초등영어 쓰기 평가에서 생성형 인공지능을 활용한 자동채점 가능성 탐색

• 영문명: Exploring the feasibility of automated scoring using generative AI in elementary English writing
• 발행기관: 한국초등영어교육학회
• 저자명: 김은송(Eunsong Kim) 곽현석(Hyunseok Kwak) 홍선호(Sunho Hong)
• 간행물 정보: 『초등영어교육』제31권 1호, 25~47쪽, 전체 23쪽
• 주제분류: 사회과학 > 교육학
• 파일형태: PDF
• 발행일자: 2025.03.31

국문 초록

This study explores the feasibility of automated scoring using generative AI (ChatGPT, Claude, and WorkAI) in an elementary English writing assessment. Essays from 87 elementary school students for English essay contest were analyzed using intraclass correlation coefficients (ICC) to examine agreement levels between human raters, among generative AI models, and between human raters and generative AI models. The results are as follows: (1) Agreement between human raters was generally high, but differences were observed in Item 2, particularly in the 'organization' criterion. (2) Agreement among generative AI models was good overall, with lenient scoring resulting in higher agreement for 'content,' 'organization,' and 'total score,' while stricter scoring worked better for 'accuracy' and 'range & mechanics.' (3) Agreement between human raters and generative AI models ranged from moderate to high. Stricter scoring produced higher agreement for Item 1 across all criteria, while lenient scoring worked better for 'content' and 'organization' in Item 2. These findings suggest that generative AI has potential as an automated scoring tool. Adjusting rater tendencies based on tasks and criteria allows generative AI to reflect human scoring patterns, supporting its use as a complementary tool in educational assessment.

목차

I. 서론

II. 선행연구

III. 연구방법

IV. 연구 결과

V. 결론 및 제언

참고문헌

키워드

자동채점 생성형 인공지능 채점자 신뢰도 구성형 문항 검사 초등영어교육 automated scoring generative AI rater reliability constructed response item test elementary English education