AI-Automated EFL Writing Assessment Using Standardized Rubrics

• 발행기관: 팬코리아영어교육학회(구 영남영어교육학회)
• 저자명: Damian Heywood Joseph Carrier Kyu-Hong Hwang
• 간행물 정보: 『영어교육연구』제37권 1호, 21~38쪽, 전체 18쪽
• 주제분류: 사회과학 > 교육학
• 파일형태: PDF
• 발행일자: 2025.03.31

국문 초록

This study investigates the reliability of AI-automated writing assessment in university-level EFL composition classes. The research compares human assessment and AI-generated assessment of 32 paragraphs written by Korean university students, using a standardized rubric adapted from the University of Michigan Writing Center. The study utilized Claude 3.5 Sonnet, accessed via API calls, to conduct multiple assessment iterations under controlled conditions. The assessment criteria included Content and Ideas, Organization and Structure, Language Use and Vocabulary, and Grammar and Mechanics, each scored on a five-point scale. Statistical analysis revealed high consistency in AI assessment (r=0.979). While correlation between human raters was moderate (r = 0.507), the AI system demonstrated reliable assessment patterns that aligned significantly with human scoring trends. The findings suggest that AI-automated assessment systems can provide reliable evaluation of student writing, though with some notable differences from human assessment patterns. This study contributes to the emerging field of AI-assisted writing assessment and provides a foundation for developing more sophisticated automated feedback systems for EFL writing instruction.

목차

Ⅰ. INTRODUCTION

Ⅱ. THEORETICAL BACKGROUND

Ⅲ. METHOD

Ⅳ. RESULTS

Ⅴ. DISCUSSION

Ⅵ. CONCLUSION

REFERENCES

키워드

AI-automated assessment writing assessment EFL composition standardized rubrics Large Language Models (LLMs) Computer-Assisted Language Learning (CALL) human assessment reliability assessment correlation