AI-Generated Rubrics for ESL Writing: Proficiency-Based Shifts in Assessment Priorities

• 발행기관: 한국영어평가학회
• 저자명: Nicholas Duvernay
• 간행물 정보: 『English Language Assessment』Vol.20 No.1, 11~34쪽, 전체 24쪽
• 주제분류: 인문학 > 언어학
• 파일형태: PDF
• 발행일자: 2025.06.30

국문 초록

This study examines how large language models (LLMs) develop and shift priorities in assessment variables across ESL proficiency levels. Using an extracted sample of 60 ESL essays categorized by learner proficiency from the Pittsburgh English Language Institute Corpus (PELIC), analyses were made on essay- and level-specific assessment rubrics generated by a large language model (Claude 3.7 Sonnet), finding shifts in assessment priorities dependent upon proficiency levels. Results found a clear developmental trajectory as proficiency increased, with assessment emphasis shifting from grammar and mechanics to content and rhetorical effectiveness. Five core assessment dimensions were identified, with language control decreasing in importance while rhetorical sophistication increased across proficiency levels. Principal component analysis showed increasing integration between assessment components at higher levels, with critical thinking and syntactic complexity found to be the strongest predictors of advanced writing proficiency. While the developmental patterns generally aligned with the Common European Framework of Reference for Languages (CEFR) framework, notable differences were found, particularly in vocabulary assessment weighting. The findings contribute to an understanding of writing development across proficiency levels and suggest that AI-generated rubrics may offer practical models for ESL writing assessment while providing new perspectives on the changing priorities in language evaluation.

목차

Ⅰ. INTRODUCTION

Ⅱ. LITERATURE REVIEW

Ⅲ. RESEARCH METHOD

Ⅳ. RESULTS

Ⅴ. DISCUSSION & CONCLUSION

REFERENCES

APPENDIX

키워드

artificial intelligence ESL writing assessment assessment rubrics proficiency levels large language models writing development