Examining the Role of Generative AI in Academic Writing Assessment: A Mixed-Methods Study in Higher Education

• 발행기관: 한국ESP학회
• 저자명: Noble Lo Sumie Chan
• 간행물 정보: 『ESP Review』Vol.7 No.2, 7~45쪽, 전체 39쪽
• 주제분류: 어문학 > 영어와문학
• 파일형태: PDF
• 발행일자: 2025.08.31

국문 초록

This study examines the use of generative artificial intelligence (AI) to assist in grading and feedback on undergraduate academic writing. While AI technologies in education have shown considerable potential, their adoption remains controversial due to contrasting evidence about their effectiveness. At a university in Hong Kong, an initiative was launched to use generative AI for providing formative feedback on students’ written drafts prior to final submission. This mixed-methods study evaluates the initiative by comparing AIgenerated grades with those assigned by human instructors and analyzing stakeholder perceptions of the AI feedback. Discrepancies between the findings of quantitative and qualitative research may be attributed in part to the research instruments used. Quantitative analysis revealed high inter-rater reliability between AI-generated scores and humanassigned grades, demonstrating the technical accuracy of the system. However, qualitative data from student interviews and instructor reflections revealed significant concerns about the AI’s ability to deliver contextually relevant, personalized, and pedagogically meaningful guidance. Stakeholders expressed scepticism and mistrust, emphasizing the lack of depth, specificity, and adaptability in AI feedback compared to human feedback. By exploring these discrepancies, this study underscores the tension between technical reliability and perceived pedagogical value in AI-assisted assessment. It advocates for stakeholder-informed, hybrid approaches that integrate AI with human oversight to ensure AI tools genuinely support student learning outcomes. Future initiatives should prioritize collaborative design, transparent communication, and stakeholder education to address these challenges effectively.

목차

I. INTRODUCTION

II. AIMS AND OBJECTIVES

III. LITERATURE REVIEW

IV. METHODOLOGY

V. FINDINGS

VI. DISCUSSION

VII. CONCLUSION

REFERENCES

키워드

artificial intelligence English Language Teaching feedback grading higher education revision