- 영문명
- Applicability of Fine-Tuned ChatGPT to Automated Essay Scoring
- 발행기관
- 한국영어평가학회
- 저자명
- Yongkook Won
- 간행물 정보
- 『English Language Assessment』Vol.18 No.2, 11~35쪽, 전체 25쪽
- 주제분류
- 인문학 > 언어학
- 파일형태
- 발행일자
- 2023.12.31
국문 초록
영문 초록
ChatGPT, released in 2022, has garnered attention due to its adaptability through
prompt engineering, enabling users to guide its responses. It is important to note
that the extent to which users can modify ChatGPT remains limited, as its core
embeddings stay unaltered through prompt engineering. Thus, this study aims to
evaluate the effectiveness of ChatGPT and its fine-tuned model in essay evaluation
compared to human raters. A total of 904 essays from the YELC 2011, on the
subject of physical punishment, were selected for this study. Among these, 723
essays were used for fine-tuning ChatGPT, and the remaining 181 were reserved
for testing the language model. Additionally, an extra set of 200 essays on different
topics, such as driving and medical issues, was included to evaluate the language
model’s performance across various themes. Inter-rater reliability indices,
including measures like correlation, agreement, Cohen’s kappa, and Krippendorff’s
alpha, along with many-facet Rasch measurement analysis, collectively indicated
that the current version of ChatGPT (gpt-3.5-turbo-0613) is not yet poised to fully
supplant human raters in essay scoring. Nevertheless, through the fine-tuning
process, the model demonstrated a significant level of agreement with human raters
and exhibited a marked degree of consistency.
목차
Ⅰ. INTRODUCTION
Ⅱ. LITERATURE REVIEW
Ⅲ. RESEARCH METHOD
Ⅳ. RESULTS
Ⅴ. DISCUSSION
Ⅵ. CONCLUSION
REFERENCES
키워드
해당간행물 수록 논문
참고문헌
최근 이용한 논문
교보eBook 첫 방문을 환영 합니다!
신규가입 혜택 지급이 완료 되었습니다.
바로 사용 가능한 교보e캐시 1,000원 (유효기간 7일)
지금 바로 교보eBook의 다양한 콘텐츠를 이용해 보세요!