Investigating Rater Behavior across Diverse English Speaking Tasks

• 발행기관: 한국외국어교육학회
• 저자명: Hyun Jung Kim
• 간행물 정보: 『Foreign Languages Education』Vol.18 No.2, 99~125쪽, 전체 27쪽
• 주제분류: 사회과학 > 교육학
• 파일형태: PDF
• 발행일자: 2011.09.30

영문 초록

As the need for performance assessment grows in second language (L2) testing, rater behavior and variability have been widely examined in speaking assessments due to rater effects on examinees’ test scores. However, rater behavior on particular tasks has not been thoroughly discussed, although recent large-scale tests require expert raters for a certain task or task type (e.g., TOEFL iBT and NEAT). To this end, the current study examined and compared raters’ scoring patterns for different English speaking tasks. Across three rating sessions, nine raters from different backgrounds scored 18 examinees’ responses on five computer-delivered, semi-direct speaking tasks using an analytic scoring rubric. Their analytic ratings were analyzed for each of the five tasks based on a series of many-facet Rasch measurement models, including the facets of examinee, rater, rating occasion, and rating scale. The longitudinal analysis of rater severity, consistency, and interaction effects revealed that the raters’ scoring behavior was not uniform across the five tasks. In particular, less experienced raters showed distinct scoring patterns for different task types. A better understanding of different groups of raters may be used for rater selection and assignment to different tasks in L2 speaking assessments.

목차

Ⅰ. Introduction

Ⅱ. Literature Review

Ⅲ. Methodology

Ⅳ. Findings and Discussion

Ⅴ. Conclusion

References

Appendix

키워드

제2언어 말하기 시험 채점자 행위 L2 speaking test rater behavior