The Effect of Task Complexity on Test-takers’ Performance in a Performance-based L2 Oral Communication Test for International Teaching Assistants

• 발행기관: 한국영어교과교육학회
• 저자명: Yongkook Won
• 간행물 정보: 『영어교과교육』영어교과교육 19권 1호, 27~52쪽, 전체 26쪽
• 주제분류: 사회과학 > 교육학
• 파일형태: PDF
• 발행일자: 2020.02.28

영문 초록

The main purpose of this study is to understand the effects of task complexity on test-takers’ linguistic output and raters’ evaluation of test-takers’ performance. One hundred and fifty-six audio clips by 52 non-native-English speaking international teaching assistants (ITAs) at a U.S. university were graded by two raters with four different rating scales a holistic scale, a pronunciation scale, a vocabulary and grammar scale, and a pace and fluency scale. The clips were then transcribed and coded for linguistic complexity, accuracy, and fluency measures per AS-unit. Task complexity was evaluated based on the resource-directing task complexity criterion from Robinson’s (2001b) Triadic Componential Framework. The grades and linguistic measures were statistically analyzed using multiple Friedman tests and follow-up pairwise comparisons with Wilcoxon signed-rank tests. It was found that only one of the fluency measures, phonation-time ratio, and fluency scores, not holistic scores, with high and low task complexity, were statistically different with mid-effect sizes. In the follow-up interview, raters reported that they tended to adjust their holistic rating severity according to task complexity. Implications of the findings for rater training and evaluation rubric design are discussed.

목차

I. INTRODUCTION

II. LITERATURE REVIEW

III. METHOD

IV. RESULTS

V. DISCUSSION

VI. CONCLUSION

REFERENCES

키워드

과제복잡성 수행평가 언어적 특징 task complexity performance-based assessment linguistic features