복합양식 텍스트 읽기 과정에 대한 시선추적 양상 연구 : 그래픽 노블과 그림책을 중심으로

• 영문명: Eye-tracking Study on Students' Reading Process of Multimodal Texts: Focusing on Graphic Novels and Picturebooks
• 발행기관: 한국교원대학교 뇌·AI기반교육연구소
• 저자명: 윤석주 이경화
• 간행물 정보: 『Brain, Digital, & Learning』제15권 제3호, 309~330쪽, 전체 22쪽
• 주제분류: 사회과학 > 교육학
• 파일형태: PDF
• 발행일자: 2025.09.30

국문 초록

The purpose of this study was to compare and analyze the gaze movement patterns of sixthgrade elementary school students while reading multimodal texts—graphic novels and picturebooks—in order to investigate their reading strategies and meaning-making processes. A crossover experimental design was employed in which the same participants read both types of texts. Quantitative and qualitative data were collected using an eye-tracking device, focusing on gaze path, heatmap, and areas of interest (AOI), and were supplemented by retrospective think-aloud interviews. The analysis of gaze paths revealed that during graphic novel reading, students followed a structured pattern in line with the sequential flow between speech balloons and images, indicating a text-centered information processing tendency. In contrast, during picturebook reading, students exhibited more fluid gaze patterns led by image exploration, highlighting the construction of meaning through the interaction between visual and verbal information. Heatmap analysis showed strong fixation on speech balloon texts and characters’ faces in graphic novels, whereas in picturebooks, fixations were concentrated on text and images marking narrative transitions. In AOI analysis, the text area in graphic novels showed the highest dwell time and number of fixations, while in picturebooks, significant gaze distribution was also observed in image areas related to the text. Interpreting these results through Serafini’s (2012) reader resource theory for successful multimodal text reading, readers of graphic novels primarily activated roles as "navigators" and "designers," whereas readers of picturebooks more prominently exhibited the roles of "interpreters" and "navigators." This study provides an empirical, process-oriented approach to multimodal text reading and offers meaningful implications for reading education and research in multimodal literacy by identifying genre-specific differences in information integration strategies.

목차

Introduction

Theoretical Background

Methods

Results

Discussion

References

키워드

Multimodal texts eye-tracking graphic novels picturebooks gaze path heatmap AOI