Abstract
Combining reading with auditory input has been shown to be an effective way of supporting reading fluency and reading comprehension in a second language. Previous research has also shown that reading comprehension can be further supported by pictorial information. However, the studies conducted so far have mainly included adults or adolescents and have been based on post-reading tests that, although informative, do not contribute to our understanding of how learners’ processing of the several sources of input in multimodal texts changes with the presence of auditory input and the effect that potential differences could have on comprehension. The present study used eye-tracking to examine how young learners process the pictorial and textual information in a graded reader under reading only and reading-while-listening conditions. Results showed that readers spent more time processing the text in the reading only condition, while more time was spent processing the images in the reading-while-listening mode. Nevertheless, comprehension scores were similar for the readers in the two conditions. Additionally, our results suggested a significant (negative) relationship between the amount of time learners spent processing the text and comprehension scores in both modes.
Funding statement: This work was supported by the Ministerio de Ciencia e Innovación, (Funder Id: http://dx.doi.org/10.13039/501100004837, Grant Number: FFI2016-80576-P).
Acknowledgements
We would like to thank Andrej Krasnansky and Radha Chandy for their help in this study.
References
Alderson, J. C. & A. H. Urquhart. 1985. The effect of students’ academic discipline on their performance on ESP reading tests. Language Testing 2. 192–204.10.1177/026553228500200207Search in Google Scholar
Anderson, R. C. & P. Freebody. 1983. Reading comprehension and the assessment and acquisition of word knowledge. In B. Hutson (ed.), Advances in reading/language research: A research annual, 231–256. Greenwich, CT: JAI Press.Search in Google Scholar
Bisson, M-J., W. Van Heuven, K. Conklin & R. Tunney. 2014. Processing of native and foreign language subtitles in films: An eye-tracking study. Applied Psycholinguistics 35. 399–418.10.1017/S0142716412000434Search in Google Scholar
Bisson, M-J., W. Van Heuven, K. Conklin & R. Tunney. 2015. The role of verbal and pictorial information in multimodal incidental acquisition of foreign language vocabulary. Quarterly Journal of Experimental Psychology 68. 1306–1326.10.1080/17470218.2014.979211Search in Google Scholar
Brown, R., R. Waring & S. Donkaewbua. 2008. Incidental vocabulary acquisition from reading, reading-while-listening, and listening to stories. Reading in a Foreign Language 20. 136–163.Search in Google Scholar
Cadwallader, J. 2009. Uncle Jack and the Bakonzi Tree. Cheltenham: ELI Pubishing.Search in Google Scholar
Carrell, P. 1991. Second language reading: Reading ability or language proficiency? Applied Linguistics 12(2). 159–179.10.1093/applin/12.2.159Search in Google Scholar
Carrell, P. L. 1983. Three components of background knowledge in reading comprehension. Language Learning 33. 183–203.10.1111/j.1467-1770.1983.tb00534.xSearch in Google Scholar
Center, Y., L. Freeman, G. Robertson & L. Outhred. 1999. The effect of visual imagery training on the reading and listening comprehension of low listening comprehenders in year 2. Journal of Research in Reading 22. 241–256.10.1111/1467-9817.00088Search in Google Scholar
Chang, A. 2009. Gains to L2 listeners from reading while listening only in comprehending short stories. System 37. 652–663.10.1016/j.system.2009.09.009Search in Google Scholar
Chang, A. & S. Millett. 2014. The effect of extensive listening on developing L2 listening fluency: Some hard evidence. ELT Journal 68. 31–40.10.1093/elt/cct052Search in Google Scholar
Chang, A. & S. Millett. 2015. Improving reading rates and comprehension through audio-assisted extensive reading for beginner learners. System 52. 91–102.10.1016/j.system.2015.05.003Search in Google Scholar
Chang, Y. & S. Choi. 2014. Effects of seductive details evidenced by gaze duration. Neurobiology of Learning and Memory 109. 131–138.10.1016/j.nlm.2014.01.005Search in Google Scholar
Conklin, K. & A. Pellicer-Sánchez. 2016. Using eye-tracking in applied linguistics and second language research. Second Language Research. Epub ahead of print doi:10.1177/0267658316637401Search in Google Scholar
Conklin, K., A. Pellicer-Sánchez & G. Carrol. 2018. Eye-tracking: A guide for applied linguistics research. Cambridge: CUP.10.1017/9781108233279Search in Google Scholar
Council of Europe. 2001. Common European framework of reference for languages. Cambridge: CUP.Search in Google Scholar
Dunn, L. M. & D. M. Dunn. 2007. Peabody picture vocabulary test, Fourth edn (PPVT™-4). Minneapolis, MN: Pearson Education.10.1037/t15144-000Search in Google Scholar
Elley, W. B. & F. Mangubhai. 1983. The impact of reading on second language learning. Reading Research Quarterly 19. 53–67.10.2307/747337Search in Google Scholar
Evans, M. A. & J. Saint-Aubin. 2005. What children are looking at during shared storybook reading: Evidence from eye movement monitoring. Psychological Science 16. 913–920.10.1111/j.1467-9280.2005.01636.xSearch in Google Scholar
Grabe, W. 2009. Reading in a second language: Moving from theory to practice. Cambridge: Cambridge University Press.10.1017/CBO9781139150484Search in Google Scholar
Han, Z. & C. A. Chen. 2010. Repeated-reading-based instructional strategy and vocabulary acquisition: A case study of a heritage speaker of Chinese. Reading in a Foreign Language 22. 242–262.Search in Google Scholar
Hill, D. 2013. Graded readers. ELT Journal 67(1). 85–125.10.1093/elt/ccs067Search in Google Scholar
Jeon, E. H. & J. Yamashita. 2014. L2 reading comprehension and its correlates: A meta-analysis. Language Learning 64. 160–212.10.1111/lang.12034Search in Google Scholar
Johnson, C. I. & R. E. Mayer. 2012. An eye movement analysis of the spatial contiguity effect in multimedia learning. Journal of Experimental Psychology: Applied. 18. 178–191.10.1037/a0026923Search in Google Scholar
Justice, L. M., L. Skibbe, A. Canning & C. Langkford. 2005. Pre-schoolers, print and storybooks: An observational study using eye movement analysis. Journal of Research in Reading 28. 229–243.10.1111/j.1467-9817.2005.00267.xSearch in Google Scholar
Kiss, T. & C. Weninger. 2016. Culture learning in the EFL classroom: The role of visuals. ELT Journal. Advance online publication: doi:10.1093/elt/ccw072Search in Google Scholar
Koda, K. 2005. Insights into second language reading: A cross-linguistic approach. NY: Cambridge University Press.10.1017/CBO9781139524841Search in Google Scholar
Kruger, J. L. & F. Steyn. 2014. Subtitles and eye tracking: Reading and performance. Reading Research Quarterly 49. 105–120.10.1002/rrq.59Search in Google Scholar
Lightbown, P.M. 1992. Can they do it themselves? A comprehension-based ESL course for young children. In R. Courchene, J. St John, C. Therien & J. I. Glidden (eds.), Comprehension-based second language teaching, 353–370. Ottawa: University of Ottawa Press.Search in Google Scholar
Lightbown, P.M., R.H. Halter, J. White & M. Horst. 2002. Comprehension-based learning: the limits of “do it yourself.”. The Canadian Modern Language Review 58. 427–464.10.3138/cmlr.58.3.427Search in Google Scholar
Marcus, N., M. Cooper & J. Sweller. 1996. Understanding instructions. Journal of Educational Psychology 88. 49–63.10.1037/0022-0663.88.1.49Search in Google Scholar
Mason, L., P. Pluchino, M. C. Tornatora & N. Ariasi. 2013a. An eye-tracking study of learning science text with concrete and abstract illustrations. Journal of Experimental Education 81. 356–384.10.1080/00220973.2012.727885Search in Google Scholar
Mason, L. & M. C. Tornatora. 2015. Integrative processing of verbal and graphical information during re-reading predicts learning from illustrated text: An eye-movement study. Reading and Writing 28. 851–872.10.1007/s11145-015-9552-5Search in Google Scholar
Mason, L., M. C. Tornatora & P. Pluchino. 2013b. Do fourth graders integrate text and picture in processing and learning from an illustrated science text? Evidence from eye-movement patterns. Computers and Education 60. 95–109.10.1016/j.compedu.2012.07.011Search in Google Scholar
Omaggio, A. C. 1979. Pictures and second language comprehension: Do they help? Foreign Language Annals 12(1979). 107.10.1111/j.1944-9720.1979.tb00153.xSearch in Google Scholar
Pickering, M., S. Frisson, B. McElree & M. Traxler. 2004. Eye movements and semantic composition. In M. Carreriras & C. Clifton (eds.), On-line study of sentence comprehension: Eye tracking, ERPs and beyond, 33–50. New York: Psychology Press.Search in Google Scholar
Qian, D. 1999. Assessing the role of depth and breadth of vocabulary knowledge in reading comprehension. The Canadian Modern Language Review 56. 282–308.10.3138/cmlr.56.2.282Search in Google Scholar
Rasinski, T. V. 1990. Effects of repeated reading and listening-while-reading on reading fluency. The Journal of Educational Research 83. 147–151.10.1080/00220671.1990.10885946Search in Google Scholar
Rayner, K. 1998. Eye movements in reading and information processing: 20 years of research. Psychological Bulletin 124. 372–422.10.1037/0033-2909.124.3.372Search in Google Scholar
Rayner, K. 2009. Eye movements and attention in reading, scene perception, and visual search. The Quarterly Journal of Experimental Psychology 62(8). 1457–1506. doi:10.1080/17470210902816461.Search in Google Scholar
Rodgers, M.P.H. 2018. The images in television programs and the potential for learning unknown words: The relationship between on-screen imagery and vocabulary. ITL - International Journal of Applied Linguistics 169(1). 191–211. doi:10.1075/itl.00012.rod.Search in Google Scholar
Roy-Charland, A., J. Saint-Aubin & M. A. Evans. 2007. Eye movements in shared book reading with children from kindergarten to grade 4. Reading and Writing 20. 909–931.10.1007/s11145-007-9059-9Search in Google Scholar
Schreiber, P. A. 1980. On the acquisition of reading fluency. Journal of Reading Behavior 12. 177–186.10.1080/10862968009547369Search in Google Scholar
Tang, G. 1992. The effect of graphic representation of knowledge structures on ESL reading comprehension. Studies in Second Language Acquisition 14. 177–195.10.1017/S0272263100010810Search in Google Scholar
Tragant, E., C. Muñoz & N. Spada. 2016. Maximizing young learners’ input: An intervention program. The Canadian Modern Language Review 72. 234–257.10.3138/cmlr.2942Search in Google Scholar
Trofimovich, P., P.M. Lightbown, R. Halter & H. Song. 2009. Comprehension based practice: The development of L2 pronunciation in a listening and reading program. Studies in Second Language Acquisition 31. 609–639.10.1017/S0272263109990040Search in Google Scholar
Webb, S. & A. Chang. 2012. Vocabulary learning through assisted and unassisted repeated reading. The Canadian Modern Language Review 68. 267–290.10.3138/cmlr.1204.1Search in Google Scholar
Webb, S. & A. Chang. 2015. Second language vocabulary learning through extensive reading with audio support: How do frequency and distribution of occurrence affect learning? Language Teaching Research 19. 667–686.10.1177/1362168814559800Search in Google Scholar
© 2019 Walter de Gruyter GmbH, Berlin/Boston
Articles in the same Issue
- Frontmatter
- Point of View
- On reelecting monolingualism: Fortification, fragility, and stamina
- Research Articles
- Learning Korean honorifics through individual and collaborative writing tasks and written corrective feedback
- Young L2 learners’ online processing of information in a graded reader during reading-only and reading-while-listening conditions: A study of eye-movements
- Exploring identities of novice mainland Chinese teachers in Hong Kong: Insights from teaching creative writing at primary schools across borders
- Attitudinal bias, individual differences, and second language speakers’ interactional performance
- Effects of self-regulated learning strategy use on motivation in EFL writing: A comparison between high and low achievers in Hong Kong primary schools
- Bilinguals and knowledge of language: a commentary to “Language Acquisition and Linguistic Theory”
Articles in the same Issue
- Frontmatter
- Point of View
- On reelecting monolingualism: Fortification, fragility, and stamina
- Research Articles
- Learning Korean honorifics through individual and collaborative writing tasks and written corrective feedback
- Young L2 learners’ online processing of information in a graded reader during reading-only and reading-while-listening conditions: A study of eye-movements
- Exploring identities of novice mainland Chinese teachers in Hong Kong: Insights from teaching creative writing at primary schools across borders
- Attitudinal bias, individual differences, and second language speakers’ interactional performance
- Effects of self-regulated learning strategy use on motivation in EFL writing: A comparison between high and low achievers in Hong Kong primary schools
- Bilinguals and knowledge of language: a commentary to “Language Acquisition and Linguistic Theory”