What watching subtitled movies does to the learner: the impact of subtitles and modality on cognitive load and vocabulary learning

References

Alshumrani, Hassan. 2019. L2 incidental vocabulary learning and retention through different modalities of audio-visual input. University of Southampton Doctoral dissertation. https://eprints.soton.ac.uk/434623/1/LIBRARY_COPY_Hassan_final_thesis_draft.pdf.Search in Google Scholar

Antonenko, Pavlo, Fred Paas, Roland Grabner & Tamara van Gog. 2010. Using electroencephalography to measure cognitive load. Educational Psychology Review 22. 425–438. https://doi.org/10.1007/s10648-010-9130-y.Search in Google Scholar

Ashcroft, Robert John, Joseph Garner & Hadingham Oliver. 2018. Incidental vocabulary learning through watching movies. Australian Journal of Applied Linguistics 1(3). 135–147. https://doi.org/10.29140/ajal.v1n3.89.Search in Google Scholar

Baranowska, Karolina. 2020. Learning most with least effort: Subtitles and cognitive load. ELT Journal 74(2). 105–115. https://doi.org/10.1093/elt/ccz060.Search in Google Scholar

Bianchi, Francesca & Tiziana Ciabattoni. 2008. Captions and subtitles in EFL learning: An investigative study in a comprehensive computer environment. In Anthony Baldry, Maria Pavesi, Carol Taylor Torsello & Christopher Taylor (eds.), From didactas to ecolingua: An ongoing research project on translation and corpus linguistics, 69–90. Trieste: Edizioni Università di Trieste.Search in Google Scholar

Bird, Stephen A. & John N. Williams. 2002. The effect of bimodal input on implicit and explicit memory: An investigation into the benefits of within-language subtitling. Applied Psycholinguistics 23(4). 509–533. https://doi.org/10.1017/s0142716402004022.Search in Google Scholar

Bjork, Elizabeth Ligon & Robert A. Bjork. 2011. Making things hard on yourself, but in a good way: Creating desirable difficulties to enhance learning. In Morton Ann Gernsbacher, Richard W. Pew, Lynn M. Hough & James R. Pomerantz (eds.), Psychology and the real world: Essays illustrating fundamental contributions to society, 56–64. New York: Worth Publishers.Search in Google Scholar

Brünken, Roland, Jan L. Plass & Detlev Leutner. 2003. Direct measurement of cognitive load in multimedia learning. Educational Psychologist 38(1). 53–61. https://doi.org/10.1207/s15326985ep3801_7.Search in Google Scholar

Brünken, Roland, Susan Steinbacher, Jan L. Plass & Detlev Leutner. 2002. Assessment of cognitive load in multimedia learning using dual-task methodology. Experimental Psychology 49(2). 109–119. https://doi.org/10.1027//1618-3169.49.2.109.Search in Google Scholar

Chang, Anna C.-S. & Sonia Millett. 2014. The effect of extensive listening on developing L2 listening fluency: Some hard evidence. ELT Journal 68(1). 31–40. https://doi.org/10.1093/elt/cct052.Search in Google Scholar

Choi, Sungmook. 2023. Visual saliency in captioned digital videos and learning of English collocations: An eyetracking study. Language Learning & Technology 27(1). 1–21.Search in Google Scholar

Clinton-Lisell, Virginia. 2023. Does reading while listening to text improve comprehension compared to reading only? A systematic review and meta-analysis. Educational Research: Theory and Practice 34(3). 133–155.10.31219/osf.io/ku6tvSearch in Google Scholar

Cobb, Tom. 2002. The Compleat lexical tutor [online resource]. https://www.lextutor.ca.Search in Google Scholar

Conklin, Kathy, Sara Alotaibi, Ana Pellicer-Sánchez & Laura Vilkaitė-Lozdienė. 2020. What eye-tracking tells us about reading-only and reading-while-listening in a first and second language. Second Language Research 36(3). 257–276. https://doi.org/10.1177/0267658320921496.Search in Google Scholar

Conway, Andrew R. A., Michael J. Kane, Michael F. Bunting, D. Zach Hambrick, Oliver Wilhelm & Randall W. Engle. 2005. Working memory span tasks: A methodological review and user’s guide. Psychonomic Bulletin & Review 12(5). 769–786. https://doi.org/10.3758/bf03196772.Search in Google Scholar

Danan, Martine. 2004. Captioning and subtitling: Undervalued language learning strategies. Meta 49(1). 67–77. https://doi.org/10.7202/009021ar.Search in Google Scholar

de Leeuw, Krista & Richard E. Mayer. 2008. A comparison of three measures of cognitive load: Evidence for separable measures of intrinsic, extraneous, and germane load. Journal of Educational Psychology 100(1). 223–234. https://doi.org/10.1037/0022-0663.100.1.223.Search in Google Scholar

Dizon, Gilbert & Benjamin Thanyawatpokin. 2021. Language learning with Netflix: Exploring the effects of dual subtitles on vocabulary learning and listening comprehension. Computer Assisted Language Learning Electronic Journal 22(3). 52–65.Search in Google Scholar

Ellis, Nick C. 1994. Consciousness in second language learning: Psychological perspectives on the role of conscious processes in vocabulary acquisition. In Jan H. Hulstijn & Richard Schmidt (eds.), Consciousness in second language learning. AILA Review, vol. 11, 37–56.Search in Google Scholar

Feng, Yanxue & Stuart Webb. 2020. Learning vocabulary through reading, listening, and viewing: Which mode of input is most effective? Studies in Second Language Acquisition 42(3). 499–523. https://doi.org/10.1017/s0272263119000494.Search in Google Scholar

Frumuselu, Anca Daniela. 2018. The implications of cognitive load theory and exposure to subtitles in English Foreign Language (EFL). Translation and Translanguaging in Multilingual Contexts 4(1). 55–76. https://doi.org/10.1075/ttmc.00004.fru.Search in Google Scholar

Frumuselu, Anca Daniela, Sven De Maeyer, Donche Vincent & María del Mar Gutiérrez Colon Plana. 2015. Television series inside the EFL classroom: Bridging the gap between teaching and learning informal language through subtitles. Linguistics and Education 32. 107–117. https://doi.org/10.1016/j.linged.2015.10.001.Search in Google Scholar

Gesa, Ferran & Imma Miralpeix. 2023. Extensive viewing as additional input for foreign language vocabulary learning: A longitudinal study in secondary school. Language Teaching Research. https://doi.org/10.1177/13621688231169451.Search in Google Scholar

Ghebghoub, Souheyla. 2021. Imagery in L2 captioned video: Investigating incidental vocabulary learning from extensive viewing as a function of modality, contiguity, and spacing. University of York Doctoral dissertation.Search in Google Scholar

Hart, Sandra G. & Lowell E. Staveland. 1988. Development of NASA-TLX (task load index): Results of empirical and theoretical research. In Peter A. Hancock & Najmedin Meshkati (eds.), Human mental workload, 139–183. North-Holland: Elsevier Science Publishers.10.1016/S0166-4115(08)62386-9Search in Google Scholar

Hsieh, Yufen. 2019. Effects of video captioning on EFL vocabulary learning and listening comprehension. Computer Assisted Language Learning 33(5–6). 567–589. https://doi.org/10.1080/09588221.2019.1577898.Search in Google Scholar

Hsu, Wenhua. 2017. The modality effect of input repetition on vocabulary consolidation. Konin Language Studies 5(4). 403–424.Search in Google Scholar

Hulstijn, Jan H. 2003. Incidental and intentional learning. In Catherine J. Doughty & Michael H. Long (eds.), The handbook of second language acquisition, 349–381. Malden, MA: Blackwell Publishing.10.1002/9780470756492.ch12Search in Google Scholar

Hummel, Kristen M. 2010. Translation and short-term L2 vocabulary retention: Hindrance or help? Language Teaching Research 14(1). 61–74. https://doi.org/10.1177/1362168809346497.Search in Google Scholar

IBM Corp. 2019. IBM SPSS Statistics for Windows. Version 26.0. Armonk, NY: IBM Corp.Search in Google Scholar

İnceçay, Volkan & Zeynep Koçoğlu. 2017. Investigating the effects of multimedia input modality on L2 listening skills of Turkish EFL learners. Education and Information Technologies 22. 901–916. https://doi.org/10.1007/s10639-016-9463-3.Search in Google Scholar

Jelani, Nurul Aini Mohd & Frank Boers. 2018. Examining incidental vocabulary acquisition from captioned video: Does test modality matter? International Journal of Applied Linguistics 169(1). 169–190. https://doi.org/10.1075/itl.00011.jel.Search in Google Scholar

Jewitt, Carey. 2013. Multimodal methods for researching digital technologies. In S. Price, C. Jewitt & B. Brown (eds.), The SAGE handbook of digital technology research, 250–265. SAGE Publications.10.4135/9781446282229.n18Search in Google Scholar

Kalyuga, Slava. 2012. Instructional benefits of spoken words: A review of cognitive load factors. Educational Research Review 7(2). 145–159. https://doi.org/10.1016/j.edurev.2011.12.002.Search in Google Scholar

Kalyuga, Slava, Paul L. Ayres, Paul Chandler & John Sweller. 2003. The expertise reversal effect. Educational Psychologist 38(1). 23–31. https://doi.org/10.1207/s15326985ep3801_4.Search in Google Scholar

Kalyuga, Slava, Paul Chandler & John Sweller. 2001. Learner experience and efficiency of instructional guidance. Educational Psychology 21(1). 5–23. https://doi.org/10.1080/01443410124681.Search in Google Scholar

Kıvrak, Cansu & Dilek Uygun Gökmen. 2019. The effects of task-induced involvement load and input modality on incidental vocabulary learning. Journal of Language and Linguistic Studies 15(4). 1355–1375.10.17263/jlls.668478Search in Google Scholar

Kress, Gunther & Theo Van Leeuwen. 2001. Multimodal discourse: The modes and media of contemporary communication. London: Arnold Publishers.Search in Google Scholar

Kruger, Jan-Louis, Este Hefer & Gordon Matthew. 2013. Measuring the impact of subtitles on cognitive load: Eye-tracking and dynamic audiovisual texts. In Proceedings of the 2013 conference on eye tracking South Africa, 62–66. New York: Association for Computing Machinery.10.1145/2509315.2509331Search in Google Scholar

Kruger, Jan-Louis, Esté Hefer & Gordon Matthew. 2014. Attention distribution and cognitive load in a subtitled academic lecture: L1 vs. L2. Journal of Eye Movement Research 7(5). 1–15. https://doi.org/10.16910/jemr.7.5.4.Search in Google Scholar

Lavaur, Jean-Marc & Dominique Bairstow. 2011. Idiomas on the screen: Is film comprehension related to viewers’ fluency level and to the idioma in the subtitles? International Journal of Psychology 46(6). 455–462.10.1080/00207594.2011.565343Search in Google Scholar

Lemhöfer, Kristin & Mirjam Broersma. 2012. Introducing LexTALE: A quick and valid lexical test for advanced learners of English. Behavior Research Methods 44. 325–343. https://doi.org/10.3758/s13428-011-0146-0.Search in Google Scholar

Leppink, Jimmie, Paas Fred, Gees P. M. Van der Vleuten, Tamara Van Gog & Jeroen J. G. Van Merriënboer. 2013. Development of an instrument for measuring different types of cognitive load. Behavior Research Methods 45(4). 1058–1072. https://doi.org/10.3758/s13428-013-0334-1.Search in Google Scholar

Leveridge, Aubrey Neil & Jie Chi Yang. 2013. Testing learner reliance on caption supports in second language listening comprehension multimedia environments. ReCALL 25(2). 199–214. https://doi.org/10.1017/s0958344013000074.Search in Google Scholar

Liando, Nihta V. F., Ray J. V. Sahetapy & Mister G. Maru. 2018. English major students’ perceptions towards watching English movies in listening and speaking skills development. Advances in Social Sciences Research Journal 5(6). https://doi.org/10.14738/assrj.56.4627.Search in Google Scholar

Liao, Sixin, Jan-Louis Kruger & Stephen Doherty. 2020. The impact of monolingual and bilingual subtitles on visual attention, cognitive load, and comprehension. The Journal of Specialised Translation 33.Search in Google Scholar

Madrid, R. Ignacio, Herre van Oostendorp & Mari Carmen Puerta Melguizo. 2009. The effects of the number of links and navigation support on cognitive load and learning with hypertext: The mediating role of reading order. Computers in Human Behavior 25(1). 66–75. https://doi.org/10.1016/j.chb.2008.06.005.Search in Google Scholar

Markham, Paul, Lizette Peter & Teresa J. McCarthy. 2001. The effects of native language vs. target language captions on foreign language students’ DVD video comprehension. Foreign Language Annals 34(5). 439–445. https://doi.org/10.1111/j.1944-9720.2001.tb02083.x.Search in Google Scholar

Mayer, Richard E., Logan Fiorella & Andrew Stull. 2020. Five ways to increase the effectiveness of instructional video. Educational Technology Research and Development 68. 837–852. https://doi.org/10.1007/s11423-020-09749-6.Search in Google Scholar

Mayer, Richard E., Hyunjeong Lee & Peebles Alanna. 2014. Multimedia learning in a second language: A cognitive load perspective. Applied Cognitive Psychology 28. 653–660. https://doi.org/10.1002/acp.3050.Search in Google Scholar

Mitterer, Holger & James M. McQueen. 2009. Foreign subtitles help but native-language subtitles harm foreign speech perception. PLoS One 4(11). https://doi.org/10.1371/journal.pone.0007785.Search in Google Scholar

Montero Perez, Maribel. 2019. Pre-learning vocabulary before viewing captioned video: An eyetracking study. The Language Learning Journal 47(4). 460–478. https://doi.org/10.1080/09571736.2019.1638623.Search in Google Scholar

Montero Perez, Maribel. 2022. Second or foreign language learning through watching audio-visual input and the role of on-screen text. Language Teaching 55(2). 1–30. https://doi.org/10.1017/s0261444821000501.Search in Google Scholar

Montero Perez, Maribel, Elke Peters, Geraldine Clarebout & Piet Desmet. 2014. Effects of captioning on video comprehension and incidental vocabulary learning. Language Learning and Technology 18(1). 118–141.Search in Google Scholar

Montero Perez, Maribel, Wim Van Den Noortgate & Desmet Piet. 2013. Captioned video for L2 listening and vocabulary learning: A meta-analysis. System 41(3). 720–739. https://doi.org/10.1016/j.system.2013.07.013.Search in Google Scholar

Muñoz, Carmen, Anastasia Pattemore & Daniela Avello. 2022. Exploring repeated captioning viewing as a way to promote vocabulary learning: Time lag between repetitions and learner factors. Computer Assisted Language Learning 37. 1–27.10.1080/09588221.2022.2113898Search in Google Scholar

Muñoz, Carmen. 2017. The role of age and proficiency in subtitle reading: An eye-tracking study. System 67. 77–86. https://doi.org/10.1016/j.system.2017.04.015.Search in Google Scholar

Muñoz, Carmen, Geòrgia Pujadas & Anastasiia Pattemore. 2021. Audio-visual input for learning L2 vocabulary and grammatical constructions. Second Language Research. 1–25.10.1177/02676583211015797Search in Google Scholar

Napikul, Supakorn, Payung Cedar & Dusadee Roongrattanakool. 2018. The effects of film subtitles on English listening comprehension and vocabulary. International Journal of Applied Linguistics & English Literature 7(6). 104–111. https://doi.org/10.7575/aiac.ijalel.v.7n.6p.104.Search in Google Scholar

Nation, I. S. P. 2006. How large a vocabulary is needed for reading and listening? The Canadian Modern Language Review 63(1). 59–82. https://doi.org/10.1353/cml.2006.0049.Search in Google Scholar

Nation, I. S. P. & S. Webb. 2011. Researching and analyzing vocabulary. Heinle Cengage Learning.Search in Google Scholar

Norouzian, Reza & Luke Plonsky. 2018. Eta- and partial eta-squared in L2 research: A cautionary review and guide to more appropriate usage. Second Language Research 34(2). 257–271. https://doi.org/10.1177/0267658316684904.Search in Google Scholar

Paas, Fred, Juhani E. Tuovinen, Huib Tabbers & Pascal W. M. van Gerven. 2003. Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist 38(1). 63–71. https://doi.org/10.1207/s15326985ep3801_8.Search in Google Scholar

Paivio, Allan. 1986. Mental representations: A dual coding approach. Oxford: Oxford University Press.Search in Google Scholar

Paivio, Allan & Wallace Lambert. 1981. Dual coding and bilingual memory. Journal of Verbal Learning and Verbal Behavior 20. 532–539. https://doi.org/10.1016/s0022-5371(81)90156-0.Search in Google Scholar

Park, Babette & Roland Brünken. 2014. The rhythm method: A new method for measuring cognitive load—an experimental dual-task study. Applied Cognitive Psychology 29(2). 232–243. https://doi.org/10.1002/acp.3100.Search in Google Scholar

Pattemore, Anastasia & Carmen Muñoz. 2022. Captions and learnability factors in learning grammar from audio-visual input. The JALT CALL Journal 18(1). 83–109. https://doi.org/10.29140/jaltcall.v18n1.564.Search in Google Scholar

Peirce, Jonathan, Jeremy R. Gray, Sol Simpson, Michael MacAskill, Richard Höchenberger, Hiroyuki Sogo, Erik Kastman & Jonas Kristoffer Lindeløv. 2019. PsychoPy2: Experiments in behavior made easy. Behavior Research Methods 51(1). 195–203. https://doi.org/10.3758/s13428-018-01193-y.Search in Google Scholar

Peters, Elke. 2018. The effect of out-of-class exposure to English language media on learners’ vocabulary knowledge. ITL – International Journal of Applied Linguistics 169(1). 142–168. https://doi.org/10.1075/itl.00010.pet.Search in Google Scholar

Peters, Elke, Eva Heynen & Eva Puimege. 2016. Learning vocabulary through audiovisual input: The differential effect of L1 subtitles and captions. System 63. 134–148. https://doi.org/10.1016/j.system.2016.10.002.Search in Google Scholar

Pujadas, Geòrgia & Carmen Muñoz. 2019. Extensive viewing of captioned and subtitled TV series: A study of L2 vocabulary learning by adolescents. The Language Learning Journal 47. 479–496. https://doi.org/10.1080/09571736.2019.1616806.Search in Google Scholar

Pusey, Kerry & Karen Lenz. 2014. Investigating the interaction of visual input, working memory, and listening comprehension. Language Education in Asia 5(1). 66–80. https://doi.org/10.5746/leia/14/v5/i1/a06/pusey_lenz.Search in Google Scholar

Richards, Jack C. 2015. Key issues in language teaching. Cambridge: Cambridge University Press.10.1017/9781009024600Search in Google Scholar

Rodgers, Michael & Stuart Webb. 2011. Narrow viewing: The vocabulary in related television programs. TESOL Quarterly 45(4). 689–717.10.5054/tq.2011.268062Search in Google Scholar

Roediger, Henry L. & JeffreyD. Karpicke. 2006. The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science 1(3). 181–210. https://doi.org/10.1111/j.1745-6916.2006.00012.x.Search in Google Scholar

Safarali, Somayeh Karbalaie & Hadi Hamidi. 2012. The impact of videos presenting speakers’ gestures and facial clues on Iranian EFL learners’ listening comprehension. International Journal of Applied Linguistics & English Literature 1(6). 106–114. https://doi.org/10.7575/ijalel.v.1n.6p.106.Search in Google Scholar

Schmidt, Richard. 2000. Attention. In Peter Robinson (ed.), Cognition and second language instruction. Cambridge: Cambridge University Press.Search in Google Scholar

Schoor, Cornelia, Maria Bannert & Ronald Brünken. 2012. Role of dual task design when measuring cognitive load during multimedia learning. Educational Technology Research & Development 60(5). 753–768.10.1007/s11423-012-9251-8Search in Google Scholar

Serrano, Raquel & Ana Pellicer-Sánchez. 2019. Young L2 learners’ online processing of information in a graded reader during reading-only and reading-while-listening conditions: A study of eye-movements. Applied Linguistics Review. https://doi.org/10.1515/applirev-2018-0102.Search in Google Scholar

Shabani, Karim & Marjan Pasha Zanussi. 2015. The impact of watching captioned TV series on vocabulary development of EFL students. Journal for the Study of English Linguistics 1. 118–129.10.5296/jsel.v3i1.8301Search in Google Scholar

Suarez, Maria del Mar, Roger Gilabert & Natalia Moskvina. 2021. The mediating role of vocabulary size, working memory, attention and inhibition in early vocabulary learning under different TV genres: An exploratory study. TESOL Journal 12(4). 1–29. https://doi.org/10.1002/tesj.637.Search in Google Scholar

Sweller, John. 2010. Element interactivity and intrinsic, extraneous, and germane cognitive load. Educational Psychology Review 22. 123–138. https://doi.org/10.1007/s10648-010-9128-5.Search in Google Scholar

Sweller, John, Paul Ayres & Slava Kalyuga. 2011. Cognitive load theory. London: Springer.10.1007/978-1-4419-8126-4Search in Google Scholar

Sweller, John, Jeroen Van Merriënboer & Fred G. Paas. 1998. Cognitive architecture and instructional design. Educational Psychology Review 10(3). 251–296.10.1023/A:1022193728205Search in Google Scholar

Sydorenko, Tetyana. 2010. Modality of input and vocabulary acquisition. Language Learning & Technology 14(2). 50–73.Search in Google Scholar

Szarkowska, Agnieszka & Olivia Gerber-Morón. 2019. Two or three lines: A mixed-methods study on subtitle processing and preferences. Perspectives 27(1). 144–164. https://doi.org/10.1080/0907676x.2018.1520267.Search in Google Scholar

Teng, Feng. 2019. Maximizing the potential of captions for primary school ESL students’ comprehension of English-language videos. Computer Assisted Language Learning 32(7). 665–691. https://doi.org/10.1080/09588221.2018.1532912.Search in Google Scholar

Vanderplank, Robert. 1988. The value of teletext subtitling in language learning. ELT Journal 42. 272–281. https://doi.org/10.1093/elt/42.4.272.Search in Google Scholar

Vanderplank, Robert. 2010. Deja vu? A decade of research on language laboratories, television and video in language learning. Language Teaching 43(1). 1–37. https://doi.org/10.1017/s0261444809990267.Search in Google Scholar

Vanderplank, Robert. 2016. Captioned media in foreign language learning and teaching: Subtitles for the deaf and hard-of-hearing as tools for language learning. Palgrave Macmillan.10.1057/978-1-137-50045-8Search in Google Scholar

Wang, Yangting. 2019. Effects of L1/L2 captioned TV programs on students’ vocabulary learning and comprehension. CALICO Journal 36(3). 204–224. https://doi.org/10.1558/cj.36268.Search in Google Scholar

Wang, Andi & Ana Pellicer-Sánchez. 2022. Incidental vocabulary learning from bilingual subtitled viewing: An eye-tracking study. Language Learning 72(3). 765–805. https://doi.org/10.1111/lang.12495.Search in Google Scholar

Yang, Jimijin. 2018. Use of L1 and L2 subtitles for incidental vocabulary learning in a second language: Focusing on word frequency. STEM Journal 19(3). 127–147. https://doi.org/10.16875/stem.2018.19.3.127.Search in Google Scholar

Yang, Chunliang, Rosalind Potts & David R. Shanks. 2018. Enhancing learning and retrieval of new information: A review of the forward testing effect. npj Science of Learning 3(8). 1–9. https://doi.org/10.1038/s41539-018-0024-y.Search in Google Scholar

Zarei, Abbas Ali. 2009. The effect of bimodal, standard, and reversed subtitling on L2 vocabulary recognition and recall. Pazhuhesh-e Zabanha-ye Khareji 49. 65–85.Search in Google Scholar

Zhao, Ting & Ernesto Macaro. 2016. What works better for the learning of concrete and abstract words: Teachers’ L1 use or L2-only explanations? International Journal of Applied Linguistics 26(1). 75–98. https://doi.org/10.1111/ijal.12080.Search in Google Scholar