The effects of watching subtitled videos on the perception of L2 connected speech by L1 Chinese-L2 English speakers
-
Yanjiao Zhu
and
Jiehui Hu
Abstract
The current study explores whether watching subtitled videos could facilitate L1 Chinese-L2 English speakers’ perception of L2 English connected speech. Three hundred ninty seven Chinese college students of L2 English completed a video-based spot dictation task after watching English videos with or without L1/L2 subtitles, featuring various connected speech types (e.g., linking, deletion, and their combinations). Results suggested an overall facilitation effect of watching videos on L2 connected speech perception, which was modulated by proficiency, subtitle form, and the complexity of connected speech. First, subtitled videos were more facilitative than non-subtitled videos in L2 perception. Second, participants with higher L2 proficiency better perceived English connected speech than those with lower proficiency. Third, the more connective devices an item used, the more difficult it was for L2 perception. When this complexity was controlled, the L2 perception was not influenced by connected speech type. Finally, the complexity of connected speech also mediated the subtitle facilitation effects. When the connected speech involved triple connective devices, L2 speakers benefited more from L1 subtitles than L2 subtitles. The findings can provide insights into multi-modal speech perception and English connected speech learning.
Funding source: Sichuan Federation of Social Science Associations
Award Identifier / Grant number: SC23BS019
Funding source: Sichuan Federation of Social Science Associations
Award Identifier / Grant number: SC23BS019
-
Ethics statement: All participants were provided written informed consent in accordance with the Declaration of Helsinki. The ethics committee of the University of Electronic Science and Technology of China approved this study (protocol code #225956), including its participant recruitment procedure and methodology.
-
Conflict of interest: The authors declare no conflict of interest.
-
Research funding: This work was funded by Sichuan Federation of Social Science Associations (Award no.: SC23BS019).
Appendix A: Counts of the errors
| Item | Missing out words | Wrong words but sound similar | Change wording | Tense or aspect | Extra words | Irrelevant words |
|---|---|---|---|---|---|---|
| It’s | 98 | 0 | 16 | 9 | 4 | 57 |
| Been | 0 | 58 | 0 | 65 | 10 | 45 |
| All this | 129 | 16 | 16 | 0 | 0 | 61 |
| In his | 48 | 13 | 56 | 0 | 0 | 100 |
| But also let’s just | 87 | 25 | 4 | 0 | 0 | 80 |
| Gonna scroll up | 4 | 7 | 4 | 0 | 0 | 223 |
| Can | 0 | 0 | 23 | 11 | 1 | 40 |
| What | 0 | 22 | 9 | 0 | 5 | 10 |
| Decided | 0 | 53 | 3 | 63 | 3 | 33 |
| With her | 0 | 21 | 2 | 0 | 0 | 11 |
| Need to | 15 | 9 | 4 | 1 | 0 | 128 |
| Would | 0 | 2 | 38 | 0 | 2 | 65 |
| With us | 10 | 6 | 4 | 1 | 0 | 18 |
| I don’t like | 16 | 36 | 44 | 2 | 4 | 91 |
| Take care of | 44 | 4 | 14 | 0 | 2 | 48 |
| Leg and all | 72 | 28 | 56 | 0 | 0 | 50 |
| Just | 0 | 17 | 28 | 0 | 8 | 64 |
| Some | 0 | 6 | 3 | 0 | 0 | 1 |
| We’ve been | 37 | 10 | 18 | 26 | 0 | 85 |
| To show us | 102 | 41 | 37 | 9 | 0 | 12 |
| Way to go | 4 | 167 | 18 | 0 | 0 | 52 |
| All the fun | 12 | 11 | 60 | 33 | 8 | 31 |
| Want your | 58 | 13 | 13 | 11 | 0 | 5 |
| Mocked you to | 73 | 5 | 29 | 0 | 1 | 122 |
| With an eye | 48 | 36 | 7 | 7 | 0 | 139 |
| You are | 6 | 7 | 6 | 1 | 7 | 15 |
| Could dream of | 91 | 10 | 19 | 15 | 2 | 31 |
| All this | 52 | 23 | 26 | 6 | 2 | 122 |
| Rotate out | 1 | 51 | 17 | 1 | 0 | 173 |
| At least | 3 | 134 | 4 | 0 | 2 | 67 |
Appendix B: Transcription of the testing materials
Video 1
I said, yes, it’s [ɪ̥ts] always been [bɪ̥n] a dream of mine to have done a Ted talk in the past. But in the middle of all this [əvɔːlɪs] excitement, the rational decision maker seemed to have something else in his [ɪnɪs] mind. He was saying, are we clear on what we just accepted? Do we get what’s going to be now happening one day in the future? We need to sit down and work on this right now. The monkey said totally agree, but also let’s [bə ɔːlsəʊlɛts] just open Google Earth, and let’s zoom into the bottom of India, like 200 feet above the ground, and we’re gonna scroll up [ɡənə skrəʊləp̚] for 2.5 h till we get to the top of the country, so we can [kən] get a better feel for India. So that’s what [wɒt̚] we did that day. And 6 months turned into four and then two and then one. The people of Ted decided to [dɪ sʌɪdɪdtʊ] release the speakers. And I opened up the website and there was my face staring right back at me, and guess who woke up.
Video 2
Speaker A: Missy invited me to see a movie with her [wɪð ər].
Speaker B: I think she’s aware of that, Lucas. No need to [niːtʊ] make her feel worse.
Speaker A: And I was wondering if maybe you guys would like to [wʊ lʌɪk tə] come along with us [wɪ ðʌs].
Speaker C: You want us?
Speaker B: I’m sorry, Lucas. That wasn’t the invitation.
Speaker A: Then I’m sorry, Missy. But I can’t go.
Speaker B: What are you talking about? Nobody’s ever turned me down in my life.
Speaker A: We’ll see. These are my friends. And I don’t like [ʌɪ dɒnt̚lʌɪk̚] doing anything without my friends. Right? Riley. I certainly appreciate you want me to take care of [ʌɪ dɒnt̚lʌɪk̚] you during a scary movie and you showing me your leg and all [lɛɡənd̥ ɔːl]. But back here in the 7th grade, I think maybe we’d have more fun just [dʒəst̚] hanging out together.
Speaker B: Grow up!
Speaker C: Not yet!
Video 3
Speaker A: I have some [səm] good news. We’ve been [wivbɪ̥n] spending the last several weeks studying careers.
Speaker B: Are we?
Speaker A: So I’m pleased to announce that several of your parents, Murffy’s father, Brien’s mother, and yes, Francis’ father have volunteered to show us [tʊ ʃəʊəs] around their places of work, which means in short …
Speaker B: Field trip!
Speaker C: A field trip with dad? Are you out of your mind?
Speaker D: What’s wrong with that?
Speaker C: Francine, don’t you realize what this means? Have you no pride?
Speaker D: Way to go [weɪtəɡəʊ]! Dad
Speaker E: Come on down. You’re missing all the fun [ɔːlð̥ə̥fʌn].
Speaker C: Do you really want your [wɒntjʊr] friends to know that our father is there? A garbage Man?
Video 4
Speaker A: Oh Candice. All the times I called you delusional and mocked you to [mɒkt͡ʃuːtʊ] my friends behind your back. All those journals I filled with an eye [wɪðn ʌɪ] toward a standup comedy, but you were telling the truth, I’m so sorry. You’re [jʊər] the best daughter any mother could [kəd] dream of having.
Speaker B: Finally, you realize that.
Speaker C: What’s all this [wɑtsɔːl ðɪ̥s] yelling about? My word. That is impressive!
Speaker A: Did you know about this?
Speaker C: No.
Speaker B: What are you gonna do to them? No TV for a month? Or they have to serve me for a year! How about confiscating their toolbox? I know, take away their seats on the city council.
Speaker A: You have seats on the city council?
Speaker E: We rotate out [rə teɪtaʊ]with the border selectman.
Speaker C: Well some punishment may be in order, but look, there’s no harm done, really Ferb.
Speaker E: Here’s bolt number 473. I hope that wasn’t important. Well, at least [atliːst̚] the self-parking works.
Speaker A: I’m sorry, but this is really for your own good. We need you both to understand that what you’ve been doing is way too dangerous. I’d never be able to live with myself if you got hurt. E: We were wearing helmets.
References
Ai, Mingyao, Kang Li, Senmao Liu & Dennis K. J. Lin. 2013. Balanced incomplete Latin square designs. Journal of Statistical Planning and Inference 143(9). 1575–1582. https://doi.org/10.1016/j.jspi.2013.05.001.Search in Google Scholar
Alameen, Ghinwa & John M. Levis. 2015. Connected speech. In Marnie Reed & John M. Levis (eds.), The handbook of English pronunciation, 159–174. Hoboken, NJ: Wiley Blackwell.10.1002/9781118346952.ch9Search in Google Scholar
Bates, Douglas, Martin Maechler, Ben Bolker & Steve Walker. 2015. Fitting linear mixed-effects models using lme4. Journal of Statistical Software 67(1). 1–48. https://doi.org/10.18637/jss.v067.i01.Search in Google Scholar
Bird, Stephen A. & John N. Williams. 2002. The effect of bimodal input on implicit and explicit memory: An investigation into the benefits of within-language subtitling. Applied Psycholinguistics 23(4). 509–533. https://doi.org/10.1017/S0142716402004022.Search in Google Scholar
Bryła-Cruz, Agnieszka. 2021. The gender factor in the perception of English segments by non-native speakers. Studies in Second Language Learning and Teaching 11(1). 103–131. https://doi.org/10.14746/ssllt.2021.11.1.5.Search in Google Scholar
Ceuleers, Dorien, Ingeborg Dhooge, Sofie Degeest, Hanneleen Van Steen, Hannah Keppler & Nele Baudonck. 2021. The effects of age, gender and test stimuli on visual speech perception: A preliminary study. Folia Phoniatrica et Logopaedica 74(2). 131–140. https://doi.org/10.1159/000518205.Search in Google Scholar
Chen, Ying Yu, Yu Sheng Chang, Jia Ying Lee & Ming Huei Lin. 2021. Effects of a video featuring connected speech instruction on EFL undergraduates in Taiwan. SAGE Open 11(2). 1–12. https://doi.org/10.1177/21582440211019746.Search in Google Scholar
Cunnings, Ian. 2012. An overview of mixed-effects statistical models for second language researchers. Second Language Research 28. 369–382. https://doi.org/10.1177/0267658312443651.Search in Google Scholar
Davies, Mark. 2004. British National Corpus (from Oxford University Press). Available at: http://www.english-corpora.org/bnc/.Search in Google Scholar
Davies, Mark. 2008. The Corpus of Contemporary American English (COCA). Available at: https://www.english-corpora.org/coca/.Search in Google Scholar
Dixon, Peter. 2008. Models of accuracy in repeated-measures designs. Journal of Memory and Language 59(4). 447–456. https://doi.org/10.1016/j.jml.2007.11.004.Search in Google Scholar
Duanmu, San. 2007. The phonology of standard Chinese. New York: Oxford University Press.10.1093/oso/9780199215782.001.0001Search in Google Scholar
Field, John. 2008. Bricks or mortar: Which parts of the input does a second language listener rely on? TESOL Quarterly 42(3). 411–432. https://doi.org/10.1002/j.1545-7249.2008.tb00139.x.Search in Google Scholar
Gass, Susan, Paula Winke, Daniel R. Isbell & Jieun Ahn. 2019. How captions help people learn languages: A working-memory, eye-tracking study. Language Learning and Technology 23(2). 84–104.Search in Google Scholar
Guichon, Nicolas & Sinead McLornan. 2008. The effects of multimodality on L2 learners: Implications for CALL resource design. System 36(1). 85–93. https://doi.org/10.1016/j.system.2007.11.005.Search in Google Scholar
Hayati, Abdolmajid & Firooz Mohmedi. 2011. The effect of films with and without subtitles on listening comprehension of EFL learners. British Journal of Educational Technology 42(1). 181–192. https://doi.org/10.1111/j.1467-8535.2009.01004.x.Search in Google Scholar
Horwitz, Elaine K., Michael B. Horwitz & Joann Cope. 1986. Foreign language classroom anxiety. The Modern Language Journal 70(2). 125–132. https://doi.org/10.1111/j.1540-4781.1986.tb05256.x.Search in Google Scholar
Huang, Jinyan. 2006. English abilities for academic listening: How confident are Chinese students? College Student Journal 40(1). 218–226.Search in Google Scholar
Jaeger, Tim Florian. 2008. Categorical data analysis: Away from ANOVAs (transformation or not) and towards logit mixed models. Journal of Memory and Language 59(4). 434–446. https://doi.org/10.1016/j.jml.2007.11.007.Search in Google Scholar
Jin, Yan, Wei Jie & Wei Wang. 2022. Daxue Yingyu Si, Liuji Kaoshi yu Yuyan Nengli Biaozhun de Duijie Yanjiu [Exploring the alignment between the College English Test and language standards]. Waiyu Jie [Foreign Language World] 209(2). 24–32.Search in Google Scholar
Jin, Jin & Huizhong Yang. 2006. The English proficiency of college and university students in China: As reflected in the CET. Language, Culture and Curriculum 19(1). 21–36. https://doi.org/10.1080/07908310608668752.Search in Google Scholar
Khoshsaligheh, Masood, Saeed Ameri & Farzaneh Shokoohmand. 2019. Amateur subtitling in a dubbing country: The reception of Iranian audience. Observatorio (OBS*) Journal 13. 71–94. https://doi.org/10.15847/obsOBS13320191439.Search in Google Scholar
Koolstra, Cees M. & Jonannes W. J. Beentjes. 1999. Children’s vocabulary acquisition in a foreign language through watching subtitled television programs at home. Educational Technology Research and Development 47. 51–60. https://doi.org/10.1007/bf02299476.Search in Google Scholar
Lenth, Russell V. 2016. Least-squares means: The R package lsmeans. Journal of Statistical Software 69(1). 1–33. https://doi.org/10.18637/jss.v069.i01.Search in Google Scholar
Leveridge, Aubrey Neil & Jie Chi Yang. 2013. Testing learner reliance on caption supports in second language listening comprehension multimedia environments. ReCALL 25(2). 199–214. https://doi.org/10.1017/S0958344013000074.Search in Google Scholar
Liang, Danxin. 2015. Chinese learners’ pronunciation problems and listening difficulties in English connected speech. Asian Social Science 11(16). 98–106. https://doi.org/10.5539/ass.v11n16p98.Search in Google Scholar
Lwo, Laurence & Michelle Chia-Tzu Lin. 2012. The effects of captions in teenagers’ multimedia L2 learning. ReCALL 24(2). 188–208. https://doi.org/10.1017/S0958344012000067.Search in Google Scholar
Mao, Huai Zhou & Hua Ying Chen. 2013. Exploring elision of schwa of /ə/ in English utterances by C & U English majors. International Journal of Applied Linguistics and English Literature 2(1). 117–125. https://doi.org/10.7575/ijalel.v.2n.1p.117.Search in Google Scholar
Markham, Paul L., Lizette A. Peter & Teresa J. McCarthy. 2001. The effects of native language vs. target language captions on foreign language students’ DVD video comprehension. Foreign Language Annals 34(5). 439–445. https://doi.org/10.1111/j.1944-9720.2001.tb02083.x.Search in Google Scholar
Mayer, Richard E. 2005. The Cambridge handbook of multimedia learning. (Ed.) Richard E. Mayer. Cambridge: Cambridge University Press.10.1017/CBO9780511816819Search in Google Scholar
Montero Perez, Maribel, Elke Peters & Piet Desmet. 2014. Is less more? Effectiveness and perceived usefulness of keyword and full captioned video for L2 listening comprehension. ReCALL 26(1). 21–43. https://doi.org/10.1017/S0958344013000256.Search in Google Scholar
Montero Perez, Maribel, Wim Van Den Noortgate & Pietw Desmet. 2013. Captioned video for L2 listening and vocabulary learning: A meta-analysis. System 41(3). 720–739. https://doi.org/10.1016/j.system.2013.07.013.Search in Google Scholar
Moyer, Alene. 2016. The puzzle of gender effects in phonology. Journal of Second Language Pronunciation 2(1). 8–28. https://doi.org/10.1075/jslp.2.1.01moy.Search in Google Scholar
Paivio, Allan. 1986. Metal representations: A dual coding approach. Oxford England: Oxford University Press.Search in Google Scholar
Paivio, Allan. 2007. Mind and its evolution: A dual coding theoretical approach. Mahwah, NJ: Erlbaum.Search in Google Scholar
Pavlenko, Aneta & Ingrid Piller. 2008. Language education and gender. In Stephen May (ed.), The encyclopedia of language and education, 12nd edn. 57–69. New York, NY: Springer.10.1007/978-0-387-30424-3_5Search in Google Scholar
Pujolà, Joan Tomàs. 2002. CALLing for help: Researching language learning strategies using help facilities in a web-based multimedia program. ReCALL 14(2). 235–262. https://doi.org/10.1017/S0958344002000423.Search in Google Scholar
Quené, Hugo & Huub van den Bergh. 2008. Examples of mixed-effects modelling with crossed random effects and with binomial data. Journal of Memory and Language 59. 413–425. https://doi.org/10.1016/j.jml.2008.02.002.Search in Google Scholar
R Core Team. 2015. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.Search in Google Scholar
Roach, Peter. 2009. English phonetics and phonology: A practical course. New York: Cambridge University Press.Search in Google Scholar
Rodgers, Michael P. H. & Stuart Webb. 2017. The effects of captions on EFL learners’ comprehension of English-language television programs. CALICO Journal 34(1). 20–38. https://doi.org/10.1558/cj.29522.Search in Google Scholar
Setter, Jane, Peggy Mok, Ee Low, Donghui Zuo & Ran Ao. 2014. Word juncture characteristics in world Englishes: A research report. World Englishes 33(2). 278–291. https://doi.org/10.1111/weng.12085.Search in Google Scholar
Shevenock, Sarah. 2022. Global streaming: By the numbers. Morning Consult Pro. Available at: https://pro.morningconsult.com/trend-setters/subtitles-dubbing-streaming.Search in Google Scholar
Stoll, Julia. 2022. Audience preferences for subtitles or dubbing 2021, by country. Statista. Available at: https://www.statista.com/statistics/1289864/subtitles-dubbing-audience-preference-by-country/.Search in Google Scholar
Sweller, John, Paul Ayres & Slava Kalyuga. 2011. Cognitive Load Theory. New York, NY: Springer.10.1007/978-1-4419-8126-4Search in Google Scholar
Teng, Mark Feng. 2022. Incidental L2 vocabulary learning from viewing captioned videos: Effects of learner-related factors. System 105. 102736. https://doi.org/10.1016/j.system.2022.102736.Search in Google Scholar
Thompson, Irene & Joan Rubin. 1996. Can strategy instruction improve listening comprehension? Foreign Language Annals 29. 331–342. https://doi.org/10.1111/j.1944-9720.1996.tb01246.x.Search in Google Scholar
Vanderplank, Robert. 2016. Captioned media in foreign language learning and teaching: Subtitles for the deaf and hard-of-hearing as tools for language learning. London: Palgrave Macmillan.10.1057/978-1-137-50045-8Search in Google Scholar
Walker, Robin. 2010. Teaching the pronunciation of English as a Lingua Franca. Oxford; New York: Oxford University Press.Search in Google Scholar
Wang, Shuaiguo. 2017. Rain classroom: The wisdom teaching tool in the context of mobile internet and big data. Modern Educational Technology 27(5). 26–32.Search in Google Scholar
Wang, Xiuli & Shuiyi Shen. 2020. Effects of intelligent teaching tools in colleges and universities: A case study of “Rain Classroom”. Journal of Beijing University of Aeronautics and Astronautics 33(3). 126–132.Search in Google Scholar
Winke, Paula, Susan Gass & Tetyana Sydorenko. 2010. The effects of captioning videos used for foreign langauge listening activities. Language Learning and Technology 14(1). 65–86.Search in Google Scholar
Wong, Simpson W. L., Jessica Dealey, Vina W. H. Leung & Peggy P. K. Mok. 2021a. Production of English connected speech processes: An assessment of Cantonese ESL learners’ difficulties obtaining native-like speech. Language Learning Journal 49(5). 581–596. https://doi.org/10.1080/09571736.2019.1642372.Search in Google Scholar
Wong, Simpson W. L., Vina W. H. Leung, Jenny K. Y. Tsui, Jessica Dealey & Anisa Cheung. 2021b. Chinese ESL learners’ perceptual errors of English connected speech: Insights into listening comprehension. System 98. 102480. https://doi.org/10.1016/j.system.2021.102480.Search in Google Scholar
Wong, Simpson W. L., Cherry C. Y. Lin, Isabella S. Y. Wong & Anisa Cheung. 2020. The differential effects of subtitles on the comprehension of native English connected speech varying in types and word familiarity. SAGE Open 10(2). 1–13. https://doi.org/10.1177/2158244020924378.Search in Google Scholar
Yang, Jie Chi & Peichin Chang. 2014. Captions and reduced forms instruction: The impact on EFL students’ listening comprehension. ReCALL 26(1). 44–61. https://doi.org/10.1017/S0958344013000219.Search in Google Scholar
© 2024 Walter de Gruyter GmbH, Berlin/Boston
Articles in the same Issue
- Frontmatter
- Research Articles
- The role of place and manner of articulation in Kurtöp tonogenesis: refining the model
- Exploring and explaining variation in phrase-final f0 movements in spontaneous Papuan Malay
- The effects of watching subtitled videos on the perception of L2 connected speech by L1 Chinese-L2 English speakers
Articles in the same Issue
- Frontmatter
- Research Articles
- The role of place and manner of articulation in Kurtöp tonogenesis: refining the model
- Exploring and explaining variation in phrase-final f0 movements in spontaneous Papuan Malay
- The effects of watching subtitled videos on the perception of L2 connected speech by L1 Chinese-L2 English speakers
