Are L2 listener verbal reports reactive? Insights from the learners
-
Michael Yeldham
und
Rainbow Tsai-Hung Chen
Abstract
This study examined L2 listener verbal reports (VRs) for reactivity, in terms of the VR possibly impacting participants’ listening comprehension. The research built on two prior studies. The first was a qualitative investigation, where the study’s three participants had each undertaken VRs using three different VR styles: two mediated (employing researcher question prompts), and one unmediated. In that study the participants perceived there was reactivity in various aspects of the VR. However, the second study, an experimental investigation, utilizing these same three VR styles, found no evidence of reactivity. The current study sought, in part, to address these conflicting findings, while finding out more about potential reactivity in L2 listener VRs. To accomplish this, we gathered data through immediate post-VR interviews with all 14 learners in the experimental group from the aforementioned experimental study, to explore their views on reactivity: (1) for the VRs, in general; (2) on the researcher regularly stopping the text for their report; (3) for mediated VRs, and (4) for the three VR styles. The study found the VRs largely did not appear to be reactive, although participants did perceive reactivity in each area investigated. Based on the results, we provide advice for L2 listener VRs.
Learners’ calibrated (scaled, percentage) scores for mediated and unmediated VRs.
| Learner | Mediated VR (%)a | Unmediated VR (%) |
|---|---|---|
| Sal | 29.46 b | 29.23 |
| Kev | 51.84 | 27.11 |
| Dot | 71.18 | 67.97 |
| Wen | 54.19 | 44.84 |
| Cate | 46.69 | 50.90 |
| Sam | 39.42 | 64.43 |
| Kim | 58.64 | 75.27 |
| Nina | 42.92 | 34.83 |
| Dave | 34.21 | 39.46 |
| Di | 70.51 | 61.29 |
| Bev | 59.82 | 58.85 |
| Al | 65.81 | 59.04 |
| Liz | 55.51 | 36.93 |
| Rob | 66.93 | 70.47 |
-
aAverage of mediated styles 1 and 2. bEach learner’s higher score shown bold, underlined.
Learners’ calibrated (scaled, percentage) scores for the 3 mediation styles.
| Learner | Mediated style 1 (%) | Mediated style 2 (%) | Unmediated style 3 (%) |
|---|---|---|---|
| Sal | 21.99 | 36.93 | 29.23 |
| Kev | 43.61 | 60.06 | 27.11 |
| Dot | 68.86 | 73.49 | 67.97 |
| Wen | 55.72 a | 52.65 | 44.84 |
| Cate | 37.32 | 56.05 | 50.90 |
| Sam | 30.03 | 48.80 | 64.43 |
| Kim | 58.73 | 58.54 | 75.27 |
| Nina | 41.25 | 44.58 | 34.83 |
| Dave | 23.62 | 44.79 | 39.46 |
| Di | 60.54 | 80.47 | 61.29 |
| Bev | 60.90 | 58.73 | 58.85 |
| Al | 62.46 | 69.15 | 59.04 |
| Liz | 67.77 | 43.24 | 36.93 |
| Rob | 62.47 | 71.39 | 70.47 |
-
aEach learner’s highest score shown bold, underlined.
References
Al-Maani, Alaa, Bara’ah AlAbabneh, Bassil Mashaqba & Anas Huneety. 2024. Investigating second language learning strategies using think aloud protocols: evidence from Jordanian EFL learners. Eurasian Journal of Applied Linguistics 10(2). 12–22.Suche in Google Scholar
Bacon, Susan M. 1992. Authentic listening in Spanish: how learners adjust their strategies to the difficulty of the input. Hispania 75(2). 398–412. https://doi.org/10.2307/344077.Suche in Google Scholar
Bowles, Melissa A. 2010. The think-aloud controversy in second language research. New York: Routledge.10.4324/9780203856338Suche in Google Scholar
Bowles, Melissa A. 2019. Verbal reports in instructed SLA research. In Ronald P. Leow (ed.), The Routledge handbook of second language research in classroom learning, 31–43. New York: Routledge.10.4324/9781315165080-3Suche in Google Scholar
Brunfaut, Tineke & Luke Harding. 2014. Linking the GEPT listening test to the Common European Framework of Reference. LTTC-GEPT Research Reports RG-05. LTTC, Taiwan.Suche in Google Scholar
Buck, Gary. 1990. The testing of second language listening comprehension. Lancaster, UK: University of Lancaster. Unpublished doctoral dissertation.Suche in Google Scholar
Bulut, Berker & İhsan S. Ertem. 2018. A think-aloud study: listening comprehension strategies used by primary school students. Journal of Education and Training Studies 6(5). 135–143. https://doi.org/10.11114/jets.v6i5.3050.Suche in Google Scholar
Cohen, Andrew D. 1998. Strategies in learning and using a second language. Harlow, UK: Longman.Suche in Google Scholar
Cohen, Andrew D. 2011. Strategies in learning and using a second language, 2nd edn. Abingdon, UK: Routledge/Pearson Education.Suche in Google Scholar
Cohen, Andrew D. 2023. System’s 50th anniversary special issue in conversation with Andrew D. Cohen. System 118. Article 103153. https://doi.org/10.1016/j.system.2023.103153.Suche in Google Scholar
Cowan, John. 2019. The potential of cognitive think-aloud protocols for educational action-research. Active Learning in Higher Education 20(3), 219–232, https://doi.org/10.1177/1469787417735614.Suche in Google Scholar
Creswell, John W. & Vicki L. Plano Clark. 2018. Designing and conducting mixed methods research, 3rd edn. Thousand Oaks: SAGE Publishing.Suche in Google Scholar
Deschambault, Ryan. 2018. Actively managed products: think-aloud data and methods in applied linguistics research. Applied Linguistics Review 9(4). 539–562. https://doi.org/10.1515/applirev-2017-0028.Suche in Google Scholar
Ericsson, K. Anders & Herbert A. Simon. 1984. Protocol analysis: verbal reports as data. Cambridge, MA: MIT Press.Suche in Google Scholar
Ericsson, K. Anders & Herbert A. Simon. 1993. Protocol analysis: verbal reports as data, Revised ed. Cambridge, MA: MIT Press.10.7551/mitpress/5657.001.0001Suche in Google Scholar
Foster, Pauline, Alan Tonkin & Gillian Wigglesworth. 2000. Measuring spoken language: a unit for all reasons. Applied Linguistics 21(3). 345–375.10.1093/applin/21.3.354Suche in Google Scholar
Goh, Christine C. M. 1998. How ESL learners with different listening abilities use comprehension strategies and tactics. Language Teaching Research 2(2). 124–147.10.1191/136216898667461574Suche in Google Scholar
Goh, Christine C. M. 2002. Exploring listening comprehension tactics and their interaction patterns. System 30(2). 185–206. https://doi.org/10.1016/s0346-251x-02-00004-0.Suche in Google Scholar
Goh, Christine C. M. 2023. Learners’ cognitive processing problems during comprehension as a basis for L2 listening research. System 119. Article 103164. https://doi.org/10.1016/j.system.2023.103164.Suche in Google Scholar
Goh, Christine C. M. & Larry Vandergrift. 2022. Teaching and learning second language listening, 2nd edn. New York: Routledge.10.4324/9780429287749Suche in Google Scholar
Goo, Jaemyung. 2010. Working memory and reactivity. Language Learning 60(4). 712–752. https://doi.org/10.1111/j.1467-9922.2010.00573.x.Suche in Google Scholar
Graham, Suzanne. 1997. Effective language learning: Positive strategies for advanced level language learning. Clevedon, UK: Multilingual Matters.Suche in Google Scholar
Graham, Suzanne, Denise Santos & Robert Vanderplank. 2011. Exploring the relationship between listening development and strategy use. Language Teaching Research 15(4). 435–456. https://doi.org/10.1177/1362168811412026.Suche in Google Scholar
Green, Alison. 1998. Verbal protocol analysis in language testing research: a handbook. Cambridge: Cambridge University Press.Suche in Google Scholar
Gu, Peter Y., Guangwei Hu & Lawrence J. Zhang. 2009. Listening strategies of Singaporean primary pupils. In Rita E. Silver, Christine C. M. Goh & Lubna Alsagoff (eds.), Language learning in new English contexts: studies of acquisition and development, 55–74. London: Continuum.Suche in Google Scholar
Jourdenais, Renée. 2001. Cognition, instruction and protocol analysis. In Peter Robinson (ed.), Cognition and second language instruction, 354–375. Cambridge: Cambridge University Press.10.1017/CBO9781139524780.014Suche in Google Scholar
Kazemi, Ali & Soraya Kiamarsi. 2017. An investigation into listening comprehension strategies and the relationship between listening comprehension strategies and overall proficiency level of intermediate and advanced learners. Journal of Language Teaching and Research 8(1). 149–156. https://doi.org/10.17507/jltr.0801.18.Suche in Google Scholar
Kim, Heejung S. 2002. We talk, therefore we think? A cultural analysis of the effect of talking on thinking. Journal of Personality and Social Psychology 83. 828–842. https://doi.org/10.1037//0022-3514.83.4.828.Suche in Google Scholar
Laviosa, Flavia. 2000. The listening comprehension processes and strategies of learners of Italian: a case study. Rassegna Italiana Di Linguistica Applicata 32(2). 129–159.Suche in Google Scholar
Leow, Ronald P. 2015. Explicit learning in the L2 classroom: a student-centered approach. New York: Routledge.10.4324/9781315887074Suche in Google Scholar
Leow, Ronald P., Sarah Grey, Silvia Marijuan & Colleen Moorman. 2014. Concurrent data elicitation procedures, processes, and the early stages of L2 learning: a critical overview. Second Language Research 30(2). 111–127. https://doi.org/10.1177/0267658313511979.Suche in Google Scholar
Li, Hongli & Hoi K. Suen. 2015. How do Chinese ESL learners recognize English words during a reading test? A comparison with romance-language-speaking ESL learners. International Multilingual Research Journal 9(2). 93–107. https://doi.org/10.1080/19313152.2014.995013.Suche in Google Scholar
Lopez, Marta N. 2017. Application, results and perceptions of a think-aloud study in listening comprehension of Spanish. International Journal of Applied Linguistics 168(2). 164–182.10.1075/itl.17007.nogSuche in Google Scholar
Macaro, Ernesto, Suzanne Graham & Robert Vanderplank. 2007. A review of listening strategies: focus on sources of knowledge and on success. In Andrew D. Cohen & Ernesto Macaro (eds.), Language learner strategies, 165–186. Oxford University Press.Suche in Google Scholar
Mackey, Alison & Susan M. Gass. 2021. Second language research: methodology and design, 3rd edn. New York: Routledge.10.4324/9781003188414Suche in Google Scholar
Miles, Matthew B. & A. Michael Huberman. 1994. Qualitative data analysis: an expanded sourcebook, 2nd edn. Thousand Oaks: SAGE Publications.Suche in Google Scholar
Murphy, John M. 1987. The listening strategies of English as a second language college students. Research and Teaching in Developmental Education 4(1). 27–46.Suche in Google Scholar
Patton, Michael Q. 2015. Qualitative research and evaluation methods: Integrating theory and practice, 4th edn. Thousand Oaks: SAGE Publications.Suche in Google Scholar
Peng, Jian-E. 2024. Willingness to communicate in a second language. Cambridge: Cambridge University Press.10.1017/9781009417884Suche in Google Scholar
Pressley, Michael & Peter Afflerbach. 1995. Verbal protocols of reading: The nature of constructively responsive reading. Hillsdale, NJ: Lawrence Erlbaum Associates.10.2307/358808Suche in Google Scholar
Roever, Carsten & Yi-Ching Pan. 2008. Test reviews. GEPT: general English proficiency test. Language Testing 25(3). 403–418.10.1177/0265532208090159Suche in Google Scholar
Santos, Denise, Suzanne Graham & Robert Vanderplank. 2008. Second language listening strategy research: methodological challenges and perspectives. Evaluation & Research in Education 21(2). 111–133. https://doi.org/10.1080/09500790802152183.Suche in Google Scholar
Sasaki, Tomomi. 2008. Concurrent think-aloud protocol as a socially situated construct. International Review of Applied Linguistics 46(4). 349–374. https://doi.org/10.1515/iral.2008.015.Suche in Google Scholar
Seliger, Herbert W. 1983. The language learner as linguist: of metaphors and realities. Applied Linguistics 4(3). 179–191. https://doi.org/10.1093/applin/4.3.179.Suche in Google Scholar
Smagorinsky, Peter. 2001. Rethinking protocol analysis from a cultural perspective. Annual Review of Applied Linguistics 21. 233–245. https://doi.org/10.1017/s0267190501000149.Suche in Google Scholar
Smith, Patriann, Deoksoon Kim, Oksana Vorobel & James R. King. 2020. Verbal reports in the reading processes of language learners: a methodological review. The Review of Education 8(1). 37–114. https://doi.org/10.1002/rev3.3170.Suche in Google Scholar
Suh, Bo-Ram. 2023. Are think-alouds reactive? Evidence from an L2 written corrective feedback study. Language Teaching Research 27(5). 1099–1119. https://doi.org/10.1177/1362168820967166.Suche in Google Scholar
Vandergrift, Larry. 1998. Successful and less successful listeners in French: what are the strategy differences? French Review 71(3). 370–395.Suche in Google Scholar
Vandergrift, Larry. 2003. Orchestrating strategy use: toward a model of the skilled second language listener. Language Learning 53(3). 463–496. https://doi.org/10.1111/1467-9922.00232.Suche in Google Scholar
Wagner, Matthew. 2006. Utilizing the visual channel: An investigation of the use of video texts on tests of second language listening ability. New York: Teachers College, Columbia University. Unpublished doctoral dissertation.Suche in Google Scholar
Yeldham, Michael. 2009. Approaches to second language listening instruction: investigating the ‘top-down/bottom-up debate. Melbourne, Australia: University of Melbourne. Unpublished doctoral dissertation.Suche in Google Scholar
Yeldham, Michael & Rainbow T.-H. Chen. 2014. Conducting verbal reports to study Chinese-speaking English learners’ listening: the use of prompts. The European Journal of Applied Linguistics and TEFL 3(1). 55–74.Suche in Google Scholar
Yeldham, Michael & Rainbow T.-H. Chen. 2016. Investigating mediation styles of second language listener verbal reports. Applied Linguistics Review 7(2). 203–233. https://doi.org/10.1515/applirev-2016-0006.Suche in Google Scholar
Yeldham, Michael & Rainbow T.-H. Chen. 2025. Investigating reactivity in L2 listener verbal reports. AILA Review 38(1). 66–104. https://doi.org/10.1075/aila.24035.yel (Epub ahead of print).Suche in Google Scholar
Yeldham, Michael & Paul Gruba. 2016. The development of individual learners in an L2 listening strategies course. Language Teaching Research 20(1). 9–34. https://doi.org/10.1177/1362168814541723.Suche in Google Scholar
Zhang, Lawrence J. & Donglan Zhang. 2020. Think-aloud protocols. In Jim McKinley & Heath Rose (eds.), The Routledge handbook of research methods in applied linguistics, 302–311. London: Routledge.10.4324/9780367824471-26Suche in Google Scholar
Supplementary Material
This article contains supplementary material (https://doi.org/10.1515/iral-2024-0262).
© 2025 Walter de Gruyter GmbH, Berlin/Boston
