Connections between measured and assessed fluency in L2 peer interaction: a problem-solving perspective
-
Pauliina Peltonen
Abstract
This article examines the connections between ratings and measures of second language (L2) fluency based on dialogue data, extending previous studies on L2 individual fluency. The links between fluency and strategic competence are studied from a problem-solving perspective. Fifteen raters’ assessments and comments of individual fluency, interactional fluency, and strategic competence based on six samples of L2 peer interaction were examined. Fluency measures corresponded to the rated dimensions. In a mixed-methods analysis, correlational analyses across the ratings and between the ratings and fluency measures were performed. The raters’ comments were analyzed qualitatively. The results demonstrated correlations between the ratings and measures of fluency for individual and interactional fluency, but not for strategic competence. The mean length of turn pauses correlated strongly with interactional fluency ratings. The raters’ comments revealed a multifaceted conceptualization of interactional fluency and strategic competence. The results have implications for L2 fluency measurement and assessment.
Appendix 1 Rating sheet
The instructions were translated by the author from the Finnish originals. Only the rating sheet for sample 1 is presented here; the sheets for other samples (2–6) were identical.
Instructions: Rate the six dialogue samples numerically according to the four criteria. Comment on your ratings for fluency, interactional fluency, and strategic competence in the boxes below the numeric scales. For oral proficiency, the numeric rating is enough.
Sample 1 (01-G-004-005)
I) Oral proficiency
Rate the oral proficiency of the speaker on the left on a scale from 1 to 9 (1 = extremely weak oral proficiency, 9 = extremely good oral proficiency). Mark the number in bold.
1 2 3 4 5 6 7 8 9
Rate the oral proficiency of the speaker on the right on a scale from 1 to 9 (1 = extremely weak oral proficiency, 9 = extremely good oral proficiency). Mark the number in bold.
1 2 3 4 5 6 7 8 9
II) Fluency
When rating fluency, pay attention to the temporal aspects of speech: speech rate, pausing (silent pauses and filled pauses, e.g. um) and repair (for instance reformulations, repetitions). Accuracy and complexity should not be a part of fluency ratings.
Rate the fluency of the speaker on the left on a scale from 1 to 9 (1 = extremely disfluent, 9 = extremely fluent). Mark the number in bold.
1 2 3 4 5 6 7 8 9
Rate the fluency of the speaker on the right on a scale from 1 to 9 (1 = extremely disfluent, 9 = extremely fluent). Mark the number in bold.
1 2 3 4 5 6 7 8 9
Which aspects did you pay attention to when rating the sample? Comment on and justify the ratings in the box below (note that you can enlarge the box if needed). The ratings should be based on concrete features of speech.
III) Interactional fluency
Rate the pair’s interactional fluency on a scale from 1 to 9 (1 = extremely disfluent, 9 = extremely fluent). Mark the number in bold.
1 2 3 4 5 6 7 8 9
Which aspects did you pay attention to when rating the sample? Comment on and justify the rating in the box below. The rating should be based on concrete features of speech.
IV) Strategic competence
When rating strategic competence, pay particular attention to the use of communication strategies (for instance paraphrases).
Rate the strategic competence of the speaker on the left on a scale from 1 to 9 (1 = extremely weak strategic competence, 9 = extremely good strategic competence). Mark the number in bold.
1 2 3 4 5 6 7 8 9
Rate the strategic competence of the speaker on the right on a scale from 1 to 9 (1 = extremely weak strategic competence, 9 = extremely good strategic competence). Mark the number in bold.
1 2 3 4 5 6 7 8 9
Which aspects did you pay attention to when rating the sample? Comment on and justify the ratings in the box below. The ratings should be based on concrete features of speech.
References
Bachman, Lyle F. & Adrian S. Palmer. 1996. Language testing in practice. Oxford: Oxford University Press.Search in Google Scholar
Boersma, Paul & David Weenink. 2015. Praat: Doing phonetics by computer. Version 6.0.07, http://www.praat.org/.Search in Google Scholar
Borger, Linda. 2019. Assessing interactional skills in a paired speaking test: Raters’ interpretation of the construct. Apples – Journal of Applied Language Studies 13(1). 151–174.10.17011/apples/urn.201903011694Search in Google Scholar
Bosker, Hans Rutger, Anne-France Pinget, Hugo Quené, Ted Sanders & Nivja H. De Jong. 2013. What makes speech sound fluent? The contributions of pauses, speed and repairs. Language Testing 30(2). 159–175.10.1177/0265532212455394Search in Google Scholar
Brand, Christiane & Sandra Götz. 2011. Fluency versus accuracy in advanced spoken learner language: A multi-method approach. International Journal of Corpus Linguistics 16(2). 255–275.10.1075/bct.52.05braSearch in Google Scholar
Chambers, Francine. 1997. What do we mean by fluency? System 25(4). 535–544.10.1016/S0346-251X(97)00046-8Search in Google Scholar
Council of Europe. 2001. Common European framework of reference for languages. Learning, teaching, assessment. Cambridge: Cambridge University Press.Search in Google Scholar
Cucchiarini, Catia, Helmer Strik & Lou Boves. 2002. Quantitative assessment of second language learners’ fluency: Comparisons between read and spontaneous speech. Journal of the Acoustical Society of America 111(6). 2862–2873.10.1121/1.1471894Search in Google Scholar
De Jong, Nivja H. 2018. Fluency in second language testing: Insights from different disciplines. Language Assessment Quarterly 15(3). 237–254.10.1080/15434303.2018.1477780Search in Google Scholar
De Jong, Nivja H. & Ton Wempe. 2009. Praat script to detect syllable nuclei and measure speech rate automatically. Behavior Research Methods 41(2). 385–390.10.3758/BRM.41.2.385Search in Google Scholar
Derwing, Tracey M., Marian J. Rossiter, Murray J. Munro & Ronald I. Thomson. 2004. Second language fluency: Judgments on different tasks. Language Learning 54(4). 655–679.10.1111/j.1467-9922.2004.00282.xSearch in Google Scholar
Dörnyei, Zoltán & Judit Kormos. 1998. Problem-solving mechanisms in L2 communication: A psycholinguistic perspective. Studies in Second Language Acquisition 20(3). 349–385.10.1017/S0272263198003039Search in Google Scholar
Dörnyei, Zoltán & Mary Lee Scott. 1997. Communication strategies in a second language: Definitions and taxonomies. Language Learning 47(1). 173–210.10.1111/0023-8333.51997005Search in Google Scholar
Ducasse, Ana Maria & Annie Brown. 2009. Assessing paired orals: Raters’ orientation to interaction. Language Testing 26(3). 423–443.10.1177/0265532209104669Search in Google Scholar
Freed, Barbara F. 1995. What makes us think that students who study abroad become fluent? In Barbara F. Freed (ed.), Second language acquisition in a study abroad context, 123–148. Amsterdam & Philadelphia: John Benjamins.10.1075/sibil.9.09freSearch in Google Scholar
Galaczi, Evelina D. 2014. Interactional competence across proficiency levels: How do learners manage interaction in paired speaking tests? Applied Linguistics 35(5). 553–574.10.1093/applin/amt017Search in Google Scholar
Galaczi, Evelina D. & Lynda Taylor. 2018. Interactional competence: Conceptualisations, operationalisations, and outstanding questions. Language Assessment Quarterly 15(3). 219–236.10.1080/15434303.2018.1453816Search in Google Scholar
Ginther, April, Slobodanka Dimova & Rui Yang. 2010. Conceptual and empirical relationships between temporal measures of fluency and oral English proficiency with implications for automated scoring. Language Testing 27(3). 379–399.10.1177/0265532210364407Search in Google Scholar
Götz, Sandra. 2013. Fluency in native and nonnative English speech. Amsterdam & Philadelphia: John Benjamins.10.1075/scl.53Search in Google Scholar
Housen, Alex, Folkert Kuiken & Ineke Vedder (eds.). 2012. Dimensions of L2 performance and proficiency: Complexity, accuracy and fluency in SLA. Amsterdam & Philadelphia: John Benjamins.10.1075/lllt.32Search in Google Scholar
Iwashita, Noriko, Annie Brown, Tim McNamara & Sally O’Hagan. 2008. Assessed levels of second language speaking proficiency: How distinct? Applied Linguistics 29(1). 24–49.10.1093/applin/amm017Search in Google Scholar
Jarvis, Scott. 2017. Grounding lexical diversity in human judgments. Language Testing 34(4). 537–553.10.1177/0265532217710632Search in Google Scholar
Kahng, Jimin. 2014. Exploring utterance and cognitive fluency of L1 and L2 English speakers: Temporal measures and stimulated recall. Language Learning 64(4). 809–854.10.1111/lang.12084Search in Google Scholar
Kormos, Judit & Mariann Dénes. 2004. Exploring measures and perceptions of fluency in the speech of second language learners. System 32(2). 145–164.10.1016/j.system.2004.01.001Search in Google Scholar
Lemhöfer, Kristin & Mirjam Broersma. 2012. Introducing LexTALE: A quick and valid lexical test for advanced learners of English. Behavior Research Methods 44. 325–343.10.3758/s13428-011-0146-0Search in Google Scholar
Lennon, Paul. 1990. Investigating fluency in EFL: A quantitative approach. Language Learning 40(3). 387–417.10.1111/j.1467-1770.1990.tb00669.xSearch in Google Scholar
Lennon, Paul. 2000. The lexical element in spoken second language fluency. In Heidi Riggenbach (ed.), Perspectives on fluency, 25–42. Ann Arbor, MI: University of Michigan.Search in Google Scholar
Lintunen, Pekka, Maarit Mutta & Pauliina Peltonen (eds.). 2020. Fluency in L2 learning and use. Bristol: Multilingual Matters.10.21832/9781788926317Search in Google Scholar
Magne, Viktoria, Shungo Suzuki, Yui Suzukida, Kazuya Saito, Meltem Ilkan & Mai Tran. 2019. Exploring the dynamic nature of second language listeners’ perceived fluency: A mixed-methods approach. TESOL Quarterly 53(4). 1139–1150.10.1002/tesq.528Search in Google Scholar
May, Lyn. 2009. Co-constructed interaction in a paired speaking test: The rater’s perspective. Language Testing 26 (3). 397–421.10.1177/0265532209104668Search in Google Scholar
McCarthy, Michael. 2010. Spoken fluency revisited. English Profile Journal 1(1). 1–15.10.1017/S2041536210000012Search in Google Scholar
Peltonen, Pauliina. 2017a. Temporal fluency and problem-solving in interaction: An exploratory study of fluency resources in L2 dialogue. System 70. 1–13.10.1016/j.system.2017.08.009Search in Google Scholar
Peltonen, Pauliina. 2017b. L2 fluency in spoken interaction: A case study on the use of other-repetitions and collaborative completions. In Mikko Kuronen, Pekka Lintunen & Tommi Nieminen (eds.), Näkökulmia toisen kielen puheeseen – Insights into second language speech, 118–138. Jyväskylä: Finnish Association of Applied Linguistics AFinLA.10.30660/afinla.73130Search in Google Scholar
Peltonen, Pauliina. 2020. Individual and interactional speech fluency in L2 English from a problem-solving perspective: A mixed-methods approach. Turku: University of Turku.Search in Google Scholar
Plonsky, Luke & Frederick L. Oswald. 2014. How big is “big”? Interpreting effect sizes in L2 research. Language Learning 64(4). 878–912.10.1111/lang.12079Search in Google Scholar
Préfontaine, Yvonne & Judit Kormos. 2016. A qualitative analysis of perceptions of fluency in second language French. International Review of Applied Linguistics in Language Teaching 54(2). 151–169.10.1515/iral-2016-9995Search in Google Scholar
Préfontaine, Yvonne, Judit Kormos & Daniel Ezra Johnson. 2016. How do utterance measures predict raters’ perceptions of fluency in French as a second language? Language Testing 33(1). 53–73.10.1177/0265532215579530Search in Google Scholar
Révész, Andrea, Monika Ekiert & Eivind Nessa Torgersen. 2016. The effects of complexity, accuracy, and fluency on communicative adequacy in oral task performance. Applied Linguistics 37(6). 828–848.Search in Google Scholar
Riggenbach, Heidi. 1991. Toward an understanding of fluency: A microanalysis of nonnative speaker conversations. Discourse Processes 14(4). 423–441.10.1080/01638539109544795Search in Google Scholar
Roever, Carsten & Gabriele Kasper. 2018. Speaking in turns and sequences: Interactional competence as a target construct in testing speaking. Language Testing 35(3). 331–355.10.1177/0265532218758128Search in Google Scholar
Rossiter, Marian J. 2009. Perceptions of L2 fluency by native and non-native speakers of English. The Canadian Modern Language Review 65(3). 395–412.10.3138/cmlr.65.3.395Search in Google Scholar
Salaberry, Rafael M. & Silvia Kunitz. 2019. Introduction. In Rafael M. Salaberry and Silvia Kunitz (eds.), Teaching and testing L2 interactional competence: Bridging theory and practice, 1–22. New York: Routledge.10.4324/9781315177021-1Search in Google Scholar
Sato, Masatoshi. 2014. Exploring the construct of interactional oral fluency: Second language acquisition and language testing approaches. System 45. 79–91.10.1016/j.system.2014.05.004Search in Google Scholar
Segalowitz, Norman. 2010. The cognitive bases of second language fluency. New York: Routledge.10.4324/9780203851357Search in Google Scholar
Skehan, Peter. 2009. Modelling second language performance: Integrating complexity, accuracy, fluency, and lexis. Applied Linguistics 30(4). 510–532.10.1093/applin/amp047Search in Google Scholar
Suzuki, Shungo & Judit Kormos. 2020. Linguistic dimensions of comprehensibility and perceived fluency: An investigation of complexity, accuracy, and fluency in second language argumentative speech. Studies in Second Language Acquisition 42(1). 143–167.10.1017/S0272263119000421Search in Google Scholar
Tavakoli, Parvaneh. 2016. Fluency in monologic and dialogic task performance: Challenges in defining and measuring L2 fluency. International Review of Applied Linguistics in Language Teaching 54(2). 133–150.10.1515/iral-2016-9994Search in Google Scholar
Taylor, Lynda & Gillian Wigglesworth. 2009. Are two heads better than one? Pair work in L2 assessment contexts. Language Testing 26(3). 325–339.10.1177/0265532209104665Search in Google Scholar
Van Batenburg, Eline S. L., Ron J. Oostdam, Amos J. S. Van Gelderen & Nivja H. De Jong. 2018. Measuring L2 speakers’ interactional ability using interactive speech tasks. Language Testing 35(1). 75–100.10.1177/0265532216679452Search in Google Scholar
Van Os, Marjolein, Nivja H. De Jong & Hans Rutger Bosker 2020. Fluency in dialogue: Turn-taking behavior shapes perceived fluency in native and non-native speech. Language Learning 70(4). 1183–1217.10.1111/lang.12416Search in Google Scholar
Witton-Davies, Giles. 2014. The study of fluency and its development in monologue and dialogue. Lancaster: Lancaster University dissertation.Search in Google Scholar
© 2021 Walter de Gruyter GmbH, Berlin/Boston
Articles in the same Issue
- Frontmatter
- Research Articles
- Contribution of English aural vocabulary size levels to L2 listening comprehension
- The effects of extensive reading on young Korean students’ construction development
- Connections between measured and assessed fluency in L2 peer interaction: a problem-solving perspective
- Transfer and unlearning of topic prominence by Chinese learners of English
- The influence of L1 script directionality and L2 proficiency on Hanzi learning among Arabic and English learners of L2 Chinese
- The effects of task types on L2 oral production and learner engagement
- Unlearning the boundary-crossing constraint: processing instruction and the acquisition of motion event construal
- Teaching South African Sign Language as a second language to university students: an integrated pedagogy
- Intra-language: the study of L2 morpheme productivity as within-item variance
- Advanced learners’ responses to Chinese greetings in study abroad
- Individual perceptions of group work environment, motivation, and achievement
- Fossilized mistakes in Spanish relative clauses learned by Chinese students
- Learning phrasal verbs in the EFL classroom: the effect of prior vocabulary knowledge and opacity
- L2 use of pragmatic markers in peer talk: Mandarin utterance-final particles
Articles in the same Issue
- Frontmatter
- Research Articles
- Contribution of English aural vocabulary size levels to L2 listening comprehension
- The effects of extensive reading on young Korean students’ construction development
- Connections between measured and assessed fluency in L2 peer interaction: a problem-solving perspective
- Transfer and unlearning of topic prominence by Chinese learners of English
- The influence of L1 script directionality and L2 proficiency on Hanzi learning among Arabic and English learners of L2 Chinese
- The effects of task types on L2 oral production and learner engagement
- Unlearning the boundary-crossing constraint: processing instruction and the acquisition of motion event construal
- Teaching South African Sign Language as a second language to university students: an integrated pedagogy
- Intra-language: the study of L2 morpheme productivity as within-item variance
- Advanced learners’ responses to Chinese greetings in study abroad
- Individual perceptions of group work environment, motivation, and achievement
- Fossilized mistakes in Spanish relative clauses learned by Chinese students
- Learning phrasal verbs in the EFL classroom: the effect of prior vocabulary knowledge and opacity
- L2 use of pragmatic markers in peer talk: Mandarin utterance-final particles
