Abstract
This study examines the co-occurrence of speech and gestures based on a multimodal corpus to investigate the extent to which speech-accompanying gestures differ between Chinese as L1 and English as L2 speech among speakers of different L2 proficiency levels. Thirty-two Chinese-speaking learners of English, equally distributed between advanced (C1) and low-intermediate (B1) proficiency levels were recruited. The face-to-face casual conversation in L1 and L2 among friends were video-recorded, and speech-accompanying gestures were then coded for different types: deictic gestures, iconic gestures, metaphoric gestures, beats, and others. The analysis of overall frequencies of L2 gestures, English proficiency level was found to have a significant effect; in particular, speakers at higher levels of L2 proficiency are more likely to produce more beats and iconic gestures in conversational interaction for additional emphasis, while less proficient L2 speakers tended to produce more deictic gestures, accompanied with other communication strategies. Furthermore, the comparison of the types of gestures that accompanied the speakers’ L1 and L2 conversation show that except metaphoric gestures, speakers tended to produce more beats, deictic and iconic gestures in L2 than in L1, with similar results found in both advanced and low-intermediate learners.
Funding statement: This work was supported by Ministry of Science and Technology, Taiwan, Funder Id: http://dx.doi.org/10.13039/501100004663, Grant Number: 105-2410-H-027-013
Appendix A: Transcription codes
Transcription convention | Symbol | Explanation |
speaker codes | <S1>,<S2>, etc. | TW and BT refer to Taiwanese and British participants respectively, and each speaker is numbered. |
Extralinguistic information | a square bracket ‘[]’ | [laughter], [coughing], [inaudible speech], etc. |
interrupted sentences | a plus ‘+’ | <BT18>: and water melon is a lot fresher+ |
<TW16>: Yeah. | ||
<BT18>: +than we have here. | ||
lengthened sounds | colons ‘:’ or ‘::’ | exceptionally long sounds (i. e. approximating 2 seconds or more) are marked with a double colon ‘::’ |
brief break | a sequence of two dots (.) | a longer pause is marked as a sequence of three dots (…) |
punctuation | . ?, | A full stop or question mark is used to mark the end of a sentence. A common indicates that the speaker has re-cast what he/she was saying. |
References
Adolphs, S. & R. Carter. 2013. Spoken corpus linguistics: From monomodal to multimodal. London: Routledge.10.4324/9780203526149Search in Google Scholar
Alibali, M., S. Kita & A. Young. 2000. Gesture and the process of speech production: We think, therefore we gesture. Language and Cognitive Processes 15. 593–613.10.1080/016909600750040571Search in Google Scholar
Allwood, J. 2008. Multimodal corpora. In Anke Lüdeling & Merja Kytö (Eds.), Corpus linguistics: An international handbook, 207–225. Berlin: Mouton de Gruyter.Search in Google Scholar
Berman, R. A. & D. I. Slobin. 1994. Relating events in narrative: A cross-linguistic developmental study. Hillsdale, NJ: Lawrence Erlbaum Associates.Search in Google Scholar
Brown, A. 2015. Universal development and L1–L2 convergence in bilingual construal of manner in speech and gesture in Mandarin, Japanese, and English. The Modern Language Journal 99. 66–82.10.1111/j.1540-4781.2015.12179.xSearch in Google Scholar
Brown, A. & M. Gullberg. 2013. L1-L2 convergence in clausal packaging in Japanese and English. Bilingualism: Language and Cognition 16. 477–494.10.1017/S1366728912000491Search in Google Scholar
Canale, M. 1983. From communicative competence to communicative language pedagogy. In J. C. Richards & R. W. Schmidt (eds.), Language and communication, 2–27. New York: Longman.Search in Google Scholar
Colletta, J. M., M. Guidetti, O. Capirci, C. Cristilli, O.E. Demir, R. Kunene-Nicolas & S. Levine. 2015. Effects of age and language on co-speech gesture production: An investigation of French, American, and Italian children’s narratives. Journal of Child Language 42(1). 122–145.10.1017/S0305000913000585Search in Google Scholar
Colletta, J. M., R. N. Kunene, A. Venouil, V. Kaufmann & J. P. Simon. 2009. Multi-track annotation of child language and gestures. In M Kipp, J.C Martin, P. Paggio & D. Heylen (eds.), Multimodal corpora: From models of natural interactions to systems and applications, 54–72). Berlin: Springer.10.1007/978-3-642-04793-0_4Search in Google Scholar
Danielsson, K. 2016. Modes and meaning in the classroom–The role of different semiotic resources to convey meaning in science classrooms. Linguistics and Education 35. 88–99.10.1016/j.linged.2016.07.005Search in Google Scholar
de Ruiter, J. P. 2000. The production of gesture and speech. In D. Mc-Neill (Ed.), Language and gesture. 284–311. Cambridge: Cambridge University Press.10.1017/CBO9780511620850.018Search in Google Scholar
Efron, D. 1972. Gesture, race, and culture. The Hague: Mouton.Search in Google Scholar
Fromkin, V. 2016. An introduction to language, 10th edn. CA: Content Technologies, Inc.Search in Google Scholar
Goldin-Meadow, S. 2005. Hearing gesture: How our hands help us think. Cambridge, MA: Harvard University Press.10.2307/j.ctv1w9m9dsSearch in Google Scholar
Gregersen, T., G. Olivares-Cuhat & J. Storm. 2009. An examination of L1 and L2 gesture use: What role does proficiency play?. The Modern Language Journal 93(2). 195–208.10.1111/j.1540-4781.2009.00856.xSearch in Google Scholar
Gullberg, M. 1998. Gesture as a communication strategy in second language discourse: A study of learners of French and Swedish. Lund, Sweden: Lund University Press.Search in Google Scholar
Gullberg, M. 1999. Communication strategies, gestures, and grammar. Acquisition Et Interaction En Langue Étrangère: Publiè Par l’Association ENCRAGES 2. 61–71.Search in Google Scholar
Gullberg, M. 2006. Some reasons for studying gesture and second language acquisition (Hommage à Adam Kendon). IRAL-International Review of Applied Linguistics in Language Teaching 44(2). 103–124.10.1515/IRAL.2006.004Search in Google Scholar
Hadar, U., R. Dar & A. Teitelman. 2001. Gesture during speech in first and second language: Implications for lexical retrieval. Gesture 1(2). 151–165.10.1075/gest.1.2.04hadSearch in Google Scholar
Holler, J. & K. Wilkin. 2009. Communicating common ground: How mutually shared knowledge influences speech and gesture in a narrative task. Language and Cognitive Processes 24(2). 267–289.10.4324/9781003059783-6Search in Google Scholar
Holler, J. & K. Wilkin. 2011. An experimental investigation of how addressee feedback affects co-speech gestures accompanying speakers’ responses. Journal of Pragmatics 43(14). 3522–3536.10.1016/j.pragma.2011.08.002Search in Google Scholar
Hussain, M. 2014. Cross linguistic variation in the gestures accompanying manner of motion event descriptions by native speakers of English and Urdu. Balochistan Journal of Linguistics 2. 139–167.Search in Google Scholar
Jacobs, N. & A. Garnham. 2007. The role of conversational hand gestures in a narrative task. Journal of Memory and Language 56(2). 291–303.10.1016/j.jml.2006.07.011Search in Google Scholar
Johansson, N. & J. Zlatev. 2013. Motivations for sound symbolism in spatial deixis: A typological study of 101 languages. Public Journal of Semiotics 5(1). 3–20.10.37693/pjos.2013.5.9668Search in Google Scholar
Kendon, A. 2004. Gesture: Visible action as utterance. Cambridge: Cambridge University Press.10.1017/CBO9780511807572Search in Google Scholar
Kida, T. (2005). Appropriation du geste par les étrangers: Le cas d’étudiants japonais apprenant le français [Acquisition of gesture by foreigners: The case of Japanese students learning French]. Unpublished doctoral dissertation, Université de Provence, Aix-en-Provence, France.Search in Google Scholar
Kita, S. & A. Özyürek. 2003. What does cross-linguistic variation in semantic coordination of speech and gesture reveal?: Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and Language 48(1). 16–32.10.1016/S0749-596X(02)00505-3Search in Google Scholar
Knight, D. 2011. Multimodality and active listenership: A corpus approach: Corpus and discourse. London: Bloomsbury.Search in Google Scholar
Krauss, R. M., Y. Chen & R. F. Gottesman. 2001. Lexical gestures and lexical access: A process model. In D. McNeill (ed.), Language and gesture, 261–283. New York: Cambridge University Press.10.1017/CBO9780511620850.017Search in Google Scholar
Leonard, T. & F. Cummins. 2011. The temporal relation between beat gestures and speech. Language and Cognitive Processes 26(10). 1457–1471.10.1080/01690965.2010.500218Search in Google Scholar
Lin, Y. L. 2017. Co-occurrence of speech and gestures: A multimodal corpus linguistic approach to intercultural interaction. Journal of Pragmatics 117. 155–167.10.1016/j.pragma.2017.06.014Search in Google Scholar
Littlemore, J. & G. Low. 2006. Metaphoric competence and communicative language ability. Applied Linguistics 27(2). 268–294.10.1093/applin/aml004Search in Google Scholar
McCafferty, S. G. 2004. Space for cognition: Gesture and second language learning. International Journal of Applied Linguistics 14(1). 148–165.10.1111/j.1473-4192.2004.0057m.xSearch in Google Scholar
McCafferty, S. G. 2006. Gesture and the materialization of second language prosody. IRAL-International Review of Applied Linguistics in Language Teaching 44(2). 197–209.10.1515/IRAL.2006.008Search in Google Scholar
McNeill, D. 1992. Hand and mind: what gestures reveal about thought. Chicago: University of Chicago Press.Search in Google Scholar
McNeill, D. 2005. Gesture & thought. Chicago: The University of Chicago Press.10.7208/chicago/9780226514642.001.0001Search in Google Scholar
McNeill, D. 2016. Why we gesture: The surprising role of hand movement in communication. Cambridge: Cambridge University Press.10.1017/CBO9781316480526Search in Google Scholar
Morsella, E., & Krauss, R. M. 2004. The role of gestures in spatial working memory and speech. American Journal of Psychology, 117. 411–424.10.2307/4149008Search in Google Scholar
Nagpal, J., E. Nicoladis & P. Marentette. 2011. Predicting individual differences in L2 speakers’ gestures. International Journal of Bilingualism 15(2). 205–214.10.1177/1367006910381195Search in Google Scholar
Negueruela, E. & J.P. Lantolf. 2008. The dialectics of gesture in the construction of meaning in second language oral narratives. In S. McCafferty & G. Stam (eds.), Gesture: Second language acquisition and classroom research, 88–106. NY: Routledge.Search in Google Scholar
Negueruela, E., J. P. Lantolf, S. R. Jordan & J. Gelabert. 2004. The ‘private function’ of gesture in second language speaking activity: A study of motion verbs and gesturing in English and Spanish. International Journal of Applied Linguistics 14(1). 113–147.10.1111/j.1473-4192.2004.00056.xSearch in Google Scholar
Nicoladis, E., S. Pika, H. Yin & P. Marentette. 2007. Gesture-use in story recall by Chinese–English bilinguals. Applied Psycholinguistics 28. 721–735.10.1017/S0142716407070385Search in Google Scholar
Parrill, F., J. Bullen & H. Hoburg. 2010. Effects of input modality on speech-gesture integration. Journal of Pragmatics 42(11). 3130–3137.10.1016/j.pragma.2010.04.023Search in Google Scholar
Pika, S., E. Nicoladis & P. F. Marentette. 2006. A cross-cultural study on the use of gestures: Evidence for cross-linguistic transfer?. Bilingualism: Language and Cognition 9(03). 319–327.10.1017/S1366728906002665Search in Google Scholar
Roth, W. M. 2001. Their role in teaching and learning. Review of Educational Research 71(3). 365–392.10.3102/00346543071003365Search in Google Scholar
Sherman, J. & E. Nicoladis. 2004. Gestures by advanced Spanish-English second-language learners. Gesture 4(2). 143–156.10.1075/gest.4.2.03sheSearch in Google Scholar
Slobin, D. I. 1991. Learning to think for speaking: Native language, cognition, and rhetorical style. Pragmatics 1. 7–26.10.1075/prag.1.1.01sloSearch in Google Scholar
So, W. C., S. Kita & S. Goldin-Meadow. 2009. Using the hands to identify who does what to whom: Gesture and speech go hand-in-hand. Cognitive Science 33(1). 115–125.10.1111/j.1551-6709.2008.01006.xSearch in Google Scholar
So, W. C., S. Kita & S. Goldin-Meadow. 2013. When do speakers use gestures to specify who does what to whom? The role of language proficiency and type of gestures in narratives. Journal of Psycholinguistic Research 42(6). 581–594.10.1007/s10936-012-9230-6Search in Google Scholar
Stam, G. & S. McCafferty. 2008. Gesture studies and second language acquisition: A review. In S. McCafferty & G. Stam (eds.), Gesture: Second language acquisition and classroom research, 3–24. NY: Routledge.10.4324/9780203866993Search in Google Scholar
Tutton, M. 2011. How speakers gesture when encoding location with English on and French sur. Journal of Pragmatics 43(14). 3431–3454.10.1016/j.pragma.2011.07.012Search in Google Scholar
Wundt, W. 1973. The language of gestures. Berlin: Walter de Gruyter.10.1515/9783110808285Search in Google Scholar
Yanagimachi, T. (1999). The acquisition of referential form use in L2 oral narrative discourse by adult English-speaking learners of Japanese. Doctoral dissertation, University of Minnesota.Search in Google Scholar
Yoshioka, K. 2008. Linguistic and gestural introduction of Ground reference in L1 and L2 narrative. In S. McCafferty & G. Stam (eds.), Gesture: Second language acquisition and classroom research, 211–230. NY: Routledge.Search in Google Scholar
Yoshioka, K. & E. Kellerman. 2006. Gestural introduction of Ground reference in L2 narrative discourse. IRAL-International Review of Applied Linguistics in Language Teaching 44(2). 173–195.10.1515/IRAL.2006.007Search in Google Scholar
Zhao, J. 2006. The communicative functions of gestures in L2 Speech. Arizona Working Papers in SLAT 13. 1–17.Search in Google Scholar
Zhao, S., E. Djonov & T. van Leeuwen. 2014. Semiotic technology and practice: A multimodal social semiotic approach to PowerPoint. Text & Talk 34(3). 349–375.10.1515/text-2014-0005Search in Google Scholar
© 2019 Walter de Gruyter GmbH, Berlin/Boston
Articles in the same Issue
- Frontmatter
- Research Articles
- Speech-accompanying gestures in L1 and L2 conversational interaction by speakers of different proficiency levels
- Transfer in L3 cognate language acquisition: The role of language background on instructed L3 Portuguese acquisition
- Task complexity, language proficiency and working memory: Interaction effects on second language speech performance
- Typology and contexts of article errors: Investigation into the use of English articles by Hong Kong Cantonese ESL learners
- Gender assignment strategies used by L1 and L2 speakers of Spanish
- The refusal of request speech act in Persian, English, and Balouchi languages: A cross-cultural and cross-linguistic study
- Non-nativelike outcome of naturalistic child L2 acquisition of Japanese: The case of noun–verb collocations
- Effects of mixed instruction on Chinese EFL learners’ perception of phonemic contrasts
- Dynamism of collocation in L2 English writing: A bigram-based study
- Nominal agreement in the interlanguage of Dutch L2 learners of Spanish
- Picture or non-picture? The influence of narrative task types on lower- and higher-proficiency EFL learners’ oral production
- Grammatical aspect and world knowledge in second language reading
- An exploratory study on pro-drop in a written description task in L2 Spanish
- Negotiation of meaning in child-child vs. adult-adult interactions: Evidence from low proficiency EFL learners
- Aspect semantics and ESL article use
Articles in the same Issue
- Frontmatter
- Research Articles
- Speech-accompanying gestures in L1 and L2 conversational interaction by speakers of different proficiency levels
- Transfer in L3 cognate language acquisition: The role of language background on instructed L3 Portuguese acquisition
- Task complexity, language proficiency and working memory: Interaction effects on second language speech performance
- Typology and contexts of article errors: Investigation into the use of English articles by Hong Kong Cantonese ESL learners
- Gender assignment strategies used by L1 and L2 speakers of Spanish
- The refusal of request speech act in Persian, English, and Balouchi languages: A cross-cultural and cross-linguistic study
- Non-nativelike outcome of naturalistic child L2 acquisition of Japanese: The case of noun–verb collocations
- Effects of mixed instruction on Chinese EFL learners’ perception of phonemic contrasts
- Dynamism of collocation in L2 English writing: A bigram-based study
- Nominal agreement in the interlanguage of Dutch L2 learners of Spanish
- Picture or non-picture? The influence of narrative task types on lower- and higher-proficiency EFL learners’ oral production
- Grammatical aspect and world knowledge in second language reading
- An exploratory study on pro-drop in a written description task in L2 Spanish
- Negotiation of meaning in child-child vs. adult-adult interactions: Evidence from low proficiency EFL learners
- Aspect semantics and ESL article use