The sequential organization of text and speech in multimodal synchronous computer-mediated communication

Hanh thi Nguyen

doi:10.1515/text-2016-0039

Article

The sequential organization of text and speech in multimodal synchronous computer-mediated communication

Hanh thi Nguyen
Hanh thi Nguyen (PhD, University of Wisconsin-Madison) is an associate professor of applied linguistics at Hawaii Pacific University. Her research interests include the discourse processes in learning situations involving face-to-face or computer-mediated communication, language socialization, the development of interactional competence in second language learning or professional communication training, and Vietnamese applied linguistics.

Published/Copyright: January 7, 2017

Published by

Become an author with De Gruyter Brill

Submit Manuscript Author Information Explore this Subject

From the journal Text & Talk Volume 37 Issue 1

Abstract

This study uses conversation analysis to describe the sequential and functional relationship between text and speech turns in an English conversational lesson conducted in multimodal synchronous computer-mediated communication (SCMC) involving text and speech modes. Focusing on repair sequences, I examine the relative timing of turns in each mode, the interactional practices that participants employed to handle timing discrepancy, and how both modes were utilized to maintain the pedagogical and interpersonal purposes of the encounter. The analysis shows that synchronous timing between text and speech turns was rare. In time lags between text and speech turns, if the repair was a self-initiated other-repair initiated by the tutee, speech turns did not seem to orient to the time lag. In other types of repair, the tutor utilized a range of practices to accommodate for the time lags, such as extreme slow speech tempo, pivot turns, and topic pursuits. The tutor also used the silent and visual features of text to insert and project an upcoming teaching episode in the midst of unfolding topical talk. The findings suggest that multimodal SCMC is a holistic process in which the affordances of modes can be employed dynamically and integratively to achieve social actions.

Keywords: multimodal discourse; SCMC; text-and-voice chat; repair; conversation analysis

About the author

Hanh thi Nguyen

Hanh thi Nguyen (PhD, University of Wisconsin-Madison) is an associate professor of applied linguistics at Hawaii Pacific University. Her research interests include the discourse processes in learning situations involving face-to-face or computer-mediated communication, language socialization, the development of interactional competence in second language learning or professional communication training, and Vietnamese applied linguistics.

Acknowledgements

I wish to thank the participants for allowing their conversational lesson to be recorded and analyzed. I thank Andre Langevin for providing the data and Guy Kellogg, Jeff Mehring, and the anonymous reviewers for their constructive feedback on this paper’s earlier versions. Hawaii Pacific University’s Scholarship Endeavor Program supported the writing of this paper. Any remaining errors are mine.

Appendix. Transcription conventions (based on Jefferson [2004])

.: falling intonation
?: rising intonation
,: slightly rising intonation
:: sound stretch or lengthening (more colons indicate longer stretch)
[: beginning of overlapped speech or actions
underline: stress (more underlined letters indicate more stress)
CAPS: louder volume than surrounding speech
°word°: quieter speech than surrounding speech (double degree signs indicate even quieter)
<word>: slowed speech tempo (double symbols indicate even slower tempo)
>word<: fastened speech tempo
hh.: audible out-breaths, often associated with laughter
heh heh: laughter tokens
↑: raised pitch
↓: lowered pitch
(.): a pause about one-tenth of a second
(number): silence measured in seconds
((words)): transcriber’s notes
“text”: beginning of typing action of text in quotation by the tutor, typing continues until message is sent
↵“text”: sending of text in quotation
“text”: typed text
italics: typing actions accompanying speech in line above

References

Betz, Emma. 2008. Grammar and interaction: Pivots in German conversations. Amsterdam: Benjamins.10.1075/sidag.21Search in Google Scholar

Bolden, Galina B. 2009. Implementing incipient actions: The discourse marker “so” in English conversation. Journal of Pragmatics 41(5). 974–998.10.1016/j.pragma.2008.10.004Search in Google Scholar

Brazil, David. 1997. The communicative value of intonation in English. Cambridge: Cambridge University Press.Search in Google Scholar

Button, Graham & Neil Casey. 1984. Generating topic: The use of topic initial elicitors. In Maxwell Atkinson & John Heritage (eds.), Structures of social action: Studies in conversation analysis, 167–190. Cambridge: Cambridge University Press.10.1017/CBO9780511665868.013Search in Google Scholar

Button, Graham & Neil Casey. 1985. Topic nomination and pursuit. Human Studies 8(3). 3–55.10.1007/BF00143022Search in Google Scholar

Gibson, Will. 2014. Sequential order in multimodal discourse: Talk and text in online educational interaction. Discourse and Communication 8(1). 63–83.10.1177/1750481313503222Search in Google Scholar

Goodwin, Charles. 1994. Professional vision. American Anthropologist 96(3). 606–633.10.1525/aa.1994.96.3.02a00100Search in Google Scholar

Goodwin, Charles. 2000. Action and embodiment within situated human interaction. Journal of Pragmatics 32. 1489–1522.10.1016/S0378-2166(99)00096-XSearch in Google Scholar

Goodwin, Charles. 2007. Participation, stance and affect in the organization of activities. Discourse & Society 18. 53–73.10.1177/0957926507069457Search in Google Scholar

Hall, Joan Kelly. 2007. Redressing the roles of correction and repair in research on second and foreign language learning. Modern Language Journal 91(4). 511–526.10.1111/j.1540-4781.2007.00619.xSearch in Google Scholar

Hampel, Regina & Ursula Stickler. 2012. The use of videoconferencing to support multimodal interaction in an online language classroom. ReCALL 24(2). 116–137.10.1017/S095834401200002XSearch in Google Scholar

Have, Paul ten. 2007. Doing conversation analysis: A practical guide, 2nd edn. London: Sage Publications.10.4135/9781849208895Search in Google Scholar

Holt, Elizabeth & Paul Drew. 2005. Figurative pivots: The use of figurative expressions in pivotal topic transitions. Research on Language and Social Interaction 38(1). 35–61.10.1207/s15327973rlsi3801_2Search in Google Scholar

Jefferson, Gail. 1972. Side sequences. In David Sudnow (ed.), Studies in social interaction, 294–451. New York: The Free Press.Search in Google Scholar

Jefferson, Gail. 1987. On exposed and embedded correction in conversation. In Graham Button & John R. E. Lee (eds.), Talk and social organization, 86–100. Clevedon: Multilingual Matters.Search in Google Scholar

Jefferson, Gail. 1993. Caveat speaker: Preliminary notes on recipient topic-shift implicature. Research on Language and Social Interaction 26(2). 1–30.10.1207/s15327973rlsi2601_1Search in Google Scholar

Jefferson, Gail. 2002. Is “no” an acknowledgement token? Comparing American and British use of (+)/(-) tokens. Journal of Pragmatics 34. 1345–1383.10.1016/S0378-2166(02)00067-XSearch in Google Scholar

Jefferson, Gail. 2004. Glossary of transcript symbols with an introduction. In Gene H. Lerner (ed.), Conversation analysis: Studies from the first generation, 13–31. Amsterdam & Philadelphia: John Benjamins.10.1075/pbns.125.02jefSearch in Google Scholar

Jenks, Christopher. 2009. When is it appropriate to talk? Managing overlapping talk in multi-participant voice-based chat rooms. Computer Assisted Language Learning 22(1). 19–30.10.1080/09588220802613781Search in Google Scholar

Jewitt, Carey. 2004. Multimodality and new communication technologies. In Philip Levine & Ron Scollon (eds.), Discourse and technology: Multimodal discourse analysis, 184–195. Washington, DC: Georgetown University Press.Search in Google Scholar

Jucker, Andreas H. & Sara Smith. 1998. And people just you know like “wow”: Discourse markers as negotiating strategies. In Andreas H. Jucker & Yael Ziv (eds.), Discourse markers: Theory and descriptions, 171–201. Amsterdam: John Benjamins.10.1075/pbns.57.10jucSearch in Google Scholar

Kasper, Gabriele. 2004. Participation orientation in German conversation-for-learning. The Modern Language Journal 88(4). 551–567.10.1111/j.0026-7902.2004.t01-18-.xSearch in Google Scholar

Kitzinger, Celia. 2013. Repair. In Jack Sidnell & Tanya Stivers (eds.), The handbook of conversation analysis, 229–256. Malden & Oxford: Wiley-Blackwell.10.1002/9781118325001.ch12Search in Google Scholar

Kost, Claudia R. 2008. Use of communication strategies in a synchronous CMC environment. In Sally S. Magnan (ed.), Mediating online discourse, 153–189. Amsterdam: John Benjamins.10.1075/aals.3.11kosSearch in Google Scholar

Kozar, Olga. 2015. Language education via audio/ videoconferencing (LEVAC): A discursive investigation. Linguistics and Education 31. 86–100.10.1016/j.linged.2015.05.007Search in Google Scholar

Kozar, Olga. 2016. Text chat during video/audio conferencing lessons: Scaffolding or getting in the way? CALICO Journal 33(2). 231–259.10.1558/cj.v33i2.26026Search in Google Scholar

Kress, Gunther & Theo van Leeuwen. 2001. Multimodal discourse: The modes and media of contemporary communication. Oxford: Oxford University Press.Search in Google Scholar

Lai, Chun & Yong Zhao. 2006. Noticing and text-based chat. Language Learning and Technology 10(3). 102–120.Search in Google Scholar

Levy, Mike & Rod Gardner. 2012. Liminality in multitasking: Where talk and task collide in computer collaborations. Discourse in Society 41. 557–587.10.1017/S0047404512000656Search in Google Scholar

Macbeth, Douglas. 2004. The relevance of repair for classroom correction. Language in Society 33(5). 703–736.10.1017/S0047404504045038Search in Google Scholar

Markee, Numa. 2000. Conversation analysis. Mahwah & London: Lawrence Erlbaum Associates Publishers.10.4324/9781410606471Search in Google Scholar

Maynard, Douglas W. 1980. Placement of topic changes in conversation. Semiotica 30(3/4). 263–290.10.1515/semi.1980.30.3-4.263Search in Google Scholar

McHoul, A. W. 1990. The organization of repair in classroom talk. Discourse in Society 19(3). 349–377.10.1017/S004740450001455XSearch in Google Scholar

Mondada, Lorenza & Kimmo Svinhufvud. 2016. Writing-in-interaction: Studying writing as a multimodal phenomenon in social interaction. Language and Dialogue 6(1). 1–53.10.1075/ld.6.1.01monSearch in Google Scholar

Mortensen, Kristian. 2013. Writing aloud: Some interactional functions of the public display of emergent writing. Paper presented at the Participatory Innovation Conference, Lahti, Finland.Search in Google Scholar

Nevile, Maurice, Pentti Haddington, Trine Heinemann & Mirka Rauniomaa. 2014. Interacting with objects: Language, materiality, and social activity. Amsterdam & Philadelphia: John Benjamins.10.1075/z.186Search in Google Scholar

Nguyen, Hanh thi & Andre Langevin. 2016. Some interactional functions of text in a text-and-voice SCMC chat session for language learning. International Journal of Computer-Assisted Language Learning and Teaching 6(1). 1–23.10.4018/IJCALLT.2016010101Search in Google Scholar

Norén, Niklas & Per Linell. 2013. Pivot constructions as everyday conversational phenomena within a cross-linguistic perspective: An introduction. Journal of Pragmatics 54. 1–15.10.1016/j.pragma.2013.03.006Search in Google Scholar

Sacks, Harvey. 1995. Lectures on conversation, Vol. I & II. Oxford: Blackwell Publishers.10.1002/9781444328301Search in Google Scholar

Sacks, Harvey, Emmanuel A. Schegloff & Gail Jefferson. 1974. A simplest systematics for the organization of turn taking for conversation. Language 50(4). 696–735.10.1353/lan.1974.0010Search in Google Scholar

Sauro, Shannon. 2009. Strategic use of modality during synchronous CMC. CALICO Journal 27(1). 101–117.10.11139/cj.27.1.101-117Search in Google Scholar

Scheffel-Dunand, Dominique. 2006. Bimodal communication over webcasts: From CSCL to CALL. Computer Assisted Language Learning 19(4). 341–355.10.1080/09588220601043107Search in Google Scholar

Schegloff, Emmanuel A. 1992. Repair after next turn: The last structurally provided defense of intersubjectivity in conversation. American Journal of Sociology 97(5 Mar). 1295–1345.10.1086/229903Search in Google Scholar

Schegloff, Emmanuel A. 1996. Third-turn repair. In Gregory R. Guy, Crawford Feagin, Deborah Schiffrin & John Baugh (eds.), Towards a science of language: Papers in honor of William Labov, 31–40. Amsterdam & Philadelphia: John Benjamins.Search in Google Scholar

Schegloff, Emmanuel A. 2000. When “others” initiate repair. Applied Linguistics 21(2). 205–243.10.1093/applin/21.2.205Search in Google Scholar

Schegloff, Emmanuel A. 2007. Sequence organization in interaction: A primer in conversation analysis, vol. 1. Cambridge: Cambridge University Press.10.1017/CBO9780511791208Search in Google Scholar

Schegloff, Emmanuel A., Gail Jefferson & Harvey Sacks. 1977. The preference for self-correction in the organization of repair in conversation. Language 53(2). 361–382.10.1353/lan.1977.0041Search in Google Scholar

Schiffrin, Deborah. 1987. Discourse markers. Cambridge: Cambridge University Press.10.1017/CBO9780511611841Search in Google Scholar

Schönfeldt, Juliane & Andrea Golato. 2003. Repair in chats: A conversation analytic approach. Research on Language and Social Interaction 36(3). 241–284.10.1207/S15327973RLSI3603_02Search in Google Scholar

Schwienhorst, Klaus. 2002. Evaluating tandem language learning in the MOO: Discourse repair strategies in a bilingual internet project. Computer Assisted Language Learning 15(2). 135–145.10.1076/call.15.2.135.8194Search in Google Scholar

Sinclair, John McHardy & Malcolm Coulthard. 1975. Towards an analysis of discourse: The English used by teachers and pupils. London: Oxford University Press.Search in Google Scholar

Smith, Bryan & Shannon Sauro. 2009. Interruptions in chat. Computer Assisted Language Learning 22(3). 229–247.10.1080/09588220902920219Search in Google Scholar

Wootton, Anthony J. 1981. The management of grantings and rejections by parents in request sequences. Semiotica 37. 59–90.10.1515/semi.1981.37.1-2.59Search in Google Scholar

Published Online: 2017-1-7

Published in Print: 2017-1-1

You are currently not able to access this content.

Articles in the same Issue

https://doi.org/10.1515/text-2016-0039

Keywords for this article

multimodal discourse; SCMC; text-and-voice chat; repair; conversation analysis