Transcribing and annotating multimodality
-
Agnès Millet
Abstract
This paper deals with the central question of transcribing deaf children’s productions. We present the annotation grid we created on Elan®, explaining in detail how and why the observation of the narrative productions of 6 to 12 year-old deaf children led us to modify the annotation schemes previously available. Deaf children resort to every resource available in both modalities: voice and gesture. Thus, these productions are fundamentally multimodal and bilingual. In order to describe these specific practices, we propose considering verbal and non-verbal, vocal and gestural, materials as parts of one integrated production. A linguistic-centered transcription is not efficient in describing such bimodal productions, since describing bimodal utterances implies taking into account the ‘communicative desire’ (‘vouloir-dire’) of the children. For this reason, both the question of the transcription unit and the issue of the complexity of semiotic interactions in bimodal utterances need to be reconsidered.
Abstract
This paper deals with the central question of transcribing deaf children’s productions. We present the annotation grid we created on Elan®, explaining in detail how and why the observation of the narrative productions of 6 to 12 year-old deaf children led us to modify the annotation schemes previously available. Deaf children resort to every resource available in both modalities: voice and gesture. Thus, these productions are fundamentally multimodal and bilingual. In order to describe these specific practices, we propose considering verbal and non-verbal, vocal and gestural, materials as parts of one integrated production. A linguistic-centered transcription is not efficient in describing such bimodal productions, since describing bimodal utterances implies taking into account the ‘communicative desire’ (‘vouloir-dire’) of the children. For this reason, both the question of the transcription unit and the issue of the complexity of semiotic interactions in bimodal utterances need to be reconsidered.
Chapters in this book
- Prelim pages i
- Table of contents vii
- About the authors ix
- Gesture and multimodal development 1
- Pointing gesture in young children 7
- Support or competition? 27
- From gesture to sign and from gesture to word 49
- How the hands control attention during early word learning 79
- Infant movement as a window into language processing 99
- Children’s lexical skills and task demands affect gestural behavior in mothers of late-talking children and children with typical language development 129
- The type of shared activity shapes caregiver and infant communication 157
- Transcribing and annotating multimodality 175
- Mathematical learning and gesture 199
- Index 221
Chapters in this book
- Prelim pages i
- Table of contents vii
- About the authors ix
- Gesture and multimodal development 1
- Pointing gesture in young children 7
- Support or competition? 27
- From gesture to sign and from gesture to word 49
- How the hands control attention during early word learning 79
- Infant movement as a window into language processing 99
- Children’s lexical skills and task demands affect gestural behavior in mothers of late-talking children and children with typical language development 129
- The type of shared activity shapes caregiver and infant communication 157
- Transcribing and annotating multimodality 175
- Mathematical learning and gesture 199
- Index 221
