Chapter 6. Quantifying the interplay of gaze and gesture in deixis using an experimental-simulative approach
-
Thies Pfeiffer
Abstract
Gaze and gestures have been studied qualitatively, e.g., by Kendon and others (Kendon, 1990; McNeill, 1992; Kendon, 2004; McNeill, 2006). A quantitative assessment of gaze and gestures in dialogue, in particular regarding precise orientations, positions and timings, however, has only been possible with the advent of advanced measuring technologies, such as motion capturing or eye tracking. Especially in dynamic natural environments, when interlocutors are concerned with their surrounding three-dimensional environment, a precise three-dimensional reconstruction of the set-up is required to analyze the produced multimodal utterances.
In this article we review several of our past projects with a focus on our experimental-simulative approach in which we combine state-of-the-art tracking technologies with 3D representations and computer simulations to test different hypotheses in the context of deixis in human-human interaction.
Abstract
Gaze and gestures have been studied qualitatively, e.g., by Kendon and others (Kendon, 1990; McNeill, 1992; Kendon, 2004; McNeill, 2006). A quantitative assessment of gaze and gestures in dialogue, in particular regarding precise orientations, positions and timings, however, has only been possible with the advent of advanced measuring technologies, such as motion capturing or eye tracking. Especially in dynamic natural environments, when interlocutors are concerned with their surrounding three-dimensional environment, a precise three-dimensional reconstruction of the set-up is required to analyze the produced multimodal utterances.
In this article we review several of our past projects with a focus on our experimental-simulative approach in which we combine state-of-the-art tracking technologies with 3D representations and computer simulations to test different hypotheses in the context of deixis in human-human interaction.
Kapitel in diesem Buch
- Prelim pages i
- Table of contents v
- Chapter 1. Introduction 1
-
Part 1. Theoretical considerations
- Chapter 2. Eye gaze as a cue for recognizing intention and coordinating joint action 21
- Chapter 3. Effects of a speaker’s gaze on language comprehension and acquisition 47
- Chapter 4. Weaving oneself into others 67
- Chapter 5. On the role of gaze for successful and efficient communication 91
-
Part 2. Methodological considerations
- Chapter 6. Quantifying the interplay of gaze and gesture in deixis using an experimental-simulative approach 109
- Chapter 7. Gaze and face-to-face interaction 139
- Chapter 8. Automatic analysis of in-the-wild mobile eye-tracking experiments using object, face and person detection 169
-
Part 3. Case studies
- Chapter 9. Gaze, addressee selection and turn-taking in three-party interaction 197
- Chapter 10. Gaze as a predictor for lexical and gestural alignment 233
- Chapter 11. Mobile dual eye-tracking in face-to-face interaction 265
- Chapter 12. Displaying recipiency in an interpreter-mediated dialogue 301
- Index 323
Kapitel in diesem Buch
- Prelim pages i
- Table of contents v
- Chapter 1. Introduction 1
-
Part 1. Theoretical considerations
- Chapter 2. Eye gaze as a cue for recognizing intention and coordinating joint action 21
- Chapter 3. Effects of a speaker’s gaze on language comprehension and acquisition 47
- Chapter 4. Weaving oneself into others 67
- Chapter 5. On the role of gaze for successful and efficient communication 91
-
Part 2. Methodological considerations
- Chapter 6. Quantifying the interplay of gaze and gesture in deixis using an experimental-simulative approach 109
- Chapter 7. Gaze and face-to-face interaction 139
- Chapter 8. Automatic analysis of in-the-wild mobile eye-tracking experiments using object, face and person detection 169
-
Part 3. Case studies
- Chapter 9. Gaze, addressee selection and turn-taking in three-party interaction 197
- Chapter 10. Gaze as a predictor for lexical and gestural alignment 233
- Chapter 11. Mobile dual eye-tracking in face-to-face interaction 265
- Chapter 12. Displaying recipiency in an interpreter-mediated dialogue 301
- Index 323
