Home Chapter 6. Quantifying the interplay of gaze and gesture in deixis using an experimental-simulative approach
Chapter
Licensed
Unlicensed Requires Authentication

Chapter 6. Quantifying the interplay of gaze and gesture in deixis using an experimental-simulative approach

  • Thies Pfeiffer and Patrick Renner
View more publications by John Benjamins Publishing Company
Search
Eye-tracking in Interaction
This chapter is in the book Eye-tracking in Interaction

Abstract

Gaze and gestures have been studied qualitatively, e.g., by Kendon and others (Kendon, 1990; McNeill, 1992; Kendon, 2004; McNeill, 2006). A quantitative assessment of gaze and gestures in dialogue, in particular regarding precise orientations, positions and timings, however, has only been possible with the advent of advanced measuring technologies, such as motion capturing or eye tracking. Especially in dynamic natural environments, when interlocutors are concerned with their surrounding three-dimensional environment, a precise three-dimensional reconstruction of the set-up is required to analyze the produced multimodal utterances.

In this article we review several of our past projects with a focus on our experimental-simulative approach in which we combine state-of-the-art tracking technologies with 3D representations and computer simulations to test different hypotheses in the context of deixis in human-human interaction.

Abstract

Gaze and gestures have been studied qualitatively, e.g., by Kendon and others (Kendon, 1990; McNeill, 1992; Kendon, 2004; McNeill, 2006). A quantitative assessment of gaze and gestures in dialogue, in particular regarding precise orientations, positions and timings, however, has only been possible with the advent of advanced measuring technologies, such as motion capturing or eye tracking. Especially in dynamic natural environments, when interlocutors are concerned with their surrounding three-dimensional environment, a precise three-dimensional reconstruction of the set-up is required to analyze the produced multimodal utterances.

In this article we review several of our past projects with a focus on our experimental-simulative approach in which we combine state-of-the-art tracking technologies with 3D representations and computer simulations to test different hypotheses in the context of deixis in human-human interaction.

Chapters in this book

  1. Prelim pages i
  2. Table of contents v
  3. Chapter 1. Introduction 1
  4. Part 1. Theoretical considerations
  5. Chapter 2. Eye gaze as a cue for recognizing intention and coordinating joint action 21
  6. Chapter 3. Effects of a speaker’s gaze on language comprehension and acquisition 47
  7. Chapter 4. Weaving oneself into others 67
  8. Chapter 5. On the role of gaze for successful and efficient communication 91
  9. Part 2. Methodological considerations
  10. Chapter 6. Quantifying the interplay of gaze and gesture in deixis using an experimental-simulative approach 109
  11. Chapter 7. Gaze and face-to-face interaction 139
  12. Chapter 8. Automatic analysis of in-the-wild mobile eye-tracking experiments using object, face and person detection 169
  13. Part 3. Case studies
  14. Chapter 9. Gaze, addressee selection and turn-taking in three-party interaction 197
  15. Chapter 10. Gaze as a predictor for lexical and gestural alignment 233
  16. Chapter 11. Mobile dual eye-tracking in face-to-face interaction 265
  17. Chapter 12. Displaying recipiency in an interpreter-mediated dialogue 301
  18. Index 323
https://doi.org/10.1075/ais.10.06pfe

Chapters in this book

  1. Prelim pages i
  2. Table of contents v
  3. Chapter 1. Introduction 1
  4. Part 1. Theoretical considerations
  5. Chapter 2. Eye gaze as a cue for recognizing intention and coordinating joint action 21
  6. Chapter 3. Effects of a speaker’s gaze on language comprehension and acquisition 47
  7. Chapter 4. Weaving oneself into others 67
  8. Chapter 5. On the role of gaze for successful and efficient communication 91
  9. Part 2. Methodological considerations
  10. Chapter 6. Quantifying the interplay of gaze and gesture in deixis using an experimental-simulative approach 109
  11. Chapter 7. Gaze and face-to-face interaction 139
  12. Chapter 8. Automatic analysis of in-the-wild mobile eye-tracking experiments using object, face and person detection 169
  13. Part 3. Case studies
  14. Chapter 9. Gaze, addressee selection and turn-taking in three-party interaction 197
  15. Chapter 10. Gaze as a predictor for lexical and gestural alignment 233
  16. Chapter 11. Mobile dual eye-tracking in face-to-face interaction 265
  17. Chapter 12. Displaying recipiency in an interpreter-mediated dialogue 301
  18. Index 323
Sign up now to receive a 20% welcome discount
Institutional Access
How does access work?
Have an idea on how to improve our website?
Please write us.
© 2025 De Gruyter Brill
Downloaded on 12.9.2025 from https://www.degruyterbrill.com/document/doi/10.1075/ais.10.06pfe/html
Scroll to top button