Home Linguistics & Semiotics Chapter 12. Computational gesture research
Chapter
Licensed
Unlicensed Requires Authentication

Chapter 12. Computational gesture research

Studying the functions of gesture in human-agent interaction
  • Stefan Kopp
View more publications by John Benjamins Publishing Company
Search
Why Gesture?
This chapter is in the book Why Gesture?

Abstract

Embodied artificial agents, like humanoid robots or virtual characters, can produce a variety of co-speech gestures in interactive settings. This enables a computational branch of gesture research that offers opportunities for (1) investigating production and comprehension processes through computational cognitive modeling, and (2) studying systematically the effects and functions of gesturing in human-agent interaction. In this chapter we review current approaches to synthesize gestures, and we discuss in detail findings from experiments on the effects of gestures produced by virtual characters or robots. Current evidence suggests that synthetic gesturing has considerable effects on how an artificial agent is perceived and attributed with human-like properties. In contrast, so far, there is little evidence showing that contemporary synthetic gesturing helps human addressees to better understand or retain the information communicated by an agent. We discuss what these findings imply for the potential functions that gesture may serve in improving human-agent interaction, and more generally for our understanding of gesture through computational modeling efforts.

Abstract

Embodied artificial agents, like humanoid robots or virtual characters, can produce a variety of co-speech gestures in interactive settings. This enables a computational branch of gesture research that offers opportunities for (1) investigating production and comprehension processes through computational cognitive modeling, and (2) studying systematically the effects and functions of gesturing in human-agent interaction. In this chapter we review current approaches to synthesize gestures, and we discuss in detail findings from experiments on the effects of gestures produced by virtual characters or robots. Current evidence suggests that synthetic gesturing has considerable effects on how an artificial agent is perceived and attributed with human-like properties. In contrast, so far, there is little evidence showing that contemporary synthetic gesturing helps human addressees to better understand or retain the information communicated by an agent. We discuss what these findings imply for the potential functions that gesture may serve in improving human-agent interaction, and more generally for our understanding of gesture through computational modeling efforts.

Chapters in this book

  1. Prelim pages i
  2. Table of contents v
  3. Introduction
  4. Chapter 1. Understanding gesture 3
  5. Section 1. The function of gesture production for language
  6. Chapter 2. Representational gestures help speakers package information for speaking 15
  7. Chapter 3. Function and processing of gesture in the context of language 39
  8. Chapter 4. The asymmetric redundancy of gesture and speech 59
  9. Chapter 5. Gesture-speech unity 77
  10. Chapter 5 Supplement. Exchange on gesture-speech unity 103
  11. Section 2. The function of gesture for cognition and social interaction
  12. Chapter 6. The function of gesture in learning and memory 129
  13. Chapter 7. Gestures highlight perceptual-motor representations in thinking 155
  14. Chapter 8. One function of gesture is to make new ideas 175
  15. Chapter 9. Gesture in socio-moral reasoning 197
  16. Chapter 10. Multi-modal communication of common ground 213
  17. Part 2. The function of gesture comprehension
  18. Chapter 11. Exploring the boundaries of gesture-speech integration during language comprehension 243
  19. Chapter 12. Computational gesture research 267
  20. Chapter 13. Making and breaking common ground 285
  21. Chapter 14. The function of gesture in mathematical and scientific discourse in the classroom 317
  22. Chapter 15. Gesture’s role in learning interactions 331
  23. Chapter 16. The sound of silence 353
  24. Part 3. Why gesture?
  25. Chapter 17. Understanding gesture as representational action 381
  26. Chapter 18. So how does gesture function in speaking, communication, and thinking? 397
  27. Author index 413
  28. Subject index 421
https://doi.org/10.1075/gs.7.13kop

Chapters in this book

  1. Prelim pages i
  2. Table of contents v
  3. Introduction
  4. Chapter 1. Understanding gesture 3
  5. Section 1. The function of gesture production for language
  6. Chapter 2. Representational gestures help speakers package information for speaking 15
  7. Chapter 3. Function and processing of gesture in the context of language 39
  8. Chapter 4. The asymmetric redundancy of gesture and speech 59
  9. Chapter 5. Gesture-speech unity 77
  10. Chapter 5 Supplement. Exchange on gesture-speech unity 103
  11. Section 2. The function of gesture for cognition and social interaction
  12. Chapter 6. The function of gesture in learning and memory 129
  13. Chapter 7. Gestures highlight perceptual-motor representations in thinking 155
  14. Chapter 8. One function of gesture is to make new ideas 175
  15. Chapter 9. Gesture in socio-moral reasoning 197
  16. Chapter 10. Multi-modal communication of common ground 213
  17. Part 2. The function of gesture comprehension
  18. Chapter 11. Exploring the boundaries of gesture-speech integration during language comprehension 243
  19. Chapter 12. Computational gesture research 267
  20. Chapter 13. Making and breaking common ground 285
  21. Chapter 14. The function of gesture in mathematical and scientific discourse in the classroom 317
  22. Chapter 15. Gesture’s role in learning interactions 331
  23. Chapter 16. The sound of silence 353
  24. Part 3. Why gesture?
  25. Chapter 17. Understanding gesture as representational action 381
  26. Chapter 18. So how does gesture function in speaking, communication, and thinking? 397
  27. Author index 413
  28. Subject index 421
Sign up now to receive a 20% welcome discount
Institutional Access
How does access work?
Have an idea on how to improve our website?
Please write us.
© 2025 De Gruyter Brill
Downloaded on 14.9.2025 from https://www.degruyterbrill.com/document/doi/10.1075/gs.7.13kop/html
Scroll to top button