Thinking in Action
-
Barbara Tversky
Abstract
When thought overwhelms the mind, the mind uses the body and the world. Several studies reveal ways that people alone or together use gesture and marks on paper to structure and augment their thought for comprehension, inference, and discovery. The studies show that the mapping of thought to gesture or the page is more direct than the arbitrary mapping to language and suggest that these forms of visual/spatial/action representation are used to “translate” language into mental representations. It is argued that actions in space create patterns in the world that reflect abstractions, that the actions are incorporated into gestures and the patterns into diagrams, a network that integrates gesture, action, the designed world, and abstraction dubbed spraction.
Abstract
When thought overwhelms the mind, the mind uses the body and the world. Several studies reveal ways that people alone or together use gesture and marks on paper to structure and augment their thought for comprehension, inference, and discovery. The studies show that the mapping of thought to gesture or the page is more direct than the arbitrary mapping to language and suggest that these forms of visual/spatial/action representation are used to “translate” language into mental representations. It is argued that actions in space create patterns in the world that reflect abstractions, that the actions are incorporated into gestures and the patterns into diagrams, a network that integrates gesture, action, the designed world, and abstraction dubbed spraction.
Chapters in this book
- Frontmatter I
- Preface and Acknowledgements V
- Contents VII
-
Part I: Introduction
- Empirical Multimodality Research: The State of Play 1
-
Part II: Charting Paths for Empirical Research: Theoretical and Methodological Reflections
- Dimensions of Materiality 35
- From Data to Patterns 65
- Thinking in Action 91
- Computational Approaches for the Interpretation of Image-Text Relations 109
-
Part III: Empirical Inroads: Case Studies and Results
- “I can’t see why you’re laughing”: Multimodal Analysis of Emotionalized Political Debate 141
- A Corpus-Based Approach to Color, Shape, and Typography in Logos 159
- Pixel Surgery and the Doctored Image 187
- Multimodal Discourse Analysis Based on the GeM Model 211
- Conventions in How Korean Films Mean 237
- An Empirical Multimodal Approach to Open-World Video Games 259
- List of Contributors 281
- Index 285
Chapters in this book
- Frontmatter I
- Preface and Acknowledgements V
- Contents VII
-
Part I: Introduction
- Empirical Multimodality Research: The State of Play 1
-
Part II: Charting Paths for Empirical Research: Theoretical and Methodological Reflections
- Dimensions of Materiality 35
- From Data to Patterns 65
- Thinking in Action 91
- Computational Approaches for the Interpretation of Image-Text Relations 109
-
Part III: Empirical Inroads: Case Studies and Results
- “I can’t see why you’re laughing”: Multimodal Analysis of Emotionalized Political Debate 141
- A Corpus-Based Approach to Color, Shape, and Typography in Logos 159
- Pixel Surgery and the Doctored Image 187
- Multimodal Discourse Analysis Based on the GeM Model 211
- Conventions in How Korean Films Mean 237
- An Empirical Multimodal Approach to Open-World Video Games 259
- List of Contributors 281
- Index 285
