Empirical Multimodality Research: The State of Play
-
Jana Pflaeging
Abstract
Multimodality research has always shown a strong reliance on data. However, the field has primarily developed around more exploratory, descriptive, and interpretative work on smaller data sets - as suggested by results we present from a meta-study of contributions to three multimodality-close international journals (Social Semiotics, Visual Communication, Multimodal Communication). Framed by a discussion of the qualitative-quantitative dichotomy and a deliberately broad working definition of empirical, we argue that it is not sample size or quantitative methods alone that support a more solid empirical grounding of multimodality research, but rather an explicit orientation to just how theory and analysis make contact with data. To this end, we propose five quality criteria of empirical practice, that is, completing the empirical feedback loop: from theory to data and back, ensuring objectivity, reliability, and validity in research, and acknowledging the inherent tentativeness of results.We thereby seek to chart paths for an appropriate and productive application of various empirical methods to novel (and supposedly familiar) forms of meaning-making in order to further strengthen the development of theory and methods in multimodality, and to encourage an even more intense exchange among the diverse communities of multimodalists.
Abstract
Multimodality research has always shown a strong reliance on data. However, the field has primarily developed around more exploratory, descriptive, and interpretative work on smaller data sets - as suggested by results we present from a meta-study of contributions to three multimodality-close international journals (Social Semiotics, Visual Communication, Multimodal Communication). Framed by a discussion of the qualitative-quantitative dichotomy and a deliberately broad working definition of empirical, we argue that it is not sample size or quantitative methods alone that support a more solid empirical grounding of multimodality research, but rather an explicit orientation to just how theory and analysis make contact with data. To this end, we propose five quality criteria of empirical practice, that is, completing the empirical feedback loop: from theory to data and back, ensuring objectivity, reliability, and validity in research, and acknowledging the inherent tentativeness of results.We thereby seek to chart paths for an appropriate and productive application of various empirical methods to novel (and supposedly familiar) forms of meaning-making in order to further strengthen the development of theory and methods in multimodality, and to encourage an even more intense exchange among the diverse communities of multimodalists.
Chapters in this book
- Frontmatter I
- Preface and Acknowledgements V
- Contents VII
-
Part I: Introduction
- Empirical Multimodality Research: The State of Play 1
-
Part II: Charting Paths for Empirical Research: Theoretical and Methodological Reflections
- Dimensions of Materiality 35
- From Data to Patterns 65
- Thinking in Action 91
- Computational Approaches for the Interpretation of Image-Text Relations 109
-
Part III: Empirical Inroads: Case Studies and Results
- “I can’t see why you’re laughing”: Multimodal Analysis of Emotionalized Political Debate 141
- A Corpus-Based Approach to Color, Shape, and Typography in Logos 159
- Pixel Surgery and the Doctored Image 187
- Multimodal Discourse Analysis Based on the GeM Model 211
- Conventions in How Korean Films Mean 237
- An Empirical Multimodal Approach to Open-World Video Games 259
- List of Contributors 281
- Index 285
Chapters in this book
- Frontmatter I
- Preface and Acknowledgements V
- Contents VII
-
Part I: Introduction
- Empirical Multimodality Research: The State of Play 1
-
Part II: Charting Paths for Empirical Research: Theoretical and Methodological Reflections
- Dimensions of Materiality 35
- From Data to Patterns 65
- Thinking in Action 91
- Computational Approaches for the Interpretation of Image-Text Relations 109
-
Part III: Empirical Inroads: Case Studies and Results
- “I can’t see why you’re laughing”: Multimodal Analysis of Emotionalized Political Debate 141
- A Corpus-Based Approach to Color, Shape, and Typography in Logos 159
- Pixel Surgery and the Doctored Image 187
- Multimodal Discourse Analysis Based on the GeM Model 211
- Conventions in How Korean Films Mean 237
- An Empirical Multimodal Approach to Open-World Video Games 259
- List of Contributors 281
- Index 285
