Multimodal Discourse Analysis Based on the GeM Model
-
Jiaping Kang
Abstract
Multimodal research is naturally interdisciplinary, allowing research studied from different disciplines and theoretical perspectives. The main challenge is the lack of empirical research with large corpora, especially when applied to visuo-verbal discourse. This chapter takes the GeM (Genre and Multimodality) model as the theoretical framework to analyze environment protection posters from China and the USA. The data set consists of ten posters, with five posters in each sub-corpus, from governmental agencies and organizations. The GeM model adopts XML (eXtensible Markup Language) to annotate the rhetorical structure of the posters, and uses GeM-Tools to conduct the research. The research analyzes the semiotic resources (language, images, and layout) participating in meaning generation to investigate how modes construct meaning together, and to clarify the typical features and relations between language and images of environment protection posters as multimodal discourses. Through an empirical research, the chapter undertakes a contrastive analysis of characteristics in semiotic resources of Chinese and American environment protection posters, conducts statistical tests to examine the significance of the differences, and explores the reasons for differences from a cross-cultural perspective.
Abstract
Multimodal research is naturally interdisciplinary, allowing research studied from different disciplines and theoretical perspectives. The main challenge is the lack of empirical research with large corpora, especially when applied to visuo-verbal discourse. This chapter takes the GeM (Genre and Multimodality) model as the theoretical framework to analyze environment protection posters from China and the USA. The data set consists of ten posters, with five posters in each sub-corpus, from governmental agencies and organizations. The GeM model adopts XML (eXtensible Markup Language) to annotate the rhetorical structure of the posters, and uses GeM-Tools to conduct the research. The research analyzes the semiotic resources (language, images, and layout) participating in meaning generation to investigate how modes construct meaning together, and to clarify the typical features and relations between language and images of environment protection posters as multimodal discourses. Through an empirical research, the chapter undertakes a contrastive analysis of characteristics in semiotic resources of Chinese and American environment protection posters, conducts statistical tests to examine the significance of the differences, and explores the reasons for differences from a cross-cultural perspective.
Chapters in this book
- Frontmatter I
- Preface and Acknowledgements V
- Contents VII
-
Part I: Introduction
- Empirical Multimodality Research: The State of Play 1
-
Part II: Charting Paths for Empirical Research: Theoretical and Methodological Reflections
- Dimensions of Materiality 35
- From Data to Patterns 65
- Thinking in Action 91
- Computational Approaches for the Interpretation of Image-Text Relations 109
-
Part III: Empirical Inroads: Case Studies and Results
- “I can’t see why you’re laughing”: Multimodal Analysis of Emotionalized Political Debate 141
- A Corpus-Based Approach to Color, Shape, and Typography in Logos 159
- Pixel Surgery and the Doctored Image 187
- Multimodal Discourse Analysis Based on the GeM Model 211
- Conventions in How Korean Films Mean 237
- An Empirical Multimodal Approach to Open-World Video Games 259
- List of Contributors 281
- Index 285
Chapters in this book
- Frontmatter I
- Preface and Acknowledgements V
- Contents VII
-
Part I: Introduction
- Empirical Multimodality Research: The State of Play 1
-
Part II: Charting Paths for Empirical Research: Theoretical and Methodological Reflections
- Dimensions of Materiality 35
- From Data to Patterns 65
- Thinking in Action 91
- Computational Approaches for the Interpretation of Image-Text Relations 109
-
Part III: Empirical Inroads: Case Studies and Results
- “I can’t see why you’re laughing”: Multimodal Analysis of Emotionalized Political Debate 141
- A Corpus-Based Approach to Color, Shape, and Typography in Logos 159
- Pixel Surgery and the Doctored Image 187
- Multimodal Discourse Analysis Based on the GeM Model 211
- Conventions in How Korean Films Mean 237
- An Empirical Multimodal Approach to Open-World Video Games 259
- List of Contributors 281
- Index 285
