Extending local grammar to multimodal meaning-making: a case study of conflict resolution in picture books
-
Ding Huang
Ding Huang is a post-doctoral researcher at the National Research Centre for Foreign Language Education, Beijing Foreign Studies University, China. She obtained her Ph.D. from Ruprecht-Karls-Universität Heidelberg, with a dissertation titled
Formulaic Sequences in Early Modern English: A Corpus-assisted Historical Pragmatic Study . Her research interests include multimodal discourse analysis, corpus linguistics, construction grammar, formulaic sequences, and pragmatics of Early Modern English.
and
Jiajin Xu
Jiajin Xu is professor of linguistics at the National Research Centre for Foreign Language Education, Beijing Foreign Studies University as well as secretary general of Corpus Linguistics Society of China. His research interests span corpus linguistics, discourse studies, second language acquisition, contrastive linguistics and translation studies. He has published papers in international journals including
Corpora ,Corpus Linguistics and Linguistic Theory ,Corpus Pragmatics ,Discourse & Society ,English for Specific Purposes ,International Journal of Corpus Linguistics ,Journal of Historical Pragmatics ,Lingua, Language Sciences andSemiotica .
Abstract
This case study attempts to describe the mechanism of meaning-making in picture books as the unity of texts and images. The study proposes that the applicability of meaning/function-based local grammar extends to multimodal discourse. By examining six picture books considered prototypical conflict narratives, our study identified and analyzed multimodal local grammar patterns accounting for three meanings: evaluation, ownership, and identification. The findings revealed that some meaning units in verbal and visual modes possess stronger semantic and pragmatic relationships among themselves than with others. These multimodal meaning units further co-select to form larger meaning units. The study also demonstrated how these patterns reflect conflict resolution strategies. Three strategies were observed: updating specific functional elements of a multimodal local grammar pattern, replacing the entire pattern, and forming non-conventional verbal-visual co-selection relationships within a pattern. These findings may contribute to communication analysis and foreign language education.
Funding source: The China MOE Major Project Fund of Key Research Center for Humanities and Social Sciences
Award Identifier / Grant number: 22JJD740012
About the authors
Ding Huang is a post-doctoral researcher at the National Research Centre for Foreign Language Education, Beijing Foreign Studies University, China. She obtained her Ph.D. from Ruprecht-Karls-Universität Heidelberg, with a dissertation titled Formulaic Sequences in Early Modern English: A Corpus-assisted Historical Pragmatic Study. Her research interests include multimodal discourse analysis, corpus linguistics, construction grammar, formulaic sequences, and pragmatics of Early Modern English.
Jiajin Xu is professor of linguistics at the National Research Centre for Foreign Language Education, Beijing Foreign Studies University as well as secretary general of Corpus Linguistics Society of China. His research interests span corpus linguistics, discourse studies, second language acquisition, contrastive linguistics and translation studies. He has published papers in international journals including Corpora, Corpus Linguistics and Linguistic Theory, Corpus Pragmatics, Discourse & Society, English for Specific Purposes, International Journal of Corpus Linguistics, Journal of Historical Pragmatics, Lingua, Language Sciences and Semiotica.
-
Conflict of interest: None.
-
Research funding: The work was supported by the China MOE Major Project Fund of Key Research Center for Humanities and Social Sciences. Grant number: 22JJD740012.
References
Allwood, Jens & Elisabeth Ahlsén. 2015. On stages of conflict escalation. In Francesca D’Errico, Isabella Poggi, Alessandro Vinciarelli & Laura Vincze (eds.), Conflict and multimodal communication: Social research and machine intelligence, 53–70. Dordrecht: Springer International Publishing.10.1007/978-3-319-14081-0_3Search in Google Scholar
Barnbrook, Geoff. 2002. Defining language: A local grammar of definition sentences. Amsterdam: John Benjamins.10.1075/scl.11Search in Google Scholar
Bateman, John. 2014. Text and image: A critical introduction to the visual/verbal divide. London: Routledge.10.4324/9781315773971Search in Google Scholar
Brahm, Eric. 2003. Conflict stages. In Guy Burgess & Heidi Burgess (eds.), Beyond intractability. Conflict information consortium. Boulder: University of Colorado. http://www.beyondintractability.org/essay/conflict-stages (accessed 12 January 2024).Search in Google Scholar
Castelfranchi, Cristiano. 2015. The cognition of conflict: Ontology, dynamics, and ideology. In Francesca D’Errico, Isabella Poggi, Alessandro Vinciarelli & Laura Vincze (eds.), Conflict and multimodal communication: Social research and machine intelligence, 3–32. Dordrecht: Springer International Publishing.10.1007/978-3-319-14081-0_1Search in Google Scholar
Chen, Yaoyao & Svenja Adolphs. 2023. Towards a speech-gesture profile of pragmatic markers: The case of “you know”. Journal of Pragmatics 210. 36–51. https://doi.org/10.1016/j.pragma.2023.03.004.Search in Google Scholar
D’Errico, Francesca, Isabella Poggi, Alessandro Vinciarelli & Laura Vincze (eds.). 2015. Conflict and multimodal communication: Social research and machine intelligence. Cham: Springer.10.1007/978-3-319-14081-0Search in Google Scholar
Greiffenhagen, Christian. 2013. Visual grammar in practice: Negotiating the arrangement of speech bubbles in storyboards. Semiotica 195. 127–167. https://doi.org/10.1515/sem-2013-0008.Search in Google Scholar
Gross, Maurice. 1993. Local grammars and their representation by finite automata. In Michael Hoey (ed.), Data, description, discourse: Papers on the English language in honour of john McH Sinclair on his sixtieth birthday, 26–38. London: HarperCollins.Search in Google Scholar
Halliday, Michael A. K. 1985. An introduction to functional grammar. London: Edward Arnold.Search in Google Scholar
Hilpert, Martin. 2014. Construction grammar and its application to English: Edinburgh textbooks on the English language (Advanced). Edinburgh: Edinburgh University Press.Search in Google Scholar
Hoffmann, Thomas. 2021. Multimodal construction grammar: From multimodal constructs to multimodal constructions. In Wen Xu & John R. Taylor (eds.), The Routledge handbook of cognitive linguistics, 78–92. London: Routledge.10.4324/9781351034708-6Search in Google Scholar
Hunston, Susan. 2002. Pattern grammar, language teaching, and linguistic variation: Applications of a corpus-driven grammar. In Randi Reppen, Susan Fitzmaurice & Douglas Biber (eds.), Using corpora to explore linguistic variation, 167–183. Amsterdam: John Benjamins.10.1075/scl.9.11hunSearch in Google Scholar
Hunston, Susan & John Sinclair. 2000. A local grammar of evaluation. In Susan Hunston & Geoff Thompson (eds.), Evaluation in text: Authorial stance and the construction of discourse, 74–101. Oxford: Oxford University Press.10.1093/oso/9780198238546.003.0005Search in Google Scholar
Hunston, Susan & Hang Su. 2019. Patterns, constructions, and local grammar: A case study of ‘evaluation’. Applied Linguistics 2017. 1–28. https://doi.org/10.1093/applin/amx046.Search in Google Scholar
Kress, Gunther & Theo van Leeuwen. 2021 [2006]. Reading images: The grammar of visual design, 3rd edn. London: Routledge.10.4324/9781003099857Search in Google Scholar
Moya-Guijarro, Arsenio Jesús. 2015. Visual metonymy in children’s picture books. In María Jesús Pinar Sanz (ed.), Multimodality and cognitive linguistics, 115–130. Amsterdam: John Benjamins.10.1075/bct.78.08moySearch in Google Scholar
Moya-Guijarro, Arsenio Jesús. 2019. Textual functions of metonymies in Anthony Browne’s picture books: A multimodal approach. Text & Talk 39(3). 389–413. https://doi.org/10.1515/text-2019-2034.Search in Google Scholar
Muller, Stacy & Amanda Willcox. 2018. 11.2 four major types of conflict, chapter 11. Identifying and understanding how to manage conflict. In Joan Wagner (ed.), Leadership and influencing change in nursing. https://pressbooks.pub/leadershipandinfluencingchangeinnursing/chapter/chapter-11-identifying-and-understanding-how-to-manage-conflict/ (accessed on 12 January 2024).Search in Google Scholar
Northrup, Mary. 2012. Picture books for children: Fiction, folktales, and poetry. Chicago: American Library Association.Search in Google Scholar
Painter, Clare, James R. Martin & Len Unsworth. 2013. Reading visual narratives: Image analysis of children’s picture books. Sheffield: Equinox Publishing Ltd.Search in Google Scholar
Pérez-Sobrino, Paula. 2017. Multimodal metaphor and metonymy in advertising. Amsterdam: John Benjamins.10.1075/ftl.2Search in Google Scholar
Poggi, Isabella, Francesca D’Errico & Laura Vincze. 2011. Agreement and its multimodal communication in debates: A qualitative analysis. Cognitive Computation 3. 466–479. https://doi.org/10.1007/s12559-010-9068-x.Search in Google Scholar
Poggi, Isabella, Francesca D’Errico & Laura Vincze. 2013. Comments by words, face and body. Journal on Multimodal User Interfaces 7. 67–78. https://doi.org/10.1007/s12193-012-0102-z.Search in Google Scholar
Pontrandolfo, Gianluca & Stanisław Goźdź-Roszkowski. 2014. Exploring the local grammar of evaluation: The case of adjectival patterns in American and Italian judicial discourse. Research in Language 12(1). 71–91. https://doi.org/10.2478/rela-2014-0014.Search in Google Scholar
Porello, Daniele, Emanuele Bottazzi & Roberta Ferrario. 2015. Group conflict as social contradiction. In Francesca D’Errico, Isabella Poggi, Alessandro Vinciarelli & Laura Vincze (eds.), Conflict and multimodal communication: Social research and machine intelligence, 33–52. Dordrecht: Springer International Publishing.10.1007/978-3-319-14081-0_2Search in Google Scholar
Royce, Terry. 1998. Synergy on the page: Exploring intersemiotic complementarity in page-based multimodal text. Japan Association for Systemic Functional Linguistics Occasional Papers 1(1). 25–49.Search in Google Scholar
Serafini, Frank & Stephanie F. Reid. 2024. Analyzing picturebooks: Semiotic, literary, and artistic frameworks. Visual Communication 23(2). 330–350. https://doi.org/10.1177/14703572211069623.Search in Google Scholar
Sinclair, John. 2008. The phrase, the whole phrase, and nothing but the phrase. In Sylviane Granger & Fanny Meunier (eds.), Phraseology: An interdisciplinary perspective, 407–410. Amsterdam: John Benjamins.10.1075/z.139.33sinSearch in Google Scholar
Stubbs, Michael. 2009. Memorial article: John Sinclair (1933–2007): The search for units of meaning: Sinclair on empirical semantics. Applied Linguistics 30(1). 115–137. https://doi.org/10.1093/applin/amn052.Search in Google Scholar
Su, Hang. 2017. Local grammars of speech acts: An exploratory study. Journal of Pragmatics 111. 73–82. https://doi.org/10.1016/j.pragma.2017.02.008.Search in Google Scholar
Su, Hang. 2018. “Thank bloody God it’s Friday”: A local grammar of thanking. Corpus Pragmatics 2. 83–105. https://doi.org/10.1007/s41701-017-0024-9.Search in Google Scholar
Su, Hang. 2020. Local grammars and diachronic speech act analysis: A case study of apology in the history of American English. Journal of Historical Pragmatics 21(1). 109–136. https://doi.org/10.1075/jhp.00038.su.Search in Google Scholar
Su, Hang & Xiaofei Lu. 2022. Assessing pragmatic performance in advanced L2 academic writing through the lens of local grammars: A case study of “exemplification”. Assessing Writing 54. 1–12. https://doi.org/10.1016/j.asw.2022.100668.Search in Google Scholar
Su, Hang & Xiaofei Lu. 2023. Local grammars and intercultural speech act studies: A study of apologies in four English varieties. Intercultural Pragmatics 20(4). 377–404. https://doi.org/10.1515/ip-2023-4002.Search in Google Scholar
Su, Hang & Naixing Wei. 2018. “I’m really sorry about what I said”: A local grammar of apology. Pragmatics 28(3). 439–462. https://doi.org/10.1075/prag.17005.su.Search in Google Scholar
Swanström, Niklas L. P. & Mikael S. Weissmann. 2005. Conflict, conflict prevention, conflict management and beyond: A conceptual exploration. Washington: Central Asia-Caucasus Institute & Silk Road Studies Program.Search in Google Scholar
Torrent, Tiago Timponi, Ely Edison da Silva Matos, Frederico Belcavello, Marcelo Viridiano, Maucha Andrade Gamonal, Alexandre Diniz da Costa & Mateus Coutinho Marim. 2022. Representing context in FrameNet: A multidimensional, multimodal approach. Frontiers in Psychology 13. 1–20. https://doi.org/10.3389/fpsyg.2022.838441.Search in Google Scholar
Wei, Naixing. 2011. Theory of co-selection and the corpus-driven phraseology. Journal of PLA University of Foreign Languages 35(1). 1–5.Search in Google Scholar
Xu, Jiajin. 2023. Co-selection and categorization. In Presentation at the international conference on advances in corpus-based interdisciplinary research, Xi'an: Xi’an Jiaotong University, 27–29 October.Search in Google Scholar
Zhang, Lei, Rui Jiang & Junmei Zhang. 2024. Table 1 shows that…: A local grammar of graphic data commentary in discourse of Economics. English for Specific Purposes 74. 68–81. https://doi.org/10.1016/j.esp.2024.01.001.Search in Google Scholar
Zhang, Yimin & Hang Su. 2022. “We define X as…”: A local grammar of definition in linguistics research articles and its pedagogical value. Journal of English for Academic Purposes 59. 1–12. https://doi.org/10.1016/j.jeap.2022.101143.Search in Google Scholar
© 2025 Walter de Gruyter GmbH, Berlin/Boston
