Startseite Growing theory for practice: empirical multimodality beyond the case study
Artikel
Lizenziert
Nicht lizenziert Erfordert eine Authentifizierung

Growing theory for practice: empirical multimodality beyond the case study

  • John A. Bateman ORCID logo EMAIL logo
Veröffentlicht/Copyright: 30. September 2021

Abstract

Many studies investigating the use and effectiveness of multimodal communication are now confronting the need to engage with larger bodies of data in order to achieve more empirically robust accounts, moving beyond the earlier prevalence of small-scale ‘case studies’. In this article, I briefly characterise how recent developments in the theory of multimodality can be drawn upon to encourage and support this change in both scale and breadth. In particular, the contribution will show how refinements in the degree of formality of definitions of the core multimodal constructs of ‘semiotic mode’ and ‘materiality’ can help bridge the gap between exploratory investigations of complex multimodal practices and larger-scale corpus studies.


Corresponding author: John A. Bateman, Linguistics, Fachbereich 10 Sprach- und Literaturwissenschaften, Universitat Bremen, Bremen, Germany, E-mail:

References

Ayaß, R. (2015). Doing data: the status of transcripts in conversation analysis. Discourse Stud. 17: 505–528, https://doi.org/10.1177/1461445615590717.Suche in Google Scholar

Bateman, J.A. (2020). The foundational role of discourse semantics beyond language. In: Zappavigna, M. and Dreyfus, S. (Eds.), Discourses of hope and reconciliation. On J. R. Martin’s contribution to systemic functional linguistics. Routledge Studies in Multimodality. Bloomsbury, London, New York, pp. 39–55.10.5040/9781350116092.ch-003Suche in Google Scholar

Bateman, J.A. (2021a). Dimensions of materiality: towards an external language of description for empirical multimodality research. In: Jana, P., Wildfeuer, J., and Bateman, J.A. (Eds.), Empirical multimodality research: methods, evaluations, implications. de Gruyter, Berlin, pp. 35–63.10.1515/9783110725001-002Suche in Google Scholar

Bateman, J.A. (2021b). What are digital media? Discourse, Context & Media 41, https://doi.org/10.1016/j.dcm.2021.100502.Suche in Google Scholar

Bateman, J.A., Veloso, F.O.D., and Lau, Y.L. (2021). On the track of visual style: a diachronic study of page composition in comics and its functional motivation. Vis. Commun. 20: 209–247, doi:https://doi.org/10.1177/1470357219839101.Suche in Google Scholar

Bateman, J.A., Wildfeuer, J., and Hiippala, T. (2017). Multimodality – foundations, research and analysis. A problem-oriented introduction. de Gruyter Mouton, Berlin.10.1515/9783110479898Suche in Google Scholar

Bezemer, J. (2014). Multimodal transcription: a case study. In: Norris, S. and Maier, C.D. (Eds.), Interactions, images and texts: a reader in multimodality. de Gruyter Mouton, Berlin, pp. 155–170.10.1515/9781614511175.155Suche in Google Scholar

Bezemer, J. and Mavers, D. (2011). Multimodal transcription as academic practice: a social semiotic perspective. Int. J. Soc. Res. Methodol. 14: 191–206, https://doi.org/10.1080/13645579.2011.563616.Suche in Google Scholar

Cowan, K. (2020). Tracing the ephemeral: mapping young children’s running games. Des. Learn. 12: 81–93, https://doi.org/10.16993/dfl.90.Suche in Google Scholar

Cowan, K. and Kress, G. (2017). Documenting and transferring meaning in the multimodal world: reconsidering “transcription”. In: Frank, S. and Gee, E. (Eds.), Remixing multiliteracies: theory and practice from New London to new times. Teachers College Press, New York, London, pp. 50–61.Suche in Google Scholar

Crawford Camiciottoli, B. and Bonsignori, V. (2015). The pisa audiovisual corpus project: a multimodal approach to ESP research and teaching. E.S.P. Today: J. Engl. Specif. Purp. Tert. Level 3: 139–159.Suche in Google Scholar

Flewitt, R. (2006). Using video to investigate preschool classroom interaction: education research assumptions and methodological practices. Vis. Commun. 5: 25–51, https://doi.org/10.1177/1470357206060917.Suche in Google Scholar

Heath, C., Hindmarsh, J., and Paul, L. (2010). Video in qualitative research: analysing social interaction in everyday life. Sage, Los Angeles, London.10.4135/9781526435385Suche in Google Scholar

Kress, G., Carey, J., Ogborn, J., and Tsatsarelis, C. (2000). Multimodal teaching and learning. Continuum, London.Suche in Google Scholar

Kress, G. and van Leeuwen, T. (2006 [1996]). Reading images: the grammar of visual design. Routledge, London, New York.10.4324/9780203619728Suche in Google Scholar

Lapadat, J.C. (2000). Problematizing transcription: purpose, paradigm and quality. Int. J. Soc. Res. Methodol. 3: 203–219, https://doi.org/10.1080/13645570050083698.Suche in Google Scholar

Levinson, S.C. (2006). On the human ‘interaction engine’. In: Enfield, N.J. and Levinson, S.C. (Eds.), Roots of human socialiy: culture, cognition and interaction. Berg, Oxford, New York, pp. 39–69.10.4324/9781003135517-3Suche in Google Scholar

MacNaught, L. (2019). Multimodal metalanguage. In: de Silva Joyce, H. and Feez, S. (Eds.), Multimodality across classrooms. Learning about and through different modalities. Routledge, London, New York, pp. 144–159.10.4324/9780203701072-10Suche in Google Scholar

Malinowski, B. (1923). The problem of meaning in primitive languages. Harcourt, Brace, and Co., Inc, New York, pp. 451–510, Supplement I to C.K.I.A. Ogden and Richards The Meaning of Meaning.Suche in Google Scholar

Mills, K.A., and Unsworth, L. (2017). Multimodal literacy. In: Oxford research encyclopedia of education. Oxford University Press, New York. http://education.oxfordre.com/view/10.1093/acrefore/9780190264093.001.0001/acrefore-9780190264093-e-232.10.1093/acrefore/9780190264093.013.232Suche in Google Scholar

Mondada, L. (2007). Commentary: transcript variations and the indexicality of transcribing practices. Discourse Stud. 9: 809–821, https://doi.org/10.1177/1461445607082581.Suche in Google Scholar

Mondada, L. (2008). Using video for a sequential and multimodal analysis of social interaction: videotaping institutional telephone calls [88 paragraphs]. Forum Qual. Sozialforsch./Forum Qual. Soc. Res. 9: Art.-39, https://doi.org/10.17169/fqs-9.3.1161.Suche in Google Scholar

Murphey, K.M. (2005). Collaborative imagining: the interactive use of gestures, talk, and graphic representation in architectural practice. Semiotica 156: 113–145.10.1515/semi.2005.2005.156.113Suche in Google Scholar

Norris, S. (2014). Modal density and modal configurations: multimodal actions. In: Carey, J. (Ed.), The Routledge handbook of multimodal analysis, 2nd ed. Routledge, London, pp. 86–99.Suche in Google Scholar

Ochs, E. (1979). Social foundations of language. In: Freedle, R.O. (Ed.), Discourse processes: advances in research and theory. Volume 2: new directions in discourse processing. Norwood, New Jersey, Ablex, pp. 207–221.Suche in Google Scholar

Pike, K.L. (1967). Language in relation to a unified theory of the structure of human behaviour, 2nd ed. The Hague: Mouton.10.1515/9783111657158Suche in Google Scholar

Potter, J. and Cowan, K. (2020). Playground as meaning-making space: multimodal making and re-making of meaning in the (virtual) playground. Global Stud. Child. 10: 248–263, https://doi.org/10.1177/2043610620941527.Suche in Google Scholar

Prior, P. (2013). Multimodality and ESP research. In: Paltridge, B. and Starfield, S. (Eds.), The handbook of English for specific purposes. Wiley-Blackwell, Chichester, U.K., pp. 519–534.10.1002/9781118339855.ch27Suche in Google Scholar

Steen, F.F., Anders, H., Joo, J., Olza, I., Cánovas, C.P., Anna, P., Ray, S., Peter, U., Valenzuela, J., and Woźny, J., et al.. (2018). Toward an infrastructure for data-driven multimodal communication research. Ling. Vanguard 4: 20170041, doi:https://doi.org/10.1515/lingvan-2017-0041.Suche in Google Scholar

Received: 2021-03-09
Accepted: 2021-09-17
Published Online: 2021-09-30

© 2021 Walter de Gruyter GmbH, Berlin/Boston

Heruntergeladen am 26.11.2025 von https://www.degruyterbrill.com/document/doi/10.1515/mc-2021-0006/html
Button zum nach oben scrollen