Startseite Peircean Semiotics and Multimodality: Towards a New Synthesis
Artikel
Lizenziert
Nicht lizenziert Erfordert eine Authentifizierung

Peircean Semiotics and Multimodality: Towards a New Synthesis

  • John A. Bateman EMAIL logo
Veröffentlicht/Copyright: 27. April 2018

Abstract

The account of signs, signification and meaning set out by the philosopher Charles Sanders Peirce around the beginning of the twentieth century is a foundation stone of modern semiotics. In Peirce’s conception, semiotics concerned the process of signification at its most general and was intrinsically multimodal. It is then logical that contemporary multimodal research should engage with the Peircean view of semiotics. This in turn makes it particularly important that better understandings of the potential interconnections between Peircean semiotics and multimodality be achieved. Until now, however, such interconnections have been limited not only with respect to what has been drawn from Peirce’s account but also with respect to the areas of multimodality concerned. To deepen and broaden communication among approaches to multimodality and between those approaches and Peircean semiotics, this article sets out a reading of Peirce’s ‘mature’ 1903 position on semiotics aimed specifically at improving the utility of the account for multimodal analysis. The article argues that a lack of methodological precision within semiotics has led to relatively impoverished and empirically unreliable analyses. A new synthesis between Peirce’s account and multimodality is proposed as a stronger metatheoretical framework both for addressing multimodality and for drawing more from Peirce.

Acknowledgements

Several people have provided feedback on previous versions of this article or material presented here, including George Rossolatos, David Machin, Søren Vigild Poulsen, participants in the “Pictures and Texts–Pictures as Texts: Iconicity and Indexicality in Graphic Communication” conference of the Topoi Excellence Cluster of the Humboldt-University Berlin and the 3rd Bremen International Conference on Multimodality “Multimodality – Towards a new discipline”, as well as in courses held on Peirce at Bremen University. Thanks also go to particularly helpful comments from two anonymous reviewers. Any errors and misrepresentations remain the responsibility of the author.

References

Asher, Nicholas, and Lascarides, Alex. (2003). Logics of Conversation. Cambridge: Cambridge University Press.Suche in Google Scholar

Atã, Pedro, and Quieroz, João. (2014). Icon and abduction: Situatedness in Peircean cognitive semiotics. In: Model-Based Reasoning in Science and Technology (Studies in Applied Philosophy, Epistemology and Rational Ethics 8), L. Magani (Ed.), 301–313. Springer.10.1007/978-3-642-37428-9_17Suche in Google Scholar

Atkin, Albert. (2016). Peirce. Abingdon and New York: Routledge.Suche in Google Scholar

Bal, Mieke, and Bryson, Norman. (1991). Semiotics and art history. Art Bulletin, 73(2):174–208.10.2307/3045790Suche in Google Scholar

Barsalou, Lawrence W., Breazeal, Cynthia, and Smith, Linda B. (2007). Cognition as coordinated non-cognition. Cognitive Processes, 8:79–91.10.1007/s10339-007-0163-1Suche in Google Scholar

Barthes, Roland. (1964). Elements of Semiology. London: Jonathan Cape. Translated by Annette Lavers and Colin Smith.Suche in Google Scholar

Barthes, Roland. (1977 [1964]). The rhetoric of the image. In: Image–Music–Text, Stephen Heath (Ed.), 32–51. London: Fontana.Suche in Google Scholar

Bateman, John A. (2011). The decomposability of semiotic modes. In: MultiModal Studies: Multiple Approaches and Domains Routledge Studies in Multimodality, Kay L. O’Halloran and Bradley A. Smith (Eds.), 17–38. London: Routledge.Suche in Google Scholar

Bateman, John A. (2014). The constitutive role of semiotic modes for the theory and practice of multimodal analysis. In: Language Arts in Asia 2: English and Chinese through Literature, Drama and Popular Culture, Christina DeCoursey (Ed.), 8–33. Newcastle, UK: Cambridge Scholars Publishing.Suche in Google Scholar

Bateman, John A. (2016). Methodological and theoretical issues for the empirical investigation of multimodality. In: Sprache im Multimodalen Kontext / Language and Multimodality (Handbooks of Linguistics and Communication Science (HSK) 7), Nina-Maria Klug and Hartmut Stöckl (Eds.), 36–74. Berlin: de Gruyter Mouton.Suche in Google Scholar

Bateman, John A., and Wildfeuer, Janina. (2014). A multimodal discourse theory of visual narrative. Journal of Pragmatics, 74:180–218. doi:10.1016/j.pragma.2014.10.001.Suche in Google Scholar

Bateman, John A., Wildfeuer, Janina, and Hiippala, Tuomo. (2017). Multimodality – Foundations, Research and Analysis. A Problem-Oriented Introduction. Berlin: Mouton de Gruyter.Suche in Google Scholar

Bergman, Mats. (2009). Peirce’s Philosophy of Communication. London and New York: Contninuum.Suche in Google Scholar

Bezemer, Jeff, and Kress, Gunther. (2016). Multimodality, Learning and Communication: A Social Semiotic Frame. New York and London: Routledge.Suche in Google Scholar

Chandler, Daniel. (2007). Semiotics: The Basics. 2nd edn. London and New York: Routledge.10.4324/9780203014936Suche in Google Scholar

Cienki, Alan. (1998a). Metaphoric gestures and some of their relations to verbal metaphorical expressions. In: Discourse and Cognition: Bridging the Gap, Jean-Pierre Koenig (Ed.), 189–204. Stanford: Center for the Study of Language and Information.Suche in Google Scholar

Cienki, Alan. (1998b). STRAIGHT: An image schema and its metaphorical extensions. Cognitive Linguistics, 9:107–149.10.1515/cogl.1998.9.2.107Suche in Google Scholar

Clark, Andy, and Chalmers, David. (1998). The extended mind. Analysis, 58(1):7–19.10.1093/analys/58.1.7Suche in Google Scholar

Colapietro, Vincent. (2011). Image, diagram and metaphor. Unmined resource and unresolved questions. In: Semblance and Signification, Christina Ljungberg, Olga Fischer, and Pascal Michelucci (Eds.), 157–171. Amsterdam: John Benjamins.10.1075/ill.10.09colSuche in Google Scholar

Danaher, David. (1998). Peirce’s semiotic and cognitive metaphor theory. Semiotica, 119(1/2):171–207.Suche in Google Scholar

De Waal, Cornelus. (2013). Peirce: A Guide for the Perplexed. London: Bloomsbury.Suche in Google Scholar

Deledalle, Gérard. (1979). Théorie et pratique du signe: Introduction à la sémiotique de Charles S. Peirce. Paris: Payot.Suche in Google Scholar

Deledalle, Gérard. (2000). Charles S. Peirce’s Philosophy of Signs: Essays in Comparative Semiotics chap. Semeiotic and Semiology: Peirce and Saussure, 100–114. Bloomington and Indianopolis, IN: Indiana University Press.Suche in Google Scholar

Deppermann, Arnulf. (2013). Multimodal interaction from a conversation analytic perspective. Journal of Pragmatics, 46(1):1–7.10.1016/j.pragma.2012.11.014Suche in Google Scholar

Eco, Umberto. (1976). A Theory of Semiotics. Bloomington: Indiana University Press.10.1007/978-1-349-15849-2Suche in Google Scholar

Elleström, Lars. (2013). Spatiotemporal aspects of iconicity. In: Iconic investigations, Lars Elleström, Olga Fischer, and Christina Ljungberg (Eds.), Amsterdam: John Benjamins.10.1075/ill.12.09ellSuche in Google Scholar

Freadman, Anne. (1986). Structuralist uses of Peirce: Jakobson, Metz et al. In: Semiotics – language – ideology (Sydney Studies in Society and Culture 3), T. Threadgold, E. A. Grosz, Gunther Kress, and Michael A. K. Halliday (Eds.), 93–124. Sydney: Sydney Association for Studies in Society and Culture.Suche in Google Scholar

Fricke, Ellen. (2007). Origo, Geste und Raum: Lokaldeixis im Deutschen. Berlin: de Gruyter.10.1515/9783110897746Suche in Google Scholar

Fricke, Ellen. (2012). Grammatik multimodal: Wie Wörter und Gesten zusammenwirken. Berlin and New York: Mouton de Gruyter.10.1515/9783110218893Suche in Google Scholar

Fricke, Ellen. (2013). Between reference and meaning: Object-related and interpretant-related gestures in face-to-face interaction. In: Body–Language–Communication. An International Handbook on Multmodality in Human Interaction (Handbücher zur Sprach- und Kommunikationswissenschaft / Handbooks of Linguistics and Communication Science (HSK) 38/1), Cornelia Müller, Alan Cienki, Ellen Fricke, Silva Ladewig, David McNeill, and Sedinha Tessendorf (Eds.), 1788–1802. Berlin and New York: Mouton de Gruyter.10.1515/9783110302028.1788Suche in Google Scholar

Halliday, Michael A. K. (1978). Language as Social Semiotic. London: Edward Arnold.Suche in Google Scholar

Hjelmslev, Louis. (1961 [1943]). Prolegomena to a Theory of Language. Madison, Wisconsin: University of Wisconsin Press. Originally published 1943; Translated by F.J. Whitfield.Suche in Google Scholar

Hobbs, Jerry R. (1990). Literature and Cognition. Stanford, California: CSLI. CSLI Lecture Notes.Suche in Google Scholar

Hoven, Paul J. van den. (2010). The intriguing iconic sign. Interstudia (Review of Interdisciplinary Centre for Studies of Contemporary Discursive Forms) 6. 24–37. Issue on: Cultural Spaces and Identities in (inter)action.Suche in Google Scholar

Iversen, Margaret. (1986). Saussure v. Peirce: Models for a semiotics of visual art. In: The New Art History, A.L. Rees and Frances Borzello (Eds.), 82–94. London: Camden Press.Suche in Google Scholar

Jakobson, Roman. (1965). Quest for the essence of language. Diogenes, 13:21–37.10.1177/039219216501305103Suche in Google Scholar

Jakobson, Roman. (1990 [1956]). The two aspects of language and two types of aphasic disturbances. In: On Language, Linda R. Waugh and Monique Monville-Burston (Eds.), 115–133. Cambridge, MA and London: Harvard University Press.Suche in Google Scholar

Jappy, Tony. (2013). Introduction to Peircean Visual Semiotics Bloomsbury Advances in Semiotics. London and New York: Bloomsbury.Suche in Google Scholar

Kappelhoff, Hermann, and Müller, Cornelia. (2011). Embodied meaning construction. Multimodal metaphor and expressive movement in speech, gesture, and in feature film. Metaphor and the Social World, 1(2):121–153.10.1075/msw.1.2.02kapSuche in Google Scholar

Kendon, Alan. (2014). Semiotic diversity in utterance production and the concept of ‘language’. Philosophical Transactions of the Royal Society, B369(1651):20130293.10.1098/rstb.2013.0293Suche in Google Scholar

Kress, Gunther. (2003). Literacy in the New Media Age. London: Routledge.Suche in Google Scholar

Kress, Gunther. (2010). Multimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge.Suche in Google Scholar

Kress, Gunther, and van Leeuwen, Theo. (2001). Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold.Suche in Google Scholar

Kress, Gunther, and van Leeuwen, Theo. (2006). Reading Images: The Grammar of Visual Design. 2nd Edition. London and New York: Routledge.10.4324/9780203619728Suche in Google Scholar

Lakoff, George, and Johnson, Mark. (1980). Metaphors We Live by. Chicago, Il: Chicago University Press.Suche in Google Scholar

Lakoff, George, and Johnson, Mark. (1999). Philosophy in the Flesh: The Embodied Mind and Its Challenge to Western Thought. New York: Basic Books.Suche in Google Scholar

Lefebvre, Martin. (2007). The Art of Pointing. On Peirce, Indexicality, and Photographic Images. In: Photography theory, James Elkins (Ed.), 220–244. London: Routledge. The art seminar, Volume II.Suche in Google Scholar

Magnani, Lorenzo. (2001). Abduction, Reason, and Science. Processes of Discovery and Explanation. New York: Kluwer Academic/Plenum Publishers.10.1007/978-1-4419-8562-0Suche in Google Scholar

Manghani, Sunil. (2013). Image Studies: Theory and Practice. London: Routledge.Suche in Google Scholar

Mitchell, W.J.T. (1994). Picture Theory: Essays on Verbal and Visual Representation. Chicago: University of Chicago Press.Suche in Google Scholar

Mitchell, W.J.T. (2005). There are no visual media. Journal of Visual Culture, 4(2):257–266. doi:10.1177/1470412905054673.Suche in Google Scholar

Mittelberg, Irene. (2006). Metaphor and Metonymy in Language and Gesture: Discourse Evidence for Multimodal Models of Grammar. Ann Arbor, MI: UMI. Cornell University Dissertation.Suche in Google Scholar

Mittelberg, Irene. (2008). Peircean semiotics meets conceptual metaphor. iconic modes in gestural representations of grammar. In: Metaphor and Gesture, Alan Cienki and Cornelia Müller (Eds.), 115–154. Amsterdam and Philadelphia: John Benjamins Publishing Company.Suche in Google Scholar

Mittelberg, Irene, and Waugh, Linda R. (2009). Metonymy first, metaphor second: A cognitive-semiotic approach to multimodal figures of thought in co-speech gesture. In: Multimodal Metaphor, Charles J. Forceville, and Eduardo Urios-Aparisi (Eds.), 329–356. Berlin/New York: Mouton de Gruyter.Suche in Google Scholar

Mittelberg, Irene, and Waugh, Linda R. (2014). Gestures and metonymy. In: Body – Language – Communication. An International Handbook on Multimodality in Human Interaction, Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeil, and Jana Bressem (Eds.), vol. 2, 1747–1766. Berlin/Boston: Mouton.Suche in Google Scholar

Müller, C., and Cienki, Alan. (2009). Words, gestures, and beyond: Forms of multimodal metaphor in the use of spoken language. In: Multimodal Metaphor, Charles J. Forceville and Eduardo Urios-Aparisi (Eds.), 297–328. Berlin/New York: Mouton de Gruyter.Suche in Google Scholar

Müller, Cornelia. (1998). Redebegleitende Gesten: Kulturgeschichte – Theorie – Sprachvergleich. Berlin: Berliner Wissenschafts-Verlag.Suche in Google Scholar

Nöth, Winfried. (2010). The criterion of habit in Peirce’s definitions of the symbol. Transactions of the Charles S. Peirce Society, 46(1):82–93.10.2979/tra.2010.46.1.82Suche in Google Scholar

Nöth, Winfried. (2011). Visual semiotics: Key features and an application to picture ads. In: The SAGE Handbook of Visual Research Methods, Eric Margolis and Luc Pauwels (Eds.), 298–316. London.10.4135/9781446268278.n16Suche in Google Scholar

Nöth, Winfried. (2014). The growth of signs. Sign Systems Studies, 42(2/3):172–192.10.12697/SSS.2014.42.2-3.02Suche in Google Scholar

Nöth, Winfried, and Jungk, Isabel. (2015). Peircean visual semiotics: Potentials to be explored. Semiotica, 207:657–673.10.1515/sem-2015-0058Suche in Google Scholar

Parret, Herman. (1984). Peirce and Hjelmslev: The two semiotics. Language Sciences, 6(2):217–227.10.1016/S0388-0001(84)80017-0Suche in Google Scholar

Peirce, Charles Sanders. (1931–1958). Collected Papers of Charles Sanders Peirce. Cambridge, MA: Harvard University Press. Vols. 1–6, 1931–1935, edited by Charles Hartshorne and Paul Weiss; Vols. 7–8, 1958, edited by Arthur W. Burks.Suche in Google Scholar

Peirce, Charles Sanders. (1998 [1893–1913]). The Essential Peirce – Volume 2. Selected Philosophical Writings (1893–1913). Bloomington: Indiana University Press.Suche in Google Scholar

Queiroz, João, and Merrell, Floyd. (2009). On Peirce’s pragmatic notion of semiosis a contribution for the design of meaning machines. Minds & Machines 19:129–143.10.1007/s11023-008-9129-zSuche in Google Scholar

Scott, David. (1997). Indexical/iconic tensions: The semiotics of the postage stamp. In: Semiotics of the Media. State of the Art, Projects, and Perspectives (Approaches to Semiotics 127), Winfried Nöth (Ed.), 191–201. Berlin/New York: Mouton de Gruyter.10.1515/9783110803617-014Suche in Google Scholar

Short, Thomas L. (1982). Life among the legisigns. Transactions of the Charles S. Peirce Society, 18(4):285–310.Suche in Google Scholar

Short, Thomas L. (1996). Interpreting Peirce’s Interpretant: A response to Lalor, Liszka, and Meyers. Transactions of the Charles S. Peirce Society, 32(4):488–541.Suche in Google Scholar

Short, Thomas L. (2007). Peirce’s Theory of Signs. Cambridge: Cambridge University Press.10.1017/CBO9780511498350Suche in Google Scholar

Skagested, Peter. (2004). Peirce’s semeiotic model of the mind. In: The Cambridge Companion to Peirce Cambridge Companions to Philosophy, Cheryl Misak (Ed.), 241–256. Cambridge: Cambridge University Press.10.1017/CCOL0521570069.010Suche in Google Scholar

Stjernfelt, Frederik. (2000). Diagrams as centerpiece of a Peircean epistemology. Transactions of the Charles S. Peirce Society, 36(3): 357–384.Suche in Google Scholar

Stjernfelt, Frederik. (2014). Natural Propositions: The Actuality of Peirce’s Doctrine of Dicisigns. Boston, MA: The Docent Press.Suche in Google Scholar

Stöckl, Hartmut. (2004). In between modes: language and image in printed media. In: Perspectives on Multimodality, Eija Ventola, Cassily Charles, and Martin Kaltenbacher (Eds.), 9–30. Amsterdam: John Benjamins.10.1075/ddcs.6.03stoSuche in Google Scholar

Varela, Francisco J., Thompson, Evan, and Rosch, Eleanor. (1991). The Embodied Mind: Cognitive Science and Human Experience. Cambridge, MA: MIT Press.10.7551/mitpress/6730.001.0001Suche in Google Scholar

Wharton, Tim. (2009). Pragmatics and Non-Verbal Communication. Cambridge: Cambridge University Press.10.1017/CBO9780511635649Suche in Google Scholar

Published Online: 2018-4-27

© 2018 Walter de Gruyter GmbH, Berlin/Boston

Heruntergeladen am 27.10.2025 von https://www.degruyterbrill.com/document/doi/10.1515/mc-2017-0021/html
Button zum nach oben scrollen