Abstract
The account of signs, signification and meaning set out by the philosopher Charles Sanders Peirce around the beginning of the twentieth century is a foundation stone of modern semiotics. In Peirce’s conception, semiotics concerned the process of signification at its most general and was intrinsically multimodal. It is then logical that contemporary multimodal research should engage with the Peircean view of semiotics. This in turn makes it particularly important that better understandings of the potential interconnections between Peircean semiotics and multimodality be achieved. Until now, however, such interconnections have been limited not only with respect to what has been drawn from Peirce’s account but also with respect to the areas of multimodality concerned. To deepen and broaden communication among approaches to multimodality and between those approaches and Peircean semiotics, this article sets out a reading of Peirce’s ‘mature’ 1903 position on semiotics aimed specifically at improving the utility of the account for multimodal analysis. The article argues that a lack of methodological precision within semiotics has led to relatively impoverished and empirically unreliable analyses. A new synthesis between Peirce’s account and multimodality is proposed as a stronger metatheoretical framework both for addressing multimodality and for drawing more from Peirce.
Acknowledgements
Several people have provided feedback on previous versions of this article or material presented here, including George Rossolatos, David Machin, Søren Vigild Poulsen, participants in the “Pictures and Texts–Pictures as Texts: Iconicity and Indexicality in Graphic Communication” conference of the Topoi Excellence Cluster of the Humboldt-University Berlin and the 3rd Bremen International Conference on Multimodality “Multimodality – Towards a new discipline”, as well as in courses held on Peirce at Bremen University. Thanks also go to particularly helpful comments from two anonymous reviewers. Any errors and misrepresentations remain the responsibility of the author.
References
Asher, Nicholas, and Lascarides, Alex. (2003). Logics of Conversation. Cambridge: Cambridge University Press.Suche in Google Scholar
Atã, Pedro, and Quieroz, João. (2014). Icon and abduction: Situatedness in Peircean cognitive semiotics. In: Model-Based Reasoning in Science and Technology (Studies in Applied Philosophy, Epistemology and Rational Ethics 8), L. Magani (Ed.), 301–313. Springer.10.1007/978-3-642-37428-9_17Suche in Google Scholar
Atkin, Albert. (2016). Peirce. Abingdon and New York: Routledge.Suche in Google Scholar
Bal, Mieke, and Bryson, Norman. (1991). Semiotics and art history. Art Bulletin, 73(2):174–208.10.2307/3045790Suche in Google Scholar
Barsalou, Lawrence W., Breazeal, Cynthia, and Smith, Linda B. (2007). Cognition as coordinated non-cognition. Cognitive Processes, 8:79–91.10.1007/s10339-007-0163-1Suche in Google Scholar
Barthes, Roland. (1964). Elements of Semiology. London: Jonathan Cape. Translated by Annette Lavers and Colin Smith.Suche in Google Scholar
Barthes, Roland. (1977 [1964]). The rhetoric of the image. In: Image–Music–Text, Stephen Heath (Ed.), 32–51. London: Fontana.Suche in Google Scholar
Bateman, John A. (2011). The decomposability of semiotic modes. In: MultiModal Studies: Multiple Approaches and Domains Routledge Studies in Multimodality, Kay L. O’Halloran and Bradley A. Smith (Eds.), 17–38. London: Routledge.Suche in Google Scholar
Bateman, John A. (2014). The constitutive role of semiotic modes for the theory and practice of multimodal analysis. In: Language Arts in Asia 2: English and Chinese through Literature, Drama and Popular Culture, Christina DeCoursey (Ed.), 8–33. Newcastle, UK: Cambridge Scholars Publishing.Suche in Google Scholar
Bateman, John A. (2016). Methodological and theoretical issues for the empirical investigation of multimodality. In: Sprache im Multimodalen Kontext / Language and Multimodality (Handbooks of Linguistics and Communication Science (HSK) 7), Nina-Maria Klug and Hartmut Stöckl (Eds.), 36–74. Berlin: de Gruyter Mouton.Suche in Google Scholar
Bateman, John A., and Wildfeuer, Janina. (2014). A multimodal discourse theory of visual narrative. Journal of Pragmatics, 74:180–218. doi:10.1016/j.pragma.2014.10.001.Suche in Google Scholar
Bateman, John A., Wildfeuer, Janina, and Hiippala, Tuomo. (2017). Multimodality – Foundations, Research and Analysis. A Problem-Oriented Introduction. Berlin: Mouton de Gruyter.Suche in Google Scholar
Bergman, Mats. (2009). Peirce’s Philosophy of Communication. London and New York: Contninuum.Suche in Google Scholar
Bezemer, Jeff, and Kress, Gunther. (2016). Multimodality, Learning and Communication: A Social Semiotic Frame. New York and London: Routledge.Suche in Google Scholar
Chandler, Daniel. (2007). Semiotics: The Basics. 2nd edn. London and New York: Routledge.10.4324/9780203014936Suche in Google Scholar
Cienki, Alan. (1998a). Metaphoric gestures and some of their relations to verbal metaphorical expressions. In: Discourse and Cognition: Bridging the Gap, Jean-Pierre Koenig (Ed.), 189–204. Stanford: Center for the Study of Language and Information.Suche in Google Scholar
Cienki, Alan. (1998b). STRAIGHT: An image schema and its metaphorical extensions. Cognitive Linguistics, 9:107–149.10.1515/cogl.1998.9.2.107Suche in Google Scholar
Clark, Andy, and Chalmers, David. (1998). The extended mind. Analysis, 58(1):7–19.10.1093/analys/58.1.7Suche in Google Scholar
Colapietro, Vincent. (2011). Image, diagram and metaphor. Unmined resource and unresolved questions. In: Semblance and Signification, Christina Ljungberg, Olga Fischer, and Pascal Michelucci (Eds.), 157–171. Amsterdam: John Benjamins.10.1075/ill.10.09colSuche in Google Scholar
Danaher, David. (1998). Peirce’s semiotic and cognitive metaphor theory. Semiotica, 119(1/2):171–207.Suche in Google Scholar
De Waal, Cornelus. (2013). Peirce: A Guide for the Perplexed. London: Bloomsbury.Suche in Google Scholar
Deledalle, Gérard. (1979). Théorie et pratique du signe: Introduction à la sémiotique de Charles S. Peirce. Paris: Payot.Suche in Google Scholar
Deledalle, Gérard. (2000). Charles S. Peirce’s Philosophy of Signs: Essays in Comparative Semiotics chap. Semeiotic and Semiology: Peirce and Saussure, 100–114. Bloomington and Indianopolis, IN: Indiana University Press.Suche in Google Scholar
Deppermann, Arnulf. (2013). Multimodal interaction from a conversation analytic perspective. Journal of Pragmatics, 46(1):1–7.10.1016/j.pragma.2012.11.014Suche in Google Scholar
Eco, Umberto. (1976). A Theory of Semiotics. Bloomington: Indiana University Press.10.1007/978-1-349-15849-2Suche in Google Scholar
Elleström, Lars. (2013). Spatiotemporal aspects of iconicity. In: Iconic investigations, Lars Elleström, Olga Fischer, and Christina Ljungberg (Eds.), Amsterdam: John Benjamins.10.1075/ill.12.09ellSuche in Google Scholar
Freadman, Anne. (1986). Structuralist uses of Peirce: Jakobson, Metz et al. In: Semiotics – language – ideology (Sydney Studies in Society and Culture 3), T. Threadgold, E. A. Grosz, Gunther Kress, and Michael A. K. Halliday (Eds.), 93–124. Sydney: Sydney Association for Studies in Society and Culture.Suche in Google Scholar
Fricke, Ellen. (2007). Origo, Geste und Raum: Lokaldeixis im Deutschen. Berlin: de Gruyter.10.1515/9783110897746Suche in Google Scholar
Fricke, Ellen. (2012). Grammatik multimodal: Wie Wörter und Gesten zusammenwirken. Berlin and New York: Mouton de Gruyter.10.1515/9783110218893Suche in Google Scholar
Fricke, Ellen. (2013). Between reference and meaning: Object-related and interpretant-related gestures in face-to-face interaction. In: Body–Language–Communication. An International Handbook on Multmodality in Human Interaction (Handbücher zur Sprach- und Kommunikationswissenschaft / Handbooks of Linguistics and Communication Science (HSK) 38/1), Cornelia Müller, Alan Cienki, Ellen Fricke, Silva Ladewig, David McNeill, and Sedinha Tessendorf (Eds.), 1788–1802. Berlin and New York: Mouton de Gruyter.10.1515/9783110302028.1788Suche in Google Scholar
Halliday, Michael A. K. (1978). Language as Social Semiotic. London: Edward Arnold.Suche in Google Scholar
Hjelmslev, Louis. (1961 [1943]). Prolegomena to a Theory of Language. Madison, Wisconsin: University of Wisconsin Press. Originally published 1943; Translated by F.J. Whitfield.Suche in Google Scholar
Hobbs, Jerry R. (1990). Literature and Cognition. Stanford, California: CSLI. CSLI Lecture Notes.Suche in Google Scholar
Hoven, Paul J. van den. (2010). The intriguing iconic sign. Interstudia (Review of Interdisciplinary Centre for Studies of Contemporary Discursive Forms) 6. 24–37. Issue on: Cultural Spaces and Identities in (inter)action.Suche in Google Scholar
Iversen, Margaret. (1986). Saussure v. Peirce: Models for a semiotics of visual art. In: The New Art History, A.L. Rees and Frances Borzello (Eds.), 82–94. London: Camden Press.Suche in Google Scholar
Jakobson, Roman. (1965). Quest for the essence of language. Diogenes, 13:21–37.10.1177/039219216501305103Suche in Google Scholar
Jakobson, Roman. (1990 [1956]). The two aspects of language and two types of aphasic disturbances. In: On Language, Linda R. Waugh and Monique Monville-Burston (Eds.), 115–133. Cambridge, MA and London: Harvard University Press.Suche in Google Scholar
Jappy, Tony. (2013). Introduction to Peircean Visual Semiotics Bloomsbury Advances in Semiotics. London and New York: Bloomsbury.Suche in Google Scholar
Kappelhoff, Hermann, and Müller, Cornelia. (2011). Embodied meaning construction. Multimodal metaphor and expressive movement in speech, gesture, and in feature film. Metaphor and the Social World, 1(2):121–153.10.1075/msw.1.2.02kapSuche in Google Scholar
Kendon, Alan. (2014). Semiotic diversity in utterance production and the concept of ‘language’. Philosophical Transactions of the Royal Society, B369(1651):20130293.10.1098/rstb.2013.0293Suche in Google Scholar
Kress, Gunther. (2003). Literacy in the New Media Age. London: Routledge.Suche in Google Scholar
Kress, Gunther. (2010). Multimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge.Suche in Google Scholar
Kress, Gunther, and van Leeuwen, Theo. (2001). Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold.Suche in Google Scholar
Kress, Gunther, and van Leeuwen, Theo. (2006). Reading Images: The Grammar of Visual Design. 2nd Edition. London and New York: Routledge.10.4324/9780203619728Suche in Google Scholar
Lakoff, George, and Johnson, Mark. (1980). Metaphors We Live by. Chicago, Il: Chicago University Press.Suche in Google Scholar
Lakoff, George, and Johnson, Mark. (1999). Philosophy in the Flesh: The Embodied Mind and Its Challenge to Western Thought. New York: Basic Books.Suche in Google Scholar
Lefebvre, Martin. (2007). The Art of Pointing. On Peirce, Indexicality, and Photographic Images. In: Photography theory, James Elkins (Ed.), 220–244. London: Routledge. The art seminar, Volume II.Suche in Google Scholar
Magnani, Lorenzo. (2001). Abduction, Reason, and Science. Processes of Discovery and Explanation. New York: Kluwer Academic/Plenum Publishers.10.1007/978-1-4419-8562-0Suche in Google Scholar
Manghani, Sunil. (2013). Image Studies: Theory and Practice. London: Routledge.Suche in Google Scholar
Mitchell, W.J.T. (1994). Picture Theory: Essays on Verbal and Visual Representation. Chicago: University of Chicago Press.Suche in Google Scholar
Mitchell, W.J.T. (2005). There are no visual media. Journal of Visual Culture, 4(2):257–266. doi:10.1177/1470412905054673.Suche in Google Scholar
Mittelberg, Irene. (2006). Metaphor and Metonymy in Language and Gesture: Discourse Evidence for Multimodal Models of Grammar. Ann Arbor, MI: UMI. Cornell University Dissertation.Suche in Google Scholar
Mittelberg, Irene. (2008). Peircean semiotics meets conceptual metaphor. iconic modes in gestural representations of grammar. In: Metaphor and Gesture, Alan Cienki and Cornelia Müller (Eds.), 115–154. Amsterdam and Philadelphia: John Benjamins Publishing Company.Suche in Google Scholar
Mittelberg, Irene, and Waugh, Linda R. (2009). Metonymy first, metaphor second: A cognitive-semiotic approach to multimodal figures of thought in co-speech gesture. In: Multimodal Metaphor, Charles J. Forceville, and Eduardo Urios-Aparisi (Eds.), 329–356. Berlin/New York: Mouton de Gruyter.Suche in Google Scholar
Mittelberg, Irene, and Waugh, Linda R. (2014). Gestures and metonymy. In: Body – Language – Communication. An International Handbook on Multimodality in Human Interaction, Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeil, and Jana Bressem (Eds.), vol. 2, 1747–1766. Berlin/Boston: Mouton.Suche in Google Scholar
Müller, C., and Cienki, Alan. (2009). Words, gestures, and beyond: Forms of multimodal metaphor in the use of spoken language. In: Multimodal Metaphor, Charles J. Forceville and Eduardo Urios-Aparisi (Eds.), 297–328. Berlin/New York: Mouton de Gruyter.Suche in Google Scholar
Müller, Cornelia. (1998). Redebegleitende Gesten: Kulturgeschichte – Theorie – Sprachvergleich. Berlin: Berliner Wissenschafts-Verlag.Suche in Google Scholar
Nöth, Winfried. (2010). The criterion of habit in Peirce’s definitions of the symbol. Transactions of the Charles S. Peirce Society, 46(1):82–93.10.2979/tra.2010.46.1.82Suche in Google Scholar
Nöth, Winfried. (2011). Visual semiotics: Key features and an application to picture ads. In: The SAGE Handbook of Visual Research Methods, Eric Margolis and Luc Pauwels (Eds.), 298–316. London.10.4135/9781446268278.n16Suche in Google Scholar
Nöth, Winfried. (2014). The growth of signs. Sign Systems Studies, 42(2/3):172–192.10.12697/SSS.2014.42.2-3.02Suche in Google Scholar
Nöth, Winfried, and Jungk, Isabel. (2015). Peircean visual semiotics: Potentials to be explored. Semiotica, 207:657–673.10.1515/sem-2015-0058Suche in Google Scholar
Parret, Herman. (1984). Peirce and Hjelmslev: The two semiotics. Language Sciences, 6(2):217–227.10.1016/S0388-0001(84)80017-0Suche in Google Scholar
Peirce, Charles Sanders. (1931–1958). Collected Papers of Charles Sanders Peirce. Cambridge, MA: Harvard University Press. Vols. 1–6, 1931–1935, edited by Charles Hartshorne and Paul Weiss; Vols. 7–8, 1958, edited by Arthur W. Burks.Suche in Google Scholar
Peirce, Charles Sanders. (1998 [1893–1913]). The Essential Peirce – Volume 2. Selected Philosophical Writings (1893–1913). Bloomington: Indiana University Press.Suche in Google Scholar
Queiroz, João, and Merrell, Floyd. (2009). On Peirce’s pragmatic notion of semiosis a contribution for the design of meaning machines. Minds & Machines 19:129–143.10.1007/s11023-008-9129-zSuche in Google Scholar
Scott, David. (1997). Indexical/iconic tensions: The semiotics of the postage stamp. In: Semiotics of the Media. State of the Art, Projects, and Perspectives (Approaches to Semiotics 127), Winfried Nöth (Ed.), 191–201. Berlin/New York: Mouton de Gruyter.10.1515/9783110803617-014Suche in Google Scholar
Short, Thomas L. (1982). Life among the legisigns. Transactions of the Charles S. Peirce Society, 18(4):285–310.Suche in Google Scholar
Short, Thomas L. (1996). Interpreting Peirce’s Interpretant: A response to Lalor, Liszka, and Meyers. Transactions of the Charles S. Peirce Society, 32(4):488–541.Suche in Google Scholar
Short, Thomas L. (2007). Peirce’s Theory of Signs. Cambridge: Cambridge University Press.10.1017/CBO9780511498350Suche in Google Scholar
Skagested, Peter. (2004). Peirce’s semeiotic model of the mind. In: The Cambridge Companion to Peirce Cambridge Companions to Philosophy, Cheryl Misak (Ed.), 241–256. Cambridge: Cambridge University Press.10.1017/CCOL0521570069.010Suche in Google Scholar
Stjernfelt, Frederik. (2000). Diagrams as centerpiece of a Peircean epistemology. Transactions of the Charles S. Peirce Society, 36(3): 357–384.Suche in Google Scholar
Stjernfelt, Frederik. (2014). Natural Propositions: The Actuality of Peirce’s Doctrine of Dicisigns. Boston, MA: The Docent Press.Suche in Google Scholar
Stöckl, Hartmut. (2004). In between modes: language and image in printed media. In: Perspectives on Multimodality, Eija Ventola, Cassily Charles, and Martin Kaltenbacher (Eds.), 9–30. Amsterdam: John Benjamins.10.1075/ddcs.6.03stoSuche in Google Scholar
Varela, Francisco J., Thompson, Evan, and Rosch, Eleanor. (1991). The Embodied Mind: Cognitive Science and Human Experience. Cambridge, MA: MIT Press.10.7551/mitpress/6730.001.0001Suche in Google Scholar
Wharton, Tim. (2009). Pragmatics and Non-Verbal Communication. Cambridge: Cambridge University Press.10.1017/CBO9780511635649Suche in Google Scholar
© 2018 Walter de Gruyter GmbH, Berlin/Boston
Artikel in diesem Heft
- Research Article
- Academic Voice Reimag(in)ed: Interest and Design in Artwork and Three Dimensional Artefacts
- Multimodal Humour: Integrating Blending Model, Relevance Theory, and Incongruity Theory
- A Multimodal Analysis of Affect and Attitude Education in China’s English Textbooks
- Peircean Semiotics and Multimodality: Towards a New Synthesis
Artikel in diesem Heft
- Research Article
- Academic Voice Reimag(in)ed: Interest and Design in Artwork and Three Dimensional Artefacts
- Multimodal Humour: Integrating Blending Model, Relevance Theory, and Incongruity Theory
- A Multimodal Analysis of Affect and Attitude Education in China’s English Textbooks
- Peircean Semiotics and Multimodality: Towards a New Synthesis