Abstract
The article presents findings from both quantitative and qualitative research regarding the resolution of cross- and multi-modal anaphoric references to objects in two Sherlock Holmes movies directed by Guy Ritchie. The goal is to illustrate how words referring to objects are linked to images depicting the objects as well as sounds produced by the objects to constitute coreferential chains with respect to the viewers’ perception. The theoretical assumptions draw concepts primarily from cognitive linguistics (such as mental spaces, compression, frame-metonymic relations), with additional notions adopted from (verbal) semantics, pragmatics, visual semantics, and film studies. A cognitive account of regular and associative cross- and multimodal anaphora is suggested, along with a model for the representation of cross- and multi-modal coreferential chains for future manual and automated annotation tools. Overall, the work emphasizes the importance of considering both verbal and nonverbal cues in understanding anaphoric references in cinematic contexts by highlighting the cognitive processes involved while viewers integrate information from different modes.
References
Apothéloz, Denis & Marie-José Reicher-Beguelin. 1999. Interpretations and functions of demonstrative NPs in indirect anaphora. Journal of Pragmatics 31(3). 363–398.10.1016/S0378-2166(98)00073-3Search in Google Scholar
Bateman, John A. & Janina Wildfeuer. 2014a. Defining units of analysis for the systematic analysis of comics: A discourse-based approach. Studies in Comics 5(2). 373–403.10.1386/stic.5.2.373_1Search in Google Scholar
Bateman, John A. & Janina Wildfeuer. 2014b. A multimodal discourse theory of visual narrative. Journal of Pragmatics 74. 180–208.10.1016/j.pragma.2014.10.001Search in Google Scholar
Bateman, John A. & Janina Wildfeuer. 2017. Film Text Analysis. New York: Routledge.Search in Google Scholar
Baumgarten, Nicole. 2008. “Yeah, that’s it!”: Verbal Reference to Visual Information in Film Texts and Film Translations. Meta 53(1). 6–25.10.7202/017971arSearch in Google Scholar
Beaver, Frank E. 2015. Dictionary of film terms: The aesthetic companion to film art. New York: Peter Lang.Search in Google Scholar
Bonifazi, Anna, Pinelopi Ioannidou & Zala Salarzai. 2022. Proper names as anaphoric expressions in short crime stories: Doing more than referring within and across paragraphs. Journal of Pragmatics 193. 88–104.10.1016/j.pragma.2022.03.009Search in Google Scholar
Bordwell, David & Kristin Thompson. 1997. Film Art: An Introduction (5th (Edition ed.). New York, NY: McGraw-Hill.Search in Google Scholar
Bosch, Peter 1983. Agreement and Anaphora. A study of the role of pronouns in syntax and discourse. London & New York: Academic PressSearch in Google Scholar
Chafe, Wallace L. 1970. Meaning and the structure of language. Chicago: University of Chicago Press.Search in Google Scholar
Cheung, Olivia S. & Isabel Gauthier. 2014. Visual Appearance Interacts with Conceptual Knowledge in Object Recognition. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2014.00793.Search in Google Scholar
Chion, Michel. 2019. Audio-Vision: Sound on Screen. New York Chichester, West Sussex: Columbia University Press. https://doi.org/10.7312/chio18588Search in Google Scholar
Clark, Hebert H. 1977. Bridging. In P. N. Johnson-Laird and P.C. Wason (eds.), Thinking: Readings in Cognitive Science. Cambridge.Search in Google Scholar
Cohn, Neil, Lincy van Middelaar, Tom Foulsham, & Joost Schilperoord. 2024. Anaphoric distance dependencies in visual narrative structure and processing. Cognitive Psychology. https://doi.org/10.1016/j.cogpsych.2024.101639Search in Google Scholar
Cohn, Neil. 2015. Narrative conjunction’s junction function: The interface of narrative grammar and semantics in sequential images. Journal of Pragmatics 88. 105–132. https://doi.org/10.1016/j.pragma.2015.09.001Search in Google Scholar
Cornish, Francis. 1999. Anaphora, discourse, and understanding: Evidence from English and French. Oxford: Clarendon Press.10.1093/oso/9780198236481.001.0001Search in Google Scholar
Crystal, David. 1997. A dictionary of linguistics and phonetics. 4th edition. Cambridge, MA: Blackwell Publishing.Search in Google Scholar
Dancygier, Barbara. 2012. Negation, stance verbs, and intersubjectivity. In:Barbara Dancygier and Eve Sweetser (eds.), Viewpoint in Language: A Multimodal Perspective, 69–93. Cambridge.10.1017/CBO9781139084727.006Search in Google Scholar
Dancygier, Barbara & Eve Sweetser. 2014. Figurative Language. Cambridge: Cambridge University Press.Search in Google Scholar
Eisenstein, Sergei M. 1991. Selected Works. Vol. II: Towards a Theory of Montage. London: British Film Institute.Search in Google Scholar
Emmott, Catherine. 1997. Narrative Comprehension. A Discourse Perspective. Oxford University Press, Oxford.10.1093/oso/9780198236498.001.0001Search in Google Scholar
Evans, Vyvyan & Melanie Green. 2006. Cognitive linguistics: An introduction. Edinburgh: Edinburgh University Press.Search in Google Scholar
Fauconnier, Gilles. 1994 [1985]. Mental spaces: Aspects of meaning construction in natural language. Cambridge University Press.10.1017/CBO9780511624582Search in Google Scholar
Fauconnier, Gilles & Eve Sweetser. 1996 [1985]. Spaces, Worlds, and Grammar. Chicago: University of Chicago Press.Search in Google Scholar
Fauconnier, Gilles & Mark Turner. 2002. The way we think: Conceptual blending and the mind’s hidden complexities. New York: Basic Books.Search in Google Scholar
Frege, Gottlob. 1892. Sinn und Bedeutung. Zeitschrift für Philosophie und philosophische Kritik 100. 25–50.Search in Google Scholar
Frith, Chris D. & Uta Frith. 2007. Social Cognition in Humans. Current Biology 17 (16). 724–732.10.1016/j.cub.2007.05.068Search in Google Scholar
Hawkins, John A. 1997. The pragmatics of definiteness. Linguistische Berichte 47 (77). 1–27.Search in Google Scholar
Huang, Yan. 2006. Anaphora, Cataphora, Exophora, Logophoricity. Encyclopedia of Language & Linguistics. 231–237.10.1016/B0-08-044854-2/01084-1Search in Google Scholar
Kleiber, Georges. 1999. Associative anaphora and part-whole relationship: The condition of alienation and the principle of ontological congruence. Journal of Pragmatics 31. 339–362.10.1016/S0378-2166(98)00072-1Search in Google Scholar
Klomberg, Bien, Irmak Hacımusaoğlu, Lenneke Doris Lichtenberg, Joost Schilperoord & Neil Cohn. 2023. Continuity, Co-reference, and Inference in Visual Sequencing. Glossa: a journal of general linguistics 8(1). 1–43. https://doi.org/10.16995/glossa.998Search in Google Scholar
Kress, Gunther R. & Theo Van Leeuwen. 2001. Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold Publishers.Search in Google Scholar
Kuhn, Markus. 2011. Filmnarratologie. Ein erzähltheoretisches Analysemodell. Berlin: de Gruyter.10.1515/9783110253559Search in Google Scholar
Kuhn Markus & Johann N. Schmidt. 2013. Narration in Film. The living handbook of narratology. Hamburg: Hamburg University Press.10.1515/9783110316469.384Search in Google Scholar
Levinson, Stephen C. 2004. Deixis. In L. Horn (ed.), The handbook of pragmatics. Oxford: Blackwell.Search in Google Scholar
Magliano, Joseph P. & Jeffrey M. Zacks. 2011. The impact of continuity editing in narrative film on event segmentation. Cognitive Science 35(8). 1489–1517. https://doi.org/10.1111/j.1551-6709.2011.01202.xSearch in Google Scholar
Metz, Christian. 1974. Film Language: A Semiotics of the Cinema. New York: Oxford University Press.Search in Google Scholar
Meyer, Josef & Robert Dale. 2002. Mining a Corpus to Support Associative Anaphora Resolution. Proceedings of the Fourth International Conference on Discourse Anaphora and Anaphor Resolution (DAAEC 2002).Search in Google Scholar
Moya Guijarro, Arsenio Jesús. 2015. Visual metonymy in children’s picture books. In M. J. Pinar Sanz (ed.) Multimodality and Cognitive Linguistics. 115–130. Amsterdam and Philadelphia: Benjamins.10.1075/bct.78.08moySearch in Google Scholar
Nahajec, Lisa. 2009. Negation and the creation of implicit meaning in poetry. Language and Literature 18 (2). 109–127.10.1177/0963947009105340Search in Google Scholar
Nunberg. Geoffrey D. 1978. The Pragmatics of Reference. Bloomington: Indiana University Linguistics Club.Search in Google Scholar
RStudio Team. 2020 (Open Access edition). RStudio: Integrated Development for R. RStudio, PBC, Boston, MA URL http://www.rstudio.com/Search in Google Scholar
Schmolz, Helene. 2020. Linguistic fundamentals of anaphors and anaphora. Anaphora Resolution and Text Retrieval: A Linguistic Analysis of Hypertexts. Berlin/Boston De Gruyter De Gruyter Mouton.Search in Google Scholar
Schubert, Christoph & Christina Sanchez-Stockhammer. 2022. Introducing cohesion in multimodal discourse. Discourse, Context & Media 45. https://doi.org/10.1016/j.dcm.2021.100565.Search in Google Scholar
Stampoulidis, Georgios. 2020. Polysemiotic Communication vs. Multimodality: a conceptual and terminological distinction applied in street art. Street Art & Urban Creativity Scientific Journal 5(2). 26–31.Search in Google Scholar
Stirling, Lesley. 2001. The multifunctionality of anaphoric expressions: a typological perspective. Aust. J. Linguist. 21 (1), 7e23.10.1080/07268600124994Search in Google Scholar
Strawson, Peter F. 1950. On referring. Mind 59. 320–344.10.1093/mind/LIX.235.320Search in Google Scholar
Sweller, John. 1988. Cognitive Load during Problem Solving: Effects on Learning. Cognitive Science 12. 257–285.10.1016/0364-0213(88)90023-7Search in Google Scholar
Turner, Mark. 2017. Multimodal form-meaning pairs for blended classic joint attention. Linguistics Vanguard 3 (s1): 20160043.10.1515/lingvan-2016-0043Search in Google Scholar
Pagán, Cristóbal Cánovas & Mark B. Turner. 2015. Generic integration templates for fictive communication. SSRN Electronic Journal.Search in Google Scholar
van Vliet, Sarah. 2002. Overspecified NPs marking conceptual shifts in narrative discourse. Ling. Neth 19. 187–198.10.1075/avt.19.20vliSearch in Google Scholar
Warne, Frederick. 1997 (revised edition). The Complete Tales of Beatrix Potter. London: Penguin.Search in Google Scholar
Wittenburg Peter, Hennie Brugman, Albert Russel, Alex Klassmann & Han Sloetjes. 2006. ELAN: A professional framework for multimodality research. Nijmegen: The Language Archive of the Max Planck Institute for Psycholinguistics. https://archive.mpi.nl/tla/elan.Search in Google Scholar
©2024 Walter de Gruyter GmbH, Berlin/Boston
Articles in the same Issue
- Frontmatter
- Editorial
- Jetzt hab ich voll die Panik: Prototype effects of NP-external intensifiers in German
- Metapragmatic markers and the instantiation of pragmatic frames: A cognitive-linguistic approach to the problem of current discourse
- Linguistic paradigms as cognitive entities: A domain-general approach
- Partial colexifications reveal directional tendencies in object naming
- The interplay of conceptualization and case marking in the directional cases of Udmurt
- Integrating approaches to the role of metaphor in the evolutionary dynamics of language
- Metaphorical meaning dynamics: Identifying patterns in the metaphorical evolution of English words using mathematical modeling techniques
- The language of gratitude: An empirical analysis of acknowledgments in German medical dissertations
- Cross- and multimodal anaphoric references in mystery movies: A cognitive perspective
- Language learners, chess champions, and piano prodigies – insights from research on language contact and expert behavior
- Adaptive language strategies of an older sibling in bilingual German-Russian acquisition: A case study
Articles in the same Issue
- Frontmatter
- Editorial
- Jetzt hab ich voll die Panik: Prototype effects of NP-external intensifiers in German
- Metapragmatic markers and the instantiation of pragmatic frames: A cognitive-linguistic approach to the problem of current discourse
- Linguistic paradigms as cognitive entities: A domain-general approach
- Partial colexifications reveal directional tendencies in object naming
- The interplay of conceptualization and case marking in the directional cases of Udmurt
- Integrating approaches to the role of metaphor in the evolutionary dynamics of language
- Metaphorical meaning dynamics: Identifying patterns in the metaphorical evolution of English words using mathematical modeling techniques
- The language of gratitude: An empirical analysis of acknowledgments in German medical dissertations
- Cross- and multimodal anaphoric references in mystery movies: A cognitive perspective
- Language learners, chess champions, and piano prodigies – insights from research on language contact and expert behavior
- Adaptive language strategies of an older sibling in bilingual German-Russian acquisition: A case study