Home Cross- and multimodal anaphoric references in mystery movies: A cognitive perspective
Article
Licensed
Unlicensed Requires Authentication

Cross- and multimodal anaphoric references in mystery movies: A cognitive perspective

  • Anna Bonifazi EMAIL logo and Pinelopi Ioannidou
Published/Copyright: November 6, 2024

Abstract

The article presents findings from both quantitative and qualitative research regarding the resolution of cross- and multi-modal anaphoric references to objects in two Sherlock Holmes movies directed by Guy Ritchie. The goal is to illustrate how words referring to objects are linked to images depicting the objects as well as sounds produced by the objects to constitute coreferential chains with respect to the viewers’ perception. The theoretical assumptions draw concepts primarily from cognitive linguistics (such as mental spaces, compression, frame-metonymic relations), with additional notions adopted from (verbal) semantics, pragmatics, visual semantics, and film studies. A cognitive account of regular and associative cross- and multimodal anaphora is suggested, along with a model for the representation of cross- and multi-modal coreferential chains for future manual and automated annotation tools. Overall, the work emphasizes the importance of considering both verbal and nonverbal cues in understanding anaphoric references in cinematic contexts by highlighting the cognitive processes involved while viewers integrate information from different modes.

References

Apothéloz, Denis & Marie-José Reicher-Beguelin. 1999. Interpretations and functions of demonstrative NPs in indirect anaphora. Journal of Pragmatics 31(3). 363–398.10.1016/S0378-2166(98)00073-3Search in Google Scholar

Bateman, John A. & Janina Wildfeuer. 2014a. Defining units of analysis for the systematic analysis of comics: A discourse-based approach. Studies in Comics 5(2). 373–403.10.1386/stic.5.2.373_1Search in Google Scholar

Bateman, John A. & Janina Wildfeuer. 2014b. A multimodal discourse theory of visual narrative. Journal of Pragmatics 74. 180–208.10.1016/j.pragma.2014.10.001Search in Google Scholar

Bateman, John A. & Janina Wildfeuer. 2017. Film Text Analysis. New York: Routledge.Search in Google Scholar

Baumgarten, Nicole. 2008. “Yeah, that’s it!”: Verbal Reference to Visual Information in Film Texts and Film Translations. Meta 53(1). 6–25.10.7202/017971arSearch in Google Scholar

Beaver, Frank E. 2015. Dictionary of film terms: The aesthetic companion to film art. New York: Peter Lang.Search in Google Scholar

Bonifazi, Anna, Pinelopi Ioannidou & Zala Salarzai. 2022. Proper names as anaphoric expressions in short crime stories: Doing more than referring within and across paragraphs. Journal of Pragmatics 193. 88–104.10.1016/j.pragma.2022.03.009Search in Google Scholar

Bordwell, David & Kristin Thompson. 1997. Film Art: An Introduction (5th (Edition ed.). New York, NY: McGraw-Hill.Search in Google Scholar

Bosch, Peter 1983. Agreement and Anaphora. A study of the role of pronouns in syntax and discourse. London & New York: Academic PressSearch in Google Scholar

Chafe, Wallace L. 1970. Meaning and the structure of language. Chicago: University of Chicago Press.Search in Google Scholar

Cheung, Olivia S. & Isabel Gauthier. 2014. Visual Appearance Interacts with Conceptual Knowledge in Object Recognition. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2014.00793.Search in Google Scholar

Chion, Michel. 2019. Audio-Vision: Sound on Screen. New York Chichester, West Sussex: Columbia University Press. https://doi.org/10.7312/chio18588Search in Google Scholar

Clark, Hebert H. 1977. Bridging. In P. N. Johnson-Laird and P.C. Wason (eds.), Thinking: Readings in Cognitive Science. Cambridge.Search in Google Scholar

Cohn, Neil, Lincy van Middelaar, Tom Foulsham, & Joost Schilperoord. 2024. Anaphoric distance dependencies in visual narrative structure and processing. Cognitive Psychology. https://doi.org/10.1016/j.cogpsych.2024.101639Search in Google Scholar

Cohn, Neil. 2015. Narrative conjunction’s junction function: The interface of narrative grammar and semantics in sequential images. Journal of Pragmatics 88. 105–132. https://doi.org/10.1016/j.pragma.2015.09.001Search in Google Scholar

Cornish, Francis. 1999. Anaphora, discourse, and understanding: Evidence from English and French. Oxford: Clarendon Press.10.1093/oso/9780198236481.001.0001Search in Google Scholar

Crystal, David. 1997. A dictionary of linguistics and phonetics. 4th edition. Cambridge, MA: Blackwell Publishing.Search in Google Scholar

Dancygier, Barbara. 2012. Negation, stance verbs, and intersubjectivity. In:Barbara Dancygier and Eve Sweetser (eds.), Viewpoint in Language: A Multimodal Perspective, 69–93. Cambridge.10.1017/CBO9781139084727.006Search in Google Scholar

Dancygier, Barbara & Eve Sweetser. 2014. Figurative Language. Cambridge: Cambridge University Press.Search in Google Scholar

Eisenstein, Sergei M. 1991. Selected Works. Vol. II: Towards a Theory of Montage. London: British Film Institute.Search in Google Scholar

Emmott, Catherine. 1997. Narrative Comprehension. A Discourse Perspective. Oxford University Press, Oxford.10.1093/oso/9780198236498.001.0001Search in Google Scholar

Evans, Vyvyan & Melanie Green. 2006. Cognitive linguistics: An introduction. Edinburgh: Edinburgh University Press.Search in Google Scholar

Fauconnier, Gilles. 1994 [1985]. Mental spaces: Aspects of meaning construction in natural language. Cambridge University Press.10.1017/CBO9780511624582Search in Google Scholar

Fauconnier, Gilles & Eve Sweetser. 1996 [1985]. Spaces, Worlds, and Grammar. Chicago: University of Chicago Press.Search in Google Scholar

Fauconnier, Gilles & Mark Turner. 2002. The way we think: Conceptual blending and the mind’s hidden complexities. New York: Basic Books.Search in Google Scholar

Frege, Gottlob. 1892. Sinn und Bedeutung. Zeitschrift für Philosophie und philosophische Kritik 100. 25–50.Search in Google Scholar

Frith, Chris D. & Uta Frith. 2007. Social Cognition in Humans. Current Biology 17 (16). 724–732.10.1016/j.cub.2007.05.068Search in Google Scholar

Hawkins, John A. 1997. The pragmatics of definiteness. Linguistische Berichte 47 (77). 1–27.Search in Google Scholar

Huang, Yan. 2006. Anaphora, Cataphora, Exophora, Logophoricity. Encyclopedia of Language & Linguistics. 231–237.10.1016/B0-08-044854-2/01084-1Search in Google Scholar

Kleiber, Georges. 1999. Associative anaphora and part-whole relationship: The condition of alienation and the principle of ontological congruence. Journal of Pragmatics 31. 339–362.10.1016/S0378-2166(98)00072-1Search in Google Scholar

Klomberg, Bien, Irmak Hacımusaoğlu, Lenneke Doris Lichtenberg, Joost Schilperoord & Neil Cohn. 2023. Continuity, Co-reference, and Inference in Visual Sequencing. Glossa: a journal of general linguistics 8(1). 1–43. https://doi.org/10.16995/glossa.998Search in Google Scholar

Kress, Gunther R. & Theo Van Leeuwen. 2001. Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold Publishers.Search in Google Scholar

Kuhn, Markus. 2011. Filmnarratologie. Ein erzähltheoretisches Analysemodell. Berlin: de Gruyter.10.1515/9783110253559Search in Google Scholar

Kuhn Markus & Johann N. Schmidt. 2013. Narration in Film. The living handbook of narratology. Hamburg: Hamburg University Press.10.1515/9783110316469.384Search in Google Scholar

Levinson, Stephen C. 2004. Deixis. In L. Horn (ed.), The handbook of pragmatics. Oxford: Blackwell.Search in Google Scholar

Magliano, Joseph P. & Jeffrey M. Zacks. 2011. The impact of continuity editing in narrative film on event segmentation. Cognitive Science 35(8). 1489–1517. https://doi.org/10.1111/j.1551-6709.2011.01202.xSearch in Google Scholar

Metz, Christian. 1974. Film Language: A Semiotics of the Cinema. New York: Oxford University Press.Search in Google Scholar

Meyer, Josef & Robert Dale. 2002. Mining a Corpus to Support Associative Anaphora Resolution. Proceedings of the Fourth International Conference on Discourse Anaphora and Anaphor Resolution (DAAEC 2002).Search in Google Scholar

Moya Guijarro, Arsenio Jesús. 2015. Visual metonymy in children’s picture books. In M. J. Pinar Sanz (ed.) Multimodality and Cognitive Linguistics. 115–130. Amsterdam and Philadelphia: Benjamins.10.1075/bct.78.08moySearch in Google Scholar

Nahajec, Lisa. 2009. Negation and the creation of implicit meaning in poetry. Language and Literature 18 (2). 109–127.10.1177/0963947009105340Search in Google Scholar

Nunberg. Geoffrey D. 1978. The Pragmatics of Reference. Bloomington: Indiana University Linguistics Club.Search in Google Scholar

RStudio Team. 2020 (Open Access edition). RStudio: Integrated Development for R. RStudio, PBC, Boston, MA URL http://www.rstudio.com/Search in Google Scholar

Schmolz, Helene. 2020. Linguistic fundamentals of anaphors and anaphora. Anaphora Resolution and Text Retrieval: A Linguistic Analysis of Hypertexts. Berlin/Boston De Gruyter De Gruyter Mouton.Search in Google Scholar

Schubert, Christoph & Christina Sanchez-Stockhammer. 2022. Introducing cohesion in multimodal discourse. Discourse, Context & Media 45. https://doi.org/10.1016/j.dcm.2021.100565.Search in Google Scholar

Stampoulidis, Georgios. 2020. Polysemiotic Communication vs. Multimodality: a conceptual and terminological distinction applied in street art. Street Art & Urban Creativity Scientific Journal 5(2). 26–31.Search in Google Scholar

Stirling, Lesley. 2001. The multifunctionality of anaphoric expressions: a typological perspective. Aust. J. Linguist. 21 (1), 7e23.10.1080/07268600124994Search in Google Scholar

Strawson, Peter F. 1950. On referring. Mind 59. 320–344.10.1093/mind/LIX.235.320Search in Google Scholar

Sweller, John. 1988. Cognitive Load during Problem Solving: Effects on Learning. Cognitive Science 12. 257–285.10.1016/0364-0213(88)90023-7Search in Google Scholar

Turner, Mark. 2017. Multimodal form-meaning pairs for blended classic joint attention. Linguistics Vanguard 3 (s1): 20160043.10.1515/lingvan-2016-0043Search in Google Scholar

Pagán, Cristóbal Cánovas & Mark B. Turner. 2015. Generic integration templates for fictive communication. SSRN Electronic Journal.Search in Google Scholar

van Vliet, Sarah. 2002. Overspecified NPs marking conceptual shifts in narrative discourse. Ling. Neth 19. 187–198.10.1075/avt.19.20vliSearch in Google Scholar

Warne, Frederick. 1997 (revised edition). The Complete Tales of Beatrix Potter. London: Penguin.Search in Google Scholar

Wittenburg Peter, Hennie Brugman, Albert Russel, Alex Klassmann & Han Sloetjes. 2006. ELAN: A professional framework for multimodality research. Nijmegen: The Language Archive of the Max Planck Institute for Psycholinguistics. https://archive.mpi.nl/tla/elan.Search in Google Scholar

Published Online: 2024-11-06
Published in Print: 2024-11-26

©2024 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 29.9.2025 from https://www.degruyterbrill.com/document/doi/10.1515/gcla-2024-0010/html?lang=en
Scroll to top button