Visual affect in films: a semiotic approach

Yi Jing

doi:10.1515/sem-2019-0081

Article

Visual affect in films: a semiotic approach

Published/Copyright: February 5, 2021

Published by

Become an author with De Gruyter Brill

Submit Manuscript Author Information Explore this Subject

From the journal Semiotica Volume 2021 Issue 239

Abstract

This study investigates affective meanings expressed in facial expressions and bodily gestures from a semiotic perspective. Particularly, the study focuses on disentangling relations of affective meanings and exploring the meaning potential of facial expressions and bodily gestures. Based on the analysis of over three hundred screenshots from two films (one animation and one live-action film), this study proposes a system of visual affect, as well as a system of visual resources involved in the expression of visual affect. The system of visual affect makes a further step in the investigation of affective meanings afforded by facial expressions and bodily gestures, and can provide methodological insights into the examination of affective meanings expressed visually. The system of visual resources provides a more meaning-motivated framework for systematic tracking of the visual resources, which may be applied to the analysis of other visual media apart from films.

Keywords: affect; facial expression; film; gesture; visual semiotics

Corresponding author: Yi Jing, Harbin Institute of Technology, Shenzhen, China, E-mail: jingyi2020@hit.edu.cn

Funding source: China Scholarship Council–University of New South Wales

Award Identifier / Grant number: 201508440230

Funding source: Harbin Institute of Technology, Shenzhen

Award Identifier / Grant number: IA45001003

Acknowledgments

The research reported in this article was conducted at The University of New South Wales. I am extremely grateful to Dr. Peter R.R. White for his critical feedback on earlier drafts of this article. I would also like to thank Prof. Theo van Leeuween, Assoc. Prof. Louise Ravelli, Prof. Jim Martin, Dr. Maryam Alavi, Dr. Yufei He, and Qingxin Xu for their comments and suggestions on this research. My gratitude further extends to Universal Studios Media Licensing, whose license fee greatly stimulates my creativity and motivates me to develop my own drawing skills.

References

Ameka, Felix K. 1992. Interjections: The universal yet neglected part of speech. Journal of Pragmatics 18(2–3). 101–118. https://doi.org/10.1016/0378-2166(92)90048-g.Search in Google Scholar

Bednarek, Monika. 2008. Emotion talk across corpora. Houndmills & New York: Palgrave Macmillan. https://doi.org/10.1057/9780230285712.Search in Google Scholar

Chuang, Ying-Ting. 2006. Studying subtitle translation from a multi-modal approach. Babel 52(4). 372–383. https://doi.org/10.1075/babel.52.4.06chu.Search in Google Scholar

Darwin, Charles. 1872 [1979]. The expression of emotions in man and animals. London: Julian Friedmann.10.1037/10001-000Search in Google Scholar

Eastman, Carol M. 1992. Swahili interjections: Blurring language-use/gesture-use boundaries. Journal of Pragmatics 18(2–3). 273–287. https://doi.org/10.1016/0378-2166(92)90055-g.Search in Google Scholar

Economou, Dorothy. 2009. Photos in the news: Appraisal analysis of visual semiosis and visual-verbal intersemiosis. University of Sydney Unpublished PhD Dissertation.Search in Google Scholar

Ekman, Paul. 2003. Emotions revealed: recognizing faces and feelings to improve communication and emotional life. New York: Times.Search in Google Scholar

Ekman, Paul & Wallace V. Friesen. 1969. The repertoire of nonverbal behavior: Categories, origins, usage and coding. Semiotica 1(1). 49–98. https://doi.org/10.1515/semi.1969.1.1.49.Search in Google Scholar

Ekman, Paul & Wallace V. Friesen. 1975. Unmasking the face: A guild to recognizing emotions from facial clues. Englewood Cliffs, NJ: Prentice-Hall.Search in Google Scholar

Ekman, Paul & Wallace V. Friesen. 2003. Unmasking the face: A guild to recognizing emotions from facial expressions. Cambridge MA: Malor.Search in Google Scholar

Ekman, Paul, Wallace V. Friesen & Joseph C. Hager. 2002. Facial action coding system: The manual on CD ROM. Salt Lake City: Research Nexus Division of Network Information Research Corporation.Search in Google Scholar

Faigin, Gary. 1990. The artist’s complete guide to facial expression. New York: Watson-Guptill.10.1097/00006534-199108000-00037Search in Google Scholar

Feng, Dezheng & Kay L. O’Halloran. 2012. Representing emotive meaning in visual images: A social semiotic approach. Journal of Pragmatics 44(14). 2067–2084. https://doi.org/10.1016/j.pragma.2012.10.003.Search in Google Scholar

Georgakopoulou, Panayota. 2009. Subtitling for the DVD industry. In Jorge Díaz Cintas & Gunilla Anderman (eds.), Audiovisual translation: Language transfer on screen, 21–35. Basingstoke: Palgrave Macmillan. https://doi.org/10.1057/9780230234581_2.Search in Google Scholar

Goddard, Cliff. 2014. Interjections and emotion (with special reference to “surprise” and “disgust”). Emotion Review 6(1). 53–63. https://doi.org/10.1177/1754073913491843.Search in Google Scholar

Halliday, Michael A. K. 1994. An introduction to functional grammar, 2nd edn. London: Arnold.Search in Google Scholar

Halliday, Michael A. K. & Christian M. I. M. Matthiessen. 2014. Halliday’s introduction to functional grammar, 4th edn. London: Routledge. https://doi.org/10.4324/9780203431269.Search in Google Scholar

Leeuwen, Theo van. 2016. A social semiotic theory of synesthesia? A discussion paper. Hermes 55. 105–119. https://doi.org/10.7146/hjlcb.v0i55.24292.Search in Google Scholar

Maestri, George. 2005. Digital character animation 3. Thousand Oaks, CA: New Riders.Search in Google Scholar

Martin, James R. & Peter R. R. White. 2005. The language of evaluation: Appraisal in English. London: Palgrave Macmillan. https://doi.org/10.1057/9780230511910.Search in Google Scholar

Martinec, Radan. 2001. Interpersonal resources in action. Semiotica 135(1/4). 117–145. https://doi.org/10.1515/semi.2001.056.Search in Google Scholar

Matthiessen, Christian M. I. M. 2007. The multimodal page: A systemic functional exploration. In Terry D. Royce & Wendy Bowcher (eds.), New directions in the analysis of multimodal discourse, 1–62. Mahwah, NJ: Lawrence Erlbaum.Search in Google Scholar

McCloud, Scott. 2006. Making comics: Storytelling secrets of comics, manga and graphic novels. New York: Harper.Search in Google Scholar

Ngo, Thu & Len Unsworth. 2015. Reworking the appraisal framework in ESL research: Refining attitude resources. Functional Linguistics 2(1). 1–24. https://doi.org/10.1186/s40554-015-0013-x.Search in Google Scholar

Rosa, Alexandra Assis. 2001. Features of oral and written communication in subtitling. In Yves Gambier & Henrik Gottlieb (eds.), (Multi)media translation: Concepts, practices, and research, 213–221. Amsterdam: Benjamins. https://doi.org/10.1075/btl.34.26ass.Search in Google Scholar

Stange, Ulrike. 2016. Emotive interjections in British English: A corpus-based study on variation in acquisition, function and usage. Amsterdam: Benjamins. https://doi.org/10.1075/scl.75.Search in Google Scholar

Tian, Ping. 2011. Multimodal evaluation: Sense and sensibility in Anthony Browne’s picture books. University of Sydney Unpublished PhD Dissertation.Search in Google Scholar

Wilkins, David P. 1992. Interjections as deictics. Journal of Pragmatics 18(2–3). 119–158. https://doi.org/10.1016/0378-2166(92)90049-h.Search in Google Scholar

Received: 2019-07-16

Accepted: 2019-10-05

Published Online: 2021-02-05

Published in Print: 2021-03-26

You are currently not able to access this content.

Articles in the same Issue

https://doi.org/10.1515/sem-2019-0081

Keywords for this article

affect; facial expression; film; gesture; visual semiotics