Exerting control: the grammatical meaning of facial displays in signed languages
-
Sara Siyavoshi
and
Sherman Wilcox
Abstract
Signed languages employ finely articulated facial and head displays to express grammatical meanings such as mood and modality, complex propositions (conditionals, causal relations, complementation), information structure (topic, focus), assertions, content and yes/no questions, imperatives, and miratives. In this paper we examine two facial displays: an upper face display in which the eyebrows are pulled together called brow furrow, and a lower face display in which the corners of the mouth are turned down into a distinctive configuration that resembles a frown or upside-down U-shape. Our analysis employs Cognitive Grammar, specifically the control cycle and its manifestation in effective control and epistemic control. Our claim is that effective and epistemic control are associated with embodied actions. Prototypical physical effective control requires effortful activity and the forceful exertion of energy and is commonly correlated with upper face activity, often called the “face of effort.” The lower face display has been shown to be associated with epistemic indetermination, uncertainty, doubt, obviousness, and skepticism. We demonstrate that the control cycle unifies the diverse grammatical functions expressed by each facial display within a language, and that they express similar functions across a wide range of signed languages.
Acknowledgments
The authors wish to thank the comprehensive and immensely helpful comments and suggestions made by three reviewers. We also want to acknowledge and thank our collaborators listed in the Appendix.
In this appendix we provide details on the data sources used. With one exception (Figure 17, noted below), data came from the following sources:
Published articles. When data from published articles was used, citations and references are contained in the text.
Dictionary entries. Radutzky (1992) for Italian Sign Language and Girod (1997) for French Sign Language.
Modality data set. Three modality data sets were used. These data sets were collected by creating a series of questions designed to elicit answers expressing a range of modal meanings. The interview surveys were adapted for each language and cultural context. The participants interviewed were in all cases native signers of their language. The questions were designed with the assistance of hearing bilingual signers in each language, who also conducted the interview. The interviewer was encouraged to ask follow up questions in order to clarify or elicit a discussion containing modal verbs. Thus, the data sets consist of extended natural conversation.
Story retelling. Pear Story (Chafe 1980) and “Frog Where Are You?” (Mayer 2003) data sets.
SignStream™ (Neidle 2000). The SignStream database consists of a collection of utterances, where each utterance is associated with one or more synchronized media clips and a detailed, fine-grained, multi-level transcription.
Online media. In some cases, online videos were used (YouTube, Instagram, etc.).
When appropriate the semantic interpretations and translations were discussed with deaf and hearing colleagues: André Xavier, Neiva Aquino Albres, Sylvia Lia Grespan Neves (Libras native signer); Ardavan Guity (Iranian Sign Language native signer); Li Heng, Wu Ling, Du Yinling (Chinese Sign Language native); Rocío Martínez, Diego Morales (Argentine Sign Language native signer).
| Figure | Language | Data source |
|---|---|---|
| Figure 2 | Chinese Sign Language | Modality data set |
| Figure 3 | Iranian Sign Language | Online media |
| Figure 4 | Italian Sign Language | Dictionary entry |
| Figure 5 | Italian Sign Language | Dictionary entry |
| Figure 6 | Argentine Sign Language | Online media |
| Figure 7 | Sign Language of the Netherlands | Published article |
| Figure 8 | French Sign Language | Dictionary entry |
| Figure 9 | Iranian Sign Language | Modality data set; published article |
| Figure 10 | Iranian Sign Language | Modality data set; published article |
| Figure 12 | Iranian Sign Language | Story retelling |
| Figure 13 | Iranian Sign Language | Modality data set; published article. |
| Figure 14 | Brazilian Sign Language (Libras) | Modality data set; published article |
| Figure 15 | American Sign Language | SignStream |
| Figure 16 | Brazilian Sign Language (Libras) | Published article |
| Figure 17 | American Sign Language | Elicited (context described in text) |
| Figure 18 | Brazilian Sign Language (Libras) | Story retelling |
| Figure 19 | Chinese Sign Language | Modality data set |
| Figure 20(a) | Brazilian Sign Language (Libras) | Story retelling |
| Figure 20(b) | Italian Sign Language | Story retelling |
References
Aikhenvald, Alexandra Y. 2016. Sentence types. In Jan Nuyts & Johan van der Auwera (eds.), The Oxford handbook of modality and mood, 141–165. Oxford: Oxford University Press.10.1093/oxfordhb/9780199591435.013.8Search in Google Scholar
Barsalou, Lawrence W. 1999. Perceptual symbol systems. Behavioral and Brain Sciences 22. 577–660. https://doi.org/10.1017/s0140525x99002149.Search in Google Scholar
Bell, Charles. 1806. Essays on the anatomy of expression in painting. London: Longman, Hurst, Rees, and Orme.Search in Google Scholar
Bolden, Galina B. 2009. Implementing incipient actions: The discourse marker ‘so’ in English conversation. Journal of Pragmatics 41(5). 974–998. https://doi.org/10.1016/j.pragma.2008.10.004.Search in Google Scholar
Bolinger, Dwight. 1986. Intonation and its parts: Melody in spoken English. Stanford, CA: Stanford University Press.10.1515/9781503622906Search in Google Scholar
Branchini, Chiara & Lara Mantovan (eds.). 2020. A grammar of Italian Sign Language (LIS). Venice: Edizioni Ca’Foscari.10.30687/978-88-6969-474-5Search in Google Scholar
Cabeza-Pereiro, Carmen & Silvia Iglesias-Lago. 2015. In Julie B. Jepsen, Goedele De Clerck, Sam Lutalo-Kiingi & William B. McGregor (eds.), Sign languages of the world: A comparative handbook, 729–769. Berlin: Walter de Gruyter GmbH & Co KG.10.1515/9781614518174-037Search in Google Scholar
Camras, Linda A., Harriet Oster, Roger Bakeman, Zhaolan Meng, Tatsuo Ujiie & Joseph J. Campos. 2007. Do infants show distinct negative facial expressions for fear and anger? Emotional expression in 11‐month‐old European American, Chinese, and Japanese infants. Infancy 11(2). 131–155. https://doi.org/10.1111/j.1532-7078.2007.tb00219.x.Search in Google Scholar
Chafe, Wallace. 1980. The pear stories: Cognitive, cultural, and linguistic aspects of narrative production. Norwood, NJ: Ablex.Search in Google Scholar
Chovil, Nicole. 1991. Discourse-oriented facial displays in conversation. Research on Language and Social Interaction 25(1–4). 163–194. https://doi.org/10.1080/08351819109389361.Search in Google Scholar
Dachkovsky, Svetlana & Wendy Sandler. 2009. Visual intonation in the prosody of a sign language. Language and Speech 52. 287–314. https://doi.org/10.1177/0023830909103175.Search in Google Scholar
Darwin, Charles. 1872. The expression of the emotions in man and animals. London: J. Murray.10.1037/10001-000Search in Google Scholar
de Morree, Helma M. & Samuele M. Marcora. 2010. The face of effort: Frowning muscle activity reflects effort during a physical task. Biological Psychology 85(3). 377–382. https://doi.org/10.1016/j.biopsycho.2010.08.009.Search in Google Scholar
de Morree, Helma M. & Samuele M. Marcora. 2012. Frowning muscle activity and perception of effort during constant-workload cycling. European Journal of Applied Physiology 112(5). 1967–1972. https://doi.org/10.1007/s00421-011-2138-2.Search in Google Scholar
Debras, Camille. 2017. The shrug: Forms and meanings of a compound enactment. Gesture 16(1). 1–34.https://doi.org/10.1075/gest.16.1.01deb.Search in Google Scholar
Dondi, Marco, Maria T. Gervasi, Angela Valente, Tiziana Vacca, Gianna Bogana, I. De Bellis, Silvia Melappioni, Maria Tran & Harriet Oster. 2014. Spontaneous facial expressions of distress in fetuses. In Cristina de Sousa & Armando M. Oliviera (eds.), Proceedings of the 14th european conference on facial expression, 34–37. Coimbra: IPCDVS.Search in Google Scholar
Geraci, Carlo. 2015. Italian sign language. In Julie B. Jepsen, Goedele De Clerck, Sam Lutalo-Kiingi & William B. McGregor (eds.), Sign languages of the world: A comparative handbook, 473–510. Berlin: Walter de Gruyter GmbH & Co KG.10.1515/9781614518174-025Search in Google Scholar
Gianfreda, Gabriele, Virginia Volterra & Andrej Zuczkowski. 2014. L’espressione dell’incertezza nella Lingua dei Segni Italiana (LIS). Ricerche di Pedagogia e Didattica. Journal of Theories and Research in Education 9(1). 199–234.Search in Google Scholar
Girod, Michel. 1997. La langue des signes: Tome 1. Dictionnaire bilingue elementaire. Vincennes: International Visual Theatre.Search in Google Scholar
Glenberg, Arthur M., Marc Sato & Luigi Cattaneo. 2008. Use-induced motor plasticity affects the processing of abstract and concrete language. Current Biology 18. R290–R291. https://doi.org/10.1016/j.cub.2008.02.036.Search in Google Scholar
Huang, Ding-Hau, Shih-Wei Chou, Yi-Lang Chen & Wen-Ko Chiou. 2014. Frowning and jaw clenching muscle activity reflects the perception of effort during incremental workload cycling. Journal of Sports Science and Medicine 13(4). 921.Search in Google Scholar
Humphries, Tom, Carol Padden & Terrence O’Rourke. 1980. A basic course in American Sign Language. Silver Spring, MD: TJ Publishers, Inc.Search in Google Scholar
Iglesias-Lago, Silvia I. 2006. Uso del componente facial para la expresión de la modalidad en lengua de signos española. Vigo, Spain: Universidade de Vigo, PhD dissertation.Search in Google Scholar
Johnston, Trevor & Adam Schembri. 2007. Australian Sign Language (Auslan): An introduction to sign language linguistics. Cambridge: Cambridge University Press.10.1017/CBO9780511607479Search in Google Scholar
Kimmelman, Vadim & Roland Pfau. 2014. Information structure in sign languages. In Caroline Féry & Shinichiro Ishihara (eds.), The Oxford handbook of information structure. Oxford: Oxford University Press.10.1515/9781501510045Search in Google Scholar
Lackner, Andrea. 2018. Functions of head and body movements in Austrian Sign Language. Berlin: Walter de Gruyter.10.1515/9781501507779Search in Google Scholar
Lackner, Andrea. 2019. Describing non manuals in sign language. Grazer Linguistische Studien 91(S). 45–103.Search in Google Scholar
Langacker, Ronald W. 2002. The control cycle: Why grammar is a matter of life and death. Proceedings of the Annual Meeting of the Japanese Cognitive Linguistics Association 2. 193–220.Search in Google Scholar
Langacker, Ronald W. 2008. Cognitive grammar: A basic introduction. Oxford: Oxford University Press.10.1093/acprof:oso/9780195331967.001.0001Search in Google Scholar
Langacker, Ronald W. 2009. Investigations in cognitive grammar, 42. Berlin: Walter de Gruyter.10.1515/9783110214369Search in Google Scholar
Langacker, Ronald W. 2013. Modals: Striving for control. In Juana I. Marín-Arrese, Marta Carretero, Jorge A. Hita & Johan van der Auwera (eds.), English modality: Core, periphery and evidentiality, 3–56. Berlin: Walter de Gruyter.10.1515/9783110286328.3Search in Google Scholar
Langacker, Ronald W. 2014. Culture and cognition, lexicon and grammar. In Masataka Yamaguchi, Dennis Tay & Benjamin Blount (eds.), Approaches to language, culture, and cognition: The intersection of cognitive linguistics and linguistic anthropology, 27–49. New York, NY: Palgrave Macmillan.Search in Google Scholar
Langacker, Ronald W. 2016. Nominal structure in cognitive grammar. Lubin & Poland: Marie-Curie Skłodowska University Press.Search in Google Scholar
Langacker, Ronald W. 2017. Evidentiality in cognitive grammar. In Juana I. Marín-Arrese, Gerda Haßler & Marta Carretero (eds.), Evidentiality revisited, 13–55. Amsterdam: John Benjamins.10.1075/pbns.271.02lanSearch in Google Scholar
Langacker, Ronald W. 2019. Levels of reality. Languages 4(22). 1–20. https://doi.org/10.3390/languages4020022.Search in Google Scholar
Liddell, Scott K. 1986. Head thrust in ASL conditional marking. Sign Language Studies 52(1). 244–262. https://doi.org/10.1353/sls.1986.0003.Search in Google Scholar
Maier, Emar, Kees D. Schepper & Martine Zwet. 2013. The pragmatics of person and imperatives in sign language of the Netherlands. Research in Language 11(4). 359–376. https://doi.org/10.2478/rela-2013-0010.Search in Google Scholar
Martínez, Rocío & Sherman. Wilcox. 2019. Pointing and placing: Nominal grounding in Argentine Sign Language. Cognitive Linguistics 30(1). 85–121. https://doi.org/10.1515/cog-2018-0010.Search in Google Scholar
Massone, Maria I. & Rocío A. Martinez. 2012. Capítulo 7: Morfología de la LSA: Procesos flexionales. In Maria I. Massone & Roció A. Martinez (eds.), Curso de Lengua de Señas Argentina. Mendoza/Berlin: Cultura Sorda.Search in Google Scholar
Mayer, Mercer. 2003. Frog, where are you? (A Boy, a Dog, and a Frog). New York: Penguin Books.Search in Google Scholar
McGregor, William B., Janne B. Niemelä & Julie B. Jepsen. 2015. Danish sign language. In Julie B. Jepsen, Goedele De Clerck, Sam Lutalo-Kiingi & William B. McGregor (eds.), Sign languages of the world: A comparative handbook, 195–233. Berlin: Walter de Gruyter GmbH & Co KG.10.1515/9781614518174-013Search in Google Scholar
McKee, Rachel & Sophia Wallingford. 2011. ‘So, well, whatever’: Discourse functions of palm-up in New Zealand Sign Language. Sign Language and Linguistics 14(2). 213–247. https://doi.org/10.1075/sll.14.2.01mck.Search in Google Scholar
McKenzie, Robert T. 1924. Exercise in education and medicine, 3rd edn. Philadelphia: W.B. Saunders Company.Search in Google Scholar
Neidle, Carol. 2000. SignStreamTM: A database tool for research on visual-gestural language. Boston, MA: Boston University Linguistic Research Project.Search in Google Scholar
Nikolaeva, Irina. 2016. Analyses of the semantics of mood. In Jan Nuyts & Johan van der Auwera (eds.), The Oxford handbook of mood and modality, 68–85. Oxford: Oxford University Press.Search in Google Scholar
Nuyts, Jan. 2001. Subjectivity as an evidential dimension in epistemic modal expressions. Journal of Pragmatics 33(3). 383–400. https://doi.org/10.1016/s0378-2166(00)00009-6.Search in Google Scholar
Oster, Harriet. 1978. Facial expression and affect development. In Michael Lewis & Leonard A. Rosenblum (eds.), The development of affect, 43–75. New York: Springer.10.1007/978-1-4684-2616-8_3Search in Google Scholar
Oster, Harriet, Douglas Hegley & Linda Nagel. 1992. Adult judgments and fine-grained analysis of infant facial expressions: Testing the validity of a priori coding formulas. Developmental Psychology 28(6). 1115–1131. https://doi.org/10.1037/0012-1649.28.6.1115.Search in Google Scholar
Pfau, Roland & Josep Quer. 2010. Nonmanuals: Their grammatical and prosodic roles. In Diane Brentari (ed.), Sign languages Cambridge language surveys, 381–402. Cambridge: Cambridge University Press.10.1017/CBO9780511712203.018Search in Google Scholar
Reilly, Judy. 2006. How faces come to serve grammar: The development of nonmanual morphology in American Sign Language. In Brenda Schick, Marc Marschark & Patricia E. Spencer (eds.), Advances in the sign language development of deaf children, 262–290. Oxford: Oxford University Press.10.1093/acprof:oso/9780195180947.003.0011Search in Google Scholar
Radutzky, Elena. 1992. Dizionario bilingue elementare della lingua italiana dei segni. Rome: Edizioni Kappa.Search in Google Scholar
Reilly, Judy S., Marina McIntire & Ursula. Bellugi. 1990. The acquisition of conditionals in American Sign Language: Grammaticized facial expressions. Applied PsychoLinguistics 11. 369–392. https://doi.org/10.1017/s0142716400009632.Search in Google Scholar
Sandler, Wendy & Diane Lillo-Martin. 2006. Sign language and linguistic universals. Cambridge: Cambridge University Press.10.1017/CBO9781139163910Search in Google Scholar
Schalber, Katharina. 2015. Austrian sign language. In Julie B. Jepsen, Goedele De Clerck, Sam Lutalo-Kiingi & William B. McGregor (eds.), Sign languages of the world: A comparative handbook, 105–128. Berlin: Walter de Gruyter.10.1515/9781614518174-009Search in Google Scholar
Shaffer, Barbara & Terry Janzen. 2016. Modality and mood in American Sign Language. In Jan Nuyts & Johan van der Auwera (eds.), The Oxford handbook of mood and modality, 448–469. Oxford: Oxford University Press.10.1093/oxfordhb/9780199591435.013.17Search in Google Scholar
Shaffer, Barbara, Maria J. Jarque & Sherman. Wilcox. 2011. The expression of modality: Conversational data from two signed languages. In Márcia Teixeira Nogueira & Maria Fábiola Vasconcelos Lopes (eds.), Modo e modalidade: gramática, discurso e interação, 11–39. Fortaleza: Edições UFC.Search in Google Scholar
Shenhav, Amitai, Sebastian Musslic, Falk Lieder, Wouter Kool, Thomas L. Griffiths, Jonathan D. Cohen & Matthew M Botvinick. 2017. Toward a rational and mechanistic account of mental effort. Annual Review of Neuroscience 40. 99–124. https://doi.org/10.1146/annurev-neuro-072116-031526.Search in Google Scholar
Siyavoshi, Sara. 2019a. Hands and faces: The expression of modality in ZEI, Iranian Sign Language. Cognitive Linguistics 30(4). 655–686. https://doi.org/10.1515/cog-2018-0130.Search in Google Scholar
Siyavoshi, Sara. 2019b. The expression of modality in Iranian Sign Language (ZEI). Albuquerque, NM: University of New Mexico.10.1515/cog-2018-0130Search in Google Scholar
Streeck, Jürgen. 2009. Gesturecraft: The manu-facture of meaning. Amsterdam & Philadelphia: John Benjamins.10.1075/gs.2Search in Google Scholar
Sullivan, Margaret W. & Michael Lewis. 2003. Emotional expressions of young infants and children: A practitioner’s primer. Infants & Young Children 16(2). 120–142. https://doi.org/10.1097/00001163-200304000-00005.Search in Google Scholar
Tai, James & Jane Tsay. 2015. In Julie B. Jepsen, Goedele De Clerck, Sam Lutalo-Kiingi & William B. McGregor (eds.), Sign languages of the world: A comparative handbook, 771–809. Berlin: Walter de Gruyter GmbH & Co KG.10.1515/9781614518174-038Search in Google Scholar
Takkinen, Ritva, Tommi Jantunen & Outi Ahonen. 2015. Finnish sign language. In Julie B. Jepsen, Goedele De Clerck, Sam Lutalo-Kiingi & William B. McGregor (eds.), Sign languages of the world: A comparative handbook, 253–272. Berlin: Walter de Gruyter.10.1515/9781614518174-015Search in Google Scholar
Taylor, Lawrence J., Shiri Lev-Ari & Rolf A. Zwaan. 2008. Inferences about action engage action systems. Brain and Language 107. 62–67. https://doi.org/10.1016/j.bandl.2007.08.004.Search in Google Scholar
Thompson, Sandra A. 2002. “Object complements” and conversation towards a realistic account. Studies in Language 26(1). 125–163. https://doi.org/10.1075/sl.26.1.05tho.Search in Google Scholar
Veinberg, Silvana. 1993. La función lingüística de los rasgos no-manuales en la aseveración, la negación y la interrogación en la Lengua de Señas Argentina. Signo & Seña 2. 101–125.Search in Google Scholar
Wilbur, Ronnie B. 2000. Phonological and prosodic layering of nonmanuals in American Sign Language. The signs of language revisited: An anthology to honor Ursula Bellugi and Edward Klima, 215–244.Search in Google Scholar
Wilcox, Sherman. 2006. Modality in American Sign Language. In William Frawley (ed.), The expression of modality, 207–237. Berlin: Mouton de Gruyter.10.1515/9783110197570.207Search in Google Scholar
Wilcox, Sherman & Barbara Shaffer. 2017. Evidentiality and information source in signed languages. In Alexandra Y. Aikhenvald (ed.), Oxford handbook of evidentiality. Oxford: Oxford University Press.10.1093/oxfordhb/9780198759515.013.36Search in Google Scholar
Wilcox, Sherman & Phyllis P. Wilcox. 1995. The gestural expression of modality in American Sign Language. In Joan L. Bybee & Suzanne Fleischman (eds.), Modality in grammar and discourse, 135–162. Amsterdam: John Benjamins Publishing Company.10.1075/tsl.32.07wilSearch in Google Scholar
Wilcox, Sherman & André Xavier. 2013. The grammaticalization of embodied action: Is constructed action gesture or language? Paper presented at International Cognitive Linguistics Association, Edmonton, Canada.Search in Google Scholar
Woll, Bencie. 1981. Question structure in British Sign Language. In Bencie Woll, James Kyle & Margaret Deuchar (eds.), Perspectives on British Sign Language and deafness. London: Croom Helm.Search in Google Scholar
Xavier, André N. & Sherman. Wilcox. 2014. Necessity and possibility modals in Brazilian Sign Language (Libras). Linguistic Typology 18. 449–488. https://doi.org/10.1515/lingty-2014-0019.Search in Google Scholar
Zeshan, Ulrike. 2004. Interrogative constructions in signed languages: Crosslinguistic perspectives. Language 80(1). 7–39. https://doi.org/10.1353/lan.2004.0050.Search in Google Scholar
Zwaan, Rolf A. 2003. The immersed experiencer: Toward an embodied theory of language comprehension. Psychology of Learning and Motivation 44. 35–62. https://doi.org/10.1016/s0079-7421(03)44002-4.Search in Google Scholar
Zwaan, Rolf A. 1999. Embodied cognition, perceptual symbols, and situation models. Discourse Processes 28. 81–88. https://doi.org/10.1080/01638539909545070.Search in Google Scholar
© 2021 Walter de Gruyter GmbH, Berlin/Boston
Articles in the same Issue
- Frontmatter
- Research articles
- What can cognitive linguistics tell us about language-image relations? A multidimensional approach to intersemiotic convergence in multimodal texts
- Articulatory features of phonemes pattern to iconic meanings: evidence from cross-linguistic ideophones
- Exerting control: the grammatical meaning of facial displays in signed languages
- The linguistic dimensions of concrete and abstract concepts: lexical category, morphological structure, countability, and etymology
- A cognitive account of subjectivity put to the test: using an insertion task to investigate Mandarin result connectives
- The intrinsic frame of reference and the Dhivehi ‘FIBO’ system
Articles in the same Issue
- Frontmatter
- Research articles
- What can cognitive linguistics tell us about language-image relations? A multidimensional approach to intersemiotic convergence in multimodal texts
- Articulatory features of phonemes pattern to iconic meanings: evidence from cross-linguistic ideophones
- Exerting control: the grammatical meaning of facial displays in signed languages
- The linguistic dimensions of concrete and abstract concepts: lexical category, morphological structure, countability, and etymology
- A cognitive account of subjectivity put to the test: using an insertion task to investigate Mandarin result connectives
- The intrinsic frame of reference and the Dhivehi ‘FIBO’ system
