Introduction to “The visage as text”

1 The face as a sign system: limits and value of physiognomy

The present collection of articles discusses the meaning of faces and face-reading systems from antiquity to artificial intelligence in an interdisciplinary fashion. Indeed, this work draws on the history of physiognomy, the history of semiotics, and other disciplines such as cultural anthropology, psychology, and art history as well as computer science and artificial intelligence in order to discuss, problematize, and re-think a subject that is as compelling as it is vast and ineffable and inexhaustible. The interest in the study of the human face is constant in the history of humanity, a proclivity that resurfaces quite vividly today in the digital domain of “cyberfaces” (Belting 2017: 226), “simulacra” (Baudrillard 1995), “inter-faces” (Jirsa and Rosenberg 2019), and artificial intelligence and data-driven visual studies (Milton et al. 2020; Morin 2013). As we will see in what follows, the study of physiognomy also has some overlaps with the history of semiotics, especially with Greek medical semeiotics or diagnostic semiotics, where the signs of the face and skin have been treated as symptoms of diseases. For these reasons, we decided to devote a special issue of Chinese Semiotic Studies to the topic of face-reading systems from antiquity to artificial intelligence. To be sure, this editorial project places itself in a long-lasting research interest in the study of the aesthetics of face representations, with special attention to the contemporary digital culture. Moreover, the present endeavor stems from the research carried out within the European Research Council (ERC) project FACETS (Face Aesthetics in Contemporary E-Technological Societies), of which the present special issue constitutes a product. Indeed, the present thematic issue was not born in a vacuum and, therefore, its forerunners need to be acknowledged here. In this regard, we should mention the double special issue of the journal Sign Systems Studies, entitled “Cultures of the ace” (Gramigna and Leone 2021), the special issue published in the journal Topoi, entitled “What’s so special about the face? Visages at the crossroads between philosophy, semiotics and cognition” (Viola and Leone 2022), the “hors série” issue of the journal De Signis devoted to “El rostro en el horizonte digital de America Latina” [The face on the digital horizon of Latin America] (Voto et al. 2021), as well as the special issue devoted to “Volti Artificiali/Artificial faces” (Leone 2021) published in the journal Lexia. Rivista di semiotica, which all tackle a vast array of theoretical and applied issues of different facets of the face.

While the face is the most visible and characteristic part of the appearance and the anatomy of the human species – the “public” façade and the “prime symbol of the self” (Synnott 1989: 607) – it nonetheless remains a quite enigmatic and very difficult subject to fathom, firstly for its complexity and secondly for its ultimate ineffability. Moreover, the study of the face in its many nuances encompasses a large set of disciplines with heterogeneous research agendas (from literature and the arts, from criminology and psychology to evolutionary biology and neuroscience) that do not always intersect. And yet, from ancient treatises on physiognomy or physiognomics (Evans 1969; Magli 1988, 1995) to visual arts, from the introduction of automated face detection (Kosinsky 2021) and AI face-recognition systems in the modern-day “culture of surveillance” (Gates 2011) to the culture of “selfies” (Rettberg 2014; Tiidenberg and Cruz 2015), the face still remains a subject of ever-lasting interest.

As the Göttingen physicist Georg Christoph Lichtenberg – a leading figure in the debate on physiognomy in the eighteenth century and a fierce critic of Johann Caspar Lavater and his system of physiognomy – once wrote: “the most entertaining surface on earth […] is that of the human face” (Lichtenberg 1984: 245).^[1] Indeed, faces are expressive in many respects and are symbols with manifold meanings. Not surprisingly, the human face is thought of as a “living hieroglyphics” (Tomkins 1995: 263), “the signifier par excellence” (Ruesch and Kees 1959: 171), a “message board” (Birdwhistell 1968), and an “advertisement to the world” (Thorek 1946: 1). As a range of research conducted in cognitive psychology and neuroscience has shown, the face is a broadcast screen from which a multilayered array of stimuli is elicited and constantly monitored by others in real life settings. Identity, age, gender, skin pigmentation, basic emotions, affects and states of mind, and other information are “read” and decoded from faces in a game of guessing and outguessing others through facial expressions. As is well known, Darwin (1872) maintained that humans and nonhuman animals share a common ground in the expressions of emotions through the face. The tendency to decode human faces in order to underscore and unveil an inner language, a syntax, or a grammar of face features has continued throughout the centuries, as is witnessed by the attempts made by the painter Le Brun (1698), among others.

A glance at the history of physiognomy suffices to realize how much the human face has taken on different connotations and is often enmeshed in a sort of dichotomous logic. It is no coincidence that the study of the face has given rise to multiple cultural models and various interpretations or reading systems, schemes, and modes of representation. From Aristotle to the Neapolitan scientist and “magician” Giambattista Della Porta, from Lavater to Franz J. Gall up to Cesare Lombroso in Italy, the attempt of all physiognomists has always been to tame, as it were, and master an object of study – precisely the human face – that is intrinsically indomitable and incommensurable. As one of the finest interpreters of the human face once said, for faces “there is neither a barometer nor a system” (Kassner 1997: 39), and yet there have been many attempts to harness it into well-defined categories.

Ancient physiognomy was based on the principle of identity that is predicated upon the existence of a correspondence, a parallelism between two opposite poles: being and appearance, the inner (invisible) and the outer (visible), character and face, soul and body. It is, evidently, a quite rigid and deterministic scheme, which contributes to dichotomizing, pigeonholing it into quite rigid binarisms: face and mask, essence and appearance, dissimulation and expression, zoomorphic traits and human features. This type of approach, based on the principle of identity, held sway from Aristotle until the nineteenth century.

In Johann Caspar Lavater’s definition of physiognomy, this aspect is still quite evident. Physiognomy means the ability to recognize the inwardness of individuals from their outer appearance (Lavater 1991: 31). Thus, the challenge and aim, often disattended, of ancient physiognomy has always been to enclose the human face within certain categories, types, codes, characters, patterns, etc. (Lavater 1991: 31). This attitude to determining the face, typical of all physiognomy, constitutes both its profound ambition and its greatest limitation. This limit, which characterizes any physiognomic attempt at “reading” the face, or to put it with Stoicea (2020: 9), to attain a “corporeal legibility,” was well underscored by Rudolf Kassner, who wrote: “Physiognomy is not, nor does it need to be, nor can it be a science. On the contrary, most of what is false about it stems from its aspiration to scientific exactitude” (Kassner 1997: 69). The human face, thus, as an object of scientific rationality, has been analyzed, studied, and articulated by a series of assumptions and characteristics imposed from outside onto the object of study, which have made the face something potentially dissectible into various parts, manipulable, quantifiable, statistically predictable, and measurable.

2 Semiotics and physiognomy

As mentioned at the outset of this introduction, the history of physiognomy and the history of semiotics often intersect. Since classical antiquity, medical semeiotics as well as physiognomy of different traditions underscored the human face as an important element of the human appearance and bestowed it with a plethora of meanings. Indeed, the signs of the face found a special place and treatment in the symptomatology of certain diseases, as is apparent in Hippocrates. In this tradition, the signs of the face and skin were treated as symptoms of various pathologies.

Fourteenth- and fifteenth-century European physiognomy was widely accepted as part of a vast network of pseudo- and para-sciences that encompassed magical, esoteric, astrological, alchemical, chiromantic, and divinatory approaches. Physiognomy as a system of face reading was influential in Western culture but was by no means limited to this area. It suffices to mention the practice of reading the body as a fortune-telling practice that was prominent in the texts of Ming China, which feature a profound philosophy of the body and a sophisticated physiognomic system: xiangshu (Wang 2020). Ancient Chinese physiognomy was part of a larger field of knowledge, “somatomancy,” which sought to predict the future through reading the body (Leone 2019). Moreover, face reading is used in traditional Chinese medicine, too.

Besides the pseudo-Aristotelian physiognomic tradition, it is also important to mention the existence of another tradition, running parallel to the Western one: the Arab-Islamic physiognomic one. In particular, the study published in 1939 by Yousef Mourad commenting on the Fakiir Al-Dīn Al-Rāzī’s text Kitāb Al-Firāsa is noteworthy (Mourad 1939). This study discusses the practice of firāsa, which means “discernment,” “insight,” or “wisdom.” This term designates the deciphering and interpretation of signs, and especially the signs of the face and the body. It is also understood as a kind of “second sight” (Courtine and Haroche 1992 [1988]: 39): “Firāsa is the practice of glance and detail: the use of the perceptual intuition infers from the details of the face and body – a furtive movement of the gaze, a trait of the morphology of the nose or a certain distance between the teeth – the truth of a soul or the secrets of a heart” (Courtine and Haroche 1992 [1988]: 39).^[2] Thus, firāsa is the immediate extrapolation that occurs through the gaze of certain details of the face or body in order to trace relevant information. Within the tradition of Arab physiognomy, firāsa played a pivotal role and principally had a practical function (Courtine and Haroche 1992 [1988]: 39). Although it no longer has any particular value today, in the past, the physiognomist was instead considered in high regard as a very authoritative person. Indeed, as Courtine and Haroche pointed out, “the physiognomist [was] called upon in court to attribute paternity and establish guilt; at the slave market he advise[d] on the robustness of males or on the sexual attitudes of the females; for the sultan he recommend[ed] the choice of advisors, camels, horses; he predict[ed] the future (Courtine and Haroche 1992 [1988]: 39).^[3] Indeed, physiognomy was recognized as having a practical and scientific function: “physiognomy [was] then accredited as the most elective part of medicine, indeed as a summit of the scientific pyramid, the only one that by defining the human model gives reason to the entire cosmic system (Getrevi 1991: 10).^[4]

It is important to note that these ancient practices of deciphering the body and reading the face have played a pivotal role in the genesis of the “circumstantial paradigm” (Ginzburg 1986) and have a connection with the history of semiotics. In this regard, it suffices to mention that Johann Caspar Lavater’s physiognomy and Charles Darwin’s study on the expression of emotions in animals and humans were included among the key texts for a history of semiotics laid out by Umberto Eco (1979) at the inception of the development of the discipline.

Physiognomy quite never achieved a scientific status, and today the “physiognomic paradigm” (Gurisatti 2006) has lost the central position it had in the past. While we witness the dawn of physiognomy in its traditional meaning, new forms of face reading are presently resurfacing through the use of technology and artificial intelligence. The establishment of advanced machine learning, artificial intelligence-based algorithms, and network-based, automated services has altered the way we think of faces, providing some benefits, but also presenting new challenges in the years ahead. What in the past was thought of as a skill acquired through biological evolution and engrained in the genetic make-up of a species, today has become the hallmark of intelligent systems capable of performing complex operations that involve, to some extent, unprecedented face-recognition processes. In a nutshell, the way we conceive of faces has dramatically changed and opened up new scenarios.

3 Comparative semiotics of physiognomic cultures

As pointed out earlier, face reading plays a crucial role in ancient China, notably in the Shih-chi ^[5] and in the Han-shu.^[6] At the beginning of the “Annals of Liu Pang,” the founder of the Han dynasty, it is related that he had been engendered by a dragon and therefore had a face resembling that of the mythical creature. An old lady, Madame Hsu, who had the reputation of being able to read the future of people in their faces, was impressed by the resemblance and prophesized good fortune for the whole family (Hsu 1975).

The cultural semiotics of the face has indeed evolved in relation to the notions of power, individuality, and exceptionality: in the remote past, physiognomy was especially cultivated in relation to the “unique” faces of the emperors. In ancient Rome, for instance, the profile of the emperors’ face circulated through effigies on coins, and ancient authors tried to deduce, from those facial features, both the characters of the emperors and their fate.

The biographies of Suetonius methodically describe the physical countenance of the emperors. This offered a model to further Latin writers in the fourth century, such as Ammianus Marcellinus and the anonymous author of the Historia Augusta. It is well known that physiognomic literature flourished under the Roman Empire. Indeed, although physiognomy in Latin probably had its roots in the Greek Empire, many more ancient texts on physiognomy in Latin are extant than in Greek (Förster 1893). That shows a probable connection between the flourishing of physiognomy and the handling of power across the centuries and the civilizations (Barton 1994). The more power is centralized and acts in mysterious ways, the more – at least this is the impression – physiognomic literature, and especially manuals, multiply: there is an ample market and readership for the teaching of (pseudo-) strategies for the decoding of faces in societies that become more and more chaotic and incomprehensible, and, above all, in those imperial capital cities where people of mysterious background and ambiguous countenance converge from everywhere. It is urgent, in these circumstances, to learn how to “read” the faces of others, and especially those of the powerful ones.

The simple juxtaposition of texts from very different cultural areas and historical epochs shows a similarity that cultural semiotics is called to explore: in the West as in the East, in ancient Rome as in early imperial China, the face was not only turned into a surface for the reading of both the personality and the future of people, but also made the object of a specialistic literature (the manuals of physiognomy), which could grant a lot of fame, social success, and economic income to their authors. Physiognomy, indeed, was presented even in ancient times more as an art than as a technique, as an activity that would leave plenty of space for interpretation and, therefore, both required and legitimized the figure of the expert physiognomist (Rohrbacher 2010). The expert could choose to share some of his knowledge with a vaster audience, yet this sharing was always vague and partial, because it served more the purpose of building the reputation of the face-reader than that of disseminating his supposed knowledge in a systematic way. The world was chaotic, and face reading was urgent, above all in relation to those distant and semi-divine faces, those of the emperors, whose personality could change the destiny of thousands of lives. Therefore, physiognomy was needed, as well as its manuals and experts, both in ancient Rome and in ancient China.

Besides this point of comparison and similarity, however, the semiotician of physiognomic cultures must also point out the differences. A crucial one concerns the adoption of either a divinatory paradigm, which would see the face as the immutable omen of a future to come, or a medical paradigm, which, instead, would see the face as something that, to a certain extent, human beings, and especially the powerful ones, could control and even change through the mastery of their habits, thanks to a continence of life that would indeed turn into the countenance of life, into a visage. The face, in ancient Rome, was also a medical object to transform as the counterpart of a life of rectitude. Ancient manuals of physiognomy, and above all those that were offered for deciphering the faces of emperors, show this oscillation between a face that one receives from destiny, and is therefore an omen of the future, and a face that one forges by one’s own will, and is therefore a sign of the personality more than one of fate. The two lines of reading diverge and intersect along the centuries and the geographies of physiognomy, and arrive intertwined until us, and until our times, in which the face is still at the crossroad of opposite ideologies of meaning, stressing either its immutable nature or its cultural malleability. Semiotics, which has its ancient roots in both the paradigm of divination and in that of medical semeiotics, is in the perfect position to further explore this dialectic, especially through that comparison of apparently distant civilizations that the semiotic eye, and a forum like Chinese Semiotic Studies, can grant.