Signs of life: a process philosophy of audible semiotics

1 A brief introduction to semiotics

Semiotics and semiology are terms that describe methods of classifying ambiguous sign relations in language, symbolic thought, and everyday experience. Aristotle’s (1936) sēmeion (σημεῖον) in “On Interpretation” is primarily dealing with words and their signatory functions. Medieval scholarship, lifting off from the work of St. Augustine, similarly focuses on the rhetorical sign in a speaking relation. But Augustine’s writings also present a distinction between natural signs (signa naturalia) and given signs (signa data; Meier-Oeser 2011). The second involves the communication of abstract ideas between people while the first are described as ideas formed through visual interpretation – fire from smoke, animal from tracks on the ground, etc. Augustine’s originality, influential in the development of a Medieval semiotics, lies in his account of natural, indexical signs on the one hand and linguistic signs on the other (Meier-Oeser 2011).^[1] Natural signs would return to prominence through Roger Bacon’s system of signs, accounting for the importance of relations and inference; but as with Medieval semiotics, the focus is linguistic. A consequence of this development, sometimes explicit, is that signs-as-language is differentiated from images-as-representations. The latter is a separate discipline, working through issues of likeness, semblance, identity, and other aspects of Aristotelian mimesis.

The Stoics conceived of three parts to a sign: “the mental representation,” “the real thing,” and “the utterable” (Barthes 1967: 43). In all these cases, we find the dual beacons of human knowledge – the visual and the stated – at the forefront of epistemology. As philosophy has furthered its discourse on what is seen or spoken, it has historically neglected states of hearing and listening that involve neither. Key to an audible semiotics is coexistence and simultaneity in the passage of time: While we are consciously involved in visual and spoken phenomena, the audible sign is other than, but not divided from, the more conscious-level objects, statements, and image processes and events that are also underway. Any event of signification is simultaneous with other events that occur, forming a temporal coexistence.

The field of semiotics gained its most influential status in the late-nineteenth and early-twentieth centuries through Charles Saunders Peirce (and to a lesser extent, William James) in the United States and Ferdinand de Saussure in France. Peirce began working on a theory of signs in the 1860s. James completed his text The Principles of Psychology in 1890, which included foundational research on signs, language, and consciousness. Saussure’s lectures from 1906 to 1911 were collected into the Course in General Linguistics in 1916. Peirce and Saussure have garnered the most influence due to their systemic taxonomies. Saussure adopts the word “sémiologie” for his study of linguistics.^[2] This term therefore denotes the study of linguistic signs and the laws that govern them. He developed the now well-known signified/signifier relationship that constitutes a ‘sign’ (signe; 1959). The example he offered is the word “tree” as signifier and the concept tree as signified. The two comprise a unit – the meeting of a signified in a signifier is a sign of potential meaning. This dialectical relation “carries” an arbitrary association of the concept to imply “the idea of the whole” (1959: 67). Listening, for Saussure, is to, as or in language. Listening to language is less an interpretation of things and their actions and more an interpretation that functions within the system of language as such.

2 Semiotics and cinema through Peirce and Deleuze

Saussure’s semiology carries over to French cinema through theorists such as Christian Metz. Through cinema, semiology came to inform how we think about representations, images, and spoken sound. Metz’s formal elements of cinema present images, sounds, symbols, and iconography, which function as a connotative and denotative structure that is named as a “language” (Metz 1974). For Metz, cinema can only be cinema because it operates within this connotative and denotative structure. That is, without it being a language, it is simply a series of images and sounds. This is a debatable conclusion in film theory. Gilles Deleuze (1989), for example, took Metz to task for the latter’s claim that images are reducible to analogical signs and for presuming a language or language system that does not exist. Deleuze instead regards cinema as a simultaneous movement of images and signs. “Sign” here is neither linguistic not analogical but rather expressive. Neither image nor sign “refers,” as a language does, rather they do their own work of expressing and differentiating.^[3]

This is why Deleuze picks up from Peirce, whose “semeiotic”^[4] emphasized experience rather than language. Through his Kantian rather than linguistic approach, Peirce builds a network of perceptual laws as structural elements. Although he is working within a pragmatic, human framework, his theory of signs became a benchmark for today’s studies in biosemiotics and zoösemiotics,^[5] which will be addressed later. The following brief overview will help us to situate his terms:^[6]

1. Three modes of being/experience: Firstness (qualities), Secondness (facts, events), Thirdness (law, completion of the sign).^[7]

2. Triadic logic of signs: representamen (stands for something), interpretant (interpretation that creates the sign), object (thing or idea).^[8]

3. Three trichotomies of signs: For brevity, we focus on his second, which consists of the categories icon, index, and symbol. These are the types of signs that manifest in cognitive experiences of 1 and 2 above.

From the above, two elements are particularly relevant to a process philosophy of signs and an audible semiotics: the index (in secondness) and the interpretant. For Peirce, the creation of the sign comes in interpretation (interpretant). “The index,” which manifests in secondness, “is a sign which refers to the Object that it denotes by virtue of being really affected by that Object” (1955: 102). An index works for sound and audibility because it describes the activity of a real, individual unit that directs the attention, but which does not resemble the object with which it shares the semiotic relation. Peirce uses examples of hearing a sound in his descriptions, but he places the point of emphasis on the sound rather than the hearing of it.^[9]

Deleuze (1989) applies Peirce’s trichotomy to produce a temporal semiotics of images which themselves function in semiotic relations, expressing in their composition and recombinations. That is, while Deleuze writes that cinema is made of images and signs, the latter constitutes the field of relationships that makes images expressive. Deleuze’s cinematic metaphysics is underdeveloped in regard to sound outside of music or language. Yet his semiotics evokes a complex matrix of relations, which functions well for audible signs. For Deleuze (1989: 30), the function of a sign is a “cognitive one: not that the sign makes its object known; on the contrary, it presupposes knowledge of the object in another sign, but adds new elements of knowledge to it as a function of the interpretant.” If we apply his thought specifically to listening, the act is less about identifying the object and more about the action of the mind in interpreting what is underway in the relationship. Further, in the cinematic arts of “sound design,” the creative act becomes less about an adherence of sound-to-object or -image and instead the development of an audible matrix through an array of sounds and hearings for interpretation.

If we regard the sign in the Aristotelian or Augustinian denotation, a sign is composed of its relation. It is not composed of identity (sound A equals idea B) but of how it is gathered (A leads to B).^[10] The “leads to” is the significance of the sign and there is no sign without significance. An audible philosophy of signs emphasizes not the object but rather the conscious relation – the gathering of the sign as an interpretive act. This composes the following tripartite audible relation: an object-event (that which moves and collides), a sonic-event (that which sounds and forms the index), and an audible-event (the significant relation born from gathering and making the sign). A semiotic hearing is one in which one’s mind must do the interpretive work from the ambiguity that makes a sign – that is, rendering an audible-event from a sonic-event. Cinema is an interesting experiment in thinking this relationship because of its creative and interpretive manipulation of all three. Ambiguity is most artfully designed in inferences rather than references. That is, each element gains significance psychologically and narratively more than materially. A sound designer is able to find different combinations among the three event types, detach each from the others, or create ambiguous closures from assumed meanings and relationships. In cinema sound design, it is not important that the object, sound and hearing match (reference), rather that the very relation is ambiguous, inferential, and interpretable.^[11] Further, returning us to Deleuze, any existing condition under which hearing occurs opens to a new set or series of audible and/or visual relations that may follow. A new matrix is born of any prior matrix, and the whole weave of activity comes alive in their processes of becoming, coexistence, and multiplicity.

3 Introducing and critiquing Williams’ concept of signs

These elements of relation and becoming are important in turning to a critique of Williams’ process philosophy of signs. Process philosophy is a deep topic, frequently attributed to thinkers such as Deleuze, Henri Bergson, and Alfred North Whitehead. Its general aim is to move philosophical ideas away from mechanistic, teleological, deterministic, and static concepts toward events and becomings in flux and change. Whitehead, for example, views process as a corrective to philosophy’s tendency to think of perceptual experience as the limit, which presupposes and maintains a dualism between consciousness and nature.^[12] Particular to the area of a process semiotics, the dualisms of philosophy (mind/body, consciousness/nature, etc.) are replaced with processional relations of interdependence, exchange, indiscernibility, etc. Like the three philosophers already mentioned, Williams is emphasizing process to overcome limited dualities of consciousness and language, such as subject/object and signifier/signified. Through process, a sign is free to enter into endless relations as itself.

Process, Williams writes, involves “a multiplicity of changing relations and values” (2016: 79). Specifically in regard to signs, process philosophy describes a sign as “not restricted by any rules, definitions, laws or qualities” (2016: 80). It is “not attached to a referent or to a meaning.” It instead “comes about as a connecting process, a change.” Rather than signifier, signified, mediation or “analogical translation,” a “process account” describes multiplicity and intensity (2016: 116). All of this is to say that the sign for Williams is not determined. A process sign is “topological rather than structural” (2016: 119). Its activity involves a selection of a set of changing intensities and relations, which draw up a diagram and modify all other relations at various points of intersection (2016: 116, 132). Put more simply, a sign is not bound to any pre-existing set of relations; it is free to enter into a range of other topologies. Williams takes from Deleuze the notion of “intensive” relations, rather than extensive ones. This means that it is not closed off from the outside; it is active within its “virtual” process of self-differentiation and self-creation, allowing it as well to move into other relations. For Williams, the sign pre-exists; it has its own life independent of any consciousness. Williams argues that we should not conceive of different typologies of signs or a variety of sign worlds; instead, “every sign is a world connected to all others” (2016: 126, emphasis added).

The launching point for Williams’ process philosophy of signs is a critique of Ludwig Wittgenstein, who states in the Philosophical Investigations that a sign is “dead” until it is given “life” through use. The fragment (frag. 432) in Wittgenstein is “Jedes Zeichen scheint allein tot. Was gibt ihm Leben? – Im Gebrauch lebt es. Hat es da den lebenden Atem in sich? – Oder ist der Gebrauch sein Atem?” This is rendered in the popular 2009 English translation of the Philosophical Investigations as ‘Every sign by itself seems dead. What gives it life? – In use it lives. Is it there that it has living breath within it? – Or is the use its breath?’ (2009: 135). However, Williams cites an older 1953/1958 version, which incorrectly translates the final “Atem” as ‘life’ rather than ‘breath’.^[13] This misrepresentation does not defeat Williams’ own vibrant concept of a process philosophy of signs. But this language of a living use does function as the crux of his turn against Wittgenstein and toward his process philosophy of signs. Where Wittgenstein emphasizes breath prior to use, Williams counters, “signs as process are alive prior to use” (2016: 14). It is likely, although one can only speculate, that Wittgenstein is drawing from the tradition of pneuma in Aristotle, Stoic metaphysics, and early Christianity. This Hellenistic pneuma concept is bound to processes of life, spirit, and soul. Regardless of his inferences, Wittgenstein is clearly stating the importance of an activity, that is, a process: in its use is its breathing; in its breathing is its use. There is no dead/alive duality here as Williams claims, but a co-relational and active process.

The crux of Williams’ text is to grant material being to the sign itself as it moves in processes of becoming. Williams’ sign is a relational thing but it also exists independent of pre-existing structures of relation, such as language or consciousness. This also reflects current trends, in this case, philosophies that de-emphasize a subject as the locus of activity in lived relations. Williams’ sign is able to engage in multiple processes. For any “sign” in existence, there is “a selection of a set, the condition for the appearance of any sign, before there can be meaning” (2016: 2). Saussure’s sign is the union of the signifier and signified, or the acoustic image and concept (Barthes 1967: 38). Within this nomenclature, to which semiotics and semiology is always responding, Williams’ sign is only the signifier, alive as itself. That is to say, “the sign” has its own real relations. This is something Saussure explicitly aimed to avoid, as Roland Barthes points out, in order to maintain the relation (1967: 38–39). It also defies Peirce’s first trichotomy of signs, notably the concept of the “representamen … which stands to somebody for something in some respect or capacity” and, as a concept, requires a mental cognition of it (1955: 99). Williams acknowledges his movement away from Peirce and Wittgenstein. His reason is that these thinkers have limited sign relations to strictly human relations. As evidence, he identifies their focus on use, meaning, reason, and pragmatics, but more particularly, in their concept of the sign as confined to what is linguistic or visual (or both).^[14]

Williams addresses an important problem shared with an audible philosophy as proposed here – the need to break the entrenched dependence on language, logic, reason, and subjectivity in Western philosophy. His method of overcoming this is to describe the process of signs as if they themselves are the life that is significant, that they have their own life within a “substratum” of activity (2016: 4). In reading Wittgenstein’s sign as dead until given life through use, Williams renders a problematic dualism of inert matter and meaningful sign. He replaces “meaning” with “intensity,” claiming this as the basis of a sign that is “haunted” by its prior life. “Meaning involves linguistic understanding,” Williams states, while “significance” is broader (2016: 19). When Williams writes of intensity, it is usually material rather than a conscious intensity, occurring at the level of a “substratum” (2016: 27). His effort to counter Wittgenstein is to de-psychologize signs. What follows is a semiotics that describes signs as “continuous processes unfolding at different speeds in many tangled lines and according to different diagrams” (2016: 29). Signs are “continuous,” “multiple,” “self-generating and self-dividing zombies, the living dead from a substratum we cannot escape” (2016: 31). Williams goes so far as to suggest a near-Kantian, noumenal realm: “The sign is outside space and outside time because the processes constitute the sign before it is situated… The sign is immanent to the processes rather than to a location” (2016: 89). Further, his sign is “self-sufficient… There is no subject of the selection” and no “goal… no law governing it.” Instead of the more familiar conception of semiotics as reference, interpretation and signification, Williams emphasizes “a priority of selection,” that is, a selected set in process “against a wider background.”^[15]

As to why “selection” is preferable to, or more ontologically accurate than, “use” is an argument that is never clearly made. Also unclear is whether, in discounting consciousness, aspects like sensation, attention or attraction have any importance. Instead, Williams writes in language that is both abstract and materialist: Signs by his definition are “continuous processes… self-generating and self-dividing… self-sufficient.”^[16] They have their own life that “is continuous and multiple” (2016: 31). He is careful in his language to remove consciousness from the process, to give “self” to the sign as a means of giving independence to a sign’s own real existence. Because it has its own intensive relations, the wide dynamic “diagram” can produce changes that are “topological rather than structural” (2016: 119). In his effort to counter a linguistic dominance in regard to signs, he has decoupled the abstract role of communication and de-psychologized signs. Granted, there is great value here in opening the possibilities of expression beyond the limitations of a rational subject. The problem is that in his opposition, Williams’ response reifies the limits he aims to overcome. He shares with Saussure a presumption that consciousness is only capable of linguistic thought. For Williams, it seems that there is only language-thought or there is sign-matter-life. In his fight against dualism, he has produced his own opposition of matter against consciousness.

Williams thereby falls into the same trap he accuses Wittgenstein of – that language is the foundation for regarding what is nonlinguistic. If we read Wittgenstein’s “meaning” and “use” as linguistic, the sign is limited in its possibilities. Williams’ “selection,” meanwhile, limits signs by its material opposition to a solely linguistic consciousness. Again, we can turn to cinema as a framework for thinking through what this suggests. Williams’ process philosophy of signs aligns with Metz and others who can only regard cinema as a language against a random chaos of images and sounds.^[17] In fact, cinema transcends language, and its best means of doing so is not through the representation of images, or material reality, but through non-linguistic, non-musical audible-events.

On one level, Williams’ conception of signs is dubious as to whether it overcomes dualities and whether it manages to escape language, representation and analogy for its own intensive processes. But it fails outright when it is considered audibly. The nonlinguistic audible sign can only be that which is gathered as a sign, whether by a human or some other life form with the capacity to hear. But because audibility is historically disregarded in favor of the dominance of language and images, this vital aspect has not been well considered. The sign resides in the object/sonic/audible relation, which is the locus of its gathering and articulation as a sign of possible significance. The word “meaning” is loaded with human coding, which is why Williams sets it aside. Yet a deer hearing the approach of (perhaps?) a predator certainly “means” something to that individual’s survival. A broader audible notion of meaning comes in its interpretation and “significance” from what is signified. Significance arises through the inferential relation of overlapping object-, sonic-, and audible-events.^[18] Significance is what develops in the attentive relation and the resulting interpretation of what the sign expresses. Its vitality is that it becomes meaningful through a duration of attentiveness.

4 Auditory biology: neurocognition, attention, working memory, and biosemiotics

The importance of attention was addressed in William James’ The Principles of Psychology and is key to his friend Henri Bergson’s concept of duration. It is also central to Edmund Husserl’s internal time consciousness, specifically the phenomenological-psychological states of retention and protention, which Husserl explains using experiences of listening to music (Gallagher and Zahavi 2008: 75–84; Husserl 1991: 25–29). In all three, there is a width or breadth in the engagement of what can be described as an event, sign or duration.

Two sub-disciplines of the cognitive sciences that study audible attention are relevant to an audible semiotics as proposed here. One involves case studies of “auditory attention” in psychology, neurology, and cognitive neuroscience. This centers on the neural activity of “sustained, selective, and divided attention” in everyday listening situations (Alain et al. 2013: 215). The other works under the moniker of “working memory,” which studies multi-modal durations of attentiveness. The former is specifically auditory and has a shorter history of research; it studies which areas of the brain are activated in processing sonic information in an environment. The second studies short-term memory attention, retention, and activation across sensory modalities, with particular focus on spoken and visual data.

Working memory (WM) refers to a “temporary storage system” that is cognitively accessible in short-term memory (Buchsbaum and D’Esposito 2013: 389). A “central executive” (CE) selects, rejects, and activates what is “‘just past’” (2013: 392, 389). Cognitive studies have historically divided working memory research into two areas: (1) the phonological loop (the “verbal” element; language and speech) and (2) the visual-spatial sketchpad (the “object and spatial” element; 2013: 393).^[19] Thus, the CE constitutes two separate neurological domains of “auditory-verbal and visual information” (2013: 392). The CE is necessarily limited in order to narrow and maintain the mind’s attention for WM to function. Its task is to select, suppress, and reactivate items in the prefrontal cortex (PFC), dispensing “attentional resources to the subsidiary components.”^[20] The CE thereby “intervenes to determine how the pool of attention is to be divided among the many possible actions” (2013: 392).

Working memory research helps to map brain activity in durations of attention. An event of actual signification emerges in such durations, with audible signs in particular requiring an attentive duration of working through the significance of what is unseen and unstated. This is why WM studies are relevant to an audible semiotics, specifically for the following reasons:

1. It is the prevailing science in the field of memory and attention;

2. it supports much of the work by philosophers such as Husserl, James and Bergson;

3. it describes what is underway in a semiotic engagement, namely:

   1. an intentional and attentional selection and rejection process,

   2. the formation of representations delivered to the PFC for conscious reasoning,

   3. our ability to maintain a duration of semiological (language) engagement,

   4. such a duration is limited to what can be stored and used in a moment of signification, significance or potential meaning;

4. most importantly, the science of WM is notable in that what we here call audibility is, as of now, a nascent and poorly developed field of research.

Regarding Point 4, empirical research in working memory has for decades branched into separate verbal (“phonological”) or visual (“sketchpad”) domains. These are the two stores of attentive cognitive processing in the CE that generate scientific interest, and their research is grounded on different empirical data sets. An audible semiotics (in the conceptual sense of non-visual and nonverbal signs) exists outside of CE consideration. Buchsbaum and D’Esposito mention that what little has been studied in regard to audible working memory is extrapolated from phonological test studies based on language processing.^[21] In the separate field of auditory attention, Alain et al. (2013) mention that visual attention dominates research and is often then “applied to… auditory attention,” pointing out that this “may be misleading” to what is distinctly auditory. As of now, there is little science on what is identified here as audibility – the distinct condition of being audible.

5 Biosemiotics: are sign and life the same?

Any biological species with the ability to process audible durations interprets signs from within a living sonic milieu. The problem with Williams’ account of signs is that he has conflated two different terms, indeed two different processes: sign and life. In this, Williams aligns with prevailing thought in biosemiotics. John Deely (2014: 375), for example, proposes “that semiosis not only surrounds life but pre-existed living things, and indeed shaped the universe so as to make living things possible in the first place.” To ground his argument, he looks to Peirce’s triadic relationship of signs to argue that Peirce’s “interpretant” needs no conscious subject. Partially quoting Peirce, Deely writes that “the ‘third term’ attained in the triadic sign relation ‘need not be of a mental mode of being,’ and hence that there need not be an interpreter in order for semiosis, i.e., the action of signs, to occur in the physical universe” (2014: 391–392). Deely states that what Peirce advanced was not an “interpreter… but rather the ‘neutral’ term interpretant, which, then, emphatically, need not be mental” (2014: 392). This, Deely continues, was “foundational to, the establishment of biosemiotics” and paved the way for zoösemiotics (2014: 392). It further grounds Deely’s own concept of a physiosemiotics – signs structured through the very inception of the physical universe.

Deely aims to incorporate Peircian semiotics while Williams discards it. It is important to note that (1) the two are working on different projects and (2) the former reads as more deterministic than the latter.^[22] Yet the authors are in agreement that life and signs are the same.^[23] Because Peirce’s statement on the interpretant is the launching point for biosemiotics and anchors Deely’s thesis, we need to closely examine what Peirce intended. First, Peirce’s statement comes in a discussion of semiotics in regard to pragmatism not biological life. In this section on signs, he begins by stating that “every thought is a sign” and then moves to scrutinizing “the action of a sign” (Peirce 1955: 274). His interest is in moving from a dyadic sign relation to a triadic one, which Deely notes as key to a bio-/physio-semiotics. Peirce’s triadic relation is “intelligent.” His example is a military commander ordering his soldiers to lower their muskets, which they do. The verbal command is the “object,” the source of the sign; it “represents the will of the officer” (Peirce 1955: 275). Peirce’s point is that this is not merely dyadic; the action is not like switching on a light. A sign only becomes triadic through a “mental representation” of an index. This action produces the effect (or “immediate object”) of the sign, which forms “another mental sign.” Without this third element, there is no “semiosy,” rather, an “‘automatic regulation’.” From this, Peirce offers his concept of the “interpretant” and only then states his claim, quoted by Deely: “The example of the imperative command shows that it need not be of a mental mode of being.” Peirce then immediately moves to defending the triadic necessity. He does not elaborate on what he means by “mental mode,” why it is unnecessary, or why he made such a qualification. Nothing else in this section makes reference to non-mental interpretation in an essay about the distinctly human philosophy of pragmatism. The grammatical subject is “the imperative command,” which is a human voice, but perhaps he means that it could be something other than a command; or perhaps he means that the sign need not be a rational (“thirdness”) element of consciousness, since his example is an imperative command that is responsive, that is, unmediated by consciousness. The imperative command can only have significance – to use Peirce’s term, “significat… logical depth” – if it is stated, heard, and interpreted, something he underscores. As Peirce writes in his definition of the interpretant, it stands for its “idea” – the representamen that is the object of thought (Peirce 1955: 99).

It is for these very reasons that Williams chooses not to engage Peirce in his process philosophy of signs.^[24] The interpretant is an engaged act of interpretation that can only be triadic. But in Williams’ process philosophy the sign is equivalent to life in its activity, without any need for interpretation. As Peirce (1955: 100) states, the interpretant is “mental.”^[25] When we conceive of an audible semiotics, we see the limits of both Williams and Deely: Both conceive thought and interpretation as exclusively pragmatic or rational. Thus, both remove human consciousness to make the sign into its own living activity. Life and sign become conflated wherein any sign pre-exists its use or meaning.

What Williams describes is life, not a sign. He describes the activity of signs as “special kinds of process which brings things into relation” (2016: 16). And: “signs may well be processes before they are in use” (2016: 17). Further, “that signs are always networks of continuous intensive processes…” As with Deely, there is no gatherer as the instigator of any of this, no person, animal or thing taking account. “The sign is always alive,” Williams writes, “always in process” (2016: 20). But this is true only insofar as it is made through attention. If not, it is life itself, not a sign. Williams offers “a continuity of living processes across all states of the sign” (2016: 22). But this is not the sign, these are the transcendental conditions of any sign, the living activity of possible signification, human or otherwise. Wittgenstein was right: The sign requires a duration of enlivening – a breathing process – to become a sign. Life is the process that requires becoming as a sign.

Williams correctly states that “Categories should only emerge later” (2016: 126). Life in its own network of relations does not care about categories. In this regard, his process semiotics is valuable for giving life its own network, or diagram, where relations are active and activated. But he misuses the concept of sign by conflating it with processes of life. Responding to Deleuze, Williams does not want to “allow for separate typologies of the sign. We should not have different worlds of signs, because every sign is a world connected to all others” (2016: 126). Such a claim only works if signs are regarded exclusively as abstract entities. It seems Williams wants it both ways: On one hand, signs are independent of human use and ontologically, materially and objectively real on their own; on the other hand he separates life/sign into two worlds, one human and the other non-human, while also claiming to be non-dualistic. Life indeed has a multiplicity of semiotic relations, human and non-human, but the sign is not what lives. Life lives. Williams’ formulation fails for audibility because he differentiates only the linguistic and the visual in his examples. This limits the full breadth of experience, produces a division, and fails to regard cognition and affectivity outside the old models of signification. He splits existence in a manner far more divisive than Wittgenstein.

An audible process philosophy of signs avoids these trappings. It does not limit meaning to any pre-existing diagram, because hearing and listening have no a priori structure other than what biological existence has enabled. It is human and non-human in its process of activity and thus has no division between species. Cognition and audibility are plural across various life forms and the overlapping worlds of such existence. Borrowing from the pioneering work in zoösemiotics by Thomas A. Sebeok and others, audible processes envelop all terrestrial life in ways that are phenomenal, interpretive, communicative, and creative.^[26] It recognizes that life moves of its own accord and that this process is infinitely plural, coexisting and simultaneous. From this, a sign may be gathered. Only after this can we say anything – engaging at this point in a different kind of semiosis – even as we continue the hearing and listening that never stops.

To be clear: life and sign are not synonymous. Life is in process regardless of any duration in which a sign emerges from within its activity. Rather than use or meaning, the essence of a sign is that it is given attention. A sign does not exist unless it is noticed, gathered, attended to, drawn forth from and as life within the process of life. A sign does not require thought, function or language; it only requires manifestation as a sign in its significance. A sign is nothing until it becomes one. This is the same for any element of life that is responsive, attentive and active.

6 An audible semiotics: interpretation both human and beyond

The audible sign is not automatic (dyadic), rather it composes through a triadic process: the physical world and its infinite collisions in time (object-events), giving birth to the physical sign becoming mental in hearing and listening (sonic-event), and the durational engagement of interpretation and interest that produces a response or idea as a sign (audible-event). The significance of any such overlapping, attentive duration of life’s collisions and their interpretation is not necessarily pragmatic, linguistic, rational, or reserved for human use. Any mind with the capacity to hear and interpret is engaged with an audible semiotics. It is inclusive of what Williams aims to bracket out (language, consciousness) while not being limited by those processes. In audibility, there is no sign without its interpretation, regardless of (to apply Linnaean taxonomy) the species, class or phylum in the animal kingdom.^[27] The requirement for an audible sign is the triadic overlapping of object/sonic/audible events in time and within that its element of interpretation via the index. A deer, turning its head toward a sonic-event and alert to the danger of an unseen object-event, is in an ambiguous state of responsiveness and interpretation. An audible-event does not necessarily require any cognitive, conscious or rational capacity, only that the form of life, in any individual’s moment of attention, has the capacity to gather meaning and interpret significance from listening.

To put forth an audible process philosophy of signs, we turn to a semiotics which is non-human and which escapes the human propensity toward visual and linguistic necessity. One of the most influential thinkers in this area is Jakob von Uexküll and his concept of “Umwelt.” Famously using a tick as an example, Uexküll builds a network of “perceptive signs” and “effect signs,” with meaning and use bound up in the effect mark. This occurs across all variety of life forms, with each form of life having its own “world” of signification (Uexküll 2010: 47–48). To have its own “semiotics,” there would need to be its own mode of gathering and attentiveness. But this would constitute entities having their own world and topology, which Williams dismisses. Williams criticizes Uexküll for producing closed worlds – a “closed nature of circuits and signs” – to advance his process substratum that is limitless, continuous and intensive (2016: 61). Williams implies that Uexküll’s account produces worlds that are transcendent and external on the basis that such worlds are closed off from each other, which is not the case. Each “bubble” maintains a capacity to interact with others and is thus involved in a local, plural, immanent environment. In Williams’ attempt to extract consciousness from his topology, he discounts how we give attention and care to what is local. This does not mean entities are isolated from one another. Williams would have benefitted from the help of Eduardo Kohn, whom he does not mention. Kohn (2013) writes that a sign is “alive” and not exclusive to humans. If we take human life as an example, a person is involved in both human and non-human semiotics of experience. Yet where Williams wants to break the dependency on consciousness and psychology, thus producing a duality, Kohn resists such a move. Instead, he is calling for an ethics of joining with the forces, movements and signs of life that become divided by efforts to exclude the non-human.

Deleuze is another target, whom Williams criticizes for having “a mistaken view of signs as many different kinds, with their own ways of functioning and their own laws” (2016: 125). Such a pluralism leads to “a divisive practice of signs” and a closure of “essential or natural types of signs” that cannot cross over into other types (2016: 126–127). But there is nothing in Deleuze that suggests a non-interaction nor a division of practice/type. Further, Deleuze is giving signs a relationality and creativity within a milieu of activity and becoming.

For Williams it seems that everything exists at the level of matter in action, conflating rather than distinguishing life and signs. If signs and life are the same, what is the point of saying any of this? His account becomes strictly nominal, a reorientation of terms to explain a non-human and highly presumptive ontology. In doing so, he has elevated explanation above all else and thereby falls into an exercise of language, a trap he expressly sought to avoid.

An audible sign in process is not a base materiality. The sonic-event is not the same, conceptually, as a sound. A sound is material, but a sonic-event is life becoming “the sign,” which constitutes a relationship in audibility. The audible sign is one kind of sign, a different sign than other types. It is a semiotics that at the conceptual level describes an engagement with sensory and cognitive life, the life of all who live in durational processes of hearing and listening.^[28] For Williams, the sign determines life; in an audible semiotics, the sign gives life to thought, affectivity, action and modification – and vice versa. Again, Williams reserves “meaning” only to what is linguistic, while signs are about a broader “significance” (2016: 17–18). This dualism separates the linguistic and conscious from signs and their life. Williams’ semiotics also evokes a Kantian metaphysics. On one hand, he argues for the multiplicity and variability of material relations, immanent to themselves and their processes. But on another, he claims that the sign, as intensive, resides “outside space and outside time” prior to it being situated in a location. An audible semiotics avoids these dualities and metaphysical speculations. It instead works through the multiplicity and variability of material and sensory/affective relations that are immanent and endlessly interpretable. The true variation – the true potential for modification, change and action – is made in the processual development and emergence of the semiotic relation. It lives and breathes in modifying the gathered situation – in how one responds, creates and acts upon such relations. This is how life is transformed.

An audible semiotics retains and emphasizes the interpretive and the sensory to seek other avenues of discovery and thought beyond language and vision. In audibility, the process sign retains its relational element, not by returning to language, nor by narrowing to some material status, but acknowledging that a sign can only be a sign in its gathering as a sign. Two arguments need to be reiterated here: (1) “life” and “sign” are not the same, and (2) semiotics is not concerned over what is, but how one lives, in this case how one lives audibly. Research in cognitive science and “working memory” have traditionally been split between rational/human and animalistic studies (Buchsbaum and D’Esposito 2013). But as the research continues, and these binary conclusions become outdated,^[29] studies need to account for the wholeness of audible experience across species. Audible life emphasizes how the multiplicity of forms hear and listen, respond and habituate, and move toward thought, interpretation or action. Audible awareness gives rise to a different set of concerns, while living within the same world of the stated and the seen, the linguistic and the visual. “Life” is not the same as “human” or “consciousness” – for who knows whom or what is listening? – rather it is what is required for a sign to become integrated within its processes. Life lives, and within life a sign is gathered.

7 The audible sign

What then is an audible sign, one that has neither note nor word, one that defies the convenient anthropomorphisms of “music” and “language”? We can think first of the idea of a “signal” from William James. In describing his “passive sensorial attention,” James writes that we need some motive for attention in order for a sound to become a signal (1950a: 418). We may hear a sound, but the “attention” requires a “signal.” From this, we can return to James’ friend Peirce. Again, Williams criticizes Peirce for his pragmatism, dependence on “reason and truth” and the linking of signs with sensations, functions and modes of thought” (Williams 2016: 183).^[30] He does not take value in Peirce’s notion of “carrying,” nor how Peirce’s semiotics features a sign that changes in meaning through use and experience. Yet these very elements that he discounts are critically important for an audible process philosophy of signs. First, the index is the sonic element of the sign, not a mode of thought. For Peirce (1955: 107), the index is “dynamical” in its relation to both an object and the experience we have of it, but he does give more weight to the object. Even as he writes of a “junction,” an index is a sound for Peirce, more than a hearing. As he writes: a rap on the door, anything that focuses attention, anything which startles us. The object may be unclear, but it is “physically connected” with the index (1955: 114). He states that without an object there is no index, and that an index can occur if there is no interpretant (1955: 104). That said, the index is not the limit of the sign. Peirce’s semiotics is rich in that “the sign” is less important than the mode in which it is gathered and interpreted, which is the point where Williams asserts his critique. The sign is also the mode of cognizing it as a sign – how it is represented and interpreted. The audible “sign” is therefore not only a sound; it is rather that the sign is being formed and reformed in a process unfolding to and within consciousness.

The audible sign that is neither spoken nor musical is a coexisting and unfolding relation of matter, sound, sensation and cognition. It is a plurality of durations that intertwine and resonate, thereby creating what we may audibly engage as “a sign.” As attended, its index points thought to what is possible within a vibratory duration of affectivity. But we must be cautious in our language. Rather than thinking the audible sign as some thing or phenomenon, we must instead consider the semiotic activity as occurring in audibility. Audibility is where and when the sign emerges, becomes significant, and transforms any situation. Williams’ semiotics struggles to extract language from the process. An audible becoming is a semiotics that loosens this dependence. This is because hearing and listening to that which is not language requires no language. Not only does it escape words and terms, it has no visual image, icon or symbol to inject prior meaning into the experience.^[31] Because the audible sign is far more ambiguous and arbitrary in its signification, it has far more freedom than both linguistic and image-based signs. But it cannot be removed from animal or human cognition and affect. These are the levels at which a sign is given its expressive breath, as Wittgenstein suggested.

8 Williams’ scenario: an audible critique

To understand why Williams’ concept of signs fails when regarded audibly, we have to critique our biases regarding visuality, language, and the language of visuality. Williams acknowledges that most of his examples of process signs are visual, even as he states that signs are linguistic by their “real nature.” He asserts here that signs should be considered linguistic because they are about “meaning.” In this, Williams is following the common thread in philosophy that conceptions of meaning can only be linguistic, which is an odd admission given that he is trying to strip language from his process. Nevertheless he does offer an audible example of his process signs to counter Wittgenstein’s living/dead distinction. This example allows us to both critique his claims and present how a process semiotics would function from an audible standpoint. Williams gives an example of the sound – but more a hearing of – a group of deer in a forest, which presents a sign that their activity may trigger an avalanche. Here it is in full:

For example, as you trek along an unstable ice and snow front, you hear the soft but growing crunching sound of deer leaping above and to the side of you. They are hungry and panicked. The cliff is melting and unstable. So you begin to compute the grim possibilities around the impending avalanche, listening to the hoof impacts. Where is the border between the dead sign and the live ones? Is it at the point where you once studied about deer coming lower down the mountain due to starvation, or when you first unconsciously registered their leaps as a presentiment of bad things about to happen? Or is it where you begin to act? Or where the deer first suffer from the effects of the harsh winter? Or simply where you begin some kind of linguistic analysis of the languages of avalanche, survival techniques and animal movements? According to my claim, any cut or definitive answer at the point of use is an abstraction from continuous processes because the cut will draw an artificial line between connected processes. (Williams 2016: 24–25)

Let us take a close look at every phase of the scenario he depicts and critique his statements from an audible perspective:

1. Williams writes: “as you trek along an unstable ice and snow front, you hear the soft but growing crunching sound…” At this point in his description, you have heard a sound: a sonic-event has become an audible-event. The conclusion he presumes you drew came in the continuance of his sentence: “…of deer leaping above and to the side of you. They are hungry and panicked.” He does not say how you know that the sound is of deer as opposed to elk or foxes. The error here is in presumptively naming a specific entity from a sound. He also does not say whether you only heard it, or saw-and-heard it, or if one sensation came before the other. Each of these states would unfold a different process of interpretation and understanding. However, we can assume from the complete narrative that you only heard and did not see it. The first critique therefore is that Williams is discounting the ambiguity that lives in any audible-event, presuming that you can automatically identify and name the object-event. Second, he presumes that from your hearing you also know, through the sonic-event, that they are hungry and panicked.

2. “The cliff is melting and unstable.” This is the prior and continuing condition of an environment in which you’ve found yourself. You know what the situation is and you are in the midst of it. Since you have consciously confirmed that deer are panicking within your current condition, you have interpreted the sign and concluded you are in danger. This is what brings the meaning of the sign, which requires no language.

3. “So you begin to compute the grim possibilities around the impending avalanche, listening to the hoof impacts.” This is the point of attention and action, which is the central executive processing of your working memory.^[32] We can presume you are considering the implications of this danger, as both the sonic-event and your listening continues within the audible-event. In other words, listening to a sound doesn’t stop, it continues and merges with thinking. There is a gathering and a taking-account within the condition you are already immersed in. This brings a durational coexistence of sound, sense, thought, mood and environment that builds into a response to this situation.

4. “Where is the border between the dead sign and the live ones?” This question is the heart of Williams’ description of this scenario and also presents his overall thesis. The answer is simple. There is no dead sign, there is the breathing flux of sonic and audible life. From this, a sign emerged and became enlivened through your condition of hearing that is turning toward thought in listening.

5. “Is it at the point where you once studied about deer coming lower down the mountain due to starvation…”? The answer is no. Your knowledge of deer is your experience and memory that you have gathered in life that preceded this event. “…or when you first unconsciously registered their leaps…”? This is the sign. But it is not “leaps.” As William James reminds us, a word within the formal structures of language and an audible sensation are not the same. Naming is its own mental process, different from the sensation Williams describes.^[33] A more accurate description of this event would be, when you first became attuned to an event that is interpretable in its plurality. The index manifests the co-compositional relation of the object-event, sonic-event and audible-event that manifests as the sign. Williams again: “…as a presentiment of bad things about to happen?” This is the duration in which listening is doing the work of reasoning, which is different from hearing.

6. “Or simply where you begin some kind of linguistic analysis of the languages of avalanche, survival techniques and animal movements?” Here the sign remains only as a resonance into a more reasoned organization of what has been gathered as the audible-event and translated into language. James (1950b: 240) calls the change to language-thought as the moment “we abandon mental images for the definite and unchangeable names which they suggest.” Working memory might identify this moment as a transfer from the central executive to the prefrontal cortex. This activity may occur or it may not. If it does, it enters into conscious, frontal-lobe thinking and reasoning within the pre-existing semiological structure of language. If it does not, perhaps it becomes something else – a moving image, a reverie, a Zen-like calm, a flight or otherwise body-affective response. But the broader point is that language is not commandeering sensation, rather we have a multiplicity that we can describe as a vibration or resonance of many durations of signification. In some sense, the audible and sonic relation carries on at various levels of the psyche, even while one’s conscious tendency is to renounce the sign and abandon audibility, as they are no longer useful, functional or meaningful for the situation. We can borrow from David Hume (2011) here, that an impression is made that gives way to an idea.^[34] This is the process in which sensation and perception have transitioned into an abstraction that arises out of the impression. At this moment, the sign has likely left; it is probably forgotten, leaving space in consciousness for reason and action. This is how working memory selects, divides and narrows attention. It is the kind of pragmatic activity of the mind Williams aims to discount, yet as audible, this is the outcome that is his question here.

7. “According to my claim, any cut or definitive answer at the point of use is an abstraction from continuous processes because the cut will draw an artificial line between connected processes.” This assessment is only true regarding what happens from gathering the audible sign. The cut of abstraction is not the sign, it is the use, which usually constitutes some state of attention that blends into action or some other response. Therefore, as audible, use and sign are not the same but they are intimately connected.

9 Conclusion: an audible process philosophy

The critique offered here is not that Williams’ concept of signs is wrong. He is correct that there is a network of life that is semiotic. Uexküll and Kohn also write of this, and as noted, against Williams’ claims, it is a network with its own receptivity and gathering. The critique instead is that when we consider signs as audible, his divisions and descriptions fail. This is evidenced when we consider the audible rather than the linguistic, visual, or Williams’ intensive topology of life. An audible semiotics reveals Williams’ process philosophy to fail, both in its dualism (a separation of consciousness and life) and its synthesis (that life and signs are the same). An audible semiotics, which describes another relational matrix, is the multiplicity and plurality that Williams is aiming to reveal. But it can only be audible. This isn’t merely psychology (which Williams rightly wishes to draw a process semiotics away from), because the sonic-event composes the sign. But to answer the age-old question, a sound unheard is not a sign, it’s a sound, that is, a process of life.^[35]

The issue with both Wittgenstein and Williams is the presumption that language is the limit of thought. Where the former dwells thoughtfully in this limit, the latter aims to abandon it. This notion of language as the limit it is as old as philosophy itself. By maintaining this limit, we miss out on the broader experience that gives life to an audible thinking, a thinking that is different. This is why, in so many cases, an audible semiotics is the best means of tackling the problems of a living semiotics. Audibility escapes the limits of language and images by coexisting with and passing through both. As interpretive and living within an unseeing state irreducible to language, an audible semiotics is more deeply interpretable, opening to thought that includes what is non-human.

While every semiotician struggles to get “outside of language,”^[36] one must caution against inclinations to do so in such a way that sets up a new duality, as Williams does. An audible process philosophy of signs shares in the need to turn away from language to emphasize instead the ambiguous plurality of lived experience. This experience can be anyone or anything with the capacity to hear – human or non-human. The main problems in Williams’ thesis and account is the conflation of life and signs and the removal of use from the relation. In this, it is a return to Wittgenstein, but without the emphasis on language nor the limits of the human.

An audible process philosophy of signs advances the processional, interpretive, world-opening work of Peirce, Uexküll and Deleuze. None of them gave particular attention how audibility is different, but their conceptual ground is implicitly audible, allowing work to continue from these frameworks. What is new in an audible process philosophy of signs is that audibility opens to a different way of gathering and interpreting what is semiotic in biological existence. The critiques of Williams and Deely have salience only because they are approached from an audible way of regarding semiotics. Contra Williams, an audible process philosophy of signs recognizes a distinction of life, events and signs in their plurality and their difference (both extensively, from each other, and as themselves in processes of intensive differentiation). Life exits and moves; it happens. Life lives, responds, acts, and in some cases thinks. Events are occurrences of life in the things that happen. Signs exist only in their attention, within the flux of life and the events that compose it. Life lives while the sign is only alive as it is a sign. Its process is in its being taken as a sign. Symbols and icons can be made and ignored. But an audible sign ignored never happened.

Finally, there is fertile space in an audible process philosophy of signs for cognitive neurosciences to test audible selection, rejection and activation in attentive states. But to do so, empirical research would need to recognize the all-important unfolding of ambiguity within attentive and creative acts composed in durations of working memory. WM research must work toward breaking the phonological/visual duality that restricts the breadth of interpretive cognition that is spread across a range of brain processes and areas. Auditory WM studies would be better served by collaborating with auditory cognition research, which uses time-variable sonic-events in their process. The outcome would be a better understanding of the roles of attention and interest in the interpretation of sonic-events. Studies of AWM and “auditory scene analysis,” for example, are predicated on selection, attention, and interest. Test material would need to be more experientially interesting to generate and study semiotic attention. To advance in this area, object-events must have the ambiguity and curiosity worthy of attention to generate audible-events worthy of interpretation. Finally, these sub-fields of auditory cognitive science and working memory are too often limited to human concerns. This is where philosophies of audibility and biosemiotics can help develop understandings about how audibility is distinct as a field of semiotic research.