Word formation patterns in the perception domain: a typological study of cross-modal semantic associations

1 Introduction

Meanings that are grouped together under a single word (‘colexified’, following François 2008) or linked via word formation (‘partially colexified’, following List 2023) tend to be conceptually related (excluding homonymy: Brugman 1988; Georgakopoulos and Polis 2022; Györi 2006; Traugott and Dasher 2002; Urban 2012, among others). These conceptual associations may be culture-bound, drawing on foundations unique to a particular language community or area (Evans and Wilkins 2000; Koptjevskaja-Tamm et al. 2022; Wilkins 1997). Alternatively, they may be observed across unrelated languages and therefore suggestive of pan-human cognitive underpinnings (Xu et al. 2020; Youn et al. 2016). A major goal in lexical typology is to delimit the variability in how languages distribute meanings across words and identify the motivations underlying recurrent patterns.

Recent years have witnessed an increase in typological studies of (strict) colexification^[1] (see, e.g., Di Natale et al. 2021; Georgakopoulos et al. 2022; Jackson et al. 2019; Norcliffe and Majid 2024; Tjuka 2024; Xu et al. 2020; Youn et al. 2016), driven in part by the compilation of large-scale databases such as the Database of Cross-Linguistic Colexifications (CLICS; Rzymski et al. 2020) and Lexibank (List et al. 2022). Probably to some extent due to the relative difficulty of creating equivalent typological databases of morphologically glossed vocabulary, the semantically-oriented study of word formation is still in its relative infancy. Notable exceptions include Urban (2012) and Schapper (2021), who study semantic associations in word formation and in multi-word expressions in the nominal domain from typological and areal perspectives.

In this study, we examine word formation in the verbal domain, specifically in verbs of perception (i.e., verbs denoting the concepts of seeing, hearing, smelling, etc.). Building on earlier seminal studies of perception verb typology (Evans and Wilkins 2000; Viberg 1984), Norcliffe and Majid (2024) compiled a geographically and genetically balanced sample of perception verb lexicons in 100 languages and examined patterns of colexification between sensory meanings. Certain pairs of sense modalities – in particular, hearing–touch, hearing–smell, and touch–taste – were found to be recurrently co-expressed in the same perception verb form across languages and geographical areas, while vision showed a robust tendency to remain lexically differentiated from all other senses. Here, we draw on the same language sample to examine patterns of word formation. As in Norcliffe and Majid’s (2024) study, our focus is on semantic associations between sense modality meanings; henceforth cross-modal associations. The phenomenon is exemplified in the two languages below, in which a source perception verb expressing one (or more) sense modality meaning combines with another element to express a different sense modality meaning:

We ask whether there are asymmetries in how sensory meanings are linked in word formation and to what extent the typological patterning mirrors that observed for colexification. To do this, we focus on two complementary aspects of cross-modal word formation: (i) the directionality of the semantic shifts that accompany word formation and (ii) the cross-modal associations themselves.

The directionality question can be stated as follows: Given a source (i.e., originating) sense modality meaning and target (i.e., extended or derived) sense modality meaning, is there a typological bias for some sense modalities to be sources and others to be targets? Early research on this topic suggests there is. In a classic typological study based on 53 languages, Viberg (1984) proposed that the perception verb domain is shaped by a hierarchy of senses (3), grounded in the dominance of vision, and to some extent audition, in human perception and neuroanatomy.

Viberg argued that the hierarchy is reflected in a number of asymmetries in sensory language, including asymmetries in the relative token frequency of different perception verbs, their morphological complexity and diachronic stability. Most relevant, Viberg also proposed the hierarchy constrained the directionality of semantic extensions within the perception domain. His evidence came from patterns of word formation as well as polysemy (colexification). With respect to word formation, Viberg found that a perception verb denoting a ‘higher’ sense modality (e.g., hearing) might extend its meaning – e.g., by combining with a noun or verb – to cover a ‘lower’ sensory meaning (e.g., touch or smell), but not the reverse (i.e., from smell to hearing or hearing to sight). For polysemy proper, Viberg (1984: 136) argued that it was generally possible to identify a prototypical as well as a secondary or extended meaning of the polysemous verb, based on various diagnostics, for example, how it is translated out of context. In such cases, Viberg claimed the prototypical meaning was always higher on the hierarchy than the secondary or extended meaning. Detailed research on Australian languages by Evans and Wilkins (2000) supported Viberg’s unidirectionality proposal. Evans and Wilkins examined patterns of word formation (“indirect semantic extensions” in their terminology) as well as diachronic extensions that result in synchronic polysemy (“direct semantic extensions”) and found meaning extensions in both conformed to Viberg’s hierarchical proposal.

While unidirectionality in the perception domain is sometimes held up as a semantic universal (Evans and Wilkins 2000), the typological robustness of the claim is difficult to assess based on past work: as Viberg was careful to acknowledge, his language sample was highly skewed towards European languages. Moreover, neither Viberg nor Evans and Wilkins’ study included information about the cross-linguistic frequencies of different extension types. In subsequent years, a scattering of counterexamples to Viberg’s unidirectionality proposal have been identified (Brenzinger and Fehn 2013; Ibarretxe-Antuñano 1999; Maslova 2004; Nakagawa 2012), casting some doubt on the generalizability of the original finding. To address this, we quantitatively test the unidirectionality claim for word formation, using a set of complex perception verbs drawn from Norcliffe and Majid’s (2024) balanced sample of languages.

In the second part of the paper, we examine the cross-modal associations themselves and ask whether there are cross-linguistic regularities with respect to which sensory concepts are linked via processes of word formation. Previous work suggests that not all combinations of sense modality meanings are equally likely to be associated in complex words. Viberg (1984) found that although semantic extensions proceeded unidirectionally from higher to lower sense modalities (3), they could skip modalities, with some cross-modal combinations unattested in extensions. He captured the attested patterns of semantic extensions in a refined ‘hierarchy’ (Figure 1).

Figure 1:

Patterns of semantic extensions in Viberg (1984).

Viberg suggested the observed extensions reflected ‘natural’ semantic relations between certain senses. The bifurcation between the contact senses (touch, taste) and the distal senses (hearing, smell) is noteworthy. Viberg also suggested there was a close relationship between smell and taste, reflecting the fact that smell is an integral part of flavor perception. Evans and Wilkin’s (2000) study of Australian languages revealed a similar pattern of associations, including the same bifurcation between touch–taste and hearing–smell and a direct link between taste and smell. Given that sensory meanings belong to the same ontological domain and are therefore all semantically close, it is not altogether surprising that languages should frequently link them in processes of word formation. More striking is the possibility raised by Viberg’s study that some pairs of sensory meanings are regularly conceptualized across cultures as more similar than others.

Because neither Viberg nor Evans and Wilkins presented frequency information, it is difficult to assess both the typological robustness of the generalizations captured in Figure 1, and whether there might be substantial differences in the cross-linguistic frequency of the different attested pairings. So, although Figure 1 shows vision linked to all modalities except smell, in later work, Viberg (2001: 1297) also claimed that vision has a strong tendency across languages not to colexify with non-visual modalities in perception verbs (see Norcliffe and Majid 2024 for discussion).^[3] Moreover, neither Viberg’s nor Evans and Wilkins’ study was based exclusively on evidence from word formation, rendering it difficult to isolate the patterns of semantic associations arising from this type of lexicalization process alone; both relied interchangeably on cases of word formation as well as on polysemy proper (strict colexification). Evans and Wilkins (2000: 553) in fact argue explicitly that both should be considered together because the same semantic association found in one language based on full identity between forms may be encountered in another based on partial identity (see also Vanhove 2008).

Although colexification and word formation can reflect the same kinds of semantic associations (Urban 2012), it is, however, not necessarily the case that they always do. At the most extreme, word formation processes can express semantic relationships that are, for reasons of interpretability, unlikely to arise via colexification. François (2008), for instance, offers as an example the derivational relationship between Latin spīro ‘breathe’ and ex-spīro ‘die’. Beyond these kinds of antonymic examples, it remains an empirical question to what extent colexification and word formation pattern in the same way with respect to the concepts they link.

For the perception verb domain, recent typological work has helped to clarify one half of this question, by focusing exclusively on typological patterns of colexification. Georgakopoulos et al. (2022) present a quantitative study of colexification in the perception and cognitive domains, based on data from CLICS², the second installment of the Database of Cross-Linguistic Colexifications. Georgakopoulos and colleagues found a tendency for the non-visual senses to colexify with each other, while vision, by contrast, was only linked to the non-visual senses via the cognitive domain. This latter finding is consistent with Viberg’s (2001) claim that vision tends to be kept apart from the other sense modalities in words. As Georgakopoulos and colleagues discuss, however, sampling imbalances in CLICS² made it difficult to reach firm typological conclusions: for colexifications in the perception domain, CLICS² was “massively overrepresented” for the Eurasian macro-area, “while Australia and North America together amount to less than 5 % of the colexification patterns” (2022, 461).

Norcliffe and Majid (2024) addressed these empirical limitations in a study of colexification based on a genetically and geographically balanced sample of 100 languages and replicated with an independent sample of 271 languages drawn from CLICS³. Confirming earlier reports, the study found that vision exhibited a strong bias to be lexically differentiated from the other senses across languages – that is, to not colexify with other sensory meanings. We refer to this as vision’s unimodal bias. For the remaining senses, not all cross-modal pairings were equally represented. Some combinations, in particular hearing–touch, as well as hearing–smell and touch–taste, were found recurrently across genetically and geographically unrelated languages. The frequent grouping of hearing–smell and touch–taste recapitulates Viberg’s original observation of a split between contact and non-contact senses. Critically, however, there was no evidence for a “close relationship” between taste and smell, contrary to Viberg (2001: 1300); these two senses were rarely co-expressed in perception verbs in a balanced sample of languages.^[4]

The negative bias against the colexification of smell and taste in perception verbs is unexpected: these senses are known to be closely associated in other areas of sensory vocabulary, a fact which has been ascribed to their close alignment at perceptual and neural levels (e.g., Winter 2019). Sensory norming studies, in which speakers are asked to rate how much a given property or object is experienced by each of the senses, also reveal the conceptual interdependency of smell and taste across languages (e.g., Chen et al. 2019; Lynott et al. 2020; Morucci et al. 2019; Speed and Majid 2017). As discussed in Norcliffe and Majid (2024), it is possible that their lack of association in perception verbs has a communicative, rather than conceptual basis. Experimental and corpus research shows that colexification is less common in contexts where confusability is particularly at stake (Brochhagen and Boleda 2022; Karjus et al. 2021). Due to their close alignment in other lexical categories, it is plausible that linguistic context cannot reliably disambiguate between an intended smell or taste interpretation of a perception verb that colexifies both meanings. So smell–taste colexifications in verbs may be inhibited for this reason. Similarly, vision’s unimodal bias may also have a basis in communicative need (Norcliffe and Majid 2024). Sensory norming studies show that words that strongly evoke one or more non-visual modalities also tend to be associated with vision (Chen et al. 2019; Lynott et al. 2020; Morucci et al. 2019; Speed and Majid 2017). This suggests that linguistic context cannot reliably disambiguate between visual and non-visual readings of ambiguous perception verbs.

If the colexification of certain sense modalities is inhibited for communicative reasons, this raises the interesting possibility that colexification and word formation may, despite previous assumptions to the contrary, show divergent typological patterning in the perception domain. This is because in word formation, the meanings of the source and target words are formally differentiated. Word formation – unlike colexification – does not give rise to ambiguity. Given their assumed close conceptual resemblance, we might expect smell and taste to associate via word formation more frequently than colexification. We might also expect that cross-modal word formations involving vision will not be inhibited to the extent observed for colexification. We address these possibilities here.

The remainder of this paper is structured as followed. In Section 2 we briefly describe the Perception Verb Database and present some initial typological generalizations regarding cross-modal word formation. Next, in Section 3 we address our first research question (directionality asymmetries), and in Section 4 the second research question (cross-modal associations). Finally, in Section 5, we consider the diachronic origins of complex perception expressions, which may provide some insight into the pattern of results observed.

2 The Perception Verb Database (PVDB) and some initial typological generalizations

In order to make robust typological generalizations, we must consider how language sampling methods influence outcomes. Previous studies of perception verbs have either relied on convenience samples (e.g., Georgakopoulos et al. 2022; Viberg 1984) or focused on one language area in depth (e.g., Evans and Wilkins 2000). Such studies, although informative in their own right, have limitations. In this study we use the Perception Verb Database (PVDB; Norcliffe and Majid 2024), which was manually created from high quality dictionaries and grammars. The language sampling followed Miestamo et al.’s (2016) method, in which each of six macro-areas – based on Hammarström and Donohue’s (2014) definitions – was sampled in proportion to its genealogical diversity. The procedure is described in detail in Norcliffe and Majid (2024) and the database is available online.^[5]

The PVDB contains perception verbs with non-controlled experiencer meanings (translational equivalents of ‘see’, ‘hear’ etc.) as well as controlled activity meanings (translational equivalents of ‘look’, ‘listen’, etc.). Only perceiver-oriented verbs are represented (i.e., where the experiencer is linked to the subject role in an active clause). Stimulus-oriented meanings (in which the perceptual stimulus is linked to the subject role, as in smell in the pancakes smell delicious) are not included (see Norcliffe and Majid 2024 for discussion). Lexical entries in the PVDB are coded for whether they are formally simple or complex.

A form is considered ‘complex’ if, semantically, it was listed in the source as the standard expression for designating perception via one of the five sense modalities and, formally, if it went beyond a simple, non-decomposable verb stem. Note that according to these criteria, our set of complex forms includes both complex lexical constituents (e.g., morphologically derived verb forms and compounds) as well as multi-word expressions (see Koptjevskaja-Tamm and Veselinova 2020 for a detailed discussion of simple and complex word structures in the lexicon). While multi-word expressions go beyond the level of the “word” (e.g., Haspelmath 2023), we consider them together with word formation proper for two reasons.

The first is a functional one: both morphologically complex words and complex expressions are used to name perceptual concepts (see Masini et al. 2022 for a similar argument with respect to “binominal compounds”). For example, in Ulwa (Keram) [yaul1241], the concept ‘to smell’ is designated with a verb + noun compound nambït-wana (‘odor’ + ‘sense’). The compound form functions as a single, polymorphemic lexical item according to language specific criteria, for example permitting object markers (preceding the entire complex word) and TAM suffixes (following the entire word), just like simple verbs in the language (Barlow 2018: 219). In Italian, by contrast, the concept ‘to smell’ is conventionally named with a multi-word expression in which a general perception verb sentire combines with an entire noun phrase l’odore di X ‘the odor of X’ (Collins 2024).

The second reason for considering complex words and multi-word expressions together is a practical one: in many cases we are unable to determine the correct structural analysis of a given complex form (according to the appropriate language-specific definitions of wordhood), due to lack of information in the source dictionary or grammar.

In the remainder of this paper, we use the term ‘complex verb’ as a cover term for both complex lexical items and multi-word expressions, and the term ‘word formation’ to refer to the process of creating such forms. Defined in this way, we took the set of cross-modal complex verbs in the PVDB and annotated them for their constituent parts (see Section 2.2).

2.1 Distribution of colexification and word formation strategies across languages

To obtain an overview, we classified the 100 languages in the PVDB into four types (see Figure 2):

1. Languages that lexicalize each sense modality meaning with a unique unimodal verb

2. Languages with at least one instance of cross-modal colexification, but no cross-modal word formation

3. Languages with at least one instance of cross-modal word formation, but no cross-modal colexification

4. Languages with at least one instance of both cross-modal colexification and cross-modal word formation.

Figure 2:

Perception verb paradigm types. (A) Examples of paradigm types according to presence versus absence of cross-modal colexification and word formation. (B) Frequencies of paradigm types in the PVDB.

For 34 languages there is a one-to-one mapping between sense modality meanings and verb forms. While this is evidently not a typologically rare strategy, it is more common for languages to have cross-modal colexification or word formation of some kind. Thirty-eight languages have at least one instance of cross-modal colexification, 13 have cross-modal word formation, and 16 have both strategies. Cross-modal word formations are found in all six macro-areas (see Table 1).

Table 1:

Languages with cross-modal word formation in the PVDB.

Languages (families)	Macro-area	Number of languages
Bokobaru (Mande) East Taa (Tuu) Luo (Nilotic)	Africa	3
Kayardild (Tangkic) Wardaman (Yangmanic)	Australia	2
Abkhazian (Abkhaz-Adyge) Italian (Indo-European) Korean (Koreanic)	Eurasia	4
Francisco León Zoque (Mixe-Zoque) Koasati (Muskogean) Mayangna (Misumalpan) Nisga’a (Tsimshian)	North America	4
Ambulas (Ndu) Duna (Isolate) Grass Koiari (Koiarian) Kalam (Nuclear Trans New Guinea) Kaluli (Bosavi) Konai (East Strickland) Mende (Sepik) Momu-Fas (Baibai-Fas) Rumu (Turama-Kikori) Ulwa (Keram) Yareba (Yareban)	Papunesia	11
Cofán (Isolate) Guambiano (Barbacoan) Kotiria (Tucanoan) Qawasqar (Kawesqar) Warao (Isolate)	South America	5

Together there are 44 unique cross-modal complex perception verbs in the PVDB. This is greater than the total number of languages with cross-modal complex perception verbs (n = 28), because languages sometimes have multiple complex forms (e.g., Luo in Figure 2A). There are 62 unique multi-sense verbs (i.e., verbs that colexify two or more modalities), differing in the number (range = 2–4) and type of sensory meanings encoded (see Norcliffe and Majid 2024).

3 Directionality biases: are cross-modal semantic shifts unidirectional?

Viberg claimed cross-modal semantic extensions are constrained by the proposed sense modality hierarchy, with the shift always proceeding from a higher to lower sense modality, for example sight > touch or hearing > smell. In this section we revisit this claim and test it quantitatively against the cross-modal complex verbs in the PVDB. We use two complementary approaches. We first determine whether, in accordance with the unidirectionality hypothesis, sources are more frequently associated with ‘higher’ than ‘lower’ sense modalities, and whether targets are more frequently associated with lower than higher sense modalities. To do this, we aggregate over sources and over targets of cross-modal semantic shifts, and use Bayesian logistic regression models to establish whether all five sense modalities exhibit the predicted asymmetry across languages. Second, we inspect the within-language source-target pairings and calculate how often the directionality of the shifts is consistent with the proposed hierarchy.

3.1 Source modalities versus target modalities

We extracted the set of 28 languages from the PVDB whose perception verb lexicons included one or more instances of cross-modal complex verbs and created a predictor variable Sense Modality (with five levels: sight, hearing, touch, taste and smell), which collapsed over agentive versus experiencer meanings of perception verbs. For each sense modality meaning in each language, we coded whether that modality was associated with a source or target perception verb. Specifically, if in a language a sense modality was encoded in at least one source verb, Source was coded as 1 for that modality, and 0 otherwise. The same procedure was followed for Target coding. Thus, we analyze separately whether there are cross-linguistic frequency differences between sense modalities with respect to whether they are encoded in source or target verbs.

To do this, we used Bayesian Logistic Mixed Effects Regression. The analysis was conducted with R 4.2.1 (R Core Team 2022). We used the package ‘tidyverse’, (version 1.3.2; Wickham et al. 2019) for data processing and visualization and ‘brms’ (version 2.17.0; Bürkner 2017, 2018, 2021) for Bayesian mixed effects regression models.^[6]

We fit separate models for sources (Model 1) and targets (Model 2). In both models the sole predictor was Sense Modality (sight, hearing, touch, taste, smell) which was Helmert-coded, so each level of the categorical predictor variable is compared to the mean of subsequent levels of that variable. This allowed us to compare (1) smell versus taste, (2) touch versus smell and taste, (3) hearing versus touch, smell and taste, and (4) sight versus hearing, touch, smell and taste. With Helmert coding, we can determine whether sense modality meanings that are increasingly higher on the proposed modality hierarchy are more likely to be encoded in source verbs, and those increasingly lower on the hierarchy encoded in target verbs.

In both models we included a random intercept for Macro-Area, which allowed macro-areas to have different baseline preferences. Fixed slopes had weakly informative priors (normal distribution centered at 0 with a standard deviation of 1: Lemoine 2019; McElreath 2020). We used the default brms priors for the standard deviation of random effects and residual errors, which take only positive values (Vasishth et al. 2018: 150). We report whether the 95 % credible intervals of the posterior distributions for each predictor included zero. Credible intervals not containing zero are interpreted as providing the strongest evidence for the effect of the predictor on the dependent variable. We also report the posterior probability of the effect being above or below zero. Markov chain Monte Carlo (MCMC) sampling was performed with four chains of 4,000 iterations each (including 2,000 warm-up iterations). We set the drift parameter delta to 0.99 to ensure convergence. Visual inspection of the chains and R ^ values of 1 indicated both models converged. Posterior predictive checks showed a good fit to the data for both models.

3.1.1 Model 1: Sense modalities associated with source verbs

The model revealed a reliable hierarchical effect (Figure 3), consistent with Viberg’s proposal. Source verbs were more likely to be associated with the visual modality than non-visual modalities (log odds = 0.24, SE = 0.09, 95 % credible interval [0.07, 0.42]), with a very high posterior probability of the effect being positive (β₁ > 0 = 99.7). Hearing meanings were also more likely to be encoded in source verbs compared to the lower modalities (log odds = 0.25, SE = 0.11; 95 % credible interval [0.02, 0.48]), again with a very high posterior probability of the effect being positive (β₁ > 0 = 98.7). Touch meanings likewise were encoded more often in source verbs compared to smell and taste (log odds = 0.42, SE = 0.17, 95 % credible interval [0.08, 0.77], β₁ > 0 = 99.1). However, there was no reliable difference between smell and taste (log odds = 0.23, SE = 0.34), with the posterior distributions firmly overlapping with zero (95 % credible interval [−0.41, 0.91]). Model stacking weights (leave-one-out cross validation with Pareto importance sampling) showed that the model including the Modality predictor provided considerably better predictive performance than a corresponding null model (null model weight = 0.268, predictor model weight = 0.732).

Figure 3:

Source verb coding results. (A) The proportion of languages encoding any of the five sense modalities in the source verb of a cross-modal shift. (B) Posterior distribution of logistic regression coefficients predicting whether languages have a source verb or not, for each of the five Helmert-coded Modality levels. 

3.1.2 Model 2: Sense modalities associated with target verbs

Once again, consistent with Viberg’s proposal, the model revealed a reliable hierarchical effect for target verb coding (Figure 4). The visual modality was reliably less likely to be coded as a target verb compared to non-visual modalities (log odds = −0.42, SE = 0.16, 95 % credible interval [−0.79, −0.14]), with a very high posterior probability of the effect being negative (β₁ < 0 = 99.9). Similarly, hear meanings were less likely to be encoded in target verbs compared to touch, taste and smell (log odds = −0.53, SE = 0.17; 95 % credible interval [−0.91, −0.23], β₁ < 0 = 100), and touch meanings were encoded in target verbs reliably less often than smell and taste (log odds = −0.42, SE = 0.17; 95 % credible interval [−0.77, −0.10], β₁ < 0 = 99.5). Again, there was no reliable difference between smell and taste (log odds = −0.29, SE = 0.27; 95 % credible interval [−0.83, 0.25]). Model stacking weights (leave-one-out cross validation with Pareto importance sampling) showed the model including Modality as a predictor provided considerably better predictive performance than a corresponding null model (null model weight = 0.147, predictor model weight = 0.853).

Figure 4:

Target verb coding results. (A) The proportion of languages encoding any of the five sense modalities in the target verb of a cross-modal shift. (B) Posterior distribution of logistic regression coefficients predicting whether languages have a target verb or not, for each of the five Helmert-coded Modality levels. 

3.1.3 Summary

The logistic regression models showed robust hierarchical effects consistent with Viberg’s original proposal. Source verbs of cross-modal semantic shifts were reliably more likely to encode modalities increasingly higher on the proposed hierarchy, while target verbs of cross-modal semantic shifts were reliably less likely to encode modalities increasingly higher on the hierarchy.

3.2 Directionality of the different shift types

Having established that sources and targets are, in the aggregate, associated with different ends of the sense modality hierarchy, we turn to specific within-language source-target pairings, to establish whether individual shifts are consistent with the proposed directionality. To do this, we considered separately each of the semantic shift types in Table 2, excluding shifts where the target verb is polysemous. This shift type is infrequent; only 3 out of 48 shifts (6 %) fall into this category. In all three cases, the target verb forms are morphologically derived and show lack of semantic transparency between constituent parts of the complex form. It is plausible that the derived verb form extended its meaning to cover additional sense modalities, facilitated by the loss of semantic transparency between the constituent parts of the complex form. Given the difficulty of establishing the original target meaning in these cases, we do not consider this shift type further.

For the remaining shift types, we consider each separately because categorizing the directionality requires a partially different approach in each case. While the directionality of simple shifts (A → B) can be easily identified, shifts involving multi-sense sources are potentially harder to classify, because the source verb could encode sensory meanings that fall hierarchically on either side of the target sensory meaning (e.g., if a complex verb denoting ‘feel’ is based on a source verb denoting both ‘hear’ and ‘smell’). We therefore deemed it appropriate to inspect this set separately. Type 3, which involves semantic narrowing, falls outside the scope of the unidirectionality proposal altogether, in that the modality of the target verb is included in the set of meanings of the source verb, so the target modality cannot properly be defined as either ‘lower’ or ‘higher’ than the source modalities. We can, nevertheless, inspect the semantic shifts in this set to determine whether the sense modality of the target verb tends to be the lowest among the set of meanings associated with the source.

The frequencies of the different shift types are presented in Figure 5. Below, we discuss each in turn.

Figure 5:

Frequencies of semantic shift types.

3.2.1 Directionality of simple shifts (A → B)

Inspection of simple A → B shifts reveals a robust directional asymmetry across languages, consistent with Viberg’s proposal (Figure 5A). Twenty-one of the total 22 shifts in this category involve a source verb that is higher on the proposed hierarchy than the target (see examples (9) and (10)).

(9)

see > taste
Yareba (Yareban) [yare1248]
ie + erásu ('eat '+ 'see, look')
→ ie erásu ‘to taste’	(Weimer and Weimer 1964: 151)

(10)

hear > smell
Grass Koiari (Koiarian) [gras1249]
vusika + uhuia (‘stink, bad odor’ + ‘hear/listen’)
→ vusika uhuia ‘to smell’	(Dutton 2003: 176)

The one case that stands apart is an instance of a shift from smell to taste, but since Viberg (1984) treated smell and taste as unordered on the hierarchy this is not strictly speaking an exception. Evans and Wilkins (2000) write that in Kayardild, the verb banyji has a basic meaning of ‘to smell’, but in combination with a verb of eating in a coverb construction, expresses the meaning ‘to taste’:

(11)

smell > taste
Kayardild (Tangkic) [kaya1319]
banyji-ja + diya-ja (‘smell’ + ‘eat’)
→ banyji-ja diya-ja ‘to taste’	(Evans and Wilkins 2000: 559)

3.2.2 Directionality of multi-sense source shifts (A + B(+) → C)

The second shift type, in which the source verb is a multi-sense verb, also conforms to the unidirectionality proposal (Figure 5B). In eight out of nine instances, the target modality does not outrank any of the modalities associated with the source verb (examples (12) and (13)). The one exception is Ulwa (Keram), in which the target verb kïkal wana ‘to hear, listen’ is based on the source verb wana ‘to feel, taste’, in combination with the body part noun kïkal ‘ear’. In this case, the target verb denotes a sense modality (‘hearing’) higher than the source verb (‘touch’, ‘taste’).

(12)

hear, listen, feel, sense > taste
Mende (Sepik) [mend1268]
a + misi (‘eat’ + ‘hear/listen/feel/sense’)
→ amisi ‘to taste’	(Nozawa 2006: 6)

(13)

hear, feel > smell
Rumu (Turama-Kikori) [rumu1243]
há + wara (‘smell (n.)’ + ‘hear/feel’)
→ há wara ‘to smell’	(Petterson 1999: 25)

3.2.3 Directionality of semantic narrowing (A + B(+) → B)

For semantic narrowing, there is a tendency (eight out of 10 instances) for target modalities to be the lowest of the set of source modalities (Figure 5C; see (14) and (15)). In one case, Kalam, the sense modality of the target verb is not the lowest, but neither is it the highest: the very general perception verb nŋ combines with the contact verb d ‘touch’ to express the agentive perceptual meaning ‘touch’ (16). Pawley (2020) has written extensively about the semantics of nŋ; in addition to denoting basic perception meanings, it also expresses a broad range of cognitive meanings. It is worth highlighting that this multi-word expression does not represent a strict semantic narrowing of the source verb nŋ, as the complex form carries an explicitly agentive meaning not present in the source verb meaning ‘feel’ (Kalam shows the same dynamicity shift in the complex form pug nŋ (sniff + nŋ), which expresses agentive ‘smell’).

(14)

hear, feel, taste, smell, sense > taste
Guambiano (Barbacoan) [guam1248]
ma + mɨrɨp (‘eat’ + ‘hear/feel/taste/smell/sense’)
→ ma mɨrɨp ‘to taste’	(Norcliffe, field notes)

(15)

hear, feel, taste, smell, perceive > smell
East Taa (Tuu)
|núm + tá̰ã (‘smell (itr.)’ + ‘hear/feel/taste/smell/perceive/understand’)
→ |núm tá̰ã ‘smell’	(Traill 1994: 154, 272)

(16)

see, look, hear, feel, smell, perceive > touch
Kalam (Nuclear Trans New Guinea) [kala1397]
d + nŋ (‘touch’ + ‘see/hear/feel/smell/perceive’)
→ d nŋ ‘to touch’	(Pawley 2020: 390)

Finally, Kalam is exceptional as it uses a complex verb form to express vision, based on the same highly polysemous source verb nŋ. In this case the target meaning is the highest of the modalities co-expressed by the source verb, according to Viberg’s hierarchy. This is the only case in the PVDB of a cross-modally derived vision verb (17).

(17)

see, look, hear, feel, smell, perceive > see
Kalam (Nuclear Trans New Guinea)
wdn + nŋ (‘eye’ + ‘see/hear/feel/smell/perceive’)
→ wdn nŋ ‘to see’	(Pawley 2020: 391)

3.2.4 Summary

Collapsing over simple and multi-sense source shift types, we find that 30 of the 31 – spanning 23 languages from 23 different language families and all six macro-areas – conform to the unidirectionality proposal. Consistent with the general trend, semantic narrowing shows an overall tendency – although not without exception – for the target meaning to be the lowest of the set of sense modalities co-expressed by the source verb.

4 Associative biases: are some cross-modal associations more typologically prevalent than others?

Having established that cross-modal semantic shifts based on word formation exhibit directionality, we turn to the possible combinations of sense modality meanings linked by these shifts. To examine whether some sense modalities are more likely to be associated via word formation, we visualize the attested cross-modal associations using a weighted directed graph, which we plotted using the R package ‘ggraph’ (version 2.0.5; Pedersen 2021). Because we are interested in recurrent cross-linguistic patterns, we pruned the edges of the graph so only cross-modal complex verb types attested more than once in the PVDB were included (see Croft 2022; Georgakopoulos et al. 2022 for a similar approach with respect to classical semantic maps). The final graph is based on seven complex verb types from 20 languages and 18 language families (Figure 6).^[7]

Figure 6:

A weighted directed graph of cross-modal associations based on word formation in the PVDB. The graph is directed (edges have arrow heads conveying the direction of shifts); cross-linguistic frequencies of associations are represented as edge weights (the thicker the edge, the greater the number of attested links between sources and targets); the size of the nodes represents their ‘out-degree’ score (calculated as the sum of the weights of the outgoing edges of a node).

The lower senses of touch, taste and smell have the lowest out-degree scores (0, calculated as the sum of the weights of the outgoing edges of a node) – there are no outgoing edges from these meanings, indicating they are never the source of a semantic shift. Sight has the highest out-degree score (15); this is driven mainly by the high frequency of shifts to taste. The multi-sense source hearing–touch–taste–smell has the next highest out-degree score (6), followed by hearing–touch (5) and hearing (4). Sight > taste is the most frequent shift overall (n = 12), followed by hearing > smell (n = 4). Notably, there are no shifts from hearing to touch (or touch to hearing). Smell and taste are also not linked.

Touch is linked to smell only when touch also colexifies hearing, and hearing is linked to taste only when hearing also colexifies touch. This suggests the association between touch and smell is a second order one, mediated via hearing and, similarly, the association between hearing and taste is a second order one, mediated via touch. These first and second order associations are schematized in Figure 7.

Figure 7:

A schematic semantic map of first and second order associations involving smell, hearing, touch and taste (the undirected dotted line between hearing and touch represents colexification; solid directed lines indicate word formation).

The association of hearing–smell and touch–taste possibly reflects the conceptual alignment of contact versus non-contact sensory perception (cf. Viberg 1984). While these groupings would appear to have a natural conceptual basis, the high frequency of sight > taste shifts is surprising, as it does not have an obvious conceptual basis. We therefore return to the original data for further qualitative study.

4.1 A closer look at the sight–taste pairings

The set of sight–taste pairings are found predominantly in verb + verb expressions, nine of 11 cases (Table 3); the remaining two involve verb + noun expressions. There is a striking consistency in the composition of the verb + verb forms: in all cases the non-perception verbal constituent is a verb of consumption, such as ‘eat’, or an activity verb referring to manner of consumption, such as ‘lick’. Together, the combination of the consumption plus vision verb yields the meaning ‘to taste’ or ‘to try food’.^[8] This specific strategy of taste predicate formation is found in three separate macro-areas: North America, South America and Papunesia (possibly reflecting a Pacific Rim areal trait, cf. Nichols 1992).

Table 3:

Complex V + V taste verbs composed of ‘eat’ + ‘see’.

Macro-area	Language (family)	Form	Reference source
North America	Francisco León Zoque (Mixe-Zoquean) [fran1266]	cyu’tu + isu (ate + saw) → cyu’tisu ‘tasted/tried (food)’	Engel et al. (1987, 32, 63)
	Koasati (Muskogean) [koas1236]	í:pat + hí:can (eat + see) → í:pat hí:can ‘taste by eating’	Kimball (1994, 94, 123)
		lásaplit + hí:can (lick once, touch with tongue, nibble + see) → lásaplit hí:can ‘taste by taking a small amount’	Kimball (1994, 94, 123)
Papunesia	Duna (Isolate) [duna1248]	neya + ke (eat + see) → neya ke ‘taste’	San Roque (2010)
	Kaluli (Bosavi) [kalu1248]	dabe + bo:ba (lick + see) → dabe bo:ba ‘try to taste’	Schieffelin and Feld (1988, 25)
	Mende (Sepik) [mend1268]	a + heye (eat + see, look, watch, know, try and find out how it is) → aheye ‘eat to find out how it tastes, smell to find out’	Nozowa (2006, 5)
	Momu-Fas (Baibai-Fas) [fass1245]	kiy-on (consume + see, watch, visit) → kiy-on ‘taste’	Honeyman (2017, 544)
South America	Cofán (Isolate) [cofa1242]	an + cañe (eat + look) → ancañe ‘try the taste’	Borman (1976, 17)
	Kotiria (Tucanoan) [guan1269]	chʉ + ñʉna (eat + see, look) → chʉ ñʉna ‘try the taste’	Waltz (2007, 74)

The vision verb, in most of these examples, contributes an attemptive (conative) modal meaning to the main predicate of consumption. In Kotiria (Tucanoan), Waltz (2007) notes that ñʉna, with the basic meaning of ‘to see/look’, functions as an auxiliary with the meaning ‘to try’ in combination with other verbs, including those from other semantic domains. In Francisco León Zoque, the vision verb isu ‘saw’ occurs as the final element in numerous complex verbs with a ‘try’ meaning, e.g., va’nisu ‘tried to sing’ (< v’anu ‘sang’), myesisu ‘tried on (clothes)’ (< myesu ‘put on clothes’), indicating a generalized attemptive function in such expressions (Engel et al. 1987).^[9] In Kaluli (Bosavi), the vision verb bo:ba also has the meaning ‘to try’ and occurs in a number of compound constructions in this function (Schieffelin and Feld 1988). Kaluli is consistent with Foley’s (1986: 152) general observation for Papuan languages: “the conative modality (the actor tries to perform the action) is almost universally signalled … with a serial verb construction involving the verb stem ‘see’”. We cannot verify whether the two remaining non-Papuan languages (Cofán, Koasati) also utilize their vision verb as a conative or semantically similar modal expression, but it is plausible. The lexical semantic relationship between seeing and trying has, moreover, been documented elsewhere for a number of languages, including Ewe (Ameka 2008), Korean (Lee 1993), Mongsen Ao (Coupe 2007), Sayan Turkic (Anderson 2004), Tariana (Aikhenvald 2003), and Chintang (San Roque et al. 2018); see Voinov (2013) for a cross-linguistic survey.

So, although vision is frequently linked to taste in V + V constructions, the association appears to be a second order one, mediated by an initial semantic extension from see > try (Figure 8).

Figure 8:

A schematic semantic map showing the associations between see, try, and taste.

The two instances of V + N taste predicates based on a vision verb may also involve the vision verb having a ‘try’ meaning. In these cases, the vision verb combines with a noun meaning ‘taste’ or ‘flavor’ to express the meaning ‘to taste’ (i.e., ‘to see the taste’).

(18)

see > taste
Korean (Koreanic) [kore1280]
mas + pota (taste (n.), flavor + see/look at, perceive, set eyes on)
→ mas pota ‘taste, try the flavor of, experience, learn’	(Martin et al. 1967: 588, 788)

(19)

see > taste
Warao (Isolate)
a riaba + mī (sweetness, flavor + see, look, find, achieve)
→ a riaba mī ‘to try/taste the flavor of’	(Barral 1979: 30, 293)

In Korean, Lee (1993) observes that the verb po- ‘to see’ can be used as an auxiliary with an attemptive function, so it follows that its complex predicate for ‘taste’ is also built on this extended ‘try’ meaning (i.e., ‘try the taste’). For Warao, a language isolate of Guyana, the picture is unclear, though we note the existence of other South American languages that use the semantically similar complex expression ‘test its sweetness’ to refer to ‘taste’, for example the Guaicuruan languages Mocoví [moco1246] (Buckwalter and Ruiz 2023) and Toba [toba1269] (Buckwalter and Sánchez 2023).

Turning briefly to the three instances of see > touch extensions, two are found in languages where ‘see’ is used with an extended ‘try’ meaning (Kotiria and Duna), showing in these cases too, the association between vision and touch is a second order one. The third, Qawasqar, Kaweskar [qawa1238] is unique in using a reflexive pronoun in combination with a vision verb to express ‘to feel’.

5 Diachronic considerations

Semantic extensions based on word formation and those based on colexification are often implicitly treated as independent kinds of lexical change (Evans and Wilkins 2000; Viberg 1984). Yet it is possible that one process is historically dependent on the other. It has sometimes been suggested that complex expressions may develop out of an earlier stage of polysemy (Brown and Witkowski 1983; Maslova 2004). Maslova (2004) discusses this possibility for perception verbs specifically, a process she refers to as “lexical specification”.^[10] Building on Maslova’s proposal, we can sketch the following diachronic scenario: A perception verb (e.g., ‘hear’) first extends its meaning to an additional sensory meaning (e.g., ‘smell’), resulting in polysemy (Stage 2). Due to the resulting ambiguity, speakers may opt to disambiguate the intended sensory meaning by collocating additional material (e.g., a sensory noun) (Stage 3). For example, speakers might use an odor noun to specify the intended meaning of a hear–smell verb is ‘smell’. Over time, through frequency of use, the collocated element may become obligatory when referring to smell, giving rise to a lexicalized complex expression designating smell. As a result, the polysemous hear–smell verb may drop ‘smell’ as a referent, narrowing its meaning to ‘hear’ (Stage 4). The end state is a complex expression carrying the meaning ‘smell’, which is based on a source verb that now only expresses ‘hear’.

When viewed synchronically at Stage 4, the word formation process appears to be based on a direct extension from a source verb expressing A to a derived verb expressing B (e.g. ‘hear’ > ‘smell’). Diachronically, however, there is no semantic extension at all, but instead semantic narrowing from a polysemous source verb co-expressing {A, B}, to a derived target verb expressing only {B} (e.g. ‘hear/smell’ > ‘smell’). Crucially, on this scenario, the functional motivation for the development of complexity is disambiguation of the intended meaning of a polysemous word.

There are languages in the complexity database that provide initial support for this scenario. As we saw in Section 3.2.3, the database contains cases of semantic narrowing, in which the modality meaning of the complex expression also belongs to the set of meanings co-expressed by a polysemous source verb. Such cases are compatible with Stage 3 of Figure 9, in which a polysemous verb optionally alternates with a more specified complex expression. In East Taa (Tuu), for example, the verb tá̰ã, which co-expresses any non-visual modality (including smell), alternates with a complex form |núm tá̰ã which is used specifically for smell (Traill 1994: 154, 272). It is based on the collocation of the intransitive verb |núm ‘to smell’ with the multi-sense verb tá̰ã. We might view such languages as at the incipient phase of the lexicalization process, before the collocated item is obligatory when referencing the target modality (i.e., Stage 4).

Figure 9:

The stages of the diachronic scenario of lexical specification (after Maslova 2004). f = form and m = meaning.

Available descriptions of East Taa do not provide details about the context of use for each of these alternating forms, but for two other cases we do have additional information. In Kalam, the general verb nŋ can, on its own, express any of the sensory meanings ‘see’, ‘look’, ‘hear’, ‘listen’, ‘feel by touch’ and ‘smell’, in addition to various cognitive meanings (Pawley 2020). Pawley writes that, where necessary, ‘see’ can be disambiguated from other meanings by optionally using a complex expression wdn nŋ (literally, ‘eye perceive’) or wdn-magi nŋ (‘eyeball perceive’). Speakers make use of various clausal strategies for disambiguating non-visual sense modality meanings (see Pawley 2020). Kalam thus constitutes a documented case where a complex expression is selected in place of an ambiguous polysemous form in cases of contextual ambiguity, consistent with Maslova’s original proposal.

Italian offers another supporting case. The general perception verb sentire, which can express any non-visual modality, typically combines with the phrases l’odore di ‘the odor of’ and il sapore di ‘the taste of’ to express ‘to smell’ and ‘to taste’ respectively. The complex expressions appear to be relatively conventionalized – in the Collins Italian-English Dictionary, for example, they are provided as the translation of English ‘to smell’ and ‘to taste’ rather than the simple verb.^[11] Nevertheless, the complex forms are not obligatory. In contexts where the intended meaning is recoverable from context, the noun phrase does not have to be expressed (C. Mazzuca, pers. comm.). Thus, just as in Kalam, we find optionality between a complex expression, and a simple polysemous source verb, based on whether there is ambiguity in context. Critically, for Italian we have information about earlier stages of history. Italian sentire goes back to Latin sentire, which could refer to any sense modality meaning (Galac 2020). We can, therefore, confirm that the collocation of a sensory noun phrase in Italian to express smell and taste is a case of specification of a sensory meaning of an originally polysemous/more semantically general source verb, rather than a semantic extension to a new modality meaning.

Importantly, the specification scenario provides a straightforward explanation for the directionality effects we reported in Section 3. Speakers tend to use longer or more complex linguistic forms where an intended meaning is less frequent – and hence less expected – in its context (e.g., Gibson et al. 2019; Haspelmath 2021; Zipf 1935). Accordingly, theories of communicative efficiency hold that language is shaped by the trade-off between competing pressures for simplicity and informativeness (e.g., Gibson et al. 2019; Haspelmath 2021; Levshina 2022). When an intended meaning is more frequent or easily inferable from context, less linguistic code is needed to convey the meaning, so it is more efficient to use a shorter form. By contrast, where the intended meaning is less frequent and hence less predictable in context, more linguistic code is required to successfully transmit the message. Thus, in cases of polysemy, speakers will be more likely to explicitly mark the less frequent meaning. Given an ambiguous hear–smell verb, for example, a speaker is more likely to explicitly mark or specify the less frequent meaning (smell).

Returning to the set of associative puzzles, the specification scenario would explain why smell–taste and visual–non-visual meanings are so rarely linked via word formation. If, diachronically, cross-modal derived perception expressions tend to develop out of multi-sense source verbs, it follows that combinations of sense modality meanings that do not typically colexify in languages (e.g., for communicative reasons) will also not come to be linked via word formation – the preceding historical stage (Stage 2 of Figure 9) on which such a process is dependent seldom arises to begin with.

The lexical specification scenario may also explain the peculiar discrepancy between colexification and word formation with respect to the cross-modal pairing hear–feel. Hear–feel is the pair of senses that most frequently colexifies across languages and geographic areas (Norcliffe and Majid 2024), yet we find that no languages in our sample link these two modalities via word formation. At first glance, this appears counter to the lexical specification hypothesis – if hear and feel frequently colexify, why are there no languages in the PVDB that have gone on to grammaticalize a complex expression based on such a polysemous source verb? The answer may lie in the linguistic resources that languages typically utilize for specifying the tactile modality in ambiguous contexts (Stage 3 of the lexical specification scenario).

While the available data are limited, reports in the literature suggest that in cases of contextual ambiguity involving multi-sense verbs, languages tend to distinguish the tactile modality by means of modifying verbal clauses that specify a particular manner of movement or pressure of the stimulus object on the body. In contrast to collocated nouns (e.g., a general stimulus noun such as ‘odor’), these are presumably less likely to combine with a polysemous perception verb and grammaticalize into complex perception expressions. Van Putten (2020: 433) observes for Avatime, for example, that to differentiate between ‘hear’ and ‘feel’ interpretations of the multi-sense verb nu (which can refer to any non-visual sense), speakers will make use of various ad-hoc constructional strategies that invoke touching the relevant body part or reference to the skin. Kalam is also reported to rely on clausal strategies to disambiguate tactile perception from other senses. Pawley (2020: 391) gives an example in which the general perception verb nŋ is interpreted as expressing tactile sensation since a preceding clause specifies contact of the stimulus object on the skin. In Guambiano (Barbacoan), when using the multi-sense verb mɨrɨp with an intended tactile meaning, speakers tend to make use of subordinate clauses that specify the nature of the tactile stimulus (Norcliffe, field notes).

The above preliminary datapoints suggest a preference to rely on clausal strategies to distinguish the tactile modality in cases of ambiguity. This may be because, as Strik Lievers and Winter (2018) observe, tactile perception is especially temporally dynamic, involving changing forces in contact with the skin. Because languages tend to lexicalize transient phenomena such as events and movements as verbs (Givón 2001; Murphy 2010 among others), it follows that languages will rely on verbal clauses for expressing tactile information. Consistent with this, Strik Lievers and Winter (2018) found for English that sensory meanings are differentially encoded with respect to lexical categories, with verbs over-represented for the domains of touch and hearing, which is also temporally dynamic.

Altogether, in the context of the lexical specification hypothesis, the different lexical affordances of each sense modality could provide an explanation for why it is typologically rare for hearing and touch to be linked via word formation. In the case of polysemies involving smell or taste, sensory nouns are typically recruited to disambiguate; these may grammaticalize into V + N compounds. In the case of hear–feel polysemies, however, languages may tend to rely on ad-hoc and diverse clausal strategies to specify tactile perception, possibly due to the greater dynamicity of the modality.

To take stock, a diachronic scenario in which complex expressions develop out of earlier polysemous source verbs would provide an account of the typological patterning: it would explain not only the observed directionality effects (for efficiency reasons, languages will tend to formally mark meanings that are less frequent in their context), but also the parallelisms with and divergences from colexification patterns in the same domain.

Logically, a historical relationship between polysemy and word formation could run in the other direction, i.e., polysemous words could develop out of earlier complex expressions. Urban (2011) proposes such a scenario in the context of complex nominal expressions. In the first stage of this scenario, a word designating one meaning combines with another word to produce a complex expression designating a new meaning. For example, a word meaning ‘skin’ combines with the noun ‘tree’ to express the meaning ‘bark’. Over time, due to “general principles of linguistic economy” (Urban 2011: 26), the complex form may undergo ellipsis, such that the original form comes to express both meanings (i.e., the form is now polysemous). For example, the element ‘tree’ is dropped from the complex form, with the simple word now designating both ‘bark’ and ‘skin’. While there are certainly documented cases of this kind of formal reduction of complex words (Brown and Witkowski 1983), this diachronic trajectory does not account for lexicalization patterns in the perception domain straightforwardly. If polysemous perception verbs regularly emerge from the formal reduction of erstwhile complex forms, then we cannot explain why hear–feel colexifications are so frequent cross-linguistically, and yet these two modalities are seldom linked via word formation. If complex forms are the diachronic source of cross-linguistically frequent polysemous hear–feel verbs, then we should regularly encounter such lexical expressions in languages.

6 Conclusion

Cross-modal word formation shows striking regularities across languages. The direction of cross-modal semantic shifts is asymmetric (running from the ‘higher’ senses to the ‘lower’) and the sensory meanings linked by such shifts show associative biases. While previous work has emphasized the role of human biology and neurophysiology to account for these lexicalization patterns, we argue instead that general conceptual, communicative, and diachronic principles better account for the mapping between words and meanings. Such an account is also preferable because it better captures both universal and culture-specific forces in meaning-making across languages.