Semantic micro-dynamics as a reflex of occurrence frequency: a semantic networks approach

2.1 Linguistic knowledge is usage-based

A linguistic category is created through the establishment of a mental pairing of (usually, but not necessarily) arbitrary acoustic, gestural or graphemic form with conceptual or grammatical function (Croft 2001; Goldberg 1995, 2006; Hoffmann and Trousdale 2013; Langacker 1987). This is achieved through entrenchment, that is, the mental integration of a form–function mapping as a result of repeated perception of the association between form and function in usage (Bybee 2007). Langacker (1987: 59) characterizes the process as follows:

Every use of a structure has a positive impact on its degree of entrenchment, whereas extended periods of disuse have a negative impact. With repeated use, a novel structure becomes progressively entrenched, to the point of becoming a unit […].

Thus, a linguistic category emerges cumulatively as positive evidence for the form–function mapping becomes available to the language learner (Behrens 2009; Bybee 2010; Ellis et al. 2013). Conversely, lack of attestations results in failure to establish a mental link and therefore a lack of acceptability (Ambridge et al. 2015; Braine and Brooks 1995). New, unattested links can nevertheless be formed by reanalysis or creative and context-dependent usage (cf. Section 2.4).

2.2 Linguistic representations are exemplar-based and display prototype effects

Exemplar theory (Bybee 2007; Goldinger 1998; Nosofsky 1988a; Pierrehumbert 2001, 2003) serves as a bridge between the usage-based approach and the theoretical requirement that models of linguistic representation be compatible with domain-general psychological mechanisms. The concept has been particularly prominent in phonology, where categories are taken to represent an individual’s aggregated experience with speech sounds. This takes the form of “a large cloud of remembered tokens of that category […] organized in a cognitive map, so that memories of highly similar instances are close to each other and memories of dissimilar instances are far apart” (Pierrehumbert 2001: 140). The idealized ‘default value’ of a phonological category, then, will be the value at the center of the phonetic region covered by the exemplars in the phonetic memory space. Statistically, this corresponds to the mode of the probability distribution of its occurrence in usage. Due to the focus on the probabilistic grounding of categories in linguistic experience, the main contribution of exemplar theory has been to account for observed frequency effects at the interface between phonology and morpho-syntax (Bybee 2007; Bybee and Thompson 1997) as well as the considerable amount of phonetic detail remembered by language users (Goldinger 1998; Jusczyk 2000; Miller 1997; Nygaard et al. 2000).

Exemplar models naturally accommodate prototype effects. This includes the common finding that some items are judged to be better or more basic representatives of a category than others and that these items are also retrieved more efficiently (Rosch 1975, 1978). The prototype structure of categories is a foundational notion in cognitive theory, particularly in semantics (Coleman and Kay 1981; Craig 1986; Geeraerts 1985, 1997). Its central claim is that concepts themselves display a radial or network-like structure, with prototypical members at the center and less prototypical ones at increasing distances from it. Group membership is established through similarity, which is judged based on a variable selection of ‘family resemblance’ features rather than a clearly delineated set of ‘necessary and sufficient’ ones (Rosch and Mervis 1975). Polysemy relations are also accommodated by this model: Basic meanings of a word form the prototype center, from which links extend to additional meanings at the margins (Cuyckens 1995; Lakoff 1987).

In analogy to the exemplar model in phonology, lexical concepts can also be thought of as exemplar clouds made up of rich, multi-modal memory traces of situations, objects, actions, or, broadly speaking, contexts in which a speaker has experienced the lexeme being applied. Exemplar theory would suggest that the placement of a category member within the graded prototype structure is to a large extent determined by frequency of occurrence (Nosofsky 1988b). A type of bird that is commonly encountered in a speaker’s native land (e.g., titmouse in Central Europe), may be more prototypical to a speaker than other species (e.g., parrots), as they may hear the term relatively more often applied to the former than the latter. Several factors may qualify the relationship between frequency and typicality, however. Prototypes do not need to match actual exemplars but may often take the shape of abstractions combining salient features from various exemplars (Divjak and Arppe 2013). Also, it has been reported that typicality correlates with age of acquisition, so that typical exemplars tend to be ones that are acquired earlier (Holmes and Ellis 2006), even allowing for the fact that age of acquisition and frequency are closely correlated (Kuperman et al. 2012).

If one accepts the programmatic non-modularity of linguistic knowledge in Cognitive Linguistics (Langacker 1987), the structure of lexical representations should in principle be no different from phonological or grammatical categories. While the variability captured by the cloud-like structure of phonological exemplars is mostly restricted to the formal side (i.e., the range of encountered articulations), variability in lexical semantics primarily concerns the functional side of the form–meaning mappings as referents denoted by the same item can differ from each other (e.g., leg ‘limb’ vs. ‘support for table top’ vs. ‘stage of journey or process’). Based on the assumptions of the exemplar model, conceptual knowledge can afford to be inclusive, capturing rich multi-modal information about referents and usage contexts. Thus, the exemplar-based representation of a lexical category is one in which “the tokens of words […] are represented in memory along with the situations they have been associated with in experience” (Bybee 2013: 64, cf. Croft 2021: 245–272). This is not to deny that abstraction and schematicity form an integral part of semantic representations. Rather, the existence of frequency and prototype effects suggests that an exemplar-like architecture lies at the basis of such abstractions and has a decisive influence on them.

2.3 Lexical meaning reflects distributional properties of words in language use

Not all lexical items refer to tangible objects that can be perceived with one’s senses. The meaning of many items may be almost entirely based on intralinguistic associations without direct recourse to extralinguistic reality. Yet, this does not seem to pose much of a problem to either communication or learning. It is a common experience that the meaning of a new word can be inferred from its linguistic context. Crucially, this is often possible even when the item relates to a relatively concrete referent. Observations such as these have prompted some linguists and lexicologists to formulate strong assumptions about the role of a lexeme’s distribution and collocational preferences in the representation of its meaning, best encapsulated by Firth’s famous quotation: “You shall know a word by the company it keeps!” (Firth 1957: 11).

The approach that regards collocational preferences as central to conceptual structure has become known as the distributional hypothesis. This line of thinking integrates well with an exemplar-based viewpoint, since the linguistic context of words forms part of “the situations they have been associated with in experience” (Bybee 2013: 64) and is stored in episodic memory alongside it. There are of course legitimate reasons to ask whether linguistic context should be considered the only kind of context that is relevant for conceptual representations. There is an inherent circularity in distributional models that try to explain the meaning of words solely by their associations with other words, which is sometimes referred to as the grounding problem. Studies informed by embodiment theory have pointed to sensomotoric experience to circumvent this problem (Barsalou 2010; Pulvermüller 2005). Nonetheless, it has been demonstrated that a concept’s perceptual features, i.e., the aspects that refer to non-linguistic modes of cognition such as vision, sound, touch and smell, can be successfully predicted based on linguistic input data alone (Johns and Jones 2012; Louwerse and Connell 2011; cf. Arbib et al. 2014). In other words, even though distributional theories may fall short of explaining how lexico-semantic representations form in the mind, they can still be used to model them, including properties that form part of the extralinguistic context (for hybrid approaches in the distributional vs. embodied debate, cf. Louwerse 2007; Vigliocco et al. 2009).

The distributional hypothesis has given rise to a thriving research tradition in computer linguistics, where it serves as the theoretical foundation for a range of methods for measuring and comparing lexical meaning(s) based on large collections of natural language data (Griffiths et al. 2007; Kutuzov et al. 2018; Landauer and Dumais 1997; Mikolov et al. 2013). The present paper takes inspiration from this rich fund of methods (cf. Section 3).

2.4 Semantic change originates in language usage

Since the linguistic categories in the minds of speakers emerge from usage, variation and change are considered integral features of language within the usage-based model (Ellis and Larsen-Freeman 2006; Langacker 1999: 91–146). Semantic change occurs when the conventionalized mapping between a lexical item and its referent(s) is altered in some way over time. Depending on whether the alteration takes place on the formal or functional end of the pairing, the change may be said to be onomasiological (what is X called) or semasiological (what does X mean), although changes that purely affect only one dimension while leaving the other untouched are probably rare (e.g., lexical replacement of body part vocabulary). For methodological reasons we are chiefly concerned with the semasiological perspective here (cf. Section 3).

On the one hand, semantic changes happen through rather subtle shifts in the pragmatic aspects of a word’s meaning as it is construed through usage. This phenomenon has been analyzed in terms of implicature and inference: Part of the pragmatic context in which an item is used is reinterpreted as an essential part of its conceptual structure (Traugott 2018; Traugott and Dasher 2002). On the other hand, semantic change may also come about as the result of creative language use for the purpose of effective communication (Geeraerts 1997). Thus, established form–meaning pairings can be extended to include new concepts that are related to the original one(s) by similarity (through metaphor) or contiguity (through metonymy), so that some conceptual features are shared between them (Lakoff and Johnson 1980; Sweetser 1990). Language is constantly used creatively in this way, to express concepts that still lack conventional linguistic form (e.g., the novel use of file for a section of memory space with a retrievable path on a computer hard drive) or to frame messages in specific ways by highlighting some features at the expense of others (e.g., as part of political rhetoric, Lakoff 2002). Mechanisms such as invited inferencing and metaphorical extension create polysemies that are themselves organized in terms of prototype relations. The most conspicuous semantic changes are those where the prototype center shifts over time so that a former peripheral conceptual cluster becomes more central (Györi 2002), not least due to changes in the relative frequency of usage. An example is the word bead, whose Middle English ancestor bede meant ‘prayer’. The original link between the two is metonymic: Beads assembled on a rosary are used to count prayers in Christian practice.

Assuming, as we do, that conceptual structure is highly inclusive in nature, these points suggest that semantic change is gradient, ranging from subtle contextual differences, over fairly independent uses, to perceived unrelatedness. In addition, change is diachronically gradual, as the prototypes inspired by input-based exemplars wax and wane with usage. In terms of a distributional operationalization of conceptual representation, this means that the similarity between the meanings of an item at different points in time can be regarded as the amount of similarity between their collocation networks in a distributional model. This is also the way that semantic similarity will be measured in this study (cf. Section 3).

2.5 The role of frequency in semantic change

Frequency of occurrence has long been recognized as an influential factor in explaining aspects of language behavior and language change. Frequency effects are commonly observed in psycholinguistic experiments, where higher frequency facilitates lexical access, processing speed and accuracy (Ellis 2002; Howes and Solomon 1951; Whaley 1978). This is in line with the predictions made by the exemplar model, according to which the most frequent instantiations of a category are also the most entrenched, located at the center of the exemplar cloud, and therefore accessed more easily (Bybee and Hopper 2001; Ellis et al. 2016: 45–60).

The effects of frequency on diachronic change are variable, depending on the kind of phenomenon in question. On the one hand, high-frequency items are more likely to undergo phonological reduction changes than less frequent ones (Bybee 2007). This often affects grammaticalizing items, such as gonna (< going to), which are routinely found in prosodically weak positions, but the same kind of routinization has also led to reductions in main class words such as cupboard. On the other hand, frequency can have the seemingly opposite effect of preserving forms through time: High-frequency items tend to resist paradigm leveling more than low-frequency items. Thus, irregular verb forms such as swim–swam–swum or eat–ate–eaten are among the most frequent verbs in English (Bybee 2007). The link between frequency and irregularity had previously been pointed out by Greenberg (1966). A similar phenomenon is observed in rates of lexical replacement, where words for frequent concepts display higher diachronic stability (Lieberman et al. 2007; Pagel et al. 2007). Such effects follow directly from the exemplar model, where high-frequency items can resist paradigmatic analogy forces due to their stable entrenchment within a tight exemplar cloud.

In comparison to phonology and morpho-syntax, the relationship between frequency and semantics had long remained relatively underexplored. This is somewhat surprising, considering that speculations about a possible correlation between semantic change and occurrence frequency go as far back as the writings of Paul (1995 [1880]), who speculated that rarer words display a higher tendency to undergo semantic reanalysis because occasion-bound ‘misinterpretations’ of such items are less likely to be corrected by repeated exposure to the ‘correct’ form–meaning mapping.^[1] A more recent corpus-driven study by Hamilton et al. (2016) has investigated the relationship between frequency and semantic change from a distributional perspective, drawing on two of the largest available historical language datasets, (Google N-Grams and COHA). They find that the rate of semantic change – measured as the cosine distance between the semantic vectors representing a word’s meaning at different points in time – is indeed negatively correlated with token frequency. This holds true even when a measure of a word’s degree of polysemy is included in the regression model. Polysemy itself seems to have the opposite effect of increasing the rate of semantic change, despite frequency and polysemy being positively correlated (i.e., other things being equal, the more frequent a word, the more polysemous it is, cf. also Casas et al. 2019; Jager et al. 2016).

Further word embeddings-based studies of different semantic change measures (Cafagna et al. 2019; Haider and Eger 2019; Rodina et al. 2019; Tahmasebi 2018) generally support the findings in Hamilton et al. (2016). However, a few studies employing short-term time spans (as we do here, cf. Section 3), have indicated that high word frequency does not necessarily imply stability of meaning or context-specific word usage (Del Tredici et al. 2019; Kahmann et al. 2017; Vylomova et al. 2019). Others suggest that the effect of frequency on the rate of semantic change could also be explained as model artifacts (Dubossarsky et al. 2016, 2017). There is a risk to overinterpret differences in word meaning representations that actually stem from noise in the data, although the issue is rather intricate as diachronic frequency movements may themselves contribute to such noise. Another study suggests a new approach for investigating the relationship between semantic change and frequency by comparing semantic divergence in a set of cognates across languages (Uban et al. 2019). That study finds an effect opposite to that observed by Hamilton et al. (2016). However, the authors concede that their word frequency ranks are not contemporaneous with their measures of semantic similarity. In contrast, Pagel et al. (2007) have adopted an onomasiological perspective, concluding that concepts that have high associated word frequencies in modern languages are more likely to be expressed through the cognates inherited from the common proto-language, while less frequent concepts are more prone to lexical replacement.

The present paper is strongly influenced by Hamilton et al. (2016) as we also study the effect of frequency on semantic change. In addition, an explicit aim is to integrate the findings with a Cognitive Linguistic framework. Since we assume, in line with current theory, that semantic change is historically gradual and fundamentally based on small usage variations that play out on the pragmatic level, we hypothesize that the effects found by Hamilton et al. (2016) should also be visible at much smaller time scales (e.g., months and years instead of centuries) given adequate data resolution. To accommodate this difference in perspective, our approach departs from previous research regarding the data used as well as the methods applied for modeling meaning, measuring semantic change over time and statistical analysis.

3.1 Corpus data

We base our analysis on the Austrian Media Corpus (AMC, Ransmayr et al. 2017), which is currently the largest diachronic corpus representing the Austrian variety of German. It collects texts (journalistic prose) from the majority of Austrian print and online media and spans a period of more than three decades. In total, it consists of about 11 billion word tokens (of 8.5 billion word-form types) distributed over 45 million texts. For our study, we use all print media texts from 1998 to 2018 (because sub-corpus sizes are considerably smaller for previous years). Within this period, yearly sub-corpus sizes range between 1.3 and 2.1 million texts. All texts are time-stamped and linguistically pre-processed (in particular, part-of-speech-tagged).

The size and granularity of the corpus are ideal for our purposes, since we are interested in semantic-contextual developments within a short time span. To ensure that derived time series are based on a large enough number of data points, a high temporal resolution is necessary. Fortunately, the AMC is large enough to extract reliable word-frequency estimates for monthly sub-periods: There are between 100,000 and 170,000 texts per month, so that the sizes of the 252 monthly sub-corpora range from 25 to 43 million word tokens (to provide some context: The well-studied Corpus of Historical American English, COHA, accommodates about 23 million word tokens, on average, per decade; Davies 2010). The sub-corpus sizes are certainly sufficient for the computational steps applied in the analysis (cf. Section 3.5).

The corpus represents a relatively confined variety (Austrian German) and genre (journalistic prose). Although this is an incidental characteristic of the corpus, which we have selected for its size and granularity, we do not consider this a disadvantage since any cognitively motivated relationships between frequency and contextual usage should be expected to be independent from the genre they are detected in. Clearly, frequency effects in contextual shifts operate subconsciously regardless of the text type. If such frequency effects can be detected in such a relatively standardized context (written, non-colloquial, journalistic language) they should be expected to extend to less normalized domains of linguistic usage.

3.2 Data set

Addressing the relationship between contextual-semantic change and occurrence frequency in a systematic way requires a data set that is representative of the predictor variables under investigation. Our starting point is a representative sample of lexical items, for which we then construct semantic network representations. Meaning variation and change will be measured by comparing the network representations associated with these lexical targets (cf. Section 3.3).

Since words in natural languages are distributed across the frequency spectrum in a notoriously uneven way (Zipf 1935, 1949), we may not succeed in detecting the expected effects if we relied on a purely random sample. Additionally, the occurrence frequency of any linguistic item should not be regarded as an invariant feature either, a fact that the ever-increasing number of historical linguistics studies documenting the waxing and waning of morphemes, constructions and lexemes testifies to. The lexical items’ actual semantic content, on the other hand, is of relatively minor importance for structuring our sample. It may safely be assumed that a good deal of semantic space will be covered by the sample given that it is large enough.^[2] Based on these considerations, we stratified our sample according to three frequency-related measures: (1) mean monthly frequency, (2) diachronic frequency development, and (3) frequency fluctuation. The corpus lexicon was divided into tertiles on each of these measures. For each combination of sampling strata, the maximal number of words allowing an equal number of items across all combinations was randomly selected, summing up to just below 3,000 items altogether (37 lemmas × 3 mean FRQ tertiles × 3 FRQ development tertiles × 3 FRQ fluctuation tertiles × 3 PoS = 2,997; see Supplementary Material A.1 for further details on the sampling procedure).^[3]

3.3 Lexical networks

We make use of network graphs to represent the context-based semantics of a target word and to measure semantic change. Graph-based methods have been employed in various areas of natural language processing (NLP). Lexical networks in particular have been used for unsupervised word sense induction (Akkasi and Snajder 2021; Biemann 2006; Hope and Keller 2013; Navigli and Lapata 2009), word sense disambiguation (Bevilacqua and Roberto 2020; Dorow et al. 2004), lexical semantic relatedness computation (Hughes and Ramage 2007), automatic synsets induction (Ustalov et al. 2017), unsupervised POS-tagging (Degórski 2013), unsupervised lexical acquisition (Biemann 2006; Widdows and Dorow 2002) or induction of lexical taxonomies in a semi-automated mode (Navigli et al. 2011). Recently, network-based algorithms have been applied to efficiently infer various similarity measures by computing node embeddings (Kutuzov et al. 2019). The structure of lexical networks also makes it possible to cluster nodes with high interconnectedness into sense-level groups, which helps determine the type of change (i.e., narrowing, broadening, birth, death), in addition to detecting the fact or degree of change (Jana et al. 2019; Mitra et al. 2014, 2015; Tahmasebi and Risse 2017). That is, lexical networks implicitly contain information about the semantic structure of words (and, in particular, polysemy). A promising application of graph theory in semantics can be found in a series of works on word usage graphs (McCarthy et al. 2016; Schlechtweg et al. 2021a, 2021b), where human-annotated similarity ratings of corpus examples of target words are used to create evaluation datasets for semantic change research.

Like many of the studies listed above, we apply an ego network approach: Each individual network embodies the contextual semantics of a target word based on its distributional characteristics in a given monthly sub-corpus. The nodes of the networks represent lexemes that have similar distributional usage characteristics to the target, while the edges in turn represent similarity relations among them (Figure 1). Thereby, we derive a densely interconnected and thus computationally stable representation of a target words’ semantic content based on the corpus data (cf. Supplementary Material A.2 for further details on network construction). Meaning change can then be detected by comparing ego-networks of the same target word across different time periods. The general idea of this approach is straightforward: Target words whose ego-networks are structured similarly (i.e., have similar nodes and edge structure) are also semantically similar.^[4]

Figure 1:

Hypothetical ego network for the target item dog. Nodes represent lexemes with similar distributional usage characteristics, edges represent similarity relations among them, which may be paradigmatic (e.g., to bark vs. to growl) or syntagmatic (e.g., to bark vs. cat). Since all nodes are by definition connected to the target, the target itself is excluded from the network.

3.4 Network distance as a measure of contextual variability and change

Change in contextual usage is captured by the quantified difference between two networks, each representing the semantics of the target word at a given point in time. The information about which nodes in a network are connected by an edge can be expressed in the form of a data matrix (Figure 2). We then determine how dissimilar two networks are by using the Frobenius norm, which defines a distance metric quantifying the difference between two network matrices in Euclidean space (Horn and Johnson 2012).

Figure 2:

Distance between two adjacency matrices M1 and M2 of two networks in terms of the Frobenius norm. For a matrix M, the Frobenius norm is defined as ‖ A ‖ F = ∑ i = 1 m ∑ j = 1 n | a i j | 2 (effectively the Euclidean norm of the flattened matrix).

There are two ways in which change in contextual usage can be measured (Figure 3): (a) by measuring the distance between semantic representations of consecutive periods (t_i vs. t_i+1 ); or (b) by measuring the distance between each of the individual representations in the time series from a common point of origin (t ₀ vs. t _i ). The former has been employed by Hamilton et al. (2016). Strictly speaking, therefore, the results of that study imply that occurrence frequency is negatively correlated with semantic variability rather than semantic change in the traditional sense, as each of their data points represents one decade-long time interval and not a time series-style progression of change from a fixed starting point. In the present study, both perspectives are employed to address slightly different aspects of the relationship between contextual-semantic change and frequency of occurrence.

Figure 3:

Two ways of measuring change in contextual usage: (a) Distances between pairs of networks in consecutive years measure the degree of contextual variability. (b) Pairwise distances between any network and the network in the first period measure the extent to which a word shifts away from its original usage at t ₀.

The item transparent serves to illustrate the amount of change that can be expected from our data (Figure 4). Like its English counterpart, the adjective has two metaphorically related meanings in German, viz. ‘clear, see-through’ (as in transparent glass) and ‘obvious, understandable, open to public scrutiny’ (as in transparent hiring practices). The network from the beginning of the period distinguishes clearly between those two meanings. One usage cluster relates to the context of architectural or possibly textile design (Farbe ‘color’, optisch ‘visual(ly)’, hell ‘bright’, elegant ‘elegant’), while the other, although somewhat vague, seems to capture the context of public or managerial processes (Anforderung ‘requirement’, effizient ‘efficient’, Struktur ‘structure’).

Figure 4:

Development of the German adjective transparent (‘transparent’). Mid-upper panel: change in contextual usage (measured as distance from common origin, i.e., network in the first month) depending on time measured in months. The increasing line indicates that transparent shifts away from its original usage. Left (petrol) and right (magenta) panel: example networks and contexts for an early (02/1999) and a late period (12/2018), respectively. Mid-lower panel: the adjective transparent has increased in frequency (left) and network size (right).

In contrast, the network from the end of the period distinguishes between the context of digital technology and consumerism (Anwendung ‘application’, Verbesserung ‘improvement’, Konsument ‘consumer’, innovativ ‘innovative’ etc.) and the public/legal/regulatory sphere (gesetzlich ‘legal’, Regel ‘rule, regulation’, prüfen ‘probe, inspect’, Richtlinie ‘guideline’, etc.). Crucially, in 2018 the literal meaning seems to have become marginalized by the densely represented metaphorical use, to the extent that it is not recognized by the underlying embedding algorithm anymore. This may well reflect an ongoing semantic shift away from the literal to the metaphorical meaning in Austrian German usage.^[5]

That the two example networks are representative of a larger trend for the item transparent can be seen in the upper middle panel. Here, change in usage is measured in terms of how distant each network in the time series is from the very first network of the investigation period, as described under (b) above. Thus, each data point stands for the semantic distance between the network representing a given month and the network at t ₀, i.e., January 1998. It emerges that network distance as a measure of change in contextual usage increases over time.

Following the literature and assuming that the observed subtle alterations in usage represent the incipient stages of more drastic semantic change, we would expect this to coincide with low frequency of occurrence. However, the relationship is complicated by the fact that the target word’s frequency does not remain constant, but is itself undergoing a marked upward shift. In other words, changes in contextual usage and frequency coincide in this case.

The same is true of network size, i.e., the number of nodes populating the network, as is reflected in the noticeably larger network from 2018. Although it would be a gross simplification to equate network size with the range of meanings linked to an item, it may nevertheless be regarded as a proxy for usage flexibility, in the sense that a higher number of nodes suggests a higher number of items which the target is related to through usage. To some degree, network size may also be determined by the embedding algorithm itself, so it is worth treating this feature with caution. Regardless of its interpretation, network size needs to be controlled for to get a clear picture of the effects of frequency on contextual change. In the case of transparent, all three variables seem to be positively correlated. This would clearly conflict with earlier findings should it turn out to be a general trend throughout our data set.

3.5 Description of variables and modeling procedure

In order to disentangle the relationships among these variables, we proceed as follows: First, we relate the mean contextual variability of our target items, measured as the network distance between consecutive months averaged across the investigation period (cf. Figure 3a above), to mean frequency, on the one hand, and mean network size, on the other (both averaged across the period). part of speech is also taken into account as a predictor. All three continuous variables are heavily skewed towards the lower end of their respective ranges. For mean frequency this was to be expected, keeping in mind the uneven Zipfian distribution of lexical items across the frequency spectrum (Zipf 1935), despite our efforts to stratify the sample with respect to frequency (cf. Section 3.2). Similar to utterance frequency, corpus size and network size are positively correlated (cf. Supplementary Material A.3 for a detailed discussion of corpus size and frequency effects on network similarity).^[6] Hence, we normalized all network-based measures, i.e., network size and distances between networks, with respect to corpus size (per month, employing a multiplicative constant of 10⁷ to avoid numerical issues). To improve the model’s quality, mean contextual variability, mean network size and mean frequency were transformed logarithmically. The two continuous predictors mean network size and mean frequency are positively correlated. In order to avoid problems of multicollinearity, this needs to be monitored closely for the regression model.

In a second step, we consider change in usage not as variability between consecutive period intervals, but as movement away from the semantic representation at point t ₀ (cf. Figure 3b above). We do this by calculating for every month the distance of the targets’ networks from the very first month of the investigation period, resulting in a time series of distances from the first month (again normalized with respect to corpus size). Then, we fit a generalized additive model (Wood 2006) to each time series, in order to obtain a smoothly curved distance trajectory. After that, we compute the area under this curve and define this area as contextual change. The rationale behind this operationalization is this: A word with contextual semantics moving away from the original meaning will show a strongly increasing distance trajectory (e.g., transparent in Figure 5). In contrast, a word with relatively stable contextual semantics throughout the observation period like the adjective fraglich (‘questionable’) is expected to show a relatively flat trajectory not far from the first month (i.e., close to a distance of zero). However, words could also first move away from their original contextual meaning, but, after some time, return again. This is illustrated by the word fett (‘fat, adipose, bold’) in Figure 5 (besides ‘adipose’, the adjective fett can also mean ‘drunk’, ‘great’, and can be used similarly to the affirmative expression awesome in English; apparently, some of these uses were more prominent one decade ago than they are now). Obviously, fett has changed to a larger extent than fraglich, although both end up close to their initial contextualized meaning. Simply computing the distance between the first and the final month would hide dynamics like this, as would the computation of the slope (or some other measure of average increase) of the trajectory. The area under the trajectory, however, does capture such subtle differences between the respective dynamics.^[7]

Figure 5:

Trajectories of contextual deviation from the first month for three different words: transparent (‘transparent’), fett (‘fat’), and fraglich (‘questionable’). Depending on the semantic dynamics, trajectories show differential areas below them.

In addition, we also account for the changeable character of frequency as well as network size by computing Spearman correlations for their diachronic trajectories, defining this as frequency change and network size change. In short, we model the extent to which changes in contextual usage can be explained by changes in occurrence frequency while controlling for changes in network size. The variable part of speech is kept unchanged.

We opt for linear regression as the method of statistical analysis and compute two models. In both models, each data point represents one word. In the first model, mean contextual variability depends on mean frequency, mean network size and part of speech. In addition, since we were mostly interested in frequency effects on semantic dynamics, we implemented multiplicative interactions between mean frequency and mean network size, as well as between mean frequency and part of speech as control terms. We also weighted mean contextual variability by the reciprocal of its margin of error (based on the 95 % confidence intervals of the estimated means) in order to account for the fact that some trajectories of pairwise differences fluctuate to a larger extent than others (which immediately affects the accuracy of the mean estimates). All predictors are implemented as fixed effects.^[8]

The second model features contextual change as outcome variable depending on the predictors (fixed effects) frequency change, network size change and part of speech. In addition, we include interactions between frequency change and network size change, as well as between frequency change and part of speech as control terms (Table 1).

To discriminate among the range of models that are possible given our predictor sets and interaction terms, we employ the Akaike information criterion (AIC) (Burnham et al. 2011; Johnson and Omland 2004). AIC is preferred over plain goodness-of-fit measures (such as R ²) as it penalizes models for their complexity, thus striking a balance between complexity and descriptiveness. Additionally, we test for multicollinearity using the variance inflation factor (VIF), a measure quantifying the extent to which a regression term is affected by multicollinearity (Grueber et al. 2011). All statistical operations are carried out with R (R Core Team 2023, version 4.2.2).

4.1 Mean contextual variability

The first set of results concerns the extent to which mean contextual variability of the target words is explained by mean frequency, mean network size and part of speech.^[9] The best-performing model (as determined by AIC and VIF analyses) is summarized in Table 2 and Figure 6.

Figure 6:

Effects on mean contextual variability. Each data point represents one target word. Upper panel: mean frequency has a negative impact on mean contextual variability (left). Verbs show lower mean contextual variability (right). Lower panel: mean network size has a positive effect on mean contextual variability (left). Interaction of mean network size and mean frequency (right). Medium and large networks display a negative effect of frequency on contextual variability. In small networks, frequency correlates positively with contextual variability.

The model coefficients confirm our expectations. When network size is controlled for, contextual variability is inversely dependent on frequency, i.e., items that are more frequent tend to show less contextual variability between months than less frequent items. Put differently, the more frequent the item, the more stable its usage context. Although frequency is significant as a main effect, its effect size is very small. It has to be kept in mind that both the predictor and the response variables have been transformed in preprocessing, so the actual relationship between the two is not linear. The coefficients and effect sizes need to be viewed in this light.

In contrast, network size has a strong positive effect on contextual variability and is by far the major contributor to the general predictiveness of the model. As a main effect, this relationship is rather difficult to interpret, due to the fact that network size itself factors into the computation of both the networks and the network distances, which is why it needed to be added as a control variable to begin with.^[10] Importantly, network size cannot be regarded as a straightforward measure of a target’s semantic scope. At the same time, it cannot be ruled out that semantic scope may still be reflected by network size to some extent. In this regard, inspecting the interaction between network size and frequency, rather than the main effect, may be more informative, since it allows us to partition the range of network sizes into smaller slices and observe the effect of frequency in each slice individually. This suggests that the negative effect of frequency is not constant across words with different average network sizes, but that it is stronger in words with larger networks.

Three tentative conclusions can be drawn from this part of the analysis: (1) in line with our expectations, words that occur more often have more stable usage contexts; (2) verbs display slightly lower contextual variability than nouns and adjectives; (3) the effect of frequency is somehow moderated by the size of the networks.

4.2 Contextual change

Up to this point, the measures for contextual variability, frequency and network size have been treated as averages across the whole period. This does not only entail a considerable amount of fuzziness in the data, but leaves the crucial question of larger diachronic change, in contrast to monthwise variability, unaddressed. In the second part, therefore, we consider contextual change as a function of frequency change while controlling for network size change and part of speech. We do this to do justice to the fact that frequency and network size are themselves changeable (cf. Section 3.5).

The best-performing model (as determined by AIC and VIF analyses) is summarized in Table 3 and Figure 7.

Figure 7:

Effects on contextual change. Top panel: frequency change (rate of growth) has a negative impact on contextual change. Middle panel: network size change (rate of growth) and contextual change are positively correlated. The effect of frequency change on contextual change is affected considerably by part of speech and reversed for expanding networks. Bottom panel: effect of part of speech on contextual change: nouns display slightly higher rates of contextual change than adjectives and verbs show lower contextual change than adjectives. The negative effect of frequency change on contextual change is not affected by part of speech.

With regard to the main effects, the results from this analysis are similar to what we have seen in the previous section. frequency change is negatively related to contextual change. In other words, items that have become less frequent over the investigation period tend to display more change in their usage contexts than items that have become more frequent. Once again, the effect size is small, but the effect is statistically significant. network size change is positively related to contextual change. part of speech also emerges as a significant factor in this model. In particular, nouns seem to have slightly higher rates of change than verbs (with adjectives placed in-between).

More intriguing than the individual factors’ main effects are the interactions among the variables, in particular the interaction between frequency change and network size change. As can be seen in Figure 7, the negative effect of frequency change on contextual change is particularly marked in items whose networks have become smaller over time (i.e., words with contracting networks). In such networks, an increase in frequency leads to more stable usage contexts, while decreasing frequency entails changing usage contexts. These effects match the expectations derived from the literature. Interestingly, however, the effect is reversed in items whose networks become larger over time. That is, in expanding networks, growth in frequency leads to larger amounts of contextual change.

The intricate relationship between frequency and network size in this model points to the possibility that network size might encode more than a computational confound after all. If it was nothing else than an artifact introduced by the method, we would not expect it to moderate the effect of frequency in the way that we have observed. On the contrary, the fact that change in frequency profiles seems to have different effects in expanding or contracting networks suggests a more ambivalent role of occurrence frequency when it comes to stabilizing the semantics of a word.