Accounting for the relationship between lexical prevalence and acquisition with Bayesian networks and population dynamics

1 Introduction

Lexical dispersion – that is, the extent to which words are used within a population of speakers and distributed in linguistic usage – is evidently connected with lexical acquisition. After all, words need to be acquired first by either children or adults before they are used. Conversely, it is evident that words which are used frequently and in many different contexts (like food or walk) are also acquired earlier than comparably rare lexical items (like sonogram or hyrax; e.g., Tomasello 2003; Kuperman et al. 2012).

Lexical dispersion can be measured in various different ways, such as through usage frequency (Bybee 2007); in terms of the number of texts, contexts, or utterances a word surfaces in (i.e., lexical dispersion in the narrow sense; Gries 2016); or by determining the area in which a word is established or the variety of social strata in which a word is known (Trudgill 2001; Mufwene 2001). More recently, Keuleers et al. (2015) highlighted the concept of “word prevalence”, that is, the percentage of people that know a word; a concept also referred to as “abundance” earlier by Nowak (2000) in the context of population dynamic models of lexical spread. It is expected that all these variables are negatively related to age of acquisition (i.e., they promote or are promoted by early acquisition). The question is, which of these variables affects or is affected most strongly by acquisition?

In this paper, we compare different measures of lexical dispersion and empirically assess the extent to which each of these measures relates to lexical acquisition. We show that lexical prevalence is most closely related to the acquisition of words and argue that this close relationship is to be expected given (a) cognitive mechanisms in language acquisition and usage and (b) that lexical proliferation represents a population dynamic spreading phenomenon.

Why is this relevant? Lexical dispersion measures differ as to how they are distributed. As is well known, token frequency typically follows a Zipfian distribution. A few words are highly frequent, while the majority of words displays a relatively low token frequency (Zipf 1949; Piantadosi 2014; Corominas-Murtra and Solé 2010). Because of this, the relationship between frequency rank and token frequency follows an inverse power law (with an exponent of about −1, depending on the actual language). This relationship is displayed in Figure 1 (dark orange line). Lexical prevalence, in contrast, follows a substantially different distribution. Here, most words are known by (almost) everyone while relatively few words are known only by a smaller share of the speaker population. When plotting prevalence against (logged) frequency rank (Figure 1, light orange line), the relationship is concave, rather than convex as is the case for Zipf’s law, and almost linear up to a knee at about 10,000. That is, the two dispersion measures seem to contain different amounts of information about lexical core and periphery. This can be clearly seen in Figure 1. While frequency accounts for much of the variability in the core lexicon (maybe up to 5,000 words), prevalence can be exploited to differentiate between lexical periphery items (about rank 5,000–10,000 and upward).

Figure 1:

Normalized token frequency (dark; diamonds) and prevalence (light; circles) depending on frequency rank, for about 30,000 words (cf. the description of Data Set 2 below). In the case of token frequency, the relationship is convex and follows Zipf’s frequency law (dark orange line), i.e., an inverse power law with an exponent of about −0.73. In contrast, the relationship between rank and prevalence is concave and relatively flat in the core-lexicon regime, a result formally derived by Nowak (2000: Figure 3 and Section 4.2).

The question remains as to whether these distributional differences influence how strongly the respective dispersion measures (in this case, frequency and prevalence) relate to age of acquisition. Indeed, Brysbaert et al. (2016) reveal a processing advantage of words with high prevalence, which is robust even if additional variables like word frequency or phonological properties are considered. Thus, prevalence seems to provide information about linguistic processing that is not already contained in token frequency. By consequence, we argue that if prevalence and age of acquisition turn out to be closely linked, lexical prevalence should not be neglected as relevant factor in studies of lexical acquisition. In this contribution, we provide arguments for this link by means of empirical data analysis and theoretical considerations.

In Section 2, we describe the data sets that we used and combined for the purpose of our analysis, as well as the crowdsourcing study that we conducted to gather information in addition to that provided by extant resources. More specifically, we collected estimates for passive as well as active prevalence (i.e., the difference between having only heard or also actively used a given word), detailed geographic information, and social information. In Section 3, we present our data analysis involving statistical modeling and Bayesian networks to learn about dependencies among dispersion measures. This is complemented with a follow-up analysis of the functional relationship between age of acquisition and prevalence based on a larger data set (Brysbaert et al. 2019). In Section 4, we first discuss cognitive factors involved in the relationship between prevalence and acquisition. Second, we relate our empirical findings to population dynamic models of linguistic spread. We conclude our paper in Section 5.

2 Data

Our empirical analysis is partially based on existing data resources and partially on data that we collected for this study. Age-of-acquisition (aoa) ratings were taken from a data set compiled by Kuperman et al. (2012). It consists of subjective age-of-acquisition estimates for about 30,000 words collected through crowdsourcing efforts. These estimates were shown to correlate robustly with age-of-acquisition norms obtained under laboratory conditions (Kuperman et al. 2012). Word frequencies (log freq; normalized) were also taken from this data set and subsequently log-transformed.

The data set compiled by Kuperman et al. (2012) also contains information about the fraction of participants who know a lexical item (“fraction known”), which already provides an estimate of lexical prevalence. However, estimates in that study are only based on roughly 20 participants per word. Brysbaert et al. (2019) collected prevalence data for more than 60,000 words with high accuracy (more than 300 data points per word); however, we wanted to add socio-geographic information, and to differentiate between passive and active knowledge. Thus, we extracted a stratified sample of 500 words from the list of Kuperman et al. (2012) in such a way that 50 words were randomly chosen for every decile of “fraction known”, thus covering rare as well as commonly known items. In addition, we sampled 25 high-frequency core items that we assumed to be known by all participants (e.g., rabbit). Together with a list of 25 nonce words,^[1] these core lexemes were added to the 500 words as control items. The distributions of age of acquisition in the data compiled by Kuperman et al. (2012) and that of this study are shown in Figure 2.

Figure 2:

Densities of age of acquisition (aoa) in the data collected by Kuperman et al. (2012; roughly 30,000 words; cf. Data Set 2 below; light orange) and in this study (500 words; cf. Data Set 1 below; dark orange). Mean aoa is µ = 11 and µ = 13.9, respectively. The present study largely covers relatively late acquired items.

Subsequently, we conducted a crowdsourcing survey to collect relevant information for each word. Participants were asked to decide whether every single word has been (1) heard but not used, (2) heard and used already, or (3) neither used nor heard at all. This enables a more differentiated assessment of lexical prevalence. Each participant had to rate all 550 items in this way. In addition, we collected geographic information (central coordinates of their hometown, i.e., longitude and latitude), and educational status (10 different options).

In total, we collected data from 100 participants. All data were collected with Prolific Academic (https://www.prolific.co/) and SoSci Survey (https://www.soscisurvey.de/), filtering for first language (English), nationality, and country of residence beforehand, and randomizing items. Participants received GBP 3.09 for their efforts. All answers were given anonymously, and no personal data was collected in the survey. After excluding unreliable responses based on the control items (threshold: at least 80% correct) and including only answers from Great Britain, 88 ratings per word remained in our data. This geographic limitation was employed to facilitate measuring spatial dispersion (see description of the variable area below).

Based on these data, we computed the fraction of speakers that have already heard (prev heard) and used (prev used) a word, giving us two different measures of lexical prevalence, corresponding to passive and active lexical knowledge, respectively. Given 88 ratings per word, the margin of error of these estimates is at most ±5%. This roughly corresponds to the accuracy for British participants in Brysbaert et al. (2019) and essentially doubles accuracy compared with the data of Kuperman et al. (2012). Note that lexical prevalence was assessed subjectively, very much like age-of-acquisition estimates in Kuperman et al. (2012) and scores based on vocabulary tests in Brysbaert et al. (2019).^[2]

We derived a proxy for the area in which the word is known (area) by means of the rectangle defined by the most extreme coordinates for each word. Thus, this proxy is monotonously related to the maximum geographic distance between speakers that know a word. In order to obtain a proxy for social dispersion (social disp), we computed entropy of educational status per word. Thus, high entropy relates to usage in diverse social strata while low entropy indicates a socially restricted usage (e.g., academic language).

Dispersion across texts (range) was taken from the Corpus of Contemporary American English (COCA, Davies 2008; SOAP sub-corpus; see Appendix A1). This score measures the number of texts in which a word is used (it was shown to be immediately related to lexical processing; cf. Gries 2008). Due to its skewed distribution, this variable was log-transformed as well (after uniformly adding 1 to all counts to avoid negative infinity values). Finally, dispersion across genres (genre disp) was determined by extracting token frequency distributions across all genres in the British National Corpus (BNC) for each word, and subsequently computing entropy for each distribution. Again, high entropy indicates usage in diverse genres whereas low entropy indicates a restriction to few genres (or just one). In case of a word not surfacing in BNC, entropy was set to zero.^[3]

After eliminating the control items, this leaves our data set at 500 instances with estimates for eight properties each (Data Set 1; see supplementary materials). Table 1 shows a sample of 10 entries in that data set. Table 2 shows summary measures for all variables.

For a follow-up analysis of the relationship between age of acquisition and prevalence based on a larger number of words, we combined the data sets from Kuperman et al. (2012) and Brysbaert et al. (2019). This resulted in a set of age-of-acquisition norms and prevalence estimates (again, not probit-transformed) for 30,794 words (Data Set 2).

3 Empirical data analysis

The statistical analysis unfolds in three steps. First, we computed straightforward linear regression models, one for each dispersion measure (as dependent variable) in Data Set 1, in which aoa figures as predictor. All variables were z-transformed to obtain standardized regression coefficients. Results are shown in Figure 3b. It can be seen, first, that as expected lexical dispersion is inversely correlated with aoa in all cases. More importantly, it is both measures of prevalence which are most strongly correlated with acquisition (effect of ß = −0.42 and −0.4 for prev heard and prev used, respectively).

Figure 3:

(a) Bayesian network of aoa (dark orange) and all lexical dispersion measures (light orange). Edge weights correspond to absolute Gaussian regression coefficients. (b) Separate linear models of lexical dispersion depending on aoa together with standardized regression coefficients.

In the second step of the statistical analysis, we were interested in identifying dependencies between all dispersion measures and acquisition. To this end, we fitted a Bayesian network to our data (Pearl 2004). Bayesian networks are probabilistic acyclic graph models which have been developed to identify conditional relationships between variables (although it is debated if this is linked to the concept of causality). In these graphs, variables figure as nodes. A pair of nodes is linked through a directed edge if one node conditionally depends on the other. For example, an edge aoa → prev heard means that passive prevalence conditionally depends on age of acquisition. In the continuous case, under the assumption of normally distributed variables, the distribution of each node is given by a normal distribution determined by the nodes pointing to it. That is, the computation of a Bayesian network boils down to fitting linear multivariate Gaussian models for sets of nodes, one of them being the dependent variable in that model, and selecting models based on some learning algorithm (here, greedy hill-climbing; Scutari and Denis 2014). That is, in our analysis both the structure of the network and the weights on its edges are derived automatically from the data and not determined a priori. The Bayesian network fitted to our data is visualized in Figure 3a, where the weights of the edges correspond to absolute regression coefficients in the respective Gaussian models. Coefficients with signs are shown in Table A2.

In Figure 3a, prev heard is most closely linked with acquisition, followed by genre disp and social disp. All other dispersion measures are only indirectly connected to aoa. In particular, the often-investigated measure of frequency is least closely correlated with acquisition. This indicates that acquisition is more strongly connected with – and likely also driven by – social interactions rather than simply frequency of exposure.^[4]

The variable genre disp is interesting in that it features a weak relationship with aoa in the linear regression analysis; in the Bayesian network, however, genre disp seems to directly depend on many variables, one of them being aoa. A closer look at the sign of the coefficient (of the impact of aoa on genre disp) in Table A2 reveals that, if log range, log freq, and prev heard are also considered, then genre disp in fact increases with aoa. That is, compared to the univariate linear model (in Figure 3b), the sign of the relationship is reversed in the multivariate model. Dispersion across genres seems to interact nontrivially with acquisition, albeit weakly so.

In a third step, we assessed the shape of the functional relationship between age of acquisition and prevalence based on Data Set 2, that is, the data set of 30,794 words. To do so, we fitted a generalized additive model (GAM, Wood 2006) to our data in which age of acquisition and prevalence figure as dependent and independent variable, respectively. Crucially, prevalence entered the model as a smooth term. The initial number of knots (k = 30) in this term was determined through an AIC-based grid search over k (Burnham and Anderson 2002; COCA, Davies 2008; see Figure A1). The resulting model is visualized in Figure 4a.

Figure 4:

(a) GAM of age of acquisition depending on prevalence based on Data Set 2 (gray area indicates a 95% confidence band; adjusted R ² = 38.4). The functional relationship is roughly linear up to a knee close to p = 1.0. A small set of lexical examples was sampled randomly for each decile. (b) Illustration of the theoretical approach described in Section 4. Left: Schematic representation of the relationship between a word’s age of acquisition and prevalence at equilibrium under the assumption of a rectangular age distribution. Right, first row: Logistic model of word prevalence in a speaker population. Right, second row: Age of acquisition is linearly related to equilibrium prevalence.

4 Discussion and theoretical considerations

Our analysis corroborates the close connection between acquisition and lexical prevalence, that is, the dispersion of words in speaker populations. This relationship could go in two directions: on the one hand, early acquisition could be a reflex of high lexical prevalence; on the other hand, early lexical acquisition might drive lexical prevalence.

Let us discuss the former direction first. To begin with, we can argue that acquisition is promoted by a high abundance of linguistic informants. A word that is known (and subsequently also used) by many speakers has a higher chance to be picked up by someone who does not yet know that word than a word known only by a small set of individuals.

In addition to this essentially probabilistic argument, which we will come back to below, cognitive factors and generalization across situations and communication events are likely to play a role (Kemler Nelson et al. 2000). For example, Schwartz and Terrell (1983) show that in infants, distributed exposure promotes lexical acquisition more strongly than massed exposure (i.e., many instances at the same time). The same was found by Childers and Tomasello (2002), who show that exposure across several events facilitates word learning over single massed exposure even if overall frequency is kept constant. Since the present study focuses on the acquisition of words that are acquired relatively late (cf. Figure 2), insights drawn from the study of early lexical acquisition (Goldfield and Reznick 1990) must be treated with caution in this context. However, the promoting effect of repetition in language learning is also visible more generally in adolescents and adults in various linguistic domains (Dempster 1996). Arguably, lexical prevalence, social dispersion, or areal dispersion are more closely related to distributed exposure than to frequency, as operationalized in our analysis. In particular, high prevalence implies that a word can be used to communicate with many different speakers rather than being employed idiosyncratically.

Another possibility is that our results may also be a reflex of word semantics. There is robust evidence that polysemous words are more frequent (Casas et al. 2019) and processed faster than words with only a single meaning (at least if they are abstract; Jager and Cleland 2016). If polysemy reflects dispersion across different contexts and if contextual versatility is correlated with the number of interlocutors, this might explain the relationship between lexical prevalence and acquisition, at least partially. However, when looking at the roles of range and dispersion across genres in our analysis (Figure 3), the situation becomes less clear. Both variables may relate to polysemy, but range is only weakly related to age of acquisition and high genre dispersion even seems to be associated with late acquisition if other covariates are also considered.^[5] Potentially, this mismatch can be explained by semantic bleaching effects, which are typically associated with late acquisition and impeded learning (Brown 1973; De Groot and Keijzer 2000; Hopper and Traugott 2003). We conclude that the role of semantics in the relationship between acquisition and prevalence needs to be investigated in further detail.

Let us now consider the other direction, that is, prevalence being a reflex of acquisition. One argument is that words which are acquired early are highly entrenched, which has an immediate effect on the chances of stably using (and not forgetting) those words. This might be grounded in increased cognitive plasticity at early ages (cf. Monaghan 2014) or cumulative frequency (Ghyselinck et al. 2004). Arguably, if an individual acquires a word earlier, they have more time to produce that word which increases the chances of passing it on to other speakers.

This probabilistic argument is closely related to population dynamic accounts of language. Linguistic constituents such as words are transmitted through speaker populations and across generations (Croft 2000; Ritt 2004), so that the proliferation of words represents a spreading phenomenon (Barabási 2016). Consequently, the spread of linguistic constituents has been modeled in terms of population dynamics (Cavalli-Sforza and Feldman 1981; Nowak 2000; Solé et al. 2010; cf. Figure 4b, right). In these models, linguistic spread is always a function of population structure and the number of interactions among individuals. In particular, we have shown elsewhere in terms of a simple logistic model of learner-user interactions that age of acquisition and prevalence (next to diachronic growth) are both related to the reproduction rate of linguistic constituents (Baumann and Ritt 2018).

From this model, it can be derived that age of acquisition and prevalence are linearly related to each other (Figure 4b, right). The argument is straightforward and analogous with considerations about age of first infection in epidemiology (Dietz 1993; Heffernan et al. 2005). Let us assume that a particular word is close to its population dynamic equilibrium (i.e., its prevalence does not grow or decline strongly). Let us furthermore assume a roughly rectangular (i.e., equal) age distribution. Then the ratio of people not knowing the word and the total population size must be the same as the ratio of age of acquisition and life expectation. This is illustrated in Figure 4b (left). From this it follows that age of acquisition is a linearly decreasing function of prevalence. Thus, the close relationship between acquisition and lexical prevalence that we see in our data can be interpreted as a reflex of straightforward population dynamics of the interactions among users and learners. This mathematical relationship does not hold well for words that are known by almost everyone, but it serves as a good model for the relationship between acquisition and prevalence in relatively rare words. That this holds true also empirically can be seen in Figure 4a, which shows the generalized additive model fitted to the data collected by Kuperman et al. (2012) and Brysbaert et al. (2019). Up to a knee close to a prevalence of 100%, the relationship between age of acquisition and prevalence seems to be fairly linear.

The directionality of the Bayesian network in Figure 3a hints at effects of acquisition on prevalence (and in turn other dispersion measures). However, it is important to point out that Bayesian networks are models of joint probability distributions defined by conditional probabilities among variables. This does not necessarily entail causal relationships among them (see Pearl 2004 for some discussion). Nevertheless, the network in Figure 3a does reveal plausible chains of dependencies, such as aoa → prev heard → prev used → log freq.

It is evident that our study has many shortcomings. The number of words analyzed (500) is low, in particular in comparison with the large-scale studies by Brysbaert and colleagues (Brysbaert et al. 2019; Kuperman et al. 2012). Also, the focus of our study is on analyzing relatively late-acquired periphery items. Figures 2 and 4a show that words which are not known by all speakers are acquired roughly at the age of 12 or later. For that reason, we can say little about how prevalence and age of acquisition hang together in early stages of lexical acquisition.

Furthermore, our proxies for spatial and social dispersion are rather simplistic. With more participants per word, one could define more accurate measures of geographic dispersion (e.g., by also taking areas of regions and more fine-grained diversity measures into account) rather than just assessing the most extreme area implied by the data. Likewise, having social dispersion just based on educational status is simplistic as well. Information about income class, for example, strikes us as a reasonable extension. Regarding social dispersion, another issue arises in our study. Since we rely on a relatively small set of words, chances are high that words with low prevalence are those that are related to scientific or academic contexts so that entropy of social state of a word (as computed here) is in fact correlated with the average social state of individuals knowing that word. We suggest that future studies with limited sample sizes should feature stratified sampling of words with respect to speech registers as well to account for such a bias (e.g., by including slang words like shyster).

Finally, the population dynamic model discussed above is based on simplistic assumptions. It assumes homogeneous mixing rather than a more plausible social network structure (Barabási 2016; Blythe and Croft 2012). In particular, Barabási (2016) shows that the reproduction rate of spreading phenomena is crucially influenced by social network structure in that the threshold for successful spread is decreased, which may in turn influence the relationship between age of acquisition and prevalence (Heffernan et al. 2005; Pastor-Satorras and Vespignani 2001). Since the acquisition of lexical periphery largely takes place during adolescence (cf. Figure 2), the setup and change of social networks in this age group should be considered as well. First, there is robust evidence that social networks grow most rapidly between the ages of 10 and 20 (Wrzus et al. 2013). Thus, ideally, network size should be controlled for. Second, and directly related to the latter consideration, there is evidence that adolescents tend to adapt to older peers, which violates the assumption of homogeneous mixing (Kerswill 1996). Moreover, the model is based on age-independent learning rate and it does not account for interactions among linguistic constituents (e.g., via phonological or semantic relatedness).^[6]

5 Conclusion

In this paper, we have analyzed various measures of lexical dispersion based on different data sources. We have shown that lexical prevalence seems to be relatively closely linked to the age of acquisition of words. We have argued that this relationship may be partially accounted for by cognitive mechanisms related to word learning, but that it may also be construed probabilistically. Here, we have argued that, if the proliferation of words is modeled as a spreading phenomenon, then, under certain assumptions, the close relationship between age of acquisition and prevalence is a logical consequence of such a model. Despite the limitations of our study laid out in the previous section, we consider our findings interesting enough to motivate further research in this direction. In particular, we would like to highlight the relevance of collecting and analyzing word-prevalence data next to word-frequency measures. This is in line with Brysbaert et al. (2016), who stress the relevance of prevalence for psycholinguistic research. Our research, however, emphasizes this matter with a different point of view in that we argue prevalence to be also relevant to research on linguistic dynamics, which is often restricted to studying diachronic frequency developments, as far as empirical analysis is concerned. Finally, we stress that studying lexical periphery in addition to core lexical items is important to get a more complete picture of lexical acquisition, usage, and change.