The realization of tones in spontaneous spoken Taiwan Mandarin: a corpus-based survey and theory-driven computational modeling

2.1 Spoken duration and articulation

Evidence is accumulating that subtle differences in meaning can be reflected in the fine phonetic details of how words are actually realized in corpora of natural speech, including aspects such as spoken word (Gahl and Baayen 2024) duration, segment duration (Plag et al. 2017), and tongue position (Saito 2024).

Heterographic homophones are words with the same pronunciation but different spellings and meanings, such as time and thyme. For a long time, homophones were thought to sound identical (see, e.g., Jescheniak and Levelt 1994). However, Gahl (2008), using the Switchboard corpus (Godfrey et al. 1992), reported that heterographic homophones such as time and thyme tend to have different acoustic durations, with more frequent homophones (time) being pronounced with shorter durations than their less frequent homophonic counterparts (thyme). Lohmann (2018) similarly observed that the duration of words such as cut depends on whether they are used as nouns or verbs. Both studies explain these effects in terms of how frequency of use affects lexical access in speech production. However, Gahl and Baayen (2024) reported that computational modeling with the Discriminative Lexicon Model provided strong evidence that the meanings of English homophones (represented by embeddings) are strong codeterminants of their spoken word duration, even when word frequency is controlled for. They argued that a powerful predictor of a homophone’s spoken word duration is the degree of support it receives from the semantics, such that greater semantic support predicts longer spoken word duration.

In addition to durational differences at the word level, durational differences have also been observed at the phonemic level in corpus studies, particularly for the realization of word-final /s/ or /z/ (henceforth referred to as S) in English. In the Buckeye corpus (Pitt et al. 2005), word-final S has been found to vary in duration depending on its morphological function: nonmorphemic S is pronounced longer than plural S, which, in turn, is pronounced longer than clitic S (Plag et al. 2017; Tomaschek et al. 2021; Zimmermann et al. 2016). Furthermore, Plag et al. (2020) found that genitive plural S showed significantly longer durations than plural Schmitz (2022) utilized a pseudo-word paradigm to demonstrate that the morphological category of word-final S (nonmorphemic > plural > clitics) influences its phonetic realization.

The relationship between semantics and phonetic realization has also been demonstrated beyond durational differences. Drager (2011) found that the pronunciation of the English word like varies according to its discourse or grammatical meanings, not only in the duration of the consonants but also in the degree of diphthongization of the vowel. Furthermore, in line with Gahl and Baayen (2024), Saito (2024), Saito et al. (2024) reported for the KEC corpus of German spontaneous speech (Arnold and Tomaschek 2016), which also registers tongue movements using electromagnetic articulography, that greater semantic support leads to a lower position of the tongue tip for the vowel /a/, indicating hyperarticulation.

2.2 Tone in connected speech

The preceding section reviewed recent evidence that semantics and phonetic realization are entangled to a greater extent than has often been assumed. In this section, we zoom in on how word meaning affects the realization of tone. To do so, we first need to provide some further background on the factors that have already been reported to codetermine the realization of tone.

It is well known that the way in which tones are realized in connected speech differs from their canonical realization. How tones are realized has been described as depending on the properties of the segments in the syllable that carries a given tone (Ho 1976a; Ohala and Eukel 1976; Xu and Xu 2003). Tonal variation in connected speech has also been reported to be shaped by the tones of adjacent words (tonal coarticulation Xu 1997), by speaking rate (Xu and Sun 2002), by the interaction between lexical tones and intonation (Ho 1976a; Wu et al. 2020), and by a speaker’s individual speaking style (Stanford 2016).

At the socio-geographic level, cross-dialectal research has reported that different varieties of Mandarin exhibit varying tone inventories (Chang 2010; Zhao 2023). For the present study, we note that in Standard Chinese, the realization of a neutral tone following a given lexical tone has been reported to be largely determined by this preceding tone. Furthermore, the f0 contour of a neutral tone has been claimed to approach a low pitch target by the end of the carrying syllable (Xu 2024). However, in Taiwan Mandarin, the behavior of the neutral tone has been reported to be different. It can either be indistinguishable from one of the four canonical lexical tones, or it can be realized as a static mid-low pitch target (Huang 2018).

From the above overview, it will be clear that the realization of tone is codetermined by a multitude of different factors. A newcomer in this arena is word meaning. Chuang et al. (2025) studied the pitch contours of disyllabic words with an initial rising tone followed by a falling tone (henceforth, the T2-T4 tone pattern). This study used a Generalized Additive Model (GAMM, Wood 2017) to decompose an observed pitch contour into separate pitch contours, capturing the effects over time of predictors such as speech rate, neighboring tones, and segmental properties. What Chuang et al. (2025) report is that the GAMMs provide strong support for word-specific pitch contour components, while controlling for other variables such as segments, gender, speaker, speech rate, and the tones of adjacent words. They also show that the statistical evidence is even stronger for sense-specific pitch contours, which suggests that these effects are semantic in nature. The importance of words’ meanings has been replicated for Mandarin disyllabic words with T2-T3 and T3-T3 tone pattern by Lu et al. (2024), and for monosyllabic Mandarin words by Jin et al. (2025). For disyllabic words with T2-T3 and T3-T3 tone pattern, the variable importance of words’ meanings was on a par of that of tonal context (tone sandhi), the other most important predictor of words’ pitch contours.

To illustrate the challenges that an analysis of tones in natural speech has to face, consider Figure 1, which displays the f0 contours of a selection of tokens of word types with a falling tone followed by a rising tone (the T4-T2 tone pattern) in the Corpus of Spontaneous Taiwan Mandarin (Fon 2004). In the left panel, the pitch contours of six tokens from different word types are presented. All words have the same canonical tone pattern: T4-T2. Token XMC_GY_4119_不能 (bu4neng2, “cannot”) (indicated by light blue) shows an initial sharp f0 rise, followed by a fall, and then a shallow rise. Token XMC_GY_8107問題 (wen4ti2, “problem”) (indicated by purple) has a much lower initial f0 than the other tokens. The initial f0 of token XMC_GY_1025_後來 (hou4lai2, “later”) (indicated by red) is unavailable due to the unvoiced initial /h/.^[1] In the right panel of Figure 1, we present four tokens of the same word type 幸福 (xing4fu2, “happiness”). One of its tokens is also presented in the left panel (indicated by brown). The four tokens of 幸福 also exhibit considerable variability in their f0 contours.

Figure 1:

A selection of tokens in spoken Taiwan Mandarin. Left panel: six tokens representing six different word types, all sharing the tone pattern T4-T2 (a falling tone followed by a rising tone). The tokens are 後來 (hou4lai2, “later”), 幸福 (xing4fu2, “happiness”), 去年 (qu4nian2, “last year”), 不能 (bu4neng2, “cannot”), 自然 (zi4ran2, “nature”), 問題 (wen4ti2, “problem”). Right panel: four tokens representing the word type 幸福 (xing4fu2, “happiness”). All f0 contours shown here are produced by the one female speaker. The vertical lines indicate the normalized syllable boundary for each token. The durations (in seconds) are shown in parentheses after each token name in the legend.

In what follows, we take on the challenge of modeling the realization of tone, taking into account the many factors reported to codetermine pitch contours, such as gender, speaker, neighboring tones, speech rate, word position, and bigram probability. Following up on earlier work (Chuang et al. 2025; Jin et al. 2025; Lu et al. 2024), the “pitch signatures” of individual words are of primary interest.

Two hypotheses guide our research.

1. The meanings of words codetermine the phonetic details of how the tones of these words are produced.

2. The pitch contours of word tokens as found in spontaneous Mandarin conversations can be predicted from token-specific meaning vectors with above-chance accuracy using computational modeling with the DLM.

Importantly, end-to-end modeling with the DLM does not require abstract units for syllables and segments. The second hypothesis, therefore, makes the strong empirical claim that for predicting the details of the phonetic realization of f0, such abstract units are not required.

In the next section, we describe the data that we collected from the Corpus of Spontaneous Taiwan Mandarin and present our statistical analyses. Section 4 complements this exploratory part of our study with theory-driven computational modeling.

3.1 The corpus

The data used in the present study come from the Taiwan Mandarin Spontaneous Conversation Corpus (Fon 2004). This corpus contains 30 h of speech from 55 native speakers of Taiwan Mandarin, with 31 females and 24 males (aged between 20 and 60 years old). In unstructured interviews, participants were encouraged to speak freely, instead of being guided by a standardized set of questions. As a result, this corpus consists of naturally occurring speech with a diverse and varied set of words across speakers.

The corpus was transcribed in traditional Chinese characters at the word level. The speech data were segmented at both the syllable and word levels. Forced alignment was first performed, and the results were later manually reviewed by native Taiwan Mandarin speakers with a background in phonetics. In the current study, we followed the transcriptions and segmentation provided in the corpus.

3.2 Data selection

Disyllabic words with the 20 tone patterns were extracted for analysis (see column 1 in Table 1). The original dataset comprises 93,701 tokens, representing 7,526 unique word types. Table 1 presents the counts of tokens and word types associated with each tone pattern. Among these, the T4-T4 pattern is the most frequent in disyllabic words, both token-wise and type-wise. The four tone patterns featuring a neutral tone in the second syllable (T1-T0, T2-T0, T3-T0, and T4-T0) are represented by the lowest numbers of types. There are also relatively few words with T3 (see Wu et al. 2021: for similar observations for journalistic speech).

In our dataset, we took into account the tone sandhi for 不 (bu4) and 一 (yi1). Thus, T2 is assigned to 不 (bu4) when followed by T4, such as 不要 (bu2yao4, “no”). T2 is assigned to 一 (yi1) when followed by T4, such as 一定 (yi2ding4, “must”), and T4 is assigned to 一 (yi1) when followed by other tones (T1, T2, and T3), such as 一天 (yi4tian1, “one day”) and 一點 (yi4dian3, “a little bit”).

Subsequently, we extracted the sound files of these disyllabic words and measured their f0 values using the To Pitch (cc) command in Praat (Boersma and Weenink 2020). For female speakers, the pitch floor was set at 75 Hz and the pitch ceiling at 400 Hz. For male speakers, the pitch floor was set at 50 Hz and the pitch ceiling at 300 Hz. The time step was set to 0.001 s, and a Gaussian window was used for optimal F0 estimation. The To PointProcess command was then applied to identify the time points of glottal pulses in the voiced sections, from which the corresponding F0 values were extracted. No f0 values were returned when there was no vocal fold vibration due to the presence of voiceless plosives or fricatives, or when creaky voice occurred.

Words with fewer than five tokens were excluded from our dataset, to ensure that each word type had a sufficient number of tokens for analysis. For high-frequency words with more than 200 tokens, we randomly sampled 200 tokens, to prevent model predictions from being biased toward high-frequency words. Furthermore, words contributed by only female speakers or only by male speakers were excluded. This ensured that the tokens of a given word type were contributed by at least two speakers, preventing bias from one speaker’s specific way of speaking.

Lastly, tokens with f0 extraction errors were excluded from analysis. These errors typically resulted from pitch halving or doubling. We calculated, for each token, the standard deviation of the differences between consecutive measurements. A large standard deviation indicated high likelihood of discontinuous f0 measurements with abrupt fluctuations. Tokens with a standard deviation greater than the 9th decile of the distribution were considered outliers. This resulted in a dataset with 24,284 tokens, representing 524 unique word types.

3.3 Predictors

The response variable of interest is f0. We log-transformed f0 to obtain a response variable that approximately follows a Gaussian distribution. As our interest is in production rather than comprehension, we did not make use of modifications of the logarithmic transformation such as the MEL or BARK scales, which are optimized for human perception. The predictors for logf0 are as follows.

Of the above list of predictors, the factor tonal_context poses a special challenge for the analysis. tonal_context provides information about the preceding and following tones. Due to the pervasiveness of tonal coarticulation, it is highly probable that the effect of tonal_context varies with the tone pattern of the target word. For example, a preceding high tone will have an effect on a word-initial dipping tone that differs from its effect on a word-initial rising tone. Accounting for such coarticulation is essential for modeling f0 in connected speech. In principle, one could introduce a variable that represents the interaction between tonal_context and tone_pattern (cf. Jin et al. 2025). However, for our dataset, this would result in a variable with 720 levels that is strongly confounded with word and sense_type.

Therefore, we opted to fit separate regression models for f0 across the four most frequent tonal contexts in our dataset, excluding any contexts that involved a “pause” in the preceding or following tone, i.e., the contexts 4.4, 3.4, 4.1, and 4.0 (cf. Table 2). We chose not to include tonal contexts involving a “pause,” for two reasons. First, when a tone is preceded or followed by a pause, several context-related variables, such as norm_utt_pos, bg_prob_prev, and bg_prob_fol, are missing, leading to data loss. Second, pauses in speech often signal utterance boundaries, hesitations, or breaths, making the “pause” category inherently heterogeneous.

As shown in Table 2, the final dataset contains 4,283 tokens representing 313 unique word types. Across all tonal contexts combined, the minimum number of tokens per word type is five, and the maximum is 56. On average, each word type was produced by 9.37 different speakers (range: 2–30). Additionally, each speaker contributed an average of 53.31 different word types (range: 4–119). For each tonal context, all 20 tone patterns are represented.

3.4 Statistical analysis

In what follows, we carry out a word-based and a sense-based analysis. Following Chuang et al. (2025), we expect that including a factor smooth of word will substantially improve model fit, thereby extending the observed word-specific tonal realization to 20 tone patterns in spoken Taiwan Mandarin. Furthermore, replacing the factor smooth for word with a factor smooth for sense may lead to further improvement of the model fit. Given the complexity of GAMMs, a certain degree of concurvity is inherent and unavoidable. However, we avoid severe concurvity by specifying models that do not include both segmental predictors and word or sense smooths simultaneously.

3.4.1 Models with word as predictor

The Generalized Additive Model (GAMM, Wood 2017) was used for the statistical analyses, with the bam() function from mgcv package (Wood 2017) implemented in R (Team 2020). Four GAMMs were fitted to each of the four datasets, using the same model specification:

logf0	∼ gender +
	s(normalized_t, by=gender,k=4) +
	s(speaker, bs=“re”) +
	s(normalized_t, tone_pattern, bs=“fs,” m=1)+
	s(normalized_t, word, bs=“fs,” m=1) +
	s(duration, by=gender, k=4) +
	ti(normalized_t, duration, k=c(4,4)) +
	s(norm_utt_pos, k=4) +
	ti(normalized_t, norm_utt_pos, k=c(4,4)) +
	s(bg_prob_prev, k=4)+
	ti(normalized_t, bg_prob_prev, k=c(4,4))+
	s(bg_prob_fol, k=4)+
	ti(normalized_t, bg_prob_fol, k=c(4,4))

To account for differences in average pitch height between genders, we included gender as a fixed effect. We added a by-gender thin plate regression smooth of normalized_t, which allows us to capture differing relationships between normalized time and f0 across genders. Other continuous variables, including speech_rate, norm_utt_pos, bg_prob_prev, and bg_prob_fol, were likewise modeled with thin plate regression splines. Interactions of covariates with normalized time were modeled with tensor product smooths, using the ti() function.

Furthermore, random intercepts were requested for speaker to account for individual variability in pitch height by speaker. Other discrete variables, including tone_pattern and word, were modeled using factor smooths (nonlinear random effects).

We implemented an AR(1) process (first-order autoregressive model) in the residuals to take into account the autocorrelations in the time series of pitch measurements. The inclusion of the AR(1) process with an autocorrelation coefficient of rho = 0.95 effectively removed nearly all autocorrelation from the residuals. Summaries of the four models are provided in the Appendix.

Akaike’s Information Criterion (AIC) was used to assess variable importance. Figure 2 shows the increase in AIC (indicating a lower-quality fit to the data) resulting from withholding individual predictors from the model specification.^[2] A greater increase in AIC when a predictor is excluded suggests a higher importance of that predictor in the model. As shown in Figure 2, across all tonal contexts, withholding the predictor word leads to a substantial increase in AIC, ranging from 7,269.27 to 12,345.54. The increase in AIC when word is omitted from the model specification substantially exceeds the corresponding change observed for any other predictor.

Figure 2:

The increase in AIC scores when a predictor is withheld from the best-fit model. The AIC increase when word or tone_pattern is withheld is shown in red, and the increase for other predictors is shown in blue. Panels 1 to 4 represent four GAMMs with tonal contexts 4.4, 3.4, 4.1, and 4.0, respectively.

Surprisingly, withholding tone_pattern has a small impact on the model fit with increases in AIC of around 11.81 units (20.08 units for 4.4, 9.25 units for 3.4, 3.43 units for 4.1, and 14.5 units for 4.0). One possible explanation is that word is nested within tone_pattern. When word is removed from the best-fit GAMM, withholding tone_pattern results in a larger AIC increase by 7,223.17 units for 4.4, 4,139.11 units for 3.4, 2,973.78 units for 4.1, and 3,594.93 units for 4.0. This suggests that tone_pattern still contributes to the model fit, though not as strongly as word. When word is included in the model, the effect of tone_pattern is overshadowed by the stronger effect of word.

Concurvity, analogous to collinearity in linear regression, measures how much a predictor’s effect can be explained by other predictors in the model. Concurvity scores range from 0 to 1, with lower values indicating that the contribution of a predictor is less confounded with the contributions of other predictors. As shown in Figure 3, concurvity scores follow a similar pattern for all four GAMMs, with the lowest concurvity scores for word. speaker also has relatively low concurvity.^[3]

Figure 3:

Concurvity scores for selected terms in the four GAMMs. The concurvity scores for word and tone_pattern are shown in red, and those for other predictors are shown in blue. From left to right, it presents tonal context 3.4, 4.0, 4.1, and 4.4, respectively. Concurvity scores were calculated based on the best-fit GAMMs with all predictors included.

By contrast, the predictor tone_pattern exhibits extremely high concurvity, ranging from 0.998 to 1. This is due to tone pattern being fully predictable given the word. When word is excluded from the model, the concurvity of tone_pattern drops substantially to 0.09. This indicates that word captures information about the word’s tone pattern, so when word is included in the model, the tone pattern is also included implicitly. However, if only tone_pattern is specified, word-specific information is not available. This results in substantially worse fits, which aligns with the AIC changes discussed in preceding subsection.

Finally, we note that the by-gender smooths for time (normalized_t:female and normalized _t:male) show very high concurvity – unsurprisingly, as the tonal contours for both genders are highly similar (see Figure 4). Without accounting for other effects, these general contours primarily reflect the pure influence of time on pitch, illustrating how pitch contours change over time. The overall curves exhibit falling contours, which suggests a general declination trend in pitch contours for disyllabic words.

Figure 4:

The partial effect of general smooth for the normalized_time for female and male speakers, in different tone contexts. The orange curves indicate the general contours for female speakers, and the blue curves indicate the general contours for male speakers. Vertical gray dashed lines indicate the average syllable boundary, and the horizontal gray dashed line represents the y = 0 reference line.

Figure 5 illustrates the partial effects of the 20 tonal patterns across the four tonal contexts under investigation, using color coding to distinguish between the tonal contexts. Within each panel, the various colored curves represent specific tone patterns associated with different tonal contexts. For example, the orange curve in the upper-left panel represents the T4-T0 tone pattern in the 3.4 tonal context. In other words, it represents a tonal sequence T3-T4-T0-T4, as in the phrase 有這麼重 (you3zhe4me0zhong4, “…is this heavy”). The black curves were obtained from averaging the four colored curves representing tone patterns under different tonal contexts. The deviations of the colored curves from the corresponding black curves highlight how the actual realizations of a tonal pattern in context differ from the expected effect of tone pattern, irrespective of context.

Figure 5:

The effect of tone pattern. The colored curves represent the partial effects of the factor smooth for tone_pattern, combined with the general smooth of normalized_t for female speakers, based on the best-fit models that include the word effect. There is one GAMM for each tonal context, resulting in four colored curves representing, in a given panel, the four tonal contexts. The black curves present the mean f0 contours of a tone pattern, calculated by averaging the four f0 contours across the tonal contexts. Thus, the colored curves in each panel illustrate how the tonal context modulates the general curve shown in black. Vertical gray dashed lines indicate the average syllable boundary, and the horizontal gray dashed line represents the y = 0 reference line.

For most of the tone patterns, the effects of the neighboring tones on the pitch contour are relatively modest, with as glaring exceptions the T2-T0 tone patterns in the 4.0 tonal context. This tonal sequence T4-T2-T0-T0 (e.g., 對孩子的, dui4hai2zi0de0, “for children’s …”) shows an unexpectedly low f0. This is probably due to the fact that this tonal sequence is underrepresented in the dataset, with only 9 tokens representing 4 unique word types (cf. Table A.1 in Appendix 1). For tone patterns T1-T0, T2-T0, T3-T0, and T4-T0, the general trend appears to approach a similar mid-low pitch target at the end of the syllable, regardless of the following tone.

For the 3.4 tonal context, 14 of the tonal patterns begin with the lowest f0. This may be a straightforward consequence of Tone 3 being often realized as a low tone in Taiwan Mandarin (Fon and Chiang 1999). For the 4.0 tonal context, by the end of the word, the f0 tends to be the lowest across all panels. This is probably due to the general curve of female speakers in 4.0 tonal context. The female curve of 4.0 tonal context has a particularly salient falling trend (cf. Figure 4). Apparently, the following neutral tone is magnifying the final downward inclination observed in the vast majority of tone patterns.

Figure 6 displays predicted pitch contours estimated by the factor smooth for word, combined with the partial effects of the factor smooth for tone_pattern. These contours are predicted from an omnibus model fitted to the entire dataset, with tonal_context included as a random intercept. These partial effects exclude the general intercept and do not account for pitch differences between female and male speakers. The red dashed curves represent the partial effect of the factor smooth for tone_pattern only, without incorporating the word-specific pitch contours, and are shown to provide a reference for assessing the word-specific effects. The deviation of the blue curves from the red dashed curves reflects the differences between the predicted pitch contours and the general tone pattern.

Figure 6:

The predicted f0 contours for words with the T4-T2 tone pattern. These contours are estimated by combining the partial effects of the factor smooth for word and the corresponding factor smooth for tone_pattern (T4-T2). The dashed red curve represents the partial effect of the T4-T2 tone pattern alone, which is identical across all panels. Vertical gray dashed lines indicate the average syllable boundary, while the horizontal gray dashed line represents the y = 0 reference line. The “n=” in the upper-right corner of each panel shows the token counts for the word types. Panels are ordered by the token counts in the dataset, from highest to lowest.

Figure 6 presents words arranged by frequency. Each word exhibits its own “pitch signature,” independent of frequency. For further discussion of a possible role of frequency, see Appendix 2. It can be observed that the pitch contours of 後來 (hou4lai2, “later”), 不然 (bu4ran2, “otherwise”), and 不能 (bu4neng2, “cannot”) closely align with the predicted tone pattern but are overall shifted upward. Similarly, the pitch contour of 認為 (ren4wei2, “to believe”) also follows a similar shape but is shifted downward. However, other words, such as 幹嘛 (gan4ma2, “What for?”), 目前 (mu4qian2, “at present”), and 化學 (hua4xue2, “chemistry”), largely deviate from the general tone pattern. Two words beginning with 不 (bu4, expressing negation), 不然 (bu4ran2, “otherwise”) and 不能 (bu4neng2, “cannot”), have very similar contours that run parallel to the contour of the T4-T2 pattern. However, 不行 (bu4xing2, “not okay”) displays a steeper fall.

The word-specific tonal realizations observed here are similar to those reported for Mandarin disyllabic words with T2-T4 tone patterns (Chuang et al. 2025), as well as words with the T2-T3 and T3-T3 tone patterns (Lu et al. 2024).

3.4.2 Sense-specific tonal realization

In the preceding section, we documented that the tonal realization of Mandarin disyllabic words varies systematically by word. It is possible that words’ segmental make-up is the crucial factor. Alternatively, it is theoretically possible that it is words’ meanings that shape their pitch contours, just as in English, the duration of homophones is to a considerable extent codetermined by their meanings (Gahl and Baayen 2024). If this hypothesis is on the right track, then word sense should be a more precise predictor than word identity. In the following analysis, we explore whether we can replicate previous studies in which sense emerged as an even better predictor of disyllabic words’ pitch contours than the word itself (Chuang et al. 2025; Lu et al. 2024). If we can show that a word with different meanings exhibits varying pitch realizations, this will provide further evidence that words’ meanings codetermine tonal realization.

In order to explore the value of this hypothesis, we make use of the fact that our data are taken from a corpus, and not a word list. As a consequence, we can estimate a word token’s most likely sense in the exact context in which it was used. To determine these most likely senses in context, we made use of the sense identification system proposed by Hsieh et al. (2024), which uses BERT in combination with the Chinese WordNet (Huang et al. 2010). For example, this system identifies the word 先生 (xian1sheng1, “husband, sir” as “a woman’s spouse in a marital relationship”) in sentences such as 我先生認為 (wo3xian1sheng1ren4wei2, “My husband thinks …”) or 我先生去睡覺 (wo3xian1sheng1qu4shui4jiao4, “My husband went to sleep …”). It assigns 先生 (xian1sheng1, “husband, sir”) the sense “a man addressed in a social context” to when it appears in the phrase 那位先生 (na4wei4xian1sheng1, “That gentleman over there …”).

Since not all words in the dataset could be assigned a sense, we excluded words for which no sense type was identified. Second, we removed sense types represented by fewer than six tokens to ensure that each sense type had sufficient data for meaningful analysis. To prevent the model’s predictions from being biased toward high-frequency sense types, we limited the maximum number of tokens per sense type. Specifically, for any sense type represented by more than 50 tokens, we randomly sampled 50 tokens from all tokens. We then grouped the dataset by tonal context, as in the previous analysis, resulting in four subdatasets (see Table 3). The final dataset consists of 3,525 tokens representing 290 unique sense types. After the trimming process, 252 unique word types remain from the initial 313. All 20 tone patterns are present for each tonal context. The distribution of sense types and word types follow the similar pattern as in the dataset shown in Table 1.

Table 3:

Overview of trimmed datasets grouped by the four tonal contexts, for the sense analysis.

Tonal context	Tokens	Sense types	Word types	Tone patterns
4.4	1,512	266	233	20
3.4	740	220	195	20
4.1	716	228	200	20
4.0	557	171	157	20
Total	3,525	290	252	20

For the sense analysis, we replaced the factor smooth for word with a factor smooth for sense_type, while keeping all other predictors from the previous analysis.

logf0	∼ gender +
	s(normalized_t, by=gender,k=4) +
	s(speaker, bs=“re”) +
	s(normalized_t, tone_pattern, bs=“fs,” m=1)+
	s(normalized_t, sense_type, bs=“fs,” m=1) +
	s(duration, by=gender, k=4) +
	ti(normalized_t, duration, k=c(4,4)) +
	s(norm_utt_pos, k=4) +
	ti(normalized_t, norm_utt_pos, k=c(4,4)) +
	s(bg_prob_prev, k=4)+
	ti(normalized_t, bg_prob_prev, k=c(4,4))+
	s(bg_prob_fol, k=4)+
	ti(normalized_t, bg_prob_fol, k=c(4,4))

An AR(1) process in the errors was also included to account for the autocorrelation in the pitch time series. The model summary is available in Appendix 1.

To assess the relative importance of sense_type, word, and tone_pattern, we compared models with different predictor structures: (1) all other predictors + tone_pattern, (2) all other predictors + word, (3) all other predictors + tone_pattern + word, and (4) all other predictors + tone_pattern + sense_type. Table 4 presents the AIC differences resulting from adding the given variable(s), relative to the all other predictors model.

Table 4:

AIC scores for models with different structures of word, sense_type, and tone_pattern, fitted separately to datasets for the four tonal contexts.

Tonal context	Model	AIC	ΔAIC
4.4	All other predictors	−210,904.67	–
4.4	All other predictors + tone_pattern	−217,017.19	−6,112.52
4.4	All other predictors + word	−226,514.74	−15,610.07
4.4	All other predictors + tone_pattern + word	−226,532.42	−15,627.76
4.4	All other predictors + tone_pattern + sense_type	−226,981.87	−16,077.20
3.4	All other predictors	−107,031.04	–
3.4	All other predictors + tone_pattern	−111,086.38	−4,055.34
3.4	All other predictors + word	−116,996.00	−9,964.96
3.4	All other predictors + tone_pattern + word	−116,989.72	−9,958.67
3.4	All other predictors + tone_pattern + sense_type	−117,572.92	−10,541.88
4.1	All other predictors	−104,637.05	–
4.1	All other predictors + tone_pattern	−107,224.95	−2,587.90
4.1	All other predictors + word	−113,207.07	−8,570.02
4.1	All other predictors + tone_pattern + word	−113,206.72	−8,569.67
4.1	All other predictors + tone_pattern + sense_type	−113,808.24	−9,171.19
4.0	All other predictors	−85,980.81	–
4.0	All other predictors + tone_pattern	−88,837.72	−2,856.91
4.0	All other predictors + word	−93,647.14	−7,666.33
4.0	All other predictors + tone_pattern + word	−93,659.45	−7,678.64
4.0	All other predictors + tone_pattern + sense_type	−93,876.38	−7,895.57

1. ΔAIC represents the difference in AIC between a given model and the baseline model including all other predictors (ΔAIC = AIC_model − AIC_baseline), with more negative values indicating improved model fit relative to the baseline. The model with the greatest AIC improvement is highlighted in bold.

First, comparing row 2 and row 5, the inclusion of tone_pattern and sense_type(ΔAIC: −16,077.20) led to a more substantial model fit improvement than tone_pattern by itself (ΔAIC: −6,112.52). This indicates that tone_pattern contributes to the model fit, albeit with a relatively minor effect compared to when sense_type is also included. A similar AIC pattern across the 3.4, 4.1, and 4.0 tonal contexts further reinforces the stronger influence of sense_type over tone_pattern in modeling f0 contours.

Second, consider the GAMMs where word was replaced by sense_type. In the case of the 4.4 tonal context, replacing word with sense_type (comparing row 4 and row 5) led to a substantial AIC decrease of 449.44 units, suggesting that sense_type is a stronger predictor than word for modeling f0 contours. The stronger effect of sense is also observed across other tonal contexts.

Figure 7 displays the predicted tonal contours for different sense types of 另外 (ling4wai4, “in addition”), calculated by combining the partial effects of sense_type and tone_pattern. Similar to the red dashed curves in Figure 6, the red dashed curves in Figure 7 again represent the general tone pattern, which is T4-T4 in this case. The three sense types of 另外 (ling4wai4, “in addition”) are “others” (sense1), “in addition to” (sense2), and “totally different” (sense3). The word 另外 (ling4wai4, “in addition”) exhibits clear variations across the three sense types compared to the general tone pattern. The pitch contours of sense 1 (shown in purple) are generally shifted below the general tone pattern, while those of sense 2 (shown in blue) are shifted above it. The pitch contour of sense 3 (shown in yellow) displays two rises, as in the general tone pattern, but is shifted upward.

Figure 7:

A selection of the predicted f0 contours for different sense_type of 另外 (ling4wai4, “in addition”) across tonal contexts. The predicted pitch contours represent the partial effect of the factor smooth for sense_type, combined with the corresponding factor smooth for tone_pattern (T4-T4 in this case). The red dashed curves represent the partial effect of T4-T4 tone pattern alone, averaged across four tonal contexts, so the red dashed curve is the same across all panels. Vertical dashed lines indicate the average syllable boundary, and the horizontal gray dashed line represents the y = 0 reference line.

3.5 Summary

This section addressed our first hypothesis, namely, that the meanings of words codetermine the phonetic details of how the tones of these words are produced. Our results show that word emerged as a more powerful predictor than all other predictors. Surprisingly, the variable importance of word was substantially greater than that of tone_pattern. The strong effect of word that we observed is in line with the results of Jin et al. (2025); Lu et al. (2024); Jin et al. (2025) also observed, albeit for monosyllabic words, that word was a stronger predictor than tone_pattern. In the study by Lu et al. (2024), however, the variable importance of word was similar to that of tone_pattern.

A further analysis clarified that sense_type is an even better predictor of pitch contours than word. The substantial improvement in model fit contributed by sense_type provides further support for the hypothesis that words’ meanings codetermine the fine detail of their pitch contours, replicating the findings of earlier studies (Chuang et al. 2025; Jin et al. 2025; Lu et al. 2024).

4.1 Fixed-length pitch vectors

To implement a linear mapping within the DLM framework, given n words, we need an n × p matrix C to represent words’ pitch contours, and an n × q matrix S for words’ meanings. Consider the form matrix C , and recall that the tokens in our dataset have unequal numbers of pitch measurement points because tokens with longer durations contain more measurement points. Furthermore, the raw data include missing values due to gaps in the pitch contours. However, the row vectors of C need to have the same fixed length p. To achieve this, we used GAMMs to obtain pitch contours represented by p = 100-dimensional vectors in normalized time. There are several ways in which such fixed-length vectors can be generated, of which we explored three.

After obtaining the estimated pitch vectors from the GAMMs, we applied by-token normalization by centering and scaling each predicted pitch vector. By doing so, the mapping from meaning to form is forced to learn to predict the shape of pitch contours rather than the absolute pitch values of each token, which vary substantially across word types and speakers.

4.2 Contextualized embeddings

We made use of Contextualized Embeddings (CEs) to represent words’ meanings. Word embeddings (semantic vectors) represent meanings in a distributed manner, building on the hypothesis that similar words occur in similar contexts (Harris 1954; Mikolov et al. 2013). Semantic vectors are more than mathematical constructs, they offer a cognitive framework for modeling human semantic memory, as shown by Landauer and Dumais (1997); Bruni et al. (2014) shows that embeddings accurately predict human semantic similarity ratings. An increasing body of research shows that semantic vector spaces encode meaningful psychological dimensions and may capture aspects of conceptual representation that are salient to human cognition (Westbury and Wurm 2022; Westbury et al. 2025). Contextualized embeddings probe aspects of semantic structure that are difficult to capture by traditional linguistics (Günther et al. 2019). For instance, Shafaei-Bajestan et al. (2024) show, using embeddings, that in English plural semantics vary by semantic class.

The first generation of semantic embeddings, such as fastText (Bojanowski et al. 2017), is fully determined by words’ orthographic forms. However, a single orthographic form can express different meanings (e.g., English “bank”), or different senses (e.g., Mandarin 水平 [shui3ping2, “level or horizontal position” or “skill or proficiency”]). Typically, the context in which a word is used provides disambiguating information. Contextualized Embeddings (CEs) were developed to provide token-specific, context-sensitive embeddings that capture the subtle differences in what a word may actually mean in context.

The CEs used in the current study were derived from a pretrained unidirectional language model based on the GPT-2 architecture, developed by CKIP, Academia Sinica. Each token in our dataset was assigned a 768-dimensional vector representing its contextualized embedding.

To inspect the quality of the contextualized embedding space, we reduced the 768-dimensional semantic space to two dimensions using t-SNE (Van der Maaten and Hinton 2008). Figure 8 displays embeddings in the resulting 2-D plane, with convex hulls highlighting clusters of tokens corresponding to different word types. Tokens clearly cluster by word, as expected. Furthermore, some semantically related words have clusters that are close to each other. For instance, in the middle-right of the Figure, the tokens of 大學 (da4xue2, “university”), 學校 (xue2xiao4, “school”), 國中 (guo2zhong1, “middle school”), and 高中 (gao1zhong1, “high school”) occur closely together, which makes sense as they are all semantically related to educational institutions. Other school-related words such as 學生 (xue2sheng1, “students”) and 老師 (lao3shi1, “teacher”) also appear near these words. Some word clusters contain outliers. For instance, in the center of the figure, 上面 (shang4mian4, “above”) has an outlier positioned near 以後 (yi3hou4, “in the future”), and 後來 (hou4lai2, “afterwards”) has an outlier near 之後 (zhi1hou4, “after”).

Figure 8:

Contextualized embeddings, obtained from a pretrained Chinese GPT-2 model, are shown in a two-dimensional plane obtained with t-SNE. Convex hulls (gray polygons) highlight the clusters of word types.

4.3 Methods

Modeling was conducted using the same dataset that we used above for the word-type–based analysis (see Table 2), which contains 4,283 tokens. This dataset, comprising all four tonal contexts, was split into a training dataset (80.39 %) and a testing dataset (19.61 %). Every word type was represented in both the training and testing data, with tokens per word being split roughly proportionally with 80 % in the training dataset and 20 % in the testing dataset.

Using the training data, we computed a linear mapping G from a 3,443 × 768 semantic matrix S to a 3,443 × 100 form matrix C by solving SG = C (for technical details, see Gahl and Baayen 2024; Heitmeier et al. 2025). We then evaluated the quality of the mapping for both the training and the testing dataset.

The accuracy of a predicted pitch vector c ̂ was evaluated as follows. For a given c ̂ , we calculated its Euclidean distance to all gold-standard pitch vectors in C . We then identified its closest form neighbor of c ̂ . If this nearest neighbor was a token of the same word type as the target token, the predicted form vector was assessed as correct; otherwise, it was considered incorrect.

4.4 Results

We estimated three linear mappings from the same semantic matrix S with CEs to three different form matrices C , one for each of the three kinds of smoothed pitch contours introduced above.

The mean accuracy of method I was 2.8 % on the training dataset and 1.4 % on the testing dataset. The mean accuracy of method II was 23.5 % on the training dataset and 15.1 % on the testing dataset. The mean accuracy of method III was 12.0 % on the training dataset and 6.9 % on the testing dataset. All accuracies were above a permutation baseline of 0.4 % and a majority baseline 1.3 %, albeit by only a small margin for method I. That method I has the lowest accuracy is unsurprising, fitting GAMMs to individual pitch contours unavoidably comes with overfitting and a loss of generalizability. Methods II and III gain strength from other tokens and incorporate less by-item observation noise. For these two methods, the mapping from meaning to pitch contours is substantially more accurate than would be expected under chance conditions. The best-performing method is method II, which abstracts away from the influences of contextual factors on the realization of pitch. This suggests that some abstraction away from the immediate context is helpful, possibly because the contextualized embeddings are not precise enough. After all, these embeddings come from a general large language model trained on large volumes of data that most likely diverge considerably for the language experience of the speakers interviewed for the Corpus of Spoken Taiwan Mandarin.

The results obtained with especially methods II and III clarify that there appears to be considerable isomorphy between the contextualized embedding space and the pitch space of word tokens. This isomorphy implies that if we take the most typical embedding for a given tone pattern and map it into the pitch space, the resulting predicted pitch contours should closely resemble the pitch contours identified by the GAMM for that tone pattern. Figure 9 shows that this prediction is on the right track. The pitch contours shown in black are the contours predicted by the GAMMs for the different tone patterns. They represent the best denoised estimates of the average tone-pattern–specific pitch contours and serve as our gold-standard pitch contours. These GAMM-based pitch contours were obtained by first extracting the tone-pattern–specific pitch contours for each of the four tonal contexts, and then averaging these (these GAMM-based contours were shown above in red in Figure 5 before). An alternative method, which results in nearly indistinguishable pitch contours, combines the data for all four contexts and adds the tone-pattern–specific contours to the general contour for female speakers.

Figure 9:

Pitch contours of 20 tone patterns in selected four tonal contexts. The black curves present pitch contours identified by GAMM, estimated by the partial effect for tone_pattern, combined with general contours of time female speakers (shown as the red curves in Figure 5, which is reproduced here). The blue, green, and red curves represent pitch contours predicted from the centroid of the contextualized embeddings using the three methods, respectively. For the blue curves, form vectors were obtained with GAMM smooths fitted to individual word tokens. For the green curves, form vectors were obtained from a GAMM fitted to the tokens of all words with a given tone pattern. For the red curves, form vectors were obtained with GAMM smooths that included all contextual factors.

We now consider how well these average pitch contours can be predicted from words’ contextualized embeddings. The most typical embedding for a given tone pattern can be approximated by calculating the centroid of the contextualized embeddings of the tokens with this particular tone pattern. The centroid is simply the mean of the semantic vectors. When we think of embeddings as points in a high-dimensional space, the centroid is located at the center of these points. To obtain the centroid of a given tone pattern, we first obtained the centroid of every relevant word type by averaging the CEs of its tokens. We then obtained the centroid of the tone pattern by averaging the centroids of the word types associated with that tone pattern. In this way, every word type is given equal weight when determining the centroids for the tone patterns.

In order to get a sense of the semantics represented by these centroid vectors, we calculated, for each tone pattern, which contextualized embeddings are closest to the corresponding centroids. Table 5 lists, for each tone pattern, the two word types with embeddings that are closest to the centroids. These two word types provide an indication of the prototypical meaning of a tone pattern. For example, 她們 (ta1men0, “they (female)”) and 他們 (ta1men0, “they (male or mix gender groups)”) are the most prototypical word types for T1-T0 tone pattern. The tone pattern T2-T0 appears to have 兒子 (er2zi0, “son”) and 孩子 (hai2zi0, “child”) as prototypical members. Most tone patterns, however, are characterized by function words.^[4]

To obtain the pitch contours predicted for the tone patterns, we provided the centroids of the 20 tone patterns as input to the three linear mappings defined above. The resulting predicted pitch contours are shown in Figure 9. Each panel in this trellis graph presents the estimates for a given tone pattern. The gold-standard pitch contours (obtained with our GAMM models as described above) are presented in black, and the contours predicted by the three DLM mappings are color-coded. The contours obtained with method I are shown in blue, those obtained with method II in green, and those obtained with method III in red. For all three methods, the resulting predicted contours are similar, and often remarkably similar, to the gold-standard contours.

To assess the similarity between the gold-standard pitch contours and the pitch contours predicted using the meaning-to-pitch mappings, we first calculated the cosine similarity, averaged across the 20 tone patterns, between the gold-standard contours and each of the three DLM pitch contours. The contours from method II show a closer fit (cosine similarity 0.93) compared to the contours from method I (cosine similarity 0.73) and III (cosine similarity 0.82). The mean correlation between GAMM-predicted pitch contours and the three DLM-derived contours is 0.85, 0.98, and 0.96, respectively. However, when using Euclidean distance to evaluate similarity, method II scores slightly worse than the other two (1.24, 1.50, and 1.09, respectively). Figure 10 presents the distributions of these three measures for the three methods, using boxplots. The three methods appear to perform with comparable accuracy, with a slight advantage for Method II when evaluated with the correlation and cosine similarity measures.

Figure 10:

Boxplot showing the correlation, cosine similarity, and Euclidean distance between GAMM-predicted pitch contours and DLM-derived pitch contours across 20 tone patterns.

4.5 Summary

In this section, we have shown that a simple linear mapping can predict the realization of token-specific pitch contours from its token-specific meaning in context with above-chance accuracy. This finding extends the earlier results of Chuang et al. (2025), which focused on disyllabic words with one specific tone pattern only (the rise-fall tone pattern T2-T4). Mapping accuracy for all 20 tone patterns is unsurprisingly somewhat lower than that observed by Chuang et al. (2025) for the T2-T4 tone pattern (30%–40 % for training data and 27%–35 % for testing data). Nevertheless, even for the present more varied dataset, accuracy is substantially above the majority baseline. This result is surprising in the light of the measurement noise that is inevitably present in both our pitch measurements and in the contextualized embeddings. The contextualized embeddings represent the knowledge of an artificial intelligence, trained on vast amounts of texts. The embeddings it generates must diverge from the meanings that the individual speakers had in mind. Nevertheless, the contextualized embeddings are sufficiently precise to enable far above chance prediction accuracy for word tokens’ pitch contours. Interestingly, the 20 canonical tone patterns are surprisingly well approximated by projecting the centroids of the contextualized embeddings of the words with these tone patterns into the f0 space. In other words, the average pitch contours identified by the GAMMs correspond to average contextualized embeddings in semantic space.