Do letters matter? The influence of spelling on acoustic duration

1.1 Why letters might matter

In order to better understand the effects of spelling in speech production, it seems promising to take a closer look at the acoustic properties of linguistic forms with different spellings. Specifically, a spelling effect on segment durations would lend support to those views that argue that the acquired orthographic information either permanently alters the lexical entries, or that this information is co-activated online during speech production. The existing studies on spelling and acoustic duration can be split up into two groups: those that look at the word level and those that look at sublexical units below the word level. As examples of the latter group, Warner et al. (2004) and Warner et al. (2006) look at potential effects of spelling on acoustic duration in the case of incomplete neutralization. The researchers report durational differences for consonants and vowels in Dutch homophone pairs such as heten ‘to be called’ versus heetten ‘were called’. The main result reported by Warner and colleagues is that consonants are produced significantly longer for words spelled with a double consonant letter, which the authors attribute directly to the differences in spelling. Similarly, in a study on frequency effects in German homophones, Hellwig and Indefrey (2017) find durational differences only for heterographic homophone pairs such as Seite ‘page’ versus Saite ‘string’, while no effect of frequency can be found in homographic pairs such as Melone ‘melon’ versus Melone ‘bowler hat’. Most recently, and for a non-alphabetic language, Grippando (2021) finds that Japanese homophones also exhibit comparable durational differences related to the number of characters in their orthographic form. In short, Grippando (2021) reports that words represented by two characters were produced significantly longer than those words that are represented by just one character. In contrast, there are also a number of studies on differences in acoustic duration that do not find an effect of spelling, specifically not one that is related to the number of letters or signs. Gahl (2008), Plag et al. (2020), and Seyfarth et al. (2018), for example, all report null results for an influence of spelling on the acoustic duration. Below the word-level, a seminal study by Brewer (2008) reports systematic durational differences for homophonous sounds that allow for heterographic representation. The main effect reported in this study is that segment duration is positively correlated with length in letters. For instance, the word-final /k/ in clique, which is spelled with three letters, appears to be significantly longer than /k/ in click (two letters), or /k/ in tic spelled with just one letter.

To summarize at this point, a growing number of results suggest a direct link between the orthographic representation of a word or phoneme and its acoustic properties – a link that is not explicable by most established models of speech production such as Levelt et al. (1999), as the forms investigated in the pertaining studies are traditionally considered to be homophonous despite their heterography. These models conceptualize homophonous forms as having identical underlying representations and phonetically identical realizations (see also Roelofs and Ferreira 2019 for an updated version of a feed-forward architecture model). Recent findings on the interaction of morphological structure with phonetic detail have increasingly called such strictly categorical approaches to homophony into question (e.g., Drager 2011; Gahl 2008; Plag et al. 2017, 2020; Schmitz et al. 2021; Seyfarth et al. 2018). It has to be noted in this context, though, that, with the exception of Brewer (2008) and Grippando (2021), the studies mentioned were not designed to specifically test the influence of orthographic information on speech production. Rather, spelling emerged as a potential post-hoc explanation of findings once some of the pertinent covariates were controlled for statistically. It is therefore not surprising that these studies did not find convincing evidence for an effect of spelling, where a study with a clear focus on spelling may have yielded different results.

As was just mentioned, Brewer (2008) presents one of two exceptions here. The series of experiments and the corpus study described in this unpublished doctoral thesis were among the first to specifically examine a direct correlation of acoustic duration and graphemic length in number of letters. The main result reported is that there is a positive correlation between the number of letters used in the representation of a word-final obstruent and the duration of this obstruent as well as the duration of the whole word. Brewer (2008) interprets her findings as an indication that an increased orthographic length increases the acoustic duration of sounds. Brewer also discusses other spelling characteristics, such as the number of possible variants and the frequency of each variant, that potentially impact segment duration which intricately interacts with orthographic length. These findings are theoretically very relevant for a number of reasons, as was explained above. The effects reported by Brewer support the notion that there is a link between the orthographic form and the phonetic realization of phonemes. As evident from the discussion above, such a link has immediate consequences for theoretical models of speech production, and it is therefore not surprising that Brewer’s (2008) doctoral thesis, even though unpublished, has been cited by a number of later publications, such as Grippando (2021), Schmitz et al. (2021), Alarifi and Tucker (2023), or Hammond (2020).

However as will be described below, the analysis in Brewer (2008) may not stand up to close scrutiny. There appear to be considerable weaknesses in some of the methodological decisions as well as in the quantitative analysis, and consequently, the effects reported in Brewer (2008) are far less robust than one may desire given the potential theoretical impact. Hence, we decided to carry out a modified replication of Brewer’s corpus study. There are at least three strong arguments why a replication is valuable at this point: Firstly, as has been mentioned, the evidence presented by Brewer (2008) has been picked up by a number of influential studies. Yet, it appears to be far from robust. We aimed to overcome the methodological and technical flaws so that our analysis produces clear evidence either in favor or against the described direct influence of spelling on articulation. Secondly, we consider Brewer’s design an ideal test case for investigating such an influence. The hypothesis under examination is very straightforward, namely that more letters equal longer durations. The fact that the study looks at segment durations at the end of words is highly relevant because spelling effects have mostly been shown using reaction times as a response variable as was discussed above. Consequently, it appears plausible to interpret them as related to lexical access and retrieval. In contrast, examining durational differences at the end of words may reveal an effect that is rather located in the articulation or production stages. If we want to further our understanding of the locus of spelling effects, we need studies that help disentangle preparation and execution phase (cf. Brewer 2008: 25). Moreover, the present study uses a corpus of conversational speech, which has the advantage of limiting task specific orthography-effects that have been discussed to potentially arise from an interference of shallow visual processing in experimental tasks (cf. Mitterer and Reinisch 2015). The third argument for a principled replication and expansion of Brewer (2008) is a methodological one: by now, there is a considerable amount of research that investigated the variables affecting segment durations. By incorporating these variables, our analysis can considerably reduce the risk of unknown confounding factors, which in effect will substantiate any effect that remains visible once the confounding factors are accounted for.

For these reasons, we decided to re-examine Brewer’s quantity hypothesis with a broader empirical base employing state of the art statistical analysis in order to substantiate the evidence for spelling effects on speech duration. We suggest that if systematic durational differences can be found for heterographic but seemingly homophonous segments in corpus data, these spelling effects can be assumed to be robust rather than a by-product of confounding morphological, lexical or contextual factors. This would in turn call for a revision of models on how orthographic knowledge is integrated in spoken word production. Ultimately, this adds to our general understanding of how lexical representations are created and maintained.

As a last remark, a thorough re-examination of a theoretically relevant finding also ties in with the general discussion around the replicability of, specifically, small effects in speech production research in linguistics. Given the small magnitude of some of the effects we are often referencing and the ever-growing body of speech data that is available for analysis, it seems crucial to also re-examine and verify previous findings documented in the literature. Replications are meant to test the validity of results. This becomes especially relevant if the results stem from a single study, which is constrained to its specific research methodology and environment as well as the state of the art of this time (cf., e.g., Roettger and Baer-Henney 2019; Strycharczuk 2019). Confirming or rejecting existing results can pave the way forward, either with substantiated findings or with new insights on why results can also vary.

3.1 Data set and pre-processing

Unfortunately, the original data set was no longer available, and could not be completely reconstructed on the basis of the documentation in Brewer (2008).^[1] In order to be able to reanalyze the data from Brewer (2008) we wrote a Python script that followed the steps outlined in the original study as closely as possible except for two slight tweaks. First, we did not include non-final segments in the data set to increase the comparability of the phonological and orthographic environment in question. In other words, for words like risk, we excluded the target fricative that occurred in non-final position. Second, we added two more voiceless obstruents that were not examined in the original study: [ʃ] and [θ].

As was explained above, the data selection in Brewer (2008) was strictly phonetically driven. This means, every token was included that was phonetically realized with a target obstruent in a matching environment, regardless of the underlying phonological form. For this, the PERL script used the time-aligned written and phonetic transcription provided in the Buckeye corpus. We adopted this strategy for the present study. Our script retrieved each instance of a monosyllabic English word with [p, t, k, f, s, tʃ, ʃ, θ] in word-final position. The resulting data set was complemented using data from the English Lexicon Project (ELP, Balota et al. 2007) to obtain additional information on orthographic variables, additional frequency measures, biphone frequencies and biphone conditional probability. Items with missing values, i.e. those tokens from the Buckeye corpus that are not listed in the ELP, were removed from the data set. We excluded tokens with exceptionally short word durations (i.e. shorter than 0.6 s) and target segment durations (i.e. shorter than 0.3 s), or an implausibly slow speech rate (i.e. a value below 1 at this preprocessing stage, which corresponds to less than 1 segment per second). Additionally, we manually removed the item prompts from the data set, as it was the only item with more than six segments and was thus classified as an outlier (see OSF repository at https://osf.io/pt65r/).

The final data-set contains 21,679 monosyllabic tokens (700 types) ending in one of eight voiceless obstruents [p, t, k, f, s, tʃ, ʃ, θ]. Table 4 gives an overview of the extracted obstruents and their respective frequencies.

As was described above, there were some issues with how the word-final sounds are aligned with letters or letter combinations. In order to increase the systematicity of the procedure, we aligned all letters following the representation of the vowel in writing with the consonant sound, starting with the first consonant letter. We wanted to keep the alignment as straightforward as possible and associated all letters that followed the vowel representation with the unit to the right, even if there might not be a 1:1 correspondence of sounds and letters. These units comprise one to three letters and can contain both consonant and vowel letters. This way, we ensure that the procedure remains phonetically driven and avoid any a priori restrictions on which letter combinations should be included in the analysis that could prove theoretically hard to justify, as we have discussed above.

This approach is of course complicated by units that combine vowel and consonant letters or by units that contain silent letters. Silent letters have been discussed as both modifying vowel phonemes (cf. Kessler and Treiman 2001; Venezky 1967, 1999) and being part of the consonantal realization (cf. Treiman et al. 1995). We account for this complication by adding a categorical variable called graphemic complexity, which distinguishes five different types of letter combinations: portmanteau, simple, compound, silent, and discontinuous. We use the term portmanteau to classify a specific case with only one exemplar: the single letter unit ‘x’ in words like six or flex realizes two sounds at the same time. Letter strings with a straightforward correspondence of sound and letter are classified as simple. For [p, t, k, f, s] each there is a consonant letter that realizes these sounds word-finally. Compound strings are defined as combinations of two or three consonant letters, such as ‘ck’ in sick or ‘tch’ in stitch. This category also includes the past-tense ‘ed’ that is a combination of a vowel and a consonant letter. It was classified with the remaining compound strings because there is consensus in the literature to assign these letter strings with the unit to the right, in our case, the word final consonant phoneme (cf. Kessler and Treiman 2001: 598). Combinations of two or three postvocalic consonant letters, which have been discussed as both modifying the phoneme to the left and to the right, were classified as silent (cf. Kessler and Treiman 2001: 598). Examples would be ‘bt’ in debt or ‘lf’ in words like half. Lastly, discontinuous letter strings contain postvocalic consonant letters followed by one or two vowel letters that can also be aligned with both vowel and consonant phoneme, such as ‘se’ in house, ‘ke’ in like or ‘que’ in clique. Table 5 summarizes the classification scheme with examples from the corpus.

This new typology offers two major advantages. First, it allows us to include all graphemic variants that are encountered in the data set as a realization of the consonant sounds under investigation without restricting ourselves to a specific set of variants. This increases the data set’s balance and allows the data selection to remain phonetically driven. Second, we can now distinguish systematically between the different letter-sound correspondences we find in the data. We presume that a more fine-grained distinction between types of letter-sound correspondences will allow us to probe whether sequences with silent letters or consonant letter doubling can really all be treated the same as was done in the original analysis.

Done this way, the final data set contains 21,679 monosyllabic words (700 types) ending in one of eight different voiceless obstruents [p, t, k, f, s, tʃ, ʃ, θ], realized by the 29 different word-final letters and letter combinations shown in Table 5.

3.2 Variables and covariates

The primary response variable FinalSegDur was obtained from the time-aligned written and phonetic transcriptions available for the Buckeye Corpus (Pitt et al. 2007) using the abovementioned Python script (see OSF repository). The Buckeye corpus text grids are based on a partially automatic phonetic annotation. Pitt et al. (2007) state that measurements for stops are made on the basis of spectrogram and wave-form analysis from the beginning of closure to the end of the burst, and for unreleased stops to the onset of the next segment. For fricatives, the measurements were made from the onset to end of frication. The distribution of the measured durations of each target consonant are given in Figure 1. The violin shapes represent the density function of the respective observations: wider areas indicate data ranges with relatively many observations, while narrow areas indicate a relatively sparsely populated data range.

Figure 1:

Distribution of durations by target sounds (in seconds).

As can be seen from Figure 1, the individual sounds show roughly the expected inherent length differences: stops are generally shorter on average than fricatives (cf. Umeda 1977, among others). It can also be seen from the plots that for some of the obstruents, the distribution is more scattered than for others and non-normally distributed. This was addressed in the modeling procedure (see Section 3.3).

Let us now look at the variables of interest and covariates that were included in the present analysis. One of the more fundamental modifications of our analysis in comparison to Brewer (2008) concerns the choice of covariates that are included in the analysis. As noted above, previous studies on the acoustic duration of individual sounds have identified a wide range of important acoustic and non-acoustic factors, including effects of the segmental context, syntactic structure, speech rate, or informativity, not all of which were included in the original analysis. Other changes in the variable selection concerned factors that were considered in Brewer (2008) but which may introduce potentially harmful collinearity effects. Table 6 gives an overview of all variables considered for our analysis. In analogy to Table 2 above, each variable is classified either as a response variable, a variable of interest, or a covariate. The table also indicates whether the variable has been modified or added in comparison to the original analysis. The unmodified variables were gathered in the same way as described in the original analysis (see Table 2). We will briefly discuss the modified variables following the table.

GraphemicComplexity This categorical variable with the five different levels portmanteau, simple, compound, discontinuous and silent has already been described in detail in Section 3.1. It is a typology of the different letters and letter strings that may be aligned with a target segment in the data set. Together with NumLetSeg and NumLetWord (the number of letters aligned with the target segment and the number of letters in the target word, respectively), this variable concludes the set of spelling-related predictors.

NumGraphVariants This numeric variable depicts the variability in the orthographic form of a sound. We counted the number of variants that were found to realize a given sound in the data. As summarized in Table 4 above, in total, the eight different target obstruents are realized by 29 letters and letter strings. The values range from one (1) variant (/θ/) to six (6) variants (/t, k/); the remaining consonants fall between (two (2) variants for /p, ʃ, tʃ/; five (5) variants for /f, s/). The idea that the degree of variability in sound-to-spelling mapping might have an effect on speech stems from research on perception, where (in)consistency effects are well-documented (e.g., Cassar and Treiman 1997; Ehri and Wilce 1980; Nayernia et al. 2019; Pattamadilok et al. 2007; Taft 2006).

GraphVariantFreq This is another numeric predictor that depicts the raw frequency of each of the graphemic variants, i.e. how often a given variant occurs in the corpus as a realization of the aligned target consonant. This, too, is a predictor that was inherited from Brewer (2008). It presumably taps into similar facilitative effects as variability, which has also been discussed in the context of speech perception. It has been suggested that more frequent variants facilitate lexical access, for example, in phoneme monitoring tasks (Dijkstra et al. 1995), which in turn may influence the acoustic duration of the target consonant.

FinalSeg This categorical variable indicates the target consonant of the associated observation (i.e. /p, t, k, f, θ, s, ʃ, tʃ/) and captures any expected inherent differences as a random effect in the analysis.

IsMorph This is a binary variable that distinguishes between the cases in which segments have morphemic status, namely {past-ed} in taxed or {plural-s} in months, and cases where segments are non-morphemic. It has been repeatedly shown that the morphemic status of a segment can influence its duration (e.g., Plag et al. 2017; Schmitz et al. 2021) and the contrast of morphemic versus non-morphemic segment was already a prevalent variable in the original analysis (Brewer 2008: 113–115).

NumSegConsClus This categorical variable encodes the number of segments that the speaker used to realize the word containing the target segment and the presence or absence of a consonant cluster. It is based on the actual pronunciation of the word as transcribed in the Buckeye Corpus and not on the canonical phonological form. In other words, segment deletions and reductions are taken into account by this variable. However, the number of segments in monosyllabic words is strongly correlated with the presence or absence of a consonant cluster in the coda, i.e. with the presence/absence of other consonants immediately preceding the target consonant. This is due to the phonotactic structure of English syllables: since the maximum number of segments in the onset of a syllable is restricted to three and the nucleus is an obligatory part of the syllable, higher number of segments in a word are increasingly more likely if the coda contains a consonant cluster (Giegerich 1992: Ch. 6). In order to account for this, we decided to combine the number of segments and the presence of a consonant cluster in the coda into the categorical variable NumSegConsClus with the eight levels 2n, 3n, 3y, 4n, 4y, 5n, 5y and 6y (‘n’ indicates that the target consonant is the only coda consonant, ‘y’ indicates that there is at least one additional consonant in the coda, forming a cluster).

UtterancePos There is rich evidence that the presence of a prosodic boundary has a lengthening effect on the preceding linguistic material (cf. Turk and Shattuck-Hufnagel 2007; White et al. 2014). In order to account for this potential prosodic effect, we included the categorical variable UtterancePos with the values ‘final’ for a word at the end of a turn, ‘prepause’ and ‘prevoc’ if it occurred before a pause or a vocalization such as laughter, respectively, or ‘internal’ if the target occurs within a turn. As the Buckeye Corpus does not contain syntactic parsing, and given the often interrupted nature of naturally spoken utterances in general, no further attempt was made to identify syntactic boundaries beyond this classification.

FreqCOCA This variable was included to incorporate the well-documented effect that word durations correlate negatively with their frequency, so that, all other things being equal, more frequent words are produced shorter on average than less frequent words. We obtained the frequency of each word containing a target consonant from the Corpus of Contemporary American English (COCA, Davies 2008–) using the corpus query tool Coquery (Kunter 2017) and the DVD version of COCA spanning years 1998–2012. In order to match the spoken nature of the Buckeye Corpus, the frequency count was restricted to the spoken component of COCA.

OLD20 The Orthographic Levenshtein Distance (OLD, Levenshtein 1966) was extracted from the ELP (Balota et al. 2007). OLD is a numeric predictor that indicates the number of deletions, insertions, and substitutions needed to generate one orthographic string from another. In other words, it is a measure of relatedness. For example, the Levenshtein Distance from BUS to FUSS is 2, reflecting one deletion and one substitution. For the present analysis, we used OLD20 measures, which is defined as the mean OLD from a word to its 20 closest orthographic neighbors (Yarkoni et al. 2008: 972). Neighborhood effects are well documented in lexical processing and there is growing evidence that letter-based similarity intricately interacts with phonological and semantic relatedness (e.g., Grainger et al. 2005).

AgeOfAcquisition As was discussed above, research on second language learning suggests that spelling effects might be more pronounced for learners of language because the learning process usually involves increased exposure to written language, whereas first languages are usually learned on the basis of phonology (Bürki et al. 2012: 451). We included this predictor as a first proxy of whether a later acquisition – and hence a potentially closer connection of phonological form and orthographic information – would show any effects (see also Afonso et al. 2018; Treiman et al. 2022; Qu and Damian 2019, for a discussion of L1–L2 differences and experience as a potential amplifier for spelling effects). Like OLD20, the average age of acquisition was also obtained from the ELP (Balota et al. 2007).

We considered two further variables that addressed the fact that the distribution of the individual variants may be somewhat skewed in that for some consonants, one or more variants occur much more frequently than other variants. The first variable GraphVariantProb expresses the probabilities that a given consonant is realized by the associated graphemic variants by dividing the raw frequencies of each graphemic variant by the overall frequency of the respective sound. The second variable GraphVariantEntropy assesses the informativity of the set of graphemic variants for a target segment. For example, when comparing a consonant for which all graphemic variants have similar probabilities to a consonant that has a small number of graphemic variants that occur much more frequently than the other possible variants, the latter will have a much lower entropy (and hence, will be less informative) than the former. As low informativity appears to correlate with short acoustic durations (e.g., Seyfarth 2014 for word informativity), a similar pattern may be expected for the graphemic variant entropy as well.

Despite the reasonable motivations for these variables, we decided against incorporating these two informativity-related variables into our model after an inspection of the potential predictor variables. The correlation matrix of the numeric predictors using the cor() function from the corrplot package (Wei and Simko 2021) revealed high correlations between LogGraphVariantFreq and GraphVariantProb (r = 0.65) and in particular between GraphVariantProb and GraphVariantEntropy (r = −0.69). Of course, the correlation between probabilities, entropy and frequency is no surprise: if one spelling variant is much more frequent, it will also be much more probable than the others and in turn much more expected, but this increased probability will decrease the entropy for the set of grapheme variants as a whole. However, as discussed in Tomaschek et al. (2018), including variables that contribute very similar information to the model can cause harmful side effects such suppression (the apparent insignificance of a variable that should by all expectations emerge as significant) and enhancement (an exaggerated contribution of a variable that may be expected to have a much smaller effect). In other words, including highly correlated variables as predictors may severely impair the interpretability of the resulting model. Consequently, of the three highly correlated variables, we decided to retain only GraphVariantFreq. It may be reasonably argued based on the correlational structure that much of the information contributed by the other variables is already accounted for by this variable.

Table 7 summarizes all variables included in our analysis.

3.3 Statistical analysis

Perhaps the most obvious change in the statistical procedure is to replace the OLS regression analyses that were conducted by Brewer (2008) with mixed-effects linear regression analyses, as implemented in the packages lme4 (Bates et al. 2015) and lmerTest (Kuznetsova et al. 2017) for R (R Core Team 2022; RStudio Team 2022). Mixed-effects models offer at least two advantages over OLS models (Bates et al. 2015). First, they distinguish between fixed effects (typically variables which may be expected to contribute systematically to the variance of the dependent variable) and random effects (sometimes termed noise variables that introduce unsystematic variation). A mixed-effects model may account, for example, for speaker-specific or item-specific variation that is introduced to the data because the speakers or items were randomly sampled. Second, mixed effects models are generally better at dealing with unbalanced data sets, i.e., data in which some categories or category combinations are much more frequent than others. These properties are highly useful for the present analysis for two reasons. First, we are looking at fine phonetic detail in speech data, which means the effects will have a potentially small magnitude (Strycharczuk 2019: 8–10). This calls for a maximized control of random variation to be able to identify any real effects. At the same time, due to the nature of the English lexicon and spelling system, not all combinations of values of independent variables are represented in the data with equal frequency (e.g., Berg 2019 on the peculiarities of the English orthographic system). This skewness is best addressed with a mixed-effects model.

The main hypothesis by Brewer (2008) states that spelling properties of a sound will affect the sound’s duration. In order to directly test this hypothesis, we fitted two separate models. The QUANT model closely replicates Brewer’s (2008) analysis in that it tests the influence of the mere number of letters NumLetSeg on the sound’s duration. The QUAL model is the modified version of the analysis which includes our newly added variable GraphemicComplexity that captures not only information on the quantity of letters in the spelling of the word-final sound but also the quality, i.e., the specific make up (see Section 3.1 for a detailed explanation). As such, the QUAL model may be considered a superset model that may detect the influence of different types of orthographic representations beyond merely the number of letters. Apart from the diverging main variable of interest, the remaining structures of the models were exactly the same. Following the principle of maximal models (cf. Barr et al. 2013), we included all explanatory variables and covariates in the initial models that were explained in Table 7. In order to reduce the strong skew of the two frequency measures GraphVariantFreq and FreqCOCA, these variables were log-transformed prior to their inclusion in the model. Similarly, we log-transformed the conditional probability BiphoneCondProb to eliminate the problem that probabilities are intrinsically bound between 0.0 and 1.0, which is a property that linear models cannot fully account for. The numerical variables (including the log-transformed ones) were normalized prior to their inclusion. In addition to the predictor variables, we also included random intercepts for the variables Filename (i.e., the name of the Buckeye recording, which captures speaker-specific variation), Word (which captures word-specific variation), and FollowingSegment (which is derived from the context in the Buckeye Corpus, since the phonological environment may be considered to be another source of variation). We decided to also add a random intercept for FinalSeg (which captures the intrinsic variation of the individual consonant that is represented by the grapheme).

The specification of our initial QUANT and QUAL models are shown in (A) and (B), respectively, using the R formula notation.

(A) QUANT

FinalSegDur ∼ NumLetSeg + GraphVariantFreq + NumGraphVariants + OLD20 + AgeOfAcquisition + SpeechRate + LogFreqCOCA + LogBiphoneCondProb + IsMorph + NumSegConsClus + UtterancePos + (1|Filename) + (1|Word) + (1|FollowingSeg) + (1|FinalSeg)

(B) QUAL

FinalSegDur ∼ GraphemicComplexity + NumLetSeg + GraphVariantFreq + NumGraphVariants + OLD20 + AgeOfAcquisition + SpeechRate + LogFreqCOCA + LogBiphoneCondProb + IsMorph + NumSegConsClus + UtterancePos + (1|Filename) + (1|Word) + (1|FollowingSeg) + (1|FinalSeg)

The decision to treat FinalSeg as a random effect in these models requires some explanation. It is a well-known fact that different speech segments can have notably different segment-intrinsic durations (cf., e.g., Klatt 1976 for a discussion). For American English, Umeda (1977) finds that fricatives have generally longer durations than plosives, with the longest fricatives being /s/, and the shortest plosives being /t/ (but see, e.g., Crystal and House (1988) for differing evidence with regard to segment durations in connected speech). It seems obvious to incorporate this information in a statistical analysis. Indeed, the various models in Brewer (2008) show a significant effect of the target consonant that matches closely the average durations reported in Umeda (1977). However, including a predictor FinalSeg that expresses the identity of a consonant may conflict with other variables in the model. As can be seen in Table 5, there are consonants with only one orthographic variant (/ʃ/ and /θ/). For these specific consonants, the variable that identifies the target consonant reduplicates the information provided by the variable that identifies the orthographic variant. However, a linear model with standard dummy coding will estimate a separate parameter for each additional level of a categorical variable. If two parameters from different categorical variables account for the same variation in the response variable, this will result in a rank-deficient model, in which one of the reduplicated parameters will be discarded from the model. When Brewer (2008: 100) describes that “(…) ’k’ is dropped from the model, mathematically necessary to uniquely identify the added variable VARIATON”, she is probably referring to exactly such a rank-deficient model. We can avoid rank-deficiency if FinalSeg is treated not as a fixed effect but as a random effect instead. Fixed effects estimate one parameter for each variable level in addition to the reference level, while a random effect in a mixed-effects model is not considered a parameter regardless of the number of categorical levels, and random effects are calculated separately from the fixed effects (cf. Baayen et al. 2008). However, the mathematical properties of random effects and in particular, the fact that the adjustments are unbiased and hence have an average of zero mean that the resulting random intercepts for each level of FinalSeg cannot be interpreted as representing the average duration of the corresponding consonant. While we expect to find that plosives have lower random intercepts than fricatives to reflect the shorter intrinsic duration of plosives, the actual numerical values will not correspond directly to the average intrinsic durations. We will return to this point when we look at the results of our final models.

After deciding on the set of predictors and the random effects structure, we applied the following modeling strategy for both QUANT and QUAL model: First, we fitted a linear mixed effects model according to the model specification in (A) and (B), respectively, using the lmer() function from the lme4 package (Bates et al. 2015). An inspection of the QQ plot of the residuals of both models revealed severe deviations of the residuals from a normal distribution. We addressed this potential violation of the linearity assumption by applying power translations to the response variable in each model (Box and Cox 1964; Venables and Ripley 2002; λ _QUANT = 0.075, λ _QUAL = 0.070). Refitting the initial models with the transformed variables greatly reduced the initial deviations from normality of the residuals. Following the suggestions from Baayen and Milin (2010) for mixed-effects modeling, data points with residuals larger than 2.5 standard deviations were removed from the resulting models (257 and 256 out of 21,679 data points for the QUANT and QUAL models, respectively, i.e., ∼1.2 percent of the total number of observations). Both models were evaluated for the presence of potentially harmful collinearity using the collin.diag() function from the misty package (Yanagida 2023). The VIFs suggested no serious risk of collinearity for both the QUANT and QUAL model. Finally, the fully specified models based on the formulas in (A) and (B) were subdued to a stepwise elimination procedure using the step() function with the Satterthwaite method implemented in the lmerTest package (Kuznetsova et al. 2017). This ensures that all insignificant predictors were removed one after another. The results from the final QUANT and QUAL models are described in the next section.

5.1 Explanations and theoretical implications: consistency, visibility, or discriminative power?

As was outlined in the Introduction, one of the best-documented effects of spelling arises from (in)consistencies in the spelling-to-sound or sound-to-spelling mapping. Across different languages and a range of paradigms, it has been repeatedly shown that consistent mapping of sound and spelling can facilitate response times while, conversely, diverging pronunciation or spelling will slow speakers down (e.g., Pattamadilok et al. 2007; Perre and Ziegler 2008; Perre et al. 2009; Seidenberg and Tanenhaus 1979; Taft 2011; Taft et al. 2008; Ziegler and Ferrand 1998; Ziegler et al. 1997a, 1997b). Now, as was already mentioned, the letter x exhibits a near perfect consistency. In other words, there is a high likelihood that x in word-final position is going to be realized as /ks/ in speech. This could give speakers a processing advantage, which would help to explain why the /s/ in fox is shorter than the /s/ in toss.

At the same time, another line of evidence from perception studies suggests that the presence of corresponding letters raises speakers’ awareness for sounds, which presumably translates into differences in categorization. Such an effect has been discussed early on, for example, by Ehri and Wilce (1980), who show that literate speakers of English tend to identify one more sound in words like pitch with a tch spelling compared to rich with a ch spelling. Comparable effects were reported by, for example, Dijkstra et al. (1995), or Hallé et al. (2000), Bürki et al. (2012), which were discussed above. Althaus et al. (2022) similarly argue for a reciprocal influence of phonological and orthographic representations, stressing that phonological alternations that are orthographically represented have processing advantages as their discriminative power over the competitor is increased. Althaus et al. (2022: 222) further argue that spelling has the ability to boost discrimination, which they view as an acquired processing advantage of literate speakers. This view would also readily accommodate other findings on heterographic versus homographic homophones (Hellwig and Indefrey 2017), effects of pseudohomophones (Taft 2006), pseudohomographs (Taft et al. 2008), unrepresented phonemes (Nayernia et al. 2019), and cross-modal neighborhood effects (e.g., Carrasco-Ortiz et al. 2017; Grainger et al. 2005). Taking these discussions into account, the shorter duration of /s/ in words like fox compared to toss could arise from the lack of a direct orthographic representation for /s/ in fox or else the unambiguous presence of such a representation in words like toss. More generally, speakers presumably conceptualize sounds (partially) represented by silent letters or with an unclear sound-letter-alignment differently from sounds represented by just one letter or a clearly aligned letter combination.

To complicate matters even further, there is a third possible source of influence. In English, monosyllabic words that end in the letter sequences cs, ks or cks as representations of /ks/ are always morphologically complex, with the s representing either the plural morpheme, third person singular or a non-standard written form of clitic-s. Monosyllabic words ending in x, on the other hand, are not only fewer in numbers but also all simplex words, such as fox or box. Similarly, monosyllabic English words ending in a double s are most likely simplex words, such as toss or fuss. In other words, word-final /s/ in these words is always non-morphemic. Now, a growing body of studies on the phonetics of word-final /s/ in English suggests that – as a category – non-morphemic /s/ is actually the longest in comparison to third person singular, plural and/or clitic /s/ (cf. Plag et al. 2017; Schmitz et al. 2021; Tomaschek et al. 2021). Do these results contradict each other? Not necessarily. If morphological structure is reflected in the phonetics of words and a words’ spelling also exerts an influence, it might very well be that the two are connected. Strong effects could arise where spelling not only consistently represents sound but additionally reflects morphological structure, as is the case with x. This would render x ‘the odd one out’ in the category of non-morphemic /s/, but, given the small size of the set of words ending in x, it is no wonder that these would not have an overriding effect on the presumably otherwise longer non-morphemic /s/.

To summarize, the shorter durations of segments represented by a single letter or a doubling of unequivocally aligned letters could be a reflex of three presumably interconnected factors: the differences in sound-to-spelling consistency, the differences in sound categorization for different orthographic representations, and the potentially boosted discriminatory effect of consistent spellings and morphological status. Our interpretation is that, taken together, these explanations all support the view that literate language users utilize a bi- or even multimodal processing architecture. In other words, letters matter, but we do not see an isolated effect of spelling. More likely, we are observing an intricate patterning of reciprocal influence of language use in all modalities (cf. Bürki et al. 2012; Qu and Damian 2019). Such a view could also help to explain why Brewer (2008) did not find any quantity effects for pseudowords in both a reading task and a learning task, which she conducted prior to the corpus analysis replicated for the present study. The lack of results for these pseudowords is discussed as evidence for an underlying effect of lexicality. At the same time, the finding is used as a counter argument to the claim that the lengthening effect is merely an effect of shallow visual processing and task-specific to reading. If this were the case, speakers would be expected to treat words and pseudowords equally and distinguish between different spellings with distinct or lengthened articulation (Brewer 2008: 117). We would further argue that the lack of effect for pseudowords supports the view that it is only in connection with an underlying lexical representation and an acquired link between spoken and written language processing, that the orthographic representation can exert discriminative power, which could manifest, for example, in longer pronunciation durations.

How could such a link between spoken and written – and presumably also signed (cf. Cohen-Goldberg 2017) – language in both production and perception be explained within existing frameworks and theories? To date, there is a tug-of-war between the two main accounts that have been put forward (Brewer 2008; Grippando 2021; Perre et al. 2009). As was outlined in the introduction, the on-line activation account assumes that speakers co-activate orthographic information during speech processing and production and that any of the observed effects arise from active cross-talk between the two modalities during lexical retrieval (cf. Grainger and Ferrand 1996; Ziegler and Ferrand 1998). The off-line restructuring account assumes that the acquisition of orthographic information permanently alters the abstract representations and turns them into phonographic representations, enriched with spelling information (Pattamadilok et al. 2014). Beyond that, it was also suggested that spelling might be utilized by speakers in specific contexts where it warrants an advantage to discriminate between sounds or words (e.g., Mitterer and Reinisch 2015). Following Brewer (2008: 25), the present study was designed to seek evidence that furthers our understanding of the locus of spelling effects. What have we gained with regard to this? The fact that we find spelling effects in corpus data can be seen as evidence in favor of a view that spelling effects are not simply task-specific (see also, e.g., Pattamadilok et al. 2014 for a more detailed discussion of task-specificity in these effects). With regard to the exact locus of spelling effects, our conclusion is twofold. We do not find a straight-forward quantity effect of number of letters on speech duration, which can be seen as evidence against an on-line activation. Instead, we find a pattern of complexity effects that seem to be in line with the overall rather inconsistent results previously found. Our interpretation would be that the presumed interaction of spelling with both pronunciation duration and morphological status can be seen as evidence in favor of the off-line restructuring account (cf. Aronoff et al. 2016; Gahl and Plag 2022; Sandra 2010 for a discussion of how morphology and spelling might interact). We would further argue that language users seem to draw on spelling as an additional component of language units, not altering existing fixed (and minimal) abstract representations but instead dynamically evolving them as they acquire and use language in all its modalities and capacities. The result might be described as morpho-phonographic representations.

Given the complexity and dynamicity of these integrations, the theoretical models that would best accommodate such a view are probably exemplar-based models (e.g., Bybee 2006; Pierrehumbert 2002). Such models presume that language use is shaped by a collection of memory tokens or exemplars that are used to “recognize[…] inputs and generate[…] outputs by analogical evaluation across a lexicon of distinct memory traces of remembered tokens of speech” (Gahl and Yu 2006: 213). To the best of our knowledge, however, these models currently do not integrate linguistic experience in all modalities but instead restrict experience to speech, as is illustrated by the above quote. Yet, as Bürki et al. (2012: 463) emphasize, the impact of exposure to written language is not to be underestimated (cf. Bürki et al. 2019). Maybe it is high time that we take this potency into account.

Finally, the present study has several limitations, some of which are inherited from the original study design, while others are inherent in data from spontaneous speech. For example, the choice of word-final consonant in monosyllabic words does not come without issues and it seems that – besides it being a stable environment – the degree of variation introduced by this configuration is hard to control. However, as we are trying to replicate the effects as claimed to be present in the data, we, to some extent, have to live with this weakness in the design. To address this concern at least to some degree, we introduced our variable NumSegConsClus, that captures some of the duration variation in connected speech. Yet, it might be that the target consonants were elided and thus the measurements would not be fully reliable (cf., e.g., Johnson 2004). One way of addressing this in future studies would be, for example, to focus on one specific consonant (and its orthographic realizations) in different phonological environments and then look at more fine-grained parameters of phonetic realization, maybe also beyond the segmental level. This would also address the rightful criticism that individual sounds can always be clearly demarcated and segmented.