Measuring and assessing indeterminacy and variation in the morphology-syntax distinction

1 Introduction

A global distinction between morphology and syntax is presupposed across much linguistic work, whether it professes to be theoretical or descriptive (Stewart 2016). Many theoretical approaches assume that the internal arrangement of words is distinct from that of syntactic structures such as phrases and clauses. For instance, many morphologists argue that morphology is realizational rather than combinatorial like syntax (Anderson 1992; Blevins 2016; Stump 2016; Zwicky 1985). Theories differ with respect to how these two systems are integrated or interact and what their essential properties are. Theories vary in the extent to which words play an important part in the overall distinction between morphology and syntax (Stewart 2016) and in terms of how they deal with purported ‘boundary cases’ such as clitics, compounds, incorporation, and periphrasis. Clitic position, for example, can be treated morphologically or syntactically, depending on the type of clitic (e.g. ‘simple’ vs. ‘special’) and the theoretical orientation of the author (Spencer and Luís 2012b). Similarly, incorporation can be treated strictly morphologically (Rosen 1989; Spencer 1995), involve a syntactic movement operation (Baker 1988), or overlapping morphological and syntactic constituent structures (Sadock 1980).

A global morphology-syntax distinction is an important part of the organization of many descriptive grammars or the description of individual languages. Descriptive grammars vary in the extent to which they imply the distinction is arbitrary, motivated, or necessary for describing the language or the languages in question. For instance, in his description of Central Alaskan Yupik (CAY, cent2127) Miyaoka (2012: 18) states that “The ‘word’ weighs heavily in the grammar of CAY (…) perhaps much more than those of a majority of the world’s languages”. In contrast, Epps (2004: 125) refers to the suffix label in her grammar of Hup as “to some degree a language-specific convenience”. Grammars typically discuss numerous intermediate or indeterminate cases (affixes that behave as clitics, clitics that behave as words, etc.) without necessarily questioning whether the ubiquity of such cases undermines the basic distinction between morphology and syntax or whether it suggests that a different partition is necessary. While some linguists have assumed that words (and therefore the morphology-syntax distinction) are given (Sapir 1921: 33–34), others have discussed recalcitrant problems in word-segmentation (Peterson 2006) and have even suggested that for some languages the distinction is not motivated or important (Pike 1972; Tallman 2018a). Finally, even if these issues regarding the description of specific languages are ignored, the issue of comparability across descriptions remains: the notion of word or what phenomena are considered morphological is not the same across descriptions (Haspelmath 2011; Russell 1999). One then wonders whether the distinction between morphology and syntax in descriptive grammars is a matter of expositional convenience, rather than one that is motivated linguistically or instantiated in the minds of speakers.

A common response to the latter concerns is to state that the data suggest a continuum approach. Morphology and syntax should not be thought of as discrete components, rather formatives and constructions are distributed on a cline from more morphology-like to more syntax-like phenomena. This idea introduces the question as to how specific languages distribute constructions and elements on such a cline. If elements are distributed uniformly, then it is unclear how the morphology-syntax distinction is motivated. If elements cluster into groups along this cline, then perhaps a fuzzy distinction between the two systems can be motivated. For typologists, these considerations mean moving away from categorizing constructions and elements with respect to presupposed components or sub-components (e.g. Is this clitic positioned with respect to the normal syntax or phrasal morphology of the language? Is this compound a word or a phrase?) to investigating language-internal and cross-linguistic variation with respect to patterns related to the variables that could make up the morphology-syntax distinction cross-linguistically. We refer to this approach to the issue as the ‘variationist’ approach to the morphology-syntax distinction, since it is takes as its starting point the goal of measuring cross-linguistic variation in the domain under question (Bickel 2010a).This paper explores some of the methodological problems involved in such a perspective.

We identify three problems that a variationist approach to the morphology-syntax distinction could address.

1. Boundary strength problem: Languages may vary in the degree to which morphology and syntax are distinct. Some languages might display more indeterminacy than others.

2. Composition problem: Languages may differ with respect to how the distinction between morphology and syntax is made. Certain properties (e.g. high degree of allomorphy) may distinguish morphology and syntax in one language and not another. Languages may also vary in the degree to which certain properties help distinguish morphology and syntax.

3. Architecture problem: Languages may vary with respect to whether morphology and syntax are distinct at all. Languages may have more than two systems for organizing form-meaning relations or just one.

Section 2 provides further background on the morphology-syntax distinction focusing on the concept of morphological autonomy. The concept of morphological autonomy posits that morphology is distinct from syntax because morphology displays deviations from biunique mappings between form and meaning, while syntax (typically) does not. This idea is crucial for some of the statistical methods used throughout the paper because it allows us to theoretically ground a distinction between independent and dependent variables. Section 3 provides a description of the languages used in this study and some discussion concerning why they are interesting with respect to the issues laid out above. Section 4 addresses the boundary strength problem, providing different ways of measuring morphological autonomy in individual languages as a typological index (Tallman and Epps 2020). We argue that languages can be distinguished according to how much indeterminacy there is between morphology and syntax. Section 5 describes some aspects of the data and provides a basic methodology that approaches the boundary strength problem. Section 6 discusses the issues of addressing the composition problem using Random Forest models for illustration. We argue that the problem is difficult to address because it requires selection of a dependent variable, but it is not clear what this should be. Section 7 discusses the architecture problem. Addressing the architecture problem involves leveraging statistical methods to assess whether the morphology-syntax distinction is motivated at all in specific languages and across them (Haspelmath 2011). To a certain extent, exploratory methods combined with validation techniques can determine whether a language makes a basic partition into two systems or not, but it is much less clear how to determine what the optimal number of systems actually is. Section 8 provides some concluding remarks and suggestions for future research.

2 Exponence complexity inside and outside morphology

In this section, we focus on the issue of morphological autonomy and its relationship to categories which seem to be indeterminate between morphological and syntactic analyses. Morphological autonomy refers to the general idea that there are properly morphological phenomena which cannot be accounted for by reference to syntax or general phonological rules (Anderson 1992, 2015; Aronoff 1994; Booij 1997a, 1997b; Matthews 1991). A popular argument in favor of autonomy with relation to syntax refers to differences in the way form and meaning relate to each other in each of these domains.

Since Matthews (1991) it has been common to emphasize that morphology differs from syntax in that morphological units and constructions display deviations from biunique correspondences between form and meaning. A biunique relation between form and meaning is one where form is predictable from meaning and meaning is predictable from form. Words are biunique units of syntax because one can predict their meaning from their form. Phrases are biunique insofar as their meaning is predictable from the composition of biunique words. These conditions are not supposed to hold of the units of words or morphological constructions. Below we describe different types of deviations from biuniqueness.

Multiple forms can correspond to a single meaning. There are two ways this can occur: (i) extended (or multiple) exponence and (ii) allomorphy. Extended exponence refers to cases where a specific meaning appears to be realized by more than one formative. For instance, negation in Araona (Bolivia, Takana, arao1248) is expressed by a circumfix (two formatives) pi-… -ma as in pi-mimi-ma ‘not speak’. Allomorphy refers to cases where a single meaning maps to multiple forms, but the forms are mutually exclusive such that they do not occur in the same context. For instance, the inflectional suffixes of Romance verb conjugations display (suppletive) allomorphy in this sense (e.g. -aba/-ía both mark imperfective in Spanish (Indo-European: Italic, stan1288) depending on the inflectional class). To the extent that variations in form occur for elements at the syntactic level, that variation should be predictable from general phonological rules or constraints of the language.

A single form can correspond to multiple meanings. Again, there are two types: (i) syncretism and (ii) cumulative exponence. Syncretic exponence typically refers to cases where a single form fills out two or more forms in a paradigm. An example is the widespread pattern in Indo-European languages whereby the nominative and accusative are expressed by the same form in neuter paradigms (e.g. Sanskrit (Indo-European: Indo-Aryan, sans1269) man-as ‘mind-nom/acc.sg.n’ and man-āṃsi ‘mind-nom/acc.pl.n’). Cumulative exponence refers to cases where a single form fuses together multiple semantic concepts – typically understood as abstract or grammatical concepts rather than lexical ones. For instance, Tariana (Arawak, tari1256) contains a set of morphemes that express tense and evidentiality together (Aikhenvald 2003: 289) (=naka ‘visual, present’; mha ‘non-visual present’).

The absence of form can correspond to a single meaning. Morphological analyses sometimes posit that the absence of a certain morph in some position indexes a specific meaning: zero or null morphs (Jakobson 1939; Mel’čuk 2006: 308). For instance, in Algonquian languages (algo1257), independent verb forms call for positing person number affixes and a theme affix that indicates which participant is acting on which (ni-wa:pam-a-w 1sg-see-sap-3-3 ‘I see him’). In the conjunct mode (roughly a type of subordinate clause), the third person and the theme marker could be analyzed as zero because there is only one analyzable suffix and the prefix slot is taken up by the conjunct marker (e-wa:pam-ø-ak-ø [conj-see-sap>3-1-3] ‘(that) I saw him’) (Wolfart 1973: 51–56).

A form can correspond to no meaning. A frequently cited example is the third stem -t of Latin (lati1261) (e.g. laudā-t ‘have been praised’) which occurs in the perfect participle form. The future participle form seems to be built from the perfect participle in that it always contain -t (laudā-t-ūr ‘will be praising’). According to (Aronoff 1994: 32) the perfect participle is ‘usually passive’ and, thus, the -t appears to lose its meaning in the context of the perfect participle (see Mel’čuk 1993: 46–50 for discussion).

There are two types of arguments against the claim that deviations from biuniqueness distinguish morphology and syntax. The first is that not all units described as ‘morphological’ necessarily display such deviations. The sheer volume of languages described as ‘agglutinating’ suggests that languages the world over are full of counterexamples to the purported tendency for morphological relations to display deviations from biuniqueness. For instance, the suffixes of South Bolivian Quechua (sout2991), illustrated in example (1), do not vary in form nor substantially in meaning across different verb forms and constructions.

The second type of counterexample comes from cases where deviations from biuniqueness appear to occur at the level of syntax (Haspelmath 2011: 54–58). examples of extended exponence (one meaning with multiple forms) occurring at the level of syntax are negative ne …pas in French (Indo-European: Italic, stan1290) and certain negation strategies in Teotitlán del Valle Zapotec (Otomanguean: Zapotecan, teot1238) (Gutiérrez Lorenzo 2018) as in example (2).

Examples of cumulative exponence at the level of syntax can be found in function words that encode more than one category and do not appear to be altogether that uncommon. An example is the relative pronoun from Kashmiri (Indo-European: Indo-Iranian, kash1277), which encodes grammatical relations (nominative), number (singular), and gender (masculine), as can be seen from example (3).

Thus, biunique relations appear in morphology and non-biunique relations appear in syntax. For those who wish to maintain that deviations from biuniqueness are still important for distinguishing between morphology and syntax, there are two ways to respond to the aforementioned counterexamples: (i) deny that there is necessarily a tight relationship between wordhood and morphological status; (ii) posit that the boundaries between morphology and syntax are fuzzy.

With regards to the first response, some linguists conceptualize morphology as autonomous, but do not view morphology as necessarily architecturally encapsulated in words (understood as the smallest unit of syntax). For instance, in Autolexical syntax, the morphological component can project phrase structures that overlap with syntax (or vice versa). In this framework, incorporated nouns in Greenlandic, for example, are simultaneously represented as heads of syntactic noun phrases and as dependents in the morphological structure of the verb (Sadock 1991). If the Autolexicalist perspective is combined with the idea that morphology be signaled by deviations from biuniqueness, we should expect that some syntactic elements display biuniqueness insofar as some of these are also morphological elements.^[1]

In Anderson’s A-morphous morphology perspective (Anderson 1992, 2005), there is a special component of ‘phrasal morphology’ that operates over categories in syntax, or post-lexical phonological constituents. Phrasal affixes (or ‘special clitics’) might be realized at the end or beginning of a syntactic or phonological phrase (the possessive ’s in English and some Wackernagel clitics), or even, phonological word (e.g. Romance pronominal clitics). If this is a general property of languages, we would except deviations from biuniqueness to be ubiquitous inside words as well as outside of them in ‘syntax’.

However, in their current formulations neither Autolexical syntax nor A-morphous morphology help distinguish between morphology and syntax because they provide no diagnostics for distinguishing these phenomena in a consistent fashion. For example, Anderson’s arguments for a phrasal morphology depend on identifying ‘special clitics’ which cannot be placed by the ‘normal rules of syntax’ (Anderson 1992: 200) or the ‘independently motivated syntax of free elements’ (Anderson 2005: 31). But the normal rules of syntax are never defined in any general way, nor are clear diagnostics set up that help the linguist decide whether they are dealing with special or normal syntax in specific cases. Tallman (2018b) takes up this problem in the context of clitic phenomena in Pano languages. While previous studies describe or posit certain clitics as ‘phrasal affixes’ and morphological in some sense, it only takes a minor adjustment (or no adjustment at all) in what one means by ‘normal syntax’ or ‘independently motivated free elements’ to incorporate the relevant phenomena into syntactic structure. Similar considerations carry over to Autolexical syntax: all the various modules of grammar are constrained to be ‘simple’, but it is not clear when complexification in one domain is justified by simplification in another.^[2] Currently, there appears to be no way of evading the wordhood issue vis-à-vis the morphology-syntax distinction without lapsing into circularity (Russell 1999).

One might posit that the boundaries between morphology and syntax are fuzzy (Vincent 2011: 434) or that morphological autonomy ‘is not absolute’ and is a ‘matter of degree’ (Aronoff 1994: 166; Blevins 2016: 61–62; Cruschina et al. 2013: 2; Maiden 2013: 42–43, 2011: 49; Smith 2013: 248). In such a perspective, formatives and constructions would vary in the extent to which they are morphology-like or syntax-like. Some constructions and formatives are perhaps indeterminate between which systems they belong to, but the overall patterning is such to justify a global distinction between morphology and syntax. For instance, in a discussion of case paradigms in Estonian (Uralic, esto1258), Blevins (2006: 444) suggests that the agglutinative (biunique) suffixes are marginal in some sense when he states that they ‘can be seen to be a limiting rather than a normative case’. The normative situation must be for words to be biunique and word-internal morphological elements to display deviations from biuniqueness. Some type of statistical justification ought to be provided to test whether indeterminate cases are really so marginal as to never challenge the idea that morphology is autonomous (Haspelmath 2011).

At first glance it might seem that the problem can be addressed by coding morphemes or constructs as either ‘morphological’ or ‘syntactic’ and developing a metric that captures exponence complexity/deviations from biuniqueness. Then one could assess how well exponence complexity correlates with the word-phrase distinction. The problem with this approach is that there is no independent way of classifying elements and constructs as morphological or syntactic in the first place. As argued by Haspelmath (2011) and Tallman (2020), language-specific designations such as affix, word, and phrase are not strictly comparable between languages. A solution to this problem is to treat wordhood diagnostics as typological variables and to ask how well exponence complexity is correlated with these variables (Tallman and Epps 2020). After discussing the languages used in this study in Section 3, we develop a set of wordhood criterial variables and a method for calculating exponence complexity, which we will take to be a metric of (non-)biuniqueness in Section 4.

3 The languages and the data of the study

The sample of languages used for this study is based on an expanded data set from that used in Tallman and Epps 2020.^[3] However, we limit the current sample to those languages that have at least 50 data points (i.e. 50 morphemes coded for all variables described below), to ensure that they can be used with the statistical methods we focus on.

The languages are drawn from southwestern Amazonia, except for Central Alaskan Yupik, which was chosen because of its status in the field as being understood as canonically polysynthetic. The languages and the number of data points for each language are provided in Table 1.

Ideally, our sample would contain a genetically and areally unbiased sample of languages. However, time constraints have prevented us from developing such a sample: it takes weeks to complete a single language. Only closed class morphemes were coded for in our database. The reasons for this are practical: grammars do not (typically) provide a list of all of the open class morphemes, but closed class morphemes usually receive special treatment. Furthermore, the data were gathered from a set of functional domains (valency and argument structure such as case markers, tense/aspect and time, nominal classification and gender, evidentiality and modality) to render them more comparable (cf. Tallman and Epps 2020 for more discussion).^[4] This leads to the possibility of sampling bias and is another reason why general theoretical conclusions cannot be drawn from the sample yet. The problem of overcoming sampling bias is discussed again in Section 8. We hope, however, that the present study serves as a reasonable proof of concept that could inform a broader cross-linguistic investigation able to capture cross-linguistic trends. As stated in the introduction, the primary goal of this paper is to formulate the methodological problems and tentative solutions for investigating the morphology-syntax distinction in a variationist perspective. The focus on a small sample of mostly Amazonian languages limits what empirical conclusions can be drawn from our study. Nevertheless, there are a number of reasons why southwestern Amazonian languages are interesting for the questions at hand: (i) the languages do not have inflectional classes and, therefore, arguments concerning the relationship of paradigm complexity to morphological autonomy (Booij 1997b; Stump 2001) do not clearly apply to these languages. If such systems display morphological autonomy, the motivation is different from that typically discussed in the literature; (ii) the languages have been singled out as displaying ‘syntax-like morphology’, suggesting that they provide an interesting testing ground for basic assumptions about linguistic architecture; (iii) grammars of Amazonian languages often proliferate clitic or other indeterminate categories in their descriptions (Tallman and Epps 2020).

4 Morphosyntactic variables and exponence complexity

This section provides a description of the variables used to develop the database. They were chosen based on two criteria: (i) the variable needs to have been described in the literature as definitional or criterial for distinguishing between affixes, clitics, or words (e.g. Dixon and Aikhenvald 2002; Spencer and Luís 2012a; Zwicky and Pullum 1983); (ii) the test, criterion, diagnostic, or definition can be operationalized in a way that allows the variable to be coded in a consistent manner based on the information available in descriptive grammars. The resulting variables are defined and classified in Table 2. Below we provide a brief description of each variable defending its treatment as separate from other variables where appropriate.

Some of the values specified above cannot be coded without relating an element to a base with which it combines. For instance, ‘fixedness’ refers to a base element against which the permutability of a given element is assessed. The notion of a ‘base’ is partially a semantic notion based on the notion of ‘semantic head’ used profitably in other typological work (Anderson 2006; Croft 2001). After a brief description of the variables, the concept of a ‘base’ is discussed in more detail in Section 4.7.

5 Measuring morphological autonomy: correlation matrices

The idea that morphology is autonomous from syntax but with some indeterminacy or fuzziness introduces the question as to whether languages vary in the degree to which their morphological systems are autonomous (or, how large their space of indeterminacy or fuzziness is). This section presents a few proposals of how to visualize and measure the degree of morphological autonomy as a typological index. It also provides an overview of some the associations between the wordhood and morphological criterial variables discussed in the previous section.

Given that morphological autonomy is associated with exponence complexity in the literature, one might think that statistical summaries of exponence complexity could be taken as approximate measurements of the degree of morphological autonomy. However, as we argued in Section 4.6, what is important is the way that exponence complexity correlates with other wordhood criterial properties. The correlation matrix in 1 provides a global overview of the correlation between the wordhood criterial variables and exponence complexity. Variables were recoded as numerical such that the syntax/word-like result was set as 0 and the morphology/affix-like result was coded as 1. With fixedness, for example, elements considered to be affixes or part of morphology are expected to have a fixed position, while words or syntactic elements are expected to be able to variably order. Thus, we coded fixed as 1 and variable as 0. Exponence Complexity already is a numeric variable and thus does not need to be recoded. Table 3 provides an overview of the recoding.

Figure 2 presents correlation matrices for each language separately. The code and further details can be found in the Supplementary materials (cf. 9). It is immediately apparent that languages vary in terms of whether their variables correlate at all. When they do exhibit significant correlations, these are not necessarily in the expected direction. In Chácobo and Central Alaskan Yupik all variables except exponence complexity are positively correlated with each other. In Central Alaskan Yupik, we find perfect correlations between stress deficiency and boundedness (i.e. a morpheme that is bound never projects stress), stress deficiency and coding elaboration (i.e. all morphemes that can be modified independently from their base project stress), boundedness and coding elaboration (i.e. all morphemes that can be modified independently from their base are free) and fixedness and contiguity (i.e. all contiguous morphemes display a fixed order with respect to their base). The other languages show weaker correlations across variables. For example, Hup displays no perfect correlations. In fact, it is unclear whether there is an overall tendency for the wordhood criterial variables to be correlated with one another evidenced by the values close to zero. Other languages like Cavineña and Tariana occupy a middle ground, with only a few variables showing strong correlations. In Cavineña, most morphemes that have coding elaboration also have stress. In Tariana, most bound morphemes do not have stress and do not exhibit coding elaboration.

It is not necessarily the case that the variables are positively correlated with one another. As suggested in Tallman and Epps (2020), the particular structure of Movima’s classifier system results in negative correlations between boundedness and exponence complexity.^[7] In Tariana there is a negative correlation between fixedness and coding elaboration. Contrary to expectation, in both Tariana and Movima, there is a negative correlation between the absence of coding elaboration (simplex in the graphs above) and occurring in a fixed order in relation to the base. This can also be related to the semi-open class classifier systems of these languages. Classifiers occur in a fixed order with relation to a head-noun, but can also occur with their own morphology when they function as roots.

The question then arises as to whether some of the variables should be discarded in one or all of the languages. Considering, for example, interruptability in Tariana, it seems implausible that this variable could be used to distinguish between morphological and syntactic elements, given how low the correlations are. The method described in the next section (Section 6) addresses the issue concerning the composition of morphological autonomy more explicitly.

Mean correlation across all variables are listed in Table 4. It is quite low overall, with CAY exhibiting a mean correlation of 0.57. Hup, on the other hand, shows a much lower mean correlation at 0.13 and that of Movima is even overall slightly negative. To the extent that the global correlations between wordhood criterial variables can be used as a metric for determining morphological autonomy, Central Alaskan Yupik has a much stronger distinction than Hup.

6 The composition of morphological autonomy: random forests

In this section, we use Random Forests (RF) as a way of assessing the composition problem, i.e. the question of whether or not languages vary in terms of which variables are important for distinguishing between morphology and syntax. A Random Forest is a classification algorithm that aggregates over a multitude of decision trees (Louppe 2014). The number of variables out of all dependent variables that are tried at each split in each decision tree for best classifying the data has to be defined beforehand. This is usually determined by running multiple RF models with different numbers of variables to find the one producing the best results. RF is a powerful tool for classification tasks and variable selection (i.e. which variables should be considered in a study) and has also been used for addressing problems in linguistics (Levshina 2016; Tomaschek et al. 2018). The variables of our study are strongly correlated with one another in many cases, which makes interpretation of the results under regression very challenging (Belsely et al. 1980). Random Forests do not suffer from this issue, since they do not make any assumptions about the input. RFs still require specification of a dependent variable and it is not clear how to determine this a priori. The variation in the strength and direction in correlations between the variables show that this problem is not trivial. For instance, we pointed out that interruptability/contiguity is likely not an important variable in Movima. If this variable was chosen as the dependent variable in the classification model (i.e. asking to what extent can the other variables predict contiguity/interruptability), we are likely to get a very different picture of the relative importance of the variables compared to say, if we considered bounded/free as the dependent variable. The selection of the dependent variable should be guided by theoretical considerations, but there is no general agreement in the field regarding whether there is a crucial determinant of the morphology-syntax distinction (Haspelmath 2011).

With these problems in mind, we suggest two ways in which the importance and relevance of variables can be assessed cross-linguistically. One option is to use author classifications (simplified to be either ‘affix’, ‘clitic’ or ‘word’) as the dependent variable. This reflects an intuition that many researchers have, namely that grammar authors are mostly consistent in applying wordhood criteria, even if these criteria vary by author. The other option is to select exponence complexity as the dependent variable, which we take to be ‘theoretically grounded’ in the sense that it is considered to be particularly important for advocates of morphological autonomy (cf. Section 2). These choices in dependent variables reflect two different perspectives on how cross-linguistic variation in the morphology-syntax distinction should be understood, one grounded on the partly intuitive classifications of experts on the specific languages, and the other grounded in certain linguistic theories concerning morphological autonomy. We applied both options. The exponence complexity option is presented in this paper (see Supplementary materials for the author classification option.)

All the following RF models are implemented in R with the randomForest package (Liaw and Wiener 2002). We aggregate over 10,000 trees for each of them with two variables tried at each split. All plots were produced with ggplot2 (Wickham 2016). More details and the code can be found in the Supplementary materials. Each model outputs error rates for each level of the dependent variable in a confusion matrix and an overall error rate for the model (called out-of-bag error or estimate, OOB for short). The former provides the classification accuracy with respect to the dependent variable and the latter how accurate the classification is overall. The model also provides a plot displaying Mean Decrease in Accuracy (MDA), which can be related to the relative importance of variables. Broadly speaking, certain variables are better and more consistent at classifying the dependent variable correctly than others across all the decision trees and this is reflected by MDA. Note that the actual value does not matter, since it is not comparable across models. What is important is the ranking of the independent variables and whether or not there are clear breaks between them.

To this we add a calculation of the baseline, the accuracy, and the difference between these two. The idea is to account for the skewness of the data, since the more skewed the data are, the easier it is to get a correct classification by chance. For example, imagine two languages X and Y with 100 data points each. Language X has 34 affixes, 33 clitics, and 33 words, while language Y has 15 affixes, 5 clitics, and 85 words. If we classify all elements as words in language Y, we get 85% right. But no matter what category we choose for language X, we do not achieve over 34% correct classification. The relative skewness by language needs to be taken into account when assessing whether the RF model performed well or not. The calculation of each of these measures is very simple. The baseline is exactly what we just discussed. More simply put, it represents the maximal row-sum of the confusion matrix divided by the number of data points. The accuracy represents the sum of correct predictions (i.e. the diagonal in the confusion matrix) divided by the number of data points, i.e. it reflects the proportion of correct predictions. The difference is calculated by subtracting the baseline measure from the accuracy, i.e. it reflects how much better the RF model performed over chance.^[8]

While Random Forests over author classifications provide an intuitive idea of cross-linguistic variation concerning the composition of the morphology-syntax distinction, they are problematic. It is possible that the author’s conceptions of what an affix, clitic, and word are vary in arbitrary ways and according to theoretical biases that are not directly relevant to the morphology-syntax distinction.^[9] The results of applying RFs with author classification as a dependent variable are thus provided only in the Supplementary materials. They suggest that authors vary substantially in which variables they consider to be important in classifying elements as affixes, clitics and words.

A theoretically more grounded way of assessing the composition problem using RF might be to use exponence complexity as a dependent variable. For these models we treat exponence complexity as a factor by considering a value of 1 as ‘low’ and all values larger than 1 as ‘high’. We thus ran RF models for each language with exponence complexity as the dependent variable and all others (prominence projection, coding elaboration, fixedness, and interruptability) as predictors. More formally, the models are specified as follows:

The OOB error, baseline, accuracy, and difference values are presented in Table 5. In the majority of languages, the model either performs at chance level or even slightly below. This is due to the model classifying nearly all elements as having low exponence complexity, reflecting the skewed distribution in the data. Only in Movima, which has a higher number of elements with high exponence complexity is the model able to classify well and outperform chance. The MDA plots for this language is shown in Figure 3. We see that free occurrence is by far the most important variable for classification in Movima, followed by coding elaboration. The other three variables play virtually no role in the classification. The other MDA plots can be found in the Supplementary materials for reference, but one should abstain from interpreting them altogether due to the poor model performance.

Dixon and Aikhenvald (2002) proposed that some notion of fixedness was one of the most important criteria for identifying words. Our results, however, do not confirm this idea insofar as exponence complexity is taken as the most important property for characterizing morphology: In Movima, fixedness has the lowest variable importance and in Chácobo the second lowest one, i.e. it does not contribute much to classifying elements into high versus low exponence complexity. The results suggest that the criterion of fixed/variable order is not important in distinguishing morphology and syntax at least in some languages. In Movima, boundedness is the main contributor to the classification, while in Chácobo it is prominence. Such discrepancies suggest that languages vary in terms of which criteria are important for distinguishing morphology and syntax if exponence complexity is taken to be an important criterion for division.

A general problem with this approach is that in the languages of our sample, exponence complexity displays very weak correlations overall with any of the other wordhood criterial variables, as shown by Figure 1. It seems that RFs can be used to describe variation in the use of wordhood criterial variables in relation to some base classification. They cannot help us determine the baseline classification (what base-element pairs should be categorized as morphological versus syntactic constructions) and they cannot help us determine whether a morphology-syntax distinction is motivated to begin with.

Figure 1:

Correlation matrix of the wordhood criterial variables using Kendall’s tau over all eight languages.

Figure 2:

Correlation matrices per language.

Figure 3:

Variable importance of RF models for Movima and Chácobo with exponence complexity as the dependent variable.

To the extent that the morphology-syntax distinction is valid and we agree on some way of grounding the distinction empirically, the variables also suggest that languages may vary concerning whether and the degree to which wordhood criterial variables reflect important structural generalizations in the structuring of the morphology-syntax distinction.

7 Testing morphological autonomy: cluster validation

In his discussion of wordhood diagnostics, Haspelmath (2011) addresses the possibility that a fuzzy notion of word might be valid. Conceptualizing constructs and formatives as positioned on a unidimensional continuum of boundedness, he contrasts two hypothetical situations. In one situation grammatical units are ‘randomly distributed’ on the continuum and in another situation elements display a ‘clustering distribution’. Variations on Haspelmath’s illustrative figures are produced with the strip plots in Figure 4.^[10]

Figure 4:

The affix-word continuum: two hypothetical situations.

Haspelmath (2011: 63) argues that “If the dimension along which the units differ (the boundedness scale) can be quantified, the clustering can be demonstrated by statistical techniques.” In a footnote he describes problems in quantifying the “boundedness scale” including the problem that the sample population would be an “open-ended set” (addressed in Section 4.7), that it is unclear how to weigh the diagnostics, and finally, “finding clusters in a multidimensional space is even more difficult”. With respect to the weighting problem, at our current stage of knowledge, the issue really reduces to whether we should assign a priori weights to any of the variables, but there appears to be no clear principle for deciding on this at this point. If anything, the data reviewed above suggest that any a priori weighting based on theoretical considerations (such as the idea that exponence complexity should be weighted higher) may reflect a bias towards the languages on which those theoretical considerations are primarily based (Sneath and Sokal 1973).^[11] At present there is simply no clear empirical basis to give one category a heavier weighting than any other.

The issue of multidimensionality is not specific to the morphology-syntax distinction, but is true of problems in typology (or even linguistics) in general (Baayen 2013; Bickel 2010b; Round and Corbett 2020). Thus, in principle, there is no a priori reason why the multivariate structure of the data present a problem insofar as we are willing to import methods used successfully in other domains of linguistics.

The first real problem arises when we consider the fact that there is no obvious way in which one of the variables can be considered the dependent variable. Haspelmath’s (2011) formulation, however, suggests that such a dependent variable is not needed, because we could cluster using “statistical techniques”. We simply need to show that the data cluster into x number of clusters beyond chance or randomness. Haspelmath (2011: 64) also states that “We should be open to the possibility that other kinds of clusters, e.g. ‘affixoid’, ‘clitic group’, ‘tight phrase’, or ‘stems’ will turn out to be more significant than word clustering. This would have the consequence that the primary division within morphosyntax would not be between morphology and syntax, but along other lines.”

There is, however, a problem with this formulation apart from those mentioned by Haspelmath (2011: 64): Aldenderfer and Blashfield (1984: 33) point out that there “is no standard or even useful definition of the term ‘cluster’, and many have argued that it is either too late or irrelevant to create one”. There are a variety of different clustering algorithms and models that depend on different properties that the researcher may wish to assign to a cluster (Hartigan 1975; Jain 2010). For instance, is similarity between elements of the same cluster more important than their dissimilarity with elements outside of it? What is the correct shape for a cluster in some n-dimensional space of variables (Kaufman and Rousseeuw 2005: 44)? Clustering is an exploratory method for hypothesis development not a method of inferential statistics. Clusters arrived at through clustering algorithms need to be validated, to ensure that they are not arbitrary partitions of the data. However, cluster validation techniques are currently extremely domain specific and it is not clear whether they are generally applicable across disciplines. It is beyond the scope of this paper to provide any sort of comprehensive review of clustering methods and validation techniques applied to the morphology-syntax distinction. Rather we illustrate one clustering method and one validation technique that we think engages with Haspelmath’s (2011) formulation of the problem as we understand it. We use hierarchical clusters and assess height difference between the first and second partition relative to the same difference in a simulated ‘random’ language and a set of simulated languages, a method we illustrate below.^[12] We suggest that this method can help us determine whether a partition into two clusters for a given language accounts for the data better than chance. To the extent that a dual partition of the data can be interpreted as motivating a morphology-syntax distinction, this serves as a test for a morphology-syntax distinction that does not rely on a dependent variable. However, the results of clustering models beg another question concerning the structure of the languages cross-linguistically. This concerns what the optimal number of clusters is for a given language, a problem which Haspelmath (2011) alludes to when he mentions “other kinds of clusters”. This question is much more difficult to answer and cluster models do not offer a clear interpretation as far as we can tell.

Following Jain and Dubes (1988), we assume that cluster validation means that the researcher should overcome “the clustering tendency problem”. This “refers to the problem of deciding whether data exhibit a predisposition to cluster into natural groups without identifying the groups themselves. Clustering algorithms will create clusters whether the data are naturally clustered or purely random” (Jain and Dubes 1988: 201). Following a suggestion in Jain and Dubes (1988), we address this issue by developing a data set in which results for the variables or our study are simulated and we then compare the results of the cluster models on individual languages to the simulated data set. Since a single simulated language might not necessarily be representative of randomness, we construct a data set of 1000 simulated languages for comparison.^[13] Details of how the simulated data were constructed can be found in the Supplementary materials.

For each language, we constructed a distance matrix using the Gower distance metric appropriate for mixed data types (Gower 1971) – recall that our variables are binary, ordinal, or continuous. These distance matrices were then used as the input for hierarchical cluster models, using Ward’s minimum variance method with the cluster package in R (Maechler et al. 2021). The dendrogram for the simulated data set is provided in Figure 5 and for the languages of our sample in Figure 6.

Figure 5:

Hierarchical clustering on a simulated language.

Figure 6:

Hierarchical clustering on the eight languages of the sample.

For some of the languages, visual comparison with the simulated language (SL) lends some support to a morphology-syntax distinction. For others, the distinction is less obvious. Consider the difference between SL in Figure 5 with that of Chácobo and Movima in Figure 6. Compared to the dendrogram of the SL, both languages exhibit a striking height difference between the first two partitions. The height difference reflects the distance between clusters, and thus, shows that in Chácobo and Movima a partition into two groups creates clusters that are much better separated than in the SL. Central Alaskan Yupik, Ashéninka Perené, and Tariana display roughly the same difference compared with the SL, although to a lesser degree. That Wãnsöjöt/Puinave, Cavineña, and Hup cut base-morphemes into morphological and syntactic types better than chance is less obvious. The first versus second partition difference are closer to that of the SL. In fact, in Hup the height difference between the first partition and the second is smaller than that of the SL.

If visual inspection of the dendrograms serves as a basis for assessing the motivation of partition into morphological and syntactic constructions, the hierarchical cluster models suggest that languages vary according to how much better than chance they cluster elements into two partitions (morphology vs. syntax). However, this should not be interpreted as meaning that the wordhood criterial variables pattern no better than chance as a whole. Once we move beyond considering the first partition the situation becomes more complex, a point we now turn to with reference to the metric of cophenetic distance.

Table 6 provides an overview of the height differences between the first partition and the second across the languages of the study and also the correlation between cophenetic distance and Gower distance between the elements coded in each of the languages. We also include the values for the simulated language displayed in 5 which we use for illustration. Cophenetic distance is a measure of how (dis)similar two elements need to be in order to be grouped into the same cluster (Sokal and Rohlf 1962). It can be calculated by the height of the dendrogram where the branches of the clusters that contain the elements meet. The cophenetic correlation is used to assess how well the clusters imposed on the data fit the distances between those elements. The correlation varies between 0 and 1, with higher numbers reflecting a better fit of the model to the data. Studies on data sets simulated by Monte Carlo methods have shown that the cophenetic correlation coefficient can vary between 0.55 and 0.65 with (pseudo-)random data depending on the number of variables and data points (Rohlf and Fisher 1968: 408). In our case the cophenetic correlation coefficient of the SL varies from 0.45 to 0.60 over 1,000 simulations. We suggest that the height difference between the first and the second partition could serve as a basis for assessing the evidence for a morphology-syntax distinction. If the distinction was not motivated and elements were not clustering into two groups better than chance, as Haspelmath suggests that they might be, we would expect the height differences from the first and second partitions to be close that of the simulated language or the mean of the 1,000 simulated languages. Thus, we can assess how much evidence there is for a morphology-syntax distinction in a given language based on their position in relation to the simulated languages of our study. Figure 7 plots the first versus second partition height difference and cophenetic correlation of the 1,000 simulated languages and the languages of our study on the x and y axes respectively. We can see from this plot that all real languages have a higher cophenetic variation than all the simulated languages. This is to be expected because real languages have more groupings of morphosyntactic categories regardless of whether they display a morphology-syntax distinction. The situation is different when we consider languages in terms of the first versus second partition height difference (the y-axis). Five out of the eight languages of our study overlap with the distribution of SLs (Asheninka, Tariana, Cavineña, Puinave and Hup) on the x-axis. It is difficult to know what the strict cut off point should be for assessing whether a language has a morphology-syntax distinction, but distributional overlap with a set of random languages suggests a relatively weak division between morphology and syntax at the very least. Visual inspection of Figure 7 suggests that in Puinave, Cavineña and Hup, in particular there is not very strong evidence for a morphology-syntax distinction. These languages have first versus second partition height differences that are relatively close to the mean value of the SLs in the aggregate. Tariana and Asheninka are beyond the third quartile of the distribution of SLs (0.7933), and thus it is more likely that these languages display a morphology-syntax distinction.

Figure 7:

Cophenetic correlation and first/second partition height differences of 1,000 simulated data sets (black circles) and the languages of our sample (red dots).

Importantly, the languages vary in terms of how well distinguished they are from the SLs, with, Chácobo, Central Alaskan Yupik and Movima appearing with a more motivated distinction between morphology and syntax than the other languages. Chácobo, Movima, and Central Alaskan Yupik stand out in that they exhibit height differences outside the range of the simulated languages.

What these results suggest is that languages may vary in terms of whether they display a morphology-syntax distinction. Furthermore, we can measure the degree to which such a distinction is valid. We have, thus, moved towards providing a method that can engage with Haspelmath’s critique of the notion of fuzzy wordhood.

8 Conclusions and future research

This paper has provided an overview of three problems that a variationist perspective on the morphology-syntax distinction could address. The problems were illustrated and explored using different statistical methods. Correlation matrices can be used to provide an overall profile of associations between wordhood criterial variables. To the extent that strong positive correlations can be understood as reinforcing a more discrete distinction between morphology and syntax, correlations matrices can provide a good overview of linguistic variation in the extent to which morphology and syntax is being distinguished between two variables at a time. The differences in the associations between the variables suggest that some variables are more important than others and that languages might vary in this regard. Random Forest models provide a method for assessing this question, but they depend on assuming a dependent variable or base classification. Clustering methods do not require a dependent variable, but they require choice of a clustering algorithm, a clustering method and a validation technique. It was suggested here that the simulation of a (pseudo-)random data set that included variables of the same type and the same range of that of the languages of the study could be used to ground a clustering validation technique. We showed that some languages do indeed provide strong evidence for a binary partition of their data into morphological and syntactic constructions, but some do not. The fact that these hierarchical models still fit the latter languages better than chance invites the possibility that in these languages different types of global divisions are more important.

Throughout the study, we pointed out a number of problems that need to be overcome for variationist studies on the morphology-syntax distinction to proceed. The most obvious one is that a larger sample of languages with a larger sample of morphemes needs to be gathered in order to make empirically responsible generalizations. For the assessment of the composition problem some theoretically motivated baseline classification has to be given to the data for the relative importance of variables across languages to be appropriately tested. It is not clear that exponence complexity is the right answer in this case, because correlations between exponence complexity and other word variables were so low in the languages of our study. The architecture problem requires an assessment of the data that overcomes the ‘clustering tendency problem’. Visual inspection of the data are not enough. Rather the results of clustering models applied to different languages should be compared with some null or random distribution.

We think that an emphasis on the issue of global architecture might help bridge the gap between two broad ideas present in the literature on the morphology-syntax divide, one which emphasizes continuity and the other which emphasizes discontinuity.

On the continuity side, the grammaticalization literature emphasizes gradience between affixal and word-like elements (Hopper and Traugott 2003: 7; Traugott and Trousdale 2010: 25). However, it is not entirely clear why a language should display even a statistically motivated distinction between morphology and syntax in such a perspective. Should we not expect there to be just as many languages that are no better than random than not? In grammaticalization theory the emphasis on the history of individual elements or specific constructions typically occurs without an eye towards how such elements are embedded in a larger ecology of morphosyntactic groupings (e.g. Kuteva et al. 2019 and papers cited therein). We have established that elements and constructions can be placed on a cline from syntactic independence to morphologically integrated, but now, with the methods explored in this paper, we can move towards exploring the distribution of elements and constructions along this cline in a given language. On the discontinuity side, we find some discussions of morphological complexity emphasizing a lack of continuity between morphology and other domains and liminal elements are taken to be marginalia (see Section 1). Despite the fact that many of these researchers have been able to show that there are domains of grammar that display patterns distinct from combinatorial syntax and phonology, the idea that such patterns support an autonomous morphological representation remains a matter of controversy (Luís and Bermúdez-Otero 2016). Apart from adding more languages and data points to the sample, and developing a more rigorous sampling technique, future research will involve further developing the wordhood criterial properties, perhaps elaborating a less arbitrary and more theoretically motivated metric of exponence complexity, and exploring different types of clustering methods (both exploratory and validational). The upshot of such a project would be the development of a data base and a set of methodologies that could help in the development of more testable theories of linguistic architecture. Rather than simply emphasizing that the boundaries between morphological and syntactic structure (or whatever other domains we wish to propose) are fuzzy to some unknown degree, we could move towards developing theories that posit quantifiable constraints on the variation and (dis)continuity.