A theory for words in Georgian: traditional constructs versus corpus annotation

3.1 Form, function, and consistency

Consistent tagset design is desirable for the sake of end-users, who must ultimately learn the tags. But consistency may conflict with other design requirements, such as the need to accurately model the language’s grammar. The consideration of annotating for form versus annotating for function epitomises this tension.

In any language with inflection, a POS tagset should classify words according to the inflectional categories they exhibit. Yet for a number of reasons (the expected polysemy of linguistic forms, syncretism across paradigms, etc.) a single morphological form may correspond to multiple (morpho-)syntactic functions. In descriptive or theoretical grammar, this phenomenon needs merely to be thoroughly explained where it occurs. But for POS tagging, that will not do: annotation requires a finite set of categories and criteria for whether any word is in or out of a given category. On one hand, disambiguating forms with multiple grammatical functions is part of what POS tagging is for in the first place (cf. the classic English example, noun/verb ambiguity). On the other, distinct categories for variant uses of one and the same form may be impossible to apply without reference to syntactic information (e.g. English’s present participle versus gerund distinction). Such multifunctionality must be distinguished from categorical ambiguity, the situation that pertains where it has been resolved that two categories are needed, but some forms may represent either category depending on context (Cloeren 1999: 47–48).

Simple homonymy aside, the easiest cases to decide are those where a formal distinction mapping to a functional distinction exists, but is absent for some lemmata in the class in question. For instance, some Georgian single thema verbs^[5] exhibit no distinction in form between the nominative verbal noun and the third person singular aorist, e.g. dats’era, either ‘to write’ (verbal noun^[6]) or ‘s/he wrote it’ (finite verb). However, most verb lemmata do exhibit a formal distinction corresponding to this functional distinction, implying a true difference in morphosyntactic category. Therefore the two forms should be tagged differently. Consistency then dictates applying that judgement also to the lemmata lacking the formal distinction. For verbs like dats’era, then, we tag for function, not for form. Consequently, annotation of dats’era and similarly conjugated verbs is ambiguous when considered out of context. Even English exhibits category definition issues of this kind (e.g. weak verb past tense versus past participle), but the exceptional complexity of Georgian morphology and morphotactics foregrounds such problems.

More difficult cases involve multifunctional forms where it must be determined whether or not the functional distinction should motivate distinct categories, and there is no comparable paradigm in which that distinction is marked morphologically. For instance, languages with an explicit copula ‘to be’ often use the same verb as an auxiliary, without formal distinction. Should that verb receive a different POS tag when it is an auxiliary? Consistent tagging for function would require that it should. A Georgian example is personal pronouns. Georgian lacks dedicated third person pronouns; demonstratives are used instead, so e.g. singular is translates ‘that’ but also ‘he/she/it’. This is not unusual crosslinguistically; demonstratives so used often grammaticalise into personal pronouns (Diessel 1999), as in Germanic and Romance. To ask whether a Georgian grammar should approach these items as demonstratives or as personal pronouns is over-simplistic. A descriptive grammar can have it both ways; Shanidze (1980: 41–43) includes is and related forms in his account of personal pronouns, but notes first and second person pronouns as “genuine” pronouns and third person pronouns as simply a function of certain demonstratives. But this approach, tailored to support human understanding, is problematic for a POS tagset, where caveating whole categories as non-“genuine” is not possible. A Georgian tagset can have either a single group of tags for demonstratives (differentiated by case etc.), or two groups, one labelling is etc. demonstratives and one labelling them pronouns.^[7] There is no halfway position.

These two possibilities map directly to tagging for form (one tag per demonstrative) and tagging for function (two tags for two functions). Consistency would favour tagging for function, as per ambiguous forms like dats’era. We can imagine end-users wishing to query tagged text for only pronominal, or only demonstrative, uses of is etc. A reason not to be consistent here is the practical consequences: adding this distinction will impose ambiguity onto every token of is etc. Of course, we cannot wish away ambiguities; sometimes a form’s category being underspecified is just a fact which taggers must deal with. Yet adding ambiguity to a set of frequent forms inevitably degrades performance (since disambiguation is never perfect). Informally: sometimes we must live with ambiguity, but we shouldn’t inflict it on ourselves if we can avoid it.

Beyond practicalities, tagging according to function may imply adoption of a specific theoretical framework. Consider again copula/auxiliary status of ‘to be’ verbs. English grammars commonly distinguish copula and auxiliary be, but some scholars (e.g. Payne 2011: 266–268) argue that copula be is indistinguishable from an auxiliary. The same is argued for Arabic. Though kāna ‘to be’ is commonly described as able to be either copula or auxiliary, some (e.g. Holes 2004: 233) analyse it as a pure grammatical marker of anteriority, even in “copula” contexts. Assigning distinct POS tags to the ‘to be’ verb’s two uses in English or Arabic (tagging for function) thus implies the theoretical stance that these verbs’ nature is not as Payne or Holes argue. The aforementioned Georgian third person “pronouns” are a parallel case. By contrast, tagging for form in these cases does not imply that the distinction does not exist, since any category may exhibit functional polysemy. The (explicit or implicit) implication is rather that the demonstrative/pronoun distinction (or copula/auxiliary, etc.) is within some other domain than POS tagging (e.g. syntactic or discourse parsing), since any disambiguation of the functions must take place at one of those levels.

Given all this, need we always avoid tagging for function in such cases, despite the inconsistency with applying that principle elsewhere? We are loath to make that argument. A carefully structured tagset can allow functional distinctions to be neutralised in practice. If third person pronouns and demonstratives are treated as pronoun subcategories (e.g. P3P and P3D within P3, alongside P1/P2) it is easy to search for both using a wildcard (P3*) or regular expression (P3[PD].*). Users who want the distinction can access it; users who do not can ignore it; thereby, theoretical decisions in the tagset design are not imposed upon users, only made available. Moreover, the case for including a particular non-formalised distinction is stronger if that distinction is established in general understanding of the language, such that tagset users are likely to expect it to be present. There is then no general answer here: tagset designers must evaluate advantages and disadvantages on a case-by-case basis before deciding on formal or functional category definitions. Consistency is not the only relevant consideration.

Ultimately, we opted not to include distinct pronoun/demonstrative categories for is etc. The literature treats the distinction as marginal (not “genuine” for Shanidze; for Gogolashvili et al. 2011: 169–173 “‘third person’ is ‘non-person’”); clearly it is not so well-established that its omission will confuse. Therefore, the hard-to-resolve ambiguity it would cause weighs more heavily. These considerations dictate defining only one category for is etc. But the issue needs to be fully and publicly documented, or else users may not grasp that what they know as personal pronouns are tagged as demonstratives.

The most difficult of these problems are when a functional distinction without formal marking cuts across major categories. Consider Georgian’s noun/adjective distinction. Adjectives typically premodify a head noun, but can postmodify, possibly with other elements intervening. They may also function predicatively with a copula, or be used independently as nominal heads. Modifying adjectives inflect for number and case of the head. All this is not unusual crosslinguistically. The question we face is whether Georgian adjectives are a distinct POS from nouns at all. In a grammar for human readers, this question is not crucial. Nouns and adjectives can be distinguished by prototypical semantics (entities versus properties), and readers instructed on the shared formal properties. But in a POS tagset, defining two categories with indistinguishable formal behaviour on semantic grounds is unsatisfactory. Given, say, a newly coined word (so a tagger can have no built-in information on it) that is marked by ergative -ma, out of context there is no way to know if it is noun or adjective, since both can take -ma. But there is no way to know in context either, since syntactic positions diagnostic of ergative nouns are equally open to ergative adjectives. (Non-novel forms can be classified as adjective or noun by human input, but again only via semantics.) So should there be a single noun/adjective category?

This form-versus-function issue resists simple solution. In the end, two points resolved us to retain the adjective category. First, the Georgian grammatical tradition is quite clear that it exists. Furthermore, there is a slight formal distinction. Postmodifying adjectives inflect for case and number like nouns. However, premodifying adjectives exhibit only a subset of these inflections (plural is not marked, some case suffixes are reduced); are unaffected by suffixaufnahme (see Section 3.4); and, if their root is vowel-final, appear with no suffix at all, regardless of the head’s inflection. These factors justify the noun-adjective distinction. But for this to work in practice, noun/adjective ambiguity cannot be allowed, as it would be all but unresolvable. Therefore, a tagging lexicon must never class any form as ambiguous between noun and adjective; only unknown forms may be ambiguous. Hence the tagset cannot allow for noun-to-adjective/adjective-to-noun category conversion (“zero derivation”); an adjective very commonly used as a nominal head always must be tagged as adjective.

The same applies to other categories with nominal inflection: pronouns, participles, verbal nouns and denominal/deadjectival adverbs. In all but the last case, we arrived at the same conclusion as for adjectives, and tag for function not form; we return to adverbs below. As consideration of form versus function moves to broader distinctions, it becomes enmeshed with the wider issue of the model of the grammar that the tagset expresses.

3.2 The model of the grammar

POS tagging is an act of categorisation; as such it requires a structured schema of the phenomena under study. That schema should be motivated, that is, founded in a coherent conceptualisation of the domain. A POS schema’s motivation is some model of the grammar of the language being tagged: a TFGM (see Section 1). The statistician’s motto “all models are wrong but some are useful” (Box 1979: 202) applies here. A grammar is not equal to the whole of a language; it simplifies and approximates, as any finite model of an infinite domain must. In outlining a TFGM, then, what counts is how useful the model is for a given purpose. Usefulness for a POS tagset need not correspond to usefulness for other purposes. As we saw, the purpose of existing Georgian grammars is to support human understanding of the structure of Georgian, not POS tagging. In this section, we consider distinct qualities needed by a TFGM for Georgian, focusing on aspects that are more or less alien to other grammars. We seek, in other words, to develop not a theory of words but a theory for words: a conceptual structure suitable for implementation in token-level annotation.

This issue cuts across all others in tagset definition. It emerged at multiple points in the foregoing discussion of form versus function: in re demonstrative versus third person pronoun, copula versus auxiliary, and adjective versus noun. In each case, we noted reasons why a category asserted in existing grammars might be best omitted from a TFGM, and discussed the rationale for our decision in each case. We also observed that a TFGM may need to differ from other models because in POS tagging, in principle every token must be able to receive a single analysis, whereas some ambiguity is acceptable or even welcome in accounts for human readership.

In what other ways must a TFGM differ from other models?

3.3 Tokenisation and treatment of clitics

Clitics present a challenge for POS tagging. They have the function of a word, but lack phonetic independence; in writing they may (but need not) appear attached to the host word without space division, hyphenated or unhyphenated. If we wish to tag clitics as per their function as words, a token division must be added to separate clitic and host. Other strategies have been tried; some English tagsets have genitive noun tags for words with =’s, to avoid treating the enclitic separately. But two arguments support the clitic-separation approach. First, many clitics alternate with full words (e.g. English not/=n’t). Consistency of tagging requires both be treated alike; only clitic separation makes this possible. Second, some units are written separately but demonstrably cliticised in speech (e.g. cannot carry stress). To analyse such clitics as part of host words, orthographically separate tokens would have to be joined together. Few analysts would support that procedure. Thus treating clitics separately requires less modification to naïve, whitespace-based tokenisation. But it introduces difficulty for users: finding e.g. didn’t in tagged English text means searching for did n’t, and users have to remember this. We consider this justified by the inconsistency avoided. But we acknowledge the downside.

Georgian is rich in enclitics, most written attached without hyphenation. Attempting to treat clitics as parts of hosts would thus have an additional disadvantage: the number of POS subcategories per lemma drastically increasing, because each clitic tokenised with its host doubles the host’s possible inflectional forms (one form per inflection with the clitic plus one without). Likewise the possibility of clitic sequences would necessitate yet more subcategories. Clearly separate treatment of enclitics is essential for a Georgian TFGM.

The categories most often cliticised are postpositions (see Section 3.4), particles (see Section 3.2), and the copula. The copula is a full, albeit irregular, verb. However, its third person singular present form aris alternates with enclitic =a. This alternation further evidences the need for clitic separation in Georgian. Being identical in grammatical function, aris and =a should be tagged identically, so two clauses differing only in use of aris versus =a can receive the same string of POS tags.

3.4 Case, suffixaufnahme, and postpositions

Traditionally (Shanidze 1980: 44–108), seven cases are identified for Georgian, though both enumeration and naming are somewhat problematic: nominative, ergative, dative, genitive, instrumental, adverbial^[10] and vocative. The case suffix morphophonology is complex, but beyond our scope here. But discussing Georgian case does require the concept of alignment,^[11] the comparative treatment of intransitive subject (S), transitive subject (agent-like argument A), and object (patient-like argument P). When arguments are case-marked, the A and P are treated distinctly: both appear in transitive clauses, so ambiguity must be avoided. The S is not subject to this confusion, so may share the marking of either A or P. The two simplest systems, then, have two cases for core arguments: (i) an S+A case (called nominative) and a P case (accusative); (ii) an S+P case (absolutive) and an A case (ergative). These alignments are named respectively nominative-accusative and ergative-absolutive. Other patterns include split ergative, where a clause’s alignment depends on tense/aspect or argument animacy; and active, where the S’s treatment depends on the semantic role the verb implies.

Georgian exhibits both ergativity split by tense/aspect and active alignment.^[12] The traditional case names do not reflect their roles in the alignments. “Nominative” is both the S+A case and the S+P case, and thus better dubbed nominative-absolutive. Meanwhile, the P case is traditionally called “dative” rather than “accusative”, as it has additional functions normally associated with dative case (e.g. marking recipients) and emerged as a syncretism of a separate accusative and dative (Skopeteas et al. 2012: 148); it is better dubbed accusative-dative.

Three case-related phenomena raise problems for our purposes: zero case, postpositions, and suffixaufnahme.

3.5 The complex morphology of Georgian verbs

The key issue in modelling Georgian verbs is exhaustiveness versus abstraction, due to the high number of categories which various affixes may express on any verb. If a TFGM aims to capture these features exhaustively, the result will be a very fine-grained schema with very complex tags. Such a schema would, however, do little to abstract across that diverse array of inflectional categories; the importance of abstraction in POS annotation was discussed earlier. Conversely, abstracting away some detail to get a tractable category schema definitionally makes an analysis less than fully exhaustive.

Traditional Georgian grammars unquestionably favour exhaustiveness (as do systems like Lobzhanidze 2013; Meurer 2007: see Section 2), aiming to capture every feature expressed in verbal morphology and exhaustively catalogue formal variation. Our model instead favours abstraction: establishing categories for only the most important grammatical features and excluding distinctions arising from derivation (to be dealt with, if need be, at the level of morphological annotation). To illustrate this approach, we overview Georgian verbs as traditionally described, before arguing that practical simplifications are needed for a TFGM.

Georgian verbs are marked for voice, aspect, tense, mood, and agreement for person and number with two nominal arguments. But no straightforward association exists between any feature and a single morpheme or “slot” in the verb’s morphological template. The inflection/derivation boundary is blurred. One marker may indicate both a grammatical category and a derivational semantic change, either at the same time, or depending on context. For instance, voice is expressed partly by so-called version prefixes, but sometimes these instead code unpredictable semantic shifts; moreover, voice is also marked suffixally. Tense/aspect is coded partly by presence or omission of a preverb (prefix expressing direction), partly by other affixes. Agreement is marked by prefix/suffix combinations, where some markers sometimes reflect the first argument and sometimes the second. Often, the morphology encoding these categories varies by transitivity class and unpredictably by verb root.

Traditional grammars (Gogolashvili et al. 2011: 266–634; Shanidze 1980: 163–530) consider three verb features inflectional (person agreement, number agreement, screeve), and six derivational (direction, orientation, aspect, voice, version, contact). A screeve (mts’k’rivi, lit. ‘row’) is one of eleven enumerated “tenses”, i.e. tense-aspect-mood combinations; each screeve consists of the full set of agreement forms of that tense-aspect-mood. Traditional approaches often treat voice, direction, version etc. as forming paradigms despite being derivational, this being one reason for verbs’ exception-riddled complexity. But that traditional insight suggests an appropriate way to simplify verb classification. Only agreement and screeve are predictable, productive, and paradigmatic: a finite verb^[15] expresses exactly one screeve and one agreement pattern. Other features may be present, absent, or multiply present, with limited predictability. Thus, we model verbs as classifiable by screeve and agreement alone.

The screeves are organised into four sets; two sets, Present and Future, are grouped as a series; two others are series on their own. Table 2 lists common terms for sets and screeves (translated where need be). Though terminology reflects it only in part, sets and screeves form a simple aspect/tense-mood matrix. Each series is an aspect, respectively imperfective, perfective, and perfect (or, according to some recent work e.g. D. Melikishvili 2014, resultative/evidential), with imperfective doubled as present and future. Then, each set has a (relative) present, (relative) past, and subjunctive, except that the perfective has no relative past. While this analysis appeals for simplicity and symmetry, our model does not employ it, due to its potential unfamiliarity to users. Rather, it lists the screeves using traditional terms (Aorist Subjunctive is, exceptionally, our own suggestion) and no further organisation.

Traditional accounts also catalogue how agreement targets differ across screeves (due to aspect affecting alignment). We believe this can be disregarded in a TFGM.

Two sets of agreement markers exist,^[16] traditionally described as agreeing with subject and object. Because this actually depends upon the clause’s alignment, we suggest the terms v-agreement and m-agreement for the two sets, based on the first person prefixes. V-agreement is normally but not always subject agreement, m-agreement normally object agreement; Table 3 illustrates both. However, when an agreement pattern logically implies use of two prefixes together, e.g. v- + g- for first singular subject + second singular object, in fact only one occurs (g- in this case). Similarly, -t can pre-empt other suffixes. Prefixes mostly express person and suffixes mostly number, but not always. Rather than try to disentangle all this, we focus on distinguishing the functional agreement categories, not the actual affixes.

Three persons and two numbers for two nominals means (3 × 2) × (3 × 2) = 36 logical categories (verbs without objects get third person m-agreement, and thus motivate no additional categories). But actual morphology does not exist for all these. The eight combinations of agreement where v- and m-agreement are either both first or both second person cannot occur (this would require, e.g., prefixing both m- and v- at once, which is impossible). Because -t marks plural for both first and second person in both v- and m-agreement the pair g-…-t can express any of first singular + second plural, first plural + second singular, or first plural + second plural. -t further marks third person plural in m-agreement, so that v-…-t may be first singular + third plural, first plural + third singular, or first plural + third plural. Likewise third plural + second singular and third plural + second plural are indistinct. Functionally, third singular m-agreement may be used for plural entities, depending on their semantic role (e.g. number may be neutralised for non-applicativised patientive objects but not applicativised recipient/beneficiary objects). There are, in sum, 19 patterns, 8 of which represent multiple paradigm positions. Finally, some screeves of some verbs exhibit additional syncretisms, e.g. gzrdit ‘raise’ is marked by g-…-t and thus has the aforementioned three-way ambiguity, but the same form is also third singular + second plural ‘s/he raises you (p)’. However, not all verbs, or all screeves of this verb, exhibit this syncretism.

Three major questions arise regarding agreement. First, should a POS schema distinguish subject versus object agreement, or v- versus m-agreement? The former implies that a verb marked by, say, v- should be tagged according to what it actually agrees with: a first person subject in most cases, a first person object in clauses whose alignment so dictates. The latter implies unambiguous tagging of such a verb as having first person v-agreement. This is a form-versus-function question, easily answered: the ambiguity of subject versus object agreement could not be resolved automatically, since identifying subjects and objects^[17] belongs to a different annotation level (parsing), plus clauses can lack explicit subject/objects. Thus, the TFGM classifies each verb according to the combination of v- and m-agreement it expresses. Two verbs with first singular v-agreement and third person m-agreement receive the same analysis even if one has a first singular subject and the other a first singular object. This implies that voice marking that changes a verb’s agreement target does not affect its agreement categorisation. The second question is whether all 36 logical categories should be present in the schema, or just the distinct 19. For consistency with issues considered earlier, we take the latter approach. Even then, the number of categories is high (19 agreements × 11 screeves = 209), though many are rare (only 77 occur at all in a manually-tagged 100,000-token sample). Third, should the neutralisation of third person object number be modelled by the schema? This last is easily answered: a third person singular verb inflection is still a third singular form, and should be tagged as such, even if the target is plural, since the mismatch is purely functional-semantic.

We found the terms v-/m-agreement useful above, but users will not know them. Tag definitions therefore refer to subject/object agreement, but mean v-/m-agreement. Table 4 lists the categories, with labels and examples; for ambiguous forms, one reading’s label is given, and others listed separately. (Tags use shorter forms of the labels.)