Discourse connectives and their arguments: an experiment on anaphoricity in German

1.1 Structural versus anaphoric connectives

We focus specifically on adverbial connectives. Here, the common pattern of realization is:

[external argument] [internal argument with connective]

Mostly, there is a sentence boundary between the two arguments, and the internal argument consists of the single sentence that contains the adverbial connective. The external argument can be a sentence, but can also be longer or shorter. Furthermore, sometimes there may be intervening material between the two arguments:

[external argument] [other] [internal argument with connective]

This situation prompted Webber et al. (2003) to explicitly distinguish structural and anaphoric connectives. The former provide a syntactic link between adjacent clauses, as with preposed adverbial clauses (ex. (3)) and with paired coordinating conjunctions (ex. (4)):^[1]

Webber et al. point out that structural connectives do not admit crossing of their predicate-argument dependencies:

In contrast, adverbial connectives do allow crossing dependencies, and are thus seen as anaphoric, selecting their antecedent from the context in the same way as event anaphors do.

The combination of structural connective (because) and adverbial (then) allows attaching the discover event (d) temporally following the ordering event (b), while the causal relation holds between the discovery (d) and the cancellation (c).

In German, the same dichotomy exists, and Pasch et al. (2003: 522) also call the relation between connective and external argument anaphoric. Furthermore, certain German adverbials contain an explicitly-anaphoric morpheme, e.g., da raufhin (‘thereupon’) or trotz dem (‘however’/‘nonetheless’), while others do not, e.g., später (‘later’) or dennoch (‘however’/‘nonetheless’). Stede and Grishina (2016) reported that 79 connectives (29%) from the lexicon ‘DiMLex’^[2] have an anaphoric morpheme (prefix or suffix).

Here is another example from Webber et al. (2003), illustrating ‘long-distance arguments’:

So far, not much is known about the conditions under which readers prefer a long-distance over an adjacent argument, for instance about the discourse function of the intervening material (e.g., parenthetical, as in ex. (7)) and about the distance admissible for an argument to be picked up. In the absence of appropriate annotated German corpora, these questions are to be tackled by carefully designed experiments.

1.2 Goals and structure of the paper

We take an experimental approach (specifically, ‘crowdsourcing’) to obtain insights about the anaphoric behavior of German connectives. In particular, we study the connection between readers’ readiness to accept a long-distance argument and its semantic ‘likelihood’. Thus we vary semantic conditions, and determine readers’ preferences on argument selection by testing inferences triggered by a concessive connective. In the test items, participants mark those statements (a to d) that they understand as being entailed by the text:

From the answers we infer what external argument the participant has constructed for the connective. Our assumption is that normally, readers prefer adjacency of external argument and adverbial connective, but that this preference can be overridden by semantic features making the adjacent sentence less likely to function as external argument.

In the following, we discuss related work on the anaphoricity of connectives and pronominal abstract anaphora (Section 2). Then we introduce the rationale of our approach and our specific hypotheses (Section 3), present our experiment (Section 4) and results (Section 5). Finally, we discuss the outcome (Section 6) and draw conclusions (Section 7).

2.1 Connectives and anaphoricity

The largest text corpus annotated with connectives and their arguments is the English-language Penn Discourse Treebank or PDTB (Prasad et al. 2008), which contains texts from the Wall Street Journal with a total of 18,459 connectives. One thousand six hundred and sixty six of the 18,459 external arguments (9%) are in a non-adjacent previous sentence, and 736 of those 1,666 connectives are adverbials. Lee et al. (2008) computed some statistics of the syntactic types of external arguments. Prasad et al. (2010) found that paragraph boundaries are a strong barrier for external arguments of adverbials to stretch across, unless the sentence with the adverbial is paragraph-initial.

A related project is the discourse-level annotation in the Czech-language Prague Dependency Treebank^[3] whose size is similar to PDTB. Regarding long-distance connective arguments, Poláková and Mírovský (2019) reported that 5,455 relations range over more than one sentence, and for 11.7% of inter-sentential relations (3.5% of all coherence relations) the external and internal arguments of a connective are non-adjacent. Manual analysis of 245 instances of frequent connectives with non-adjacent arguments revealed two types of material:

1. The external argument provides a general statement/claim; the intervening material is an elaboration (e.g., detail or background information); the internal argument is strongly linked to the topic of the external one.

2. The intervening material constitutes a digression, which can be marked as parenthetical by brackets or dashes.

Further, the study by Poláková and Mírovský is to our knowledge the only one that looked for a potential effect of explicit anaphoric morphemes in antecedent selection (in the Czech data), but no evidence was found.

For German, there is PDTB-style data in the Potsdam Commentary Corpus (Stede and Neumann 2014), but with roughly 1,100 connectives it is too small to study aspects of argument distance.

2.2 Pronominal abstract anaphora

For the related phenomenon of pronominal event anaphora, a comprehensive survey is provided by Kolhatkar et al. (2018). Corpus-based work on German is to our knowledge limited to Dipper and Zinsmeister (2012). In parliamentary debate text, they annotated 225 instances of abstract anaphors (demonstratives das/dies ‘this’ and personal pronoun es ‘it’). For the antecedents, they used a set of semantic types, which are potentially relevant for resolving the anaphors, as their role in the verb complex may trigger semantic selectional restrictions on the antecedent; this is an important difference to anaphoric connectives.

Of the three pronouns, das is the one that most frequently occurs in adjacency to its antecedent, but for all pronouns there are many instances of non-adjacency (exact numbers are not given in the paper).

2.3 Other features of connectives

One of the factors we investigate in our experiment is the role of subjectivity in the statements joined by the connective. So far, the subjective/objective parameter has been studied predominantly for the connectives themselves, such as the difference between Dutch want and omdat (Canestrelli et al. 2013) or similar contrasts in Spanish (Santana et al. 2021). The influence of non-/subjective contexts on a single connective, on the other hand, has received little attention. Likewise, we are not aware of earlier work that investigated the tradeoff between the semantic plausibility of an external argument versus its linear distance from the connective.

4.1 Design

Every participant received a questionnaire containing eight items that we are interested in, and eight fillers. Items and fillers are short texts of two or three simple sentences,^[6] followed by three statements that are possibly implied by the text meaning (see Supplementary Material for a complete list of experiment items). The participants were instructed to carefully read the text and mark those statements that they could infer from the content of the given text, without relying solely on their world knowledge or opinions. As the three options are semantically not mutually exclusive, participants could select as many as they wished out of the three, or opt for ‘none of the specified options applies’. The instructions came along with an example of a text with possible inferences and explanations (see Supplementary Material).

Items in individual questionnaires come from a pool of eight item groups of four items each, i.e., sets of items that cover the same topic, but vary the parameters we study. One parameter concerns the connective with/out anaphoric morpheme (H3): Four of the eight item groups contain trotzdem, the other four dennoch. For logistical reasons, the study was split into two experiments, henceforth exp1 and exp2. Both contained the same items except that we swapped the occurrences of trotzdem and dennoch between the experiments.

To study the factuality parameter (H2), we used two types of response statements in our experiment conditions II, III and IV: In half of the item groups – two with trotzdem and two with dennoch – the response statements b, which represent an inference from the second sentence and the concessive connective, were conceived as facts (‘Cabrera’, ‘Mexico’, ‘Parasol’, ‘Vaccine’; see Supplementary Material), whereas in the other half, b expressed a subjective opinion (‘Barbecue’, ‘Birthday’, ‘Farm’, ‘Sea’). To assess our third parameter – the tradeoff between semantic plausibility and anaphoric distance (H1) – we formulate the second sentence of the short texts in variants that make it semantically more or less likely to be regarded as part of the external argument (whose core is always the first sentence).

An example of an item group (‘Cabrera’) is given in examples (11)–(14) below. The first item of each group (11) contains two sentences: a sentence with the external argument and the trigger sentence starting with trotzdem/dennoch. We use this kind of item (adjacent external argument and connective) as a control to test whether the candidate inference induced by the concessive connective can actually be drawn by participants (condition I). Texts in items (12)–(14) comprise three sentences: the same two as in condition I plus an intervening sentence in between. Readers can regard this intervening sentence either as part of the external argument of the connective or not. We vary the degree of semantic incompatibility between the intervening sentence and the sentence with the connective: In (12), the intervening sentence coheres easily with the trigger sentence (condition II); in (13), the relation is less plausible (condition III); and in (14), the compatibility is the lowest (condition IV). Taken together, these items test how an inference based on a long-distance argument is drawn in the presence of varying amount of intervening information between an anaphoric connective and its antecedent.

The options offered to participants (a to d) for judging plausible inferences were selected for the four conditions as follows: a is the target inference (based on the adjacent argument in condition I, but on the long-distance argument in conditions II – IV); it stays unchanged throughout the group. Option b is either an invalid inference (condition I) or represents a statement implied by the respective intervening sentence, i.e., corresponds to adjacency between the connective and external argument (conditions II – IV). For this statement, we contrast versions of items with a factual statement on the one hand, and a subjective opinion on the other, in order to test our hypothesis on the role of subjectivity (H2).^[7] Option c serves as distraction material that is kept unchanged throughout the group. Option d can be selected if none of the above seems applicable.

4.2 Technical implementation

We recruited participants via the crowdsourcing platform Prolific^[8] and implemented our questionnaires using _magpie.^[9] The condition number (I – IV) from each group was assigned to the participants automatically at random, and the order of presentation of the items and fillers was shuffled.

For both exp1 and exp2, we initially collected judgments from 100 participants per experiment. After a manual check of all submissions, we rejected those with low-effort results, e.g., no correct Prolific ID submitted or very short completion time (under 4 min for reading the instructions and judging all items). Those participants were automatically replaced by Prolific with new candidates.

As a further measure of quality control, we used MACE (Hovy et al. 2013) to detect unreliable participants on the basis of the content of their submissions.^[10] We excluded 33 submissions with MACE score below 0.5, which left us with 78 submissions in exp1 and 89 in exp2. The resulting number of judgments per item ranges from 12 to 23 in exp1, and from 13 to 34 in exp2, due to random assignment of items to participants and MACE filtering.

5.1 Condition I

In exp1 (Figure 1), the target inference (a) was selected by the majority of participants (>60% in the prevailing number of items). The invalid inferences b and c, on the contrary, were selected only occasionally (‘Farm’, ‘Vaccine’ and ‘Parasol’).^[11] This demonstrates that in general our candidate inference induced by the concessive connective was recognized well by the readers. However, option d (no inference applies) was selected quite frequently, especially in the subjective items (ca. 40–70% per item). We regard this as a first pointer to the fact that material presented as subjective opinion poses a greater challenge for the reader, thus generating more heterogeneous judgments compared to facts.

Figure 1:

Results for condition I in exp1 across the item groups: a (target inference), b and c (distraction material), d (no inference selected). Factual items are marked in bold in this and all subsequent figures.

The results for exp2 (Figure 2) show a similar pattern, although its items project somewhat greater overall uncertainty – option d is more frequent in all but two items compared to exp1. In one half of the items the target inference predominates (>50% per item). In the other half, it is either a tie between the options a and d (‘Parasol’) or the prevailing majority of participants did not consider any of the offered inferences to be valid. A statistical model fitted to the relevant subset of the experimental data, i.e. condition I, revealed a significant preference for the target inference over option d in exp1 (Est. = 1.104, SE = 0.235, Z = 4.696, p = 0.0). However, no significance was determined for exp2 (Est. = 0.261, SE = 0.209, Z = 1.250, p = 0.211). Full models can be found in Appendix B (Table 2).^[12]

Figure 2:

Results for condition I in exp2 across the item groups: a (target inference), b and c (distraction material), d (no inference selected).

The generalization that factual items appear more straightforward than subjective ones is confirmed by the models reporting a significantly higher probability of selecting the target inference in factual than in subjective items in exp1 (Est. = 1.164, SE = 0.417, Z = 2.789, p = 0.005) and exp2 (Est. = 1.257, SE = 0.453, Z = 2.774, p = 0.006); for the full models see Appendix B (Table 4). The other way around, option d is significantly more frequent in subjective than in factual items in exp1 (Est. = 1.533, SE = 0.626, Z = 2.449, p = 0.014) and in exp2 (Est. = 1.344, SE = 0.407, Z = 3.30, p = 0.001); for the full models see Appendix B (Table 5).^[13]

5.2 Condition II

The results for condition II (high coherence of the intervening sentence with the trigger sentence) make the contrast between factual and subjective items even more prominent (Figures 3 and 4). Both the target (a) and adjacent (b) inference were selected in all items (except for ‘Farm’ in exp2); they both prevail in factual items, whereas in the subjective items option d (>60% per item) predominates. Furthermore, as in condition I, exp2 seems to be associated with higher uncertainty compared to exp1.

Figure 3:

Results for condition II in exp1 across the item groups: a (target inference), b (adjacent inference), c (distraction material), d (no inference selected).

Figure 4:

Results for condition II in exp2 across the item groups: a (target inference), b (adjacent inference), c (distraction material), d (no inference selected).

In exp1, in three out of four factual items, most participants preferred the target inference over the adjacent inference, although this difference is rather minor. In exp2, the picture is similar. In both experiments, ‘Parasol’ triggered the adjacent inference noticeably more frequent than the target inference, but this item sets itself apart from the rest, as mentioned above.

The probability of selecting the adjacent inference is significantly higher in factual than in subjective items in exp1 (Est. = 2.694, SE = 0.684, Z = 3.936, p = 0.0) and exp2 (Est. = 2.641, SE = 0.852, Z = 3.099, p = 0.002); for the full models see Appendix B (Table 3). Since in this condition, option b is inferred from the coherent intervening material, its frequent selection is not surprising. Interestingly, its presence did not prevent the participants from making the target inference; rather, they deemed both implications possible.^[14]

5.3 Condition III

In this condition (less plausible relation between the intervening and trigger sentence compared to condition II), all items in exp1 (Figure 5), except for ‘Barbecue’ and ‘Farm’, show a sharp reduction in the frequency of the adjacent inference compared to condition II. The same holds for exp2 (Figure 6), except for ‘Farm’. This is in line with our prediction that an inference based on the long-distance argument becomes more apparent in the absence of a plausible adjacent alternative. Similarly to previously discussed conditions, the gap between the factual and subjective items is apparent here as well.

Figure 5:

Results for condition III in exp1 across the item groups: a (target inference), b (adjacent inference), c (distraction material), d (no inference selected).

Figure 6:

Results for condition III in exp2 across the item groups: a (target inference), b (adjacent inference), c (distraction material), d (no inference selected).

5.4 Condition IV

In condition IV (lowest compatibility between the intervening and trigger sentence), we observe a clear case of target inference drawing in the items ‘Cabrera’ and ‘Vaccine’. It was chosen almost exclusively (>95%) in exp1 (Figure 7) and in >75% in exp2 (Figure 8), whereas the adjacent inference is absent completely. The items ‘Mexico’ and ‘Parasol’ show rather mixed results, but the target inference prevails over the adjacent inference there as well. In the subjective items, the target inference predominates over the adjacent inference for the most part, especially in exp1. However, in general, option d is most frequent.

Figure 7:

Results for condition IV in exp1 across the item groups: a (target inference), b (adjacent inference), c (distraction material), d (no inference selected).

Figure 8:

Results for condition IV in exp2 across the item groups: a (target inference), b (adjacent inference), c (distraction material), d (no inference selected).