Iconicity and gradient at-issueness: insights and future avenues

1 Introduction

In our target article, we made the claim, based on experimental data on gestures and ideophones, that at-issueness is not a binary notion, but rather a gradient one. We showed how various factors can cause iconic enrichments to fall at different points along a scale of at-issueness. We then proposed a formalisation of this concept of at-issueness based on relevance to the Question Under Discussion (QUD). The responses to our article are, we hope, just the beginning of an interesting discussion not only on the nature of at-issueness and how to model this in formal semantics, but also on the role and contribution of a variety of iconic enrichments in natural language meaning. We would like to thank all the authors for their replies, which have brought us new insights into our research, suggested avenues for future research and raised issues in our current model of gradient at-issueness.

This article is structured as follows; in Section 2 we will discuss the insights into iconic phenomena that have been raised by the responses to our article, before discussing how best to model iconicity in the semantics in Section 3. In Section 4, we will move on to at-issueness and reiterate why we believe at-issueness truly is gradient, before finally discussing necessary refinements to our formalism in Section 5. Section 6 concludes the paper.

2 Insights into iconic phenomena

3 Modelling iconicity

Given that our approach can be applied to a range of iconic phenomena beyond co-speech gestures and ideophones, the next question to be addressed is how exactly we want to model this iconicity. Schlenker suggests that, while our analysis for gestures and ideophones is a useful beginning, future research should aim for a more explicit semantics of iconicity that goes beyond demonstrations and a similarity relation. Similarly, Laparle proposes that instead of simply aiming to integrate multimodal forms into a semantics of natural language, we should aim to reformulate semantics as a whole. While it is clear that iconic forms such as gestures and the iconic components of ideophones add meaning on top of spoken language, we are rather more tentative to categorise this iconicity as inherently linguistic. Instead, we propose that iconic forms and enrichments receive their interpretation compositionally from their combination with standard descriptive language and are not necessarily meaningful in isolation.

As evidence for this, consider the examples of iconicity in the lexicon of both spoken and sign languages, where this iconicity is not necessarily interpreted semantically. For example, Blasi et al. (2016) have shown that many words for body parts have iconic properties, such as the nasal consonant n in nose or the sign BOOK in American Sign Language (ASL), as in Figure 1. While both of these lexical items are transparently iconic, their iconicity is not necessarily relevant to the interpretation of their lexical semantics (see Emmorey 2001; Taub 2012 for discussion of the interpretation of iconicity in sign languages). Were we to argue for an explicit iconic semantics beyond demonstrations, then it would have to account for why iconicity is interpreted in some cases and not others. In our approach, however, we capture that it is the way that iconic forms occur alongside descriptive language that causes them to be interpreted iconically, with the interpretation of a gesture being fully dependent on its temporal alignment with the expression it co-occurs with. Similarly, the similarity relation between the utterance of an adverbial ideophone and the reported event is triggered compositionally by the utterance of the ideophone as an adverbial. As discussed in Section 2.2, this is not the case for ideophonic verbs such as platschen, although these contain potentially iconic components similar to those of ideophones.

Figure 1:

BOOK in ASL (Taub 2001).

4 Is at-issueness gradient?

Koev and Schlenker have suggested interpretations of our experimental results, which could indicate that the effects seen are not due to gradient at-issueness but rather other factors. Koev in particular discusses how considering at-issueness and truth values as gradient may be too severe and instead suggests that the results of our experiment can be captured with an approach that models the gradiency as being due to uncertainty around a speaker’s intention to address the QUD. However, this intuition is not necessarily entirely incompatible with our proposal. While we maintain that at-issueness is indeed gradient, we need not assume that truth values are. A critical point raised here is how to interpret speakers’ ratings of the iconic enrichments in matching and mismatching environments; are these ratings representative of truth values? Or are these rather something else? Kroll and Rysling (2019), for example, argue that we should distinguish between semantic truth and speakers’ evaluations of how appropriate and felicitous a given utterance is in a particular context. Perhaps our previous analysis of such ratings as reflecting the truth values may have been too strong and it would be better to follow Kroll and Rysling (2019) and assume that speakers’ evaluations are distinct from truth values. If we were to adopt this approach, then our experimental results should be reinterpreted as reflecting the speakers’ evaluations of the truth of the utterances in the given contexts, with the prediction being that these evaluations are influenced more towards a true judgement by at-issue content than by less at-issue content. This would of course also require a reformulation of our gradient truth value calculation. Alternatively, we could adopt an approach wherein truth values, as well as at-issueness could in fact be gradient.

Turning now to Schlenker, we would like to take the opportunity to address the explanations that he proposes for our experimental results, as we believe that this discussion will yield interesting insights. Schlenker provides the following potential reasons for the reduced mismatch effect seen for gestures and ideophones in comparison to standard arbitrary content:

1. Modality (gesture only): gestures can be ignored due to their occurrence in the visual modality.

2. Semantic failure: mismatching ideophones and gestures trigger some sort of semantic failure instead of falsity.

3. Underspecification: the relation between the gesture or ideophone and the expression it modifies is underspecified.

4. Imprecision, vagueness: gestures and ideophones are less precise than their adjectival or adverbial equivalents.

Firstly, we would like to address the third point of underspecification. Schlenker suggests that participants could have interpreted the circular gesture accompanying window as some sort of prototypical gesture, for example a window, the kind of object that’s typically round. However, there was a pre-experiment conducted prior to Ebert et al.’s (2020) main experiment, which established what were seen as prototypical gestures for the objects to be used in the experiment. This then informed the gestures used in the main experiment and prototypical gestures were not included as items in the final experiment. A similar pre-experiment was conducted before the Barnes et al. (2022) experiment to establish the best adverbial equivalents for the ideophones to be used in the main experiment. Here participants were shown the target sentences to be used in the main experiment and had to select the best adverbial to replace the ideophone. While this does not mean that the ideophones have exactly the same meaning as the adverbials, it indicates that participants believe they modify the sentences in a similar manner. Hence it seems unlikely that the results are due to an underspecified relation between the gesture and/or ideophone and the modified expression.

Turning to the remaining points, the question of whether we are seeing semantic failures in the experiment as opposed to falsity (cf. 2. above) relates to what we discussed above; are the ratings measuring gradient truth values or speaker evaluations of appropriateness? If we indeed argue that what is happening in the experiments is some sort of semantic failure then it is not clear what exactly the distinction between semantic failure and falsity is; i.e. how at-issue must a meaning contribution be for it to result in falsity rather than semantic failure? One potential approach would be to assume that content which is above a certain contextual threshold of relevance to the QUD will result in falsity, whereas anything below this will be a semantic failure. For the moment, if we assume that this is the case, then we propose that the remaining explanations given by Schlenker are not incompatible with our analysis, but rather are contributing factors as to why the iconic enrichment is less at-issue and therefore results in semantic failure rather than falsity in mismatching conditions. That is to say that an iconic enrichment occurring in a different modality compared to the main utterance (cf. 1. above) or being somewhat vague and imprecise (cf. 4. above) will result in that enrichment being less at-issue and therefore having less influence on speaker evaluations of truth.

While we have previously discussed the effect of modality on at-issueness, we have not yet discussed the role of precise versus vague meaning on at-issueness. Since writing the target article, we have conducted research into ideophones in Akan, a prototypical ideophonic language, where ideophones are used frequently in every day speech and are highly conventionalised, with highly specified meanings (cf. Agyekum 2008). This experimental work indicated that in contrast to German, adverbial ideophones in Akan are no less at-issue than standard adverbials (Asiedu et al. 2023). Our proposal is that through frequent use, the descriptive meaning component of ideophones in Akan has become more conventionalised and therefore the ideophones have more specified and precise descriptive meaning components compared to ideophones in German. As to why a more precise meaning would result in a more at-issue content, our tentative proposal is that a more specified meaning component will be more informative with respect to resolving the QUD and as such will be automatically more relevant to said QUD in comparison to other alternative propositions. Therefore, while the depictive component of the Akan ideophones remains non-at-issue, the descriptive component being more at-issue causes the ideophone to shift towards at-issueness. Steinbach has also suggested that a modality specific approach to iconicity may be necessary due to iconicity playing a role in the logical core of sign languages. We argue, however, that our Akan results indicate that this is not necessarily an effect related to modality, but rather due to the presence and frequency of iconicity in a language. This is in line with what Steinbach conjectures about the at-issue status of classifiers as opposed to idiomatic signs, namely that classifiers are expected to be less at-issue since the demonstrations used in classifier constructions are less conventionalised than those in idiomatic signs. But in general, similarly to how ideophones are more at-issue in a language where they occur much more frequently and are therefore much more conventionalised, iconicity may be more at-issue in sign languages due to it being a much more pervasive and commonly occurring part of the languages.

In summary then, our experiments demonstrate the gradient nature of at-issueness. The mismatch effect is smaller for German ideophones than German adverbials because speakers do not rate their occurrence in mismatching conditions as poorly as they do for adverbials, due to them being less at-issue than said adverbials. This lower at-issue status is likely due to their imprecise descriptive meanings, which make them less informative with respect to the QUD. Iconic co-speech gestures exhibit an even smaller mismatch effect as they are even less at-issue than ideophones, not only having a less precise meaning than their adjectival equivalents, but also occurring in the visual modality while the main utterance is in the spoken modality.

5 Refining the analysis

Having reiterated our commitment to at-issueness as a gradient property, we now turn to issues raised around our formalisation of gradient at-issueness. One question posed by Gutzmann suggests an interesting future avenue for research into at-issueness; defining what exactly at-issueness is. Gutzmann’s solution is to view at-issueness as propositional prominence, which we find a very interesting proposal and definitely agree that this could be an appropriate explanation. The prominence lending properties listed by Gutzmann certainly overlap a good deal with the properties which we have described for both gestures and ideophones. In particular, Gutzmann highlights that this understanding of at-issueness as propositional prominence provides a means to overcome problems with the appropriateness condition, repeated in (7).

Gutzmann points out that this condition makes incorrect predictions with respect to utterances where the matrix clause alone does not answer the QUD, but only when combined with the appositive, as in (8).

Here, the question posed by A is only answered via the combination of the matrix and appositive clause and is a perfectly acceptable sentence. However, the condition in (7) would predict it to be infelicitous.

On the other hand, Koev claims that the appropriateness condition is too weak – in a situation where matrix clause and appositive propositions are the only available possible alternatives, our account would predict a sentence as in (9) to be felicitous, no matter how irrelevant both are for the QUD (because the matrix proposition would be assigned a relevance of 1 in such a situation).

However, as Koev admits, this is only the case if the space of alternatives is extremely limited, which we might assume will not be the case in any communicative situation. But if it were the case, i.e. there are only two possible propositional options, a minimally relevant one and one that is not relevant at all, an utterance with the minimally relevant one as matrix clause assertion seems indeed acceptable, simply because there is no other viable alternative.

Gutzmann proposes two additional principles to replace the minimum appropriateness condition, based on viewing at-issueness as propositional prominence. These are as follows:

We think these new conditions combine well with the theory we originally developed and certainly seem to address most of the issues raised by Gutzmann and Koev wrt. the minimum at-issueness condition. Future research should definitely be conducted into at-issueness as propositional prominence, particularly given the overlap between prominence lending properties and properties of non-at-issue content.

However, one issue, which Gutzmann raises, cannot be resolved via these reformulations. According to Gutzmann, certain constructions simply cannot be used to address the QUD, no matter how relevant they are to said QUD, as in (12).

If Gutzmann is right and B’s reactions to A are indeed infelicitous,^[1] it is true that the minimum appropriateness condition does not account for this straightforwardly, but at this point neither does propositional prominence. As of now, we have no solution to offer for such problems, but one potential way to go would be to assume that certain constructions, such as certain types of exclamatives, are simply specified for low propositional prominence. Or alternatively, that exclamatives cannot answer an explicitly given QUD, due to their non-propositional nature and they hence have no relevance for it. And even if exclamatives are defined as constructions with a low threshold of relevance, their actual relevance in a given situation might be even lower and hence not high enough to make an exclamative a felicitous reaction to a given QUD.

Aside from the minimum at-issueness condition, Koev brings forth two main concerns with respect to our theory: 1) he takes issue with the way gradient at-issueness is formally spelled out in our theory, and 2) he questions the link between at-issueness and truth and hence the gradient nature of at-issueness. Wrt. the latter, he argues that at-issueness is binary and proposes to take an alternative path where gradience arises from the fact that a listener might be uncertain about a speaker’s intention of making certain parts of their utterance at-issue. He spells this out in an interesting proposal couched in the framework of Rational Speech Act theory (Frank and Goodman 2012). As we pointed out above, in Section 4, we believe that this analysis is not necessarily incompatible with our proposal.

Concerning his criticism of our formal implementation, Koev correctly captures our picture of how alternatives determine the relevance measure for individual propositions with his calculation of the value for r in his formula (5). But we would like to point out an apparent misconception that appears to inform his criticism of our formalism. Specifically, we do not introduce the measure in (5) “with the apparent goal of ‘normalizing’ van Rooy’s measure in (4)”, as Koev writes. Crucially, it is not r that enters our computation of the joint truth value T(u), as Koev mistakenly suggests in his reformulation of it (his (9)) and in the text following it. We repeat the formula for the computation of the joint truth value given in our target article in (13).

As can be seen, it is the normalization r ̃ of r that we use for the computation of the joint truth value in order to achieve the desired result, namely that “the joint (graded) truth value T(u) of the overall utterance u is the result of weighting the individual truth values of its component propositions in the actual world w* by their normalised relevance r ̃ (to range between [0, …, 1]) wrt. to the constructed QUD Q*” [our target article, emphasis added]. To be perfectly explicit, the normalization of r is defined as in (14), which is standard for normalisation and hence pretty much in line with the normalisation proposed by Koev (his (10)).

6 Conclusions

Overall, we believe that our article and these responses have initiated an important discussion on the nature of at-issueness, which could have important consequences for natural language semantics. The contribution of iconic enrichments in natural language is important for understanding how meaning can be contributed at different levels and in different manners within language and establishing the information status of such enrichments can not only help us to understand the difference between spoken and sign languages, but also the properties of non or less at-issue content that is not arbitrary. Conceptualising at-issueness as gradient also allows us to capture subtle differences between both iconic and arbitrary constructions and may be able to provide insight into discourse theories and management of the common ground. While our formalism likely requires further refinement and future research will be necessary to provide empirical support for the theory, we believe that this is a promising start.