The colexification of vision and cognition in Mandarin: controlled activity surpasses uncontrolled experience

2.1 Classification of vision verbs: the control in a perceptual event

Cross-linguistically, there are three basic types of vision verbs, i.e., look₁, see, and look₂, with English being the metalanguage (cf. Viberg 1983, 2001). Differing from the first two types, look₂ represents a link verb, e.g., “the flowers look beautiful”. Following Evans and Wilkins (2000: 555), we do not classify the copulatives, such as 看起来 kàn-qǐlái and 看上去 kàn-shàngqù, into a distinct type, but rather as the “source-based” derivatives. Therefore, the focus of our study lies in the look₁-see dichotomy, where a variety of criteria have been adopted in previous research. Some representatives are listed in Table 1.

As shown in Table 1, there are basically three approaches to classification. First, many scholars attempt to classify vision verbs according to their visual event properties, such as Scovel’s (1971) classification of “activity” and “state” and Viberg’s (1983) far-reaching typological studies based on “activity versus experience” classification. The second approach emphasizes the semantic roles of arguments. For example, Quirk et al. (1985) state that there are agentive and non-agentive vision verbs, and Gisborne (2010) classifies them as agentive verbs and experiencer verbs. Similarly, the third approach focuses on subjective will and personal state for classification. For instance, Rogers (1974) distinguishes active visual verbs from those with a cognitive sense. Nevertheless, there are some classifications with inconsistent criteria. In Kopytko’s (1990) componential analysis, subjective [+Active] and eventual [+Result] parameters are used simultaneously as the decisive features. As one possible solution to these miscellaneous distinctions, the classification of “controlled activity” and “uncontrolled experience”, proposed by Evans and Wilkins (2000), pays equal attention to the properties of visual events and arguments, and has been widely used in many recent studies (see Georgakopoulos et al. 2022 among others). Table 2 provides some examples illustrating the distinction between verbs of controlled visual activity and verbs of uncontrolled visual experience across languages.

As has been pointed out in previous studies (cf. Chén 2014; Lǚ 1980; Zhāng 2006), the classification of the basic Mandarin vision verbs kàn and jiàn aligns with the distinction between the verbs of controlled visual activity and uncontrolled visual experience. This suggests that Mandarin has something in common with many other languages in terms of visual representations, and lays the foundation for our inquiry into their colexification patterns. There are also some frequently used expressions, e.g., 看见 kàn-jiàn, 看到 kàn-dào, and 见到 jiàn-dào, which express the meaning of uncontrolled visual experience in Mandarin. According to Chao (1979: 208) and Liú et al. (1983: 330), they should be construed as verb-complement constructions, derived from the basic vision verbs kàn or jiàn (Chén 2014: 95–96). However, it is worth noting that most existing works about the classification of these words are based on intuition and introspection. Following a cognitive-functional research paradigm, we believe that the classification of vision verbs should be based on prototypical identification.

2.2 Transfield colexification: semantic extension from vision to cognition

The study on the colexification of vision and cognition has a long history, as the primary target domain of vision verbs is the “intellect or mental activity” semantic field (Ibarretxe-Antuñano 1999, 2008). Sweetser (1990: 45) claims that the connection between vision and knowledge may “be fairly common cross-culturally, if not universal”, which has been corroborated by an abundance of evidence across languages (see, inter alia, Aikhenvald and Storch 2013; Ibarretxe-Antuñano 1999, 2008; Vanhove 2008).

In general, there are two lines of research in the study of vision-cognition colexification. One line of research deals with the classification of vision verbs. Caplan (1973: 273) notices that in English see permits abstract readings, i.e., “realize” or “come to know”, but look and watch do not carry such permissions. Divjak (2021: 180) points out that there are some constructions in which only verbs of experience could work, e.g., “He saw (*looked) that something was wrong”. The common idea across these studies, which are primarily based on Indo-European languages, is that the colexification of vision and cognition relies on uncontrolled visual experience. In a recent study, Georgakopoulos et al. (2022) collate a large amount of empirical evidence from the Multilingual WordNet dataset (Bond and Paik 2012) and the Database of Cross-Linguistic Colexifications (CLICS², List et al. 2018) to assess this claim. A closer look at the language distribution of these datasets, however, reveals that most of the Sinitic languages are not represented in their investigation.

The other line of study tackles internal differences among cognitive meanings. Three of the cognitive senses are at the center of the discussion, that is, know, understand, and think (Aikhenvald and Storch 2013). Ibarretxe-Antuñano (2008) presents understand as a type of cognition that has more mental manipulations than does know, suggesting that a further connection to vision is established. Yet the data from Vanhove (2008) demonstrate that the connection between visual perception and understand is the most widely distributed among the three senses. As pointed out by van Putten (2020: 450), “the connections between vision and particular cognitive meanings are more language specific” (see also San Roque et al. 2018), it is thus intriguing to further explore these relationships in other non-European languages.

The close connection between vision and cognition in Mandarin has already been attested. However, only a few works have investigated the influence of visual and cognitive classifications on the phenomena of colexification. Through exemplification, a few scholars (e.g., Fáng 2018) have come up with some distinctive features of Mandarin visual representations, including their particular reliance on light and visual activity. Nevertheless, it remains unclear how Mandarin basic vision verbs behave in context, and whether such patterns are consistent with the proposal that Mandarin differs from other languages in terms of the colexification of vision and cognition.

2.3 Behavioral profile analysis: corpus-based lexical semantic research

Behavioral Profile analysis (BP analysis) is one of the most powerful corpus-based research methods for lexical semantics (see Divjak 2006, 2015; Divjak and Gries 2006, 2008; Glynn 2009, 2010; Glynn and Robinson 2014; Gries 2006; Jansegers et al. 2015 among others). The underlying assumption of BP analysis is the Distribution Hypothesis (cf. Bolinger 1968; Firth 1957; Harris 1954, 1970), which holds that different distributions reflect differences in function or meaning. In this way, the corpus-based BP analysis can help to identify the semantic structure of given verbs more objectively and efficiently (Gries and Divjak 2009: 60).

BP analysis was first developed by Divjak and Gries (2006). It has been adopted to provide a comprehensive enquiry into the multiple senses of the English verb run in Gries (2006). Gries (2006: 57) points out that BP analysis may help to tackle “some notoriously difficult problems”, such as prototype identification and polysemous distinction. For assessing prototypicality, frequency of the senses and their ID tag attributes are counted; and a hierarchical agglomerative cluster analysis is proposed as a helpful technique for polysemous distinction. Similar statistical methods, especially hierarchical agglomerative cluster analysis, are used in most of the subsequent works, for example, on the polysemous Spanish sentir ‘feel’ (Jansegers et al. 2015), Estonian nägema ‘see’ (Proos 2019), and Mandarin 想 xiǎng ‘think’ (Wú et al. 2021).

Apart from its application in the study of individual words, BP analysis has also been used in contrastive research (Enghels and Jansegers 2013; Divjak and Gries 2009), entering the field of lexical typology. Built on a usage-based foundation, the study of lexical typology takes polysemy (termed cross-linguistically as “colexification”; see François 2008) as one of its central issues and as a result has turned to a number of empirical methods for semantic studies (Koptjevskaja-Tamm 2012). Data-based studies like these have unprecedented advantages in discovering colexification patterns on a larger scale, though there are nevertheless some limitations. For instance, the database CLICS² (List et al. 2018), based upon which Georgakopoulos et al. (2022) draw their conclusions, does not cover some Sinitic languages in the perceptual and cognitive domains. Large-scale studies rely heavily on the quality of the database, and may fall short when dealing with in-depth questions of lexical semantic analysis. To help address this issue without undermining the empirical foundation, our study applies BP analysis while taking these limitations into account. As a corpus-based lexical semantic approach, BP analysis provides both manual analysis and quantitative statistics, yet it still has been limited in its application in contrastive studies.

3.1 Data collection

The data for this study are retrieved from the oral section of the Center for Chinese Linguistics Corpus (CCL, Peking University) (Zhān et al. 2019), which consists of 3,081,723-character transcriptions of Mandarin surveys, talks, and TV shows. Oral materials are considered reliable sources for the study of meaning extensions of perception verbs (San Roque et al. 2018). All concordances of the lemmas kàn and jiàn are retrieved and non-relevant instances are filtered manually. In this process, we rule out the cases where kàn and jiàn are not used as verbs, for example, in the nominal compounds like 看法 kànfǎ ‘view’ and 意见 yìjiàn ‘opinion’, or as grammaticalized particles. For concordances with more than one verbal use of kàn and jiàn, only the first one is retained since their meanings are highly overlapping under the same context. The numbers of all concordances and retained samples of kàn and jiàn for this study are presented in Table 3.

3.2 Data annotation

Based on previous studies and taking into account the features of Mandarin (see, inter alia, Divjak 2006; Divjak and Gries 2006; Gries 2006; Glynn 2014; Jansegers et al. 2015), we design our following annotation scheme. All 2,721 of the retained sampled concordances are annotated manually according to 10 variables (ID tags, Atkins 1987) which cover verbal, syntactic, and discursive levels: Semantic property, Morphological feature, Sentence type, Transitivity, Negation, Subject, Object, Adverbial, Complement and Discourse marker. Table 4 presents an overview of the annotating scheme.

The semantic properties are mainly taken from three authoritative Chinese dictionaries: Xiàndài Hànyǔ Cídiǎn (Dìqībǎn) 《现代汉语词典 (第七版)》 [Modern Chinese Dictionary (the 7th edn.)] (Zhōngguó Shèhuìkēxuéyuàn Yǔyányánjiūsuǒ Cídiǎnbiānjíshì 2016), Dāngdài Hànyǔ Cídiǎn 《当代汉语词典》 [Contemporary Chinese Dictionary] (Dāngdài Hànyǔ Cídiǎn Biānwěihuì 2009), and Xiàndài Hànyǔ Guīfàn Cídiǎn (Dìsānbǎn) 《现代汉语规范词典(第三版)》 [Modern Chinese Standard Dictionary (the 3rd edn.)] (Lǐ 2014). The latest version of the Database of Cross-Linguistic Colexifications (CLICS³, Rzymski et al. 2020) is also referenced for cross-linguistic alignment. The selection of morphological features is generally based on Yú’s (1954) study of Mandarin morphology with its application in Wú et al.’s (2021) research. Syntactic level involves sentence type, transitivity, positive or negative, and the semantic roles of subject, object, adverbial, and complement. At the discursive level, we focus on the formation of discourse markers. In Table 5, examples are given to illustrate how the concordances are annotated for the 72 ID tag levels.

For the semantics of visual perception, we only distinguish between the senses of controlled activity (look) and uncontrolled experience senses (see) in this study. This means that some visual perceptual specifications (e.g., the meanings of watch, glance, glimpse, etc.) are divided into one of these two groups. Following Divjak (2006: 36) and Divjak and Gries (2006: 35), we adopt Vendler’s (1967) stativity tests to minimize the subjective bias in the process of manual semantic annotation, as follows: (1) Imperative test. look verbs are able to co-occur with imperative mood, e.g., in the form of “persuade to” and “force to” constructions, while most see verbs fail to do so. For example, the sentence “I am forced to look at it” is valid but “I am forced to see it” is semantically anomalous; (2) Adverb test. A few of see verbs that may slip through the imperative test will be recognized in the adverb test. look verbs can be modified by some intentional adverbs, e.g., “deliberately” or “carefully”, whereas see verbs can only co-exist with particular unconscious adverbs like “accidentally” or “inadvertently”. For instance, “I deliberately look at it” is acceptable, whereas “I deliberately see it” lacks semantic validity. In addition, based on the features of Mandarin morphology the meanings of look and see can be distinguished by the complement after the main verb. If there is a resultative complement, it is usually in the sense of uncontrolled visual experience, for example, 看见 kàn-jiàn and 看到 kàn-dào (cf. Viberg 1983). This methodology also applies in annotating the two senses of listen and hear in our sample.

3.3 Data analysis

The annotated data are further analyzed with respect to the behavioral profiles of kàn and jiàn, centering on their semantic prototypes and colexification patterns. The statistical procedures are performed by R 4.2.2 with R studio. The R packages involved are pvclust and Rling. The latter is designed specifically for linguistic studies by Levshina (2015). In the procedures, we first present a frequency and percentage description for the semantic properties of kàn and jiàn and the relevant ID tag levels. Second, the data are transformed into BP vectors using the bp function in the Rling package. Third, we measure the distances by “Canberra distance”, which treats the minimal differences in the BP vectors proportionally rather than absolutely (Levshina 2015: 307). Fourth, we adopt the “Ward method” to derive more compact clusters in the hierarchical agglomerative cluster analyses (Levshina 2015: 311). Finally, we test the validation of each cluster with the help of “Multiscale bootstrap resampling” using the pvclust package. The results of the bootstrapping are represented by two probability numbers: the AU (Approximately Unbiased) p-value and the BP (Bootstrap Probability) value; the former is considered more precise in displaying the significance of clustering (Levshina 2015: 316).

4.1 Semantic prototypes

In our sample, the verb kàn is found to have 15 different senses generally. Given that frequency is one of the important indicators of prototypicality and the senses with higher prototypicality simply occur more frequently (Schmid 2000: 39), Table 6 provides a list of the senses of kàn, arranged according to the frequencies and percentages of their concordances and ID tag levels.

As shown in Table 6, visual perceptual and cognitive meanings occupy the top five places, which corroborates the primary status of the cognitive sense in vision verbs (Ibarretxe-Antuñano 1999, 2008). Among them, look ranks first (602, 26.73 %) of all the senses, and there are less than 20 % cases where kàn is used for the sense of see (383, 17.01 %). Examples (7a–b) below are typical representations of kàn’s primary semantic property. In (7a), the sense of activity is apparent since kàn is used in an imperative mood and the audience is invited to focus on a visible entity (大城砖 dà chéngzhuān ‘big city brick’). Similarly, in (7b), the uncertainty of the result and the inclination to focus on a visual target confirm it as an instance of controlled visual activity.

In several contexts, kàn can be used to mean see. For example, kàn by itself is able to present a realized visual experience, as shown in (8a). Moreover, the construction made up of the verb kàn and the resultative complement -jiàn (viz., 看见 kàn-jiàn) is a common expression for uncontrolled visual experience in Mandarin (Chao 1979: 208; Liú et al. 1983: 330). Therefore, the zero-form kàn in (8a) can also be regarded as a case where kàn-jiàn omits the resultative complement -jiàn in order to achieve a more economical sentence. Additionally, 到 -dào is another frequent resultative complement that enables kàn to present the meaning of visual experience, as shown in (8b).

Another important parameter for prototypicality is multiplicity of context. Elements with higher prototypicality are likely to be the least formally constrained, showing a greater flexibility in contexts (Gries 2006). In a behavioral profile analysis, this indicator can be measured by the number of ID tag attributes: the more contexts wherein the verb is able to occur, the larger the number of ID tag levels it owns (Gries 2006: 76). As can be seen in Table 6, look covers the most ID tag levels (49, 87.50 %) in the BP vectors, showing the highest contextual flexibility among all senses. Taking into consideration these indicators, it can be concluded that kàn’s prototypical meaning is in fact look, thereby empirically verifying that kàn can be characterized as a controlled visual activity verb.

The indicators are also used for the prototype identification of jiàn. As shown in Table 7, seven meanings are embodied in jiàn with see ranking the highest (231, 49.25 %) and look the lowest (2, 16.07 %) according to frequency and percentage of concordances. This suggests that there should be a great disparity in status between the two visual semantics, with jiàn being barely used in the sense of controlled visual activity.

Examples (9a–b) present the typical usage scenarios where jiàn is used in the sense of see. The events of “(he) acts like a spoiled child” in (9a) and “(he) doesn’t cry” in (9b) both indicate the reactions and results of the visual activities.

In some rare cases, jiàn can be interpreted as a verb of activity. In (10a-b), the imperative sentences require that the visual activities for “a detailed analysis” and “the full text of the document” are under the control of participants.

As another indicator of prototypicality, the sense of see also owns the most ID tag levels (41, 73.21 %), which confirms its highest contextual flexibility compared with other possible meanings. Taking these indicators together, see is confirmed as the prototypical meaning of jiàn, thereby empirically verifying that jiàn can be characterized as an uncontrolled visual experience verb.

4.2 Colexification patterns

As mentioned above, kàn is such a highly polysemous verb that it prompts us to assess its inter-sense relationships, especially with respect to its visual and cognitive meanings. In this section, we report a pair of hierarchical agglomerative cluster analyses, which are plotted with validation from multiscale bootstrap resampling, in order to explore how the different senses of kàn and jiàn show their similarity and distinctiveness.

The clustering results for the senses of kàn are presented in Figure 1. It is shown that the meaning of look clusters with see first in the tree, which reveals the similarity between these two visual senses of kàn (Divjak 2006; Divjak and Fieller 2014; Divjak and Gries 2006). The same situation also appears in the clustering of the two auditory senses, viz., listen and hear. It is no surprise that these two pairs of activity and experience meanings have close connections, since they all take visually or audibly capable subjects or accessible objects as the main arguments.

Figure 1:

Hierarchical agglomerative clustering for the senses of kàn.

The cognitive meanings know and understand are the second to cluster together, followed by their clustering with the remaining cognitive sense think. These three cognitive meanings can be observed and recognized by the semantic properties of the objects. The examples below illustrate the differences and connections among the three cognitive meanings (see examples below).

In (11a–b), the event objects “sowing season had come” and “my eldest sister is almost forty” cannot be perceived directly from visual perception, which justifies kàn’s semantic extension from vision to cognition. However, it is only loosely detached from vision since one of the most important ways to “know” is to perceive with one’s eyes, for instance, by observing the state of the soil or the change of weather, or noticing graying hair and wrinkles on the sister’s face. This is harder to realize in (12a) since the subject “we” has to mobilize a higher level of cognitive ability to process “people, things, and all kinds of principles in the world” instead of a passive acceptance of visually hinted information. And in (12b), the word 懂 dǒng ‘clear’ highlights the cognitive meaning of understanding. The cognitive processing is even greater in (13a–b), since the objects “it is good” and “the current students are OK” are propositions or statements that express one’s feelings or points of view, rather than the external things or objective events as in (11) and (12). In these examples, the semantic change of meaning from visual perception to cognition shows a gradient of cognitive processing, i.e., know > understand > think, during which the first two meanings demonstrate a slightly closer relationship.

In addition to the clustering of care and like, which share a connection of emotional affection, and the clustering of depend and seem, which normally take the perceived as subjects, the visual perceptual senses look and see and the cognitive meanings, know, understand, and think, have a notable association in kàn, since they cluster in the tree with an AU value as high as 93 %.

The situation for jiàn is less complicated in terms of the colexification pattern. As mentioned in Section 4.1 above, jiàn includes the two visual perceptual senses and one cognitive meaning know in its polysemous network. Our investigation of the inter-sense relationships for jiàn is presented as a hierarchical agglomerative clustering (see Figure 2).

Figure 2:

Hierarchical agglomerative clustering for the senses of jiàn.

It can be noticed that although jiàn also extends to the cognitive meaning know, the relationships between the visual and cognitive senses are not so close in comparison with those in kàn. The relatively remote and non-significant clustering results suggest that the cognitive representation of jiàn is often limited to certain contexts of usage (see examples below).

Examples (14a–b) present two typical situations where jiàn is used for cognitive purposes in contemporary Chinese. The first is produced in a style reminiscent of ancient Chinese, and in the second jiàn is embedded in a discourse marker kě jiàn. It is indicated that in many cases the cognitive use of jiàn can only be obtained by means of conventionalization, which largely reduces its flexibility in behavioral profiles.

The results of the above analyses of semantic prototypes and colexification patterns indicate that the Mandarin kàn is a prototypically controlled visual activity verb with a close connection with cognition. In comparison, the colexification pattern of jiàn indicates that although this prototypically uncontrolled visual experience verb is also sometimes associated with cognition, this connection is relatively weak and its usage for cognitive interpretation is rather limited.