The multimodality of complaint: a case study of Chinese EFL learners

2.1 Complaints in L2 pragmatics

Research on the speech act of complaint in L2 pragmatics primarily delves into comparing the pragmatic ability between first language (L1) speakers and L2 learners in the realization of complaints. It examines the learners’ performance and related influencing factors in three aspects: directness (e.g. DeCapua 1989; Murphy and Neu 2006; Olshtain and Weinbach 1987), external modifiers (e.g. DeCapua 1989; Murphy and Neu 2006), and internal modifiers (e.g. Trosborg 1995). Studies have illuminated that L2 proficiency (e.g. Azarmi and Behnam 2012; Beltrán-Palanques and Querol-Julián 2018), age (e.g. Li et al. 2006), gender (e.g. Boxer 1993), the language used in communication (e.g. DeCapua 1989), social distance and power (Olshtain and Weinbach 1987), and cultural values and social norms (e.g. Arent 1996) have impacts on the use of complaints.

Among these factors, the directness of complaints functions as a pivotal indicator of speaker’s consideration for the listener’s face and their attitude toward repairing the situation (Olshtain and Weinbach 1987). Additionally, it affects the listener’s interpretation of the speech act (DeCapua 1989). Research indicates that the directness of speech acts realized by speakers is jointly influenced by their own culture and specific communicative contexts (Blum-Kulka 1982).

As a manifestation of specific communicative contexts, the language used in communication serves as a crucial factor affecting the directness of complaints. DeCapua (1989) adopted discourse completion tasks to reveal that German learners of English, influenced by their L1 communicative patterns, exhibited greater directness in their complaints compared to English L1 speakers. Further research by Yang and Yuan (2019) has shown that Chinese learners of English, shaped by their mother tongue, presented more directness in English complaints. Conversely, Gallaher’s (2011) study on American learners of Russian observed less influence from their mother tongue in some scenarios, as long-term interaction with Russian L1 speakers led to the adoption of complaint strategies akin to those of these L1 speakers.

The role of social distance as a critical sociolinguistic factor has been extensively examined, with studies like Boxer (1993) and Azarmi and Behnam (2012) highlighting its impact on the realization of complaints. For instance, Azarmi and Behnam (2012) found that low-intermediate Iranian learners of English were more direct in their complaints in English when facing strangers than acquaintances. Hong and Shih (2013) found that both high-level and low-level Chinese learners of English presented a gradual decrease in directness in English when addressing complaints to siblings, neighbors, and strangers. Nevertheless, some studies have noted no effect of social distance on the learners’ choice of complaint strategies. One such study was conducted by Olshtain and Weinbach (1993), who found no significant difference in the directness of complaints among Hebrew learners in scenarios with acquaintances and strangers.

2.2 Speech acts in L2 pragmatics from multimodal perspectives

While the majority of speech act research in L2 pragmatics has concentrated on aspects of language, a select few studies have adopted multimodal interaction analysis (Norris 2004) to explore the multifaceted characteristics of speech acts. Beltrán-Palanques and Querol-Julián (2018), for example, analyzed the multimodal features of complaints among four Spanish learners of English, observing that advanced learners frequently utilized gestures, whereas less proficient learners relied more on gaze and facial expressions to convey their complaints. Similarly, Pei et al. (2020) identified varying modal densities in the apology strategies of Chinese and Zimbabwean learners of English. These studies offer fresh insights into the convergence of L2 pragmatics and multimodality. However, the research design and analytical framework of the above studies have certain limitations: the number of research participants is small and the research variables are narrowly defined, making it difficult to draw generalizable conclusions. In addition, while multimodal interaction analysis predominantly considers time as a key measurement, it is not always feasible to gauge the intensity of all modes based solely on time duration. Such considerations underscore the need for further research that embraces a more expansive and nuanced approach to understanding the multimodal nature of L2 pragmatics.

In response to recognized gaps in the literature, the current study employs a multimodal lens to explore the intricate dynamics of complaints among 18 pairs of advanced Chinese learners of English by harnessing role-play tasks for data elicitation. It scrutinizes the multimodal expressions of these learners as they navigate through scenarios that encompass different languages and social distances, thereby painting a comprehensive picture of how complaints are produced and perceived. The research questions are as follows:

1. How do advanced Chinese EFL learners perform multimodally when complaining in different linguistic contexts (English versus Chinese)?

2. What are the similarities and differences in the multimodal expressions of complaints by advanced Chinese EFL learners in English scenarios with different social distances (acquaintances versus strangers)?

3.1 Participants

The participants in this study were 36 female postgraduate students majoring in English from a university in China (average age = 24.8). All the participants had no experience of studying abroad. Moreover, they all passed the Test for English Majors 8 (TEM-8). Testers who pass TEM-8 are generally acknowledged as advanced English learners in China, so they were regarded as advanced learners of English. The participants were randomly divided into 18 pairs. The relationship between the participants in each pair is that of ordinary classmates who only interact in the classroom setting.

3.2 Instrument

This study employed role-play tasks for data collection. In the preliminary stage, we crafted three stranger scenarios and three acquaintance scenarios based on the campus experiences of the two researchers. Then, a group of ten female graduate students who were not involved in the formal experiment were tasked with choosing two scenarios each from the stranger and acquaintance categories based on the parameters of familiarity and the degree of imposition. Following this, we obtained four scenarios with high familiarity and consistent degrees of imposition. Subsequently, we selected one scenario from the stranger category and one from the acquaintance category from the pool of the chosen scenarios for translation into English. A brief overview of these four scenarios and their corresponding experimental variables is as follows (for detailed scenario descriptions, refer to Appendix):

3.3 Procedure

To ensure that the participants were well acquainted with the role-play tasks, they were initially asked to engage in warm-up exercises. After obtaining the consent of each of the participants, we proceeded to capture video recordings of the formal tasks. During the formal tasks, participants were given the autonomy to select their roles as either the complainer or the complainee. This role selection was maintained consistently across the four selected scenarios. To mitigate the potential impact of task order, the participants were asked to engage in the Chinese acquaintance scenario and the English stranger scenario in the first week, and the remaining two scenarios were conducted one week later. In the post-experiment interviews, the participants conveyed that they were either personally familiar with or could readily envision the scenarios presented. They further expressed a belief that their actions within the role-play tasks closely aligned with their behaviors in real-life situations, thereby reinforcing the validity and reliability of the study’s findings.

3.4 Ethical issues

The participants in this study were informed that their involvement in the role-play tasks would be captured on video, with the assurance that such recordings were intended exclusively for academic examination. Only after the participants granted their informed consent did the experiments commence. Upon completion of the experiments, as a token of appreciation, each participant was rewarded with a selection of gifts.

3.5 Data analysis

The video of this study was transcribed using the online software program iFLYTEK^[1] first and then manually corrected by researchers to achieve a 100 % accuracy rate. In the initial phase of analysis, we coded the speech acts of complaint on the linguistic level, based on the seminal work of Olshtain and Weinbach (1987) on the taxonomy of complaint directness. We made a refinement that “expression of annoyance or disapproval” was modified to “implicit complaint,” resulting in the following coding framework for verbal complaints:

1. Below the level of reproach: speaker chooses a complete avoidance of direct or indirect reference to either the event or the hearer;

2. Implicit complaints: speaker performs an obvious hint with avoidance of direct and explicit mention of event or hearer;

3. Explicit complaints: speaker expresses disappointment, annoyance, and criticism toward the hearer by reference to either the event or the hearer, or both, but no sanctions are instigated;

4. Accusation and warning: speaker performs an open face-threatening act and even implies potential sanctions for the hearer;

5. Threat: speaker performs an open face-threatening act and even implies immediate sanctions for the hearer.

Subsequently, we coded the speech acts of complaint on the multimodal level. The study leveraged the multimodal coding scheme proposed by Sun et al. (2021), and it was adapted to align with the specific context of the current investigation (Table 1):

According to the coding frame, we embarked on a joint analysis of the data extracted from a single scenario in terms of linguistic, paralinguistic, and nonverbal modes. Then we independently analyzed the entire dataset, reaching a high degree of agreement at 87.27 % agreement for the verbal modes and 80.73 % agreement for paralinguistic and nonverbal modes. Finally, we discussed the discrepancies until a consensus was reached.

4.1 Complaints in English and Chinese scenarios

The results, as presented in Table 2, reveal that, in both the English and the Chinese scenarios, none of the complainers chose the strategy below the level of reproach, which corresponds to directness level 1, signifying that all participants showed a clear intention to voice their complaints. A significant majority of the participants, comprising at least 15 individuals, preferred to use explicit complaint, which is categorized under directness level 3. Conversely, a minimal number of complainers chose to employ a higher directness level, such as accusation and warning, which falls under directness level 4, with 2 participants adopting this strategy. Furthermore, even fewer, limited to 1 person, opted for the threat strategy, the most direct form of complaint, at directness level 5.

A noteworthy difference was observed in the adoption of directness level 2 between the English and Chinese scenarios. Implicit complaints, represented by the counts (3, 4), were scarcely utilized in the Chinese context. In contrast, this form of complaint was more prevalent in the English scenario, with counts of (16, 16). To further illustrate this point, compare the following two dialogues:

Examples 1 and 2 illustrate a set of complaints between acquaintances within Chinese and English contexts respectively. In both scenarios, the complainers articulated their grievances. However, the approach to managing the complaints diverged between the two language settings. In the Chinese scenario, as shown in Example 1, the complaint was introduced in a swift and straightforward manner, as the complainer promptly raised the issue in turn 2 (你怎么才来啊?! ‘why did it take you so long to get here?!’), reflecting a tendency to address problems without delay. Conversely, the English scenario exhibited a more circumspect approach to avoid potential confrontation evidenced by the complainer’s employment of a series of mitigating strategies, gradually elucidating the reasons for the complaint across turns 1 (Did you see the grade that we got in the literature class?), 3 (We failed it. Do you know about that?), and 5 (Don’t you wonder why we failed it?). Upon perceiving a lack of accountability from the complainee, the complainer opted to explicitly voice her complaint in turn 7 (The professor told me that you didn’t write a good research paper.).

The study’s quantitative analysis of the paralinguistic and nonverbal features of the participants’ complaints, as detailed in Table 3, reveals no significant difference in paralinguistic aspects between the English and Chinese scenarios. Nevertheless, a marked disparity was observed in nonverbal modes, with the English scenario featuring a pronounced use of gestures and head movements. Further examination, as indicated in Table 4, highlights that the frequency of metaphorics, deictics, and beats gestures, along with nodding and head tilting, in the English scenarios was significantly higher than those in the Chinese scenarios.

To verify the aforementioned data and to comprehend the intricate tapestry of multimodal synergy that emerged in the context of complaints, we re-compared Example 1 and Example 2 from the perspective of paralinguistic and nonverbal modes. In both scenarios, confronted with the complainees’ justification for their offensive conduct (turns 5 and 7 in Example 1; turns 12 and 14 in Example 2), the complainers stood firm, refusing to concede or forgive. They grounded their demands in moral norms (turn 8 in Example 1; turn 15 in Example 2), urging the complainees to assume appropriate responsibility. Moreover, upon delving deeper into the paralinguistic and nonverbal cues exhibited by the complainers in these two scenarios, a distinct contrast emerged. In the Chinese context (Example 1), the complainer’s sparing use of head tilts and furrowed brows was often synchronized with emphatic adverbs, such as 才 ‘so,’ 怎么 ‘how come,’ 又 ‘again,’ and 好歹 ‘at least.’ In contrast, the English scenario (Example 2) unveiled more diverse paralinguistic and nonverbal modes. The complainer also adopted facial expressions, such as frowning when uttering awful. Beyond this, she also used head movements, such as tilting the head to the right or front when saying that, you, need, and disappointed. Moreover, the English scenario was marked by a paralinguistic presence; the declaration of disappointed was not only underscored by a forward head tilt but also by an elevation in vocal pitch. The final flourish in the English narrative was the abundant use of gestures, which was absent in the Chinese scenario. This includes gestural emphasis, such as clenching both fists to symbolize the concept of “co-worker” and moving them up and down, pointing to denote the parties involved (you and I).

4.2 English complaints in different social distance scenarios

Table 2 indicates that, in terms of linguistic perspective, the directness of the complaints did not significantly vary with social distance in the English scenarios. This suggests that regardless of whether the complainers were interacting with an acquaintance or a stranger, they tended to opt for moderate to low levels of directness (directness level 2 and 3) in their complaint strategies, where individuals were often cautious not to offend the other party, even when expressing dissatisfaction. On the other hand, Table 3 illustrates that more abundant gestures, head movements, and change of volume were displayed by the complainers when expressing complaints to acquaintances rather than to strangers in the English settings. Table 4 provides evidence of the use of nonverbal communication. It shows that the complainers employed a wider variety of metaphorics, deictics, and beats, as well as nodding, shaking, and head tilting, when complaining to acquaintances. To attain a deeper comprehension of the underlying significance encapsulated within the aforementioned data, we compared Example 2 with Example 3:

The complainer in Example (3), akin to Example (2), underwent a transition from subtle to overt expressions of discontent. In turn 2, the complainee declared the seat was occupied and inquired if the complainer could seek out an alternative. Confronted with this directive, the complainer eschewed a direct critique of the complainee’s misconduct, through voicing frustration over the crowdedness of the library and the considerable time spent in search of a seat (turn 3), thereby indirectly rejecting the complainee’s request. In response to this veiled refusal, the complainee showed no sign of yielding but reiterated the seat’s occupancy and mentioned the imminent return of its rightful user (turn 4).

When faced with the complainee’s persistent refusal to relinquish the seat, the complainer articulated her rationale for its occupancy (There’s nothing here to say it’s taken), and invoked societal norms, specifically the library’s policy against seat occupation, to underscore the impropriety of the complainee’s action (turn 5), thereby expressing her dissatisfaction. Despite this, the complainee neither addressed nor countered the complainer’s argument, instead citing her own motives for holding the seat (I’m just trying to help my friend out) and reiterating the friend’s swift return (turn 6), thereby urging the complainer to give up her claim to the seat. The complainer, in response, articulated two clear complaints in turn 7: the complainee’s justification was deemed irrelevant (That’s not the point), and her action was at odds with the norms of public spaces (You can’t just occupy a seat and leave it empty). In the face of these explicit grievances, the complainee maintained her stance, once again referencing the friend’s prompt return (turn 8).

This repetitive justification further aggravated the complainer, who, in turn 9, emphatically restated the legitimacy of her claim to the seat (There are signs everywhere saying “No seats occupation”). The complainee ultimately acquiesced, yet in the subsequent turns (turn 10, 12), accused the complainer of improper conduct (Keep your voice down. I didn’t think it would be a big deal; You don’t have to be so rude about it). Despite such accusations, the complainer remained steadfast in her defense, invoking the regulations of public spaces in turn 11 (It is a big deal when I can’t find a place to study because people ignore the rules) and turn 13 (If you can’t follow the rules, then you shouldn’t expect others to do so).

Examples 2 and 3 illustrate that in the English contexts, the complainers achieved their purpose of lodging complaints, whether dealing with acquaintances or strangers. By employing strategies that transition from hints (directness level 2) to overt complaints (directness level 3), they effectively circumvented the potential for exacerbating interpersonal tensions. This measured tactic ensured that the essence of their discontent was communicated without inciting undue conflict. Within these interactions, the complainers leveraged social conventions to substantiate their grievances. This is evidenced in turn 15 of Example 2 and turn 5, 7, 9, 11, 13 of Example 3, where they drew upon shared societal expectations to articulate the inappropriateness of the complainees’ misconduct. By anchoring their complaints in commonly accepted norms, they rendered their objections more palatable. In the English scenario featuring acquaintances (Example 2), the complainer utilized a rich array of paralinguistic and nonverbal resources, namely five gestures, four movements of head, one facial expression, and one change of volume, which we explored in detail in the latter part of Section 4.1. However, when communicating in English with a stranger (Example 3), the complainer employed only five instances of detectable paralinguistic and nonverbal expressions in five turns (turn 3, 5, 7, 9, 13), namely, three movements of the head and two changes of volume.

5.1 Complaints in verbal language modes

The complainers in this study demonstrate a reluctance to employ highly face-threatening complaint strategies in both language contexts, indicating a strategic avoidance of escalating interpersonal conflicts while simultaneously achieving the intended communicative objectives. The complainers in this investigation display a pattern of behaviors in both settings that aligns with the observations of Yang and Yuan (2019), who reported that high-level English learners were inclined to adopt explicit complaint strategies in English contexts. However, their study relied on discourse completion tasks, which inherently limited the analysis of the complainers’ dynamic performances throughout the communicative exchange. In contrast, the present study employs role-play tasks to elicit communication intentions among the participants, thereby capturing the evolving nature of the speech act of complaints.

An examination of the complaint process reveals that the English complaints issued by the learners in this study progress from implicit to explicit, which is a trend that echoes the behaviors observed among most Korean learners of English as documented by Murphy and Neu (2006). In contrast, the complaints articulated by the learners in Chinese scenarios are characterized by a more direct and swift manner. Within the contemporary Chinese sociolinguistic landscape, politeness serves to underscore the importance of social harmony and the mitigation of interpersonal discord (Gu 1990). Nevertheless, this study, along with other relevant empirical research (Hong and Shih 2013; Yang and Yuan 2019), reveals a propensity among complainers to adopt a forthright style of communication. This suggests that, in Chinese settings, when politeness principles come into conflict with the communicative objective of lodging a complaint, the principle of politeness may be subordinate to the fulfillment of the communicative aim, but in English contexts, interlocutors strive to strike a balance between adhering to politeness norms and the pursuit of specific communicative goals.

Previous L1 research has consistently highlighted the essential role of social distance in realizing speech acts. Comparative pragmatic investigations have uncovered that the ramifications of social distance are intricately intertwined with the social and cultural background of the communicators, with its impact manifesting differently across diverse linguacultural landscapes. However, contrasting findings have emerged from other scholarly works. Hong and Shih (2012) noted that social distance impacted the directness of complaints among both Chinese L1 and English L1 speakers, with the use of direct complaint with acquaintances and indirect complaint with strangers. However, a few studies presented confronting findings. Olshtain and Weinbach (1987) observed that Hebrew L1 speakers employed analogous complaint strategies regardless of whether they communicated with acquaintances or strangers. This led to the conclusion that within Israeli linguaculture, social distance exerted a less pronounced influence on the selection of complaint strategies. Yuan (2011) corroborated this pattern among English L1 speakers but observed a nearly uniform level of directness in the complaints of Chinese L1 speakers with both familiar and unfamiliar interlocutors, maintaining a moderately low intensity. The learners in the current study mirror Yuan’s (2011) findings in their Chinese complaints. The learners in this study, like those in Yuan’s (2011) research, hail from mainland China, whereas the Chinese participants in Hong and Shih’s (2012) study are from Taiwan. This distinction underscores the fact that within Chinese subcultures, the influence of social distance on interlocutors’ choice of complaint strategy can vary significantly. Based on this, we are convinced that culture exerts a profound influence on the way language is employed. However, culture itself is complex and layered. When dissecting linguistic phenomena, it is imperative to refrain from oversimplifying when using the term “culture.” Instead, we should enrich our analysis by integrating linguistic evidence to provide a more comprehensive understanding of language use within its cultural context.

The existing body of research into L2 complaint speech acts, akin to the results of L1 research mentioned above, presents a diverse picture of the role of social distance. Some studies (e.g. Azarmi and Behnam 2012; Trosborg 1995) have demonstrated that the complaint strategies of English learners are molded by considerations of social distance. In contrast, our study, along with the research by Olshtain and Weinbach (1993), finds no significant differences in the directness of complaints among L2 learners when interacting with acquaintances versus strangers. The divergent outcomes of these studies may be attributed to the varying levels of L2 proficiency among learners and the distinct sociocultural contexts they inhabit. These factors are significant in shaping learners’ perceptions of social distance, thereby influencing their performance in realizing complaints. This further underscores the profound regulatory influence of culture on social distance as a sociopragmatic factor, reinforcing the complexity and richness of the intercultural communication.

The learners in this study did not exhibit a discernible variation in their English complaint strategy, regardless of the social distance between the complainers and complainees. This consistency in directness between the two languages, based on the above discussion, suggests that these advanced learners may still be influenced by the conventions of Chinese complaint behaviors when voicing their grievances in English, potentially resulting in negative pragmatic transfer. A methodological issue worth discussing here is that some studies of speech acts in L2 pragmatics, particularly those concerning pragmatic transfer, have often adhered to Selinker’s (1969) research paradigm. This paradigm involves the selection of participants as follows: (1) L1 speakers of the learners’ first language; (2) the learners; (3) L1 speakers of the learners’ second language (see Beebe et al. 1990; Félix-Brasdefer 2004; Kasper 1992; Wang and Li 2007). This approach can, to a certain degree, influence experimental outcomes due to the differences among individuals in categories (1) and (2). Therefore, we believe that a design similar to the one used in our experiment could be attempted. To mitigate this potential impact, we instructed the learners in our study to voice complaints in analogous scenarios, employing their L1 and L2 respectively. This methodological strategy is designed to bolster the reliability and validity of the collected data.

5.2 Complaints in multimodality

This investigation revealed a noteworthy phenomenon: the learners utilize a more abundant array of multimodal expressions, including gestures and head movements, when performing complaints in English compared to their Chinese interactions. This observation can be ascribed to two primary factors.

Firstly, the learners may have experienced limitations in their oral English proficiency, prompting them to leverage multimodal resources as a compensatory mechanism to achieve their communicative objectives. This aligns with the findings from Gullberg’s (1999) study on intermediate-level Swedish learners of French and Nicoldis et al.’s (2007) research on intermediate-level Chinese learners of English, both of which observed an increased reliance on gestures and head movements in the target language. These studies suggest that learners employ nonverbal modes to offset their linguistic shortcomings. While the participants in the current study are recognized as advanced English learners, exhibiting fluency and pragmatic competence at the linguistic level, their vivid multimodal expressions in English scenarios indicate that they depend on certain paralinguistic and nonverbal cues to bolster their linguistic expressions.

The second rationale pertains to the participants’ utilization of multimodality to unveil their emotional states. Multimodality serves a pragmatic purpose in relaying personal emotions during communication, aiding in the maintenance of social relationships and management of face (Locher and Langlotz 2008). The complainers in this study often imply early on, in the English contexts, that they expect the complainees to assume responsibility. When such intentions are rebuffed or overlooked, they may perceive a thwarting of their communicative goals, leading to the emergence of negative emotions. Consequently, gestures and head movements become more frequent, intensifying the conveyance of interpersonal emotions during the exchanges. This observation resonates with the findings of McClave (2000) and Busso and Narayanan (2007), which indicate that the gestures and head movements of interlocutors are reflective of psychological imagery and emotional states, particularly when complaints are voiced but negotiations are impeded. The complainers may employ high-frequency, large-amplitude gestures and head movements to express their anger. In this study, the learners tend to mobilize a greater array of multimodal resources when complaining to acquaintances as opposed to strangers, utilizing an abundance of gestures, head movements, and vocal pitch. This not only reinforces the linguistic modes but also accentuates the speakers’ conveyed meaning and emotion of the complaints. Thus, there is a divergence from the conclusions drawn by Olshtain and Weinbach (1993). It is posited that, from a multimodal performance perspective, social distance is indeed a determinant in the selection of complaint strategies in English scenarios and is also a significant factor influencing the harmony of interpersonal relationships.

Finally, as can be seen from our findings, multimodal resources have a significant strengthening effect on the illocutionary force of complaints – an effect that linguistic pragmatics alone cannot clearly identify. Therefore, analyzing solely from the linguistic level has become insufficient to capture the full picture of language use in context to a certain extent. Multimodal resources such as gestures and head movements, which were previously considered as contextual factors by pragmatics, should be given renewed attention. Beyond this, it is worthwhile engaging in a discussion on the imperative to explore the multimodality that characterizes not only the speech act of complaints but the vast expanse of human interaction itself. During the communication process, speakers may jointly adopt verbal and nonverbal modes to highlight the focus of interaction. These modes work together to reinforce, complement, regulate, and synergize the conveyance of information, which serves to help the listener understand the purpose of the speaker’s discourse (Zhang 2009). Given that human interaction is inherently a sign behavior, the significance imparted by each symbolic entity merits our earnest consideration. The multimodality of interaction prompts us to re-evaluate the essence of the modern usage of “pragmatics” as originally established by Morris in 1938. Under the influence of analytic philosophy, the domain of pragmatics has been somewhat constricted, predominantly concentrating on language as the principal subject of scrutiny and relegating nonverbal cues within the communicative act to the status of contextual data. We argue that this is a digression from Morris’s conception, which takes pragmatics to be multimodally semiotic in nature. We echo Gu’s (2019) proposal that multimodal semiotic pragmatics serves as a complement to the current mainstream linguistic pragmatics.