Understanding online advice-giving evaluations through the politeness evaluation model

2.1 Audience design

Owing to the often public nature of online platforms, the relationship between various interlocutors within online social platforms is important. On X (formerly Twitter), for example, audiences have been shown to judge and hold broadcasters accountable for the content they publish (Draucker 2015). Similarly, Luo and Hancock’s (2020) study revealed that audiences judged self-praise speech acts within Facebook status updates negatively. The work done on this area has predominately drawn from Bell’s (1984) framework of audience design. In this seminal study, Bell (1984) argued that our linguistic choices are strongly influenced by our tendency to adapt to the potential audiences of our spoken utterances. Bell (1984) claimed that a speaker has several types of potential audiences, which may or may not be ratified (i.e., acknowledged by the speaker), addressed (i.e., directly spoken to), and/or known (i.e., the speaker has an awareness of). These audience types are conceived of as the addressee (listeners who are known, addressed, and ratified), the auditor (listeners who are not directly addressed, but are known and ratified), the overhearer (non-ratified listeners who the speaker has an awareness of), and the eavesdropper (those that are non-ratified, who the speaker is also unaware of). Bell (1984) further argued that our language choices may be influenced by how we might adapt to these potential audiences.

Tagg and Seargeant’s (2014) study on Facebook has further developed the concept of audience design within online platforms. They argue that strategies related to audience design were of utmost importance for social media users, as they tend to “tailor their posts to the expectations of their imagined readership” (Tagg and Seargeant 2014: 162). These strategies included more traditional linguistic strategies, such as considerations of register, tone, and style, but also the use of Facebook’s affordances (e.g., tagging, the @ and # signs, groups, and status updates).

Unger (2020) has claimed that Reddit is similar to discussion forums. The public nature of most discussion forums has been argued to influence interactional behaviors because there are a potentially limitless number of unratified audiences. These unratified audiences can take the role of auditors, overhearers, and eavesdroppers. Moreover, their judgments have been claimed to be as important as other forum members (Arendholz 2013). Scholars have argued that contributors to polylogal online contexts, like discussion forums, are aware of these imagined, passive audiences (Bou-Franch and Blitvich 2014; Lee 2014; Marwick and Boyd 2011).

All these studies point to the pressing need for researchers to also bear in mind the perceptions from unratified audiences (e.g., those that view the interactions on a forum, but do not post). Unfortunately, this aspect has received very little attention, perhaps because of the difficulty of ascertaining who the unratified audiences of a discussion forum are, as well as robustly measuring their attitudes. If we were to take this together with Nielsen’s (2006) 90-9-1 rule of participation inequality, where it is theorized that 90 % of participants online are unratified audiences who only observe but do not contribute, 9 % are occasional contributors, while 1 % are the dominant contributors, then understanding the perceptions of this majority group of online users is all the more important. Yet, with very few exceptions (e.g., Gauducheau and Marcoccia 2021; West 2015), discussions of these audiences are mostly theoretical, with little empirical evidence that helps us understand how the vast majority of CMC users might feel, think, or influence CMC practices.

2.2 Online advice and its perceptions

Language scholars have done much good work in striving to understand advice in online contexts. For instance, Locher’s (2006) seminal study examined how advice-seekers linguistically oriented toward the technological affordances of the online context where they sought advice. Similarly, Stommel and te Molder (2015) also explored how the affordances of an online chat counseling platform influenced how advice interactions progressed on the site. The body of research conducted thus far on online advice has shown that users often do orient toward the features and affordances of the online platforms in which the advice interaction occurs, with much of the features of the advice formulations themselves bearing strong similarities to what has been observed in offline contexts.

The paradigms and methodological frameworks involved in these studies are typically conversation and discourse analytical approaches (e.g., Kouper 2010; Lindholm 2017; Locher 2006; Morrow 2006, 2012]; Placencia 2012; Stommel 2016; Vepsäläinen 2022). Descriptive approaches such as these are important in examining how authentic online advice interactions take place, yet we still have room to explore advice from other perspectives. Specifically, adopting methods that can also account for indirect participants of the interaction (i.e., the 90 % of online users that observe, but do not contribute) seems to be important but missing in our current understanding of online advice.

Capturing evidence of how participants perceive advice is difficult, especially in digital contexts where a person can view an advice interaction but leave no visible record of their opinion. Nevertheless, Spencer-Oatey and Xing (2003) have argued for the value of perception data and the dangers of overrelying on the analyst. Responses and perceptions to online advice have only been accounted for in a handful of linguistic studies (see Harrison and Barlow 2009; Morrow 2006; Page 2012, for examples). In most of these studies, perceptions of advice are evidenced when the advice recipient explicitly reports back their attitudes. Relying solely on explicit reporting back by the advice recipient can be overly limiting. For example, the data in Morrow’s (2006) work on advice interactions within an online discussion forum indicated that there were very few instances of explicit negative perceptions of advice, and zero evidence of positive perceptions.

More research on how advice is perceived, along with understanding what dimensions these evaluations take place, is sorely needed. Descriptive linguistic analyses of advice formulations are very useful, yet they fall short of helping us understand how these formulations affect our perceptions of the appropriateness, usefulness, or value of the advice itself. Moreover, because viewership of online advice can presumably extend beyond the advice recipient, it is therefore also important to look beyond the speaker-addressee dyad. Outside of linguistics, there has been some work done that explores advice perceptions. For example, Feng and MacGeorge (2010) and MacGeorge et al. (2004), within communication studies, have attempted to theorize and test the factors that affect our perceptions of advice through surveys. There is, therefore, the potential to approach the study of advice perceptions from an interdisciplinary perspective, drawing on the linguistic features of advice as outlined by linguists, while also utilizing methodologies used to study advice perceptions from other disciplines.

3.1 Experiment design

A scenario-based between-subjects single-factor experiment was designed. Due to time and research constraints, two moves from Lu’s (2024) study were chosen to be tested as the main independent variables. The two moves tested were the “background” move and the “encouragement” move. These two moves were selected because they naturally co-occurred together in the same text within Reddit, ensuring a good standard of ecological validity, and maximizing authenticity to the corpus. Explanations and examples of these two moves are shown in Table 1.

The experiment was designed, and data was collected through Qualtrics (www.qualtrics.com). In total, there were four experimental conditions: the background move condition, the encouragement move condition, the “all moves” condition, and the “no moves” condition. Participants were evenly and randomly assigned to one of the four conditions. In all four conditions, the participants read a stimulus text that included two parts: one advice-seeking post, and one advice-giving reply. The text was designed to imitate the appearance of the default desktop version of Reddit by including a post title, using the same font as Reddit, and having fictionalized usernames for both the advice-seeker (nicknamed Adviceseeker22) and the advice-giver (nicknamed Advicegiver101).

The fictionalized advice scenario was identical in all four conditions. Adviceseeker22 provides a description of their current relationship with their son, then mentions how their son has become very upset with them and left home, provides a self-assessment, and finally asks for advice. The content and format of this scenario was designed based on a genuine advice post observed in Lu’s (2024) corpus of Reddit advice interactions. The choice of this particular advice post and advice category (a family advice scenario) was due to the fact that the “background” and “encouragement” moves both often naturally co-occurred within advice responses to family advice scenarios in that corpus.

The fictionalized response post by Advicegiver101 was therefore also based on a genuine reply to the advice post, where the “background” and “encouragement” moves co-occurred naturally. This advice-giving reply post was manipulated in each of the four conditions. The background move condition included the “background” move as the initial move, the encouragement move condition included the “encouragement” move as the initial move, the “all moves” condition included both the “background” and the “encouragement” move in the positions where they most naturally occurred in the corpus, while the “no moves” condition did not contain any of these aforementioned moves (i.e., served as the baseline).

An example of one of the experimental conditions is presented in Figure 1. This condition was designed to test the effect of the “background” move. The “background” move was inserted in the first line of the advice-giving reply, underlined in the following example: “As a parent of an 18 year old teenager myself, I want to say that the feeling of disconnection from parents is a common and normal thing for every teen”. The experimental conditions and their respective text prompts can be found in Figures A5-A8 in the appendices.

Figure 1:

The background move condition.

3.2 Procedures

A four-page survey was administered. The first page provided background information about the researcher, the study’s purpose, the approximate time needed to complete the study, and how the data will be used. This same page also asked for participants’ consent to join the study, while also confirming that they were at least 18 years old, and a native-level English speaker before being allowed to proceed.

On the second page, participants were informed that they would be reading two texts and then answering questions based on the reading. They were told that the two texts were posts from an online forum, and text one was written by a user named Adviceseeker22 asking for advice, while text two was written by a user named Advicegiver101, giving advice. They were also told that the advice situation involves a relationship issue between a parent and their teenage child.

Once participants proceeded to the third page of the survey, one of the four stimulus text conditions were presented. Regardless of the condition, the survey items they were presented with were identical. Participants responded to 12 Likert scale-type questions, and two open-ended short response questions. The Likert scales, designed to invoke evaluations of the version of the advice and the advice-giver they were assigned to in the survey, were adapted from work in social psychology pertaining to warmth and competence (Cuddy et al. 2008), and in communication studies regarding advice message factors (Feng and MacGeorge 2010; MacGeorge et al. 2004). The two open-response questions aimed to qualitatively elicit both instanced and wider attitudes and beliefs about advice and as a way to screen out inattentive participants. Participants were also invited to choose to either use Reddit’s upvote or downvote icon to evaluate the given advice. This paper’s focus is on the results of the two open-ended short response questions. A more detailed discussion of the other elements of the experiment will be outlined in a forthcoming paper.

The fourth and final page of the survey asked participants about their familiarity with Reddit, and if they have given advice in an online forum before. Participants were also asked to provide demographic information including age, gender, education level, and occupation before the survey finished. A visualization of the experiment procedure is shown in Figure 2.

Figure 2:

Experimental procedure.

3.3 Participant recruitment

Participants were recruited through the online crowdworking platform Prolific (www.prolific.co). Participants were each compensated at a rate equivalent to £6.50 per hour for completing the study. Palan and Schitter (2018) have described how the qualities of Prolific’s participant pool allows for generating more robust data. Participants from Prolific have also been argued to be more honest in their responses (Peer et al. 2017).

3.4 Implementing the politeness evaluation model

The 180 participants’ responses to the two open-ended responses (360 datapoints in total) yielded significant and interesting findings, meriting an in-depth discussion. In the subsequent parts of this paper, I will report on the results from the open-ended responses of the experiment by painting a general picture of the findings, followed by a more detailed discussion of how the composition of the moves in the different experimental conditions were associated with a prevalence of certain types of participant responses.

The first open-response question can be seen in Figure 3. This question intended to prompt participants to tap into their wider (i.e., context-spanning) perceptions and attitudes about advice. On the other hand, the second question (see Figure 4) intended to provide an opportunity for participants to reveal their evaluations of the specific advice given in the experiment’s scenario. Example responses to these items in this paper are unedited save for anonymization.

Figure 3:

Question item related to what generally constitutes good advice.

Figure 4:

Question item related to participant evaluations of the advice in the scenario.

Vepsäläinen (2022) has noted that advice-seeking on Reddit often involves a moral dilemma. Spencer-Oatey and Kádár’s (2021) politeness evaluation model therefore seemed to be an apt framework to analyze the responses, because the model is suitable for analyzing both instanced and wider social and moral attitudes.

Spencer-Oatey and Kádár (2021: 81) theorize that the evaluation of a behavior or the person performing the behavior is formed through appeals to “two interconnected elements: individuals’ interpersonal sensitivities and concerns, and the underpinning socio-moral order”. The first element – individuals’ interpersonal sensitivities and concerns – is monitored through rapport management, which is comprised of three interlinked dimensions: face sensitivities, sociality rights and obligations, and interactional goals (Spencer-Oatey 2008), while the second element pertains to the socio-moral order.

Spencer-Oatey and Kádár’s (2021) conception of face pays particular attention to how intentional or unintentional acts of facework carry the potential to affect face sensitivities; for instance, threatening or losing face, gaining face, or preserving face. They suggest that there is an array of face types, and each type has its own sensitivities. A summary of face types can be found in Spencer-Oatey and Kádár’s (2021: 162) book. My analysis of the participants’ open-ended responses will draw on the notion of face types to provide a more granular understanding of the face concerns relevant to advice-giving.

Sociality rights and obligations refers to what is supposed to happen between people in context. Often this supposed nature is formed due to repeated instances of occurrence (e.g., having the same or similar interactions regularly). Spencer-Oatey (2008: 15) argues that these expectations usually come from contractual/legal agreements and requirements, conceptualizations of roles and positions, as well as behavioral conventions, styles, and protocols.

Interactional goals refer to the objectives of a social interaction. Spencer-Oatey describes interactional goals as follows:

People often (although not always) have specific goals when they interact with others. These can be relational as well as transactional (i.e. task-focused) in nature. These ‘wants’ can significantly affect their perceptions of rapport because any failure to achieve them can cause frustration and annoyance (Spencer-Oatey 2008: 17).

Advice-giving may seem to have a very straightforward interactional goal at first (i.e., giving advice), but in reality, it can have both transactional and relational properties. I draw from this notion in my own understanding of the interactional goal of advice as well.

Spencer-Oatey and Kádár (2021) have also argued that evaluations of politeness behaviors invoke an underpinning socio-moral order. The socio-moral order involves two extremities along a continuum; with social rules on one end, and morality on the other. The moral order is described by Kádár and Haugh (2013: 6) as “a set of expectancies through which social actions and meanings are recognizable as such, and consequently are inevitably open to moral evaluation”. Understanding the moral order involves understanding how an interpersonal exchange forms and makes evident broader context-spanning norms and expectations. These in turn shape the moral order, or the social rules which we are all expected to adhere to. In other words, evaluations of the moral order are grounded in something common, yet outside of an individual context.

On the other end of the spectrum, the concept of social order is an attempt by Spencer-Oatey and Kádár (2021) to decouple evaluations of norms and expectations that do not necessarily invoke morality. Spencer-Oatey and Kádár (2021: 165) describe social rules as “local protocols and simply facilitate social interaction through a shared understanding of group etiquette”, while moral rules are defined as “behavior that is more wrong and more punishable, with the wrongness being more authority independent”.

Spencer-Oatey and Kádár’s (2021) politeness evaluation framework has not been used, as far as I am aware, to analyze evaluations of advice-giving. Nevertheless, my experiment’s question item “In up to three sentences, please describe what you think characterizes good advice”, prompts participants to tap into their perceptions of the socio-moral order of advice-giving, i.e., conventions and etiquette (social order), and also what they believe are right and wrong (moral order). Moreover, responses to this item could also indicate the types of face that are ideal, as well as deepen our understandings of the interactional goal(s) of advice-giving. On the other hand, responses to the experiment question “In up to three sentences, please describe what you think about the given advice” can be analyzed in relation to how well participants believed that Advicegiver101 has adhered to face sensitivities, sociality conventions, interactional goals, and the socio-moral order within the context of this particular online advice-giving scenario. Data captured from both of these questions can therefore be analyzed under the politeness evaluation model.

3.5 Analyzing the open-ended responses

The analysis of the open-ended responses started with an inductive approach, where the data was scrutinized without any preconceptions of how it would or should be coded. Instead, categories were coded and labeled as they emerged, which were then recoded into recurrent themes. The early stages of my coding are therefore similar to what Charmaz (2014) describes as initial coding, where the first round of coding occurs during the initial readthrough of the data. During the initial coding stage, lexical signals were used to identify coding labels, i.e., lexis carrying similar semantic meaning or recurrent lexical items. During subsequent rounds of coding, the coding labels and categories that emerged from the initial coding were further grouped into thematic categories.

4.1 Face sensitivities

4.2 Sociality rights and obligations

The open-ended responses have also provided insights into what is to be expected in an advice-giving situation. Participants repeatedly mentioned that advice-giving should be novel and/or specific.

4.3 Interactional goals – transactional

Offering information can be considered the task-oriented goal of advice-giving. While offering information, the advice-giver needs to not only be mindful of relevant face sensitivities (in particular, autonomy face, competence face, inclusion face, reliability face, as my data has suggested), but they also ultimately need to offer information that is viewed as high quality and task-fulfilling (i.e., demonstrating utility). To this end, participants also provided evaluations related to the perceived task fulfillment of the advice.

4.4 Interactional goals – relational

Besides offering information, advice can also take on the role of emotional support. Emotional support is related to an advice-giver’s demonstration of their warmth qualities, as well as their attempts to cater to the inclusion face of themselves and/or the advice-seeker. Many participants evaluated the advice positively in achieving its relational goals. Examples which discuss the relational goal fulfillment of the advice are shown as follows.

1. The advice shows empathy with the affected person and seeks a positive approach without pointing any blame of her.

2. The wording is very empathetic and attempts to assure the advice seeker that this problem is a normal one.

3. It is emphatic and is given in a compassionate voice that I would hope to hear if in the same situation.

As these replies suggest, discussions of relational goals also often go together with discussions of inclusion face. Perceptions of relational goal fulfillment via catering to inclusion face sensitivities were generally regarded positively, as the following excerpts highlight.

1. She (advicegiver101) seems like she understands what the mother (adviceseeker22) is going through

2. Appeared to be considerate of the feeling of the person posting

3. Advicegiver101 sounds like they understand the situation

4.5 The advice goal continuum

Interestingly, some participants separated the relational and transactional goals, while also emphasizing the value of one over the other. In the following responses (emphasis mine), these participants evaluated Advicegiver101’s attempts at fulling the interactional goal of advice-giving as unsatisfactory, due to a perceived overfocus on the relational goal.

1. There is nothing in the advice given which someone could not work out for themselves. It is more about making the questioner feel better about themselves than about resolving the real problem.

2. It is reasonable advice, but fairly obvious in nature. It will, however, help to reassure Adviceseeker that there are some simple things they can do which might help the situation. The advice is more of a help to Adviceseekers emotional state than practical help with the relationship issues she has.

3. It was quite broad and I felt like the advice giver was just trying to make the advice seeker feel better rather than actually giving much advice

These responses offer additional evidence that interactional goals are scalable, and provide further support to Lu’s (2024) argument that relational goals are less important for certain audience types (e.g., auditors and overhearers).

4.6 The socio-moral order

Commonly discussed moral foundations in the literature include care-harm, fairness-cheating, loyalty-betrayal, authority-subversion, and purity-degradation (Graham et al. 2018; Haidt and Kesebir 2010; Spencer-Oatey and Kádár 2021). My results suggest that fairness is an important moral concept related to advice-giving, which can be linked to participant appeals to impartiality and being non-judgmental.

4.7 Politeness evaluations related to each condition

The between-subjects design of my experiment allows me to also identify which evaluations were made in response to which advice text manipulation, while my coding of the politeness evaluation themes were based on instances of occurrence, which, combined, allows for cross comparison. The next and final section of this article will outline evaluations that are associated to the manipulations of the advice text (e.g., whether the participants were exposed to the “no moves” condition, background move condition, encouragement move condition, or “all moves” condition). The sunburst charts in Figures A1-A4 in the appendices depict the relative frequency of evaluation types across the four experimental conditions.

I will explore how these manipulations may have affected the participants’ evaluations of the advice. To do this, I use the “no moves” condition as a baseline from which the evaluation of the advice related to the manipulated moves can be considered (Figure A1). It is these datapoints which I compare the evaluations observed in the other three conditions against.