Morphosyntactic stereotypes of speakers with different genders and sexual orientations: an experimental investigation

2.1 Gender norms and roles in Mexican and Spanish societies

In general, Hispanic societies have traditionally been characterized by rigid gender roles and a divide between the masculine public sphere and the feminine domestic sphere (Herrera and Agoff 2018: 1325). However, the rapid transformation of both Mexico and Spain from rural, traditional societies to modern, industrialized ones has led to an increased participation rate of women in the job market (World Bank 2020). The rapid societal changes are reflected in differences towards gender roles and norms between generations (Calvo-Salguero et al. 2008; Rojas 2014) and socioeconomic groups (Herrera and Agoff 2018; Szasz 1998). Thus, different age groups have different views on gender roles and expectations (cf. Rojas 2014), which may also be reflected in the existence and strength of stereotypical beliefs and attitudes about language use (see Section 3.1).

Regarding gender roles and values, Mexican society and culture has traditionally been characterized by the concepts of machismo and marianismo (Stevens 1973), i.e., the exaltation of stereotypically masculine and feminine identities and ideals that are deeply entrenched in societal and, in particular, religious beliefs. Marianismo has been considered a form of ambivalent sexism fostering both misogynist and protective attitudes towards women (Hietanen and Pick 2015), while machismo refers to a traditional active and dominant role reserved for men, marginalizing other kinds of male identities, especially non-heterosexual ones (Prieur 1996; Rojas 2014).

While not addressing differences between Spain and Mexico directly, various statistical surveys comparing countries suggest that these two societies do differ with regard to gender roles and expectations towards women and men, and that Mexican society may foster more conservative values and attitudes. For example, the United Nations Gender Social Norms Index measures the occurrence of gender biases in the society (e.g., “Men make better political leaders than women”, “Men should have more right to a job than women”). In the Mexican sample analyzed for the survey (UNDP 2020), 87.70 % of the informants had at least one such belief, compared to 50.50 % of the Spanish informants. Surveys also show that religiosity plays a higher role in Mexico than Spain: in 2020, 77.7 % of Mexicans (INEGI 2020) and 59.2 % of Spaniards self-identified as Catholic (Centro de Investigaciones Sociológicas 2020: 21). Religiosity, in particular Catholicism, has been linked with more conservative attitudes towards gender and sexuality, including negative attitudes towards homosexuality (Štulhofer and Rimac 2009).

Based on the differences observed between Mexico and Spain with regard to attitudes towards gender, and religiosity, as well as more conservative attitudes observed in the older generations, our initial expectation in this study is that beliefs about gender differences in language use would be steeper among the Mexican sample and the older generations in our survey. This is reflected in our hypotheses (see Section 3.1).

2.2 Gender stereotypes

In social psychology, stereotypes can be defined as associations between a social group and certain characteristics (Fitzgerald et al. 2019), with gender stereotypes referring to attributions of characteristics along a single and particular axis of social differentiation. Social psychological research shows that women and men are judged differently on the two fundamental dimensions of person perception, i.e., warmth/communion/expressivity and competence/agency/instrumentality (Fiske et al. 2002; Spence and Helmreich 1978), the so-called “Big Two” (Abele and Wojciszke 2014) of person perception. Men are more likely to be ascribed task-oriented traits such as being assertive, competent, and independent, whereas women are more likely to be associated with socioemotional traits such as being caring, communal, and warm (Froehlich et al. 2020; Heilman 2001). These patterns of gender stereotyping are partly reflected in language-related stereotypes, with men being associated with a more impersonal and women a more personal communication style, as discussed in Section 2.3. Stereotypes make generalizing statements about all members of a particular group and negate the heterogeneity of individuals within a group as well as situational influences (Doosje et al. 1995; Judd and Park 1988). For example, not all men display an impersonal communication style, and even if a man uses impersonal constructions on average quite often, it is conceivable that there are situations in which he is more likely to use more personal constructions (e.g., in the presence of an interlocutor subjectively perceived as trustworthy).

Stereotypes related to sexual orientation can be considered a subtype of gender stereotypes because they follow a gender inversion heuristic (Kite and Deaux 1987): mental representations of gay men and lesbian women are characterized by low levels of gender conformity compared to heterosexual people. Collective stereotypes, i.e., widely shared beliefs of certain groups of people, apply not only to others but also to one’s own self (Simon and Hamilton 1994). For example, compared to gay men, not only are straight men described by others as more masculine, but they also describe themselves that way (Kachel et al. 2016; Lippa 2005).

In sociolinguistics, since Labov’s (1972) seminal distinction between indicators, markers, and stereotypes, the term stereotype has been used to refer to ideas about language use that speakers are very much aware of to the extent that they “may become the overt topic of social comment” and even “may become increasingly divorced from the forms actually used in speech” (Labov 1972: 180). Linguistic stereotypes, in the Labovian sense, differ from indicators and markers that are linguistic forms associated with groups of speakers that the speakers themselves are either completely unaware of (in the case of indicators) or only partially aware of (in the case of markers). Labov’s typology resembles the distinction between explicit and implicit stereotypes in social psychology: explicit stereotypes represent verbalized (or verbalizable) knowledge about groups that is subject to cognitive control and therefore susceptible to processes of social desirability (Stangor and Schaller 2000). These stereotypes are often captured via self-report, for example, by asking people what linguistic features they associate with certain groups of people (Kachel et al. 2018). Implicit stereotypes, on the other hand, operate subconsciously, are automatically activated when members of certain groups of people are present, and predict spontaneous rather than planned behavior (Ebert et al. 2014; Haslam et al. 2002).

Since the present research does not directly address the question of speakers’ awareness of the stereotypical differences in language use, we employ the term stereotype in the social psychological rather than in the Labovian sense, however not distinguishing between explicit and implicit stereotypes. We refrain from using the terms indicator and marker that are related with actual language use, which is not the object of the present study.

2.3 Empirical evidence and stereotypes of gender differences in language use

While tangible or purported differences in men’s and women’s use of language have received considerable attention in previous research, as part of the “difference approach” to language and gender (Tannen 1991), these studies tend to focus either on general features of language use such as quantity of speech (Crawford 1995), the use of certain lexemes such as swear words or “empty” adjectives (Lakoff 1975), the use of pragmatic expressions or politeness strategies such as mitigation or hedges (Coates 2004), or the preference for personal/interactional aspects of communication as opposed to conveying information (Holmes 1995). Lakoff’s (1973, 1975 well-known description of “women’s language”, although based on her own impressions, is in line with the findings of empirical works on English (Berryman and Wilcox 1980; Berryman-Fink and Wilcox 1983; Edelsky 1976; Haas 1979; Kramer 1977; Siegler and Siegler 1976). These early studies on gender stereotypes suggest that female language is perceived as softer, more polite, weaker, emotional, gossip-centered, whereas males’ use of language is thought to be more aggressive, direct, and blunt.

Despite the vast body of research on language and gender, there are few references to morphosyntactic differences, apart from general observations that nonstandard grammar (Berryman and Wilcox 1980; Kramer 1977) and incomplete sentences (Berryman and Wilcox 1980; Haas 1979) are connected to female speech, whereas standard grammar has been linked to male speech (Berryman and Wilcox 1980). In particular, research on stereotypes related with morphosyntactic features in language use remains scarce, in particular in other languages than English. In the case of Spanish, Hadodo and Kanwit (2020) constitute a notable exception. Their study will be discussed in Section 2.4, since it also deals with sexual orientation.

In English, gendered variation in the use of morphosyntactic features has been studied using large text corpora consisting of texts authored by women and men. Argamon et al. (2003) analyzed a subset of the British National Corpus consisting of 604 documents and over 25 million words authored by male and female writers and pertaining to different formal genres (i.e., literary fiction and nonfiction). They found that female writers use personal pronouns more often than male writers – in particular first and second-person pronouns referring to the writer and the addressee – and argue that this finding can relate to a more intersubjective or involved feminine writing style as opposed to masculine writing, which they consider less intersubjective or involved. In addition, they found that the use of quotation marks after punctuation “indicates that the female texts introduce other people’s words into their writing more than the male texts do” (Argamon et al. 2003: 338). In other words, female writers in their data used more direct reported discourse than men. By consequence, we can assume that male writers may use more indirect reported speech.

In spoken Peninsular Spanish, tendencies similar to those reported by Argamon et al. (2003) in written English have been found by Serrano and Aijón Oliva (2014) and Posio (2016). Serrano and Aijón Oliva (2014: 247) report that women used the “objectifying”, i.e., generic or impersonal, second-person singular more frequently than men in their oral data, but not in their written media data. While the authors admit that these findings are based on limited data, they suggest that a more frequent use of this referential device by women in speech may be indicative of a communicative style involving a higher use of epistemic constructions and mitigation strategies to reduce the illocutionary force of the speech (Coates 1996), while the gendered distribution of news topics and genres may explain why male journalists use the second-person singular more than female journalists (Serrano and Aijón Oliva 2014: 247). Posio (2016) also found that female speakers of Peninsular Spanish used the impersonal second-person singular more frequently (i.e., with a normalized frequency of 12.11 occurrences per 1,000 words) than male speakers (i.e., with a normalized frequency of 7.30 times per 1,000 words).

Similarly, the use of first-person singular and plural has been reported to be more characteristic of female speech by Aijón Oliva and Serrano (2013) and by Posio (2016). Although the differences between male and female speakers regarding the use of first-person forms were smaller than for second-person forms in Posio’s (2016) data, the more frequent use of deictic (i.e., first and second) persons by women consequently means that male speakers must use a higher proportion of third-person forms than female speakers (given that all clauses must contain a verb form in some grammatical person).

Previous quantitative analyses like the ones cited above seem to corroborate the more qualitative findings – or even impressionistic observations – according to which male and female speakers differ in their use of morphosyntactic resources, potentially reflecting either differences in the choice of topics being discussed or the speakers’ personal involvement and, in particular, the ways in which they involve their interlocutor in the conversation. For example, Coates (2004: 133) argues that men tend to avoid self-disclosure and speak of “more impersonal topics such as current affairs, modern technology, cars or sport” when speaking with other men. Hirschman (1994: 434), in her pioneering study about female/male differences in communication, also argues that women use more deictic pronouns because they “tend to talk more about their own experiences and feelings, while the males tend to generalize and talk rather abstractly”. However, it is unclear to what extent such explanations apply to different speaker communities and discourse types.

Although various studies suggest that women use more deictic person forms than men, this observation does not hold in all kinds of data and, crucially, the use of a certain grammatical person depends on what functions that person form has in different textual genres. For instance, Posio and Heikkilä (2023), analyzing a corpus consisting of 11,775 articles from six European newspapers from 1960 to 2010, found that in journalistic texts female writers did not use more first-person singulars than men, but rather the other way round, although the gender difference was only significant in their Spanish data, not in the other languages included. A qualitative scrutiny of this finding suggests that male writers in Spanish newspapers adopt an authoritative stance more often than female writers, and they use the first-person singular to enforce their authoritative stance rather than to express personal involvement or mitigation (Posio and Heikkilä 2023). As is usual in gender-based variation, textual genres, mediums, topics, and roles condition the variation to a great extent, questioning whether the variation is based on gender or whether the role of gender is rather coincidental to something else.

Variation in the use of reported speech has also been studied in connection with gender. In Spanish, several studies have researched this topic using a pan-Hispanic corpus of sociolinguistic interviews (PRESEEA). They all report a higher frequency of reported speech in female speakers of different Spanish dialects: Fernández (2012) in Mérida (Venezuela), Gallucci (2018) in Caracas (Venezuela), San Martín and Guerrero (2013) in Santiago de Chile (Chile), De la Mora (2018) in Mexico City (Mexico) and Repede (2018) in Seville (Spain). Gender has also been considered when analyzing the distribution of the subtypes or reported speech, i.e., direct and indirect reported speech. Although direct reported speech is preferred over indirect reported speech in spoken Spanish, most studies have found that women use also indirect speech more frequently than men (e.g., Fernández 2012; Gallucci 2018; San Martín and Guerrero 2013), with the exception of the results reported by Repede (2018).

Other gender differences in the use of reported speech are found by De la Mora (2018) focusing on the constructions used to introduce reported speech: women use speech act verbs more often than men, whereas men employ more null markers. San Martín and Guerrero (2013) identify a gender-specific pragmatic function linked to a concatenated use of direct reported speech. According to these authors, when several instances of direct speech are found concatenated in storytelling parts, women tend to perform “mini-dramas”, i.e., dramatizations where they perform and switch between different speakers’ voices.

2.4 Empirical evidence and stereotypes of sexual orientation and use of morphosyntactic features

Speech differences and stereotypes involving sexual orientation have not been as extensively studied as their counterparts relating to gender (see Section 2.2). The literature aimed at describing gay and, to a lesser extent, lesbian speech has focused on studying phonetic features (see, e.g., Kachel et al. 2017, 2018). Research on morphosyntactic differences and stereotypes remains more limited, in particular for Spanish.

While previous studies on English suggest that there are culturally salient stereotypical representations of how gay male speech sounds (see, e.g., Mack and Munson 2012 and references therein), lesbian speech styles or stereotypical ideas of how lesbians sound are much less salient. In a study of phonetic features and perceptions of the speech of women with different sexual orientations, Barron-Lutzross (2018) notes that informants tend to be very consistent in their evaluations of certain female speakers as lesbians, and that these evaluations generally coincide with the actual sexual orientation of the speakers. However, it is difficult to pinpoint what speech attributes exactly cause this association and how.

One of the few morphosyntactic features of Spanish that has been studied in this respect is the choice of address pronouns: in Colombian Spanish, the address pronoun usted is associated with straight men and tú with gay men as well as with femininity (Cepeda Ruiz 2017; Zwisler 2017). Similar perceptions are also found in the Spanish of Costa Rica (Castillo and Marín 2009; Fernández-Mallat and Dearstyne 2021; Marín 2013). However, these results are not relevant for Peninsular and Mexican Spanish, where tú is the most frequently used address pronoun that encodes proximity and informality while usted is used in more formal or distant settings.

Feminine pronouns and feminine flexion in nouns and adjectives are reported to be used by gay men in Mexico (Eller 2019) and by the LGBT community in Spain (Navarro-Carrascosa 2021). However, it must be noted that this use of feminine flexion primarily occurs within the gay community. In both studies it is described to be part of what characterizes gay speech, rather than a resource of the general population to refer to gay men.

The work of Eller (2019) analyzes stereotypes of the gay male community in Mexico City using several methods to study the linguistic perceptions of people outside and within the gay community. They use sociolinguistic interviews to study speech stereotypes that gay men hold about themselves. Informants report mainly on the lexical expressions and the phonetic traits. To examine the stereotypes held by the general Mexican population, they carried out a survey where informants were asked to assess the sexual orientation of seven men based on their recorded speech. Respondents were able to successfully identify the straight men and three out of the four gay men included in the speech samples. The gay man they failed to identify was the one that was reported to have a “less stereotypical” gay voice. When informants were asked what “sounded gay” about the people that were identified as such, there was consensus in ascribing this sound to “effeminate” talk, a particular intonation and the pronunciation of the [s] sound.

Hadodo and Kanwit (2020) study how the diminutive is perceived in terms of gender, age, and sexual orientation of the speaker in Madrid (Spain). By means of a questionnaire, respondents assessed whether a sentence had been uttered by a girl, boy, woman, man, elderly man, or elderly women. In an open question task informants were asked about their perceptions about the use of diminutives in connection with the variables studied. The results indicate that a more frequent use of diminutives is perceived to be characteristic of women, children, and, indirectly, gay men: respondents considered this trait as “immature” and “childish” and thus also indexing gayness and femininity when used by men.

As seen in this and the previous section, the existence of stereotypical beliefs about interconnections between speakers’ gender or sexual orientation and their use of morphosyntactic features has received little attention so far. As Hadodo and Kanwit (2020: 148) state, “morphosyntactic traits should be observed in addition to phonetic features in research concerning gender and sexuality”.

3.1 Research questions and hypotheses

As seen in the previous section, the literature documents that there are actual differences as well as perceived stereotypes related to gender and sexual orientation. However, most of the work has focused on phonetic features and on the English language, leaving ample room to further research other languages and linguistic levels. This paper is conceived precisely as a contribution aimed to shed some light on gender and sexual orientation stereotypes ascribed to morphosyntactic traits in two different Spanish-speaking societies. Previous work on morphosyntactic differences regarding gender allows us to test whether they are also perceived as stereotypical features not only regarding gender but also sexual orientation. Moreover, comparing two different societies sharing the same language allows us to address the potential of this factor to impact the results on stereotypes. Thus, our research questions are the following:

1. Do Mexican and Spanish informants display linguistic stereotypes regarding the use of certain morphosyntactic features found to differ between speakers representing different genders or sexual orientations?

2. If such stereotypes are found, are they consistent with previous empirical findings?

3. Are there differences regarding the existence or strength of such stereotypes between the two countries, male versus female informants, or different age groups?

The review of previous studies permits us to elaborate three hypotheses:

In order to investigate morphosyntactic gender stereotypes and test the above-mentioned hypotheses, we developed a two-part questionnaire that aims for indirect, example-based stereotype measurement. This questionnaire is described in Section 3.2.

3.2 Morphosyntactic Gender Stereotype Questionnaire (MOGEST)

The first part of the Morphosyntactic Gender Stereotype Questionnaire, MOGEST-d, is discourse-related and consists of four items, each comprising one question with two response options. The response options consist of several sentences with varying morphosyntactic features for which different frequencies in the use between women and men have been reported in previous studies (e.g., Argamon et al. 2003; Posio 2016; see Table 1). Therefore, it could be assumed that one response option represents a stereotypically feminine and the other a stereotypically masculine variant of contents with the same truth value, i.e., in Labovian terms, two “alternate ways of saying ‘the same thing’” (Labov 1972: 188). The inclusion of various different features in each variant maximizes the contrast between each variant in a pair and, consequently, increases the likelihood of finding effects. For three of the four text pairs, the stereotypically masculine version was presented first (i.e., on the left-hand side of the computer screen), and the stereotypically feminine version was presented second (i.e., on the right), and vice versa for the other pair. For an overview on all MOGEST-d items, please see Appendix I, and for an analysis on how order of versions of MOGEST-d items affected the results, please see Appendix III and Section 5.

The second part of the questionnaire, MOGEST-s, consists of twelve pairs of sentences. Again, each pair is constituted by a stereotypically feminine and masculine version of the same content. However, only one morphosyntactic feature was varied within each pair of sentences: grammatical person, referential expression, (im)personal style, or type of reported speech (direct vs. indirect). By varying only one morphosyntactic feature within each sentence pair, gender stereotypes concerning individual features could be identified. Each of the four morphosyntactic features was represented by three sentence pairs. For half of the sentence pairs, the stereotypically feminine version was shown on the left and the stereotypically masculine version on the right side, and vice versa for the other half. For an overview of all MOGEST-s items, please see Appendix II. For an analysis on how order of versions of MOGEST-s items affected the results, please see Appendix III and Section 5.

In our study, the participants were randomly assigned to one out of three experimental conditions: they were asked to indicate which variant within the pair of texts (MOGEST-d) or sentences (MOGEST-s) had more likely been uttered by (a) a woman rather than a man (experimental condition: gender identity), (b) a gay man rather than a straight man (experimental condition: men’s sexual orientation), and (c) a lesbian woman rather than a straight woman (experimental condition: women’s sexual orientation). For the sake of comparability, all responses were recoded in such a way that higher scores correspond to higher levels of stereotype adherence, i.e., when the stereotypically feminine variant was selected by participants who were asked to identify a woman or a gay man, and the stereotypically masculine variant was selected by participants who were asked to identify a lesbian woman.

All MOGEST items (see Appendix I and II) were created in Peninsular Spanish and then localized to Mexican Spanish in order to adjust to lexical differences. The validity of both language versions was approved by several native speakers of each variety before the data collection.

3.3 Data and methods

Overall, N = 365 participants were recruited for the study using two crowdsourcing platforms (Clickworker and Amazon Mechanical Turk) and the social networks of the research assistants. The data were collected between July 1, 2020 and January 18, 2021. The informants also provided other linguistic and social psychological data that are not used in the current study. They were told that they were participating in a study examining the relationship between society and language, not highlighting the more specific research objective. According to the University of Helsinki guidelines, no statement or license from an Ethics Review Board was required for the study.

According to the original research question, part of the exclusion criteria was based on sociodemographic characteristics. Individuals who reported to be neither Mexican nor Spanish citizens (n = 13), held dual citizenships (n = 4), or fell outside the age categories (n = 11) were excluded. Because gender was included as a between-groups factor in the design, the one participant reporting that they did not identify as a woman or a man (n = 1; non-binary) was not included in the data analyses, as their inclusion would have resulted in a highly imbalanced distribution of self-reported gender identities. Another set of exclusion criteria was related to data quality measures and adherence to ethical guidelines. The following participants were excluded: those who did not proceed to the end of the study (n = 48), those who gave no consent to the terms of data use (n = 11), and those who showed certain patterns in their responses reflecting a low level of engagement (n = 4), did not answer attention checks correctly (e.g., “Please mark ‘I completely agree’ here to control your concentration”), or showed low self-ratings regarding concentration (n = 21).

The final sample comprises 252 participants who were subject to further data analysis. They were either Spanish (n = 132) or Mexican citizens (n = 120), self-identified as women or men (n_women = 117, n_men = 135) and were between eighteen and fifty-six years old (M = 36.10, SD = 10.35). Based on ethical considerations, we recruited only participants who were of legal age (>18). The upper age limit of fifty-six years was affected by the low number of older people registered on the microwork platforms that we used for recruiting participants. For the purposes of the present study, the participants were divided into three age categories each comprising twelve years: 18–30, 31–43, and 44–56. Although there is no wide consensus on the definition of age groups, these groupings can be considered as representing young adulthood, transition to midlife, and midlife, respectively.

Most of the participants stated that they had a job (n = 173) or were unemployed (n = 47). Some were studying at a university (n = 29) or reported going to school or being engaged in an apprenticeship (n = 3). With respect to sexual orientation, most participants self-identified as straight (n = 213), some as bisexual (n = 23), or lesbian/gay (n = 13), two as pansexual, and one as heteroflexible.

An a priori power analysis was done to test differences based on the three between-participant factors nationality (Spain vs. Mexico), gender (women vs. men), and experimental condition (gender identity, men’s sexual orientation, or women’s sexual orientation). To obtain a medium main or interaction effect (f = 0.25) for the factor with the largest number of levels (i.e., experimental condition) with a high test power (1 − β = 0.95) and a standard alpha error probability (α = 0.05), 251 participants are required overall. Test power calculation was conducted with G*Power (Version 3.1.9.4; Faul et al. 2007).

Participants were first presented with study information and terms of participation (e.g., voluntariness, anonymity). Those who consented were asked to provide data on several sociodemographic characteristics (e.g., gender, age, nationality). Subsequently, participants were randomly assigned one of three experimental conditions (gender identity, men’s sexual orientation, women’s sexual orientation) and were instructed to fill in the MOGEST questionnaire (see Section 3.2). We also collected data on further psychological characteristics of the participants not relevant for the present study. Hidden in the questionnaire there were three items to control for the participants’ attention (e.g., “Please mark ‘I completely agree’ here to control your concentration”). Finally, we also asked the participants to indicate their subjective level of concentration on a seven-point scale ranging from 1 (no concentration at all) to 7 (highly concentrated). At the end, the participants were informed about the exact purpose of the study (i.e., investigating gender stereotypes between Mexican and Spanish women and men), and were asked to consent on the data use again. The study lasted approximately 40 min and participants were financially compensated with 3–5 euros.

4.1 MOGEST-d analyses

One-sample t-tests against chance level were used to determine whether participants selected gendered discourses varying in morphosyntax in a stereotypical manner. Whether stereotype-consistent responses differed between experimental conditions was tested by applying one-factorial ANOVAs and simple-effects tests in case of significant group differences. A combined illustration for both analyses is presented in Figure 1.

Figure 1:

Percentage of stereotype-consistent responses for each MOGEST-d item separated for experimental conditions (n = 252). Note. The item labels contain two kinds of information: (a) the item number corresponding to the position of the item in the questionnaire (e.g., 01 represents the first item that the participants were faced with) and (b) whether the stereotypically masculine (m) or the feminine (f) item version was presented on the left side of the screen and therefore was presented first to the participants. Percentages of stereotype-consistent responses above chance level (0.5) indicate that the stereotypically feminine (compared to the masculine) item version was attributed to a woman or a gay man more often and that the stereotypically masculine (compared to the feminine) item version was attributed to a lesbian more often. Asterisks below the bars indicate that the respective percentage significantly differed from the chance level (one-sample t-test against a fixed value of 0.5). Asterisks placed next to the brackets mark significant differences between experimental conditions (simple-effects tests with Bonferroni adjustment). Error bars represent 95 % CIs. ^*p < 0.05, ^**p < 0.01, ^***p < 0.001.

Five out of twelve one-sample t-tests against chance level (0.5) were significant, t(82) > 2.37, p < 0.020, Cohen’s d > 0.26, in the expected direction: the stereotype-consistent variant within a text pair was selected significantly more often than the stereotype-inconsistent variant. There were medium to large effects for gender identity as experimental condition (Cohen’s d ranging between 0.52 and 0.94), but small effects for the participants assigned to men’s (Cohen’s d = 0.26) and the women’s sexual orientation condition (Cohen’s d = 0.28). In other words, participants with gender identity as experimental condition attributed the “feminine” variant to a woman far more often than the participants with the men’s and women’s sexual orientation as experimental condition attributed the “feminine” variant to a gay man or the “masculine” variant to a lesbian woman. Interestingly, there seems to be a clear divide for effects of gender identity versus men’s and women’s sexual orientation between items: while participants with gender identity as experimental condition responded stereotype-consistently for items number 1, 3, and 4, participants with men’s and women’s sexual orientation as experimental condition selected the stereotype-consistent version of item 2 significantly more often than could be expected by chance. This inconsistency is discussed in Section 5.

We additionally observed differences between experimental conditions only for those three items where the stereotype-consistent responses in the gender identity condition differed from chance level, F(2,249) > 4.42, p < 0.013, η² > 0.034. For item number 3, no significant group differences occurred, p > 0.1. Simple-effects tests showed that the likelihood of stereotype-consistent responses was higher in the group of participants with gender identity as experimental condition than in the group of informants with women’s sexual orientation as experimental condition for all three items, p < 0.010. For the first item, the difference between the groups with gender identity and men’s sexual orientation as experimental condition was significant as well, p = 0.033.

Overall, there were no substantial intercorrelations between the four MOGEST-d items (for an overview, see Appendix VI), with one exception: the third and the fourth item showed a high correlation (r = 0.75, p < 0.001; all others, |r| < 0.07, p > 0.264). Looking at the three experimental conditions separately, this effect is clearly driven by the group of participants assigned to men’s sexual orientation as experimental condition (r = 0.88, p < 0.001). The correlations for the group of participants assigned to the women’s sexual orientation as experimental condition (r = 0.33, p < 0.003) and for the group assigned to the gender identity as experimental condition (r = 0.23, p = 0.033) were significantly smaller, z > 6.47, p < 0.001. There was no significant difference between experimental conditions for any other MOGEST-d item correlations, |z| < 0.78, p > 0.219.

Since MOGEST-d items differed strongly in the extent to which they elicited stereotype-consistent responses, the responses depended on the experimental condition, and only a few significant intercorrelations occurred between the items, it seems difficult to integrate the test items to one scale. This judgment is further supported by principal axis factoring and a reliability analysis. Conducting a PAF to explore how many factors underly the MOGEST-d, Kaiser-Meyer Olkin criterion = 0.504 indicates low sample adequacy. As a result, the PAF failed. Connectedly, the internal consistency of the overall scale was low (Cronbach’s α = 0.424), which can be interpreted as poor reliability. Thus, integrating the single MOGEST-d items into one scale was not successful.

4.2 MOGEST-s analyses

In contrast to MOGEST-d items, each MOGEST-s item contained a single morphosyntactic feature connected with either female or male language users in previous research. Four features were measured by clusters of three items each: (a) grammatical person, (b) referential expression, (c) (im)personal style, and (d) reported speech. Findings for the first two clusters are depicted in Figure 2a, and for the last two clusters in Figure 2b. For an overview of the correlations between MOGEST-s items, see Appendix VII.

Figure 2a:

Percentage of stereotype-consistent responses for MOGEST-s items measuring preference for grammatical person (A) and preference for referential expressions (B) separated for experimental conditions (n = 252). Note. The item labels contain three kinds of information: (a) the morphosyntactic feature with respect to which gender-associated stereotypes are measured (indicated by a capital letter), (b) the item number corresponding to the position of the item in the questionnaire (e.g., 01 represents the first item that the participants were faced with), and (c) whether the stereotypically masculine (m) or the feminine (f) item version was presented on the left side of the screen and therefore was presented first to the participants. Percentages of stereotype-consistent responses above chance level (0.5) indicate that the stereotypically feminine (compared to the masculine) item version was attributed to a woman or a gay man more often and that the stereotypically masculine (compared to the feminine) item version was attributed to a lesbian more often. Asterisks below the bars indicate that the respective percentage significantly differed from chance level (one-sample t-test against a fixed value of 0.5). Asterisks placed next to the brackets mark significant differences between experimental conditions (simple-effects tests with Bonferroni adjustment). Error bars represent 95 % CIs. ^*p < 0.05, ^**p < 0.01, ^***p < 0.001.

Figure 2b:

Percentage of stereotype-consistent responses for MOGEST-s items measuring preference for personal versus impersonal style (C) and preference for reported speech expression (D) separated for experimental conditions (n = 252). Note. See note to Figure 2a.

For the cluster on grammatical person, more than 80 percent of the participants assigned to the gender identity condition attributed the “feminine” variant of all three items to a woman, t(88) > 6.95, p < 0.001, Cohen’s d > 0.74. Lower deviations of stereotype-consistent responses from chance level were found for the men’s sexual orientation condition for two out of three items, t(82) > 3.12, p < 0.003, Cohen’s d > 0.34. For women’s sexual orientation, the percentage of stereotype-consistent responses did not differ from what could be expected by chance, p > 0.1. Correspondingly, for all three items an effect of experimental condition was observed, F(2,249) > 3.31, p < 0.038, with small to medium differences between the groups (η² ranging between 0.026 and 0.089). The group of participants assigned to gender identity as experimental condition provided significantly more stereotype-consistent responses than the group of participants with women’s sexual orientation as experimental condition for all three items, p < 0.035, and the group of participants with men’s sexual orientation as experimental condition for one item, p = 0.005. Moreover, only one out of three possible correlations between the three items were significant (r = 0.43, p < 0.001; all others, r < 0.12, p > 0.069): the more stereotype-consistent responses the participants gave on one item, the more they did so on the other item. This effect was descriptively weaker for the gender identity condition (r = 0.24, p = 0.026) compared to men’s sexual orientation (r = 0.45, p < 0.001) and women’s sexual orientation condition (r = 0.44, p < 0.001), but the conditions did not differ significantly (|z| < 1.55, p > 0.061). For an overview on the intercorrelations between all MOGEST-s items, please see Appendix VII.

For the cluster of referential expressions, the findings did not match our hypotheses. We either found no deviations of stereotype-consistent responses from chance level, t(88) < 1.83, p > 0.071, Cohen’s |d| < −0.19, or we observed negative deviations two times in the sense that participants chose the stereotype-inconsistent variant significantly more often, t(82) > 2.19, p < 0.032, Cohen’s |d| < −0.24. There was no significant effect of experimental condition, F(2,249) > 2.74, p > 0.067, η² < 0.022. As for the associations between all items designed to represent referential expressions, they correlated significantly with each other and to a large degree (r > 0.58, p < 0.001). This effect was driven by the participants assigned to men’s sexual orientation as experimental condition (r ranging between 0.83 and 0.89): the difference to both other conditions was large for all three items (|z| > 5.57, p < 0.001).

For the cluster on (im)personal style, the group of participants assigned to women’s sexual orientation as experimental condition chose the stereotype-consistent option of all three items significantly more often than the other option, t(88) < 1.83, p > 0.071, Cohen’s |d| < −0.19. For gender identity and men's sexual orientation as experimental conditions, findings were more mixed. For two items, we found that participants assigned to gender identity as experimental condition picked the stereotype-consistent version more often, t(88) < 2.05, p < 0.043, Cohen’s d > 0.22, while participants with men’s sexual orientation as experimental condition showed no deviation from chance level, t(82) < 0.33, p > 0.742, Cohen’s d < 0.04. However, for the same item, both groups showed a negative deviation and selected the stereotype-inconsistent variant with a higher likelihood, t(82) > 3.92, p < 0.001, Cohen’s d > 0.43. For this item only, differences between experimental conditions were observed, F(2,249) > 15.72, p < 0.001, η² = 0.112. Participants assigned to women’s sexual orientation as experimental condition showed a substantially larger number of stereotype-consistent responses than participants with gender identity and men’s sexual orientation as experimental condition, p < 0.001. Regarding the relationships between items measuring (im)personal style, only two out of three items correlated significantly (r = 0.68, p < 0.001; all others, r < 0.10, p > 0.117). This effect, again, was driven by the group of participants with men’s sexual orientation as experimental condition (r = 0.86, p < 0.001), because the corresponding correlation was not significant for participants with gender identity (r = 0.12, p = 0.280), nor with women’s sexual orientation (r = 0.20, p = 0.074) as experimental condition

For the cluster on reported speech, participants with men’s sexual orientation as experimental condition showed a significantly higher percentage of stereotype-consistent responses for all three items than can be expected by chance, t(82) > 4.51, p < 0.001, Cohen’s d > 0.50. For the other two conditions, there was only one significant deviation from chance level: participants with gender identity as experimental condition showed more stereotype-consistent response than can be expected by chance, t(88) = 4.57, p < 0.001, Cohen’s d = 0.49; all others t(88) < 1.17, p > 0.246, Cohen’s d < 0.12. Significant differences between experimental conditions were observed for all three items, F(2,249) > 5.41, p < 0.005, η² > 0.042. Participants assigned to men’s sexual orientation as experimental condition showed more stereotype-consistent responses than participants with women’s sexual orientation as experimental condition for all three items, p < 0.007, and compared to participants in the gender identity group for two items, p < 0.013. As for the items representing referential expression, all items measuring reported speech intercorrelated with each other (r > 0.26, p < 0.001). However, compared to all previous correlational analyses, men’s sexual orientation did not cause the effect. Here, all correlations were significant for the gender identity experimental condition only (r > 0.27, p < 0.009).

When designing MOGEST-s, each item was assumed to represent one of four morphosyntactic clusters. Although MOGEST-s items varied in the way they elicited stereotype-consistent responses and differed in the extent to which they intercorrelated with each other, we conducted PAF in order to explore the number of factors measured by the items. After excluding six items with low KMO-specific values and communalities < 0.5, final PAF showed adequate sample fit (Kaiser-Meyer-Olkin criterion = 0.82). A single factor solution was most appropriate according to graphical scree-plot analysis and eigenvalue criterion > 1.0. A steep decline of explained variance occurred between the first (69 %) and the subsequent factor (8 %). All six items represented the factor well with factor loadings ranging from 0.75 to 0.92. The factor was composed of items originally designed to measure different clusters (A 10, B 01, B 05, B 09, C 08, and C 11). Although the reliability can be regarded as excellent according to a high internal consistency (Cronbach’s α = 0.91), the interpretation of the factor and its validity seems questionable since it is composed of three items (B 01, B 05, B 09) showing low mean stereotype consistencies (0.41–0.47).

5.1 Grammatical persons and referential expressions

As predicted by the results of previous research (Argamon et al. 2003; Posio 2016; Serrano and Aijón Oliva 2014) participants attributed the “feminine” variants, i.e., those with frequent use of first and second-person singular and pronominal subjects as opposed to noun phrase subjects to female speakers in items 1, 3, and 4 of the first part of the experiment (MOGEST-d).^[1] These items were presented as two alternative replies to a short question. For the sake of illustration, let us consider item 1, presented here as example (2):

Since our research did not directly address the reasons that made participants choose the stereotype-consistent option, we can only hypothesize that speakers have a stereotypical representation of female speech as more intersubjective or involved as opposed to male speech (see, e.g., Argamon et al. 2003). This interpretation is supported by the MOGEST-s items, where the use of first and second-person in all three items testing this feature was strongly associated with female speakers. However, we did not get similar results from the three items testing the use of noun phrase versus personal pronoun subjects. The lack of clear results from these items may be a consequence of the difficulty in construing natural-sounding alternatives for the versions with noun phrases rather than pronouns: it may be that participants reacted to the repetition of the noun phrase in the “masculine” variant rather than the type of subject used. Consider, for example, item 5, presented in example (3).

Regarding the inconsistency detected between the four items of the MOGEST-d, it is interesting to observe which features of item 2 (i.e., the only item that triggered more stereotype-consistent responses in participants with men’s sexual orientation and women’s sexual orientation as experimental condition than in participants with gender as experimental condition) may have caused this difference. Even though all MOGEST items were designed to be as gender neutral as possible, it is obvious that semantic and pragmatic features of the items may also affect participants’ responses.

In item 2, both variants were presented as possible answers to the question ‘What made you study economics?’ and the two alternative answers both list the same reasons (the speaker had friends studying economics and it is a study track leading to lots of different job opportunities). Our interpretation of the fact that this item yielded less stereotype-consistent responses in the “gender” experimental condition group is that studying economics is considered more prototypical of men rather than women, which would cause participants to hesitate between attributing this item to female versus male speakers more than with the other three items. On the other hand, participants in the two sexual orientation experimental condition groups did not have to choose between a female or a male speaker and were thus more likely to choose the more stereotype-consistent response, i.e., the “feminine” version for gay men and the “masculine” version for lesbian women. In retrospect, it might have been more suitable for the purpose of the present study to construct this item so that no specific field of study was mentioned (e.g., ‘What made you choose this study track?’). Another option that could be explored in future research would be to study the impact of semantic contents of the items in addition to morphosyntactic features and to include, for instance, items referring to stereotypically feminine and masculine study tracks.

Although the eventual effect of the order of presentation of the two variants in the MOGEST test was not among our initial research questions, we did observe an unexpected pattern: participants with gender identity and men’s sexual orientation as experimental conditions selected the stereotype-consistent variant more often when the stereotypically masculine version was presented first (i.e., on the left side of the screen; for a more detailed account, see Appendix III). One possible explanation is that presenting the masculine variant first is perceived as more stereotypical – this is typically the order used in dictionaries and forms with, e.g., masculine and feminine titles – and thus results in more stereotype-consistent answers. The observed effect may also relate to the effect known as “spatial agency bias” (Chatterjee 2002; Maass et al. 2009) according to which speakers of languages written from left to right tend to interpret leftmost figures as possessing more agency than rightmost figures; men are also stereotypically perceived as possessing more agency than women (Froehlich et al. 2020). However, more research would be needed to account for this finding.

5.2 Impersonality

The preference for impersonal style (i.e., the “masculine” variant) versus personal style (i.e., the “feminine” variant) show rather unexpected results: the “masculine” variant was more frequently attributed to women in two out of the three items measuring this dimension, and all three items were more frequently attributed to lesbian women rather than straight women. Only one of the items was associated more frequently with gay men rather than straight men. This seemingly chaotic result start to make sense when we look more closely at the different items. Items 8 and 11 (presented in example 4), where the “masculine” variant containing an impersonal construction was frequently attributed to women, are declarative sentences, while item 2 (presented in example 5), where the “masculine” variant was frequently attributed to lesbian women and the “feminine” variant to gay men, is an interrogative sentence.

It may be that the sentence type affects the interpretation of the impersonal construction: while Posio (2016) found that male speakers used more impersonal constructions instead of personal ones in sociolinguistic interviews – a less interactive and more monological discourse type than conversations – in dialogical situations impersonality may serve a mitigating function, as it reduces the speaker’s involvement. Mitigation, in turn, can be considered a prototypically feminine activity (Albelda Marco 2018), which may explain the association between female speakers and the impersonal variants of items 8 and 11.

However, in the case of the interrogative sentence in item 2 (presented here as example 5), several native speakers of Spanish that we consulted judged the whole sentence assertive and even aggressive rather than mitigating. The inquisitive tone of this request of information was considered to be augmented by the use of the impersonal se construction which is characteristic of formal rather than colloquial spoken registers in Peninsular Spanish (Posio 2015) in the stereotypically masculine variant. The domineering tone of this particular variant may have caused our informants to link it with certain stereotypes of lesbian women as more masculine presenting or aggressive than straight women (see Walker et al. 2012). Further research would be needed to elucidate to elucidate the existence of such stereotypes about how lesbian women speak.

5.3 Reported speech

The results show that for reported speech there was a clear association of the “feminine” variant with gay men, even more so than with women. In other words, our informants thought that the “feminine” variant, i.e., direct reported speech, was more likely to have been uttered by a gay man in all three items of MOGEST-s measuring this feature, whereas the association with women was only significant in one of the items. On the other hand, the “masculine” version (i.e., the use of indirect reported speech) did not present any strong associations with either gender or sexual orientation. For the sake of example, consider item 12, presented here as example (6).

The consistent association of direct reported speech (DRS) with gay men and, in particular, the lack of a similar pattern with women, contrasts with the ample evidence found in the literature. Several studies confirm that women use more DRS than men across different Spanish dialects (De la Mora 2018; Fernández 2012; Gallucci 2018; Repede 2018; San Martín and Guerrero 2013). However, this connection between women and DRS may constitute a linguistic indicator or marker in the Labovian sense, rather than a stereotype that speakers would be aware of. The strong association between gay men and DRS in our experiment seems to signal the existence of a belief according to which gay men frequently use DRS. Our results might even suggest that speakers associate enactment of sequences of reported speech, perhaps with performing different voices (cf. the “mini-dramas” discussed by San Martín and Guerrero 2013), by discourse practices favored by gay men. Once again, we signal the need for further research to elucidate the existence and extent of a stereotype relating speakers’ sexual orientation and the use of certain morphosyntactic structures.