Sexist jokes don’t appear to increase rape proclivity among men high in hostile sexism: Evidence from two pre-registered direct replications of Thomae and Viki (2013)

1 Introduction

Prejudiced norm theory (Ford and Ferguson 2004; Ford et al. 2001) explains the apparent prejudice-releasing function of disparagement humor through several interacting social and psychological processes (e.g., Ford et al. 2008): According to the theory, disparagement humor activates a humorous mindset in the audience. For this non-critical humor mindset to be activated, listeners must interpret and accept the communication as funny and, consequently, not assess the communicated content in a serious or critical manner. Moreover, listeners must interpret the social situation in which disparagement humor is presented as tolerant towards the expression of prejudices (Ford and Ferguson 2004). Together, the humor mindset and the interpretation of the social norms increase the likelihood of observers expressing their pre-existing prejudices.

Evidence for the prejudiced norm theory has been established in a long program of empirical research (e.g., Ford 2000; Ford et al. 2001, 2002, 2008). Moreover, in recent years, the research efforts putting prejudiced norm theory to the test have evolved from an initial focus of disparagement humor targeting women (Ford et al. 2008, 2013; Thomae and Viki 2013; Viki et al. 2007) to a wider range of groups (Muslims, gay men, politicians, lawyers; e.g., Ford et al. 2013; Mendiburo-Seguel and Ford 2019; Thomae and Page 2016). An important theoretical development was the finding that disparagement humor particularly harms the attitudes towards groups whom societal attitudes are ambivalent (Ford et al. 2013; Mendiburo-Seguel and Ford 2019). Only for groups occupying a position of normative ambiguity disparagement humor increases the acceptability of prejudice expressed against the group (e.g., Muslims, Ford et al. 2013; gay men, Mendiburo-Seguel and Ford 2019). For groups generally perceived without ambiguity (e.g., terrorists, Ford et al. 2013; politicians, Mendiburo-Seguel and Ford 2019), adopting a humor mindset does not seem to increase the expression of prejudices against them.

Arguably, women have been a societal group viewed with normative ambiguity (see Glick and Fiske 1996, 2001) and sexist humor has been linked to men’s rape myth acceptance (Viki et al. 2007), decreased attributions of sexism (Mallett et al. 2016) and the triggering of self-objectification among women (Ford et al. 2015). But can simply reading a series of sexist jokes enhance the reader’s self-reported likelihood of engaging in hypothetical acts of rape? A number of studies suggests so (Romero-Sánchez et al. 2010; Thomae and Viki 2013; Viki et al. 2007). These studies report that, after being presented with a series of sexist jokes, men report an increased willingness to engage in hypothetical acts of rape. That is, sexist humor increases self-reported rape proclivity (Malamuth 1981).

Furthermore, research has identified several moderators for this effect. Thomae and Viki (2013; TV2) had participants complete a measure of hostile sexism (HS; Glick and Fiske 1996) before asking them to read a series of jokes. In the sexist humor condition, participants read four sexist jokes (e.g., “Q. Why do women have small feet? A. So they can get closer to the sink!”) and one non-sexist joke (e.g., “Q. Why was the leper stopped for speeding? A. He couldn’t take his foot off the accelerator!”), while participants in the control condition read four non-sexist jokes and one sexist joke. The study then measured the participants’ self-reported proclivity towards committing hypothetical acts of rape and found that rape proclivity increased following exposure to sexist jokes, but only for men high in HS. Likewise, Viki et al. (2007) also reported that rape proclivity increased, but only when the rape victim was an acquaintance (versus a stranger). Romero-Sanchez et al. (2010) found the effect only when the reader did not have an aversive response to the joke. More recently, Romero-Sánchez et al. (2017) extended this research by demonstrating that anti-female sexist humor enhanced rape proclivity especially when the joke was told by a woman rather than a man. If accurate, these results suggest that merely reading a series of sexist jokes can increase the willingness of some men to engage in acts of rape, at least in some situations having what researchers have called “devastating” consequences (p. 12, Thomae and Pina 2015). Support for the auxiliary hypothesis that sexist humor increases rape proclivity, especially for men high in hostile sexism, is thought to corroborate the prejudiced norm theory (Romero-Sánchez et al. 2010; Thomae and Viki 2013).

Science is a cumulative process, and replicability is a core feature of a thriving field of science. Yet, large-scale replication efforts have highlighted the difficulty of reproducing psychological findings (Open Science Collaboration 2015). The literature investigating the impact of sexist humor on rape proclivity includes a number of successful internal and direct replications (e.g., Romero-Sánchez et al. 2010; Thomae and Viki 2013, Study 3). However, to our knowledge, there has been no attempted pre-registered replications of the effect. Pre-registration requires experimenters to specify their hypotheses, materials, and analytic plan prior to recruitment. Furthermore, studies that do not obtain support for the experimental hypothesis will often remain unpublished (the file-drawer problem; Rosenthal 1979) which can result in the field overestimating the size of the effect if one exists. We therefore conducted two pre-registered replications of Thomae and Viki (2013, Study 2; hereafter referred to as TV2).

To our knowledge, there have been two direct replications of TV2. In their third study, Thomae and Viki (2013) conducted a replication while controlling for the sexual content of the jokes. They reported that the significant interaction between joke condition and HS replicated successfully, although the main effect of joke condition was no longer significant. Additionally, Romero-Sanchez et al. (2010) attempted to replicate the study using jokes from the Escala de Apreciación Humor (Humor Appreciation Scale; Carretero-Dios et al. 2009), and while they concluded that sexist jokes were shown to enhance rape proclivity relative to non-sexist control jokes, they failed to replicate the interaction between joke condition and hostile sexism. However, sensitivity analyses indicate that Romero-Sanchez et al. (2010) only had sufficient power to detect medium to large effect sizes, and so might not have been sufficiently powered to detect the interaction effect.

Our first replication used identical materials to TV2, while the second replication used updated jokes based on a recent pilot study conducted to ensure that the jokes were considered equally funny and sexist to a contemporary sample. We report how we determined our sample sizes, all data exclusions, all manipulations, and all measures used in the studies. Experimental pre-registrations, data and code are available on the Open Science Framework (https://osf.io/7g3t2/).

2 Deviations from original study

The original study reported using a 7-point Likert scale to measure how funny and sexist participants considered the jokes, but reviewing the original dataset showed they had in fact used a 9-point Likert scale (this has been confirmed by the first author of TV2, an author on the current article). This was only noticed after the completion of the replication studies. We have therefore scaled the results accordingly whenever the original study’s results were used to model the experimental hypotheses when calculating Bayes factors.

While Replication 1 used otherwise identical materials to the original study, Replication 2 deviated in several ways. First, we used the full Social Desirability Scale (Crowne and Marlowe 1960) instead of the five items used by TV2 (items: 6, 12, 21, 28, 30) as we felt using the full validated and reliable measure of SDS would be a better measure than the selection of items used in both the original study and Replication 1. The first author of TV2 confirmed prior to data collection that this change was not expected to impact the results. Second, while the jokes used in Replication 1 were identical to those used by TV2 (originally used by Ford 2000, and Ford et al. 2001), Replication 2 used different jokes (see Appendix A for all jokes used in the current studies). These jokes were selected based on the results of a recent pilot study conducted by the lead author of TV2, in an effort to ensure that the jokes used across conditions were considered equally funny but that the sexist jokes were considered to be more sexist. We updated the jokes for Replication 2 as the pilot study for the original jokes had been conducted approximately 15 years prior (Thomae 2004).

3 Deviations from pre-registration

4 Analyses

Bayes factors were conducted to determine whether the results of the replication studies supported the experimental hypotheses (H1, modelled using the raw effect sizes reported by TV2) or the null hypothesis (H0, modelled assuming effect size = 0). Bayes factors provide a continuous measure of how probable the obtained data are under one hypothesis relative to another. Bayes factors >3 or <0.33 were interpreted as “moderate” evidence for the experimental hypothesis and null hypothesis respectively, and Bayes factors >10 or <0.10 were interpreted as strong evidence for the experimental hypothesis and null hypothesis respectively. Bayes factors between 0.33 and 3 were interpreted as weak and inconclusive. Bayes factors are a continuous measure, and we report these thresholds only to make transparent the decision criteria used throughout the replication studies. The result of the Bayes factor depends on the specific models used to represent the hypotheses, and so we also report robustness regions to indicate the smallest and largest effect size that could be used to model H1 and still yield the same conclusion as the model of H1 used to calculate the reported Bayes factor (for an introduction to the use of robustness regions, see p. 76 McLatchie et al. 2020).

In addition to reporting the results of the pre-registered Bayes factors and significance tests, and in line with other recent replication attempts in social psychology (see e.g., Gervais et al. 2020), we also report the results of a small-telescopes approach and a mini meta-analysis. The small telescopes approach assesses the detectability of effects by retrospectively evaluating the power of the original study to detect the effect sizes obtained by the replications. If an original study lacks sufficient power to detect the replication’s effect size (here, we consider sufficient power to be >0.80), the replication is considered unsuccessful (Simonsohn 2015). We also report the results of a mini meta-analysis of the original study and the two replication studies. The meta-analysis provides the best estimate of the effects from the three studies (we provide the results with and without the original study), and Bayes factors were conducted on the meta-analytic results. The mini meta-analysis serves only to provide an indication as to whether the data from TV2 and the current replication studies relatively support the experimental hypothesis, the null, or are inconclusive, and is not meant to summarize the literature overall.

5 Design

The design of the replications was identical to TV2, with only a few minor exceptions detailed above. A between-subjects design was employed with participants randomly allocated to either the sexist joke condition (4 sexist jokes, 1 non-sexist joke) or the control joke condition (4 non-sexist jokes, 1 sexist joke) using the online research suite Qualtrics (www.Qualtrics.com). Participants for both replications were recruited via Amazon’s Mechanical Turk online recruitment pool (www.mturk.com; see Paolacci et al. 2010). The replications took approximately 10–15 min to complete, and participants received $1.25 for their participation.

6 Results

The results of the original and replication studies are presented in Table 1.

6.1 Replication 1

In contrast to the results reported by TV2, there was moderate evidence for the null hypothesis against a model of H1 predicting a positive raw slope for the interaction between HS and joke condition, B _H(0,0.09) = 0.17, RR[0.05, ∞] (see Figure 1, centre panel). Simple slope analyses confirmed that the data provided moderate evidence for the null hypothesis for participants high in HS, b = −0.18, SE = 0.09, p = 0.05, B _H(0,0.24) = 0.13, RR[0.09, ∞], although the data were inconclusive for participants low in HS, b = 0.15, SE = 0.11, p = 0.15, B _H(0,0.24) = 1.73, RR[0, 1.63]. The data from Replication 1 were inconclusive as to whether joke condition predicted rape proclivity, B _H(0,0.14) = 0.36, RR[0,0.15], and whether exposure to sexist jokes (M = 2.36, SD = 1.42) relative to control jokes (M = 2.40, SD = 1.42) enhanced rape proclivity, F(1,265) = 0.04, p = 0.85, η ² _G  < 0.001, B _H(0,0.26) = 0.57, RR[0, 0.52].

Figure 1:

Interaction plots showing the impact of hostile sexism on rape proclivity as a function of joke condition for the original study (left), replication 1 (centre) and replication 2 (right). Created using the R package interactions (Long 2019).

Figure 2:

The perceived funniness (left panel) and sexism (right panel) of the sexist and control jokes reported by Thomae and Viki (2013) and the current replication studies. Means (horizontal bars) and 95 % Highest Density Intervals are presented. Created using the R package yarrr (Phillips 2017).

6.1.1 Small telescopes approach

For the key outcome of the HS*joke condition interaction, the original study had a power of 0.30 for the effect size obtained by Replication 1. Furthermore, for the outcome of whether joke condition predicted rape proclivity, the original study had a power of 0.03 for the effect size obtained by Replication 1. The original study would therefore be considered severely under-powered to detect the effect sizes obtained by the current research.

7 Mini meta-analysis

A mini meta-analysis was conducted with Bayes factors to assess whether the data from all three studies, as well as the two replication studies only, supported the experimental hypothesis or the null hypothesis. As one should not use the results of a study (e.g., the results of TV2) to model H1 for a Bayes factor that assesses results that include that study, the experimental hypothesis was specified using heuristics outlined by Dienes (2019) that provide an approximate prediction for a theory. Using the ratio-of-scales heuristic, the results of the meta-analysis revealed that the results from all three studies indicated that the interaction between joke condition and HS provided strong evidence for the null hypothesis, b _Meta = −0.01, SE_Meta = 0.03, B _H(0,0.5) = 0.04, RR[0.06, ∞], when predicting rape proclivity, and similarly strong support for the null when including only the replications, b _Meta = −0.08, SE_Meta = 0.04, B _H(0,0.5) = 0.02, RR[0.04, ∞]. The effect sizes were also considerably smaller and in the opposite direction than in the original study.

Furthermore, the combined evidence from all three studies was inconclusive as to whether joke condition predicted rape proclivity, b _Meta = 0.05, SE_Meta = 0.04, B _H(0,0.5) = 0.34, RR[0, 0.51], although the evidence from only the replications provided strong support for the null hypothesis, b _Meta = −0.005, SE_Meta = 0.05, B _H(0,0.5) = 0.09, RR[0.13, ∞]. Compared to the results obtained by the original study, the effect sizes were considerably smaller for the meta-analytic results, and in the opposite direction.

8 Exploratory analyses

Here we explore a number of factors that might have affected the outcome of the replications. These analyses were not pre-registered and should be considered exploratory.

9 Discussion

A series of studies have previously reported that exposure to sexist jokes can enhance rape proclivity among certain men (Romero-Sanchez et al. 2010, 2017; Thomae and Viki 2013; Viki et al. 2007). We attempted two sufficiently powered pre-registered direct replications of one such study, Study 2 by Thomae and Viki (2013), which reported that men who scored high in hostile sexism demonstrated increased proclivity towards rape following exposure to sexist jokes. Neither replication study successfully reproduced the results reported in the original study, and the combined datasets provide evidence for the null hypothesis for all key analyses over the alternative hypothesis.

For the critical test of the interaction between joke condition and hostile sexism, Bayes factors provided strong support for the null hypothesis, as did the evidence from only the replications. Similarly, the key null-hypothesis significance tests were not significant. Although Replication 1 did obtain a significant effect for the interaction between joke condition and HS, it was in the opposite direction to the results reported by TV2. Lastly, the small telescopes approach revealed that the original study was not sufficiently powered to detect the effect sizes obtained by the replications.

In summary, the results of our replication studies do not support the idea that sexist humor releases pre-existing hostile sexist attitudes (see Ford et al. 2008; Martineau 1972), at least not if expressed through self-reported rape proclivity (Bohner et al. 1998). Our findings do not necessarily imply that the interacting social and psychological processes as proposed by prejudiced norm theory no longer apply to the release of sexist prejudices, only that the mechanism investigated and replicated previously (Romero-Sanchez et al. 2010, 2017; Viki et al. 2007) does not seem to be reproducible in the current studies. Future research could investigate whether and where along the explanatory chain of social and psychological processes (levels of pre-existing prejudice, the activation of the non-critical humor mindset, the interpretation of the communication as funny, the non-critical assessment of the humor content, the interpretation of the social context as tolerant of sexism; see Ford 2000; Ford et al. 2008; Ford and Ferguson 2004; Ford et al. 2002; Ford et al. 2001) required for the prejudice-releasing function of sexist humor to take effect, changes may have occurred.

There are several possible reasons why the current replications did not successfully reproduce the findings reported by TV2. It is possible that the samples differed along dimensions we did not measure. For example, Romero-Sanchez et al. (2010) reported that joke condition only impacted on rape proclivity if participants did not find the jokes aversive. Sample differences between the original and the replication studies along such unmeasured moderator variables could explain why the effects did not successfully replicate. Still, using the same methodology as Thomae and Viki (2013), neither of the current replications support the idea that hostile sexism interacts with joke condition to increase men’s rape proclivity.

There was also some evidence that a relatively large minority of our participants did not pay attention to certain questions (e.g., when asked to enter a joke of their own). A weakness of the current replications was that we did not include attention checks to ensure participants were paying attention and answering questions in an appropriate manner, and this could have reduced the validity and reliability of the current results. Future studies should look to include attention checks throughout to ensure the quality of their data (for examples see Curran and Hauser 2019; Meade and Craig 2012). Although the current replications did not include such checks, our exploratory meta-analytic analyses showed that our conclusions were robust even after excluding participants suspected of not paying attention. There was also evidence that, despite our best efforts to match the jokes, the control jokes in the current study were perceived as more sexist than the control jokes in the original study. It is possible that the replications did not reproduce the results because the differences in the perceived sexism of the sexist jokes relative to the control jokes is less.

It is also possible that the effects of sexist humor on rape proclivity have been over-estimated by published findings in the literature. TV2 recruited 107 participants, which, given the average effect size in social psychology (d = 0.43, Richard et al. 2003), suggests the original study was likely underpowered. In conjunction with publication bias, such under-powered studies can result in published studies over-estimating effect sizes because only those studies with sufficiently large effect sizes to obtain significant results get published (resulting in the so-called winner’s curse; Button et al. 2013). Additionally, the p-values reported by TV2 were, despite being significant, relatively large (joke condition: p = 0.03, HS*joke condition: p = 0.05). Such large p-values are more likely under the null hypothesis (which is expected to produce a uniform distribution of p-values) than when genuine effects exist (Simonsohn et al. 2014). Finally, the evidence provided by the original study was relatively weak. Bayes factors show that although TV2 obtained significant results, the evidence in favor of the experimental hypothesis relative to the null was only just moderate for the effect of joke condition (B _H(0,0.33) = 3.99, RR[0.04, 0.46]) and was inconclusive for the interaction between joke condition and HS (B _H(0,0.33) = 1.85, RR[0, 1.92]).^[5]

Regardless of the reasons why the results reported by TV2 did not replicate, we encourage researchers to exercise caution before claiming that brief exposure to sexist humor increases men’s rape proclivity, or that hostile sexism moderates this effect, until further pre-registered and transparent research has been conducted to demonstrate the effect and the social and psychological steps preceding it (see Ford 2000, Ford et al. 2008; Ford and Ferguson 2004; Ford et al. 2002; Ford et al. 2001) are reproducible and can reliably obtained today.