Did Shy Trump Supporters Bias the 2016 Polls? Evidence from a Nationally-representative List Experiment
-
Alexander Coppock
Abstract
Explanations for the failure to predict Donald Trump’s win in the 2016 Presidential election sometimes include the “Shy Trump Supporter” hypothesis, according to which some Trump supporters succumb to social desirability bias and hide their vote preference from pollsters. I evaluate this hypothesis by comparing direct question and list experimental estimates of Trump support in a nationally representative survey of 5290 American adults fielded from September 2 to September 13, 2016. Of these, 32.5% report supporting Trump’s candidacy. A list experiment conducted on the same respondents yields an estimate 29.6%, suggesting that Trump’s poll numbers were not artificially deflated by social desirability bias as the list experiment estimate is actually lower than direct question estimate. I further investigate differences across measurement modes for relevant demographic and political subgroups and find no evidence in support of the “Shy Trump Supporter” hypothesis.
Appendix
Comparing Direct Question and List Experimental Estimates of Trump Support.
| Subgroup | N | Unadjusted estimates |
Adjusted estimates |
||||
|---|---|---|---|---|---|---|---|
| Direct question | List experiment | Difference | Direct question | List experiment | Difference | ||
| Entire sample | 5290 | 32.5 (0.8) | 29.6 (3.4) | 3.0 (3.4) | 33.6 (0.7) | 33.0 (2.6) | 0.6 (2.6) |
| Strong democrat | 726 | 4.5 (0.8) | −4.3 (7.5) | 8.8 (7.6) | 5.0 (0.6) | 0.8 (2.4) | 4.2 (2.4) |
| Not very strong democrat | 1140 | 9.2 (1.2) | 5.0 (7.0) | 4.2 (6.9) | 8.0 (0.7) | 2.5 (3.2) | 5.5 (3.2) |
| Lean democrat | 409 | 13.0 (2.1) | 8.4 (11.0) | 4.6 (11.1) | 13.0 (1.4) | 7.6 (5.9) | 5.5 (6.1) |
| Independent | 1131 | 20.1 (1.5) | 24.3 (8.0) | −4.2 (8.1) | 18.8 (1.4) | 26.5 (6.9) | −7.7 (6.9) |
| Lean republican | 351 | 73.4 (2.7) | 55.9 (11.0) | 17.6 (11.1) | 59.8 (2.4) | 48.5 (12.5) | 11.3 (12.3) |
| Not very strong republican | 990 | 68.1 (1.9) | 72.1 (8.2) | −4.0 (8.0) | 72.4 (1.3) | 73.2 (6.9) | −0.8 (6.9) |
| Strong republican | 543 | 90.4 (1.3) | 80.5 (9.7) | 9.9 (9.7) | 84.1 (1.3) | 89.6 (6.5) | −5.5 (6.6) |
| Less than high school | 157 | 30.7 (4.2) | −4.3 (18.4) | 35.1 (18.4) | 33.6 (4.2) | 15.3 (9.1) | 18.3 (9.7) |
| High school or some college | 2793 | 34.4 (1.1) | 35.8 (4.7) | −1.4 (4.6) | 36.2 (1.0) | 36.7 (4.2) | −0.5 (4.2) |
| College | 1465 | 30.1 (1.4) | 27.4 (5.0) | 2.7 (5.0) | 32.5 (1.3) | 35.5 (4.7) | −3.0 (4.6) |
| Graduate school | 875 | 25.8 (1.7) | 10.9 (6.7) | 14.9 (6.5) | 27.1 (1.6) | 20.5 (4.7) | 6.6 (4.5) |
| Below 20th income percentile | 1038 | 30.7 (1.8) | 15.6 (8.2) | 15.1 (7.8) | 33.3 (1.2) | 31.0 (4.0) | 2.4 (3.9) |
| 20th–40th Income percentile | 1361 | 36.6 (1.7) | 38.4 (6.3) | −1.8 (6.2) | 35.6 (1.1) | 34.6 (2.8) | 0.9 (2.7) |
| 40th–60th Income percentile | 936 | 34.0 (2.1) | 19.7 (7.3) | 14.3 (7.3) | 34.0 (1.3) | 34.8 (3.3) | −0.8 (3.1) |
| 60th–80th Income percentile | 1151 | 33.9 (1.8) | 40.7 (7.4) | −6.8 (7.4) | 35.4 (1.4) | 39.7 (5.0) | −4.3 (4.9) |
| Above 80th income percentile | 804 | 27.2 (2.0) | 30.5 (8.8) | −3.4 (9.0) | 27.5 (1.4) | 21.4 (4.7) | 6.1 (4.7) |
| Men | 2332 | 36.9 (1.3) | 33.5 (5.3) | 3.5 (5.2) | 38.0 (1.1) | 33.5 (3.7) | 4.5 (3.7) |
| Women | 2958 | 28.4 (1.1) | 26.4 (4.4) | 2.0 (4.5) | 30.1 (1.0) | 32.7 (4.6) | −2.6 (4.6) |
| White | 3544 | 39.4 (1.1) | 38.0 (4.1) | 1.4 (4.1) | 39.9 (0.9) | 40.5 (3.3) | −0.6 (3.3) |
| Black | 446 | 10.3 (1.7) | 16.6 (10.9) | −6.3 (11.0) | 11.2 (1.7) | 22.0 (9.0) | −10.7 (9.0) |
| Hispanic | 804 | 24.4 (2.0) | 2.8 (9.7) | 21.6 (9.4) | 26.2 (1.9) | 9.0 (5.5) | 17.2 (5.4) |
| Other race | 496 | 20.3 (2.2) | 24.0 (11.6) | −3.8 (11.2) | 20.7 (2.2) | 28.5 (10.1) | −7.8 (10.0) |
| Unlikely voter | 1420 | 24.2 (1.4) | 20.7 (7.1) | 3.5 (7.1) | 24.3 (1.3) | 24.9 (5.9) | −0.5 (5.9) |
| Likely voter | 3870 | 36.5 (1.0) | 33.6 (3.7) | 2.9 (3.7) | 37.0 (0.9) | 36.0 (2.9) | 1.0 (2.8) |
-
All estimates incorporate sampling weights.
-
Bootstrapped standard errors are in parentheses.
-
Adjusted direction question estimates are predictions from a logistic regression.
-
Adjusted list experiment estimates are predictions from Imai’s (2011) NLS regression model.
References
Aronow, P. M., A. Coppock, F. W. Crawford and D. P. Green (2015) “Combining List Experiment and Direct Question Estimates of Sensitive Behavior Prevalence,” Journal of Survey Statistics and Methodology, 3:43–66.10.1093/jssam/smu023Search in Google Scholar
Blair, G. and K. Imai (2012) “Statistical Analysis of List Experiments,” Political Analysis, 20:47–77.10.1093/pan/mpr048Search in Google Scholar
Curtice, J. (1997) “So How Well Did They Do? The Polls in the 1997 Election,” Journal of the Market Research Society, 39:449–462.10.1177/147078539703900303Search in Google Scholar
Dropp, K. (2015) “Why Does Donald Trump Perform Better in Online Versus Live Telephone Polling?”.Search in Google Scholar
Dropp, K. (2016) “Yes, There Are Shy Trump Voters. No, They Won’t Swing the Election,” URL https://morningconsult.com/2016/11/03/yes-shy-trump-voters-no-wont-swing-election/.Search in Google Scholar
Durand, C., A. Blais and S. Vachon (2001) “Review: A Late Campaign Swing or a Failure of the Polls? The Case of the 1998 Quebec Election,” The Public Opinion Quarterly, 65:108–123.10.1086/320041Search in Google Scholar
Frye, T., S. Gehlbach, K. L. Marquardt and O. J. Reuter (2016) “Is Putin’s Popularity Real?” Post-Soviet Affairs, 1–15.10.1080/1060586X.2016.1144334Search in Google Scholar
Glynn, A. N. (2013) “What Can We Learn with Statistical Truth Serum? Design and Analysis of the List Experiment,” Public Opinion Quarterly, 77:159–172.10.1093/poq/nfs070Search in Google Scholar
Hopkins, D. J. (2009) “No More Wilder Effect, Never a Whitman Effect: When and Why Polls Mislead about Black and Female Candidates,” The Journal of Politics, 71:769–781.10.1017/S0022381609090707Search in Google Scholar
Imai, K. (2011) “Multivariate Regression Analysis for the Item Count Technique,” Journal of the American Statistical Association, 106:407–416.10.1198/jasa.2011.ap10415Search in Google Scholar
Kuklinski, J. H., P. M. Sniderman, K. Knight, T. Piazza, P. E. Tetlock, G. R. Lawrence and B. Mellers (1997) “Racial Prejudice and Attitudes Toward Affirmative Action,” American Journal of Political Science, 41:402–419.10.2307/2111770Search in Google Scholar
LaBrie, J. W. and M. Earleywine (2000) “Sexual Risk Behaviors and Alcohol: Higher Base Rates Revealed Using the Unmatched-Count Technique,” Journal of Sex Research, 37:321–326.10.1080/00224490009552054Search in Google Scholar
Lax, J. R., J. H. Phillips and A. F. Stollwerk (2016) “Are Survey Respondents Lying About Their Support for Same-Sex Marriage? Lessons from a List Experiment,” Public Opinion Quarterly, 80:510–533.10.1093/poq/nfv056Search in Google Scholar
Lyall, J., G. Blair and K. Imai (2013) “Explaining Support for Combatants duringWartime: A Survey Experiment in Afghanistan,” American Political Science Review, 107:679–705.10.1017/S0003055413000403Search in Google Scholar
Mellon, J. and C. Prosser (2017) “Missing Nonvoters and Misweighted Samples: Explaining the 2015 Great British Polling Miss,” Public Opinion Quarterly, forthcoming.10.1093/poq/nfx015Search in Google Scholar
Miller, J. (1984) A New Survey Technique for Studying Deviant Behavior, Phd thesis., George Washington University.Search in Google Scholar
Payne, J. G. (2010) “The Bradley Effect: Mediated Reality of Race and Politics in the 2008 US Presidential Election,” American Behavioral Scientist, 54: 417–435.10.1177/0002764210381713Search in Google Scholar
Powell, R. J. (2013) “Social Desirability Bias in Polling on Same-sex Marriage Ballot Measures,” American Politics Research, 41:1052–1070.10.1177/1532673X13484791Search in Google Scholar
Streb, M. J., B. Burrell, B. Frederick and M. A. Genovese (2008) “Social Desirability Effects and Support for a Female American President,” Public Opinion Quarterly, 72:76–89.10.1093/poq/nfm035Search in Google Scholar
©2017 Walter de Gruyter GmbH, Berlin/Boston
Articles in the same Issue
- Frontmatter
- Articles
- The Representativity of Election Polls
- Less Supervision, More Satisficing? Comparing Completely Self-Administered Web-Surveys and Interviews Under Controlled Conditions
- Did Shy Trump Supporters Bias the 2016 Polls? Evidence from a Nationally-representative List Experiment
- Understanding the 2016 US Presidential Polls: The Importance of Hidden Trump Supporters
- Roosevelt Predicted to Win: Revisiting the 1936 Literary Digest Poll
- Predicting the Brexit Vote by Tracking and Classifying Public Opinion Using Twitter Data
- What do we Lose with Online-Only Surveys? Estimating the Bias in Selected Political Variables Due to Online Mode Restriction
Articles in the same Issue
- Frontmatter
- Articles
- The Representativity of Election Polls
- Less Supervision, More Satisficing? Comparing Completely Self-Administered Web-Surveys and Interviews Under Controlled Conditions
- Did Shy Trump Supporters Bias the 2016 Polls? Evidence from a Nationally-representative List Experiment
- Understanding the 2016 US Presidential Polls: The Importance of Hidden Trump Supporters
- Roosevelt Predicted to Win: Revisiting the 1936 Literary Digest Poll
- Predicting the Brexit Vote by Tracking and Classifying Public Opinion Using Twitter Data
- What do we Lose with Online-Only Surveys? Estimating the Bias in Selected Political Variables Due to Online Mode Restriction
