Volume 8, Issue 1

Election polls are conducted in many countries during election campaigns. Provided such polls are set up and carried out correctly, they can give an accurate indication of the voting intentions of people. However, the last couple of years these polls seem to be less able to predict election results. Examples are the polls for the general election in the UK of 2015, the Brexit referendum in the UK, and the presidential election in the US of 2016. The polls in the UK and the US have all in common that they are either telephone polls or online polls. It is shown in this paper that both type of polls suffer from lack of representativity. The compositions of their samples differ from that of the population. This can have several causes. For telephone polls, problems are mainly caused by increasing nonresponse rates, and lack of proper sampling frames. Most online polls are based on samples from web panels that are recruited by means of self-selection instead of random samples. Such web panels also not representative. The paper analyses the shortcomings of these election polls. The problems are illustrated by describing the polls in the UK and the US in some more detail.

Although online surveys are becoming more and more prominent, the quality of the resulting data is still contested. One potential caveat of web surveys is the absence of an interviewer who controls the interview situation, can motivate respondents and prevent them from satisficing, i.e. answering questions with minimal cognitive effort. While there is evidence for differences between data gathered in interviewer-administered surveys and data from self-administered questionnaires, it has not yet been studied whether the sheer presence of an interviewer affects data quality. The present article addresses this research gap. Based on a recent panel study of young unemployed adults, we compare the results from a completely self-administered web survey with those from interviews which were self-administered but conducted in the presence of an interviewer. In particular, we look for differences concerning drop-out, speed, item-non-response, and item-non-differentiation. While we do find significant differences in drop-out rates, we do not find any evidence for interviewer-absence leading to less diligence in filling in the questionnaire. We thus conclude that the presence of an interviewer does not enhance data quality for self-administered questionnaires, but positively affects completion rates.

Explanations for the failure to predict Donald Trump’s win in the 2016 Presidential election sometimes include the “Shy Trump Supporter” hypothesis, according to which some Trump supporters succumb to social desirability bias and hide their vote preference from pollsters. I evaluate this hypothesis by comparing direct question and list experimental estimates of Trump support in a nationally representative survey of 5290 American adults fielded from September 2 to September 13, 2016. Of these, 32.5% report supporting Trump’s candidacy. A list experiment conducted on the same respondents yields an estimate 29.6%, suggesting that Trump’s poll numbers were not artificially deflated by social desirability bias as the list experiment estimate is actually lower than direct question estimate. I further investigate differences across measurement modes for relevant demographic and political subgroups and find no evidence in support of the “Shy Trump Supporter” hypothesis.

Following Donald Trump’s unexpected victory in the 2016 US presidential election, the American Association for Public Opinion Research announced that “the polls clearly got it wrong” and noted that talk of a “crisis in polling” was already emerging. Although the national polls ended up being accurate, surveys just weeks before the election substantially over-stated Clinton’s lead and state polls showed systematic bias in favor of Clinton. Different explanations have been offered for these results, including non-response bias and late deciders. We argue, however, that these explanations cannot fully account for Trump’s underperformance in October surveys. Utilizing data from two national polls that we conducted in October of 2016 ( n >2100 total) as well as 14 state-level polls from October, we find consistent evidence for the existence of “hidden” Trump supporters who were included in the surveys but did not openly express their intention to vote for Trump. Most notably, when we account for these hidden Trump supporters in our October survey data, both national and state-level analyses foreshadow Trump’s Election Day support. These results suggest that late-breaking campaign events may have had less influence than previously thought and the findings hold important implications for how scholars, media, and campaigns analyze future election surveys.

The Literary Digest poll of 1936, which incorrectly predicted that Landon would defeat Roosevelt in the 1936 US presidential election, has long been held up as an example of how not to sample. The sampling frame was constructed from telephone directories and automobile registration lists, and the survey had a 24% response rate. But if information collected by the poll about votes cast in 1932 had been used to weight the results, the poll would have predicted a majority of electoral votes for Roosevelt in 1936, and thus would have correctly predicted the winner of the election. We explore alternative weighting methods for the 1936 poll and the models that support them. While weighting would have resulted in Roosevelt being projected as the winner, the bias in the estimates is still very large. We discuss implications of these results for today’s low-response-rate surveys and how the accuracy of the modeling might be reflected better than current practice.

We use 23M Tweets related to the EU referendum in the UK to predict the Brexit vote. In particular, we use user-generated labels known as hashtags to build training sets related to the Leave/Remain campaign. Next, we train SVMs in order to classify Tweets. Finally, we compare our results to Internet and telephone polls. This approach not only allows to reduce the time of hand-coding data to create a training set, but also achieves high level of correlations with Internet polls. Our results suggest that Twitter data may be a suitable substitute for Internet polls and may be a useful complement for telephone polls. We also discuss the reach and limitations of this method.

This paper adds to the discussion on the value of online surveys for political science research. Mainly because of the lower costs, collecting survey data over the web has become increasingly popular in recent years, despite the higher sampling and coverage error in web-only surveys, especially online access polls. Recruiting respondents for the actual panel surveys based on a representative sample using a different mode is regarded as a solution to the sampling problem. Two approaches have been used to tackle the problem of coverage error: Providing respondents with computers (e.g. LISS, ELIPPS, GIP) and offering a different mode than online to respondents, thereby adopting a mixed mode design (e.g. paper as in GALLUP, GESIS Panel). The literature suggests that offering participation in the respondent’s preferred mode affects response rates positively but not much is known about respondents’ reasoning to choose a specific mode. We argue that it is important to understand this decision to evaluate the selection into online surveys and the consequences this has for data quality. We investigate this question by drawing on data from the GESIS Panel face-to-face recruitment interview for building a mixed mode access panel (paper and web) in Germany that gave a mode choice to internet users. Our results suggest that web literacy, age and education alone do not explain the mode choice but that affinity towards the technology related to the online mode has an independent effect. In a second step, we analyse the effect of this selection mechanism on answers to questions on typical variables used in political participation research such as media competence, political interest and civic duty in the subsequent mixed mode survey. We assess the added value of adopting a mixed-mode strategy. The results inform the evaluation of biases in unimode online surveys.