The Results of Exit Polls in Kansas to Verify Voting Machine Counts in the November 2016 Election

2.1 Machine Vote Counts are Not Secure

Fundamental Flaw of Voting Computers: Whoever programs the computer, decides what election results are reported by the computer program inside the voting machine. This was the testimony of Dr. Andrew Appel on Sept. 12th, 2017 at the meeting of the Presidential Advisory Commission on Election Integrity Resources, chaired by Kris Kobach, Secretary of State for Kansas.

Independent research from multiple sources indicate electronic voting equipment is insufficiently secure to provide citizens a reasonable basis for faith in the reported outcomes.

Our review concludes that the vendor systems lack basic technical protections necessary to guarantee a trustworthy election. (Patrick McDaniel 2007)

Accuracy of machine counts should not be assumed without post election audits. (Dill 2017)

Def Con, a conference for hackers, set up a challenge in July 2017: Hackers were provided with voting machines and the goal of getting access. It didn’t take long. All were hacked within 90 min (Porter 2017).

In March of 2018, Sen. Ron Wyden (D-Ore.) wrote to Election Systems & Software (ES&S), cautioning that malicious hackers could seek to exploit such software if it is built into the machines or other election-management products and wanting to know if such software had ever been included in their delivered products (Beavers 2018). On July 17, 2018 their response was made public, admitted in a letter to a U.S. senator that some of its past election-management systems had remote-access software preinstalled, despite past denials that any of its systems were equipped with such software (Ramsey 2018).

2.2 Machine Vote Counts are Not Transparent

It is worth noting that after the Nov 8th election, Jill Stein failed in her attempt to get a human count of votes. Her effort served only to illuminate the flaws in the process and the difficulty of obtaining a human verified count.

The Michigan recount was shut down after just three days; a federal judge rejected a request to recount paper ballots in Pennsylvania; and while Wisconsin did conduct a recount, in many counties, officials neglected to hand-count paper ballots and did not examine vulnerable software in electronic voting machines. (Eisen 2017)

A number of individuals, such as Dr. Laura Pressley of Texas and Tim Canova of Florida, have documented their failure to obtain appropriate verification of machine vote counts even when required by law. In Kansas, the author, a PhD statistician and Certified Quality Engineer with the American Society for Quality had been attempting, unsuccessfully, since November of 2012 to obtain access to the necessary records to conduct an audit in Sedgwick County Kansas, where she resides. Her attempts have included filing two lawsuits and have been chronicled in Appendix C. Here is the final response from the Sedgwick County Kansas Elections Office in an email correspondence requesting information on whether their office intended to conduct a post-election audit to assess the accuracy of the new election equipment used in the April 2017 Special election.

State statutes have not changed regarding the ability of an election official to conduct post-election audits of voting equipment. Until such time as that occurs, we are unable to audit the voting equipment. Sedgwick County and this office strongly support legislation that permits post-election audits but this is a matter to be decided in the state legislature.

Email 10-19-2017

The laws in Kansas have been changed since then. Election officials are now able to audit election results. However, the software that runs on the machines in Kansas is proprietary. No one who isn’t employed by the company, not even the officials entrusted to run the elections, are allowed to inspect proprietary software for trapdoors and other malicious code. Hacking by anyone, from foreign governments to motivated amateurs, is undetectable in many electronic voting systems.

The people most important to hide such manipulation from are the local election workers. An honest worker will report and correct any malfeasance they discover. There are hundreds of them in Sedgwick County alone. A hacker working alone does not have the ability to subvert and corrupt election workers all across even one county. Instead, they will attempt to make their electronic changes invisible to the people running the machines. Find access to the right point in the process and changes could be made or a virus inserted into the code before it was loaded onto the actual voting machines. Or it could be remotely accessed and altered via built in software (Kim 2018). A subversion of the system right under the noses of those employed to run it securely yet who are simultaneously forbidden from ever looking for that sort of flaw.

With the system in Kansas, we can assume that election workers conscientiously perform their duties properly, but this would not guarantee a secure system. The machine counting process is opaque to the people involved in certifying the results accurate. No steps were taken to measure the accuracy of the count produced. The ease of hacking combined with the lack of minimal checks on the results makes it nearly inevitable that attempts would be made and successful ones would go undetected by anyone with the authority to rectify the situation.

2.3 Political Context for 2016 Kansas Election

Governor Brownback took office in 2011. Five Supreme Court Justices (SCJ) were up for retention votes in the 2016 Election, but only one of the five was a Brownback appointee. 2016 was a far more politicized contest for those positions than has occurred in the past. All of the Justices were members of the Republican Party. Republican affiliated organizations were actively seeking to oust the four Justices up for retention vote that were appointed by Brownback’s predecessors. They were encouraging their supporters to vote ‘No’ on all except Stegall, Brownback’s lone appointee among the five. Documentation of this situation is provided in Appendix D.

3.1 Ballot Options

In Kansas on Election Day voters may vote in one of three ways. The majority of people vote on electronic voting machines (DREs), but paper ballots are available on request. In Wichita voters place the paper ballots into the scanning machines themselves. In Winfield and Wellington, the paper ballots are placed in a secure location and counted with the mail-in ballots, so no analysis of paper ballot results from those sites is possible. Individuals who have problems with the registration or ID requirements are given a provisional ballot. After they fill out their provisional ballot, they are given an envelope to place it in, which is sealed, signed, and given to the election judge. These are only opened and counted if the signer is verified as a registered voter. While it is expected that people will be able to correctly identify whether they voted by machine or on paper, the only difference between the paper and provisional ballots was whether they were counted that night after the polls closed or later, after first determining that the individual was verified to be a registered voter. In retrospect, it’s not surprising that some voters were confused about which type of paper ballot they had voted on. For example the Southwest Wichita site collected 79 surveys from citizens who indicated they had received a provisional ballot while the official results for that location indicated only 78 provisional ballots. For this reason, only the votes cast on machines were compared to the official results.

3.2 Electronic Voting Equipment

The electronic voting machines (DREs) used in all three Kansas counties were Election Systems and Software (ES&S) Ivotronic. Sedgwick County had the optional RTAL (Real Time Audit Log) for those machines while Sumner and Cowley counties had no paper trail with their voting machines. The model of paper ballot scanning equipment varied, but all were from the same manufacturer: ES&S.

Official results were tabulated election night for each polling place separately for the machine votes and, in Sedgwick County, the scanned paper ballot votes as well. Citizens are allowed to inspect the totals on election night at the polling location. Most of the exit poll managers were able to do that. Provisional votes were counted later, and only after the voter was found to be properly registered.

The machine ballot results from the exit polls were compared with the official totals given for the votes cast on machines and counted on site immediately after the close of polls. The official counts for each site on Election Day by type of equipment are shown in Appendix B.

3.3 Survey Design

To do a direct comparison of exit poll results with machine counts, the smallest sampling unit was a polling place, which contains multiple precincts. Every voter leaving the polling site on Election Day was asked to complete the survey, but could not be expected to know their precinct number. This sampling plan was considered best because it would maximize the exit poll sample size relative to the total votes cast in each unit and thus minimize the size of a detectible discrepancy.

This approach is well-suited to citizen activists who are concerned about their own voting processes. It makes a direct comparison of the people who voted at the polling sites’ surveys with their own numbers, collected and counted for themselves. Citizens working the exit poll receive a certainty of the outcome that cannot be had from entrusting it to others.

3.4 Site Management

Each polling location was managed by a volunteer willing to take on the responsibility of running the exit polling site from the time the polls opened until they closed. These site managers scheduled the other volunteers to cover the entire time polls would be open and made sure all necessary supplies were ready and available.

The sites for the surveys to take place were not chosen randomly. Wichita had sufficient volunteers to staff three locations. The three locations were chosen to make sure that there was one majorly democrat site, one majorly republican site, and one site evenly split between the two parties, while also minimizing travel time for the site managers. Sumner and Cowley county sites were chosen to be located at their respective county courthouses – the major voting site for both of those counties. All site managers contacted the property owners of the polling location to let them know what was planned and obtain permission to set up a booth. The election offices in all three counties were informed of these plans as well.

Volunteers were recruited and trained in how to approach voters and what to say (and not say). Volunteers attended a training session, explaining the set-up and procedure, and emphasizing that they should accept “No” as an answer. A copy of the training materials is provided in Appendix A.

All voters were to be approached as they left and given the opportunity to participate in the survey. A flyer was provided explaining our purpose was given out to voters along with the survey. A copy of the flyer is provided in Appendix A.

As people exited the polling place, they would be asked whether they had just voted. The voters exiting the polling place were then asked to fill out a survey form identifying how they voted (by machine, a scanned paper ballot or a provisional paper ballot). They were given a clipboard, pen and survey and asked to place their completed survey in a box. They were given space to fill out the survey without being observed. No identifying information about the voter was collected in order to keep their responses anonymous.

Site managers had final approval on the survey form. Three sites included additional questions specific to their site which were not include in this analysis. Copies of the survey forms used in each site are provided in Appendix A. The exit poll results and corresponding official vote counts are provided in Appendix B. Electronic copies of the completed surveys are available on request for Sedgwick and Sumner Counties.

3.5 Voting Equipment

The electronic voting machines (DREs) used in all three Kansas counties were ES&S Ivotronic. Sedgwick County had the optional RTAL (Real Time Audit Log) for those machines while Sumner and Cowley counties had no paper trail with their voting machines. The DRE machines were purchased in 2006.

The results were tabulated election night for each polling place. Results for machine votes and scanned paper ballot votes were tabulated separately in the official counts so direct comparisons to votes cast by machine can be made. Citizens are allowed to inspect the totals on election night at the polling location. Provisional votes were counted later, and only after the voter was found to be properly registered. Figure 1 shows the response rate for each site for the machine cast ballots versus the paper and provisional ballots.

Figure 1:

Response rate for by site and method of voting.

4.1 Wichita, Sedgwick County – Three Locations

Exit polls were run at three locations in Wichita selected to cover disparate voter demographics (see Table 1). At the Countryside Christian Church in Southeast Wichita, registered voters were evenly divided between the parties. At the Asbury Church in Southwest Wichita, voters are primarily registered Republicans while the Urban Wichita voters at the County Health Department were primarily registered Democrats. Unaffiliated voters were approximately equal at all three locations.

The percent of votes cast by machine each candidate received in the official count and the exit poll counts were computed for each candidate and for the yes/no responses on the question of whether to retain the Supreme Court Justice. This was used to then compute the % non-responders required for each candidate in order to match the official results. The data is shown by site in Tables 2–7.

4.2 Winfield, Cowley County (Tables 8 and 9)

4.3 Wellington, Sumner County

Because the Sumner County Survey did not include the option of ‘left blank’ for the Judges, their results are presented both with and without including those who did not indicate a choice on the Kansas Supreme Court Justices.

The differences in response rate for non-responders are extremely high for those polling stations that managed to obtain response rates in excess of 60 percent and above. These high response rates allow us a high confidence rate in our conclusions, aiding in eliminating non-response bias as an alternative explanation for the results obtained (Tables 10–12).

4.4 Comparison of Exit Poll Results with Machine Counts

The pattern of the statistical flags can allow us determine whether that cause is likely to have been inadvertent or intentional. Different causes generate different patterns in these statistics. Frequent large errors randomly distributed across parties should be followed by a rigorous evaluation of the fitness of the voting machines, looking for defective output. Patterns favoring one party across different races is expected under the hypothesis of deliberate manipulation and should be investigated to verify the accuracy of the result. Should deliberate manipulation be confirmed, we must redesign our voting process to eliminate the security flaws that permitted it to happen.

Charts in this section have the axes set to ±5% to allow visual comparison between them.

Sumner County did not include a ‘left blank’ option for the Supreme Court Justice questions; undervotes are therefore not included for that county. The tables for the other four sites show the results for the judges both with and without the ‘left blank’ option so comparisons can be made with the Sumner County results.

4.5 Hypotheses to be Tested

A general expression of the null hypothesis for all tests can be expressed as follows:

Null (H₀) – The null hypothesis is that no tampering of machine counts occurred. This would be supported by a random distribution of differences not statistically distinguishable from zero.

Alternative (H₁) – The alternative hypothesis is that there is a statistically significant difference between the exit poll results and the official count. The pattern of errors will be used to try and determine cause.

Failing to reject the null means there is no statistically significant difference between the exit poll and the official count – i.e. no explanation beyond random variation is needed. When we reject the null, we need to explain the cause of the observed values being so different from the expected values.

4.6 Pattern Analysis with Respect to Determination of Cause of Discrepancies

A statistically significant result in these tests does not allow us to conclude fraud occurred. We must also consider the possibility that the statistical result is an unintended side effect of some other error.

Unintentional design error s can lead to a bias in the differences. The Florida 2000 presidential race ballot [11] design problem is an example of unintentional error resulting in a statistically significant discrepancy between exit poll results and the official results. Indeed, a discrepancy with exit poll results was how that particular design error was discovered. Such unintentional errors cannot be excluded as a potential explanation for any statistically significant deviation between exit poll results and official results. However, such errors would be limited to a single race and county combination with multiple errors expected to be randomly distributed between parties with regard to direction of the bias. Thus, inadvertent error is not a plausible explanation for a pattern that affects multiple races within a single site.

Non-Response bias occurs where voter response to the exit poll is correlated with a particular party (e.g. Republicans are less likely to participate in the poll). This is the most difficult alternative to distinguish from fraud. Results in southeast Kansas show that it is possible to eliminate or support that hypothesis with data from multiple races and sites. If response bias occurs, the differences will be consistent within each polling site; we expect similar size errors in the same direction across races and exit poll sites. Bias due to fraud might or might not show this pattern. If all races show the similar differences in the same direction, then non-response bias is a reasonable hypothesis, but if the bias within a polling site varies from race to race within a site, then non-response bias is not a reasonable hypothesis as the cause of the bias.

When bias is apparent within a site, but the size of the differences varies significantly between races, and the direction of the bias is different between demographically similar polling sites, non-response bias is not a plausible explanation. Both of these characteristics are part of the Kansas exit poll database. This leaves deliberate fraud via tampering of the machine vote counts as the only plausible conclusion for some sets of errors.

4.6.1 Analysis by Political Party

Eight ballot questions were included on all five exit poll surveys. Three were for multiple candidate races and five were Kansas Supreme Court Justice Retention votes. All five judges were Republicans although as discussed in section 2.3 and documented in Appendix D, the Republican party campaigned against four of the five justices up for retention votes. The retention vote is a yes/no question so the judges were analyzed separately from the candidate races (section 4.6.3).

The three races with multiple parties competing were analyzed to assess response bias by party. Units were defined as the difference between the vote share for each candidate in each race by polling location and method. This gives 15 units to assess each of the three major parties and five units for the Green party which fielded a candidate in only the presidential race and five units for the Independent candidate in the fourth district congressional representative race. The differences by party for those three races are shown for the Libertarians in Figure 2, the Republicans in Figure 3 and the Democrats in Figure 4. This data meets all requirements for both the chi-squared test and the t-test. The chi-squared test results were computed using exit poll counts against the expected values based on the official results. A two-sided paired t-test was performed on the % differences.

Figure 2:

Differences by party results.

Figure 3:

Republican party results.

Figure 4:

Democratic party results.

H₀:

The differences between the official results and the exit poll results are randomly distributed across races and polling locations for each party (D, R, & L)

H₁:

The differences between the official results and the exit poll results are not randomly distributed across the races and polling locations for each party (D, R, & L).

The p-value of the t-statistic for the Libertarian party results is 0.0046. The p-value of the χ ² test statistic is <0.0001. The null hypothesis of no difference between exit poll results and official results for the votes cast on machine can be rejected by both tests at the 99% confidence level. The average difference of 0.81% is statistically significant. The results for Senator and fourth District Representative show a clear bias against the Libertarian candidate in the exit poll (exit poll results are less than official results) while the votes for President do not. We could surmise that the Senator and fourth District Representative races are due to Libertarians being less inclined to participate in exit polls. The consistent difference in both races is what we expect to see in that scenario. The Presidential race does not follow that pattern.

The p-value of the t-statistic for the Republican party results is 0.4102. The p-value of the χ ² test statistic is 0.0853. The null hypothesis of no difference between exit poll results and official results for the votes cast on machine cannot be rejected by either the χ ² test or the t-test. They do not appear to be consistent in direction the way the libertarian results were. The overall average difference of 0.36%, given the large variability of the data, is not large enough to reject the null that this difference is statistically significant.

The p-value of the t-statistic for the Democrat party results is 0.0022. The p-value of the χ ² test statistic is 0.0084. The null hypothesis of no difference between exit poll results and official results for the votes cast on machine can be rejected by both tests at the 99% confidence level indicating that the differences are larger than would be expected by chance. They also appear to be consistent in direction with an overall average difference of −1.31%. This difference is statistically significant and fall primarily in one direction. Response bias might be an explanation. This would require an assumption that democrats were significantly more likely than Republicans or Libertarians to respond to the exit poll.

Analysis of the Green Party and Independent Party results and the Left Blank results are included in Appendix E.

4.6.2 The Presidential Race

H0:

The differences between the official results and the exit poll results are randomly distributed across polling locations for all candidates in the Presidential Election.

H1:

The differences between the official results and the exit poll results are not randomly distributed across polling locations for all candidates in the Presidential Election.

The machine results for all three Sedgwick county polling locations and the Cowley polling site show statistically significant drops in votes for Hillary Clinton and statistically significant gains for Donald Trump in those same locations. Sumner County does not follow the pattern, but shows large % differences in the opposite direction. Non-response bias does not fit the pattern of differences for the presidential race. These differences are consistent with a small proportional siphoning of votes to Donald Trump from all of the other candidates. The gains in the Wichita and Wellington sites were statistically significant according to the test results while the Sumner results are not. The most plausible explanation is deliberate fraud in the Sedgwick and Cowley County sites.

The Libertarian Candidates show a pattern (Figure 2) in the Senate and fourth District races, which looks like a sample bias due to differential non-response by Libertarians. The Presidential race does not conform to that pattern (Figure 5). The Presidential results were skewed toward Trump except in Sumner, which is opposite. If results of the presidential race were due to deliberate manipulation, then non-response bias is a plausible explanation for the pattern of Libertarian differences in the Senate and fourth District Congressional races.

Figure 5:

Presidential race results.

If we assume a non-response bias for Libertarians accounts for the differences favoring Libertarians in the Senate and fourth District races, then we conclude votes were also taken from Johnson. We can compute the difference between those races and the presidential race to estimate what % of Libertarian votes were siphoned to President Trump. The average difference for all five polling stations for the Libertarian Senate and fourth District Rep candidates was 1.22% while the average difference for all five polling stations for the Libertarian Presidential Candidate was −0.31%. It’s reasonable to conclude that the non-response bias for Libertarian votes is ∼1.22% yielding a rough estimate of 1.5% votes siphoned from Johnson under the hypothesis of vote counting malfeasance.

The Democrats show some consistency in the direction of their differences but not for the size of the discrepancies. Democrats received an average of 0.94% more votes in the exit polls than the official counts. The majority of that being large discrepancies in the presidential race. This pattern is consistent with deliberate manipulation in the presidential race.

The size of these differences are small relative to the overall vote share Trump received in Kansas. He won Kansas 56–36% so, even assuming he had extracurricular help from unknown parties, the outcome for Kansas can be considered correct with a high level of confidence.

4.6.3 Kansas Supreme Court Justices

H0:

The differences between the official results and the exit poll results are randomly distributed across the five polling locations for each judge.

H1:

The differences between the official results and the exit poll results are not randomly distributed across the five polling locations for each judge

H0:

The differences between the official results and the exit poll results are randomly distributed across the five judges within each polling locations.

H1:

The differences between the official results and the exit poll results are not randomly distributed across the five judges within each polling locations.

Looking at the judges individually, the chi-squared results clearly show non-randomness for the four judges with organized Republican opposition to their retention; the t-test results do not. Looking at the results by site, the t-tests show statistically significant differences for all sites except SE Wichita while the chi-squared results show statistically significant differences only for Cowley and Sumner (see Tables 13 and 14).

Table 13:

Supreme court justice retention race analysis results by judge.

Judge	No. samples	t-Test p-values	χ 2 Test p-values	Average difference
Beier	5	0.3036	0.0001	−2.00%
Biles	5	0.4577	0.0035	−1.24%
Luckert	5	0.2309	0.0002	−2.32%
Nuss	5	0.6183	0.0027	−0.84%
Stegall	5	0.7744	0.4186	−0.26%

Table 14:

Supreme court justice retention race analysis results by site.

Unit	No. samples	t-test p-values	χ 2 Test p-values	Average difference
SE Wichita	5	0.1112	0.8737	−1.43%
Urban Wichita	5	0.0070	0.1502	−3.30%
SW Wichita	5	0.0041	0.3231	−2.34%
Cowley	5	0.0145	0.0001	−3.64%
Sumner	5	0.0001	0.0329	4.05%

The average difference for each judge falls within the bounds of a 95% confidence interval around zero even if all five differences are negative. The chi-squared test examines the size of the differences, but not their direction. Within each site, the judge’s results are of similar size, hence the chi-squared results don’t show anything unusual. However, the t-test, which looks at average difference compared to zero does show problems. The differences are statistically significant for all sites except SE Wichita Machine votes.

While the results for SE Wichita are not statistically significant by themselves, they follow the same pattern as the Sedgwick and Cowley sites casting further doubt on the official results. The graph of the differences display a pattern consistent with the narrative of sites being vulnerable to hackers (see Figure 6). Sumner results are clearly different from the others; They exhibit the same pattern, but the direction is flipped.

Figure 6:

Supreme court justices results.

The pattern across sites for all five judges is the same as the pattern seen in the presidential race. In addition, the magnitude of the errors also exceeds (approximately doubles) that found in the other three races. The results for the Judge Retention votes indicate signs of deliberate manipulation at all five sites, although not all counties were discrepant in the same direction.

Non-response bias correlated with party does not make sense as an explanation for these retention votes given the difference in direction for Sumner County. The magnitude of the differences for these races is significantly larger than for the multiparty races. While inadvertent process problems could increase variability they would not show consistency in one direction. Given the previous results we have seen in the candidate races, the only conclusion with any plausibility is deliberate manipulation of the machine vote counts.

However, all the Supreme Court Justices retained their positions with more than 55% of the voters choosing yes. As with the presidential results, the outcome for all five judges can be considered correct with greater than 99.99% confidence.