6.1 Analysis of the Vote Shares

This test looks for a systematic bias in the reported results of the outlying observations. We do so by comparing the vote shares for each candidate between our outlying observations and their corresponding control units. This approach allows us to identify potential trends on the vote results over time and assess the overall effect on the electoral result.

The comparisons are illustrated in Figure 10, which shows the differences between the vote shares of the three main candidates for each outlying observation and its correspondent control unit. We separate the figure by each of the matching approaches referenced above. Dots above the horizontal line indicate a positive difference for the candidate in the outlying observations compared with their control units. By contrast, a value below the horizontal line denotes a lower candidate’s vote share for the outlying observation compared with that reported by its control units. The plots show no observable trend between the reported time of the outlying observation and the differences between the candidates’ vote shares. Overall, the discrepancies for each candidate cancel each other out, failing to back the argument of a potential sorting of the results in favor of or against a particular candidate.

Figure 6:

Histogram for the values of Crossings. Notes: The plots show the frequency for the number of times in which the leading candidate overturns the vote aggregates of the runner-up in the simulated elections (Plot 1(A)) and permuted results (Plot 1(B)) described in Section 3.1. The dashed line represents the observed number of crossings in the candidates’ vote shares reported by the PREP. We log-transform the number of overturns per iteration to facilitate its visualization.

Figure 7:

Histogram for the values of Last overturns. Notes: The plots show the frequency for the number of reported tallies before the leading candidate gets an unsurpassable vote share in the simulated elections (Plot 2(A)) and permuted results (Plot 2(B)) described in Section 3.1. The dashed line represents the observed last overturn in the candidates’ vote shares reported by the PREP. We log-transform the number of last overturns overturns per iteration to facilitate its visualization.

Figure 8:

Presidential candidates’ vote shares by the time the results of the polling stations were reported. Mexico, 2006. Notes: The figure shows the vote shares for the candidates given the reported time of the preliminary results. The dashed lines show the vote shares for the candidates after adjusting the reporting times of the outlying units.

Figure 9:

Geographic location of the outliers identified on Section 3.2.

Figure 10:

Difference in vote shares for each outlier by candidate and comparison method.

Notes: The plots show the difference in the vote shares for each candidate between those observations identified as outliers and their matched polling stations. Dots above (below) the red line depict higher (lower) vote shares of the outliers when compared to their matched observations.

To estimate the overall effect of the outlying observations, we compare their mean vote share for each candidate with those reported by the control units. The results are summarized in Figure 11, which shows the estimated difference of means and 95% confidence intervals of the vote shares between the outliers and their respective counterfactual sets when comparing adjacent polling stations and using sparse optimal matching, respectively. In both cases, the differences are never statistically different from 0, suggesting that the results reported later than expected had no effect on the reported trend for each candidate.

Figure 11:

Difference in means for the candidates’ vote shares. Notes: The dots and horizontal lines show the mean differences and 95% confidence intervals for comparing the vote shares of the outliers to the matched observations.

In sum, the method described above suggests a way to identify suspicious observations given the time in which a polling station reported the results. In the case of the 2006 presidential election in Mexico, we find that polling stations reporting their results with an undue delay represent less than 0.5% of all observations. Moreover, and against the conventional wisdom regarding the preliminary vote count in the country, we did not find significant biases in the vote shares of these outlying units. Our model shows that the reporting time of every polling station mostly corresponds to its geographic and logistic context, and that the late results show no particular pattern.

As an extension of the proposed methodology, the next subsection shows the analysis of similar cases for the 2016 local elections in two Mexican provinces. In both elections, the losing candidates claimed fraud and blamed the electoral authorities for multiple irregularities during the preliminary vote count. Using data for those elections, we find that the (un)timely reporting of the results had no effect on the overall trend for each of the candidates in their respective elections.

6.2 2016 Local Elections

We use the proposed methodology to assess the fraud allegations on the preliminary counts of two gubernatorial elections in Oaxaca and Veracruz on June 5, 2016. Both states are often identified among those states with the lowest levels of sub-national democracy (Gibson 2013; Giraudy 2015). In the case of Oaxaca, the PRI’s candidate won the election with 32% of the votes, followed by the PAN and PRD coalition and the recently formed National Regeneration Movement (MORENA) with 25% and 23% of the vote, respectively. This result gave back control of the state to the PRI after losing the election 6 years earlier. Meanwhile, the PAN’s candidate won the governorship in Veracruz with 34% of the vote, followed by the candidates of the PRI and MORENA with 30% and 27%, respectively. This result ended the PRI’s 87-year rule of the state.

In both elections, losing candidates claimed fraud and blamed the electoral authorities for multiple irregularities during the preliminary vote count. In Oaxaca, after the PREP showed a lead for the PRI candidate, opposition parties alleged that the PREP was biased, suggesting a potential manipulation of the results to match them with the pre-electoral polls.^[28] Similarly, the candidate of MORENA in Veracruz, Cuitláhuac García, accused the PREP of being “statistically manipulated,” and argued that the electoral authorities inflated the preliminary results with more than 100,000 fake votes.^[29]

To assess the validity of these claims we proceeded in a similar way to the analysis described in the previous section. First, we obtained the coordinates of the local district offices in each state and estimated their driving distance to the polling stations from which they should receive the results. Then, using sociodemographic data from the 2010 census, we estimate a duration model for the time in which the results were reported in the PREP.^[30] Unlike the analysis for the 2006 election, we lack information on the presence of party representatives and poll workers. For Oaxaca, we also lack information on the number of registered voters.

Following the procedures described above, we estimate the survival models and correct for non-proportionality. The results of the survival models for Oaxaca and Veracruz are shown in the Appendix in Tables 5 and 6, respectively. We then obtain the Deviance residuals of the final model for each election and identify as outliers those observations with a residual value below −3; the geographic locations of these outliers are shown in Figure 12. Finally, we create potential counterfactuals to estimate the potential effect of the unexpected survival time of these outliers on the electoral result. The complete results are in the Appendix.

Figure 12:

Geographic location of the outliers. Notes: The plots in each map show the geographic location of those polling station which reporting times were underestimated by the model.

Figure 13 shows the overall effects of the outliers on vote shares in Oaxaca and Veracruz, respectively. The difference in means for each candidate’s vote share are plotted in both figures, along with 95% confidence intervals. As in the case of the 2006 Election, the model once again shows no statistical difference in outcomes when comparing the outliers to their respective counterfactuals, suggesting that the (un)timely reporting of the results had no effect on the overall trend for each of the candidates in their respective elections.

Figure 13:

Difference in means for the candidates’ vote shares. Notes: The dots and horizontal lines show the mean differences and 95% confidence intervals comparing the vote shares of the outliers to the matched observations.

Figure 14:

Schoenfeld residuals showing violations of proportionality assumption (Oaxaca Elections, Table 5).

Figure 15:

Schoenfeld residuals for corrected model (Oaxaca Elections).

Figure 16:

Deviance residuals and time in which each observation was reported by the PREP. Oaxaca.

Figure 17:

Schoenfeld residuals showing violations of proportionality assumption (Veracruz Elections, Table 6).

Figure 18:

Schoenfeld residuals for corrected model (Veracruz elections).

Figure 19:

Deviance residuals and time in which each observation was reported by the PREP. Veracruz.

Figure 20:

Residuals Oaxaca (under-neighbors).

Figure 21:

Residuals Veracruz (under-neighbors).

Figure 22:

Residuals Oaxaca (under-optimal).

Figure 23:

Residuals Veracruz (under-optimal)

6.3 Alternative Specifications

The baseline model contains the same base covariates as the model in Table 2, but does not correct for violations of non-proportionality.

Table 7 shows the results of the Cox Proportional Hazard Model prior to correcting for violations of the proportionality assumption. Overall, the direction of each covariate’s effect is as expected, and most variables attain statistical significance. However, as detailed above, because these variables violate the proportionality assumption, this model is incorrectly specified, and, therefore, inaccurate. This is further supported by comparing the plot of the Deviance Residuals of the baseline model (Figure 24) to the plot of the Deviance Residuals of the corrected model (Figure 4). In the former, the distribution of residuals is far from being symmetrically distributed around 0, tending instead towards the negative values for those units with the longest duration rates. This suggests that the model progressively underpredicts observations with the passage of time, as demonstrated by the larger residuals in the lower right quadrant of the figure. In contrast, the residuals for the time-corrected model show a more symmetric distribution of the residuals around 0, and the flatness of the loess line suggests that the corrected model overperforms the baseline model. Moreover, as seen in Table 8 and Figure 26, the Schoenfeld Residuals further support the presence of violations of the proportionality assumption in the baseline model.

Figure 24:

Deviance residuals and time in which each observation was reported by the PREP. Mexico, 2006. Baseline Model. Note: The scatter plot show the relationship between the deviance residual of each observation and its correspondent time that reported the results. We denote outlying observations in light blue and define them as those with a deviance residual outside the [−3, 3] range. The plot displays the results of the baseline model that does not correct for violations of non-proportionality. Locally-weighted regression lines are shown in blue.

Figure 25:

Survival function of baseline cox model. Note: The baseline model contains the same base covariates as the model in Table 2, but does not correct for violations of non-proportionality.

Figure 26:

Schoenfeld residuals for baseline model, Table 7.

As an additional check, we rerun the hazard model with the same specification as our main model (Table 2), but with the right-censored observations included. Results of this model are shown in Table 9, and a plot of this model’s Deviance Residuals is displayed in Figure 27. The right-censored model is not directly comparable to our main model due to the inability to correct for violations of proportionality using time interactions. This is because, by definition, right-censored observations are those still present after the reporting period is over; that is, these observations do not have a measure of time with which to create the interactions. However, the Deviance Residuals display a pattern similar to that of the uncorrected, baseline model: we observe a larger clustering of residuals in the right, bottom quadrant. Compared to the plot of the Deviance Residuals of our main model, this suggests that the main model outperforms the right-censored model, and is therefore more suitable for the analyses conducted above.

Figure 27:

Deviance residuals and time in which each observation was reported by the PREP. Mexico, 2006. Censored Model. Note: The scatter plot shows the relationship between the deviance residual of each observation and its correspondent time that reported the results. We denote outlying observations in light blue and define them as those with a deviance residual outside the [−3, 3] range. The plot displays the results from a model with the same specifications as the one found in Table 2, but includes the right censoring information with the survival time. Locally-weighted regression lines are shown in blue.

6.4 Robustness Checks

To further verify that the outliers identified in the above analyses are not effecting an undue bias on our model estimates, we conduct two additional robustness checks. The first of these is shown in Table 10. Here, Model (1) is the model used for the above analyses, whereas Model (2) excludes all outlying observations. A comparison of the two models reveals that no major differences are observed, providing evidence that the outlying polling stations are not having a disproportionate effect on our model estimates. For the second robustness check, we take the following steps. First, we drop outlying observations with 0.20 probability. Then, we rerun the model and store the new model’s estimates. We repeat this process one hundred times, each time storing the new coefficient estimates. Finally, we plot a histogram displaying the distribution of the coefficients derived from each of the one hundred simulations. Results can be seen in Figure 23A–S. In each figure, the red, vertical, dotted line represents our model’s coefficient estimate. As we can see, with the exception of illiteracy (and its interaction term), the model estimates are fairly centered around the mean of the distributions, providing further evidence that the outliers are not biasing our model estimates.

6.5 k-means Approach

This section shows the results of our identification strategy when we cluster observations using the k-means classifier. This approach separates observations into a given set of clusters with similar characteristics by estimating the Euclidian distance between observations. The outliers are those observations with the largest distance from the closest location of the group’s mean.

For our goal, we use this approach as follows. First, we estimate the intra-cluster distances between points and its variance to estimate the optimal number of clusters using the “elbow method.” This approach finds the number of clusters that explain most of the variance. Then we group the observations into the number of clusters defined in the previous step to then estimate the Euclidean distance between each observation and the mean of its respective cluster, labeling as outliers those observations within the largest 5% distance.

We use the same battery of variables that we employ in our survival model. In this case, we standardize our time measurements to facilitate the clustering. As Figure 29 below shows, the elbow method suggests that the optimal number of clusters is 15. We then group observations in 15 clusters to then estimate the Euclidean distance between the center of each cluster and the location of their respective observations. The summary statistics of every cluster are below. Finally, we will label as outliers those observations whose Euclidean distances with their cluster’s center are in the top five percent.

Figure 28:

Comparison of coefficient estimates from Table 2 to coefficient distributions of bootstrapped model. (A) Absent poll workers; (B) registered voters; (C) driving distance; (D) PAN observers; (E) PRD observers; (F) PRI observers; (G) 65-years-old; (H) high school; (I) computer; (J) no Spanish; (K) illiteracy; (L) PAN*log(min); (M) PRD*log(min); (N) PRI*log(min); (O) 65-years-old*log(min); (P) high school*log(min); (Q) computer*log(min); (R) no Spanish*log(min); (S) illiteracy*log(min). Note: Each figure above displays the distribution of each of the Model’s Coefficients, derived from the simulations described above. The red vertical line represents the estimated coefficients from our original model.

Figure 29:

k-means clustering sum of square errors (SSE). Note: The figure shows the sum of square errors of the first 100 clusters using the k-means classifier.

We replicate the analysis described in the main body of the manuscript considering only those observations classified as outliers in both the k-means approach and the deviance analysis. This leaves us to 202 observations. Figure 30 shows that the differences between these observations and their respective control units is statistically indistinguishable from zero.

Figure 30:

Results. Note: The dots and horizontal lines show the mean differences and 95% confidence intervals for comparing the vote shares of (1) those observations classified as outliers by both the deviance analysis and the k-means classifier and (2) their corresponding control units.

This table shows the average values of the variables for the first 15 largest clusters.