Impact of Voucher Design on Public School Performance: Evidence from Florida and Milwaukee Voucher Programs

National Commission of Excellence in Education (1983),“A Nation at Risk: The Imperative for Educational Reform,” Washington, DC: U.S. Government Printing Office.

The Florida Department of Education (DOE) classified schools according to five grades: A, B, C, D, and F (A – highest, F – lowest). In 1999, 78 schools got an “F”, and students in two of those schools became eligible for vouchers. In 2000, four elementary schools got an F, although none became eligible for vouchers. In 2001, no school got an F. In 2002, 64 schools got an F. Students in 10 of those schools became eligible for vouchers.

I will focus on the Milwaukee program up to 1996–1997. This is because following a 1998 Wisconsin Supreme Court ruling, there was a major shift in the program when religious private schools were allowed to participate in the program and the program entered into its second phase. Moreover, financing of the Milwaukee program saw some crucial changes, so that voucher amounts and revenue loss per student due to vouchers were not comparable between Florida and second-phase Milwaukee.

Neal (2009) also argues that choice and accountability should not be seen as policy substitutes, but argues in favor of combining both policies.

It should be noted here that the combination of private school vouchers and public school monitoring modeled in Ferreyra and Liang (2012) is different from the TOV scheme modeled in this article. In the vouchers – public monitoring scheme that they model, vouchers are already imposed. In addition, they supplement this voucher system with public monitoring. In contrast, the key feature of the TOV system is that vouchers are not already imposed. Rather, the policymaker issues a threat or establishes a target/cutoff, and vouchers are introduced only if the school fails to meet that cutoff. Also, while the policy intervention in the case of TOV is announcement of a cutoff, the public monitoring in Ferreyra and Liang (2012) takes the form of “detailed evaluations of public school performance, direct observation of classroom and administrative practices etc.,” and is costly.

The Cleveland voucher program (implemented in the 1995–1996 school year), the District of Columbia voucher program (implemented 2004–2005) and the Indiana voucher program (implemented 2011–2012) are also of the VS type, while the Ohio voucher program (implemented 2006–2007), and Louisiana voucher program (implemented 2012–2013) are of the TOV type. Among the VS-type programs, the Milwaukee program is the oldest; while among the TOV-type programs, Florida program is the oldest. Another crucial aspect is that apart from the differences in designs highlighted in the article (VS versus TOV), the other features of the programs were very similar between the Florida and Milwaukee programs (which is essential for credible comparisons of the VS and TOV designs). As a result, I focus on the Milwaukee and Florida programs.

Exceptions are Figlio and Rouse (2006) that control for all three, Hoxby (2003b) that controls for differences in pre-existing trends, and Peterson and West (2006) that controls for changes in student composition of schools. Unlike this study, the analysis in Figlio and Rouse (2006) focuses on the low stakes test. While Greene (2001) and Greene and Winters (2003) attempt to assess mean-reversion effect, their analysis is based on post-program effects of schools. Hence, any mean-reversion effect is likely to be confounded with post-program response of schools. For a more detailed literature review as well as effect on low stakes test scores in Florida, see Chakrabarti (2008a).

For the remainder of the article, I will refer to school years by the calendar year of the spring semester.

All results continue to hold if public schools are assumed to maximize enrollment subject to cost. An alternative formulation could be to model the public school as a quality maximizer. However, in that case, there would be no argument for voucher programs as far as improving public school quality is concerned.

Note that at equilibrium, private school quality will always exceed public school quality. Otherwise, no household would pay to attend a private school. Chakrabarti (2009) shows that for voucher students, random selection by private schools has indeed taken place.

Public school quality can be thought of as being embodied in public school scores. The notion here is that public school scores reflect both public school effort and public school peer-group quality, which in turn depends on the abilities of the public school students. In other words, both public school and student characteristics contribute to school scores.

For proofs and discussion of existence and uniqueness of equilibrium, see working paper version of this article, Chakrabarti (2008a). The equilibrium is unique, if the marginal utility from peer quality is not too high.

. There are two effects. Vouchers lead to an exodus of relatively high-ability households (at each income level) to private schools, so that the new marginal household has a relatively lower marginal valuation of quality. Consequently, the number of students gained due to a marginal increase in effort is lower under vouchers. This is captured by the negative first term. Second, since the marginal utility from school quality decreases with quality (u_qq0), the marginal number of students due to an increase in effort decreases with an increase in peer quality . Since vouchers lead to a fall in peer quality, the marginal number of students increases due to this factor (which is captured by the positive second term).

Note that since peer quality is known, announcing a cutoff in terms of effort is equivalent to announcing a corresponding cutoff in terms of quality.

The intuition can be laid down in two steps. Call the VS program where all students are eligible the universal voucher shock (UVS) program and where only the low-income students are eligible the targeted voucher shock (TVS) program and the corresponding equilibrium number of students and equilibrium effort N_UVS, N_TVS, and e_UVS, e_TVS, respectively. First, note that the equilibrium rent under the TVS is greater than that under the UVS. Under the TVS, the school can attract N_UVS students by giving a lower effort than under the UVS (follows from the discussion earlier and Claim 1 in Appendix A) and hence at a rent higher than under the UVS. Since the school chooses to attract N_TVS students, it must be the case that rent is higher under the TVS. Second, if vouchers when imposed in the Florida-type TOV program took a targeted form, then following the argument in Proposition 2(i), the program could implement a cutoff . But vouchers take the universal form in Florida, which implies that the rent would be smaller than the TVS rent if the school failed to meet the cutoff. This implies that there exists a cutoff which satisfies the school’s incentive constraint with equality and hence can be implemented by the TOV program. Any cutoff in the range induces higher effort under the TOV program than under the TVS program. To summarize the above discussion, there are two features in the design of the Florida TOV that induce a higher effort than the TVS: (i) vouchers are not already imposed and (ii) the potential loss of students is greater. But, as it follows from the above discussion, the first factor is sufficient to induce a higher effort under the TOV.

In the TOV program, it may be reasonable to think that there is a stigma attached to being labeled as a “voucher public school”. For example, Maureen Backentoss, assistant superintendent of curriculum and instruction of Lake County School District, refers to it as a “glass of cold water in the face.” In the presence of such a stigma, the public schools gain an additional utility if they are able to escape vouchers. This feature is absent in the VS program. Note that this will weigh results in favor of the TOV and will induce an even higher improvement under the TOV.

It follows from the above discussion that the corollary holds not just for the UVS system but also the TVS system.

The measure of public (private) competition here is the number of public (private) schools within a certain radius. I use 1, 2, 3, and 5 mile radii.

I restrict my analysis to elementary schools in both Florida and Milwaukee, as there were too few treated middle and high schools in the respective places to justify analysis.

In fact, there is some anecdotal evidence that D schools might have responded to the program. The superintendent of Hillsborough county, which had no F schools in 1999, announced that he would take a 5% pay cut if any of his 37 D schools received an F on the next school report card. (For more evidence, see Innerst 2000).

While 175% was the cutoff poverty level for eligibility, this cutoff was not strictly enforced (Hoxby 2003b) and households within this 10% margin (175% and free or reduced-price lunch eligibility cutoff 185%) were often allowed to apply. Also there were very few students who fell in the 175–185% range.

Note that the test scores under consideration in Florida are high stakes FCAT test scores. Since the threat in the Florida program was given in terms of grade, the response of the Florida threatened schools has to be assessed in terms of the high stakes test. Since this study is interested in examining the response of the threatened schools, it focuses on the high stakes tests in Florida. For example, even if there is no improvement in the low stakes test, it cannot be concluded that the public schools did not respond to the program. The tests under consideration in Wisconsin are state tests (WRCT or WKCE) and district test (ITBS). These tests are low stakes. Note that the comparison of the effects on the more treated schools in the two places in terms of these tests is legitimate, as the Florida FCAT tests are high stakes to the threatened schools precisely because of the Florida accountability-tied voucher system, that is, the Florida accountability-tied voucher program had the capacity to make the corresponding tests high stakes unlike the Milwaukee-type voucher program (that did not have an accountability component).

When data are available for only 2 years before program (for example, reading and math), the pre-program difference between treatment and control groups can be either a trend difference or a year effect. Specification 1 controls for this pre-program difference assuming it is a trend difference, and specification 3 controls for it assuming this difference is a year effect. Results from regressions without controlling for these pre-program differences are qualitatively similar.

In many of the tables, only the fixed-effects estimates are reported. The OLS results are very similar to the FE estimates and hence are omitted.

It should be noted here that both F and D schools received additional funds from the state. However, all results above are obtained after controlling for real per pupil expenditure. The results neither are sensitive to inclusion of real per pupil expenditure nor do change after including a polynomial in real per pupil expenditure. Moreover, even in the pre-1999 period, the critically low performing schools received extra assistance – however, this did not result in improved performance of this group in this period, as Table 4 shows in a different context.

I also consider two corresponding discontinuity samples where both groups fail the minimum criteria in reading and writing (math and writing). F schools fail the minimum criteria in math (reading) also, unlike D schools. In these samples, the probability of treatment changes discontinuously as a function of the percentage of students at or above level 2 in math (reading), and there is a sharp cutoff at 60%.

I do not use the pre-1999 reading and math NRT scores, because different districts used different NRTs during this period, which varied in content and norms. Also districts often chose different NRTs in different years. Thus, these NRTs were not comparable across districts and across time. Moreover, since districts could choose the specific NRT to administer each year, the choice was likely related to time varying (and also time-invariant) district unobservable characteristics which also affected test scores.

I would like to thank David Figlio for suggesting this strategy.

Figlio and Lucas (2004) find that following the first assignment of school grades in Florida, the better students differentially selected into schools receiving grades of “A,” though this differential sorting tapered off over time.

I also do a pair-wise non-parametric test (sign test), where I ignore the significance of coefficients and consider only their signs. Under the null of equal effects, the probability that any one effect size in Florida exceeds the corresponding one in Milwaukee is . Under the null, D Florida effect–Milwaukee effect follows a binomial distribution. D is positive in all cases. The probability of getting all positive D under the null is very small, and hence, the null of equal effects can be comfortably rejected.

It might be worth comparing these effect sizes with those obtained in the literature. Greene (2001) and Figlio and Rouse (2006) study the effect of the 1999 Florida program on F schools in the first year after program. Greene (2001) finds F-school effects (relative to C schools) to be 0.42 standard deviations in reading and 0.67 standard deviations in math, while Figlio and Rouse (2006) finds 0.09 of a standard deviation improvement in reading and 0.23 of a standard deviation improvement in math for the F schools. These effects are not directly comparable to those in this study due to the differences in methodology, specifications, and samples. For example, the estimates in Greene (2001) do not control for mean reversion or pre-program trends or changes in demographic compositions of schools. The effects in Figlio and Rouse (2006) are relative to all higher-graded schools (so their comparison group was different), do not control for pre-existing trends or mean reversion, and also pertain to data on the group of Florida districts for which they had micro-data. Note that the main focus of Figlio and Rouse (2006) was looking at low stakes tests (not high stakes tests, so the estimates above are their initial estimates). Chiang (2009) and Rouse et al. (forthcoming) look at the second phase of the Florida program after the 2002 changes. Thus, they look at a different time period and the program parameters had also changed, and their methodology and samples were also different, so their effect sizes are not directly comparable to those in this article. Using a RD estimation strategy with a 28-point bandwidth, Chiang finds effect sizes of 0.11 in reading and 0.12 in math 1 year after the program shift. Rouse et al. (forthcoming) finds 0.6–0.14 standard deviation effect sizes in math and 0.6–0.10 standard deviations effect in reading for F schools (relative to the sample of all higher-graded schools) also 1 year after the program shift.

However, although rent falls at each realization of e′ with vouchers, this fall (or alternatively, the gain in rent from avoiding vouchers) may either increase or decrease with e′. Depending on this, under certain circumstances as Proposition 3 indicates, at very low levels of cutoff, the public school effort under TOV may be less than VS and at very high levels of cutoff, effort under TOV may be less than PP. The intuition is as follows. First consider TOV versus VS. If is low, schools escape vouchers for low values of e′ also. If it is the case that the gain in rent from avoiding vouchers is largest for lower values of e′, then since an increase in effort decreases the probability of occurrence of lower values of e′, public school may find it profitable not to increase effort. Now, consider TOV versus PP. If is high, vouchers will be incurred at high values of e′ also. If it is the case that the fall in rent due to vouchers is highest for high values of e′, then the school may not have an incentive to increase effort since an increase in effort increases the probability of occurrence of higher values of e′.