Yesterday, Media Matters tried to refute a blog post in which I point out, among other things, that the impact of voucher use in the D.C. Opportunity Scholarship Program resulted in a 91 percent graduation rate compared to 70 percent in the control group. The findings are from the U.S. Department of Education’s final evaluation of the voucher program, authored by Dr. Patrick Wolf.
Walid Zafar writes via Media Matters:
Where does Burke get the 91 percent figure from? Well, not this [the Department of Education’s] report. It’s hard not concluding that she made that statistic up. The report puts the graduation rate for students receiving vouchers at 82 percent.
Not so fast. The report does in fact find that the use of voucher resulted in a 91 percent graduation rate. On page 20 of the report’s executive summary, Wolf writes:
The offer of an OSP scholarship raised students’ probability of completing high school by 12 percentage points overall. The graduation rate based on parent-provided information was 82 percent for the treatment group compared to 70 percent for the control group. There was a 21 percent difference (impact) for using a scholarship to attend a participating private school. [Emphasis added]
The 21 percentage point difference for impact means the typical student who received a voucher and actually used it to attend a private school had a graduation rate of 91 percent, compared to 70 percent for non-voucher students. Here’s exactly how the graduation rates break down:
- D.C. Public Schools graduation rate: 49 percent.
- Control group (those students who applied for a voucher but did not receive one) graduation rate: 70 percent.
- Voucher recipient group (students who applied for a voucher, won the lottery to receive one, but did not necessarily use it) graduation rate: 82 percent.
- Impact of voucher use: (students who applied for, received, and actually used the voucher to attend a private school) graduation rate: 91 percent.
Zafar also argues that the results of the study are minimized due to the increased motivation of parents who applied for a scholarship:
You can’t compare the graduation rate at DC Public Schools (which take in all who apply, regardless of learning disabilities and level of parental involvement) to a lottery based voucher system to which only the most highly motivated students (and parents) choose to apply.
First, the D.C. Opportunity Scholarship Program does have to take all students who apply. When applications exceed scholarships, officials use a lottery to determine which students receive vouchers. In fact, because evaluators anticipated objections like Zafar’s, they controlled for the students who applied for a voucher but were ultimately not offered one. These presumably highly motivated students were evenly distributed across the treatment and control group, which is probably why the control group graduation rate of 70 percent was higher than the overall DCPS rate of 49 percent. The voucher students significantly outperformed the control group on the crucial measure of high school graduation even though the lottery ensured that both groups were equally stocked with motivated students and parents.
While it’s true that parents have to have a certain level of interest in the educational opportunities of their children in order to apply for a voucher, thousands of low-income families in the District jumped at the opportunity to do so when given the chance. In fact, there were four applicants for every available scholarship.
Finally, Zafar argues that the DCOSP had no impact on academic achievement:
In the area of student achievement, the report concludes, “Overall reading and math test scores were not significantly affected by the Program, based on our main analysis approach.” Most crucially, the report notes that “No significant impacts on achievement were detected for students” who “were lower performing academically when they applied.” In other words, the students who did well on the voucher program were those who were already doing well in public school.
While the final evaluation did not find a statistically significant impact on academic achievement (which was not the main point of our argument), it did find that the scholarships had a positive impact on academic outcomes for some subgroups of students. Moreover, Dr. Wolf, the lead researcher on the OSP study, explains in a statement from the University of Arkansas that the significant positive impact on graduation rates is more important than the impact on academic achievement:
These results are important because high school graduation is strongly associated with a large number of important life outcomes such as lifetime earnings, longevity, avoiding prison and out-of-wedlock births, and marital stability. Academic achievement, in contrast, is only weakly associated with most of those outcomes.
In the area of education, how far you go is more important than how much you know, and D.C. students went farther with the assistance of a school voucher.
Facts matter, and we hope we’ve stated them clearly enough so that even Media Matters can’t deny them.
