K-12 education in the nation’s capital has struggled for decades. Yet since 2003, the D.C. Opportunity Scholarship Program—a voucher program open to low-income children in the District of Columbia—has been a bright spot for educational opportunity.
But a new report by the Institute of Education Sciences at the U.S. Department of Education, released Tuesday, found that students who participate in the D.C. scholarship program had worse math scores after two years than a control group of students who applied for a scholarship but did not receive one.
The scientifically rigorous evaluation found that scholarship recipients were doing worse in math after two years, but that the program had increased parent and student perceptions of school safety.
Study authors Mark Dynarski, Ning Rui, Ann Webber, and Babette Gutmann found that the D.C. Opportunity Scholarship Program had a statistically significant negative impact on scholarship users’ math achievement after two years of participation. Students with a scholarship had math scores that were 10 percentile points lower than students who applied for, but were not awarded, a scholarship.
Reading achievement scores were 3 percentile points lower for scholarship students, but the difference was not significant.
The D.C. scholarship program appears to be demonstrating in real time what the University of Arkansas’ Jay Greene suggested recently in a widely cited post:
Schools and programs that increase test scores quite often do not yield higher high school graduation or college attendance rates. Conversely, schools and programs that fail to produce greater gains in test scores sometimes produce impressive improvements in high school graduation and college attendance rates, college completion rates, and even higher employment and earnings.
Indeed, an experimental evaluation of the scholarship program found that participating students graduated at a rate 21 percentage points higher than their peers who applied for, but did not receive, a scholarship. That is, the impact of voucher use in the program resulted in a 91 percent graduation rate compared to 70 percent in the control group.
So, while the D.C. Opportunity Scholarship Program might not be having an immediate positive effect on the standardized test scores prized by researchers and policy bean counters, it is having a potentially more meaningful, long-term impact on participants’ academic attainment and future life outcomes.
Moreover, the most extensive study to date of the intersection of test scores and later life outcomes demonstrated a weak relationship between the two, finding that a school choice program’s effect on student test scores was a weak predictor of longer-term outcomes such as graduation and college attendance.
The study’s authors, Collin Hitt, Michael Q. McShane, and Patrick J. Wolf, found that “achievement impact estimates appear to be almost entirely uncorrelated with attainment impacts.” The authors urge humility on the part of policymakers when it comes to “what they believe that test scores tell them about the performance of schools of choice.”
It’s also worth noting that while researchers often look to test scores to determine school quality, parents do so to a far lesser extent. Parents prioritize school safety, while student performance on standardized tests is one of the least important factors parents cite.
It is notable, then, that the D.C. Opportunity Scholarship Program had a considerable impact on parent and student perceptions of school safety. The new evaluation found a 19.5 percentage point difference between parents of scholarship users and the control group in their perception of their school as being very safe—a statistically significant difference.
Fully 74.2 percent of parents of scholarship users rated their child’s school as “very safe,” compared with 54.7 percent of parents of students who applied for, but were not awarded, a scholarship.
Students themselves also felt safer in their new schools: 59.2 percent of students who used a D.C. scholarship to attend a private school of choice rated their schools as “very safe,” compared to 43.7 percent of students who applied for, but did not receive, a scholarship—a statistically significant difference.
In a city that has long struggled with school safety, it is no wonder that parents would prioritize finding a school in which they feel their children are safe. The D.C. Opportunity Scholarship Program has enabled them to do that.
The bottom line is this: The new evaluation shows the program did not move the needle on something parents care little about anyway (standardized test outcomes), but had a significant impact on something parents prioritize highly (the safety of the school to which they entrust their children for the day).
The D.C. Opportunity Scholarship Program is producing two critical outcomes for participants: significantly higher graduation rates and safer students. And the fact that this program does so for fractions on the dollar spent in D.C. Public Schools is just icing on the cake of school choice.
