Heritage Foundation Analyst Makes Up Statistics (Updated)

October 13, 2010 1:25 pm ET — Walid Zafar

Lindsey Burke, the Heritage Foundation's education analyst, is not happy that DC Public Schools Chancellor Michelle Rhee is resigning at the end of the month.  In a post, Burke blames Rhee's departure on the big, bad teachers' union, which work to "thwart attempts to reform the failed status quo, seeing any opening for children to escape monopoly public school systems as a threat to their power."  Since no Heritage analysis is complete without an attack on President Obama, Burke concludes that the reason Obama is opposed to "the best drop-out prevention program in the country" is because he's afraid of "losing the support of 'big' education."

Yet, just outside the doors of the White House, the best drop-out prevention program in the country has been in place for the past five years: The D.C. Opportunity Scholarship Program. In fact, 91 percent of students who used a scholarship to attend a private school of their choice graduated high school. Just 49 percent of children in D.C. public schools graduate.

It's no surprise that the Heritage Foundation supports school vouchers.  But are the statistics Burke cites and the conclusions she draws from them accurate?  Let's take a look.  Burke links to a report from the Department of Education which looks at the effectiveness of the DC Opportunity Scholarship Program.  From the get go, the report notes, "There is no conclusive evidence that the [Opportunity Scholarship Program] affected student achievement."  The report continues, "On average, after at least four years students who were offered (or used) scholarships had reading and math test scores that were statistically similar to those who were not offered scholarships.

Where does Burke get the 91 percent figure from? Well, not this report.  It's hard not concluding that she made that statistic up. The report puts the graduation rate for students receiving vouchers at 82 percent.  But it's not that simple.  You can't compare the graduation rate at DC Public Schools (which take in all who apply, regardless of learning disabilities and level of parental involvement) to a lottery-based voucher system to which only the most highly motivated students (and parents) choose  to apply.  When you adjust for that important factor, the report puts the graduation rate difference at 12 percent.  That's still significant, but nowhere near the 42 percent difference Burke passes off as proof of the program's success. 

Is the 12 percent difference reflected in academic success?  In the area of student achievement, the report concludes, "Overall reading and math test scores were not significantly affected by the Program, based on our main analysis approach."  Most crucially, the report notes that "No significant impacts on achievement were detected for students" who "were lower performing academically when they applied."  In other words, the students who did well on the voucher program were those who were already doing well in public school.

That's all the proof the Heritage Foundation and its education analyst need to conclude that the program is the "best drop-out prevention program in the country."

UPDATE: The Department of Education study did in fact note a 21 percent difference between those who used the vouchers and the control group (made up of those who applied for the program but were not accepted).  In a response to our post, Burke points out that we didn't differentiate between those who received vouchers and those who actually used them.  We apologize for making that mistake.