Research brings positive news for vouchers in North Carolina

New research from North Carolina State University breaks a streak of widely publicized negative test results for voucher students. It shows modest to strong positive effects for low-income students who use North Carolina’s Opportunity Scholarship Program.

The study, “An Impact Analysis of North Carolina’s Opportunity Scholarship Program on Student Achievement,” by Anna Egalite, D.T. Stallings and Stephen R. Porter, is the first statewide voucher study in several years to show immediate and strong positive effects right out of the gate.

“The effects sizes observed are positive, large, and statistically significant,” the researchers wrote.

The report comes after several studies over the last two years have shown negative impacts in Louisiana, Ohio, Indiana and D.C. Overall the research still trends modestly positive for voucher recipients with eight of the 15 “gold-standard” random-assignment studies showing positive effects. Only three show negative effects.

North Carolina’s Opportunity Scholarship program was launched in 2013 to low-income students eligible for free and reduced-price lunch. It offers scholarships worth up to $4,200 to attend private schools. Participating private schools do not have to use the same state curriculum or standardized test as public schools.

Last year the scholarship program served 7,344 students attending 405 private schools statewide.

To measure the program’s impact, researchers asked student volunteers in public and private schools to take the Iowa Test of Basic Skills, a nationally norm-referenced test. They collected scores from students in 24 private schools and 14 public schools. Of the private schools, 53 percent were Catholic, 26 percent were nondenominational and 16 percent were Baptist. Of the public schools, 93 percent were traditional and 7 percent were magnet schools. All public schools qualified for federal Title I funding, a proxy for a high-poverty school.

The average participating private school had an enrollment of 225 compared to 650 students in the public school.  The pupil-to-teacher ratio in the participating private schools was 12, compared to 14 in the public schools. Though the participating schools came from similar geographic regions, private schools were more likely to be urban.

Test-takers at private schools were more likely to be male, less likely to be black or Hispanic, and had modestly higher test scores while they were public school students than the comparison group in public schools.

Once researchers collected the test results, they controlled for demographics, disciplinary records and prior test scores. Using 698 test scores from low-income students, the researchers were able to compare 245 private school students with matches in public schools.

They found “first year impact estimates of .36 standard deviations in math and .44 standard deviations in language. Two-year impact estimates suggest that student achievement in language improved by .52 standard deviations.”

Because the research was not a random-assignment study, other unobserved characteristics with students or schools may explain the results.

The findings could shine an interesting light on other voucher studies in states like Indiana and Louisiana.

The North Carolina researchers compared public and private school students using the Iowa Test of Basic Skills. It’s a norm-referenced assessment popular in private schools. They found the test-score impact of vouchers remained positive, but became statistically insignificant, when they excluded test scores from students attending private schools that already used the Iowa test — and therefore might have been better prepared to do well on the assessments.

Similar effects might have been at play in states like Louisiana, where voucher students took the same state assessments as children in public schools. The private schools may not have aligned their curricula as closely to the state standards as the public schools did, placing the private schools in those states at a disadvantage.

The same test-allignment issues that may have placed Lousiana students using vouchers at a disadvantage compared to their public-school peers may have given a slight edge to students who used vouchers in North Carolina.

However, the researchers suggest other factors could also be at play in their results because unlike criterion referenced tests, norm-referenced tests like the Iowa Test of Basic Skills aren’t aligned to any one curriculum. “We cannot rule out the possibility that the large effect sizes for this analysis may reflect factors other than curricular alignment with the assessment,” they wrote in their working paper.

2 Responses to Research brings positive news for vouchers in North Carolina

    • Patrick R. Gibbons
      Patrick R. Gibbons June 20, 2018 at 1:44 pm #

      Hi Kris,

      Thanks for the link. For your readers and ours, I’d like to address some of the issues you raised.
      1) The average scholarship student in North Carolina lives in a household that is eligible for FRL the same threshold used in public schools. While the limit is higher, the average is about the same, thus controlling for race and prior test score would, in fact, be far more important as the researchers state.
      2) Researchers told us that the private schools were from some of the poorest areas, however they cannot identify the schools because of confidentiality agreements.
      3) The public school students were also recruited, so the same limitations/complaints would apply to their test results as well.
      4) It is possible the voucher students were more motivated, as this was not a random assignment test, we don’t know. Researchers mention this too.
      5) A more accurate comparison could be made by requiring norm-referenced testing in private and public schools (no need for a statewide criterion referenced test).
      6) Calling PEFNC “disreputable” is a bit harsh. It is no more disreputable than NC Policy Watch.
      7) This is, by far, the best study on NC vouchers to date. Even if you don’t like the outcomes (or question them) you can’t deny that. That said, more research should be done.

      Thanks for commenting.