Last week Diane Ravitch warned her readers not to trust the findings of the latest Center for Research on Education Outcomes (CREDO) report highlighting student achievement in charter schools. To make her case, she cited a review of the CREDO study written by the National Education Policy Center (NEPC). Diane Ravitch citing NEPC struck me as humorous for a very good reason.
As the CREDO study results changed direction in favor of charter schools, both Ravitch and NECP took a u-turn of their own.
Now there is nothing wrong with changing your mind. In fact, I applaud people who review research and reassess their previous held beliefs, because it takes considerable courage to change your mind or admit you were wrong. However, the CREDO research methods didn’t change and neither did the concerns raised by NEPC. The only thing that did change was the CREDO results, and it no longer supported Ravitch’s or NEPC’s professional opinion.
Back in 2009, the CREDO report concluded, “Despite promising results in a number of states and within certain subgroups, the overall findings of this report indicate a disturbing — and far reaching — subset of poorly performing charter schools.”
When Ravitch accepted the National Education Association's “Friend of Education” award in 2010, she cited CREDO findings stating, “five out of six charters will get no different results or worse results than the regular public schools.” Ravitch was still highlighting CREDO findings as late as the summer of 2011, but today she attacks CREDO as part of a corporate education reform plot to privatize education.
Like Ravitch, the NEPC had nice things to say about CREDO back in 2009. Despite very little change in NEPC’s own concerns, or even in the CREDO methodology, the organization’s opinion turned decidedly negative in 2013 when the CREDO results shifted in favor of charter schools.
In 2009, NEPC wrote that the “CREDO study strengthens the well-established, broader body of evidence showing average charter performance to be equal to, or perhaps lower than, the performance of traditional schools ...” [emphasis added] NEPC did find areas to critique and described these problems as “some weaknesses” and “commonplace limitations.” But it rated the study as pretty solid overall: “The relative strength and comprehensiveness of the data set used for this study, as well as the solid analytic approaches of the researchers, makes this report a useful contribution to the charter school research base.”
In 2013, with the results more favorable to charter schools, NEPC’s largely similar criticisms were suddenly stated with more forcefulness. They wrote, “There are significant reasons for caution in interpreting the results.” [emphasis added] Regarding the now-positive impacts of attending a charter school, NEPC writes “these effects are so small as to be regarded, without hyperbole, as trivial.” Of the research methodology, NEPC concluded: “This review has noted reasons for caution when making inferences to a true causal effect of charter schools.”
To be clear, the CREDO methodology did not change in any significant way in the intervening years and neither did the size of the observed effects. The results and effect sizes were no less “trivial” in 2009, though the NEPC concluded charter schools could be performing worse than public schools. But instead of agreeing with CREDO's observations, like it did in 2009, NEPC concludes that charter schools are “essentially indistinguishable from public schools in terms of their effects on academic performance.”
To be fair to the NEPC, the critiques in 2009 and 2013 were written by different authors. But consistency of language and tone matters. The contradictory attitudes coincide with a change in results more favorable to charter schools, and that might tarnish a reasonable person’s opinion of the NEPC.
