Demographic Research

A recent editorial in the Wall Street Journal focused on the work of researchers at the University of Arkansas’ School Choice Demonstration Project that measured school choice environments in the states and Washington, D.C. The Education Freedom Index considered offerings for private school choice such as vouchers and tax credit scholarships, homeschooling, public school choice and charter schools.

The original Education Freedom Index, written by Jay Greene in 2001 and published by the Manhattan Institute, captured broad elements of public-school choice, private choice and home-schooling, and found a positive statistical relationship between choice and state academic performance.

I had the opportunity to work with Greene and his associates Patrick J. Wolf and James D. Paul on the updated index. Once again, a positive association was found between education freedom and outcomes after controlling for other factors.

Journal editors were curious about how these regression coefficients might translate into student outcomes. They wrote:

How could this translate in the real world? Study author Jay Greene says to consider a student about to enter kindergarten in New York, which ranks 35th in the Education Freedom Index.

If his family chose to move to Florida—which ranks 7th—for the child’s K-8 education, the child would climb 10 points on the NAEP (e.g., to the 60th from 50th percentile in achievement) by the time he reached eighth grade compared to if he had stayed in New York.  If the family moved to Arizona, the child would see an 18-point improvement. That’s using the researchers’ most conservative model.

Is this possible?

Outside evidence points in this direction (discussed below) but also the need for caution. Many factors influence academic achievement, both within school and outside of school. Scholars have, for instance, begun to quantify the level of education enrichment spending going on in America. Wealthy families were spending more than $9,000 per child per year in 2006.

We were not able to control for this or for other potential factors in the analysis. New York is the home to far more well-to-do families than either Arizona or Florida, and it is impossible to estimate the role this and other outside factors play in achievement.

We also don’t understand the lags involved between policy changes and outcomes changes. Arizona ranked first in the 2001 index and first again in 2021. Indiana ranked second in 2021, but Indiana’s choice programs are much younger than Arizona’s. Choice in Arizona debuted during the first Clinton administration; it came much later in Indiana.

Divining the drivers of academic achievement is more like a mystery with clues than a science lab with test-tubes. Daniel Patrick Moynihan’s warning should always be borne in mind: “It is quite possible to live with uncertainty, with the possibility, even the likelihood that one is wrong. But beware of certainty where none exists. Ideological certainty easily degenerates into an insistence upon ignorance.”

In that spirit, I will note there are more things in heaven and earth than are dreamt of in any regression analysis.

With that said, both Arizona and Florida made large amounts of academic progress during the period between the two Education Freedom Index studies, New York not so much despite spending far, far more per pupil than either. Arizona’s choice system may have (uniquely?) reached a point of being primarily driven by school districts.

Arizona has the nation’s largest charter school sector at 20% of students, but district open enrollment transfers outnumber charter students nearly two to one in a study of enrollment patterns in the Phoenix area – 31% for open enrollment to 16% for charter schools. Arizona may have achieved, after decades and multiple policies, a sort of escape velocity relative to other states.

The chart below shows the trend in test scores of low-income students in grades 3-8 versus socioeconomic status between 2007 and 2018.

The reader should note that this period starts after Florida’s sizeable improvement in NAEP scores between the late 1990s and 2007 but covers the period of Arizona’s NAEP surge (2009-2015 with mixed results after 2015). If this chart had been available for 1998 to 2009, Arizona and Florida’s positions likely would be reversed. New York’s NAEP scores, meanwhile, show a broad pattern of stagnation and decline after 2003.

The horizontal axis measures the relative socio-economic status of low-income students in each state (most disadvantaged in Mississippi on the far left, most advantaged in New Hampshire on the far right). The vertical axis measures the trend in academic achievement for low-income students in percentages of a grade level gain or decline per year.

The average poor student in Arizona (1) gained .15 grade levels per year during this period, the nation’s highest. Low-income students in Florida (2) gained .04 per year grade levels per year, in New York (3) zero.

See: https://media.tenor.com/images/82136cbb2661bfbc7f079bb3cab40b97/tenor.gif

In New York, high demand charter schools seeking to create more seats for students must battle a mayor determined to make life tough for them. Even in Arizona, the pace at which high demand district schools expand or replicate has been glacial (although that may be changing), and charter schools face practical obstacles like paying for facilities out of operating budgets while receiving fewer public dollars per pupil.

Arizona parents practice frontier justice style accountability with charter schools, closing them down without pity if they don’t desire them. Districts, however, keep many low-demand schools open, with school boards fearful of community backlash accompanying a closure.

A similar political dynamic keeps districts from expanding or replicating high demand schools; the low-demand schools in the district feel threatened by it, keeping it off the table for consideration in most districts. The problem, as always, lies with politics. The status quo is deeply flawed and inequitable, but powerful groups benefit from inertia and resist change.

No state has yet captured anything close to the full potential benefits of choice. The struggle continues.

In the meantime, New York families should take the Wall Street Journal’s hint and move to Arizona or Florida. Super high-cost schools that do too little to teach kids suit certain interests just fine, but they are a terrible deal for you as a taxpayer and parent.