Testing and Accountability

USA K-12 achievement stalled on international exams

Matthew Ladner December 9, 2019

PISA, the OECD’s Programme for International Student Assessment, measures 15-year-olds’ ability to use their reading, mathematics and science knowledge and skills to meet real-life challenges.

The performance of American 15-year-olds in reading and math has remained stagnant for the past two decades according to results released this past week from the Organisation for Economic Co-operation and Development (OECD). Meanwhile, the achievement gap in reading between high- and low-performing students has grown wider.

The less-than-stellar results from the exam, the Program for International Student Assessment (PISA), mirror recently released scores from the National Assessment of Educational Progress (NAEP), which I described in an earlier post as an Agincourt-level disaster.

Chile is the country most closely resembling the math and reading scores of American black students; Turkey most closely matches the combined achievement of American Hispanic students. Yet the United States spends more than twice as much per pupil as either Chile or Turkey.

Even the scores for American white students, while internationally competitive, appear less than impressive.

Estonia scores a bit higher while spending half as much per pupil as the United States. Moreover, scholars have made us aware that higher-income American families spend lavishly on enrichment options for their K-12 students (tutoring, club sports, etc.), and that this spending has increased steadily over time.

How many Estonian families do you reckon spend $8,872 per child per year on enrichment spending? Since the average American income is more than $26,000 per person higher than that in Estonia after adjusting for purchasing power, I’m going to walk out on a limb and dare a guess: Not many.

And, while enrichment spending apparently has little to do with improvement among low-income students (the trend in such spending is flat since the early 1980s), that isn’t the case among advantaged students. Richer in schools twice as well funded but underperforming is not a great place for America’s highest-performing subgroup to find itself vis-à-vis Estonia.

Even without these latest PISA results, but reinforced in light of them, it’s clear that without the benefit of lavish enrichment spending and other related advantages, the high levels of spending in American schools appears broadly ineffectual for students of color.

December 9, 2019 3 comments

Commentary and opinion Education and Public Policy Jonathan Butcher School Choice Testing and Accountability

Guest commentary: NAEP should be a wake-up call for ed reformers, policymakers

Jonathan Butcher November 21, 2019

Jonathan Butcher

A recent comparison of K-12 children around the U.S. brought bad news for education reformers—an amorphous group of policymakers and advocates who are akin to locksmiths searching for the right combination of resources and policy ideas to unlock student potential. The news was bad for students, too, but since the scores do not affect a student’s report card, the results mattered more to the aforementioned locksmiths today.

The results will matter for students tomorrow.

As readers of this blog will know, the Nation’s Report Card, also known as the National Assessment of Educational Progress (NAEP), is a set of math and reading tests given to a sample of fourth- and eighth-graders around the country every two years. The U.S. Department of Education also administers NAEP tests in other subjects and specific cities.

The more frequent math and reading results allow education locksmiths to gauge whether any number of inputs in the nation’s schools—from spending increases to new laws—are having their desired effects. With both more money and federal law firmly in place, observers were frustrated that national averages for 2019 reading and math scores fell from 2017 (with the 1-point improvement in fourth-grade math a small exception).

Rigorous research is required to appropriately link policies and school budgets to NAEP outcomes, but Florida’s aggressive upgrade to its K-12 education design in the early 2000s tracked closely with improved NAEP scores among minority students. The additional parent options in education, along with a focus on reading among third-graders and more attention to the use of Advanced Placement testing has been the subject of policy discussion around the country since.

This year, reformers pointed to similar reading-related policies in Mississippi to explain the state’s improvement in fourth-grade scores, while calling for “urgent action” elsewhere. The Council of Chief State School Officers even said the “urgency of improving outcomes for all students” was enough to plan … a meeting. This “urgent action” is why the results will matter for students tomorrow: That is when the changes will affect the classroom. Parents can only hope it is not too late by then.

Nationally, average scores among fourth- and eighth-grade students increased in math by 27 points and 20 points, respectively, between 1990 and 2009. Yet since 2009, and a decade is almost the length of a child’s K-12 career, the improvement stalled. Reading scores have improved by just three points in both grades since 1992.

It is these scores and other comparisons that have not changed that should bother reformers and policymakers alike in 2019. Last summer, Harvard University’s Paul Peterson wrote in Education Next that the test score gap on NAEP and international tests between students from low-income families and their more affluent peers has not changed since the 1960s. Peterson wrote later that the “performances on math, reading, and science tests of the most advantaged and the most disadvantaged students differ by approximately four years’ worth of learning, a disparity that has remained essentially unchanged for nearly half a century.”

Such findings temper enthusiasm for any year-to-year increases. So, too, do the results from NAEP’s other test, the Long-Term Trend Assessment (LTT). Average scores for 17-year-olds in math and reading have not changed since the 1970s (the LTT was last given in 2012 and is scheduled to resume in 2020). Students appear to lose any gains made in elementary and middle school by graduation.

Over time, these results have been depressingly more of the same, which should incentivize education reformers and policymakers not to do the same things. Such as: In the last two years, many state policymakers were either absent or complicit in teacher union attempts to keep students in assigned district schools by curbing charter school growth in Los Angeles and Chicago and blocking new private learning options in Kentucky and West Virginia.

These options are the most significant departures from the routine of school-by-zip-code because parents can help a struggling child immediately by moving them to a new setting with a charter school, private school scholarship or education savings account. Students will not have to wait for the urgent meetings to finish, hoping to find success this time.

Now that would be different.

November 21, 2019 0 comment

Analysis Education and Public Policy Education politics Testing and Accountability

NAEP 2019: Where do we go from here?

Matthew Ladner November 14, 2019

Matthew Ladner

The aughts were a time of K-12 academic improvement. For some states, including Florida, that improvement started earlier, but included the era illustrated in this chart.

All states began participating in NAEP exams in 2003. Most states experienced a pronounced improvement in eighth-grade reading and eighth-grade math from 2003 to 2009. On these tests, 10 points equals approximately one grade level’s worth of average progress.

As you can see, there are far more bars going in a positive direction in the first chart and relatively few going in a negative direction between 2003 and 2009. Now let’s examine the same sort of chart between 2009 and 2019.

Things went badly after 2009 in most states. Some broad theories readily come to mind but are weaker than they might seem.

Most states adopted the Common Core standards after 2009, for instance. Prominent analysts of the center right and left have studied the impact of Common Core on academic achievement and have reached a common conclusion: not much positive or negative impact. Look at Texas on both charts; positive experience in 2003 to 2009, deeply negative experience from 2009 to 2019. Texas never adopted Common Core.

Obviously, lots of states that did adopt also had bad decades, but Mississippi had the biggest 2009 to 2019 gains. Judged against lofty promises of improvement, the Common Core project clearly fell flat at a high cost in terms of political and financial capital. That is very different than saying they worsened outcomes, but it’s still very bad.

It was the economy! My friend Mike Petrilli has been exploring the theory that the Great Recession is a driver of decline in NAEP performance. Texas, again, is a fly in this ointment.

The Texas economy got off light in the Great Recession and boomed economically for much of the 2009 to 2019 period, but Texas scores worsened statewide. The so-called “Sand States” – Arizona, California, Florida and Nevada – were the hardest hit in the housing downturn, but they all had a much better decade driving academic improvement than Texas.

Per-pupil spending has been increasing since fourth-graders of 2019 took the NAEP, and for most of the time eighth-graders have been in school. We should hope that Mike is wrong about this given that the country is overdue for a recession by historical standards.

Sandy Kress makes the case that the blame lies in the Obama administration’s decision to let the pressure off test-based accountability. The trends certainly show something went south after 2009, and Sandy may be right. It’s worth considering whether something had to give with the 100 percent-ish proficiency on a deadline NCLB requirement.

Before President Obama took office, I had feared states might simply drop their passing thresholds if the federal government stuck to its guns. Moreover, the Common Core project coupled with teacher evaluation can be seen as a doubling down on high-stakes testing strategies, one that radicalized opponents countered in a variety of (largely predictable) ways.

So, what to make of all of this?

Providing academic transparency to parents, taxpayers and policymakers remains vital, but it’s clear the public is not on board for heavy-handed, top-down testing mandates. We live in a democracy, and the demos is not enamored with the idea of closing schools based on test scores and firing teachers based on test scores, and are sick of the amount of test prep going on in school these days. If you doubt any of that, feel free to focus-group it, or alternatively go outside and talk to normal people.

In many states, much of the low-hanging academic improvement fruit of the NCLB era has been reversed post 2009. Sun Tzu wrote that “a victorious general wins and then seeks battle whereas a defeated army seeks battle and then seeks victory.” It’s clear which of these two profiles education reformers more closely resemble today.

A good first step to recovery would be to focus on policies that have the prospect of developing supportive constituencies rather than annoying the public. Those who adhere to the status quo have the upper hand in most states even without antagonizing the public.

November 14, 2019 1 comment

Achievement Gap Analysis Education reporting Testing and Accountability

NAEP 2019: Where does Florida stand?

Patrick R. Gibbons November 5, 2019

Patrick R. Gibbons

Year-to-year student achievement will always fluctuate, but how does Florida stand in the larger picture? While results from the 2019 National Assessment of Educational Progress should concern everyone involved with K-12 education, Florida’s long-term trend is holding despite the recent score stagnation. And as we reported last week, Florida maintained high rankings when adjusting for demographics.

Among eighth-grade math students, Florida has improved 24 points compared to the national average of 19. Black students are up 28 points compared to 23 points nationally, and Hispanic students are up 30 points compared to 22 points nationally.

Eighth-grade reading, typically a sore spot for Florida given the tremendous success in the early years, is up 8 points compared to 21 years ago. But the situation is much worse for the national average, where there has been no improvement on eighth-grade reading scores since 1998.

Black eighth-graders in Florida, despite being down from last year, still have improved by 8 points since 1998. The average black student nationally has seen no improvement since 1998.

Hispanic students fare better on English reading, up 12 points in Florida and 9 points nationally since 1998.

November 5, 2019 0 comment

Demographic Research Education reporting Testing and Accountability

NAEP 2019: Mississippi improvement more impressive under closer examination

Matthew Ladner November 4, 2019

Matthew Ladner

Federal officials released results from the 2019 National Assessment of Educational Progress, or NAEP, Oct. 30. The congressionally mandated project administered by the National Center for Education Statistics provides group-level data on student achievement in reading and mathematics for grades 4 and 8.

While last week’s NAEP news was glum nationwide, Mississippi students performed relatively well. You have to dig into the details to see just how well.

First, a bit of backstory.

Mississippi is one of the nation’s poorest states and has the largest African-American student population in the country. The state has ranked at or near the bottom on NAEP scores for many years. Over the past decade, policymakers have adopted several K-12 policies familiar to Floridians: A-F letter grades for schools, a strong emphasis on early literacy, a charter school law and an education savings account program for students with disabilities.

The state’s choice sector went from non-existent to nascent in recent years, although it’s not yet at a scale where it can play more than a complimentary role to the public education system. Yet based on this year’s NAEP, as seen in this crosstab of student race/ethnicity on fourth-grade reading, something seems to have gone very right.

Average scale scores for Grade 4 reading by National School Lunch Program eligibility across race/ethnicity for Mississippi, nation

Free and reduced-price meal eligible white students in Mississippi, as well as free and reduced-price meal eligible black students, outscored their peers by 9 points. Not to be outdone, eligible Hispanic students beat the national average by 15 points. These differences are statistically significant.

Meanwhile, Mississippi black students who do not qualify for free and reduced-price meals (middle to high income) outscored their peers nationwide by 22 points.

Mississippi didn’t just clobber national averages; the state took it to the nation’s wealthiest states as well. Northeast states (Connecticut, Maine, Massachusetts, New Hampshire, New Jersey, New York, Pennsylvania, Rhode Island and Vermont) have long held the highest NAEP scores, but they tend to hold the same advantaged end of various achievement gaps as wealthy states. Many of those states spend far more per pupil than Mississippi as well.

NAEP allows comparisons not just between states, but also between states and regions. So, here are the trends for fourth-grade reading broken down by student free and reduced-price meal eligibility status.

Back in 2009, students in Northeastern states in both low-income and middle/high income categories were well ahead of that cohort in Mississippi, but in 2019, Mississippi led among low-income students and tied among non-eligible free and reduced-price meal students. Mississippi students progressed from being well behind students in Northeastern states to being virtually tied in both categories on eighth-grade reading over the past decade.

As a relatively late adopter of reform efforts, Mississippi had the chance to learn from earlier successes and earlier mistakes. Alas, there have been more of the latter than the former.

Massachusetts began its reform efforts in 1993 and had the highest scores on all four NAEP exams until 2019. Florida’s scores surged after 1998 with a different, but also multi-faceted set of reforms. Arizona students were unique in achieving statistically significant gains on all six NAEP exams between 2009 and 2015, and Mississippi joined Arizona in that category in 2017.

Initial success does not guarantee sustained success, but for now, Mississippi leads in the academic improvement clubhouse. I hope it continues its success – and that the rest of us can give the state some competition.

November 4, 2019 0 comment

Achievement Gap Education reporting News Testing and Accountability

NAEP 2019: Florida maintains high marks despite national slide

Patrick R. Gibbons November 1, 2019

Patrick R. Gibbons

In a statement released Wednesday by the U.S. Department of Education, Secretary of Education Betsy DeVos called 2019 NAEP results “devastating,” opining that the country is in a “student achievement crisis” that has continued to worsen over the past decade.

While the results aren’t great, they’re not quite that bad. One bright spot is that Florida maintains high rankings when adjusting for demographics.

Florida ranks first in fourth-grade reading, first in fourth-grade math, third in eighth-grade reading and eighth in eighth-grade math, according to the Urban Institute’s adjustment of rankings for race, income and English language learner status – and maintains those rankings despite drops in scores for students in the Sunshine State. Nevertheless, those declines are worth exploring.

Florida’s fourth-grade reading dropped 3 points, with Hispanic students seeing a 4-point decline and low-income students seeing a 3-point decline. Eighth-grade reading dropped 4 points, with black and Hispanic students dropping 6 and 3 points, respectively. Florida’s overall eighth-grade reading scores tie the national average, consistent with last year’s scores.

Florida maintained its scores in fourth-grade math and eighth-grade math, however black students declined 3 points while eighth-grade Hispanic students improved by 3 points. Florida’s eighth-grade black students scored at the national average relative to their peers, while Hispanic students maintained a healthy 8-point lead. Florida’s eighth-grade Hispanic math students are the state’s lone bright spot within the 2019 NAEP results.

As the charts show, Florida’s results in 2019 are still far beyond where Florida students started back in 1998, despite the decline.

November 1, 2019 0 comment

Achievement Gap Education reporting Testing and Accountability

NAEP 2019: Agincourt-level disaster

Matthew Ladner October 31, 2019

Matthew Ladner

Editor’s note: redefinED executive editor Matt Ladner, responding to a request from EducationNext, made a prediction about 2019 NAEP scores prior to their release Wednesday. Ladner’s subsequent commentary for EducationNext, following the release, is published below. You can read reactions from other national education policy experts here.

Shakespeare’s Henry V includes a scene in which the young King of England reads off a list of casualties his army inflicted upon the French after the Battle of Agincourt, including a long list of French nobles.

“Here was a royal fellowship of death…” Henry notes grimly. This scene comes to mind when reviewing the 2019 NAEP, and states are generally not, alas, playing the role of Henry.

NAEP 2019: Hispanics perform better in Republican-held states

Patrick R. Gibbons October 30, 2019

Patrick R. Gibbons

Between January and March, students who participated in NAEP were assessed in mathematics, reading, and science. Most students took the assessment on tablets, while a small subset took paper-and-pencil versions to help NAEP evaluate any differences in student performance between the two methods of test administration.

Last week Education Next published predictions from leading education scholars about how the 2019 National Assessment of Education Progress (NAEP) test results would shake out. The boldest prediction came from Morgan Polikoff, an associate professor at the University of Southern California.

Polikoff anticipated that President Donald Trump’s “racism” and “attacks on Latinix people in particular” would cause an increase in the achievement gap between whites and Hispanics, with the largest gaps growing in border states and states controlled by the Republican Party.

You can read his full prediction here.

Polikoff is correct that America is more politically polarized today than in many decades, so it’s clearly worth exploring the prediction. But the results suggest Polikoff appears to either grant Trump far more power over student achievement than is realistic, or assumes students are more fragile than they really are. Perhaps both.

Either way, his hypothesis is not only incorrect, but we find larger achievement gaps in Democratic-controlled states than Republican ones. In fact, Democratic-controlled states have less achievement equity than Republican states.

First, the border states:

Border states’ achievement gaps and changes in achievement gaps are similar to the rest of the U.S. Border states controlled by Democrats perform slightly worse than Republican ones, though this difference is mainly driven by California.

States with a Republican “trifecta” — that is, one-party control of both houses of the legislature and the governor’s office — see smaller racial achievement gaps in fourth-grade reading, eighth-grade reading, fourth-grade math and eighth-grade math than Democratic trifectas. Democratic-controlled states, on average, performed worse compared to states with divided governments. States with Republican legislatures and Democratic governors, on average, performed better than states with Democratic legislatures and Republican governors.

There were 13 states with Democratic trifectas and 21 states with Republican trifectas. Twelve states had divided governments, though Nebraska has a unicameral legislature instead of a traditional bicameral (house and senate). The National Conference of State Legislature’s database was used to determine party control.

The results are the same when examining control of the state legislature only. There were 29 states with full Republican control of the legislature and 16 states with full Democratic control. States with insufficient Hispanic students to provide a score were dropped from the comparison.

The achievement gap between 2017 and 2019 was also more likely to decline in Republican trifectas than Democratic trifectas. Republican trifectas did see an increase in the eighth-grade reading gap, but Democratic trifectas saw a much larger widening of the fourth-grade math gap. Divided governments performed better than Democratic trifectas, except on eighth-grade reading. States with Republican legislatures, but Democratic governors, performed better than those with Democratic legislatures and Republican governors.

Negative numbers in the chart below mean the gap between white and Hispanic students closed. A positive number represents an increase in the achievement gap.

When examining only one-party control of the legislature, Democratic-controlled states perform worse across the board on the change in the achievement gap.

Finally, I checked to see if the Democratic achievement gaps might simply be due to higher overall performance. Democratic-controlled states are, on average, much better at teaching white students than Republican-controlled states but are worse at teaching Hispanics. This remains true for trifectas and one-party legislative control.

In other words, Republican-controlled states are more likely to provide equitable education outcomes than Democratic states. If racism is the cause of achievement gaps, as Polikoff hypothesized, it certainly isn’t more pronounced in Republican states. States under Democratic control, however, might want to look inward.

October 30, 2019 0 comment

Load More Posts