Testing and Accountability

Test scores plus responsibility

John E. Coons November 13, 2020

John E. Coons

“Liberty means responsibility.”

George Bernard Shaw, Man and Superman

The school choice movement continues to plead its case by focusing on “results.” That is, the experts compare the test scores of low-income students whose parents have transferred them to a “charter” school against those of age-mates who remain in the public school to which they were assigned according to their residence.

This fixation on numbers is understandable; the media need to keep the tale as simple as possible, and scores do exactly that.

Up to now, the outcomes so measured favor the performance of charters to a fair degree; in the world of numbers, choice appears to work. Of course, in the hands of anti-choice professionals, these reports can be and are portrayed as the effect of greater sophistication of those parents who made the choice of charters and of their children who have already profited by living with them in such a home.

In any case, the public who consume such conflicting news can, and too often do, conclude that choice would be okay – but it’s no big deal.

But it is. The comparison of scores is, of course, relevant to the wisdom of subsidizing choice for the poor. If, instead of this positive picture, there were a gulf in favor of those students whose families who decided their child should stay put, we might worry about aiding parents to make the escape.

But, in fact, choice of charters and private schools seems not merely to do no harm, but rather to raise scores and make subsidized choice possible. Society can now turn to address the more profound social problems it has created by its disabling of the parent.

The core argument for empowering the poor lies not in statistics but rather in the centrality of parental responsibility in creating and maintaining a stable home and, in so doing, increases our hope for the good society.

Here is a reality not so easily tested; there is no standard mathematics to report the spirit and functioning of parents and its link to the maturing of the heart and mind of their child. There is only our human experience of families.

I fear that, in the eyes of the child, by deposing the parent we have rendered fathers and mothers figures of impotence. They may ask their child at dinner how the school is shaping up, but, when given a negative response, they are helpless to aid their own.

And to the child, the overall message is clear: For 180 days of the year, the raising of a family is out of the parents’ jurisdiction. They may be loving people, but they remain powerless subjects of the public school. The design and pursuit of the good life are in government hands.

One must ask: Just what is the point of this peculiar subordination of fellow citizens according to their bank account? And just what is its effect upon the child’s respect for the very idea of family, and upon the parents’ respect for themselves?

And finally, what will be the contribution of each to the civil order?

November 13, 2020 0 comment

2020 Presidential Election Analysis Commentary and Opinion Coronavirus / COVID-19 Demographic Research Education Spending Featured Testing and Accountability

NAEP scores: American public schools spend more but deliver less for students, families, taxpayers

Matthew Ladner October 29, 2020

Matthew Ladner

The performance of American public schools was in decline before the pandemic struck; based on the latest results from the National Assessment of Educational Progress released Wednesday by the National Center for Education Statistics, things are likely to only get worse from here.

The data show the average reading score for the nation’s 12^th-graders declined between 2015 and 2019. Meanwhile, there was no statistically significant change in 12^th-graders’ average mathematics score for the same time period.

Bottom line: considerably more money per pupil was spent to get the same not-so-great results.

The tests upon which the data is based were given in spring 2019, before the COVID-19 pandemic. The indications are now worse: Inflation-adjusted spending per pupil is up, childhood poverty is down, and scores are down rather than flat.

The earliest 12^th-grade reading score in this series comes from 1992. The Class of 1992 benefited from a nationwide average of $105,560 in 2018 constant dollars spent on their K-12 education. The Class of 2017, the cohort from which we have the most recently available data, had a nationwide average of $158,431 in constant dollars spent on their education – approximately 50% more. The figure for the Class of 2019 will be even higher.

Which class, 1992 or 2017, demonstrated better reading ability? Let’s break down the results by parental education.

Regardless of the level of a parent’s education, reading scores were lower for the Class of 2019 than the Class of 1992. All of the above differences are statistically significant.

Now take a look at the chart below, provided by Michael J. Petrilli from the Thomas B. Fordham Institute, showing the decline in childhood poverty rates from the 1980s and 1990s.

What the chart shows: higher spending, less poverty and lower national achievement.

Mind you, this was the trend before the current massive decline in instruction time due to the pandemic. I’ll dare to predict that if the NCES manages to conduct the scheduled 2021 NAEP, exams scores will decline across the board and achievement gaps will grow. K-3 kids who are in their literacy acquisition windows, for instance, in districts like Los Angeles Unified, Clark County Nevada and New York have been receiving less than half the amount of instruction time delivered during a normal school year.

And finally, special education trends were a disaster in many states before the pandemic, as detailed in this chart.

It’s difficult to imagine that this already dismal chart won’t look even worse with 2021 data, coming in the aftermath of generally reduced instruction time and special education being attempted using the Zoom platform. We are not out of the pandemic yet, but the academic damage seems likely to greatly outlive the virus.

These most recent data came among favorable conditions of declining poverty and increased spending. Very soon, we’ll be forced to face what happens when you reverse these favorable trends and we end up with a large percentage of students with huge academic deficits.

Buckle up.

October 29, 2020 0 comment

Analysis Catholic Schools Featured Homeschooling School Choice Testing and Accountability

NAEP History, Geography, Civics scores paint dismal picture

Matthew Ladner April 27, 2020

Matthew Ladner

Diffusion, then, rather than discovery, is the duty of our government. With us, the qualifications of voters are as important as the qualifications of governors, and even comes first in the natural order. -Horace Mann

Preparing students to responsibly exercise citizenship represented a foundational aspiration in the founding of the American public-school system. Then as now, we did not simply aspire to have schools equip students with the knowledge and skills necessary for economic flourishing; we also wanted students to develop a grounding in history and civics.

Our education system is failing to accomplish this goal.

Last week, the National Assessment of Educational Progress (NAEP) released nationwide eighth-grade data on the 2018 exams taken in United States History, Geography and Civics. Fewer than one-third of American public-school students scored proficient on any of the three exams. In fact, on each exam, the percentage of public-school students scoring “Below Basic” exceeded the percentage scoring “Proficient.”

For every eighth-grade public-school student who scored “Proficient,” there were more than 2.5 who scored “Below Basic,” the lowest achievement level. We can’t be sure what the ratio of “Proficient” to “Below Basic” would be among American eighth-graders if we abolished their history classes, gave them a library card and offered to pay them a dollar for every history book they read, but it might surpass 2.5 to 1.

While scores aren’t available for private schools in general, scores for Catholic school students are – and they tell a different story.

Scores of Catholic school students show an advantage, in several ways: when comparing them to free and reduced-price lunch-eligible public-school students; non-eligible free and reduced-price lunch populations; and racial and ethnic groups. If you compare Catholic school students of parents without college degrees to public-school students of parents without college degrees, Catholic school students get the better of it. The same holds true when you compare Catholic school students of college-educated parents to public-school students of college-educated parents.

Ironically, we heard last week a faint echo of the Know-Nothing Party’s anti-Catholic school rhetoric directed at home schooling. Harvard Magazine quoted Harvard Law professor Elizabeth Barthole:

“From the beginning of compulsory education in this country, we have thought of the government as having some right to educate children so that they become active, productive participants in the larger society,” she says. This involves in part giving children the knowledge to eventually get jobs and support themselves.

“But it’s also important that children grow up exposed to community values, social values, democratic values, ideas about nondiscrimination and tolerance of other people’s viewpoints,” she says, noting that European countries such as Germany ban homeschooling entirely and that countries such as France require home visits and annual tests.

People can have honest disagreements about home-school policy, but I invite the reader to ponder the first chart in this post and attempt to square it with the quote above. Should we be worried about the civic values and knowledge of the 3 percent of students who home-schooled before the pandemic? Or might the appalling lack of history, geography and civics displayed by the 87 percent or so students attending public schools constitute a more pressing concern?

April 27, 2020 1 comment

Analysis Charter Schools Education Choice Education Spending factcheckED Featured Private School Scholarships School Choice Testing and Accountability

FACT-CHECKED: Tallahassee educator’s attack on ‘monstrous’ system is misleading

Jon East January 23, 2020

Jon East

Editor’s note: The goal of fact-checkED is to bring clinical precision to complex issues that are easily misunderstood, aiming to counteract incorrect information before it continues to circulate.

Sally Butzin has a rich and accomplished history in public education, as a classroom teacher, instructional innovator and published author. But her latest column in the Tallahassee Democrat offers an unfortunate brew of bitterness and hyperbole about modern reforms.

Butzin cast Florida today as something of an educational wasteland, reminiscing about the early 1990s when “innovation was encouraged” and contrasting that with the election of Gov. Jeb Bush in 1998 when “the bottom fell out.” She deplored the “emphasis on testing and accountability” but especially blames charter schools and private school vouchers. They constitute a “monstrous parallel system” that “has bled the life out of public education” and “left millions of children behind.”

We’ll leave to others the political defense of Bush, who ushered in a wave of education reform in his eight years in office, and instead focus on three of her more tangible claims.

Claim No. 1: “These new schemes had no accountability to taxpayers.”

The “schemes,” in this context, refer to charter schools and private school scholarships.

First, charter schools are created through a contract with each public school district that involves oversight into educational goals and tax dollars received. Their students are required to take the same state assessment tests as all other district students and the schools receive the same letter grade ratings as public schools. Further, under state law, a charter school can be shut down, and many have been, for receiving two grades of F.

Second, the state’s largest scholarship program, the Florida Tax Credit Scholarship serving 108,570 economically disadvantaged students in 1,836 private schools, is overseen by the state Department of Education. Students are required to take nationally norm-referenced tests approved by the state and those test results are reported publicly each year. Schools that receive more than $250,000 in scholarship dollars are required to submit financial reports by certified public accountants, and scholarship funding organizations are audited each year by the State Auditor General and private accounting firms. The students, schools and scholarship organizations are subject to roughly 17,000 words of statutory and agency regulations.

Third, both charter schools and scholarship schools face a form of accountability that can be more intense than state regulations and audits. Because they are schools of choice and students are free to attend or not, they can go out of business if parents are dissatisfied and choose elsewhere.

These points are brief in part because Butzin’s claim is categorical. She wrote that they have “no accountability,” which is flatly false.

Claim No. 2: “This parallel system has bled the life out of public education, as billions of dollars flowing to it have left millions of children behind.”

This claim seems to have a life of its own, despite the fact that it is refuted by extensive independent research. At its core, it assumes that public schools should be paid tax dollars without regard to how many students attend them, even though Florida has always funded education on a per-student basis. And it ignores the fact that students choosing these two alternative options receive fewer dollars than they would if they remained in a traditional public school.

This is the third time in the short history of fact-checkED that we have analyzed this financial cost claim, and for more detail we invite you to read our first effort. Once again, we rate this claim as false.

Claim No. 3: “Now, it would be one thing if this parallel system was improving outcomes for all our children. But this is simply not the case.”

Butzin did not share her data on outcomes, but the National Assessment of Educational Progress (NAEP) is generally regarded as one of the most reliable indicators of academic achievement in elementary and secondary schools. NAEP, often called the “Nation’s Report Card,” paints an entirely different picture.

By every measure, the NAEP reading and math test scores of fourth- and eighth-graders have increased over the past two decades – the period after which Butzin says “the bottom fell out.” The percentage of fourth-graders who meet basic standards in math is up by nearly two-thirds, and the percentage rated proficient or above has tripled. The percentage of fourth-graders who meet basic standards in reading is up by a third, and the proficient and above is up by two-thirds. Similarly, the percentage of eighth-graders who meet basic in math is up by a fifth, and proficient is up by four-fifths. The percentage of eighth-graders at basic in reading is up by a tenth, and the percentage above proficient up by half.

And, according to the Urban Institute’s adjustment of rankings for race, income and English language learner status, Florida ranks first in fourth-grade reading, first in fourth-grade math, third in eighth-grade reading and eighth in eighth-grade math.

A few other measures are worth mentioning. The most recent Education Week Quality Counts report ranked Florida fourth in the nation in terms of K-12 academic achievement. The College Board ranked Florida third in percentage of graduating high school seniors who have passed college-caliber Advanced Placement exams. The high school graduation rate has jumped from 60.1 percent to 86.9 percent in the past two decades.

We don’t know the data Butzin used, but we rate her claim as highly misleading.

January 23, 2020 0 comment

2020 Presidential Election Analysis Education Choice Education Politics Education Spending factcheckED Featured School Choice Tax Credit Scholarships Testing and Accountability

fact-checkED: Sanders’ education choice claims in Florida op-ed misleading

Jon East January 16, 2020

Jon East

With the presidential primary election in delegate-rich Florida just two months away, Democratic candidates are beginning to knock on the door.

In a Monday column in the South Florida Sun-Sentinel, Vermont Sen. Bernie Sanders found little to like about Florida’s public education system. But our focus is on educational choice and three assertions he made while insisting that “Florida is ground zero of a school privatization movement intent on destroying public education.”

“Each year almost $1 billion in state money goes to private schools instead of public schools.”

“These private schools operate with little to no accountability.”

“In many cases, their students’ math and reading skills have declined.”

These claims are misleading, at best.

Claim No. 1: The $1 billion citation draws attention but does not prove Sanders’ point. He implies that private school voucher and scholarship programs harm public schools financially, which is a longstanding and erroneous argument advanced by the Florida Education Association (FEA) that we have dealt with previously in this space. There is simply no evidence to support it.

Eight different independent financial studies all have concluded the scholarships, which cost less per student than the total amount spent in a traditional public school, save tax money that can be used to enhance public schools. Florida TaxWatch weighed in last year, finding existing scholarships cost 60 cents on the public school dollar. And the FEA lost its legal challenge to the 19-year-old Tax Credit Scholarship program in part because the trial and appellate courts summarily rejected its argument that public schools “have been and will continue to be injured by the scholarship program’s diversion of resources from the public schools.”

Claim No. 2: The qualifier in this assertion – that private schools operate with “little to no” accountability – gives Sanders some wiggle room in part because accountability is hard to objectively measure. But Sanders seems to dismiss entirely the accountability that is inherent with any school that survives only if parents choose it for their children.

That market concept may not appeal to a self-professed socialist but does have real-world consequences and, in the case of the state’s two scholarships for underprivileged students, gives genuine educational power to parents who typically have little.

As to the regulatory oversight, Florida’s largest program, the Tax Credit Scholarship serving 108,570 economically disadvantaged students this year, is ranked among the nation’s most aggressive. The students, schools and scholarship organizations are subject to roughly 17,000 words of statutory and agency regulations.

Among those requirements: scholarship students must take state-approved standardized tests; those test results are reported publicly every year; schools must submit annual financial reports by certified public accountants; and scholarship organizations must be audited each year by the state Auditor General and independent accounting firms.

Sanders is entitled to believe the program is not subject to sufficient regulation, but we rate his claim of “little to no” accountability as misleading.

Claim No. 3: Once again, Sanders uses an interesting qualifier – “in many cases” – in his assertion that scholarship students’ “math and reading skills have declined.” Eleven years’ worth of standardized test reports refute his claim.

Students who receive the Tax Credit Scholarship must take a nationally norm-referenced test each year approved by the state, and most take the Stanford Achievement. The test scores are sent to Florida State University’s Learning Systems Institute, which is paid to aggregate and report them publicly.

For the most recent year, 2017-18, scholarship students scored on average at the 47.4 percentile in reading and 45.2 percentile in math. That’s basically average, which is encouraging given that these scholarships students are among the poorest in the state and were among the lowest-performing students in the public schools they left behind.

More to the point, their annual gains have been remarkably consistent. The scores reflect that these low-income students have achieved the same annual gains as students of all income levels nationally.

In other words, they have increased, not declined.

Certainly the average doesn’t reflect every student. In the most recent report, about as many students – 0.6 percent – achieved extraordinary gains of more than 40 percentile points as those who dropped an equivalent amount. But Sanders’ qualifier that “many” students declined seems intended to distort the overall test findings. We rate it mostly false.

January 16, 2020 0 comment

Achievement Gap Analysis Commentary and Opinion Education Spending Featured Testing and Accountability

USA K-12 achievement stalled on international exams

Matthew Ladner December 9, 2019

Matthew Ladner

PISA, the OECD’s Programme for International Student Assessment, measures 15-year-olds’ ability to use their reading, mathematics and science knowledge and skills to meet real-life challenges.

The performance of American 15-year-olds in reading and math has remained stagnant for the past two decades according to results released this past week from the Organisation for Economic Co-operation and Development (OECD). Meanwhile, the achievement gap in reading between high- and low-performing students has grown wider.

The less-than-stellar results from the exam, the Program for International Student Assessment (PISA), mirror recently released scores from the National Assessment of Educational Progress (NAEP), which I described in an earlier post as an Agincourt-level disaster.

Chile is the country most closely resembling the math and reading scores of American black students; Turkey most closely matches the combined achievement of American Hispanic students. Yet the United States spends more than twice as much per pupil as either Chile or Turkey.

Even the scores for American white students, while internationally competitive, appear less than impressive.

Estonia scores a bit higher while spending half as much per pupil as the United States. Moreover, scholars have made us aware that higher-income American families spend lavishly on enrichment options for their K-12 students (tutoring, club sports, etc.), and that this spending has increased steadily over time.

How many Estonian families do you reckon spend $8,872 per child per year on enrichment spending? Since the average American income is more than $26,000 per person higher than that in Estonia after adjusting for purchasing power, I’m going to walk out on a limb and dare a guess: Not many.

And, while enrichment spending apparently has little to do with improvement among low-income students (the trend in such spending is flat since the early 1980s), that isn’t the case among advantaged students. Richer in schools twice as well funded but underperforming is not a great place for America’s highest-performing subgroup to find itself vis-à-vis Estonia.

Even without these latest PISA results, but reinforced in light of them, it’s clear that without the benefit of lavish enrichment spending and other related advantages, the high levels of spending in American schools appears broadly ineffectual for students of color.

December 9, 2019 3 comments

Commentary and Opinion Education and Public Policy Jonathan Butcher School Choice Testing and Accountability

Guest commentary: NAEP should be a wake-up call for ed reformers, policymakers

Jonathan Butcher November 21, 2019

Jonathan Butcher

A recent comparison of K-12 children around the U.S. brought bad news for education reformers—an amorphous group of policymakers and advocates who are akin to locksmiths searching for the right combination of resources and policy ideas to unlock student potential. The news was bad for students, too, but since the scores do not affect a student’s report card, the results mattered more to the aforementioned locksmiths today.

The results will matter for students tomorrow.

As readers of this blog will know, the Nation’s Report Card, also known as the National Assessment of Educational Progress (NAEP), is a set of math and reading tests given to a sample of fourth- and eighth-graders around the country every two years. The U.S. Department of Education also administers NAEP tests in other subjects and specific cities.

The more frequent math and reading results allow education locksmiths to gauge whether any number of inputs in the nation’s schools—from spending increases to new laws—are having their desired effects. With both more money and federal law firmly in place, observers were frustrated that national averages for 2019 reading and math scores fell from 2017 (with the 1-point improvement in fourth-grade math a small exception).

Rigorous research is required to appropriately link policies and school budgets to NAEP outcomes, but Florida’s aggressive upgrade to its K-12 education design in the early 2000s tracked closely with improved NAEP scores among minority students. The additional parent options in education, along with a focus on reading among third-graders and more attention to the use of Advanced Placement testing has been the subject of policy discussion around the country since.

This year, reformers pointed to similar reading-related policies in Mississippi to explain the state’s improvement in fourth-grade scores, while calling for “urgent action” elsewhere. The Council of Chief State School Officers even said the “urgency of improving outcomes for all students” was enough to plan … a meeting. This “urgent action” is why the results will matter for students tomorrow: That is when the changes will affect the classroom. Parents can only hope it is not too late by then.

Nationally, average scores among fourth- and eighth-grade students increased in math by 27 points and 20 points, respectively, between 1990 and 2009. Yet since 2009, and a decade is almost the length of a child’s K-12 career, the improvement stalled. Reading scores have improved by just three points in both grades since 1992.

It is these scores and other comparisons that have not changed that should bother reformers and policymakers alike in 2019. Last summer, Harvard University’s Paul Peterson wrote in Education Next that the test score gap on NAEP and international tests between students from low-income families and their more affluent peers has not changed since the 1960s. Peterson wrote later that the “performances on math, reading, and science tests of the most advantaged and the most disadvantaged students differ by approximately four years’ worth of learning, a disparity that has remained essentially unchanged for nearly half a century.”

Such findings temper enthusiasm for any year-to-year increases. So, too, do the results from NAEP’s other test, the Long-Term Trend Assessment (LTT). Average scores for 17-year-olds in math and reading have not changed since the 1970s (the LTT was last given in 2012 and is scheduled to resume in 2020). Students appear to lose any gains made in elementary and middle school by graduation.

Over time, these results have been depressingly more of the same, which should incentivize education reformers and policymakers not to do the same things. Such as: In the last two years, many state policymakers were either absent or complicit in teacher union attempts to keep students in assigned district schools by curbing charter school growth in Los Angeles and Chicago and blocking new private learning options in Kentucky and West Virginia.

These options are the most significant departures from the routine of school-by-zip-code because parents can help a struggling child immediately by moving them to a new setting with a charter school, private school scholarship or education savings account. Students will not have to wait for the urgent meetings to finish, hoping to find success this time.

Now that would be different.

November 21, 2019 0 comment

Analysis Education and Public Policy Education Politics Testing and Accountability

NAEP 2019: Where do we go from here?

Matthew Ladner November 14, 2019

Matthew Ladner

The aughts were a time of K-12 academic improvement. For some states, including Florida, that improvement started earlier, but included the era illustrated in this chart.

All states began participating in NAEP exams in 2003. Most states experienced a pronounced improvement in eighth-grade reading and eighth-grade math from 2003 to 2009. On these tests, 10 points equals approximately one grade level’s worth of average progress.

As you can see, there are far more bars going in a positive direction in the first chart and relatively few going in a negative direction between 2003 and 2009. Now let’s examine the same sort of chart between 2009 and 2019.

Things went badly after 2009 in most states. Some broad theories readily come to mind but are weaker than they might seem.

Most states adopted the Common Core standards after 2009, for instance. Prominent analysts of the center right and left have studied the impact of Common Core on academic achievement and have reached a common conclusion: not much positive or negative impact. Look at Texas on both charts; positive experience in 2003 to 2009, deeply negative experience from 2009 to 2019. Texas never adopted Common Core.

Obviously, lots of states that did adopt also had bad decades, but Mississippi had the biggest 2009 to 2019 gains. Judged against lofty promises of improvement, the Common Core project clearly fell flat at a high cost in terms of political and financial capital. That is very different than saying they worsened outcomes, but it’s still very bad.

It was the economy! My friend Mike Petrilli has been exploring the theory that the Great Recession is a driver of decline in NAEP performance. Texas, again, is a fly in this ointment.

The Texas economy got off light in the Great Recession and boomed economically for much of the 2009 to 2019 period, but Texas scores worsened statewide. The so-called “Sand States” – Arizona, California, Florida and Nevada – were the hardest hit in the housing downturn, but they all had a much better decade driving academic improvement than Texas.

Per-pupil spending has been increasing since fourth-graders of 2019 took the NAEP, and for most of the time eighth-graders have been in school. We should hope that Mike is wrong about this given that the country is overdue for a recession by historical standards.

Sandy Kress makes the case that the blame lies in the Obama administration’s decision to let the pressure off test-based accountability. The trends certainly show something went south after 2009, and Sandy may be right. It’s worth considering whether something had to give with the 100 percent-ish proficiency on a deadline NCLB requirement.

Before President Obama took office, I had feared states might simply drop their passing thresholds if the federal government stuck to its guns. Moreover, the Common Core project coupled with teacher evaluation can be seen as a doubling down on high-stakes testing strategies, one that radicalized opponents countered in a variety of (largely predictable) ways.

So, what to make of all of this?

Providing academic transparency to parents, taxpayers and policymakers remains vital, but it’s clear the public is not on board for heavy-handed, top-down testing mandates. We live in a democracy, and the demos is not enamored with the idea of closing schools based on test scores and firing teachers based on test scores, and are sick of the amount of test prep going on in school these days. If you doubt any of that, feel free to focus-group it, or alternatively go outside and talk to normal people.

In many states, much of the low-hanging academic improvement fruit of the NCLB era has been reversed post 2009. Sun Tzu wrote that “a victorious general wins and then seeks battle whereas a defeated army seeks battle and then seeks victory.” It’s clear which of these two profiles education reformers more closely resemble today.

A good first step to recovery would be to focus on policies that have the prospect of developing supportive constituencies rather than annoying the public. Those who adhere to the status quo have the upper hand in most states even without antagonizing the public.

November 14, 2019 1 comment