AEI: Voiding the Choice Warrantee

The American Enterprise Institute has a new report  that calls into question one of the foundational fallacies of the entire reform movement. Think of it as the latest entry in the Reformster Apostasy movement.

Do Impacts on Test Scores Even Matter? Lessons from Long-Run Outcomes in School Choice Research asks some important questions. We know they are important questions because some of us have been asking and answering them for twenty years.

Here are the key points as AEI lists them:

For the past 20 years, almost every major education reform has rested on a common assumption: Standardized test scores are an accurate and appropriate measure of success and failure.

This study is a meta-analysis on the effect that school choice has on educational attainment and shows that, at least for school choice programs, there is a weak relationship between impacts on test scores and later attainment outcomes.

Policymakers need to be much more humble in what they believe that test scores tell them about the performance of schools of choice: Test scores should not automatically occupy a privileged place over parental demand and satisfaction as short-term measures of school choice success or failure.

Yup. That's just about it. The entire reformster movement is based on the premise that Big Standardized Test results are a reliable proxy for educational achievement. They are not. They never have been, and some of us have been saying so all along. Read Daniel Koretz's book The Testing Charade: Pretending To Make Schools Better for a detailed look at how this has all gone wrong, but the short answer is that when you use narrow unvalidated badly designed tests to measure things they were never meant to measure, you end up with junk.

AEI is not the first reform outfit to question the BS Tests' value. Jay Greene was beating this drum a year and a half ago:

But what if changing test scores does not regularly correspond with changing life outcomes?  What if schools can do things to change scores without actually changing lives?  What evidence do we actually have to support the assumption that changing test scores is a reliable indicator of changing later life outcomes?

Greene concluded that tests had no real connection to student later-in-life outcomes and were therefor not a useful tool for policy direction. Again, he was saying what teachers and other education professionals had been saying since the invention of dirt, but to no avail.

In fact, if you are of a Certain Age, you may well remember the authentic assessment movement, which declared that the only way to measure any student knowledge and skill was by having the student demonstrate something as close to the actual skill in question. IOW, if you want to see if the student can write an essay, have her write an essay. Authentic assessment frowned on multiple choice testing, because it involves a task that is not anything like any real skill we're trying to teach. But ed reform and the cult of testing swept the authentic assessment movement away.

Really, AEI's third paragraph of findings is weak sauce. "Policymakers should be much more humble" about test scores? No, they should be apologetic and remorseful that they ever foisted this tool on education and demanded it be attached to stern consequences, because in doing so the wrought a great deal of damage on US education. "Test scores should not automatically occupy a privileged place..."? No, test scores should automatically occupy a highly unprivileged place. They should be treated as junk unless and until someone can convincingly argue otherwise.

But I am reading into this report a wholesale rejection of the BS Test as a measure of student, teacher, or school success, and that's not really what AEI is here to do. This paper is focused on school choice programs, and it sets out to void the warrantee on school choice as a policy.

Choice fans, up to and including education secretary Betsy DeVos, have pitched choice in terms of its positive effects on educational achievement. As DeVos claimed, the presence of choice will not even create choice schools that outperform public schools, but the public schools themselves will have their performance elevated. The reality, of course, is that it simply doesn't happen.The research continues to mount that vouchers, choice, charters-- none of them significantly move the needle on school achievement. And "educational achievement" and "school achievement" all really only mean one thing-- test scores.

Choice was going to guarantee higher test scores. They have had years and years to raise test scores. They have failed. If charters and choice were going to usher in an era of test score awesomeness, we'd be there by now. We aren't.

So what's a reformster to do?

Simple. Announce that test scores don't really matter. That's this report.

There are several ways to read this report, depending on your level of cynicism. Take your pick.

Hardly cynical at all. Reformsters have finally realized what education professionals have known all along-- that the BS Tests are a lousy measure of educational achievement. They, like others before them,  may be late to enlightenment, but at least they got there, so let's welcome them and their newly-illuminated light epiphanic light bulbs.

Kind of Cynical. Reformsters are realizing that the BS Tests are hurting the efforts to market choice, and so they are trying to shed the test as a measure of choice success because it clearly isn't working and they need reduce the damage to the choice brand being done.

Supremely Cynical. Reformsters always knew that the BS Test was a sham and a fraud, but it was useful for a while, just as Common Core was in its day. But just as Common Core was jettisoned as a strategic argument when it was no longer useful, the BS Test will now be tossed aside like a used-up Handi Wipe. The goal of free market corporate reformsters has always been to crack open the vast funding egg of public education and make it accessible to free marketeers with their education-flavored business models. Reformsters would have said that choice clears up your complexion and gives you a free pony if they thought it would sell the market based business model of schooling, and they'll continue to say-- or stop saying-- anything as long as it helps break up public ed and makes the pieces available for corporate use.

Bottom line. Having failed to raise BS Test scores, some reformsters would now like to promote the entirely correct idea that BS Tests are terrible measures of school success, and so, hey, let's judge choice programs some other way. I would add, hey, let's judge ALL schools some other way, because BS Testing is the single most toxic legacy of modern ed reform.