Is Testing Accountability Dead Yet?

Today Education Next features a three-headed take on the question, "Is test-based accountability dead?" Three prominent reformy thinkers address the question. Do they come up with any useful answers? Let's see.

 







Why Accountability Matters, and Why It Must Evolve

Morgan Polikoff (USC Rossier) leads us off. Polikoff is a long-time Big Standardized Test supporter and logged some time with the Gates effective teaching project. And he is making a very creative case for the BS Test here.

What positively affects student outcomes, has "overwhelming" support of parents and voters, supports various policies and research, and has been used widely for a decade? "School accountability" is his broad and inclusive answer. But when it comes to test-based accountability specifically, I think he's only batting .500 here.

Does BS Testing work? It's a tricky question only because so much of the research is so bad, boiling down mostly to "ever since we started giving the BS Test for high stakes, students have scored higher on the BS Test." That may be true, but so what? That tautological progression holds whether the BS Test is a good test, a bad test, or a test of how well students can recite the Preamble to the Constitution. Update: Matt Barnum suggests that I'm conflating accountability tests with the NAEP, which has been used in some studies to "measure" effectiveness of other reformy ideas. This steers us back towards places like the Honesty gap, which is a side trip we don't have time for, but my point-- that taking lots of standardized tests makes students better at taking standardized tests-- remains the same.

Polikoff does pull off a masterful piece of data-juggling. You may recall a CREDO study suggesting that students in urban charters lost ground compared to their NCLB public school counterparts. Polikoff flips that around and tells us that NCLB caused public school students gains "equivalent to the gain from spending three or four years in an average urban charter." That is some fancy baloney slicing there, showing once again that anything can be made to look good if you compare it to the right Brand X. Update: Okay, Barnum kindly referred me to the study that Polikoff was referring to, and it does say something more like what he argued in his piece. This time, my snark is misplaced.

Polikoff cites Education Next's poll that that folks agree that schools should be accountable for providing a good education. Sure. This conveniently skips past the question that really matters-- are the BS Tests a good measure of whether or not a school is providing a good education.

So, 0 for 2 so far. His final two points are valid, but irrelevant. He asserts that high stakes testing puts weight behind certain policies and generates data for certain studies. Again, no word on whether any of these policies or studies are actually valid. And we've been doing this test-centered education thing for a while. No argument there. Are we tired of winning yet?

What you might find fun about Polikoff's essay is he's not arguing with those of us in the non-reformy camp.

Despite this track record of modest success, many parties seem poised to throw the policy overboard and use the guise of “parental choice” or “local control” to return us to a time when we had little idea which schools were educating children well and which were not.

What fun times we live in, when reformers scold other reformers. Polikoff also alludes to the idea that people who hate the Common Core only because they associate it with President Obama. And then he addresses some real concerns.

He touches on the issue of test-centered education narrowing and shallowing and hollowing out the curriculum, but that is totes going to get better when PARCC and SBA unveil their new! improved! tests. This has been the promise for years, and it has always been an empty one-- standardized tests, particularly if they are to be administered on a grand scale, will always be severely limited. Polikoff also responds to the notion that BS Tests do not predict "life outcomes," but unfortunately his response is to bring up Raj Chetty again, and Chetty has no real answer to this criticism, no matter how many times testocrats trot him out.

Polikoff acknowledges that the "accountability coalition" has frayed, and he restates his belief that choice must travel hand-in-hand with accountability or we are wasting tax dollars. He sees hope in ESSA's call for broader measures of school quality. And he swears we're really making progress and we can't give up now.

Polikoff's problem remains-- the BS Tests are junk that provide junk data and damage schools in the process. Accountability is a good idea, but the standards-based high-stakes tests that we've been subjected to for the past more-than-ten years are junk, and they do not provide a useful, reliable, or valid measure of school quality-- not even sort of. Nor have they helped-- not even incrementally. They have hurt, and hurt badly, a system that is now geared toward test prep and a narrow, stunted version of what education even means. Accountability matters, but Polikoff is asking all the wrong questions, ultimately getting the way of true accountability rather than supporting it.

Futile Accountability Systems Should Be Abandoned

Jay Greene (no relation) speaks up next. And as usual, he is not entering the conversation gently:

Is test-based accountability “on the wane”? The question is based on a fallacy. For something to be on the wane, it has to exist, and test-based accountability has never truly existed in the United States. Holding people accountable requires that they face significant consequences as a result of their actions.

So, the current system fails because it doesn't punish people hard enough. Greene also notes that it "has distorted the operation of schools to the detriment of educational quality." It is a Soviet-style central planning system that cannot possibly "capture the diverse spectrum of local priorities in our nation." It focuses on math and reading and ignores learning to be good citizens. It has crappy metrics. And schools are figuring out that the punishment for "failing" isn't so great (Greene juxtaposes this with stalled NAEP scores, as if a lack of fear among school personnel has caused growth to stop). Greene also, as he has before, notes that there's no proven or apparent connection between scoring well on the BS Test and doing well at life.

Greene also argues that test-based accountability is politically weak:

Rather, accountability that centers on testing is doomed because it has many political adversaries but no enduring political constituency. Parents have never rallied to demand that their children be tested more, that tests be used to retain students or prevent them from graduating, or that tests be used to determine teacher pay or employment. Educators revile test-based accountability even more. Test-based accountability was initiated by policy elites frustrated over rising education costs and subpar results. But elites cannot sustain such a policy in the face of opposition from educators and families. American politics is shaped by the activity of organized interests, not poll results.

Common Core is the canary in the coal mine, the demonstration of how centralized planning with no political backing is doomed to collapse.

All of this leads Greene to conclude that real accountability can only come from families exercising local control and local choice. Interestingly, though, his argument actually undercuts the typical choice-voucher argument. Choice fans argue that rich and middle class parents get to choose a school they like, but Greene talks about the power of those families "to exercise control over how and what their children are taught." In other words, maybe those families don't so much choose their school as the force their school to shape itself to their preferences. That's an idea I'd like to come back to on another day.

If Parents Push for It, Accountability Can Work

Says Kevin Huffman, a guy who has never made it work anywhere despite having control of an entire state's education system (Tennesee). But he will also lead with the idea that reformsters never had a big enough stick. "Shockingly few public school educators" lost salary or jobs or raises or promotions because of BS Test results. That, I would argue, is because most people on the local level recognized that tying those things to student results on bad standardized tests was A) unfair and B) unlikely to accomplish anything useful.

Huffman will go ahead and claim that BS Test results did predict life outcomes, by which he means early tests predict results on later tests. This is unsurprising, since all test results (and much of later life results as well) correlate most closely with socio-economic background. But Huffman wants to tell us a sad tale about an eighth grader facing a "lifetime of truncated opportunities dictated by weak performance at an incredibly young age." Because the nation is filled with people who are now poor because they did badly on the BS Test back in eighth grade.

Huffman is a long-standing member of the Everything Is The Teacher's Fault club. He joined in with Arne Duncan in claiming that students with special needs could be "fixed' by having teachers with high expectations ("Go on, Chris-- your dyslexia won't be a problem if you just listen to the sound of my expectations and try harder.")

Huffman is also going to chime in on the testing ouroborus-- students who take tests get better at taking tests, and therefor test-based accountability works. And as a leading test cultist, he is going to boldy assert as "fact" that higher scores on a narrowly-focused badly-designed standardized math and reading test prove that students are getting a better education. He's upset that the response to "improved results" in Tennessee and DC has been "a deafening silence," but it doesn't occur to him that the results are an unimpressive mirage. He might ask Chris Barbic, the head of Tennessee's Achievement School District who left the post early because he discovered he couldn't get results.

Huffman poses a question:

If test-based accountability works to improve student results but is unpopular with people who make their living in schools, can we reasonably expect it to find a foothold?

This is the wrong question. Let me suggest a rewrite:

If test-based accountability shows no independently verifiable improvement for students and is largely criticized by the trained professionals and experts in the education field, is there any reason to hold onto it?

 But Huffman is an education amateur who holds tight to what he doesn't know.

We know beyond a shadow of a doubt that some schools, districts, and states are doing better work than their peers. Some are getting better results, and some are driving faster improvements. How do we know this? Because of tests.

Man, I disagree with some of what Greene says and most of what Polikoff says, but this is just dumb. Does Huffman really, truly believe that nobody can tell the difference between a good school and a bad school except by looking at BS Test scores? Oh, and we can use test results to improve schools, somehow, except that of course we've been claiming to do that for over a decade.I guess we didn't threaten and punish teachers enough.

 Huffman is Arne Duncan lite, right down to insisting that test opponents are those rich, white suburbanites. Well, Duncan had seven years to make his pitch, and he failed to demonstrate any significant successes ever. Huffman is banging a drum that stopped making any semblance of music ages ago.  

So, Is It Dead ?

Oh, if only. But BS Testing is still enshrined in ESSA and in many state systems. It will continue to be a toxic drag on the school system, providing no useful information and warping the very idea of education.

So it won't go away, and it can't do anything useful or nurturing, and in fact can be destructive and damaging at times. Let's call test-centered accountability one more zombie loose in the education world.

Do I have some ideas about how to do better? I do, but this has dragged on enough for today.