School Accountability Camps

Now that ESSA has opened the door (maybe, kind of) to new approaches to school accountability. What are the possibilities?

Mike Petrilli of the Fordham Institute has outlinbed four possibilities in a list calculated to help us all conclude that only one of the possibilities is really legit. The list apparently grows out of the Fordham contest to design a compatibility system; I entered that competition but was not a finalist, though I swear, I'm not bitter. Someday some rich benefactor will give me a stack of money, and I'll start my own thinky tank, and then I'll have ed system design competitions all the time. so there.

Anyway. let's look at the four camps that Petrilli sees. He labels them by their supposed slogans.

Every School is A-OK!

Petrilli says that the proponents of this view are the teachers unions and "other educator groups," and it does not bode well for his list that he opens with a Gigantic Person of Straw. I read a lot of writing by a lot of people, and I'll be damned if I can think of a single person who says that every school is a-okay. So we can skip past this camp because there is not a single tent pitched there.

Attack the Algorithms

Petrilli says this camp is anti-test, but pro-accountability. "It seeks a system that uses as much human judgment as possible and captures a full, vivid, multifaceted picture of school quality." These tend to base their system on school inspection. It's more common in Europe than the US, which could be in part because the last decade or two of US law has specifically centered on bone-headed high-stakes test-based formulae that do their best to root out any trace of human judgment (except, of course, the human judgment that produced the tests, the scoring of the tests, and the algorithms for crunching the test results). I won't lie. This is my camp.

Living in the Scholars’ Paradise

Yeah, Petrilli doesn't have his poker face on, either.

This approach uses sophisticated, rigorous models to evaluate schools’ impact on student achievement, making sure not to conflate factors (like student demographics or prior achievement) that are outside of schools’ control. 

He's wrong, and he's wrong because he is focused on the single output of student test scores. He's willing (at last) to admit that there are factors that influence the test scores that are outside the school's control, but he's still just counting test scores, and that's an unacceptably low bar for what measuring what schools do. Actually, it's not a bar at all. It's like judging pole vaulters by checking their foot placement at the point of lift-off and ignoring everything else. And, to flog the simile further, under our current testing regimen, it's like judging that foot placement by comparing it to an idea about foot placement cooked up by someone whose only expertise is that several years ago they watched a couple of pole vaulters on television.

Petrilli also wants to include, in fact give most attention to, value-added measures, and at this stage of the game, backing VAM as a good measure of school quality makes even less sense than arguing that Donald Trump should be President because of his great statesmanlike qualities. VAM is simply indefensible as a measure of school or teacher quality. Petrilli has displayed flexibility in the past; why he remains devoted to this slice of junk science is a mystery to me.

NCLB Extended, Not Ended

NCLB is gone but not forgotten. Or maybe it’s not exactly gone, in the mind of folks who yearn for Uncle Sam to mandate accountability models that obsess about achievement gaps and give failing grades to any school with low proficiency rates for any subgroups.

As always, I tip my hat to Petrilli's ability to craft a sentence to stay polite and yet still make clear that he thinks someone is full of shit. Game recognizes game.

Anyway, his portrayal of this camp is interesting in the context of the ongoing reformster battle over whose reform it is, exactly, anyway. Here he says that those lefty civil rights types just want to measure raw achievement scores so as to highlight schools that are failing non-wealthy students of color. As with category #1, I'm not sure these are people who actually exist.

Tents He Forgot To Pitch

One of the critical questions that is repeatedly overlooked in accountability discussions is:

Accountable to whom?

In other contexts, reformsters will talk about being accountable to parents, that in their perfect world parents get all sorts of information that allows them to select the most awesomest school available. Yet none of Petrilli's camps, least of all his preferred one, spends any time at all asking parents what they want the schools to be held accountable for. And everything we know about parents (and personally, I'd like to consult all non-parental taxpayers as well, because, public schools) tells us that "high scores on standardized tests" does not top anybody's list. And look-- reformsters already know that, which is why they insist on talking about "student achievement" when they mean "test scores." They know that if we just told folks about "test scores" they would be unimpressed and unmoved, so we call it "student achievement," which brings to mind a whole wide spectrum of accomplishments and skills-- even though test scores is all that's really on the plate.

And of course, since we're talking about ESSA, what we're really talking about is accountability to the feds and (depending on how far rogue the USED is willing to go) the states. So while talk about accountability to parents sounds nice, that's not really what we're trying to do anyway. We can say that the bureaucrats at the capital are fine proxies and watchdogs for parent and taxpayer interests, but let's get real-- the suits at the capital have a whole list of political concerns (which are in turn tangled up with all sorts of lobbying and money concerns) that have nothing to do with what Mom and Dad are concerned about when they send Chris and Pat off to school.

Bottom line-- the school inspection, human-based accountability gets you the kind of information that parents and taxpayers (who are, by and large, human beings) want and find useful. The scholar's paradise is really a bureaucrat and policy wonk's paradise that gives primacy to the interests, concerns and policy goals of the wonks and bureaucrats over the interests and concerns of parents and taxpayers.

Ask a parent (or taxpayer)-- Do you want to know everything you'd like to know about your local school, or would you like to be confident that the bureaucrats at the department of education know everything they want to know? I have a thought about what the answer would be.

And underlining all of this? Congress and the USED still have to finishing arguing about what the department is supposed to (or allowed to) hold accountable for. Can't wait to see where that leaves us.