Pacts Americana: Bellwether Looks at Fed-State Relationship

Bellwether Education Partners is a reformy thinky tank that often stands in the shadows of the Fordham Institute. If I were to characterize the differences (beyond size-- Bellwether is a more modestly sized operation) it would be that in the great thinky tank balancing act between thinking and trying to sell something, Bellwether tilts more toward thinking and Fordham tilts toward selling. Bellwether is back in the office reading Carfax reports and meditating on the nature of transportation while Fordham is out on the lot working the hard sell.

But this morning Bellwther hit the ground running with a new report, "Pacts Americana: Balancing National Interest, State Autonomy and Educational Accountability."  The paper, by Chad Aldeman, Kelly Robson and Andy Smarick, raises some points worth talking about even though (spoiler alert) I think they get several things very wrong.

It's a thirty page document, but I've read it so that you don't have to. As always, you can thank me later.

Introduction

The paper starts with a recap of accountability over the past thirteen years. NCLB gave us a national reporting-accounting system, but congressional disagreement-lock opened the door to waivers, a "de-facto ESEA reauthorization," which opened the door to multiple accountability measures. The writers' takeaway from all of this is to view all of these various structures as deals between the state and federal government, and that's going to be the lens through which we consider accountability in this paper. I can go along with that for the moment.

Evolution of Federal Accountability

ESEA started out as a way to fund inputs, with little or no accountability measures attached. Under Nixon, the National Institute of Education was launched to see if the feds were getting any bang for their bucks; this gave us the birth of the NAEP tests. By the seventies, more folks wanted to know if the giant mountain of money was doing anything useful, egged on by the 1977 SAT investigation into why SAT scores were falling. The correct answer was "because more students of lower ability are taking it," but folks decided to push accountability anyway, getting us our first "minimums." By the early eighties, the feds were handing out block grants, and we got the premier chicken littling of A Nation at Risk.

NAR kicked off a new accountability era, with the search on for a good test-driven model. From there it's a hop-skip-jump to Clinton-era Goals 2000 and the Improving America's Schools Act, which was kind of like NCLB but without actual teeth-- states were to implement test-based accountability and measure AYP, and if they came up short the feds gave them a Stern Look.

In this context, NCLB looks like the feds simply saying, "Yeah, we say keep doing that stuff, but now we're serious." But it turned out that NCLB's tendency to be inflexible, brutish and stupid did not endear it to anybody. Cue backlash, including a desire to get the feds to back off, and our current mess.

That's the Bellwether History O'Accountability, and it's not a bad one. Let's move on and look at the lessons of the NCLB era.

Lesson 1: Test-based accountability has produced positive academic outcomes.

Man, I thought I'd get further into this before I had to flat-out disagree, but here we are already. There's a lot of research cited here; I'm not going to address it directly or in detail, because I think by looking at this stuff in detail obscures the simple truth of it, and it's in looking at the simple truth that we see why this is just dead wrong.

Here's what the research shows:

By focusing on getting higher standardized test scores, we are able to get higher standardized test scores.

The "positive academic outcomes" mentioned in the heading are really only one single outcome-- higher standardized test scores. That's it. That's all. There are no measures here of future life success of students, ability to better function in the real world, better college achievement level, better professional success, improved quality of life, deeper critical understanding of the material- nor are there any academic outcomes that aren't math and reading related.

So if you think the purpose of school is to get students to produce higher standardized test scores, then yes, this lesson is a True Thing. But if you think that's a narrow, cramped, tiny, inadequate, probably inaccurate measure of student achievement, then lesson one is no lesson at all.

Lesson 2: States vary in their implementation and success.

Again, we're talking about test scores. And since nobody knows how to convert a raw test score into a reliable proxy for educational achievement (we check our work against... what?), every state comes up with its own method-- and since the results were linked to high stakes and big money, states predictably infected the methods with means of blunting the impact of low numbers. These infections ranged from Mildly Creative to Baldfaced Baloney.

Oddly, the writers omit the biggest source of such state-level shenanigans--- under NCLB, every single school in the country was destined to fail. By 2014, given the 100% above average requirement of the law, every single school in America was either going to be failing or cheating. And so as those dates loomed closer and Congress displayed its inability to do anything about it, states and districts did whatever they could do to postpone their inevitable failure. It's the rule demonstrated with fatal consequences in Mao's China-- impossible goals plus strong punishment for failure equals lying.

The authors also do not spend any time talking about variation within the states. If standards and reporting systems really do bring everyone into line and foster achievement, shouldn't we see that within the states, at least, the NCLB era decreased the spread of achievement within states? Did that happen? I think not.

Lesson 3: State flexibility is essential

One final, critical lesson from NCLB is that state flexibility is essential. States differ widely due to their unique histories, demographics, traditions, politics, and more. The federal government should not—cannot—implement a one-size-fits-all model across such widely varying contexts.

That sounds about right. The authors cite the tradition of local control and the differences between states. They also note that top-down programs imposed from on high do not engender enthusiastic and fruitful implementation by the people on the ground. And finally, the idea of a one-size-fits-all model is just unicorn farming. There is no such thing, and any attempt to create and  implement such a thing will result in failure.

In fact, these guys say that the Obama waivers were the right move-- and may not have gone far enough in terms of returning control to the states.

Funny piece of trivia

I just noticed-- and did a document search to confirm-- that the phrase 'common core" does not appear anywhere within this paper.

The New Idea: Compacts

There follows a few pages of chartage, laying out the differences between NCLB, Waivers, and Compacts. Most of what the chart says is covered in the following, but if you're intrigued by any of this, I suggest giving the charts a look.

Principle 1: States must have flexibility to tailor their education policies to their unique local contexts.

Even as I agree with this principle, I know what the problem with it can be. The "unique local context" of some states, for instance, is "we don't want to spend more than $1.98 on education" or "we prefer not to fund schools for poor/black/brown students."  And Bellwether's idea of a range of approaches has a familiar reformy ring to it:

Some have embraced non-district chartering; others are adopting private school choice programs and others still have created statewide “extraordinary authority” entities like Louisiana’s Recovery School District.

Compacts would come with some structure and requirements. States should show they have college and career readiness standards in place (sigh). And they would need means of identifying and "addressing" pockets of incorrigible suckage. They will need a plan of attack, but the plan must be all theirs.

The writers recognize that such flexibility takes us back to Lesson 2. They suggest three ways to head this off by requiring approval from either 1) the secretary of ed, 2) a peer review group or 3) a panel of experts. These all have their own sets of drawbacks, all underlining the futility of such a system.

Here, in short form, is your problem. If your system rests on the idea that somewhere, somehow, some place the system rest on an  absolute immutable objectively verifiable vision of exactly what a school system must be and do, you are doomed because your journey, no matter how complex and far-ranging, still ends at a unicorn farm. Yes, I do have some ideas about what you do instead, but rather than add another thousand words to this piece, I'll just say that step one is accepting that you can't have perfectly objective and absolute standards of accountability any more than you can come up with such standards for kisses or marriage, because ultimately education is a web of relationships, and all relationships are primarily shaped by the people involved. I have more, but let's not wander too far off track.

Principle 2: State accountability should focus on outcomes, not inputs.

Again, I get this. A teacher whose classroom approach is, "Hey, I cover the material, and they either get it or they don't. Not my problem. I put it out there." is not a good teacher.

But there remains a huge huge huge HUGE problem with the focus on outputs school-- we still don't have any decent way to measure the outputs that matter.

But there would need to be guidance for the creation of goals. For example, goals should include clear measures of educational achievement, in particular for low-income and historically underserved populations of students. States might be required to create goals related to graduation rates but states could also develop other goals. For example, states could decide to measure the development of noncognitive skills, the percentage of fourth graders reporting a challenging and supportive school environment or the percent of high school students taking AP or IB classes.

"Educational achievement" still just means "test scores on limited one-time standardized tests," and if you ask a hundred parents what they want out of their child's school, "Get him to score well on standardized tests" is not going to be high on the list. I will admit that it is creeping up the list, primarily because reformsters have successfully hammered away at the idea that such tests are a measure of educational quality. I'm unimpressed. A well-repeated lie is still a lie.

We don't know how to provide a simple, clear objective measure of how good a school is. Like pornography or the woman who steals our heart, we know it when we see it, but we can't lay out a set of clear, objective measures of it. That's bad enough, but when we try to fake it, we end up screwing up the system and providing more examples of Campbell's Law in action.

I decide I want to measure meal quality in restaurants. I can't really measure easily a meal's goodness, so I notice that being visually appealing with a nice mix of colors and textures usually goes with it, so I measure those. If I offer rewards just to people who score high and punishment to those who don't, pretty soon I've got a world of chefs who are choosing food-ish materials based on how they look and not how they taste, and my meals taste lousy.

Bad, inexact, incorrect, incomplete measurements warp the processes that they measure. That's where we are right now. We aren't measuring the outcomes that matter.

Principle 3: Federal accountability should focus on continuous improvement.

States should have to re-up their compact with the feds, and part of that process should be showing how they're going to do better. Well, yes. Everyone who actually works in education gets that continuous improvement is part of the gig. I've often said that any teacher worth her salt can immediately list for you the areas where she is weak and is working to improve.

That, actually, was one of the immediate signs that the Common Core are crap-- there is absolutely no mechanism in place for revising, improving and upgrading them. Nothing good in education is like that. Nothing that is high quality in education stands still.

This principle does underline the problem with compacts-- they keep the feds in the driver's seat. We can institute everything that the paper talks about and still end up with a federal government that says, "No, if you don't check off the following items the way we want them, no compact for you." Which would put us right back where we are with waivers.

Negotiating the Compact

In fact, one of my big points of curiosity-- how do these things get settled? How would my state of Pennsylvania, which is incapable of settling the budget-connected policy issues even remotely on time negotiate a compact with the federal government, which is now eight years late rewriting ESEA?  If they can't agree on the terms of a compact, what happens? If the answer is, as I suspect, the feds just put their own default in place, then what pressure do they feel to negotiate when they will "win" in the end anyway?

Just wondering how all this will work.

Bottom line

Ignoring for a moment that Bellwether's measure of excellence defaults include "does it help support more charters and choice?" the biggest issue remains that this whole system depends on an objective, reliable, accurate measure of how well schools are doing, and that means that the system might as well depend on a conference of yetis meeting on the head of a pin while dancing angels cater lunch with hippogryph meat. That is a huge problem, and the paper doesn't address it at all.

On the other hand, the question of how to balance state freedom, local control, federal oversight, and some kind of accountability-- that's a good question. A much better question than "how many standardized tests should we have" or "how can we scrap tenure" or "what are some good ways to take over public schools and give them to private operators."

This question at least addresses some of the fundamental issues lurking behind many of the surface skirmishes in education. While I disagree with Bellwether hugely on many of the answers and most of the solutions, I give them props for asking a good question. And the title's cute.