Can Evidence Improve America's Schools?

That's the question asked by Mike Petrilli (Fordham Institute) asks over at the Fordham blog.

The piece harkens back to Chester Finn's 1986 book titled, unironically, What Works. And the piece also ties into Petrilli's ongoing series about how to drive change in education. After noting that nothing from Finn's work has ever really gotten traction, Petrilli bemoans our benighted edu-state.

Education remains a field in which habit, intuition, and incumbency continue to play at least as large a role as research and data analysis.

Petrilli has, as is his wont, stacked the deck from the first framing of the issue. He could as easily have written that education is a field in which experienced and trained professionals continue to rely on their own judgment rather than the kibbitzing of non-professionals outside the field. But absent the framing, I'll allow him this-- not a lot of outside research makes much of an impression on how schools function.

So the first question is why not? And Petrilli has a list of answers from Vivian Tseng (W.T.Grant Foundation), Tom Kane (Harvard Center for Reformy Education Policy Research) and Michael Barber (Overlord of Pearson's World-Conquering Data-Hungry Corporate Monstrosity).  The list is interesting, albeit not exactly on point.

Limited supply. Maybe there's more research than there used to be, but there are still many educational questions for which there are no research-based answers.

Too much junk. "Education is awash in a deluge of reports, journal articles, emails, tweets, and news stories" that all make claims to research-based pronouncements. And yet, much of it is junk.These two supply issues already raise some points for debate. Petrilli says that v"rigorous studies have made a big impact on teacher evaluation (for better or worse)" and I'd be inclined to argue just how rigorous the studies were that gave us VAM. There's also a certain irony in this observation coming from a guy at Harvard Ed Policy, arguably an advocacy group masquerading as a think tank masquerading as a university department. That kind of multi-mask mummery has given us an unending supply of marketing baloney masquerading as research.

Add to this the growing recognition of a problem in the sciences in general and education research in particular-- the paucity of research replication. Beloved long-held research findings are dropping like flies, to the point that some science is being called broken.  Bottom line-- there may be even more junk than we realize.

Poor Dissemination. School administrators don't check the What Works Clearing House very often. Research outlets do a bad job of pushing out their works of blinding genius.

Ideology. Damn ed college professors. "They’re fundamentally opposed to the reform agenda, measuring schools via student outcomes, and hard-nosed quantitative analyses." Petrelli's tongue is in his cheek here, but the list does touch on something important-- part of the reformy stance has been not so much "here's a technique for better education" as it has been "your entire map of the world is wrong." Most of us remain unconvinced.

Habits and Practices in schools and districts. So maybe we're just not open to new research. Maybe we're tired of the "reform of the month." Maybe (and this is me talking now) reformsters sound like that guy in the old joke-- "Are you going to believe me, or your own eyes?"

Because what's going on daily in schools is not just habits and practices. It's research and data collection. I start a new lesson. I scan the faces of my students, looking for signs of reactions, in particular reactions that signal understanding or confusion, engagement or cutting out. I ask some questions. I gauge the answers. I hand out a practice sheet. I watch  them work at it. We correct it, and I collect to check for particular patterns of understanding or confusion. That's ten minutes of my class, and I've already collected a ton of data and tested a mountain of techniques. I do that all day, every day. When some guy shows up to say, "I've never done your job, and I've never watched you do your job, and I don't know your kids, or your school or your community or you, but I would like to tell you how you should totally change what you do based on one flawed piece of research with ten bits of data, and by the way, I'm hoping to make a ton of money by selling this to you--" Well, that's just not a pitch I'm buying. I'm an education researcher every day of my professional life.


So the list of reasons is a little incomplete. What does Petrilli propose as possible solutions?

Book it. Basically, publish your research in book form so that people who read books will find it. "Develop evidence-based book that might have an impact." And I was almost nodding along with this one, but then he tossed out Doug Lemov's odious Teach Like a Champion as an example, which is kind of like using Art of the Deal as an example of a way to teach economics.

Get together. Again, he's onto something in noting that most teachers get techniques and ideas from people that they personally know and trust. But from there he jumps to the notion that folks with ed ideas to plug should "do a better job partnering" with groups that already exist, and that's not false, but the problem there is also not knew-- it is hard to "partner" with people when your idea of "partner" is "I'll talk and you'll listen. I'll sell and you'll buy. You'll scratch my back, and I'll help you scratch my back." He also likes the ideas of maybe forming new networks which-- come on. Reformsters have been trying to form new grass roots networks from the top down for the past several decades, from the Center for Education Reform to StudentsFirst. Surely we already have enough astroturf.

Go small. Again, not an entirely bad idea. Petrelli suggests that instead of broad global big scale studies, why not small-scale studies that give us information about very specific, very focused situations. Which-- let's be honest-- is what many education researchers have been producing without admitting it. After all, how much education research is there that applies only to small groups of sophomore from one specific university. A great deal of research does only apply to small, narrow situations-- researchers just haven't been willing to admit that they haven't found the solution for everyone everywhere.

Look, as I said above, evidence improves America's school every single day. Every half-decent classroom runs on evidence that the teacher-researcher gathers and processes constantly, often with several replications of the study in a single day. We try new things and we immediately find out how well they work. We share what our research has uncovered (at least, we do in schools where our job security and don't depend on "beating" our colleagues) and we pick up ideas from our larger web of connections (our Personal Learning Networks, as the current jargon puts it).

The problem with much of what Petrilli and Finn before him call "evidence-based" is that the evidence sucks. We are stuck right now in a period when a bunch of influential people actually imagine that the scores from a narrow, badly-constructed test taken by disinterested students is somehow evidence of anything at all. I can't say this enough times. The scores from the Big Standardized Tests are not evidence of a single damned thing.

Meanwhile, the evidence I collect in my classroom is immediate, actionable, and directly impacts instruction and my students' achievement. The kind of research and evidence and growth and development that teachers do every day is the lifeblood of pretty much every school where the process hasn't been choked to death by some lousy administrator or a dopey CMO that that already thinks it knows everything and need not listen to teachers. The real question Petrilli is asking is, "Can we tap into that, inject our own stuff into the bloodstream, and somehow get the process to move in the direction we want it to?" But if we're going to inject it into the heart and soul of the school, it had better be something other than lumpy, clotted junk.