Teacher Evaluation: Plus or Minus?

Matthew Kraft is an Assistant Professor of Education and Economics at Brown University, and it says something about where we are today that there is even such a job. I look forward to universities hiring someone as a Professor of Microbiology and Sociology, or a Professor of Astrophysics and Cheeseburgers. The notion that economists are automatically qualified to talk about education continues to be one of the minor plagues us these days.

But I digress.

Kraft is in Education Next making some big, sweeping statements about his research:

When I present my research on teacher evaluation reforms, I’m often asked whether, at the end of the day, these reforms were a good or bad thing. This is a fair question—and one that is especially important to grapple with given that state policymakers are currently deciding on whether to refine or reject these systems under ESSA. For all the nuanced research and mixed findings that concern teacher evaluation reforms and how teachers’ unions have shaped these reforms on the ground, what is the end result of the considerable time, money, and effort we have invested?

I think I could skip right ahead to a conclusion here, but Kraft has created a nice little pro and con list that both helps us address the question and gives a quick and dirty picture of what reformsters think they have done in the teacher evaluation world. So let's see what he's got.

The Pro List

So here's what Kraft thinks are the "positive consequences" of ew evaluation systems.

Growing national recognition of the importance of teacher quality.

Is that a consequence of new evaluation systems? Were that many people wandering about before saying, "You know, I don't think it matters at all whether school teachers are awesome or if they suck."  Granted, we did have one group dedicated to the notion that we should have a system in which it doesn't matter which teacher you get, that every class should be "teacher-proofed," and that if we do all that well enough, we can park any warm body in a classroom. But those were the reformsters, and I'm not sure they've gotten any wiser on this point.

A slight shift toward the belief that some teachers are better or worse than others.

Again, I'm not sure this is news to ordinary civilians, but lord knows that reformsters have been complaining loudly and constantly that schools are loaded with Terrible Teachers who must be weeded out. How is this a positive, exactly?

The widespread adoption of rigorous observational rubrics for evaluating instructional practice that provide clear standards and a common language for discussing high-quality instruction.

Nope. This is not a positive. Reducing the evaluation of teacher quality to a "rigorous rubric" is not a positive. Academians and economists like it because it lets them pretend that they are evaluating teachers via cold, hard numbers, but you can no more reduce teaching to a "rigorous rubric" than you can come up with a rubric for marital success or parental effectiveness.

For that matter, not only is this not a positive, but there's not much evidence that it has actually happened in any meaningful new way. We've seen lots of teacher eval rubrics (aka checklists) before (get out your Madeline Hunter worksheet, boys and girls) but they never last, because they turn out to be bunk. But at the moment, rubrics and checklists still take a back seat in most districts to Big Standardized Test scores soaked in some kind of VAM sauce. So this item from the pro list is wrong twice.

New administrative data from student information systems that, linked to teacher human resource systems, allow administrators and researchers to answer a range of important questions about teacher effectiveness.

Holy jargonized bovine fecal matter! This is also not a positive. But it's a sign of where this list is headed.

More and better (albeit still imperfect) teacher performance metrics to inform important human capital decisions made by administrators.

See? By the time you're talking about "human capital decisions," you've lost the right to be taken seriously by people who actually work in education. Plus, this is just purple prose dressing up the old reformster idea that we should be using teacher evaluation data to decide who to hire and fire, which is old sauce and not a positive because Kraft has mistyped "albeit still imperfect" when what he surely means is "still completely invalid and unsupported."

Increased attention to the inequitable access to highly effective teachers across racial and socio-economic lines.

First, the "data" on relative awesomeness of teacher at poor schools is almost impossible to take seriously because A) it's based on crappy BS Testing data and B) comparing teachers at wealthy and poor schools is like comparing the speeds of people running down a mountain to the speeds of people running up one.

Second, we don't need a lot of hard data to know that non-wealthy, non-white schools get less support or that state and district funding systems inevitably short change those schools. If you can only believe that because you see numbers on a spread sheet-- I mean, I guess it's swell that you've finally figured it out, but damn, what is wrong with you?

Increased turnover among less effective teachers.

You have no idea whether that is happening or not, because you have no way of knowing which teachers are doing well and which ones are doing poorly. The mere fact that you assume awesomeness or non-awesomeness is a permanent state for every teacher shows that you don't understand the issues involved here.

That's the pro list. And it's pretty much all bunk.The Con List

What downside has there been to the evaluation revolution?

The loss of principal's time to a formal evaluation process and paperwork that (often) have little value.

That is correct. New evaluation systems have created a host of hoops for administrators to jump through, most of which serve no local purpose, but are simply there to satisfy a state bureaucracy's need to see numbers on forms.

The erosion of trust between teachers and administrators. That trust would be useful for real ongoing professional development.

That is correct. Because so many modern reformster evaluation systems were designed with the idea of weeding out all the Terrible Teachers, and because those evaluation systems are often based on random data that the teacher's job performance doesn't actually affect (looking at you, BS Test scores), teachers view the whole process with distrust. One of the most powerful things an administrator can say to a teacher is, "How I can I help you do the kind of job you want to do?" These evaluation systems stand directly in the path of any such interaction.

An increased focus on individual performance at the potential cost of collective efforts.

I'm giving Kraft a bonus point for this one, because too many reformsters refuse to acknowledge that their evaluation systems set up a kind of teacher thunderdome, a system in which I can't collaborate with a colleague because I might just collaborate myself out of a raise or a job. Because a school doesn't make a profit, all teacher merit pay systems must be zero sum, which means in order for you to win, I must lose. This does not build collegiality in a building.

Decreased interest among would-be teachers for entering the profession.

There are certainly many factors at play here, but Kraft is right-- knowing that your job performance will be decided by a capricious, random and fundamentally unfair system certainly doesn't make the profession more attractive.

The costs associated with teacher turnover, particularly in a hard-to-staff schools.

This is correct. Once teachers have been driven out or fired, schools cannot just go grab new teachers of the Awesome Teacher Tree in the back yard. Costs associated with turnover include a lack of stability and continuity at the school, which is not helpful for the students who attend.

I'd add that Kraft has missed a few, but most notably, the waste of time, money, and psychological energy on a system that doesn't provide useful or accurate information, but which presents teachers with, at a minimum, an attack on their own sense of themselves as professionals and, at a maximum, an attack on their actual earning power, or even career. When it comes to teacher evaluation, we are spending a lot of money on junk.

Looking at the balance sheer.

From my perspective, teacher evaluation reforms net a modest positive effect nationally. While my judgment is informed by a growing body of scholarship, it is also subjective, imprecise, and colored by my hope that the negative consequences can be addressed productively going forward.

As I often tell one of my uber-conservative friends, "We see different things here." Since I find none of the positives convincing or compelling, but all of the negatives strike me as accurate, if understated, I see the balance as overwhelmingly negative.

Kraft does ask some good questions in his concluding section. For instance, would schools, teachers and students be better off if states had not "implemented evaluation reforms at all"? It's a useful question because it reminds us that the current sucky system replaced previous sucky systems. But the critical difference is this:

Previous sucky evaluation systems may not have provided useful information about teachers (or depended on being used by good principals to generate good data). But at least those previous systems did not incentivize bad behavior. Modern reform evaluation systems add powerful motivation for schools to center themselves not on teachers or students or even standards, but on test results. And test-centered schools run upside down-- instead of meeting the students' needs, the test-centered school sees the students as adversaries who must be cajoled, coached, trained and even forced to cough up the scores that the school needs. The Madeline Hunter checklist may have been bunk, but at least it didn't encourage me to conduct regular malpractice in my classroom.

So yes-- everyone would be better off if the last round of evaluation "reforms" had never happened.

Kraft also asks if "the rushed and contentious rollout of teacher evaluation reforms poison the well for getting evaluation right."

Hmmm. First, I'll challenge his assumption that rushed rollout is the problem. This is the old "Program X would have been great if it had been implemented properly," but it's almost never the implementation, stupid. There's no good way to implement a bad program. Bad is bad, whether it's rushed or not.

Second, that particular well has never been a source of sparkling pure water, but yes, the current system made things worse. The problems could be reversed. The solution here is the same as the solution to many reform-created education problems-- scrap test-centered schooling. Scrap the BS Test. Scrap the use of a BS Test to evaluate schools or teachers or students. Strip the BS Test of all significant consequences; make it a no-stakes test. That would remove a huge source of poison from the education well.