Did NYC Tenure Changes Chase Away Weaker Teachers?

At EdWeek, Stephen Sawchuk reports on research which suggests that NYC tenure changes chased away weaker teachers. I suspect that the research is further proof that, when it comes to teacher quality, we have no idea what the hell we're talking about.

The working paper, written by Susanna Loeb of Stanford with Luke C. Miller and James Wyckoff of the University Virginia, looks at what happened in New York City after NYC instituted an internal policy shift regarding the granting of tenure. The district pushed principals to weigh observations, lesson plans, and , in some cases, VAM scores, and provided principals with more information to consider. And the district office called principals in to second-guess their decisions ask for justification if the principal's decision didn't fit the district office's reading of the data.

In what was perhaps the most interesting twist, the district office gave the principals the option of an extension-- a year's postponement of making the final decision about a teacher.

That option was important because it was the popular one. The number of tenure denials did not significantly change, but principals really liked the "Put off till next year" option (creating a corresponding drop in tenures granted.)

Loeb et al came up with the following findings. I'm not sure they found what they think they found.

"Extended teachers" were way likely to move to another school or out of NYC.

Well, duh. Your boss calls you in and says, "I'm not so sure you've got a future here." Do you stick it out and cross your fingers, or do you go find a second opinion from someone who has not announced that he doesn't have all that much faith in your ability?

"Extended teachers" were way more likely to have been rated "less effective" by their principal and/or VAM score.

Again, duh. This is not a surprise. A principal who found a teacher to be effective, but decided not to give that teacher tenure-- that would be news.

Teachers in schools with high concentrations of black or low-performing students were more likely to be "extended," (i.e. found to be "less effective).

I've explained this phenomenon before, and the phenomenon of high-churn tough urban schools is as familiar as the phenomenon of the sun rising in the East.

Sawchuk sums up the conclusion of these unremarkable findings thus:

In sum. "nudging" some teachers out the door this way seems to have improved the overall quality of the teaching force.

I read through the full paper to see if Sawchuk just kind of skipped over the part where the researchers established that each of these nudged-out teachers was replaced with a better one. Was there some data-driven proof based on research findings that compared the Total Quality Level of schools prior to the policy change to the schools afterwards? Was there some proof that the quality of the teaching force was improved?  The long answer involves some spirited number crunching. The short answer is "No."

[Okay, there is one fun wrinkle. They couldn't compare the leavers of 2010-2011 and 2011-2012 to their actual replacements, so they compared them to the hires of 2008-2009 and 2009-2010, which, unless I am missing something, means that the leavers could have been compared in effectiveness to themselves!!]

Every piece of this research rests on the assumption that the teacher evaluations involved are absolutely accurate. But if we set that unproven and baseless assumption aside for a moment, I can reach some conclusions of my own about this research.

Teachers can read the writing on the wall about when to get out of Dodge.

So the guy who is going to decide whether to hire you or not says, "I'm not so sure. In fact, I think
you're not very good. You can stick around for a year and see if I get a better opinion of you, or maybe your students' test scores will go up. Or you could leave to find somebody else who might see more career potential in you."

What do you choose?  It depends. If you think you've really got a shot, you may stay. But if your read on that guy is not good, you cut your losses and look for greener pastures.

This process works regardless of how good the teacher actually is!

With this system, a good principal can "counsel out" plenty of bad teachers. And with this system, a bad principal can drive out plenty of good teachers. Interesting fact not included in research-- number of teachers who left one principal and were then awarded tenure by another.

Sawchuk and Loeb's reading of this research depends completely upon the assumption that the "extended" teachers were not good. But if they just had the misfortune to land in a classroom where student data, poverty, background, etc etc etc guaranteed that any teacher in that room would be rated "ineffective," then this research is really bad news, and it's the worst news for foes of tenure.

Because while you may see this as "proof" that withholding tenure leads to more excellent schools, I see the exact opposite. How do we know-- I mean, really know-- that some of those extended teachers would not have been a great addition to the schools they were chased out of?

I actually like the idea of the extension option-- handled properly, it could have some positive effects. But under current conditions, particularly with teacher evaluations such a VAM-driven mess of random crap, it would be disastrous. Because here's my finding:

In sum, when you play games with job security to the point that teachers don't think they have any, they will leave for some other school system that offers it.

This continued search for the secret of ejecting bad teachers is real cart before the horse stuff. Launching the witch hunt before you know how to identify a witch just creates bad results for everybody. And this paper provides some proof.