Is Teacher Quality Portable?

You may be old enough to remember this awesome idea from the annals of reformsterdom--

Let's use magical VAM scores to evaluate teachers. Then we'll take the super teachers with awesome VAM scores and we'll move them to struggling schools, and they will cause test scores at that school to go up and up and up. 

It was always a dumb idea in so many ways. For one thing, VAM scores are a big pile of baloney that are only slightly more reliable than evaluating teachers by giving horned toads Ouija boards to operate under a full moon. For another, it assumes teacher excellence is portable, that a teacher who does well in one school will be equally awesome in any other school. Give that teacher a different boss, a different school culture, a different type of student, a different surrounding community, and different co-workers and it won't matter a bit.

This is a bit like arguing that the teacher in the classroom with no roof keeps getting wet during the rainy season, so let's get a teacher who is always dry in her classroom and move her to the roofless classroom. Will she stay dry? 

As Matt Barnum reports in Chalkbeat, some research from 2013 said yes. They were looking at the federal program that offered "effective" teachers (aka "teachers of students with good test scores) to move into a low-performing high-poverty school. The federal Ed Department's research wing (back before Dera Leader gutted it) found that test scores went up a bit. But now new research suggests that the 2013 paper missed something.

This new working paper-- "Is Teacher Effectiveness Fully Portable? Evidence from the Random Assignment of Transfer Incentives"-- uses some dense and, honestly, off-putting language, but the results are simple enough-- when you move the dry teacher into the roofless room, she might stay a little bit dry, but not nearly as dry as she used to be. Or as the academics put it--

Using a difference-in-differences framework, we estimate that incentivized-transfer teachers’ value added dropped by 0.12 student standard deviations.

Meaning that the effective teacher did not bring all of her effectiveness with her. Maybe she pulled the class up a bit, but not nearly as much as she was elevating her class in her former classroom. The study looked at 80 high-rated teachers; when they moved, their effectiveness rating fell from the 85th percentile to the 66th. 

Why did this happen?

This decline appears to be driven by lower match quality, negative indirect school effects, and the loss of student-specific human capital.

I told you their language was sometimes off-putting. Lord save students from anyone who refers to them as "student-specific human capital." But the point is sound. Different (and new) teaching colleagues, different students (aka students at a struggling school with fewer resources), and different context in which one might not fit as well-- in other words, any teacher you put in the roofless room gets wet. If they're very good, they may be able to get teaching done while they're getting wet. The old notion that you don't have to repair the roof-- just stick a dry teacher in there-- is and was a terrible theory.

I don't want to pay a lot of attention to a study that relies so heavily on the notion that the Big Standardized Test is a valid and reliable measure of educational quality, and I note that one of the co-authors is from TNTP, creators of the execrable "Widget Effect" paperish thing. But if research like this will convince some folks that teaching is, in fact, a "team sport" and context and specifics do matter and that we can't "fix" struggling schools by porting in Very Special Savior Teachers, then by all means, let's put this research in front of those people.