|Ding dong, indeed.|
EVAAS is the VAM of choice in Houston. This is the system developed by William Sanders, an agricultural statistician who thought that a statistical model for modeling genetic and reproductive trends among cattle could be used to figure out how much value teachers were adding to students. The result was a system that nobody could really explain to anybody, but which spread like kudzu across the educational landscape because science! numbers! The explanation of the secret VAM sauce looks like this:
But I prefer this one, which is more accurate:
Experts came to testify, and laid out twelve major findings about the VAAS system:
1) Large-scale standardized tests have never been validated for this use.
2) When tested against another VAM system, EVAAS produced wildly different results.
3) EVAAS scores are highly volatile from one year to the next.
4) EVAAS overstates the precision of teachers' estimated impacts on growth
5) Teachers of English Language Learners (ELLs) and “highly mobile” students are substantially less likely to demonstrate added value
6) The number of students each teacher teaches (i.e., class size) also biases teachers’ value-added scores.
7) Ceiling effects are certainly an issue.
8) There are major validity issues with “artificial conflation.” (This is the phenomenon in which administrators feel forced to make their observation scores "align" with VAAS scores.)
9) Teaching-to-the-test is of perpetual concern.
10) HISD is not adequately monitoring the EVAAS system. HISD was not even allowed to see or test the secret VAM sauce.
11) EVAAS lacks transparency.
12) Related, teachers lack opportunities to verify their own scores.
US Magistrate Judge Stephen Smith agreed, saying that "high stakes employment decisions based on secret algorithms (are)incompatible with... due process" and the proper remedy was to overturn the policy. The Houston Federation of Teachers was pleased:
HFT President Zeph Capo: “With this decision, Houston should wipe clean the record of every teacher who was negatively evaluated. From here on, teacher evaluation systems should be developed with educators to ensure that they are fair, transparent and help inform instruction, not be used as a punitive tool.”
What happens next? Well, personally, I hope my union here in Pennsylvania, where we use PVAAS (which is EVAAS with less E and more P), will call up that bunch of experts and march them into some state court to repeat the Houston performance.
Do teachers and schools need some form of accountability to parents, students, and the taxpayers who foot the bill? Absolutely. But that form of accountability needs to be real, and not some high-tone version of bouncing dice of a horny toad's back under a full moon. Let's get rid of this bogus (but highly profitable) tool and replace it with something useful. VAM is a big fat fake; not only does the emperor have no clothes, but he's not even the emperor.
Update: More details available here.