 |
| Ding dong, indeed. |
EVAAS is the VAM of choice in Houston. This is the system developed by William Sanders, an agricultural statistician who thought that a statistical model for modeling genetic and reproductive trends among cattle could be used to figure out how much value teachers were adding to students. The result was a system that nobody could really explain to anybody, but which spread like kudzu across the educational landscape because science! numbers! The explanation of the secret VAM sauce looks like this:
But I prefer this one, which is more accurate:
Experts came to testify, and laid out twelve major findings about the VAAS system:
1) Large-scale standardized tests have never been validated for this use.
2) When tested against another VAM system, EVAAS produced wildly different results.
3) EVAAS scores are highly volatile from one year to the next.
4) EVAAS overstates the precision of teachers' estimated impacts on growth
5) Teachers of English Language Learners (ELLs) and “highly mobile” students are substantially less likely to demonstrate added value
6) The number of students each teacher teaches (i.e., class size) also biases teachers’ value-added scores.
7) Ceiling effects are certainly an issue.
8) There are major validity issues with “artificial conflation.” (This is the phenomenon in which administrators feel forced to make their observation scores "align" with VAAS scores.)
9) Teaching-to-the-test is of perpetual concern.
10) HISD is not adequately monitoring the EVAAS system. HISD was not even allowed to see or test the secret VAM sauce.
11) EVAAS lacks transparency.
12) Related, teachers lack opportunities to verify their own scores.
US Magistrate Judge Stephen Smith agreed, saying that "high stakes employment decisions based on secret algorithms (are)incompatible with... due process" and the proper remedy was to overturn the policy. The Houston Federation of Teachers was pleased:
HFT President Zeph Capo: “With this decision, Houston should wipe clean the record of every teacher who was negatively evaluated. From here on, teacher evaluation systems should be developed with educators to ensure that they are fair, transparent and help inform instruction, not be used as a punitive tool.”
What happens next? Well, personally, I hope my union here in Pennsylvania, where we use PVAAS (which is EVAAS with less E and more P), will call up that bunch of experts and march them into some state court to repeat the Houston performance.
Do teachers and schools need some form of accountability to parents, students, and the taxpayers who foot the bill? Absolutely. But that form of accountability needs to be real, and not some high-tone version of bouncing dice of a horny toad's back under a full moon. Let's get rid of this bogus (but highly profitable) tool and replace it with something useful. VAM is a big fat fake; not only does the emperor have no clothes, but he's not even the emperor.
Update: More details available here.
I continue to be amazed at how little play "student accountability" or "parent accountability" gets.
ReplyDeleteThe know-nothing reformers have treated student learning as if good teaching alone can make it happen for everyone.
But VAM/EVAAS/PVAAS all claim to control for student/family factors and for years the powers that be have bought that argument.
DeleteThe argument that needed to be made - and which HFT successfully made, thank heavens - is that VAM/EVAAS/PVAAS *don't* actually control for those factors. The next step is to show that those factors can't, in fact, be controlled for. Right now we've cut off one head of the hydra. Now we have to slay the beast itself.
No formula can control for inattention or apathy or social distraction. The negative impacts of poor attendance get seriously undervalued by educrats. Here in NYS most districts now are using shared/ditributed HS Regents scores for the 50% of teacher evaluations that must be derived from testing. This has relieved much of the test prep pressure and has eliminated scores of "local" pre and post tests; and as you would expect it has expunged all credibility from our APPR evaluations.
Delete@NY Teacher:
DeleteSocial Distraction-ha, ha.
Maybe 20 or more years ago I was doing substituting work prior to getting my K-8 Certification in the state I’d moved into. I was doing an English Lit. class on the high school level. After I passed out the assignment paper, and read the teacher’s instructions, a young woman sitting in the back of the class put her hand up with a question. I thought it might not be something that was confusing to the rest of the class, so I asked her to come up so I could go over the part of the instructions to answer her question.
She was seated in a row with four boys in front of her. As she passed up the row, she let her hand glide up the back of the neck of each boy and ruffle his hair. I should explain that she was simply gorgeous from top to bottom but especially in the important middle area. I was surprised and had nothing to say. I was able to give her an explanation of the point she was raising and she went back and began working on her paper. I always felt bad that those four boys probably were never going to understand some important story-telling detail about characters, setting, plot, -whatever it was that the paper was about.
NY Teacher is exactly right-these people have forgotten a lot about actual school.
Leila