Testing the Invisibles

Last weekend, Chad Aldeman of Bellwether Education Partners took to the op-ed pages of the NYT to make his case for annual standardized testing. I offered my response to that here (short version: I found it mostly unconvincing).

But Aldeman is back today on Bellwether's blog to elaborate on one of his supporting points, and I think it's worth responding to because it's one of the more complicated fails in the pro-testing argument.

Aldeman's point is this: NCLB's requirement that districts be accountable for subgroups forced schools to pay attention to previously-ignored portions of their student population, and that led to extra attention that paid off in test score gains for members of those groups. Aldeman did some data crunching, and he believes that they crunched results show "a move away from annual testing would leave many subgroups and more than 1 million students functionally “invisible” to state accountability systems."

This whole portion of the testing argument shows a perfect pairing of a real problem and a false solution. I just wrote about how this technique works, but let me lay out what the issue is here.

I believe that Aldeman's statement of the basic issue is valid. I believe that we are right to question just how much certain school districts hope to hide their problem students, their difficult students, their we-just-aren't-sure-what-to-do-with-them students. I believe it's right to make sure that a school is serving all students, regardless of race, ability, class, or any other differential identifier you care to name.

But where Aldeman and I part ways comes next.


Are tests our only eyes?

Aldeman adds a bunch of specific data about how many groups of students at various districts would become invisible if annual testing stopped, which just makes me ask-- is a BST the only possible way to see those students? There's no other possible measure, like, say, the actual grades and class performance in the school, that the groups could be broken out of? (And-- it should be noted that Aldeman skips right over the part where we ask if any such ignoring and invisibility was actually taking place.)

Because I'm thinking that not only are Big Standardized Tests not the only possible way to hold schools accountable for how they educate the subgroups, but they aren't even the best way. Or a good way.

Disagregated bad data is still bad data.

Making sure that we break out test results for certain subgroups is only useful if the test results tell us something useful. There's no reason to believe that the PARCC, the SBA, and the various other Big Standardized Tests tell us anything significant about the quality of a student's education.

Aldeman writes that losing the annual BST would be bad "because NCLB’s emphasis on historically disadvantaged groups forced schools to pay attention to these groups and led to real achievement gains." But by "real achievement gains" Aldeman just means better test scores, and after over a decade of test-based accountability, we still have no real evidence that test scores have anything to do with real educational achievement.

This part of the argument continues to be tautological-- we need to get these students' test scores because otherwise, how will we know what their test scores are. The testy worm continues to devour its own tail, but still nobody can offer evidence that the BST measures any of the things we are rightfully concerned about.

Still, even as bad data, it forces school districts to pay attention these "historically disadvantaged groups." That's got to be a good thing, right?

Well, no.

The other point that goes unexamined by Aldeman and other advocates of this argument is just what being visible gets these students.

Once we have disagregated a group and rendered them visible, what exactly comes next?

Does the local district say, "Wow- we must take steps to redirect resources and staff to make sure the school provides a richer, fuller, better education to these students." Does the state say, "This district needs an increase in state education aid money in order to meet the needs of these students."
Generally, no.

Instead, the students with low test scores win a free trip to the bowels of test-prep hell. Since NCLB began, we've heard a steady drip-drip-drip of stories about students who, having failed the BST (or the BST pre-test that schools started giving for precisely the purpose of spotting probable test-failers before they killed the school's numbers) lose access to art and music and gym or even science and history. These students get tagged for days filled with practice tests, test prep, test practice, test sundaes with test cherries on top. In order to insure that their test scores go up, their access to a full, rounded education goes down. This is particularly damaging when we're talking about students who have great strengths in areas that have nothing to do with taking a standardized reading and math test.

Disagregation also makes it easier to inflict Death By Subgroup on a school. Too many low BST subgroup failures, and a school can become a target for turnaround or privatization.

Visibility needs a purpose

Nobody should be invisible-- not in school, not in life. But it's not enough just to be seen. It matters what people do once they see you.


So far we have mostly failed to translate visibility into a better education for members of the subgroups. In fact, at many schools we have actually given them less education, an education in nothing but test taking. And by making them the instruments of a school's punishment, we encourage schools to view these students as problems and obstacles rather than human beings to assist and serve.

NCLB turned schools backwards, turning children from students to be served by the school into employees whose job is to earn good test scores for the school. As with many portions of NCLB, the original goal may well have been noble, but the execution turned that goal into a toxic backwards version of itself.

Making sure that "historically disadvantaged subgroups" don't become overlooked and under-served (or, for that matter, ejected by a charter school for being low achievers) is a laudable and essential goal, but using Big Standardized Tests, annually or otherwise, fails as an instrument of achieving that goal.