Mark Weber's Three Critical Testing Questions

This is not the first time I've piggy-backed on a post from Mark "Jersey Jazzman" Weber, but sometimes what he posts just sets a bright blazing lightbulb off in my head. Weber is a certified PhD-carrying academic of data crunching, and he is an absolute master of rendering complicated ideas comprehensible to civilians, and that's what he's done with his post  about why "We Don't Need Standardized Testing In A Pandemic."  

I encourage you very much to go read the whole post. But sometimes saying, "This! This! This!" isn't enough for me. So. What I'm going to center on is his three-question response to anyone who says that students must take a standardized test.

1) What are you going to do with the results?

This question is hugely powerful and by far the least answered. Here's Weber:

A core concept of assessment is that tests must be shown to be valid for the purposes in which they will be used. In other words: you should make a separate argument for every proposed use of a test. A test that may be valid to use for, say, determining whether there are enough overall resources in the education system isn't necessarily valid for the purpose of determining whether a student should pass to the next grade.

Psyshometricians often speak of making a validity argument in favor of the use of a test for a particular purpose. That argument should touch upon the relevancy of the outcomes to a specific use, the consequences of making decisions based on these outcomes, the opportunity to learn the content in the test, and other factors.

This is just so key. As Weber notes, one of the huge problems with the current Big Standardized Test system is that one test is supposed to serve a variety of purposes. Evaluate schools, evaluate teachers, assess curriculum, inform parents, plus serve as a go-to measure of whether Reform Program X is "effective" or not. Plus, now, figuring out how far "behind" students are. And most of the time nobody has made a validity argument for any of it.

The answer to the question also needs to be specific. The announcement that the administration won't waive the test leans on that old standard promise that test results will be used to "target resources and supports to the students with the greatest needs." That's great.

How? 

How do you plan to use the test results to do that targeting? Where is the bill or program proposal that shows how test scores will be linked to the targeting of resources?

Of course there is no such proposal. There never has been. What states have actually done is things like using low scores to target schools for charterization. 

When your doctor wants to perform a biopsy, there's a reason, and it's not "Just thought IK'd chop you open, dig around, see if I can figure out anything about your diet or your blood pressure or maybe check for cancer, and then we'll do, I don't know, something." Your doctor has a specific intent both for the operation and for hat's going to be done with the results.

When we get to specifics of these tests, we are forced to see how useless they are. Are they to help teachers find out how "behind" students are? How will that work-- months from now, when the teachers see the basic test scores (but not the questions that were answered)? What will this one test tell teachers (particularly those who don't teach math or reading) that they can't --and haven't-- already figure out on their own in their own classroom?

And on what planet will we find the parent (because the Biden administration throws them in the test-justification mix, too) saying this-- "I had no idea how my child was doing in school, but now that I can see some very general results on a test she took a few months ago, it's all clear to me." 

If you can't tell me exactly what you want to do with the results of a test, you have no business giving the test. 

2) What is the cost of administering the test?

Testing is, of course, a billion-dollar industry, supported with tax dollars. But as Weber points out, there are other costs. We've had the long-term costs of narrowing the curriculum, and the cost of telling children of poor families that they are academic failures.

But there is also opportunity costs. Time spent t=on test prep and test taking is time not spent on education. I guarantee you that among the first reactions among teachers to the Biden Testing Is On announcement, right after some fiery obscenity, was the thought, "Well, what I'm going to have to cut out now."

For most of my career, my year to year aspiration was to see what else I could add to my class. By becoming more efficient, more focused, and better at understanding where my students were, I could claim some space in the 180 days and fill it with something else. But towards the end, as my school became more "data driven," the question flipped, and year to year I found myself asking, "Now that I'm losing more time to testing, what can I stand to cut." 

This year, time is more valuable and rare than ever. That makes the cost of testing astronomical.

3) Is the cost of testing worth it?

Again, directly from Weber:

What, exactly, will statewide standardized test outcomes tell us that we didn't already know, or that we couldn't find out some other way? That students didn't gain as much learning as they would have without the pandemic? We already know this; and again, it's not like the tests will give educators data they couldn't get other ways that are much faster and more detailed.

Will we learn that students who are economically disadvantaged need more resources to equalize their educational opportunity? We already knew that before the pandemic. And does anyone really think that otherwise reluctant politicians will be persuaded to dole out more funds when they see this year's test scores? Really?

And on top of that, will there be any magical means of determining which part of the score is the result of student knowledge and which will be the result of students taking tests during a chaotic year under lousy circumstances under family stress? There is always reason to doubt that the tests measure what they purport to measure; this year that doubt can be multiplied a hundredfold.

It will be worth it to some folks. Test manufacturers will get their payday. And I'm betting that once test results are out, folks who like to declare that public schools are disastrous failures will have a field day pushing the critical corporate reforms that must be implemented to Save Students. The Big Standardized Test has always been a tool for those who want to break public education down and sell off the parts; these test results should be a disaster capitalist holiday, a national-scale Hurricane Katrina.

Weber's three questions are a useful response to testing in any year, yielding results that can be debated by reasonable people. But this year it's pretty clear that the cost will be high and the results will be mostly worthless. Remember these three questions for the years ahead, but this year, the Biden administration should reverse its bad decision.