Is This A Case For Standardized Testing?

For some folks, love for the Big Standardized Test just never dies. If anything, fears that the pandemess would squelch the BS Test gave testophiles an extra shot to the heart. Over at the Fordham Institute, Victoria McDougald kicked off the month by taking a shot at making the 2024 case for the BS Test. McDougald is not an educator, but a policy maven, with years logged at the Gates Foundation. So how'd she do? Let's see what the current state of the argument is.

McDougald offers six reasons to stick with the Big Standardized Test.

1. Tests provide an essential source of information for students and parents about student learning, alongside grades and teacher feedback.

Well, at least we've moved forward from the days when reformsters argued that without BS Testing, parents and students would have no idea how students were doing. But this is still a silly argument. A single multiple choice test held up against the results of regular assessments and teacher observations will do one of two things-- agree with what parents have already learned, or contradict it. If it agrees, so what? If it disagrees, which will parents find more useful- a year's worth of direct observation and assessment, or that single snapshot? 

Yes, more data is more useful than less data, but with BS Testing we must always always always talk about opportunity cost. Look at the hours and days used through the year to prep, pre-test, re-test, and test--is the tidbit of data generated by the BS Test worth giving up all the other educational stuff that could have been done with all that time?

McDougald compares the test data to a doctor's appointment. 

Just as I wouldn’t skip my child’s annual physical at the doctor’s office, I wouldn’t opt out of testing that provides important data about how my child is doing and progressing academically.

Unless my child was spending the rest of the year with a team of health care professionals and the doctor's office in question was a shady one whose credentials are not actually established. 

2. Test scores help counteract grade inflation in schools.

There are some ed reform games that schools cannot win. If student grades are staying the same, that's a sign of stagnation. If they're dropping, that's a sign of failure. If grades are improving, that's just a sign of "grade inflation." 

The idea here is the BS Tests provide an objective measure, as if the hand of God is weighing student achievement on a divine scale. But tests are manufactured by human beings and, as witnessed by decades of argument about embedded bias in the SAT, subject to those humans' subjective choices. Even the beloved NAEP has questionable value as a gold standard benchmark.

3. Tests shed light on learning successes and gaps, and help teachers address students’ unique needs.

Again, testophiles have learned to temper their declarations of usefulness here, but this is still one of those selling points that reflects a lack of experience on the ground in a real classroom. Here's the nub--

Alongside other indicators of student performance, tests provide teachers with actionable data that can help inform their instruction...

No, they don't. You get your test results long after they are of any use, and those "results" come in a black box. You are not allowed to see the actual questions your students attempted to answer (because protecting the proprietary materials of test manufacturers takes priority overt usefulness for teachers), so as a teacher, you literally do not know what your students got wrong. Scores are single numbers with no particular depth or detail (student got a 5 in "reading non-fiction").  What detail you do get will come from the practice tests that your school requires in an attempt to try to figure out where test prep might help your school make better numbers. 

Even a mediocre teacher will get more utility out of a quick "check for understanding" quiz. 

McDougald also argues that the BS Test will help administrators figure out "which teachers and schools are excelling at or struggling with helping students learn." But since test results can vary wildly--particularly if they are first being soaked in value-added measure (VAM) sauce-- that's not necessarily true. And in the real worlds, that mostly just leads to more test prep ("Here--scrap that unit on a full novel and start using these reading excerpt drill books"). 

This is how Campbell's Law activates-- when you treat the measure of the thing as if it actually is the thing, you end up focusing on the measurement instead of the thing. IOW, you start pretending that "Raise student test scores" and "teach students more and better how to read and write" are synonymous. They aren't.

Four, state tests provide policymakers with consistent, comparable data about student learning statewide.

Talking about the complex systems, goals, outcomes, and effects of education is hard. Reducing everything to a number or letter grade makes it easy. Policymakers would rather do the easy thing. 

This reductive and nuance-free approach might be tolerable if it resulted in anything good. But the simplified measurements generated by the BS Test have been used mostly for destruction. "Look at those low scores! We'd better give that public school a huge boost in resources right away," said policymakers never. Instead, low scores make public schools targets for takeover, charterization, or arguments in favor of vouchers.

Low test scores usually feed the rhetoric of "Public schools are failing" and not "Policy makers are failing public schools."

And now that we've had test-centered education for a while, it's hard not to notice that certain folks only care about the data when it doesn't make their favorite policy look bad. Josh Cowen studied voucher data for a couple of decades before concluding that the data shows that vouchers result in bad student outcomes. And let's not forget the states where non-public schools don't have to generate BS Test data for policy makers to contemplate.

Fifth, they’re an important indicator of college readiness.

Yeah, not really. McDougald provides a link to a Fordham Institute article that doesn't really say anything about the tests as a measure of college readiness. Meanwhile, there's plenty of evidence that high school grades are better predictors of college success. 

Sixth, tests are also pretty good predicters of later life success.

This oft-repeated talking point is meaningless. Correlation is not causation, particularly when it's a correlation that is pretty easily explained.

Let's say that research shows that fifth graders who have a large shoe size grow up to be tall. Should a state interested in increasing average adult height mandate that all fifth graders should wear extra large shoes, or perhaps retain students in fifth grade until they fit in extra large shoes? Because that would be silly, right?

We know that students from certain socio-economic backgrounds do better on Big Standardized Tests. We know that students from that same certain socio-economic background have better life outcomes. 

What we don't know--what no research at all shows--is whether raising a student's test score will improve that student's life outcome. And unless and until someone shows that, this talking point is just an argument that everyone should wear big shoes.

So I'm going to call McDougald 0 for 6. The Big Standardized Test can generate some marginally useful data, but the big question that McDougald doesn't answer is whether or not that small slice of data is worth all the time and money and opportunity that it costs schools to generate. All these years into test-centered education and I have yet to see one reason that justifies the cost, let alone six.