Why The Big Standardized Test Is Useless For Teachers

In schools throughout the country, it is testing season--time for students to take the Big Standardized Test (the PARCC, SBA, or your state's alternative). This ritual really blossomed way back in the days of No Child Left Behind, but after all these years, teachers are mostly unexcited about it. There are many problems with the testing regimen, but a big issue for classroom teachers is that the tests do not help the teacher do her job.

Folks outside of schools often imagine that one of the benefits of the test is to check to see how students are doing and adjust instruction accordingly. Unfortunately, the tests provide no such benefit.

First, the timing does not serve that purpose. Tests are being given now, close to the end of the year. By the time test score come back, these students will be in a different teacher’s classroom. There will be zero opportunity for a teacher to say, “Okay, these students are having trouble with fractions, so I’d better review that unit and add some extra instruction on the subject.” Those students are gone. New students arrive with their own test scores, but their new teachers have no first-hand knowledge of how instruction went last year.

But that’s not the worst of it. Even if the tests were great (that's a discussion for another day) and the results came back instantaneously, they would still be of little use for informing instruction.

Imagine that you are a basketball coach, tasked with training your team for great things. Imagine that when game day comes, you are not allowed to be in the gym with your team to see them play, and that they are forbidden to tell you anything about how the game went. You aren’t even allowed to know about the opposing team. All you are allowed to know is how many points your team scored. And yet, somehow, you are to make efficient use of practice time to strengthen their weaknesses. You can practice the kinds of skills that you imagine probably factor in a game, but you have no way of knowing how they use those skills in a game situation, or what specifically you should try to fix.

That’s the situation with the standardized test. (Well, actually, it's worse. To really get the analogy right, we'd also have to imagine that as soon as the ball left the players' hands, a blindfold slammed down over their eyes, so they don't really know how they're doing, either.)

You would think that part of analyzing the results of a test would involve looking at which items students missed and what kinds of mistakes they made, so that the teacher can identify and plug the holes in student knowledge. But as part of training in “test administration ethics,” teachers are sternly warned that they should never even look at the test questions. In many states, students are required to sign a pledge that they will not discuss any test questions with anyone at all. Nor are any of the test materials released once the test has been administered. When the results of the test come back, teachers see a score. That’s it. Nothing that shows them which questions were missed or which wrong answers were frequently selected (a good guide to where instruction is failing). Some tests provide a breakdown by content area, but often that is so vague that it’s useless; for years in PA our ELA “analysis” from the test said either that the student had problems in reading fiction or reading non-fiction.

More detailed information is certainly possible. The PSAT results include question by question breakdowns, including which answer then student selected and which answer was the correct one, and there are a very few states that do a better job.

So why don’t the big standardized state tests provide that kind of feedback? Why do the tests stay locked in a black box of secrecy?

The test manufacturers claim that this is about fairness and accuracy, and in the past the PARCC folks have fought an aggressive (and losing) battle against students on social media spreading word of testing items. That’s a legitimate concern, but it doesn’t explain the need for secrecy after the fact, unless of course the test manufacturers intend to keep costs down by recycling test items from year to year.

The secrecy also makes it difficult for teachers, parents, and students to challenge questionable content. Occasionally word gets out of a particularly egregious example (e.g. the infamous talking pineapple test items), and there is plenty to question. Now that I’m retired, I can admit it—I peeked, every year. Most teachers do. The reading portion of the tests involves many opinion questions masquerading as objective judgments with answers that are highly debatable, but because the tests are hidden under a cloak of proprietary secrecy, those debates never happen.

The secrecy ultimately serves test manufacturers, not teachers or students. As it is, even if the tests are generating data about student learning and teacher instruction, but that data is unavailable to teachers and students in levels that are granular enough to be useful. Because the tests are high stakes, they drive instruction, but because they are hidden in a black box, they drive it in vague and not-very-useful ways. The assessments that teachers create on their own, administer to students, and then examine in detail, are far more useful for informing instruction. The tests now taking up hours of school time may be helping somebody, somewhere, but not anybody who actually works inside those schools.

Originally posted at Forbes