How Assessment Ruins Standards

Long time readers know that I do not subscribe to the whole "The standards are swell; it's just those evil tests that screw everything up" school of thought. I think there are plenty of reasons to oppose national standards no matter what standards they are, and plenty of reasons to believe that no set of national standards will ever accomplish any of the goals set for them.

But let's set all of that aside for a moment and talk about how the very attempt to assess standards-based outcomes ruins those standards.

For my example, I'm going to pick the oft-noted CCSS standards about evidence.

I pick it because it's a part of the Common Core that doesn't particularly bother me. Like most English teachers, I've been encouraging (in many cases, quite vigorously) my students to provide support for whatever idea they are trying to assert. ("No, Chris-- saying Huck Finn is a dynamic character because he does dynamic stuff in a dynamic way does not really make your case"). When I assign a paper, two of the main questions I consider when assigning a grade are 1) did you actually have a point and 2) did you support it with actual evidence.

So in this area, the Core and I can co-exist peacefully.

But this kind of evidence use is a tool, a technique, and so it can't be assessed in a vacuum, just as you cannot judge somebody's hammering skills by just watching them hold a hammer or judge their free-throw shooting skills without handing them a ball. And that's where we get into trouble.

Assessing this skill is easiest when tying it to an act of critical thinking. But critical thinking has to be an open-ended activity. (Here's a quick tip-- if your question only has a single possible correct answer, it is not assessing critical thinking skills.) In my classroom, the most obvious avenue is a response to literature, though it will also work to deal with social issues, human behavior puzzles, historical research, or evaluating somebody else's essay work. Just to name a few.

To make this assessment work, I have to really know my stuff. I have to know Huck Finn frontwards and backwards, and perhaps augment my own expertise with lots of reading of scholarly critiques of the work. I have to know the work well enough that I can give the student freedom to go where his ideas lead him without me having to say, "Sorry, but I don't know anything about that, so I can't grade it, so you can't do it."

Every area of study is like a big patch of real estate, and I need to be a well-informed native guide who knows the territory so well that no matter where the student wanders off to, I'll know the terrain. When I don't know the territory, I end up fencing students in. "Just stay on the path-- don't stray." At the very worst, I will lay rails and make everyone ride through the territory on a train that only goes where it is meant to. The train tour is by far the least interesting, the least useful, the least rewarding, the least educational, and the least authentic way to explore the territory. It rules out all discovery and invention, and it is certainly hard on any prospect for joy or excitement.

But again-- we can only scrap the train and the pathway if we have a knowledgeable native guide. That's how my juniors can do a unit built on research of local history-- because I've made myself a little bit of an expert on the subject. That's how my colleague can do a massive end-of-year project with her AP seniors about Paradise Lost-- because she knows and loves that work. If we had to trade projects, we would both be lost.

But within our areas, we are qualified to tell the difference between evidence that is really evidence, and evidence that is just a piece of camouflaged baloney.

So here's why a standardized test can't test this standard.

First, a standardized tests starts with the assumption that any person (or computer program) should be able to evaluate the student's work without possessing any actual expertise at all.

Second, the answers have to be evaluatable in a very short time span.

And that means we will travel through the territory strapped into a seat on a tightly run train.

Look through the PARCC samples. These are slightly spiffed up multiple-choice single answer questions. The new SAT essay is just a wordier version of the same thing-- look at a piece of writing, determine what point the test manufacturers think you should see, and support it with the evidence that the test manufacturers believe are the correct pieces of evidence. These folks keep coming up with more complicated ways to ask closed-ended questions. This is partly, I suspect, because it's simply faster and more efficient to have a test of closed-ended questions that can be scored by any non-expert (or non-human). But also partly because some of these folks just have a narrow, cramped, stilted view of life and the world. "I handed you a brush, a small flat surface, and a jar of blue paint. What else would any normal person do except paint the flat surface solid blue?"

But that's not critical thinking, and it's not supporting your ideas with evidence.

Now, what is often shortened to "support with evidence" in discussion of the CCSS is actually mangled pretty badly in the actual wording of the standards, but even if it weren't, the Big Standardized Tests would mangle this idea to death anyway.

What this should properly mean is, "Come up with an idea that makes sense to you, and support it with evidence that you believe backs up your idea." But the only person who can evaluate that is a classroom teacher who possesses 1) enough expertise to evaluate the student's process 2) fewer than 1,000 students so that said evaluation can occur within the next week or so.

So what the standard means in a standardized test situations is, "Figure out what idea the test manufacturer wants you to find, and then locate the details that the test manufacturer wants you to pick out."

When we talk about test prep in the ELA world, we're talking about getting students into that second mindset, about training them to figure out the One Correct Answer associated with each piece of reading and the Only Correct Evidence located as well. And then repeat it, year after year after year.

And what really sucks is that we are getting good at it, and our students are paying the price. This whole blog piece is the result of a conversation I had with a colleague, both of us concerned because, despite our best efforts, we find our students over the past few years have become progressively worse at really engaging with reading and writing. They have learned that it's not about thinking or reacting or engaging and gripping the material with your own brain. What do you think? Why do you think that? Can you convince me to agree with you? I feel like my students have only recently encountered these kinds of questions, and they aren't sure what to do with them, because they've learned that there's only one right way to do each reading and writing task, and that one right way is known by someone else, and it's up to them to figure that out.

Of course, this issue didn't start with Common Core and Big Standardized Testing, but those conjoined twins have made things so much worse. When you try to make a complicated idea something you can assess "at scale," you do enormous damage. When you write a standard specifically so that you CAN test it "at scale," you break it entirely.