Why Critical Thinking Won't Be on The Test

Critical thinking is one of the Great White Whales of education. Every new education reform promises to foster it, and every new generation of Big Standardized Tests promises to measure it.

Everybody working in education has some idea of what it is, and yet it can be hard to put into a few words. There are entire websites devoted to it, and organizations and foundations dedicated to it. Here, for example, is the website of the Foundation for Critical Thinking. They've got a definition of critical thinking from the 1987 National Council for Excellence in Critical Thinking that goes on for five paragraphs. One of the shortest definitions I can pull out of their site is this one:

The intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action.

Bottom line-- critical thinking is complicated.

So can we believe test manufacturers when they say that their test measures critical thinking skills? Can a series of questions that can be delivered and scored on a national scale be designed that would actually measure the critical thinking skills of the test takers?

I think the obstacles to creating such a standardized test are huge. Here are the hurdles that test manufacturers would have to leap.

Critical thinking takes time.

Certainly there are people who can make rapid leaps to a conclusion, who can see patterns and structure of ideas quickly and clearly (though we could argue that's more intuitive thinking than critical thinking, but then, intuition might just be critical thinking that runs below the level of clear consciousness, so, again, complicated). But mostly the kind of analyses and evaluation that we associate with critical thinking takes time.

There's a reason that English teachers rarely give the assignment, "The instant you finish reading the last page of the assigned novel, immediately start writing the assigned paper and complete it within a half hour." Critical thinking is most often applied to complex constructions, and for most people it takes a while to examine, reflect, re-examine and pull apart the pieces of the matter.

If you are asking a question that must be answered right now, this second, you are at the very best asking a question that measures how quickly the student can critically think-- but you're probably not measuring critical thinking at all.

Critical thinking takes place in a personal context.

We do not do our critical thinking in a vacuum. We are all standing in a particular spot in space and time, and that vantage point gives us a particular perspective. What we bring to the problem in terms of prior understanding, background, and our own mental constructs, profoundly influences how we critically think about any problem.

We tend to make sense out of unfamiliar things by looking for familiar structures and patterns within them, and so our thinking is influenced by what we already know. I've been an amateur musician my whole life, so I can readily spot structures and patterns that mimic the sorts of things I know form the world of music. However, I am to athletics what Justin Bieber is to quantum physics, and my mental default is not to look at things in athletic terms. Think about your favorite teachers and explainers-- they are people who took something you couldn't understand and put it in terms you could understand. They connected what you didn't know to what you did know.

None of this is a revolutionary new insight, but we have to remember that it means every individual human beings brings a different set of tools to each critical thinking problem. That means it is impossible to design a critical thinking question that is a level playing field for all test takers. Impossible.

Critical thinking is social.

How many big critical thinking problems of the world were solved single-handedly by a single, isolated human being?

Our sciences have a finely-tuned carefully-structured method for both carrying on and acknowledging dialogue with the critical thinkers of the past. If a scientist popped up claiming to have written a groundbreaking paper for which he needed no citations nor footnotes because he had done it all himself, he would be lucky to be taken seriously for five minutes. The Einsteins of history worked in constant dialogue with other scientists; quantum theories were hammered out in part by dialogue by a disbelieving Einstein ("God does not play dice") and the wave of scientists building on the implications of his work.

On the less grand scale, we find our own students who want to talk about the test, want to compare answers, want to (and sometimes love to) argue about the finer points of every thinking assignment.

Look at our own field. We've all been working on a big final test question-- "What is the best way to take American education forward?"-- and almost everyone on every side of the question is involved in a huge sprawling debate that sees most of us pushing forward by trying to articulate our own perspective and thoughts while in dialogue with hundreds of other thinkers in varying degrees of agreement and disagreement. One of the reasons I trust and believe David Coleman far less than other reformsters is that he almost never acknowledges the value of any other thinker in his development of Common Core. To watch Coleman talk, you would think he developed the entire thing single-handedly in his own head. That is not the mark of a serious person.

Do people occasionally single-handedly solve critical thinking problems on their own, in isolation, like a keep-your-eyes-on-your-own-paper test? It's certainly not unheard of-- but it's not the norm. If your goal is to make the student answer the question in an isolation chamber, you are not testing critical thinking.

Critical thinking is divergent.

Let's go back to that critical thinking problem about how to best move forward with public education. You may have noticed that people have arrived a wide variety of conclusions about what the answer might be. There are two possible explanations for the wide variety of answers.

The first explanation is the childish one, and folks from both sides indulge in it-- people who have reached a conclusion other than mine are some combination of stupid, uninformed, and evil.

The more likely explanation is that, given a wide variety of different perspectives, different histories, and different values, intelligent people will use critical thinking skills and arrive at different conclusions.

Critical thinking is NOT starting with the conclusion that you want to reach and then constructing a bridge of arguments specifically designed to get you there, and yet this is perilously close to the kind of thinking a standardized test requires.

But here's a good rule of thumb for anyone trying to test critical thinking skills-- if you are designing your assessment and thinking, "Okay, any student who is really using critical thinking skills must come up with answer B," you are not testing critical thinking skills. No-- I take that back. Oddly enough this is a sort of critical thinking question, but the actual question is, "Given what you know about the people giving you the test and the clues they have left for you, what answer do you think the testmakers want you to select?" But that is probably not the question that you thought you were asking. As soon as you ask a question with one right answer (even if the one right answer is to select both correct answers), you are not testing critical thinking.

Critical thinking must be assessed by critical thinking.

How do you assess the answer to your critical thinking question? Again, I direct you to the education debates, where we "grade" each others' work all the time,  checking and analyzing, probing for logical fallacies, mis-presentation of data, mis-reading of other peoples' writing, honesty of logic, etc etc etc.

To assess how well someone has answered a critical thinking question, you need to be knowledgeable about the answerer, the subject matter, and whatever background knowledge they have brought to the table (if I answer a question using a music analogy and you know nothing about music, will you know if my analogy holds up). On top of all that, you need some critical thinking skills of your own. And that means all of the issues listed above come back into play.

What are the odds that you can get all that in a cadre of minimum-wage test-scorers who can wade through a nation's worth of tests quickly, efficiently, and accurately?

Can it be done?

When I look at all those hurdles and try to imagine a nationally scaled test that gets deals with all of them, I'm stumped. Heck, it's a challenge to come up with good measure for my own classroom, and that's because critical thinking is more of a tool than an end in itself. Testing for critical thinking skills is kind of like testing for hammering skills-- it can be done, but it will be an artificial situation and not as compelling and useful and telling as having the student actually build something.

So I try to come up with assessments that require critical thinking as a tool for completion of the assignment. Then I try to come up with the time to grade them. Could I come up with something for the entire nation? Practically speaking, no. Even if I get past the first few hurdles, when I reach the point that I need a couple million teachers to score it, I'm stumped. Plus, standardized test fans are not going to like the lack of standardization in my test.

No, I think that standardized testing and critical thinking are permanently at odds and we'd be further ahead trying to develop a test to compare the flammability of the water from different rivers.

Critical thinking is not on the BS Tests. It will not be on the new generations of the BS Tests. It will never be on the BS Tests. Test manufacturers should stop promising what they cannot hope to deliver.