Friday, September 16, 2016

What do you mean? I took the test.

My class is wasting our time engaged for two days with a popular "research-based, computer adaptive assessment that helps kids learn."  It is several hours of our lives that we can't have back, and it reminds me once again of the very basic hurdles that these standardized bubble tests have trouble overcoming.



The biggest first hurdle is that the students have to care. The students have to think that it's really, really important that they give their all for these tests. The whole business makes me wonder if any of these test manufacturers have ever met an actual human, but I suspect that the real problem is that they have themselves so convinced that these tests are really important, valuable measures that it just no longer occurs to them that other people don't see reality in the same way (is there a word for when you gaslight yourself?)

But yesterday's first round reminded me of another issue, more subtle, but equally problematic.

So my student Chris (not the actual name) completes the forty-three questions in roughly ten minutes. This strikes me as improbable. SO I ask Cris, "Did you actually read the questions"

"Sure," Chris says.

"And did you read the selections that the questions went with?"

"No," Chris replies, with a subtext of "why would I do something stupid like that?"

I believe at this point I made That Face.

"What do you mean," Chris elaborates. "I took the test."

Chris was not yanking my chain. Chris believes that the task was completed. Chris and the test manufacturers and I do not share the same definition of the task.

This is not uncommon with instruments like a bubble test which reduce complex operations to a simple job. Teachers and students often have some version of the following conversation:

Teacher: Read the selection through carefully, looking for the main ideas and trying to divine the author's purpose. You should also be on the lookout for key words and phrases. See which key details and evidence support your ideas about main ideas and purpose, and if possible, look for ways to both synthesize the different perspectives included in the excerpt and prepare yourself to compare and contrast the approach to the thematic material in this work with other similarly themed works we've read. Ideally you should read the whole selection through twice so that you can catch the deep structural choices and give yourself a better chance to close read for the role of vocabulary and sentence structure to achieve the author's purpose. Do you understand my expectations for this assessment?

Student: So, you want me to bubble in the right answers for these three questions at the end.

This does not make students unusual examples of the human race. What teacher has not sat through a big high-flying staff meeting about some bold new initiative or state-mandated reform or newly-implemented program and after a few hours of full-blown juice-packed rhetoric said (out loud or to yourself, depending on your bravery), "Yeah, but what do you actually want me to do?"

It is human to translate these kind of full-blown operations down to "Blah blah blah blah blah do Task A." The simpler Task A is, the easier it is to ignore everything in the blah department. And there is nothing simpler than "Click on one of the four options."

You can argue that the students have to read the selection in order to have a better shot at clicking on the "correct" answer, but of course, once again, they first have to care if the answer they click on is correct. On top of that, in the case of crappy standardized tests (with which my juniors have much experience), students have learned that A) you often don't need to read the selection in order to answer the question and B) reading the selection doesn't help them find the correct answer anyway because the question is from crazy alternate universe in which they do not live. And in the cost-benefits analysis of my most struggling students, reading several paragraphs in the hopes that it might give you one more correct click is like plunking down $100 for one ounce of a soup that might not even taste good.

So at the end of the day, many of my students have just defined the task down. Go through forty or so questions, and click on an answer for each one. This supposed to somehow provide me with actionable data.

And at the end of the day, I resent the assessment for more than just the wasted time. I resent it for introducing the idea that sometimes I will ask my students to spend time on pointless things. I promised them at the beginning of the year that I would never purposefully waste their time. Now that promise is suspect.

And sure-- we can argue that as the teacher I'm supposed to do a better job of convincing them that this test is valuable and important and that they need to give it their all because Important Consequences rest on it. That somehow the results of forty-some standardized multiple choice questions will tell me things about them that I could not otherwise figure out for myself. But at the beginning of the year I also promised them I would never lie to them.

Yes, there are life lessons here. Sometimes you are required to perform certain tasks that strike you as pointless because people with more power than you believe that they know better than you what you need. In those situations people can require that you perform a task, but they cannot make you care. They cannot make you do it well. You always get to choose how much commitment you bring to a task.

Most of all, there are lessons here for me as a teacher. One is a reminder about the Rule of Stupid Rules-- if you invest your authority in a silly directive, you weaken your authority in all things. If you require the passengers on your ship wear blue deely-bobbers and sing nursery rhymes whenever they're on deck, you increase the likelihood that they will not pay attention to you when it's time to implement evacuation procedures. And this is also a reminder that we are challenged as teachers to design assessment tasks that cannot be easily defined down so that they don't actually require or measure any of the things we want them to. If a half-assed job actually fulfills the basic requirements of the task, then we have noone to blame but ourselves if a student can say, "What do you mean? I took the test."

16 comments:

  1. How about you CYA and tell them that this is an "important" , state mandated test and tell them when it is a test that will cover class material and will reflect on their transcript? I think students will make a smart decision if they trust you and they respect you.

    ReplyDelete
    Replies
    1. But it won't reflect on their transcript. My children have taken standardized test upon standardized test, and the scores on not on their transcripts. Their course grades are. Everyone, including the kids, knows that these tests are BS. My children race through the North Carolina end of grade test in about 30 minutes. They know that they are smart and will score fine by racing through it. Why waste another 90 minutes to get a marginally better score? No one is every going to use that score for anything important.

      Delete
    2. CYA? Really? My students trust me not to CMA at their expense. That's actually why they trust me.

      Delete
    3. I think maybe I didn't get my point across the right way. Teachers are mandated to push the stupid High Stakes Test, so let the kids know that it's a mandated test. Let them know when the test is a teacher designed test that will be used for grades. The kids get it and if they respect the teacher and themselves, they will ditch the mandated test and try their best for the one that counts.

      Delete
    4. Eric,

      You are certainly right. For the vast majority of students, teacher written and graded tests are the only high stakes exams the student will ever take. Fail an teacher written and graded exam in a course required for graduation and you will not get a high school diploma.

      If someone is against high stakes exams, clearly they should be against these teacher written and graded exams.

      Delete
    5. TE. it's tempting not to put this up at all, since it adds nothing of substance to the conversation. But I feel it's useful to periodically see just how wrong some folks can be, whether their ignorance is willful or just plain clueless.

      It is of course not true that the vast majority of students will never take a high stakes big standardized test-- the policies of the last two administrations have mandated and guaranteed that. Nor is it true that students will fail a high school course if they fail one exam.

      Nor could any thoughtful person fail to distinguish between the "high stakes" of a teacher-created assessment, the stakes of which are usually how high or low the student's grade will be, versus the high stakes of the Big Standardized Test that can decide anything from whether or not the student passes on to the next grade to whether or not a teacher keeps their job to whether or not a school gets necessary support.

      If you want to post here, you really need to make an effort to say less stupid things.

      Delete
    6. I know exactly how you define a high stakes test: any test that has consequences for teachers. The stakes for students are irrelevant.

      Delete
    7. Once again, TE, it's unclear whether you can't read or you just don't read. The amount of space spent in this blog railing against the use of tests to punish children makes your point just silly. If you're going to contribute to the conversation, you're going to have to do better than just making shit up.

      Delete
  2. Maybe the first time but these kids have been hearing this for a long time.

    ReplyDelete
  3. I think the worst lesson for the students is that tests, and school assignments in general, become merely tasks to complete. I see this attitude more and more in my college students. They just want me to tell them what to write or what the answers are for the test. Then they will regurgitate the material. This is not learning. I want them to write for themselves, not just for the class. I can get some of them to do this, but first I have to rid them of the idea that education is merely a series of tasks to be completed. Standardized testing is reinforcing this false idea every day now.

    ReplyDelete
  4. Sorry if I cause dissension or hurt anyone's feelings (not really), but this sounds suspiciously like teacher evaluation schemes.

    I must unerringly prove that I know my students, then unerringly prove that I intend to teach them something. As if there was ever an otherwise. But this must be supported by data, analysis of said data, and submitted by a ridiculously early in the school year deadline in order to "meet growth" by some deadline in April, (like taxes), in order to show that I am not some dud. Same ridiculous crap. Over and over.

    ReplyDelete
    Replies
    1. I have also been frustrated with the new "growth" requirements and all the pre-testing and post-testing I must document. It's another huge waste of time and contributes very little to the learning process, and growth can be gamed so easily that the results are meaningless.

      Delete
    2. Yeah, it sure seems like everything the government requires is a huge waste of time and contributes very little to the learning process.

      Delete
  5. I see this "task completion" mentality even when doing labs, something students of the past loved to do and engaged in the idea of doing something to see what happens. Now they want me to tell them what is supposed to happen so that they can put down the "right answer." I think this is a side effect of the testing obsession.

    ReplyDelete
    Replies
    1. Yes. I agree that this is caused by excessive testing, especially the multiple-choice variety where there is always a "right" answer.

      Delete
  6. I stay away from Facebook a lot. But EVERY TIME I look at it, Kitty is always fighting the good fight. Showing us articles we otherwise would not have seen. Fighting for better education for students, exposing the liars, hypocrisy, and morons that have tried to ruin education (and succeeded) that has dumbed down America to such a point that millions of Americans would actually vote for Donald Trump as our PRESIDENT !?!?. Kitty, you are an ANGEL.
    YOU MUST BE EMPOWERED BY SOME DIVINE FORCE NOT TO QUIT. I am not an ANGEL. Both my wife and I were teachers of handicapped kids for 40 years. As soon as we were able to, we retired. They were making it impossible to help kids anymore. We're so happy NOT to be teaching anymore, tongue can not tell. (The phrase comes from Mark Twain.)
    But Kitty hangs in there, seemingly fighting the the good fight, almost singlehandedly, every day! You are an ANGEL sent from heaven, or however one would state the thought. Wow! Keep it up.

    ReplyDelete