Data Driven Drivel

To follow the current tempest over school reform, you would think that teachers are opposed to data. But isn't data good? And if we are going to collect data and use it to shape instruction, doesn't that make sense? If teachers don't like data-driven instruction, does that mean that they'd all rather design instruction based on oiuja boards and dowsing rods?

In fact, teachers aren't against data, and they find themselves in something of a rhetorical pickle, because on the one hand, we know that information and data are good, but on the other hand, our gut tells us that the data-driven fad is dehumanizing and bad for our students. But we have a hard time finding the right words. Let me take a shot at it.

The problem with the current data-collection fad is not that it collects too much data. It's that it doesn't collect enough.

Human beings are complicated and complex. All good teachers know that. It's why we collect data all the time. All. The. Time.

We go over a drill sheet on some simple skill. We call on students. We watch their responses. Did Johnny look puzzled or bored? Did Jane answer quickly, or do a lot of mulling? Did Ethel deliver and inspired insight or a lucky guess? Is Chris confused by the material or distracted by a fight with his best friend? Does Bob know how he got that answer, or did it come straight out his butt? We ask follow up questions, probe, watch carefully. We know there's a difference between a class that has a skill mastered and one that's just barely getting it, even if both classes get the same number of right answers.

We know all this because we collect literally thousands of data points, many of which boil down to verbal and non-verbal cues, and many of those we can interpret only because we've developed a relationship with the student. In the ten minutes it took to go over a simple worksheet, we have observed, gathered, sorted and collated thousands of data points.

These shiny new fancy data-collecting assessment whiz-bangs (available at a generous price from Pearson et al)-- how many data points do they collect?

One.

A score. A simple right or wrong number. They have to. It's all they can handle. If it's a complicated matter, they still reduce it to a fill-in-the-bubble, right-or-wrong, one-data-point number.

This is why these things are de-humanizing. Because human beings are complex creatures who generate wild and vibrant webs of complicated information, a complex of behavior so varied and stunning that the very computers that are used to analyze it cannot even begin to imitate it.

The data-driven craze is like a doctor who wants to diagnose a patient. She has available every test, every diagnostic, every lab facility in the world. But instead, she just writes down the patient's height and weight and calls it a day. Or posts it on her data wall.

We need to stop saying that we are opposed to data-driven instruction, because we're not-- we've been doing it for as long as we've been in a classroom. What we need to start saying is that the so-called data-driving tools that we're being offered (or forced) to use are crap, producing a thin sliver of useless data, a mere drop compared to the vast waterfalls of data available from the beautiful, varied human beings who are our students.

To data-driving assessment providers, we have to say, "I'm sorry that you're only capable of measuring a minute fraction of what I need to do my job. But you have to stop saying that because X is the only thing you can measure, X is the only thing that matters."