More Proof The Big Standardized Tests Mean Diddly-Squat

Chad Aldeman (Bellwether Education Partners) wants us to take heart. He's over at The 74 (Campbell Brown's House of Reform Advocacy) arguing that although reformsters are writing off the last ten years as a "lost decade" there was actually some good news-- while "educational achievement" (by which Aldeman actually means "standardized test scores") remained stagnant, college attainment has been on the rise.

Well, that's a puzzler. Adelman suggests that you might explain the incongruity by claiming that colleges and high schools just lowered their standards. But he thinks the attainment rates deserve attention.

One reason is that it’s much easier to gauge attainment than achievement. Measuring attainment is as simple as asking adults what their highest level of education is, whereas measuring achievement involves a complex process of defining what people at various ages should know and be able to do, developing a tool to measure those skills, sampling the given population and then translating the sample results into larger estimates.

Well, yes. That second thing, the measuring achievement is hard, so hard I would argue (as I have for forever) that we can't actually do it, but in the education disruptor's desire to have such a tool, we've just rushed a couple of bad proxies to market and continued to pretend that they measure what they don't.

We already know that the NAEP is not actually a "gold standard" for measuring educational achievement. And the PARCC, SBA, and all their sad cousins/replacements have never, ever made a convincing case that they can measure educational achievement. And soaking all this crappy data in a vat of VAM sauce doesn't make it any better. And even reformsters have accepted that changing Big Standardized Test scores does not have any connection to changing future life outcomes.

Aldeman goes on to parse and pick at the data a little more, but the main conclusion here is obvious. The proxy-of-choice for education achievement is supposed to tell us, among other things, how many students are how ready for college. And yet while that proxy has been screaming that students are no more ready for college than they were a decade ago, many more students are successfully completing college.

Look- if your local weather man gets the forecast wrong for ten straight years, you do not scratch your head and wonder what's gone wrong with the weather. You must conclude that whatever he's using as a tool for prognostication is busted. If your yardstick tells you that your child has not grown in ten years, and yet your household budget, your clothes shopping, and your actual eyeballs tell you that your child has grown three feet in ten years, you don't assume that your child is perpetrating an amazing illusion-- you throw out the yardstick.

The Big Standardized Tests that generate the scores that folks keep trying to pass off as a perfect proxy for student achievement? They do not work. They do not tell us what folks claim they tell us. How much more evidence do we need before we stop pretending that we know how to measure educational achievement?