Oh Nos!! NAEP Sadness!

Wednesday unleashed a torrent of bureaucratic sadness as the newest batch of NAEP numbercrunching shambled out into the light of day. NAEP stands for National Assessment of Educational Progress (and not, as I always assumed, No Actual Educational Purpose), and it involves giving math and science tests to students throughout their educational career.

The Big Sadness is because high school seniors have stayed flat. While fourth and eighth grade test results have inched up for the past couple of decades, senior results stay stuck pretty much in the same corner of the testing results sargasso sea. And we are so very dismayed!!

"Our high schools take kids who have made incredible progress in fourth grade and eighth grade," said Mark Schneider, a vice president at the American Institutes of Research who previously led the government arm that administered NAEP. "Whatever good we did is gone."

Experts are scrambling to take a shot at explaining this development. Schneider himself has a pair of theories. "Either our high schools are doing a terrible job, or 12th graders don't care about NAEP."

Oh, Mark. Marky Mark Mark. Let's play a little thought experiment. You are a high school senior in the two-thousand-teens. You have been taking standardized tests your entire school career, tests that you understand aren't really related to your education. But now you are a senior. There's prom and maybe college and some parties and that cute boy/girl that you've got only so much time left to ask out. Nothing left stands between you and your graduation. And here comes one more standardized dumb test that you know has absolutely nothing to do with your life, your future, or anything that you care about. The only thing special about it is that, out of 14,283 standardized tests you've faced down, this is one you can finally safely tell to take a flying leap.

What do you think, Mark? Do you think, "Boy, I had better do my level best to generate some really good, valid data for those number-crunching technocrats to look at. I don't want to disappoint them, so no matter how tough I will just bear down and-- wow, Sandy is looking really cute today!"

Caring is the great missing link in standardized test validity. Test oracles (oh, can I please call them testacles for short) are so devoted to their super duper magical mystical testing programs that it just doesn't occur to them that the world is filled with people who can look at big beautiful standardized tests and just not give a rat's ass. Every test result we ever look at starts with the assumption that the student cared enough to really try.

On some level, testacles know students don't. That's why we keep ramping up the stakes all the way to "Pass this test or you don't get to move out of this grade"-- because we know that without some sort of stick, the students would just start playing ACDC and doodling Spongebob pictures.

Course, not everybody gets that. Joy Resmovits quotes Cornelia Orr of the National Assessment Governing Board calling the senior disengagement an urban myth. After all (I swear I am not making this up), they see a motivational video first! Now, I should not scoff motivational videos. In high school we all saw drivers ed movies including Signal 30 and Mechanized Death and these automotive snuff films motivated us to want to walk everywhere for the rest of our lives (we got over it). So maybe the motivational film depicts Jimmy, a young man who fails to take the NAEP test seriously and ends up impaled on a stack of frozen moray eels at the prom.

There are other possible explanations for where all the awesomeness of the earlier results is vanishing to. One is that the whole model of educational scaffolding is just wrong, and that we can't just jump ahead of one developmental step and expect to stay ahead the rest of our educational career. If you're running a marathon, running your first five miles ten minutes faster than you usually do does NOT mean that you will finish the entire race ten minutes faster than usual.

Another is that the lower grades are loaded with all manner of test prep, which makes fourth and eighth graders particularly ready to ride the train to bubbleland. Meanwhile, schools have pretty much stopped the test prep stuff for seniors, whose bubblicious days are behind them.

Peggy Carr, an NAEP administrator, has an interesting theory-- since many students who would have dropped/flunked out in the past are now sticking it out, they are contributing lower scores to the aggregate and bringing down the overall awesomeness. Of course, her theory assumes that the awesome new grad rate numbers are not statistical shenanigans and hoary hocus-pocus; unfortunately, this article by smart person Julian Vasquez Heilig suggests that the new numbers are baloney.

Let me wrap up with one more quote from Mark Schneider.

"Students aren't learning what they should be learning in high school."

Students aren't learning? We don't really know that, do we? We just know they aren't getting the preferred answers on questions that purport to measure whether or not they have learned certain things. What they should be learning? Wow, there is a whole mountain of assumption in that one-- exactly whose idea of what they should be learning are we chasing here?

There is one thing I do know. We've had the test-centric federal-driven top-down standardized status quo in place for over a decade, and reformsters have had the chance to redesign schools as they wish. We have spent over a decade devoting the entire educational system to churning out exemplary standardized test-takers, and apparently we are bad at it. If stagnant NAEP results prove anything at all, they prove that the reformster agenda is a failure.