Tuesday, August 8, 2017

NWEA Thought Police

Here's an astonishing piece of news from the "You're in the Wrong Line of Work" file.

NWEA, purveyors of a whole raft of standardized computerized testing, has managed to score a grant, specifically the first annual Social-Emotional Assessment Design Challenge, a competition for assessments that measure social-emotional learning, or SEL.

I will tell you just how they scored this prize in a minute, but first, a quiz--

A student, confronted with a standardized computerized test, rips through it quickly with little regard for carefully answering the questions. The student does this because:

A) The student is bored out of their skull with this stuff and tired of taking stupid tests

B) The student knows that the test has exactly zero stakes for them

C) The student is far more concerned about problems at home, a decaying relationship, etc

D) The student wants to take a nap

E) The student always fails these stoopid tests so what's the use of wasting time trying

F) The student has not been sufficiently motivated to create good, rich data sets to benefit government bureaucrats and researchers who depend on these data to shape super-duper policy ideas

Turns out that, according to NWEA's crack research team, it's none of the above. Here's the line of reasoning, which starts off well enough, and the veers wildly into the weeds:

Rapid guessing behavior is defined as responding to assessment items too quickly to comprehend the question. Extensive research, conducted by NWEA senior research fellow Steve Wise, links rapid guessing behavior to a lack of student engagement on an assessment. Based on this research, Soland and Jensen studied rapid guessing behavior on NWEA’s MAP Growth assessment and found that it directly correlates to the social-emotional constructs of self-management and self-regulation. Students who demonstrated a pattern of rapid guessing also demonstrated a lower ability to self-manage and self-regulate in school.

(Emphasis mine). Yes, zipping through a standardized basically shows that you're immature. The test company doesn't have to ask question to police your thoughts-- they can check your brain innards just by how you click the answers.

This is a classic problem in the testing industry-- these folks are so devoted to and invested in the Big Standardized Tests that they cannot imagine any intelligent response, any display of responsible agency except to take the test slowly, carefully and seriously. If this is the research that won, I cannot even imagine the research that lost. 


  1. I have a nephew who used to answer the questions so that they formed Christmas trees and other images on the answer page. He was bored and knew the test didn't matter.

  2. Or its that the school teaches them to rely on visual aids (younger kids) like a number grid and spend a week just starting multiplication of simple nunbers then they take this test and its 16 x ? = 32 and theyre flipping lost cuz they have to figure it out in their head. So after a few like that the kid is either emotionally having a meltdown feeling stupid or gives up thinks this is bs and just fills in random numbers to move on with their life.

  3. I think the same could be said for some forms of neuropsychological evaluation. My kid got bored after a full day of testing by a bunch of different strangers and his evaluation results reflected that. The head evaluator, a PhD, didn't even catch what was obvious to me. Early in the day, his assessors reported him to be neurotypical. Testers who worked with him toward the end of the day reported him to be "highly autistic". The end result was that I was given an expensive eval that left me more confused than ever. He was four years old at the time.