Does The Nation's Report Card Have A New Reading Problem

Chester "Checker" Finn has concerns about the National Assessment Governing Board (NAGB), the folks who bring us the National Assessment of Education Progress (NAEP), aka The Nation's Report Card, aka that Big Standardized Test that periodically fuels massive freak outs about the Youths and their Learnings.

Finn recently expressed concern over a "gag order" placed on the group by the chair, Haley Barbour, former Mississippi governor and RNC chair, which is a legit concern for any such body. This week he's worrying about a shift in the framework for reading assessments, both because it will interrupt the stream of consistent data and because there's some concern handling the issue of background knowledge and the socio-cultural context of reading, and this leads to a host of the usual debates surrounding reading tests.

None of this is new. What is reading exactly? What you believe shapes how and what you test. If you think, as the Common Core years encouraged us to, that reading is some set of discrete skills that somehow exist in your brain separate from knowledge etc, then you want to design tests in which knowledge doesn't matter, so, for example, giving third graders reading passages about ancient Turkish trading history or some other topic on which you can expect all students to be equally ignorant. If you think reading is basically decoding, you design extreme tests like the DIBELS test where students are required to decode nonsense syllables. If we drift to the end of the pool that's all about comprehension and higher order thinking, then it's hard to avoid giving reading tests that are not also thinking and knowledge tests. 

So I appreciate the concerns about reading tests, because the whole issue of trying to peek into a person's head to figure out what they do or don't understand is one of the great challenges of education. But here comes a place where I think I can put Finn and his fellow fretters at ease. Finn quotes a pair of professors from Johns Hopkins and Emery who are concerned about the socio-cultural framework:

“Rather than allowing poor performance to serve as a signal that large knowledge gaps should be fixed through better education,” they wrote in City Journal, “we will simply lower the impact of background knowledge on the reported test performance.... If NAEP follows this route,” they concluded, “its assessment will no longer be a reading test that we can trust to demonstrate where students need more help—and where teachers should focus their efforts.”

Never fear, folks, because the NAEP has never been useful for any of that. 

NAEP has had one primary use--to be fretted over by policy makers who then use it to flog whichever theory they have. It provides a bunch of cold hard data that does not accomplish any of the things data fans believe data can accomplish. Nobody's mind is changed or point proven.

And NAEP certainly doesn't inform the actual work of school districts. I admittedly don't have first hand knowledge of every district in the country, but I'd be very surprised to find any of them sitting down at curriculum planning time and saying, "Okay, let's be guided in this process by NAEP results." And I'd bet dollars to donuts that there are no teachers anywhere in the country depending on the NAEP to tell them where students need more help and teachers should focus their efforts. After all--teachers have the actual students right there in front of them. 

Meanwhile,  we still have to annually explain that "proficient" on the NAEP does not mean "grade level."

Fiddling with the NAEP will create more debating points for policy wonks and legislator staff members, but it won't make much difference in actual classrooms with actual teachers.

Finn goes on to describe what sounds like a horrifying/amazing fight amongst the NAGP members and staff (he calls it a "horror show") which, from out here in the cheap seats, seems like further evidence that the group really has lost any sense of what their mission is and what it means. Perhaps they might want to look around at the skipping of NAEP during the pandemic pause and ask what effect that missing data has had on teaching in this country (spoiler alert: none) and just calm themselves down.