US News, Please Knock It Off

 US News was once a magazine, but these days it's arguably most famous as a Ranker of Things, especially schools. They rank colleges and high schools annually, and despite the fact that these rankings are hugely questionable (see here, here and here), they are uncritically reprinted, quoted, and used by the fortunate top tier as a marketing tool. 

So I'm sure from their perspective it makes sense to extend the brand by ranking elementary and middle schools. This is just as bad an idea as you think it is, and raises some big questions.

How do they do it?

I first guessed a system that used darts, a blindfold, and the broad side of a barn. But no--it's worse than that.

Scoring was almost entirely rooted in students’ performance on mathematics and reading/language arts state assessments.

So, standardized test scores from 2018-2019. But also demographics worked in by soaking the test results in a sophisticated stew of argle-bargle fertilizer, because US News employs data strategists instead of journalists:

As a note, we hope readers benefit from the sophistication of our analysis. The U.S. News rankings team produced multivariate regressions that assessed student performance in the context of demographics and their states. We believe that is more useful than simply looking at test results to evaluate schools, because this process resembles to a certain extent how education administrators and researchers consider school performance. But our rankings do provide considerable weights for scores themselves, too, because we believe parents value environments where most children arrive prepared to learn and teachers can provide a culture of enrichment.

So, we try to take into account where test scores are affected by the fact that non-wealthy, non-white students tend to do more poorly on these tests, but at the same time, we know that lots of parents value environments that are wealthy and white. 

What next?

Parents should use these results to get "insights into a key element of school quality." Parents can use "rankings, our data and word-of-mouth research" to shop for schools, and it's a close-to-honest statement that ranks US News ranking as tied for first place with "Talk to the ladies at the hair salon." 

As many Wags on Twitter (a fine band name) observed, we can look for US News to continue to expand its brand. First obvious choice is rankings for pre-schools, but why stop there? America needs to know--where are the top-ranked playgrounds in the country? Whose mini-van back seat is producing the leaders of tomorrow? Which were the top-ranked fetuses of the year, and which uteruses are the best? Top-ranked sperm? 

My dream is that the world greets this latest rank adventure with a massive yawn, but they won't. People love rankings, love them so much that too many don't even pause to ask, "Rank based on what, exactly?" Nobody anywhere is going to benefit from the sophistication of their analysis; the best we can hope for is that schools do not follow the lead of colleges and some high schools and start trying to game the system ("Sorry, Mrs. Potts, but your child is going to bring down our ranking with their test scores, so we're booting little Pat out of kindergarten.")

Just stop, US News. Just stop.