A Reminder About The Uselessness Of Test Scores

 As we move through the latest stage of the pandemic in schools, we still get a lot of noise about how we Really Need to get those Big Standardized Test scores collected and crunched, because only then can we address Learning Loss or Pandemic Stumble or general Falling Behind. 

In doing so, we once again make the same old mistake of trying to use Big Standardized Test scores as a measure of future success (at its most extreme in the "students will suffer with years of lost earnings" think pieces).There is no particular reason to believe this is true. 

Let me remind you of this old graph.

In other words, a rich kid who drops out of high school has as much a chance of success as a poor kid who graduates from college. 

There are plenty of theories about why this is so. A Georgetown study concluded that early tests scores are less predictive of future success than socio-economic status. Those researchers point to an idea that echoes the issue of social capital that Robert Putnam explores in Our Kids-- that wealthier families have connections that both help locate opportunities for children (My kid really likes ponies, and I know a guy who runs a stable) as well as providing a safety net. As the Georgetown report puts it

When students from affluent families stumble, they have a softer landing and assistance getting back on track, while those in adverse environments are more likely to land on rocky ground and never recover.

The lead author of the report told CNBC

People with talent often don’t succeed. What we found in this study is that people with talent that come from disadvantaged households don’t do as well as people with very little talent from advantaged households.

The Georgetown report, like most such studies, is using test scores as a proxy for talent or smartitude. So what we're seeing here repeatedly is that tests are a lousy predictor of future earnings, life outcomes, etc. Which means that if we are concerned about those future outcomes for students, we need to look for better predictors.

There is a lot of legitimate concern right now over the fallout from pandemic. But obsessing over BS Test scores and throwing all our energy into trying to lift those scores is not the answer. The scariest part of that Georgetown report is in the last part of the sentence-- "those in adverse environments are more likely to land on rocky ground and never recover." If it's not too late to keep students from landing on rocky ground, we should try to prevent, and for those who have already landed, we should be helping them get back up. Hammering them to prep for the Test so they can Raise Those Scores is not the way.