The Fallacy In Learning Loss Panic

Back in March of 2021 (roughly a thousand years ago in pandemic time), I made the argument that Learning Loss is educational halitosis; you start with a real thing, dress it up in some faux science, and use the ensuing panic to sell your preferred remedy. 

The tricky thing about Learning Loss panic is that it's not entirely made up--there are certainly some pieces of some sorts of learning that didn't happen when we were all pandemicking around with schools fully closed and half-assed distance learning and all the rest of the pendemess. But as soon as you start claiming that you can measure what has been lost in months or days or liters or cubic centimeters or hectares of learning, you are shoveling fertilizer in hopes of growing an orchard full of money trees. 

But one element of Learning Loss is just plain made up. Let me tell you what it is, and how to respend to people who try to push it on you.

For maximum panic, some folks are claiming that a drop in test scores due to Learning Loss indicates a future loss of earnings for individuals and economic strength for countries (for example, this from one of the leading promoters of test scores = future earnings, Eric Hanushek). All of this is based on a correlation between test score and life outcomes, except that there are problems with using this correlation.

The big one is that it is just a correlation, like noting that kids who wear larger shoes in fourth grade tend to be taller as adults. There is a connection--it's just not cause and effect. Students who come from a wealthier, whiter background tend to do better on tests. Students who come from a wealthier, whiter background tend to do better in life. In fact, let's trot out this old chart:

There has always been a critical piece of proof missing from the test score = life outcomes assertion. Does changing the score change the student's future? In other words, if I make my fourth grader wear larger shoes, will she grow taller as an adult? 

The very reformy Jay Greene, while of the very reformy University of Arkansas Education Reform Department, looked for evidence of that very thing--and found nothing. 

There is no reason to believe that getting Pat to score higher on the Big Standardized Test will earn Pat more money at work and a better life. None. Raising Pat's test score above the score that Pat would have achieved in some other unboosted alternate universe accomplishes nothing except getting Pat a higher score. (Well, unless you impaired the rest of Pat's education to get that score increase). 

By the same token, if Pat gets a lower test score than Pat would have "normally" achieved, there is no reason to believe that Pat will now suffer lower wages and lifetime earnings, an uglier spouse, unhappier children, and a less friendly dog than Pat would otherwise have experienced. The focus should not be on what a score predicts, but what a change in score from the "expected" score predicts.

People like Hanushek (and the sloppy journalists who depend on him) will say, "Research says that people with low test scores have low lifetime earnings."

The response to this is, "What does research say about how getting a lower-than-predicted test score? Is there research to show that lowering the test score lowers lifetime earnings? How does varying from the expected test score affect the student's future?" The answer is that the research says that it doesn't.

Bottom line: there is zero reason to believe that low pandemic-related test scores are indicative of future financial and economic disaster for individuals and countries.