Hanushek Plays The Hits (Chicken Littling The NAEP)

The NAEP scores have inspired a great deal of handwringing and navel gazing and at least one more addition to the edu-panic vocabulary (make way for a "learning recession"). And it provides another round of attention to economists who specialize in education blarney.

So the Washington Post's consideration of NAEP math scores, comes the headline "Math scores remain lower than a decade ago. It's a bad sign for the economy."

The second part of that headline signals the return of Eric Hanushek, the economist who has followed fellow economist Raj Chetty to the assertion that getting the right elementary school teacher will lead the student to make oh so much more money as an adult. A decade ago, this was a popular argument for those who wanted us to use Value-Added Measures to find all the Bad Teachers so schools could fire their way to excellence. That was a dumb idea.

One might ask how, exactly, any scientist could come up with evidence that the right teacher would lead to adult riches. Was it some sort of empirical anecdotal evidence, like multiple class reunions where everyone noticed that all of Mrs. Swellteach's former students were rich and all of Mrs. Dullbutt's students were poor? It was not, though one would think that if Chetty and Hanushek were correct, such anecdotes would be easy to come by.

This may sound like a silly notion, but Hanushek has been pushing it for at least fifteen years. Here's a paper from 2011 where he lays out a very specific connection between a teacher's percentile and the exact number of lifetime dollars that a student will gather. Seriously. See for yourself:

Take a good but not great teacher, one at the 69th percentile of all teachers rather than at the 50th percentile (that is, a teacher who is half a standard deviation above the average). She produces an increase of $10,600 on each student’s lifetime earnings. Even a modestly better than average teacher (60th percentile) raises individual earnings by $5,300, compared to what would otherwise be expected. While those numbers are not trivial, they burgeon dramatically once we recognize that every student in the class can expect such increases in earnings. Consider, for example, a teacher with a class of 20 students. Under such circumstances, the teacher at the 60th percentile will—each year—raise students’ aggregate earnings by a total of $106,000. The impact of one at the 69th percentile (as compared to the average) is $212,000, and one at the 84th percentile will shift earnings up by more than $400,000. But there is also symmetry to these calculations. A very low performing teacher (at the 16th percentile of effectiveness) will have a negative impact of $400,000 compared to an average teacher.

This was music to the ears of the nominally-Democrat crowd of reformsters, the folks in the Obama/Duncan axis who insisted that if everyone in the country got an advanced degree, nobody would be poor. ("Masters degree??!" exclaims the Walmart manager. "Then we'll start paying you $35 an hour!"). 

Hanushek's whole shtick is to slice test scores into pieces of standard deviation. It's Hanushek and friends who came up with the "days of learning" which is just a slice of standard deviation on a test score. 

So how did he come up with this connection between good teachers and lifetime earnings? I'm going to over-simplify here, because we're talking about economist stuff here, but it goes pretty much like this: We know that a better teacher is better because their students get higher test scores, and we know that students with higher test scores go onto have generally wealthier life outcomes.

Hmmm. Well, first we've got to ignore the fairly small teacher effect on student success in school. And maybe high test scores cause higher earnings, or maybe it's that (as we already know) students from wealthier backgrounds do better on standardized tests, and that students from wealthier backgrounds tend to grow up to be wealthy adults? At this point, we can also call out the data that aren't there. We've had plenty of time to follow the students of high and low VAM teachers to compare how they're all doing and see if there's a pattern then. And somebody could have pursued the biggest question of all-- is there a shred of evidence that raising a student's test score raises their life outcomes? 

You would think all of that is more than enough to retire this baloney. But Hanushek has adapted to the new educational preoccupations. In February of 2020 (aka The Last of the Before Times) there he was, insisting that NAEP scores showed we'd have to get better teachers in classrooms soon, issuing a full-on policy analysis from the Hoover Education Success Initiative-- "The Unavoidable: Tomorrow's Teacher Compensation." The Initiative is a gathering of the usual suspects-- the executive committee is Hanushek, Chester Finn (Fordham Institute boss-emeritus), Paul Peterson, and Margaret Raymond (CREDO chief and Hanushek's wife). Only by using test scores to select and recruit the best teachers can we usher in an era of prosperity.

Now, six years later, for some reason Lauren Lumpkin gave Hanushek a call so that he could explain that today's graduates will earn an average of 8% less through their lifetimes. Because they have fewer skills, a thing he knows because of the NAEP math scores ("Oh, you're one of those," groans the Walmart manager. "We'll start you at 8% less than these older guys.")

And that's not all--

He estimated the combined effect of those losses will cost the U.S. $90 trillion through the year 2100.

He explains

“People don’t get very concerned about this, in part because it’s sort of like blood pressure. It’s the silent killer you don’t notice until you notice it,” Hanushek said of the way math achievement will affect the economy. “What it comes out to is a huge number that we have to pay attention to because it affects our position in the world, frankly.”

 And I shouldn't just pick on Hanushek, because other economists are out there chicken littling about this, too. 

Thomas Kane and a crew at the National Bureau of Economic Research are predicting a lifetime earning loss of $900 billion for all the students enrolled in the 2020-21 school year. Kane is the Harvard GSE guy who stumped hard for Common Core and testing and once published a terrible analogy about how you can't diet without a bathroom scale and a mirror (really). 

These guys all have big ole credentials and big-time jobs, so maybe I'm just not smart enough to follow their lines of reasoning. But it sure looks like a big old pile of baloney to me. And yet somehow it just keeps coming back, floating on zombie air. 

Economist Hansuhek Gets It Wrong Again

When you want a bunch of legit-sounding baloney about education, call up an economist. I can't think of a single card-carrying economist who has produced useful insights about education, schools and teaching, but from Brookings to the Hoover Institute, economists can be counted on to provide a regular stream of fecund fertilizer about schools.

So here comes Eric Hanushek in the New York Times (staging one of their op-ed debates, which tend to resemble a soccer game played on the side of a mountain) to offer yet another rehash of his ideas about teaching. The Room for Debate pieces are always brief, but Hansuhek impressively gets a whole ton of wrong squeezed into a tiny space. Here's his opening paragaph:

Despite decades of study and enormous effort, we know little about how to train or select high quality teachers. We do know, however, that there are huge differences in the effectiveness of classroom teachers and that these differences can be observed.

This is a research puzzler of epic proportions. Hansuhek is saying, "We do not know how to tell the difference between a green apple and a red apple, but we have conclusive proof that a red apple tastes better." Exactly what would that experimental design look like? Exactly how do you compare the red and green apples if you can't tell them apart?

The research gets around this issue by using a circular design. We first define high quality teachers as those whose students get high test scores. Then we study these high quality teachers and discover that they get students to score well on tests. It's amazing!

Economists have been at the front of the parade declaring that teachers cannot be judged on qualifications or anything else except results. Here's a typical quote, this time from a Rand economist: "The best way to assess teachers' effectiveness is to look at their on-the-job performance, including what they do in the classroom and how much progress their students make on achievement tests."

It's economists who have given us the widely debunked shell game that is Valued Added Measuring of teachers, and they've been peddling that snake oil for a while (here's a research summary from 2005).  It captures all the wrong thinking of economists in one destructive ball-- all that matters about teachers is the test scores they produce, and every other factor that affects a student's test score can be worked out in a fancy equation.

And after all that, experts (and economists pretending to be experts) have figured that a teacher affects somewhere between 7.5% and 20% of the student outcome.

Now when Hanushek says that teachers make a huge difference, he is obliquely referencing his own crazy-pants assertion that having a good first grade teacher will make you almost a million bucks richer over your lifetime (you can also find the same baloney being sliced by Chetty, Friedman, and Rockoff). Both researchers demonstrate their complete lack of understanding of the difference between correlation and causation.

Remember that, as always, they believe that "test scores" equal "student achievement." They note that students who get high test scores grow up to make more money. Clearly, the test scores cause the more-money-making, right? Or could it be that (as we already know) students from wealthier backgrounds do better on standardized tests, and that students from wealthier backgrounds tend to grow up to be wealthy adults?

So, in short, what we know about the "huge difference" created by Hansuhek's idea of a "good teacher" is pretty much jack and also squat. But he's going to build a house on this sand sculpture of a foundation.

Without knowing the background, preparation or attributes that make a good teacher, we cannot rely on the credentialing process to regulate the quality of people who enter the profession. Therefore the most sensible approach is to expand the pool of potential teachers but tighten up on decisions about retention, tenure and rewards for staying in teaching.

Since we don't know how to spot good teachers, says Hanushek, we should get a bunch of people to enter the profession and then throw a bunch of them out. This is a fascinating approach, and what I really want to see is the kind of promotional brochures that Hanushek would help college programs design. "Come run up over $100K of debt on the off chance that you might be one of the lucky few to get a career in teaching." Or maybe "Do you think teaching might be the work you want to do, maybe? Well, don't get your heart set on it, but do commit to years of expensive education to test the waters." How does a career counselor even approach this subject? "We'd like all of you to commit to this profession with the understanding that we plan to find half of you unfit for it." How exactly do you talk a student into pursuing a career that you don't think he's fit for?

Evaluation of teacher performance becomes key. Gains in student achievement should be one element, because improving student achievement is what we are trying to do, but this is not even possible for most teachers. Moreover, nobody believes that decisions should be made just on test scores. What we need is some combination of supervisor judgments with the input of professional evaluators.

What? What??!! Improving student achievement aka test scores is what we're trying to do? First, which "we" do you mean, exactly, because I certainly didn't enter teaching dreaming of increasing standardized tests scores. And what do you mean "this is not even possible for most teachers"? I mean, it could be a sensible statement, acknowledging that most teachers do not teach subjects that are measured by the Big Standardized Test. And if "nobody believes" that the judgment should be made just on test scores, why would you say that raising test scores is "what we're trying to do"?

And "professional evaluators"? Really. That's a thing? People whose profession is just evaluating teachers? How do you get that job? How do you prepare for that job? Is that what we're going to do with all the people we talked into pursuing teaching as a career just so we could have excess to wash out?

Hansuhek closes by trotting out DC schools as an example of how the test and punish, carrot and stick system works so super well. Would that be the system that was revamped to not include test scores because they were such a mess? Or is he thinking of the good old days when She Who Will Not Be Named used the system to spread fear and loathing, creating an atmosphere ripe for rampant cheating?

There's no evidence, anywhere, that test-based accountability improves schools. None. Not a bit. Not when it's used for "merit pay," not when it's used for hiring and firing decisions, not when it's used for any system of carrots and sticks. Nor could there be evidence, because the only "evidence" folks like Hanushek are looking at is test scores, and test scores are a measure of one thing, and one thing only-- how well students score on the Big Standardized Test. And there is not a link anywhere that those test scores mean anything else (and that would include looking back to the days when US low test scores somehow didn't stand in the way of US economic and international success).

It's tired baloney, baloney sliced so thin that it's easy to see through it. You may want to argue that I am just a high school English teacher, so what do I know about big-brained economics stuff. I'd say that if a high school English teacher can see the big fat hole is your weak economist-generated argument, that just tells you how weak the argument is. Hansuhek has become one of those go-to "experts" whose continued credibility is a mystery to me. He may an intelligent man, a man who treats his mother well, and is fun to hang out with. But his arguments about education are baseless and unsupportable. If you're going to read any portion of the NYT debate, I recommend you skip over Hanushek and check out the indispensable Mercedes Schneider, whose piece is much more closely tied to reality.