Tuesday, September 8, 2015

Don't Fix Your Building

Up a floor from the publishing offices of the Journal of Obvious Conclusions, we find the Journal of Dumb Questions, which probably wants dibs on this working paper from the National Bureau of Economic Research-- "Investing in Schools: Capital Spending, Facility Conditions, and StudentAchievement."

I should probably be less hard on these guys-- after all, they are economists. And this is exactly the kind of stupid question we'll waste time on as long as we think the purpose of schools is to "raise student achievement" (aka "get higher test scores").

Here's the premise:

Public investments in repairs, modernization, and construction of schools cost billions. However, little is known about the nature of school facility investments, whether it actually changes the physical condition of public schools, and the subsequent causal impacts on student achievement. 

Yup. The roof in your school may be leaking, the bathrooms may be crumbling, and the heaters might not work, but before you go throwing a lot of money at the problems, let's ask the important question-- will fixing any of those things raise test scores?

The answer is, apparently, no. Now, I don't know how you pretend to research this. I could buy the paper on-line, but I can think of almost anything that might be a better use of my money (though I guess spending it on school improvement would not be on that list). The suggestion of the summary seems to be that they studied only schools that underwent big capital spending programs and looked to see if their students suddenly got better test scores. In related research, studies show that food does not make people healthy because today I ate a meal and I'm not any healthier than I was a week ago, so clearly eating food doesn't improve my health.

In other words, not only research that addresses a stupid question, but which "researches" it in a stupid way. Kudos to you, NBEC. But they persevere to arrive at this conclusion:

Thus, locally financed school capital campaigns – the predominant method through which facility investments are made – may represent a limited tool for realizing substantial gains in student achievement or closing achievement gaps. 

Also, a bicycle, because a vest has no sleeves.

Yes, I can imagine school boards across the country, meeting in emergency session. The superintendent tells them that the students are cramped, the floors are sagging, the pipes have burst, and there are alligators in the basement. "We need to make some major improvements to the building," she says. But now, armed with this research, the board members will be able to look her in the eye and ask, "Yes, but will repairs raise test scores."

This is the kind of foolishness you get when you start using one bad metric for your measure of success. Should we buy a new book series? Should we paint the crumbling walls? Should we offer ice cream in the cafeteria? Should we lower class sizes? Should we let teachers wear blue jeans? Should we make an effort to be nice to students? Should we have a new anti-bullying policy? Should we have an art show? Should we take a field trip? Should the principal let me send Alice to the office when she's having trouble getting over the death of her grandfather?

There are so many questions that a school needs to answer, and so many questions for which "But will it raise test scores" is not a useful part of the conversation. This research may be a big slice of stupid, but it's a big slice of stupid that is right in line with the times.


  1. If leaking roofs aren't fixed, then mold will develop and the building will eventually be condemned. That will be very expensive. Schools need buildings and buildings need to be maintained. I guess these people want public school children to go to schools without heat, air-conditioning, working toilets, etc. How is that going to be good for education and good for the health of the children? My children attend public schools and they deserve clean and healthy buildings with heat, air, etc. What's next? An analysis of how bus maintenance does not raise test scores? Maybe the buses don't need new tires or brakes?

    1. Haha! Yep, I'm sure that will be the next research subject for these guys.

  2. Yes, you SHOULD be hard on them precisely because they are economists, until they get the hell out of areas like education, of which they know nothing.

    Besides having facilities in good working order, since we're dealing with human psyches, there ought to be an effort to also make them aesthetically pleasing instead of feeling like an institution.

  3. I have access to the paper. Here is a relevant quote:

    The prevalence of public schools in need of repair is worrisome because poor physical environments may impede student achievement if students learn more easily in safe, clean, controlled environments (Jones and Zimmer, 2001).

    Indeed, recent evidence on the impacts of very large construction projects in contexts where school facilities were either in very poor condition or non-existent suggests that new school construction projects can improve student outcomes (Duflo, 2001; Aaronson and Mazumder, 2011; Neilson and Zimmerman, 2014). For instance, Neilson and Zimmerman (2014) find positive effects on reading achievement of a construction project financed through state and federal sources that cost $70,000 per pupil and involved rebuilding almost every school campus in an urban district (New Haven, CT). However, this type of capital campaign is atypical in the U.S. where school capital projects (both renovations and new construction) are primarily financed locally through the issuance of voter-approved bonds that are repaid with property taxes.2 For instance, the average per-pupil size of capital campaigns in Texas, the state we study in this paper, is about $7,800. The achievement effects of investments of this magnitude remain unclear.

  4. The next time your state representatives want to put the taxpayers on the hook for a new multi-million dollar NFL stadium, ask if it will make the players play better.