WSJ: Fordham's Four Kinds of Wrong

Mike Petrilli and Aaron Churchill took to the Wall Street Journal Monday to present four kinds of wrong while promoting a new Fordham report to be released today.

Before we even get into what they said or why it's baloney, let's open with the caveat that they themselves left out of the article. The study looks at the benefits of closing schools and was done in Ohio, where the Fordham operates charters that directly benefit from the closing schools. So this is, once again, a study touting the benefits of cigarette smoking brought to you by your friends at the Tobacco Institute.

Fordham's claim is simple-- when schools are closed and the students are moved to a new school, those students gain forty-nine extra days of learning. Closing the school and moving the students raises the student achievement.

That's the claim. How is it baloney? Let me count the ways.

1) "Days of learning" is not a thing.

While this concept is frequently used, I've never encountered its use by anyone except reformsters trying to make a case for Reform d'jour. I have never seen it used by a serious researcher. While that may just mean I'm not well-read enough (you can edify me in the comments), I'm still unconvinced by it as a measure.

First, it assumes that learning is a simple linear progression, like I-79 from Pittsburgh to Erie. That strikes me as a view of education so simplistic as to be useless. What is a Day of Learning, exactly, and do we distinguish between a Monday of Learning or a Friday of Learning. How about the First Day of School, which is pretty slow on the learning, even compared to a Last Day Before Vacation Day of Learning.

Second, like VAM, it assumes the power to correctly predict exactly how much learning a hypothetical student in an alternate universe would have achieved. In other words, I can't say that Chris achieved three "extra" days of learning unless I know how many days of learning Chris would have achieved in that alternate universe.

2) We're talking about test scores. That's it.

The scores on Big Standardized Tests that cover just two subject areas are not a  valid proxy for student achievement or learning in general. Reformsters like to talk about "student achievement" because that sounds like a broad, beautiful, whole-child measure. Who doesn't want to see students achieve?

But we're not talking about the whole child or real, robust measures of the complex web of skills and knowledge possessed by complicated individual human beings. We're talking about scores from a single test on two subjects. Even if it's a great test (and there really is no reason to think it is, but let's play the if game), that's not a measure of student achievement.

And when we're talking about tests scores, it's always important to remember one thing-- nobody has ever displayed a convincing causal link between test scores and anything. Nobody has ever filled in the blanks in "When a student gets higher test scores, that will lead later in life to ________" with anything other than unsupported baloney.

3) We're talking about very modest and mysterious gains.

We're talking about a move from the 20th to the 22nd or 23rd percentile. This is not a dramatic increase. This is a blip.

Put another way-- if the school that we decided to close had posted this kind of gain in the last year before closure, would the Fordham been out in front, standing in front of the bulldozers hollering "Stop! No!! You can't close this school because they are showing dramatic improvement!!" Would they declare that this climb in percentile ranking was enough to declare the turnaround of a failing school successful?

Now, the report must feature some complicated math, because the Coming Attractions trailer also claims 49 extra days of learning over three years, which would be a over a quarter of a year-- how does a blip translate to that much learning.

And if the students displayed a two or three percentile grade, does that not mean that other students were pushed down in percentile rankings? Did transferring students from the closed school make students in the receiving school worse?

4) We're ignoring some fairly significant factors.

Even some reform boosters recognize the value of social capital, and there is research to back it up. Ties to community provide a strong and valuable influence over the trajectory of a child's life, and nothing gives a student strong community ties like a neighborhood school.

So ripping a child away from a neighborhood school is not simply, as Petrilli and Churchill put it, "politically dangerous"-- it's a policy that comes with tremendous cost to the community and the students. It's politically dangerous because people understand that it comes at a huge educational and social cost. It cuts the child loose from a community support system while hollowing out the heart of that community. It tells us something about the expectations of American public schooling that we never used to say "community schools" for the same reason we don't order "wet water" in a restaurant. A community base is the foundation of US public education.

In plain English

Fordham is proposing, with plenty of bells and whistles and pretty filigree, that we close community schools, and uproot and disperse the students of those communities, so that they will score a few points better on math and reading tests.

How can this possibly be more cost-effective than investing resources and support in making the community school better? Why not spend every dollar and cent that you were going to spend reconfiguring administration and distribution of students, transportation costs, materials cost and the human cost to the social capital of the community-- why not invest all that in the school you've already got? What are you doing at this magical new school that you could not do at the school that already exists-- without gutting a community, uprooting children, and blasting away whatever social capital you may have?

Closing schools and dispersing the students weakens the community, weakens the forces that are needed to help students rise and advance. It is exactly the wrong thing to do, and therefor, proposing to do so ought to come with a pretty convincing list of large and transformative benefits. Fordham is not making that case, not even remotely.