Can Schools Play Catch-Up?

From its launch as Campbell Brown's attempt to be a major education player in anti-public ed politics, The74 has become a very mixed bag. Sometimes they publish valuable journalism about education, and sometimes they roll out junk like this article about using AI to help schools get students caught up, an article mostly impressive in how it manages to get so much wrong in such a little space.

Everyone just run faster than that guy-- go catch up

The piece is by Daniel Weisberg. Weisberg has deep reformster credentials; the former lawyer was First Deputy Chancellor of schools in NYC under Joel Klein and David Banks and is a Broad Foundation fellow. He was CEO of TNTP, the sister organization for Teach for America, where he attacked teacher job protections and oversaw blog-posts-disguised-as-reports like The Widget Effect and The Opportunity Myth that lacked substance and accuracy, but which provided cover for reformsters to act like their ideas were grounded in something other their personal preferences. He's no stranger to controversy, having been implicated in a scandal under Banks/Adams. 

Weisberg has never shown a particularly strong grasp of teaching or education, and this article doesn't break his streak. 

"America's schools are terrible at catching kids up" Weisberg says, a sentence he puts in its very own paragraph to help make it pop.

This is just so dumb. The whole discourse around "catching kids up" is just dumb.

What's the hope here? Let's take a student who is behind by, say, three months of material. So to catch that student up, the teacher needs to get that student through three months' worth of material in one month. 

If the teacher could do that, wouldn't she be doing it already?

Do catch-up fans imagine teachers are sitting there thinking, "Well, I could teach this material a lot faster, but I think I'll just poke along instead." Do catch-up fans imagine that teachers aren't already moving as quickly as they can? 

Guys like Weisberg believe in "intervention programs designed to catch kids up," but if educators knew a swifter, more efficient way to teach that material, why would it be an "intervention program" and not a "regular program"?

But Weisberg never has shown much understanding of actual classroom teaching. He argues that schools are bad at catch-up because teachers are being asked to do the impossible-- but he has the wrong idea about what the impossible is.

In a classroom of 25 to 30 students, teachers must determine who is on grade level, who is behind (and why), how to modify instruction for each struggling child and how to extend learning for advanced students — all while delivering grade-level content.

Diagnostic exams, designed to give educators information on how students are progressing, are infrequent and often test different subject matter than what is used in the classroom. Intervention programs designed to catch kids up are purchased but poorly implemented. Students needing intensive help are sometimes segregated into programs with low expectations and weak outcomes.

Sigh. Not exactly untrue, but all beside the point. Weisberg assumes that a great intervention program and intensive help could somehow cause struggling learners to learn material faster than any other students in the system. He talks about a "roadmap to acceleration," but if we had such a roadmap, why wouldn't we have all students on it (and is it possible we already do). He also connects these problems, somehow, to grade inflation. 

Weisberg thinks he know how to do achieve the great catch up miracle. Let's see. First, this:

TNTP’s study identified 1,400 schools where students consistently learned more than a year’s worth of material annually, enabling those who started behind to reach grade level.

No, they did not. They identified some schools where students scored well on the standardized test of math and reading. When someone starts talking about "1.3 years of learning" they are talking about a certain amount of a standard deviation on a test score. Can intensive test prep bring test scores up? Probably. Do we have a shred of evidence that raising that test score will improve the student's life outcomes? We do not.

Weisberg continues with his bold vision:

In other words, the Catch-up Crisis is reversible. But first. we need a bold, shared goal: that students who fall behind grade level will catch up to — or exceed — grade-level standards within two school years, and without fail by high school graduation.

This is lake Woebegone talk-- we can get all students to be above average. You know what happens when all students are at or above grade level? We start talking about "grade level inflation" and how the standards are too low. 

But Weisberg sees three obstacles to implementing his bold vision: "limited real-time insight into student learning, little evidence-based guidance on how to address specific learning gaps and minimal job-embedded coaching."

Part of Weisberg's issue is a definite lack of faith in professional educators. "Students generate enormous amounts of work daily — assignments, quizzes, writing, projects," he says, as if human children are some sort of assembly line machine and the work they do descends from nowhere. "No human can analyze all of it for 25 students every day." He should meet secondary teachers who do it for 150-200 students. Is it hard? Sure. Do you find ways to manage it without doing it every single day? Maybe. 

But you know what he thinks the solution is-- magical AI that "can surface patterns quickly and provide teachers with usable, digestible insights." Which can also "generate evidence-informed strategies for specific challenges." Here's his example:

Imagine a fifth grader who is struggling with fractions. His teacher knows he earned a C- on the last test but doesn’t know why or what to do to help. AI can analyze the student’s work in real time and discovers he tends to invert numerators and denominators; it draws on data from thousands of similar children to see what worked best to help those with the same misconceptions and recommends content for a 15-minute tutoring block for the teacher to review and revise.

I'm stuck trying to imagine a fifth grade teacher who can't spot a student who tends to invert numerator and denominator (while doing what, exactly?) Where is this data from thousands of similar children? And how would AI know what worked best? And on what planet do you find a fifth grader who can be retaught successfully in a fifteen-minute block of time? 

Weisberg's working with a manufacturing model here. The assembly line is turning out a flawed product, so we examine all the data from the equipment and figure out how to correct the problem. But there are so many steps in this process that raise huge questions. How did the AI collect data from thousands of students-- did they agree to have every step of their classroom work monitored and recorded, and why is this data available all across the country? Also, given that AI does not actually think or understand in any human sense of the word, how was the instruction modified and shaped so that the AI could spot patterns in a useful way? 

Also, I love that AI-in-education folks always turn to math for examples (even though chatbots are notoriously bad at math). What if the student is having trouble analyzing figurative language in Shakespearean sonnets? What if the student is behind because they were supposed to read The Great Gatsby and they just, you know, didn't? 

Weisberg also wants to deploy AI to coach teachers. "AI-supported coaching tools, used responsibly, could provide timely, standards-aligned feedback on recorded lessons, supplementing human coaching rather than replacing it." Never mind "Teach like a pirate"-- now you can teach like a robot. This dovetails nicely with the suggestions for students, all of which add to the offloading of professional cognitive work for teachers. I wonder how long it would take the AI to deskill the actual human teacher.

Weisberg name-checks some companies doing some pilot work and claims some of these are seeing significant progress, but he only links to corporate sites-- not any "evidence-informed" support.

Weisberg nods to the ideas that teachers should still make final choices and also maybe the district better figure out how badly this adds to their too-much-screen-time problems. So he gets a half a point for that.

But mostly this is one more case of over-promising that AI can do something it can't actually do and maybe we shouldn't be trying to get it to do in the first place and, most of all, that can't really be done. He makes the mistake of imagining that teaching is engineering (read Russell Barkley on being a shepherd rather than an engineer), a view that is doubly problematic as it treats students like pieces of sheet metal waiting to be fashioned into a shape of management's choosing. Students get no agency or choice in his vision.

And all of that in service of the notion that if a runner is lagging in a race, they just need to be properly directed to run faster (faster even than those in the front of the pack) so that they can catch up. No, thank you.