More Bogus Research from Rocketship

When you need some "research" to pump up your ad copy, what better than to just hire it yourself. In fact, why not hire the guys who have been pumping it out reliably for you for years.

Rocketship Academy is patting itself on the back over a new "research" report that it commissioned from good, old SRI International, a group that just happens to be heavily invested in technology-based education. SRI used to stand for Stanford Research Institute, but separated from Stanford in 1970 and changed its name in 1977 (which suggests the split was plenty amicable). That was long before Preston Smith tacked his Teach for America credits onto his resume and looked to enter the lucrative world of edubusiness, but since then, SRI has teamed up with Rocketship to show the awesomeness of the Rocketship Academy product.

Way back in 2011, SRI did a super-duper study of the K-5 Rocketship to show that the Dreambox program (Dreambox because it's what you bury dreams in?)  raised scores on the NWEA MAP test by a couple of points. Dreambox is a company that Reed "Elected School Boards Stink" Hastings (Netflix) bought for the Charter School Growth Fund, a fund that also invests in Success Academy and Rocketship Academies. We could play "Follow The Incestuous Privatization Ties" all day, but we'll pass for the moment. We'll also let the usefulness of NWEA results ride for a moment (though you may recognize the name from the successful boycott of the test by Seattle teachers). SRI was back with another glowing report in 2014 and were touting further awesomeness in 2015.

It's almost as if SRI was a corporate partner of Rocketship rather than an "independent nonprofit research center," and indeed, way back in 2010 we find Rocketship then-CEO John Danner  (now CEO of Zeal-- more in a second) explaining in an interview that they are teaming up with SRI

 Next year, with SRI’s help we’re going to instrument Rocketship to be a test lab where we can measure the effectiveness of every online curricula for elementary schools.

Funny side note. Zeal started out as one more adaptive instructional software start-up, with money from NewSchools Venture fund and other folks interested in a good ROI on their edubiz dollars.Visit their site today and you'll find them headlining "live, on-demand coaching" from "real coaches." So I guess Danner has lost a little of his faith in soft-ware based education.

Fine. Whatever. What about that New Research?

So Rocketship's use of SRI as an independent evaluator is about as fishy as the "volunteer" pulled out of the audience by the bad magician at the company picnic. How about the actual research? Does it look plausible anyway?

Short answer: not so much.

What the research claims is that Rocketship middle schoolers have gained a full year of learning over their peers.

After controlling for demographic differences and other key variables, SRI found that Rocketship alumni outperformed their classmates on NWEA MAP, a nationally norm referenced assessment, by approximately one year of academic growth in both math and reading. This independent evidence from a world-renowned research institute is a powerful validation of our pioneering personalized learning model. Our incredibly dedicated teachers, deeply engaged parents and purposeful approach to technology are delivering significant gains for our Rocketeers that extend into middle school.

Let me pause a moment to nod at my usual ridicule of the notion of measuring learning growth in years, and especially in parts of years. Which part of the year? Did the student gain an April or a September? The notion that a year of learning represents some sort of homogenous tofu-like spread of steady increase only make sense to someone who doesn't know many human beings. It's a statistical construct, like saying that one car is stolen every fifteen minutes, as if car theft is a regular ticking event by which we can measure time. "How long have we been trapped in this dungeon, Chris?" "I'm not sure, but I think Pat's learning has grown about this much, so I figure around three months."

But let's look at the experimental design.

The study population included Rocketship alumni enrolled in one of seven participating charter middle schools in the San Jose area between the 2012–13 and 2015–16 school years, totaling 625 students. The comparison group of non-Rocketship peers was made up of students who did not attend any Rocketship elementary school and who attended the participating charter middle schools in the same grade level and in the same academic years as the Rocketship alumni in the study, totaling 1,294 students.

There's some other baloney about controlling for demographic characteristics "using propensity score weighting and multiple linear regression" (I just hope the reset the turginator on the framistan before they did all that), but the bottom line here is that they took a bunch of Rocketship middle schoolers and compared the ones who had attended Rosketship elementary school to the ones that hadn't.

From this we can conclude that students who attended Rocketship Academy elementary schools were better prepared for Rocketship Academy middle schools than students who did not.

Other research also indicates that at most Rocketship Acdemies, the sun rose to the East of the school. Early working papers also suggest that water is wet.

Also, grit!

Just in case you think Rocketship is only focused on the quality of their test prep for a standardized math and reading test, they'd like you to know that they are also looking into the hearts of Rocketshippiteers.

Of course, test scores are only one indicator of future success. That is why we embed our core core values, socio-emotional curriculum, and positive behavior interventions and supports throughout our Rocketeers’ learning experience. And while these skills are harder to measure, they still matter tremendously, especially for the student population we serve.

Emphasis mine. And the emphasis is just in bolding because I don't have an emoticon for a dropping jaw. "Especially for the student population we serve"??!! You mean poor minority deficient defective kids? Good behavior, grit, socio-emotional qualities-- the rich white kids don't need that? On the one hand, there's a bit of a point here. We have living proof that in our society, a person born into privilege can display every socially undesirable quality known to humankind and still have a shot at being both successful and also Presidential material. On the other hand, it kind of looks like Rocketship needs to work on its savior complex and its deficit thinking model for approaching non-white, non-wealthy students. But then, wealthy kids are not the Rocketship market, nor are the privileged crowd lined up for a chance to partake of this educational awesomeness. The whole model reeks of a low-cost edu-product that's good enough for Those People. Every once in a while they just forget to avoid saying so out loud.

Also, where Smith wrote "hard to measure," I think that's a typo. "Impossible to measure" is undoubtedly what he actually meant.

Bottom Line?

It's more charter marketeering fluffed up with science-flavored PR filler. It's dishonest and not very useful in adding to a real conversation about meeting educational needs or evaluating the actual impact of charters in general and the Rocketship blended plunk-kidsd-in-front-of-computers model in particular. I'm sure we'll see this thing passed around in the weeks ahead. Do not be fooled.