Here Comes Efficacy!

I'm not sure who injected "rigor" into the education conversation in this country, but there can be no doubt who decided that we will now be talking about "efficacy"-- Pearson has made the term the centerpiece of their newest corporate initiative. And they've put a ton of their corporate information about the efficacy initiative on line where we all can get a look.

It's fascinating and rather involved reading. Michael Feldstein at e-literate has a great examination of the whole package; it's lengthy but worth the read. Feldstein breaks down much of the impetus behind the movement and encourages us not to jump to conclusions that Pearson is Darth Vadering things up, the better to see what they've really gotten right and wrong with this.

I went into the site looking for one simple answer-- what exactly does Pearson (and therefor, eventually everyone who deals with them) mean by "efficacy"?

Pearson wrote the book on efficacy, and the book is entitled "The Incomplete Guide To Delivering Learning Outcomes." The book includes a chapter entitled "What is efficacy?" and it is that chapter that I'm going to break down for you. The chapter is about ten pages long, so as I tell my students right before I cover the history of The Great European War in fifteen minutes, I may cut a few corners.

Pearson borrowed "efficacy" from the pharmaceuticals industry, specifically as it relates to "medical interventions" being proven through "systematic trials." They identify it as an "aspiration we intend to work toward," which I kind of wish it weren't because now I'm wondering how that would work. Will we have to form whole new reading groups and now instead of bluebirds and robins we will have white rats and placebos? I know there are lots of ethical safeguards and protections built into medical research, which is itself terrifically important. But the education = health care analogy is not one that I think holds up; still, I'll save that for another day and not wander off into the weeds before we're even to our second paragraph of this thing.

Here's our definition: An education product has efficacy if it has a "measurable impact on improving people's lives through learning."

They go on to explain. It's not enough that a student pass a test-- he has to experience some actual positive improvement in his life.

Pearson realizes this is a high standard, but inspired by the medical profession, they want to shoot even higher. After all some patients keep coming back (imagine) and some doctors are "incentivized" to order lots of procedures, because $$. And here we introduce one of the central shifts involved in emphasizing efficacy-- replacing focus on inputs with attention to outputs.

And then, charmingly, they confess to the hopelessness of their vision. Emphasizing outcomes requires tests we don't have and agreement on the subjective qualities of excellence, which we'll never have. But since all of that is off in the future, in the meantime we'll just have to rely on tests and graduation rates and all the same old baloney, while in the meantime argle-blargle with partners blah blah blah toward a bold vision of brighter blerg.

Next up: the Three Factors of Efficacy! They are

1) The student(s) and his/her-their incumbent level of motivation. (I'm giving them a bonus point for working "incumbent" in there)

2) The teacher and/or the technology with his/her/its capacity to make an impact. (Emphasis mine. Just in case you were worried that Pearson thought we couldn't be replaced.)

3) The interaction or relationship between them.

And as an add-on bonus, we also recognize that time on task matters as well.

"If this mix is right, learning should happen." That's a quote. So, if these things are present, six-year-olds will learn quantum physics and Shakespeare, I guess. Also, pay attention to the research (and they name check John Hattie). Leaders of schools, universities or systems (?) are the carburetors of education, properly mixing the fuel for the efficacy engine. PISA scores are cited as a useful tool, somehow.

"The idea of efficacy has a lot of support across Pearson" which I guess is how you talk about these things when you are a small corporate nation unto yourself. But this Framework of Efficacy section is interesting because it's more about how product groups within Pearson will now jockey for position.

It's a puzzler. How do we benchmark products without a "neat mathematical formula that spat out numbers"? If we had been collecting data about lifelong effects of our stuff, we'd be halfway there, but only a few divisions within the company were thinking that way. We've developed this framework based on Michael's work evaluating government programs back when he worked at 10 Downing.

The framework for evaluating very diverse products had to 1) Be constructive and practical 2) Be forward-looking and 3) Enable comparison. And there's a cool chart about mapping/evaluating outcomes, evidence, planning & implementation, and capacity to deliver. And the chapter finishes up with a look at each of these four areas.

I have to echo Michael Feldstein here-- it's kind of extraordinary that here on line we can see how Pearson is going to manage themselves and how, exactly, they will judge their various product groups. Anyway, here are the four subsections:

Goals. You remember Outcome Based Education.Apparently it's now in-house for Pearson. Tell management not just your sales goals, but what outcomes your product will create for the learners. We frankly admit that meeting this standard has been hard for some managers of pre-existing products. Yup. Pearson is here admitting that some product groups have a hard time explaining what a learner would get out using Pearson's product.

Evidence. Can you find a way to prove your stated goals have been achieved? Look at the research. And since there's not a lot of good research out there, Pearson's research division is going to focus on the parts of the education mystery that we keep not having answers to. We need more data so we can create programs with deliverable learner outcomes.

Planning. Pretty self-evident. How are you going to get this made and sold?

Capacity. This part is either creepy or encouraging, depending on how much Pearson alarms you. It starts with the observation that it's no longer enough to drop off the materials you sold and wash your hands of it, saying, "Hey, we delivered perfectly good stuff. If they screwed it up, that's their problem."

Frankly, this is not news in many other industries. I am a yearbook adviser who works with Jostens Publishing, and like most yearbook companies, if you want the sales rep to come in on a regular basis and walk you through every part of the process, they will do it-- because they want you to feel successful with their program. (They will also stay away if you ask, which I find invaluable).

If you want the encouraging view, it's a picture of a Pearson rep in your school helping you succeed with their stuff. If you want the creepy view, it's a picture of a Pearson rep in your school acting like s/he's an administrator there. Those of you who have already had some variation of this experience know the other picture-- a publisher's rep in your building demonstrating his/her ignorance of how to work with students in a classroom.

There's a rosy conclusion about how this new path for business leaders within Pearson will stride together into a great future of efficacy.

I don't think it's all bad news. Imagine, for instance, if this framework had been applied to the creation and implementation of CCSS? And the notion that I should be able to ask a Pearson rep, "So what how will my student actually benefit from this in his life, and how do you know that?" is kind of exhilarating. Yes, they may not be able to tell me, but if they get to pretend this is their corporate policy, I get to act like I expect them to follow it.

Still, it clearly doesn't address any of Pearson's terminally mistaken assumptions about how teaching actually works or the scary anti-wisdom of Pearson's one-world view. I'm not ready to cheer for Pearson's next onslaught, but I appreciate their sharing their plans for it.