CBE: Personalized Education & The Indexing Problem

There are plenty of reasons not to like Competency Based Education, which can be found these days shambling about under the nom de guerre "personalized education." It's an appealing name, as it evokes images of a student with her own personal tutor and guide, her own educational concierge. Instead, it's actually a student strapped to a personal computer screen watching a parade of adaptive software unspooling before her. As I said, there are lots of reasons not to like this, but we can skip past the philosophical issues for a moment and consider some of the technical challenges of a truly adaptive, personalized piece of educational software.

Let's talk about the indexing problem.

For over a year, my daughter worked for a start-up company that was creating a huge searchable database of art. Her job, along with several other art history degreed folks, was to index and tag all the artwork in the world. The concept would be that one could search from artist to artist, finding the creators and creations that connected in some meaningful way to the stuff you already knew and liked.

Lots of websites have tried to crack the recommendation code. Amazon tells you that if you bought this, you might like to buy that. Netflix will try to tell you what you might like to watch. Pandora will tell you if you want to listen to this artist, you probably also want to hear that artist. And iTunes genius will take one song and build you a whole playlist of songs that you'll want to hear with it.

All of these recommendation systems depend on a massive system of indexing and tags. If the software thinks that you probably like "Good Vibrations" because you like sixties pop music, it may rustle up some Monkees, but if it believes you like the song because of its acid rock qualities, you may be recommended some Iron Butterfly. If it thinks the salient quality is vocal harmonies, you may get a Mitch Miller record next, but if it thinks the theremin is the key, you may find yourself listening to Bernard Herrman's score for The Day the Earth Stood Still. How the software indexes and tags things (and how it weights those tags) makes a huge difference in what it thinks you want next.

If you've worked with any of these programs, you know they all share one quality-- they don't work super-well. Some are trainable-- if you have a few spare hours or days, you can sit and rate everything you've ever bought from Amazon to give the software a better idea where your preferences lie. But even software that tries to learn about you can be problematic. I regularly research charter schools for this blog, and so my browsers are convinced that I really want to see charter school ads, because it doesn't know the difference between positive and negative attention. I once spent a day trying to hunt down Peter Fox's Schuttel Deinen Speck including lyrics (don't ask) and for a week Google was certain I wanted all my search results to include German language entries.

The problem can be complicated by library size. Pandora does not have an infinite supply of music, and some of the music they do have is more expensive than other parts of their library. So my wife's Sara Bareilles "channel" also includes old Louis Armstrong recordings (the 20s jazz Armstrong, not the 50s easy listening Armstrong). When your iTunes has to try to come up with a good playlist based on just the music in your iPod, the challenge becomes even huger.

And this is before we run all of this past the weirdly specific tastes and interests of individuals. Speaking of Louis Armstrong, he was perhaps the top jazz cat of his generation, and yet, one of his favorite bands to go hear was the exceptionally square Guy Lombardo, who in fact could sell out Harlem's Savoy Ballroom. The connections that bind together an individual's tastes and preferences are often mysterious and elusive.

Companies have spent millions of dollars trying to solve the problem-- Netflix famously offered a prize to anyone who could improve their recommendation engine.

So what does this mean for personalized adaptive education?

First, to come up with the right personalized recommendation for the student, the bigger the library of possible assignments and modules, the better. After all, if we're saying, "Well, Pat, you've finished Module A, so let's check the software's recommendation and see if you should do Module B or Module C" isn't very personalized. True personalization calls for a near-infinite number of possible paths. If we've just got ten or twelve possible paths, that's not personalization-- it's just tracking.

But once we have hundreds of modules containing hundreds of cyber-worksheets or adapted learning activities or whatever we're going to call them, we need an absolute kick-ass indexing system, and we need an analytical engine that can figure out what the indexing system is telling us. What are the important qualities in Module A that tell us which module Pat should do next? Every indexing tag is another variable-- how will we determine which variables are the important ones? In the reading module, was it the vocabulary and if so, which words) or was it the topic-- and if it was the topic, what about the topic? Was the reading about race cars effective or ineffective because Pat likes cars, or because Pat likes things that go fast, or because Pat has an uncle who both races cars and hunts elk? Because knowing would be the difference between a next module about fast jets or classic Studebakers or hunting elk. Did Pat respond to the sentence structure or the paragraph length. For that matter, given that Knewton seriously promised us we would know what to eat for breakfast the day of a math test, do we have to cross-index Module A against what Pat ate and who Pat played with? Have we even talked about the effect of typography on Pat's reading interest and comprehension (I'm not joking)? Did Pat respond to the voice of the writer, and would most respond to another piece by that same writer regardless of any other factors-- and how exactly will we index the different qualities of a writer's voice?

I'm just warming up, but you see my point. To really truly index each module and each assignment within each module would, done well, take a gazillion person-hours and be complicated beyond belief-- then to come up with an artificially intelligent engine that can sort through all of those index tags and cross-reference them against the gazillion other pieces of content in its vast library.

Of course, you know who's really good at analyzing a human beings tastes and preferences and sorting the important details from the less important ones? That's right-- another human being.

You can hire my daughter and some other experts to spend over a year to catalog and index and just generally feed stuff into a big computer program-- or you could sit down with her and she could ask you some questions and personally come up with some recommendations for you. You can let genius put together playlists for you, or you can let someone who knows music and knows your tastes make you a playlist (which lacks the romance of a mix tape, but hey-- technology marches on).

You could try to create an enormous library of instructional modules with a gigantic and complex web of indexing and analytics. Or you could create a semi-large-ish library of units and a pretty small, superficial indexing system  and just kind of half-ass the whole thing, then try to cover it with a whiz-bang sales pitch.

Or you could just hire a competent, well-trained, knowledgeable professional human to be the classroom teacher.

I recommend that option, personally.