Tom Vander Ark and the End of the Big Test

It's good to step away from present arguments from time to time and look at what folks said about particular issues back when they were laying groundwork. And when it comes to competency based education, it is always instructive to set the Wayback Machine and travel back through the writings of Tom Vander Ark.

Vander Ark has been a prolific and active advocate for reformsterism. He graduated from the Colorado School of Mines (on a football scholarship) with a BA in Mineral Engineering (1981), then moved on to University of Denver for an MBA in energy finances (1984). Then from 1986 to 1993 he was Vice President of PACE membership warehouse, which was supposed to be K-mart's answer to Sam's Club.  From there, Vander Ark went on to a short stint at Capgemini, a multinational consulting group.

And then, somehow, Tom Vander Ark became the superintendent of Federal Way Public Schools in Washington state. It's entirely possible that somewhere in his prodigious volume of writing, Vander Ark has explained how a guy with zero education background gets hired as a school district superintendent, but I haven't found it anywhere-- the vast ocean of glowing press always recaps his career as if it began as a school superintendent. And he was hired in 1994, before every little piece of news was dutifully transcribed for the internet.

In some respects, that job really was the beginning of Vander Ark's career, because he leapt into tech in a big way, starting an internet academy, backing a high speed network (which in the late nineties was no small feat), and threw his weight into raising test scores. After five years as superintendent, he was hired by the Bill and Melinda Gates Foundation as executive director of their education initiatives. In those days, that meant being a cheerleader for small schools, the initiative that Gates was sure would change education until he decided it wouldn't, after all, and just dropped it.

After that, the sky was the limit a nd all sorts of folks wanted to work with him. Among the more notable were Edmodo, the Technology Alliance, and iNACOL, the big time boosters of on-line learning (Vander Ark was a director for six years). He has also worked with investment groups like Learning Capital, tech companies like Bloomboard, and is currently still a director with eduInnovation, a group that "aims to spread innovations through thought leadership campaigns" because there is just nothing like a really good thought leader. (He also works with Charter Board Partners, an organization that helps charter schools rustle up some board members-- presumably nice, cooperative ones).

Bottom line-- Tom Vander Ark is a reform hipster, a reformster before reformsters were cool. He has been talking down public schools for decades, and he has been pushing technology as a path to privatization for decades as well. While folks in the ed debates have been wrestling with flashy edutourists like She Who Will Not Be Named and Arne Duncan, Vabder Ark has been steadily laying out what The Next Move would be, and for the last few years, his eye has been firmly focused on competency based education.

You can find his thoughts laid out with a regularity that puts even certain bloggers to shame on the blog for the organization that is his personal brand, Getting Smart, and I could spend the rest of the week sitting here sifting through it all. But let's look at this particular post, "The End of the Big Test: Moving to Competency Based Policy." (I should note that this is from a little more than a year ago)

Here's the rationale for competency based education.

After noting that just about everyone is sick of the Big Standardized Test, Vander Ark leaps right in by quoting Clayton Christensen, the king of disruptive innovation., who reminds us that we should stay focused on the heart of adding value (because, I guess, students are toasters that we are just adding value to) which is the job to be done. In the case of testing, Vander Ark sees four jobs to be done:

1. Inform learning: continuous data feed that informs students, teachers, and parents about the learning process.
2. Manage matriculation: certify that students have learned enough to move on and ultimately graduate.
3. Evaluate educators: data to inform the practice and development of educators.
4. Check quality: dashboard of information about school quality particularly what students know and can do and how fast they are progressing

#1 is pretty vague, and might cover helping teachers tweak their instruction (a popular reformster test justification). But nowhere do we see anything about helping the student learn. The best assessments are part of the learning process, but the corporate view that Vander Ark typifies doesn't see this. It is true that on an assembly line, the process of measuring throughput and inspecting the product and counting the number of toasters come through-- all of that is separate from the actual process of making the toaster. This is not true in education-- done properly, the measurement of learning is part of the process of learning, in fact helps with the important meta-learning necessary for students to become self-regulating life long learners. This doesn't fit the corporate manufacturing model because, among other things, toasters do not inspect themselves. But like most corporate reformsters, Vander Ark is working with the wrong model, and so he comes up with incorrect ideas, like a man learning to operate on a human by practicing on a squid.

Vander Ark notes that advances from psychometricians and lawyers (!) have led BS Tests to get longer and longer in an attempt to meet all four requirements. But he notes that six things have happened since The Old Days.

* Student internet access
* Performance assessment tools make it easier to manage a bunch of assessment stuff
* Assessment stuff can be embedded in digital materials
* Formative assessment systems (he means the digital ones) have been improved
* Adaptive assessment is widely used (see MAPS from NWEA)
* We're looking for measures of non-cognitives that don't fit on BS Tests

In short, we have computers that can do all sorts of cool things. We can put together a giant library of assessment tasks. We can embed that stuff in every digital assignment. We can track it all digitally. We can do adaptive assessments with computers, and computers can measure character traits.

The first problem with this is that we don't really know that computers can do these things. I've talked about how the sheer task of compiling and indexing the kind of library of tasks needed for adaptive learning seems... daunting. Computer ability to "score" assessments is severely limited (no, they still can't actually assess writing).

The second problem is that Vander Ark, like many corporate reformsters, seems to ignore that this system has to work with live human students. It's a common reformster error-- they just kind of assume that since all of their stuff is real important to them, when they plunk it down in front of a bunch of thirteen-year-olds, those students will also take it all super-seriously. All reformsters need to sit in a room and watch students get through a computer practice module by just hitting buttons quickly in order to make the program be done as soon as possible.

The third problem is that he assumes that being able to do these things is the same as being able to do them well or effectively. But that's in part because

Fourth problem-- he is looking at all of this from a systems technocrat viewpoint. The question at the core of all of this is not how can we best help students learn, but how can we best meet the needs and requirements of this cool technosystem.

So what's the plan?

First, says Vander Ark, the states must be convinced to throw out Carnegie units and replace them with "competencies aligned to state standards." When you can do X, you are ready to move on. He notes that several New England states are moving toward proficiency based diplomas. And he lists the five items that iNACOL says must be in place for CBE to happen. See if any of these sound familiar in your neighborhood:

*     Students advance upon mastery.

Competencies include explicit, measurable, transferable objectives that empower students.

Assessment is meaningful and positive learning experience for students.

Students receive timely, differentiated support based on their individual needs.

Outcomes include application of knowledge and development of important skills and dispositions.

Well, at least, it involves these five things to the extent that they can be managed by a computer program. Competencies include objectives that are measurable by a computer. Assessment is a really meaningful computer file. Students receive differentiated support to the extent that the computer software can differentiate. Outcomes include displays of hoop-jumping that can be measured by the computer.

End of course options include either an old school test, a college test like the ACT, a "body of evidence" portfolio approach, or a "compilation" of all the many mini-assessments from throughout the course. "The test based options tend to be more reliable while the student work product approaches are more valid and authentic."

Call me crazy, but options three and four sound like exactly what classroom teachers do right now-- look at the full body of our students' work and wrap it all up into a final overall assessment. So why are we even having this conversation. It seems that we have arrived at a good argument for just letting teachers continue to do their jobs.

Could it be that my option is not desirable because it does not provide any sort of business opportunity for venture fund capitalists? Because we don't have to buy anything or hire any consulting firms or otherwise unleash an extra boatload of money to do it?

I am inclined to think yes. Vander Ark's plan is a business plan, not an education plan, a business plan that assumes that we must replace teachers with software because otherwise how are we going to creatively disrupt all those beautiful, beautiful dollars loose? Vander Ark's plan says we want to displace teachers with technology because we can, because we must, because my conclusion is actually my premise. We have entirely skipped the part where we discuss whether or not these technosolutions actually work in general, and in particular whether or not they work better than the system we have now.

I know Vander Ark's answer-- currently public schools are terrible and must be burnt to the ground with fire. All right, I may be overstating his case, but that's the basic idea. We will say that public schools are failing, though we can't offer evidence, and we will say that CBE and technobased approaches to school will be better, though we have  no evidence of that, either. Mostly it just looks as if we're saying, "Man, if we could convert the entire education system to a computer-centered magical instruction system, that would be the most amazing business and investment opportunity ever."

Vander Ark has seen for a while the end of the big test, and he sees the end of the big test as the beginning of the big bucks.