Who's Scoring the Test?

A sharp-eyed BAT spotted this ad, one more example of a genre that has become almost cliche-- the Craigslist advertisement for test scoring work.

Do You Have a College Degree?

Thank you for your interest in employment with Measurement Incorporated. We are a diverse company engaged in educational research, test development, and scoring tests that are administered throughout the world. Our company has grown to be a leader in the industry by providing consistent and reliable results to our clients. We are able to do this through the professional efforts of a flexible staff, and we welcome your interest in becoming a member.

Measurement Incorporated boasts ten scoring centers, which is a good thing because "to guarantee test security, all work has to be done at one of our Scoring Centers in Tennessee, Michigan, Florida, North Carolina, Kansas and Washington or from a secure work station in your home."

The ad, which went up ten days ago, is part of a recruiting drive for the test-correction high season of March and April. "These projects may include scoring test items in reading, math, science, social studies, or written essays. The tests come from many different states representing students at all grade levels."

The job starts at $11.20. After logging 450 hours, workers are eligible to bump up to $11.95. Day and night shifts are available, and workers are expected to put in five days a week.

Their employment website provides more details of the job in language that is apparently designed to sift out the better-educated prospects who can speak corporate balonese:

Within the field of performance assessment scoring, MI has distinguished itself by relying on an extensive and disciplined approach to training and monitoring a carefully selected workforce of qualified readers that is unparalleled in the industry. Due to the seasonal nature of scoring, MI hires and offers paid training to hundreds of temporary, highly-skilled, and well-educated employees to score tests on a project-by-project basis while maintaining strict guidelines for accuracy and quality control. 

Commenters on the job-rating site indeed made many comments about the seasonal nature of the work and the fact that it was an unreliable income. While they were mostly positive about working for MI (can I be amused that the corporate initials just take me back to a million medical tv shows and myorcardial infarctions?) it's clear that this is not a line of work for someone who has a real job. I suppose the labor pool of well-educated college grads who can't find a real job is fairly well-stocked at the moment, if employment ever does pick up in this country, the test-scoring industry could be in trouble. Well, more trouble.

This is, of course, one of the great undiscussed and unsolved issues of national-level Big Standardized Tests-- who the heck is going to grade it? The question is not just qualifications, but quantity-- how do you round up the human hours needed to score several million tests?

So how does MI stay in business if they're doing seasonal work. The NC-based company has some other products, including a service for providing test items and an AI program named PEG for assessing writing, because the computer-graded essay-scoring field can always use one more program that can't actually do the job. MI also has a writing instruction program; maybe I'll look at that another day.

I expect we'll continue to see many of these smaller companies scarfing up sub-contracts for the Big Guys and handling the business of hiring part-timers to help make decisions about the fate of America's children, teachers, and schools. Only one of two things can be true here-- either the system is so simplified and so user-proof that it doesn't really matter who's doing the scoring work (in which case it's a dopey system that gives back very little information and is easy to game) or it does matter who's doing the scoring (in which case, the use of part-time temps who are available only because they couldn't find a real job is not exactly comforting). Either way, this is one more big fat reminder that the Big Standardized Test is a dumb way to assess any part of America's education system.