Monday, March 23, 2015

Who's Scoring the Test?

A sharp-eyed BAT spotted this ad, one more example of a genre that has become almost cliche-- the Craigslist advertisement for test scoring work.

Do You Have a College Degree?

Thank you for your interest in employment with Measurement Incorporated. We are a diverse company engaged in educational research, test development, and scoring tests that are administered throughout the world. Our company has grown to be a leader in the industry by providing consistent and reliable results to our clients. We are able to do this through the professional efforts of a flexible staff, and we welcome your interest in becoming a member.

Measurement Incorporated boasts ten scoring centers, which is a good thing because "to guarantee test security, all work has to be done at one of our Scoring Centers in Tennessee, Michigan, Florida, North Carolina, Kansas and Washington or from a secure work station in your home."

The ad, which went up ten days ago, is part of a recruiting drive for the test-correction high season of March and April. "These projects may include scoring test items in reading, math, science, social studies, or written essays. The tests come from many different states representing students at all grade levels."

The job starts at $11.20. After logging 450 hours, workers are eligible to bump up to $11.95. Day and night shifts are available, and workers are expected to put in five days a week.

Their employment website provides more details of the job in language that is apparently designed to sift out the better-educated prospects who can speak corporate balonese:

Within the field of performance assessment scoring, MI has distinguished itself by relying on an extensive and disciplined approach to training and monitoring a carefully selected workforce of qualified readers that is unparalleled in the industry. Due to the seasonal nature of scoring, MI hires and offers paid training to hundreds of temporary, highly-skilled, and well-educated employees to score tests on a project-by-project basis while maintaining strict guidelines for accuracy and quality control. 

Commenters on the job-rating site indeed made many comments about the seasonal nature of the work and the fact that it was an unreliable income. While they were mostly positive about working for MI (can I be amused that the corporate initials just take me back to a million medical tv shows and myorcardial infarctions?) it's clear that this is not a line of work for someone who has a real job. I suppose the labor pool of well-educated college grads who can't find a real job is fairly well-stocked at the moment, if employment ever does pick up in this country, the test-scoring industry could be in trouble. Well, more trouble.

This is, of course, one of the great undiscussed and unsolved issues of national-level Big Standardized Tests-- who the heck is going to grade it? The question is not just qualifications, but quantity-- how do you round up the human hours needed to score several million tests?

So how does MI stay in business if they're doing seasonal work. The NC-based company has some other products, including a service for providing test items and an AI program named PEG for assessing writing, because the computer-graded essay-scoring field can always use one more program that can't actually do the job. MI also has a writing instruction program; maybe I'll look at that another day.

I expect we'll continue to see many of these smaller companies scarfing up sub-contracts for the Big Guys and handling the business of hiring part-timers to help make decisions about the fate of America's children, teachers, and schools. Only one of two things can be true here-- either the system is so simplified and so user-proof that it doesn't really matter who's doing the scoring work (in which case it's a dopey system that gives back very little information and is easy to game) or it does matter who's doing the scoring (in which case, the use of part-time temps who are available only because they couldn't find a real job is not exactly comforting). Either way, this is one more big fat reminder that the Big Standardized Test is a dumb way to assess any part of America's education system.

4 comments:

  1. A sharp-eyed BAT...you crack me up with your sly word usage, Mr. Greene. Also, the fate of everyone giving or doing the tests depends on people doing scoring only until they get their real job, what could go wrong?

    ReplyDelete
  2. I'm betting this company is also owned by Pear$on (they realize that everyone's on to them), because the ad wording is suspiciously similar.
    Not one thing has changed since Todd Farley's 2009 book was published.
    Time to cut off the head of the monster & point the finger at the perp by letting EVERYONE know just how much of their education funding taxpayer $$$ are being WA$TED on this child, parent & teacher abuse.

    ReplyDelete
  3. Measurement Incorporated is not owned by Pearson. They are an employee owned company.

    ReplyDelete
  4. Looks like they have had their share of problems. Of course they could always claim “We’re Better than Pearson”

    http://www.nytimes.com/2011/04/09/nyregion/09tests.html?_r=2&scp=1&sq=sarah%20maslin%20nir&st=cse

    http://www.myajc.com/news/news/scoring-errors-jeopardize-tests-poor-oversight-rai/nZ27M/

    http://www.eeoc.gov/eeoc/newsroom/release/1-6-11.cfm

    ReplyDelete