Sunday, August 7, 2016

Reuters: SAT, ACT, and Test Insecurity

A Reuters investigative team has been taking a look at the ACT and SAT testing industry, and finding a huge mess. We had already seen hints of the problems with, among other things, whistleblowing posts from SAT insider Manuel Alfaro. But this Reuters series, now at five articles plus sidebars, is sort of jawdropping.

The articles have maintained a remarkably low profile, so I'm going to give you links to all five with a short peek at each so you can pick and choose your faves. Bookmark this-- it may take you a while to work through all of these, but it's worth it.

Part One: Multiple Choices

Turns out the SAT has been breached, big-time and many times, overseas. And the College Board knew it. And they went ahead and used the compromised tests anyway. These "content thefts" is eastern Asia are a huge part of that regions test prep industry. Further, the investigation shows that the College Board knew that a Chinese website was the source of much leakage, but they still failed to limit seatings at Chinese administrations of the test (it would have cost them over a million dollars in revenue). Most interesting takeaway here-- the highly compromised nature of the SAT in Asia suggests that US students might be losing out on college spots to Asian students who have cheated for high SAT scores. And you know it's bad when the ever-hubristic David Coleman chooses not to comment.

Part Two: Cheat Sheet

Security for the new SATs released in March lasted roughly five minutes. The traditional low-tech solutions were used, of course-- test prep companies waiting outside test sites to ask students what was one the test. The internet was also not kind to SAT security. But the College Board's antiquated and long-porous security measures also buckled immediately. Chinese tipsters showed Reuters whole chunks of the test that had been hacked from College Board's computers. This is particularly damaging because the College Board routinely uses the form of the test given in the USA in overseas countries. And all of this despite being repeatedly warned that their security was not holding up. These security issues were clear before David Coleman implemented his "beautiful vision" of a new SAT. The College Board should have been ready to protect the new test. They weren't.

Sidebar: College Board Responds

Having been caught with their cyber-pants down, the College Board offered their own response, which breaks down basically to

A) Hey, nobody's perfect and we're working real hard on this stuff

B) We're totally working with Reuters on this because we certainly have nothing to hide

C) Look at how many people are taking our test! We makin' the money!

Part Three: Deception 101

Well, this is a new one on me. There is apparently an entire Chinese underground industry that helps students cheat to get into US schools, and then helps them cheat to get through the US schools. The story focuses on the University of Iowa, where Chinese nationals receive messages from a coaching service that will help them with homework, papers, and even take their exams for them. But Iowa is not even close to the only school where this goes on.

Part Four: Special Access 

The Global Assessment Certificate Program is supposed to help non-US students develop the skills to succeed in US schools. Turns out it also gives them an early look at the ACT so they can better succeed on that as well. The program costs about $10,000 for a student enroll, and says Reuters, "has emerged as one of many avenues in Asia used to exploit weaknesses in the U.S. college admissions process."

Oh yeah. And the GAC is owned by ACT. Reuters talks to a student who practiced the actual ACT he took a week before he actually took it. And they talk to a former GAC teacher who was sacked over complaints that he was cracking down on plagiarism and cheating. And the ACT has recently benefited from the SAT's belated attempts to create some semblance of test security in Asia.

Part Five: At Risk

When the College Board set out to redesign the SAT, they hired a consultant who told them, "Your security sucks. You need to fix that crap." (I'm paraphrasing). But that didn't happen, and a month after the new SAT was unveiled, someone came to Reuters with hundreds of leaked and/or stolen test items. Reuters sent them to the College Board to ask, "Are these real?" The College Board replied via attorney, saying that publishing the items would be Very Bad (presumably the implication was "sue-ably" bad).

Part of the issue can be laid on a procedural shift. Previously, test-manager ETS had housed test development, test items and the question bank, but under David Coleman, more of this work and storage was done in house. So, less "lock these nuclear codes in the super-secure bank vault" and more "I'll just put this in the locked drawer in my desk."

Bottom Line

There may be more to come, but this sure seems like plenty. For a guy whose "beautiful vision" of the test was one that related to the real world, David Coleman sure seems to have bungled the real world problem of test security. And how does anybody do business in China and not realize things are different there. Even Bill Gates eventually figured it out over a decade ago-- charge the Chinese too much for your intellectual property or designs, and they will just steal it and make it themselves for cheap.

The solution to the College Board's problems is simple-- they just don't like it. Most of these issues get much if every single SAT administration involves a completely different test. But the College Board doesn't want the expense involved in generating that much test material.

So here we sit, with an SAT that is increasingly useless and pointless, yet which has successfully sold itself to some states as a test for every single student in school. Coleman has, for the moment, converted his junk into highly profitable junk, but if the wheels keep coming off the car, he won't be able to drive it forever. Stay tuned. Meanwhile, two more prestigious universities announced they were dropping a portion of their SAT requirements.

No comments:

Post a Comment