The Trouble With The College Board's New Adversity Score

The College Board has for years been trying to rescue its floundering flagship, the SAT. The newly announced adversity score is just the latest unforced error from the testing giant. 

Just keep telling yourself that

For almost a decade, the company has been fighting for market share. In 2012, it hired David Coleman, fresh from his work as architect of the language portion of the Common Core Standards. The theory was that Coleman could lead a redesign of the test that would bring it in line with the Common Core, so that students steeped in the new standards would be well-prepared for the SAT. The alignment would also be a selling point for states looking for a high school exit exam, and within a few years, the College Board was lining up states to make the SAT their official test, giving the company a captive market. On top of that, Coleman proudly announced that his new, improved test would be a tool for eradicating social injustice; the test would be a great leveler.  

But a critical part of Coleman’s strategy was to get the new test to market quickly. The new test was well under way by the beginning of 2014. It launched in the 2015-2016 school year.  

There were problems. PSAT scores from the fall of 2015 were late. Test prep experts were advising students not to take the new SAT at all. In an attempt to clamp down on cheating, the College Board implemented some last minute measures for the March 2015 test that added to the confusion and chaos. By 2016, a former employee was publishing concerns about the test, including flawed items and an inadequate development process. Reuters reported on how the rush to redesign had brought Coleman into immediate conflict with some of College Board’s test designers. That report followed a five-part Reuters series in March of 2016 that laid out a whole series of problems, focusing particularly on terrible security problems. 

Meanwhile, colleges are increasingly dropping the SAT requirement, and research continues to suggest that high school GPA is a better predictor of college success than SAT scores.  

Coleman’s notion of the SAT as a means of opening college to all wasn’t working out. A stinging report showed that the SAT redesign had made it harder for female students to score in the higher bands. And after being dogged for years by charges of racial and economic bias, the College Board’s own data (most recently reported in the Wall Street Journal) shows that when you break out the scores by racial subgroups, there are obvious gaps. As a group, white students still score higher on the test than black kids.  

The College Board has tried to level the playing field by partnering with Khan Academy to provide high quality free test prep. This is meant to counteract the test prep advantage that students from wealthy families receive. For example, in Pittsburgh, you can purchase a thirty-two-hour test prep package--for $4,800. Success Academy, the charter chain based in New York City, is currently looking to hire a P/SAT Curriculum Developer to create a full PSAT and SAT curriculum. And of course recent news has highlighted how far some parents will go to bring up those SAT scores. 

The free Khan Academy test prep is supposed to counterbalance all that, promising SAT score gains of 200 points. The problem is that the offer all but acknowledges that what the SAT measures is neither scholastic aptitude nor intellectual ability, but how good the student’s test prep was. If the SAT is only a measure of test coaching, what real purpose does it serve? 

Now the College Board has announced its Adversity Score, and it promises to be another unforced error by the testing company. 

The score promises to incorporate a dozen factors, divided into neighborhood environment, family environment, and high school environment; it does not include race. It could as easily be called a privilege score as an adversity score—on the 0 to 100 scale, over 50 is disadvantaged and under 50 is privileged. A scan of reactions over the last two days suggests that the score has few fans. Bob Schaeffer of FairTest says the adversity score is an admission by the College Board that the SAT is not a “common yardstick” but is “really a measure of accumulated advantage.” Some conservative commentators have been quick to note the David Coleman connection between the College Board and the hated Common Core. Heather MacDonald (Manhattan Institute) appeared on Tucker Carlson’s show to “slam” the score and decry how it upends the proper meritocratic system.  

Most of the critics have a point. If the test is supposed to be a level playing field, and it actually isn’t, then why not rewrite the test instead of creating a new measure to travel with it (and in so doing, suggest that the SAT score itself cannot be entirely trusted)? The score is to be kept secret from everyone but college admissions offices. What is the purpose of doing that, and what reasons will admissions offices have to keep the scores secret—and what will happen when they inevitably don’t? Is the College Board using information that they've collected from students, and does the adversity score thereby violate student FERPA privacy? How will certain high schools react to knowing their privilege rating? The score is supposedly “steeped in research.” Can the College Board even give a hint at what research base is being used to set their secret proprietary formula for computing the student score? And if this is a “scientific” measure, why not fold it into SAT score computations; why hand it to colleges and say, “Just kind of use it as you think best"? I reached out to the College Board for answers, but they did not reply. 

Coleman has offered several outlets versions of this comment: 

“Since it is identifying strengths in students, it’s showing this resourcefulness that the test alone cannot measure,” Mr. Coleman, the College Board CEO, said. “These students do well, they succeed in college.” 

That’s pretty clear. The SAT cannot predict if a student will succeed in college.  

You go to the grocery store and buy a box of macaroni and cheese, and as you check out, the clerk says, “You know, that box doesn’t actually have any cheese in it. Let me give you this.” And they hand you a plastic bag with some cheese in it.  

You ask, “What kind of cheese is this? How was it made? Where did it come from? How much do I add? And what do you mean the box of macaroni and cheese doesn’t actually contain macaroni and cheese?” 

The clerk ignores most of your questions. “Just use the amount that seems right. You know—just kind of eyeball it.” 

You would not go back to that store for macaroni and cheese. The SAT is in trouble, and no amount of adversity score is going to help.  

Originally posted at Forbes