The College Board Tweaked The SAT Adversity Score. But It's Not Fixed-- Or Gone.

Since David Coleman took the helm at the College Board, its flagship product--the ubiquitous SAT, one-time queen of college entrance exams--has been the victim of a series of unforced errors. The roll-out and walk-back of the "adversity score" is only the latest--and recent reports of that score's death may be greatly exaggerated.

The company ran into some glitches in its rush to get a new, Common Core-aligned test to market. Coleman expressed a desire for the test to be a great leveler, a test that would recognize and elevate intellectual prowess wherever it was found. The SAT has long been criticized as being loaded with cultural bias, and the College Board's own data seems to support that assertion. The other knock on the test was that it could be beaten with the help of test prep and coaching (a criticism bolstered by an entire SAT test prep industry). And the College Board has been confirming that these criticisms are valid.

In 2014, the College Board entered into a partnership with Khan Academy to offer free test prep to anyone who wanted it. Rather than designing a test that was immune to test prep (which may, in fact, be impossible), the College Board appeared to be conceding that SAT scores measured, at least in part, how well a student had been coached for the test.

Then came the Environmental Context Dashboard, featuring the Adversity Score. The score was supposed to capture the social and economic background of students through a combination of fifteen dimensions. But though it was supposedly "steeped in research," the genesis of the score remained a proprietary mystery, somehow combining factors from school and community. The result would be a score between 1-100, with a score of more than 50 representing disadvantage and a score under 50 indicating some privilege. Critics attacked the notion of reducing a student's entire background to a single score. They criticized it for being an attack on meritocracy. And most of all, they criticized it for being an admission that the SAT itself is a biased test given on a tilted playing field. Meredith Twombly, vice president of admissions and financial aid, at Clark University in Worcester, Massachusetts, one of the many test-optional colleges in the U.S., had a typical response:

For decades the College Board said the SAT alone is the best unbiased, objective indicator of success and likened it to an equalizer. The creation of the ECD is the SAT proving the point they have been denying.”

Coleman responded initially with variations on this comment:

Since it is identifying strengths in students, it’s showing this resourcefulness that the test alone cannot measure. These students do well, they succeed in college.”

That comes perilously close to admitting that the SAT itself cannot actually predict college success.

Now the College Board has responded to criticism of the ECD and the Adversity Score. The headlines are reporting it as "College Board Drops Plans For SAT Student Adversity Score," but that might oversell the move, which could be described as a tweak and some rebranding.

First, the Environmental Context Dashboard has been given a new name--Landscape.

Next, the College Board has increased the transparency of the product. Students will be able to see their own Landscape information, and the process that generates that information is now somewhat more transparent. There will be a variety of information about the school itself, including information such as how rural or urban the school is, the size of the senior class, number of free-or reduced-lunch students--a batch of information taken from the National Center for Education Statistics (NCES). There will be information about AP classes and tests (the AP test is another College Board product). Landscape will also show how the student's SAT scores compare to other students in that school.

Finally, Landscape will offer high school and neighborhood average indicators. Six factors--college attendance, household structure, median family income, household stability, education level, and crime--will be "averaged and presented on a 1-100 scale." A higher value on the scale "indicates a higher level of challenge related to educational opportunities and outcomes." The two "values" will be kept separate, rather than combined, as with the Adversity Score. The College Board very carefully avoids the word "score," but it certainly looks like a scaled-down version of the Adversity Score.

In its press release, the College Board emphasizes that Landscape doesn't replace any of the information that students supply as part of a college application. It's just trying to provide admissions offices with a bit more context. Said the College Board's Coleman:

We listened to thoughtful criticism and made Landscape better and more transparent. Landscape provides admissions officers more consistent background information so they can fairly consider every student, no matter where they live and learn."

With the dashboard rebranded and the word "score" banished, the same old question remains--if an SAT score reflects coaching, and additional "context" is required to consider a score fairly, then what good is the SAT in the first place? 

Originally posted at Forbes.com