Wednesday, July 19, 2017

Did SAT Unmask Grade Inflation?

The story was carried by USA Today and rapidly picked up by much of the Kids These Days press-- the good people at the College Board have discovered rampant grade inflation as illuminated by the SAT, as witnessed by variations on this lede:

Recent findings show that the proportion of high school seniors graduating with an A average — that includes an A-minus or A-plus — has grown sharply over the past generation, even as average SAT scores have fallen.


Like many education stories, this puts some thins together that have nothing to do with each other. Let's pull apart the pieces, shall we?







SAT Score Dip

The span discussed is 1998 to 2016. During that span, the average SAT score (on the 1600 scale and without the worthless writing portion) dropped from 1,026 down to 1,002. So, just a few questions we need to answer to know how exciting this is.

First, is a 24 point dip significant? That's hard to track down. This source from the College Board suggest it would mean about 3 percentile points. But folks at Fairtest (who don't work for the College Board) peg the margin of error at 60-- so the "drop" would fall well within the margin of error.

Second, are the populations who took the test comparable? That's a no. Since 1998 the folks pushed to take the test have grown, especially in states like Illinois where every student must now take the test. So over the span we've added way more students who would not have taken the test in 1998, which would tend to lower the average.

Third, did both populations take the same test? Again, no. The College Board under new boss, Common Core creator David Coleman, has worked hard to revamp its signature product over the past decade, so we are comparing scores on two entirely different tests.

Conclusion? This reported "fall" of SAT scores is a big fat nothing burger with a side of self-serving PR sauce.

Reminder about The College Board

Despite its high-sounding name, the College Board is a business, and one of the products it sells is the SAT test. Sales of that test depend on students buying the chance to take it and college buying into the notion they need it to help make a decision.

With that in mind, note this quote from Michael Hurwitz of the College Board, the lead researcher on this little project:

He said one of the goals of the research is to "make sure that college admissions professionals are equipped to make the best decisions possible.”

Hurwitz also calls the grade inflation piece of the findings "really stunning." We'll get there in a moment, but here's what you need to remember. The number one rival of the SAT is not the ACT-- it's student GPAs. Studies show that high school GPAs are a far better predictor of college success than SAT scores, so who even needs SAT scores at all?

It is absolutely in the College Board's financial interest to try to erode trust in high school GPAs, just a surely as it makes sense for Ford to suggest that Chevy's aren't reliable. Hurwitz is not an impartial observer without a stake in the outcome of this research. This is the tobacco institute doing research on healthy lifetsyles (You know, jogging can be very harmful for you).

Grade Inflation

Recent findings show that the proportion of high school seniors graduating with an A average — that includes an A-minus or A-plus — has grown sharply over the past generation, even as average SAT scores have fallen.

In 1998, it was 38.9%. By last year, it had grown to 47%.

That's USA Today again, and boy would I like to see the specific research behind both of those numbers, because I sure don't personally know any school where half the graduating class has an A average. So I have some huge questions about where those numbers come from.

Am I going to swear up and down that grade inflation doesn't happen? No, I'm not. But it's hard to believe there could be that much (I am sure someone will correct me in the comments). But do parents push for better grades? Do students put themselves in lower-than-challenging levels (particularly in senior year)? Do some special ed supervisors declare that students with special needs must get a passing grade? Yes (but not usually an A!)

It also occurs to me that in the age of differentiation, we make much more effort to match the course requirements to the student's ability, which might lead to better grades all around. 

But I'm not going to defend grade inflation-- it's not a good thing where it happens. I'm just highly doubtful that it happens this much. And when it's being reported by someone who has a stake in discrediting grades (see above), I'm extra doubtful. I guess I'll have to wait for the book.

And as the story rips around the conservative media world, that's just how it's playing. Participation trophies for all the little snowflakes-- that's why we need leadership that will kick ass and take names and give really low grades (just not to my kid). It's the damn teachers union.

So I wish USA Today had reported this story a little more thoughtfully. Reporter Greg Toppo does report that Hurwitz is with the College Board, but doesn't really examine what that means in this research. That's a disappointment. Just remember to fill in those blanks for all the people who are about to confront you with this story and say, "So, what are all you lazy cheaters up to, anyway?"

9 comments:

  1. Great blog. College Board researchers (sic) handed Greg Toppo material that he could publish in USA Today. Why did he not ask a few questions before biting into the shiny apple?

    ReplyDelete
    Replies
    1. Fake news and alternative facts...isn't that the new mantra?

      Delete
    2. Sounds like David Coleman isn't happy with profits these days with all the scandal about the College Board and all the colleges figuring out that SAT and AP aren't really good indicators of good college students. Time to spin the web and place some blame elsewhere. Problem is that so many people believe the lies and believe that those over paid and lazy teachers (who only work 9 mos out of the year) are the problem. It's the lie that's been told so much that people are stupid enough to believe it's true.

      Delete
  2. Yeah, I don't know of any schools where half the graduating class has an A average either (except maybe private schools), so I also would like to know the methodology. I doubt that there's any more grade inflation than there ever was.

    This does make me think of how when I was at Miami University of Oxford, Ohio (the self-acclaimed "Ivy League School of Ohio" or something like that)in the early 70's, the president told the professors they were giving too many A's and they had to start giving more C's or people wouldn't believe it was such a premier school. So grades were simply seen as a PR product.

    ReplyDelete
  3. Peter, I'm 60 years old and the same things were being said in my high school days. The angst over dropping SAT scores, the grade inflation (at that time, it was claimed that teachers deliberately inflated grades so young men could get student deferments and avoid going to Vietnam via the draft) ... la plus que ca change, la plus que la meme chose.

    ReplyDelete
  4. Interesting...years ago in the mid 80s in a report that can still be found in the college board archives that says the changes from the 60s to 80s we're all due to the shift in who took the test. Why is the college board shifting o the education fads more so now?

    ReplyDelete
  5. The debate (such as it is) about grade inflation is a little different at the college level, but it often goes like this when people accuse me of it.

    Middle-class white collar manager person: You inflate grades!
    Me: Why would I do that? What's my motivation?
    MCWCMP: To boost their GPAs so their job applications look better.
    Me: Do you look at GPA when you hire people to work at your company/non-profit/accounting service?
    MC....: No, not really. I don't think it means very much.
    Me: So then why are accusing me of doing something terrible when you agree that the outcome is irrelevant?
    MC...": Uh....

    I've had nearly this exact conversation probably 50 times in the last 10 years.

    ReplyDelete
  6. I also wonder how much the push for students to take AP and IB classes affects the overall GPA? It seems like schools are offering more advanced and college courses in high school which have a 5 point possibility, thereby raising the overall high school GPA.

    ReplyDelete
  7. Your question about the source of the data is right on point. If The College Board is using its own data, it is possible that the grades of the students that are taking the SAT have risen, but that is not looking at the entire population of schools. Data on the PSAT might be more useful, as more students take the test.

    I have worked at several high schools, and have never encountered such large numbers of students at such a GPA. The numbers that I've seen for students in the CSF (California Scholastic Federation - the CA honor society) do not support such an assertion.

    It seems like a very self-serving observation by CB and a lack of journalistic ethics on the part of USA Today, in that the editors failed to ascertain that the assertions of the CB were true.

    ReplyDelete