Tuesday, December 5, 2023

Let's Do The PISA Panic Dance

Is there anything in education particularly useful or illuminating in the scores from the Programme for International Student Assessment (PISA) in this (or any) year?

Short answer: 


Long answer: 

Time for the PISA Panic Dance. You will hear, as always, that the US is woefully low in the rankings. We have always been low in these test scores gleaned from fifteen year olds. In fact, because everyone took some big hits this time, we actually climbed a bit. But we're still woefully low. 

Expect to hear from the usual suspects declaring that the woefully low ranking of the US is a monumental crisis of cosmic proportions and therefor we should rush to implement [insert name of the same policy that they push every other day of the week here]. 

Expect plenty of chicken littling. If you care to respond...

Long response to cries of "Our PISA rank is low! Our PISA rank is low!":

Do you have a research based context for your alarms? What can you tell me about the comparison-- is it between similar student populations, or do certain countries only test certain student populations? Additionally, can you cite any research that ties the PISA rankings to specific real-world outcomes for nations. For instance, Estonia routinely ranks high on these lists--in what areas do you believe Estonia is outpacing the US, and how would raising our PISA scores help counterbalance that?  

For instance, one authority says this: "Since a high ranking on PISA corresponds to economic success, researchers have concluded that PISA is one of the indicators of whether school systems are preparing students for the 21st-century global knowledge economy." Can you explain the difference between correlation and causation?

Short response to cries of "Our PISA rank is low! Our PISA rank is low!":

So what?

Okay, there are a couple of other pieces of data highlighted in the New York Times coverage that may shake up the usual PISA Panic Dance (despite the fact that NYT uses the "lost equivalent of three-quarters of a year" baloney). For instance, the US "lost less ground" aka "test score points" than some European nations that "prioritized opening schools more quickly." PISA also didn't find an increase in the gap between US highest and lowest students. 

But it's the final paragraph of Sarah Mervosh's article that is most concerning.
On other measures, the United States stood out for having more children living with food insecurity (13 percent, compared with an average of 8 percent in other O.E.C.D. countries), more students who are lonely at school (22 percent, versus 16 percent) and more students who do not feel safe at school (13 percent, versus 10 percent).

Here's hoping that someone in the education policy world chooses to stop the PISA Panic Dance long enough to look these data and declares that we should do something about them. Instead of worrying about our international bragging rights, maybe we could focus on the lives of the young human beings in our schools. That would be worth a real dance.


1 comment:

  1. The handwringing and pearl clutching is simply misguided. Media know-nothings (especially NYT "education reporters") should take the time to read the Common Core(ish) math standards, and then take a look at the PISA sample test items.

    Spoiler Alert: There is virtually zero correlation.

    Then they can redirect their handwringing and pearl clutching at our misguided sets of math standards that almost completely ignore real world applications and measurement.