PA: Graduation Test In Trouble (Again)

Pennsylvania's education bureaucrats had high hopes for the Keystone exams.

Back in 2010, the idea was that there would be at least ten of them-- one for each major course-- and students would take them at the end of the year as a final qualifying test for course credit (and therefor graduation). Donna Cooper (now of Public Citizens for Children and Youth) was part of the Rendell administration pushing for the tests, and like all good reformsters of the era, all she wanted was perfect standardization so that every student in every state was learning exactly the same thing.  "It would seem to me that a parent in Norristown and a parent in Johnstown, their kids should know the same things to graduate.” 

And like good reformy bureaucrats, neither the Rendell administration that cooked this up, nor the Corbett administration that cemented it into law, envisioned the state providing any resources at all to help students over this new hurdle. The Keystone exam system was the biggest unfunded mandate the state had ever seen.

The fiddling began immediately. Maybe the Keystones would count for a third of the full year grade. And somehow we'd have to roll the tests out over several years, only they turned out to be hard to just whip up quickly. And they were expensive, too.

Soon enough, the state decided that math, biology and English literature (aka "reading") would be the Keystone exams offered. And students and schools would get a couple of years to get up to speed before the test became a graduation requirement for the Class of 2017. 

But there was a problem. It quickly became clear that if the Keystones were required for graduation, a whole bunch of students weren't going to graduate. In more than 100 districts (and charters) well over half the students would not get a diploma.

Nobody in Harrisburg wanted to attach his name to that, so Keystones-as-grad-requirements were pushed back to 2019, and to insure that the results looked better by then, the state government did-- well, they did nothing. Hopes and dreams, thoughts and prayers, maybe. But still no resources to help in those afflicted districts. And sure enough-- as 2019 approaches (those students are taking the tests this year in many districts) things still look bad.

So buried within Harrisburg's most recent attempt to enact a budget, there's another postponement for the Keystones.   

Fans of local control are pleased-- let local districts set their own graduation requirements. But members of the Cult of Testing and Standardization are unhappy. Like Cooper, they argue that the previous Big Standardized Test, the PSSA, proved that local standards were inadequate.

“58,000 students were graduated and given diplomas who could not pass the state’s 11th grade PSSA,” said Cooper.  “That’s a real indication of the failure of local control to understand the market signals of what is needed for a kid to succeed in today’s economy.”

Cooper and others like her might have a point-- if there were a speck of evidence that the PSSA was a valid measure of what is needed for a kid to succeed in today's economy, or college, or anything. There was no such evidence for the PSSA, nor is there any such evidence for the Keystone. And the PSSA, which is still given in lower grades, is a norm-referenced test with cut scores reset every year -- so somebody has to fail. 

The Keystone exams are theoretically standards-referenced, which should mean that everyone can pass. But it should also mean that we can get test results literally five minutes after the student finishes the test, but we're still waiting months. Why is that? Maybe because of something called scaling, which seems like a fancy way to explain different weights for different questions on different forms of the test. Or maybe it has to do with rangefinding, which seems an awful lot like norm-referencing-- collect answers and see what their distribution looks like. 

Trying to uncover the problems of the Keystone exams can be a daunting task. The technical reports from every year are available (here's 2015), with hundreds of pages each, including illuminating passages like this one 

Reading these technical reports might suggest to a layperson, even a highly educated one, that testocrats have disappeared so far up their own butts that they are now viewing the world through the tiny little lenses of their own belly buttons.

But the answer to the Keystone problem may be much simpler than psychometric gobbledeegook or legislative refusal to fund what they demand. 

It could be that the Keystones are just bad tests.

Mind you, I'm not supposed to know that. The PA DOE Code of Ethics says that I should never set eyes on the test itself, and if I accidentally see anything, I should make myself forget it. We are supposed to remain ignorant of the test except in the most general terms, and the students have to swear not to talk about them either. We are all not only supposed to hit the target, but we're supposed to do it blindfolded. That way the test manufacturers won't have to spend money rebuilding the test every year the integrity of the test will be maintained.

But, scofflaw that I am, I look anyway. And I'm telling you that if someone offered me the test to use for free in my class so that I could have more time with my children and less time test writing, I would turn it down. It's junk. Questions with no objectively supportable single correct answer. Questions that are simple vocabulary tests. Questions that require the students to use psychic powers on the authors of the passages. These tests do not measure anything except the students' ability to navigate a bunch of trick questions and guess what the test manufacturers are thinking. 

You know who else knows that? My students. Like most districts, we have made the tests mandatory for graduation because we want the students to try because our school rating depends largely on those test results. But the students know that they'll do a performance task (the Binder of Doom) if they fail, and many of them are not only tired of taking stupid tests year after year, but they have long since concluded that they might as well roll dice, because success or failure feel pretty much random to them. 

Pennsylvania is having the same damn argument as much of the rest of the education world, with accountability mavens arguing that we must test to have accountability, but skipping over the entire question of whether the test being used is actually measuring anything worth measuring. It's like listening to someone insist, "We have to know whether or not you have cancer, so you must wave your hand over this horny toad under a full moon." It's the same old reformy disconnect-- establishing that something is a real problem is not the same as establishing that you are proposing a real solution (and for those of us who don't agree that local control and variation is a real problem, you are even further off base).

Without a decent, fair, valid test, and without resources to back up this unfunded  mandate, and without a reason for students to care, the Keystones will always be a disaster. We can only hope that the state legislature stops kicking this can down the road and finally just throws it in the dumpster of history, where it belongs.