Saturday, March 14, 2015

Pearson Proves PARCC Stinks

When I was in tenth grade, I took a course called Biological Sciences Curriculum Studies (BSCS). It was a course known for its rigor and for its exceedingly tough tests.

The security on these tests? Absolutely zero. We took them as take-home tests. We had test-taking parties. We called up older siblings who were biology majors. The teacher knew we did these things. The teacher did not care, and it did not matter, because the tests required reasoning and application of the basic understanding of the scientific concepts. It wasn't enough, for instance, to know the parts of a single-celled organism-- you had to work out how those parts were analogous to the various parts of a city where the residents made pottery. You had to break down the implications of experimental design. And as an extra touch, after taking the test for a week outside of class, you had to take a different version of the same test (basically the same questions in a different order) in class.

Did people fail these zero-security take home tests? Oh, yes. They did.

I often think of those tests these days, because they were everything that modern standardized test manufacturers claim their tests are.

Test manufacturers and their proxies tell us repeatedly that their tests require critical thinking, rigorous mental application, answering questions with more than just rote knowledge.

They are lying.

They prove they are lying with their relentless emphasis on test security. Teachers may not look at the test, cannot so much as read questions enough to understand the essence of them. Students, teacher, and parents are not allowed to know anything specific about student responses after the fact (making the tests even less useful than the could possibly be).

And now, of course, we've learned that Pearson apparently has a super-secret cyber-security squad that just cruises the interwebs, looking for any miscreant teens who are violating the security of the test and calling the state and local authorities to have that student punished(and, perhaps, mounting denial of service attacks on any bloggers who dare to blog about it).

This shows a number of things, not the least of which is what everyone should already have know-- Pearson puts its own business interests ahead of anything and everything.

But it also tells us something about the test.

You know what kind of test need this sort of extreme security? A crappy one.

Questions that test "critical thinking" do not test it by saying, "Okay, you can only have a couple of minutes to read and think about this because if you had time to think about it, that wouldn't be critical thinking." A good, solid critical thinking question could take weeks to answer.

Test manufacturers and their cheerleaders like to say that these tests are impervious to test prep-- but if that were true, no security would be necessary. If the tests were impervious to any kind of advance preparation aimed directly at those tests, test manufacturers would be able to throw the tests out there in plain sight, like my tenth grade biology teacher did.

A good assessment has no shortcuts and needs no security. Look at performance-based measures-- no athlete shows up at an event and discovers at that moment, "Surprise! Today you're jumping over that bar!"

Authentic assessment is no surprise at all. It is exactly what you expect because it is exactly what yo prepared for, exactly what you've been doing all along-- just, this time, for a grade.

Big Stupid Test manufacturers insist that their test must be a surprise, that nobody can know anything about it, is a giant, screaming red alarm signal that these tests are crap. In what other industry can you sell a customer a product and refuse to allow them to look at it! It's like selling the emperor his new clothes and telling him they have to stay in the factory closet. Who falls for this kind of bad sales pitch? "Let me sell you this awesome new car, but you can never drive it and it will stay parked in our factory garage. We will drive you around in it, but you must be blindfolded. Trust us. It's a great car." Who falls for that??!!

The fact that they will go to such extreme and indefensible lengths to preserve the security of their product is just further proof that their product cannot survive even the simplest scrutiny.

The fact that product security trumps use of the product just raises this all to a super-kafka-esque level. It is more important that test security be maintained than it is that teachers and parents get any detailed and useful information from it. Test fans like to compare these tests to, say, tests at a doctor's office. That's a bogus comparison, but even if it weren't, test manufacturers have created a doctors office in which the doctor won't tell you what test you're getting, and when the test results come back STILL won't tell you what kind of test they gave you and will only tell you whether you're sick or well-- but nothing else because the details of your test results are proprietary and must remain a secret.

Test manufacturers like Pearson are right about one thing-- we don't need the tests to know how badly they suck, because this crazy-pants emphasis on product security tells us all we need to know. These are tests that can't survive the light of day, that are so frail and fragile and ineffectual that these tests can never be tested, seen, examined, or even, apparently, discussed.

Test manufacturers are telling us, via their security measures, just how badly these tests suck. People just have to start listening.

15 comments:

  1. In your doctor analogy, the doctor would not be allowed to look at the test (so how would he know if it was useful for your condition or not?), and would not get the test results for a year. The results he will see (in a year) will be so generalized that it won't be very useful for intervention. But when the patient dies the doctor will be deemed inefficient and if too many die he will be fired.

    ReplyDelete
  2. OK just did a little google prowling on Tracx and came up with pics of tweets about Pearson that the Tracx company used as a promotion. They are not just spying on students who take the tests. They are spying on everyone, all the time, anytime the Pearson brand gets mentioned...on twitter or FB.

    ReplyDelete
  3. In addition we know these tests suck because of the sample questions that were released. We have to assume that released sample questions would be the best of the pack but yet the samples I have seen in the press have glaringly obvious flaws. If those are the best crayons in the box then I fear for what the rest of the box contains. The emphasis on security is like the Wizard of Oz shouting 'Don't look behind that curtain.'

    ReplyDelete
  4. In addition we know these tests suck because of the sample questions that were released. We have to assume that released sample questions would be the best of the pack but yet the samples I have seen in the press have glaringly obvious flaws. If those are the best crayons in the box then I fear for what the rest of the box contains. The emphasis on security is like the Wizard of Oz shouting 'Don't look behind that curtain.'

    ReplyDelete
  5. In addition we know these tests suck because of the sample questions that were released. We have to assume that released sample questions would be the best of the pack but yet the samples I have seen in the press have glaringly obvious flaws. If those are the best crayons in the box then I fear for what the rest of the box contains. The emphasis on security is like the Wizard of Oz shouting 'Don't look behind that curtain.'

    ReplyDelete
  6. "They are spying on everyone, all the time, anytime the Pearson brand gets mentioned...on twitter or FB."

    Simple solution then - give them something to look at. Lots of somethings, actually. People should start tweeting, Facebooking, blogging, emailing, etc. as much as they can about Pearson and tests and PARCC and whatever other keywords might get such communication noticed. Include those words in every communication, even if (especially if) the communication is irrelevant to testing. When you send your sister your fabulous chicken salad recipe, be sure to talk about Pearson. And when you Facebook about how beautiful the weather is, include something about PARCC. Eventually we'll bore them all to death.

    ReplyDelete
    Replies
    1. LOL! That is exactly what Badass Teachers are doing on Wednesday!

      Delete
    2. Badass Teachers are already busy doing this. Someone suggested we create our own fake test questions and inundate the Internet with them so Pearson stays busy checking them out, too!

      Delete
    3. LMAO! I'm going to fabricate a Facebook post about erectile dysfunction and mention Pearson in it. Think they'll grasp the allusion? Not if they take their own tests!

      Delete
  7. the original intent of some of the early tests like SAT -- was to keep students out of Yale and Harvard because they only wanted the elite…. this purpose continues today ; they want to show in 2nd grade which children deserved educational opportunities so they can deny to the rest.

    ReplyDelete
  8. there was an article in the Atlantic not so long ago that also stated that some of the tests were devised to screen IN the WASP and to screen out the jewish student. I should go through the Atlantic archives and find that… (I know they have put all back issues into searchable format)… the Binet test was commissioned to determine "which students are likely to profit from an education"… so I think we are turning that on it's head with our understanding of ALL children, equity. etc. The test makers would like to tell us that our students are "children of a lesser god"

    ReplyDelete
  9. when the WISC IV was reformed students who were faster at perceptual motor speed became the new "norm" and even our gifted students were penalized; all students make speed accuracy trade-offs. The students who are reflective thinkers or who take a bit longer because the examine the questions and think about various responses are always penalized when this WISC IV is used. Why should we change the definition of "gifted" to someone who can perform routines that are mechanized in rapid succession -- especially when we are working with young children? That to me is conformity to society and what they want in the "pipeline" to "careers" set up the way corporate defines things.

    ReplyDelete
    Replies
    1. I meant re-normed of course not reformed

      Delete