Thursday, July 16, 2015

Can Third Graders "Fail"

A twitter conversation this morning turned into a discussion of the semantics of talking about test results, but any conversation that turns to phrases like "semantics of talking about test results" (39 characters) is kind of doomed on twitter. Jennifer Borgioli (@JennBinis) referred me to her article from last summer addressing the issue, "The Semantics of Test Scores."

There's a fairly detailed illustration of her argument, but her general point is that people play pretty fast and loose with the term "fail," particularly with the 3-8 grade range of tests.

Generally speaking, when we talk about a test where the resulting score is described as passing or failing, it’s in relation to the consequences for the test taker. Fail your driver’s test? You can’t drive. Pass your boards? Welcome to the profession, Doctor. 

While that holds true for many high school Big Standardized Tests that are used as graduation requirements, Bergioli argues that no such "bright line" exists for BS Tests for grades 3 through 8. 

There are no short-term negative consequences for students in grades 3-8 based on their performance on the state assessments. 

Bergioli's example is rooted in New York; this "bright line" assertion is, of course, flat out false if we throw in states like Mississippi that like the idea of holding back third graders who score too low on the reading assessment. That, I think, qualifies pretty clearly as failing.

I do get her point. Language choice with children (particularly younger ones) is important, and it is particularly important to choose carefully when discussing success or the lack thereof. When my kids were little, their mother and I were careful to use phrases like "haven't succeeded yet" in place of "failed." When we designed graduation projects for my high school, the only outcomes we made a place for in the evaluation stage were "successfully completed" and "not successfully completed yet." 

I get that some test-loving reformsters imagine a perfect world where tests are given, tests come back, and nine year olds say, "Well, that was not what I had hoped for. But I can see that I need to enrich my study and practice of identifying main ideas in paragraphs, so I guess I'll just hunker down and do that." 

But in this, as in so many areas, folks who design and promote this stuff are kidding themselves.

So, while there is once again the possibility of larger, longer-term consequences to the school, the district, and the community based on how a group of students do on these tests, in the absence of a clear and bright line of a relationship between student performance on the test and consequences to the student, it’s misleading to say the student “failed” the test.

No, I don't think it's misleading at all.

First of all, the larger, long-term consequences are based on drawing a clear and bright line. The state says, "This number of your students fell on the wrong side of this clear, bright line, so you are not an effective teacher." Depending on your state and its laws, that ineffectiveness may be reflected in your evaluation, your pay, and your future employment. It will also be reflected in the rating of your school, and of course in many states that rating will be a nice, neat letter grade. Why do some policymakers like giving schools letter grades? Because it makes it easy to tell if the school is passing or failing.

Second, children are not dopes. Decades of sorting students into bluebirds and chickenhawks have fooled almost nobody-- students know whether they're winning or losing in the Big Game O' Learning Stuff. And because the stakes on test results are so high for schools, students who score Not So High on BS Tests will find themselves rewarded with extra work, extra practice, extra time in the albatross reading group. It can take a tough little kid to look at the evidence and not reach unflattering conclusions about herself, and most teachers I know do their best to keep children from reaching those conclusions. But students know when they've failed, whatever we try to call it.

We can say that these students have not technically failed, and in an academic technical sense, we are correct. But eight year olds are not known for their ability to look at things in an academic technical sense. That's one of the truly toxic effects of badly written tests-- young students lack the capacity to say, "Well, this was a poorly designed assessment." They just think, "I must be stupid."

Nor do we get a lot of nuance from the policymakers and politicians who keep talking about failing schools and failing teachers, all of which underlines clearly that there's a bright clear line, and anybody falling below it has failed. 

Test manufacturing experts sometimes remind me of sad scientists in old SF movies. They design these instruments to try to tease out nuanced granular pictures of student strengths and weaknesses and then policymakers just grab the tests and say, "Never mind all that. I just wanna know how many of these kids failed."
 

I agree that it would be better for everyone if we could deal with these issues in a nuanced thoughtful manner. But then, if the education discussion were being run by policymakers who valued nuance, detail, and the expertise of people in the field, we'd be in a far different place than we are today. We can try to shade the meaning of "fail" so that it has a very specific meaning in very specific circumstances, but that's a hopeless exercise. Everyone knows what "fails," means-- you came in below a particular mark. And that definition fits for every test in the history of ever.

 

6 comments:

  1. I have had my share of twitter skirmishes with Jennifer Borgioli (@JennBinis). She is flat out wrong about 3rd graders not "failing" a NYS standardized test. If a student earns a 1 or a 2, they are immediately eligible for Academic Intervention Services. That pretty clearly says to the child, and the child's peers, who "failed" and who did not. And, the child loses art, music, gym, or recess.

    ReplyDelete
  2. This comment has been removed by the author.

    ReplyDelete
  3. Why hello there stranger with whom I've exchanged Tweets with! What's compelling in your comment is the implication that needing extra help, AIS, is somehow "failing". In a way, youve supported my point around the importance of language. After all, if a school is telling an 8 year old " You're a failure because you need extra help" it says a lot about how we communicate about these tests.

    ReplyDelete
    Replies
    1. Nobody has to tell a third grader who just lost her art class or music class or recess because she has to sit and do extra work that her other scored-better-on-the-test peers do not-- nobody has to tell her that she "failed." She will supply that interpretation of events all on her own. Nobody, especially an eight year old, thinks, "I need extra help! I must be extra awesome." We can try our damnedest to make sure no stigma is attached to the situation, but that will not make the absence of success feel like success to that child.

      Delete
    2. No doubt the AIS system in NYS (which I think the poster was referring to) is broken and the regs haven't been adjusted in a while.

      My wondering: is it helping or hurting that third grader when the adults around her are talking about how many kids "failed" a test she never has a chance to retake? That's the mixed message here: One on hand, we want students to know failing is okay, it's a part of learning. On the other, we deride "failure" rates on state tests or explain why it's okay to refer to a child whose scores fell in Level 1 or 2 on a rubric as a "failure." And then we wonder why kids become obsessed with points and grades as they get older or annoyed at "reformers" who peddle the school "failure" message. Ravitch's post about the changing cut scores provides pretty compelling evidence that their claim "schools are failing! Just look at how many kids failed the state tests!" is right. Want to change the narrative on failing schools? Stop saying that kids are failing a once a year test they have no chance to retake.

      Delete
    3. Want to change the narrative on failing schools? Opt out/ refuse the test.

      Delete