It is one of the great fallacies you will frequently encounter in the work of education reform.
I most recently encountered a very striking version of it in a new position-paper-advocacy-research-report-white-paper-thingy from FEE, the reformster group previously working for Jeb Bush and handed over (at least until Bush finishes trying to be President) to Condoleezza Rice.
The report (which I went over in more detail here) wants to make the case for charters and choice in education, and it starts by arguing that soon there will be way too few employed people paying for way too many children and retired geezers, therefore, school choice. The "report" runs to almost 100 pages, and ninety-some of those are devoted to mapping out the severe scrariosity of the upcoming crisis. The part that explains how school choice would fix this-- that gets a couple of pages. At its most critical juncture, the argument depends on one previously debunked study.
This is a relatively common fallacious argument structure, but if you are going to spend time in the education debates, it's useful to know it when you see it. The basic outline of the argument looks like this:
1) SOMETHING AWFUL IS GOING TO HAPPEN OH MY GOOD LORD IN HEAVEN LOOOK I EVEN HAVE CHARTS AND GRAPHS AND IT IS SOOOOOOOOO TERRIBLE THAT IT WILL MAKE AWFUL THINGS HAPPEN, REALLY TERRIBLE AWFUL THINGS LET ME TELL YOU JUST HOW AWFUL OH GOD HEAVENS WE MUST ALL BEWARE--- BEEE WAAAARREEEEEEE!!!!!!!!!
2) therefore for some reason
3) You must let me do X to save us!
The trick here is to load up #1 with facts and figures and details and specifics. Make it as facty and credible as you possibly can (even if you need to gin up some fake facts to do it).
#3 is where you load in your PR for whatever initiative you're pushing.
And #2 you just try to skate past as quickly as possible, because #2 is the part that most needs support and proof and fact-like content, but #2 is also the place where you probably don't have any.
In a normal, non-baloney argument, #2 is the strongest point, because the rational, supportable connection between the problem and the solution is what matters most. But if you are selling baloney, that connection is precisely what you don't have. So instead of actual substance in #2, you just do your best to drive up the urgency in #1.
For example:
1) The volcano is gigantic and scary and when lava comes pouring out of it WE ARE ALL GOING TO DIE HOT FLAMING DEATHS AND SUFFOCATE IN ASH AND IT WILL BE TERRIBLE
2) Therefore, for some reason
3) We should sacrifice some virgins
Or:
1) We are falling behind other countries and if we don't get caught back up we will be BEHIND ESTONIA!! ESTONIA!!!! GOOD GOD, WE MUST NOT FALL BEHIND THESE OTHER NATIONS ON THE TOTALLY MADE-UP INTERNATIONAL AWESOMENESS INDEX
2) Therefore, for some reason
3) We should adopt Common Core
You can manufacture the #1 crisis if necessary. But this can be even more effective if you use an actual real problem for #1:
1) Poor and minority children in this country keep getting the short end of the stick educationally, with fewer resources and less opportunity to break out of the cycle of poverty. This is a crappy way for our fellow Americans to have to live, and certainly leaving no pathway out of poverty is a violation of the American dream
2) Therefore, for some reason
3) We should make sure they all have to take a Big Standardized Test every year.
You just have to convey a sense of urgency about #1 and never ever let the conversation drift to #2. If people start trying to ask exactly how #3 actually helps with #1, you just rhetorical question them into silence.
Treat questioning #2 as if it's the same as questioning #1.Can't for the life of you see how the #1 of poverty and under-resourced schools is solved by more charter schools that drain resources from public education and only agree to teach the handful of students that they accept, while remaining unaccountable to anyone? Condoleezza Rice says you're a racist.
But it's #2 where the most important questions lie. Even if I accept that US schools are in some sort of crisis (which I don't, but if), exactly how would Common Core fix that? I do believe that we have a real problem with poverty in this country, but how, exactly, will giving poor kids standardized tests help with that?
If you have a gut feeling that a great deal of the reformster just doesn't make sense, #2 is where the problem mostly lies. Most reformster arguments involve using a loud #1 and a slick #3 to cover up a non-existent #2.
1) Some students score low on Big Standardized Tests-- They GET LOW SCORES! LOW SCORES THAT ARE A BAAAAAAD THING! True, they're a bad thing because we've set up a system of artificial imposed punishments for low scores but hey, still-- LOOOOWWWW SCOOORESSSSSS!!!!!
2) Therefore, for some reason
3) There should be no tenure for teachers
There's no connection at all. We could just as easily say
3) Taxpayers should buy charter operators a pony
3) The National Guard should shoot a badger
3) We should sacrifice a virgin
But of course badgers and ponies and virgins aren't nearly as profitable as charters and tests. That, and I think some folks really believe that #2 is there when it just isn't. Either way, it's important to know what the real connection is before you start sacrificing virgins.
Sunday, February 8, 2015
Sampling the PARCC
Today, I'm trying something new. I've gotten myself onto the PARCC sample item site and am going to look at the ELA sample items for high school. This set was updated in March of 2014, so, you know, it's entirely possible they are not fully representative, given that the folks at Pearson are reportedly working tirelessly to improve testing so that new generations of Even Very Betterer Tests can be released into the wild, like so many majestic lion-maned dolphins.
So I'm just going to live blog this in real-ish time, because we know that one important part of measuring reading skill is that it should not involve any time for reflection and thoughtful revisiting of the work being read. No, the Real Readers of this world are all Wham Bam Thank You Madam Librarian, so that's how we'll do this. There appear to be twenty-three sample items, and I have two hours to do this, so this could take a while. You've been warned.
PAGE ONE: DNA
Right off the bat I can see that taking the test on computer will be a massive pain in the ass. Do you remember frames, the website formatting that was universally loathed and rapidly abandoned? This reminds me of that. The reading selection is in its own little window and I have to scroll the reading within that window. The two questions run further down the page, so when I'm looking at the second question, the window with the selection in it is halfway off the screen, so to look back to the reading I have to scroll up in the main window and then scroll up and down in the selection window and then take a minute to punch myself in the brain in frustration.
The selection is about using DNA testing for crops, so fascinating stuff. Part A (what a normal person might call "question 1") asks us to select three out of seven terms used in the selection, picking those that "help clarify" the meaning of the term "DNA fingerprint," so here we are already ignoring the reader's role in reading. If I already understand the term, none of them help (what helped you learn how to write your name today?), and if I don't understand the term, apparently there is only one path to understanding. If I decide that I have to factor in the context in which the phrase is used, I'm back to scrolling in the little window and I rapidly want to punch the test designers in the face. I count at least four possible answers here, but only three are allowed. Three of them are the only answers to use "genetics" in the answer; I will answer this question based on guesswork and trying to second guess the writer.
Part B is a nonsense question, asking me to come up with an answer based on my first answer.
PAGE TWO: STILL FRICKIN' DNA
Still the same selection. Not getting any better at this scrolling-- whether my mouse roller scrolls the whole page or the selection window depends on where my cursor is sitting.
Part A is, well... hmm. If I asked you, "Explain how a bicycle is like a fish," I would expect an answer from you that mentioned both the bicycle and a fish. But PARCC asks how "solving crop crimes is like solving high-profile murder cases." But all four answers mention only the "crop crime" side of the comparison, and the selection itself says nothing about how high-profile murder cases are solved. So are students supposed to already know how high-profile murder cases are solved? Should they assume that things they've seen on CSI or Law and Order are accurate? To answer this we'll be reduced to figuring out which answer is an accurate summary of the crop crime techniques mentioned in the selection.
This is one of those types of questions that we have to test prep our students for-- how to "reduce" a seemingly complex question to the simpler question. This question pretends to be complex; it is actually asking, "Which one of these four items is actually mentioned in the selection?" It boils down to picky gotcha baloney-- one answer is going to be wrong because it says that crop detectives use computers "at crime scenes"
Part B.The old "which detail best supports" question. If you blew Part A, these answers will be bizarrely random.
PAGE THREE: DNA
Still on this same damn selection. I now hate crops and their DNA.
Part A wants to know what the word "search" means in the heading for the final graph. I believe it means that the article was poorly edited, but that selection is not available. The distractor in this set is absolutely true; it requires test-taking skills to eliminate it, not reading skills.
Part B "based on information from the text" is our cue (if we've been properly test prepped) to go look for the answer in the text, which would take a lot less time if not for this furshlugginer set up. The test writers have called for two correct answers, allowing them to pretend that a simple search-and-match question is actually complex.
PAGE FOUR: DNA GRAND FINALE, I HOPE
Ah, yes. A test question that assesses literally nothing useful whatsoever. At the top of the page is our selection in a full-screen width window instead of the narrow cramped one. At the bottom of the page is a list of statements, two of which are actual advantages of understanding crop DNA. Above them are click-and-drag details from the article. You are going to find the two advantages, then drag the supporting detail for each into the box next to it. Once you've done all this, you will have completed a task that does not mirror any real task done by real human beings anywhere in the world ever.
This is so stupid I am not even going to pretend to look for the "correct" answer. But I will remember this page clearly the next time somebody tries to unload the absolute baloney talking point that the PARCC does not require test prep. No students have ever seen questions like this unless a teacher showed them such a thing, and no teacher ever used such a thing in class unless she was trying to get her students ready for a cockamamie standardized test.
Oh, and when you drag the "answers," they often don't fit in the box and just spill past the edges, looking like you've made a mistake.
PAGE FIVE: FOR THE LOVE OF GOD, DNA
Here are the steps listed in the article. Drop and drag them into the same order as in the article. Again, the only thing that makes this remotely difficult is wrestling with the damn windows. This is a matching exercise, proving pretty much nothing.
PAGE SIX: APPARENTLY THIS IS A DNA TEST TEST
By now my lower-level students have stopped paying any attention to the selection and are just trying to get past it to whatever blessed page of the test will show them something else.
Part A asks us to figure out which question is answered by the selection. This is one of the better questions I've seen so far. Part B asks which quote "best" supports the answer for A. I hate these "best" questions, because they reinforce the notion that there is only one immutable approach for any given piece of text. It's the very Colemanian idea that every text represents only a single destination and there is only one road by which to get there. That's simply wrong, and reinforcing it through testing is also wrong. Not only wrong, but a cramped, tiny, sad version of the richness of human understanding and experience.
PAGE SEVEN: SOMETHING NEW
Here comes the literature. First we get 110 lines of Ovid re: Daedelus and Icarus (in a little scrolling window). Part A asks which one of four readings is the correct one for lines 9 and 10 (because reading, interpreting and experiencing the richness of literature is all about selecting the one correct reading). None of the answers are great, particularly if you look at the lines in context, but only one really makes sense. But then Part B asks which other lines support your Part A answer and the answer here is "None of them," though there is one answer for B that would support one of the wrong answers for A, so now I'm wondering if the writers and I are on a different page here.
PAGE EIGHT: STILL OVID
Two more questions focusing on a particular quote, asking for an interpretation and a quote to back it up. You know, when I say it like that, it seems like a perfectly legitimate reading assessment. But when you turn that assessment task into a multiple choice question, you break the whole business. "Find a nice person, get married and settle down," seems like decent-ish life advice, but if you turn it into "Select one of these four people, get married in one of these four ceremonies, and buy one of these four houses" suddenly it's something else.
And we haven't twisted this reading task for the benefit of anybody except the people who sell, administer, score and play with data from these tests.
PAGE NINE: OVID
The test is still telling me that I'm going to read two selections but only showing me one. If I were not already fully prepped for this type of test and test question, I might wonder if something were wrong with my screen. So, more test prep required.
Part A asks what certain lines "most" suggest about Daedelus, as if that is an absolute objective thing. Then you get to choose what exact quotes (two, because that makes it more complex) back you up. This is not constructing and interpretation of a piece of literature. Every one of these questions makes me angrier as a teacher of literature and reading.
PAGE TEN: ON TO SEXTON
Here's our second poem-- "To a Friend Whose Work Has Come To Triumph." The two questions are completely bogus-- Sexton has chosen the word "tunneling" which is a great choice in both its complexity and duality of meaning, a great image for the moment she's describing. But of course in test land the word choice only "reveals" one thing, and only one other piece of the poem keys that single meaning. I would call this poetry being explained by a mechanic, but that's disrespectful to mechanics.
PAGE ELEVEN: MORE BUTCHERY
Determine the central idea of Sexton's poem, as well as specific details that develop the idea over the course of the poem. From the list of Possible Central Ideas, drag the best Central Idea into the Central Idea box.
Good God! This at least avoids making explicit what is implied here-- "Determine the central idea, then look for it on our list. If it's not there, you're wrong." Three of the four choices are okay-ish, two are arguable, and none would impress me if they came in as part of a student paper.
I'm also supposed to drag-and-drop three quotes that help develop the One Right Idea. So, more test prep required.
PAGE TWELVE: CONTRAST
Now my text window has tabs to toggle back and forth between the two works. I'm supposed to come up with a "key" difference between the two works (from their list of four, of course) and two quotes to back up my answer. Your answer will depend on what you think "key" means to the test writers. Hope your teacher did good test prep with you.
PAGE THIRTEEN: ESSAY TIME
In this tiny text box that will let you view about six lines of your essay at a time, write an essay "that provides and analysis of how Sexton transforms Daedelus and Icarus." Use evidence from both texts. No kidding-- this text box is tiny. And no, you can't cut and paste quotes directly from the texts.
But the big question here-- who is going to assess this, and on what basis? Somehow I don't think it's going to be a big room full of people who know both their mythology and their Sexton.
PAGE FOURTEEN: ABIGAIL ADAMS
So now we're on to biography. It's a selection from the National Women's History Museum, so you know it is going to be a vibrant and exciting text. I suppose it could be worse--we could be reading from an encyclopedia.
The questions want to know what "advocate for women" means, and to pick an example of Adams being an advocate. In other words, the kinds of questions that my students would immediately id as questions that don't require them to actually read the selection.
PAGE FIFTEEN: ADAMS
This page wants to know which question goes unanswered by the selection, and then for Part B asks to select a statement that is true about the biography but which supports the answer for A. Not hopelessly twisty.
PAGE SIXTEEN: MORE BIO
Connect the two central ideas of this selection. So, figure out what the writers believe are the two main ideas, and then try figure out what they think the writers see as a connection. Like most of these questions, these will be handled backwards. I'm not going to do a close reading of the selection-- I'm going to close read the questions and answers and then use the selection just as a set of clues about which answer to pick. And this is how answering multiple choice questions about a short selection is a task not much like authentic reading or pretty much any other task in the world.
PAGE SEVENTEEN: ABIGAIL LETTER
Now we're going to read the Adams family mail. This is one of her letters agitating for the rights of women; our questions will focus on her use of "tyrant" based entirely on the text itself, because no conversation between Abigail and John Adams mentioning tyranny in 1776 could possibly be informed by any historical or personal context.
PAGE EIGHTEEN: STILL VIOLATING FOUNDING FATHER & MOTHER PRIVACY
Same letter. Now I'm supposed to decide what the second graph most contributes to the text as a whole. Maybe I'm just a Below basic kind of guy, but I am pretty sure that the correct answer is not among the four choices. That just makes it harder to decide which other two paragraphs expand on the idea of graph #2.
PAGE NINETEEN: BOSTON
Now we'll decide what her main point about Boston is in the letter. This is a pretty straightforward and literal reading for details kind of question. Maybe the PARCC folks are trying to boost some morale on the home stretch here.
Oh hell. I have a message telling me I have less than five minutes left.
PAGE TWENTY: JOHN'S TURN
Now we have to pick the paraphrase of a quote from Adams that the test writers think is the berries. Another set of questions that do not require me to actually read the selection, so thank goodness for small favors.
PAGE TWENTY-ONE: MORE JOHN
Again, interpretation and support. Because making sense out of colonial letter-writing English is just like current reading. I mean, we've tested me on a boring general science piece, classical poetry, modern poetry, and a pair of colonial letters. Does it seem like that sampling should tell us everything there is to know about the full width and breadth of student reading ability?
PAGE TWENTY-TWO: BOTH LETTERS
Again, in one page, we have two sets of scrollers, tabs for toggling between works, and drag and drop boxes for the answers. Does it really not occur to these people that there are students in this country who rarely-if-ever lay hands on a computer?
This is a multitask page. We're asking for a claim made by the writer and a detail to back up that claim, but we're doing both letters on the same page and we're selecting ideas and support only from the options provided by the test. This is not complex. It does not involve any special Depth of Knowledge. It's just a confusing mess.
PAGE TWENTY-THREE: FINAL ESSAY
Contrast the Adams' views of freedom and independence. Support your response with details from the three sources (yes, we've got three tabs now). Write it in this tiny text box.
Do you suppose that somebody's previous knowledge of John and Abigail and the American Revolution might be part of what we're inadvertently testing here? Do you suppose that the readers who grade these essays will themselves be history scholars and writing instructors? What, if anything, will this essay tell us about the student's reading skills?
DONE
Man. I have put this off for a long time because I knew it would give me a rage headache, and I was not wrong. How anybody can claim that the results from a test like this would give us a clear, nuanced picture of student reading skills is beyond my comprehension. Unnecessarily complicated, heavily favoring students who have prior background knowledge, and absolutely demanding that test prep be done with students, this is everything one could want in an inauthentic assessment that provides those of us in the classroom with little or no actual useful data about our students.
If this test came as part of a packaged bunch of materials for my classroom, it would go in the Big Circular File of publishers materials that I never, ever use because they are crap. What a bunch of junk. If you have stuck it out with me here, God bless you. I don't recommend that you give yourself the full PARCC sample treatment, but I heartily recommend it to every person who declares that these are wonderful tests that will help revolutionize education. Good luck to them as well.
So I'm just going to live blog this in real-ish time, because we know that one important part of measuring reading skill is that it should not involve any time for reflection and thoughtful revisiting of the work being read. No, the Real Readers of this world are all Wham Bam Thank You Madam Librarian, so that's how we'll do this. There appear to be twenty-three sample items, and I have two hours to do this, so this could take a while. You've been warned.
PAGE ONE: DNA
Right off the bat I can see that taking the test on computer will be a massive pain in the ass. Do you remember frames, the website formatting that was universally loathed and rapidly abandoned? This reminds me of that. The reading selection is in its own little window and I have to scroll the reading within that window. The two questions run further down the page, so when I'm looking at the second question, the window with the selection in it is halfway off the screen, so to look back to the reading I have to scroll up in the main window and then scroll up and down in the selection window and then take a minute to punch myself in the brain in frustration.
The selection is about using DNA testing for crops, so fascinating stuff. Part A (what a normal person might call "question 1") asks us to select three out of seven terms used in the selection, picking those that "help clarify" the meaning of the term "DNA fingerprint," so here we are already ignoring the reader's role in reading. If I already understand the term, none of them help (what helped you learn how to write your name today?), and if I don't understand the term, apparently there is only one path to understanding. If I decide that I have to factor in the context in which the phrase is used, I'm back to scrolling in the little window and I rapidly want to punch the test designers in the face. I count at least four possible answers here, but only three are allowed. Three of them are the only answers to use "genetics" in the answer; I will answer this question based on guesswork and trying to second guess the writer.
Part B is a nonsense question, asking me to come up with an answer based on my first answer.
PAGE TWO: STILL FRICKIN' DNA
Still the same selection. Not getting any better at this scrolling-- whether my mouse roller scrolls the whole page or the selection window depends on where my cursor is sitting.
Part A is, well... hmm. If I asked you, "Explain how a bicycle is like a fish," I would expect an answer from you that mentioned both the bicycle and a fish. But PARCC asks how "solving crop crimes is like solving high-profile murder cases." But all four answers mention only the "crop crime" side of the comparison, and the selection itself says nothing about how high-profile murder cases are solved. So are students supposed to already know how high-profile murder cases are solved? Should they assume that things they've seen on CSI or Law and Order are accurate? To answer this we'll be reduced to figuring out which answer is an accurate summary of the crop crime techniques mentioned in the selection.
This is one of those types of questions that we have to test prep our students for-- how to "reduce" a seemingly complex question to the simpler question. This question pretends to be complex; it is actually asking, "Which one of these four items is actually mentioned in the selection?" It boils down to picky gotcha baloney-- one answer is going to be wrong because it says that crop detectives use computers "at crime scenes"
Part B.The old "which detail best supports" question. If you blew Part A, these answers will be bizarrely random.
PAGE THREE: DNA
Still on this same damn selection. I now hate crops and their DNA.
Part A wants to know what the word "search" means in the heading for the final graph. I believe it means that the article was poorly edited, but that selection is not available. The distractor in this set is absolutely true; it requires test-taking skills to eliminate it, not reading skills.
Part B "based on information from the text" is our cue (if we've been properly test prepped) to go look for the answer in the text, which would take a lot less time if not for this furshlugginer set up. The test writers have called for two correct answers, allowing them to pretend that a simple search-and-match question is actually complex.
PAGE FOUR: DNA GRAND FINALE, I HOPE
Ah, yes. A test question that assesses literally nothing useful whatsoever. At the top of the page is our selection in a full-screen width window instead of the narrow cramped one. At the bottom of the page is a list of statements, two of which are actual advantages of understanding crop DNA. Above them are click-and-drag details from the article. You are going to find the two advantages, then drag the supporting detail for each into the box next to it. Once you've done all this, you will have completed a task that does not mirror any real task done by real human beings anywhere in the world ever.
This is so stupid I am not even going to pretend to look for the "correct" answer. But I will remember this page clearly the next time somebody tries to unload the absolute baloney talking point that the PARCC does not require test prep. No students have ever seen questions like this unless a teacher showed them such a thing, and no teacher ever used such a thing in class unless she was trying to get her students ready for a cockamamie standardized test.
Oh, and when you drag the "answers," they often don't fit in the box and just spill past the edges, looking like you've made a mistake.
PAGE FIVE: FOR THE LOVE OF GOD, DNA
Here are the steps listed in the article. Drop and drag them into the same order as in the article. Again, the only thing that makes this remotely difficult is wrestling with the damn windows. This is a matching exercise, proving pretty much nothing.
PAGE SIX: APPARENTLY THIS IS A DNA TEST TEST
By now my lower-level students have stopped paying any attention to the selection and are just trying to get past it to whatever blessed page of the test will show them something else.
Part A asks us to figure out which question is answered by the selection. This is one of the better questions I've seen so far. Part B asks which quote "best" supports the answer for A. I hate these "best" questions, because they reinforce the notion that there is only one immutable approach for any given piece of text. It's the very Colemanian idea that every text represents only a single destination and there is only one road by which to get there. That's simply wrong, and reinforcing it through testing is also wrong. Not only wrong, but a cramped, tiny, sad version of the richness of human understanding and experience.
PAGE SEVEN: SOMETHING NEW
Here comes the literature. First we get 110 lines of Ovid re: Daedelus and Icarus (in a little scrolling window). Part A asks which one of four readings is the correct one for lines 9 and 10 (because reading, interpreting and experiencing the richness of literature is all about selecting the one correct reading). None of the answers are great, particularly if you look at the lines in context, but only one really makes sense. But then Part B asks which other lines support your Part A answer and the answer here is "None of them," though there is one answer for B that would support one of the wrong answers for A, so now I'm wondering if the writers and I are on a different page here.
PAGE EIGHT: STILL OVID
Two more questions focusing on a particular quote, asking for an interpretation and a quote to back it up. You know, when I say it like that, it seems like a perfectly legitimate reading assessment. But when you turn that assessment task into a multiple choice question, you break the whole business. "Find a nice person, get married and settle down," seems like decent-ish life advice, but if you turn it into "Select one of these four people, get married in one of these four ceremonies, and buy one of these four houses" suddenly it's something else.
And we haven't twisted this reading task for the benefit of anybody except the people who sell, administer, score and play with data from these tests.
PAGE NINE: OVID
The test is still telling me that I'm going to read two selections but only showing me one. If I were not already fully prepped for this type of test and test question, I might wonder if something were wrong with my screen. So, more test prep required.
Part A asks what certain lines "most" suggest about Daedelus, as if that is an absolute objective thing. Then you get to choose what exact quotes (two, because that makes it more complex) back you up. This is not constructing and interpretation of a piece of literature. Every one of these questions makes me angrier as a teacher of literature and reading.
PAGE TEN: ON TO SEXTON
Here's our second poem-- "To a Friend Whose Work Has Come To Triumph." The two questions are completely bogus-- Sexton has chosen the word "tunneling" which is a great choice in both its complexity and duality of meaning, a great image for the moment she's describing. But of course in test land the word choice only "reveals" one thing, and only one other piece of the poem keys that single meaning. I would call this poetry being explained by a mechanic, but that's disrespectful to mechanics.
PAGE ELEVEN: MORE BUTCHERY
Determine the central idea of Sexton's poem, as well as specific details that develop the idea over the course of the poem. From the list of Possible Central Ideas, drag the best Central Idea into the Central Idea box.
Good God! This at least avoids making explicit what is implied here-- "Determine the central idea, then look for it on our list. If it's not there, you're wrong." Three of the four choices are okay-ish, two are arguable, and none would impress me if they came in as part of a student paper.
I'm also supposed to drag-and-drop three quotes that help develop the One Right Idea. So, more test prep required.
PAGE TWELVE: CONTRAST
Now my text window has tabs to toggle back and forth between the two works. I'm supposed to come up with a "key" difference between the two works (from their list of four, of course) and two quotes to back up my answer. Your answer will depend on what you think "key" means to the test writers. Hope your teacher did good test prep with you.
PAGE THIRTEEN: ESSAY TIME
In this tiny text box that will let you view about six lines of your essay at a time, write an essay "that provides and analysis of how Sexton transforms Daedelus and Icarus." Use evidence from both texts. No kidding-- this text box is tiny. And no, you can't cut and paste quotes directly from the texts.
But the big question here-- who is going to assess this, and on what basis? Somehow I don't think it's going to be a big room full of people who know both their mythology and their Sexton.
PAGE FOURTEEN: ABIGAIL ADAMS
So now we're on to biography. It's a selection from the National Women's History Museum, so you know it is going to be a vibrant and exciting text. I suppose it could be worse--we could be reading from an encyclopedia.
The questions want to know what "advocate for women" means, and to pick an example of Adams being an advocate. In other words, the kinds of questions that my students would immediately id as questions that don't require them to actually read the selection.
PAGE FIFTEEN: ADAMS
This page wants to know which question goes unanswered by the selection, and then for Part B asks to select a statement that is true about the biography but which supports the answer for A. Not hopelessly twisty.
PAGE SIXTEEN: MORE BIO
Connect the two central ideas of this selection. So, figure out what the writers believe are the two main ideas, and then try figure out what they think the writers see as a connection. Like most of these questions, these will be handled backwards. I'm not going to do a close reading of the selection-- I'm going to close read the questions and answers and then use the selection just as a set of clues about which answer to pick. And this is how answering multiple choice questions about a short selection is a task not much like authentic reading or pretty much any other task in the world.
PAGE SEVENTEEN: ABIGAIL LETTER
Now we're going to read the Adams family mail. This is one of her letters agitating for the rights of women; our questions will focus on her use of "tyrant" based entirely on the text itself, because no conversation between Abigail and John Adams mentioning tyranny in 1776 could possibly be informed by any historical or personal context.
PAGE EIGHTEEN: STILL VIOLATING FOUNDING FATHER & MOTHER PRIVACY
Same letter. Now I'm supposed to decide what the second graph most contributes to the text as a whole. Maybe I'm just a Below basic kind of guy, but I am pretty sure that the correct answer is not among the four choices. That just makes it harder to decide which other two paragraphs expand on the idea of graph #2.
PAGE NINETEEN: BOSTON
Now we'll decide what her main point about Boston is in the letter. This is a pretty straightforward and literal reading for details kind of question. Maybe the PARCC folks are trying to boost some morale on the home stretch here.
Oh hell. I have a message telling me I have less than five minutes left.
PAGE TWENTY: JOHN'S TURN
Now we have to pick the paraphrase of a quote from Adams that the test writers think is the berries. Another set of questions that do not require me to actually read the selection, so thank goodness for small favors.
PAGE TWENTY-ONE: MORE JOHN
Again, interpretation and support. Because making sense out of colonial letter-writing English is just like current reading. I mean, we've tested me on a boring general science piece, classical poetry, modern poetry, and a pair of colonial letters. Does it seem like that sampling should tell us everything there is to know about the full width and breadth of student reading ability?
PAGE TWENTY-TWO: BOTH LETTERS
Again, in one page, we have two sets of scrollers, tabs for toggling between works, and drag and drop boxes for the answers. Does it really not occur to these people that there are students in this country who rarely-if-ever lay hands on a computer?
This is a multitask page. We're asking for a claim made by the writer and a detail to back up that claim, but we're doing both letters on the same page and we're selecting ideas and support only from the options provided by the test. This is not complex. It does not involve any special Depth of Knowledge. It's just a confusing mess.
PAGE TWENTY-THREE: FINAL ESSAY
Contrast the Adams' views of freedom and independence. Support your response with details from the three sources (yes, we've got three tabs now). Write it in this tiny text box.
Do you suppose that somebody's previous knowledge of John and Abigail and the American Revolution might be part of what we're inadvertently testing here? Do you suppose that the readers who grade these essays will themselves be history scholars and writing instructors? What, if anything, will this essay tell us about the student's reading skills?
DONE
Man. I have put this off for a long time because I knew it would give me a rage headache, and I was not wrong. How anybody can claim that the results from a test like this would give us a clear, nuanced picture of student reading skills is beyond my comprehension. Unnecessarily complicated, heavily favoring students who have prior background knowledge, and absolutely demanding that test prep be done with students, this is everything one could want in an inauthentic assessment that provides those of us in the classroom with little or no actual useful data about our students.
If this test came as part of a packaged bunch of materials for my classroom, it would go in the Big Circular File of publishers materials that I never, ever use because they are crap. What a bunch of junk. If you have stuck it out with me here, God bless you. I don't recommend that you give yourself the full PARCC sample treatment, but I heartily recommend it to every person who declares that these are wonderful tests that will help revolutionize education. Good luck to them as well.
Saturday, February 7, 2015
Aldeman in NYT: Up Is Down
In Friday's New York Times, Chad Aldeman of Bellwether offered a defense of annual testing that is a jarring masterpiece of backwards speak, a string of words that are presented as if they mean the opposite of what they say. Let me hit the highlights.
The idea of less testing with the same benefits is alluring.
Nicely played, because it assumes that we are getting some benefits out of the current annual testing. We are not. Not a single one. The idea of less testing is alluring because the Big Standardized Test is a waste of time, and less testing means less time wasting.
Yes, test quality must be better than it is today.
Other than that, Mrs. Lincoln, how did you like the play. Again, this assumes that there is some quality in the tests currently being used. There is not. They don't need to be improved. They need to be scrapped.
And, yes, teachers and parents have a right to be alarmed when unnecessary tests designed only for school benchmarking or teacher evaluations cut into instructional time.
A mishmosh of false assumptions. First, there are no "necessary" tests, nor have I ever read a convincing description of what a "necessary" test would be nor what would make it "necessary." And while there are no Big Standardized Tests that are actually designed for school benchmarking and teacher evaluation, in many states that is the only purpose of the BS Test! The only one! So in Aldeman's view, would those tests be okay because they are being used for purposes for which they aren't designed?
But annual testing has tremendous value. It lets schools follow students’ progress closely, and it allows for measurement of how much students learn and grow over time, not just where they are in a single moment.
Wait! What? A test is, in fact, single snapshot from a single day or couple of days-- that doesn't just give a picture of where students are at a single moment? Taking a single moment from four or five consecutive years does not let anybody follow students progress closely. This style of measurement is great for measuring student height-- and nothing else. This is like saying that the best way to assess the health of your marriage is to give your spouse a quiz one day a year.
Aldeman follows with several paragraphs pushing the disagregation argument-- that by forcing schools to measure particular groups, somebody somewhere gets a better picture of how the school is doing. It is, as always, unclear who needs this picture. You're the parent of a child in one of the groups. You believe your child is getting a good education or a bad education based on what you know about your child. How does getting disagregated data from the school change your understanding?
Besides, I thought we said a few paragraphs back that tests for measuring the school were bad and to be thrown out?
And of course that entire argument rests on the notion that the BS Test measures educational quality and there is not a molecule of evidence out there that it does so. Not. One. Molecule.
Coincidentally, the push for limiting testing has sprung up just as we’re on the cusp of having new, better tests. The Obama administration has invested $360 million and more than four years in the development of new tests, which will debut this spring. Private testing companies have responded with new offerings as well.
Oh, bullshit. New, better tests have been coming every year for a decade. They have never arrived. They will never arrive. It is not possible to create a mass-produced, mass-graded, standardized test that will measure the educational quality of every school in the country. It is like trying to use a ruler to measure the weight of a fluid-- I don't care how many times you go back to drawing board with the ruler-- it will never do the job. Educational quality cannot be measured by a standardized test. It is the wrong tool for the job, and no amount of redesign will change that.
Good reminder though that while throwing money at public schools is terrible and stupid, throwing money at testing companies is guaranteed awesome.
Annual standardized testing measures one thing-- how well a group of students does at taking an annual standardized test. That's it. Even Aldeman here avoids saying what exactly it is that these tests (you know, the "necessary ones") are supposed to measure.
Annual standardized testing is good for one other thing-- making testing companies a buttload of money. Beyond that, they are simply a waste of time and effort.
The idea of less testing with the same benefits is alluring.
Nicely played, because it assumes that we are getting some benefits out of the current annual testing. We are not. Not a single one. The idea of less testing is alluring because the Big Standardized Test is a waste of time, and less testing means less time wasting.
Yes, test quality must be better than it is today.
Other than that, Mrs. Lincoln, how did you like the play. Again, this assumes that there is some quality in the tests currently being used. There is not. They don't need to be improved. They need to be scrapped.
And, yes, teachers and parents have a right to be alarmed when unnecessary tests designed only for school benchmarking or teacher evaluations cut into instructional time.
A mishmosh of false assumptions. First, there are no "necessary" tests, nor have I ever read a convincing description of what a "necessary" test would be nor what would make it "necessary." And while there are no Big Standardized Tests that are actually designed for school benchmarking and teacher evaluation, in many states that is the only purpose of the BS Test! The only one! So in Aldeman's view, would those tests be okay because they are being used for purposes for which they aren't designed?
But annual testing has tremendous value. It lets schools follow students’ progress closely, and it allows for measurement of how much students learn and grow over time, not just where they are in a single moment.
Wait! What? A test is, in fact, single snapshot from a single day or couple of days-- that doesn't just give a picture of where students are at a single moment? Taking a single moment from four or five consecutive years does not let anybody follow students progress closely. This style of measurement is great for measuring student height-- and nothing else. This is like saying that the best way to assess the health of your marriage is to give your spouse a quiz one day a year.
Aldeman follows with several paragraphs pushing the disagregation argument-- that by forcing schools to measure particular groups, somebody somewhere gets a better picture of how the school is doing. It is, as always, unclear who needs this picture. You're the parent of a child in one of the groups. You believe your child is getting a good education or a bad education based on what you know about your child. How does getting disagregated data from the school change your understanding?
Besides, I thought we said a few paragraphs back that tests for measuring the school were bad and to be thrown out?
And of course that entire argument rests on the notion that the BS Test measures educational quality and there is not a molecule of evidence out there that it does so. Not. One. Molecule.
Coincidentally, the push for limiting testing has sprung up just as we’re on the cusp of having new, better tests. The Obama administration has invested $360 million and more than four years in the development of new tests, which will debut this spring. Private testing companies have responded with new offerings as well.
Oh, bullshit. New, better tests have been coming every year for a decade. They have never arrived. They will never arrive. It is not possible to create a mass-produced, mass-graded, standardized test that will measure the educational quality of every school in the country. It is like trying to use a ruler to measure the weight of a fluid-- I don't care how many times you go back to drawing board with the ruler-- it will never do the job. Educational quality cannot be measured by a standardized test. It is the wrong tool for the job, and no amount of redesign will change that.
Good reminder though that while throwing money at public schools is terrible and stupid, throwing money at testing companies is guaranteed awesome.
Annual standardized testing measures one thing-- how well a group of students does at taking an annual standardized test. That's it. Even Aldeman here avoids saying what exactly it is that these tests (you know, the "necessary ones") are supposed to measure.
Annual standardized testing is good for one other thing-- making testing companies a buttload of money. Beyond that, they are simply a waste of time and effort.
Friday, February 6, 2015
Goliath and the Changing Ed Conversation
If you want to see a story confirming that there are, in fact, limits to what one can accomplish with money, power and connections, look no further that Education Post. It's a giant, dusty monument to some of the differences that truly separate the reformsters from the defenders of traditional public education.
Education Post debuted on September 1st to considerable fanfare, including a nice infomercial on the launch in Washington Post. The head honcho was (and still is) Peter Cunningham. Cunningham is an old Chicago hand who traveled to DC with Arne Duncan to become the voice of Duncan's office (some others characterized him as its brains). The site was bankrolled ($12 million) by money from Bloomsburg, Broad and Walton philanthropies. It proposed to make the education debates more civil and pleasant and reasoned and based on facts-not-anecdotes, and all of that noble purpose lasted about as long as it took to post the first handful of articles that established that Education Post would be shilling hard for the Obama administration's reformster agenda. Fittingly enough, their logo features a bulhorn, not ordinarily a weapon of choice for civil, reasoned conversation.
EdWeek covered the launch and tossed up this detail about the site's function:
Education Post also will have a “rapid response” capacity to “knock down false narratives” and will focus on “hot spots” around the country where conflicts with national implications are playing out, Cunningham said.
The Washington Post profile included this:
Cunningham said some of the group's work will be behind the scenes, drafting op-ed articles for policymakers, educators, and others, as well as providing strategic advice. But a more public effort
will involve writing blog posts and responding to public misconceptions.
So what we're really talking about is a campaign politics style PR attack office determined to blitzkreig its way into control of the narrative. And they followed through swiftly. The very day I ran my first piece about the site, I had two contributors? employees? operatives? whatever you want to call thems all up in my twitter with some spicy "So when did you stop beating your wife?" challenges. Cunningham called out Jose Luis Vilson within the first week on the site.
Three weeks later the site tried to take on Carol Burris, decided to dial it back, and still mounted a weak non-conversational assault. And after that, things just got quiet.
In the first few days, the site had drawn many dissenting posts in the comments section. Those were swiftly erased. In response to the complaints, EdPost tweeted "Hoping for a better conversation. Stay tuned." But that conversation never happened-- not even a chorus of happy sock puppets to sing the praises of the stable of writers. Education Post became one more demonstration that the opposite of love is indifference.
It certainly wasn't that people on either side of the education debates hate to converse. Mike Petrilli, Andy Smarick, and Rick Hess are just three examples of hard-driving reformsters who are perfectly capable of having intelligent conversations with public school advocates.
But Education Post was not really interested in a conversation. Instead, they revealed themselves fairly quickly to be a twelve million dollar troll. They had said they wanted to amplify the voices of reformy success stories, but they also devoted time to playing gotcha with voices on the side of public education. They added a feature where they marked up pro-public-ed documents with red pen, like a petulant schoolmarm, and that didn't seem like a conversation starter, either. But clearly they had hoped that they could be at the center of education policy firestorms, and they had a box of matches and a tank of gasoline already to go but... well, nobody wanted to play. Time and again they set out the bait, grabbed ahold of their club, and waited under their bridge but.... crickets.
This is not the first time reformsters have tried to harness the interwebs and some of that social medias the kids are all tweetering about, and it's not the first time that reformsters have failed miserably doing so (see Jeb Bush/FEE's now defunct "Learn More Go Further" campaign for another example). But this might be the most expensive.
I thought I'd check to see how big the fail was, and plugged some sites into the admittedly-imperfect site Alexa.com, which ranks all the websites in the world by traffic. Here's what I got (we'll stick with US ranks and ignore the international). This is the rank in America as roughly estimated by Alexa:
EducationPost-- 223,516
Diane Ravitch's blog-- 20,380
So, Ravitch, with a staff of one and a budget of maybe a hundred bucks, cleaned their clocks. Is it their politics? Let's see what the very-reformy thinky tank Fordham Foundation site clocks in at:
EdExcellence-- 67,360
So, no, it's possible to draw some attention from their side of the tracks. Maybe other sites rank higher because they've been around longer? How about Living in Dialogue, a pro-public ed website launched at just about the same time, for considerably less that $12 million.
Living in Dialogue-- 138,616
How do they compare to a simple high school English teacher who (even though I've been online longer) just blogs in his spare time with a budget of $0.00?
Curmudgucation-- 119,612
I've checked other independent public ed bloggers, and the results are similar. We can also check metrics like sites linking in to the site-- EducationPost has 81, which is not an impressive number.
Bottom line-- in money spent per number people getting the message, EducationPost is at the bottom of the heap. It's proof once again that while the reformsters can keep outspending everybody else, that doesn't mean they're actually convincing anybody else. The reformster movement is lifted up by a giant bag of hot air, and that air is heated by constantly burning a giant pile of money. When the money runs out, or is withdrawn, the balloon will deflate and the reformster initiative will float back to earth with the rest of us.
It can seem like the reformsters are winning-- they have the pretty sites, the shiny PR, the well-paid PR rapid response operatives. What they don't have are the people who are pouring their blood and sweat and heart and soul into a cause that is bigger than profit and power.
Meanwhile, EducationPost continues to troll hard, most recently going after activist mom/blogger Sarah Blaine (because you have to stop those moms from messing wit the narrative) and Diane Ravitch herself by pointing out that she used to say different things than she does now, trying to discredit today's education activity by bringing up what she said way back in the day, as if Ravitch hadn't already written a book herself explaining what beliefs changed and why. These trolling runs have not made EducationPost a center of conversation. No firestorm. Not even a smokescreen. Just a short quiet correction from Mercedes Schneider. It is possible that EducationPost could be more efficient by simply posting, "Notice Me, Dammit" as a headline.
But it's a 2015 world, and people mostly understand that you don't feed the trolls (which is why you'll find no links in this story, or any of my newer stuff, to the EducationPost website). More than that, defenders of US public education are coming to understand that not every reformster requires or deserves a response. Paul Thomas once called for Phase Three in the resistance, and perhaps this is it-- a phase in which we realize that we are no longer backed into a corner and no longer have to respond to every cockamamie attack on public education, even as some reformsters try to get us to start up the same old fight. Maybe EducationPost is not about trying to go forward to better conversations, but to actually sucker us into the same old dynamic and thereby preserve the narrative that reformsters are the ones with all the power, while we have to fight and scrape to get our point across. They aren't Goliath. They're just a big troll on life support.
If that's the case, than the irrelevance of EducationPost (because, really, does it matter whether they close up shop or not?) is one more true sign that Things Have Changed, that money can't win everything, and that we all need to have real conversations about the future of American public education, not simply a battle of rapid-response PR blitzes and stale talking points.
The premise of EducationPost was that the conversation about public education was their conversation to be held at their table under their terms. But now they are sitting at the table alone, while more important conversations are held elsewhere. Good news for the rest of us, but if I were Bloomberg, Broad and Walton, I'd want my $12 million back.
Education Post debuted on September 1st to considerable fanfare, including a nice infomercial on the launch in Washington Post. The head honcho was (and still is) Peter Cunningham. Cunningham is an old Chicago hand who traveled to DC with Arne Duncan to become the voice of Duncan's office (some others characterized him as its brains). The site was bankrolled ($12 million) by money from Bloomsburg, Broad and Walton philanthropies. It proposed to make the education debates more civil and pleasant and reasoned and based on facts-not-anecdotes, and all of that noble purpose lasted about as long as it took to post the first handful of articles that established that Education Post would be shilling hard for the Obama administration's reformster agenda. Fittingly enough, their logo features a bulhorn, not ordinarily a weapon of choice for civil, reasoned conversation.
EdWeek covered the launch and tossed up this detail about the site's function:
Education Post also will have a “rapid response” capacity to “knock down false narratives” and will focus on “hot spots” around the country where conflicts with national implications are playing out, Cunningham said.
The Washington Post profile included this:
Cunningham said some of the group's work will be behind the scenes, drafting op-ed articles for policymakers, educators, and others, as well as providing strategic advice. But a more public effort
will involve writing blog posts and responding to public misconceptions.
So what we're really talking about is a campaign politics style PR attack office determined to blitzkreig its way into control of the narrative. And they followed through swiftly. The very day I ran my first piece about the site, I had two contributors? employees? operatives? whatever you want to call thems all up in my twitter with some spicy "So when did you stop beating your wife?" challenges. Cunningham called out Jose Luis Vilson within the first week on the site.
Three weeks later the site tried to take on Carol Burris, decided to dial it back, and still mounted a weak non-conversational assault. And after that, things just got quiet.
In the first few days, the site had drawn many dissenting posts in the comments section. Those were swiftly erased. In response to the complaints, EdPost tweeted "Hoping for a better conversation. Stay tuned." But that conversation never happened-- not even a chorus of happy sock puppets to sing the praises of the stable of writers. Education Post became one more demonstration that the opposite of love is indifference.
It certainly wasn't that people on either side of the education debates hate to converse. Mike Petrilli, Andy Smarick, and Rick Hess are just three examples of hard-driving reformsters who are perfectly capable of having intelligent conversations with public school advocates.
But Education Post was not really interested in a conversation. Instead, they revealed themselves fairly quickly to be a twelve million dollar troll. They had said they wanted to amplify the voices of reformy success stories, but they also devoted time to playing gotcha with voices on the side of public education. They added a feature where they marked up pro-public-ed documents with red pen, like a petulant schoolmarm, and that didn't seem like a conversation starter, either. But clearly they had hoped that they could be at the center of education policy firestorms, and they had a box of matches and a tank of gasoline already to go but... well, nobody wanted to play. Time and again they set out the bait, grabbed ahold of their club, and waited under their bridge but.... crickets.
This is not the first time reformsters have tried to harness the interwebs and some of that social medias the kids are all tweetering about, and it's not the first time that reformsters have failed miserably doing so (see Jeb Bush/FEE's now defunct "Learn More Go Further" campaign for another example). But this might be the most expensive.
I thought I'd check to see how big the fail was, and plugged some sites into the admittedly-imperfect site Alexa.com, which ranks all the websites in the world by traffic. Here's what I got (we'll stick with US ranks and ignore the international). This is the rank in America as roughly estimated by Alexa:
EducationPost-- 223,516
Diane Ravitch's blog-- 20,380
So, Ravitch, with a staff of one and a budget of maybe a hundred bucks, cleaned their clocks. Is it their politics? Let's see what the very-reformy thinky tank Fordham Foundation site clocks in at:
EdExcellence-- 67,360
So, no, it's possible to draw some attention from their side of the tracks. Maybe other sites rank higher because they've been around longer? How about Living in Dialogue, a pro-public ed website launched at just about the same time, for considerably less that $12 million.
Living in Dialogue-- 138,616
How do they compare to a simple high school English teacher who (even though I've been online longer) just blogs in his spare time with a budget of $0.00?
Curmudgucation-- 119,612
I've checked other independent public ed bloggers, and the results are similar. We can also check metrics like sites linking in to the site-- EducationPost has 81, which is not an impressive number.
Bottom line-- in money spent per number people getting the message, EducationPost is at the bottom of the heap. It's proof once again that while the reformsters can keep outspending everybody else, that doesn't mean they're actually convincing anybody else. The reformster movement is lifted up by a giant bag of hot air, and that air is heated by constantly burning a giant pile of money. When the money runs out, or is withdrawn, the balloon will deflate and the reformster initiative will float back to earth with the rest of us.
It can seem like the reformsters are winning-- they have the pretty sites, the shiny PR, the well-paid PR rapid response operatives. What they don't have are the people who are pouring their blood and sweat and heart and soul into a cause that is bigger than profit and power.
Meanwhile, EducationPost continues to troll hard, most recently going after activist mom/blogger Sarah Blaine (because you have to stop those moms from messing wit the narrative) and Diane Ravitch herself by pointing out that she used to say different things than she does now, trying to discredit today's education activity by bringing up what she said way back in the day, as if Ravitch hadn't already written a book herself explaining what beliefs changed and why. These trolling runs have not made EducationPost a center of conversation. No firestorm. Not even a smokescreen. Just a short quiet correction from Mercedes Schneider. It is possible that EducationPost could be more efficient by simply posting, "Notice Me, Dammit" as a headline.
But it's a 2015 world, and people mostly understand that you don't feed the trolls (which is why you'll find no links in this story, or any of my newer stuff, to the EducationPost website). More than that, defenders of US public education are coming to understand that not every reformster requires or deserves a response. Paul Thomas once called for Phase Three in the resistance, and perhaps this is it-- a phase in which we realize that we are no longer backed into a corner and no longer have to respond to every cockamamie attack on public education, even as some reformsters try to get us to start up the same old fight. Maybe EducationPost is not about trying to go forward to better conversations, but to actually sucker us into the same old dynamic and thereby preserve the narrative that reformsters are the ones with all the power, while we have to fight and scrape to get our point across. They aren't Goliath. They're just a big troll on life support.
If that's the case, than the irrelevance of EducationPost (because, really, does it matter whether they close up shop or not?) is one more true sign that Things Have Changed, that money can't win everything, and that we all need to have real conversations about the future of American public education, not simply a battle of rapid-response PR blitzes and stale talking points.
The premise of EducationPost was that the conversation about public education was their conversation to be held at their table under their terms. But now they are sitting at the table alone, while more important conversations are held elsewhere. Good news for the rest of us, but if I were Bloomberg, Broad and Walton, I'd want my $12 million back.
Reading As Relationship
Russ Walsh is an expert in reading instruction, a blogger, and (as near as I can tell) a gentleman. A recent post of his is, for my money, one of his most important ones because it collects some research and clear thinking to remind us of one of the great truths of both reading instruction in particular and education itself in general.
The post sets out to take a more nuanced look at text complexity, leaning particularly on the work of Lauren Anderson and Jamy Stillman, (Over)Simplifying Complexity: Interrogating the Press for a more Complex Text.
Both Russ's post and the original article are well worth reading in their entirety; I'm going to oversimplify them here because that's how I roll.
First, Anderson and Stillman re-support what teachers and other humans with common sense already know-- that giving a student a text above her frustration level does not actually help anything, at all. But there's more than that. Writes Walsh
They were increasingly aware that they needed to revise their definition of text complexity to include the context of the reading situation, the background knowledge and skills of the students and the reading instruction goals.
In other words, the level of challenge in any text is not something that exists as a discrete quality, separate from all others. Text difficulty (or complexity or level or whatever other name tag you want to put on these various measures) is not an objective immutable quality. How challenging a text is depends on context, on whose hands are holding it, on what purpose has been attached to it.
Instruction-- the directions and pedagogy that a teacher attaches to the text-- can change the level of challenge. If I hand first graders a copy of War and Peace and tell them to tell me how many pages or how many chapters are in the book, there is only a little bit of challenge there. If I hand seniors a copy of Green Eggs and Ham, and assign a paper using the book as basis for an analysis of social pressures on the individual as experienced in a post-agrarian society resisting the imperialism of other oppressive cultures, it is now a highly challenging text.
I have been a voracious reader for most of my life (miraculous, considering my parents failed to give me the benefit of high quality pre-pre-K when I was three). Early on, I fell in love with dinosaurs and devoured everything by Roy Chapman Andrews. When I had run out of kid books about dinosaurs, I moved on to grown-up books that were, technically, way above my reading level. But at that point I knew an awful lot about dinosaurs, so between the background knowledge I already had and my high degree of interest and motivation, I managed. On the other hand, the first book I was ever unable to finish was the classic Black Beauty, a stirring tale of some horse who does something or other and then zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz. Horses-- I neither knew nor cared about anything horsey. I returned to that book as late as 8th or 9th grade but still found that I was easily distracted from it by, well, watching grass grow would do it. It was like Black Beauty was surrounded by a special impenetrable anti-reading forcefield that would push my eyes off in any direction. Still one of the biggest textual challenges I have ever faced, and I've read Moby Dick.
We can play this anecdotal game all day. Whenever I teach seniors, I always teach them Macbeth, no matter what level students they are. But how I teach, what I teach about it, what I expect them to get out of it, what I assess them for-- that varies widely depending on the students.
Bottom line-- I cannot assess the challenge level of any reading material as a specific, objective quality in and of itself. I can do broad strokes (I feel comfortable saying that Macbeth is more challenging than Green Eggs and Ham), but the real classroom challenge of a work comes down to the relationship between the specific work, the specific students, and the pedagogical approach and techniques of the teacher.
The notion of reading difficulty as some static objective isolated quality is a common mental mistake of the reformsters, and it completely misses the importance of relationships. Current education policy is so off track that it qualifies as both necessary and radical to say that relationships matter.
Yet policy is built on ignoring relationships. Teachers are evaluated in a manner that suggests that a teacher's quality and effectiveness are somehow static, absolute, objective, isolated qualities that exist outside of any context, background or purpose. It's like insisting that if a man is a Good Husband, he will be a Good Husband for any woman selected at random from any place, age or location in the world.
Context matters. Background matters. Purpose matters. And relationships matter most of all. Relationships between students and text, students and teachers, students and each other. The fact that we don't have a handy lexile score or quality index or piece of inanely-generated "data" to measure relationships does not mean they aren't important. That's true for reading and for everything else in education as well.
The post sets out to take a more nuanced look at text complexity, leaning particularly on the work of Lauren Anderson and Jamy Stillman, (Over)Simplifying Complexity: Interrogating the Press for a more Complex Text.
Both Russ's post and the original article are well worth reading in their entirety; I'm going to oversimplify them here because that's how I roll.
First, Anderson and Stillman re-support what teachers and other humans with common sense already know-- that giving a student a text above her frustration level does not actually help anything, at all. But there's more than that. Writes Walsh
They were increasingly aware that they needed to revise their definition of text complexity to include the context of the reading situation, the background knowledge and skills of the students and the reading instruction goals.
In other words, the level of challenge in any text is not something that exists as a discrete quality, separate from all others. Text difficulty (or complexity or level or whatever other name tag you want to put on these various measures) is not an objective immutable quality. How challenging a text is depends on context, on whose hands are holding it, on what purpose has been attached to it.
Instruction-- the directions and pedagogy that a teacher attaches to the text-- can change the level of challenge. If I hand first graders a copy of War and Peace and tell them to tell me how many pages or how many chapters are in the book, there is only a little bit of challenge there. If I hand seniors a copy of Green Eggs and Ham, and assign a paper using the book as basis for an analysis of social pressures on the individual as experienced in a post-agrarian society resisting the imperialism of other oppressive cultures, it is now a highly challenging text.
I have been a voracious reader for most of my life (miraculous, considering my parents failed to give me the benefit of high quality pre-pre-K when I was three). Early on, I fell in love with dinosaurs and devoured everything by Roy Chapman Andrews. When I had run out of kid books about dinosaurs, I moved on to grown-up books that were, technically, way above my reading level. But at that point I knew an awful lot about dinosaurs, so between the background knowledge I already had and my high degree of interest and motivation, I managed. On the other hand, the first book I was ever unable to finish was the classic Black Beauty, a stirring tale of some horse who does something or other and then zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz. Horses-- I neither knew nor cared about anything horsey. I returned to that book as late as 8th or 9th grade but still found that I was easily distracted from it by, well, watching grass grow would do it. It was like Black Beauty was surrounded by a special impenetrable anti-reading forcefield that would push my eyes off in any direction. Still one of the biggest textual challenges I have ever faced, and I've read Moby Dick.
We can play this anecdotal game all day. Whenever I teach seniors, I always teach them Macbeth, no matter what level students they are. But how I teach, what I teach about it, what I expect them to get out of it, what I assess them for-- that varies widely depending on the students.
Bottom line-- I cannot assess the challenge level of any reading material as a specific, objective quality in and of itself. I can do broad strokes (I feel comfortable saying that Macbeth is more challenging than Green Eggs and Ham), but the real classroom challenge of a work comes down to the relationship between the specific work, the specific students, and the pedagogical approach and techniques of the teacher.
The notion of reading difficulty as some static objective isolated quality is a common mental mistake of the reformsters, and it completely misses the importance of relationships. Current education policy is so off track that it qualifies as both necessary and radical to say that relationships matter.
Yet policy is built on ignoring relationships. Teachers are evaluated in a manner that suggests that a teacher's quality and effectiveness are somehow static, absolute, objective, isolated qualities that exist outside of any context, background or purpose. It's like insisting that if a man is a Good Husband, he will be a Good Husband for any woman selected at random from any place, age or location in the world.
Context matters. Background matters. Purpose matters. And relationships matter most of all. Relationships between students and text, students and teachers, students and each other. The fact that we don't have a handy lexile score or quality index or piece of inanely-generated "data" to measure relationships does not mean they aren't important. That's true for reading and for everything else in education as well.
Thursday, February 5, 2015
Sen. Alexander's Choice
A reminder that if you've been thinking that Lamar Alexander is going to emerge as a public school champion, well, not so much.
Alexander spoke at Brookings (never a good sign, because what Brookings doesn't understand about education would fill a Death Star) at the release of their Education Choice and Competition Index. Arianna Prothero provides a handy short version of his remarks over at EdWeek, and the even shorter version of those remarks is "School choice is awesome and magical."
Alexander would like to "put money in the kid's backpack," because of course education is not a public trust for all citizens of the country, but a private service for families that just happens to be funded by public tax dollars.
Alexander also lays out four Things The Feds Should Do To Help School Choice.
But I want to go back to the first two for a second, because I think they have a backhanded kind of promise.
Note the phrase "of their choice."
As we've been noting for some time, school choice usually turns out to actually mean school's choice. Only Mike Petrilli at the Fordham is honest enough to 'fess up to this, but most charters reserve the right to determine who is deserving.
Imagine what might happen if the feds threw their weight behind that "of their choice" language. Imagine what might happen if charters could not turn away students who wanted to attend, no matter what. Imagine what would happen if a low-income student or a student with disabilities could not be turned down, if they could say, "I choose this charter school" and the charter school could not say no or chase them away or counsel them out or push them toward the door. Imagine if that were part of federal charter law.
I don't expect that charter operators would let such a thing happen without a fight, but it would be an awfully hard point to argue in public. Such a rule would be disastrous for modern charters, whose whole model of success rests on their ability to take and keep only the students they pick and choose.
I am no fan of school choice. But in most places we don't even have that; we have a charter system that allows the schools to do the choosing and the students just have to take what they are given, and as much as a school choice system would stink, a school's choice system is even worse.
Let's see if Alexander wants to take a real whack at that problem.
Alexander spoke at Brookings (never a good sign, because what Brookings doesn't understand about education would fill a Death Star) at the release of their Education Choice and Competition Index. Arianna Prothero provides a handy short version of his remarks over at EdWeek, and the even shorter version of those remarks is "School choice is awesome and magical."
Alexander would like to "put money in the kid's backpack," because of course education is not a public trust for all citizens of the country, but a private service for families that just happens to be funded by public tax dollars.
Alexander also lays out four Things The Feds Should Do To Help School Choice.
- Allow states to use federal dollars to create scholarships to follow low-income students to any school of their choice;
- Allow students with disabilities to spend the federal dollars allocated to them on schools of their choice;
- Expand the District of Columbia's school voucher program, called the Opportunity Scholarship Program, which is funded by Congress;
- Encourage the expansion of high-quality charter schools in the states through federal grant programs.
But I want to go back to the first two for a second, because I think they have a backhanded kind of promise.
Note the phrase "of their choice."
As we've been noting for some time, school choice usually turns out to actually mean school's choice. Only Mike Petrilli at the Fordham is honest enough to 'fess up to this, but most charters reserve the right to determine who is deserving.
Imagine what might happen if the feds threw their weight behind that "of their choice" language. Imagine what might happen if charters could not turn away students who wanted to attend, no matter what. Imagine what would happen if a low-income student or a student with disabilities could not be turned down, if they could say, "I choose this charter school" and the charter school could not say no or chase them away or counsel them out or push them toward the door. Imagine if that were part of federal charter law.
I don't expect that charter operators would let such a thing happen without a fight, but it would be an awfully hard point to argue in public. Such a rule would be disastrous for modern charters, whose whole model of success rests on their ability to take and keep only the students they pick and choose.
I am no fan of school choice. But in most places we don't even have that; we have a charter system that allows the schools to do the choosing and the students just have to take what they are given, and as much as a school choice system would stink, a school's choice system is even worse.
Let's see if Alexander wants to take a real whack at that problem.
What Role Should the Feds Have?
In the edubloggoverse, we've moved quickly from a consideration of a
possible ESEA rewrite to the real issue that will lurk behind all the
upcoming deliberations, negotiations, and arguments with your
brother-in-law at family gatherings—just how much involvement
should the federal government have in the world of public education?
This argument has percolated below the surface for quite a while now, but the ESEA and the U.S. Department of Education itself have turned the heat up by their very existence. Time to stake your position between one of two poles.
The Federal Government Should Maintain a Strong Presence in Regulating Public Education
This is the position advocated by Arne Duncan in his impassioned plea Monday to choose a new direction by staying the course (the Secretary of Education's speech may have been a little muddled). It is also the position preferred by the ACLU, the NAACP, the Business Roundtable, and a whole lot of people who hope to make a bundle in the charter school biz.
Pros:
Accountability. Standardization. A uniformity of schooling. A demand for transparency that will make it harder for states to hide their educational misbehavior. And taxpayers get to know how their money is spent. Well, the money that was borrowed in their name, anyway. National-scale resources can be brought to bear on the problems of education.
Cons:
The concentration of power and control all in one place. There are huge problems with this. With diffuse and dispersed power, you have increased probability that somebody, somewhere is coming up with the right answer. Centralized power assumes that there is One Right Answer for everyone, and that the central office always knows what that answer is. This is unlikely in the best of circumstances. If you put the central office far away from actual schooling and deep in the heart of Politicsland, you make it likely that your Secretary of Education will know way more about power and politics than about education.
Centralized power also creates a one-stop shop for powermongering. If the centralized power controls access to a large, lucrative market, it invites people who want access to that market to do their best to insert themselves into the lawmaking process. How many well-paid lobbyists did Pearson et al keep in DC before there was a Department of Education?
Centralizing power also makes a statement about what you think the "center" actually is. Centralized control by the federal government builds in the assumption that DC is, in fact, the center, and that all those local school districts are just out there on the periphery somewhere, away from the Really Important Stuff. It also re-enforces the idea that people from The Center of Really Important Stuff are best suited to travel out to the distant outposts to bring people living there the school-leading wisdom that only DC has. This is patronizing, paternalistic poop. It first creates an un-meetable necessity that those from The Center must always be right, and quickly leads to an assumption (on their part) that since they are from The Center they must be correct.
The Congressional hearings kicking off today are an example of everything bad about a centralized approach. The hearing room is far, far away from any actual school or classroom, and the entire setting and approach favors people who know how to work the politics and optics of the situation. The hearings will generate lots of sound bites and debate fodder (already those of us in the edubloggoverse are sifting through the quotes and tweets to see what we can fall upon with kisses and/or knives), and Senators will say Very Dumb Things because they don't know for sure what they're talking about, but everyone's paying attention, so they'd better say something.
Control of Education Should Rest With State and Local Authorities
This is the position favored by fans of traditional public education.
Pros:
Local control is the best guarantee that schools meet the needs and goals of the communities they serve. Direct democracy is certainly more in keeping with our nation's traditions. It acknowledges people in those communities are important, that the school and community are not outposts of the Center of All Things Important off in DC, and that those people know best how to manage the ins and outs and resources and needs and culture of their community. They are best positioned to decide what "success" should mean in their local schools.
If a local school district makes a bad policy choice, they're only making it for community (not the entire country) and therefore bad policy decisions can be recognized and contained before they make a hash of every school in the country.
Cons:
A crazy-quilt patchwork pattern of different educational programs across the country, making it impossible to accurately compare and rank different school districts and different educational programs. I'll confess—that prospect doesn't bother me in the slightest, but I understand how reasonable people can think it would be a problem.
True local control would not help us fix the problems of equity. Without federal involvement, it's far more likely that poor schools would suffer from a lack of resources, while wealthy schools flourished.
The Sticking Point Is Money
No matter how much local control fans want local control, they still need and want federal money, and federal money does not come string-free. "Have the taxpayers back a truck of money up to our door, drop it off, and never look back," is not a reasonable expectation.
Meanwhile, folks who want the federal government to drive the national education policy bus have to bump up against their own unwanted consequence—if you want to drive the bus, you have to buy the gas.
In the ideal world of Federal Control fans, the feds hand down the rules on how education ought to work, but they never have to spend a penny of taxpayer money to make it happen. In the ideal world of Local Control fans, the feds dispense as much money as it takes to make things right, but they never say a word about what to do with it.
Neither of these ideal worlds will ever happen. There are big debates in education about how to separate standards from testing, but the big inseparable pair are the conjoined twins, money and control. Every debate about federal versus local control must ultimately come back to those twins.
Originally published at View from the Cheap Seats
This argument has percolated below the surface for quite a while now, but the ESEA and the U.S. Department of Education itself have turned the heat up by their very existence. Time to stake your position between one of two poles.
The Federal Government Should Maintain a Strong Presence in Regulating Public Education
This is the position advocated by Arne Duncan in his impassioned plea Monday to choose a new direction by staying the course (the Secretary of Education's speech may have been a little muddled). It is also the position preferred by the ACLU, the NAACP, the Business Roundtable, and a whole lot of people who hope to make a bundle in the charter school biz.
Pros:
Accountability. Standardization. A uniformity of schooling. A demand for transparency that will make it harder for states to hide their educational misbehavior. And taxpayers get to know how their money is spent. Well, the money that was borrowed in their name, anyway. National-scale resources can be brought to bear on the problems of education.
Cons:
The concentration of power and control all in one place. There are huge problems with this. With diffuse and dispersed power, you have increased probability that somebody, somewhere is coming up with the right answer. Centralized power assumes that there is One Right Answer for everyone, and that the central office always knows what that answer is. This is unlikely in the best of circumstances. If you put the central office far away from actual schooling and deep in the heart of Politicsland, you make it likely that your Secretary of Education will know way more about power and politics than about education.
Centralized power also creates a one-stop shop for powermongering. If the centralized power controls access to a large, lucrative market, it invites people who want access to that market to do their best to insert themselves into the lawmaking process. How many well-paid lobbyists did Pearson et al keep in DC before there was a Department of Education?
Centralizing power also makes a statement about what you think the "center" actually is. Centralized control by the federal government builds in the assumption that DC is, in fact, the center, and that all those local school districts are just out there on the periphery somewhere, away from the Really Important Stuff. It also re-enforces the idea that people from The Center of Really Important Stuff are best suited to travel out to the distant outposts to bring people living there the school-leading wisdom that only DC has. This is patronizing, paternalistic poop. It first creates an un-meetable necessity that those from The Center must always be right, and quickly leads to an assumption (on their part) that since they are from The Center they must be correct.
The Congressional hearings kicking off today are an example of everything bad about a centralized approach. The hearing room is far, far away from any actual school or classroom, and the entire setting and approach favors people who know how to work the politics and optics of the situation. The hearings will generate lots of sound bites and debate fodder (already those of us in the edubloggoverse are sifting through the quotes and tweets to see what we can fall upon with kisses and/or knives), and Senators will say Very Dumb Things because they don't know for sure what they're talking about, but everyone's paying attention, so they'd better say something.
Control of Education Should Rest With State and Local Authorities
This is the position favored by fans of traditional public education.
Pros:
Local control is the best guarantee that schools meet the needs and goals of the communities they serve. Direct democracy is certainly more in keeping with our nation's traditions. It acknowledges people in those communities are important, that the school and community are not outposts of the Center of All Things Important off in DC, and that those people know best how to manage the ins and outs and resources and needs and culture of their community. They are best positioned to decide what "success" should mean in their local schools.
If a local school district makes a bad policy choice, they're only making it for community (not the entire country) and therefore bad policy decisions can be recognized and contained before they make a hash of every school in the country.
Cons:
A crazy-quilt patchwork pattern of different educational programs across the country, making it impossible to accurately compare and rank different school districts and different educational programs. I'll confess—that prospect doesn't bother me in the slightest, but I understand how reasonable people can think it would be a problem.
True local control would not help us fix the problems of equity. Without federal involvement, it's far more likely that poor schools would suffer from a lack of resources, while wealthy schools flourished.
The Sticking Point Is Money
No matter how much local control fans want local control, they still need and want federal money, and federal money does not come string-free. "Have the taxpayers back a truck of money up to our door, drop it off, and never look back," is not a reasonable expectation.
Meanwhile, folks who want the federal government to drive the national education policy bus have to bump up against their own unwanted consequence—if you want to drive the bus, you have to buy the gas.
In the ideal world of Federal Control fans, the feds hand down the rules on how education ought to work, but they never have to spend a penny of taxpayer money to make it happen. In the ideal world of Local Control fans, the feds dispense as much money as it takes to make things right, but they never say a word about what to do with it.
Neither of these ideal worlds will ever happen. There are big debates in education about how to separate standards from testing, but the big inseparable pair are the conjoined twins, money and control. Every debate about federal versus local control must ultimately come back to those twins.
Originally published at View from the Cheap Seats
Subscribe to:
Posts (Atom)