Last Friday, the LA Times brought the news that "a major charter expansion" is "in the works for LA Unified students." It might have also noted that the expansion was in the works for parents and taxpayers, but I suppose that's not as powerful as noting that this is For The Children.
But the lede will give you an idea of whence this wind is blowing:
A prominent local education foundation is discussing a major expansion of
charter schools in Los Angeles aimed at boosting academic achievement
for students at the lowest performing campuses.
The prominent foundation is, of course, the Eli and Edythe Broad Foundation, though apparently the folks at Keck and Walton are in on this, too, and my hat is once again off to folks who have the chutzpah to unilaterally declare themselves the head of a previously-democratic sector of society. Did somebody elect the Broad Foundation to the school board of the LA USD? No? Well, why let that stop them from going ahead and setting policy. I think I may go ahead and declare myself the chief of police here in my town, stop down to City Hall, and let them know what the new policies are going to be.
"People have been demanding better public schools forever and not
getting them," said Swati Pandey, a spokeswoman for the foundation."But we say, screw public schools-- let's just replace them with privately owned and operated charters." Ha! Okay, she's only quoted as saying that first part. I filled in the rest for her.
Folks who have attended the meetings about this unelected initiative have shared other tidbits, like a goal to enroll half of all LA students in charters over the next eight years. There also seemed to be a lot of looking at maps of where all the students trapped in failing schools are, and discussing how to get charters operating for those students.
Although they note that "an ambitious expansion of charter schools would be costly and would likely face a political fight," there's no indication of a discussion about the relative expense of supporting and improving those public schools as compared to the expensive charter-launching approach.
There's also no indication that any part of this conversation was held with the actual public school system. LAUSD board president Steve Zimmer, whatever his faults, has a quote in the article that shows he understands the problem.
"The most critical concern would be the collateral damage to the children left behind," he said.
Because this charter plan for a huger private school system (and all the major players, from Green Dot to ICEF are apparently in on this) would get its operating expenses by stripping resources from the public system.
And if you're a fan of LA school foolishness, you'll love this final line from the Times article:
The foundation declined to discuss what role, if any, Deasy is playing in the new effort.
Yesterday, the LA School School Report followed up on this "bombshell story" by getting Broad to offer some non-clarification clarification. The foundation sent an email saying, "Some schools bad. All students should have the benefit of contributing to the financial health of a privately operated charter school." I'm paraphrasing.
Because when you are announcing your intention to launch a hostile takeover of the entire public school system in a major city (or at least a takeover of its funding), the last thing you need to do is clarify yourself to the taxpayers, voters, elected officials, parents, and all those other little people that you don't have to answer to.
Tuesday, August 11, 2015
Monday, August 10, 2015
Seriously, CAP?
So there I am, passing some quick minutes on twitter, when this appears.
First of all, calling to stop politicizing Common Core is like demanding that we stop making water wet. Common Core was born in politics, sold to politicians, and pushed into the world using strictly political means and methods. It was not created by educators, and it was not pitched to educators. Instead the creators of CCSS, a group of politically-connected members of the educational-industrial complex, went straight over the heads of educators and started in finding ways to start pushing the Core. The feds helped promote it. Bill Gates became its sponsor (because, again, the fathers of Common Core did not say, "Let's see what actual teachers think about this" but instead said, "Let's go get a really rich, powerful person to become our patrons and push the crap out of this.") If you want to read about it all in painstaking detail, try Mercedes Schneider's book :
Bottom line-- you cannot "politicize" Common Core any more than you can make salt salty. And that's before we get to your real problem, which is that folks are not so much politicizing Common Core as just ignoring it as the meaningless amateur hash it is. Yes, plenty of damage is still being done in its name, but CCSS as it was originally conceived and created is pretty much dead.
Furthermore, nobody "knows" that higher standards will prepare our students for future success because A) "higher standards is a meaningless phrase and B) there isn't an iota of evidence that high standards have any effect on future success (whatever that means, exactly-- higher standards will insure you have a more compatible spouse and more attractive children? that you will be happier than low-standards people?)
This is baloney-- and stale baloney at that. And, it should be noted, being served up by a group with close ties to candidate Hillary Clinton.
Hard to believe that anybody is still pushing this ridiculous poop sandwich this late in the game. CAP, if you are going to continue to pipe up about education, at least say things that aren't so obviously foolish.
Every child deserves a high-quality education: Let’s stop politicizing the #CommonCore pic.twitter.com/Cfr1V5Ddyk
— CAP Education (@EdProgress) August 10, 2015
I checked quickly to see if it was a retweet from 2013. But no. It's the Center for American Progress, trying to sell something that I don't think anybody, anywhere, is buying any more.First of all, calling to stop politicizing Common Core is like demanding that we stop making water wet. Common Core was born in politics, sold to politicians, and pushed into the world using strictly political means and methods. It was not created by educators, and it was not pitched to educators. Instead the creators of CCSS, a group of politically-connected members of the educational-industrial complex, went straight over the heads of educators and started in finding ways to start pushing the Core. The feds helped promote it. Bill Gates became its sponsor (because, again, the fathers of Common Core did not say, "Let's see what actual teachers think about this" but instead said, "Let's go get a really rich, powerful person to become our patrons and push the crap out of this.") If you want to read about it all in painstaking detail, try Mercedes Schneider's book :
Bottom line-- you cannot "politicize" Common Core any more than you can make salt salty. And that's before we get to your real problem, which is that folks are not so much politicizing Common Core as just ignoring it as the meaningless amateur hash it is. Yes, plenty of damage is still being done in its name, but CCSS as it was originally conceived and created is pretty much dead.
Furthermore, nobody "knows" that higher standards will prepare our students for future success because A) "higher standards is a meaningless phrase and B) there isn't an iota of evidence that high standards have any effect on future success (whatever that means, exactly-- higher standards will insure you have a more compatible spouse and more attractive children? that you will be happier than low-standards people?)
This is baloney-- and stale baloney at that. And, it should be noted, being served up by a group with close ties to candidate Hillary Clinton.
Hard to believe that anybody is still pushing this ridiculous poop sandwich this late in the game. CAP, if you are going to continue to pipe up about education, at least say things that aren't so obviously foolish.
Fighting Democracy
There are two questions coming out of the discussions about New Orleans and its privatized school system.
1) Did student achievement actually improve?
2) If that improvement did happen, was it worth the price?
The answer to the first question is "Probably not" (though the careful secrecy and hoarding of data makes a definitive answer difficult), but we can still move on to the second question without a definitive answer to the first, because the price in NOLA was the suspension of "local control," which is another way to say "democracy."
On twitter and in the blogs, reformsters like to frame the struggle as one between the rights of students and the preservation of the institution. "I won't sacrifice the needs of students to preserve the privilege of the school system," is a familiar construction.
But the school system is an arm of democracy.
Granted, democracy has some problems these days. We can get angry about outside interests taking over the Douglas County school board (check out the new documentary Education, Inc for a closer look)-- but only 17% of the voters actually voted. Large cities like Chicago and New York have long since mastered the art of subverting democracy. And that's before we get to the areas where politicians have come up with new and creative ways to keep the non-white and the non-wealthy from voting.
But the solution to a problem of Not Enough Democracy is not Less Democracy.
When certain areas of the country worked to disenfranchise black citizens, the best solution, the right solution, the democratic solution was NOT for wealthy, connected folks from outside those communities to come in and say, "Tell you what. We will go ahead and get the people elected that we think you need. We're not going to give your own vote, but we will go ahead and be sure to elect people who will create programs that we think you need."
No. The solution when an American citizen is deprived of his or her right to democracy is to restore that right.
For year upon year, the response to a movement to give women a vote was to say, "Hey, they don't need it. The menfolk will vote in their best interest. "
For year upon year, the response to calls for emancipation was, "Black folks don't need to be freed. The slaveowners look after them and see that their best interests are taken care of."
It would be bad enough if the policy makers who have descended on Newark and Camden and New Orleans and Philadelphia and Chicago were simply saying, "It's okay. The parents and voters and taxpayers and citizens of these places don't need an elected school board. They don't need a vote. Wise folks from Out Of Town will look out for their best interests."
That would be bad enough. But the subtext is often worse-- These People can't be trusted to run their own schools or raise their own children, so for their own good, we're going to have to suspend democracy for them.
And so they get systems in which they have no say. Schools open and close based on business decisions, and local citizens have no say. Tax dollars are thrown left and right, past schools and into the pockets of private interests, and taxpayers have no say. Children are shipped back and forth across a city, ripping their neighborhoods apart, and the residents of those neighborhoods have no say.
Are there places where the schools are failing-- abysmally, utterly, systematically-- to serve the needs of their constituents? Absolutely. And that is a failure of democracy in and of itself (unless you're telling that some urban schools are poor and ineffective because that's what the residents of that community are demanding), a failure of elected officials to respond to the needs of their constituents. But you cannot tell me that the solution for too little democracy or ineffectively implemented democracy is to simply do away with democracy.
We don't suspend democracy or local control often in this country because it is foundational to who we are. In fact, in times like the civil rights era, we have suspended "local control" because it was not really local control at all, but an anti-democratic attempt to silence members of a community.
So how can buy the idea that among the legitimate reasons to suspend local control, to rip away an entire community's democratic power to run their own schools as a backbone of their own community, is to get better test scores on a single narrowly focused standardized test? How did we end up handing so much power to people who not only don't believe that democracy is a fundamental value, but that democracy is a problem to be stamped out?
The defenders of the NOLA privatization experiment are not just arguing for better test scores, but are arguing that stamping out local voices and stifling democratic process are a great thing for the mostly-not-white, mostly-not-wealthy people of New Orleans. I can believe that some really believe that getting those test scores up is just that important, but as long as this is the United States, they are absolutely wrong.
1) Did student achievement actually improve?
2) If that improvement did happen, was it worth the price?
The answer to the first question is "Probably not" (though the careful secrecy and hoarding of data makes a definitive answer difficult), but we can still move on to the second question without a definitive answer to the first, because the price in NOLA was the suspension of "local control," which is another way to say "democracy."
On twitter and in the blogs, reformsters like to frame the struggle as one between the rights of students and the preservation of the institution. "I won't sacrifice the needs of students to preserve the privilege of the school system," is a familiar construction.
But the school system is an arm of democracy.
Granted, democracy has some problems these days. We can get angry about outside interests taking over the Douglas County school board (check out the new documentary Education, Inc for a closer look)-- but only 17% of the voters actually voted. Large cities like Chicago and New York have long since mastered the art of subverting democracy. And that's before we get to the areas where politicians have come up with new and creative ways to keep the non-white and the non-wealthy from voting.
But the solution to a problem of Not Enough Democracy is not Less Democracy.
When certain areas of the country worked to disenfranchise black citizens, the best solution, the right solution, the democratic solution was NOT for wealthy, connected folks from outside those communities to come in and say, "Tell you what. We will go ahead and get the people elected that we think you need. We're not going to give your own vote, but we will go ahead and be sure to elect people who will create programs that we think you need."
No. The solution when an American citizen is deprived of his or her right to democracy is to restore that right.
For year upon year, the response to a movement to give women a vote was to say, "Hey, they don't need it. The menfolk will vote in their best interest. "
For year upon year, the response to calls for emancipation was, "Black folks don't need to be freed. The slaveowners look after them and see that their best interests are taken care of."
It would be bad enough if the policy makers who have descended on Newark and Camden and New Orleans and Philadelphia and Chicago were simply saying, "It's okay. The parents and voters and taxpayers and citizens of these places don't need an elected school board. They don't need a vote. Wise folks from Out Of Town will look out for their best interests."
That would be bad enough. But the subtext is often worse-- These People can't be trusted to run their own schools or raise their own children, so for their own good, we're going to have to suspend democracy for them.
And so they get systems in which they have no say. Schools open and close based on business decisions, and local citizens have no say. Tax dollars are thrown left and right, past schools and into the pockets of private interests, and taxpayers have no say. Children are shipped back and forth across a city, ripping their neighborhoods apart, and the residents of those neighborhoods have no say.
Are there places where the schools are failing-- abysmally, utterly, systematically-- to serve the needs of their constituents? Absolutely. And that is a failure of democracy in and of itself (unless you're telling that some urban schools are poor and ineffective because that's what the residents of that community are demanding), a failure of elected officials to respond to the needs of their constituents. But you cannot tell me that the solution for too little democracy or ineffectively implemented democracy is to simply do away with democracy.
We don't suspend democracy or local control often in this country because it is foundational to who we are. In fact, in times like the civil rights era, we have suspended "local control" because it was not really local control at all, but an anti-democratic attempt to silence members of a community.
So how can buy the idea that among the legitimate reasons to suspend local control, to rip away an entire community's democratic power to run their own schools as a backbone of their own community, is to get better test scores on a single narrowly focused standardized test? How did we end up handing so much power to people who not only don't believe that democracy is a fundamental value, but that democracy is a problem to be stamped out?
The defenders of the NOLA privatization experiment are not just arguing for better test scores, but are arguing that stamping out local voices and stifling democratic process are a great thing for the mostly-not-white, mostly-not-wealthy people of New Orleans. I can believe that some really believe that getting those test scores up is just that important, but as long as this is the United States, they are absolutely wrong.
Sunday, August 9, 2015
Another Writing Robo-teacher
This summer the University of Delaware was happy to unveil yet more research on yet another attempt to argue that computer software has a place in writing instruction.
Being up front
As a high school English teacher, I've thought about this a great deal, and written about it on several occasions (here, here and here, for example). And mostly I think actual useful essay-grading computers are about as probable as unicorns dancing with chartreuse polar bears in fields of asparagus. We could safely label me "Predisposed to be Skeptical."
And yet I'm determined to have an open mind. But I've been down this road before, and I recognize the Big Red Flags when I see them.
Red Flag #1: Who's Paying for This?
Assistant professor Joshus Wilson, from UD's School of Education, set out to see if the software PEGWriting could be used not just to score student writing, but to inform and assist instruction throughout the year. Why would he want to look into that?
The software Wilson used is called PEGWriting (which stands for Project Essay Grade Writing), based on work by the late education researcher Ellis B. Page and sold by Measurement Incorporated, which supports Wilson's research with indirect funding to the University.
So, the software maker paid for and perhaps commissioned this research. Just to be clear, the fact that there's no direct quid pro quo makes it worse-- if I'm counting on your funding to pay for the project I'm doing, the funding and the project can go away together and life can go on for the rest of my department. But if I'm doing research on your product over here and you're paying for, say, all our office furniture over there, the stakes are higher.
At any rate, this is clear built-in bias. Anything else?
Red Flag #2: You're Scoring What??!!
The software uses algorithms to measure more than 500 text-level variables to yield scores and feedback regarding the following characteristics of writing quality: idea development, organization, style, word choice, sentence structure, and writing conventions such as spelling and grammar.
First, I know you think it's impressive that it's measuring 500 variables ("text-level"-- as opposed to some other level?? Paper-level?), but it's not. It's like telling me that you have a vocabulary of 500 words. not so impressive, given the nature of language.
But beyond that-- PEGWriting wants to market itself as being tuned into six trait writing. I have no beef with the six traits-- I've used them myself for decades. And that's how I know that no software can possibly do what this software claims it can do.
Idea development? Really? I will bet you dollars to donuts that if I take my thesis statement ("Abe Lincoln was a great peacemaker") and develop it with absolute baloney support ("Lincoln helped bring peace by convincing Hitler to give up his siege of the Alamo"), the software will think that's swell. The software cannot read or understand ideas. It cannot assess this trait. Nor can it assess organization beyond looking for recycled prompts and transition words (Next, Furthermore, On the other hand). Nor can it have the slightest idea whether my word choices were best suited to the ideas in my essay. Any evaluation of sentence structure or style will be restricted to simply counting up types of sentences that it can (mostly) identify based on structure words and punctuation.
But robo-writing software always hits the same barrier-- the basic unit of writing is ideas, and if the software could understand ideas, the software developers would have created artificial intelligence and they'd have far more interesting things to spend time on than student writing.
No consideration of this topic can be complete without invoking my hero Les Perelman, who has made a career out of making essay-grading software look stupid. He has demonstrated over and over and over again that software does not know the difference between good writing and gibberish.
PEGWriting does enlist the teacher's help in scoring for textual evidence and content accuracy, so that's better than simply claiming the computer can do it.
Hey! A Non-red Flag
The idea is to give teachers useful diagnostic information on each writer and give them more time to address problems and assist students with things no machine can comprehend
This is a True Thing. I have my students do some fairly low-brain diagnostics on their own writing-- how many forms of "be," how many sentences, what sentence lengths, etc. Software could totally do this work and the information, when run through a human brain, can be useful, particularly in helping writings identify their tendencies and weak areas. That is the kind of thing software could totally do.
Red Flag #3: Know Your Research
Researchers have established that computer models are highly predictive of how humans would have scored a given piece of writing, Wilson said, and efforts to increase that accuracy continue.
Well, no. You can look at my short take on Perelman's work or the whole piece, but the bottom line is that the research that Wilson is most likely referring to is thoroughly unconvincing and shot full of huge holes.
Red Flag #4: That's Not a Good Thing
Wilson's research involved handing out free copies of PEGWriting to third, fourth and fifth grade classes.
Teachers said students liked the "game" aspects of the automated writing environment and that seemed to increase their motivation to write quite a bit. Because they got immediate scores on their writing, many worked to raise their scores by correcting errors and revising their work over and over.
Um, no. That's not entirely a good thing. I'll give you the positive side effect of making writing seem more fun than chorelike, but otherwise, the idea of having students learn that writing is like a game where you mess with words to score points-- well, that might prepare them for careers as internet trolls, but as with most bad writing instruction, it takes directly away from the actual point and purpose of writing, which is to say what you have to or want to say in the clearest way possible. Anything that reduces writing a mechanical activity completely divorced from the acual meaningful expressions of live humans is a Bad Thing. What could be worse than the approach described above? Oh, I know--
That same quick score produced discouragement for other students, though, teachers said, when they received low scores and could not figure out how to raise them no matter how hard they worked.
Emphasis mine. Because that "hard work" will be composed entirely of trying to mechanically manipulate pieces parts of written stuff. It will be no more about learning to write well that Super Mario Brothers is about learning how to talk to girls.
Is It All Bad?
The software does seem to offer some useful features, including an interactivity with both teacher and peer reviewers that could be handy. And I confess that I find the idea of a writing instruction platform on line to hold onto all the pieces parts of writing instruction.
Meanwhile, Wilson is looking for "efficiencies" in an approach that does seem to suggest some evolution in the marketing approach of software companies, as well as clarifying the teacher role in collaboration with the software. The old approach was to present software that would do everything for you; this research seems more focused on figuring out ways in which the software can help with instruction by saving time on things that software can actually do.
The bad news for the software manufacturers is that the answer to "what parts of writing assessment can software actually do" is "not many." I do think it's possible to create useful software, but unfortunately given how many teachers and administrators looking for a quick and easy way to writing instruction, it's unlikely that vendors won't keep trying to cash in on that market with crappy products that try to do too much that computers cannot do, resulting in more of these crappy pieces of crappity crap.
But But But
You may say that I'm quibbling about a level of writerly sophistication that only comes into play in older students, and that as long as we're just talking about elementary students, this sort of mechanical trained-monkey approach is fine.
I vehemently disagree.
The most important thing that young students learn about writing is what it is, what it's for, and how to engage with it. When we teach young students that writing is a series of mechanical tasks performed to make some teacher or software happy, we do huge long-term damage, and we turn potential writers into people who don't even know what writing is. From day one, we should be teaching them that writing is a cool way to communicate what you think and feel to other human beings, and that it starts inside your own brain and heart, not in some set of instructions.
Writing instruction done well is powerful, because young humans who are aching to be heard can discover a way to put their voice into the world, to be heard and responded to by other human beings. There is no similar excitement to be found in gaming a computer.
Being up front
As a high school English teacher, I've thought about this a great deal, and written about it on several occasions (here, here and here, for example). And mostly I think actual useful essay-grading computers are about as probable as unicorns dancing with chartreuse polar bears in fields of asparagus. We could safely label me "Predisposed to be Skeptical."
And yet I'm determined to have an open mind. But I've been down this road before, and I recognize the Big Red Flags when I see them.
Red Flag #1: Who's Paying for This?
Assistant professor Joshus Wilson, from UD's School of Education, set out to see if the software PEGWriting could be used not just to score student writing, but to inform and assist instruction throughout the year. Why would he want to look into that?
The software Wilson used is called PEGWriting (which stands for Project Essay Grade Writing), based on work by the late education researcher Ellis B. Page and sold by Measurement Incorporated, which supports Wilson's research with indirect funding to the University.
So, the software maker paid for and perhaps commissioned this research. Just to be clear, the fact that there's no direct quid pro quo makes it worse-- if I'm counting on your funding to pay for the project I'm doing, the funding and the project can go away together and life can go on for the rest of my department. But if I'm doing research on your product over here and you're paying for, say, all our office furniture over there, the stakes are higher.
At any rate, this is clear built-in bias. Anything else?
Red Flag #2: You're Scoring What??!!
The software uses algorithms to measure more than 500 text-level variables to yield scores and feedback regarding the following characteristics of writing quality: idea development, organization, style, word choice, sentence structure, and writing conventions such as spelling and grammar.
First, I know you think it's impressive that it's measuring 500 variables ("text-level"-- as opposed to some other level?? Paper-level?), but it's not. It's like telling me that you have a vocabulary of 500 words. not so impressive, given the nature of language.
But beyond that-- PEGWriting wants to market itself as being tuned into six trait writing. I have no beef with the six traits-- I've used them myself for decades. And that's how I know that no software can possibly do what this software claims it can do.
Idea development? Really? I will bet you dollars to donuts that if I take my thesis statement ("Abe Lincoln was a great peacemaker") and develop it with absolute baloney support ("Lincoln helped bring peace by convincing Hitler to give up his siege of the Alamo"), the software will think that's swell. The software cannot read or understand ideas. It cannot assess this trait. Nor can it assess organization beyond looking for recycled prompts and transition words (Next, Furthermore, On the other hand). Nor can it have the slightest idea whether my word choices were best suited to the ideas in my essay. Any evaluation of sentence structure or style will be restricted to simply counting up types of sentences that it can (mostly) identify based on structure words and punctuation.
But robo-writing software always hits the same barrier-- the basic unit of writing is ideas, and if the software could understand ideas, the software developers would have created artificial intelligence and they'd have far more interesting things to spend time on than student writing.
No consideration of this topic can be complete without invoking my hero Les Perelman, who has made a career out of making essay-grading software look stupid. He has demonstrated over and over and over again that software does not know the difference between good writing and gibberish.
PEGWriting does enlist the teacher's help in scoring for textual evidence and content accuracy, so that's better than simply claiming the computer can do it.
Hey! A Non-red Flag
The idea is to give teachers useful diagnostic information on each writer and give them more time to address problems and assist students with things no machine can comprehend
This is a True Thing. I have my students do some fairly low-brain diagnostics on their own writing-- how many forms of "be," how many sentences, what sentence lengths, etc. Software could totally do this work and the information, when run through a human brain, can be useful, particularly in helping writings identify their tendencies and weak areas. That is the kind of thing software could totally do.
Red Flag #3: Know Your Research
Researchers have established that computer models are highly predictive of how humans would have scored a given piece of writing, Wilson said, and efforts to increase that accuracy continue.
Well, no. You can look at my short take on Perelman's work or the whole piece, but the bottom line is that the research that Wilson is most likely referring to is thoroughly unconvincing and shot full of huge holes.
Red Flag #4: That's Not a Good Thing
Wilson's research involved handing out free copies of PEGWriting to third, fourth and fifth grade classes.
Teachers said students liked the "game" aspects of the automated writing environment and that seemed to increase their motivation to write quite a bit. Because they got immediate scores on their writing, many worked to raise their scores by correcting errors and revising their work over and over.
Um, no. That's not entirely a good thing. I'll give you the positive side effect of making writing seem more fun than chorelike, but otherwise, the idea of having students learn that writing is like a game where you mess with words to score points-- well, that might prepare them for careers as internet trolls, but as with most bad writing instruction, it takes directly away from the actual point and purpose of writing, which is to say what you have to or want to say in the clearest way possible. Anything that reduces writing a mechanical activity completely divorced from the acual meaningful expressions of live humans is a Bad Thing. What could be worse than the approach described above? Oh, I know--
That same quick score produced discouragement for other students, though, teachers said, when they received low scores and could not figure out how to raise them no matter how hard they worked.
Emphasis mine. Because that "hard work" will be composed entirely of trying to mechanically manipulate pieces parts of written stuff. It will be no more about learning to write well that Super Mario Brothers is about learning how to talk to girls.
Is It All Bad?
The software does seem to offer some useful features, including an interactivity with both teacher and peer reviewers that could be handy. And I confess that I find the idea of a writing instruction platform on line to hold onto all the pieces parts of writing instruction.
Meanwhile, Wilson is looking for "efficiencies" in an approach that does seem to suggest some evolution in the marketing approach of software companies, as well as clarifying the teacher role in collaboration with the software. The old approach was to present software that would do everything for you; this research seems more focused on figuring out ways in which the software can help with instruction by saving time on things that software can actually do.
The bad news for the software manufacturers is that the answer to "what parts of writing assessment can software actually do" is "not many." I do think it's possible to create useful software, but unfortunately given how many teachers and administrators looking for a quick and easy way to writing instruction, it's unlikely that vendors won't keep trying to cash in on that market with crappy products that try to do too much that computers cannot do, resulting in more of these crappy pieces of crappity crap.
But But But
You may say that I'm quibbling about a level of writerly sophistication that only comes into play in older students, and that as long as we're just talking about elementary students, this sort of mechanical trained-monkey approach is fine.
I vehemently disagree.
The most important thing that young students learn about writing is what it is, what it's for, and how to engage with it. When we teach young students that writing is a series of mechanical tasks performed to make some teacher or software happy, we do huge long-term damage, and we turn potential writers into people who don't even know what writing is. From day one, we should be teaching them that writing is a cool way to communicate what you think and feel to other human beings, and that it starts inside your own brain and heart, not in some set of instructions.
Writing instruction done well is powerful, because young humans who are aching to be heard can discover a way to put their voice into the world, to be heard and responded to by other human beings. There is no similar excitement to be found in gaming a computer.
ICYMI: Great Eduposts This Week
I missed last Sunday because vacation. But here's some of what you could be reading today.
About those charter school waiting lists
Here's a nice clear first-person account of how those wait lists get to be so large, and how little wait lists tell us about demand for charter schools.
Why schools need more teachers of color-- for white students
We already know that there's no question about the benefits of having students see teachers who look like them. But having teachers of color is also hugely beneficial for white students as well.
Is NOLA Experiencing Slave Market Education Reform
Reformsters are bummed. The release of the ERA report was supposed to unleash a storm of excitement about how awesomely successful the New Orleans school privatization experiment has been. Instead, pieces like Jennifer Berkshire's Salon article have folks questioning the whole business all over again. Jullian Vasquez Heilig storified some of the critical conversations coming out of #NOLAEdWarning so we can see just why folks aren't dancing in the streets, and just how much isn't being included in the NOLA PR blitz.
Want to know how a student is doing?
Wendy Lecker reminds us that parents already know how to find out how well their children are doing in school-- and it isn't to march to the office and demand to see Junior's standardized test results.
The Hefty ad campaign
Yes, the Hefty trash bag folks, of all people, have launched a campaign hitting the idea that schools and teachers are underfunded and undersupported.
The suicide of the liberal arts
John Agresto takes to the pages of the Wall Street Journal to make a case for the liberal arts, and to argue that the discipline needs to be a better friend to itself.
About those charter school waiting lists
Here's a nice clear first-person account of how those wait lists get to be so large, and how little wait lists tell us about demand for charter schools.
Why schools need more teachers of color-- for white students
We already know that there's no question about the benefits of having students see teachers who look like them. But having teachers of color is also hugely beneficial for white students as well.
Is NOLA Experiencing Slave Market Education Reform
Reformsters are bummed. The release of the ERA report was supposed to unleash a storm of excitement about how awesomely successful the New Orleans school privatization experiment has been. Instead, pieces like Jennifer Berkshire's Salon article have folks questioning the whole business all over again. Jullian Vasquez Heilig storified some of the critical conversations coming out of #NOLAEdWarning so we can see just why folks aren't dancing in the streets, and just how much isn't being included in the NOLA PR blitz.
Want to know how a student is doing?
Wendy Lecker reminds us that parents already know how to find out how well their children are doing in school-- and it isn't to march to the office and demand to see Junior's standardized test results.
The Hefty ad campaign
Yes, the Hefty trash bag folks, of all people, have launched a campaign hitting the idea that schools and teachers are underfunded and undersupported.
The suicide of the liberal arts
John Agresto takes to the pages of the Wall Street Journal to make a case for the liberal arts, and to argue that the discipline needs to be a better friend to itself.
So, I Have a Book
I am a fan of self-publishing. The new world of Print on Demand makes it possible to put just about anything in a book at little-to-no cost. I've used it to publish collections of pieces from my newspaper column, create book-like memorabilia for my tiny region, and to publish the history of the 159-year-old town band I play in. Every year I now collect the major project from my honors students and publish it in book form. For our first anniversary, I made my wife a book.
I'm old school. I think books are awesome. I think holding the object, turning the pages, feeling that sheet of paper move under my fingers as it shifts to reveal another field of words-- it's just awesome. Other things in the world, particularly the cyber-world, could vanish tomorrow without a trace, but a book is real, solid (yes, I know it will dry up and blow away eventually, but somehow, that's different).
So I've published a book for this blog, and I am pleased to say that it's available to the world.
The book is a collection of ninety-some pieces from this blog, and I feel it only fair to point out that there isn't anything in the book that you can't read right here for free. Some popular old favorites are in the book, like the Arne Duncan spleen piece and the Hard Part, a blog that blew up on Huff Post. If you would like your dosage of Curmudgucation in a handy read-on-the-porch-with-a-cup-of-coffee format, or if you've been trying to get people in your life to read about what's happening in education but they just hate that whole interwebs thing, this is one more available option.For me, this will be a handy Christmas gift for family members.
And if you prefer not to fund amazon, you can buy a copy directly from the publisher (which is owned by amazon, but Bezos will still get less of your money less directly).
At any rate, I'm pleased to offer rantings in a different format as one more way to get the word out. One thing I've learned over the years is that I'm very bad at doing the whole "Hey, buy my book," thing. But if you have enjoyed the blog and would like some copies of classic material in book form, hey, buy my book.
I'm old school. I think books are awesome. I think holding the object, turning the pages, feeling that sheet of paper move under my fingers as it shifts to reveal another field of words-- it's just awesome. Other things in the world, particularly the cyber-world, could vanish tomorrow without a trace, but a book is real, solid (yes, I know it will dry up and blow away eventually, but somehow, that's different).
So I've published a book for this blog, and I am pleased to say that it's available to the world.
The book is a collection of ninety-some pieces from this blog, and I feel it only fair to point out that there isn't anything in the book that you can't read right here for free. Some popular old favorites are in the book, like the Arne Duncan spleen piece and the Hard Part, a blog that blew up on Huff Post. If you would like your dosage of Curmudgucation in a handy read-on-the-porch-with-a-cup-of-coffee format, or if you've been trying to get people in your life to read about what's happening in education but they just hate that whole interwebs thing, this is one more available option.For me, this will be a handy Christmas gift for family members.
And if you prefer not to fund amazon, you can buy a copy directly from the publisher (which is owned by amazon, but Bezos will still get less of your money less directly).
At any rate, I'm pleased to offer rantings in a different format as one more way to get the word out. One thing I've learned over the years is that I'm very bad at doing the whole "Hey, buy my book," thing. But if you have enjoyed the blog and would like some copies of classic material in book form, hey, buy my book.
Saturday, August 8, 2015
Evaluating Evaluation
When discussing the current flock of evaluation tools, both for teachers and for schools, the defense always seems to work back around to this:
How are we going to know which teachers and schools are doing well or not? It's better than nothing, and we have to do something.
It's a specious argument. If I'm on a sidewalk bleeding and broken and a random stranger approaches with a running chainsaw offering to chop away, I am not going to say, "Sure, go ahead. It'll be better than nothing." Very often things that are "better than nothing," are, in fact, worse than nothing, and I would argue that VAM, for instance, is as destructive as any sidewalk chainsaw medical practice. And it's worth noting that reformsters never use the "better than nothing" argument to justify leaving an ineffective teacher or school in place.
That said, we cannot simply go to a system in which the taxpayers insert their money into a big, black box marked "School" and trust that everything inside the box is hunky dory (a system used in public schools decade ago and in charter schools currently). "Take our word for it," is not an accountability system.
I have some ideas about how to run an evaluation system, but rather than push those today (still saving them for when I retire to start my million-dollar consulting business), let's do something else. We know there are lots of ideas out there about how to evaluate schools and teachers. How can we tell the good plans from the bad plans? What are the characteristics of a good evaluation system? How can we evaluate the evaluation?
Here are the traits that are essential for a useful, viable, good evaluation system.
Give the community voice
One of the challenges of evaluation schools and teachers is that we have about seventy-eight gadzillion ideas about what schools and teachers are supposed to be doing. Teachers often find themselves in the position of a person who thought she was hired to bake pies and finds herself in trouble for having ugly upholstery on her couch.
When it comes to the question of what the schools and teachers are supposed to be doing, the primary voice that must be heard is the voice of the community that the school serves. In other words, any evaluation system that involves outside folks or government officials coming into a community and saying, "Be quiet. We will tell you what your schools should be doing" is a crappy system.
Give a survey, form a committee, do regular outreach, put community leaders in positions of power-- whatever you do, your evaluation system must be based on the priorities set and chosen by all the members of the community. If some guy in the state capital or a policy making group thinks those priorities are "wrong," that's tough. Welcome to democracy.
Embrace the chaos
Somewhere between the deeply sainted Mrs. McAwesometeach and the widely loathed Mrs. O'Suxbuckets are many teachers living in a greyer area (and, honestly, you can still find students who hated the former and who loved the latter). Teacher and school performance vary over time and over student, and the complex constellation of skills involved in teacher guarantee that there are a million different ways to be good at the job.
Any system that draws a hard, bright line between the effective and the ineffective is a crappy system, because no such line can be drawn. Can we find individuals on both extremes on which we can have clear agreement? Probably. But any system that assumes that we can clearly and decisively sort every single school and teacher is a fool's game.
This includes systems that try to distribute teachers or schools on a bell curve. The bell curve guarantees that, even if all the teachers in a school are awesome, some of them will be labeled "sucky" by the system. This also includes any system that tries to reduce teacher ratings to a single score and then tries to create a cut-off line for those scores.
I know some of you want solid hard numberfied data on schools and teachers. You can't have it. You just can't. You can't go through your neighborhood and give each couple a hard data numerical rating of their marriage, and you can't give each of your children a hard data numerical rating on how swell they are and rank their family standing accordingly.
The fundamental basis of education is relationships. You can roughly sort into "probably keepers," "probably not keepers," and "somewhere in the middle." The more precise you attempt to make your system, the more mistakes your system is going to make and the more your system is going to warp and twist and generally screw up your school.
Neither carrots nor sticks, but helping hands
The purpose of an evaluation system is to make the school better. Isn't it? I mean, was there some other purpose I'm missing? No? Good.
The stack-ranking, reward and punish systems completely abdicate systemic responsibility for improvement. They are bosses that say, "Hey, something needs to be fixed here. You, buddy!! You figure out how to fix this right now, or else" or "Hey, this needs to work better. I've got a fiver here for the first person who can get it fixed for me." In both cases, the system sloughs off all responsibility for analyzing, addressing or ameliorating any problems-- it just pushes all of that off on the evaluatees. It is the world's worst coach-- "Hey, you suck. Get back in there and suck less, somehow."
An evaluation system should produce actionable recommendations, and it should result in the necessary assistance to pursue those actions. It does not help to say to a school, "Hey, you are too poor. Be less poor, will you?" Find the problem, and get help for the problem from the appropriate source.
That means that a good evaluation system must also value--
Richness over granules
We've been saying it over and over, but it needs to be repeated until policy makers act as if they get it: a single poorly-constructed narrow standardized test of reading and math does not give us any useful any information about teacher performance.
When I was student teaching, my co-op worked with me daily, and my supervisor worked with me weekly. In my first year, that same supervisor worked with me monthly. He knew tons about me in the classroom, not just in terms of pedagogical techniquery, but how I interacted with different kinds of students in different sorts of situations. His knowledge of my skills (or lack thereof) was rich and deep, and resulted in direct coaching that was specifically tailored to me and my needs. It allowed us to address what I needed as a teacher quickly and effectively.
Compare that to someone handing me (or my principal) a bunch of student scores and saying, in effect, "Your kids' scores last year weren't good enough. Make sure they're better this year."
Information that is rich, deep, and personal is the key to driving meaningful improvement and growth. We know that to be true for students; why would it not be true for teachers and schools?
"Multiple measures" are generally a dodge by the same folks who believe that only numbers count as information. Multiple observations. Walk-throughs. Student and alumni interviews and questionnaires. Peer review. Gather a ton of information-- not data, but information. And then, the hardest part.
Judgment
All of that information has to be weighed, sown together, and judged by a live local human being.
I understand the desire to get human judgment out of the system. I am well aware that there are Jerks With Power out there, and that a big JWP can make an ungodly mess.
But you cannot create an unbiased system. You can't. Systems that are set in stone and automatically triggered by data points are just a faceless form of human bias, with every mechanized lever of the system an expression of the biases of the person who designed it (and who doesn't actually have to look anybody in the face when the system implements the designer's judgment).
You cannot take judgment out of the system. What you can do is take it out from behind the curtains and machinery that try to obscure it. What you can do is put it in the hands of professionals who understand their field well enough to put the work ahead of personal bias. What you can do is create a system where there are redundancies (many peoples' judgment is weighed) and counterbalances as simple as having to deliver your judgment face-to-face instead of through a digitized report from faceless software. What you can do is create a system where the people who exercise judgment have to own it.
Your goal is not a system devoid of human judgment, but a system where that judgment reflects the priorities of the community, the realities of the school, teachers and students, and the professionalism of the person making the call. Your goal is a culture of support and excellence and humanity; not one of data, punishment and fear.
Simple enough
Come up with a system that includes all of these features, and I think you may have something worthwhile, something that can actually help grow schools and teachers who are the best they can be. It will be far better than better than nothing.
How are we going to know which teachers and schools are doing well or not? It's better than nothing, and we have to do something.
It's a specious argument. If I'm on a sidewalk bleeding and broken and a random stranger approaches with a running chainsaw offering to chop away, I am not going to say, "Sure, go ahead. It'll be better than nothing." Very often things that are "better than nothing," are, in fact, worse than nothing, and I would argue that VAM, for instance, is as destructive as any sidewalk chainsaw medical practice. And it's worth noting that reformsters never use the "better than nothing" argument to justify leaving an ineffective teacher or school in place.
That said, we cannot simply go to a system in which the taxpayers insert their money into a big, black box marked "School" and trust that everything inside the box is hunky dory (a system used in public schools decade ago and in charter schools currently). "Take our word for it," is not an accountability system.
I have some ideas about how to run an evaluation system, but rather than push those today (still saving them for when I retire to start my million-dollar consulting business), let's do something else. We know there are lots of ideas out there about how to evaluate schools and teachers. How can we tell the good plans from the bad plans? What are the characteristics of a good evaluation system? How can we evaluate the evaluation?
Here are the traits that are essential for a useful, viable, good evaluation system.
Give the community voice
One of the challenges of evaluation schools and teachers is that we have about seventy-eight gadzillion ideas about what schools and teachers are supposed to be doing. Teachers often find themselves in the position of a person who thought she was hired to bake pies and finds herself in trouble for having ugly upholstery on her couch.
When it comes to the question of what the schools and teachers are supposed to be doing, the primary voice that must be heard is the voice of the community that the school serves. In other words, any evaluation system that involves outside folks or government officials coming into a community and saying, "Be quiet. We will tell you what your schools should be doing" is a crappy system.
Give a survey, form a committee, do regular outreach, put community leaders in positions of power-- whatever you do, your evaluation system must be based on the priorities set and chosen by all the members of the community. If some guy in the state capital or a policy making group thinks those priorities are "wrong," that's tough. Welcome to democracy.
Embrace the chaos
Somewhere between the deeply sainted Mrs. McAwesometeach and the widely loathed Mrs. O'Suxbuckets are many teachers living in a greyer area (and, honestly, you can still find students who hated the former and who loved the latter). Teacher and school performance vary over time and over student, and the complex constellation of skills involved in teacher guarantee that there are a million different ways to be good at the job.
Any system that draws a hard, bright line between the effective and the ineffective is a crappy system, because no such line can be drawn. Can we find individuals on both extremes on which we can have clear agreement? Probably. But any system that assumes that we can clearly and decisively sort every single school and teacher is a fool's game.
This includes systems that try to distribute teachers or schools on a bell curve. The bell curve guarantees that, even if all the teachers in a school are awesome, some of them will be labeled "sucky" by the system. This also includes any system that tries to reduce teacher ratings to a single score and then tries to create a cut-off line for those scores.
I know some of you want solid hard numberfied data on schools and teachers. You can't have it. You just can't. You can't go through your neighborhood and give each couple a hard data numerical rating of their marriage, and you can't give each of your children a hard data numerical rating on how swell they are and rank their family standing accordingly.
The fundamental basis of education is relationships. You can roughly sort into "probably keepers," "probably not keepers," and "somewhere in the middle." The more precise you attempt to make your system, the more mistakes your system is going to make and the more your system is going to warp and twist and generally screw up your school.
Neither carrots nor sticks, but helping hands
The purpose of an evaluation system is to make the school better. Isn't it? I mean, was there some other purpose I'm missing? No? Good.
The stack-ranking, reward and punish systems completely abdicate systemic responsibility for improvement. They are bosses that say, "Hey, something needs to be fixed here. You, buddy!! You figure out how to fix this right now, or else" or "Hey, this needs to work better. I've got a fiver here for the first person who can get it fixed for me." In both cases, the system sloughs off all responsibility for analyzing, addressing or ameliorating any problems-- it just pushes all of that off on the evaluatees. It is the world's worst coach-- "Hey, you suck. Get back in there and suck less, somehow."
An evaluation system should produce actionable recommendations, and it should result in the necessary assistance to pursue those actions. It does not help to say to a school, "Hey, you are too poor. Be less poor, will you?" Find the problem, and get help for the problem from the appropriate source.
That means that a good evaluation system must also value--
Richness over granules
We've been saying it over and over, but it needs to be repeated until policy makers act as if they get it: a single poorly-constructed narrow standardized test of reading and math does not give us any useful any information about teacher performance.
When I was student teaching, my co-op worked with me daily, and my supervisor worked with me weekly. In my first year, that same supervisor worked with me monthly. He knew tons about me in the classroom, not just in terms of pedagogical techniquery, but how I interacted with different kinds of students in different sorts of situations. His knowledge of my skills (or lack thereof) was rich and deep, and resulted in direct coaching that was specifically tailored to me and my needs. It allowed us to address what I needed as a teacher quickly and effectively.
Compare that to someone handing me (or my principal) a bunch of student scores and saying, in effect, "Your kids' scores last year weren't good enough. Make sure they're better this year."
Information that is rich, deep, and personal is the key to driving meaningful improvement and growth. We know that to be true for students; why would it not be true for teachers and schools?
"Multiple measures" are generally a dodge by the same folks who believe that only numbers count as information. Multiple observations. Walk-throughs. Student and alumni interviews and questionnaires. Peer review. Gather a ton of information-- not data, but information. And then, the hardest part.
Judgment
All of that information has to be weighed, sown together, and judged by a live local human being.
I understand the desire to get human judgment out of the system. I am well aware that there are Jerks With Power out there, and that a big JWP can make an ungodly mess.
But you cannot create an unbiased system. You can't. Systems that are set in stone and automatically triggered by data points are just a faceless form of human bias, with every mechanized lever of the system an expression of the biases of the person who designed it (and who doesn't actually have to look anybody in the face when the system implements the designer's judgment).
You cannot take judgment out of the system. What you can do is take it out from behind the curtains and machinery that try to obscure it. What you can do is put it in the hands of professionals who understand their field well enough to put the work ahead of personal bias. What you can do is create a system where there are redundancies (many peoples' judgment is weighed) and counterbalances as simple as having to deliver your judgment face-to-face instead of through a digitized report from faceless software. What you can do is create a system where the people who exercise judgment have to own it.
Your goal is not a system devoid of human judgment, but a system where that judgment reflects the priorities of the community, the realities of the school, teachers and students, and the professionalism of the person making the call. Your goal is a culture of support and excellence and humanity; not one of data, punishment and fear.
Simple enough
Come up with a system that includes all of these features, and I think you may have something worthwhile, something that can actually help grow schools and teachers who are the best they can be. It will be far better than better than nothing.
Subscribe to:
Posts (Atom)