The George Washington University will no longer require most
undergraduate applicants to submit SAT or ACT test scores, effective
Aug. 1.
With one sentence, the folks at GW made life just a little bit harder for standardized test manufacturers.
It's
not just that another college has dropped the popular testing product
requirement. Doing so touched off the usual round of press. The folks at FairTest
reminded us that over 850 colleges and universities no longer require
students to jump through the old hoops. Valerie Strauss helpfully broke out some striking lists
of top-ranked (well, top-ranked by the not-entirely-believable US News
ranking system) schools that don't require students to plunk down good
money for a bad evaluation of their post-secondary prospects. Bowdoin!
Bryn Mawr! Wake Forest! Hey, even my own alma mater—way to go,
Allegheny!
It's not just that this calls up all the old
objections, all the things we already knew about the tests. Cue the
regular research showing cultural and racial bias in the tests. Trot out the 2014 research showing that high school grades are better predictors of
college success than SAT's. Discuss the validity of tests so game-able
that an entire industry has sprung up around gaming them.
But the
bigger problem is the continued drawing of blood. Like the god-king who
is nicked by a spear, the SAT and ACT are most hurt by the wound that
reveals they are not what they say they are.
For generations here
in the East, the SAT was just something you do. In the seventies, my
classmates and I didn't even ask where the SAT came from—as
far as we knew it was some sort of government-college co-operative,
mandated and required, like a driver's test or vaccinations. Even
"College Board" sounds like some sort of official regulatory agency.
The
testing companies have fought to maintain that illusion. David Coleman
and the College Board can still magically turn a drop in market share
into a referendum on America's education system. They've worked to market a racially biased instrument as a tool for social justice. And Coleman has stumped hard for the virtues of his new, improved version of the SAT, continuing his work to redefine what college-ready actually means.
They've also managed the impressive trick of getting states to include their product as part of school evaluation formulas. This is tantamount to getting the government to require cars to have a Ford nameplate in order to pass inspection.
But
the continued defection of top schools has drawn blood, and the growth
of the opt-out movement has armed the peasants with even more spears.
More and more people are coming to see the SAT and ACT for what they are—products
for sale. Much of the SAT and ACT customer base is people who pay for
the product because, well, you know, you have to, right? There's no
choice, right?
But with every passing defection it becomes clear
there is a choice. There's a choice for students, and there's most
especially a choice for colleges and universities. The SAT is not a
necessary rite of passage. It's just another six-pack of snake oil, an
expensive con that's long on drawbacks and short on benefits. The SAT
and ACT may still rule the majority of college domains for now, but it
is increasingly clear that the emperor is both fully mortal and mostly
naked.
Originally published at View from the Cheap Seats
Wednesday, August 12, 2015
Tuesday, August 11, 2015
LA Charter Onslaught
Last Friday, the LA Times brought the news that "a major charter expansion" is "in the works for LA Unified students." It might have also noted that the expansion was in the works for parents and taxpayers, but I suppose that's not as powerful as noting that this is For The Children.
But the lede will give you an idea of whence this wind is blowing:
A prominent local education foundation is discussing a major expansion of charter schools in Los Angeles aimed at boosting academic achievement for students at the lowest performing campuses.
The prominent foundation is, of course, the Eli and Edythe Broad Foundation, though apparently the folks at Keck and Walton are in on this, too, and my hat is once again off to folks who have the chutzpah to unilaterally declare themselves the head of a previously-democratic sector of society. Did somebody elect the Broad Foundation to the school board of the LA USD? No? Well, why let that stop them from going ahead and setting policy. I think I may go ahead and declare myself the chief of police here in my town, stop down to City Hall, and let them know what the new policies are going to be.
"People have been demanding better public schools forever and not getting them," said Swati Pandey, a spokeswoman for the foundation."But we say, screw public schools-- let's just replace them with privately owned and operated charters." Ha! Okay, she's only quoted as saying that first part. I filled in the rest for her.
Folks who have attended the meetings about this unelected initiative have shared other tidbits, like a goal to enroll half of all LA students in charters over the next eight years. There also seemed to be a lot of looking at maps of where all the students trapped in failing schools are, and discussing how to get charters operating for those students.
Although they note that "an ambitious expansion of charter schools would be costly and would likely face a political fight," there's no indication of a discussion about the relative expense of supporting and improving those public schools as compared to the expensive charter-launching approach.
There's also no indication that any part of this conversation was held with the actual public school system. LAUSD board president Steve Zimmer, whatever his faults, has a quote in the article that shows he understands the problem.
"The most critical concern would be the collateral damage to the children left behind," he said.
Because this charter plan for a huger private school system (and all the major players, from Green Dot to ICEF are apparently in on this) would get its operating expenses by stripping resources from the public system.
And if you're a fan of LA school foolishness, you'll love this final line from the Times article:
The foundation declined to discuss what role, if any, Deasy is playing in the new effort.
Yesterday, the LA School School Report followed up on this "bombshell story" by getting Broad to offer some non-clarification clarification. The foundation sent an email saying, "Some schools bad. All students should have the benefit of contributing to the financial health of a privately operated charter school." I'm paraphrasing.
Because when you are announcing your intention to launch a hostile takeover of the entire public school system in a major city (or at least a takeover of its funding), the last thing you need to do is clarify yourself to the taxpayers, voters, elected officials, parents, and all those other little people that you don't have to answer to.
But the lede will give you an idea of whence this wind is blowing:
A prominent local education foundation is discussing a major expansion of charter schools in Los Angeles aimed at boosting academic achievement for students at the lowest performing campuses.
The prominent foundation is, of course, the Eli and Edythe Broad Foundation, though apparently the folks at Keck and Walton are in on this, too, and my hat is once again off to folks who have the chutzpah to unilaterally declare themselves the head of a previously-democratic sector of society. Did somebody elect the Broad Foundation to the school board of the LA USD? No? Well, why let that stop them from going ahead and setting policy. I think I may go ahead and declare myself the chief of police here in my town, stop down to City Hall, and let them know what the new policies are going to be.
"People have been demanding better public schools forever and not getting them," said Swati Pandey, a spokeswoman for the foundation."But we say, screw public schools-- let's just replace them with privately owned and operated charters." Ha! Okay, she's only quoted as saying that first part. I filled in the rest for her.
Folks who have attended the meetings about this unelected initiative have shared other tidbits, like a goal to enroll half of all LA students in charters over the next eight years. There also seemed to be a lot of looking at maps of where all the students trapped in failing schools are, and discussing how to get charters operating for those students.
Although they note that "an ambitious expansion of charter schools would be costly and would likely face a political fight," there's no indication of a discussion about the relative expense of supporting and improving those public schools as compared to the expensive charter-launching approach.
There's also no indication that any part of this conversation was held with the actual public school system. LAUSD board president Steve Zimmer, whatever his faults, has a quote in the article that shows he understands the problem.
"The most critical concern would be the collateral damage to the children left behind," he said.
Because this charter plan for a huger private school system (and all the major players, from Green Dot to ICEF are apparently in on this) would get its operating expenses by stripping resources from the public system.
And if you're a fan of LA school foolishness, you'll love this final line from the Times article:
The foundation declined to discuss what role, if any, Deasy is playing in the new effort.
Yesterday, the LA School School Report followed up on this "bombshell story" by getting Broad to offer some non-clarification clarification. The foundation sent an email saying, "Some schools bad. All students should have the benefit of contributing to the financial health of a privately operated charter school." I'm paraphrasing.
Because when you are announcing your intention to launch a hostile takeover of the entire public school system in a major city (or at least a takeover of its funding), the last thing you need to do is clarify yourself to the taxpayers, voters, elected officials, parents, and all those other little people that you don't have to answer to.
Monday, August 10, 2015
Seriously, CAP?
So there I am, passing some quick minutes on twitter, when this appears.
First of all, calling to stop politicizing Common Core is like demanding that we stop making water wet. Common Core was born in politics, sold to politicians, and pushed into the world using strictly political means and methods. It was not created by educators, and it was not pitched to educators. Instead the creators of CCSS, a group of politically-connected members of the educational-industrial complex, went straight over the heads of educators and started in finding ways to start pushing the Core. The feds helped promote it. Bill Gates became its sponsor (because, again, the fathers of Common Core did not say, "Let's see what actual teachers think about this" but instead said, "Let's go get a really rich, powerful person to become our patrons and push the crap out of this.") If you want to read about it all in painstaking detail, try Mercedes Schneider's book :
Bottom line-- you cannot "politicize" Common Core any more than you can make salt salty. And that's before we get to your real problem, which is that folks are not so much politicizing Common Core as just ignoring it as the meaningless amateur hash it is. Yes, plenty of damage is still being done in its name, but CCSS as it was originally conceived and created is pretty much dead.
Furthermore, nobody "knows" that higher standards will prepare our students for future success because A) "higher standards is a meaningless phrase and B) there isn't an iota of evidence that high standards have any effect on future success (whatever that means, exactly-- higher standards will insure you have a more compatible spouse and more attractive children? that you will be happier than low-standards people?)
This is baloney-- and stale baloney at that. And, it should be noted, being served up by a group with close ties to candidate Hillary Clinton.
Hard to believe that anybody is still pushing this ridiculous poop sandwich this late in the game. CAP, if you are going to continue to pipe up about education, at least say things that aren't so obviously foolish.
Every child deserves a high-quality education: Let’s stop politicizing the #CommonCore pic.twitter.com/Cfr1V5Ddyk
— CAP Education (@EdProgress) August 10, 2015
I checked quickly to see if it was a retweet from 2013. But no. It's the Center for American Progress, trying to sell something that I don't think anybody, anywhere, is buying any more.First of all, calling to stop politicizing Common Core is like demanding that we stop making water wet. Common Core was born in politics, sold to politicians, and pushed into the world using strictly political means and methods. It was not created by educators, and it was not pitched to educators. Instead the creators of CCSS, a group of politically-connected members of the educational-industrial complex, went straight over the heads of educators and started in finding ways to start pushing the Core. The feds helped promote it. Bill Gates became its sponsor (because, again, the fathers of Common Core did not say, "Let's see what actual teachers think about this" but instead said, "Let's go get a really rich, powerful person to become our patrons and push the crap out of this.") If you want to read about it all in painstaking detail, try Mercedes Schneider's book :
Bottom line-- you cannot "politicize" Common Core any more than you can make salt salty. And that's before we get to your real problem, which is that folks are not so much politicizing Common Core as just ignoring it as the meaningless amateur hash it is. Yes, plenty of damage is still being done in its name, but CCSS as it was originally conceived and created is pretty much dead.
Furthermore, nobody "knows" that higher standards will prepare our students for future success because A) "higher standards is a meaningless phrase and B) there isn't an iota of evidence that high standards have any effect on future success (whatever that means, exactly-- higher standards will insure you have a more compatible spouse and more attractive children? that you will be happier than low-standards people?)
This is baloney-- and stale baloney at that. And, it should be noted, being served up by a group with close ties to candidate Hillary Clinton.
Hard to believe that anybody is still pushing this ridiculous poop sandwich this late in the game. CAP, if you are going to continue to pipe up about education, at least say things that aren't so obviously foolish.
Fighting Democracy
There are two questions coming out of the discussions about New Orleans and its privatized school system.
1) Did student achievement actually improve?
2) If that improvement did happen, was it worth the price?
The answer to the first question is "Probably not" (though the careful secrecy and hoarding of data makes a definitive answer difficult), but we can still move on to the second question without a definitive answer to the first, because the price in NOLA was the suspension of "local control," which is another way to say "democracy."
On twitter and in the blogs, reformsters like to frame the struggle as one between the rights of students and the preservation of the institution. "I won't sacrifice the needs of students to preserve the privilege of the school system," is a familiar construction.
But the school system is an arm of democracy.
Granted, democracy has some problems these days. We can get angry about outside interests taking over the Douglas County school board (check out the new documentary Education, Inc for a closer look)-- but only 17% of the voters actually voted. Large cities like Chicago and New York have long since mastered the art of subverting democracy. And that's before we get to the areas where politicians have come up with new and creative ways to keep the non-white and the non-wealthy from voting.
But the solution to a problem of Not Enough Democracy is not Less Democracy.
When certain areas of the country worked to disenfranchise black citizens, the best solution, the right solution, the democratic solution was NOT for wealthy, connected folks from outside those communities to come in and say, "Tell you what. We will go ahead and get the people elected that we think you need. We're not going to give your own vote, but we will go ahead and be sure to elect people who will create programs that we think you need."
No. The solution when an American citizen is deprived of his or her right to democracy is to restore that right.
For year upon year, the response to a movement to give women a vote was to say, "Hey, they don't need it. The menfolk will vote in their best interest. "
For year upon year, the response to calls for emancipation was, "Black folks don't need to be freed. The slaveowners look after them and see that their best interests are taken care of."
It would be bad enough if the policy makers who have descended on Newark and Camden and New Orleans and Philadelphia and Chicago were simply saying, "It's okay. The parents and voters and taxpayers and citizens of these places don't need an elected school board. They don't need a vote. Wise folks from Out Of Town will look out for their best interests."
That would be bad enough. But the subtext is often worse-- These People can't be trusted to run their own schools or raise their own children, so for their own good, we're going to have to suspend democracy for them.
And so they get systems in which they have no say. Schools open and close based on business decisions, and local citizens have no say. Tax dollars are thrown left and right, past schools and into the pockets of private interests, and taxpayers have no say. Children are shipped back and forth across a city, ripping their neighborhoods apart, and the residents of those neighborhoods have no say.
Are there places where the schools are failing-- abysmally, utterly, systematically-- to serve the needs of their constituents? Absolutely. And that is a failure of democracy in and of itself (unless you're telling that some urban schools are poor and ineffective because that's what the residents of that community are demanding), a failure of elected officials to respond to the needs of their constituents. But you cannot tell me that the solution for too little democracy or ineffectively implemented democracy is to simply do away with democracy.
We don't suspend democracy or local control often in this country because it is foundational to who we are. In fact, in times like the civil rights era, we have suspended "local control" because it was not really local control at all, but an anti-democratic attempt to silence members of a community.
So how can buy the idea that among the legitimate reasons to suspend local control, to rip away an entire community's democratic power to run their own schools as a backbone of their own community, is to get better test scores on a single narrowly focused standardized test? How did we end up handing so much power to people who not only don't believe that democracy is a fundamental value, but that democracy is a problem to be stamped out?
The defenders of the NOLA privatization experiment are not just arguing for better test scores, but are arguing that stamping out local voices and stifling democratic process are a great thing for the mostly-not-white, mostly-not-wealthy people of New Orleans. I can believe that some really believe that getting those test scores up is just that important, but as long as this is the United States, they are absolutely wrong.
1) Did student achievement actually improve?
2) If that improvement did happen, was it worth the price?
The answer to the first question is "Probably not" (though the careful secrecy and hoarding of data makes a definitive answer difficult), but we can still move on to the second question without a definitive answer to the first, because the price in NOLA was the suspension of "local control," which is another way to say "democracy."
On twitter and in the blogs, reformsters like to frame the struggle as one between the rights of students and the preservation of the institution. "I won't sacrifice the needs of students to preserve the privilege of the school system," is a familiar construction.
But the school system is an arm of democracy.
Granted, democracy has some problems these days. We can get angry about outside interests taking over the Douglas County school board (check out the new documentary Education, Inc for a closer look)-- but only 17% of the voters actually voted. Large cities like Chicago and New York have long since mastered the art of subverting democracy. And that's before we get to the areas where politicians have come up with new and creative ways to keep the non-white and the non-wealthy from voting.
But the solution to a problem of Not Enough Democracy is not Less Democracy.
When certain areas of the country worked to disenfranchise black citizens, the best solution, the right solution, the democratic solution was NOT for wealthy, connected folks from outside those communities to come in and say, "Tell you what. We will go ahead and get the people elected that we think you need. We're not going to give your own vote, but we will go ahead and be sure to elect people who will create programs that we think you need."
No. The solution when an American citizen is deprived of his or her right to democracy is to restore that right.
For year upon year, the response to a movement to give women a vote was to say, "Hey, they don't need it. The menfolk will vote in their best interest. "
For year upon year, the response to calls for emancipation was, "Black folks don't need to be freed. The slaveowners look after them and see that their best interests are taken care of."
It would be bad enough if the policy makers who have descended on Newark and Camden and New Orleans and Philadelphia and Chicago were simply saying, "It's okay. The parents and voters and taxpayers and citizens of these places don't need an elected school board. They don't need a vote. Wise folks from Out Of Town will look out for their best interests."
That would be bad enough. But the subtext is often worse-- These People can't be trusted to run their own schools or raise their own children, so for their own good, we're going to have to suspend democracy for them.
And so they get systems in which they have no say. Schools open and close based on business decisions, and local citizens have no say. Tax dollars are thrown left and right, past schools and into the pockets of private interests, and taxpayers have no say. Children are shipped back and forth across a city, ripping their neighborhoods apart, and the residents of those neighborhoods have no say.
Are there places where the schools are failing-- abysmally, utterly, systematically-- to serve the needs of their constituents? Absolutely. And that is a failure of democracy in and of itself (unless you're telling that some urban schools are poor and ineffective because that's what the residents of that community are demanding), a failure of elected officials to respond to the needs of their constituents. But you cannot tell me that the solution for too little democracy or ineffectively implemented democracy is to simply do away with democracy.
We don't suspend democracy or local control often in this country because it is foundational to who we are. In fact, in times like the civil rights era, we have suspended "local control" because it was not really local control at all, but an anti-democratic attempt to silence members of a community.
So how can buy the idea that among the legitimate reasons to suspend local control, to rip away an entire community's democratic power to run their own schools as a backbone of their own community, is to get better test scores on a single narrowly focused standardized test? How did we end up handing so much power to people who not only don't believe that democracy is a fundamental value, but that democracy is a problem to be stamped out?
The defenders of the NOLA privatization experiment are not just arguing for better test scores, but are arguing that stamping out local voices and stifling democratic process are a great thing for the mostly-not-white, mostly-not-wealthy people of New Orleans. I can believe that some really believe that getting those test scores up is just that important, but as long as this is the United States, they are absolutely wrong.
Sunday, August 9, 2015
Another Writing Robo-teacher
This summer the University of Delaware was happy to unveil yet more research on yet another attempt to argue that computer software has a place in writing instruction.
Being up front
As a high school English teacher, I've thought about this a great deal, and written about it on several occasions (here, here and here, for example). And mostly I think actual useful essay-grading computers are about as probable as unicorns dancing with chartreuse polar bears in fields of asparagus. We could safely label me "Predisposed to be Skeptical."
And yet I'm determined to have an open mind. But I've been down this road before, and I recognize the Big Red Flags when I see them.
Red Flag #1: Who's Paying for This?
Assistant professor Joshus Wilson, from UD's School of Education, set out to see if the software PEGWriting could be used not just to score student writing, but to inform and assist instruction throughout the year. Why would he want to look into that?
The software Wilson used is called PEGWriting (which stands for Project Essay Grade Writing), based on work by the late education researcher Ellis B. Page and sold by Measurement Incorporated, which supports Wilson's research with indirect funding to the University.
So, the software maker paid for and perhaps commissioned this research. Just to be clear, the fact that there's no direct quid pro quo makes it worse-- if I'm counting on your funding to pay for the project I'm doing, the funding and the project can go away together and life can go on for the rest of my department. But if I'm doing research on your product over here and you're paying for, say, all our office furniture over there, the stakes are higher.
At any rate, this is clear built-in bias. Anything else?
Red Flag #2: You're Scoring What??!!
The software uses algorithms to measure more than 500 text-level variables to yield scores and feedback regarding the following characteristics of writing quality: idea development, organization, style, word choice, sentence structure, and writing conventions such as spelling and grammar.
First, I know you think it's impressive that it's measuring 500 variables ("text-level"-- as opposed to some other level?? Paper-level?), but it's not. It's like telling me that you have a vocabulary of 500 words. not so impressive, given the nature of language.
But beyond that-- PEGWriting wants to market itself as being tuned into six trait writing. I have no beef with the six traits-- I've used them myself for decades. And that's how I know that no software can possibly do what this software claims it can do.
Idea development? Really? I will bet you dollars to donuts that if I take my thesis statement ("Abe Lincoln was a great peacemaker") and develop it with absolute baloney support ("Lincoln helped bring peace by convincing Hitler to give up his siege of the Alamo"), the software will think that's swell. The software cannot read or understand ideas. It cannot assess this trait. Nor can it assess organization beyond looking for recycled prompts and transition words (Next, Furthermore, On the other hand). Nor can it have the slightest idea whether my word choices were best suited to the ideas in my essay. Any evaluation of sentence structure or style will be restricted to simply counting up types of sentences that it can (mostly) identify based on structure words and punctuation.
But robo-writing software always hits the same barrier-- the basic unit of writing is ideas, and if the software could understand ideas, the software developers would have created artificial intelligence and they'd have far more interesting things to spend time on than student writing.
No consideration of this topic can be complete without invoking my hero Les Perelman, who has made a career out of making essay-grading software look stupid. He has demonstrated over and over and over again that software does not know the difference between good writing and gibberish.
PEGWriting does enlist the teacher's help in scoring for textual evidence and content accuracy, so that's better than simply claiming the computer can do it.
Hey! A Non-red Flag
The idea is to give teachers useful diagnostic information on each writer and give them more time to address problems and assist students with things no machine can comprehend
This is a True Thing. I have my students do some fairly low-brain diagnostics on their own writing-- how many forms of "be," how many sentences, what sentence lengths, etc. Software could totally do this work and the information, when run through a human brain, can be useful, particularly in helping writings identify their tendencies and weak areas. That is the kind of thing software could totally do.
Red Flag #3: Know Your Research
Researchers have established that computer models are highly predictive of how humans would have scored a given piece of writing, Wilson said, and efforts to increase that accuracy continue.
Well, no. You can look at my short take on Perelman's work or the whole piece, but the bottom line is that the research that Wilson is most likely referring to is thoroughly unconvincing and shot full of huge holes.
Red Flag #4: That's Not a Good Thing
Wilson's research involved handing out free copies of PEGWriting to third, fourth and fifth grade classes.
Teachers said students liked the "game" aspects of the automated writing environment and that seemed to increase their motivation to write quite a bit. Because they got immediate scores on their writing, many worked to raise their scores by correcting errors and revising their work over and over.
Um, no. That's not entirely a good thing. I'll give you the positive side effect of making writing seem more fun than chorelike, but otherwise, the idea of having students learn that writing is like a game where you mess with words to score points-- well, that might prepare them for careers as internet trolls, but as with most bad writing instruction, it takes directly away from the actual point and purpose of writing, which is to say what you have to or want to say in the clearest way possible. Anything that reduces writing a mechanical activity completely divorced from the acual meaningful expressions of live humans is a Bad Thing. What could be worse than the approach described above? Oh, I know--
That same quick score produced discouragement for other students, though, teachers said, when they received low scores and could not figure out how to raise them no matter how hard they worked.
Emphasis mine. Because that "hard work" will be composed entirely of trying to mechanically manipulate pieces parts of written stuff. It will be no more about learning to write well that Super Mario Brothers is about learning how to talk to girls.
Is It All Bad?
The software does seem to offer some useful features, including an interactivity with both teacher and peer reviewers that could be handy. And I confess that I find the idea of a writing instruction platform on line to hold onto all the pieces parts of writing instruction.
Meanwhile, Wilson is looking for "efficiencies" in an approach that does seem to suggest some evolution in the marketing approach of software companies, as well as clarifying the teacher role in collaboration with the software. The old approach was to present software that would do everything for you; this research seems more focused on figuring out ways in which the software can help with instruction by saving time on things that software can actually do.
The bad news for the software manufacturers is that the answer to "what parts of writing assessment can software actually do" is "not many." I do think it's possible to create useful software, but unfortunately given how many teachers and administrators looking for a quick and easy way to writing instruction, it's unlikely that vendors won't keep trying to cash in on that market with crappy products that try to do too much that computers cannot do, resulting in more of these crappy pieces of crappity crap.
But But But
You may say that I'm quibbling about a level of writerly sophistication that only comes into play in older students, and that as long as we're just talking about elementary students, this sort of mechanical trained-monkey approach is fine.
I vehemently disagree.
The most important thing that young students learn about writing is what it is, what it's for, and how to engage with it. When we teach young students that writing is a series of mechanical tasks performed to make some teacher or software happy, we do huge long-term damage, and we turn potential writers into people who don't even know what writing is. From day one, we should be teaching them that writing is a cool way to communicate what you think and feel to other human beings, and that it starts inside your own brain and heart, not in some set of instructions.
Writing instruction done well is powerful, because young humans who are aching to be heard can discover a way to put their voice into the world, to be heard and responded to by other human beings. There is no similar excitement to be found in gaming a computer.
Being up front
As a high school English teacher, I've thought about this a great deal, and written about it on several occasions (here, here and here, for example). And mostly I think actual useful essay-grading computers are about as probable as unicorns dancing with chartreuse polar bears in fields of asparagus. We could safely label me "Predisposed to be Skeptical."
And yet I'm determined to have an open mind. But I've been down this road before, and I recognize the Big Red Flags when I see them.
Red Flag #1: Who's Paying for This?
Assistant professor Joshus Wilson, from UD's School of Education, set out to see if the software PEGWriting could be used not just to score student writing, but to inform and assist instruction throughout the year. Why would he want to look into that?
The software Wilson used is called PEGWriting (which stands for Project Essay Grade Writing), based on work by the late education researcher Ellis B. Page and sold by Measurement Incorporated, which supports Wilson's research with indirect funding to the University.
So, the software maker paid for and perhaps commissioned this research. Just to be clear, the fact that there's no direct quid pro quo makes it worse-- if I'm counting on your funding to pay for the project I'm doing, the funding and the project can go away together and life can go on for the rest of my department. But if I'm doing research on your product over here and you're paying for, say, all our office furniture over there, the stakes are higher.
At any rate, this is clear built-in bias. Anything else?
Red Flag #2: You're Scoring What??!!
The software uses algorithms to measure more than 500 text-level variables to yield scores and feedback regarding the following characteristics of writing quality: idea development, organization, style, word choice, sentence structure, and writing conventions such as spelling and grammar.
First, I know you think it's impressive that it's measuring 500 variables ("text-level"-- as opposed to some other level?? Paper-level?), but it's not. It's like telling me that you have a vocabulary of 500 words. not so impressive, given the nature of language.
But beyond that-- PEGWriting wants to market itself as being tuned into six trait writing. I have no beef with the six traits-- I've used them myself for decades. And that's how I know that no software can possibly do what this software claims it can do.
Idea development? Really? I will bet you dollars to donuts that if I take my thesis statement ("Abe Lincoln was a great peacemaker") and develop it with absolute baloney support ("Lincoln helped bring peace by convincing Hitler to give up his siege of the Alamo"), the software will think that's swell. The software cannot read or understand ideas. It cannot assess this trait. Nor can it assess organization beyond looking for recycled prompts and transition words (Next, Furthermore, On the other hand). Nor can it have the slightest idea whether my word choices were best suited to the ideas in my essay. Any evaluation of sentence structure or style will be restricted to simply counting up types of sentences that it can (mostly) identify based on structure words and punctuation.
But robo-writing software always hits the same barrier-- the basic unit of writing is ideas, and if the software could understand ideas, the software developers would have created artificial intelligence and they'd have far more interesting things to spend time on than student writing.
No consideration of this topic can be complete without invoking my hero Les Perelman, who has made a career out of making essay-grading software look stupid. He has demonstrated over and over and over again that software does not know the difference between good writing and gibberish.
PEGWriting does enlist the teacher's help in scoring for textual evidence and content accuracy, so that's better than simply claiming the computer can do it.
Hey! A Non-red Flag
The idea is to give teachers useful diagnostic information on each writer and give them more time to address problems and assist students with things no machine can comprehend
This is a True Thing. I have my students do some fairly low-brain diagnostics on their own writing-- how many forms of "be," how many sentences, what sentence lengths, etc. Software could totally do this work and the information, when run through a human brain, can be useful, particularly in helping writings identify their tendencies and weak areas. That is the kind of thing software could totally do.
Red Flag #3: Know Your Research
Researchers have established that computer models are highly predictive of how humans would have scored a given piece of writing, Wilson said, and efforts to increase that accuracy continue.
Well, no. You can look at my short take on Perelman's work or the whole piece, but the bottom line is that the research that Wilson is most likely referring to is thoroughly unconvincing and shot full of huge holes.
Red Flag #4: That's Not a Good Thing
Wilson's research involved handing out free copies of PEGWriting to third, fourth and fifth grade classes.
Teachers said students liked the "game" aspects of the automated writing environment and that seemed to increase their motivation to write quite a bit. Because they got immediate scores on their writing, many worked to raise their scores by correcting errors and revising their work over and over.
Um, no. That's not entirely a good thing. I'll give you the positive side effect of making writing seem more fun than chorelike, but otherwise, the idea of having students learn that writing is like a game where you mess with words to score points-- well, that might prepare them for careers as internet trolls, but as with most bad writing instruction, it takes directly away from the actual point and purpose of writing, which is to say what you have to or want to say in the clearest way possible. Anything that reduces writing a mechanical activity completely divorced from the acual meaningful expressions of live humans is a Bad Thing. What could be worse than the approach described above? Oh, I know--
That same quick score produced discouragement for other students, though, teachers said, when they received low scores and could not figure out how to raise them no matter how hard they worked.
Emphasis mine. Because that "hard work" will be composed entirely of trying to mechanically manipulate pieces parts of written stuff. It will be no more about learning to write well that Super Mario Brothers is about learning how to talk to girls.
Is It All Bad?
The software does seem to offer some useful features, including an interactivity with both teacher and peer reviewers that could be handy. And I confess that I find the idea of a writing instruction platform on line to hold onto all the pieces parts of writing instruction.
Meanwhile, Wilson is looking for "efficiencies" in an approach that does seem to suggest some evolution in the marketing approach of software companies, as well as clarifying the teacher role in collaboration with the software. The old approach was to present software that would do everything for you; this research seems more focused on figuring out ways in which the software can help with instruction by saving time on things that software can actually do.
The bad news for the software manufacturers is that the answer to "what parts of writing assessment can software actually do" is "not many." I do think it's possible to create useful software, but unfortunately given how many teachers and administrators looking for a quick and easy way to writing instruction, it's unlikely that vendors won't keep trying to cash in on that market with crappy products that try to do too much that computers cannot do, resulting in more of these crappy pieces of crappity crap.
But But But
You may say that I'm quibbling about a level of writerly sophistication that only comes into play in older students, and that as long as we're just talking about elementary students, this sort of mechanical trained-monkey approach is fine.
I vehemently disagree.
The most important thing that young students learn about writing is what it is, what it's for, and how to engage with it. When we teach young students that writing is a series of mechanical tasks performed to make some teacher or software happy, we do huge long-term damage, and we turn potential writers into people who don't even know what writing is. From day one, we should be teaching them that writing is a cool way to communicate what you think and feel to other human beings, and that it starts inside your own brain and heart, not in some set of instructions.
Writing instruction done well is powerful, because young humans who are aching to be heard can discover a way to put their voice into the world, to be heard and responded to by other human beings. There is no similar excitement to be found in gaming a computer.
ICYMI: Great Eduposts This Week
I missed last Sunday because vacation. But here's some of what you could be reading today.
About those charter school waiting lists
Here's a nice clear first-person account of how those wait lists get to be so large, and how little wait lists tell us about demand for charter schools.
Why schools need more teachers of color-- for white students
We already know that there's no question about the benefits of having students see teachers who look like them. But having teachers of color is also hugely beneficial for white students as well.
Is NOLA Experiencing Slave Market Education Reform
Reformsters are bummed. The release of the ERA report was supposed to unleash a storm of excitement about how awesomely successful the New Orleans school privatization experiment has been. Instead, pieces like Jennifer Berkshire's Salon article have folks questioning the whole business all over again. Jullian Vasquez Heilig storified some of the critical conversations coming out of #NOLAEdWarning so we can see just why folks aren't dancing in the streets, and just how much isn't being included in the NOLA PR blitz.
Want to know how a student is doing?
Wendy Lecker reminds us that parents already know how to find out how well their children are doing in school-- and it isn't to march to the office and demand to see Junior's standardized test results.
The Hefty ad campaign
Yes, the Hefty trash bag folks, of all people, have launched a campaign hitting the idea that schools and teachers are underfunded and undersupported.
The suicide of the liberal arts
John Agresto takes to the pages of the Wall Street Journal to make a case for the liberal arts, and to argue that the discipline needs to be a better friend to itself.
About those charter school waiting lists
Here's a nice clear first-person account of how those wait lists get to be so large, and how little wait lists tell us about demand for charter schools.
Why schools need more teachers of color-- for white students
We already know that there's no question about the benefits of having students see teachers who look like them. But having teachers of color is also hugely beneficial for white students as well.
Is NOLA Experiencing Slave Market Education Reform
Reformsters are bummed. The release of the ERA report was supposed to unleash a storm of excitement about how awesomely successful the New Orleans school privatization experiment has been. Instead, pieces like Jennifer Berkshire's Salon article have folks questioning the whole business all over again. Jullian Vasquez Heilig storified some of the critical conversations coming out of #NOLAEdWarning so we can see just why folks aren't dancing in the streets, and just how much isn't being included in the NOLA PR blitz.
Want to know how a student is doing?
Wendy Lecker reminds us that parents already know how to find out how well their children are doing in school-- and it isn't to march to the office and demand to see Junior's standardized test results.
The Hefty ad campaign
Yes, the Hefty trash bag folks, of all people, have launched a campaign hitting the idea that schools and teachers are underfunded and undersupported.
The suicide of the liberal arts
John Agresto takes to the pages of the Wall Street Journal to make a case for the liberal arts, and to argue that the discipline needs to be a better friend to itself.
So, I Have a Book
I am a fan of self-publishing. The new world of Print on Demand makes it possible to put just about anything in a book at little-to-no cost. I've used it to publish collections of pieces from my newspaper column, create book-like memorabilia for my tiny region, and to publish the history of the 159-year-old town band I play in. Every year I now collect the major project from my honors students and publish it in book form. For our first anniversary, I made my wife a book.
I'm old school. I think books are awesome. I think holding the object, turning the pages, feeling that sheet of paper move under my fingers as it shifts to reveal another field of words-- it's just awesome. Other things in the world, particularly the cyber-world, could vanish tomorrow without a trace, but a book is real, solid (yes, I know it will dry up and blow away eventually, but somehow, that's different).
So I've published a book for this blog, and I am pleased to say that it's available to the world.
The book is a collection of ninety-some pieces from this blog, and I feel it only fair to point out that there isn't anything in the book that you can't read right here for free. Some popular old favorites are in the book, like the Arne Duncan spleen piece and the Hard Part, a blog that blew up on Huff Post. If you would like your dosage of Curmudgucation in a handy read-on-the-porch-with-a-cup-of-coffee format, or if you've been trying to get people in your life to read about what's happening in education but they just hate that whole interwebs thing, this is one more available option.For me, this will be a handy Christmas gift for family members.
And if you prefer not to fund amazon, you can buy a copy directly from the publisher (which is owned by amazon, but Bezos will still get less of your money less directly).
At any rate, I'm pleased to offer rantings in a different format as one more way to get the word out. One thing I've learned over the years is that I'm very bad at doing the whole "Hey, buy my book," thing. But if you have enjoyed the blog and would like some copies of classic material in book form, hey, buy my book.
I'm old school. I think books are awesome. I think holding the object, turning the pages, feeling that sheet of paper move under my fingers as it shifts to reveal another field of words-- it's just awesome. Other things in the world, particularly the cyber-world, could vanish tomorrow without a trace, but a book is real, solid (yes, I know it will dry up and blow away eventually, but somehow, that's different).
So I've published a book for this blog, and I am pleased to say that it's available to the world.
The book is a collection of ninety-some pieces from this blog, and I feel it only fair to point out that there isn't anything in the book that you can't read right here for free. Some popular old favorites are in the book, like the Arne Duncan spleen piece and the Hard Part, a blog that blew up on Huff Post. If you would like your dosage of Curmudgucation in a handy read-on-the-porch-with-a-cup-of-coffee format, or if you've been trying to get people in your life to read about what's happening in education but they just hate that whole interwebs thing, this is one more available option.For me, this will be a handy Christmas gift for family members.
And if you prefer not to fund amazon, you can buy a copy directly from the publisher (which is owned by amazon, but Bezos will still get less of your money less directly).
At any rate, I'm pleased to offer rantings in a different format as one more way to get the word out. One thing I've learned over the years is that I'm very bad at doing the whole "Hey, buy my book," thing. But if you have enjoyed the blog and would like some copies of classic material in book form, hey, buy my book.
Subscribe to:
Posts (Atom)