Just before Christmas, the Seattle Times provided coverage of the Gates Foundation's report about their decade's worth of progress with their goal of fixing the world (the Grand Challenge). After a billion dollars spent on improving lives and health care in the developing world, Gates had to report, "I was pretty naive about how long the process would take."
In his quest to make the world a better place, Gates invested in all sorts of research. But it turns out that research can only happen as fast as it can happen. Sometimes science takes time.
Not only did he underestimate some of the scientific hurdles, Gates
said. He and his team also failed to adequately consider what it would
take to implement new technologies in countries where millions of people
lack access to basic necessities such as clean water and medical care.
The foundation has tweaked the Grand Challenges approach in a variety of ways, but still doesn't really know whether any of it is actually succeeding. In many cases, they know it is not. There are several examples, but let's look at toilets.
Gates funded high-tech toilets in the Indian city of Raichur, at a cost of $8,000 each. These beauties have automatic sensors that run lights, fans and FM radio when a patron uses them. Some prototypes in the toilets project wing of Grand Challenges also throw in solar power and other amenities. But in Raichur, the rollout had some technical difficulties, and then- the public just didn't use them.
As it turns out, there are already people working on the toilet problem, but not with high tech answers. Jason Kass, founder of Toilets for People (which, as a name-- really? to distinguish them from Toilets for Cattle?) took Gates to task in a New York Times piece "Bill Gates Can't Build a Toilet" in which he notes, “If the many failed development projects of the past 60 years have
taught us anything, it’s that complicated, imported solutions do not
work."
But Gates is a technocrat with a deep commitment to techno-solutions to problems, like a plan to stop Dengue Fever by injecting bio-engineering mosquitoes to with bacteria that block disease transmission, instead of more directly addressing the living conditions and general health of the affected regions.
Drawing a line between Gates's naivete about fixing world health problems and his naivete about education systems is like shooting fish in a barrel, but some fish just need to be shot.
My impression of Gates is not a power-hungry greed-hound who is somehow trying to leverage the world's suffering into personal gain, but someone who is blinded to any view of the world but his own. He's used to being the smartest guy in the room, the boss, the man. It would be understandable if he had succumbed to a belief that he's fundamentally better, wiser, cleverer than most other people. He is a computer engineering systems guy. Systems are his hammer and everything in the world is a nail.
With much of his health initiative, you see the same basic outline-- technology will allows us to set up this awesome system, and because it is so obviously the Right Way To Do Things, people will just fall in line, and if they won't we'll just have to find a way to get them to. It's the same pattern some techno-critics see in many Microsoft failed products-- this is how people ought to want to do things, so this should work (and if it doesn't, it's the people, not our product). How any of you got a Zune for Christmas?
Gates wants to use systems to change society, but his understanding of how humans and culture and society and communities change is faulty. It's not surprising that Gates is naive-- it's surprising that he is always naive in the same way. It always boils down to "I really thought people would behave differently." And although I've rarely seen him acknowledge it print, it also boils down to, "There were plenty of people who could have told me better, but I didn't listen to them."
The non-success of Grand Challenges is just like the failure of the Gates Common Core initiative. Gates did not take the time to do his homework about the pre-existing structures and systems. He did not value the expertise of people already working in the field, and so he did not consult it or listen to it. He put an unwarranted faith in his created systems, and imagined that they would prevail because everyone on the ground would be easily assimilated into the new imposed-from-outside system. He became frustrated by peoples' insistence on seeing things through their own point-of-view rather than his. And he spent a huge amount of money attempting to impose his vision on everybody else.
We can say, "He's a rich guy. He can spend his money on what he wants," and that's true. But the opportunity cost here is staggering. Imagine what could be done if we started with, "Here's a billion dollars. Let's get the experts together and decide how it could best be spent." Instead of "Here's a billion dollars that we're going to spend on my solution no matter what."
I keep wondering when the light bulb is going to go off. After the failure of his small schools initiative, Gates had the chance to say either "This small schools things didn't work" or "My whole approach to finding solutions for education is messed up," and he seems to have chosen the former. It's nice that he can occasionally look back and call himself naive. It would be nicer if he could look around and recognize when he's doing it again, right now. It would be best if he could really recognize what he's being naive about.
Friday, December 26, 2014
Thursday, December 25, 2014
Merry Christmas
It's a complicated holiday as we celebrate it in this country, a horrendous mash-up of religion, commercialism, and bits and pieces of left-overs lying about. But it is also a perfect example of what it is to be human in the world-- there's sort of a pattern and set of traditions (one could almost say standards) for how we are "supposed to" celebrate, and yet all of us have modified and created our own family traditions, depending on where we are and what is going on and what we believe and how we like to celebrate. Much of how we choose to approach the day is private and personal, and that's not a bad thing at all. So as a nation, we are all spending today kind of the same way, and kind of not. Because we are human beings.
None of my children were able to be home this year, and they are far apart from each other, but since they were old enough to do it, they have played Battleship on Christmas Eve. Last night they played over skype (the series remains tied). Here in northwestern PA it's a lot more like April than December, but the flock of geese is still hanging out on the river behind the house. Last night my wife and I played in a brass ensemble at a church service; this morning my wife is sleeping in as her first Christmas Day treat.
I wouldn't presume to tell you how to celebrate the day, though I do have one suggestion. Decades ago at my college graduation, a baccalaureate speaker talked about the idea of sabbath, a holy day on which we were to behave as if we didn't have stuff or the obligations that came with it. It really struck a chord and though the speaker didn't suggest it, connected for me with John Lennon's "Imagine."
What would you do if you didn't have to worry about your stuff, your obligations, your business, your "have to's" but could simply stop and breathe and pay attention to the people and the world around you? That, suggested the speaker, would be a sabbath. We were supposed to be stopping our business and taking stock once a week; we kind of lost that. But Christmas, once a year, provides us with a sabbath of sorts.
So whether you're doing it in celebration of the day when the Creator of All That Is became flesh and dwelt among us and God came to help us get our collective shit together, or whether you're simply celebrating an occasion for family and friends, or whether you're honoring some other faith or tradition, or whether you're having none of that, or even if this is just a day to get through as best you can, may this day be one that allows you to feed your spirit, catch your breath, trim your sails, and move forward with a renewed focus.
Well, that's my suggestion. My wish for you is that the day be exactly what you had hoped it would be. Merry Christmas!
None of my children were able to be home this year, and they are far apart from each other, but since they were old enough to do it, they have played Battleship on Christmas Eve. Last night they played over skype (the series remains tied). Here in northwestern PA it's a lot more like April than December, but the flock of geese is still hanging out on the river behind the house. Last night my wife and I played in a brass ensemble at a church service; this morning my wife is sleeping in as her first Christmas Day treat.
I wouldn't presume to tell you how to celebrate the day, though I do have one suggestion. Decades ago at my college graduation, a baccalaureate speaker talked about the idea of sabbath, a holy day on which we were to behave as if we didn't have stuff or the obligations that came with it. It really struck a chord and though the speaker didn't suggest it, connected for me with John Lennon's "Imagine."
What would you do if you didn't have to worry about your stuff, your obligations, your business, your "have to's" but could simply stop and breathe and pay attention to the people and the world around you? That, suggested the speaker, would be a sabbath. We were supposed to be stopping our business and taking stock once a week; we kind of lost that. But Christmas, once a year, provides us with a sabbath of sorts.
So whether you're doing it in celebration of the day when the Creator of All That Is became flesh and dwelt among us and God came to help us get our collective shit together, or whether you're simply celebrating an occasion for family and friends, or whether you're honoring some other faith or tradition, or whether you're having none of that, or even if this is just a day to get through as best you can, may this day be one that allows you to feed your spirit, catch your breath, trim your sails, and move forward with a renewed focus.
Well, that's my suggestion. My wish for you is that the day be exactly what you had hoped it would be. Merry Christmas!
Wednesday, December 24, 2014
A Free Market Lesson from the Airlines
In today's Slate, Alison Griswold addresses the issue of cranky travelers and the ever-shrinking world of air travel amenities, and in the process, she delivers a lesson in how the free market sometimes works.
Everyone complains about the shrinking world of air travel. Smaller seats. Less legroom. Fees for every conceivable. I fully expect that at some point airlines will charge not only to transport the clothing in our luggage but also to let us transport the clothes on our back. Why, oh why, do the airlines keep providing less and less for travelers at increasingly nickle-and-dime prices? Griswold has an answer--
Because air travelers choose to make it so.
One pair of stats tell the story. The most loathed, the most complained-about, the most customer-annoying airline by far is Spirit. And the airline that is tops in turning a profit? Also Spirit. While people may pay lip service to the idea of quality, when it comes time to dig out the wallets, people cast their vote for As Cheap As We Can Stand, partly because people are cheap and partly because people just don't have that much money to spend.
Here is yet another example of how the free market does not necessarily foster a drive for excellence at all. In the case of the airlines, the Invisible Hand is apparently tightly grasped around a wallet.
Once again, the idea that free market competition in schools will somehow bring about an era of educational excellence-- well, it just doesn't look like how things work in the real world. In the real world, the free market drives competition to make some products and services just as cheap as we can stand. Many folks in the education biz will recognize As Cheap As We Can Get Away With as a not-uncommon guiding principle in school system budgeting.
But the airline example is not completely applicable to schooling-- schooling is actually even worse.
If an education is a plane ticket to San Jose, then we might well figure we can put up with a lot just to keep things cheap. But since our schools are funded by all taxpayers, for many folks an education is really like buying a plane ticket to San Jose for somebody else. And if somebody else is going to ride in that tiny seat with no leg room-- well, heck, that doesn't really inconvenience me much here at home at all. Even if we assume a voucher system in which taxpayers give students a lump of money to go buy their ticket to San Jose, the amount taxpayers will pitch in for the ticket will still tend to be low.
So the favored few will travel in First Class because they can afford it, and occasionally the airline will upgrade some lucky traveler, but most folks will travel in crappy cramped coach conditions. And the invisible hand of the free market will not make the slightest move toward pursuing excellence.
Everyone complains about the shrinking world of air travel. Smaller seats. Less legroom. Fees for every conceivable. I fully expect that at some point airlines will charge not only to transport the clothing in our luggage but also to let us transport the clothes on our back. Why, oh why, do the airlines keep providing less and less for travelers at increasingly nickle-and-dime prices? Griswold has an answer--
Because air travelers choose to make it so.
One pair of stats tell the story. The most loathed, the most complained-about, the most customer-annoying airline by far is Spirit. And the airline that is tops in turning a profit? Also Spirit. While people may pay lip service to the idea of quality, when it comes time to dig out the wallets, people cast their vote for As Cheap As We Can Stand, partly because people are cheap and partly because people just don't have that much money to spend.
Here is yet another example of how the free market does not necessarily foster a drive for excellence at all. In the case of the airlines, the Invisible Hand is apparently tightly grasped around a wallet.
Once again, the idea that free market competition in schools will somehow bring about an era of educational excellence-- well, it just doesn't look like how things work in the real world. In the real world, the free market drives competition to make some products and services just as cheap as we can stand. Many folks in the education biz will recognize As Cheap As We Can Get Away With as a not-uncommon guiding principle in school system budgeting.
But the airline example is not completely applicable to schooling-- schooling is actually even worse.
If an education is a plane ticket to San Jose, then we might well figure we can put up with a lot just to keep things cheap. But since our schools are funded by all taxpayers, for many folks an education is really like buying a plane ticket to San Jose for somebody else. And if somebody else is going to ride in that tiny seat with no leg room-- well, heck, that doesn't really inconvenience me much here at home at all. Even if we assume a voucher system in which taxpayers give students a lump of money to go buy their ticket to San Jose, the amount taxpayers will pitch in for the ticket will still tend to be low.
So the favored few will travel in First Class because they can afford it, and occasionally the airline will upgrade some lucky traveler, but most folks will travel in crappy cramped coach conditions. And the invisible hand of the free market will not make the slightest move toward pursuing excellence.
What Do We Do About NCLB
The return of a GOP majority to DC has renewed talk of the Great White Whale of education reform-- completing the long-overdue rewrite of ESEA, currently commonly known as No Child Left Behind.
Lest we forget, NCLB is an actual law, and every state in the union is in violation. At this point the early predictions about the law are true-- every school is either failing or cheating. It's that universal violation that makes the extra-legal legerdemain of Race to the Top/waivers possible. Change the law so that it no longer requires 100% of US students to be above average, and the waivers become unnecessary, and the current administration's legisltion-free rewrite of US education collapses. If I were cynical, I might conclude that it's that chance to hand the President a policy defeat, and not a desire to restore the promise of public education, that motivates some GOPpers on this issue.
But an ESEA rewrite brings us up against the same obstacle that has been clogging the pipes since 2007-- what to put in its place? Nobody has yet found the proper education enema to get things moving.
Recently Andy Smarick and Jack Schneider took on the question, and while they are both bright and learned men, I find that I disagree with both of them in some substantial ways.
Smarick, both in the EdWeek piece and in his writing elsewhere, has recognized that education now falls into a classic conservative conundrum-- on the one hand, conservatives want government to leave people alone and stop telling them what to do, but on the other hand, when government leaves people alone, they often run out and start doing things conservatives think they ought not to.
Smarick writes that "when states made virtually all K-12 decisions absent federal accountability rules--call this the "pre-NCLB" era--our nation didn't get the results we wanted." Schneider questions whether that's actually true. I'd like to ask who "we" are.
Smarick is concerned that "too many disadvantaged kids were not well-served" in pre-NCLB America. That's a legitimate concern, but at this stage of the game, there's no sign that NCLB/RttT made anything better.
And while I get his concern that a government that hands over giant honking bales of cash can reasonably expected to hear the banging of its bucks, all that gets us is an influx of companies that are good at filling out accountability paperwork.
This is part of what Schneider likes about NCLB-- transparent school accountability, and disaggregation of data. I'm not convinced, because the "data" we're talking about are inevitably test scores. The breaking out of sub-groups has had some deeply unpleasant side effects. For instance, the common practice of pre-testing students and targetting the failures. This gives us a two-tier system; pre-test winners get a full, rich day of varied class offerings, while pre-test losers (or last year's test losers) get a day filled with math and English and math and English and test prep and more test prep.
In my area, where our most common sub-groups are low-income students, we would be further ahead to hire enough of those parents at well-paying jobs doing anything at all, so that their children were no longer part of the subgroup and we could make the sub-group small enough not to destroy our numbers.
So here's my rewrite of NCLB:
The federal government will distribute its giant mountain of imaginary education-cash in a manner designed to offset the varied levels of poverty across the US.
Somebody can punch up the language. But that's it. In all other respects, the federal government will butt out of the education biz.
Oh, I know I'm fantasizing. If they rewrite it, it will be yet again a labyrinthian mess of federal overrreach and mandated malpractice. Here's why--
Any federal law about education will be written not by people who are good at education, but by people who are good at the business of politics and regulation. Accountability will continue to be based on a politically-favored business model, which will reward people who are good at business and government accountability paperwork. At no point will the rewrite be under the control or direction of people who will be primarily concerned with the educational aspects of the bill.
Look at NCLB. The insane Lake Woebegone clause (that all students will be made above average) was all about politics, and even today, Smarick was trotting out the old "If you don't want it to be 100%, then you go ahead and pick out which students will be left behind" which is all about political leverage and not one iota about education. The politicians who put it there created a time bomb that they never expected to go off. The steep climb of the AYP wasn't scheduled to start until 2008, after Pres. Bush was done and after Congress was supposed to rewrite the bill. But 2007 came and they couldn't get the job done.
When Congress set that reality-impaired goal, they weren't over-estimating teachers. They were over-estimating themselves. Politics stuck us with an idea that was political gold, but educationally impossible (and the only support ever offered is essentially a political dare-- "go ahead and say something that can be used against you.")
Revise ESEA? The feds can't do this job right (and maybe not even at all). The best solution, the solution that would actually take US education forward, is for the feds to back up and get out of the business of doing anything except trying to level the financial playing field.
Federal involvement in education has not solved a single problem, or fixed a single broken thing. If you think the states do a lousy job of handling education, please note that federal involvement has only made things worse. The problems of big city schools are problems of politics and money; NCLB and RttT have simply injected more money-fueled politics into state-level education, and it has gotten us nothing good. Nothing. Urban schools are a problem in search of a solution, but the solution does not lie in ESEA. Nor has the unending, ever-growing mountain of reporting to the federal government helped anybody fix anything, with the possible exception of increased employment for administrators and administrative assistants hired by school districts to cope with government reporting requirements.
My solution is both radical and reactionary. The cry of fans of federalism is, "Without accountability and reporting to the federal government, how will we know that schools are doing well." My response is-- who needs to know? Who, beyond the teachers and administrators and local taxpayers and parents, needs to know how a particular school is doing, and what could the federal government do to inform them? My answers-- nobody, and nothing. It is, in fact, the system that allowed us the robust freedom and flexibility that coincided with the 20th century rise of the US as a world power.
All I want to do with NCLB is blow it up. I realize I'm dreaming, but so is anyone who thinks we can have 100% above average students or who thinks that free market forces could possibly help education. I like my dream better.
Originally posted at View from the Cheap Seats
Lest we forget, NCLB is an actual law, and every state in the union is in violation. At this point the early predictions about the law are true-- every school is either failing or cheating. It's that universal violation that makes the extra-legal legerdemain of Race to the Top/waivers possible. Change the law so that it no longer requires 100% of US students to be above average, and the waivers become unnecessary, and the current administration's legisltion-free rewrite of US education collapses. If I were cynical, I might conclude that it's that chance to hand the President a policy defeat, and not a desire to restore the promise of public education, that motivates some GOPpers on this issue.
But an ESEA rewrite brings us up against the same obstacle that has been clogging the pipes since 2007-- what to put in its place? Nobody has yet found the proper education enema to get things moving.
Recently Andy Smarick and Jack Schneider took on the question, and while they are both bright and learned men, I find that I disagree with both of them in some substantial ways.
Smarick, both in the EdWeek piece and in his writing elsewhere, has recognized that education now falls into a classic conservative conundrum-- on the one hand, conservatives want government to leave people alone and stop telling them what to do, but on the other hand, when government leaves people alone, they often run out and start doing things conservatives think they ought not to.
Smarick writes that "when states made virtually all K-12 decisions absent federal accountability rules--call this the "pre-NCLB" era--our nation didn't get the results we wanted." Schneider questions whether that's actually true. I'd like to ask who "we" are.
Smarick is concerned that "too many disadvantaged kids were not well-served" in pre-NCLB America. That's a legitimate concern, but at this stage of the game, there's no sign that NCLB/RttT made anything better.
And while I get his concern that a government that hands over giant honking bales of cash can reasonably expected to hear the banging of its bucks, all that gets us is an influx of companies that are good at filling out accountability paperwork.
This is part of what Schneider likes about NCLB-- transparent school accountability, and disaggregation of data. I'm not convinced, because the "data" we're talking about are inevitably test scores. The breaking out of sub-groups has had some deeply unpleasant side effects. For instance, the common practice of pre-testing students and targetting the failures. This gives us a two-tier system; pre-test winners get a full, rich day of varied class offerings, while pre-test losers (or last year's test losers) get a day filled with math and English and math and English and test prep and more test prep.
In my area, where our most common sub-groups are low-income students, we would be further ahead to hire enough of those parents at well-paying jobs doing anything at all, so that their children were no longer part of the subgroup and we could make the sub-group small enough not to destroy our numbers.
So here's my rewrite of NCLB:
The federal government will distribute its giant mountain of imaginary education-cash in a manner designed to offset the varied levels of poverty across the US.
Somebody can punch up the language. But that's it. In all other respects, the federal government will butt out of the education biz.
Oh, I know I'm fantasizing. If they rewrite it, it will be yet again a labyrinthian mess of federal overrreach and mandated malpractice. Here's why--
Any federal law about education will be written not by people who are good at education, but by people who are good at the business of politics and regulation. Accountability will continue to be based on a politically-favored business model, which will reward people who are good at business and government accountability paperwork. At no point will the rewrite be under the control or direction of people who will be primarily concerned with the educational aspects of the bill.
Look at NCLB. The insane Lake Woebegone clause (that all students will be made above average) was all about politics, and even today, Smarick was trotting out the old "If you don't want it to be 100%, then you go ahead and pick out which students will be left behind" which is all about political leverage and not one iota about education. The politicians who put it there created a time bomb that they never expected to go off. The steep climb of the AYP wasn't scheduled to start until 2008, after Pres. Bush was done and after Congress was supposed to rewrite the bill. But 2007 came and they couldn't get the job done.
When Congress set that reality-impaired goal, they weren't over-estimating teachers. They were over-estimating themselves. Politics stuck us with an idea that was political gold, but educationally impossible (and the only support ever offered is essentially a political dare-- "go ahead and say something that can be used against you.")
Revise ESEA? The feds can't do this job right (and maybe not even at all). The best solution, the solution that would actually take US education forward, is for the feds to back up and get out of the business of doing anything except trying to level the financial playing field.
Federal involvement in education has not solved a single problem, or fixed a single broken thing. If you think the states do a lousy job of handling education, please note that federal involvement has only made things worse. The problems of big city schools are problems of politics and money; NCLB and RttT have simply injected more money-fueled politics into state-level education, and it has gotten us nothing good. Nothing. Urban schools are a problem in search of a solution, but the solution does not lie in ESEA. Nor has the unending, ever-growing mountain of reporting to the federal government helped anybody fix anything, with the possible exception of increased employment for administrators and administrative assistants hired by school districts to cope with government reporting requirements.
My solution is both radical and reactionary. The cry of fans of federalism is, "Without accountability and reporting to the federal government, how will we know that schools are doing well." My response is-- who needs to know? Who, beyond the teachers and administrators and local taxpayers and parents, needs to know how a particular school is doing, and what could the federal government do to inform them? My answers-- nobody, and nothing. It is, in fact, the system that allowed us the robust freedom and flexibility that coincided with the 20th century rise of the US as a world power.
All I want to do with NCLB is blow it up. I realize I'm dreaming, but so is anyone who thinks we can have 100% above average students or who thinks that free market forces could possibly help education. I like my dream better.
Originally posted at View from the Cheap Seats
When Testing Drives the Bus
From my spiritual brother over at Curmudgeon Central comes an instructive tale from East Rockaway about what can happen when testing drives the bus. It's a tale of triumph and tragedy, and it comes in two chapters.
Chapter One: How To Achieve Excellence
As reported in a Newsday story now behind a paywall, but excerpted at Curmudgeon Central, Voula Coyle's method of preparing her fourth graders for the state test was neither new nor novel. In fact, it was exactly the technique that we already know is the best, most effective way of preparing students for a standardized tests:
The arbitrator, in her ruling, noted that fourth-graders took more than 60 practice tests from January 2012 through April 2012, virtually a daily routine. The arbitrator also noted that the elementary school's principal had endorsed the practice sessions.
Yessir. The best way to get good at taking standardized tests is to practice taking standardized tests (because the only thing a standardized test really measures is your ability to take a standardized test). And so the seventeen-year veteran regularly achieved the highest possible rating in the Students Improve On The Test portion of her evaluation.
Chapter Two: The Rewards of Excellence
You might have noticed that the above quote mentions an arbitrator and thought, "Well, that can't be good." And Coyle did find herself in trouble. She was not in trouble for wasting educational time every day with stupid practice tests, however, or for making a mockery of the educational process by suspending it in order to do straight-up test prep.
She was in trouble for getting good test results.
Most coverage of the story tracks back to this piece from Fios1 news, which explains that Coyle's high scores in fourth grade made it difficult for fifth grade teachers to get any more improvement out of the students. This gave fifth grade teachers low ratings which in turn hurt the school ratings. Coyle felt pushback from her administration.
The instructor said her superiors have regularly encouraged her and her colleagues to avoid overachieving and to keep their scores from exceeding the state rating of "effective."
"One faculty member said our job is not to be optimal, but to be adequate. That underlying message of mediocrity was promoted," Coyle said. Teachers were also told to accept that a third of their students would understand the material, another third would be average and the rest would fail, she said.
First, Coyle was accused of cheating by telling students to go back to particular questions they had gotten wrong. That case (which, among other things, depended on not including testimony from the aid in the room during testing) fell apart, so administration simply re-assigned Coyle to a job pushing paper and replaced her with a teacher who didn't do so well.
So What Have We Learned?
The story has two morals.
First, excellence in test-taking comes from excellence in test-taking practice. Teach to that test.
Second, don't teach to the test too well, because it messes up the VAM sauce. Get test scores that are good, but not too good.
Remember-- it's not about education or learning. It's about generating the right set of numbers to keep the state happy .
Chapter One: How To Achieve Excellence
As reported in a Newsday story now behind a paywall, but excerpted at Curmudgeon Central, Voula Coyle's method of preparing her fourth graders for the state test was neither new nor novel. In fact, it was exactly the technique that we already know is the best, most effective way of preparing students for a standardized tests:
The arbitrator, in her ruling, noted that fourth-graders took more than 60 practice tests from January 2012 through April 2012, virtually a daily routine. The arbitrator also noted that the elementary school's principal had endorsed the practice sessions.
Yessir. The best way to get good at taking standardized tests is to practice taking standardized tests (because the only thing a standardized test really measures is your ability to take a standardized test). And so the seventeen-year veteran regularly achieved the highest possible rating in the Students Improve On The Test portion of her evaluation.
Chapter Two: The Rewards of Excellence
You might have noticed that the above quote mentions an arbitrator and thought, "Well, that can't be good." And Coyle did find herself in trouble. She was not in trouble for wasting educational time every day with stupid practice tests, however, or for making a mockery of the educational process by suspending it in order to do straight-up test prep.
She was in trouble for getting good test results.
Most coverage of the story tracks back to this piece from Fios1 news, which explains that Coyle's high scores in fourth grade made it difficult for fifth grade teachers to get any more improvement out of the students. This gave fifth grade teachers low ratings which in turn hurt the school ratings. Coyle felt pushback from her administration.
The instructor said her superiors have regularly encouraged her and her colleagues to avoid overachieving and to keep their scores from exceeding the state rating of "effective."
"One faculty member said our job is not to be optimal, but to be adequate. That underlying message of mediocrity was promoted," Coyle said. Teachers were also told to accept that a third of their students would understand the material, another third would be average and the rest would fail, she said.
First, Coyle was accused of cheating by telling students to go back to particular questions they had gotten wrong. That case (which, among other things, depended on not including testimony from the aid in the room during testing) fell apart, so administration simply re-assigned Coyle to a job pushing paper and replaced her with a teacher who didn't do so well.
So What Have We Learned?
The story has two morals.
First, excellence in test-taking comes from excellence in test-taking practice. Teach to that test.
Second, don't teach to the test too well, because it messes up the VAM sauce. Get test scores that are good, but not too good.
Remember-- it's not about education or learning. It's about generating the right set of numbers to keep the state happy .
Tuesday, December 23, 2014
A Teaching Ambassador!
I've been looking at the application materials for the Department of Education's teacher ambassadorship. After all, I am always up to Learn New Things, and I feel that I have a few things to share with Arne Duncan and the USED (Exhibit A: Any of the last seven hundred or so posts). So let's take a look at what would be required.
First, we actually have a choice of two programs.
For Washington Fellowships, teachers head to DC to become full-time employees of the department (a deal has to be cut between the feds and your employer). This requires a teacher who is ready, willing and able to be uprooted-- and not just to anywhere, but to Washington DC (Moto: Mostly Not Swamp Now). "They contribute valuable school and classroom level knowledge and perspective to the Department, collaborate to provide specific outreach to other teachers, and greatly increase their knowledge and understanding of federal education policies and programs in order to share with other teachers."
There are also Classroom Fellowships. For these, the teacher stays put at her regular gig, earning an hourly rate for the 20-40 hours per month of gummint work she would do. This fellowship appears to be mostly doing outreach and marketing for department policies out in the hinterlands, although, like the DC fellowship, this gig supposedly involves telling the department what you think. See? I could totally do this!
The eligibility requirements are simple enough. To qualify for one of the fellowships, you need to
Five essays that will cover
* Your contributions to student learning, including how you work with your particular demographic. You should have some qualitative and quantitative evidence of your learning impact. I don't see anything that specifically requires you to submit your VAM scores, but I'm sure they'd be acceptable..
* Your leadership experience. Show some experience that highlights your style, approach and achievements. The USED needs to know that when you start selling, somebody will be buying.
* Written and oral communication skills. Show some examples of where you have presented about ed stuff or "led discourse." What professional networks are you already plugged in to? It's not clear if blogging meets this requirement.
* Other skills. "Ambassador Fellows work in a fast-paced and ambiguous environment in which they must quickly focus, digest complex information, and network with a wide variety of education stakeholders." Do you suppose playing tailgate trombone or jazz tuba count here?
* Interests and expertise. What can you do for the department, and what do you hope the department will do for you. And here's the money quote: "Include relevant experience with Federally-funded or key initiatives addressing: the transition college and career ready standards and instruction; support for great teachers and leaders; turning around chronically low performing schools; or providing access and opportunity for all students from cradle to career." In other words, show us how much you love everything we love here at the USED-- this is the same list of priorities you can find in Race to the Top and waiver requirements.
Employer letter of recommendation
Has to be signed and on letterhead, and indicate that administration is aware that they are handing one of their teachers over to the borg collective. It should back up whatever the applicant said in those essays, and it should highlight the specifics of how the applicant gets things done in the school and is plugged into the community. Is this someone that people will listen to when she starts pushing the party line?
Current Resume
This seems like a low bar, which makes sense because if you know classroom teachers, you know that most of us don't have very snazzy resumes."Resumes are used to help verify eligibility and should be consistent with or able to enhance information contained within your narrative responses." So, don't catch yourself lying in these.
Competition is tight because, let's face it-- this would be a pretty interesting gig, even for those of us who think the USED is pretty much wrong about many, many things. We have till January 20th to get our application in, and you can find all the pertinent information at the department's website. I suspect that, despite my communication skills and my leadership experience (president of striking local) and my professional network (if you're reading this, I'm probably counting you) and my bitchin' low brass skills-- despite all that, I am probably not a good fit for the department. But I bet we could all do a little something to help the department find just the right folks.
First, we actually have a choice of two programs.
For Washington Fellowships, teachers head to DC to become full-time employees of the department (a deal has to be cut between the feds and your employer). This requires a teacher who is ready, willing and able to be uprooted-- and not just to anywhere, but to Washington DC (Moto: Mostly Not Swamp Now). "They contribute valuable school and classroom level knowledge and perspective to the Department, collaborate to provide specific outreach to other teachers, and greatly increase their knowledge and understanding of federal education policies and programs in order to share with other teachers."
There are also Classroom Fellowships. For these, the teacher stays put at her regular gig, earning an hourly rate for the 20-40 hours per month of gummint work she would do. This fellowship appears to be mostly doing outreach and marketing for department policies out in the hinterlands, although, like the DC fellowship, this gig supposedly involves telling the department what you think. See? I could totally do this!
The eligibility requirements are simple enough. To qualify for one of the fellowships, you need to
- Anticipate employment in a teaching position (including instructional coaches/specialists) during the 2015-2016 school year in a United States school (including traditional public, charter, virtual, military, tribal and/or private schools) that serves any grade, preschool through twelfth.
- Have a minimum of five (5) years of teaching experience.
- Be a United States citizen or permanent resident.
- Be able to obtain school/district support to sign an Intergovernmental Personnel Act (IPA) agreement at the point of selection for participation in the program.
Five essays that will cover
* Your contributions to student learning, including how you work with your particular demographic. You should have some qualitative and quantitative evidence of your learning impact. I don't see anything that specifically requires you to submit your VAM scores, but I'm sure they'd be acceptable..
* Your leadership experience. Show some experience that highlights your style, approach and achievements. The USED needs to know that when you start selling, somebody will be buying.
* Written and oral communication skills. Show some examples of where you have presented about ed stuff or "led discourse." What professional networks are you already plugged in to? It's not clear if blogging meets this requirement.
* Other skills. "Ambassador Fellows work in a fast-paced and ambiguous environment in which they must quickly focus, digest complex information, and network with a wide variety of education stakeholders." Do you suppose playing tailgate trombone or jazz tuba count here?
* Interests and expertise. What can you do for the department, and what do you hope the department will do for you. And here's the money quote: "Include relevant experience with Federally-funded or key initiatives addressing: the transition college and career ready standards and instruction; support for great teachers and leaders; turning around chronically low performing schools; or providing access and opportunity for all students from cradle to career." In other words, show us how much you love everything we love here at the USED-- this is the same list of priorities you can find in Race to the Top and waiver requirements.
Employer letter of recommendation
Has to be signed and on letterhead, and indicate that administration is aware that they are handing one of their teachers over to the borg collective. It should back up whatever the applicant said in those essays, and it should highlight the specifics of how the applicant gets things done in the school and is plugged into the community. Is this someone that people will listen to when she starts pushing the party line?
Current Resume
This seems like a low bar, which makes sense because if you know classroom teachers, you know that most of us don't have very snazzy resumes."Resumes are used to help verify eligibility and should be consistent with or able to enhance information contained within your narrative responses." So, don't catch yourself lying in these.
Competition is tight because, let's face it-- this would be a pretty interesting gig, even for those of us who think the USED is pretty much wrong about many, many things. We have till January 20th to get our application in, and you can find all the pertinent information at the department's website. I suspect that, despite my communication skills and my leadership experience (president of striking local) and my professional network (if you're reading this, I'm probably counting you) and my bitchin' low brass skills-- despite all that, I am probably not a good fit for the department. But I bet we could all do a little something to help the department find just the right folks.
Setting Cut Scores
Benchmark is originally a surveying term. Benchmarks are slots cut into the side of stone (read "permanent") structures into which a bench (basically a little shelf) can be inserted for surveying purposes. We know they're at a certain level because they've been measured in relation to another marker which has been measured in relation to another marker and so on retrogressively until we arrive at a Mean Sea Level marker (everything in surveying is ultimately measured in relation to one of those).
Surveying markers, including benchmarks, are literally set in stone. Anybody with the necessary training can find them always in the same place and measure any other point in relation to them.
This metaphorical sense of unwavering objective measure is what many folks carry with them to their consideration of testing and cut scores. Passing, failing, and excellence, they figure, are all measured against some scholarly Mean Sea Level marker by way of benchmarks that have been carefully measured against MSL and set in stone.
Sorry, no. Instead, cut scores represent an ideal somewhere between a blindfolded dart player with his fingers duct-taped together, and the guy playing against the blindfolded dart player who sets the darts exactly where he wants them.
Writing in the Stamford Advocate, Wendy Lecker notes that the Smarter Balanced Assessment Consortium members (including Connecticut's own committed foe of public education Commissioner Stefan Pryor) set cut scores for the SBA tests based on stale fairy dust and the wishes of dying puppies.
People tend to assume that cut scores-- the borderline between Good Enough and Abject Failure-- mean something. If a student fails The Test, she must be unready for college or unemployable or illiterate or at the very least several grades behind where she's Supposed To Be (although even that opens up the question "Supposed by whom?")
In fact, SBAC declares that the achievement levels "do not equate directly to expectations for `on-grade' performance" and test scores should only be used with multiple other sources of information about schools and students.
Furthermore, "SBAC admits it cannot validate whether its tests measure college readiness until it has data on how current test takers do in college."
If you are imagining that cut scores for the high-stakes accountability tests are derived through some rigorous study of exactly what students need to know and what level of proficiency they should have achieved by a certain age-- well, first, take a look at what you're assuming. Did you really think we have some sort of master list, some scholastic Mean Sea Level that tells us exactly what a human being of a certain age should know and be able to do as agreed upon by some wise council of experty experts? Because if you do, you might as well imagine that those experts fly to their meetings on pink pegasi, a flock of winger horsies that dance on rainbows and take minutes of the Wise Expert meetings by dictating to secretarial armadillos clothed in shimmering mink stoles.
Anyway, it doesn't matter because there are no signs that any of these people associated with The Test are trying to work with a hypothetical set of academic standards anyway. Instead, what we see over and over (even back in the days of NCLB), is educational amateurs setting cut scores for political purposes. So SBAC sets a cut score so that almost two thirds of the students will fail. John King in New York famously predicted the percentage of test failure before the test was even out the door-- but the actual cut scores were set after the test was taken.
That is not how you measure a test result against a standard. That's how you set a test standard based on the results you want to see. It's how you make your failure predictions come true. According to Carol Burris, King also attempted to find some connection between SAT results and college success prediction, and then somehow graft that onto a cut score for the NY tests, while Kentucky and other CCSS states played similar games with the ACT.
Setting cut scores is not an easy process. Education Sector, a division of the thinky tank American Institutes for Research (they specialize in behavioral sciency thinking, and have a large pedigree in the NCLB era and beyond), issued an "explainer" in July of 2006 about how states set passing scores on standardized tests. It leads off its section on cut scores with this:
On a technical level, states set cut scores along one of two dimensions: The characteristics of the test items or the characteristics of the test takers.It is essential to understand that either way is an inescapably subjective process. Just as academic standards are ultimately the result of professional judgment rather than absolute truth, there is no “right” way to set cut scores, and different methods have various strengths and weaknesses.
The paper goes on to talk about setting cut scores, and some of it is pretty technical, but it returns repeatedly to the notion that at various critical junctures, some human being is going to make a judgment call.
Educational Testing Service (ETS) also has a nifty "Primer on Setting Cut Scores on Tests of Educational Achievement." Again, from all the way back in 2006, this gives a quick compendium of various techniques for setting cut scores-- it lists eight different methods. And it also opens with some insights that would still be useful to consider today.
The first step is for policymakers to specify exactly why cut scores are being set in the first place. The policymakers should describe the benefits that are expected from the use of cut scores. What decisions will be made on the basis of the cut scores? How are those decisions being made now in the absence of cut scores? What reasons are there to believe that cut scores will result in better decisions? What are the expected benefits of the improved decisions?
Yeah, those conversations have not been happening within anyone's earshot. Then there is this:
It is important to list the reasons why cut scores are being set and to obtain consensus among stakeholders that the reasons are appropriate. An extremely useful exercise is to attempt to describe exactly how the cut scores will bring about each of the desired outcomes. It may be the case that some of the expected benefits of cut scores are unlikely to be achieved unless major educational reforms are accomplished. It will become apparent that cut scores, by themselves, have very little power to improve education. Simply measuring a child and classifying the child’s growth as adequate or inadequate will not help the child grow.
Oh, those crazy folks of 2006. Little did they know that in a few years education reform and testing would be fully committed and devoted to the notion that you can make a pig gain weight by weighing it. All this excellent advice about setting cut scores, and none of it appears to be getting use these days.
I'm not going to go too much more into this document from a company that specializes in educational testing, except to note that once again, the paper frequently notes that personal and professional judgment is a factor at several critical junctures. I will note that they include this step--
The next step is for groups of educators familiar with students in the affected grades and familiar with the subject matter to describe what students should know and be able to do to reach the selected performance levels.
They also are clear that selecting the judges who will set cut scores means making sure they are qualified, have experience, and reflect a demographic cross section. They suggest that policymakers consider fundamental questions such as is it better to pass a student who should fail, or fail a student who should pass? And they are also clear that the full process of setting the cut scores should be documented in painstaking detail, including the rationale for methodology and qualifications of the judges.
And they do refer uniformly to the score-setters as judges, because the whole process involves-- say it with me-- judgment.
People dealing with test scores and test results must remember that setting cut scores is not remotely like the process of surveying with benchmarks. Nothing is set in stone, nothing is judged based on its relationship to something set in stone, and everything is set by people using subjective judgment, not objective standards. We always need to be asking what a cut score is based on, and whether it is any better than a Wild Assed Guess. And when cut cores are set to serve a political purpose, we are right to question whether they have any validity at all.
Surveying markers, including benchmarks, are literally set in stone. Anybody with the necessary training can find them always in the same place and measure any other point in relation to them.
This metaphorical sense of unwavering objective measure is what many folks carry with them to their consideration of testing and cut scores. Passing, failing, and excellence, they figure, are all measured against some scholarly Mean Sea Level marker by way of benchmarks that have been carefully measured against MSL and set in stone.
Sorry, no. Instead, cut scores represent an ideal somewhere between a blindfolded dart player with his fingers duct-taped together, and the guy playing against the blindfolded dart player who sets the darts exactly where he wants them.
Writing in the Stamford Advocate, Wendy Lecker notes that the Smarter Balanced Assessment Consortium members (including Connecticut's own committed foe of public education Commissioner Stefan Pryor) set cut scores for the SBA tests based on stale fairy dust and the wishes of dying puppies.
People tend to assume that cut scores-- the borderline between Good Enough and Abject Failure-- mean something. If a student fails The Test, she must be unready for college or unemployable or illiterate or at the very least several grades behind where she's Supposed To Be (although even that opens up the question "Supposed by whom?")
In fact, SBAC declares that the achievement levels "do not equate directly to expectations for `on-grade' performance" and test scores should only be used with multiple other sources of information about schools and students.
Furthermore, "SBAC admits it cannot validate whether its tests measure college readiness until it has data on how current test takers do in college."
If you are imagining that cut scores for the high-stakes accountability tests are derived through some rigorous study of exactly what students need to know and what level of proficiency they should have achieved by a certain age-- well, first, take a look at what you're assuming. Did you really think we have some sort of master list, some scholastic Mean Sea Level that tells us exactly what a human being of a certain age should know and be able to do as agreed upon by some wise council of experty experts? Because if you do, you might as well imagine that those experts fly to their meetings on pink pegasi, a flock of winger horsies that dance on rainbows and take minutes of the Wise Expert meetings by dictating to secretarial armadillos clothed in shimmering mink stoles.
Anyway, it doesn't matter because there are no signs that any of these people associated with The Test are trying to work with a hypothetical set of academic standards anyway. Instead, what we see over and over (even back in the days of NCLB), is educational amateurs setting cut scores for political purposes. So SBAC sets a cut score so that almost two thirds of the students will fail. John King in New York famously predicted the percentage of test failure before the test was even out the door-- but the actual cut scores were set after the test was taken.
That is not how you measure a test result against a standard. That's how you set a test standard based on the results you want to see. It's how you make your failure predictions come true. According to Carol Burris, King also attempted to find some connection between SAT results and college success prediction, and then somehow graft that onto a cut score for the NY tests, while Kentucky and other CCSS states played similar games with the ACT.
Setting cut scores is not an easy process. Education Sector, a division of the thinky tank American Institutes for Research (they specialize in behavioral sciency thinking, and have a large pedigree in the NCLB era and beyond), issued an "explainer" in July of 2006 about how states set passing scores on standardized tests. It leads off its section on cut scores with this:
On a technical level, states set cut scores along one of two dimensions: The characteristics of the test items or the characteristics of the test takers.It is essential to understand that either way is an inescapably subjective process. Just as academic standards are ultimately the result of professional judgment rather than absolute truth, there is no “right” way to set cut scores, and different methods have various strengths and weaknesses.
The paper goes on to talk about setting cut scores, and some of it is pretty technical, but it returns repeatedly to the notion that at various critical junctures, some human being is going to make a judgment call.
Educational Testing Service (ETS) also has a nifty "Primer on Setting Cut Scores on Tests of Educational Achievement." Again, from all the way back in 2006, this gives a quick compendium of various techniques for setting cut scores-- it lists eight different methods. And it also opens with some insights that would still be useful to consider today.
The first step is for policymakers to specify exactly why cut scores are being set in the first place. The policymakers should describe the benefits that are expected from the use of cut scores. What decisions will be made on the basis of the cut scores? How are those decisions being made now in the absence of cut scores? What reasons are there to believe that cut scores will result in better decisions? What are the expected benefits of the improved decisions?
Yeah, those conversations have not been happening within anyone's earshot. Then there is this:
It is important to list the reasons why cut scores are being set and to obtain consensus among stakeholders that the reasons are appropriate. An extremely useful exercise is to attempt to describe exactly how the cut scores will bring about each of the desired outcomes. It may be the case that some of the expected benefits of cut scores are unlikely to be achieved unless major educational reforms are accomplished. It will become apparent that cut scores, by themselves, have very little power to improve education. Simply measuring a child and classifying the child’s growth as adequate or inadequate will not help the child grow.
Oh, those crazy folks of 2006. Little did they know that in a few years education reform and testing would be fully committed and devoted to the notion that you can make a pig gain weight by weighing it. All this excellent advice about setting cut scores, and none of it appears to be getting use these days.
I'm not going to go too much more into this document from a company that specializes in educational testing, except to note that once again, the paper frequently notes that personal and professional judgment is a factor at several critical junctures. I will note that they include this step--
The next step is for groups of educators familiar with students in the affected grades and familiar with the subject matter to describe what students should know and be able to do to reach the selected performance levels.
They also are clear that selecting the judges who will set cut scores means making sure they are qualified, have experience, and reflect a demographic cross section. They suggest that policymakers consider fundamental questions such as is it better to pass a student who should fail, or fail a student who should pass? And they are also clear that the full process of setting the cut scores should be documented in painstaking detail, including the rationale for methodology and qualifications of the judges.
And they do refer uniformly to the score-setters as judges, because the whole process involves-- say it with me-- judgment.
People dealing with test scores and test results must remember that setting cut scores is not remotely like the process of surveying with benchmarks. Nothing is set in stone, nothing is judged based on its relationship to something set in stone, and everything is set by people using subjective judgment, not objective standards. We always need to be asking what a cut score is based on, and whether it is any better than a Wild Assed Guess. And when cut cores are set to serve a political purpose, we are right to question whether they have any validity at all.
Subscribe to:
Posts (Atom)