I don't usually do this, but I just hammered out a large post. I'm now to attempt a short version. If you find it intriguing, follow the link to the long version.
Some reformsters keep trying to shore up teacher evaluation systems by referring to the private sector where, we are told, folks are rewarded for doing a good job and suffer for doing poorly. Why, they ask, should teaching not come with a performance based incentive system?
I believe that's a false comparison. A really false comparison.
Teachers are not private sector employees. They are public servants, and public service does not play well with performance incentives.
Nobody proposes that we make performance incentives for police officers that reward them for higher numbers of arrests.
Nobody proposes that we create fire fighter performance incentives based on the total dollar amount of property they save in a month.
Nobody proposes that mail delivery persons be rewarded for delivering to a certain number of addresses per hour (if you can't see why not, think about Montana instead of New York City).
Nobody proposes that we give national guardsmen bonuses based on the income of the people whose property they protect.
Nobody proposes these things because public service means serving the whole public, all of them, equally, without prejudice, without ignoring a part of the public because it might hurt your numbers or screw up your monthly performance bonus.
I stretch it out and fill in the gaps in the longer post, but that's the nut of it.
Monday, October 13, 2014
How We Pay Public Servants-- and Why
We have always paid public servants a flat fee, untethered to any sort of "performance measures." That's because we want public service to be completely disconnected from any private interests. (And if you just thought, "Damn, this is a long post," you can get the basic point here and decide if you want to travel down the whole web of alleys with me.)
Fighting Fire with Money
Imagine if, for instance, we paid fire fighters on sliding scale, based on how many of which type fires they put out at a certain speed. This would be disastrous for many reasons.
Fire fighters would refuse to work in cities where there were few fires to fight, because they couldn't make a living. In cities where there were commonly multiple fires, fire fighters would look at each fire call through a lens of "What's in it for me?"
For instance, in a system where fire fighters were paid based on the value of the flame-besieged property, fire fighters might view some small building fires as Not Worth the Trouble. Why bother traveling to the other side of the tracks? It's only a hundred-dollar blaze, anyway. Let's wait till something breaks out up in the million-dollar neighborhood.
In the worst-case scenario, one of our fire fighters depending on performance-based pay to feed his family may be tempted to grab some matches and go fire up some business.
Perverse Incentivization
Occasionally we've seen these kinds of perverse incentives in action, and we don't much like it. The areas of the country where you take extra speed limit care at the end of the month because the local police have a quota to meet. The neighborhood where cops have to roust a certain number of suspects a week to keep their job ratings. Nobody thinks these are examples of excellence in public service.
In fact, we have a history of playing with private police forces and private fire companies. We don't much care for how that works out, because it creates a system that provides excellent service-- but only for the customers who are paying for it.
The idea of public service is to create a class of people who are above self-interest and who do not respond to a single boss. We are outraged when abuse of police power happens precisely because we expect the police to act as if they work for everyone, and to put their dedication to that service above any single interests, including their own.
That's the definition of public service-- service roles that are stripped of any possibility of incentives other than the mandate to serve the public good. That's what we mean by "professional"-- a person who puts all personal self-interest aside and focuses on Getting the Job Done. Trying to motivate a public servant with self-interest inevitably tends to pollute the professional setting with the very self-interest that we're trying to get out of there.
Incentives and Suck-ups
Here's the thing about performance incentives. They always come from actual individual humans. In business that's okay, because the humans are already the bosses.
In public service, we often talk about performance incentives as if they fall from the sky, descending fully-formed from some on-high objective source. They are not. They, too, are developed by actual individual humans. And those humans will invariably encode their own values and priorities into the incentives.
"I like red houses. I think they are more valuable," say our fire company evaluators. "That's why I live in one. And that's why a good fire company always gives priority to saving red houses first."
Performance incentives for public service always-- always-- involve substituting the values of the few for the values of everybody. Fire fighters are supposed to save save everybody's homes. Police are supposed to protect every citizen. The US Postal Service is supposed to deliver to every home. They are there to serve the public, and that means everybody. Public servants are supposed to support the values of all citizens. Any performance based evaluation reward system will prioritize some citizens' values over those of other citizens.
A Public Service Performance Based Incentive System In Action
You know which public servants have a fully-realized performance-based pay system in place?
Congress.
The CEO of International Whoomdinglers says, "That Senator Bogswaller has done an excellent job of looking out for the things we believe are important. He ought to keep his job. Send him a big fat check."
The head of the Society for Preservation of Free-Range Spongemonkeys says, "We appreciate the hard work that Representative Whangdoodle has done looking out for spongemonkeys. He deserves a raise. Send him a big fat check."
You (and the members of the Supreme Court who are paying attention) might call this corruption, but it's just a Performance Based Incentive System, and the high regard with which Congress is held tells you how well a PBIS mixes with public service.
But But But
But a Performance Based Incentive System put in place by the government would not be run like the hodge-podge of private interests you describe incentivizing the US Congress, you say.
And I say, baloney. We already have a Performance Based Incentive System that says you're a better school district if you sell more of the College Board's AP product line. The PBIS testing system being used to incentivize students and teachers and schools-- that system is entirely a product of private corporate interests.
The only difference between an private incentive system, like the one that runs Congress, and a public one, like Race to the Top, is whether the people with money and power have to manipulate a government middle man or can go straight to the source.
Under the Umbrella
I teach mostly juniors, sometimes seniors. There are a few things I tell them every year.
One is to make the most out of senior year, because it is the last time they will be surrounded by people who are paid to put the students' interests first. It's the last time they'll be in an institution that is organized around their concerns, their interests, their needs. After that, they're in the open market. They will always be dealing with people who are trying to sell them something.
The PSAT will collect a ton of information about them so it can turn around and sell that data to colleges. Colleges will try to sell them, particularly if they are highly desirablecustomers students. Employers will try to get the use of their talents without having to pay much for it, and politicians will piss on them and tell them it's raining so that the pols can keep their jobs.
But here, under the umbrella of the public school, my students have nothing that I need to survive or make a living. I have no reason to do this job except for the reason I took this job in the first place-- to serve the best interests of my students.
It Doesn't Make Any Difference
It makes business-oriented reformy types crazy that the way I do my job doesn't make any difference to my pay. I understand the terror for them there, but that Not Making A Difference is actually the point of how we pay public servants.
It doesn't matter it's a big fire or a small fire, a rich person's house or a poor person's house-- the fire department still does their job. It doesn't matter whether I have a classroom full of bright students or slow students, rich students or poor students, ambitious students or lazy students-- I will still show up and do my job the best I know how. I should never, ever, ever have to look at a class roster or a set of test results or a practice quiz and think, "Dammit, these kids are going to keep me from making my house payment next month."
Why I Won't Suck
Reformsters are sure that human beings must be motivated by threats and rewards, and that the lack of threats and rewards means that I can too easily choose to do a crappy job, because it won't make any difference. They are wrong. Here's why.
1) I knew the gig when I started. I knew I would not get rich, not be powerful, not have a chance to rise to some position of prominence. There was no reason to enter teaching in the first except a desire to do right by the students.
2) Teaching is too hard to do half-assed. Do a consistently lousy job, and the students will eat you alive and dragging yourself out of bed every day will be too damn much. There isn't enough money to keep people flailing badly in a classroom for a lifetime. Just ask all the TFA dropouts who said, "Damn! This is hella hard!" and left the classroom.
And Most Importantly
Threats and rewards do not make people better public servants (nor have I ever seen a lick of research that suggests otherwise, but feel free to review this oft-linked video re: motivation). Threats and rewards interfere with people's ability to get their job done. Threats and rewards motivate people to game the system.
And any time you have a complex system being measured with simple instruments, you have a system that is ripe for gaming. In fact, if your measures are bad enough (looking at you, high stakes tests and VAM), your system can only be successfully operated by gaming it.
So, No Accountability At All?
Heck, no. You need to keep an eye out for the grossly incompetent, though they will often self-identify (I've made a huge mistake) and take the next stage out of Dodge. Beyond that, you just go watch and pay attention. If you're my administration, you're welcome in my classroom at any time for as long as you'd like to stay. No, don't bring that stupid checklist. Just watch and listen and use your professional judgment. If you think I need to fix something, let's you and I talk about it. How will bringing in extra layers of bureaucracy and government make that system work any more smoothly?
Man, This Is Running Long
Agreed. Sometimes these posts get away from me.
Bottom line-- the comparison to private enterprise performance based incentive systems is bogus. Those systems may be appropriate in corporate environments where we want to enforce a bias in favor of certain actors and outcomes-- where some people are in fact more important than others.
But in public service, performance based incentive systems are contra-indicated. They by nature enforce a particular bias and cannot help but tilt the system in favor of some customers over others.
The system we have does, in fact, make sense. We stand our public servants beside a door and we say, "I'm going to pay you to stand here and wait as long as it takes and help whoever comes through that door. It doesn't matter who comes through that door-- nothing is going to affect your pay, so that's a settled and done deal-- just concentrate on watching that door and helping whoever walks through it."
Are You Still Here?
God bless you. This chunk of ideas has only just grabbed ahold of me, and I still have much to sort out, which I will undoubtedly do in future posts. Feel free to chime in in the comments.
Fighting Fire with Money
Imagine if, for instance, we paid fire fighters on sliding scale, based on how many of which type fires they put out at a certain speed. This would be disastrous for many reasons.
Fire fighters would refuse to work in cities where there were few fires to fight, because they couldn't make a living. In cities where there were commonly multiple fires, fire fighters would look at each fire call through a lens of "What's in it for me?"
For instance, in a system where fire fighters were paid based on the value of the flame-besieged property, fire fighters might view some small building fires as Not Worth the Trouble. Why bother traveling to the other side of the tracks? It's only a hundred-dollar blaze, anyway. Let's wait till something breaks out up in the million-dollar neighborhood.
In the worst-case scenario, one of our fire fighters depending on performance-based pay to feed his family may be tempted to grab some matches and go fire up some business.
Perverse Incentivization
Occasionally we've seen these kinds of perverse incentives in action, and we don't much like it. The areas of the country where you take extra speed limit care at the end of the month because the local police have a quota to meet. The neighborhood where cops have to roust a certain number of suspects a week to keep their job ratings. Nobody thinks these are examples of excellence in public service.
In fact, we have a history of playing with private police forces and private fire companies. We don't much care for how that works out, because it creates a system that provides excellent service-- but only for the customers who are paying for it.
The idea of public service is to create a class of people who are above self-interest and who do not respond to a single boss. We are outraged when abuse of police power happens precisely because we expect the police to act as if they work for everyone, and to put their dedication to that service above any single interests, including their own.
That's the definition of public service-- service roles that are stripped of any possibility of incentives other than the mandate to serve the public good. That's what we mean by "professional"-- a person who puts all personal self-interest aside and focuses on Getting the Job Done. Trying to motivate a public servant with self-interest inevitably tends to pollute the professional setting with the very self-interest that we're trying to get out of there.
Incentives and Suck-ups
Here's the thing about performance incentives. They always come from actual individual humans. In business that's okay, because the humans are already the bosses.
In public service, we often talk about performance incentives as if they fall from the sky, descending fully-formed from some on-high objective source. They are not. They, too, are developed by actual individual humans. And those humans will invariably encode their own values and priorities into the incentives.
"I like red houses. I think they are more valuable," say our fire company evaluators. "That's why I live in one. And that's why a good fire company always gives priority to saving red houses first."
Performance incentives for public service always-- always-- involve substituting the values of the few for the values of everybody. Fire fighters are supposed to save save everybody's homes. Police are supposed to protect every citizen. The US Postal Service is supposed to deliver to every home. They are there to serve the public, and that means everybody. Public servants are supposed to support the values of all citizens. Any performance based evaluation reward system will prioritize some citizens' values over those of other citizens.
A Public Service Performance Based Incentive System In Action
You know which public servants have a fully-realized performance-based pay system in place?
Congress.
The CEO of International Whoomdinglers says, "That Senator Bogswaller has done an excellent job of looking out for the things we believe are important. He ought to keep his job. Send him a big fat check."
The head of the Society for Preservation of Free-Range Spongemonkeys says, "We appreciate the hard work that Representative Whangdoodle has done looking out for spongemonkeys. He deserves a raise. Send him a big fat check."
You (and the members of the Supreme Court who are paying attention) might call this corruption, but it's just a Performance Based Incentive System, and the high regard with which Congress is held tells you how well a PBIS mixes with public service.
But But But
But a Performance Based Incentive System put in place by the government would not be run like the hodge-podge of private interests you describe incentivizing the US Congress, you say.
And I say, baloney. We already have a Performance Based Incentive System that says you're a better school district if you sell more of the College Board's AP product line. The PBIS testing system being used to incentivize students and teachers and schools-- that system is entirely a product of private corporate interests.
The only difference between an private incentive system, like the one that runs Congress, and a public one, like Race to the Top, is whether the people with money and power have to manipulate a government middle man or can go straight to the source.
Under the Umbrella
I teach mostly juniors, sometimes seniors. There are a few things I tell them every year.
One is to make the most out of senior year, because it is the last time they will be surrounded by people who are paid to put the students' interests first. It's the last time they'll be in an institution that is organized around their concerns, their interests, their needs. After that, they're in the open market. They will always be dealing with people who are trying to sell them something.
The PSAT will collect a ton of information about them so it can turn around and sell that data to colleges. Colleges will try to sell them, particularly if they are highly desirable
But here, under the umbrella of the public school, my students have nothing that I need to survive or make a living. I have no reason to do this job except for the reason I took this job in the first place-- to serve the best interests of my students.
It Doesn't Make Any Difference
It makes business-oriented reformy types crazy that the way I do my job doesn't make any difference to my pay. I understand the terror for them there, but that Not Making A Difference is actually the point of how we pay public servants.
It doesn't matter it's a big fire or a small fire, a rich person's house or a poor person's house-- the fire department still does their job. It doesn't matter whether I have a classroom full of bright students or slow students, rich students or poor students, ambitious students or lazy students-- I will still show up and do my job the best I know how. I should never, ever, ever have to look at a class roster or a set of test results or a practice quiz and think, "Dammit, these kids are going to keep me from making my house payment next month."
Why I Won't Suck
Reformsters are sure that human beings must be motivated by threats and rewards, and that the lack of threats and rewards means that I can too easily choose to do a crappy job, because it won't make any difference. They are wrong. Here's why.
1) I knew the gig when I started. I knew I would not get rich, not be powerful, not have a chance to rise to some position of prominence. There was no reason to enter teaching in the first except a desire to do right by the students.
2) Teaching is too hard to do half-assed. Do a consistently lousy job, and the students will eat you alive and dragging yourself out of bed every day will be too damn much. There isn't enough money to keep people flailing badly in a classroom for a lifetime. Just ask all the TFA dropouts who said, "Damn! This is hella hard!" and left the classroom.
And Most Importantly
Threats and rewards do not make people better public servants (nor have I ever seen a lick of research that suggests otherwise, but feel free to review this oft-linked video re: motivation). Threats and rewards interfere with people's ability to get their job done. Threats and rewards motivate people to game the system.
And any time you have a complex system being measured with simple instruments, you have a system that is ripe for gaming. In fact, if your measures are bad enough (looking at you, high stakes tests and VAM), your system can only be successfully operated by gaming it.
So, No Accountability At All?
Heck, no. You need to keep an eye out for the grossly incompetent, though they will often self-identify (I've made a huge mistake) and take the next stage out of Dodge. Beyond that, you just go watch and pay attention. If you're my administration, you're welcome in my classroom at any time for as long as you'd like to stay. No, don't bring that stupid checklist. Just watch and listen and use your professional judgment. If you think I need to fix something, let's you and I talk about it. How will bringing in extra layers of bureaucracy and government make that system work any more smoothly?
Man, This Is Running Long
Agreed. Sometimes these posts get away from me.
Bottom line-- the comparison to private enterprise performance based incentive systems is bogus. Those systems may be appropriate in corporate environments where we want to enforce a bias in favor of certain actors and outcomes-- where some people are in fact more important than others.
But in public service, performance based incentive systems are contra-indicated. They by nature enforce a particular bias and cannot help but tilt the system in favor of some customers over others.
The system we have does, in fact, make sense. We stand our public servants beside a door and we say, "I'm going to pay you to stand here and wait as long as it takes and help whoever comes through that door. It doesn't matter who comes through that door-- nothing is going to affect your pay, so that's a settled and done deal-- just concentrate on watching that door and helping whoever walks through it."
Are You Still Here?
God bless you. This chunk of ideas has only just grabbed ahold of me, and I still have much to sort out, which I will undoubtedly do in future posts. Feel free to chime in in the comments.
The Teacher Jobs Shortfall
Last week the Economic Policy Institute popped out an article attached to this graphic:
Their headline was the dramatic shortfall of jobs in public ed, particularly when compared to their projection of the number of jobs needed to keep up with enrollment.
I have a number of questions raised by the graph.
Does this reflect a shift to private employers? How many public school jobs are being shifted to private and charter schools? We know about the dramatic shifts of this nature occurring in places like New Orleans and Cleveland, where pubic schools have been shut down so that they can be replaced with charters. But is that process occurring in smaller ways throughout the country-- Bogwump High School cuts three teachers, but Bogwump Corporate Charter opens with a six teacher staff. And that raises its own second question-- is the EPI accepting the fiction that charter schools are public schools, or is it correctly counting charter teaching jobs as non public school jobs.
Public education added 400,000 jobs between 2003 and 2008?!?! Okay, not exactly a question, but-- damn. Did everyone go on a staff spending spree with their stimulus money? Did regulations and court decisions create a ballooning demand for special needs teachers? I mean, I was going to look for data to indicate whether that's abnormal growth or not, but I don't really need to. If that were normal growth, that would mean twenty yeas ago there were no teaching jobs at all, and I'm pretty sure that's wrong.
But that raises the real question-- are we looking at a shortfall, or a market correction after an employment bubble?
What is that projection based on? EPI says we should have added 123,000 jobs over the 2008 numbers (or 377,000 over where we are now) in order to keep pace with student growth. I'm curious about exactly what formula was used top determine that number.
Where is the understaffing happening? If this chart and the EPI interpretation of it are correct, we have schools that are grossly understaffed, overworked, and financially strapped. Everything I know tells me that is true-- but not everywhere. My own school, for instance, has shed many staff positions in the last ten years, but we have also dropped student population considerably. The same story is true in most districts in my region.
So has somebody done a more specific regional study? Do we know exactly where local public schools are failing to keep staffing rates up to speed with student population numbers? This is definitely more of a problem in some places than in others-- do we know where those places are? And do we have some data beyond the anecdotal?
The Great Recession was hard on teaching as a profession, and it raised the curtain on reformster policies that were also hard on the teaching profession, particularly in public schools. Given the number of people who have been chased out of the profession, this chart is but half as picture, and too broad of one to be really useful. The profession is hemorrhaging teachers, particularly teachers of color and young teachers, and where it is not hemorrhaging them, it is chasing them away vigorously by making the profession as unattractive as possible (looking at you, North Carolina).
Are we shedding jobs to keep up with shedding teachers? Are we chasing teachers away by shedding jobs? And most importantly-- how do these dynamics play out differently from place to place?
Ultimately this chart raises many questions for me. If you have the answers, let me know. If I find any answers, I'll pass them along.
Their headline was the dramatic shortfall of jobs in public ed, particularly when compared to their projection of the number of jobs needed to keep up with enrollment.
I have a number of questions raised by the graph.
Does this reflect a shift to private employers? How many public school jobs are being shifted to private and charter schools? We know about the dramatic shifts of this nature occurring in places like New Orleans and Cleveland, where pubic schools have been shut down so that they can be replaced with charters. But is that process occurring in smaller ways throughout the country-- Bogwump High School cuts three teachers, but Bogwump Corporate Charter opens with a six teacher staff. And that raises its own second question-- is the EPI accepting the fiction that charter schools are public schools, or is it correctly counting charter teaching jobs as non public school jobs.
Public education added 400,000 jobs between 2003 and 2008?!?! Okay, not exactly a question, but-- damn. Did everyone go on a staff spending spree with their stimulus money? Did regulations and court decisions create a ballooning demand for special needs teachers? I mean, I was going to look for data to indicate whether that's abnormal growth or not, but I don't really need to. If that were normal growth, that would mean twenty yeas ago there were no teaching jobs at all, and I'm pretty sure that's wrong.
But that raises the real question-- are we looking at a shortfall, or a market correction after an employment bubble?
What is that projection based on? EPI says we should have added 123,000 jobs over the 2008 numbers (or 377,000 over where we are now) in order to keep pace with student growth. I'm curious about exactly what formula was used top determine that number.
Where is the understaffing happening? If this chart and the EPI interpretation of it are correct, we have schools that are grossly understaffed, overworked, and financially strapped. Everything I know tells me that is true-- but not everywhere. My own school, for instance, has shed many staff positions in the last ten years, but we have also dropped student population considerably. The same story is true in most districts in my region.
So has somebody done a more specific regional study? Do we know exactly where local public schools are failing to keep staffing rates up to speed with student population numbers? This is definitely more of a problem in some places than in others-- do we know where those places are? And do we have some data beyond the anecdotal?
The Great Recession was hard on teaching as a profession, and it raised the curtain on reformster policies that were also hard on the teaching profession, particularly in public schools. Given the number of people who have been chased out of the profession, this chart is but half as picture, and too broad of one to be really useful. The profession is hemorrhaging teachers, particularly teachers of color and young teachers, and where it is not hemorrhaging them, it is chasing them away vigorously by making the profession as unattractive as possible (looking at you, North Carolina).
Are we shedding jobs to keep up with shedding teachers? Are we chasing teachers away by shedding jobs? And most importantly-- how do these dynamics play out differently from place to place?
Ultimately this chart raises many questions for me. If you have the answers, let me know. If I find any answers, I'll pass them along.
Sunday, October 12, 2014
What We Haven't Learned from the Widget Effect
Do you remember that awesome post I wrote that totally changed the face of American education?? You don't?? Well, let me just keep mentioning that awesome post (and how it changed the face of American education) for the next five years and maybe my massive importance will start to sink in.
That's about where we are with TNTP and "The Widget Effect," a "report" I'm not going to link to for the same reason I don't mention TNTP's leader by name or provide links to pro-anorexia sites-- some things are just already taking up too much of the internet.
The Widget Effect is celebrating its fifth anniversary of its own importance. If you're unfamiliar with the "report," let me summarize it for you:
We don't pay teachers differently based on how good they are. We should do that.
That's it. Pump it up with extra verbage and slap on some high-fallutin' graphics, and you've got a "report" that other "report" writers love when they need to add some gravitas to the footnote section of their "report." As you may have heard, there's particular interest in the "We should do that" portion; TNTP is a huge fan of teacher evaluating.
TNTP has presented several anniversary evaluation commentary-paloozas, including this one that sandwiches a thoughtful Andy Smarick piece in between two large slabs of reformy baloney. But that's not where we're headed today. Today we're going to look at "4 Things We've Learned Since the Widget Effect." Let's do a little check for understanding and see if our five years of study have paid off.
Implementation Matters More Than Design
Correct! Reformsters have learned (and are still learning) that if you promise people a warm, cuddly pet and then drop an angry badger into their home, they lose interest in your promises very quickly. Further, you do not provide useful damage control by repeating, "But it's really intended to be warm and cuddly" while the badger has the children cornered and terrified on top of the credenza. Teacher evaluation has had teachers on top of the credenza for about five years, so happy anniversary, honey badger!
TNTP offers a solution best summarized as "Do it better." Sigh. In more words, the recommendation is that if you train your key people and give them time to do a better job, the badgers will be warmer and cuddlier. TNTP describes these key people with words like "Chief Academic Officers" and "middle managers." The odd terminology leads us back to a central question-- does TNTP think the badgers are warm and cuddly, or does it just want to convince us so we'll let the badgers trash the house. I won't rule out the former, but I lean toward the latter.
Multiple Measures-- Including Data about Student Learning Growth-- Are the Way To Go
The old observation technique was a bust, TNTP says. They support this by saying that it's just common sense. So there ya go.
While the issue of evaluation remains hotly debated, multiple measures might be the one place where something resembling a consensus has emerged. That’s a positive thing we should celebrate.
Really? Which consensus would that be? There's a fairly large consensus that "including data about students learning growth" (aka VAM) is problematic because every instrument we have that claims to do it is no more reliable than having the badgers read tea leaves through a crystal ball. I'm guessing that's not the consensus being referenced.
So incorrect on the main answer. Their recommendation, however, is to have multiple observations by multiple observers. In buildings with enough administrative staff to implement it, that idea is... not stupid.
You Can't Fix Observations If Observers Don't Rate Accurately
Observations are also one of the best examples of the gap between design and implementation. If you’re concerned about the potential variability of value-added scores, you should be truly frightened by the statistical Wild West that is classroom observations.
They're onto something here. Here's the thing about administrators-- if they are even remotely competent, they know how good their teachers are. They'll use the fancy piece of paper if you make them, but if the observation instrument tells them one thing and their brain, sense, and professional judgment tell them another, guess who wins. If you ask, "What are you going to believe-- the observation form or your own eyes?" They will go with their own senses.
Now, if your principal is a boob, or hates you for some reason, this effect is Very Bad News. Maybe you call that the statistical Wild West, but that's still better than VAM, which is a statistical black hole caught in a box with Schroedinger's cat strapped into the hold of the Andrea Doria sailing through the Sargasso Sea as it falls into the Negative Zone just as the Genesis Bomb goes off.
TNTP's solution-- easier, shorter paperwork. Because reducing a complicated human observation of complex human interactions to a short, simple checklist totally works. I suggest that TNTP staffers field test the principle by piloting a spousal observation form to be tested on evaluating their wives and husbands.
Double fail on this item.
Done Right, Teacher Evaluations Really Can Help Teachers and Students
We're going to go to the research connected to the IMPACT evaluation system in DC. And damn-- these people can't really be that confused or dopey, can they? I want to believe that they are willfully manipulative and misleading, because that would at least mean they're smart enough to understand what they're saying, and as a teacher, it makes me sad to imagine a lump of dumb this large in the world.
Okay, here's the deal. They measure a teacher's awesomeness. They give the teacher feedback on the measurement. They measure again, and the teacher proves to be more awesome. Let me see if I can illustrated why this proves almost nothing.
Chris: If you pass my test for being my awesomest friend, I will give you a dollar. Now, hold up some fingers?
Pat: Okay. How'd I do?
Chris: Bummer. If you had held up four fingers instead of three, I would have known you were my awesomest friend.
[Fifteen minutes later]
Chris: Okay, let's take the awesome friend test again. Hold up some fingers.
Pat: Cool.
Chris: You did it. Four fingers!! Here's a dollar!
[Later over supper at Chris's house]
Chris: Mom, Pat and I became much better friends this afternoon!
The IMPACT system and the attendant research are not useless. They prove that teachers can be trained to respond to certain stimuli as easily as lab rats. They do not, however, prove jack or squat about how the system "improves" teaching-- only that it improves teacher response to the system.
TNTP recommends staying the course. I recommend that TNTP release a dozen honey badgers into their offices and hold some special training meetings on top of the credenza. If the credenza is all covered up with the Widget Effect's birthday cake, just feed the cake to the badgers. Tell them they're celebrating one of the most influential reports of the last five years.
That's about where we are with TNTP and "The Widget Effect," a "report" I'm not going to link to for the same reason I don't mention TNTP's leader by name or provide links to pro-anorexia sites-- some things are just already taking up too much of the internet.
The Widget Effect is celebrating its fifth anniversary of its own importance. If you're unfamiliar with the "report," let me summarize it for you:
We don't pay teachers differently based on how good they are. We should do that.
That's it. Pump it up with extra verbage and slap on some high-fallutin' graphics, and you've got a "report" that other "report" writers love when they need to add some gravitas to the footnote section of their "report." As you may have heard, there's particular interest in the "We should do that" portion; TNTP is a huge fan of teacher evaluating.
TNTP has presented several anniversary evaluation commentary-paloozas, including this one that sandwiches a thoughtful Andy Smarick piece in between two large slabs of reformy baloney. But that's not where we're headed today. Today we're going to look at "4 Things We've Learned Since the Widget Effect." Let's do a little check for understanding and see if our five years of study have paid off.
Implementation Matters More Than Design
Correct! Reformsters have learned (and are still learning) that if you promise people a warm, cuddly pet and then drop an angry badger into their home, they lose interest in your promises very quickly. Further, you do not provide useful damage control by repeating, "But it's really intended to be warm and cuddly" while the badger has the children cornered and terrified on top of the credenza. Teacher evaluation has had teachers on top of the credenza for about five years, so happy anniversary, honey badger!
TNTP offers a solution best summarized as "Do it better." Sigh. In more words, the recommendation is that if you train your key people and give them time to do a better job, the badgers will be warmer and cuddlier. TNTP describes these key people with words like "Chief Academic Officers" and "middle managers." The odd terminology leads us back to a central question-- does TNTP think the badgers are warm and cuddly, or does it just want to convince us so we'll let the badgers trash the house. I won't rule out the former, but I lean toward the latter.
Multiple Measures-- Including Data about Student Learning Growth-- Are the Way To Go
The old observation technique was a bust, TNTP says. They support this by saying that it's just common sense. So there ya go.
While the issue of evaluation remains hotly debated, multiple measures might be the one place where something resembling a consensus has emerged. That’s a positive thing we should celebrate.
Really? Which consensus would that be? There's a fairly large consensus that "including data about students learning growth" (aka VAM) is problematic because every instrument we have that claims to do it is no more reliable than having the badgers read tea leaves through a crystal ball. I'm guessing that's not the consensus being referenced.
So incorrect on the main answer. Their recommendation, however, is to have multiple observations by multiple observers. In buildings with enough administrative staff to implement it, that idea is... not stupid.
You Can't Fix Observations If Observers Don't Rate Accurately
Observations are also one of the best examples of the gap between design and implementation. If you’re concerned about the potential variability of value-added scores, you should be truly frightened by the statistical Wild West that is classroom observations.
They're onto something here. Here's the thing about administrators-- if they are even remotely competent, they know how good their teachers are. They'll use the fancy piece of paper if you make them, but if the observation instrument tells them one thing and their brain, sense, and professional judgment tell them another, guess who wins. If you ask, "What are you going to believe-- the observation form or your own eyes?" They will go with their own senses.
Now, if your principal is a boob, or hates you for some reason, this effect is Very Bad News. Maybe you call that the statistical Wild West, but that's still better than VAM, which is a statistical black hole caught in a box with Schroedinger's cat strapped into the hold of the Andrea Doria sailing through the Sargasso Sea as it falls into the Negative Zone just as the Genesis Bomb goes off.
TNTP's solution-- easier, shorter paperwork. Because reducing a complicated human observation of complex human interactions to a short, simple checklist totally works. I suggest that TNTP staffers field test the principle by piloting a spousal observation form to be tested on evaluating their wives and husbands.
Double fail on this item.
Done Right, Teacher Evaluations Really Can Help Teachers and Students
We're going to go to the research connected to the IMPACT evaluation system in DC. And damn-- these people can't really be that confused or dopey, can they? I want to believe that they are willfully manipulative and misleading, because that would at least mean they're smart enough to understand what they're saying, and as a teacher, it makes me sad to imagine a lump of dumb this large in the world.
Okay, here's the deal. They measure a teacher's awesomeness. They give the teacher feedback on the measurement. They measure again, and the teacher proves to be more awesome. Let me see if I can illustrated why this proves almost nothing.
Chris: If you pass my test for being my awesomest friend, I will give you a dollar. Now, hold up some fingers?
Pat: Okay. How'd I do?
Chris: Bummer. If you had held up four fingers instead of three, I would have known you were my awesomest friend.
[Fifteen minutes later]
Chris: Okay, let's take the awesome friend test again. Hold up some fingers.
Pat: Cool.
Chris: You did it. Four fingers!! Here's a dollar!
[Later over supper at Chris's house]
Chris: Mom, Pat and I became much better friends this afternoon!
The IMPACT system and the attendant research are not useless. They prove that teachers can be trained to respond to certain stimuli as easily as lab rats. They do not, however, prove jack or squat about how the system "improves" teaching-- only that it improves teacher response to the system.
TNTP recommends staying the course. I recommend that TNTP release a dozen honey badgers into their offices and hold some special training meetings on top of the credenza. If the credenza is all covered up with the Widget Effect's birthday cake, just feed the cake to the badgers. Tell them they're celebrating one of the most influential reports of the last five years.
Update on Pearson's Mistake
For those of you who followed my earlier post about the (latest) Pearson test screw-up, the multi-national educational juggernaut did actually respond to Sarah Blaine's original post with an apology of sorts.
Pearson did make an error on the specific quiz question in a lesson in the Envision Math textbook and we sincerely apologize for this mistake.
As corporate apologies go, it's actually refreshingly clear and unequivocally weasel-word free.
The down side, as Blaine notes, is that it doesn't acknowledge the larger issues she raised, other than to note that "trust in our products and services is key." Well, yes. And I would make fun of them for pointing out the obvious, except that we see too many examples of corporations that don't see the obvious, so bravo, Pearson, on seeing the obvious.
So the larger issues go unaddressed (a little transparency, folks?), but at least they said, "We screwed up. Sorry about that." Which by modern corporate standards is not too shabby. Is it too snarky for me to note that apparently all that practice apologizing for mistakes that Pearson has had is apparently paid off? Maybe? Sorry about that.
Pearson did make an error on the specific quiz question in a lesson in the Envision Math textbook and we sincerely apologize for this mistake.
As corporate apologies go, it's actually refreshingly clear and unequivocally weasel-word free.
The down side, as Blaine notes, is that it doesn't acknowledge the larger issues she raised, other than to note that "trust in our products and services is key." Well, yes. And I would make fun of them for pointing out the obvious, except that we see too many examples of corporations that don't see the obvious, so bravo, Pearson, on seeing the obvious.
So the larger issues go unaddressed (a little transparency, folks?), but at least they said, "We screwed up. Sorry about that." Which by modern corporate standards is not too shabby. Is it too snarky for me to note that apparently all that practice apologizing for mistakes that Pearson has had is apparently paid off? Maybe? Sorry about that.
College-ready Five Year Olds
We periodically hear of the notion of college-ready five year olds. Not that they are ready to go to college while still that young, but that we can clearly tell in kindergarten whether these children are on the collegiate trajectory or not.
Recently a pair of teachers attempted a response to Carol Burris's Real Clear Education interview. Since the two work for Student Achievement Partners, a group started by CCSS architect David Coleman and financed by Bill Gates, what the two SAPs is not exactly a surprise.
The kindergarten SAP argues that her students (at her select charter school in Oakland, California) are able to do super-hard things that let her know that they are ready for college. In particular she is arguing for having kindergartners count to 100. She does not clarify whether she uses the technique of Rote Repetition of the Numbers With No Idea What They Mean or the technique of Counseling Out Students Who Can't Count To 100.
I'm excited about being able to ID college-ready five year olds. This presents a host of opportunities including the chance to start applying to college at age six. I mean, my high school juniors and seniors get very stressed about the whole application process. Imagine how much more relaxed and focused they could be if they had locked up that collegiate spot by age seven. They are just childhood as an excuse to be lazy anyway.
Of course, deciding college that early would really mess with David Coleman"s College Board SAT revenue stream. There's a pretty hefty industry driven by the general college-seeking panic of teenagers and their parents, so even as Core boosters claim that we can determine the college prospects of small children, reformsters once again face two challenging choice:
1) Shift the industry around to monetize the new impact areas or
2) Pretend they don't understand the implications of what they're saying.
For cradle-to-career railroad, it's a big number two all the way.
Mind you, they've occasionally admitted that they really do want to be able to predict the adult life of a small child with a "seamless web that literally extends from cradle to grave." But nobody who A) knows who Big Brother is or B) wants a future in American politics is going to hold up that infamous Marc Tucker "Dear Hillary" letter and say, "Yes, this is what we should do." Not out loud.
Reformsters could argue that the very notion of being able to place a five year old in a college is silly because, of course, any number of things in his life outside of school could happen in the next thirteen years to interfere with his college readiness-- but they can't make that argument because then they would have to admit that life factors outside of school affect the child's education.
No, we have to pretend that the educational journey is a train-- one track, one beginning and ending, everyone traveling along the undeviating, uninterrupted trail.
So if that's true, why wouldn't we fill out those college applications at the end of kindergarten? If all students are going to meet the same standards at the same time, and we can tell whether kindergartners are on track, and there's only one track, why isn't that good enough?
I suspect that in a dark moment of honesty, some reformsters would say it was good enough, that they already knew that Chris was destined for a life of corporate servitude and all we're doing is waiting for the sapling to grow large enough to harvest.
But in the meantime, reformsters will at once pretend that it's not absurd to declare a five year old on track for college, even as they fail to acknowledge the implications of that college-ready declaration. If we know that a five year old is on track for college, why not sign her up now? The answer-- sort of-- is that reformsters can't explain why a five year old's college application is absurd without also explaining why reform itself is absurd.
Recently a pair of teachers attempted a response to Carol Burris's Real Clear Education interview. Since the two work for Student Achievement Partners, a group started by CCSS architect David Coleman and financed by Bill Gates, what the two SAPs is not exactly a surprise.
The kindergarten SAP argues that her students (at her select charter school in Oakland, California) are able to do super-hard things that let her know that they are ready for college. In particular she is arguing for having kindergartners count to 100. She does not clarify whether she uses the technique of Rote Repetition of the Numbers With No Idea What They Mean or the technique of Counseling Out Students Who Can't Count To 100.
I'm excited about being able to ID college-ready five year olds. This presents a host of opportunities including the chance to start applying to college at age six. I mean, my high school juniors and seniors get very stressed about the whole application process. Imagine how much more relaxed and focused they could be if they had locked up that collegiate spot by age seven. They are just childhood as an excuse to be lazy anyway.
Of course, deciding college that early would really mess with David Coleman"s College Board SAT revenue stream. There's a pretty hefty industry driven by the general college-seeking panic of teenagers and their parents, so even as Core boosters claim that we can determine the college prospects of small children, reformsters once again face two challenging choice:
1) Shift the industry around to monetize the new impact areas or
2) Pretend they don't understand the implications of what they're saying.
For cradle-to-career railroad, it's a big number two all the way.
Mind you, they've occasionally admitted that they really do want to be able to predict the adult life of a small child with a "seamless web that literally extends from cradle to grave." But nobody who A) knows who Big Brother is or B) wants a future in American politics is going to hold up that infamous Marc Tucker "Dear Hillary" letter and say, "Yes, this is what we should do." Not out loud.
Reformsters could argue that the very notion of being able to place a five year old in a college is silly because, of course, any number of things in his life outside of school could happen in the next thirteen years to interfere with his college readiness-- but they can't make that argument because then they would have to admit that life factors outside of school affect the child's education.
No, we have to pretend that the educational journey is a train-- one track, one beginning and ending, everyone traveling along the undeviating, uninterrupted trail.
So if that's true, why wouldn't we fill out those college applications at the end of kindergarten? If all students are going to meet the same standards at the same time, and we can tell whether kindergartners are on track, and there's only one track, why isn't that good enough?
I suspect that in a dark moment of honesty, some reformsters would say it was good enough, that they already knew that Chris was destined for a life of corporate servitude and all we're doing is waiting for the sapling to grow large enough to harvest.
But in the meantime, reformsters will at once pretend that it's not absurd to declare a five year old on track for college, even as they fail to acknowledge the implications of that college-ready declaration. If we know that a five year old is on track for college, why not sign her up now? The answer-- sort of-- is that reformsters can't explain why a five year old's college application is absurd without also explaining why reform itself is absurd.
How To Align Painlessly
(Originally post at View from the Cheap Seats)
The Dream
The ideal, as imagined by the Common Core Crowd looks something like this:
A group of fresh-scrubbed teachers gather in a room with a consultant (because, after all, they're only teachers and lack the expertise to do this work on their own). The CCSS expert teaches them how to "unpack" the standards, and over the course of many days, the teachers unpack a standard and decide how best to implement that standard in a well-constructed CCSS-aligned lesson. Depending on the consultant, they will probably also incorporate some features that are not actually in the Core, but which the consultant likes.
Once implemented, this new curriculum will totally revolutionize the way these teachers teach, and soon, awesome test scores will descend upon them like manna from heaven. Because once you align the to the standards, your students will automatically be prepared for the aligned standardized tests.
The Reality
Well, sadly, some peoples' reality will be a pursuit of the dream. If your administration has soaked too long in the special Common Core Coolade, the Dream will be your goal. Good luck with that.
For the earthbound districts, the process looks something more like this.
1) Spread out printouts on big desk. On one side, the printout of the Common Core National Standards (or the lightly edited version that your state is passing off as state-developed standards). On the other side, your pre-existing curriculum.
2) Go through curriculum, cross-referencing against standards. Look for standards that you can reasonably claim are being met by old lessons (e.g. "This paper I have them write about Great Expectations requires them to support their statements with evidence from the text. Check!") Check standards off of list.
3) Gather up standards you didn't meet. Make one last attempt to justify attaching them to pre-existing units. Rig up some new unit to meet those standards. Finish checking off list.
4) Go to PARCC/SBA/Whatever websites and gather up sample questions. Buy some sample question practice books. Schedule test prep units strategically through year.
5) Wait for High Stakes Test results. Analyze results to see what test prep units you need to beef up. Start collecting materials for next year. decide which units you can cut to make room for additional test prep. Do not forget to count your test prep units as part of your aligned curriculum.
Alternate approach
For the above steps 1-4, substitute the folowing:
1) Buy pre-aligned book and materials from dependable vendor like Pearson. Print out their alignment materials. Insert printout in alignment report for district.
Anyone Can Play
Slapping some Common Core numbers on the same old same old is a popular game these days. For instance, at a site called ELA Common Core Lesson Plans, we can find a deadly dull lesson on humor in literature that could easily be from ten, twenty, thirty years ago, except for the CCSS standards tags at the end. Or take this site that cheerfully plugs literature circles as a great CCSS technique, as if lit circles haven't been around almost as long as crop circles. Watch a video of a teacher using basic techniques that were actually well known long before CCSS. Or check out this lesson using song lyrics to teach figurative language, a technique used by every English teacher in the history of ever. But hey-- there are standards numbers attached to it, so it must be Common Core.
In fact, these lessons cover the other challenge of alignment-- what to do with the parts of lessons that actually contradict the Core approach. Teaching humor in texts? You're probably cheating by bringing in information from outside the four corners of the text, because humor's pretty hard to get in a context-free vaccuum. And we'd better hope that the fifth grade teacher working with the text about Harriet Tubman and slavery is not actually answering her students' questions about the origins of racism and slavery, because that would be contrary to the Core's love of Close Reading 2.0.
So painless alignment is not only about finding ways to line up things you already do with the Core-- it also involves ignoring the Core when it wants you to stop using tried and true successful practices you've used in the past.
Painful Alignment
There are of course more painful ways to align. Teachers who have been through a reality-based alignment in which CCSS has no major effect on their classroom may well wonder what the big fuss is. But the Dream described above is pretty painful.
Alignment by textbook series can be fairly painful until you have worked out your own unoffical editted edition of modern classics like the elementary math textbooks that require you to perform twenty sensless math activities in thirty minutes.
It can get worse. Core-soaked administrators may anticipate your roguish behavior, or may simply not trust you to behave yourself and stay within the lines. In that case, you may be subjected to the most painful alignment of all-- alignment by script, in which you are required to simply do exactly what is laid out in the pre-crafted modules. If you are in a scripted district, my condolences-- you work for dopes.
The Realest Alignment
All of these approaches bring us back to the bottom line. We've seen this movie before, under NCLB-- the ultimate alignment instrument for the district is the High Stakes Test. Advocates of the Core may swear that if you follow the standards, the test scores will take care of themselves. This is simply not true, and we all know it. Test prep was the order of the day under NCLB, and under the Core, the high stakes test will drive the curriculum bus once again. All other alignment is just doing the deck chair dance of doom.
The Dream
The ideal, as imagined by the Common Core Crowd looks something like this:
A group of fresh-scrubbed teachers gather in a room with a consultant (because, after all, they're only teachers and lack the expertise to do this work on their own). The CCSS expert teaches them how to "unpack" the standards, and over the course of many days, the teachers unpack a standard and decide how best to implement that standard in a well-constructed CCSS-aligned lesson. Depending on the consultant, they will probably also incorporate some features that are not actually in the Core, but which the consultant likes.
Once implemented, this new curriculum will totally revolutionize the way these teachers teach, and soon, awesome test scores will descend upon them like manna from heaven. Because once you align the to the standards, your students will automatically be prepared for the aligned standardized tests.
The Reality
Well, sadly, some peoples' reality will be a pursuit of the dream. If your administration has soaked too long in the special Common Core Coolade, the Dream will be your goal. Good luck with that.
For the earthbound districts, the process looks something more like this.
1) Spread out printouts on big desk. On one side, the printout of the Common Core National Standards (or the lightly edited version that your state is passing off as state-developed standards). On the other side, your pre-existing curriculum.
2) Go through curriculum, cross-referencing against standards. Look for standards that you can reasonably claim are being met by old lessons (e.g. "This paper I have them write about Great Expectations requires them to support their statements with evidence from the text. Check!") Check standards off of list.
3) Gather up standards you didn't meet. Make one last attempt to justify attaching them to pre-existing units. Rig up some new unit to meet those standards. Finish checking off list.
4) Go to PARCC/SBA/Whatever websites and gather up sample questions. Buy some sample question practice books. Schedule test prep units strategically through year.
5) Wait for High Stakes Test results. Analyze results to see what test prep units you need to beef up. Start collecting materials for next year. decide which units you can cut to make room for additional test prep. Do not forget to count your test prep units as part of your aligned curriculum.
Alternate approach
For the above steps 1-4, substitute the folowing:
1) Buy pre-aligned book and materials from dependable vendor like Pearson. Print out their alignment materials. Insert printout in alignment report for district.
Anyone Can Play
Slapping some Common Core numbers on the same old same old is a popular game these days. For instance, at a site called ELA Common Core Lesson Plans, we can find a deadly dull lesson on humor in literature that could easily be from ten, twenty, thirty years ago, except for the CCSS standards tags at the end. Or take this site that cheerfully plugs literature circles as a great CCSS technique, as if lit circles haven't been around almost as long as crop circles. Watch a video of a teacher using basic techniques that were actually well known long before CCSS. Or check out this lesson using song lyrics to teach figurative language, a technique used by every English teacher in the history of ever. But hey-- there are standards numbers attached to it, so it must be Common Core.
In fact, these lessons cover the other challenge of alignment-- what to do with the parts of lessons that actually contradict the Core approach. Teaching humor in texts? You're probably cheating by bringing in information from outside the four corners of the text, because humor's pretty hard to get in a context-free vaccuum. And we'd better hope that the fifth grade teacher working with the text about Harriet Tubman and slavery is not actually answering her students' questions about the origins of racism and slavery, because that would be contrary to the Core's love of Close Reading 2.0.
So painless alignment is not only about finding ways to line up things you already do with the Core-- it also involves ignoring the Core when it wants you to stop using tried and true successful practices you've used in the past.
Painful Alignment
There are of course more painful ways to align. Teachers who have been through a reality-based alignment in which CCSS has no major effect on their classroom may well wonder what the big fuss is. But the Dream described above is pretty painful.
Alignment by textbook series can be fairly painful until you have worked out your own unoffical editted edition of modern classics like the elementary math textbooks that require you to perform twenty sensless math activities in thirty minutes.
It can get worse. Core-soaked administrators may anticipate your roguish behavior, or may simply not trust you to behave yourself and stay within the lines. In that case, you may be subjected to the most painful alignment of all-- alignment by script, in which you are required to simply do exactly what is laid out in the pre-crafted modules. If you are in a scripted district, my condolences-- you work for dopes.
The Realest Alignment
All of these approaches bring us back to the bottom line. We've seen this movie before, under NCLB-- the ultimate alignment instrument for the district is the High Stakes Test. Advocates of the Core may swear that if you follow the standards, the test scores will take care of themselves. This is simply not true, and we all know it. Test prep was the order of the day under NCLB, and under the Core, the high stakes test will drive the curriculum bus once again. All other alignment is just doing the deck chair dance of doom.
Subscribe to:
Posts (Atom)