First, they can't work like a private sector bonus system. In the business world, bonuses and incentives are based on a clear idea of success. Whether the definition of success at Widgetcorps this year is "We made a bunch more money" or "We increased the value of Widgetcorps stock this year" or even "We managed to strip a pile of dollars out of Widgetcorps this year," the end result is a pile of "extra" money (the end result, depending on the definition of success, may be a less healthy company, but let's set that aside for now). The bonus deal between Widgetcorps and its bonus-eligible employees is simple--if there's a pile of extra money here at the end of the year, you get to have some of it.
Public schools, on the other hand, don't make money, and school district success is hugely hard to define. More high school graduates? The football team won a championship? The school musical was really good this year? More students really got into Shakespeare? Larger number of students earned welding certification? The school's culture and atmosphere was so healthy that students really love attending? All of these might count as success, but none of them result in a school district ending the year with "extra" money to be split up for bonuses.
Instead, the pile of bonus money has to be pre-budgeted, and then teachers get to compete with each other for a share of a fixed pie. This may not establish the kind of in-house atmosphere you want for your school ("No, colleague, you may not see my materials for how I teach fractions--my child needs braces this year!") We already know this is bad news; this kind of internal competition is arguably part of what destroyed Sears.
"But, see, I think teachers should be paid more, but only the good ones. So I want merit pay so that we only have to give good teachers raises." This is an understandable impulse. The problem is that we have no universal agreement on who the great ones are or how to reliably measure their greatness. Settling that issue would involve a long and complicated discussion that we still haven't had. Instead, policy makers mostly just said, "Use scores from the Big Standardized Test."
Again, we could have a long conversation about the merits and demerits of this approach, but to understand how basing "merit" on standardized test scores affects teacher behavior, all we have to grasp is this-- from a teacher standpoint, judging teacher merit based on test scores is only slightly more reliable that having a horned toad toss dice under a full moon. Basing merit pay on test results divides the teaching staff into two groups who will hear two different messages. The non-English, non-math teachers hear, "You have no control over whether you get merit pay or not." This is not hyperbole; in many regions, teachers have been judged by the test results in subjects they don't teach from students they've never met. English and math teachers also hear the message, "You will be judged on how well you teach to the test." Merit pay based on test scores does not incentivize better teaching.
"Well, what if I want to use bonuses to push teachers in directions that help the district, such a teaching at hard to staff schools?" That was a big part of the goal for Denver, and it hasn't worked. Again, teachers feel at the mercy of forces outside their control (one batch of teachers lost their incentive pay for teaching in a high-poverty neighborhood when new luxury townhouses were built nearby). And a one-time pay incentive does not outweigh a bad boss or building culture.
A teacher bonus system is prone to other problems. To fund that pile of bonus money, districts often lower base pay for teachers. This is not just bad financial news if the horned toad who's tossing dice for your bonus comes up snake eyes; the base pay is what determines your pension, and what you write on applications for loans and mortgages.
And at its heart, a merit pay system is insulting. It imagines a world of teachers who sit at their desk thinking, "I have the perfect lesson for effectively teaching pronouns right over there in my filing cabinet--but I'm not going to get it out until someone offers me a bonus."
Here is a thing actual teachers have all said, more or less, at one time or another: "Today was awesome. I was on fire. The kids were right here, taking it all in. I could see lights going on all over the room as they got what we were doing. I taught the crap out of that unit. I am ready to get back in there tomorrow."
Actual teachers have also said, more or less, this: "Today was a thousand hours long. They wouldn't focus. They wouldn't stop hollering at each other. I couldn't find my rhythm. I sucked. It was awful. If I thought this was what it was always going to be, I'd think hard about quitting."
Here's what no actual teachers have ever said in any way, shape, or form. "Today was awful and excruciating. I was terrible. The lesson was awful. Mind you, I could have had a great day. I could have been awesome in there instead of suffering endlessly, but I won't do it unless someone offers me a bonus."
I would not go so far as to say that teaching well is its own reward, because teachers need food and shelter, too. But teaching badly is definitely its own punishment. No merit pay system--particularly not one based on small rewards for student test scores--is likely to exert more power over teachers than the immediate results in the classroom.
Merit pay for teachers has been tried many times over the last several decades. It never works. It doesn't raise test scores, or improve teacher attraction and retention. It doesn't improve morale. A district can garner better results by spending the money to raise base wages, or to decrease class sizes and provide other supports for classroom teachers.