Thursday, August 6, 2015

TNTP: Why Does Professional Development Suck?

TNTP has released a new report-ish papery thing addressing the state of teacher professional development in the US. Their conclusion? It sucks.

To their credit, they're apparently unhappy with those results. I'm not sure they need to be quite so discouraged; as with most TNTP researchy products, this one has some problems. Still, if you think I'm going to stick up for the awesomeness of professional development, I do hope you have another think coming. But let's go ahead and take a quick look at The Mirage: Confronting the Hard Truth about Our Quest for Teacher Development.

Do we know how to help teachers get better? 

I confess that I find this particular reporty thing oddly fascinating, because at its root it is a slow-motion collision between a fundamental reformster flaw and reality. So what we're looking at is TNTP trying to jump their unicorns over an existential chasm while trying not to look directly into the abyss.

The intro is almost plaintive-- we were so sure that if we could just figure out what teachers should be doing and then train them to do it, students would start racking up the big test scores (I'm paraphrasing, but only a little). I want to feel bad for these guys, because right out of the box they are designing a whole study based on fundamental misunderstandings of how teaching works, what teachers do, and what student success looks like. Really, we've driven right past the unicorn farm to a special lab where scientists are trying to make kumquats produce better pork chops by developing techniques for making the number nine smell more like lilacs.

I'm not entirely sure how to address the issues here, but let's start by looking at what they think they've learned.

Districts are spending big bucks

Well, they can't screw this up-- it's just running the numbers, right? TNTP found that districts are spending astronomical funds on teacher development, including the district that spends more on PD than on transportation. Okay, then.

Despite the buck spendage, most teachers do not appear to improve from year to year

How would you even measure something like that? TNTP went with just looking at the districts' own ratings of teachers, and discovered that "the difference in performance between an average first-year teacher and an average fifth-year teacher was more than nine times the difference between an average fifth-year teacher and an average twentieth-year teacher." So, beginning teachers improve at a greater rate than veteran teachers. Is this news to anybody, anywhere, at all?

But TNTP is concerned, particularly because the veteran teachers "still have ample room to improve." They know this because they find that many teachers were less than "effective" in critical thinking. This is probably a good time to mention two things-- the reporty thing is based primarily on three unnamed districts, and the "effective" rating is one computed by TNTP, not the district's rating. It's all in the big fat appendix.

Those teacher ratings go back to 2011, and include plenty of special VAM sauce. So they are, as always, questionable. Nor does TNTP mention if Districts A, B and C use one of the many teacher evaluation systems that require the school to keep everyone on a bell curve, therefor insuring that few teachers ever make it into the top levels.

Nor does TNTP offer any justification that its measure of teacher improvement can in any way, shape or form be tied to the professional development inflicted on that particular staff. This is like saying, "Well, we planted these begonias out in the swamp and watered them every day, but they grew poorly, so clearly water does not help begonias grow."

TNTP kind of acknowledges that

We couldn't find any improvey link to PD

They looked at teachers who had improved and tried to pin down any one aspect of PD that was linked to improviness. There's a full page chart showing that when it comes to many traits, attitudes and experiences, improvers are are pretty much just like non-improvers. Yup-- incredibly, there doesn't appear to be a single one-size-fits-all approach to professional development that works for every single human teacher. Not even the ones that TNTP folks "believed to be most promising." Do these guys get out much? Have they met humans?

School systems are not helping teachers understand how to improve-- or even that they have room to improve.

TNTP's position here is not unexpected. In the reformster world, remember, parents and students and teachers have no idea how students are doing unless they have datafied test results to look at. So if course, in that same universe, teachers would not know anything about their own strengths and weaknesses unless the system told them. TNTP believes the system is not doing that.

As evidence they offer that old standard, beloved by education experts like Andrew Cuomo and Campbell Brown-- if 40% of the students got low test scores, then 40% of the teachers must be ineffective. This is reasoning that stuns with its dumbosity. Since my favorite baseball team lost 40% of their games last year, 40% of the players must be terrible. Since 40% of my flowers died in the garden last year, we must conclude that 40% of the days had bad growing conditions.

Further proof of teachers' universal lack of self-knowledge? 60% of teachers who were low-rated didn't think they were terrible teachers. Which is foolish, because we all know that teacher evaluation systems are perfect. But we're not done. Come with me now as TNTP takes a flying leap over Logic Gulch. Two-thirds of teachers surveyed didn't think their PD was useful, and only 40% thought the PD was a good use of their time. Can you guess why? No, no, you can't-- this low appreciation of the PD sessions is a product of "a pervasive culture of low expectations for teacher development and performance."

Sigh. You know what causes a culture of low expectations about professional development? A long history of crappy professional development. You know what that has to do with expectations about teacher performance? Absolutely nothing at all.


"We bombard teachers with help, but most of it is not helpful--"

Like much of the report, this sits right on the cusp of actual understanding. Who are "we" exactly, anyway, and why is PD bombarded, a word that perfectly elicits the image of things being dropped from above on helpless folks below. And why is the help not helpful? These would all be useful questions, and TNTP is not going to answer them.

Here's a whole paragraph's worth of climbing right up to the edge of the pond and refusing to drink:

In spite of this, the notion persists that we know how to help teachers improve and could achieve our goal of great teaching in far more classrooms if we just applied that knowledge more widely. It’s a hopeful and alluring vision, but our findings force us to conclude that it is a mirage. Like a mirage, it is not a hallucination but a refraction of reality: Growth is possible, but our goal of widespread teaching excellence is further out of reach than it seems. 

On the other hand, there are moments when it appears that TNTP really has learned something:

Teacher development appears to be a highly individualized process, one that has been dramatically oversimplified. The absence of common threads challenges us to confront the true nature of the problem—that as much as we wish we knew how to help all teachers improve, we do not.

Well, they're proposing some solutions here, so let's see how many clues they bring to that process.

So what do we do?

Redefine what it means to help a teacher improve

The new definition includes "measurable defined progress toward an ambitious standard," so we're back to the over-reliance on what can be measured. It's not that I think teacher development should live in a fuzzy land of fluffy clouds, but if you insist on concrete deliverables, you will end up with the kind of crap chosen because it's easily measurable, not because it's useful.

The new definition is also supposed to be about increasing teacher self-knowledge which, okay, maybe. But then we follow that with rewards and punishments-- I mean, "consequences." Which takes us back to the old idea that doing a good job in a classroom somehow has no intrinsic reward or feedback, which is just a meagre view of human beings.

Reevaluate current stuff that's out there

Sort through what's on the market and distinguish crap from gold. Because once you've labeled the crap as crap, the company that's making a living pushing it will certainly fold up and go away. Surely TNTP understands what most teachers who have ever sat through a PD session understand-- much if not most of this stuff is not being put out by people who think they can make teaching better, but by people who think they can make a living selling their particular program.

Reinvent how we support effective teaching at scale

Wow. That's all wrong. All wrong. Because "we" (again-- who is the we here) don't know how to identify effective teaching, let alone support it. And "at scale"? Why? Why why why WHY? The only reason to care about doing this "at scale" is so that it can be more effectively widely marketed, so we're right back to the wrong question-- the question of how to operate the PD business as a more efficient money-making business.

These guys actually got it right a few paragraphs ago-- teacher development is highly individual. Such individuality suggests that trying to operate at scale is a fools game, and likely to work directly against the goal of effective development.

But TNTP is thinking big. Under this item they would like to reconstruct the job of teaching, redesign schools, and reimagine how teachers are trained and certified. So, just redo everything.

My advice for free

If I were to address the issue of teacher development stuff, there are a few other things that might help.

Start with teachers

One striking chart in the papery thing shows that the highest-rated "I find this helpful" activity listed by teachers is informal collaboration.

The worst professional development (and probably the most common) is done to teachers, not for them or with them. You're lucky if you teach in a district where your preferences count, and even then, that may not count for much because your state government may have helpfully created rules about what things may be done for professional development.

The most useful professional development for me is that which addresses needs that I've identified for myself. Period. Some presenter who is a glorified salesperson (or in the case of PD provided by textbook companies, an actual salesperson) does not have my attention. Somebody sent by the state to tell me what the state has decided I need to do does not have my attention (Common Core test prep sessions, anybody).

Most professional development is like a restaurant where you don't get to see a menu, you don't get to pick your order, and the waitpersons don't even ask if you have food allergies.

Never mind measuring what you can't measure

TNTP's reporty thing acknowledges this, but sloughs it off because, gosh, if we did give teachers what they wanted, would they improve? This avoids the big question once again-- if I improve as a teacher, how will you know? Answer-- particularly if I'm teaching high school history or some other completely untested subject-- is probably not. If I have a good principal and an evaluation system that doesn't depend on VAMmy foolishness, we may have a shot. But hey-- the feds have mandated that everybody suffers through VAMmy foolishness, so probably not.

Evaluate the PD directly

TNTP used a long convoluted chain of possible cause and improbable effect to evaluate development. We could do better just by handing every teacher in the session a single question: was the session useful, middling, or a waste of time? Granted, some of my colleagues will frustrate me by "being nice" on an instrument that blunt, but still-- is there anything you need to know about the development that the answer to that question won't tell you?


  1. A lot of our PD days last year took us out of the classroom all day long, which of course, really improves student learning. :(

  2. Great post, as usual. This will reverberate with every veteran teacher who has spent literally months of their life sitting through crappy PD that they know in the first minute will not help them be a better teacher in any way. I particularly liked two things you wrote:

    (1) "we've driven right past the unicorn farm to a special lab where scientists are trying to make kumquats produce better pork chops by developing techniques for making the number nine smell more like lilacs." I just like this because of the word picture. I love how you often take a concept that could be described adequately in edujargon and make it really hit home with analogous humor, and this is one of your best yet.

    (2) "Surely TNTP understands what most teachers who have ever sat through a PD session understand-- much if not most of this stuff is not being put out by people who think they can make teaching better, but by people who think they can make a living selling their particular program." As soon as I saw the report, I thought the same exact thing. I am a very efficient and effective teacher and I have worked really hard to develop some systems that work well for me. Many times over the years I have considered taking my show on the road and pitching PD. Never once was my motivation to make teachers everywhere better at their jobs. Every time, I was envisioning how much each book would sell for and how much I could charge for speaking engagements. In the end, it was never the lack of potential value of my "product" to teachers that caused me to not go commercial, it was the unlikeliness of making a good enough profit.

  3. Would be very interested to hear comments on the, "Capturing Kids Hearts", Flipp Flippen group. Also interested in comments regarding Doug Lemov, (Teach Like a Champion).

    1. From what I've read, Flipp Flippen is a motivational speaker/businessman, no educational background. His schtick that motivated students learn better is nothing new. A district paid tons of money for his program and dumped it two years later.

      To me, Lemov's book sounds like it might make a good handbook for new teachers, but there's nothing much in it veteran teachers don't already know and use.

    2. For TNTP to put out a statement analyzing professional development is a ploy to disguise the fact that they implement very poor teacher development. Look into these organizations and what they are upholding. He who barks is the first to see.

  4. My district is embracing the "Ed Camp" practice. To oversimplify: teachers lead collaborative conversations with other teachers about subjects that they CHOOSE -- all FREE. SOOOO much better than the "alleged, self-appointed expert is hired to tell teachers what to do -- and in the telling, reveal that they themselves have no idea whatsoever how to do the teacher's job and would not last two months....or two weeks."

  5. "It’s a hopeful and alluring vision, but our findings force us to conclude that it is a mirage. Like a mirage, it is not a hallucination but a refraction of reality". Wow. What were they smoking?

    Another delightful, insightful analysis. My favorite analogy: TNTP "trying to jump their unicorns over an existential chasm while trying not to look directly into the abyss."

    It was interesting that the teachers in the three districts thought that informal collaboration was more useful to them than any other kind of PD. The "improvers" felt that their own independent efforts were second most helpful; "non-improvers" didn't seem quite as confident about their own efforts. They also all felt that PD was not specific enough for their own needs, and complained that there was no follow-up.

    Even though the authors admit that teaching is very complex, and teacher development is highly individualized and they don't know what to do to improve it, they seem to think that what is most needed is having everyone on the same page about the "urgency" of getting those scores up.

    Besides the three unnamed school districts, they also studied an unnamed mid-sized charter school. Although they still couldn't say why some teachers "improved" and some didn't, their "findings" were that charter teachers improved more than district teachers. It isn't real clear how many of the teachers had been there very long, and if most of their teachers are TFA people with only five weeks of teacher preparation, no wonder they had lots of room for improvement and 81% of teachers "agreed that they have weakness in their instruction."

    I never got very much out of workshops, even when they were specific to my subject or on good, solid, cognitive science. I had to read the research myself in order to understand why and how it worked before I could apply it.

  6. I just found your analysis of the 'papery thing'called the Mirage. I missed it in August. Your reading of the Mirage is perfect. I came upon it because somebody was tweeting out the number $8B for professional development and I asked where she got that number. When I got to the Blurry report I reacted much the same way you did. I did notice that this report is written by people who have about a years worth of classroom experience between the two of them, and neither of them appear to have any education credits. I will be the first to point out that teacher training programs need to be significantly improved, but I think actual experience (more than a year or two, at least) in a classroom should be a prerequisite for designing the new professional preparation programs.

    As you pointed out, Peter, not identifying the particular districts from which they derived their data was a huge problem. They seem to think that all classrooms all over the country are the same with students who are all pretty much the same, too. They also fail to differentiate between any of the various teaching roles that are part of K12 district. There a some similarities in teaching elementary reading and AP English, but ...
    Thanks for a good post on the papery thingy by the TNTP

  7. Flip is an idiot. His program is a scam. Taxpayers are being screwed.