NCTQ New Report on Teacher Evaluation

It's a big report, over a hundred pages, and I've read it so you don't have to. But that doesn't mean you don't have your work cut out for you here on this blog. Let's get going.

Who are these people?

The National Council on Teacher Quality's continued presence in the education world is one of the great mysteries of the reformster era (or maybe just one of the great con jobs). This "national council" includes a staff composed almost exclusively of former TFA folks and professional bureaucrats and a board of directors that contains no teachers.

Let me say that again-- this group that has declared itself the arbiter of teacher quality for the country has no career teachers in positions of authority. None.

They have been an excellent tool for reformsters, which may be why their funders list is a who's who of reformy money (Gates, Broad, Walton, Joyce, and even Anonymous). Like other heavy-hitters (or at least heavy cash-checkers) of the "non-partisan research and policy organization" world, they specialize in putting a glossy figleaf of research study paper over the big ugly naked truth of reformster advocacy.

Their particular brand is about assaulting the teaching profession with a concern trolling spin. From their mission statement:

We recognize that it is not teachers who bear responsibility for their profession's many challenges, but the institutions with the greatest authority and influence over teachers. To that end we work to achieve fundamental changes in the policy and practices of teacher preparation programs, school districts, state governments, and teachers unions. 

In other words, teachers suck, but it's not their fault, poor dears, because they are helpless, powerless tools of Important Forces. Oddly enough, I have never come across anything from NCTQ suggesting that empowering teachers might be a useful solution.

Let me be up front about NCTQ

There are people and organizations in the reformster world that can, I believe, be taken seriously. I may disagree with almost everything they conclude, but they are sincere, thoughtful, and at least to some degree intellectually honest. They raise questions that are worth wrestling with, and they challenge those of us who support public schools in ways that are good for us. I have a whole list of people with whom I disagree, but whom I'm happy to read or talk to because they are serious people who deserve to be taken seriously.

NCTQ is not on that list.

NCTQ once issued a big report declaring that college teacher education programs were much easier than other programs. Their research-- and I swear I am not making this up-- was to look through a bunch of college commencement programs and course syllabi.

This may actually be better than their signature report ranking the quality of various teacher education programs, a program infamous in my neck of the woods for rating a college on a program that didn't actually exist. This list is published in US News (motto: "Listicles make better click bait than new stories"), so it makes some noise, leading to critiques of NCTQ's crappy methodology here and here and here, to link to just a few. NCTQ's method here again focuses on syllabi and course listings, which, as one college critic noted, "is like a restaurant reviewer deciding on the quality of a restaurant based on its menu alone, without ever tasting the food." That college should count its blessings; NCTQ has been known to "rate" colleges without any direct contact at all.

The indispensable Mercedes Schneider has torn NCTQ apart at great length; if you really want to know more, you can start here. Or check out Diane Ravitch's NCTQ history lesson. That will, among other things, remind you that She Who Must Not Be Named, the failed DC chancellor and quite possibly the least serious person to ever screw around with education policy, was also a part of NCTQ.

Bottom line. Everything I know about NCTQ makes me inclined to expect that any report they put out is intellectually dishonest crap designed to further an agenda of braking down teaching as a profession.

So we are ever going to look at this new thing? 

Yes, sure. I just wanted to make sure your expectations were set low enough.

State of the States 2015: Evaluating Teaching, Leading and Learning

That's the report, and here's your first spoiler alert: the report isn't really going to look at evaluating learning at all.

In fact, it will help to understand the report if you do not jump to the mistaken conclusion that NCTQ is asking, "Have we found effective ways to do these things?" Because the question NCTQ is really asking is, "How many of our preferred policies have we gotten people to implement?" At no point will they ever, ever ask, "Hey, are any of our preferred policies actually any good?"

If you understand the questions we're really asking (and not asking), the report makes a lot more sense.

Key Findings about Teacher Evaluation

NCTQ is happy to report that more states are falling in line. Almost all include student results in teacher evals, and some include those results extra hard. This is super-not-surprising, as such linkage was mandated by Race To The Top and the waivers that states pretty much had to try for. And we're super-happy that twenty-three states now require use of student test scores evidence of teacher results to decide teacher tenure.

Oh, but there is sad news, too. A "troubling" pattern.

The critique of old evaluation systems was that the performance of 99 percent of teachers was rated satisfactory, regardless of student achievement. Some policymakers and reformers have naively assumed that because states and districts have adopted new evaluations, evaluation results will inevitably look much different. But that assumption continues to be proven incorrect. We think there are several factors contributing to the lack of differentiation of performance:

Dammit!! The new evaluation systems were supposed to root out the terrible teachers in schools ("look much different" means "look more faily"), because if ten percent of students fail the Big Standardized Test, that must mean that ten percent of the teachers stink. It's common sense. Like if a football team loses ten percent of its games, ten percent of its players must be bad. Or if ten percent of the patients in a hospital die, ten percent of the doctors must be terrible. Come on, people-- it's just common sense.

So what do they think screwed things up? Well, lots of states only do one observation a year. Okay-- so is there a correlation between number of observations and number of "ineffective" ratings? Cause that seems like an easy thing to check, unless you were the laziest research group on the planet. Don't have that data? Okay then.

The other possible culprits are SLOs, which NCTQ suggests might be a disorderly vague mess. Well, I can't really argue with that conclusion, though its effect on evaluations is unclear, other than I'd bet lots of principals are reluctant to give lousy teacher ratings based on a technique less reliable than throwing dice through the entrails of a brown snake under a full moon.

Also, NCTQ knows that implementing both new "college and career standards" and new test-based teacher evaluation systems created an "unfortunate collision." Yeah, implementing new amateur hour standards along with crappy tests to be used in junk science evaluation schemes, and doing it all at once-- that's a thing that just kind of happened and wasn't at all the result of deliberate poorly-thought out plans of the educational amateurs running the reformy show. Honest to goodness, it will be a truly amazing day if I ever find a reformster policymaker actually say, "Yeah, we did that wrong. We screwed up. We made a bad choice and we should have listened to the ten gazillion education professionals telling us to choose better." But today is not that day.

NCTQ does think that student surveys might improve the whole evaluation thing, and boy, nobody can imagine downsides to that approach. But they are thinking basically anything that makes observations less of a piece of the evaluation, because they're pretty sure it's those damn principals messing up the system and making teachers look better than they are.

Any way, states should be "sensitive," but should not "indulge critics." And if you're looking for the part of the report that considers whether or not any of these teacher evaluation policies is valid, reliable, useful or indicative of actual teacher effectiveness-- well, that's just not going to happen.

Meanwhile, that bad old opt out movement has been all about protecting teachers from evaluations, and evaluations are much better now, so knock it off. 

Key Findings about Principal Evaluation 

Folks have figured out that we have to hold principals' feet to the fire, but states have found a wide variety of ways to do that, some of which are so sketchy that nobody even knows whose responsibility the principal eval is.

But in big bold letters, comes the pull quote: "There is insufficient focus on meaningful consequences for ineffective school leaders." So whatever system we come up with for evaluating principals, it really needs to punish people harder.


Connecting the Dots

What NCTQ would like to see more than anything else in the whole wide world is a teacher evaluation system driven by test scores that in turn drives everything else. Hiring, firing, promotions, tenure, revoking tenure, pay level-- they would like to see all of those tied to the teacher evaluation.

NCTQ credits Delaware, Florida and Louisiana with "connecting the dots" best of all. The language used for this baloney is itself baloney-- it's like the baloney you make out of the leftover scraps of baloney. But it's worth seeing, because it's language that keeps reappearing, including in places like, say, TeachStrong.

While there has been some good progress on connecting the dots in the states, unless pay scales change, evaluation is only going to be a feedback tool when it could be so much more. Too few states are willing to take on the issue of teacher pay and lift the teaching profession by rewarding excellence.

Sigh. Yes, teachers are currently holding back their most excellent selves, but if we paid them more, they'd be motivated. Because teaching really attracts people motivated by money. Of course, that's not really the idea behind various forms of merit pay. The real idea is a form of demerit pay cuts-- let's only give good pay to only the people we've decided deserve it.

Lessons for the Future

NCTQ has a whole decade of policy-tracking under its belt, so they've reached some conclusions.

States should not go too far with teacher effectiveness policy. NCTQ actually calls out North Carolina for screwing up the teacher evaluation system and trashing pay and offering ridiculous bonus pay and trying to kill tenure and just generally being a giant jerk to all teachers. While I applaud them for noticing that North Carolina has done nobody any favors by trying to become the most inhospitable teaching environment in the country, I feel it's only fair to point out that North Carolina hasn't done anything that directly contradicts NCTQ's policy recommendations. They've just done it in an unsubtle and poorly PRed manner.

Principal and teacher evals need to be lined up.

It's important to focus on the positive and not let teachers see the evaluation process as "an ominous enterprise aimed at punishing teacher." So I guess back a few pages when NCTQ was saying it was such a huge disappointment that teacher eval systems were still finding mostly good teachers, or a few pages after that when they were saying how all employment decisions should be tied to evaluations-- those were somehow NOT talking about how evaluation should be used to punish teachers? Definite mixed message problem here.

Don't forget what this is all about. The children. We're doing all this for the children. Not that we've done a lick of study to see if our favorite policies actually help the children in any meaningful way.

Finally, "incentives" are better than "force." Bribes are superior to beatings. Sigh. Okay, let's link to Daniel Pink's "Drive" one more time. 

Finally 

We get page after page of state by state summary chart showing how well each state is doing at linking teacher evaluation to every aspect of teacher professional existence. You'll have to look your own page up. Look, I can't do everything for you.

There are also some appendices of other fun things that I'm also not going to summarize for you.

What's missing?

The report includes not a word about how we might know that any of the recommended policies actually works. We are clear that the be-all and end-all is to raise student test scores. Any proof that higher test scores are indicative of anything other than scoring higher? And as we move to teacher evaluation systems, is there any proof that, say, linking tenure to test scores improves test scores or anything that are actually related to a good education?

No. So the report is left with a basic stance of, "Here are some things everybody should be doing because we think they are good ideas, though none of us have ever been public school teachers, and none of us have any real experience in public education. But you should do these things, and if you do, education in your state will be better in ways that we can't really support or specify." And it took over 100 pages to say that. But this is NCTQ, so some bunch of media dopes are going to report on this as if it is real research from reputable experts who know what the hell they're talking about. What a world.