NCTQ: Terrible Teacher Prep and Headline Research

The National Council on Teacher Quality is one of the great mysteries of the education biz. They have no particular credentials and are truly the laziest "researchers" on the planet, but I think I may have cracked the code. Let me show you their latest piece of "research," and then we can talk about how they really work.

Their new report-- "Learning about Learning: What Every New Teacher Needs To Know" (which is a curious title-- do other teachers NOT need to know these things?)-- is yet another NCTQ indictment of current teacher education programs. The broad stroke of their finding is that teacher education programs are not teaching the proven strategies that work in education.

That's the broad stroke. As always with NCTQ, the devil is in the details. After all, that sounds like a huge research undertaking. First, you would have to identify teaching strategies that are clearly and widely supported by all manner of research. Then you would have to carefully examine a whooooooole lot of teacher education programs-- college visits, professor and student interviews, sit in classes, extensive study of syllabi-- it would be a huge undertaking.

Or you could just flip through a bunch of educational methods textbooks.

What Every Teacher Needs To Know

First, NCTQ had to select those methods that "every new teacher needs to know." Here's the methodology for that piece of research-based heavy lifting:

In Organizing Instruction and Study to Improve Student Learning: A Practice Guide, the Institute of Education Sciences (IES), the research arm of the U.S. Department of Education, identified proven practices that promote learning for all students, regardless of grade or subject, and that are especially potent with struggling students. Six practices stand out for the research behind them. There is little debate among scholars about the effectiveness of these six strategies.

Here are a few things to know about Organizing Instruction and Study To Improve Student Learning.

It was published in September of 2007. It was produced under a USED- IES contract with Optimal Solutions Group, LLC, a policy data-analysis business. It opens with a disclaimer that includes this:

The opinions and positions expressed in this practice guide are the authors’ and do not necessarily represent the opinions and positions of the Institute of Education Sciences or the U.S. Department of Education. 

The IES paper does, in fact, appear to be a group of researchers checking to see how much research basis there is for seven ideas that they think will help teaching subjects "that demand a great deal of content learning, including social studies, science, and mathematics." So, not actually "all subjects and grades" as NCTQ says. And they are based around a memory-based model of education.

More importantly, the IES paper rates the seven approaches according to strength of the research to support them. Four of the seven are rated "moderate," two are rated "low," and the seventh is rated "strong".

What Are The Must-Know Techniques?

That depends on whether you look at the original IES paper or the NCTQ "research." NCTQ drops one IES technique-- teaching students how to use time. And they convert "use quizzing to promote learning" into "assessing to boost retention." Either way, the IES paper rates the scientific basis for this technique low, with little research beyond reading instruction experiments with college students. So that whole "there is little debate" and "research-based" bullshit is, in fact, bullshit.

That is the only low-rated technique that made the list. The strong technique is "ask deep probing questions."

The other four are all moderately-rated, meaning that there is some research basis for them (back in 2007), but it's not overwhelming. Those four are "pairing graphics with words," "linking abstract concepts with concrete representations," "alternating solved problems with problems to solve," and "spread out practice over time."

These are not bad techniques, useless techniques, unwelcome techniques-- but is NCTQ suggesting that of all the educational techniques in the world, these six are the essential ones? Well, they call them "the fundamental knowledge they need to make learning 'stick.'"

NCTQ refers to these six techniques as "the field’s bedrock research as identified by IES," which is a lie. 

And they reach this scary conclusion: "If teacher candidates aren’t being taught the research-proven and workable practices that help students learn new content, they will flounder when they try to make learning last." So how do they know that teacher candidates aren't being taught these techniques. 

How Are Ed Schools Failing?

They looked in a bunch of textbooks. They looked at 48 college programs (at 28 different colleges). They selected books "assigned in educational psychology, general methods and secondary subject-specific methods courses." And because NCTQ really is the laziest research group on the planet, there's this:

We note that textbooks unique to subject-specific elementary methods courses were not reviewed in depth. What examination we did of these textbooks indicated that had we reviewed them, none would have received credit for covering the strategies.

We glanced at some books and they looked like they were fer suresies losers, so we just skipped those.

Ultimately they settled on 48 books. You can see the breakdown here. A team of four scanned through the books for signs of references to the techniques with blah blah blah I'm not sure they didn't just use [Control + F] here, but they have a complex-sounding technique for deciding if the technique was fully and accurately presented in the textbook. You can look through their methodology if you like, but the bottom line is that references to the techniques had to be absolutely on the nose.

They also claim to have done some study of programs, going back to their file folder of course syllabi, because that totally tells you exactly what goes on in classes. As with books, the requirement was to learn the technique as a general truth. It looks as if, for instance, learning all about teaching maths with manipulatives does not count as "linking abstract ideas to concrete representations" because that's only for a math class, and dopey teachers might not understand that linking abstract and concrete could be used in other classes.

Seriously?

There are so many other questions to ask, such as, "Do teachers actually use their methods and general ed psych books?" Is the main pathway for teaching prospective teachers through traditional lecture and textbooks? That would be an interesting question to study, but NCTQ does not go there. Heck, asking any teacher in any classroom, "Can you put your hands on your college methods textbook right now?" would be entertaining. But as always, NCTQ has more important things to do than try to find out any useful truths.

Who are these people?

NCTQ has appointed themselves the arbiters of teacher quality because reasons. Would you like to guess how many career teachers are actually involved in running NCTQ? Did you answer zero? Good for you. Would you like to guess how many former TFA temps are running the group? Did you guess many? Good for you again.

What is their research specialty?

NCTQ is the group that once declared that college teacher programs are too easy, and their research was (and I swear I am not making this up) to look through college commencement programs.

NCTQ is the group that cranked out a big report on teacher evaluation whose main point was, "It must not be right yet, because not enough teachers are failing."

NCTQ creates the college rankings list published every year by US News leading to critiques of NCTQ's crappy methodology here and here and here, to link to just a few. NCTQ's method here again focuses on syllabi and course listings, which, as one college critic noted, "is like a restaurant reviewer deciding on the quality of a restaurant based on its menu alone, without ever tasting the food." That college should count its blessings; NCTQ has been known to "rate" colleges without any direct contact at all.

NCTQ's history has been well-chronicled by both Mercedes Schneider and Diane Ravitch. It's worth remembering that She Who Must Not Be Named, the failed DC chancellor and quite possibly the least serious person to ever screw around with education policy, was also a part of NCTQ.

NCTQ depends on the reluctance of people to read past the lede. For this piece, for instance, anybody who bothered to go read the old IES paper that supposedly establishes these as "bedrock" techniques would see that the IES does no such thing. Anyone who read into the NCTQ "research" on teacher program difficulty would see it was based on reading commencement programs. The college president I spoke to was so very frustrated because anybody who walked onto her campus could see that the program NCTQ gave a low ranking was a program that did not actually exist.

But NCTQ specializes in headline research-- generate an eye-catching pro-reform headline and hope that if you follow it with a bunch of words, folks will just say, "Well, there's a lot of words there, so they must have a real research basis for what they're saying."

There are reform advocates who are, I believe, sincere and intellectually honest. But for NCTQ to, for instance, transcribe the 2007 IES report of the quizzing technique as a "bedrock" of learning when IES clearly ranks it as having little real research to back it up-- that requires either unbelievable stupidity, incredible laziness, or just flat-out lying. NCTQ might very well be the least serious outfit in the education biz, and yet they still draw press attention.

But if you run across references to this report (which is part of a broader assault on teacher education programs), rest assured that it's rubbish, and don't hesitate to encourage people to ignore it. Never has a group so justly deserved to be completely ignored.