Sunday, February 7, 2016

NCTQ: Terrible Teacher Prep and Headline Research

The National Council on Teacher Quality is one of the great mysteries of the education biz. They have no particular credentials and are truly the laziest "researchers" on the planet, but I think I may have cracked the code. Let me show you their latest piece of "research," and then we can talk about how they really work.

Their new report-- "Learning about Learning: What Every New Teacher Needs To Know" (which is a curious title-- do other teachers NOT need to know these things?)-- is yet another NCTQ indictment of current teacher education programs. The broad stroke of their finding is that teacher education programs are not teaching the proven strategies that work in education.

That's the broad stroke. As always with NCTQ, the devil is in the details. After all, that sounds like a huge research undertaking. First, you would have to identify teaching strategies that are clearly and widely supported by all manner of research. Then you would have to carefully examine a whooooooole lot of teacher education programs-- college visits, professor and student interviews, sit in classes, extensive study of syllabi-- it would be a huge undertaking.

Or you could just flip through a bunch of educational methods textbooks.

What Every Teacher Needs To Know

First, NCTQ had to select those methods that "every new teacher needs to know." Here's the methodology for that piece of research-based heavy lifting:

In Organizing Instruction and Study to Improve Student Learning: A Practice Guide, the Institute of Education Sciences (IES), the research arm of the U.S. Department of Education, identified proven practices that promote learning for all students, regardless of grade or subject, and that are especially potent with struggling students. Six practices stand out for the research behind them. There is little debate among scholars about the effectiveness of these six strategies.

Here are a few things to know about Organizing Instruction and Study To Improve Student Learning.

It was published in September of 2007. It was produced under a USED- IES contract with Optimal Solutions Group, LLC, a policy data-analysis business. It opens with a disclaimer that includes this:

The opinions and positions expressed in this practice guide are the authors’ and do not necessarily represent the opinions and positions of the Institute of Education Sciences or the U.S. Department of Education. 

The IES paper does, in fact, appear to be a group of researchers checking to see how much research basis there is for seven ideas that they think will help teaching subjects "that demand a great deal of content learning, including social studies, science, and mathematics." So, not actually "all subjects and grades" as NCTQ says. And they are based around a memory-based model of education.

More importantly, the IES paper rates the seven approaches according to strength of the research to support them. Four of the seven are rated "moderate," two are rated "low," and the seventh is rated "strong".

What Are The Must-Know Techniques?

That depends on whether you look at the original IES paper or the NCTQ "research." NCTQ drops one IES technique-- teaching students how to use time. And they convert "use quizzing to promote learning" into "assessing to boost retention." Either way, the IES paper rates the scientific basis for this technique low, with little research beyond reading instruction experiments with college students. So that whole "there is little debate" and "research-based" bullshit is, in fact, bullshit.

That is the only low-rated technique that made the list. The strong technique is "ask deep probing questions."

The other four are all moderately-rated, meaning that there is some research basis for them (back in 2007), but it's not overwhelming. Those four are "pairing graphics with words," "linking abstract concepts with concrete representations," "alternating solved problems with problems to solve," and "spread out practice over time."

These are not bad techniques, useless techniques, unwelcome techniques-- but is NCTQ suggesting that of all the educational techniques in the world, these six are the essential ones? Well, they call them "the fundamental knowledge they need to make learning 'stick.'"

NCTQ refers to these six techniques as "the field’s bedrock research as identified by IES," which is a lie. 

And they reach this scary conclusion: "If teacher candidates aren’t being taught the research-proven and workable practices that help students learn new content, they will flounder when they try to make learning last." So how do they know that teacher candidates aren't being taught these techniques. 

How Are Ed Schools Failing?

They looked in a bunch of textbooks. They looked at 48 college programs (at 28 different colleges). They selected books "assigned in educational psychology, general methods and secondary subject-specific methods courses." And because NCTQ really is the laziest research group on the planet, there's this:

We note that textbooks unique to subject-specific elementary methods courses were not reviewed in depth. What examination we did of these textbooks indicated that had we reviewed them, none would have received credit for covering the strategies.

We glanced at some books and they looked like they were fer suresies losers, so we just skipped those.

Ultimately they settled on 48 books. You can see the breakdown here. A team of four scanned through the books for signs of references to the techniques with blah blah blah I'm not sure they didn't just use [Control + F] here, but they have a complex-sounding technique for deciding if the technique was fully and accurately presented in the textbook. You can look through their methodology if you like, but the bottom line is that references to the techniques had to be absolutely on the nose.

They also claim to have done some study of programs, going back to their file folder of course syllabi, because that totally tells you exactly what goes on in classes. As with books, the requirement was to learn the technique as a general truth. It looks as if, for instance, learning all about teaching maths with manipulatives does not count as "linking abstract ideas to concrete representations" because that's only for a math class, and dopey teachers might not understand that linking abstract and concrete could be used in other classes.

Seriously?

There are so many other questions to ask, such as, "Do teachers actually use their methods and general ed psych books?" Is the main pathway for teaching prospective teachers through traditional lecture and textbooks? That would be an interesting question to study, but NCTQ does not go there. Heck, asking any teacher in any classroom, "Can you put your hands on your college methods textbook right now?" would be entertaining. But as always, NCTQ has more important things to do than try to find out any useful truths.

Who are these people?

NCTQ has appointed themselves the arbiters of teacher quality because reasons. Would you like to guess how many career teachers are actually involved in running NCTQ? Did you answer zero? Good for you. Would you like to guess how many former TFA temps are running the group? Did you guess many? Good for you again.

What is their research specialty?

NCTQ is the group that once declared that college teacher programs are too easy, and their research was (and I swear I am not making this up) to look through college commencement programs.

NCTQ is the group that cranked out a big report on teacher evaluation whose main point was, "It must not be right yet, because not enough teachers are failing."

NCTQ creates the college rankings list published every year by US News leading to critiques of NCTQ's crappy methodology here and here and here, to link to just a few. NCTQ's method here again focuses on syllabi and course listings, which, as one college critic noted, "is like a restaurant reviewer deciding on the quality of a restaurant based on its menu alone, without ever tasting the food." That college should count its blessings; NCTQ has been known to "rate" colleges without any direct contact at all.

NCTQ's history has been well-chronicled by both Mercedes Schneider and Diane Ravitch. It's worth remembering that She Who Must Not Be Named, the failed DC chancellor and quite possibly the least serious person to ever screw around with education policy, was also a part of NCTQ.

NCTQ depends on the reluctance of people to read past the lede. For this piece, for instance, anybody who bothered to go read the old IES paper that supposedly establishes these as "bedrock" techniques would see that the IES does no such thing. Anyone who read into the NCTQ "research" on teacher program difficulty would see it was based on reading commencement programs. The college president I spoke to was so very frustrated because anybody who walked onto her campus could see that the program NCTQ gave a low ranking was a program that did not actually exist.

But NCTQ specializes in headline research-- generate an eye-catching pro-reform headline and hope that if you follow it with a bunch of words, folks will just say, "Well, there's a lot of words there, so they must have a real research basis for what they're saying."

There are reform advocates who are, I believe, sincere and intellectually honest. But for NCTQ to, for instance, transcribe the 2007 IES report of the quizzing technique as a "bedrock" of learning when IES clearly ranks it as having little real research to back it up-- that requires either unbelievable stupidity, incredible laziness, or just flat-out lying. NCTQ might very well be the least serious outfit in the education biz, and yet they still draw press attention.

But if you run across references to this report (which is part of a broader assault on teacher education programs), rest assured that it's rubbish, and don't hesitate to encourage people to ignore it. Never has a group so justly deserved to be completely ignored.

5 comments:

  1. What is a deep probing question? In the history of Western thought, Socrates is frequently held up as a model of a great teacher, and he asks a lot of questions. However, he rarely asks a single question that is, by itself, deep and probing. Many of his questions are quite simple, but it a series of these simple questions that leads his interlocutor (and the reader of Plato's dialogues) to philosophical depth.

    ReplyDelete
  2. CONTINUED FROM ABOVE:

    ———————————————
    WHERE DO WE GO FROM HERE? article:

    “A lack of funding over four decades has left our educational system in shambles. We have schools that are legally unsafe, children do not have enough materials, books or workbooks to help them achieve the best that they can be.

    “In Seattle the students get ‘Art on a cart’ because of a lack of funding in many schools. Schools have had to shorten their days and increase class sizes due to a lack of funding. If we financially supported our educational system to the degree that we do our military/industrial complex or to the extent that we bailed out several corporations or subsidize oil companies, we would not be where we are today.

    “To put the emphasis on teachers as the sole problem in our educational system is naive at best.”
    ———————————————

    DORA:

    “And one last point. Everything that I wrote in this post was true, with enough links to each fact to validate all of my statements.”
    ————————————————————-
    ————————————————————-

    That’s the end of the dialogue, from either of them.

    ReplyDelete
  3. PART ONE:

    Sometimes the best and most revealing information comes from the COMMENTS section in response to articles.

    An example of this is when this NCTQ functionary “Emily” gets into it with Seattle parent “Dora” in the COMMENTS section of Dora’s article — posted on a Seattle parents’ blog called SEATTLE SCHOOLS.”

    https://seattleducation2010.wordpress.com/2011/01/23/kate-walsh-nctq-and-us-news-world-report/

    In her article, Dora eviscerates NCTQ and its motives and tactics, and by extension, the motives and tactics of its funders — the usual suspects, Broad, Gates, Walton, etc. In particular, Dora also calls out NCTQ’s leader Kate Walsh’s lack of qualifications — i.e. Walsh’s minimal background or experience in education, and the fact that Walsh regularly introduces herself oddly for someone who’s out to help teachers and children.

    Walsh leads with her self-description “I’m an entrepreneur”, not “I’m an educator”, as Walsh did at a presentation that she gave to parents and community members at a Seattle forum that Dora attended.

    NCTQ’s Emily, who apparently works far and admires Kate, is quite livid and vocal in her retort defending NCTQ and her boss Kate.

    First, here’s some of what Dora said that provoked Emily:

    ————————————————————-
    ————————————————————-
    DORA, parent writing at “Seattle Schools”:

    “When Kate Walsh came to Seattle to present her NCTQ report, it was just before union negotiations were to begin with the Seattle Education Association. The terms that were introduced in the report were echoed by ed reform groups during that time, terms like “teacher effectiveness and ‘performance pay’, items that our Broad–trained superintendent had put on the table for negotiations six months earlier.

    “The report was waved around in front of the press and community leaders by the faux grassroots ed reform organizations that were spawned by Gates and Broad, causing many community organizations to sign onto a Community Values Statement. That statement was brought in front of the teacher’s union during negotiations as a way to make teachers feel that the entire community was demanding that they relent on the issues of merit pay and seniority.

    “It was a scam plain and simple, and we will be written about in detail in a future post. That document also made its’ way to Olympia and our state legislators who then thought that everyone wanted these measures put into place and from that our ed reform Bill 6696 was born and adopted.

    “All of this done without anyone coming to the rest of us, parents, students and teachers, and asking us to participate in the formulation of our vision for education in Seattle.”
    ————————————————————-
    ————————————————————-

    ReplyDelete
  4. PART TWO:

    In response to Dora, Emily, the NCTQ functionary, then gets out her cat claws:

    ————————————————————-
    ————————————————————-

    NCTQ’s EMILY-to-DORA:

    “Did you know that (NCTQ head Kate Walsh) started a school for underprivileged boys from inner city Baltimore, and that shucks, she didn’t do it to win praise from people like you? She has devoted her entire life to improving education standards for schools, and she is doing is trying her best to think outside the box in a creative, constructive way.

    “Her entire staff is made up of former public school teachers, did you know that? All of the people producing and advocating this research- every single one of them has taught in a classroom.

    “What I find really hilarious is that you think that NCTQ and the rest of the cohort are out to hurt teachers or something? They want to get good teachers into school classrooms. Do you as a parent — which you so often like to point out — have issues with people who are trying to find other ways to evaluate our nations teachers?

    “Our government (i.e. its school system, JACK) is not performing. Mrs. Walsh’s statement about (the worthlessness of Education Dept.) Masters degrees is true. Look at what YOU have written. You’re getting fired up about people who are actually trying to do good in this world. Pick a different victim and — as a parent — look at people who are trying to harm your children and make it a more challenging world for them.

    “And please, do your homework before you write a slandering blog about someone you know absolutely nothing about.

    ” … ”

    “Dora, if you want to have a look at Kate’s CV and past experience, you just have to click on her name on the NCTQ website- pretty straight forward.

    “Or, for that matter give her a ring at her office.”
    ————————————————————-
    ————————————————————-

    CONTINUED BELOW:

    ReplyDelete
  5. PART FOUR:

    DORA-to-EMILY: (CONTINUED FROM ABOVE)

    “And to the point of teachers, there are evaluation systems in place. One is a four-tiered system that our state superintendent of schools has endorsed and is supported by our local teachers’ union. If the teachers in Seattle are confident that it will work, then I have the confidence that it will also. It has been explained to me by the union president and appears rational and reasonable.

    “The other point about teachers is that they (alleged “bad teachers”, JACK) are not the reason for our faltering educational system. See: Where Do We Go From Here?.

    ———————————————
    WHERE DO WE GO FROM HERE? article:

    “A lack of funding over four decades has left our educational system in shambles. We have schools that are legally unsafe, children do not have enough materials, books or workbooks to help them achieve the best that they can be.

    “In Seattle the students get ‘Art on a cart’ because of a lack of funding in many schools. Schools have had to shorten their days and increase class sizes due to a lack of funding. If we financially supported our educational system to the degree that we do our military/industrial complex or to the extent that we bailed out several corporations or subsidize oil companies, we would not be where we are today.

    “To put the emphasis on teachers as the sole problem in our educational system is naive at best.”
    ———————————————

    DORA:

    “And one last point. Everything that I wrote in this post was true, with enough links to each fact to validate all of my statements.”
    ————————————————————-
    ————————————————————-

    That’s the end of the dialogue, from either of them.

    ReplyDelete