The National Council on Teacher Quality is one of the great mysteries of the education biz. They have no particular credentials and are truly the laziest "researchers" on the planet, but I think I may have cracked the code. Let me show you their latest piece of "research," and then we can talk about how they really work.
Their new report-- "Learning about Learning: What Every New Teacher Needs To Know" (which is a curious title-- do other teachers NOT need to know these things?)-- is yet another NCTQ indictment of current teacher education programs. The broad stroke of their finding is that teacher education programs are not teaching the proven strategies that work in education.
That's the broad stroke. As always with NCTQ, the devil is in the details. After all, that sounds like a huge research undertaking. First, you would have to identify teaching strategies that are clearly and widely supported by all manner of research. Then you would have to carefully examine a whooooooole lot of teacher education programs-- college visits, professor and student interviews, sit in classes, extensive study of syllabi-- it would be a huge undertaking.
Or you could just flip through a bunch of educational methods textbooks.
What Every Teacher Needs To Know
First, NCTQ had to select those methods that "every new teacher needs to know." Here's the methodology for that piece of research-based heavy lifting:
In Organizing Instruction and Study to Improve Student Learning: A Practice Guide, the Institute of Education Sciences (IES), the research arm of the U.S. Department of Education, identified proven practices that promote learning for all students, regardless of grade or subject, and that are especially potent with struggling students. Six practices stand out for the research behind them. There is little debate among scholars about the effectiveness of these six strategies.
Here are a few things to know about Organizing Instruction and Study To Improve Student Learning.
It was published in September of 2007. It was produced under a USED- IES contract with Optimal Solutions Group, LLC, a policy data-analysis business. It opens with a disclaimer that includes this:
The opinions and positions expressed in this practice guide are the authors’ and do not necessarily represent the opinions and positions of the Institute of Education Sciences or the U.S. Department of Education.
The IES paper does, in fact, appear to be a group of researchers checking to see how much research basis there is for seven ideas that they think will help teaching subjects "that demand a great deal of content learning, including social studies, science, and mathematics." So, not actually "all subjects and grades" as NCTQ says. And they are based around a memory-based model of education.
More importantly, the IES paper rates the seven approaches according to strength of the research to support them. Four of the seven are rated "moderate," two are rated "low," and the seventh is rated "strong".
What Are The Must-Know Techniques?
That depends on whether you look at the original IES paper or the NCTQ "research." NCTQ drops one IES technique-- teaching students how to use time. And they convert "use quizzing to promote learning" into "assessing to boost retention." Either way, the IES paper rates the scientific basis for this technique low, with little research beyond reading instruction experiments with college students. So that whole "there is little debate" and "research-based" bullshit is, in fact, bullshit.
That is the only low-rated technique that made the list. The strong technique is "ask deep probing questions."
The other four are all moderately-rated, meaning that there is some research basis for them (back in 2007), but it's not overwhelming. Those four are "pairing graphics with words," "linking abstract concepts with concrete representations," "alternating solved problems with problems to solve," and "spread out practice over time."
These are not bad techniques, useless techniques, unwelcome techniques-- but is NCTQ suggesting that of all the educational techniques in the world, these six are the essential ones? Well, they call them "the fundamental knowledge they need to make learning 'stick.'"
NCTQ refers to these six techniques as "the field’s bedrock research as identified by IES," which is a lie.
And they reach this scary conclusion: "If teacher candidates aren’t being taught the research-proven and workable practices that help students learn new content, they will flounder when they try to make learning last." So how do they know that teacher candidates aren't being taught these techniques.
How Are Ed Schools Failing?
They looked in a bunch of textbooks. They looked at 48 college programs (at 28 different colleges). They selected books "assigned in educational psychology, general methods and secondary subject-specific methods courses." And because NCTQ really is the laziest research group on the planet, there's this:
We note that textbooks unique to subject-specific elementary methods courses were not reviewed in depth. What examination we did of these textbooks indicated that had we reviewed them, none would have received credit for covering the strategies.
We glanced at some books and they looked like they were fer suresies losers, so we just skipped those.
Ultimately they settled on 48 books. You can see the breakdown here. A team of four scanned through the books for signs of references to the techniques with blah blah blah I'm not sure they didn't just use [Control + F] here, but they have a complex-sounding technique for deciding if the technique was fully and accurately presented in the textbook. You can look through their methodology if you like, but the bottom line is that references to the techniques had to be absolutely on the nose.
They also claim to have done some study of programs, going back to their file folder of course syllabi, because that totally tells you exactly what goes on in classes. As with books, the requirement was to learn the technique as a general truth. It looks as if, for instance, learning all about teaching maths with manipulatives does not count as "linking abstract ideas to concrete representations" because that's only for a math class, and dopey teachers might not understand that linking abstract and concrete could be used in other classes.
Seriously?
There are so many other questions to ask, such as, "Do teachers actually use their methods and general ed psych books?" Is the main pathway for teaching prospective teachers through traditional lecture and textbooks? That would be an interesting question to study, but NCTQ does not go there. Heck, asking any teacher in any classroom, "Can you put your hands on your college methods textbook right now?" would be entertaining. But as always, NCTQ has more important things to do than try to find out any useful truths.
Who are these people?
NCTQ has appointed themselves the arbiters of teacher quality because reasons. Would you like to guess how many career teachers are actually involved in running NCTQ? Did you answer zero? Good for you. Would you like to guess how many former TFA temps are running the group? Did you guess many? Good for you again.
What is their research specialty?
NCTQ is the group that once declared that college teacher programs are too easy, and their research was (and I swear I am not making this up) to look through college commencement programs.
NCTQ is the group that cranked out a big report on teacher evaluation whose main point was, "It must not be right yet, because not enough teachers are failing."
NCTQ creates the college rankings list published every year by US News leading to critiques of NCTQ's crappy methodology here and here and here, to link to just a few. NCTQ's method here again focuses on syllabi and course listings, which, as one college critic noted, "is like a restaurant reviewer deciding on the quality of a restaurant based on its menu alone, without ever tasting the food." That college should count its blessings; NCTQ has been known to "rate" colleges without any direct contact at all.
NCTQ's history has been well-chronicled by both Mercedes Schneider and Diane Ravitch. It's worth remembering that She Who Must Not Be Named, the failed DC chancellor and quite possibly the least serious person to ever screw around with education policy, was also a part of NCTQ.
NCTQ depends on the reluctance of people to read past the lede. For this piece, for instance, anybody who bothered to go read the old IES paper that supposedly establishes these as "bedrock" techniques would see that the IES does no such thing. Anyone who read into the NCTQ "research" on teacher program difficulty would see it was based on reading commencement programs. The college president I spoke to was so very frustrated because anybody who walked onto her campus could see that the program NCTQ gave a low ranking was a program that did not actually exist.
But NCTQ specializes in headline research-- generate an eye-catching pro-reform headline and hope that if you follow it with a bunch of words, folks will just say, "Well, there's a lot of words there, so they must have a real research basis for what they're saying."
There are reform advocates who are, I believe, sincere and intellectually honest. But for NCTQ to, for instance, transcribe the 2007 IES report of the quizzing technique as a "bedrock" of learning when IES clearly ranks it as having little real research to back it up-- that requires either unbelievable stupidity, incredible laziness, or just flat-out lying. NCTQ might very well be the least serious outfit in the education biz, and yet they still draw press attention.
But if you run across references to this report (which is part of a broader assault on teacher education programs), rest assured that it's rubbish, and don't hesitate to encourage people to ignore it. Never has a group so justly deserved to be completely ignored.
Showing posts with label NCTQ. Show all posts
Showing posts with label NCTQ. Show all posts
Sunday, February 7, 2016
Wednesday, November 18, 2015
Gates Takes Aim at Teacher Education
As noted today at Education Week, the Gates Foundation has fastened its aim on teacher preparation programs. The Bill & Melinda Gates Foundation is ready to drop $34 million cool ones on "cooperative initiatives designed to improve teacher-preparation programs' overall effectiveness."
So what does that mean? Good news? Bad news?
The three year grants are based on four principles:
* developing strong partnerships with school districts
* giving teacher-candidates opportunities to refine a specific set of teaching skills
* using data for improvement and accountability
* ensuring that faculty mentors are effective at guiding novices into the profession
The first sounds great. The second sounds... well, I don't know. Exactly what specific set are we talking about, and what does that even mean? Becoming an interrogatory specialist? Learning to be excellent at teaching fractions? I'm worried that the Gates tendency to believe that all complex activities can be broken down into disconnected, context-free skills is at play here, in which case I'm doubting this will be useful.
Third? Well, if I thought "data" meant what I mean by "data, I'd think this was fine. I use data every minute of every day. But since this is Gates, I'm afraid that "data" means "results from a computer-based bunch of competency-based-baloney" or even "more of the useless data from those dreadful Big Standardized Tests."
Fourth point. Yes, excellent idea, if in fact you have any idea of how to tell that mentors are effective at guiding etc etc. Which I'm betting you don't, or worse, you have some sort of "based on student test scores VAM sauce baloney," which won't do anyone any good.
But hey, maybe the recipients of the Gates money will give us a clue about where this is headed.
Grantee #1 is TeacherSquared (you know-- a place that makes teacher teachers) which is mostly "nontraditional" preparation programs. In fact, it's mostly RelayGSE, a fake teacher school set up by charters so that non-teachers with a little experience could teach non-teachers with no experience how to be teachers. So that is not a good sign.
#2 Texas Tech University, "which will head the University-School Partnerships for the Renewal of Educator Preparation National Center" which is six Southern universities welded together. Lord only knows what that will look like.
#3 Massachusetts Department of Education, which will head up an EPIC (Elevate Preparation, Impact Children) center to work with all the teacher ed programs in the state. This is just going to be confusing, because the EPIC acronym has been used before-- including by charter schools in Massachusetts (Effective Practice Incentive Community). But the Massachusetts DoE has a mixed track record on reformy issues, so we'll see.
#4 National Center for Teacher Residencies, which is promoting a full-year residency model which has been popping up around the country and which I think could actually be a great idea.
TeachingWorks at University of Michigan will be a coordinating hub for all the cool things these other grantees will come up with.
According to EdWeek's Stephen Sawchuk, Gates wants each of these "centers" to crank out 2,500 teachers per year which is-- well, that is huge. I'm pretty sure that's more than most entire states produce. It is a grand total of 10,000 teachers. Per year. At a time when enrollment in teacher education programs is plummeting. The USPREPNC would have to get upwards of 600 teacher-grads per year out of its six member universities. I mean, we can turn this number around many ways, and from every angle, it's a huge number. Of the four grantees, only the state of Massachusetts seems likely to handle that kind of capacity.
Want more bad signs? Here's a quote from Vicki Phillips:
“The timing is great because of having great consistent, high standards in the country and more meaningful, actionable teacher-feedback systems and some clear definitions about what excellence in teaching looks like,” said Vicki Phillips, the Gates Foundation’s director of college-ready programs.
In other words, this is way to drive Common Core up into teacher education programs, where it can do more damage.
Anissa Listak of the NCTR points out that making sure clinical faculty (i.e. co-operating teachers) are top notch will be a game changer, and I don't disagree. But it sidesteps the question of how the top notch faculty will be identified, and it really side steps the issue of how the program will find 10,000 master teachers who want to share their classes with a student teacher for a whole year-- especially in locations where test scores will reflect on their own teacher ratings (including, perhaps, the ratings that marked them as "qualified" to host a teacher-resident in the first place).
The Gates has identified a need here-- evaluating teacher preparation programs. Nobody is doing it (well, nobody except the scam artists at NCTQ who do it by reading commencement programs and syllabi), and if we had a legitimate method of measuring program quality, it could be helpful to aspiring teachers. But we don't, and it's not clear that any of these grantees have a clue, either.
It all rests on knowing exactly how to measure and quantify teacher excellence. With data. And boy, there's no way that can end badly.
Will the Gates money be well-spent? I'm not optimistic-- particularly not with an outfit like Relay GSE on the list of recipients. And the Gates has a bad history of using grants to push a narrow and unbending agenda that it has already formed rather than truly exploring an issue or trying to get ideas from people who might know something. In other words, if this is all just a way for Gates to impose his own ideas of what teacher training should look like, then it's likely to be as wasteful and destructive as his championing of Common Core.
So what does that mean? Good news? Bad news?
The three year grants are based on four principles:
* developing strong partnerships with school districts
* giving teacher-candidates opportunities to refine a specific set of teaching skills
* using data for improvement and accountability
* ensuring that faculty mentors are effective at guiding novices into the profession
The first sounds great. The second sounds... well, I don't know. Exactly what specific set are we talking about, and what does that even mean? Becoming an interrogatory specialist? Learning to be excellent at teaching fractions? I'm worried that the Gates tendency to believe that all complex activities can be broken down into disconnected, context-free skills is at play here, in which case I'm doubting this will be useful.
Third? Well, if I thought "data" meant what I mean by "data, I'd think this was fine. I use data every minute of every day. But since this is Gates, I'm afraid that "data" means "results from a computer-based bunch of competency-based-baloney" or even "more of the useless data from those dreadful Big Standardized Tests."
Fourth point. Yes, excellent idea, if in fact you have any idea of how to tell that mentors are effective at guiding etc etc. Which I'm betting you don't, or worse, you have some sort of "based on student test scores VAM sauce baloney," which won't do anyone any good.
But hey, maybe the recipients of the Gates money will give us a clue about where this is headed.
Grantee #1 is TeacherSquared (you know-- a place that makes teacher teachers) which is mostly "nontraditional" preparation programs. In fact, it's mostly RelayGSE, a fake teacher school set up by charters so that non-teachers with a little experience could teach non-teachers with no experience how to be teachers. So that is not a good sign.
#2 Texas Tech University, "which will head the University-School Partnerships for the Renewal of Educator Preparation National Center" which is six Southern universities welded together. Lord only knows what that will look like.
#3 Massachusetts Department of Education, which will head up an EPIC (Elevate Preparation, Impact Children) center to work with all the teacher ed programs in the state. This is just going to be confusing, because the EPIC acronym has been used before-- including by charter schools in Massachusetts (Effective Practice Incentive Community). But the Massachusetts DoE has a mixed track record on reformy issues, so we'll see.
#4 National Center for Teacher Residencies, which is promoting a full-year residency model which has been popping up around the country and which I think could actually be a great idea.
TeachingWorks at University of Michigan will be a coordinating hub for all the cool things these other grantees will come up with.
According to EdWeek's Stephen Sawchuk, Gates wants each of these "centers" to crank out 2,500 teachers per year which is-- well, that is huge. I'm pretty sure that's more than most entire states produce. It is a grand total of 10,000 teachers. Per year. At a time when enrollment in teacher education programs is plummeting. The USPREPNC would have to get upwards of 600 teacher-grads per year out of its six member universities. I mean, we can turn this number around many ways, and from every angle, it's a huge number. Of the four grantees, only the state of Massachusetts seems likely to handle that kind of capacity.
Want more bad signs? Here's a quote from Vicki Phillips:
“The timing is great because of having great consistent, high standards in the country and more meaningful, actionable teacher-feedback systems and some clear definitions about what excellence in teaching looks like,” said Vicki Phillips, the Gates Foundation’s director of college-ready programs.
In other words, this is way to drive Common Core up into teacher education programs, where it can do more damage.
Anissa Listak of the NCTR points out that making sure clinical faculty (i.e. co-operating teachers) are top notch will be a game changer, and I don't disagree. But it sidesteps the question of how the top notch faculty will be identified, and it really side steps the issue of how the program will find 10,000 master teachers who want to share their classes with a student teacher for a whole year-- especially in locations where test scores will reflect on their own teacher ratings (including, perhaps, the ratings that marked them as "qualified" to host a teacher-resident in the first place).
The Gates has identified a need here-- evaluating teacher preparation programs. Nobody is doing it (well, nobody except the scam artists at NCTQ who do it by reading commencement programs and syllabi), and if we had a legitimate method of measuring program quality, it could be helpful to aspiring teachers. But we don't, and it's not clear that any of these grantees have a clue, either.
It all rests on knowing exactly how to measure and quantify teacher excellence. With data. And boy, there's no way that can end badly.
Will the Gates money be well-spent? I'm not optimistic-- particularly not with an outfit like Relay GSE on the list of recipients. And the Gates has a bad history of using grants to push a narrow and unbending agenda that it has already formed rather than truly exploring an issue or trying to get ideas from people who might know something. In other words, if this is all just a way for Gates to impose his own ideas of what teacher training should look like, then it's likely to be as wasteful and destructive as his championing of Common Core.
Saturday, May 2, 2015
NY: Eval Overhaul In Scary Hands
The expert names for the New York teacher evaluation high speed overhaul panel are in, and it is, at best, a mixed bag.
* Thomas Kane, an economist from Harvard. Kane thinks that evaluation should be directly linked to the Common Core via high stakes testing; he likes to compare this to using a bathroom scale when dieting. He thinks too few NY teachers were evaluated as sucky last year, and he imagines that maybe video-based observation would be swell. And he was an expert witness for the Vergara trial (can you guess on which side?) He headed up the Gates Measures of Effective Teaching study, and he thinks Cuomo is pretty much on the right track.
* Catherine Brown, vice-president of the Center for American Progress, a thinky tank invariably billed as "left-leaning" despite their general on-boardedness with assaults on the teaching profession. CAP has issued any number of sloppy and ill-supported attempts to push Common Core and VAM.
* Sandi Jacobs, vice-president of the National Council on Teacher Quality, a group that has taken the position that US teachers are low-quality hacks. These are the guys who help US News and World rate college teacher programs (including programs that don't actually exist) and who cobbled together a report on the rigor of college teacher prep programs by sitting in their offices and looking through a stack of commencement programs.
* Leslie Guggenheim of TNTP (The New Teacher Project), a group that really wants to see more personnel decisions, including pay, based on test results. They'd kind of like to get rid of tenure, too. Their big claim to fame is a paper called "The Widget Effect," that argues that teachers are not interchangeable widgets, but are in fact interchangeable widgets of varying degrees of quality.
I will go out on a limb and predict right now, today, that these four will declare that Cuomo's evaluation plan is okee dokee. But in the interests of not-entirely-kangaroo courtage (and perhaps additional entertainment value), the group also includes:
* Jesse Rothstein is a professor at Berkeley who has spent some time shooting holes in the research of both Kane and Raj Chetty. Starting with the same data, he found far less to love about VAM.
* Stephen Caldas is a professor at Manhattanville College who tagged the NY evaluation system with the delightful term "psychometrically indefensible."
* Aaron Pallas of Teachers College. He's been busily pointing out the problems with VAMmy systems for a few years now.
Those of you who have scored proficient in counting will notice that the majority of the committee seats are occupied by fans of reformy nonsense. But wait-- there's more.
Cuomo's insanely accelerated timeline (why get things right when you can get them done quickly) means that the usual 45-day post-draft comment period on proposed regulations is being waived because, well, if you had it, people might comment. Hey, it's not like anything else about supposed ed reform has suffered from being rammed through too quickly.
So NYSUT (which you may or may not love-- honestly, you New Yorkers and your intra-union alliances and battles) is on point when they say that everybody had better start making comments and making them now. President Karen MaGee says that folks need to speak up.
"NYSUT is well aware of the unrealistic deadlines contained in the governor's convoluted and unworkable plan, and the pressure that puts on the Regents and SED to try and mitigate the worst of it. Still, those deadlines do not absolve them of their responsibility to listen carefully to parents and practitioners and make any necessary adjustments to the draft regulations they wind up writing," Magee said. "One month is plenty of time for SED and the Regents to hold public hearings and still meet their deadlines."
So if you're a New York teacher or parent, it's time (right now-- the committee meets May 7) to get word to a Regent or the State Education Department. Tell them you want hearings on the draft. Tell them what you want in the evaluation system. Tell them why the stuff the committee is about to okay is a bunch of hooey (I'd suggest a more professional word than "hooey")
You can find a guide to individual Board of Regents members right here, complete with email links. You can find some NYSED phone numbers here and a whole department index starting with the A's right here. The clock is ticking. Time to make some noise. You might want to let the non-junk-science portion of the group know you support them-- they may be feeling a bit lonely soon. Heck-- you can even send word to Andrew Cuomo himself. It looks like this whole mess isn't going to be pretty-- but it doesn't have to be ugly and quiet both.
* Thomas Kane, an economist from Harvard. Kane thinks that evaluation should be directly linked to the Common Core via high stakes testing; he likes to compare this to using a bathroom scale when dieting. He thinks too few NY teachers were evaluated as sucky last year, and he imagines that maybe video-based observation would be swell. And he was an expert witness for the Vergara trial (can you guess on which side?) He headed up the Gates Measures of Effective Teaching study, and he thinks Cuomo is pretty much on the right track.
* Catherine Brown, vice-president of the Center for American Progress, a thinky tank invariably billed as "left-leaning" despite their general on-boardedness with assaults on the teaching profession. CAP has issued any number of sloppy and ill-supported attempts to push Common Core and VAM.
* Sandi Jacobs, vice-president of the National Council on Teacher Quality, a group that has taken the position that US teachers are low-quality hacks. These are the guys who help US News and World rate college teacher programs (including programs that don't actually exist) and who cobbled together a report on the rigor of college teacher prep programs by sitting in their offices and looking through a stack of commencement programs.
* Leslie Guggenheim of TNTP (The New Teacher Project), a group that really wants to see more personnel decisions, including pay, based on test results. They'd kind of like to get rid of tenure, too. Their big claim to fame is a paper called "The Widget Effect," that argues that teachers are not interchangeable widgets, but are in fact interchangeable widgets of varying degrees of quality.
I will go out on a limb and predict right now, today, that these four will declare that Cuomo's evaluation plan is okee dokee. But in the interests of not-entirely-kangaroo courtage (and perhaps additional entertainment value), the group also includes:
* Jesse Rothstein is a professor at Berkeley who has spent some time shooting holes in the research of both Kane and Raj Chetty. Starting with the same data, he found far less to love about VAM.
* Stephen Caldas is a professor at Manhattanville College who tagged the NY evaluation system with the delightful term "psychometrically indefensible."
* Aaron Pallas of Teachers College. He's been busily pointing out the problems with VAMmy systems for a few years now.
Those of you who have scored proficient in counting will notice that the majority of the committee seats are occupied by fans of reformy nonsense. But wait-- there's more.
Cuomo's insanely accelerated timeline (why get things right when you can get them done quickly) means that the usual 45-day post-draft comment period on proposed regulations is being waived because, well, if you had it, people might comment. Hey, it's not like anything else about supposed ed reform has suffered from being rammed through too quickly.
So NYSUT (which you may or may not love-- honestly, you New Yorkers and your intra-union alliances and battles) is on point when they say that everybody had better start making comments and making them now. President Karen MaGee says that folks need to speak up.
"NYSUT is well aware of the unrealistic deadlines contained in the governor's convoluted and unworkable plan, and the pressure that puts on the Regents and SED to try and mitigate the worst of it. Still, those deadlines do not absolve them of their responsibility to listen carefully to parents and practitioners and make any necessary adjustments to the draft regulations they wind up writing," Magee said. "One month is plenty of time for SED and the Regents to hold public hearings and still meet their deadlines."
So if you're a New York teacher or parent, it's time (right now-- the committee meets May 7) to get word to a Regent or the State Education Department. Tell them you want hearings on the draft. Tell them what you want in the evaluation system. Tell them why the stuff the committee is about to okay is a bunch of hooey (I'd suggest a more professional word than "hooey")
You can find a guide to individual Board of Regents members right here, complete with email links. You can find some NYSED phone numbers here and a whole department index starting with the A's right here. The clock is ticking. Time to make some noise. You might want to let the non-junk-science portion of the group know you support them-- they may be feeling a bit lonely soon. Heck-- you can even send word to Andrew Cuomo himself. It looks like this whole mess isn't going to be pretty-- but it doesn't have to be ugly and quiet both.
Thursday, November 13, 2014
NCTQ: It's So Easy
The National Council on Teacher Quality is one of the leaders in the production of education-related nonsense that is somehow taken seriously. The offices of NCTQ may not produce much of anything that provides real substance, but somewhere in that cushy suite there must be the best turd-polishing machine ever built.
NCTQ has published a new "report" that "seeks" to "answer" two questions:
Are teacher candidates graded too easily, misleading them about their readiness to teach? Are teacher preparation programs providing sufficiently rigorous training, or does the approach to training drive higher grades?
And when I say "seeks to answer," what I mean is "tries to cobble together support for the answer they've already settled on." There is no indication anywhere that NCTQ actually wondered what the answers to these questions might be. No, it appears that they set out to prove that teacher candidates are graded too easily as they meander through their rigorless teacher programs.
Who are these guys?
Does the NCTQ moniker seem familiar? That's because these are the guys who evaluated everyone's education program and ran it as a big story in US News (motto "When it comes to sales, lists are better than news"). That evaluation list caused a lot of stir. A lot.
Funny story. I interviewed a college president from a local school who was steamed about that list in part because it slammed them for the low quality of a teacher program that they don't even have. Turns out a lot of people had problems with NCTQ methodology, which involved not actually talking to anybody at the schools, but collecting information less detailed than what you can get from your high school guidance counselor. You can read a cranky critique here and a more scholarly one here. Bottom line-- they get great media penetration with a report that has less substance than my hair.
NCTQ has tried to blunt some of the criticism with moves like adding a Teacher Advisory Group composed of Real Live Teachers. Also, NCTQ uses very pretty graphics in soothing colors.
The new report-- Easy A's (they even italicize the title, like a book title, because it's more hefty and important than a mere "article title")-- is billed as "the latest latest installment of the National Council on Teacher Quality’s Teacher Prep Review, a decade-old initiative examining the quality of the preparation of new teachers in the United States." This is supposed to be part of their "growing body of work designed to ensure that teacher preparation programs live up to the awesome responsibility they assume." For the moment, let's look at the "findings" in this "report."
And the bottom line is...
They studied about 500 schools; these schools are collectively responsible for about half the teacher degrees granted. Two "findings" here.
First, they find the majority of institutions studied (58%) grading standards are lower than for students in other majors on the same campus.
Second, they find a strong link "between high grades and a lack of rigorous coursework, with the primary cause being assignments that fail to develop the critical skills and knowledge every new teacher needs."
Wow!!
I know! My mind boggles at the huge amount of research involved here. This must have required an extensive study of each of the 500 institutions studied. I mean, we're talking about comparing the rigor of assignments in both education and non-education courses, so researchers must have had to dig through tens of thousands of college course assignments in addition to an extensive study of the grading standards of thousands of professors to be able to make these comparisons.
And then to do all the research and number crunching needed to establish a correlation between the rigor of assignments and grades achieved-- this would be a more complicated model than VAM, to tease out all those data,
Also, it's worth noting that NCTQ knows what critical skills and knowledge all new teachers need, which must have been a huge research project all by itself. I do hope they publish that one soon, because if we had such a list, it would certainly revolutionize teacher evaluation and training. In fact, if they've done all this research and know all these answers, why aren't they just traveling from college to college and saying, "Here-- this is what our proven research shows you should be teaching teacher trainees."
Never mind.
Apparently the minds at NCTQ boggled at that research challenge as well. So let's look at what they actually did.
For the first point-- the idea that teacher grads are the recipients of departmental grade inflation-- NCTQ looked at commencement brochures. They checked commencement brochures to see a) who graduated from a teaching program and b) who had an honors grad designation based on GPA.
It is not clear how many years are spanned. There were 500-ish schools studied, and footnotes in an appendix indicate that a total of 436 commencement brochures were discarded for insufficient data. Yet the executive summary says that 44% of all teacher grads in all 509 schools earned honors, while only 30% of all graduating students did. And in 214 schools, there's no real difference, and in 62 schools, teachers had fewer honors grads. How did they get such precise numbers? That is not explained.
There's some more detailed breakdown of methodologies of teasing the data, but that's the data they accumulated from graduation brochures, and the whole argument boils down to "Barely more teachers graduate with honors than do other majors." Oddly enough, this does not lead NCTQ to conclude, "Good news, America! Teachers are actually smarter than the average college grad." I guess it comes down to how you interpret the data. That you collected from graduation brochures.
But what about that lack of rigor?
Having somehow concluded that teacher programs are hotbeds of easy grades, NCTQ turns to the question of who let the rigor out. Once again, their methodology is itself as rigorous as a bowl of noodles left to boil for twelve hours.
Multiple theories as to why students in education majors might appear to excel so often were also examined (e.g., clinical coursework that lends itself to high grades, too many arts and crafts assignments, too much group work, particularly egregious grade inflation, better quality instruction, more female students who tend to get higher grades, opportunities to revise work, and higher caliber students), but none appears to explain these findings as directly as the nature of the assignments.
First of all, interesting list of theories there. Second of all-- "none appears to explain"?? How did you judge that appearance, exactly. Did we just put each theory on a post-it note, stick it to the wall, stand back and hold up our thumbs and squint to see which one looked likely? Because usually the way you reject theories in research is with research.
The Big NCTQ Thumb came to rest on "criterion-referenced" and "criterion-deficient" assignments. We'll come back to that in a moment-- it deserves its own subheading. Just one more note about te methodology here.
NCTQ got their hands on syllabi for 1,161 courses, "not just on teacher education but across an array of majors." Except-- wait. We're looking at 509 schools, so 1,161 works out to a hair over two courses per school, including schools with multiple education programs. Oh, no-- never mind. Here it is in the appendix-- we're only going to do this in depth analysis for seven schools. Plus at thirty-three other schools, we will look at just the education programs. Oh, and on this chart it shows that of the seven in-depth schools, we'll look at only teacher programs in two. So, "wide array" means six other majors at five of the 509 schools.
And yes-- the data comes from course syllabi. At seven schools. Not the course, professors, students-- just the syllabi. So our first assumption will be that these syllabi with their lists of course assignments will tell us everything we need to know about how rigorous the coursework is.
Creating the right hatchet for the job
"Criterion-referenced" is a thing, and it basically means an objective test. "Criterion-deficient," on the other hand, will actually win you a game of googlewhacking because apparently nobody uses the term except NCTQ. "Criterion deficiency" is a real thing, used it seems mostly in the non-teaching world to describe a test that fails to assess an important criterion (e.g. you want a secretary who can word process, but the job evaluation doesn't check for word processing). I bring this up only because now I have a great fancy word for discussing high stakes standardized tests-- they suffer from citerion deficiency.
But back to our story.
NCTQ cross-reference course syllabi with grade records posted publicly by registrars and open records requests (Schools DO that?! Seven years ago I wasn't allowed to know the grades of my own children for whom I was paying bills because of privacy laws!) NCTQ "applauds the commitment to transparency" of those schools willing to complete ignore student privacy concerns.
NCTQ looked at 7,500 assignments and ranked them as either CR or CD (that's my abbreviation--if they can be lazy researchers, I can be a lazy typist). Here are the criteria used to tell the difference:
An assignment is considered criterion-referenced when it is focused on a clearly circumscribed body of knowledge and the assignment is limited so that the instructor can compare students’ work on the same assignment.
Qualities that indicate an assignment is criterion-referenced include
* a limited scope
* evaluation based on objective criteria;
* students’ work products similar enough to allow comparison.
Qualities that indicate an assignment is criterion-deficient include
* an unlimited or very broad scope
* evaluation based on subjective criteria
* students’ work products that differ too much to be compared.
NCTQ provides a sample of each. A CR lesson would be one in which the student is to apply a specific tool to critique videotaped (quaint) lessons. This is good because everyone uses the same tool for the same lessons so that the instructor knows exactly what is going on. A CD assignment would be to teach something-- anything-- to the class using information from the chapter. This is bad because everybody teaches something different, uses different specific parts of the chapter, and teaches in different ways. This would be bad because there would be too many differences the instructor would be unable to determine who is best at the teaching of stuff.
How is this distinction a useless one? Let me count the ways.
1) It assumes that the purpose of an assignment is to allow comparison of one student to another. This would be different from, say, using assignment as a way to develop and synthesize learning, or to mark the growth and development of the individual student.
2) It assumes that all good teachers look exactly the same and can be directly compared. It imagines that difference is a Bad Thing that never intrudes in a proper classroom. This is bananas.
3) It assumes that the best assignments are close-ended assignments that have only one possible outcome. Even reformsters steeped in Depth of Knowledge and Rigorology tout the value of open-ended response tasks with a variety of correct answers without demanding that the many correct responses be ranked in order of most correct.
4) It appears to place the highest value on objective testing. If that is true for my teacher training, is it not true for my practice? In other words, is NCTQ suggesting that the best assessments for me to use with my students are true-false and multiple choice tests rather than any sort of project-based assessment. Because, no.
5) It assumes that all students in the future teacher's future classroom will also be one size fits all. When I ask my students to prepare oral presentations, should I require that they all do reports on Abraham Lincoln so that they can be more easily and accurately compared?
6) It discounts the truth that part of being a professional is being ready and able to exercise subjective judgment in an objective manner. In other words, free from personal prejudice, but open to the ways that the individual student's personality, skills, and history play into the work at hand (without excusing crappy work).
7) They are trying to make this distinction based on assignments listed in syllabi.
There's more
I did look for the appendix evaluating how well five weeks will prepare you for teaching if you come from a super-duper university, but that was one aspect of teacher training ease NCTQ did not address.
There are other appendices, examining ideas such as Why High Grades Are Bad (including, but not limited to, if grades are divorced from learning, would-be employers will find grades less useful). But I'm not going to plow through them because at the end of the day, this is a report in which some people collected some graduation brochures and course syllabi and close read their way to an indictment of all college teacher training programs.
It is not that these questions are without merit. Particularly nowadays, as teacher programs are increasingly desperate to hold onto enough paying customers to keep the ivy covered lights on, teacher training programs are undoubtedly under increasing pressure to manufacture success for their students, one way or another. Nor is it unheard of for co-operating teachers to look at student teachers and think, "Who the hell let this hapless kid get this far, and who the hell is going to actually give him a teaching certificate?"
The question of how well training programs are preparing teachers for real jobs in the real world ought to be asked and discussed regularly (more colleges and universities might consider the radical notion of consulting actual teachers on the matter). And as more reformster foolishness infects college ed departments, the problem of Useless Training only becomes worse. So this is absolutely a matter that needs to be examined and discussed, but the method should be something more rigorous than collecting some commencement brochures and course syllabi and sitting in an office making ill-supported guesses about what these random slips of paper mean.
And yet I feel in my bones that soon enough I'll be reading main stream media accounts of the important "findings" of this significant "research." Talk about your easy A.
NCTQ has published a new "report" that "seeks" to "answer" two questions:
Are teacher candidates graded too easily, misleading them about their readiness to teach? Are teacher preparation programs providing sufficiently rigorous training, or does the approach to training drive higher grades?
And when I say "seeks to answer," what I mean is "tries to cobble together support for the answer they've already settled on." There is no indication anywhere that NCTQ actually wondered what the answers to these questions might be. No, it appears that they set out to prove that teacher candidates are graded too easily as they meander through their rigorless teacher programs.
Who are these guys?
Does the NCTQ moniker seem familiar? That's because these are the guys who evaluated everyone's education program and ran it as a big story in US News (motto "When it comes to sales, lists are better than news"). That evaluation list caused a lot of stir. A lot.
Funny story. I interviewed a college president from a local school who was steamed about that list in part because it slammed them for the low quality of a teacher program that they don't even have. Turns out a lot of people had problems with NCTQ methodology, which involved not actually talking to anybody at the schools, but collecting information less detailed than what you can get from your high school guidance counselor. You can read a cranky critique here and a more scholarly one here. Bottom line-- they get great media penetration with a report that has less substance than my hair.
NCTQ has tried to blunt some of the criticism with moves like adding a Teacher Advisory Group composed of Real Live Teachers. Also, NCTQ uses very pretty graphics in soothing colors.
The new report-- Easy A's (they even italicize the title, like a book title, because it's more hefty and important than a mere "article title")-- is billed as "the latest latest installment of the National Council on Teacher Quality’s Teacher Prep Review, a decade-old initiative examining the quality of the preparation of new teachers in the United States." This is supposed to be part of their "growing body of work designed to ensure that teacher preparation programs live up to the awesome responsibility they assume." For the moment, let's look at the "findings" in this "report."
And the bottom line is...
They studied about 500 schools; these schools are collectively responsible for about half the teacher degrees granted. Two "findings" here.
First, they find the majority of institutions studied (58%) grading standards are lower than for students in other majors on the same campus.
Second, they find a strong link "between high grades and a lack of rigorous coursework, with the primary cause being assignments that fail to develop the critical skills and knowledge every new teacher needs."
Wow!!
I know! My mind boggles at the huge amount of research involved here. This must have required an extensive study of each of the 500 institutions studied. I mean, we're talking about comparing the rigor of assignments in both education and non-education courses, so researchers must have had to dig through tens of thousands of college course assignments in addition to an extensive study of the grading standards of thousands of professors to be able to make these comparisons.
And then to do all the research and number crunching needed to establish a correlation between the rigor of assignments and grades achieved-- this would be a more complicated model than VAM, to tease out all those data,
Also, it's worth noting that NCTQ knows what critical skills and knowledge all new teachers need, which must have been a huge research project all by itself. I do hope they publish that one soon, because if we had such a list, it would certainly revolutionize teacher evaluation and training. In fact, if they've done all this research and know all these answers, why aren't they just traveling from college to college and saying, "Here-- this is what our proven research shows you should be teaching teacher trainees."
Never mind.
Apparently the minds at NCTQ boggled at that research challenge as well. So let's look at what they actually did.
For the first point-- the idea that teacher grads are the recipients of departmental grade inflation-- NCTQ looked at commencement brochures. They checked commencement brochures to see a) who graduated from a teaching program and b) who had an honors grad designation based on GPA.
It is not clear how many years are spanned. There were 500-ish schools studied, and footnotes in an appendix indicate that a total of 436 commencement brochures were discarded for insufficient data. Yet the executive summary says that 44% of all teacher grads in all 509 schools earned honors, while only 30% of all graduating students did. And in 214 schools, there's no real difference, and in 62 schools, teachers had fewer honors grads. How did they get such precise numbers? That is not explained.
There's some more detailed breakdown of methodologies of teasing the data, but that's the data they accumulated from graduation brochures, and the whole argument boils down to "Barely more teachers graduate with honors than do other majors." Oddly enough, this does not lead NCTQ to conclude, "Good news, America! Teachers are actually smarter than the average college grad." I guess it comes down to how you interpret the data. That you collected from graduation brochures.
But what about that lack of rigor?
Having somehow concluded that teacher programs are hotbeds of easy grades, NCTQ turns to the question of who let the rigor out. Once again, their methodology is itself as rigorous as a bowl of noodles left to boil for twelve hours.
Multiple theories as to why students in education majors might appear to excel so often were also examined (e.g., clinical coursework that lends itself to high grades, too many arts and crafts assignments, too much group work, particularly egregious grade inflation, better quality instruction, more female students who tend to get higher grades, opportunities to revise work, and higher caliber students), but none appears to explain these findings as directly as the nature of the assignments.
First of all, interesting list of theories there. Second of all-- "none appears to explain"?? How did you judge that appearance, exactly. Did we just put each theory on a post-it note, stick it to the wall, stand back and hold up our thumbs and squint to see which one looked likely? Because usually the way you reject theories in research is with research.
The Big NCTQ Thumb came to rest on "criterion-referenced" and "criterion-deficient" assignments. We'll come back to that in a moment-- it deserves its own subheading. Just one more note about te methodology here.
NCTQ got their hands on syllabi for 1,161 courses, "not just on teacher education but across an array of majors." Except-- wait. We're looking at 509 schools, so 1,161 works out to a hair over two courses per school, including schools with multiple education programs. Oh, no-- never mind. Here it is in the appendix-- we're only going to do this in depth analysis for seven schools. Plus at thirty-three other schools, we will look at just the education programs. Oh, and on this chart it shows that of the seven in-depth schools, we'll look at only teacher programs in two. So, "wide array" means six other majors at five of the 509 schools.
And yes-- the data comes from course syllabi. At seven schools. Not the course, professors, students-- just the syllabi. So our first assumption will be that these syllabi with their lists of course assignments will tell us everything we need to know about how rigorous the coursework is.
Creating the right hatchet for the job
"Criterion-referenced" is a thing, and it basically means an objective test. "Criterion-deficient," on the other hand, will actually win you a game of googlewhacking because apparently nobody uses the term except NCTQ. "Criterion deficiency" is a real thing, used it seems mostly in the non-teaching world to describe a test that fails to assess an important criterion (e.g. you want a secretary who can word process, but the job evaluation doesn't check for word processing). I bring this up only because now I have a great fancy word for discussing high stakes standardized tests-- they suffer from citerion deficiency.
But back to our story.
NCTQ cross-reference course syllabi with grade records posted publicly by registrars and open records requests (Schools DO that?! Seven years ago I wasn't allowed to know the grades of my own children for whom I was paying bills because of privacy laws!) NCTQ "applauds the commitment to transparency" of those schools willing to complete ignore student privacy concerns.
NCTQ looked at 7,500 assignments and ranked them as either CR or CD (that's my abbreviation--if they can be lazy researchers, I can be a lazy typist). Here are the criteria used to tell the difference:
An assignment is considered criterion-referenced when it is focused on a clearly circumscribed body of knowledge and the assignment is limited so that the instructor can compare students’ work on the same assignment.
Qualities that indicate an assignment is criterion-referenced include
* a limited scope
* evaluation based on objective criteria;
* students’ work products similar enough to allow comparison.
Qualities that indicate an assignment is criterion-deficient include
* an unlimited or very broad scope
* evaluation based on subjective criteria
* students’ work products that differ too much to be compared.
NCTQ provides a sample of each. A CR lesson would be one in which the student is to apply a specific tool to critique videotaped (quaint) lessons. This is good because everyone uses the same tool for the same lessons so that the instructor knows exactly what is going on. A CD assignment would be to teach something-- anything-- to the class using information from the chapter. This is bad because everybody teaches something different, uses different specific parts of the chapter, and teaches in different ways. This would be bad because there would be too many differences the instructor would be unable to determine who is best at the teaching of stuff.
How is this distinction a useless one? Let me count the ways.
1) It assumes that the purpose of an assignment is to allow comparison of one student to another. This would be different from, say, using assignment as a way to develop and synthesize learning, or to mark the growth and development of the individual student.
2) It assumes that all good teachers look exactly the same and can be directly compared. It imagines that difference is a Bad Thing that never intrudes in a proper classroom. This is bananas.
3) It assumes that the best assignments are close-ended assignments that have only one possible outcome. Even reformsters steeped in Depth of Knowledge and Rigorology tout the value of open-ended response tasks with a variety of correct answers without demanding that the many correct responses be ranked in order of most correct.
4) It appears to place the highest value on objective testing. If that is true for my teacher training, is it not true for my practice? In other words, is NCTQ suggesting that the best assessments for me to use with my students are true-false and multiple choice tests rather than any sort of project-based assessment. Because, no.
5) It assumes that all students in the future teacher's future classroom will also be one size fits all. When I ask my students to prepare oral presentations, should I require that they all do reports on Abraham Lincoln so that they can be more easily and accurately compared?
6) It discounts the truth that part of being a professional is being ready and able to exercise subjective judgment in an objective manner. In other words, free from personal prejudice, but open to the ways that the individual student's personality, skills, and history play into the work at hand (without excusing crappy work).
7) They are trying to make this distinction based on assignments listed in syllabi.
There's more
I did look for the appendix evaluating how well five weeks will prepare you for teaching if you come from a super-duper university, but that was one aspect of teacher training ease NCTQ did not address.
There are other appendices, examining ideas such as Why High Grades Are Bad (including, but not limited to, if grades are divorced from learning, would-be employers will find grades less useful). But I'm not going to plow through them because at the end of the day, this is a report in which some people collected some graduation brochures and course syllabi and close read their way to an indictment of all college teacher training programs.
It is not that these questions are without merit. Particularly nowadays, as teacher programs are increasingly desperate to hold onto enough paying customers to keep the ivy covered lights on, teacher training programs are undoubtedly under increasing pressure to manufacture success for their students, one way or another. Nor is it unheard of for co-operating teachers to look at student teachers and think, "Who the hell let this hapless kid get this far, and who the hell is going to actually give him a teaching certificate?"
The question of how well training programs are preparing teachers for real jobs in the real world ought to be asked and discussed regularly (more colleges and universities might consider the radical notion of consulting actual teachers on the matter). And as more reformster foolishness infects college ed departments, the problem of Useless Training only becomes worse. So this is absolutely a matter that needs to be examined and discussed, but the method should be something more rigorous than collecting some commencement brochures and course syllabi and sitting in an office making ill-supported guesses about what these random slips of paper mean.
And yet I feel in my bones that soon enough I'll be reading main stream media accounts of the important "findings" of this significant "research." Talk about your easy A.
Thursday, June 19, 2014
TNTP Evaluates NCTQ
It's a rare day when one titan reformsylvania decides to critique another's work, but that's what we have over at the TNTP blog, where Karolyn Belcher suggests that the NCTQ evaluation of teacher programs could use some tweaks.
Mind you, they think that NCTQ is doing God's work, and doing it well. Their ratings will stir up controversy and "it's the right conversation to have on a critical issue." TNTP is also excited " to see NCTQ begin to put alternative programs on equal footing with traditional programs, and welcome the outside scrutiny." And they did okay, ratings-wise ("solid Bs").
So, really, they don't want to complain. But they have some suggestions, some things deserving of tweakery (aka some ways they would look better). Do tell! Let's see the list!
A more focused approach to evaluating teacher support.
NCTQ dinged TNTP for not providing a co-teaching period for their recruits. "That’s fine in principle, but there is little or no evidence connecting it to improved practice or student learning. Other approaches, such as focused coaching (even in virtual environments), have a broader research base."
I left the links in that quote so you could see TNTP's support. The first unfortunately leads to a log-in page. Next we get a study of literacy coaches in a Florida middle school, a coaching "experiment run in the NOLA RSD, and a study of coaching in choice schools. You can wade through them, or you can take my word for it that they are not exactly a mountain of compelling evidence.
A more nuanced view of content preparation.
TNTP agrees that teachers should know what the hell their talking about in their content area (I'm paraphrasing) but takes umbrage that NCTQ suggests more coursework and tests than the state requires, because the research doesn't support it.And by research, once again, we mean "proof that it raises test scores." Keep that in mind because number three is--
A stronger push for evidence of effectiveness.
We know that NCTQ shares our belief that the best measure of a program’s effectiveness isn’t what programs put into their candidates, but whether the teachers it prepares advance student learning. NCTQ rightfully incorporated value-added data in its assessments of programs where available.
VAM is like the flippin' educational bureaucrat zombie. Kill it over and over again and it just keeps coming back. Take mountains of evidence of VAM's uselessness as a measure of teacher effective, and pile that mountain on top of VAM and it just claws its way out and stumbles onward.
But as bizarre as it is that anyone would think you could go a step further and evaluate the program that created a teacher based on the standardized test score of that teacher's students, it is even more bizarre that TNTP would insist on being judged by it, that they would stand there, in the middle of the street screaming, "Come here, zombie VAM! Come and eat OUR brains!!"
It's like insisting on playing Russian Roulette because you are certain you have a method to beat the game. It is at times like this that I suspect TNTP leadership does not have a brain in its collective head. Maybe they are Wile E. Coyote smart; perhaps they feel that public schools will take more damage than they will with this bogus instrument, or that their superior marketing will withstand it (after all, their titular launcher is a woman who still carries weight in the education world despite never having succeeded at anything in education except making money).
The article wraps up with general glad-handing. Glad to work with NCTQ. Everybody should do it (but bring your shiny evidence-like research). And in the meantime, don't forget that NCTQ is giving pretty much everybody a bad grade.
Mind you, they think that NCTQ is doing God's work, and doing it well. Their ratings will stir up controversy and "it's the right conversation to have on a critical issue." TNTP is also excited " to see NCTQ begin to put alternative programs on equal footing with traditional programs, and welcome the outside scrutiny." And they did okay, ratings-wise ("solid Bs").
So, really, they don't want to complain. But they have some suggestions, some things deserving of tweakery (aka some ways they would look better). Do tell! Let's see the list!
A more focused approach to evaluating teacher support.
NCTQ dinged TNTP for not providing a co-teaching period for their recruits. "That’s fine in principle, but there is little or no evidence connecting it to improved practice or student learning. Other approaches, such as focused coaching (even in virtual environments), have a broader research base."
I left the links in that quote so you could see TNTP's support. The first unfortunately leads to a log-in page. Next we get a study of literacy coaches in a Florida middle school, a coaching "experiment run in the NOLA RSD, and a study of coaching in choice schools. You can wade through them, or you can take my word for it that they are not exactly a mountain of compelling evidence.
A more nuanced view of content preparation.
TNTP agrees that teachers should know what the hell their talking about in their content area (I'm paraphrasing) but takes umbrage that NCTQ suggests more coursework and tests than the state requires, because the research doesn't support it.And by research, once again, we mean "proof that it raises test scores." Keep that in mind because number three is--
A stronger push for evidence of effectiveness.
We know that NCTQ shares our belief that the best measure of a program’s effectiveness isn’t what programs put into their candidates, but whether the teachers it prepares advance student learning. NCTQ rightfully incorporated value-added data in its assessments of programs where available.
VAM is like the flippin' educational bureaucrat zombie. Kill it over and over again and it just keeps coming back. Take mountains of evidence of VAM's uselessness as a measure of teacher effective, and pile that mountain on top of VAM and it just claws its way out and stumbles onward.
But as bizarre as it is that anyone would think you could go a step further and evaluate the program that created a teacher based on the standardized test score of that teacher's students, it is even more bizarre that TNTP would insist on being judged by it, that they would stand there, in the middle of the street screaming, "Come here, zombie VAM! Come and eat OUR brains!!"
It's like insisting on playing Russian Roulette because you are certain you have a method to beat the game. It is at times like this that I suspect TNTP leadership does not have a brain in its collective head. Maybe they are Wile E. Coyote smart; perhaps they feel that public schools will take more damage than they will with this bogus instrument, or that their superior marketing will withstand it (after all, their titular launcher is a woman who still carries weight in the education world despite never having succeeded at anything in education except making money).
The article wraps up with general glad-handing. Glad to work with NCTQ. Everybody should do it (but bring your shiny evidence-like research). And in the meantime, don't forget that NCTQ is giving pretty much everybody a bad grade.
Tuesday, June 17, 2014
NCTQ Coverage- How's It Look So Far?
The release of this year's NCTQ teacher prep school ratings is another opportunity for news organizations to practice press release journalism. The good news is that as of noon-time on June 17, most outfits aren't even bothering to do that.
Foxnews.com is headlining a list of twelve terrible teachers who are being protected by tenure. A search on USAToday.com for "nctq" produces only a press release from February of 2013. On cnn.com, a similar search turns up nothing at all. Even MSNBC, whose love for reformy things is deep and abiding, is more excited about the Starbucks college (sort of) plan.
That's understandable. The NCTQ report is proprietary USNews stuff, and rival organizations aren't going to do someone else's PR work for them. That reality, it should be noted, underscores the degree to which the report is not news.
Even a google of the report turns up mostly various folks debunking the whole ridiculous mess. (There's some good debunkery here and here.) So it's possible that this story may get the kind of attention it actually deserves, which is to say none except those deflating the self-inflated bubble of importance.
However, if we want to see the template for how some news outlets will inevitably be elevated to a level of importance and legitimacy that it does not deserve, we can stroll over to see what Joy Resmovits is covering it at HuffPost.
We're going to skip over the inflammatory headline-- in the world of journalism, writers generally do not write their own headlines, so it's no fair holding journalists responsible for the clickbait banners that fly above their work.
We hit our first bit of trouble in graf two: "The National Council on Teacher Quality, a Washington group that advocates tougher teacher evaluations..."
Resmovits gets a point for the verb "advocates," because NCTQ is nothing more than that. But to say that they advocate for tougher teacher evaluations is to echo their own press releases. I might as easily say that NCTQ advocates for their own ideas about what teacher evaluations should look like, or advocates for evaluations that reflect their own narrow interests. By characterizing them as "tough" Resmovits gives them a badge of legitimacy and positive focus that they do not deserve, and which incorporates a judgment about their agenda that is not really reportage.
Resmovits then summarizes their big findings, unchallenged, and lets NCTQ honcho Kate Walsh have the floor for some lengthy quotage-- again, unchallenged.
Then we throw in a graph. With "raw scores," because, you know, data. Resmovits does not address the question of where the raw scores come from. Not entirely surprising, because any positive reporting of NCTQ ratings has to pussy-foot around the question of where the numbers come from. Number of times that Common Core and Phonics are mentioned in the course catelog? Quality of department stationary? Evaluator's mood on the afternoon they were throwing numbers around? We don't know.
Resmovits does report that there is Broad and Arnold money behind the report, but again characterizes the report as aiming "to improve teacher training, modeling itself after broad reforms to medical schools a century ago." And boy, there's a sentence that just begs further explanation. I'm going to guess it doesn't mean "more teachers should train with cadavers" but beyond that, who knows. Whatever it is, we're once again going to characterize it as "improving" education rather than any number of other verbs such as "alter" or "twist" or even "take over."
Resmovits says HuffPo tried getting quotes from the bottom three schools; I could have told her that was unlikely, given that these schools are just today finding out that they (apparently) suck and have had no real chance to figure out why they got the ax. That might have been something worth noting, though one school did indicate that they had declined to cooperate with NCTQ. That's doing better than some, since NCTQ doesn't always bother to even speak to the school. A teacher school up the road from me was taken aback last year when they found themselves rated failing for a program that they don't even have. "Had anyone talked to us," the university president told me, shrugging.
But Resmovits lets Walsh bat clean-up as well, and Walsh assures us that even if it seems like everyone in public is ignoring their bogus bulk of bulbous baloney (I'm paraphrasing), behind closed doors, policymakers are getting ready to use this important information in the upcoming federal takeover of teacher prep programs.
The AACTE, teacher prep school lobby group, offers a politely worded raspberry about the report and its origins. That's in the very last paragraph.
Resmovits frustrates me. She clearly has the skill and brains to do better reportage, but invariably she fails to challenge the reformster narrative, or even offer enough details to let it be seen in all its awesome glory.
For instance, there's my favorite fun fact from the report. The top teacher prep school for secondary teachers is an on-line school. Wouldn't that be a fun lead. Wouldn't America's parents just be delighted a few years down the road to find out that their kid's teacher was certified at a cyber school.
At any rate, that's how things look just a few hours after the release of the report. Arne Duncan has kept quiet so far, as have many major media outlets. Let's see what happens as the week goes on.
Foxnews.com is headlining a list of twelve terrible teachers who are being protected by tenure. A search on USAToday.com for "nctq" produces only a press release from February of 2013. On cnn.com, a similar search turns up nothing at all. Even MSNBC, whose love for reformy things is deep and abiding, is more excited about the Starbucks college (sort of) plan.
That's understandable. The NCTQ report is proprietary USNews stuff, and rival organizations aren't going to do someone else's PR work for them. That reality, it should be noted, underscores the degree to which the report is not news.
Even a google of the report turns up mostly various folks debunking the whole ridiculous mess. (There's some good debunkery here and here.) So it's possible that this story may get the kind of attention it actually deserves, which is to say none except those deflating the self-inflated bubble of importance.
However, if we want to see the template for how some news outlets will inevitably be elevated to a level of importance and legitimacy that it does not deserve, we can stroll over to see what Joy Resmovits is covering it at HuffPost.
We're going to skip over the inflammatory headline-- in the world of journalism, writers generally do not write their own headlines, so it's no fair holding journalists responsible for the clickbait banners that fly above their work.
We hit our first bit of trouble in graf two: "The National Council on Teacher Quality, a Washington group that advocates tougher teacher evaluations..."
Resmovits gets a point for the verb "advocates," because NCTQ is nothing more than that. But to say that they advocate for tougher teacher evaluations is to echo their own press releases. I might as easily say that NCTQ advocates for their own ideas about what teacher evaluations should look like, or advocates for evaluations that reflect their own narrow interests. By characterizing them as "tough" Resmovits gives them a badge of legitimacy and positive focus that they do not deserve, and which incorporates a judgment about their agenda that is not really reportage.
Resmovits then summarizes their big findings, unchallenged, and lets NCTQ honcho Kate Walsh have the floor for some lengthy quotage-- again, unchallenged.
Then we throw in a graph. With "raw scores," because, you know, data. Resmovits does not address the question of where the raw scores come from. Not entirely surprising, because any positive reporting of NCTQ ratings has to pussy-foot around the question of where the numbers come from. Number of times that Common Core and Phonics are mentioned in the course catelog? Quality of department stationary? Evaluator's mood on the afternoon they were throwing numbers around? We don't know.
Resmovits does report that there is Broad and Arnold money behind the report, but again characterizes the report as aiming "to improve teacher training, modeling itself after broad reforms to medical schools a century ago." And boy, there's a sentence that just begs further explanation. I'm going to guess it doesn't mean "more teachers should train with cadavers" but beyond that, who knows. Whatever it is, we're once again going to characterize it as "improving" education rather than any number of other verbs such as "alter" or "twist" or even "take over."
Resmovits says HuffPo tried getting quotes from the bottom three schools; I could have told her that was unlikely, given that these schools are just today finding out that they (apparently) suck and have had no real chance to figure out why they got the ax. That might have been something worth noting, though one school did indicate that they had declined to cooperate with NCTQ. That's doing better than some, since NCTQ doesn't always bother to even speak to the school. A teacher school up the road from me was taken aback last year when they found themselves rated failing for a program that they don't even have. "Had anyone talked to us," the university president told me, shrugging.
But Resmovits lets Walsh bat clean-up as well, and Walsh assures us that even if it seems like everyone in public is ignoring their bogus bulk of bulbous baloney (I'm paraphrasing), behind closed doors, policymakers are getting ready to use this important information in the upcoming federal takeover of teacher prep programs.
The AACTE, teacher prep school lobby group, offers a politely worded raspberry about the report and its origins. That's in the very last paragraph.
Resmovits frustrates me. She clearly has the skill and brains to do better reportage, but invariably she fails to challenge the reformster narrative, or even offer enough details to let it be seen in all its awesome glory.
For instance, there's my favorite fun fact from the report. The top teacher prep school for secondary teachers is an on-line school. Wouldn't that be a fun lead. Wouldn't America's parents just be delighted a few years down the road to find out that their kid's teacher was certified at a cyber school.
At any rate, that's how things look just a few hours after the release of the report. Arne Duncan has kept quiet so far, as have many major media outlets. Let's see what happens as the week goes on.
Thursday, April 10, 2014
A Mercedes Schneider Reader
When it comes to fired-up scholarship, passionate digging out of detail, and supplying simple facts for the Resistance, it is hard to beat Mercedes Schneider.
Schneider has one of the most varied backgrounds in the field. She started out as a classroom teacher of German and English, then acquired a PhD in applied statistics and research methods from the University of Northern Colorado and moved to Indiana to teach at Ball State. But there were curves yet to negotiate; Katrina hit her family head on. She decided to go home, and took a job teaching high school English in 2007. The university let her know that professors don't get to recover from that sort of step backwards. Writes Schneider, "But I love to teach. High school, I decided, would be fine with me."
When I started wading into the school reform swamp, I realized that many of the posts that I kept returning to for facts, data, numbers, details, context and sequence-- so many of them were on her blog. Schneider has a book coming out sometime this month, and I have no doubt that it will be valuable and necessary reading for everyone who cares about public education and what is happening to it. But until it arrives, I've collected some of my favorite Schneider pieces. If you are not a regular Schneider reader, you should be, and these are a fine place to start.
Schneider has one of the most varied backgrounds in the field. She started out as a classroom teacher of German and English, then acquired a PhD in applied statistics and research methods from the University of Northern Colorado and moved to Indiana to teach at Ball State. But there were curves yet to negotiate; Katrina hit her family head on. She decided to go home, and took a job teaching high school English in 2007. The university let her know that professors don't get to recover from that sort of step backwards. Writes Schneider, "But I love to teach. High school, I decided, would be fine with me."
When I started wading into the school reform swamp, I realized that many of the posts that I kept returning to for facts, data, numbers, details, context and sequence-- so many of them were on her blog. Schneider has a book coming out sometime this month, and I have no doubt that it will be valuable and necessary reading for everyone who cares about public education and what is happening to it. But until it arrives, I've collected some of my favorite Schneider pieces. If you are not a regular Schneider reader, you should be, and these are a fine place to start.
Cheating as We Worship: The Almighty Standardized Test
A personal but still fact-loaded reflection on the many ways in which the worship of The Test leads everybody involved to cheat in a variety of creative and not-always-obvious ways.Twelve Embarrassing Years of NCLB and RTTT: Time for Arne to Blame USDOE
Schneider examines the long list of people that Arne likes to blame for the alleged problems in education, from moms to teachers to lawmakers to-- well, you name it. And then she lays out exactly how each of those attempts to shift blame is a lie.Success Academy Tax Documents: Moskowitz Can Afford the Rent
Here's an example of what Schneider does so well. While everyone else was trading allegations and rhetoric about what Eva Moskowitz could or couldn't afford and her charters did or didn't need, Schneider went to the paperwork, dug through the publicly available tax records, and published the real numbers.NCTQ Letter Grades and the Reformer Agenda–Part VIII
After this piece had run, NCTQ removed Michelle Rhee from its board of directors, but this remains one of the best dissection of Rhee's dismal career, going back to her own stories about her failed attempt to be a TFA classroom body.The Common Core Memorandum of Understanding: What a Story
Still one of the most jaw-dropping narrative-destroying pieces of investigative journalism anyone has done about the genesis of the Common Core. Schneider unearth's the memorandum of understanding that lays out who will create them and how they will be implemented. This is your go-to link for every time someone tries to tell you that the Core are the result of a state-led initiative.Gates Money and Common Core– Part VI
I'll give you part six so you can work your way backward through the links to the other sections. Schneider has done massive amounts of work tracking the Gates money and discovering where it went. Who did Gates pay off? This series lays it out for you, thanks to what had to be long and tedious research.
More on the Common Core: Achieve, Inc., and Then Some
More on the core and the roots from which it emerged, with particular attention to Achieve as well as some of the principle architects.
Beware of Data Sharing Cheerleaders Offering Webinars
An example of the kind of respect and attention that Schneider draws in the education world. Come for this column, but stay for the comments in which the some of the top scholars and business leaders of the data mining world get into a spirited discussion of the issues at hand. You could pay for a fancy webinar and not see a discussion this exciting and illuminating.
There are plenty of other classics-- some great work on the data track miners and a whole cast of reformy characters whose names you may not know. Schneider employs her tireless research and her scathing wit on each. Read these, explore some more, and get ready to grab a copy of her book the moment it becomes available.
Subscribe to:
Posts (Atom)