PDK & Marketing for the Core

One of the features of Common Core has always been the ability to market materials on a national scale. A national set of standards should help edubizes get away from having to marker fifty different sets of materials, but it only partially solves the problem of millions of individual teachers who think they have the professional expertise to think and choose for themselves.

We've already covered the creation of EdReports, a site intended to be a Consumer Reports style recommender of education materials. But here comes a puff piece in the Phi Delta Kappan that read likes the advertising insert in a glossy magazine.

"Support the Common Core with the Right Instructional Materials" authors Rachel Leifer and Denis Udall both have nifty education pedigrees. Leifer did stint with TFA in DC ("where more than 80 percent of her students advanced at least 1.5 years in academic skills annually")and is now a program officer for the Helmsley Foundation. Udall graduated from Harvard's Graduate School of Education and went on to found a charter school; these days he works for the Hewlett Foundation. So, big fans and supporters of public education.

Leifer and Udall open with an anecdote about a school in New York that used EngageNY materials and -- whoosh!-- for the first time in years "test data show that nearly every student at Ripley is making substantial learning gains." Or at least test data show that students are generating better test data. But it wouldn't be another day in Reformsterland if we didn't blithely assume that test scores = learning. The conclusion Leifer and Udall reach in this introductory anecdote is that having the right materials makes all the difference!

So advertising point one-- you need good materials.

Point number two-- the good materials are essential, but they are scarce.

Well, damn. If only there were some expert organization that could direct me to the Right Stuff!

CCSS supporters "realized early that they would need to prod the marketplace to respond to the standards." So "working with educators," the Student Achievement Partners (the non-profit profiteering group founded by CCSS writers David Coleman, Susan Pimentel and Jason Zimba) decided they would whip something up.

Instructional Materials Evaluation Tool (IMET) is a product of SAP. It is

a set of rubrics designed to support educators and administrators tasked with developing, evaluating, or buying full-year or multi-year curricula. The rubrics distill the standards into non-negotiable criteria for alignment with tangible metrics.

Doesn't that sound grand and technical and like the kind of thing you'd need experts for-- real smart experts and not just classroom teachers? And so we consider the process of suggesting that classroom teachers are not knowledgeable enough to select classroom materials. I know that's not a new idea, but the CCSS marketing plan requires it.

Educators Evaluating Quality Instructional Products (EQuIP) is from our good friends at Achieve.

This rubric evaluates a lesson or unit on four dimensions: alignment to the depth of the standards, key shifts required by the standards, instructional supports, and assessments. Scores for each dimension classify materials as exemplar, exemplar if improved, revision needed, or not ready.

Achieve has trained a boatload of teachers to use this (because, again, the poor dears certainly couldn't have mastered it on their own) and has a cadre of fifty evaluators ready to give materials a look-see and the stamp of approval (or not).

It's about here, in a small-print paragraph, that Leifer and Udall note that both of these groups being advertised here get grant money from Helmsley and Hewlett.

The author's cite two benefits of using the rubrics. First, they will create "smart demand." In other words, these rubrics are a way for the rubric designers to coach the market, to encourage the market to want what the rubric designers think the market should want. Second, the rubrics will make everyone who uses them more familiar with the Core. Presumably not in the "Now that I understand what the Core is, I do not support it" manner documented in recent polls.

Two shining lights

Leifer and Udall go on to discuss two states that have had super-duper success with this sort of thing: Louisiana (where Leifer worked for a while) and New York.

For Louisiana, they talked to John White as well as touting the use of materials from some of the same folks who helped write EngageNY's lesson plan straightjackets. At any rate, they claim that LA reviewed textbooks so rigorously that only one each for math and ELA made the grade. White is proud of judging publishers transparently. The article does not in any way address that giant regulatory clusterfinagle that is currently LA education.

In New York, we just go ahead and declare EngageNY a success, based on anecdotes from a couple of administrators. This alleged success is due to three factors:

         1) Using the EQuIP rubric real hard
         2) Training many educators
         3) Facilitating adaptions instead of requiring scripts

Because EngageNY is just famous for its lack of scripting and its enormous freedom for teachers. Which, given what I've been hearing for the last year or more, will come as real news to some folks.

The Five Main Steps

So what does it take to come up with great materials? Five steps, it turns out.

1) Build on previous efforts and existing resources. By which they mean, use the techniques that have already worked for places like Louisiana and New York.

2) Make sure educators are involved and trained. The training is important because, remember, teachers are not sufficiently knowledgeable or professional to select their own classroom materials without first being properly indoctrinated trained.

3) Have non-negotiables. In the dating world, these are called dealbreakers. In this case, it means don't try to make your own revisions to the rubrics-- if the rubric says no go, then listen tot it. Remember, teachers and principals and curriculum directors-- you are not professionals and these sorts of decisions are beyond your ability to make unaided.

4) Provide detailed feedback. To textbook companies, that is. It's your job to help them make the sale.

5) Enable teachers to supplement and adapt material on their own. By which they apparently mean to allow teachers to go to "online libraries of vetted materials" (EQuIP and SAP both have them), not actually write or adapt materials themselves. Good lord, they're only classroom teachers-- how could they possibly do that?


It's a pretty little advertising insert and really, what better message to send out to the members of a society of professional educators that they can relax, because education is in the hands of people more capable than professional educators.

7 Reasons To Send $$ To Teachout/Wu NY Campaign

Why contribute to a New York gubernatorial campaign when you don't live in New York?

Zephyr Teachout is challenging Andrew Cuomo for the Democrat position on the ballot. While a victory is unlikely, it's not impossible. And you, dear reader, are probably not even a New York resident. Here's why you should support Zephyr Teachout and Tim Wu anyway.

Send a message to the Democratic Party

The Democratic party has taken to defining their traditional constituencies as "those people whose interests we don't have to represent because they will vote for us even if we punch them directly in the nose."

"Vote for us," Democrats tell teachers. "We're the ones who like teachers." And then they punch teachers in the nose, and trash education. The Democratic party of New York is so sure they don't have to reach out to Democratic voters that Cuomo has barely pretended to be a Democrat at all. If being a Democrat were a crime, I'm not sure Cuomo could be convicted.

Bottom line-- people will take you for granted just as much as you let them. The Democratic Party needs to stop taking education voters and labor voters and not-actually-rich-guy voters for granted.

The race has national implications

See above. New York is not the only place where Democrats have decided they can turn their back on education. The Obama administration has been as anti-public education as any Bush ever dreamed of being. And since Cuomo has his eye on the White House, lessons from this governor's race are also lessons about 2016. If the lesson of this race is that being a pro-corporate, anti-public ed tool will cost you at the polls, that's a lesson that will affect any Democrat running in 2016.

Teachout doesn't have to win

It would be great if she did. And she still could. But she doesn't have to. Cuomo is supposed to be invincible, untouchable. This supposed to be a walk in the park.

This is like one of those stories in which somebody is pretending to be a god. To bring him down, you don't have to outright kill him-- you just have to make him bleed. He's already showing strain and a hint of flop sweat.

There's another race, here

Quick-- name Cuomo's running mate! Yeah, I can't, either. Tim Wu, on the other hand, is the father of net neutrality. There are two races here, and since only about twelve people in New York pay attention to the Lt. Governor race, mobilizing voters could let Wu walk off with that office, thereby handcuffing Cuomo to his own opposition.

This is what rich people do

Looking across the country and picking out people who stand for your own values is a rich person's game. Thanks to the Supremes, we don't even know how many gazillions of dollars have been spread around the country into various races, but based on what we can see, we know it's not peanuts. Rich individuals like the Koch brothers, advocacy groups like StudentsFirst-- this is what they do. Swoop in, bankroll a guy who Sees Things Your Way, and hope it helps.

So here's a chance to live like a rich person and support candidates in races you won't even vote in. Maybe your contribution will be more Grey Poupon than beluga caviar, but you can still feel fancy.

Teachout/Wu stand for the right stuff

Teachout is Not a Politician in all the best ways. She's not ignorant or naive, but savvy and knowledgeable, and she gets it. This is not a protest campaign or a stunt campaign-- this is a campaign of substance and thought. For public education fans, she sees what is going wrong with public ed in this country, but she sees it in the context of larger issues.

People are out of power now, not just in their politics where they feel that their voices don't matter, but in their workplace and in the marketplace. I want to revive the old American belief -- exemplified by Jefferson (who wanted an anti-monopoly clause in the Constitution), Teddy Roosevelt and FDR -- that concentrated private power threatens democratic institutions.

If you don't ordinarily contribute to political campaigns because it's all political crap and the candidates are all bought and paid for by Big Money anyway, here's a chance to support the kind of politician you wish were running. Here's your chance to say to The Machine, "This is the kind of thing I want to see."


Politics are a free market of sorts. Much of the market is driven by advertising and entrenched power, but the consumer always has the power to demand certain choices, and we exercise that power by standing up for those choices when they appear.

Celebrate Labor Day

I live in Pennsylvania, but I have contributed to the Teachout/Wu campaign, and I will do it again. I will not contribute much, but every reader of this blog chipped in ten or fifteen bucks, it would add up to some real money. Teachout has a real chance to make a real difference, but she is up against fully entrenched and well-financed power. There's just nine days left till the primary, and lots of phone calling, flier distributing, sign posting, and general campaigning to do.

So what better way to celebrate Labor Day than by supporting someone who is trying to put the voice of regular citizens back into the political conversation.

Click on this link, contribute some money. Do it in the next 24 hours. Step up and help out.

Why Aren't More Women in Tech? Here's One Thought...

If you need a reminder just how bad misogynistic behavior can be in our current culture, there is news this week to remind you.

Anita Sarkeesian is someone you should know. Her vlog series "Tropes vs. Women" has over the last few years dissected and explained the sexism of the video gaming world. Her videos are thoughtful and scholarly. Situated on the website Feminist Frequency, they provide an intelligent, considered look at how the tropes and traditions of video games reinforce some of the worst sexist attitudes of the culture. Sarkeesian's presentations are calm, clear, and non-ranty; were it not for the subject matter, you could easily imagine her videos running on PBS.

So, of course, she has received constant threats. The level of harassment is stunning, a degree of ugliness that makes you want to just powerwash your computer after you look at it. Beyond the standard fare of trolling (known in the meat world as Being an Asshole) Sarkeesian has been the threatened with violence, rape and death. Trolls created a video game "Beat Up Anita Sarkeesian" and sent her screen shots of herself being assaulted. All this, I'll remind you, because she is a woman daring to point out sexism in video games. Video games.

Sarkeesian has put up with this since Day One, but last week, things took a turn for the even worse when someone let her know that they had tracked down her home address, as well as the names and address of her parents, and threatened to kill them all. Sarkeesian called the police and moved out of her home.

When discussing the shortage of women in the tech industry, it's standard to observe that the frat boy atmosphere can make them feel unwelcome. Frankly, saying that the tech industry makes women feel unwelcome is like saying being mugged makes people feel uncomfortable. We have example after example-- the launching of the app "titstare," the tinder lawsuit, the endless tales of Comic Con misbehavior. And every example of tech world sexism and harassment comes with its own second helping of "How dare you call us sexist, you ugly woman who probably can't get laid."

The culture in general and teachers in particular have got to update our image of what sexism looks like. The classic sexist stereotype (macho, strutting, physically powerful and confident) is being replaced by a new harasser-- the smart guy using his techy device to blast his ideas and images out into the world, never having to even look his victim in the face, and demanding that no women enter his domain without submitting to him.

It doesn't have to be that way. My son-in-law works in the tech industry, and he and my daughter make a fine feminist couple. But somehow we've raised a whole host of young men who think that it's okay to threaten women with rape and death.

I've taught Kate Chopin's The Awakening for years, and it always sparks some important and revealing conversations among my students. But lately I'm feeling that a discussion of the subtle and powerful ways in which society can pressure women into certain roles hardly prepares us for a world in which women who dare to call out little boys for their misogyny can expect relentless threats of death and rape-- how the hell did we end up with that world?? I should have done it before now, but this year we'll be upping our game when it comes to talking about gender in American lit.

One thing's for sure-- in that world, providing women with access to STEM education is only a tiny part of the solution. We like to think of ourselves as far more advanced than countries where women are threatened, harassed and assaulted for trying to get an education. I wish we were separated from that kind of thinking by a far larger gulf than we apparently are.

NEA Gets It Wrong Again

I would probably not pick at NEA so much, were it not that I am paying them enough money each year to buy myself a nice toy, support a local food pantry, or get a bit ahead on paying off my kids' college loans. But never have I forked over so much money for the privilege of belonging to an organization that annots me so much.

I say all that because I'm about to bitch about something that, on its face, is a trivial complaint. And yet, it seems completely symptomatic of NEA's problems as an organization.

President-elect Lily Eskelsen Garcia used to have a blog; a nice personal site where she infrequently posted. But hey-- she was actually using the internet, which seems to be a technological leap that the NEA leadership is largely unwilling to make. When it comes to technology, this is not your father's NEA-- it's your grandfather's. The NEA continues to closely resemble the GOP of the last two national elections-- they know that the young people are out there playing with their twitters and using those interwebs, but they can't seem to think of anything to do with the technology except either lock it up tight or use it to make cyber-versions of print magazines and glossy brochures.

So LEG's blog was a nice touch, threatening to bring the NEA up to at least a decade ago. But now the blog is gone.

The title is still in place ("Lily's Blackboard") but the website has been replaced with a slick, glossy, cold, corporate website resplendent with press releases and articles. Logos and links mark it clearly as part of the family of bloodless NEA websites. Her old posts have been warehoused in a special category, a section of the website set apart, I guess, for when LEG might actually write something herself. Wowee.

The NEA corporate communication guys seem to have realized on some level that LEG's personal touch is part of her appeal, so they not only kept her original title, but they put it in one of those cute fonts that's supposed to look handwritten.

But one of NEA's problems continues to be a culture at the top levels that comes across as detached, disconnected, aloof, cold, corporate, and far away from the world of classrooms. I'm always left with the impression that my union leaders are far more comfortable talking to an Arne Duncan or a Bill Gates than to an actual real live teacher.

So when you elect a new president whose signature strength is an ability to communicate personally and powerfully with people, it seems kind of dopey to surround her with a bunch of virtual handlers and whisk her off to the boardroom so that the executive assistants can start communicating through press releases.

She has too many important things to do? Bullshit. What does the president of NEA have to do that's more important than communicate with her members? And if you know what you're doing, maintaining a blog or twitter account or social media presence does not take dozens of hours a week.

The bottom line appears to be that the NEA simply doesn't know how to make social media work. Their GPS network, an attempt at running a bulletin board system to discuss and share education materials and topics is still a slick, glossy ghost town. And while I was initially excited that LEG had a twitter account, NEA seems to have no idea what to do with it beyond PR blurbs.

Look, folks. Age is not an excuse. I'm fifty-seven years old. My first lessons in computer programming involve BASIC and punch cards. And yet I seem to have figured out some basics here and there. And here's the most basic thing to get about social media-- it is a way to communicate with people, not to manage them. Lily's New Blackboard is a bummer, because it replaces something that could have been personal and effective with one more piece of shiny impersonal plastic. Wrong again, NEA.

First Loser in NY Anti-Tenure Lawsuit

In the end, there can be only one.

Even though it's all about the kids, according to the NY Post only one group gets to stand center stage for the big New York Tenure Takedown Lawsuit.

Previously the Campbell Brown nameplate case was going to have to share the big Caring for Kids spotlight with Mona Davids (of New York Parents Union), but Davids is now saying, none too cheerfully, that her law firm has dropped the case. Davids charges that the firm was chased off by "bullying" tactics by Brown and her camp, but Davids also announced that her lawsuit will continue. So this fight to be the Big Name in New York anti-tenure lawsuits is not over yet.

According to "multiple sources" the firm Gibson Dunn actually pulled out because they have education clients who didn't take kindly to Davids working with the firm. Brown's lawsuit is, of course, is being handled by several big-name lawyers and PR flacks with close ties to the Obama administration, who don't much care if they bother any people in the education world or not.

Students Matter, the Vergara-sponsoring advocacy group composed primarily of David Welch and his giant pile of money, has also withdrawn from Davids' suit.

No news yet about a press conference in which Campbell and Davids stand together and announce that since they are only concerned about what's best for the children and not who gets to be Really Important, they're putting aside all their competition for the best position in front of the camera.

UPDATE: Eclectablog has a more detailed account and timeline of how Brown et al squashed these pretenders to the thrown and competitors for the spotlight like bugs.

TNTP's Teacher Pay Proposal Is Dopey

I picked the title because I didn't want to make you wade through this whole post just to get to my bottom line. That's one thing I learned from reading the TNTP report "Shortchanged: The Hidden Cost of Lockstep Teacher Pay" The other thing I learned is that you can't be afraid to recycle the same old shinola with a spiffy new spin. I can promise you that you probably won't find a thing in this post that I haven't said before. But I can be just as relentless as TNTP. So let's crack open this report and see what creamy nougat is inside.

Introduction

Money is really important. Followed by an abstract of what's to come.

Why lockstep pay doesn't work

Because it doesn't include any punishments for being bad or rewards for being good. And by rewards, we mean money. Because why else would anybody want to do a good job, except to get more money?

From here the nameless authors conclude that school districts are wasting buskets of money paying for crappy teachers.

Low Entry-Level Salaries Keep People from Considering Teaching

Pay attention, because here TNTP starts to show their hand. We should offer more money to starting teachers so they will choose teaching. Part of the problem-- we're paying too much money to high-step teachers. We should move that money down to the bottom of the scale. Though they also offer a chart showing that teacher salaries rise too slowly compared to other fields.

Great teachers feel pressure to leave classroom while lousy ones feel pressure to stay

Okay-- the great teachers will be downtrodden and discouraged because they don't get recognition and more money that the craptastic guy next door, while Mr. Craptastic will stick around forever for the paycheck. So...let me think this through. The great teacher is not motivated by the paycheck, and that's why he needs a bigger one? The guy who doesn't like his job feels highly motivated to keep showing up for the job that he hates?

It doesn't exactly scan, but maybe if we threw in some dubious statistics.

The amount of taxpayer money that goes toward rewarding poor teaching is staggering. Last year, schools in the U.S. spent a conservative estimate of $250 million giving pay increases to teachers identified by their districts as ineffective.

"Estimate" is a generous word to use here for this mostly made-up number. Or this:

This goes a long way toward explaining why 75 percent of low-performing teachers remain at the same school from one year to the next, and half say they plan to remain a teacher for at least another decade.

Well, this one has a source, anyway. A previous paper by TNTP. They finish with their real point, which is that it is annoying as hell that good teachers and bad teachers might be paid the same.

The Best Teachers Aren't Recognized for Leading the Classrooms Where They Are Needed 
Most

The assertion here is that people in different settings working with different populations are paid the same amount. The truth of which depends an awful lot on your location. But then, the value of your pay varies with location, too. I don't make huge dollars compared to folks teaching in, say, Pittsburgh, but I live in a nice 10 room house with a finished basement and two baths, residential neighborhood, large yard which butts up against a river that I paid a five-figure price for. I'm not sure they have any point here except that it's hard to compare apples and anteaters.

What's the return on a Master's Degree?

Teachers have a financial incentive to pursue advanced degrees. Some advanced degrees are not awesome, yet teachers still get paid more for having them. TNTP thinks those raises should not happen.

A Roadmap for Building Smarter Compensation Systems

This is a three-step process. More pay for beginners. Offer raises for strong performance (and nothing else). Give incentives for working in tough areas. Let's take a closer look at each, shall we?

Competitive Early Career Salaries

One cute idea-- sign-on bonuses that you have to be vested to get. Otherwise, they want high starting, and a scale that goes up quickly, within, say, the first five years. Six figures in six years is their idea. They do get one thing wrong in asserting that in most districts it takes a whole career to get to the top. Unions figured out that problem ages ago and compacted schedules. For instance, in my district you get to the top of the pay scale in about thirteen years.

I can tell you the problem with that. People at the top of the scale sometimes get testy about having fairly stagnant wages while young folks are getting big raises. Of course, you have to stick around for over a decade for that to matter. Hmmmm....

Merit Based Raises

Nameless Author carefully avoids the M word, but that's what they want-- raises based on performance, with little or no increase for longevity. Their proposal does not include the exact wording of a pitch where schools say, "Come work for us, and you may or may not get a raise, based on measures that we haven't perfected and which you can't affect."

Incentives for High Needs Schools

Do that. But don't give money to bad teachers. Just the good ones. Since high needs schools will always have low test scores, proving that the teachers there are ineffective, you will never have to give anyone a raise, ever. Score!!

Is Performance Based Pay Affordable?

Now, eighteen pages in, we get to the heart of the matter. I've argued over and over that merit pay will never work because in education, you cannot grow the pie-- you can only slice it thinner and thinner. Here's the TNTP answer

It does, however, require that school systems make a fundamental choice: Do they want to pay for years of service, advanced degrees and everything else they are currently buying with teacher salaries? Or do they want to pay for great teaching? Doing both is not an affordable option.

But paying those six-figure salaries year after year to 100% of your teaching staff is not an option either. So they are correct-- you can have this model, or you can have longevity. Their model works as long as you keep churning and turning over your staff. That's been the argument for the first eighteen pages-- TNTP doesn't want to look at career earnings, but just the first five or six years, because not only do they not care if you stick around longer, but they need you to get the heck out so that they can repurpose your salary to pay for two newbies.

And that is how we use the argument of better teacher pay to drive the McDonaldization of education. As long as we can churn and burn, this model is sustainable. But it cannot sustain a school full of lifers who all make top dollar all the time.

Examples

The paper goes on to offer some studies of school districts that have done some version of what they propose. What can I say, except that the first example is Newark. Newark. Followed by Bridgeport's Academy school. Followed by Louisiana, Tennessee, Indiana and Florida as examples of state-led salary reform. If you had somehow still been taking them seriously up until this point, you may now get off that train.

They also have ideas about how to implement this smart compensation program, finishing with the motto "Give great teaching the compensation it deserves." And that certainly sings with ambiguity, doesn't it.

It's the same old bad ideas constructed to model the charter/TFA model of schools as dispensers of speedy education product, staffed by temps who are just passing through and cashing in on their way. It's a dark and dopey vision of education, no matter how many pretty graphics you stick on the page.

Spell Check & Educrats

A few days ago I tried composing a post on my tablet. I like the equipment; it's new and shiny and it lets me get things done while sitting on the couch with my wife instead of hunched over my desk.

But for whatever reason (no doubt a setting that I haven't located and changed yet) it is an aggressive fixer of my spelling. And my battles with the spell checker remind me of the role that technocrats have tried to play in education.

Spell check seems like such a helpful idea in theory. Whether you mistype or mis-spell, the power of the computer will correct you, help you get things right. Except that instead of helping you get things right, it helps you write things that the programmer judges as right. And here we hit trouble.

Granted, the only thing worse than my spelling is my typing. I have a cadre of loyal readers who regularly direct my attention to mistakes I've typed. The process there is that they send me a message about what they've spotted, I check it, and if need be, I fix it.

This is different than the process of spell-checking, in which the computer program substitutes its judgment for my own without asking. In the case of my tablet, it is so insistent that even when I think I have overruled it, I find out later that it simply changed things back. It's not trying to be oppressive. I have no doubt that the software writers felt that they were offering a helpful feature, that this overruling of my judgment was for my own good. But it does not need to know me, meet me, even make an attempt to understand what I'm trying to say. The content of my writing doesn't matter; I will be assimilated.

Technocrats dream this dream a great deal. They dream of an elegant system, a perfectly produced piece of software that will make human judgment (so messy, so flawed) unnecessary. So Google tries to finish my words for me even as they work on a car that will drive for me.

It's a weird warping of time. It feels as if the software overrides my judgment right now, but of course the decisions that created that moment were made by programmers a while ago, long before they would even know who would be using their program. They could never have known that I like to bend and twist words, make new illegitimate ones. They could never have known that I think one of life's great little moments is when somebody puts together a sentence that has never been spoken or written before.

It also makes subtle value judgments. Most spell checkers assume that people either never would or never should use words like shit, and never, ever drop the F bomb. So spell checkers are also politeness checkers.

You can see all of these drives in the technocrat approach to education reform. The dream is a system so smooth and uniform that it can be implemented anywhere and, more importantly, the people who are actually pressing the buttons don't ever use their personal judgment because the system renders their judgment unnecessary. Teachers and students should be able to just boot things up and step back and wait for their prompt. Their individual qualities and preferences shouldn't matter. When the system and the people clash, the system is keyed to simply override the humans. Human judgment as exercised by humans, after all, is messy and sloppy and unpredictable; it reacts to too much, is steered too much by its senses and surroundings. On the other hand, human judgment enshrined in the form of software is solid and unchanging and smooth and unaware of anything except its own directives.

When live humans and software collide, technocrats blame the humans for not getting in harmony with the system. I can tolerate that in my word processing, but not in my school. Or to speak in programmer terms, human chaos and responsiveness to the world, the tendency to change and grow and change some more in relationship with the richness of the world and other humans-- these are not bugs. They are features. They are the whole point.

Bellwether Flubs Teacher Evaluation Argument

I am fascinated by the concept of think tank papers, because they are so fancy in presentation, but so fanceless in content. I mean, heck-- all I need to do is give myself a slick name and put any one of these blog posts into a fancy pdf format with some professional looking graphic swoops, and I would be releasing a paper every day.

Bellwether Education, a thinky tank with connections to the standards-loving side of the conservative reformster world, has just released a paper on the state of teacher evaluation in the US. "Teacher Evaluation in an Era of Rapid Change: From 'Unsatisfactory' to 'Needs Improvement.'" (Ha! I see what you did there.) Will you be surprised to discover that the research was funded by the Bill and Melinda Gates Foundation?

In this paper, Chad Aldeman (Associate Partner on the Policy and Thought Leadership Team-- his mom must be proud) and Carolyn Chuong (Analyst-- keep plugging, kiddo) lay out what they see as current trends which they evaluate in terms of what they think the trends should be. So, see? A smattering of factish information all filtered through a set of personal ideas about how education should be going-- just like me! Let's see what they came up with.

The Widget Effect

Oh, this damn thing. You can go back and read the original TNTP paper, which was earthshattering and galvanized governments to leap up and start using a new piece of jargon. Just in case you missed it, the whole point was that school systems should not treat teachers as if they are interchangeable widgets, but instead should treat them as interchangeable widgets, some of which do widgetty things better than others. In other words, under this approach, all teachers are still widgets in a big machine; it's just that some widgets are better than others. But this theoretical thought-leadery framework is still influential today in the sense that it influenced this paper that I'm reading and you are reading about.

So what did Aldeman and Chuong find? Five things, it turns out. Here they are.

1) Districts are starting to evaluate teachers as individuals.

The "most dramatic finding" in The Widget Effect was that school districts were using binary pass/fail. Now states are moving toward a four- or five-tiered system. Woot!

Some people, apparently, quibble because the new system still finds a small percentage of teachers are in the suck zone, and for many reformsters, a teacher eval system is only good if it finds the gazillions of bad teachers that reformsters just know are out there. But Aldeman and Chuong say that criticism misses two points.

First, they say, don't look at the percent-- look at the number. See how high that number is? That's lots of bad teachers, isn't it. Also, they cite the New York report about tenure rule changes. They think the research says that if you're a bad teacher and your administration says so, you might leave. I think the research also says that if you're a good teacher and your boss gives you a bad evaluation, you might think twice about wanting to work for that boss. But here, as throughout, we will see that the question "Is the evaluation accurate" never appears on the radar.

Second, did we mention there are more than two categories. And the categories are named with words, and the words are very descriptive. That allows us to give targeted support, which we totally could never do under the old system, because-- I don't know.  Principals are dopes and the evaluation rating is the one and only source of data they have about a teacher's job performance?

2) Schools are providing teachers with better, timelier feedback on their practice.

There's no question that this is a need. Traditional evaluations in many states involved getting a quick score sheet as part of a teacher's end-of-the-year check-out process. Not exactly useful in terms of improving practices.

But in this section the writers come close to acknowledging the central problem-- the ineffectiveness of the actual evaluation. They note that research shows that teachers with higher-functioning students tend to get better evaluations.

However, they correctly note that new evaluation techniques encourage a more thorough and useful dialogue between the teacher and the administrator. But, of course, the new evaluation system are based on the same old true (and only) requirement-- certain paperwork must be filled out. The new models put huge time requirements on principals who still have a school to run, and the pressure to the letter of the paperwork law met while trampling the spirit are intense. We'll see how that actually works out.

3) Districts still don't factor student growth into teacher evals

Here we find the technocrat blind faith in data rearing its eyeless head again

While raw student achievement metrics are biased—in favor of students from privileged backgrounds with more educational resources—student growth measures adjust for these incoming characteristics by focusing only on knowledge acquired over the course of a school year.

This is a nice, and inaccurate, way to describe VAM, a statistical tool that has now been discredited more times than Donald Trump's political acumen. But some folks still insist that if we take very narrow standardized test results and run them through an incoherent number-crunching, the numbers we end up with represent useful objective data. They don't. We start with standardized tests, which are not objective, and run them through various inaccurate variable-adjusting programs (which are not objective), and come up with a number that is crap. The authors note that there are three types of pushback to using said crap.

Refuse. California has been requiring some version of this for decades. and many districts, including some of the biggest, simply refuse to do it.

Delay. A time-honored technique in education, known as Wait This New Foolishness Out Until It Is Replaced By The Next Silly Thing. It persists because it works so often. 

Obscure. Many districts are using loopholes and slack to find ways to substitute administrative judgment for the Rule of Data. They present Delaware as an example of how futzing around has polluted the process and buttress that with a chart that shows statewide math score growth dropping while teacher eval scores remain the same.

Uniformly high ratings on classroom observations, regardless of how much students learn, suggest a continued disconnect between how much students grow and the effectiveness of their teachers.

Maybe. Or maybe it shows that the data about student growth is not valid.

They also present Florida as an example of similar futzing. This time they note that neighboring districts have different distributions of ratings. This somehow leads them to conclude that administrators aren't properly incorporating student data into evaluations.

In neither state's case do they address the correct way to use math scores to evaluate history and music teachers.

4) Districts have wide discretion

Their point here is simply that people who worry about the state (and federal) government using One Size Fits All to intrude local autonomy into oblivion are "premature" in their concern. "Premature" is a great word here, indicating that the total control hasn't happened yet-- it's just going to happens later.

5) Districts continue to ignore performance when making decisions about teachers

Let me be clear. I used the heading of this section exactly as Adelman and Chuong wrote it, because it so completely captures a blind spot in this brand of reformster thought.

Look at that again, guys. Is that really what you meant to say? Districts completely ignore performance when making decisions about teachers? Administrators say to each other, "Let's make our decisions about staff based on hair color or height or shoe size, but whatever we do, let's not consider any teacher's job performance ever, at all."

No, that would be stupid. What Adelman and Chuong really mean is that districts continue to ignore the kind of performance measures that Adelman and Chuong believe they should not ignore. Administrators insist on using their own professional judgment instead of relying on state-issued, VAM-infested, numbly numbery, one-size-measures-all widget wizardy evaluation instruments. Of course districts make decisions about teachers based on job performance; just not the way Adelman and Chuong want them to.

Also, districts aren't rushing to use these great evaluation tools to install merit pay or to crush FILO. They are going to beat the same old StudentsFirst anti-tenure drum. I have addressed this business at great length here and here and here and here (or you can click on the tenure tag above), but let me do the short version-- you do not retain and recruit great teachers by making their continued pay and employment dependent on an evaluation system that is no more reliable than a blind dart player throwing backhand from a wave-tossed dinghy.

Recommendations

It's not a fancy-pants thinky tank paper until you tell people what you think they should do. So Adelman and Chuong have some ideas for policymakers.

Track data on various parts of new systems. Because the only thing better than bad data is really large collections of bad data. And nothing says Big Brother like a large centralized data bank.

Investigate with local districts the source of evaluation disparities. Find out if there are real functional differences, or the data just reflect philosophical differences. Then wipe those differences out. "Introducing smart timelines for action, multiple evaluation measures including student growth, requirements for data quality, and a policy to use confidence intervals in the case of student growth measures could all protect districts and educators that set ambitious goals."

Don't quit before the medicine has a chance to work. Adelman and Chuong are, for instance, cheesed that the USED postponed the use of evaluation data on teachers until 2018, because those evaluations were going to totally work, eventually, somehow.

Don't be afraid to do lots of reformy things at once. It'll be swell.

Their conclusion

Stay the course. Hang tough. Use data to make teacher decisions. Reform fatigue is setting in, but don't be wimps.

My conclusion

I have never doubted for a moment that the teacher evaluation system can be improved. But this nifty paper sidesteps two huge issues.

First, no evaluation system will ever be administrator-proof. Attempting to provide more oversight will actually reduce effectiveness, because more oversight = more paperwork, and more paperwork means that the task shifts from "do the job well" to "fill out the paperwork the right way" which is easy to fake.

Second, the evaluation system only works if the evaluation system actually measures what it purports to measure. The current "new" systems in place across the country do not do that. Linkage to student data is spectacularly weak. We start with tests that claim to measure the full breadth and quality of students' education; they do not. Then we attempt to create a link between those test results and teacher effectiveness, and that simply hasn't happened yet. VAM attempted to hide that problem behind a heavy fog bank, but the smoke is clearing and it is clear that VAM is hugely invalid.

So, having an argument about how to best make use of teacher evaluation data based on student achievement is like trying to decide which Chicago restaurant to eat supper at when you are still stranded in Tallahassee in a car with no wheels. This is not the cart before the horse. This is the cart before the horse has even been born.

Expecting Less Than Excellence

Most teachers have heard it in the last year or two. It is apparently hardwired into all administrative training about new evaluation methods.

You will not live in Excellent (or above average or super-duper proficient or whatever language your state prefers). You will only visit. You will live in Mostly Pretty Okay (or whatever).

Imagine if we started out the year by telling our students, "You'll only get a couple of A's this year. You are never going to excel. You will only be mostly pretty okay the majority of the time." And you'll have to imagine it, because who would actually say that?! Not any Mostly Pretty Okay teacher, because we know that expectations matter. I tell my students every year that we are shooting for awesome. I tell them a gajillion times they can do this and they will be great. Because expectations matter.

Even Arne Duncan believes in expectations, to the point of imagining that great expectations can cure students of any disabilities they might have.

But for some bizarre reason, the US has adopted an approach to teacher evaluation that starts with the premise that the teaching staff will be usually Mostly Pretty Okay and rarely Great. How does that expectation lead us to excellence?

Districts that are operating with some sort of merit pay system only make matters worse. They can't afford-- literally cannot financially afford-- to have a staff of uniformly excellent teachers because they don't have the money to pay them all big-time quality pay. So those districts have an actual financial incentive to make sure that their teaching staff is Mostly Pretty Okay.

And so we flounder on in upside-down education world, where we talk about the need to foster and promote excellence in teaching while we structure the system to avoid and smother excellence. It's a reverse emperor's new clothes-- teachers appear clothed in excellence and the emperor insists that they are naked. The good news for students is that teachers will continue to produce excellence whether anybody in power claims they can see it or not.

Mike Petrilli Interprets Reform Backlash (Part 1)

This week Mike Petrilli took a stab at interpreting some of the pushback on reformster programs in what we can hope is a step in his journey to a more enlightened opinion. The column is actually an excerpt from a speech that he delivered to the Oklahoma City Chamber of Commerce, which is interesting in that Chambers have not generally have not been open to a message of "Hey, we might have gotten a few things wrong on this ed reform thing."

More than a marketing miscalculation

Petrilli, to his credit, is not here to explain why those who are pushing back are deluded and wrong. Instead, he's asking reformsters to take a look in the mirror.

If we’re going to succeed over the long haul, we need to take a hard look not just at how we’re selling, but also at what we’re selling. We need to look at our reform agenda and ask ourselves: Is it working? Do the pieces fit well together? Does it diagnose the problem correctly and offer the right cures?

This is where we’ve made our biggest mistakes: getting the diagnosis wrong. Specifically, we have diagnosed all of our schools as having the same disease, and prescribed the same medicine for all of them.

Petrilli's first piece of analysis is that reformsters over-stated their case, suggesting that the US landscape was peppered with failing schools and a national system that needed to be creatively disrupted into oblivion and beyond. But Petrilli knows that his own kids go to a decent public school, and he suspects that most well-to-do parents feel the same. So when reformsters started to threaten to blow up those schools and build oppressive testing factories on top of the rubble, parents became cranky. (This is not a new insight for Petrilli-- check out Fordham's House of Cards parody in which Petrilli dupes a clueless Secretary of Education into shooting himself in the foot with a "white suburban moms" gaffe). No word on why low income parents also became cranky.

Mediocrity on the march

Petrilli's larger move in this piece is to downgrade the State of our Schools threat level from Defcon 2 to Defcon 4. It's not that all US public ed is a massive pile of disaster circling a great inexorable suck. No, US public ed is just mostly mediocre. There are very few great schools. That's our new Big Problem in Education.

How can we make sure that every professional in our building is excellent, always improving, and giving 110 percent?


I agree that "always improving" is a goal to shoot for. But Petrilli should know better than the 110% line, which is the pep talk equivalent of an amp that goes up to 11. No human being will ever "give 110%"

Most of us are now teaching under an evaluation system in which we are routinely cautioned that we won't live in "outstanding," but will only visit it occasionally, like a really expensive time-share that actually belongs to a rich uncle. We will live, we're repeatedly told, in "just pretty okay enough."

And Petrilli does dance around some of the definition and explanation of the alleged mediocrity. He suggests, for instance, that his son's elementary school has been neglecting history and science because they are complacent. I know plenty of schools that have cut back on science and history, and it has a lot less to do with complacency than with Not-on-the-test-itis.

The issue of challenging students is also a tricky one-- plenty of students will choose a comfortable A over a challenging B, and without any push from home, it's pretty hard to change that. I teach the most challenging class for juniors in my department, and every year, plenty of students choose not to be challenged. Not sure that's an indicator of school mediocrity.

Petrilli still believes that CCSS and "rigorous, aligned tests" are a solution. I remain convinced they are now part of the problem. But we do agree on this, with one exception:

What’s not a good fit for these middle class schools are policies that take power away from local school boards and local educators, such as a mandatory state curriculum or a formulaic system to evaluate teachers using a template created by a far-away state bureaucrat, and one that encourages teaching to the test.

Note that Petrilli says this is bad policy for middle class schools. I think it's bad policy in any school.

A two tiered system

On the one hand, Petrilli now makes a point that is rather huge coming from a reformster. He moves on to talking about high-poverty schools, and he says this:

From my experience, and from my examination of the data, most of even these schools are not “failing.” ... But on the whole, high poverty schools tend to be no better and no worse than the average school in the affluent suburbs. Their teachers work just as hard, the curriculum and methods they use are much the same.

So, high-poverty schools are not the victims of substandard staff and terrible teaching. Good to hear it.

But this takes us to the heart of Petrilli's point, and it's a dangerous and difficult point to address. Basically, here's how the argument breaks down. US public schools are mediocre. Middle-and-higher class students will be okay anyway, because they have access to resources that will get them where they want to go. But students from high-poverty schools can't settle for mediocre, because poverty puts them at too much of a disadvantage-- a disadvantage that schools have to make up for.

There are two ways (at least) to read this argument. The exceptionally bold one would be to read it as an argument that we should be focusing resources on high-poverty districts to ensure that those students have the best schools in the country. That would be awesome, but hard to sell, because there's no way to get around the reality that such a refocusing means collecting tax dollars from the well-to-do and pumping them into poverty-stricken schools.

The not-so-bold way to read this argument is that only poor students should have to suffer through all the reformster crap. Middle and upper class kids can have the school system their parents want for them, and poor kids can get the school system that bureaucrats and reformsters decide they should have in order to make up for their many failings.

So, which door will we choose

If you're wondering which reading Petrilli is advocating, take a look at this close-to-the-conclusion graf:

The most excellent urban schools in the country tend to be high performing No-Excuses charter schools that have the freedom and drive to obsess about excellence every day, to ensure that every adult in the building is top-notch and giving his or her all, to uphold high standards for student behavior and effort, and to create a culture of success. I’m doubtful that big bureaucratic districts can replicate that kind of school, and for that reason I think most big cities are going, ten or twenty years from now, to have systems dominated by charter schools, instead of school systems as we know them today. And if we can get the policies right and the accountability piece right, our kids will be better off for it.

So, charters. There are a few problems with Petrilli's solution.

First, the "success" of charters (whether they allow excuses or not) has been repeatedly shown to be illusory. Any public, private or charter school can make great numbers as long as they have the power to rid themselves of every under-achieving student.

Second, I agree that big bureaucratic districts are at a distinct disadvantage. But it's becoming rapidly clear that the typical charter of tomorrow (and probably today) is, in fact, part of a big corporate bureaucracy larger than any single school district. K12 is just one example of how the real money in charters is in massive scalability. Charters are going to be just about as nimble and responsive as the phone company.

Waiting for Part 2

So Petrilli has some new insights and ideas, and some of them are admirable and welcome, but they seem to be leading him to an old conclusion, a vision of districts where charters run most of the schooling, but public schools are still kept around because all those students who are run out of No Excuses charters have to be stuck somewhere (thereby keeping public schools in a perpetual state of failure).

I welcome Petrilli's evolution, and his willingness to consider the reformster need to look, not at their marketing, but at their product. I'm just hoping that step #2 in this journey of a thousand miles is forthcoming.

PARCC and Test Prep

When PARCC wrote a press release earlier this month addressing some testing concerns, most commentators focused on this quote:

“High quality assessments go hand-in-hand with high quality instruction based, on high quality standards,” said Laura Slover, the Chief Executive Officer of the PARCC nonprofit. “You cannot have one without the other. The PARCC states see quality assessments as a part of instruction, not a break from instruction.”

It's a noteworthy quote, given its baldfaced admission that testing, curriculum, and the standards are all of a piece. The connection between all the parts is not a surprise, but reformsters rarely divert from the standard story of separate lives for their reformy wares.

But plenty of folks have hit that point, and I don't want you to miss this quote that appears a bit further down the page.

“The PARCC assessment system is a new way of testing that reduces time spent on ‘test prep,’ because the only way to prepare for these more sophisticated assessments is through good teaching and learning all year long," Slover said.

That, in one short sentence, attempts the bank shot of changing the definitions of both test prep and good teaching by throwing them into a big cauldron in which they can smush together.

"Test prep" is just "good teaching," and "good teaching" is that which gets students ready for the test. Which is of course what we usually call "test prep."

Law Professor Says Duncan's Actions Un-Constitutional

An upcoming article in the Vanderbilt Law Review argues that the administration's waiver program is both illegal and a very, very bad precedent. University of South Carolina law professor Derek W. Black has written articles about the intersection of federal power and school law before, but none quite as feisty as "Federalizing Education By Waiver." And folks have questioned the legality of Duncan's waivers all along, but this takes that game to a whole new level.

Black opens with one of the most concise summaries of the current reformster wave you'll ever see

Two of the most significant events in the history of public education occurred over the last year. First, after two centuries of local control and variation, states adopted a national curriculum. Second, states changed the way they would evaluate and retain teachers, significantly altering teachers’ most revered right, tenure. Not all states adopted these changes of their own free will. The changes were the result of the United States Secretary of Education exercising unprecedented agency power in the midst of an educational crisis: the impending failure of almost all of the nation’s schools under the
No Child Left Behind Act (NCLB). The Secretary invoked the power to impose new conditions on states in exchange for waiving their obligations under NCLB....As a practical matter, he federalized
education in just a few short months.

This allows the kibbitzing to start immediately in response. Black does not distinguish at all between Common Core Standards and a national curriculum, a distinction without a difference that reformsters have fought hard to maintain. Nor will reformsters care for the assertion that states did not all adopt reform measures of their own free will. But all of that background in the first paragraph of the article is simply setting the stage for Black's main point.

This unilateral action is remarkable not only for education, but from a constitutional balance-of-power perspective. ... Yet, as efficacious as unilateral action through statutory waiver might be, it is unconstitutional absent carefully crafted legislative authority. Secretary Duncan lacked that authority. Thus, the federalization of education through conditional waivers was momentous, but unconstitutional. [emphasis mine]

I should note that all of his material comes heavily laden with footnotes.

There follows a more detailed recap of the Tale of NCLB and Creeping Federalism. 

Once upon a time, Congress created NCLB which kept the line on states' rights by making the states accountable for educational results. Scholars called it "cooperative federalism" and it was a new role for the feds, but a limited one. But NCLB was flawed, and as early as 2008, Congress and the President were looking to stop the train. The President proposed a fix in 2010, but Congress was not having it. However, NCLB came with its own magic beans-- the Sec of Ed had the power to waive noncompliance consequences for the states.

The Sec of Ed broke out the magic beans, but he said they will come at a price-- a price, it turns out, remarkably identical to the 2010 proposal ideas. And here's the thing about that 2010 blueprint-- it was proposed as a way to take education in a completely different direction, away from NCLB.

That means that the waiver requirements were decidedly NOT an outgrowth of the underlying legislation, and were in fact meant to bury it, not to save it. That means that waivers took us into a magical new land of Not Actually Legal.

Specifically, it assaults the magical fairies of the Spending Clause in these two ways. 1) You can't use federal money to change the rules that the money is attached to and 2) Congress can't use federally funded programs to coerce states into adopting federal policies. And so it's time to break out the magic wands and zap some naughty federal fairies.

There's not a lot of scholarship about this web of fine legal detail, and so Black sets out to fill in the gap with four very erudite and legally sections of the article. I am going to summarize them in very non-legally ways, and any damage I do to Black's arguments is on me, not him.

PART I: No Changing the Rules

When the feds pass a law, they have to lay out all the rules that do and will apply to that law. You can't pass a law, start folks working under it, and then years later announce, "Oh, yeah, and by the way, we've changed this law about making cheese sandwiches so that it also covers sloppy joes, and also, if you don't go along with us on this, we get to take your car." Also, you can't suddenly say, "We've given my brother-in-law the power to judge your sloppy joes."

Conditions for receiving federal fund must be "unambiguous" and non-coercive. Also, you can't suddenly delegate Congressional authority to an agency of the Executive branch.

There's not a lot of constitutional case law related to waivers, but Black is pretty sure that insufficient notice of waiver conditions as well as "leverage and surprise at the point of waiver" (wasn't that an Alan Parson's Project album?) are problematic. Even more problematic is the issue of an agency of the federal government using waiver conditions to rewrite laws passed by Congress. And then he takes a few pages to explain how these issues should be navigated.

PART II: Using NCLB Waiver To Impose New Policy

If you're going to understand why this was bad, it helps to understand how it happened. The smoking gun for Black is the President's Blueprint for Reauthorizing the Elementary and Secondary Education Act. He is painstakingly specific in this (reading this 51 page article has helped me remember why I'm not a lawyer), but the upshot seems to be this:

The President said quite plainly that his blueprint was meant to erase, replace, and supersede No Child Left Behind. So when the same requirements appear in the waivers, that makes it hard to argue that the waivers are meant to conform with and help preserve NCLB. Put another way, a waiver cannot legitimately be based on replacing the waived law with some other law entirely. It's like those movies where federal agents offer a criminal release from jail only if he'll steal something for them-- it may be cool drama, but it is in fact coercive and ultimately illegal.

PART III: The Constitutional Flaws of NCLB Waivers

The Constitution does not give agencies (executive branch) the power to rewrite laws (legislative branch). They have some limited legal power to do waivery things, but only to the extent that the waivery things are described in the original law. NCLB does not contain any waiver descriptions that match what Arne Duncan has been doing. Duncan has no authority to offer these waivers under the conditions he's set.

PART IV: That Would Be Extraordinarily Bad

If the NCLB waivers are ruled as Constitutional, then we've just extended to an agency of the executive branch the authority to create new laws. This would be bad. Really, unprecedentedly bad.

Yes, regulatory agencies like the EPA often have to make judgments that seem tantamount to creating policy and law, but they still have to make those judgments based on facts and in ways that fit the original regulations. Agencies like the FCC have very broad legislative mandates, but other language and explication actually narrowed their scope considerably. What the ed waivers have done is create a whole new version of ESEA without the country's actual lawmakers ever touching a bit of it.

CONCLUSION

With no more power than the authority to waive noncompliance with NCLB, Secretary Arne Duncan achieved a goal that educational equality advocates had long sought, but never secured: the federalization of public education. His path to the “holy grail” of education, however, was fundamentally flawed. He only reached it by imposing waiver conditions that were neither explicitly nor implicitly authorized by the text of NCLB. Thus, he exceeded his statutory authority and violated the Constitution’s clear notice requirements regarding conditions on federal funds.

States only acceded to these new and unforeseeable terms because their impending non-compliance with NCLB put so much at stake financially, practically, and politically. By the time Secretary Duncan announced the conditions, states were out of options and left in a position where the Secretary could compel them to accept terms that, under most any other circumstances, they would reject. The administration took the states’ vulnerability as an opportunity to unilaterally impose policy that had already failed in Congress. In doing so, the administration unconstitutionally coerced states.

This is fifty-one pages of detailed argument with a mountain of footnotes and a heck of a lot of Constitutional lawyerese. But it is a thorough argument about how the current reformy wave of waiverism is not merely bad policy, but illegal. It is going to be really interesting to see what fuss is kicked up once this article hits the fan. Plenty of folks have been calling the waivers illegal since waivering first began, but now they've got a heavy-duty law professor in a professional journal to back them up. Who would like to start the countdown to lawsuit?




(h/t to Love Light for passing this article along to me)

Four Reformy Problems in a Single Tweet

@NeeravKingsland @CtrEdEcon @smarick @amandaripley How do we measure schools' progress without annual tests?
— Michael Petrilli (@MichaelPetrilli) August 24, 2014

This is the kind of statement that brings me up short. It's neither notably nefarious, nor is it larded with some deliberate obfuscation, and yet it is loaded with so many assumptions.

Measure schools' progress

Is "schools' progress" really a thing? It sounds more sciency than "is the school getting better"? But it has the same problem. Progress is a giant tribble of a word, so fuzzy that one cannot really make out a shape within the furry expanse. Progress from where to where? Become a better connector of the community? Become safer? Develop a stronger arts program, or more successful sports program? Do a better job of creating students who are able to function as self-directed learners? Graduate more students who get into college or get good jobs?

There are so many ways in which a school can progress (and most schools are tying to move in several of them at once. How exactly will standardized test results measure all of those many and varied forms of progress?

Measure schools' progress

Annual standardized tests will only measure one sort of progress-- how well the school is doing at housing students who do well on standardized tests. That is a standard for progress that can be met many ways, including paying close attention to which kind of students you're housing. This truly does measure the school's progress-- but not the progress of students, which would be a nice goal for a school. But we've moved from measuring student achievement to measuring student progress to measuring schools' progress.

Why? Why is measuring school progress useful? Why is it the end goal? It seems a little like checking the oven temperature as a way of determining if the turkey is cooked. Why not just check the turkey?

And that gets us right back to progress. When we talk about great schools, schools or students that are making progress-- what do we mean?

Without annual tests

Is that what we mean? Progress will be defined as "whatever we can measure with a standardized test?" Thomas Newkirk gets to the heart of this:

It all comes down to the parable of the drunk and his keys, an old joke that goes like this: A drunk is fumbling along under a streetlight when a policeman comes up and asks him what he doing. The drunk explains he is looking for his keys. “Do you think you lost them there?” the policeman asks.             
“No.But the light is better here.”

Neither of the necessary questions is being asked-- what do we really want to measure, and what would be the best way to measure it?

We

What do you mean "we"? Who is this we? And why do they need to have a measure of schools' progress?

Is it parents? I don't think so-- why would parents care about any plural schools beyond their own singular one? Who is it that needs a national scale everyschool rundown of how things are progressing? When exactly did the US appoint a Grand High Overseer to whom all schools must answer, and whom did we appoint? Is this the US DOE talking? A representative council elected by all the American taxpayers?

I know it's picky as shit to peel apart a simple tweet, but I want to highlight how many unexamined assumptions find their way into the education discussion. If we could start examining them more often, maybe we could start talking about the real concerns that should be on the table.

The Cult of Order

Many, many, many reformsters are members of the Cult of Order.

The Cult of Order believes in blind, unthinking devotion to Order. Everything must be in its proper place. Everything must go according to plan. Everything must be under control.

It is not new to find cult members in education. We all work with a least a couple. Desks must be just so. The surface of the teacher desk must be pristine and orderly enough that bacteria will avoid it and others will either stay back in awe (or experience a near-uncontrollable urge to violate it). Students lose a letter grade for putting their name in the wrong corner of a paper. In high schools, they believe that even seniors would benefit from going class to class in neat and orderly lines.

But reformster members take the Cult of Order to new levels.

They were bothered by the chaos of the crazy-quilt state standards, each different from the rest. They are alarmed at the possibility that individual teachers might be teaching differently from other teachers. Order, predictability, uniformity-- these are qualities to be pursued, not because they are a path to better outcomes of some sort, but because they are in and of themselves desirable outcomes. Standardization and a national curriculum that gets every student in every classroom on the same page at the same time-- this vision is good. Don't ask "Good for what?" To the Cult of Order that's a nonsense question, like asking about the utility of a kiss. For them, controlled, orderly  standardization is as beautiful as a sunrise.

The Cult of Order is all about fear-- fear that some sort of dark, menacing chaos lurks just beyond the borders. There's a horrible monster waiting just on the other side of that white picket fence, and the only way to keep it at bay is to make the fence just as neat and orderly as possible.

And yet, we know this is not how the world of human beings works. Human relationships are messy, wobbly, unpredictable, hard to plan. At first flutter of your heart, you cannot know how that story will end. Friendship may grow or wither, and no amount of orderly control can change it. And on the large scale, throughout human history, the dream of perfect order always travels hand in hand with aspirations of totalitarian despots.

It is only in the modern age that a true dream of perfect order seems attainable. The Romans maintained centuries of empire precisely because they developed a system that did not depend on perfect order and standardization, but left the many varied local governments in place. The Romans knew that complete order and control was unattainable. The modern Cult of Order believes it is.

Science also tells us that the Cult of Order is wrong. Chaos theory and information theory and quantum mechanics all tell the same story, The dream of simple linear Newtonian order, where insisting on A always gets you to B-- that dream is unattainable, a failed model that does not reflect how the world works. There is a kind of order in the chaos, but it is more rich and complex than we have ever imagined. More importantly, it is an order that does not respond to nor allow for planning and control.

Reformsters keep asking, "How can we precisely control the aspects of education in order to get the exact results that we want?" This is a oxymoronic nonsense question, like asking "How can I best kick that cute girl in the face in order to get her to love me?"

To try to exert that kind of exacting control over other human beings is not just futile-- it's damaging and destructive. To remodel American public education into the model preferred the Cult of Order is destructive and wrong.

ACT Report Finds ACT Really Important

Okay kids. Here's today's lesson in critical thinking (which, as you may have heard, is built right into the heart and sinew of the Common Core).

When a business releases a study showing the importance and effectiveness of that business's product, is there a possibility that the study might be aimed at something other than Telling the Whole Truth?

The folks at ACT must be really sweating these days. Their competitors at the College Board have scored a couple of coups, including A) hiring the well-connected author of the CCSS ELA standards, David Coleman and B) getting state and federal governments to adopt their line of AP products as the official education product of the US government. So the ACT people are in need of some realmPR work.

So they are here to report on the national level of college readiness.

This report provides the ACT folks with a great chance to let us know what they've been up to as they desperately try to grab some market share work to make US education sweller.

They have, for instance, released ACT Aspire, a test package for assessing student college readiness for grades 3 through 10. It aligns with the ACT College Readiness Standards, so take that, new multi-grade PSAT.

The ACT itself has been "enhanced" by the addition of stats and probability math questions, new questions that require students to "integrate knowledge and ideas across multiple texts, a STEM score, a new writing section enhanced with God-knows-what, and, of course, a computerfied version to take on line, because, computers. They will start throwing in ACT Profile, which appears to be career-suggestion software, so I guess the ACT is now an aptitude test. They also want you to know that they are committed to making sure that the test is continuously monitored for validity and evidence, which may seem obvious, but since David "An Education Is What I Say It Is" Coleman seems inclined to move SATs away from validity and evidence, it's worth the ACT's time to toot their own trumpet of obviousness.

Now, if you think any of that qualifies as a "report" and not "advertising copy," you are not exceeding expectations in your applying of critical thinking. But that's okay, because even Caralee Adams writing about the report at EdWeek, fails critical thinking regarding the meat of this report.

She takes away exactly what she was supposed to-- students who take lots of ACT courses do better in college, therefor, the ACT courses must have prepared them super-well for college.

Why is it that education "research" is so riddled with an inability or unwillingness to distinguish between correlation and causation?

If we were going to design an experiment to determine the effectiveness of a product (say, New Shiny ACT Creme) we would need to first start with subjects who were close to identical. So, let's start with 200 fifteen-year-olds who have similar amounts of acne, give 80 of them New Shiny ACT Creme, 80 of them nothing, and 40 of them mayonnaise. After several months, we'll check to see how the acne looks.

That is not what this "study" has done.

Instead, this study says, "We want the teens who think they have the best complexion to use ACT Creme. We'll let them try it, and in a few months, we'll check to see how they look."

What the ACT report seems to show is not terribly remarkable. Apparently the same high-achieving, hard-working, top-ability, ambitious students who do well in college also seek out the challenging courses in high school. Put another way, the students who value college success and take steps to achieve it in college, also take steps to achieve it in high school. In other news, the sun is expected to rise in the East tomorrow.

I'm not suggesting that the ACT is a scam and terrible waste of time. Like AP courses and the SAT, the ACT clearly has some benefits for some students, and that's a Good Thing.

But we really need to stop looking at these groups as if they are philanthropic organizations trying to make the education world a better place out of the goodness of their highly objective hearts. The ACT is a business, and any "report" they offer about their product is most reasonably viewed as advertising copy, not scientific research.

News organizations do not run stories with headlines like "Researchers Discover Frosted Flakes Are Grrrreat" or "Study Shows Men Who Wear AXE Get Laid More Often." At least not yet. In the meantime, let's keep those critical thinking caps firmly attached.

Objectivity Is a Unicorn

Objectivity is a unicorn. It's inspiring to believe in, pretty to create pictures of. Some folks love the idea of objectivity so much that they dress up horses or try their hand at photoshop. But at the end of the day, it doesn't actually exist.

Nevertheless, we continue to enshrine the idea of objectivity in places where it does not exist. Writers have repeatedly reminded us, for instance, that internet search engines are not objective. This article from Michael Kassner at Tech Republic provides a good summary of the basic arguments. When you google something, you do not pull up some objective summary of what the internet contains, but a list that has been weighted by programmers who judge that certain factors should be considered (including a cyber-reading of you and your own proclivities). Google results are just as subjective as if they were compiled by some guy and his buddy making their best guess about what you want and what you should see.

But we really like to believe in Objective Facts, and in particular the notion that certain instruments provide such facts for us.

Some simple instruments do provide some rudimentary objective data. A measuring stick will provide an objective indication of length of an object. But the stick might be measuring meters or feet, and it might include hatchmarks for very small units or not, and if you're using to measure something that's not exactly straight, you will have to make a judgment about how do that. Even the simplest measurement includes subjective judgments. And that's working with objects, not humans.

Complex measures are mostly subjective judgments, whether we are measuring all the articles on the internet and how well they match your search terms, or we are measuring how "college and career ready" your eight year old child might be.

We keep talking about standardized tests as if they are objective measures of... well... anything. They are not. A standardized test is the product of the individual personal judgment of the test writers, who have their ideas about which specific bits of knowledge and skill should be tested and who make their own judgments about what exact tasks would measure those bits. They may claim that research backs up some of their choices, but research is itself the result of individual subjective judgments and choices made by the researchers deciding what to measure, how to measure it, and how to interpret the data they generate. In some cases, such ads David Coleman's reconfiguration of SAT vocabulary, they simply baldly say, "I think we should do this, not that."

Track the elements of standardized tests in any direction you wish; you will soon arrive at human beings making personal subjective judgments about how the test should work.

Reformsters keep talking about the use of testing and data as if it will result in replacing the varied subjective judgments of a teacher with the pure objective results of the testing. No such thing is true. What they seek to do (whether they understand it or not) is replace the judgment of the teacher in the classroom with the subjective judgment of the persons who make the test.

When someone claims "this test is an objective measure of a students language use ability," they are wrong. The test is, at best, a pretty good subjective measure of some tasks that some test-writer guys believe probably indicate language use skill. It is no different from having some person come into the classroom and say, "I'm going to sit and talk to Pat for a couple of hours, and then I'll tell you how good a reader I think he is."

It is not humanly possible to remove subjective judgment from education (or, arguably, anything at all, but let's narrow our focus for now). Never even mind the question of whether or not we should-- it cannot be done.

How do we deal with the inherent subjectivity?

The problem with subjectivity is that it solves problems within a personal framework. We operate in our own little bubble or silo and we define problems and search for solutions based on what we see and know inside that little framework. The trick is to temper subjectivity by expanding that framework. We need to see and know more.

We expand the framework with professional knowledge. We train teachers to understand pedagogy and subject matter so that their instructional judgment is not based on considerations like "What kind of mood am I in" or "Do I like this kid."

We expand the framework with personal relationships. One of the terrible lies of the cult of objectivity is that we can make good decisions about students without knowing a thing about them. Management schools recommend that managers not live in the same communities as their employees, so that relationships are not part of their framework.

Baloney. One of the marks of a good decision is that you can talk about it out loud with the people who are affected by it right there in the room with you. When our framework is expanded to include the people who are part of the choices and the results, our subjective view of the situation is more complete, more useful to people beyond ourselves.

Reformsters have done their damnedest to keep teachers, students, and much of accumulated wisdom of American public ed outside of their framework. This leads them to say naive things (Hey, did you guys know that water is wet) and stupid things (Hey, if you tried hard, I bet you could build a house out of water). It doesn't ever lead them to say objective things.

They claim that their goal is to inject objectivity into the education world, when in fact they're simply trying to impose their own subjective judgments in place of others'. When someone wearing a white hat rides into town on a unicorn, you'd better check to see how the horn is attached.

I've Launched at EdWeek

As noted a while ago, I have taken on blogging for Education Week, and the new blog launched a few days ago.

The EdWeek blog will be called "View from the Cheap Seats," and I'll be posting there at least weekly. I even get a cool graphicky thing for the header!I am excited about the potential for reaching a new audience. The first post is about the absence of the teacher voice from education debates. Feel free to stop over, read it, and shower them with demands that I be paid a zillion dollars. Or just enjoy it.

Superintendents Speak Up

On the first day of school, my wife's superintendent got choked up.

He was delivering the usual kick-off speech, and she said he started to talk about testing and numbers and the students. He reminded his staff that students were not just test scores, not just a number, and that the work they did as teachers was so much more than could be measured by numbers. It looked, she said, as if he was on the verge of tears.

My wife's superintendent is my former principal. He's a good man and a fine educator. And apparently he's done pretending that chasing test scores is a good way to run a school district.

This may well be the fall we remember for the number of school district leaders who have finally had enough and begin to speak up.

Examples abound. In Peru, Illinois, Superintendent Mark Cross sent out a letter that said in part

Unfortunately, there are many federal and state education initiatives that can very much be a distraction from what matters most. These initiatives are based on good intentions and are cloaked in the concept of accountability, but unfortunately most do little to actually improve teaching and learning. Most are designed to assess, measure, rank and otherwise place some largely meaningless number on a child or a school or a teacher or a district. That is not to say that student growth data is not important. It is very critical, and it is exactly why we have our own local assessment system in place. It is what our principals and teachers use to help guide instruction and meet the needs of your kids on a daily basis. In other words, it is meaningful data to help us teach your child.

He makes this commitment

This is why I wanted to let you know that we will not be talking to you that much about the PARCC assessment or Common Core or other initiatives that have some importance, but they are not what matters most to us. YOUR CHILDREN are what matter most, and we believe that kids should be well-rounded, with an emphasis on a solid foundation for learning across all subjects by the time they get to high school and later college. We believe that kids need to be creative and learn to solve problems. We believe that exposure to music and art, science and social studies, physical education and technology and a wide variety of curricular and extracurricular activities will serve them very well as they grow into young adults.

And he delivers this pointed (if grammatically suspect) indictment

The state and federal government have failed epically in their misguided attempts at “reforming” public education. Public education does not need reformed. 

Superintendent Cross is, of course, not alone. In Washington State, the education system has lost its waiver from Arne Duncan because the state legislature would not implement the federal Department of Education's preferred method of teacher evaluation. So Washington schools are operating under No Child Left Behind, which means that all schools not meeting requirements that 100% of students be above average (aka "all schools") are failing. You may recall that one of the punishments for failing schools under NCLB is that they must send out a "We Are Failing" letter to the public.

Superintendents in WA have sent their letter. However, 28 superintendents wrote a letter which includes the observation that the label under NCLB is "regressive and punitive." The basic layout of the letter is "The feds say we have to tell you this, which we are now doing, however you should know that the feds are full of it, their policies are stupid, and we are educating your students pretty well, thank you very much."

And as I noted here yesterday, the Board of Education for the entire State of Vermont has adopted a resolution calling out the feds on their stinky testing requirements.

The tone in administrative offices is continuing to shift. Ten years ago there was a lot of kool-aid drinking. Then we had "Well, it's the law." Then we had fatalism and resignation, "Well, let's just do our best work and hope that these tests take care of themselves and somehow things work out." What we have always needed are administrators to stand up and say clearly, "This is not right. It's not right for us, and it's not right for our students."

I know there are still districts and entire states where the school leaders have not only drunk the kool-aid, but are selling it themselves out of the back of a van. But it's heartening to see and hear more who are willing to speak out in a meaningful and public way. Duncan is clearly trying to stem the tide with his waiver-waiver, the offer of "Look, we'll just wait a year and then we'll punch you in the face." But postponing a stupid thing does not make it any less stupid, and in the meantime, more and more people are starting to point out that the emperor's clothes (which are no longer new) are woven out of air and empty promises.

Vermont BOE Hammers Fed-Style Testing

With states like North Carolina and Florida doing their best to bury public education and dance on its grave, it's nice to see some states can still stand up for their schools.

Earlier this week, the Vermont State Board of Education adopted a statement and resolution on assessment and accountability. It's worth a read, but let me hit the highlights for you.

The Board starts by recognizing that uniform standardized tests can be a useful tool for helping schools chart a path toward successful delivery of well-designed standards. And then comes the pivot-

Despite their value, there are many things tests cannot tell us. Standardized tests like the NECAP and soon, the SBAC, can tell us something about how students are doing in a limited set of narrowly defined subjects overall, as measured at a given time. However, they cannot tell us how to help students do even better. Nor can they adequately capture the strengths of all children, nor the growth that can be ascribed to individual teachers. And under high-stakes conditions, when schools feel extraordinary pressure to raise scores, even rising scores may not be a signal that students are actually learning more. At best, a standardized test is an incomplete picture of learning: without additional measures, a single test is inadequate to capture a years’ worth of learning and growth.

Unfortunately, the way in which standardized tests have been used under federal law as almost the single measure of school quality has resulted in frequent misuse of these instruments across the nation.

In order to avoid that sort of foolishness getting loose in the Green Mountains, the Board lists eight guiding principals for the appropriate use of standardized tests.

1) The proper role of large scale tests must be stated before giving the test, and that use must be demonstrated as scientifically and empirically valid. That includes proof that the test can predict performance on "other indicators we care about, including post-secondary success, graduation rates and future employment." And you can't use the test all by itself-- mix it up with other measures.

2) Public reporting. Schools need to do that, but they need to report a wide variety of indicators that give a full picture of what they're doing.

3) Judicious and proportionate testing. Reduce the amount of time on summative and standardized testing. The feds should back off on multiple subject testing grades 3-8 as well as high school (so, you know, all of it). "Excessive testing diverts resources and time away from learning while providing little additional value for accountability purposes."

4) Test development criteria. Any big standardized test used in Vermont needs to be built in accordance with principles of American Educational Research Association, National Council on Measurements in Education, and the American Psychological Association.

5) Value-added scores. Near as we can tell, these are crap. We will not be using them in Vermont "for any consequential purpose."

6) Mastery level or Cut-off scores. This whole paragraph is pretty awesome.

While the federal government continues to require the use of subjectively determined cut-off score, employing such metrics lacks scientific foundation. The skills needed for success in society are rich and diverse. Consequently, there is no single point on a testing scale that has proven accurate in measuring the success of a school or in measuring the talents of an individual. Claims to the contrary are technically indefensible and their application would be unethical.

7) Use of cut scores and proficiency categories for reporting purposes. The fed since NCLB was born have required this. Here's a list of ways in which it has been documented to create negative effects. We'll keep doing what the letter of the law requires, but it's crap.

8) Just as the state high quality education, the federal, state and local governments must provide adequate resources to get the job done. If you're going to demand a report on the quality of the school's work, demand a report on the sufficiency of the resources provided to the school "in light of the school's unique needs."

These are followed by several whereas's that note that the the nation's have been spending an ever-increasing amount of time and money on testing of a sort and in ways that are known to be No Damn Help to anyone and wrapping up with

WHEREAS, the culture and structure of the systems in which students learn must change in order to foster engaging school experiences that provide joy in learning, depth of thought and breadth of knowledge for students [emphasis mine, because, damn, wouldn't you like to see that in every school's mission statement!]

And then we get the Be It Resolved portion 

-- The Secretary of Education should re-examine the accountability system and come up with one that sucks less (I'm paraphrasing)

-- Congress should get off its collective keister and amend the ESEA

-- Other state and national groups should join us in this

I grew up just across the Connecticut River from Vermont, playing in my front yard and looking at the big beautiful mountains, but I have never loved Vermont more than I do reading this resolution. If you see Vermont today, give it a big hug for me. And send Arne Duncan a copy of their resolution. It's true these are just words-- but they are damn fine words.