ICYMI: Here Comes Another Arctic Blast Edition (1/27)

So it's going to get cold again. But in the meantime there are useful things to read about education. Here's the list for this week-- remember to share what you think needs to be shared.

Denver's Portfolio Model School District is a Failure

Thomas Ultican breaks down some of the details in the long-running reform experiment in Denver schools.

Automation at Davos

This is pretty stark stuff. The difference between what the movers and shakers say about AI publicly (It will be great for workers) and what they say privately (We'll be able to fire 99% of our human workforce).

Success Academy: The High School

One would expect Eva's attempt to expand her brand into high school to be ugly and messy, but this podcast with transcript shows just how ugly and messy (spoiler alert: really). Just brace yourself for the whiplash conclusion.

Floridians Choose Public Schools

Breaking down some survey data to see what Floridians would really like.

Under-discussed Stories of 2018

Have You Heard podcast looks at five six stories that didn't quite get the attention they deserved. (There's a transcript if you aren't a podcast person).

Houses for People

Teacher Tom provides his littles a lesson in activism.

MLK's Work Precedes Us And, With Resilience, Lives After

Jose Luis Vilson and a wake up call for what MLK's legacy is about beyond pretty memes.

How Do Charter Schools Affect Students With Special Needs?

From the LAUSD strike comes this set of word delivered to the board about some of the charter claims that just don't hold up.

The Battle For New Orleans Public Schools  

If you'd like one more article for the "NOLA Is Not A Huge Success" file.

Is Competency The Hot New Thing?

Tom Vander Ark thinks that competency is the up-and-coming next big thing in education. He just said so a few weeks ago at Forbes, but he's been saying so for several years now.  Vander Ark has been at the education reform biz longer than most, but his career also includes the launch of K-Mart's competitor to Sam's Club, point man for the Gates small schools initiative, and an attempt to launch some charter schools in NYC that left a bad taste in many mouths. He's not always right. How about this time?

When we talk about competency (as in competency-based education), we need to keep a couple of things in mind.

Everyone's probably at home working on badges

First of all, it's not remotely new. For most of its history, it has been called "mastery" or "learning for mastery," and it crops up as far back as almost a century ago, when programs like the Winnetka Plan started playing with the idea that instead of focusing on the hours spent in a program, we could focus on whether or not the students had mastered a particular piece of skill or content. Mastery Learning began to catch fire again in the sixties. Most of us who went to teacher school in the seventies learned about it and were encouraged to make it a factor in our work, though nobody had yet solved one of the central problems with mastery learning. The premise was that every student could learn the material as long as she was given enough time--but there were still only 180 days in the school year.

In the classroom, mastery learning often took the form of giving students multiple, even unlimited, attempts to show mastery of the material. On the ground, this looked like, "Students, you can keep taking the unit test until you pass it." Students sometimes took advantage of the reduced sense of urgency, and parents were not always supportive--as one parent asked me, "Why should my kid try when he gets a dozen shots and everybody passes?" While mastery learning became central in very few classrooms, by the eighties, many teachers had incorporated elements of mastery learning into their practice.

In the nineties, mastery learning made a comeback with the rise of Outcome Based Education. In OBE, each lesson would culminate in a student demonstration of some particular outcome--new terminology for showing mastery (immortalized in a million million lesson plans as "The Student Will Be Able To", aka "TSWBAT"). We would have authentic assessments, where the students would demonstrate mastery in some "real" way; multiple choice tests and their ilk would be banished. Each student would have a portfolio that would show the complex web of her mastered skills, not just some simple letter grade. And finally, the motto, drilled into teachers at countless professional development sessions, was that "all can learn all."

But OBE died a quick death. Part of the opposition came from conservative parents who resisted the "values" outcomes that required students to demonstrate mastery of the skill of being a good person. The other fatal attack on OBE came from the rising tide of accountability hawks, spurred to action by A Nation At Risk and demanding the kind of cold, hard numbers and measures that led us to No Child Left Behind, Common Core and accountability based on multiple-choice standardized tests.

Competency is a new branding of a century-long thread in education. Now mastery can be marked with digital badges, the progression of skills maintained, measured and recorded by computers, the badges earned and issued in and out of school.

If it's going to finally become the big thing in education, it will have to solve some of its old central problems. How long do you give students who progress to mastery slowly? How do you sequence competencies in a way that's fair and sensible? How do you break complex skills and knowledge into competencies that are measured in authentic and valid methods? If you let anyone, anywhere issue a "badge" for a competency that's been mastered, how do we keep accountability hawks happy? And a new problem--what happens when you let all of these educational decisions be made by tech companies?

Vander Ark's evidence that competency is going to finally bust through this time is essentially a list of tech companies that are working on various parts of the problem. Some companies are working out how to issue a digital badge for a variety of mastered skills that will be assessed... somehow. Financiers like XQ and New Schools Venture Fund are throwing money at groups that want to work on these problems. But even if all these groups successfully solve the problems of mastery learning, advocates like Vander Ark will still have one more question to answer-- can you get parents to sign up for a mastery learning system on a large scale for the first time in 100 years?

Originally posted at Forbes

OK Legislator To Teachers: Shut The Hell Up

Oklahoma has worked hard to get itself in the front of the pack of States Most Hostile To Public Education. Maybe not number one (relax, Florida), but right up there. Ultra-low teacher pay. Slack charter rules. The kind of state where the idea for improving education is to gear it more toward providing meat widgets for employers. The kind of state where a serious idea about improving teacher pay is to fire half the teachers and give their money to the remaining teachers, who will all teach twice as many students.

So it wasn't a huge surprise last year when teachers in the state walked out. While they didn't get everything they wanted, they were still confident that they has sent a message to the legislature.

Apparently some legislators misunderstood the message.

Oklahoma's political leaders could have looked at the walkout and said, "Damn , we need to spend more on education" or "Damn, if we don't get our act together, we'll never recruit enough teachers to help with our ongoing teacher shortage" or even, "Damn, we have got to find a way to suck less."

Nope. The conclusion some legislators reached was, "Damn, we let teachers talk too much."

This frickin' guy. 

Meet Rep. Todd Russ. He's a hard work and Godly values guy. He has two degrees in banking and a Doctor of Ministry from Berean School of the Bible, and online university. And his wife is a public school teacher. He's been a banker most of his life, but has apparently since founded Commercial Growers Incorporated, and I can't find anything about that organization. In 2004, he founded the Burns Flat- Dill City Education Foundation, which seems to be scholarship related. And he occasionally attracts cranky attention.

And he's the guy pushing HB 2214. It doesn't have a name, so let's just call it the Shut Oklahoma Teachers The Hell Up Act. SOTTHUA is an amendment to the previous act that was supposed to already have shut teachers up by making it illegal to strike. But it's only a strike if a teachers union takes action against its local school board; when all the teachers walk out because of the state legislature, that's just a walk out and it was previously completely legal. So Russ (did I mention that his wife is a public school teacher??) has plugged the holes with this language:

It shall be illegal for the board of education or school district employees, including all those defined in Section 1-116 of this title, to strike or threaten to strike or otherwise close schools or interfere with school operations as a means of resolving differences with the board of education, the State Department of Education, the State Board of Education, the Legislature or any other public official or public body. Any person engaging in a strike, shutdown or related activities shall be denied the full amount of his or her wages during the period of such violation, and if the person holds a certificate issued by the State Board of Education, such certificate shall be permanently revoked. 

Stage a walkout against the local board or the state legislature or, well, anybody, and lose your teacher certificate. If you can't be a quiet, submissive teacher, well, then, you can't be a teacher at all.

Note, also, that this applies to boards of education, because some of those sonsabitches supported the teacher walkout of 2018 both vocally and by closing schools. So they can also shut the hell up.

I haven't found any quote from Russ discussing how he thinks this will affect Oklahoma's teacher shortage. Nor does this staunch defender of the Second Amendment said anything about how he feels this law fits in with the First Amendment. I'm no Constitutional scholar, but this seems a bit iffy to me. Heck, the courts say a politician can't block critics on Twitter, so Russ would be powerless an Oklahoma tweetstorm.

At this point, it's hard to tell if any legislators are willing to support Russ's baloney bill. And it's not the only stupid unconstitutional bill being proposed; Senator Mark Allen wants to make any group over 100 that wants to protest on the capitol grounds post a $50K bond. Both parties have already indicated that bill won't fly; let's hope similar sense prevails regarding Russ's boneheaded bill.  At any rate, here are two more things I do know about Todd Russ.

First, this is apparently not his only education-related bill. Check out HB 2208. This bill would require every school district to have a secret ballot vote at least every five years to determine whether or not the union can keep representing teachers there or not.

Second, Todd Russ won his 2018 re-election bid over Dennis Dugger by about 500 votes-- 5,698 to 5,106. It seems as if Rep. Russ might be a tad vulnerable come next election cycle. That seems worth remembering.

P.S. He's one of three legislators to vote against an equal pay act for women in the state.

WV: Legislative Extortion

Last spring, West Virginia's teachers stood up and stood up loud, shutting down every single school in the state. They were out with five demands-- better wages, health insurance, defeating an expansion of charter schools, keeping seniority, and killing a "paycheck protection" bill. They won, the governor signed a pay raise, and teachers won the right to shout at the end, "Who made history? We made history!"

They should have forced the legislature to swear "No take backs."

Republicans in the West Virginia Senate have introduced a bill that aims to undo some of the results of the 2018 strike.

"This is a vision that’s been worked on with input from many,” said Senate Education Committee Chairwoman Patricia Rucker, R-Jefferson. She forgot to say many what, as the bill, wit versions clocking in at well over 100 pages (one draft version is online here), was put together with zero input from the teachers union.

That makes sense, since the goal here is to shaft West Virginia's teachers yet again.

A picturesque WV cliff off which the WV GOP would like to throw public ed

Teachers struck for a 5% wage hike; the bill has a 5% average wage hike. Teachers struck for improved health insurance. The bill sort of does that. The bill also adds incentive pay for math teachers. It even adds a teacher expense credit (a whopping $250) for teachers. But it doesn't end there.

Since the strike, teachers in West Virginia have been playing whack-a-mole with various reformster proposals, and each one of them has been shot down. Now they are all back, wrapped into this humongous bill the includes teacher wage increases and health care. The GOP message is clear-- give us what we want, or you can't have your raise.

What do they want? It includes a "paycheck protection" clause, requiring unions to get permission annually to deduct dues from teacher paychecks. It docks teacher pay during walkouts.

It gives West Virginia, for the first time, charter school law the establishes both brick and virtual charters. The charter law takes up 32 pages of the whole bill, and covers all the bases from a state charter commission stocked with political appointees all the way to rules allowing charter takeover of public facilities in whole or in part. It institutes open enrollment.

And it creates the super-voucher education savings account system. The system would give parents 75% of the state adjusted per pupil expense. The proposed voucher may be used for private school tuition, online learning programs, tutoring, extra services like activity fees, textbooks or any other instructional materials, computer hardware or software, school uniforms, testing fees, summer school tuition, CTE tuition, services and therapies like PT, transportation to/from school, and anything else the state treasurer approves. The ESA does not require the student to be enrolled in a private school-- in other words, you can use your voucher to home school.

The parent fills out an application and promises to get the student an education in "at least" reading, language, math, science and social studies (which is certainly a "least" education). The ESA program will itself be privatized by hiring someone to manage it. A parent review committee of seven ESA parents picked by the treasurer will determine if any expenses are questionable; they'll meet when the treasurer calls them, and he'll call them when...? Are we supposed to believe he'll be monitoring all those ESAs? Because that is a lot of work, but if you don't do it, Florida history tells us that a lot of the money wanders off.

Granted, I'm looking at a draft, of which there were apparently several, but this is a bad implementation of a bad idea.

So that, in broad strokes, is the deal. If the legislature wants teachers to get the raises they were promised, they have to let the GOP blow up public education. Senate President Mitch Carmichael projects just the right weasely passive-aggressive tone:

I'm certain that there are some teachers and some union leaders that would rather just have an enormous pay raise — which is a component of this bill — and not reform the system in any manner. But I am confident that really great teachers want to have the opportunity to do their job in the best possible manner. 

The actual bill was, possibly, going to show its face today. The lesson here is not a pleasant one-- any history that is made can be unmade, and the West Virginia GOP is apparently committed to undoing as much of last spring's strike as they can.

The really unfortunate thing here is the the GOP missed the point-- the strike was not simply about money and healthcare, but about dignity and respect and building a better future for West Virginia's public schools and the students they serve. The GOP is calculating that they have made WV teachers so poor that they'll jump at the money and let all the rest go. They are calculating that the teachers can be bought. That's too bad, because while this bill can-- and should be defeated-- the disrespect for teachers that it shows cannot be taken back.

Here's hoping this bill goes down in flames and the legislature goes back to provide the teachers what they were  promised and what public education in West Virginia desperately needs.

DeVos Has A Hammer

Secretary of Education Betsy DeVos has been busy lately, making actual somewhat-public appearances and talking about all her favorites subjects. Thursday it was the 87th annual United States Conference of Mayors Winter Meeting, and her remarks included many of her favorite points. But she's not just beating a drum; she's working with a big, heavy hammer. Bang bang bang. Here are some highlights:

Droll understatement: "I was active in local politics and policy" sounds so much humbler than "I used to run the GOP in Michigan" and so much cleaner than flat-out admitting that you're buying influence. Bang.

The 100 year complaint: Everything has changed in the last hundred years but "approaches to education have largely remained the same." Bang.

Meat widgets: There are millions of unfilled jobs in this country. Somehow, she fails to see any connection between this unfilled need and the push to stop immigration. Bang.

Fake statistics: Well, at least she didn't use an actual number. But she still claimed that "majority of the jobs that today's students will do just ten short years from now haven't been invented." This oft-cited factoid is bogus. Bang.

Odd transitions: In discussing the fact that employers say they can't find trained people (could it be that they are trying to fill jobs that didn't exist ten years ago?), DeVos unleashes this transitional gem: "There is a disconnect between education and the economy, just as there is often a disconnect between a child and the school they’re assigned to." Bang.

Is it a disconnect, or just hypocrisy: DeVos says that too many students are "treated like commodities." This is shortly after she suggests that the job of education is to make students useful to employers. Bang.

More neat widgets: DeVos wants to plug Perkins V, just signed into law last summer. We need to have a conversation about this new version of the bill some time, but what she's excited about at the moment is the way it breaks the "giant silos" between educators and employers. Or, as one commenter put it a month ago, "provides some new opportunities for tighter alignment of programs of study to data-driven workforce needs." Or, as I put it now, made it clear that education exists to serve business interests. And it spreads some money around, and the mayors will want to keep their eyes peeled for that. Bang.

Something new: DeVos is anti-silo. Education has too many silos. Not like the rest of the world, which is silo-free. Which I suppose seems true if you're a really rich, well-connected person who has always been able to swoop into any place she wants, including an entire public sector in which you have no experience or expertise. I suppose to that person it must seem like the world has no boundaries. Bang.

Rethinking: DeVos wants everyone to question everything so that nothing limits students from being prepared. She has a list of questions, which are all cagily aimed at one answer-- privatized, computerized, teacherless learning.  Her list of questions does not include, for instance, why should one family get to hold onto more wealth than they can possibly need while poor students have to worry about having enough to eat. Also, she's decided that CTE is good stuff (at least, you know, for certain people) and that there should be partnerships which involve turning schools into training facilities for particular employers. Bang bang.

Quotable nonsense: DeVos always delivers at least one pull-quote-worthy line. For the mayors, I's pick this:

Well, education is the least disrupted “industry” in America. And, let’s not kid ourselves, it is an industry.

Education is not a public good or a trust or a promise to our children or a valuable institution. It's an industry, like manufacturing toasters. Yet another quote that gives a clear insight into why DeVosian policies are so hostile to public education.

One size fits all: It's one of DeVos's go-to criticisms of public education. It's useful to her not because it's true, but because it sets up choice systems as the opposite of public ed. Of course, no school is one size fits all at all, as public schools are not only inclined by nature but required by law to accommodate all kinds of students. Certainly the Common Core movement was a concerted effort to force one-size-fits-all into public ed; the degree to which it failed is a measure of how much public ed is not inclined to be one size fits all. Meanwhile, charter schools only offer one size. They don't claim their one size will fit all-- they just refuse to accommodate those students that it doesn't fit. Not clear to me how that's better. Bang.

Swell anecdote: No DeVosian talk is complete without the story of a student that she recently met (despite her schedule, she is apparently always meeting students) who benefited from exactly the policies that DeVos would like to promote. Pretty sure DeVos needs to get out more. Maybe to actual public schools. Thump.

The governor's speech was vintage DeVos, and this week she also got her hammer out for a talk at the Heritage Foundation. She was there largely talking about the DC Opportunity Scholarship, one of the programs often held up as a proof of concept for choice.

For the Heritage crowd, DeVos threw in the big two-part idea that (part one) the majority of people want choice. Her basis for that claim is a poll run by-- surprise-- the American Federation for Children, the pro-choice advocacy group funded by the DeVos family. The poll was a phone survey of 1,200 likely 2020 voters; how those voters were selected isn't covered. John Schilling, the president of AFT, calls for "policymakers to listen to these voters," which-- well, the policymakers will have to implement these voucher policies because no voters have ever actually voted for a voucher law (vouchers just died-- again-- in Texas). The only way vouchers become law is when lawmakers create and pass the laws, voters be damned.

But DeVos has her own explanation for why school choice isn't more successful-- the damned teachers union is "the only thing" standing in the way because they have a "personal vested financial interest." I don't think DeVos is stupid, so I'll attribute this line to working her highly conservative audience, because the opposition to choice has come in many forms in many places-- including conservatives who don't want to see choice turn into another government power grab. School boards, superintendents, the NAACP, a whole host of other organizations, and actual voters, when it comes time to put a choice law up to a vote, have also stood in the way of school choice many times.

Even if the "survey" were accurate, even if people did say they want choice, that does not mean they want DeVos's idea of choice. "I want my child to be able to choose a good school" is not the same as "I want to see the public school stripped and destabilized and ultimately replaced by a privatized network of education-flavored businesses that may not even accept my child if she applies to attend them."

This is a long con we don't talk about often enough, and it's embedded in every DeVos speech-- the notion that educational choice can only happen via privatization, that the public school system can only be improved by dismantling its democratic systems and replacing them with private ownership. But "privatization" is a hard sell, while "choice" sounds just divine.

DeVos has a hammer, a big expensive hammer built to smash public education into small pieces (but not so small they can't be sold for parts). Every speech, she gets out that same hammer, maybe puts some new tape on the handle, shines up the head, but it's always that one hammer, beating away, over and over. Bang bang bang.

Classroom Practice Ideas For Reformsters

So while I was working on my last post, I came across this post by Robert Pondiscio that makes a good companion piece.

It's a post you might well miss; if you're a regular reader here, you may not check the Fordham Think-Tank Reform Advocacy Blog often. But Pondiscio and I really disagree about some things and really agree about some others, so I pay attention. I give Pondisc io credit for kicking off the 2018 year of reformy navel gazing with the observation that reformers had “overplayed our hand, overstated our expertise, and outspent our moral authority by a considerable margin.'

In this post, he's also responding to Mike Petrilli's new interest in evidence-based practices, and offered five suggestions designed to keep the "golden age of educational practice" from "blowing up on the launch pad." Let's look at the five pieces of advice:

1. Ask the right questions.

“What works?” is the wrong question. “The right question is ‘Under what conditions does this work?’” observes Dylan Wiliam.

That's an excellent start. My own suggestion is instead of politicians and thinks tank guys asking each other, "How can we make these damn teachers do the right stuff," turn to teachers and ask, "How can we help you do your jobs?"

2. Understand and accept trade-offs.

Quality research, says Pondiscio via Dan Willingham, "tells you what’s likely to happen if you pull a lever. It’s silent on whether it’s a good idea, or if the trade-offs are worth it."

Reformsters were surprised and baffled by the consequences of test-based accountability. Arne Duncan kept bemoaning the way test prep dominated many schools without ever understanding that his policies helped create the problem. One of the most critical question that people (especially journalist people) don't ask when some school announces miraculous test scores is "What did you sacrifice to get them." And don't get classroom teachers started on the endless directives to add One More Thing to their day, as if they have some vast bank of unused time they can tap into.

Teachers are operating at capacity; in some cases, beyond capacity. You cannot add anything without losing something. Yet more than once I've seen a principal get angry because, after receiving a new directive, a teacher asked, "So what do you want me to stop doing," as if that were some sort of impertinent sass and not a legitimate concern. And you don't have to get into deep or complex practices to see this issue in action. How many teachers have been told, "Between classes, be outside your door, monitoring the hall. Also, between classes, make sure you are monitoring your classroom."

This issue is a great example of something thinky tank guys and politicians easily ignore because they aren't in a classroom. But every single choice in a classroom comes with an opportunity cost, something that won't be happening because the teacher is doing Thing X instead. Any discussion of classroom practices must include a discussion of the costs-- all the costs.

3. Kill education myths.

This advice is rather broad. Pondiscio brings up learning styles and other pseudo-science. But myths are hard to stifle. Take "charter schools do a better job than public schools" or "vouchers and cyber schools will work"-- very hard to stifle. But I support stamping out edu-baloney, even though we might disagree about which myths qualify, exactly.

4. Learn the lessons cognitive science.

Here we hit an area where Pondiscio and I agree-- the "skills" movement labeled many things (e.g, critical thinking) as skills that can be taught, leaned and honed in a vacuum. That's just not so. From reading to creativity, content knowledge is what makes it possible. You can't think critically about something if you're ignorant of that something. Your reading skill rests on your content knowledge; you can't decode your way to understanding a word you've never heard of in a context you don't understand. Even creativity-- you can't discover creative new ways to look at something if you don't know anything about that something. You can phrase this in fancy language and say that cognitive skills are domain specific, but the idea stays the same.

5. Stop demanding bad practice through policy.

Really, stop demanding any practice through policy, because I don't care what it is-- for some teacher with some student on some day, the policy will be bad.

This was my raging frustration through the second half of my career-- the constant demands coming from state and federal government that I commit educational malpractice. The number of times that I had to look at a particular hill and decide whether I should fight on it, die on it, sneak around it, or just live to fight another day-- there was nothing more tiring about the work than those kinds of decisions, made on a daily basis, and all the worse because these decisions were forced on me by politicians and thinky tank hot shots and rich guys who wanted to dabble in education, but not by actual educators who had a clue about my work.

Common Core, test-centered accountability, test prep-- and at the high school level the damage being done isn't as severe as what's happening on the elementary level. I could not have been angrier if I had been a surgeon told by my bosses, "Stop using scalpels and operate with this rusty shovel instead."

Guys like Bill Gates, thinky tanks like Fordham, dabblers like David Coleman, politicians like--well, all of them-- if your ideas for good classroom practices are, in fact, good, then put them out there in the field. Let the marketplace of ideas get a look at them. If teachers like them, if they work, they will spread on their own. But mandating them is a bad idea because--

First, no practice is a good practice 100% of the time and

Second, as the last twenty years show, you guys mostly don't know what the hell you're talking about.  What possessed you to appoint yourselves the Grand High Poohbahs of education, I do not know, but just stop it. Stop operating on the assumption that you Know Things that actual teachers in the field do not, and so you must go ahead and force every teacher to do things your way. We haven't even begun to unpack all the damage you've already done, so just stop. You've been consistently wrong, and yet, like some energizer bunny that thinks it's a cordon bleu chef, you won't stop making a mess in the kitchen. Stop. If you want your say, sure, go ahead, this is America. Work with teachers. Try to be helpful. But stop trying to turn your every idea into a law, rule or regulation.

Let me just catch my breath here. 

I give four of Pondiscio's ideas a gold star for being useful going forward. And if reformy leaders want to shift gears from trying to break schools to trying to help teachers do the work-- well, I'm suspicious, but I'm always willing to listen. I appreciate what Pondiscio has written here; I just hope his colleagues listen to him.

The Trouble With Evidence

So now some voices are calling for an emphasis on evidence-based practices in classrooms, and I don't disagree. Evidence-based is certainly better than intuition-based or wild-guess-based or some-guy-from-the-textbook-company-told-us-to-do-this based. But before we get all excited about jumping on this bus, I think we need to think about our evidence bricks before we start trying to build an entire house on top of them.

There are three things to remember. The mot important is this:

Not all evidence is created equal.

Your Uncle Floyd is sure that global warming is a hoax, and his evidence is that it's currently five degrees outside. The Flat Earth Society has tons of evidence that the world is not round at all. Youtube is crammed with videos showing the evidence that 9/11 was faked, that the moon landing was staged, and that the Illuminati are running a huge world-altering conspiracy via the recording industry. Every one of those folks is certain that their idea is evidence-based, and yet their evidence is  junk.

Some evidence is junk because it has been stripped of context and sense. Some is junk because it has been unmoored from any contradictory evidence that might give it nuance and accuracy. Some is junk because it has been mislabeled and misrepresented.

One of the huge problems in education evidence-based anything is that when we follow the trail, we find that the evidence is just the same old set of test scores from a single bad standardized test. Test scores have been used as evidence of student learning, of teacher quality, of school achievement. Test scores have been used as evidence of the efficacy of one particular practice, an influence somehow separated from all others, as if we were arguing that the beans Chris ate at lunch three Tuesdays ago are the cause for Chris's growth spurt. And don't even get me started on the absurd notion that a teacher's college training can be evaluated by looking at student test scores.

Certainly test scores are evidence of something, but not much that's useful. Don't tell me that Practice A improves student learning if all you really mean is that Practice A appears to make test scores go up. The purpose of education is not to get students to do well on a standardized test.

Educational evidence has one other quality problem-- experimental design doesn't allow for real control groups. For the most part, educational researchers cannot say, "We know your child is supposed to be getting an education right now, but we'd like to use her as a lab rat instead." Arguably there are huge exceptions (looking at you, Bill Gates and Common Core), but mostly educational research has to go on during the mess of regular life, with a gazillion variables in play at every moment. Yes, medical research has similar problems, but in medical research, the outcomes are more clear and easily measurable (you either have cancer, or you don't).

Bottom line: much of the evidence in evidence-based educational stuff is weak. At one point everyone was sure the evidence for "learning styles" was pretty strong. Turns out it wasn't, according to some folks. Yet teachers largely use it still, as if they see evidence it works. Apparently the evidence is not strong enough to settle the battle.

And all of this is before we even get to the whole mountain of "evidence" that is produced by companies that have a product to sell, so they go shopping for evidence that will help sell it.

This helps explain why teachers are mostly likely to trust evidence that they collect themselves,  the evidence of their own eyes and ears, invoking a million data points gathered on a daily basis. This evidence is not foolproof either, but they know where it came from and how it was collected.

After we work our way through all that data collection, we still have to interpret it.

We just came out of a harrowing weekend of national weekend over an incident in DC involving some Catholic school teens, some indigenous peoples, and the Black Israelites. We've all had the same opportunity to watch and examine the exact same evidence, and we have arrived at wildly different conclusions about what the evidence actually shows. Not only do we reach different conclusions, but we are mostly really, really certain of our conclusions. And that's just the people making good-faith efforts to interpret the data; once we throw in the people trying to push a particular agenda, matters get even worse.

My point is this-- anyone who thinks that we'll be able to just say "Well, here's the evidence..." and that just settles everything is dreaming. All evidence is not created equal, and not all interpretations of evidence are created equal.

Newton is no longer king.

I get the desire, I really do. You want to be able to say, "Push down this lever, and X happens. Pull on this rope and the force is exerted through a pulley and the object on the end of the rope moves this much in this direction ." But levers and pulleys are simple machines in a Newtonian universe. Education is not a simple machine, and we don't live in Newton's universe any more.

We left Newton behind about a century ago, but the memos haven't gotten around to everyone yet. Blame that Einstein guy. Things that we thought of as absolutes like, ay, time and space-- well, it turns out how they kind of depend on where you are and how you're moving and even that can only be measured relative to something else. And then we get to chaos theory and information theory (one of the most influential reads of my life was Peter Gleick's Chaos) and the science that tells us how complex systems work, and the short version is that complex systems do not work like simple machines. Push down the "lever" in a complex system and you will probably get a different result every time, depending on any number of tiny uncontrollable variables. Not wildly variable, but not straight line predictable, either (we'll talk about strange attractors some time).

Students and teachers in a classroom are a complex machine indeed. Every teacher already gets this-- nothing will work for every student, and what was a great lesson with last year's class may bomb this year.

But there are people who desperately long for teaching to be a simple machine in Newton's world. I've been following reading debates for weeks and there are people in that space who are just so thoroughly sure that since Science tells us how the brain works re: reading, all we have to do is put the same Science-based method of teaching reading in the hands of every teacher, then every student will learn to read. It's so simple! Why can't we do that? Hell, Mike Petrilli floated for five minutes the notion of suing teacher prep schools that didn't.

The standard response to this is that teaching isn't a science. But I'll go one better-- it is science that tells us that such a simple Newtonian machine approach will not work. The dream some evidence-based folks have that we'll just use science to determine the best practices via evidence, and then just implement that stuff-- they are misunderstanding both teaching and science.

The federal definition of evidence-based is dumb.

The ESSA includes support for evidence-based practices, and it offers definitions of different levels of evidence-basededness that are.... well, not encouraging.

Strong Evidence means there's a least one good research paper that suggests the intervention will improve student outcomes (which, of course, actually means "raise test scores") or a related outcome (which means whatever you want it to). There should be no legit research that contradicts the findings, it should have a large sample, and the sample should overlap the populations and settings involved. In other words, research about rural third graders in Estonia does not count if you're looking for an intervention to use with American urban teens.

Moderate Evidence is one good "quasi-experimental study" and then all the other stuff applies. Not really clear what a quasi-experimental study is, but the department still considers moderate evidence good enough.

Promising Evidence requires a correlation study because (and this really explains a lot) even the federal government doesn't know the difference between correlation and causation. I just smacked my forehead so hard my glasses flew off.

Demonstrates a Rationale, like Promising Evidence, somehow doesn't appear on the No This Doesn't Count list. All this means is you can make an argument for the practice.

All four of these are enshrined in ESSA  as "evidence-based," even though a layperson might conclude that at least two of them are sort of the opposite of evidence-based.

So do we throw out the whole evidence based thing?

Nope. Having evidence for a practice is smart, and it's mostly what teachers do. I don't think I've ever met a teacher who said, "Well, this didn't work for anybody last year, so I'm going to do it again this year." No, teachers watch to see how something works, and then, like any scientist, accept, reject or modify their hypothesis/practice. This, I'd argue, is how so many classroom teachers ended up modifying the baloney that was handed to them under Common Core.

Teachers are perfectly happy to borrow practices that they have reason to trust. The most powerful message implicit in a site like "Teachers Pay Teachers" is "I use this and it works for me." Government agencies and policy wonks who want to help disseminate best evidence-based practices can be useful, provided they're clear about what their evidence really is.

But evidence-based should not be elevated to the status of Next Silver Bullet That Will Fix Everything, and government definitely absolutely positively should not get involved in picking winners and losers and then mandating which will be used. In the dead-on words of Dylan Wiliam, "Everything works somewhere and nothing works everywhere."

Nor should we get trapped in the evidentiary Russian doll set, where we need evidence of the evidence really being evidence by collecting more evidence about the evidence ad infinitum. At some point, someone has to make a call about whether to use the practice or not, and that someone should be the classroom teacher. At that point, that teacher can begin collecting the evidence that really matters.