The Road Beyond The Test

I missed this piece when it first ran at KQED's education tab Mind/Shift, but Katrina Schwartz's article is worth looking at because it captures some of the inaccurate thinking that still surrounds the Common Core today.

Entitled "How To Teach the Standards without Becoming Standardized," the piece patches together some interviews at EduCon with Diana Laufenberg and  some others. Laufenberg teaches at a Philadelphia magnet school and also tells people about project based learning and "structuring modern learning ecosystems."

This was back in spring of 2014, so they could still talk about teacher "ambivalence" toward the Core rather than, say, hatred, and it is organized around a list of ways to avoid becoming standardized in your teaching. As is usually the way with such pieces, the advice is diplomatically worded versions of "ignore the standards stuff and just follow best practices in your professional judgment."

For instance, #2 is "Teach students to question. When kids develop effective questioning techniques they become active partners in constructing learning." That doesn't really have anything to do with the Core; it's just a decent piece of teacher advice.

But buried amidst the educational wonder bread is a piece of bad advice that KQED liked well enough to turn into a pull quote, but which captures a major misconception about the Core. It's a Laufenberg quote, and while she says some sensible things elsewhere in the article, this is not one of them:

Teach past the test to this other meaningful, creative work and you will get the test, but you’ll get all this other stuff too.

I've heard this sentiment expressed many times over the past decade, particularly from administrators. It's a pretty thought, but it's wrong.

The assumption here is that the Really Good Stuff is just straight on past and on beyond the "get ready for the test" stuff. Like if you're starting from Chicago and you have to go Cleveland, but you really want to go to Pittsburgh, so you just stop in Cleveland then hop on I-76 and head to Pittsburgh. Easy peasy. Only the real analogy is, starting from Chicago you're required to go to Tampa, but you really want to go to Seattle. Tampa is not on the way, not even a little.

To get to good writing, you do not test-pleasing writing, just a little more so. Test writing and authentic quality writing are two entirely different things that just happen to have a superficial resemblance to each other because they look like words arranged in sentences arranged in paragraphs. But as a classroom teacher, I would literally tell my students, "I'm now going to teach you some things to use for testing, but you should not ever use these in any other writing situation." The behaviors needed to game a writing portion (e.g. parrot the prompt, use big words, write a bunch even if it's repetitious, never worry about accuracy or internal logic) are not desirable features in real writing.

Nor is the ability to respond immediately to a multiple-choice question about a short reading excerpt taken out of context a step on the path to mature, reflective analysis of a full-sized work of literature.

This notion that the path to the test is at best a small detour and after we've touched that base we can head on into the Land of Actual Education is a snare and a delusion. The path to test readiness is a dead end' the road goes no further and all that lies beyond is just a vast, barren wasteland.

ICYMI: Tech Sunday Edition (9/23)

No, not ed tech. It's tech Sunday, the kickoff of show week for the local theater production of The Producers that I'm directing. So I may be a little more scarce this week, but I've still collected a few things for you to read.

The Case Against High School Sports

Need to start a big argument? This article should do the trick-- but it's also some things to think about.

Learning from What Doesn't Work in Teacher Evaluation

Audrey Amrein-Beardsley shares some of the lessons of bad teacher evaluation (cough cough VAM).

Finding the Holy Grail in Poverty Mining

More scary news from the world of digital balls and chains. How exactly can the rich profit from poverty and data?

Bring Me a Higher Love

Jose Luis Vilson reminds us again of the place of the L word in teaching.

13 Things I learned While Blogging        

Nancy Flanagan remains one of my blogging heroes-- and she's pulling up stakes at Ed Week and moving out on her own. Her tenure remains one of the great runs in edublogging; I look forward to reading her without a paywall.

Rep Eddie Farnsworth makes a killing in AZ charters

Arizona remains one of the great places to run a charter school scam, particularly if you're a legislator there. Here's how one guy has played the game and made himself rich while pretending to be interested in education.

Behind Closed Doors    

Sarah Blaine has been quiet on the blogging front for a while, but she returns with a vengeance, looking at a New Jersey legislator's insistence that the decisions about the future of PARCC should be made far away from the prying eyes of parents, educators, and taxpayers.  

Who Is Behind Leaders in Education PAC    

The indispensable Mercedes Schneider once again does the research to peel back the layers of a reformster group and discovers the same old rich folks.

The Educational Outcomes Fund  

A plan for using education to strip more money from Africa and the Middle East.

A Teacher's Thoughts at 2 AM  

An honest and open reporting by Cori Anderson-Lais of her inner dialogue struggling with several school issues.

How Pushback Against Reform Is Used To Push Reform Forward

The pushback against education reform ideas like Common Core and test-centered accountability has brought together a broad assortment of voices, from the right to the left, from supporters of public education to avid home schoolers. But the opposition to some forms of education reform also includes one other group--advocates of newer education reform.

It's called jiu-jitsu

Take this piece, recently being recirculated on the interwebs, entitled "The Testing Emperor Finally Has No Clothes." Writer Bruce Dixon wastes no time getting straight to the point-- he's talking about the "tyranny of testing." He's speaking to an Australian audience, but his criticisms are recognizable to U.S. and British educators critical of the practice of using Big Standardized Tests to measure students and schools and teachers.

Standardized testing policy is "intrusive, devisive, deceitful" and is "fast turning teachers into lab rats." It's an "insidious virus" based on "chicanery." It fosters the false impression that learning is a competitive sport. It "kills curiosity and penalizes diversity." He quotes writers like John Holt, Anya Kamenetz, Deborah Meier and Alfie Kohn, all critics of test-centered schooling. He invokes Finland as an education exemplar.

And he points out that the testing industry is big money, pushing the testing regimen as a giant cash cow. The choice, Dixon suggests, is between lobbyists and learners.
Up until this point in the article, Dixon's argument could have been written by any of the army of advocates for public education. But Dixon is not a soldier in that army, and he signals it as he starts to consider alternatives to a testing regimen. "Our modern world demands," he says, "a shift in thinking about credentials at every level, how and why they are awarded but more importantly why."

Dixon cites the Mastery Transcript Consortium, a group that proposes a new way of reporting student achievement based on what it calls "Mastery Credits." This is a version of what are sometimes called micro-credentials or badges. The idea behind them is to break down every bit of learning into one particular item, and once you pass some sort of competency test, a little badge goes in your permanent digital record (not unlike earning an achievement on your Xbox game). In the super-deal version, all of this is both delivered by and recorded via computer and the cloud, so that you can earn them anywhere, any time, and your digital "transcript," perhaps stored blockchain style can be accessed by any future employers, the government, and anyone who pays for the privilege, forever. If you want to know more about this, look at speculative video about The Ledger.

As you might have figured out, this approach to learning renders a traditional public school virtually obsolete and unnecessary.

So who is Bruce Dixon? He's the cofounder and president of the Anywhere Anytime Learning Foundation and a consultant who talks to education departments and tech companies and advocates for 1-to-1 learning, an approach that puts one computer in the hands of every student.

Another name for the mastery based approach is Competency Based Education, and another name for that is Personalized Learning, which many education experts see as Education Reform 2.0. Ironically, one of the great marketing tools for Education Reform 2.0 has been Education Reform 1.0. Bruce Dixon is just one example of someone whose using dissatisfaction with Education Reform 1.0 to pitch his 2.0 products.

If you are someone who is unhappy with test-centric schooling and the data gathering that goes with it, you should take a careful look at anyone who hopes to rescue you. The CBE you're considering may well get rid of the once-a-year Big Standardized Test--and replace it with small standardized tests every day. It may be heavily computer-based, and therefor hoovering huge amounts of data about your child. And rather than giving control back to your local school and your child's teacher, it may strip the local school of even more control and reduce the classroom teacher to an aide who monitors the software and keeps students on task.

And as always, the best question to ask someone who promises you to rescue from one Awful Thing is, "What are you selling?"

Originally posted at Forbes   

Field Guide To Bad Education Research

Folks in education are often  criticized for not using enough research based stuff. But here's the thing about education research-- there's so much of it, and so much of it is bad. Very bad. Terrible in the extreme. That's understandable-- experimenting on live young humans is not a popular idea, so unless you're a really rich person with the financial ability to bribe entire school districts, you'll have to find some clever ways to work your research.

The badness of education research omes in a variety of flavors, but if you're going to play in the education sandbox, it's useful to know what kinds of turds are buried there.

The Narrow Sampling

This is the research that provides sometimes shocking results-- "Humans Learn Better While Drinking Beer." But when you look more closely, you discover the sample size lacks a little breadth-- say, fifteen Advanced Psychology male college juniors at the University of Berlin. These may be experimental subjects of convenience; the above researcher may have been a U of B grad student who worked as a TA for the Advanced Psychology course.

Generally these narrow projects yield results that are not terribly useful, but if you're out shopping for research to back whatever you're selling, these can often provide the "research base" that you wouldn't otherwise find.

The Meta Study

Meta research involves taking a whole bunch of other studies and studying the studies in your study. The idea is to find patterns or conclusions that emerge from a broad field of related research. Met research is not automatically bad research. But if the meta researcher has gone shopping for studies that lean in his preferred direction, then the pattern that emerges is-- ta-da-- the conclusion he went fishing for.

This is a hard thing to check. If you know the literature really well, you might look for which studies are not included. But otherwise just keep a wary eyeball out.

The Not Actually A Study

These are cranked out pretty regularly by various thinky tanks and other advocacy groups. They come in nice slicky-packaged graphics, and they are not actual research at all. They're position papers or policy PR or just a really nicely illustrated blog post. There are many sleight of hand tricks they use to create the illusion of research-- here are just two.

Trick One: "Because there are at least ten badgers in our attic, many of the neighbors took to drinking pure grain alcohol." There will be a reference for this sentence, and it will provide a source for the number of badgers in the attic. Nothing else, including the implied cause and effect, will be supported with evidence.

Trick Two: "Excessive use of alcohol can lead to debilitating liver disease. The solution is to sell all alcoholic beverages in plastic containers." References will shore up the problem portion of the proposal, establishing clearly that the problem is real. Then the writers' preferred solution will be offered, with no evidence to support the notion that it's a real solution.

The Not Really A Study is also given away by the list of works cited, which tend to be other non-studies from other advocacy groups (or, in the case of ballsy writers, a bunch of other non-studies from the same group). No real academic peer-reviewed research will be included, except a couple of pieces that shore up unimportant details in the "study."

The Thousand Butterfly Study

Like studies of other human-related Stuff (think health-related studies), education studies can involve a constellation of causes. When researchers study data from the real world, they may be studying students over a period of time in which the teaching staff changed, new standards were implemented. administration changed, new standardized tests were deployed, new textbooks were purchased, the cafeteria changed milk suppliers, a factory closed in town, a new video game craze took off, major national events affected people, and any number of imponderables occurred in student homes. The researcher will now try to make a case for which one of those butterflies flapped the wings that changed the weather.

Some butterfly researchers will try to create a compelling reason to believe they've picked the correct butterfly, or what is more likely, they will try to make a case that the butterfly in which they have a vested interest is the one with the power wings. This case can never not be shaky; this is a good time to follow the money as well as the researcher's line of reasoning.

The worst of these will simply pretend that the other butterflies don't exist. The classic example would be everyone who says that the country has gone to hell since they took prayer out of school; crime rates and drug use and teen pregnancy, the argument goes, have all skyrocketing as a result of the Supreme's decision-- as if nothing else of importance happened in 1962 and 1963.

The Bad Proxy Study

Education research is tied to all sorts of things that are really hard, even impossible to actually measure. And so researchers are constantly trying to create proxies. We can't measure self-esteem, so let's count how many times the student smiles at the mirror.

Currently the King of All Bad Proxies is the use of standardized test scores as a proxy for student achievement or teacher effectiveness. It's a terrible proxy, but what makes matters worse is the number of researchers, and journalists covering research, who use "student achievement" and "test scores" interchangeably as if they are synonyms. They aren't, but "study shows humus may lead to higher test scores" is less sexy than "humus makes students smarter."

Always pay attention to what is being used as a proxy, and how it's being collected, measured, and evaluated.

The Correlation Study

God help us, even fancy pants ivy league researchers can't avoid this one. Correlation is not causation. The fact that high test scores and wealth later in life go together doesn't mean that test scores cause wealth (wealth later in life and high test scores are both a function of growing up wealthy). The best thing we can say about bad correlations is that it has given rise to the website and book Spurious Correlations.

Just keep saying it over and over-- correlation is not causation.

The Badly Reasoned Study and The Convenient Omission Study

For the sake of completeness, these old classics need to be included. Sometimes the researchers just follow some lousy reasoning to reach their conclusions. Sometimes they leave out data or research that would interfere with the conclusion they are trying to reach. Why would they possibly do that? Time to follow the money again; the unfortunate truth of education research is that an awful lot of it is done because someone with an ax to grind or a product to sell is paying for it.

The Badly Reported Study

Sometimes researchers are responsible and nuanced and careful not to overstate their case. And then some reporter comes along and throws all that out the window in search of a grabby headline. It's not always the researcher's fault that they appear to be presenting dubious science. When in doubt, read the article carefully and try to get back to the actual research. It may not be as awful as it seems.

Keep your wits about you and pay close attention. Just because it looks like a legitimate study and is packaged as a legitimate study and seems to come from fancy scientists or a snazzy university-- well, none of that guarantees that it's not crap. When it comes to education research, the emptor needs to caveat real hard.

Digital Dictatorship Has Arrived

Here's a rarity-- I don't have a lot to add to this article, but you really need to see it. This is what the triumph of the data overlords looks like.

What may sound like a dystopian vision of the future is already happening in China. And it’s making and breaking lives.

The Communist Party calls it “social credit” and says it will be fully operational by 2020.

Within years, an official Party outline claims, it will “allow the trustworthy to roam freely under heaven while making it hard for the discredited to take a single step”.

A social credit score. Constant video monitoring of all citizens, complete with facial recognition. All behavior, choices, actions recorded and mined for data as part of your personal record, boiled down to a simple score. Smartphone and online actions monitored and recorded. Your shopping habits tracked. And of course your school, medical and employment records included as well. As the article puts it, "no dark corner in which to hide." All included in your social credit score, which will determine how you get to live your life.

Oh, and the scores of the people you associate with-- that will factor in as well.

Possibly scariest part? The woman who is profiled in the article is cool with all of this. It makes her feel safe. And, truth be told, she expects to be a winner in this game-- the old "only people who are doing bad things worry about surveillance and law enforcement"-- which only works if you think the people running the system share your values and priorities. But someone has to decide what characteristics are displayed by a Good Citizen.

Read this. If you don't read anything else today, read this. Big Brother was a slacker.

Non-wealthy People and Choice

When it comes to school, the poor, the argument goes, should have the same choices that wealthier folks have. The ability to choose a neighborhood gives wealthier folks the ability to choose a school, so even folks who attend public school are making use of school choice, the argument goes.

Let's skip over the usually-ignored part of that argument, which suggests that the problems of school selection could be addressed via zoning. Break up the last bastions of redlining, and put low cost housing in every neighborhood, including the ritzy ones and voila! everyone can exercise real estate based choice. I wonder why we never talk about that solution.

Instead, the preferred solution is to set loose the power of the free market to provide the non-wealthy with all sorts of choicey alternatives, a rich buffet of options. Reformsters used to say that choicey competition would create excellence as well, but that's no longer part of the pitch. Choice need not promote excellence; it's enough for reformsters that choice promotes choice.

It doesn't matter; any way you frame it, you run up against the same problem-- choice will not accomplish what its fans say it will accomplish.

The problem is that the free market is not a friend of poor people.

Oh, it likes them when it comes to marketing. Note-- the unwealthy are not stupid and they are not lazy, but they are busy just trying to hold things together between jobs and families and too few resources. Just the mechanics of being a family with two or three jobs but just one car can make for a very busy week, People who are spending all their energy just to tread water don't have a lot of time to extensively research advertising and PR claims.

Add to that Greene's Law: The free market does not foster superior quality; the free market fosters superior marketing. The market has a vested interest in making sure that consumers don't make informed choices, or at the very least, choices that are informed the way the marketeers want them to be informed.

So well-informed carefully-researched decisions uninfluenced by spin and puffery are not a very common thing in the marketplace.

But the free market is also not a friend of the non-wealthy because, well, they don't have much money. And that is important because of another True Thing about the market--

The market does not provide consumers with choices because it thinks those would be nice choices to have; the market provides the choices that businesses think they can make money providing.

My town has long needed some kind of youth facility, whether it be a youth-focused dance club or a specialized recreational facility. People have been saying it as long as I've lived here. We still don't have one. This is not because of any government regulations or state-sponsored monopoly or other market impediment. It's because no business thinks they can make money, as demonstrated by the two or three who have attempted it and then closed up shop because they couldn't make money doing it. We could talk about why they don't make money, but my point is that as much as we want it, as much as we would benefit from it, the free market is not providing it.

The free market is not Santa Claus. It does not provide goods and services because people need them or even deserve them.

Combine that truth with the lack of money in poorer communities, and you have a problem with the choice theory of action.

Can you name one kind of business that is providing poor communities with a rich buffet of choices. Maybe the fast food industry, but that encompasses a range of choices that go from A to B-- upscale restaurants are not in there tryin to enrich the choice list. Not supermarkets or other food providers; their absence from poor communities is why there's such a thing as a food desert. Auto dealers? Computer hardware stores? Dress shops?

Whatever the sector, what you find are a range of downscale choices with a business plan of catering to people without a lot of money.

There are businesses that specifically target such communities-- Walmart, and now Dollar General (five years ago there were none in my area-- now there's a DG roughly every ten miles on local roads) aim for the non-wealthy crowd, and again, that crowd is offered what Walmart execs figure they can pay for. Do wealthy people go shopping at Walmart in pursuit of top quality. Does Louis Vutton open shops in poor areas because those folks also deserve a chance to check out overpriced luxury luggage? No, because that's not how the free market works.

A school choice system will claim to circumnavigate this by using government money to pay for the schooling, thereby artificially inflating the wealth level of the families involved. It's almost like choice creates a new entitlement to send students to private school at public expense, but you'll never hear choice fans describe it that way because they are mostly conservatives and the "entitle--" word is verboten.

At any rate, that doesn't really help because in many states, the per pupil spending for education is already too little to really support a school, and then, anyone who's operating a free market business expects to keep some of that revenue as salaries or profit or, if they're part5icularly shady, fun vacations and a generally cushy lifestyle. So now there's really not a lot of money left to spend on the choice school. Some of the charters deal with this by hitting up parents and wealthy donors for some more cash. The vast majority of voucher schools are church related, so the church can help chip in. But a less wealthy community is limited in the ways it can help the charter business stay solvent.

Choice schools will make decisions based on business concerns. For instance, enrolling students who have special needs-- but special needs that are not expensive to deal with. Student performance is part of the marketing, so it becomes important to push out students whose performance will mess up the PR.

All of these considerations affect how charters approach doing business in not-so-wealthy communities. Advocates will point to some charters in those communities and say, "See? Charters are providing the same options as wealthy families have." And I could run on at greater length about why that's not true, but it's quicker to ask just how many wealthy families consider these charters as good a choice as any other they've considered and decide to send their children there. (And the answer is that occasionally that does happen-- and it's Step One in gentrifying a neighborhood by pushing the locals out.) Nobody is pulling their kids out of Phillips Exeter in order to enroll the child in Success Academy.

So what you end up with is many top educators or schools or just plain entrepreneurs saying, "Well, I'm not going to try to make money running a school in that neighborhood" and a few, maybe just one, saying, "Yeah, I think I can do this cheaply enough to make a buck at it." And some students will get a small choice, but not the choice they imagine, just the choice that "make a buck" company wants to offer. And choice fans will say, "Yes, but we got a better education to some of those students," and I will say, "By leaving everyone in a system that wastes a bunch of money that could have been used to educate folks."

The public education system is riddled with inequities as it stands. A choice system doesn't propose fixing that problem; it just promises to let a few more kids get in on the high side of the inequity, while making the low side worse off. In the meantime, choice turns out to mean "give businesses the choice of cashing in on the education racket" while providing little in the way of actual, legitimate choices for the non-wealthy.

Covering The Education Horse Race

This. This right here is the kind of education policy coverage that makes me cranky. (Okay, crankier.)

I'm looking at a piece by Matt Barnum at Chalkbeat, and before I lapse into spleen ventage, let me say that Barnum often does excellent education policy journalism that avoids everything I'm about to bitch about. The following is definitely NOT meant to be a "Matt Barnum sucks" piece. But this particular article tripped many of my triggers.

"What’s next for the Laurene Powell Jobs-funded effort to rethink American high schools" has many of my least favorite features of ed reform journalism.

First, it exists. Imagine how odd it would be to open up a website and read the piece "What the guy who fixed my muffler thinks should be done about US education" or "Ed reform policy promoted by hairdresser is Pahrump." Or even, "What Mrs. McTeachalot in  Room 123 at PS 15 thinks should be done about ed policy."Laurene Powell-Jobs has no expertise in education; what she has is a giant pile of money that allows her to try to buy influence and control of a piece of the system. I get that, because she was able to buy an hour of tv packed with stars and because she was able to get schools to dance to her tune in exchange for a pile of money, she is newsworthy. But I don't have to like it, and I don't. Nor do I like that she is covered uncritically, as if her wealth is an actual qualification to try to set education policy.

Second, we get all the usual suspects in quotes. Some Maine parent gets an anonymous quote, but when you need an ed policy quote toot de suite, call Mike Petrilli or Rick Hess. In this case, Hess got the call. Skeptical voices from outside the ed reform community are not included [Update: Barnm correctly points out that Warren Simmons, who's quoted as an Annenberg guy, is now with NEPC. So this article is partially absolved.]

But mostly what trips my trigger is the horse race coverage. We talk about this during every election cycle-- it's the kind of coverage that looks at whether or not a candidate's proposed policy is gaining traction, polling well or poorly, and just generally helping or hurting. In other words, we get coverage of how the policy is affecting the race, but nothing about whether it's actually an effective policy or not. Horse race coverage tells us all about who's winning, but nothing about who we might want to actually root for.

Barnum's piece discusses Powell-Jobs's education ideas, but all the discussion is like this

Simmons said one challenge of the approach is that simply creating a handful of successful schools doesn’t mean their approaches will catch on. “That viral theory of action has failed time and time again,” he said.

When the article talks about XQ schools being "successful," it doesn't mean "successful at educating students." It means "successful at winning the horse race and giving Powell-Jobs more traction in the ed reform world." By the end of the article, the average reader has no idea whether XQ schools are on to something really great for students, or if they're just full of expensive hooey. We have a better sense of whether or not Powell-Jobs is becoming influential, but no idea at all if she should be.

This is horse race coverage-- looking at how well a policy or the policy's patron is doing, but not at whether that policy is valid, effective, or able to deliver what it promises. Horse race coverage of pharmaceuticals would cover sales figures, but not talk about whether the drug actually worked or not (unless the reporter thought that was affecting the sales).

It's doubly frustrating because, as with political races, education policy horse races have real consequences for real people. When a President makes policy choices about health care or welfare, it's not just important because it influences his poll numbers-- it's important because people are going to die if he chooses badly. When some rich person decides they wan to appoint themselves a national education tsar, their fiddling around and privatizing in search of influence doesn't just affect their personal standing-- it screws with the actual education of live human students.

When I'm emperor of the world, nobody will be allowed to write horse race coverage without including a critical evaluation of the policies being discussed and a look at the effects of those policies on real people. And if that makes the piece too long, the horse race stuff is the first thin to get cut.