I'm not a huge fan of meta-posting on the subject of navigating the education debates, but it's unavoidable because the ed debates involve such a mish-mosh of tribes.
I agree with what X says about Y, but I'm worried he's just trying to create support for Z-- particularly because he belongs to Organization Q.
I've seen some variation on this a gazillion times-- from more than one side of the debates.
It's understandable because all sides have always consisted of alliances. It's people who can only really agree for half a sentence. For instance, common core opposition included a bunch of people who could start a sentence, "Common Core is a terrible idea, and therefor..." but then would finish the sentence with "...it should be abolished so that public schools can return to their proper mission" and "...it's the final proof that the government can't be trusted to run schools at all."
Sometimes the alliances are barely alliances at all. Lots of folks see the Big Standardized Test as a huge blight on public education, but while some folks want to see those tests abolished, or at least reduced in importance, others see the opposition to the BS Test as a good motivation for jumping into Personalized Competency-Based Learning Education, which is just another heaping pile of corporate reform.
Many people cope with this kind of confusion and tension by simply sticking to their tribe, or focusing on which tribe they oppose. If it comes from our team, it must be good, and if it comes from that other team, it must be bad.
But there are several problems with this.
First of all, tribalism leads to focusing not on the issues or the message, but on the classification of the source. Instead of listening to what Pat says and deciding whether it's bunk or not, I spend my time trying to suss out which team Pat belongs to.
Second, and perhaps worse, it leads to people agreeing with really dumb things just to stay on the right "side." This is the current problem of many so-called conservatives-- they've defined conservatism or "the right position" as anything that makes liberals upset, which has led to sudden bizarre changes in direction like "Russia is swell" and "cheating on your third wife with a prostitute is totally okay."
Ultimately we end up with people hugely overthinking things to their own detriment. We are sitting in the living room of a house that is on fire, and we can see through the front door a swimming pool and an ambulance, and somebody is arguing, "How do we know they didn't set the fire on purpose so that we would go jump into the swimming pool which is actually filed with sharks we just can't see from here? Let's just stay here in the burning house. That will show them." Or even worse, "It was one of their people who yelled that this house is on fire, so I'm not convinced these flames are rea." This is crazy talk. If the house is on fire, get out. Once you're out, check the swimming pool out carefully and make your next choice accordingly.
Pay attention. Watch, listen, and think. Use every available piece of information to get a picture of the road ahead, but don't just make shit up, and don't just take somebody's word for it.
Most of all, keep your eyes on the big picture and the important issues. Don't put tribe ahead of that.
[Update: It seems worth adding that the impulse of many people in the comments section was to try to assign me to a particular tribe and then reject or accept what I said based on that assignment. That tribe assignment included the assumption that if I said something mean about conservatives, then I must belong to the anti-conservative tribe. None of that discussion really considers whether what I said was actually true or not, or if it could be negated by saying something similar about the other tribe. Though, for the record, anyone who reads regularly would find plenty of criticism of the Democratic brand of misbehavior and the faux progressivism that masquerades as "liberal" school reform. I just happened to pick up the most recent phenomenon of conservatives selling their principles for a ride on the Trump train.
At the end of the day, that's another side effect of tribalism-- in addition to actual conservatives and liberals, we have a lot of people whose only real interest is self-interest, and they'll happily pretend to be a member of whatever tribe they think will further their cause.]
Monday, April 16, 2018
Sunday, April 15, 2018
ICYMI: Finally Spring Edition (4/15)
Your reading for the week. Remember to pass along what you value. And don't forget to peruse the blogroll in the righthand column.
Dear Betsy DeVos
Marie Corfield responds to Betsy DeVos's comments about striking Oklahoma teachers. This is what it means to serve the public.
Mute the Messenger
From the Texas Observer-- here's what happened when a Texas professor spoke out against standardized testing. Pearson came after him.
What Teacher Strikes May Teach Democrats about Education Politics
The Democratic Party abandoned public education and the teachers who work there a while ago-- will the current wave of teacher strikes make them rethink their strategy?
Now Watch Republicans Blame Obama for Failed Education Reform
Ed reform isn't quite as beloved as it once was, and one sign of that is the GOP folks trying to pin it on Obama. Jeff Bryan as always brings the research and the insight.
Put Out To Pasture
Nancy Bailey walks us through all the Secretaries of Education, the policies they pursued, and where they are now. It's a useful, but not encouraging, history lesson.
Time To Ditch the SAT
From US News, another compelling argument against the old SAT warhorse.
Why the School Spending Graph Betsy DeVos Is Sharing Doesn't Mean What She Says It Means
Betsy DeVos has been using an exhibit from the Museum of Bad Graphs to make her point. Matt Barnum explains why it is misleading.
Why American Students Haven't Gotten Better At Reading in 20 Years
A compelling piece from the Atlantic making the not-new point that reading can't be taught as a set of discrete skills free of content or context.
Remembering the Holocaust
If there was ever an education story, this is it. A new study suggests that a whole bunch of Americans simply don't know about the Holocaust. Not deny it-- just don't know about it.
Billionaire Offers To Buy High School
If you missed this story, Valerie Strauss has the details. A Pennsylvania billionaire offered his alma mater $25 million-- if they'd put his name on the school, let him set some curriculum, and keep it all secret. It's a truly bizarre tale.
Wakanda Schools
Jose Luis Vilson finally saw Black Panther, and it gave him some thoughts about education and shared prosperity.
Dear Betsy DeVos
Marie Corfield responds to Betsy DeVos's comments about striking Oklahoma teachers. This is what it means to serve the public.
Mute the Messenger
From the Texas Observer-- here's what happened when a Texas professor spoke out against standardized testing. Pearson came after him.
What Teacher Strikes May Teach Democrats about Education Politics
The Democratic Party abandoned public education and the teachers who work there a while ago-- will the current wave of teacher strikes make them rethink their strategy?
Now Watch Republicans Blame Obama for Failed Education Reform
Ed reform isn't quite as beloved as it once was, and one sign of that is the GOP folks trying to pin it on Obama. Jeff Bryan as always brings the research and the insight.
Put Out To Pasture
Nancy Bailey walks us through all the Secretaries of Education, the policies they pursued, and where they are now. It's a useful, but not encouraging, history lesson.
Time To Ditch the SAT
From US News, another compelling argument against the old SAT warhorse.
Why the School Spending Graph Betsy DeVos Is Sharing Doesn't Mean What She Says It Means
Betsy DeVos has been using an exhibit from the Museum of Bad Graphs to make her point. Matt Barnum explains why it is misleading.
Why American Students Haven't Gotten Better At Reading in 20 Years
A compelling piece from the Atlantic making the not-new point that reading can't be taught as a set of discrete skills free of content or context.
Remembering the Holocaust
If there was ever an education story, this is it. A new study suggests that a whole bunch of Americans simply don't know about the Holocaust. Not deny it-- just don't know about it.
Billionaire Offers To Buy High School
If you missed this story, Valerie Strauss has the details. A Pennsylvania billionaire offered his alma mater $25 million-- if they'd put his name on the school, let him set some curriculum, and keep it all secret. It's a truly bizarre tale.
Wakanda Schools
Jose Luis Vilson finally saw Black Panther, and it gave him some thoughts about education and shared prosperity.
Saturday, April 14, 2018
The Testing Charade: Buy This Book
This is probably going to be a long post, so let me get the most important parts out of the way first. The books is The Testing Charade: Pretending To Make Schools Better.
Daniel Koretz has published a book that gathers between two covers all the things wrong with the test-centered accountability under which we all currently suffer. His explanations are clear, and his illustrations are vivid. If you want to clarify your thinking about testing-- if you know something is wrong, but it's hard to wrap your head around-- you need this book. If a loved one or a colleague or an administrator asks you, "So what's the big deal, anyway? Why get upset over some simple standardized testing and accountability?" you need to hand them a copy of this book.
Buy this book.
Other things you should know up front. Koretz is the real deal. He is a widely recognized testing expert and scholar based at the Harvard Graduate School of Education. He is not some guy I found who presents my point of view; long time readers know that I am a standardized test radical (kill them all with fire) and Koretz is not-- he sees valuable uses for the big tests done right, and he pushes some ideas I don't agree with. Nevertheless, I'm telling you to buy this book.
Okay.
There are fourteen chapters in this book. I'm going to talk about all of them, with quotes, because Koretz says a bunch of important stuff. If any of this lights a fire in your brain, my advice is simple. Buy this book.
1: Beyond All Reason
Pressure to raise scores on achievement tests dominates American education today.
In fact, the blurring of that line is one of the sobering parts of this chapter,
Koretz lists three types of bad test prep-- reallocating time between subjects, reallocating time within subjects, and coaching. Schools shift time, both between classes and within classes, to things that count on the test. No band for you, Pat-- you need to spend a period in ELA test remediation. Today, class, we're going to skip over chapter five, because that stuff is not on the test. Coaching is teaching tricks that have no use except for taking the test. For example, process of elimination, which seems harmless enough, but allows a student to get a correct answer when the student would never have been able to come up with the correct answer on her own. Life is not multiple choice; process of elimination is a skill that is only useful for taking the test.
The most alarming part of the chapter addresses the idea that test prep is corrupting the idea of good teaching. Koretz has a disturbing bank of anecdotes about teachers who are told their is not to teach their subject, but to raise test scores in it. And like anyone else who has encountered new teachers in the last decade, he's met young teachers who were taught in college that test scores are their main purpose.
They were telling me that I was missing the boat by seeing test prep as something that competes for time with good instruction. In their experience, raising scores had become the end goal, the mark of a "good" teacher. To an alarming degree, they had been taught that test prep and good instruction are the same thing.
There are so many reasons this is bad, bad news, but here's one that Koretz spots
For the present, it indicates that one of the few checks against inappropriate test prep-- teachers' own understanding of the differences between test prep and good instruction-- has been eroded.
8: Making up Unrealistic Targets
Yikes. This is a depressing chapter. Koretz notes that the "trust most people have in performance standards is essential, because the entire system now revolves around them." And yet that trust is wildly misplaced.
Koretz outlines several methods of setting performance standards, but the important thing to remember is this:
There is another, perhaps even more important, reason why performance standards can't be trusted: there are many different methods one can use, and there is rarely a really persuasive reason to select one over the other.
And yet those different methods can produce vastly different results.
There are other problems.
A primary motivation for setting a Proficient standard is to prod schools to improve, but information about quickly teachers actually can improve student learning doesn't play much, if any, of a role in setting performance standards. When panels set standards, they are not given information about practical rates of improvement, and for the most part they are not asked to consider them.
In other words, they are set up to be the educational equivalent of an agricultural board that declares, "What we need is wheat that we can harvest six hours after it's planted. Do that!"
Also, standards are set mainly on the assumption that all kids are the same, with a goal of radically reducing the variability of achievement. I'm just going to sum up on this part: that's a dumb assumption.
9: Evaluating Teachers
Koretz is pretty brutal about the VAM-sauced test-based teacher evaluation systems. He allows that the old system had a problem in that everyone ended up looking great, but the new system surprised him because it solved problems with solutions so dumb it had never occurred to him that anyone would actually use them (e.g. evaluating teachers based on the scores of students they have never met).
This chapter includes Koretz's account of his visit to the Department of Education in Duncan's early days. You'll want to read that yourself.
How have we screwed up test-based accountability for teachers. Let Koretz count the ways:
* Taking test scores out of context (a "deliberate goal" of reformers, but "one of the main reasons the reforms have failed)
* Trying to use tests to explain, not just describe
* Using "Value-Added Modeling to evaluate teachers (here's as good an explanation as you'll find for why VAM can't possibly work)
* Rating teachers with the wrong test
* Teachers ratings are inconsistent across tests
* Teacher test scores are unstable over time (there are charts and specifics here that drive home how bad this effect is).
10: Will the Common Core fix this?
In a word, no.
Want a longer explanation?
It's not just the Common Core that has been dropped into schools wholesale before we gathered any evidence about impact; this has been true of almost the entire edifice of test-based reform, time and time again. I'll argue later that putting a stop to this disdain for evidence-- this arrogant assumption that we know so much that we don't have to bother evaluating our ideas before imposing them on teachers and students-- is one of the most important changes we have to make.
Koretz also argues that CCSS tests have actually been created in a way that makes them more predictable, and therefor more susceptible to all the Campbell's Lawian Test Preppery he just spent several chapters eviscerating. Koretz dismisses the one size fits allness of the Core, noting that one official once told him that the Core eliminated the distinction between career readiness and college readiness. "Rhetorically, perhaps, but not in actuality" responds Koretz.
It is a new flavor of the same old failed approach, gives one size fits all a "grandiose rhetorical wrapper." Underneath all the noise
the basic failed model of educational improvement remains unchanged: set arbitrary performance targets on standardized tests; apply them uniformly, without regard to circumstances; and reward and punish. Whatever its other virtues and vices, the Common Core hasn't changed this. This approach hasn't worked before, and it won't work with the Common Core.
11: Did Kids Learn More?
We know less about this most fundamental of questions than we probably should.
Why not? Partly because the test data is so subject to inflation that it can't be trusted. But also-- and I know I just ran a similar quote, but this point is important
There is a second reason for the dearth of information, the blame for which lies squarely on the shoulders of many of the reformers. Time after time they declared that they had figured out what would work, and they imposed it on students and teachers on a mass scale without taking the time to evaluate their programs first. It's analogous to a drug company saying that they have figured out, based just on their own beliefs and logic, which drugs will be effective and safe, so they can skip the time-consuming and expensive burden of actually gathering some evidence before selling it to you.
Koretz spends some time crunching some numbers, asking if students learned more, if they learned it because of test-centered accountability, and if what they learned justifies the "huge" costs of these policies (including costs like the corruption of instruction on a broad scale).
The answer is pretty clearly, "No."
12: Nine Principles for Doing Better
Here they are--
Pay attention to other important stuff.
Monitor more than student achievement.
Set reasonable targets.
Stop just kicking the dog harder.
Don't expect school to do it all.
Pay attention to context.
Accept the need for human judgment.
Create counterbalancing incentives.
Monitor, evaluate and revise.
13: Doing Better
Here Koretz works out his nine principles into a more specific action plan. He considers at length some of the usual models-in-other-countries. And he lays out some ideas.
Measure what matters. Okay, no bonus points for originality here, but then one of the subtexts of this book is that apparently you have to point out the obvious to people in education reform because some of them rush straight into doing things that are obviously stupid. Here also he and I disagree on a role for well-used well-made standardized tests. That's okay; I know I'm out in left field on this. But I do like this:
This is the first and one of the most difficult tradeoffs we face: to measure learning well and to give teachers better incentives, we will have to use measures that have serious drawbacks-- in particular, potential inconsistency from classroom to classroom and school to school.
Yes. The more perfectly something is universally standardized, the useful it is to an individual teacher.
We need to measure "soft skills" well. This will be hard, and there will be disagreement. Yup-- that's already happening. And he has a useful insight-- part of the reason we got standardized testing "was the notion that educators can't be trusted to evaluate schooling or other educators." But the soft skills measures will have to "give a substantial role to the judgment of professionals." Not standardized SEL tests.
We need a sensible accountability system. Most interesting detail here-- we need various measures that are NOT too closely aligned with each other. If everything's aligned together, everything canm be wrong together.
Use tests sensibly. One test cannot do everything.
Provide support to teachers. Monitor and make midcourse corrections (you know-- the way Common Core was specifically designed not to do).
Still here? Good for you.
Chapter 14 is a wrap-up and I'll skip that, just as I've skipped over many specifics and explanations. Really, if I had time to be a better reviewer, I would have given you a much more compact look at what Koretz has created here, which is nothing less than a scholarly, thoughtful, accessible explanation of how test-based reform has taken education into the weeds. You should read this book, and you should pass it on to other folks who care about education and want to both understand the problems and envision some solutions. This is a valuable work, and I'll be coming back to it again and again, and you should, too.
Daniel Koretz has published a book that gathers between two covers all the things wrong with the test-centered accountability under which we all currently suffer. His explanations are clear, and his illustrations are vivid. If you want to clarify your thinking about testing-- if you know something is wrong, but it's hard to wrap your head around-- you need this book. If a loved one or a colleague or an administrator asks you, "So what's the big deal, anyway? Why get upset over some simple standardized testing and accountability?" you need to hand them a copy of this book.
Buy this book.
Other things you should know up front. Koretz is the real deal. He is a widely recognized testing expert and scholar based at the Harvard Graduate School of Education. He is not some guy I found who presents my point of view; long time readers know that I am a standardized test radical (kill them all with fire) and Koretz is not-- he sees valuable uses for the big tests done right, and he pushes some ideas I don't agree with. Nevertheless, I'm telling you to buy this book.
Okay.
There are fourteen chapters in this book. I'm going to talk about all of them, with quotes, because Koretz says a bunch of important stuff. If any of this lights a fire in your brain, my advice is simple. Buy this book.
1: Beyond All Reason
Pressure to raise scores on achievement tests dominates American education today.
That's the first sentence of the first chapter, which looks at where we have arrived in the bizarre and extreme pursuit of school improvement via test scores. Koretz provides multiple examples of how this accountability approach has gone off the tracks, including tales of excellent schools given "failing" grades. The evidence of the failure of this system has, Koretz says, "been accumulating for more than a quarter century. Yet it is routinely ignored-- in the design of education programs, in public reporting of educational 'progress,' and in decisions about the fates of schools, students, and educators."
Don't make the mistake of thinking that these problems will disappear now that NCLB has finally been replaced. Test-based accountability was well established in this country before NCLB, and it will continue now that ESSA has replaced it. It is true that NCLB was a very poorly crafted set of policies-- a train wreck waiting to happen, some of us said when it was enacted-- and it did substantial harm....ESSA continues the basic model of test-based accountability, while returning to states just a fraction of the discretion they had in implementing this model before NCLB was enacted.
Koretz notes that he believes that standardized tests-- properly used-- can be valuable. And while he is going to damn the current accountability system, he is not arguing against all accountability.
2: What Is a Test ?
Everyone knows a test when they see it. However, understanding tests is very different from recognizing them, and unfortunately, many of the people with their hands on the levers in education don't understand what tests are and what they can and can't do.
Koretz offers the useful analogy that a test is like a poll-- a small sampling of a much larger domains, and only useful if the small sample is properly related to the larger area. "The items on a test matter only to the extent that they allow us to predict mastery of the larger subject area from which they were sampled." Standardized tests are not good tools for measuring full mastery, and Koretz lays out the reasons.
First, standardized testing has inherent limitations. Things like complex analytical thinking and problem solving aren't best assessed with a standardized test. Second, test authors must make large numbers of decisions about what is and is not included in test items. These decisions, some deliberate and some not, narrow what the test actually tests. This point, for Koretz, is huge. The sampling decisions introduce error (as in "margin of" and not "oopsies"), The samples will be incomplete measures. And perhaps most importantly, the sampling creates perverse incentives.
High stakes testing creates strong incentives to focus on the tested sample rather than the domain it is intended to represent. If you teach a domain better-- say, geometry-- scores on a good test of that area will go up. However, if you directly teach the small sample measured by a particular test-- for example, memorization of the fact that vertical angles are equal-- scores will increase, often dramatically, but mastery of geometry as a whole will not improve much, if at all.
This is as good an explanation as I've seen of why teaching to the test is a bad idea.
3: The Evolution of Test-Based "Reform"
You know we're going back to 1983 and A Nation at Risk, as Koretz tries to lay out how we arrived at the place where tests are the most central part of any school's everyday life. He notes that nations used as examples of test-based education are actually trying to back away from it, and that even the highest high-stakes testing countries don't do what we do, don't test so often, don't use test scores to evaluate teachers and schools.
He offers the term "measurement-driven instruction" which is shorthand for a world in which "improving performance on the specific test was to be the explicit goal, and higher-quality instruction would be the consequence. This was the tail wagging the dog."
This approach did not start with NCLB, but NCLB gave it national scale. And folks who thought Obama would provide relief "were quickly and sorely disappointed." Arne Duncan's big contribution was to get states to tie individual teacher evaluation to test scores, leading to "some of the most ludicrous uses of test scores." ESSA isn't going to help, other that in allowing some states to add broader measures to the mix.
Koretz gives test-based reformsters credit for good motives, but "Whatever their motives. the proponents were wrong. The reforms caused more harm than good."
Although he'll spend the rest of the book exploring the details, he gives the broad strokes here to explain why test-based reform has failed so badly. First, it "rewards far too narrow a slice of educational practice and outcomes." Second, it is too high pressure. And third is that it left almost no room for human judgment.
...teaching is far too complex a job to evaluate without any judgment, and many of the things we value most in schools aren't captured by tests.
4: Campbell's Law
The more any quantitative social indicator is used for social decision making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processe it is intended to monitor.
Well, you knew Campbell's Law would come up. But what you might not know (well, anyway, I didn't) is that Campbell actually predicted how the law would play out in educational testing:
But when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways.
For folks who don't quite get how Campbell's Law works, or why it applies, Koretz has some great examples of the law in action (my favorite: the old Soviet shoe factory story). And he tempers the observations about the Law in action in education by arguing that we have an obligation to consider the balance; some approaches will be more heavily hurt by Campbell's Law than others, and Koretz wants us to make sure that there isn't more damage than benefit.
5: Score Inflation
This was a helpful concept for me. For Koretz, "grade inflation" doesn't mean blowing up scores or moving the cut score goal posts-- it means raising test scores by techniques that do not actually raise the level that the tests purport to measure.
Koretz talks at length about the problems of trying to study test score inflation-- turns out that people who are politically invested in "good" results on the Big Standardized Tests are not interested in letting scholars study how inaccurate those good scores are.
Koretz addresses inflation throughout the book, and his view is nuanced-- on one end of the scale is flat-out cheating, where on the other end we have coaching. At one point, my bosses expected me to put up a poster of the "anchor standards" that would be tested. Would that count as an inflation factor? I'd say yes, because it was aimed at preparing for the test and not aimed at improving reading, writing, speaking and listening skills.
Koretz says that all you need to get inflated scores is a test that is predictable. If, for instance, you are poring through banks of old test questions in order to narrow down what kinds of questions your students need to be prepared for, that's causing score inflation. And if your administrators are requiring you to do that...well, they are part of the reason that this test-driven system is a failure. High stakes matter as a motivation to do test prep, but even in lower-stakes settings, Koretz found a culture of "applied anxiety" about test scores.
Beyond corrupting the scores of individual students through things like test prep, Koretz points out you can corrupt the group results as well. One way is to focus on bubble kids-- the ones who can be dragged across the cut score line. Another is to game the system by rigging which students are tested. And of course there's plain old cheating.
Koretz explains a less-obvious problem of inflation-- it varies from school to school and classroom to classroom, which means "we are identifying the wrong teachers, schools and systems as successes and failures." And it's also worth noting that low-income schools have a higher incentive to inflate, which in turn means that the students who arguably most need an "improved" education are instead getting the education most corrupted by the testing process. IOW, they most need a "real" education, and they are most likely to get a battery of test prep instead.
6: Cheating
By the end of NCLB, as many of us noted, there were only two types of schools-- schools that were failing and schools that were cheating. Measurement-driven instruction has created enormous incentive to cheat, and created some thorny ethical dilemmas. After all, which is worse-- cheating on the student score for a single BS Test, or allowing bad educational policy to take that child's school away from her?
Koretz has a whole batch of cheating tales here, and they are depressing, but necessary if we're going to understand what high-stakes testing pushes people toward.
7: Test Prep
There is no doubt that test-based accountability has resulted in a huge commitment of time and effort to test preparation, I can't be more precise, in part because people don't agree on the dividing line between test prep and regular instruction.
2: What Is a Test ?
Everyone knows a test when they see it. However, understanding tests is very different from recognizing them, and unfortunately, many of the people with their hands on the levers in education don't understand what tests are and what they can and can't do.
Koretz offers the useful analogy that a test is like a poll-- a small sampling of a much larger domains, and only useful if the small sample is properly related to the larger area. "The items on a test matter only to the extent that they allow us to predict mastery of the larger subject area from which they were sampled." Standardized tests are not good tools for measuring full mastery, and Koretz lays out the reasons.
First, standardized testing has inherent limitations. Things like complex analytical thinking and problem solving aren't best assessed with a standardized test. Second, test authors must make large numbers of decisions about what is and is not included in test items. These decisions, some deliberate and some not, narrow what the test actually tests. This point, for Koretz, is huge. The sampling decisions introduce error (as in "margin of" and not "oopsies"), The samples will be incomplete measures. And perhaps most importantly, the sampling creates perverse incentives.
High stakes testing creates strong incentives to focus on the tested sample rather than the domain it is intended to represent. If you teach a domain better-- say, geometry-- scores on a good test of that area will go up. However, if you directly teach the small sample measured by a particular test-- for example, memorization of the fact that vertical angles are equal-- scores will increase, often dramatically, but mastery of geometry as a whole will not improve much, if at all.
This is as good an explanation as I've seen of why teaching to the test is a bad idea.
3: The Evolution of Test-Based "Reform"
You know we're going back to 1983 and A Nation at Risk, as Koretz tries to lay out how we arrived at the place where tests are the most central part of any school's everyday life. He notes that nations used as examples of test-based education are actually trying to back away from it, and that even the highest high-stakes testing countries don't do what we do, don't test so often, don't use test scores to evaluate teachers and schools.
He offers the term "measurement-driven instruction" which is shorthand for a world in which "improving performance on the specific test was to be the explicit goal, and higher-quality instruction would be the consequence. This was the tail wagging the dog."
This approach did not start with NCLB, but NCLB gave it national scale. And folks who thought Obama would provide relief "were quickly and sorely disappointed." Arne Duncan's big contribution was to get states to tie individual teacher evaluation to test scores, leading to "some of the most ludicrous uses of test scores." ESSA isn't going to help, other that in allowing some states to add broader measures to the mix.
Koretz gives test-based reformsters credit for good motives, but "Whatever their motives. the proponents were wrong. The reforms caused more harm than good."
Although he'll spend the rest of the book exploring the details, he gives the broad strokes here to explain why test-based reform has failed so badly. First, it "rewards far too narrow a slice of educational practice and outcomes." Second, it is too high pressure. And third is that it left almost no room for human judgment.
...teaching is far too complex a job to evaluate without any judgment, and many of the things we value most in schools aren't captured by tests.
4: Campbell's Law
The more any quantitative social indicator is used for social decision making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processe it is intended to monitor.
Well, you knew Campbell's Law would come up. But what you might not know (well, anyway, I didn't) is that Campbell actually predicted how the law would play out in educational testing:
But when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways.
For folks who don't quite get how Campbell's Law works, or why it applies, Koretz has some great examples of the law in action (my favorite: the old Soviet shoe factory story). And he tempers the observations about the Law in action in education by arguing that we have an obligation to consider the balance; some approaches will be more heavily hurt by Campbell's Law than others, and Koretz wants us to make sure that there isn't more damage than benefit.
5: Score Inflation
This was a helpful concept for me. For Koretz, "grade inflation" doesn't mean blowing up scores or moving the cut score goal posts-- it means raising test scores by techniques that do not actually raise the level that the tests purport to measure.
Koretz talks at length about the problems of trying to study test score inflation-- turns out that people who are politically invested in "good" results on the Big Standardized Tests are not interested in letting scholars study how inaccurate those good scores are.
Koretz addresses inflation throughout the book, and his view is nuanced-- on one end of the scale is flat-out cheating, where on the other end we have coaching. At one point, my bosses expected me to put up a poster of the "anchor standards" that would be tested. Would that count as an inflation factor? I'd say yes, because it was aimed at preparing for the test and not aimed at improving reading, writing, speaking and listening skills.
Koretz says that all you need to get inflated scores is a test that is predictable. If, for instance, you are poring through banks of old test questions in order to narrow down what kinds of questions your students need to be prepared for, that's causing score inflation. And if your administrators are requiring you to do that...well, they are part of the reason that this test-driven system is a failure. High stakes matter as a motivation to do test prep, but even in lower-stakes settings, Koretz found a culture of "applied anxiety" about test scores.
Beyond corrupting the scores of individual students through things like test prep, Koretz points out you can corrupt the group results as well. One way is to focus on bubble kids-- the ones who can be dragged across the cut score line. Another is to game the system by rigging which students are tested. And of course there's plain old cheating.
Koretz explains a less-obvious problem of inflation-- it varies from school to school and classroom to classroom, which means "we are identifying the wrong teachers, schools and systems as successes and failures." And it's also worth noting that low-income schools have a higher incentive to inflate, which in turn means that the students who arguably most need an "improved" education are instead getting the education most corrupted by the testing process. IOW, they most need a "real" education, and they are most likely to get a battery of test prep instead.
6: Cheating
By the end of NCLB, as many of us noted, there were only two types of schools-- schools that were failing and schools that were cheating. Measurement-driven instruction has created enormous incentive to cheat, and created some thorny ethical dilemmas. After all, which is worse-- cheating on the student score for a single BS Test, or allowing bad educational policy to take that child's school away from her?
Koretz has a whole batch of cheating tales here, and they are depressing, but necessary if we're going to understand what high-stakes testing pushes people toward.
7: Test Prep
There is no doubt that test-based accountability has resulted in a huge commitment of time and effort to test preparation, I can't be more precise, in part because people don't agree on the dividing line between test prep and regular instruction.
In fact, the blurring of that line is one of the sobering parts of this chapter,
Koretz lists three types of bad test prep-- reallocating time between subjects, reallocating time within subjects, and coaching. Schools shift time, both between classes and within classes, to things that count on the test. No band for you, Pat-- you need to spend a period in ELA test remediation. Today, class, we're going to skip over chapter five, because that stuff is not on the test. Coaching is teaching tricks that have no use except for taking the test. For example, process of elimination, which seems harmless enough, but allows a student to get a correct answer when the student would never have been able to come up with the correct answer on her own. Life is not multiple choice; process of elimination is a skill that is only useful for taking the test.
The most alarming part of the chapter addresses the idea that test prep is corrupting the idea of good teaching. Koretz has a disturbing bank of anecdotes about teachers who are told their is not to teach their subject, but to raise test scores in it. And like anyone else who has encountered new teachers in the last decade, he's met young teachers who were taught in college that test scores are their main purpose.
They were telling me that I was missing the boat by seeing test prep as something that competes for time with good instruction. In their experience, raising scores had become the end goal, the mark of a "good" teacher. To an alarming degree, they had been taught that test prep and good instruction are the same thing.
There are so many reasons this is bad, bad news, but here's one that Koretz spots
For the present, it indicates that one of the few checks against inappropriate test prep-- teachers' own understanding of the differences between test prep and good instruction-- has been eroded.
8: Making up Unrealistic Targets
Yikes. This is a depressing chapter. Koretz notes that the "trust most people have in performance standards is essential, because the entire system now revolves around them." And yet that trust is wildly misplaced.
Koretz outlines several methods of setting performance standards, but the important thing to remember is this:
There is another, perhaps even more important, reason why performance standards can't be trusted: there are many different methods one can use, and there is rarely a really persuasive reason to select one over the other.
And yet those different methods can produce vastly different results.
There are other problems.
A primary motivation for setting a Proficient standard is to prod schools to improve, but information about quickly teachers actually can improve student learning doesn't play much, if any, of a role in setting performance standards. When panels set standards, they are not given information about practical rates of improvement, and for the most part they are not asked to consider them.
In other words, they are set up to be the educational equivalent of an agricultural board that declares, "What we need is wheat that we can harvest six hours after it's planted. Do that!"
Also, standards are set mainly on the assumption that all kids are the same, with a goal of radically reducing the variability of achievement. I'm just going to sum up on this part: that's a dumb assumption.
9: Evaluating Teachers
Koretz is pretty brutal about the VAM-sauced test-based teacher evaluation systems. He allows that the old system had a problem in that everyone ended up looking great, but the new system surprised him because it solved problems with solutions so dumb it had never occurred to him that anyone would actually use them (e.g. evaluating teachers based on the scores of students they have never met).
This chapter includes Koretz's account of his visit to the Department of Education in Duncan's early days. You'll want to read that yourself.
How have we screwed up test-based accountability for teachers. Let Koretz count the ways:
* Taking test scores out of context (a "deliberate goal" of reformers, but "one of the main reasons the reforms have failed)
* Trying to use tests to explain, not just describe
* Using "Value-Added Modeling to evaluate teachers (here's as good an explanation as you'll find for why VAM can't possibly work)
* Rating teachers with the wrong test
* Teachers ratings are inconsistent across tests
* Teacher test scores are unstable over time (there are charts and specifics here that drive home how bad this effect is).
10: Will the Common Core fix this?
In a word, no.
Want a longer explanation?
It's not just the Common Core that has been dropped into schools wholesale before we gathered any evidence about impact; this has been true of almost the entire edifice of test-based reform, time and time again. I'll argue later that putting a stop to this disdain for evidence-- this arrogant assumption that we know so much that we don't have to bother evaluating our ideas before imposing them on teachers and students-- is one of the most important changes we have to make.
Koretz also argues that CCSS tests have actually been created in a way that makes them more predictable, and therefor more susceptible to all the Campbell's Lawian Test Preppery he just spent several chapters eviscerating. Koretz dismisses the one size fits allness of the Core, noting that one official once told him that the Core eliminated the distinction between career readiness and college readiness. "Rhetorically, perhaps, but not in actuality" responds Koretz.
It is a new flavor of the same old failed approach, gives one size fits all a "grandiose rhetorical wrapper." Underneath all the noise
the basic failed model of educational improvement remains unchanged: set arbitrary performance targets on standardized tests; apply them uniformly, without regard to circumstances; and reward and punish. Whatever its other virtues and vices, the Common Core hasn't changed this. This approach hasn't worked before, and it won't work with the Common Core.
11: Did Kids Learn More?
We know less about this most fundamental of questions than we probably should.
Why not? Partly because the test data is so subject to inflation that it can't be trusted. But also-- and I know I just ran a similar quote, but this point is important
There is a second reason for the dearth of information, the blame for which lies squarely on the shoulders of many of the reformers. Time after time they declared that they had figured out what would work, and they imposed it on students and teachers on a mass scale without taking the time to evaluate their programs first. It's analogous to a drug company saying that they have figured out, based just on their own beliefs and logic, which drugs will be effective and safe, so they can skip the time-consuming and expensive burden of actually gathering some evidence before selling it to you.
Koretz spends some time crunching some numbers, asking if students learned more, if they learned it because of test-centered accountability, and if what they learned justifies the "huge" costs of these policies (including costs like the corruption of instruction on a broad scale).
The answer is pretty clearly, "No."
12: Nine Principles for Doing Better
Here they are--
Pay attention to other important stuff.
Monitor more than student achievement.
Set reasonable targets.
Stop just kicking the dog harder.
Don't expect school to do it all.
Pay attention to context.
Accept the need for human judgment.
Create counterbalancing incentives.
Monitor, evaluate and revise.
13: Doing Better
Here Koretz works out his nine principles into a more specific action plan. He considers at length some of the usual models-in-other-countries. And he lays out some ideas.
Measure what matters. Okay, no bonus points for originality here, but then one of the subtexts of this book is that apparently you have to point out the obvious to people in education reform because some of them rush straight into doing things that are obviously stupid. Here also he and I disagree on a role for well-used well-made standardized tests. That's okay; I know I'm out in left field on this. But I do like this:
This is the first and one of the most difficult tradeoffs we face: to measure learning well and to give teachers better incentives, we will have to use measures that have serious drawbacks-- in particular, potential inconsistency from classroom to classroom and school to school.
Yes. The more perfectly something is universally standardized, the useful it is to an individual teacher.
We need to measure "soft skills" well. This will be hard, and there will be disagreement. Yup-- that's already happening. And he has a useful insight-- part of the reason we got standardized testing "was the notion that educators can't be trusted to evaluate schooling or other educators." But the soft skills measures will have to "give a substantial role to the judgment of professionals." Not standardized SEL tests.
We need a sensible accountability system. Most interesting detail here-- we need various measures that are NOT too closely aligned with each other. If everything's aligned together, everything canm be wrong together.
Use tests sensibly. One test cannot do everything.
Provide support to teachers. Monitor and make midcourse corrections (you know-- the way Common Core was specifically designed not to do).
Still here? Good for you.
Chapter 14 is a wrap-up and I'll skip that, just as I've skipped over many specifics and explanations. Really, if I had time to be a better reviewer, I would have given you a much more compact look at what Koretz has created here, which is nothing less than a scholarly, thoughtful, accessible explanation of how test-based reform has taken education into the weeds. You should read this book, and you should pass it on to other folks who care about education and want to both understand the problems and envision some solutions. This is a valuable work, and I'll be coming back to it again and again, and you should, too.
Friday, April 13, 2018
How To Discredit a Strike
From the list of must-see news is this item in The Guardian that brings us a secret-ish manual for fighting back against the teacher strikes breaking out across the US. But there is good news here-- I'll get to that in a moment.
The "messaging guide" is only three pages long, but it includes specific ideas about how to fight back against these crazy teachers and their desire to be paid a decent wage and also work in decent facilities.
1. Teacher strikes hurt kids and low-income families.
The guide cites some "research" from Brookings and an article from Oregon Public Broadcasting that actually cites research about poor attendance. But the point here is that parents working hourly-wage jobs can't just take off or suddenly get child care. But the guide says to note that it's "unfortunate" that teachers are "protesting low wages by punishing other low-wage parents and their children" when instead teachers should, I guess, punish those families the old-fashioned way-- by standing by idly while the schools that serve low-income neighborhoods are gutted and torn down in tasteful slow motion.
2. We can all agree that good teachers should get paid more.
This is just a re-purposing of an old reformster point. Only good teachers should be paid well, which is why tenure and seniority and salary schedules should be trashed, because if we only have to pay for quality, and we get to define what quality is, then we can get away with paying less.
Other ideas for messaging include:
Address the role of red tape and bureaucracy.
Administrators and non-teaching staff are proliferating. Keep harping on how the money should go to teachers and students, and not these other leaches.
Tailor your message to the specifics of your state.
For instance, since Kentucky teachers sort of kinda got a commitment to try to meet some of their demands, just say, "Hey, you got what you want. Go back to work." And don't forget that old chestnut, "We already spend Umpty-ump dollars on education!"
Most hilarious advice: If your state has increased education spending lately, harp on that. Also, if pigs have recently flown out of your butt, make bacon for breakfast.
Most realistic advice: If you have been busy cutting taxes (while claiming it wouldn't hurt anything), we have to admit that's a tough sell stacked up against visible evidence that public schools are being financially starved to death. But, um, maybe you could try selling trickle down one more time-- we cut taxes and now people are going to make more money and pay more taxes which will fund... you know what? You're on your own.
This all comes courtesy of the State Policy Network, and if you've never heard of them, well, know that they are pursuing "a vision of an America where personal freedom, innovation, opportunity, and a more peaceful society help all Americans flourish." Need to be further impressed? Well...
Founded in 1992 by Tom Roe at the urging of Ronald Reagan, State Policy Network (SPN) is the only group in the country dedicated solely to improving the practical effectiveness of independent, nonprofit, market-oriented, state-focused think tanks.
They are tied closely to ALEC, and funded by the usual gang, including the Koch brothers, the Roe family, the Coors family, the Walton family, and yes, the DeVos family. The network includes many of our old friends, like the American Enterprise Institute, CATO, and the Center for Education Reform, to name just a few.
So what's the good news?
Here's what I notice in this messaging guide.
There are many messages that the guide does not recommend. I see nothing here about "Keep saying how lazy teachers don't deserve any dam money" or " Why should we lay out tax dollars for a bunch of babysitters" or even "Their work is so simple and unimportant that anybody should be willing to do it for minimum wage." And certainly nothing along the lines of "These greedy teachers are already rich, and they work in buildings that are like the Taj Mahal."
One of the most encouraging factors of all of the statewide strikes is the amount of public support for teachers.
This guide underlines a simple truth-- after all these years of reform and attacks on teachers and assaults on our profession and complaints about our unions, people still like teachers and respect the work they do. It is still bad politics to run right at teachers-- particularly when they are clearly standing up for students and public education.
There's other evidence-- reformsters have tried to lower the bar for the profession, but they haven't tried to replace the profession. For instance, Teach for America still claims to produce teachers-- it doesn't argue that teachers can be replaced by minimally trained content delivery specialists. Likewise, states that have passed laws to allow any warm body into the classroom still call the warm bodies teachers, rather than arguing that teachers can be replaced with something else. Everywhere we look, there have been attempts to twist and alter what it takes to be called a teacher-- but they still keep the name "teacher" because that word still has power, still commands some respect among many people.
So view this messaging guide as more than an attack, but as proof that the enemies of teachers and teacher unions are still afraid to come straight at them. They come at teachers this way because the enemies of the teachers and their union are afraid.
The "messaging guide" is only three pages long, but it includes specific ideas about how to fight back against these crazy teachers and their desire to be paid a decent wage and also work in decent facilities.
1. Teacher strikes hurt kids and low-income families.
The guide cites some "research" from Brookings and an article from Oregon Public Broadcasting that actually cites research about poor attendance. But the point here is that parents working hourly-wage jobs can't just take off or suddenly get child care. But the guide says to note that it's "unfortunate" that teachers are "protesting low wages by punishing other low-wage parents and their children" when instead teachers should, I guess, punish those families the old-fashioned way-- by standing by idly while the schools that serve low-income neighborhoods are gutted and torn down in tasteful slow motion.
2. We can all agree that good teachers should get paid more.
This is just a re-purposing of an old reformster point. Only good teachers should be paid well, which is why tenure and seniority and salary schedules should be trashed, because if we only have to pay for quality, and we get to define what quality is, then we can get away with paying less.
Other ideas for messaging include:
Address the role of red tape and bureaucracy.
Administrators and non-teaching staff are proliferating. Keep harping on how the money should go to teachers and students, and not these other leaches.
Tailor your message to the specifics of your state.
For instance, since Kentucky teachers sort of kinda got a commitment to try to meet some of their demands, just say, "Hey, you got what you want. Go back to work." And don't forget that old chestnut, "We already spend Umpty-ump dollars on education!"
Most hilarious advice: If your state has increased education spending lately, harp on that. Also, if pigs have recently flown out of your butt, make bacon for breakfast.
Most realistic advice: If you have been busy cutting taxes (while claiming it wouldn't hurt anything), we have to admit that's a tough sell stacked up against visible evidence that public schools are being financially starved to death. But, um, maybe you could try selling trickle down one more time-- we cut taxes and now people are going to make more money and pay more taxes which will fund... you know what? You're on your own.
This all comes courtesy of the State Policy Network, and if you've never heard of them, well, know that they are pursuing "a vision of an America where personal freedom, innovation, opportunity, and a more peaceful society help all Americans flourish." Need to be further impressed? Well...
Founded in 1992 by Tom Roe at the urging of Ronald Reagan, State Policy Network (SPN) is the only group in the country dedicated solely to improving the practical effectiveness of independent, nonprofit, market-oriented, state-focused think tanks.
They are tied closely to ALEC, and funded by the usual gang, including the Koch brothers, the Roe family, the Coors family, the Walton family, and yes, the DeVos family. The network includes many of our old friends, like the American Enterprise Institute, CATO, and the Center for Education Reform, to name just a few.
So what's the good news?
Here's what I notice in this messaging guide.
There are many messages that the guide does not recommend. I see nothing here about "Keep saying how lazy teachers don't deserve any dam money" or " Why should we lay out tax dollars for a bunch of babysitters" or even "Their work is so simple and unimportant that anybody should be willing to do it for minimum wage." And certainly nothing along the lines of "These greedy teachers are already rich, and they work in buildings that are like the Taj Mahal."
One of the most encouraging factors of all of the statewide strikes is the amount of public support for teachers.
This guide underlines a simple truth-- after all these years of reform and attacks on teachers and assaults on our profession and complaints about our unions, people still like teachers and respect the work they do. It is still bad politics to run right at teachers-- particularly when they are clearly standing up for students and public education.
There's other evidence-- reformsters have tried to lower the bar for the profession, but they haven't tried to replace the profession. For instance, Teach for America still claims to produce teachers-- it doesn't argue that teachers can be replaced by minimally trained content delivery specialists. Likewise, states that have passed laws to allow any warm body into the classroom still call the warm bodies teachers, rather than arguing that teachers can be replaced with something else. Everywhere we look, there have been attempts to twist and alter what it takes to be called a teacher-- but they still keep the name "teacher" because that word still has power, still commands some respect among many people.
So view this messaging guide as more than an attack, but as proof that the enemies of teachers and teacher unions are still afraid to come straight at them. They come at teachers this way because the enemies of the teachers and their union are afraid.
Thursday, April 12, 2018
Why Are We Still Testing
It's the season of testing again, a season that has no come so many times that lots of folks don't even question it any more? But it's a question that needs to be asked about the Big Standardized Test-- why, exactly, are we still doing this? We've had a variety of answers over the years-- let's see how they hold up.
The Bathroom Scales
Take the test! It's just like weighing yourself on the bathroom scales! Early on many reformsters suggested that weighing the pig would make it gain weight, but that's stupid. So we weigh the school-- then what? That "then what" is a huge part of the problem, but the other part of the problem is trying to use the read-out on the bathroom scales to determine how tall, how healthy, and how well-adjusted the weigh-ee is. This was always a really dumb analogy.
Compare and Contrast
The BS Tests would let us compare a rural school in Idaho with an urban school in Michigan. It would let us compare every third grader to every other third grader. This is no longer possible-- different states have different sets of standards, and there are a wide variety of BS Tests being given, so we're back to comparing apples to watermelons.
Evaluating Educators
By soaking BS Test results in magical VAM sauce, we were supposed to get data that would let us tell the difference between good teachers and bad teachers. To begin with, that assumes that your definition of "good teacher" is "one who gts students to score well on a single math and reading test," which is not very helpful in sorting out, say, science or music teachers. And that's before we get to the insanity of using BS Test scores to "evaluate" teachers who have never even met the students whose scores are being used. Plus, VAM has problems. So many problems that there's a long list of folks who don't think it should be used for this purpose.
Informing Staffing Decisions
It was a fond dream that test-based evaluation would lead to evaluation based hiring, firing and compensation policies. This balloon never lifted off the ground, perhaps because the teacher pipeline has dried up so badly that schools are in no hurry to inflict a staffing shortage on themselves, and because some states are already paying teachers so little that they don't need a ginned-up excuse to pay teachers even less. Also, as noted, test-based evaluation of teachers doesn't work. If you're a principal, who are you going to believe-- test data, or your own eyes, ears, and brain?
Scaling Excellence
The failed evaluation piece means that another dream is also dead. That's the dream in which BS Test results are used to identify super-duper teachers, who are then tasked with spreading their super-duper teacherly wisdom to other less super-duper teachers. In fact, states were supposed to have a plan for moving good teachers to low-achieving schools, but that never happened because it turns out rendering educators is illegal in the US.
Helping Schools in Trouble
The BS Tests were going to help us identify schools that were "troubled" or "failing" or "sucky." One might argue that we can already find these schools without any trouble, but I suppose a case can be made that numbers you can wave at politicians might give some heft to that identification. The problem here is what hasn't happened. "Look, this school is clearly having trouble, so let's get them additional resources and help," said no legislature ever. Instead, the low-achievement label is used to justify targeting that school for destruction. Low scores can be used to justify the launch of charter businesses, or even the gentrification of entire sectors of a community. Low-scoring schools are not targeted for assistance; they are targeted for dismantling.
Address Inequity
We would find where non-wealthy non-white student populations were being ill-served. Anyone who can't figure that out without the BS Test is a dope. And as with the last point, the problem has been that the data hasn't so much been used to find schools that need help as it has been used to find schools that are vulnerable and ready to be turned into somebody's business opportunity. Instead of focusing our will to address educational inequity, test-based accountability has highlighted our lack of will (and wasted the good intentions of some folks).
Informing Instruction
Teachers were going to get their data spreadsheets and figure out, with laser-like precision, how they needed to change their instruction. But right off the bat it became clear that data about students in your class would only arrive long after the students had departed for their next classroom. Then the security issue reared its stupid head-- I can see student scores, but I am forbidden to see the test itself. (For that matter, students who are so inclined are unable to see their specific results to ask "What exactly did I get wrong here?") This means I can tell that Pat only got an okayish score, based on some questions that might have asked about something about reading that Pat apparently answered incorrectly. How can that inform my instruction? It can't. It doesn't. The BS Tests "inform instruction" mostly by encouraging teachers to spend more time on test prep. That's not a good thing.
Letting Parents Know How Their Children Are Doing
Under this theory, parents have no idea how their children are doing in school until the BS Test results appear. Assuming for the moment that the parents are that disconnected, the information provided is minimal, scoring a few categories on a 1-3 or 1-4 scale. A BS Test provides very non-granular data, less nuanced than a report card-- and based on just one test. There is nothing for parents to learn here.
Unmask the Lies
Of course, guys like Arne Duncan were sure that once the BS Test revealed the Truth-- that US schools are super-stinky-- folks like the fabled suburban white moms would have to face the Truth that their children were actually doing terribly. And then we tried talking about the honesty gap. Basically, a whole bunch of folks started with the premise that schools and the teachers who work in them largely suck and the BS Test would be a tool for revealing the Awful Truth (for some folks, you can also insert a screed about a vast union scam and conspiracy here). Somehow, that never happened. It's almost as if the vast majority of teachers don't actually suck.
Redefine What It Means To Be Educated
I don't know that this was a very widespread goal, but it was certainly near and dear to the hearts of guys like David Coleman, who had a good idea of what he did and didn't approve of in education, and dreamed of using standards hard-wired to high-stakes tests to force people to see things his way. Very few hearts and minds have been won at this point.
As a Backdoor Method of Imposing State and Federal Amateur Top-Down Control of Curriculum in Local Schools
Okay, this goal has kind of worked out. Many school districts have redesigned their curriculum to "align" with the BS Test (not the standards, but the "anchor" standards, or standards that will be tested). Heck, some school districts have restructured the district itself to accommodate the test (what's the best to handle the fact that 8th graders tend to do poorly on these tests? either fold them into your high school, or lower your "middle school" years so that they include elementary tests.) Test-centered curriculum affects students scheduling, with students who come up short on the practice tests may find they have to schedule around double math or double reading (no art, music, or history for you, kid). Within math and English classes, teachers are directed to use "data" from practice tests to "inform" their instruction, which fo course necessitates dropping content that is not On The Test.
So yes-- this particular goal of the BS Test is being achieved. It's just a very bad thing.
Bonus Paranoid Goal
You may believe that one of the goals of the testing regime is to destabilize, dismantle, and destroy public schools. I think some reformsters really thought this was going to help, and you can spot them because they are acknowledging that they failed. I think other reformsters weren't necessarily scheming, but when this came down the pike, they smelled an opportunity. And some reformsters absolutely want to see public education dismantled and the pieces sold off. BS Tests have been a useful tool in selling the narrative that public schools are "failing," that students are "trapped" in terrible public schools. In fact, BS Testing has been a kind of two-fer, because if you want to claim that public schools are failing, you can argue that they are now tied up in testing and following stupid government rules. Hey-- it's even a three-fer, because reformsters who want to move on to the Next Big Thing can say, "Yes, this test-centered reform is awful-- what we really need is Personalized Competency Based Learning Education Stuff!"
But How Else Will We Know How Schools Are Doing?
It's a fake question, because it assumes that BS Tests are now telling us how schools are doing, and they aren't. Nobody's definition of a Good School is "one in which students score well on a once-a-year math and reading test." There are so many things that matter in deciding if a school is a good one or not, and the vast majority (perhaps all) of them are not measured by the BS Tests.
So, Back To The Main Question
The Big Standardized Test was launched into schools with big goals, big plans, big dreams-- and none of them have come true. We've been doing this for oh so many years now, and if we were going to reap benefits, we would be awash in those benefits right now. We are not. Not by the measure of the supposedly "gold standard" NAEP test, not by college success, not by an economic and cultural renaissance caused by an influx of super5-educated young people.
Some of the goals associated with the test were not worthwhile goals to begin with. Some of the results have proven to be hugely undesirable. I don't believe that anyone associated with test-centered accountability said, "Oh, and let's try to make young children really stressed out to the point that they are crying and pulling hair-- that would be cool!" And yet, here we are.
Here we are spending a buttload of tax money on a product that has not delivered on any of its promises-- a buttload of money that could be spent to make schools better. Here we are shortening the school year so that even less instruction can take place.
Here we are continuing with the testing regimen even though, after two decades, we don't have a shred of evidence that it is doing any good, and a ton of evidence that it is doing harm.
So why are we still doing this?
Inertia? Affection for the status quo (which test-centered schooling now is)? Corporate lobbying to keep the tax dollars flowing? Policy leaders unwilling to confess they screwed up? Because legislators understand education as well as they understand the internet?
I don't know the answer. But I do know what we should do next.
Stop.
Just stop.
Cancel the BS Tests. Throw them out. Have an honest conversation about which of the above goals are worth pursuing and how best to pursue them. That will take time; it won't be easy. Maybe there will be a place for the right tests, used correctly, in the future. Maybe. But what we have now continues to do serious damage to US public education. It's costing us so much, both in terms of money and human toll and opportunity costs, and it is giving us nothing in return
Stop. Stop the testing. Stop it completely. Stop it now.
Just stop.
The Bathroom Scales
Take the test! It's just like weighing yourself on the bathroom scales! Early on many reformsters suggested that weighing the pig would make it gain weight, but that's stupid. So we weigh the school-- then what? That "then what" is a huge part of the problem, but the other part of the problem is trying to use the read-out on the bathroom scales to determine how tall, how healthy, and how well-adjusted the weigh-ee is. This was always a really dumb analogy.
Compare and Contrast
The BS Tests would let us compare a rural school in Idaho with an urban school in Michigan. It would let us compare every third grader to every other third grader. This is no longer possible-- different states have different sets of standards, and there are a wide variety of BS Tests being given, so we're back to comparing apples to watermelons.
Evaluating Educators
Informing Staffing Decisions
It was a fond dream that test-based evaluation would lead to evaluation based hiring, firing and compensation policies. This balloon never lifted off the ground, perhaps because the teacher pipeline has dried up so badly that schools are in no hurry to inflict a staffing shortage on themselves, and because some states are already paying teachers so little that they don't need a ginned-up excuse to pay teachers even less. Also, as noted, test-based evaluation of teachers doesn't work. If you're a principal, who are you going to believe-- test data, or your own eyes, ears, and brain?
Scaling Excellence
The failed evaluation piece means that another dream is also dead. That's the dream in which BS Test results are used to identify super-duper teachers, who are then tasked with spreading their super-duper teacherly wisdom to other less super-duper teachers. In fact, states were supposed to have a plan for moving good teachers to low-achieving schools, but that never happened because it turns out rendering educators is illegal in the US.
Helping Schools in Trouble
The BS Tests were going to help us identify schools that were "troubled" or "failing" or "sucky." One might argue that we can already find these schools without any trouble, but I suppose a case can be made that numbers you can wave at politicians might give some heft to that identification. The problem here is what hasn't happened. "Look, this school is clearly having trouble, so let's get them additional resources and help," said no legislature ever. Instead, the low-achievement label is used to justify targeting that school for destruction. Low scores can be used to justify the launch of charter businesses, or even the gentrification of entire sectors of a community. Low-scoring schools are not targeted for assistance; they are targeted for dismantling.
Address Inequity
We would find where non-wealthy non-white student populations were being ill-served. Anyone who can't figure that out without the BS Test is a dope. And as with the last point, the problem has been that the data hasn't so much been used to find schools that need help as it has been used to find schools that are vulnerable and ready to be turned into somebody's business opportunity. Instead of focusing our will to address educational inequity, test-based accountability has highlighted our lack of will (and wasted the good intentions of some folks).
Informing Instruction
Teachers were going to get their data spreadsheets and figure out, with laser-like precision, how they needed to change their instruction. But right off the bat it became clear that data about students in your class would only arrive long after the students had departed for their next classroom. Then the security issue reared its stupid head-- I can see student scores, but I am forbidden to see the test itself. (For that matter, students who are so inclined are unable to see their specific results to ask "What exactly did I get wrong here?") This means I can tell that Pat only got an okayish score, based on some questions that might have asked about something about reading that Pat apparently answered incorrectly. How can that inform my instruction? It can't. It doesn't. The BS Tests "inform instruction" mostly by encouraging teachers to spend more time on test prep. That's not a good thing.
Letting Parents Know How Their Children Are Doing
Under this theory, parents have no idea how their children are doing in school until the BS Test results appear. Assuming for the moment that the parents are that disconnected, the information provided is minimal, scoring a few categories on a 1-3 or 1-4 scale. A BS Test provides very non-granular data, less nuanced than a report card-- and based on just one test. There is nothing for parents to learn here.
Unmask the Lies
Of course, guys like Arne Duncan were sure that once the BS Test revealed the Truth-- that US schools are super-stinky-- folks like the fabled suburban white moms would have to face the Truth that their children were actually doing terribly. And then we tried talking about the honesty gap. Basically, a whole bunch of folks started with the premise that schools and the teachers who work in them largely suck and the BS Test would be a tool for revealing the Awful Truth (for some folks, you can also insert a screed about a vast union scam and conspiracy here). Somehow, that never happened. It's almost as if the vast majority of teachers don't actually suck.
Redefine What It Means To Be Educated
I don't know that this was a very widespread goal, but it was certainly near and dear to the hearts of guys like David Coleman, who had a good idea of what he did and didn't approve of in education, and dreamed of using standards hard-wired to high-stakes tests to force people to see things his way. Very few hearts and minds have been won at this point.
As a Backdoor Method of Imposing State and Federal Amateur Top-Down Control of Curriculum in Local Schools
Okay, this goal has kind of worked out. Many school districts have redesigned their curriculum to "align" with the BS Test (not the standards, but the "anchor" standards, or standards that will be tested). Heck, some school districts have restructured the district itself to accommodate the test (what's the best to handle the fact that 8th graders tend to do poorly on these tests? either fold them into your high school, or lower your "middle school" years so that they include elementary tests.) Test-centered curriculum affects students scheduling, with students who come up short on the practice tests may find they have to schedule around double math or double reading (no art, music, or history for you, kid). Within math and English classes, teachers are directed to use "data" from practice tests to "inform" their instruction, which fo course necessitates dropping content that is not On The Test.
So yes-- this particular goal of the BS Test is being achieved. It's just a very bad thing.
Bonus Paranoid Goal
You may believe that one of the goals of the testing regime is to destabilize, dismantle, and destroy public schools. I think some reformsters really thought this was going to help, and you can spot them because they are acknowledging that they failed. I think other reformsters weren't necessarily scheming, but when this came down the pike, they smelled an opportunity. And some reformsters absolutely want to see public education dismantled and the pieces sold off. BS Tests have been a useful tool in selling the narrative that public schools are "failing," that students are "trapped" in terrible public schools. In fact, BS Testing has been a kind of two-fer, because if you want to claim that public schools are failing, you can argue that they are now tied up in testing and following stupid government rules. Hey-- it's even a three-fer, because reformsters who want to move on to the Next Big Thing can say, "Yes, this test-centered reform is awful-- what we really need is Personalized Competency Based Learning Education Stuff!"
But How Else Will We Know How Schools Are Doing?
It's a fake question, because it assumes that BS Tests are now telling us how schools are doing, and they aren't. Nobody's definition of a Good School is "one in which students score well on a once-a-year math and reading test." There are so many things that matter in deciding if a school is a good one or not, and the vast majority (perhaps all) of them are not measured by the BS Tests.
So, Back To The Main Question
The Big Standardized Test was launched into schools with big goals, big plans, big dreams-- and none of them have come true. We've been doing this for oh so many years now, and if we were going to reap benefits, we would be awash in those benefits right now. We are not. Not by the measure of the supposedly "gold standard" NAEP test, not by college success, not by an economic and cultural renaissance caused by an influx of super5-educated young people.
Some of the goals associated with the test were not worthwhile goals to begin with. Some of the results have proven to be hugely undesirable. I don't believe that anyone associated with test-centered accountability said, "Oh, and let's try to make young children really stressed out to the point that they are crying and pulling hair-- that would be cool!" And yet, here we are.
Here we are spending a buttload of tax money on a product that has not delivered on any of its promises-- a buttload of money that could be spent to make schools better. Here we are shortening the school year so that even less instruction can take place.
Here we are continuing with the testing regimen even though, after two decades, we don't have a shred of evidence that it is doing any good, and a ton of evidence that it is doing harm.
So why are we still doing this?
Inertia? Affection for the status quo (which test-centered schooling now is)? Corporate lobbying to keep the tax dollars flowing? Policy leaders unwilling to confess they screwed up? Because legislators understand education as well as they understand the internet?
I don't know the answer. But I do know what we should do next.
Stop.
Just stop.
Cancel the BS Tests. Throw them out. Have an honest conversation about which of the above goals are worth pursuing and how best to pursue them. That will take time; it won't be easy. Maybe there will be a place for the right tests, used correctly, in the future. Maybe. But what we have now continues to do serious damage to US public education. It's costing us so much, both in terms of money and human toll and opportunity costs, and it is giving us nothing in return
Stop. Stop the testing. Stop it completely. Stop it now.
Just stop.
Tuesday, April 10, 2018
It's NAEP Day. Here's What To Remember As You Peruse All the Various PIeces Offering Reactions and Analysis of the So-Called Nation's Report Card. Really.
Even if you disagree with the valu the NAEP, it is the yardstick by which many folks, including many reformsters, choose to use in measuring educational achievement.
The 2017 tests were taken by students who have, for the most part, received an entire education shaped by ed reform.
The scores were not good.
Ed reform has failed.
Everything else is just details and noise.
The 2017 tests were taken by students who have, for the most part, received an entire education shaped by ed reform.
The scores were not good.
Ed reform has failed.
Everything else is just details and noise.
Monday, April 9, 2018
Fight for Scraps (The Real Causes of the Strikes)
In today's Washington Post, economist Robert Samuelson, lays out his vision of why states like West Virginia and Oklahoma and Kentucky (and Arizona etc etc) are squeezing their teachers economically.
The deeper cause is that teachers — and schools — are competing with the elderly for scarce funds.
As the Boomers age, grandparents and grandchildren will vie for scarce funds.
Spending on the elderly is squeezing K-12 schools, police, parks, libraries, roads and other infrastructure (water projects, sewers), mainly through two programs: (a) Medicaid, a joint state-federal program of health insurance for the poor, which pays about half of nursing-home and long-term-care costs for the aged and disabled (on average, states pay about 37% of Medicaid’s costs); and (b) contributions to underfunded pensions for state and local workers.
If nothing else, this may help clarify for some of us why lately there is so much right-tilted thinky tank interest in "fixing" pensions.
But for me it raises another question. If the underlying problem behind all these issues is competition for funding, couldn't we also say that the underlying issue is the lack of funding?
Could we not say that the underlying problem is that too few people are collecting too much of the wealth generated by the economy and paying too little tax on it?
I'm not an economist, but I continue to be mystified at how the country can get richer and richer, and yet states and communities seem to get poorer and poorer. I mean, we've seen some states run their own experiments. Minnesota increased taxes. California increased taxes. Kansas slashed taxes. It has turned out well for Minnesota and California; it has turned out poorly for Kansas. Could it be that taxing done well is helpful?
So maybe it's not necessary to have a cage match between Grampaw and Junior. Maybe it's possible to fund stuff for the aging boomers and also fund stuff for children (like, you know, education) just by collecting more funds.
A sensible society would direct its governmental programs and investments toward preparing for the future. Instead, our emphasis is backward-looking, with more and more support going to the aged. On the other hand, a compassionate and caring society — a civilized society — doesn’t discard its older members just because their self-reliance and social utility have declined.
Teachers and others will continue to battle the demographics. Until we muster the courage to be candid about the choices, we will be stuck in a place we don’t want to be.
Samuelson isn't wrong. But that candid conversation could include one other choice-- we could tax folks a bit more so that we could afford to be both sensible and civilized. We could decide that providing health care for the aged and education for the young are both more important than making sure the 1% keep their vast wealth undisturbed. I'm not a hard-core socialist, nor do I think the government needs to strip every last dollar from every rich person's hand.
But by Samuelson's own framing, doesn't the fact that some states have to choose between being sensible and being civilized-- isn't that a sign that we may have veered a bit too far in the direction of using government primarily to service the desires corporations and the rich folks who run them? Because I don't think there's anything sensible or civilized about a country that makes Grampaw and Junior fight over table scraps while the rich are grabbing more food than they know what to do with.
The deeper cause is that teachers — and schools — are competing with the elderly for scarce funds.
As the Boomers age, grandparents and grandchildren will vie for scarce funds.
If nothing else, this may help clarify for some of us why lately there is so much right-tilted thinky tank interest in "fixing" pensions.
But for me it raises another question. If the underlying problem behind all these issues is competition for funding, couldn't we also say that the underlying issue is the lack of funding?
Could we not say that the underlying problem is that too few people are collecting too much of the wealth generated by the economy and paying too little tax on it?
I'm not an economist, but I continue to be mystified at how the country can get richer and richer, and yet states and communities seem to get poorer and poorer. I mean, we've seen some states run their own experiments. Minnesota increased taxes. California increased taxes. Kansas slashed taxes. It has turned out well for Minnesota and California; it has turned out poorly for Kansas. Could it be that taxing done well is helpful?
So maybe it's not necessary to have a cage match between Grampaw and Junior. Maybe it's possible to fund stuff for the aging boomers and also fund stuff for children (like, you know, education) just by collecting more funds.
A sensible society would direct its governmental programs and investments toward preparing for the future. Instead, our emphasis is backward-looking, with more and more support going to the aged. On the other hand, a compassionate and caring society — a civilized society — doesn’t discard its older members just because their self-reliance and social utility have declined.
Teachers and others will continue to battle the demographics. Until we muster the courage to be candid about the choices, we will be stuck in a place we don’t want to be.
Samuelson isn't wrong. But that candid conversation could include one other choice-- we could tax folks a bit more so that we could afford to be both sensible and civilized. We could decide that providing health care for the aged and education for the young are both more important than making sure the 1% keep their vast wealth undisturbed. I'm not a hard-core socialist, nor do I think the government needs to strip every last dollar from every rich person's hand.
But by Samuelson's own framing, doesn't the fact that some states have to choose between being sensible and being civilized-- isn't that a sign that we may have veered a bit too far in the direction of using government primarily to service the desires corporations and the rich folks who run them? Because I don't think there's anything sensible or civilized about a country that makes Grampaw and Junior fight over table scraps while the rich are grabbing more food than they know what to do with.
Subscribe to:
Posts (Atom)