Tuesday, May 27, 2014

Test (In)Security

One of the features of High Stakes Testing is a level of security usually associated with large bales of money, important state secrets, and the recipe for Bush's Baked Beans. On facebook, in the category of "Ethical Dilemmas Nobody Ever Thought She'd Face," teachers are arguing about whether it's okay to photograph or copy any of the PARCC or SBA exams. I have a thought-- but first, let me tell you a story.

Forty years ago I took a biology course called BSCS. That stands for "Biological Sciences Curriculum Studies," though we generally interpreted it as "BS College Style."  To this day, the tests from that course are the toughest tests I've ever taken. I still remember sections of those tests (a village where they make pottery compared to a single-celled organism, an experiment on a kangaroo rat) because they were so challenging. And these very tough, very challenging tests came with zero security.

In fact, we took the tests as take-home tests. People called their college siblings, friends who were taking college biology. We had test parties, and everybody came and hammered out the answers. And then, on the due date, we handed the test in. And then we took the same test (with questions rearranged) in class.

The tests were so perfectly built around the ability to think and reason scientifically, to interpret data, to build useful analogies, that no security was needed (and no, everybody did not get an A every time-- not even close). So here's what I think about high stakes test security:

If your test needs super-secret high-and-tight lock down security, it is a crappy test.

This applies to classroom teachers as well. The better I get at assessment, the less I need test security. My students pass on year after year, like tales of the Loch Ness Bigfoot, tales of various assignments that may crop up in my class (though the assignment list does vary from year to year). It doesn't matter. If I've done my job well, there is no place on God's green earth to go look up The Answer.

The PARCC and SBA make a lot of noise about how tight security must be in order to protect the validity of the test. Baloney. Aren't these tests supposed to be impervious to test prep, completely inaccessible to the world of memorization and rote? What these folks want to protect is their delicate ears and eyes, protect them from the onslaught of outrage and ridicule that would follow if the general public and professional educators got a good look at the test.

If your model of a test is a surprise, a moment in your course when you jump out from behind a bush a try to play "gotcha" with your students, you are doing it wrong. Tests should be an opportunity to apply knowledge, to ramp the whole process up one final step that really seats the knowledge or skill in the students' brains.

But the underlying assumption in the high-stakes test-driven movement is that there are skadzillions of bad teachers and the students they have failed to teach out there, skadZILLIONS of them, and somehow they are sneaking by and we are going to have to outsmart them so that we can catch them in their consummate suckiness. The high-stakes test-driven movement is not about education-- it's about finding proof, somehow, that public schools are failing. It is nothing but a game of "gotcha."

The super-duper secret security that surrounds the assessments is further proof of their suckiness. And, for what it's worth, the ethical dilemma of copying one of the exams is about the same as the ethical dilemma of photographing a policeman who is beating a suspect.

Conservatives Don't Really Like School Choice

Conservatives often claim they are big fans of school choice. I think they're wrong. I don't mean that I want to disagree with them using fluffy progressive liberal arguments. I mean that in the world of conservative values and goals, school choice really doesn't fit. Let me explain.

Resources and Inefficiency

One of the assumptions of every choice system is that a choice system can operate for the same amount of money-- or less-- than the current system. This is clearly false.

Which will be more inexpensive and efficient-- educating 100 students in one school , or educating them in ten separate ten-student schools, each with its own group of administrative employees and each with its own physical plant and infrastructure. "We're in serious financial trouble, so let's take our set of elementary schools and break them into even more elementary schools," said no school board ever.

There are some functions that government can perform more efficiently. Nobody suggests that we open the door to any contractor who wants to set up a competing system of interstate highways. Nor do we open up each new war to bids from any private army that wants to go in there. Okay, actually that one does happen a little, and you'll notice that when it does, things get even more expensive really quickly. And when government does allow a spirit of competition, it doesn't work out all that well. We are still trying to fix the massive disconnect between competing intelligence agencies that made it easy to pull off the 9/11 attacks.

I agree that given infinite resources, a multiple service provider system would look a lot different. But that's not what we've got and it's not what we're ever going to have. School choice requires multiple school systems to live as cheaply as one, and they can't. Yes, there are charters who claim they can do it. So far, they are all liars; any lower operating costs they purport to achieve are the result of simply tossing high-cost students out of the system, and if we're willing to throw away the expensive children, we can make public schools run way cheaper tomorrow.

No, a school choice system is no financial winner. We end up with waste and inefficiency and duplication of services, and we end up with school systems that either don't have enough resources, or we simply soak the taxpayers for more money.

Big Government

Because there are not enough resources to go around, we will need some Wise and Powerful Wizard to divvy them all up. That wizard is going to be the state or federal government. For better or worse, under current market conditions starting a new competing school system to compete with the public system will be like starting a new software company to compete with Microsoft Windows. The cost of admission is way too high unless Big Government gets involved.

The only way to extend the reach of choice schools will be to extend the reach of big government. And since the choice schools will be accepting government money, they will be accepting government oversight. Yes, I know they've battled it back for now, but they will lose that war. The government will declare, as it has with public schools, that it has a responsibility to see that it's money was spent appropriately. Some choice school will get caught doing something spectacularly egregiously stupid, and big gummint will have its opening.

You know what a good example of small, local government is? Locally elected school boards. Yes, many are less than perfect. But at what point did conservatives join the chorus of, "We need to just tell the electorate what to do. It's for their own good."

Competition Does Not Foster Quality Products

I've written about this before, comparing charter schools to cable channels. The big money is in the big markets, so the big players compete for the muddled middle. Education has two particular problems-- there's not much product differentiation, and a big chunk of your market is people who don't really want your product.

The lack of product differentiation (particularly if all schools are using the same CCSS to teach to the same Big Tests) means that the game will belong to the person with the best marketing. Trot out your own examples here (I like Betamax vs. VHS) of superior products that did NOT win the marketplace because they were out-marketed by somebody else.

In a choice system, schools will compete, but not by being the highest quality educators. They'll offer programs that appeal to students who don't find school appealing ("Welcome to No Homework High!!"), and they will offer really cool and glitzy marketing. You may say, "Fine. Let the jerks send their kids to crappy schools and that will just leave my kids at Really Quality High with the other cool kids."

Except. First of all, Really Quality High has to accept you. Every admission's decision will be a marketing decision. If your child is too expensive, we don't want him. If he is going to screw with our scores, we're sending him back to you. Here's your competition-- you will compete with other parents to pull strings, make it rain, and otherwise score your kid a seat at Exclusive High (pro tip: you won't compete by making your kid suddenly smarter or a better student, because you can't do much about those things, and I bet you won't say, "Oh well, you're just not as smart as the Smith kid, so we'll settle for Average Shmoe High.")

And second of all, Really Quality High has to exist. In the early days of cable, there were some really classy channels. I liked Bravo for broadway shows and Arts&Entertainment for its highbrow culture offerings. But there wasn't enough of me to make those approaches profitable, so now Bravo and A&E broadcast the same basic sort of dreck as every other channel.

Competition Does Not Foster Competition

One of my favorite history books is The Robber Barons, a history of the great money-grubbers of the 19th century written by a 1930s-era socialist. Matthew Josephson really wants to hate these guys, but at the same time, he clearly admires them because they are economic collectivists. Rockefeller, Carnegie, et al didn't really have a beef with centralized control of an entire industry, as long as they were the people in charge.

Unbridled competition leads to centralized control. Let, say, the phone company just suck up every other phone company, and you get the telephone monopoly of the 1970s, run by a corporation just as impersonal, uncaring, inefficient, unresponsive and insulated from competition as any sector ever run by Big Gummint. What does it take to keep such monopolistic centralization from happening? Why, hello there Big Gummint!

You think this won't happen in choice schools? Of course it will-- it already is. Pearson is already assembling a vertically integrated powerhouse of Rockefellerian proportions (and do I need to remind you that they aren't even American, that as upset as we were when the Chinese were buying up America bit by bit, Pearson has already done much the same with American education), and in may states, the only charter players are the big players. And like every power centralizer before them, they did not conquer their world simply by being so much better than everyone else. They use money and influence and, when necessary, the tool of Big Government to get their way.

This is not meritocracy in action. This is corporations and big government teaming up to display exactly why conservatives who rail against Big Government have a point.

Caveats and Etc

Are there pockets of charter schools who have avoided all these pitfalls? Absolutely. But look at today's corporate-dominated landscape and tell me if you really think there's room for a small, creative edupreneur.

Do I have ideas for alternatives? You know I do, but this is already running long. But conservatives-- you need to stop promoting school choice, because you don't really want it. You just haven't figured that out yet.

Monday, May 26, 2014

I'm Not Blogging Today

I realize that title launches me into some sort of post-modern metablogging fogbank, but hey-- it's the 21st century.

Shortly I'll head out the door to march with a 158-year-old community band in a small town Memorial Day parade, followed by a program in the city park. Because this is all within walking distance, my wife and I will stroll home afterwards, stopping to visit her family.

Along the way I will see and talk to friends, neighbors, family, students, parents of students and people I know in the community (some will fill multiple roles on that list). We will pause to honor those soldiers who have passed (and who, regardless of the arguable historic truth of events, did what they thought was right), and we will honor veterans like my brother-in-law (because you shouldn't have to wait to be appreciated until after you're dead). There's talk of steak and a grill. And then I'm going to psyche myself up for the last two weeks of school.

It's good, I think, to mindfully step out of one corner of one's life and into another, to remind yourself what you want your life to be like, to be about, and to plug solidly, fully, into each community to which you belong. So today I'm stepping away from the computer and into this small town post card that I live in. You enjoy your day, too. I will see you tomorrow.

Sunday, May 25, 2014

CAP Serves Some CCSS Baloney

The Center for American Progress came down hard for the Common Core last week, providing yet another field test for the 100% baloney sandwich that is the Core's urban poor talking point menu.

In "The Common Core Is An Opportunity for Educational Equity,", CAP asserts, "The Common Core State Standards hold promise for low-income students, students of color, English language learners, and students with disabilities, who traditionally perform significantly worse than their peers." And you know that this is a serious position paper because it has footnotes and stuff. What it doesn't have is sense.

Quality Control

The standards will act as a "quality-control check," and let's just stop right there, because do you know, CAPsters, how a quality control check works? Because this seems to be a point on which many CCSS supporters are really fuzzy.

Quality control does not mean that every piece that rolls down the assembly line is now suddenly up to standard. What quality control means is that we check every piece on the line, and when we find pieces that don't meet the standard we throw them away. Quality control does not mean every toaster will be perfect-- it means that every toaster that makes it out of the factory will be perfect.

Using Common Core standards as quality control can only mean one thing-- we will find the students who don't meet the standards and we will throw them away. This is really, really wrong and completely counter to the point of American public education and I can't believe I even have to type that out, but apparently I do.

Out of the Stone Age

Students will explore concepts deeply, work together to solve complex problems, and engage in project-based learning—instead of focusing on worksheets and rote memorization. 

Yes, because no teacher in the history of teacherdom ever knew how to teach concepts or cooperative learning or anything except worksheets before CCSS.


Highlighting Educational Gaps

In this section, CAP notes that low-income students and students of color are less likely to have access to higher-level courses, are more likely to have inexperienced or out-of-area teachers, and along with ELL and students with disabilities are less likely to graduate on time. They have footnotes, and I have no reason to doubt that these are all true facts.

Students of Color and Low Income Students Have Lower College Outcomes

Fewer of these students attend college and a high percentage of them need remedial courses. Again, I believe that by and large this is all true.

And Now That We've Wound Up, The Pitch!

So having established the need, I expect we're now going to make a case for how the implementation of CCSS will help address these issues and-- wait! What? Ummm... no, this is the whole conclusion, verbatim:

The Common Core will improve education quality for all students—particularly traditionally underserved students. Raising standards and preparing all students for college and careers will help reduce the disparities identified for low-income students, students of color, ELLs, and students with disabilities.

But-but-but--HOW!! Fairy dust! Magic beans! I mean, hell, I can type "Eating a baloney sandwich every day will make me grow tall, handsome and wise," but that doesn't make it so! Are you not even going to TRY to explain how Common Core will help? Not even try a teensy weensy bit??

Because-- and I don't think you need me to tell you this, but I want you to know that I know-- those are serious issues that you've laid out. Inequality of opportunity, of education, of employment, or health care-- this is a bit of a national shame. The fact that schools intended for the urban poor are underserved, underresourced, underfunded, understaffed-- I mean, all those things you listed as gapos and problems are things that we really ought to be trying to fix.

But here we are in the hospital ER looking at a patient who has been hit by a truck, who is broken and bleeding, and you want to offer him a magical baloney sandwich??!! Come on, CAP. You can do better than this.

Saturday, May 24, 2014

Will "AP" Stand for "Assessment Prep"?

In Which I Have To Take Back Some of the Nice Things I Said About AP Courses

When I last wrote about the AP courses, I praised the looseness of the course design. I should have known better. Alert readers pointed me at the news from AP-land; apparently the looseness is now seen as a design flaw to be fixed.

Some courses, like the Literature and Composition, still sum up the basics in a half-dozen pages. But courses are being redesigned. AP US History is due to roll out new and shiny next fall, and its course summary is now close to 100 pages. Why the added detail?

The redesign of the AP U.S. History course and exam accomplishes two major goals: it maintains AP U.S. History's strong alignment with the knowledge and skills taught in introductory courses at the college level, and also offers teachers the flexibility to focus on specific historical topics, events, and issues in depth.

Yes, by providing a more specific and detailed course outline, the College Board folks will be giving teachers more flexibility, much like a straightjacket provides more freedom and ignorance is strength. In fact, "the lack of specificity put pressure on many teachers: uncertain about what the AP Exam would assess, they attempted to cover every detail of American history."

The AP folks have been working to erase some of that uncertainty for a while now. If you haven't looked under the AP hood in a while, you may not know that for the past five years or so, the College Board has required that all wannabe AP teachers must submit their syllabus for an audit annually. If that seems like a great deal of work, the College Board offers sample annotated syllabi-- in other words, you can now get AP courses in a can. In the case of AP History, it's a nineteen page can.

But if we look under the new extended course framework hood, what do we find? Some conservatives, like the folks at the Heartland Institute, a righty thinky tanky, think we find a newly biased version of history. The framework breaks the course into four areas. Let's look at each:

I. Historical Thinking Skills

These are just what the title implies-- various skills useful in organizing and interpreting historical information, like being able to determine plausible cause and effect linkage. I would be happy to teach this stuff. No problems here.

II. Thematic Learning Objectives

Now it gets dicey. Whether we're talking about the history of the rise and fall of the Ottoman Empire or the history of Bob and Ethel's tempestuous dating relationship, we're going to make some judgments about what themes/factors drive the narrative. My college history professor said 19th century European history was all about war, nationalism, class, and...um, something else. (What do you want from me? It was a long time ago!) And that in itself was a judgment.

I was going to get specific here, but there are seven large thematic objectives and each is broken down into several more for a grand total of  fifty specific learning objectives, for example:

Analyze how changing religious ideals, Enlightenment beliefs, and republican thought shaped the politics, culture, and society of the colonial era through the early Republic

Fifty of those, so basically one a week. Holy smokes.

III. The Concept Outline

Uh-oh. Here's the part that's going to start to piss people off.

This breaks down the historical periods that teachers are supposed to use, and provides the conclusions that the students are supposed to reach. For instance:

Reinforced by a strong belief in British racial and cultural superiority, the British system enslaved black people in perpetuity, altered African gender and kinship relationships in the colonies, and was one factor that led the British colonists into violent confrontations with native peoples. 

This outline is exceptionally specific, and notable for what it does and doesn't include. It favors economic factors, and is not very interested in social or cultural matters. It isn't interested in military history at all-- wars appear briefly but little is said about how they are fought and won. The outline was also clearly not the product of any historians who believe in the Great Man theory of history-- very few individuals appear at all. Lincoln, FDR, and Reagan are barely seen in relation to the historic moments associated with them. The closer we get to modern times, the more evident are the attempts to touch all the properly balanced or politically correct, depending on how you feel about such things. We cover Japanese-American internment camps, but not the national collective actions to support the fight against the Nazis. And if you think God helped make America great, you will not love the new AP version.

I'm not prepared to argue that any of these things absolutely need to be included in a summary of US history. It's very much open to debate, or at least it's open to debate any place other than in an AP History class. In this large section, the AP folks have imposed one specific reading of American history. Here comes the straightjacket of flexibility for AP History teachers.

Speaking of which, the section begins with a nice outline that tells us how much instructional time to devote to each period, and how much of the test will cover the indicated span.

IV. An intro to the test itself. Let's let that be.

This is going to anger some folks on the right (and it should anger the left as well). The same people who think creationism should be taught in science class will object to this outline's omission of America's exceptional God-given role in the world, or the implicit criticism of America in some of the goals. They are wrong. Some things just have no basis in fact and there's no reason for us to teach them in school.

But they will be right about one thing-- they will call this course outline biased, and in that they will be correct. And when studying history, I don't care whether your bias is widely accepted or Crazypants McFringebob-- part of the whole point of doing history is the give-and-take, back-and-forth, argue-and-support of differing viewpoints about exactly what happened, why it happened, why it mattered, and what happened next because of it. Real authentic history is about the never-ending wrestling matches over these questions-- not the learning to accept the answers that a current authority offers.

Beyond that-- remember back at the top when the College Board said that one benefit of this reboot would be more time to study pieces of history in greater depth? Can we talk about the history of coming up with that claim, because it had to have involved being on some historically strong drugs. Who knew that AP would turn into the class where teachers said, "Boy, I'd like to continue this discussion, but it's Tuesday and we have to move on the next unit right now."

Somehow the College Board (now run by our old buddy David Coleman) has taken AP US History from a loose framework for college-level inquiry and deep freewheeling study and exploration to a hog-tied tightly dictated connect-the-dots learn by numbers course.

But teachers can rest easy, knowing that they will now be able to do a better than ever job of prepping students to take the US History AP test.

When Data Are Not Data

My favorite story of the last week ran in Valerie Strauss's "The Answer Sheet" on the Washington Post site and dealt with Arne Duncan's reaction to the increasingly loud chorus of VAM debunkers.

The research showing VAM as ineffective is piling up. Research by Polikoff and Porter eviscerated, finding no real predictive power and suggesting that what VAM measures is not what we mean when we say "good teaching." That study was particularly telling because it had large piles of Gates dollars piled up behind it.

Perhaps my favorite VAM study was performed by Marianne P. Bitler who showed that VAM predicts a teacher's effect on student test score growth about as well as it predicts the teacher's effect on the student's height.

People who play with this type of statistical analysis have been warning that VAM was not solid for five years now, but now in addition to the professional opinions of people who do this sort of thing for a living, we have actual studies by actual scientists with actual-- what is that stuff called? Oh yeah-- DATA!

Don't we love data now? Isn't the whole foundation of the VAM system the idea that it's cold, hard data, with like numbers and stuff, so we know it has to be right. So wouldn't the US DOE want to make use of all this handy data that's rolling in about VAM? Here's the response that Strauss reports from the DOE:

Including measures of how well students are learning as part of multiple indicators of educator effectiveness is part of a set of long-needed changes that will improve classroom learning for kids. Growth measures are a significant improvement over the system that existed before, which failed to produce useful distinctions in teacher performance. Growth measures — including value-added measures — focus attention on student learning and show progress. While these measures are better than what existed before, educators will continue to improve them, and sharp, critical attention from the research community can help.

Strauss is a real journalist, so she simply reports this quote with little comment. I am not a real journalist, so I can go ahead and say this-- what a load of bullshit.

It is a distilled version of the Reformster narrative. We know (because, you know, we just know) that there are a whole ton of terrible teachers out there. The old evaluation system doesn't show the existence of these gazillion horrible educators, so the old system must not work. We will look for a new system that does work, and we will have proof that it works when it confirms our belief that a huge number of American school teachers suck. VAM may not work any better than tea leaves or reading the bumps on a frog, but it tells us that many teachers suck, and that's good enough for us and certainly an improvement of the old system.

The spokesperson also indicated that the US DOE keeps track of the research.

So bottom line-- Duncan's office knows that VAM is crap. They just don't particularly care. Because data are only data when they say what you want them to say.

AP A-OK?

The US DOE, among its many promotional and marketing activities, has been pushing hard for AP classes. This week they aimed loud praise noises at the state of Colorado for increasing AP participation in the state. The laudatory article declares success because more students take and pass the AP courses.

But before parents and students get too excited about the spreading and blooming of AP courses, they should remember a few simple facts:

AP Is A Business, Not A Public Service

AP tests are a product of the College Board, the same people who bring you the SAT, and although the name seems to suggest a group of college scholars who gather together on some altruistic mission to guard the gateways of higher education for the Greater Good, the fact is that the College Board is just a business intent on making a buck and keeping its market share (it is also currently run by David Coleman, one of the co-authors of the Common Core).

Every time a teacher goes to a seminar to learn about designing an AP course, the AP folks make money. Every time a school buys AP materials, the AP folks make money. And every time a student takes the AP test, the AP folks make money-- a bunch of money. 

It was a great day for these folks when they hopped on the Education Reform Gravy Train and became the Official Education Course Product of Race to the Top. In Pennsylvania, for instance, a school's rating factors in how many AP courses are offered. This is extraordinary, like Ford getting the government to rate school district excellence based on how many Ford school buses they used.

You May Get Absolutely Nothing For Your $$

AP stands for Advanced Placement. The whole point of AP is supposed to be that you take the exam, get a good score, and your college gives you either course credit or a higher placement. In other words, you get to either replace English for Dopes with a higher level course, or you get credit for the course without ever having to take it. The second option is appealing to students; it's not quite so appealing to colleges, because it amounts to giving away free credits. And here's the important point to remember--

What you get, or not, is completely at the discretion of your college or university.

Many schools prefer that you learn things their way. (“We want a Dartmouth education to take place at Dartmout.”) They may not grant AP credit within your major (which is likely to be your strongest AP showing), and they definitely may not be in the mood to give away credits for free. The College Board folks pooh-pooh these sorts of trends, just as they pooh-pooh the plummeting market share of the SAT test and the growing tendency of colleges not to require it, but if you think dropping $90 on the AP test is going to pay off in college, you may be better off buying your kid a nice outfit to wear to the interview.


This chart can talk about the students who "qualified for college credit" but it's baloney. Nobody is qualified for college credit until the college offers it. (The chart also provides a quick guide to how huge a financial windfall Race to the Top has provided for the College Board).

Is There No Value?

One of the odd positives in the AP marketing scam is leverage for teachers. Where administration might in the past simple axed any requests for resources by classroom teachers, the AP marketing boondoggle gives teachers a new magic power. "Well, if you want to offer AP Basketweaving," says the teacher, "You are going to have to give us double-lab periods and buy these super-duper books."

AP also has the virtue of being a loose framework. When we "added" our first AP English course, we only had to perform some minor tweakage on the honors program we already had in place. But now, you know, it was AP!!!!!!! (use your super-hero voice) and so it was totally awesome. I used to call AP the alligator on a polo shirt, and little has happened to change my mind. It's savvy marketing, but it comes without handcuffs or a program in a box. I guess it says something about the current state of education that it qualifies as one of the less malignant tumors growing on the heart of schools these days.

[Update: Annnnd I had to take back some of those nice words about the loose framework almost immediately.]