Return of the Data Overlords

When the President and Arne Duncan took to all manner of media to pretend to stand up against standardized testing (but only, you know, the bad redundant not-so-good standardized testing), the New Testing Plan included some odd language and offers of assistance, and it all struck a chord of recognition for Peggy Robertson, an education activist who blogs at Peg with Pen.

For her, the signs point toward Competency Based Education. CPE has been floating around for a while, a kind of fig leaf for the educational dream of the Data Overlords. She has written pretty compelling and researched pieces about the whole business. Emily Talmage thinks the whole anti-test spin is a trojan horse. I don't think they're over-reacting.

We haven't heard from our data overlords in a while. When Leonie Haimson et al shut down inBloom and its dreams of hoovering up every speck of student data, too many folks heaved a sigh of relief and shrugged their shoulders. "Sure glad we're done with that." Well, we're not, and we never were.

If you doubt me, take a look at "Impacts of the Digital Ocean on Education." It's from Pearson way back in February of 2014, and it is one of many documents out there underlining the unique Pearson vision for students in the world. In fact, the textbooks, the instructional programs, the standardized tests, the huge mountains of profit-- all of those serve a central vision of swimming in the digital ocean. Here's a quote from that paper from Sir Michael Barber, the biggest Pearson Data Overlord of them all:

Once much of teaching and learning becomes digital, data will be available not just from once-a-year tests, but also from the wide-ranging daily activities of individual students. Teachers will be able to see what students can and cannot do, what they have learned and what they have not,which sequences of teaching have worked well and which haven’t - and they will be able to do so in real time. 

Got it? Even Pearson understands that Big Standardized Tests don't really get the job done, that what we really need to do is collect every piece of data from every piece of work that students do. You might argue (as I did) that teachers already do this every single day. But the Data Overlords have two problems with that. 1) Meat widgets like teachers don't record data as purely and beautifully as technology and 2) the data in teachers' heads is not easily accessible to the Data Overlords. They are more than happy to talk about getting rid of tests that will not fit in with the system they have in mind. In fact, testing isn't all that important to them if they can just capture ALL student activity data 24/7.

You can read more of the same in Pearson's eighty-eight page opus about the coming "renaissance in assessment" (or you can plough through the five posts I wrote about it after I read it. You're welcome). Again, here's just one pertinent excerpt:

Through the use of rubrics, which will define performance in terms of a hierarchically ordered set of levels representing increasing quality of responses to specific tasks, and a common set of curriculum identifiers, it will be possible to not only provide immediate feedback to guide learning and teaching but also to build a digital record of achievement that can be interrogated for patterns and used to  generate individualised and pictorial achievement maps or profiles.

Pearson is okay extending this collecting work to all aspects of the child, including "non-cognitive skills," so that the Data Overlords will not only know how good your child is at math, but how good a human being they are as well.

How would all of this work? I had a thought a few years back-- what if Common Core is not standards at all, but just a system of data tags? All the time teachers are spending "aligning" every lesson, every test, and even, in some cases, every item on every worksheet and assessment-- that's so that the results can be tagged and bagged and sorted to create a more easily crunched by the Data Overlords.

Look at this article from 2013, explaining how a symbiotic relationship between Pearson, their buddies/underlings at Knewton, and a school, would work. I can give you the gist of it in one image from the article.

Or check out this video from a Knewton honcho

That video is from 2012, at an event presented by the Department of Education.

The basic ideas here have cropped up in the world of adaptive testing, personalized learning, and anything that involves putting a computer in front of a student. The idea of competency based "badges" for students is out there (and has also been floated for teachers).

After reading pages and pages of this stuff, I think the Dream of the Data Overlords looks something like this:

From early in a child's life (maybe even their fetal days), every interaction between the child and the world is collected and cataloged. By the time the child gets to school, we already know much about her, and she is met by a computerized education program shaped by what we know about her. And every exercise, practice, comment, sentence, comma or sneeze that she completes becomes part of her fully digitized record as well as being instantly used to direct her educational program to suit her. By the time she's twelve or thirteen, we know where she should best be positioned in life, and we can shape her educational program accordingly. By the time she finishes high school (or whatever sort of teen-years education program the software deems most appropriate for her), we know all about her intellectual capabilities and skills in rich detail, as well as a picture of her soft skills (aka her personality and character). This will allow us to perfectly match her with the best vocational/career fit.

Data Overlords believe that if we know everything, we can control everything. Data Overlords are in some ways the scariest of the reformster tribes because they are mostly True Believers who don't ask two critical questions-- 1) is it really possible to do all this and 2) is it a good thing to do. Personally, I go with "no" for both. Data Overlords know that they face some obstacles (if you thought Common Core was federal overreach, Knewton will totally freak you out). But  I'll finish with a Barber quote I frequently trot out, because I think it's his most scary statement.

Barber recognizes that there are systemic, logistic and human obstacles to his grand design of a Data Overlord world. He recognizes that there are so very many details to work out. But--

Be that as it may, the aspiration to meet these challenges is right

At the end of the day, the Data Overlords believe that remaking the world in their own image is not merely a business plan or a great way to make a buck. They believe it is a moral imperative to impose their vision on the rest of the world. And that is scary as hell. 

PA: GOP Walks Away from Funding Crisis

As I've outlined elsewhere, Pennsylvania suffers from long-standing school funding issues. In recent years these have been exacerbated by pension funding issues and policies that allow charter schools to suck the blood right out of public systems. Put it all together, and Pennsylvania has the widest gap between rich and poor schools in the country.

But on top of all that, Pennsylvania is suffering from the flare-up of whatever chronic problem it is that has led to five budget impasses in the past ten years. We are on day one-hundred-and-oh-hell-you-have-GOT-to-be-kidding-me of Life without Budget. And in Pennsylvania, when we don't have a budget, people don't get paid. (Well, most people. In some manner that literally nobody is prepared to explain, the state has spent $27 billion of some money on some thing.)

This is not just a school issue. Right now you can't swing a cat in the commonwealth without whacking it into a service organization or government program that's having trouble meeting payroll and taking care of the citizens it was set up to serve. Senior citizens, the disabled, the unemployed-- all are looking at a patchwork of supports that appears to have been gnawed on by some very angry goat.

But every single school district in the state is getting hit. Back in August the teachers in Chester Uplands* made headlines by offering to work for free, but they were just the canary in this particular coal mine. Districts across the state are looking for ways to beg, borrow, and steal enough money to stay in business. Some have been pretty direct in their commentary-- the Erie School District asked the state to float them an interest-free loan of $47mill to keep the lights on. Districts have felt the need to announce how long they can last on cash reserves (fun fact-- in the last decade the state has passed regulations limiting how much money a district can park in its general reserve fund). Word on the street is that the state will NOT be reimbursing districts for the money they will be spending on the millions and millions of dollars they'll be borrowing to tide them over. Okay, it's actually about half a billion dollars.

In other words, on top of interfering with the stable operation of basic government services, this budgetary cockfight will end up costing taxpayers real money. Because if there's anything that taxpayers want to do, it's finance legislative logjams.

So this week, the legislature took some real action.

They went on vacation.

I'm not kidding.

Even as the State Auditor was holding a press conference about just how awful the budget crisis has become, the legislature was voting along party lines to take their two week vaca. And because our legislature is GOP-controlled, that means that our legislators have headed home to fiddle while the state burns (it also means our Democratic legislators could vote against vacation knowing that it wouldn't change anything except how they looked to the public). Or would burn, if anyone could afford matches and firewood.

Folks are loaded with creative ideas. Let's all refuse to pay taxes for the same number of days the budget is late. Let's cut legislator pay (the second-highest in the country) by a pro-rated daily amount for all the hundred-and-what-the-hell days they don't get their job done.

It is impossible from out here in the cheap seats to tell what is really happening, or not, because what plays out here is various acts of political gamesmanship. The legislature offers a "stopgap" measure which is essentially their original budget ideas for a shorter period of time. The governor vetoes it. All done with fanfare and an eye on the press releases. Pennsylvania residents can be excused for concluding that their elected leadership sucks.

And yes, I get that sometimes when negotiations are going badly, it's good to step back and clear heads. But this billion-dollar crisis is well past that point. Arguing about Titanic deck chairs is a waste of time, and "Let's take a second to clear our heads" is inappropriate when the burning house is collapsing around you.

But if you are a Pennsylvanian and this bugs you, I suggest you let your elected representative know about your displeasure. He should be easy to find. He'll be home for the next two weeks.

*corrected- I originally gave the incorrect district name here

Eva's Very Bad Month

It has not been a fun couple of weeks for Eva Moskowitz and her Success Academy chain of test score workshops. I just want to collect all these high points in one spot.

First, she found herself the subject of a John Merrow PBS piece on the use of discipline to push students out and raise the collective scores of her schools. It's worth watching the clip to see her performance:

This is what happens when you get too used to only appearing in media settings that you totally control.

But Eva fought back quickly, demanding an apology in a letter that I won't link to because, incredibly, Moskowitz included the easily-identifiable disciplinary record of a then six year old student. Because when your charter business is under attack, you fight back with whatever weapons are at your disposal against whoever stands in your way. Talk about putting adult interests ahead of the needs of children.

Does that sound like a violation of the law? The child's mother thinks so, and has filed suit against Moskowitz.

That was two weeks ago. Then this week, Moskowitz found herself facing off against the city. City Controller Scott Stringer put it plainly:

If an organization wants to be paid New York City taxpayer dollars, they need to follow New York City rules.  

Moskowitz wants those sweet sweet Pre-K tax dollars, but as she has periodically reminded the State of New York, school regulations are for little people, and she is not a public school when it comes to accountability (only when it comes to collecting checks). Moskowitz held a press event to declare her right not to listen to Stringer, featuring parents as props and a closed setting where only those she wanted present could attend.

But even as Moskowitz was standing up for her right to take public tax dollars without having to be accountable for them, she was getting questions about an article by Kate Taylor in the New York Times laying out just how determined Success Academies can get about pushing out students that they don't want to teach. This piece includes the damning story of one SA locations "Got To Go" list in which, incredibly, a principal actually wrote out a list of students that were marked for pushing out. Stories of Moskowitz's determined work in pushing out students (and teachers) she didn't want to teach has been widely documented, but the school hit list added a new level of awfulness.

Moskowitz tried to put out that fire yesterday (twittering public ed advocates noted that admission to the event was once again carefully controlled). The offending principal apologized, complete with tears and early departure, but Moskowitz said he would not be fired:

At Success, we simply don't believe in throwing people on the trash heap for the sake of public relations.

She also released an e-mail in which she called the principal "stubborn" and "dense." She indicated that she was too cool to take PR advice, and she insisted repeatedly that this instance was an anomaly. She also tried to provide evidence that she had been all over this way back when it came to her attention.

She did not indicate that Success Academy would be making any policy changes to avoid similar events in the future.

Politico's full report on the press event is worth reading, especially such dry observations as noting that after the NYT piece ran, "other charter advocates declined to come to Success' defense." I'm not surprised. Mike Petrilli, however, did run a piece in the Daily News defending Moskowitz's right to do what she says she doesn't do. So there's that.

All in all, the fiction of Success Academy's great achievements is taking a beating. We shall see if things start looking up once November rolls around.

Petrilli: Creaming Is a Feature

You have to give Mike Petrilli, Head Honcho of the Very Reformy Fordham Foundation, credit. He will say what many charter supporters will not.

The standard charter claim is that charters can do what public schools cannot-- take the same kids and get them to score well on standardized tests raise their achievement levels. They have been hemming and hawing all week over the revelation that Eva Moskowitz's Success Academy was caught keeping a "Got To Go" list of students who were to be driven out of their Very Special Test Score Factory. Success Academy has thrown that principal under the bus, and then had him publicly drive the bus over himself, and then underline it with a classic Moskowitz quote:

At Success, we simply don't believe in throwing people on the trash heap for the sake of public relations.

Success Academy simply doesn't just toss human beings aside because those people don't serve Moskowitz's purpose. Of course, that still leaves the mystery of how SA loses half of each cohort on the way to graduation. But all those parents talking about their experience of being pushed out? Liars or deluded or something.

Mike Petrilli calls "bullshit." In fact, he calls bullshit on everybody, including the people who have been howling at the Success Academy revelations, in particular taking a shot at Randi Weingarten of AFT:

What makes this sort of demagoguery more disappointing than usual is the nature of the issue at hand. As Weingarten’s own members know all too well, classroom disruption is a major problem. In a Public Agenda survey, 85% of public school teachers said that the experience of most students suffers because of a few chronic offenders.

Petrilli's position has been consistent and clear for years--  some students are a Big Problem, and schools should be able to make those students go away, so that the deserving, worthy students can have an education untroubled by troublemakers. I have a couple of problems with Petrilli's position:

1) After we get rid of trouble students, where do they go?
2) It's a mistake to assume that being a problem student is a static, immutable, hardwired, consistent condition.
3) The whole American public education deal is that we educate everybody, not just the "deserving."

For Petrilli, the whole point of charters is to give a space where "strivers" can stop being held back by all those Other Students who create disruption and trouble. And instead of yelling at people like Eva Moskowitz who are doing such a good job of winnowing out the non-strivers, we should give public schools the same sorts of powers.

This is exactly the wrong approach. Rather than piling old restrictions on charter schools, we should be working to reduce the restrictions on traditional public schools. 

Well, sort of. Though I have to ask-- if we actually did that, why would we need charters at all? But when you say it that way, it doesn't sound so bad. But then, as his closer, Petrilli says it this way.

By all means, we should work to serve all kids well, including the serial disrupters, but not necessarily in the same classrooms or schools.

Rather than blast Moskowitz, Weingarten and others should ask that district teachers have the ability to prioritize the vast majority of their students, too. That would be worth crowing about.

Prioritize our students? Like, decide which students deserve how much education?

Petrilli's point is not completely without merit, and as teachers often lack sufficient time and resources, many do perform a certain amount of educational triage by considering which students need us most. And every teacher knows the frustration of having a classroom tyrannized by one serial disrupter. But "prioritize" students? That sounds like a level of judgmental school administration that I'm not comfy with, and I suspect would provide an avenue for biases and concerns for compliance to run roughshod over actual care and concerns for the well-being of students.

Look-- Success Academy is not nobly rescuing the top strivers from difficult situations. They are picking winners and losers based on the school's preferences and the school's convenience, based on Moskowitz's two guiding values-- compliance and test scores. When a six year old cracks under that sort of misguided pressure, that's not revealing some sort of character deficiency or lack of striverness. It's revealing an institutional incompetence in dealing with six year olds.

But I appreciate Petrilli's willingness to just say it-- charters are only for the chosen few, those that the school finds deserving. What I'd really like to know next is how a system in which a school is the final arbiter of what level of education a child deserves fits together with the reformy ideals of school choice?

Reformsters and Dinosaurs

Last night my wife and I watched our newly acquired copy of Jurassic World, a movie that doesn't have an original idea in its head, but is still plenty of fun to watch. Even more than when we saw it in the theater, I'm struck by how the themes of education reform are laced through the film, and though I wrote about the movie at the time, I want a do-over, to expand on what I originally noticed.

Virtually every reformster foible is on display in this movie.

Our leading lady is introduced with a big Marked for Redemption sign on her forehead. She refers to the animals in the park as "assets," things rather than living beings, and she prefers to manage based on data and spreadsheets-- management by screen. She follows procedure rather than listening to her expert.

The movies baddest human is Vincent D'Onofrio's ex-military corporate tool. He's most immediately marked as a bad guy with his speech about competition, and how that's the road to improvement. What I noticed more clearly this time through is that he likes the idea of competition because he believes that he will come out on top-- competition is important because it's how other things are brought up to snuff.

Paired with that belief in competition is yet another rejection of expertise. Chris Pratt (playing what we affectionately refer to as Bert Macklin, Dinosaur Hunter) tries to explain to D'Onofrio all of the specifics and understanding needed to handle the almost-trained raptors, but D'Onofrio brushes him off because D'Onofrio believes that he just has a gut-level understanding that is greater than Pratt's actual knowledge and experience. D'Onofrio is so sure that he just knows how things go that he will prevail-- right up to the moment the raptor chomps down on his arm.

Also worth noting-- the billionaire backer of the park. He seems to be a voice concerned about the right things (Are the customers happy? Are the animals happy?) but he also suffers from a hubris problem. As he climbs aboard a helicopter that he intends to pilot, another character asks if there is anyone else who can pilot the copter. The billionaire replies, "We don't need anybody else." Again he believes in his own awesomeness over any needed expertise, and the result is death and destruction for himself and others.

Add--of course-- the scientist who has met the demands for a bigger, badder dinosaur without regard for the moral and ethical issues involved. Park management needs newer, scarier "assets" to keep their numbers up, and the scientist has delivered. Had the character ever read any Michael Crichton book, he might have paused to consider Crichton's favorite idea-- that human beings always underestimate the problems that come with their technological solutions.

All of these factors-- the focus on keeping numbers up, the impersonal data focus, the creation of artificial solutions, the belief in competition, the hubristic disregard for expertise-- combine to produce a monster. The monster was supposed to be the best, the creation that would save the park. Instead, it destroys it before being itself destroyed. It's all very, very reminiscent of the education debates, of the drive by powerful people whose faith is in their own rightness and not in expertise and experience to create something that is supposed to fix everything. But their values are warped and instead of trying to do what is best for the animals or the human guests of the park, they are really driving to create weapons, to create profits that will prove they are the best. What they create is meant to be the best, a savior, but because their values and goals are wrong, their creation is a destructive monster.

I suppose I might have spoiled a few details, but in truth there are no spoilers for this film because absolutely nothing happens that comes as a real surprise. And that's what's really interesting to me-- the characters who display the coldness, the detachment, the self-importance, the hubris that we associate with reformsters, are all immediately recognizable as characters who will be dealt either redemption or destruction. I venture a guess that nobody who watches the film sees D'Onofrio's character, hears him talk about how the raptors can be used, how competition makes the world work, how expertise can be ignored because he just knows-- nobody sees all this and thinks, "Yeah, that guy is going to be the hero."

Yes, the parallels aren't perfect. I'm happy to think of Chris Pratt as a proxy for teachers, but the dinosaurs end up as proxies for students and/or traditional public ed, which is less flattering. 

So public ed fans can enjoy the movie because the good guys win and the bad guys, mostly, get their comeuppance. And public ed fans can take heart from the fact that the good guys are readily recognizable by just about anybody, that our struggle does include recognizable archetypes. Maybe that will help the public really understand what is happening to public ed.

One thing, though, that might get missed in the Big Finale-- the scientist and his engineered embryos escape unscathed.

The Charter Knives Come Out

This week you may have caught this report that ran in the Columbus Dispatch and the Washington Post and the Detroit Free Press, to name a few. Turns out there's a study that shows that online charter school students learn far less than their bricks-and-mortar counterparts.

Check out this mind-boggling statistic:

Nationally, online charter students received on average 180 fewer days of learning in math and 72 fewer days in reading during a typical 180-day school year. In Ohio, with the largest e-school enrollment, students lost 144 days of math and 79 days of reading.

180 days fewer than 180 would seem to equal, well, zero. So on average, the study suggests that studying math in cyber-school is no more productive than 180 days of watching Dr. Phil. And 180 days is an average-- which means that some cyberschools actually move students backwards during the year, I guess. ("Students, in today's lesson, we'll be forcing you to forget everything you know about the quadratic equation.")

These are stunning results, even worse than some of the mean things that I have had to say about cyber charters (and I've said some mean things about the bloodsucking cyberschool vampires that have been allowed to lay waste to much of Pennsylvania's educational landscape).

Who would go after cyber charters with such verve?

Well, let's check the pull quotes. From the Washington Post:

Nina Rees, president and CEO of the National Alliance for Public Charter Schools, said her group was “disheartened to learn of the large-scale underperformance of full-time virtual charter public schools. While we know that this model works for some students, the CREDO report shows that too many students aren’t succeeding in a full-time online environment.”  

From the Detroit Free Press:

Greg Richmond, president and CEO of the National Association of Charter School Authorizers, said in a statement that the results are "deeply troubling."

"There is a place for virtual schooling in our nation, but there is no place for results like these," he said.

And from the Dispatch

“In Ohio and across the United States, students attending virtual charter schools simply are not learning enough,” said Chad L. Aldis, vice president for Ohio policy and advocacy for the Thomas B. Fordham Institute, which sponsors 11 traditional charter schools.

“Proponents of school choice are increasingly hard-pressed to defend virtual charters when their academic gains fall so far below the traditional schools against which they are compared.”

Yes, three top charter-loving outfits, all lined up to stick it to the cybers based on the results of this study conducted by-- wait for it-- CREDO.

Yup. If you didn't read these reports carefully enough, you might have missed that the Center for Research on Education Outcomes at Stanford. This drubbing of cyber charters comes from the choice-favoring, charter-leaning folks at CREDO, and then has been enthusiastically endorsed by a brace of charter advocates.

What's going on? A new epiphany about chartering? A willingness to confront hard truths? Opposite day?

I don't know. But I can guess. Let's see.... besides advocating for choice and charters, what else does Fordham do in Ohio? That's right--they run bricks and mortar charters.

One aspect of the zero-sum charter-choice landscape has always been predictable. At some point, the charter operators will have to fight each other for those sweet slices of that finite school funding pie. The lofty picture of "competition" has always been neat and bloodless, noble charter operators simply pursuing excellence in hopes that if they built a better math trap, the students would beat a path to their school.

But of course that's not always how competition works. Some times you get out the long knives and start trying to carve up competitors. Because until we fix the stupid, stupid funding system we've got now, more for me must always mean less for you.

So, yes-- charter operators will call for reforms (like say, a New York law that only schools with "success" in the name can be authorized, or an Ohio law that allows payments only for schools that have car manufacturers' names and pork products in their organization title). It was only a matter of time before they started promoting research that proved that some charters are Bad. So they shake their heads, cluck their tongues, and express sadness that Those Terrible Charters are taking attention away from Our Awesome Charters (accepting applications for next year's class soon).

Do I think cyber charters are often dreadful? Sure. But call me a little cynical if I see a bit of self-serving cynicism in the charter community's announcement that they are shocked-- shocked!!-- at some of the awful shenanigans going on. But only Over There.

The Only Good Standardized Test

As testing has risen once again to the surface of the ed policy soup,  I have found myself in versions of the same conversation, because people who like the idea of standardized testing really like the idea of standardized testing, and because I said the number of necessary standardized tests is zero.

Jennifer Borgioli Binis challenged me to be a little more precise in my language. I took exception to Robert Pondiscio's assertion that data from tests are the life blood of education and he took exception with my exception. Someone in the comments called me a "union shill." And a reporter asked me what the alternative to standardized testing would be.

It's a fair question. Is there such a thing as a useful standardized test?

But First a Few Words about Opposition

To have this conversation, we have to get one thing out of the way first. If you believe (and I think some reformsters sincerely believe it) that the only reason that teachers oppose the current high stakes test-and-punish status quo is because their self-serving union tells them to, you are blinding yourself to some real issues. First, there is a real gulf between national union leadership and rank and file teachers precisely because union opposition to reformster policies has been tepid at best. For the most part, NEA and AFT leadership is not whipping up opposition to ed reform policies-- they are trying to tamp it down.

The teacher opposition to testing comes first and foremost from teachers who are watching testing become a toxic and destructive element in our classrooms. Testing doesn't just drive the bus, but it drives it straight toward a cliff. It gets in our way, interfering with our ability to deliver real education. It's detrimental to our students. It is educational malpractice. And on top of all that, it is used in many places to deliver a professional verdict on our schools and ourselves with an accuracy no greater than a roll of the dice.

The other opposition to testing comes from the other people who see how it plays out on the ground-- the parents. The Opt Out movement was not created by teachers, is not led by teachers and, in some places, is actually potentially damaging to teachers under the current bizarro test-driven accountability system.

So if you imagine that test opposition is some sort of political ploy engineered top-down by the unions, you are kidding yourself.

None of That Answers the Question, so Let's Get Back To It

If I am such a dedicated opponent of standardized testing, what do I propose as an acceptable substitute.

Before we go any further, we'd better clarify our rather fuzzy terms.

"Standardized" Test?

Come to think of it, we'd better clarify "test" as well. For many folks, it's only a "test" if the student is answering questions. A five page paper assignment, for instance, is usually not called a test. In fact, the more open-ended the assessment, the less likely folks are to call it a "test." In schools, a test (students must prove they know something) is different from tests anywhere else (e.g. if we test the water, it is not up to the water to prove anything, but it is up to the tester to find a way to measure the nature of the water). Requiring students to prove themselves is the very first step in developing a bad assessment.

"Standardized" when applied to a test can mean any or all (well, most) of the following: mass-produced, mass-administered, simultaneously mass-administered, objective, created by a third party, scored by a third party, reported to a third party, formative, summative, norm-referenced or criterion referenced.

This broad palate of definitions means that conversations about standardized testing often run at cross-purposes. When Binis talks about the new performance assessment task piloting in NH, she thinks she's making a case for standardization, and I'm think that performance based assessment is pretty much the opposite of standardized testing. There's a lot of this happening in the testing debates-- people arguing unproductively because they have very different things in mind.

Acceptable Substitute for What Purpose

The confusion is further exacerbated by a myriad of stated and unstated purposes for standardized testing. This confusion about purpose has emerged as a huge issue in the ed debates because far too many of the amateurs designing testing policy don't understand this at all. At. All.

It's not just that reformsters argue that you can make the pig gain weight by measuring it. It's that they also assert that the scales used for weighing the pig can also be used to measure the voltage of your house's electrical system and the rate of water flow in the Upper Mississippi.

If we want to find an acceptable test, we have to first declare what the test is going to be used for.

Ranking schools, students and teachers

This is where purpose becomes important, because I can't think of a good test for achieving these goals because I don't think these goals are worth achieving. As a teacher, I don't need to know how my student compares to students in Idaho. I don't need to know that as a parent, either.

Comparing teachers to other teachers, schools to other schools, students to other students-- it's a fool's game. First of all, I can only make the comparison based on a narrowly defined criteria. Otherwise I'm reduced to deciding if my insensitive smart flabby artist student ranks lower or higher than my sensitive tall winning cross country racer student. The comparison only has meaning if it is based on narrow criteria (which student answered the most math problems correctly on Tuesday)-- but what good is a narrowly defined comparison.

If I find that my smart, funny wife is not as smart and funny as some other woman, should I be unhappy in my marriage? If this delicious steak is not as delicious as the steak I had last night, should I spit it out? If all the teachers in my school are great, should it be closed down because some other school has greater ones?

The signature feature of a ranking system is that it locates losers. But what decent teacher would stand in front of a class of thirty on the first day of school and say, "Five of you will turn out to be losers." Testy science wonks like Binis would scold me for saying "loser" and argue for something less loaded and more clinical, but I'm working with students and all the sugar coating in the world will not hide the medicine in this model.

Ranking and rating means that even if everyone is excellent, the least excellent must be marked Below Basic or Underperforming or Just Not Good Enough. A system based on ranking and rating is a system that assumes that in every endeavor, there are people who just aren't good enough. I reject that view of the world, and so I reject any testing system designed to re-inforce that view. If everybody in my classroom does a great job, everybody in my classroom gets an A.

Providing feedback for parents

Here we have a standardization problem because not all parents want the same feedback. Is she getting an A? Is she passing? Is she developing a better grasp of abstract language particularly as used in classic literature? Is she okay? Does she seem happy? These are all types of feedback I've been asked for by some parents. What one measuring tool would satisfy all those questions?

Standardized testing is repeatedly sold with the myth of the clueless parents, the parents who have no idea how their students are doing. But the solution to this problem is transparency, the levels of which can be controlled by the parents.

For example, the electronic gradebook. Our parents can look up their students any time and see exactly what I see when I pull up the gradebook. Some of my parents look every day. Some look never. Some look and then call or email me to ask, "So what exactly was this one assignment."

When we control the available information, we do parents a disservice. Only revealing the grade at report card time is a disservice. But anyone who has taught at a school with big detailed portfolio gradeless systems can tell the story of the parent who looked at all that data and said, "Look, can you just tell me what grade she's getting?"

Parents deserve just as much feedback as they want. Standardized testing has nothing to do with providing that.

Feedback for teachers

Any decent teacher generates this kind of data daily. Any lousy teacher will have no use for standardized test data even if it arrives on gold-clothed ponies.


You are dodging the question

Okay, yeah. I've laid out my usual assortment of objections to standardized testing, but I still haven't said what would be an acceptable substitute. If you're still here, I'll try to address that now.

What qualities would an acceptable-to-me standardized test have?

If I ever were to find a standardized test that I could live with (or even date regularly), this is what it would look like.

Criterion-based (and so, objective)

If I'm going to measure my students against a standard, not against each other. I can use the test to answer the question, "Do my students know how to find verbs" or "Can my students identify dependent clauses?" If every student in my class can't potentially get a top score, I'm not interested. And if it's not objectively scoreable, it's no help. That means that no standardized test is going to be used for any higher order critical thinking type skills.

(This is part of the whole point of Depth of Knowledge testing love--it creates the illusion that higher order stuff can be scored objectively. But no, it can't).

It is possible to come up with standardized questions. I once had a textbook with great literature questions-- but I still had to evaluate the answers myself.

In fact, I can only see using a standardized test for checking the lowest levels of simple operations-- simple recall, basic application.

As Close to Authentic as Possible

I want a task that actually assesses what it claims to assess. Multiple choice questions don't assess writing skills. Click-and-drag questions don't assess critical thinking.

Transparent

This ought to go without saying, but if I don't get to see the questions, the answers, and the exact results from my students, then, no, thank you. I can do better myself.

Adaptable

I rarely re-use my own test-like assessments; instead, I make new ones each year to fit the class and the instruction. Particularly when I'm working summative assessments, I'll create something that focuses on the issues that we're struggling with. For instance, if we're solid on spotting infinitive phrases but have trouble picking out gerunds used as direct objects, I can design a test that will help both me and my students. I can adjust assessment to build confidence or prompt a come-to-Jesus moment.

Expertise and Convenience

There are lots of things I don't know. Materials prepared by people who are experts in particular areas are a necessary aid, and those sometimes include assessments. I'm happy to have an expert in a particular field in my classroom.

And at some points, I can use the convenience of having something pre-built to save me some time.

So, the acceptable alternative...?

Man, this ended up far longer than I meant it to, but I wanted to seriously examine my thoughts about this. Do I really think that there are no necessary standardized tests?

Well, Binis is correct when she argues that we all use standardization because we don't completely individualize everything from assessment through evaluation-- but that's a hugely broad definition of "standardized."(She disagrees with my reading of her "ever do this..." list.)

By that standard (har) everything used with two or more students is a standardized test-- and maybe it's useful to think of standardization as a sliding scale. The more we broaden the reach of the assessment, the more students we try to make it each, and the more we try to make the grading of the test be quick and uniform, the less useful the assessment becomes. A test that you can give to every student in America and which can be scored in just a week will by necessity be inauthentic and measure little. So for best classroom assessment, we stay as close to the individualized specifics end as we possibly can. The more that an assessment is developed in response to specific instruction by a specific teacher of specific students, the more useful that assessment will be in performing the most useful function of any test-- telling students and teachers where their strengths and weaknesses lie.

Yes, that information is not what the policymakers would really like to have. But the information they would like to have is completely useless to me in the classroom (and so far, they've found no reliable method for either collecting or using such information anyway). I'm not convinced that information can be collected by standardized tests anyway, but Good lord in Heaven am I still typing???

The number of necessary standardized tests, the number of tests I really need in my classroom? I still say zero. Mind you, I'm not saying that all standardized tests are an evil plague, and stripped of baseless high-stakes consequences, their plaguiness is greatly reduced. There are standardized tools that are tolerable, and a few that might rise to the level of useful. But necessary? Needed?

Still zero.