PA: Value-less Data

It's autumn in Pennsylvania, which means it's time to look at the rich data to be gleaned from our Big Standardized Test (called PSSA for grades 3-8, and Keytsone Exams at the high school level).

We love us some value added data crunching in PA (our version is called PVAAS, an early version of the value-added baloney model). This is a model that promises far more than it can deliver, but it also makes up a sizeable chunk of our school evaluation model, which in turn is part of our teacher evaluation model.

Of course the data crunching and collecting is supposed to have many valuable benefits, not the least of which is unleashing a pack of rich and robust data hounds who will chase the wild beast of low student achievement up the tree of instructional re-alignment. Like every other state, we have been promised that the tests will have classroom teachers swimming in a vast vault of data, like Scrooge McDuck on a gold bullion bender. So this morning I set out early to the states Big Data Portal to see what riches the system could reveal.

Here's what I can learn from looking at the rich data.

* the raw scores of each student
* how many students fell into each of the achievement subgroups (test scores broken down by 20 point percentile slices)\
* if each of the five percentile slices was generally above, below, or at its growth target

Annnnd that's about it. I can sift through some of that data for a few other features.

For instance, PVAAS can, in a Minority Report sort of twist, predict what each student should get as a score based on-- well, I've been trying for six years to find someone who can explain this to me, and still nothing. But every student has his or her own personal alternate universe score. If the student beats that score, they have shown growth. If they don't, they have not.

The state's site will actually tell me what each student's alternate universe score was, side by side with their actual score. This is kind of an amazing twist-- you might think this data set would be useful for determining how well the state's predictive legerdemain actually works. Or maybe a discrepancy might be a signal that something is up with the student. But no-- all discrepancies between predicted and actual scores are either blamed on or credited to the teacher.

I can use that same magical power to draw a big target on the backs of certain students. I can generate a list of students expected to fall within certain score ranges and throw them directly into the extra test prep focused remediation tank. Although since I'm giving them the instruction based on projected scores from a test they haven't taken yet, maybe I should call it premediation.

Of course, either remediation or premediation would be easier to develop if I knew exactly what the problem was.

But the website gives only raw scores. I don't know what "modules" or sections of the test the student did poorly on. We've got a principal working on getting us that breakdown, but as classroom teachers we don't get to see it. Hell, as classroom teachers, we are not allowed to see the questions, and if we do see them, we are forbidden to talk about them, report on them, or use them in any way. (Confession: I have peeked, and many of the questions absolutely suck as measures of anything).

Bottom line-- we have no idea what exactly our students messed up to get a low score on the test. In fact, we have no idea what they messed up generally.

So that's my rich data. A test grade comes back, but I can't see the test, or the questions, or the actual items that the student got wrong.

The website is loaded with bells and whistles and flash-dependent functions along with instructional videos that seem to assume that the site will be used by nine-year-olds, combining instructions that should be unnecessary (how to use a color-coding key to read a pie chart) to explanations of "analysis" that isn't (by looking at how many students have scored below basic, we can determine how many students have scored below basic).

I wish some of the reformsters who believe that BS Testing gets us rich data that can drive and focus instruction would just get in there and take a look at this, because they would just weep. No value is being added, but lots of time and money is being wasted.

Temporary Police Order

This weekend features the major festival of the year in my town, and as my wife and I walked back from yesterday's round of festivating, we noticed a sign posted many times along the street. It was a no parking sign, boldly lettered "Temporary Police Order," and we made a quick joke about why temporary police would be allowed to issue orders about parking.

But it got me to thinking.

The preferred method for teaching and testing reading in the [Insert Name Here] Core Standards is to treat reading as a discrete set of skills, completely unrelated to any prior knowledge. David Coleman famously admonishes us to stay within the four corners of the text.

A really great reader, Coleman and his acolytes suggest, can read anything in his native tongue, even without prior background knowledge or contextual knowledge from outside those four corners.

And yet here is a simple text, a text so short and sweet that it fits on a tiny sign. And without prior knowledge, without information from outside the four corners, we can't understand it. We can't "read" it.

Does it announce a order issued by police who are only hired for a short period of time? Is it an order issued by the regular police that will be in effect only for a short period of time? In truth, the former seems more likely if I stay within the four corners-- wouldn't the latter be better expressed by "Police Temporary Order," as awkward and ungainly as that sounds?

Of course, the perceived awkwardness is a function of my prior knowledge of how those three words are best arranged. So using that prior knowledge is cheating.

In fact, I'm already cheating by being aware of the context of the text, which is a sign stapled to a piece of wood driven into the ground beside parking places. If I did not have the "no parking" portion of the sign in front of me, I would have no way of knowing if "order" meant a command or directive as opposed to the absence of chaos or a particular arranged sequence of police. So "temporary police order" could be a reference to the sequence in which some part-time police officers might be standing in line, or it could mean that police have quelled ongoing chaos, but that chaos can be expected to erupt again at any moment. Without the context of the sign, I might imagine that the text originally appeared on a pizza shop takeout form, and now a whole new set of possibilities open up.

The clarification can further depend on my knowledge of local history. Is the use of temporary police  pretty standard fare here, or does this town depend on a standing police force? Have I encountered rent-a-cops in municipalities often enough for me to think of them as common, or am I unacquainted with that police hiring technique? My own understanding will influence my ideas about which reading of the text seems more probable.

My understanding of the text rests firmly on my prior knowledge and the context in which it appears. In fact, without employing prior knowledge and context, I cannot reach a definitive reading of the text.

My point? Folks like Coleman whose conception of reading is that it is a simple decoding exercise (like dialing in the combination of a safe) or a set of skills that can be simply exercised cut off from any prior knowledge or understanding outside the four corners-- those folks have a poor understanding of just how complex the act of reading actually is and just how difficult it is to measure. You will find it nearly impossible to create a reliable measure of reading that would cut out prior knowledge, restrict readers to the four corners, and still somehow meaningfully measure reading skills. I'm not sure you could do it at all-- not even with a temporary police order.

Driven By Parent Choice

That's the dream for some reformsters-- a school system that is driven not by regulation or government edict and certainly not by the professional educators who work there, but a system that is driven by parent choice.

I believe there are probably idealistic purists among the visionaries who really do believe that we'd have a system where parents peruse sets of pure and transparent data and then make informed choices based on that data, allowing schools to compete and improve by responding to how their data plays out in the marketplace. These purists are sweet, like delightful five year olds who still wait breathlessly for Christmas morning and the proof of Santa's arrival.

But there are also advocates of choice who are less idealistic and less pure. They understand what having a system "driven by parent choice" really means. I'll give you a hint-- it doesn't mean a system driven by parent choice.

Idealists talk about parent choice as if it occurs in some sort of iron-clad vacuum, influenced by nothing by clean data and clear thinking.

That's silly. Parent choice is malleable, shapeable, bendable, and open to influence. Yes, technically, in a choice system, the ultimate decision is theoretically made by parent choice. But that parent choice does not begin and end with parents-- it is molded and directed by a hundred other forces.

Purists imagine parents poring over spreadsheets, consuming mounds of impartial school information with a tasty topping of crunched data. But as we have already seen, charters take a strong hand in what data gets to the public, spinning and angling for a good marketing picture. They direct marketing toward particular parents, and they offer benefits that go far beyond any data that can be crunched (I'm remembering Pennsylvania cyber-school ads that suggested cyber-school would make your child happier than public school).

To get money out of the tightish fist of a government agency can be tricky, what with all those rules and regulations. But to get money out of a parent (especially if it isn't actually the parent's personal money), you can appeal to emotions and prejudices with slanted, incomplete and just-plain-false pitches.

Saying that such a system would be driven by parent choice is like saying the fast food, soft drink and automotive industries are driven by consumer choice. They are-- but that consumer choice is driven by marketing that involves everything from emotional appeals that have nothing to do with product quality, to manipulating the market to do things like buy up all the shelf space in the store, to simply carpet-bombing the airways so that your product is best-known. And that at least is a market in which you aren't allowed to offer battery-acid cola or brakeless automobiles; some states (looking at you, Ohio) have not yet figured out how to keep flat-out charlatans from bilking parents with built-to-fail charter flim flams.

"Driven by parent choice" just means "run on the free market," and the free market runs on marketing. "Driven by parent choice" just means opening parents up to every marketing maneuver and sales shenanigans in the book.

Greene's Law of the Free Market: The free market does not foster superior quality; the free market fosters superior marketing.

Am I saying that parents are just too dumb to navigate a free marketty education system? Not at all. But I am saying that parents, like most US consumers in today's marketplace, are people who are bringing a butter knife to a gun fight. I am saying that parents who have their hands full with the daily business of holding a home and family together may have trouble doing the full-time research work necessary to cut through the fog and smoke and the lies and half-truths and spun baloney of charter marketing. I am saying that schools should not have "caveat emptor" stamped on their front doors.

And I'm saying that folks who say "driven by parent choice" as if that is a pure, clear, clean solution are either fooling themselves or trying to fool everyone else.

ICYMI: This Week's Edubloggery Sampler

Well, this has been a week. I will leave you to find your own pieces about John King failing upward and the NEA deciding to shove a Hillary recommendation down membership's throat. In fact, maybe you'll enjoy some reading to take your mind off all of that.

Highlighting Websites and Blogs


Nancy Bailey offers a great, mostly-organized list of some of the folks writing about the Great Public Education Debate. I include this not because she says nice things about me, but because it's always good to expand your reading horizons

Does Subject Matter Knowledge Matter As Much As We Think?

A look at some research about what has impact in the classroom. The most powerful part is seeing what does have an impact

A Charter Place Parable

Short and sweet tale of why charters are a bad idea, from the superintendent of Chapel Hill schools (so you really might have missed it)


AP Classes Are a Scam

This article from the Atlantic is actually three years old, but it's still worth a read now that the PSAT is more directly set up as a marketing tool for AP courses.

A Look Back at DeVos

Links to three videos of reformsters gabillionaire Richard DeVos talking about how public education needs to be the next frontier of trashing democracy for fun and profit. Oh, and the videos go back to 2002.

The Reformy Arguments Are Getting Worserer

This week the Jazzman got himself into an argument with one of the crack writers over at Education Post. This is the final piece of the series-- you can backtrack if you like, but this captures the Jazzman at his fact-wielding best.

Class Is In Session With Jose Luis Vilson

Vilson was profiled this week, and actually, sadly, it would be notable enough that any publication decided to profile an actual working teacher, Vilson is also an accomplished author and activist, so this profile is doubly rewarding. Lift yourself up by reading about one of the good guys who is actually making his voice heard from inside the profession.

An Open Letter to Jerry Oleksiak (PSEA President)

Dear Jerry:

It has been a disappointing couple of days, what with the announcement of the announcement that the terrible Arne Duncan will be replaced with the even-worse John King. On top of that, we get the NEA's announcement of the early endorsement of Hillary Clinton. Now, today, on Facebook, I find your rationalized spin for that endorsement. I am beyond disappointed, but rather than simply stomp off angrily without saying anything except, "Cancel my membership," I want to address your comments, despite the fact that this endorsement fiasco drives home the fact that nobody in the state or local union is paying the slightest damn attention to the members. I am in my thirty seventh year of teaching in a classroom. I'm a registered Democrat who votes in every single election, and I'm a past local president of my union. So understand that I am not just some random crank, but an experienced and long-involved crank.

After presenting the background of your involvement in NEA's decision, you ask this question.

As you make your own decisions about whom you will support, I would encourage you to ask yourself this question: "Does this candidate support public education, our schools, and our students?"

It is an excellent question to ask, and one that I have asked myself. In fact, it's in asking this question that I arrive at my personal decision not to support Clinton. But before I talk about my own data on the issue, I want to see how that question leads you and NEA to an endorsement of Hillary Clinton. So let's look at what you've got.

I have spoken to many PSEA members over the past year, and one concern I hear again and again is that public education in the United States is at a crossroads. Our colleagues you elected to the NEA board have heard similar sentiments, as have their NEA Board colleagues in other states. That is why they felt it was so important to act early and decisively to recommend someone for president who is the right person to take our country in the right direction.

Do not talk to us as if we are children. First of all, the NEA higher-ups did not get together to endorse "somebody" as the right person. You got together to consider endorsing Hillary Clinton, and you did it in hopes that by stepping in to save her flagging candidacy at the point it's beginning to circle the drain, the NEA would earn some favors, or at least some standing with a possible Clinton administration. If Clinton were not starting to worry about Bernie Sanders, the NEA would never have been having this conversation.

Secretary Clinton is a longtime supporter of public education and a longtime friend of NEA. She has worked throughout her career to make sure every child has access to preschool, and every child has access to quality healthcare through the CHIP program. 

Those two examples mean nothing in terms of supporting public education. And if you want to convince me she's a friend of NEA, you'd better get specific, because while it's hard to prove a negative, I'm damned if I can think of a single time she's done something for the union.


She opposes over-testing of students, and has fought for greater resources for our schools. 

When? How? You follow with a quote in which she says something nice about public education, but general platitudes are not reassuring, particularly when we're talking about someone who praised Jeb Bush's work on education in Florida. 

Secretary Clinton will also be a voice for our professions. She supports higher salaries for educators and collective bargaining rights. Secretary Clinton has said, "It's time to stand up to efforts across our country to undermine worker bargaining power, which has been proven again and again to drive up wages."

Surely the NEA leadership is not that obtuse. Higher salaries for teachers is popular talking point among many education reformers, who see higher pay-- for some-- as a useful tool in de-professionalizing teaching. Some like the idea of higher pay--linked to test results. Others like the idea of well-paid super-sardinemasters, with teachers handling hundreds of students in a single class.

And if Senator Clinton feels strongly about bargaining rights, why has she not spoken out strongly about any of the direct attacks on those rights.

She will listen to our ideas, be sensitive to our policy recommendations, and appoint a secretary of education who will do what's right for our schools and students.

Who do you mean by "our" exactly? Because apparently I can't even get my own union to listen to my ideas, so I'm not thinking members concerns will get passed on. "Be sensitive to" is the weakest kind of weasel language, as is the line about the secretary of education.

I mean, part of my issue here is that this is a bunch of weak-sauced general pablum at a time in which we face very specific, very direct attacks on our profession. If the candidate can't talk about specifics, I'm going to assume that the candidate is either ignorant of them or chooses not to address them because the candidate thinks I wouldn't like what she/he has to say. If soldiers from East Bottlevania were landing on the beaches of New Jersey, slaughtering civilians and blowing up cities, I would not be looking for a President who says, "In general, we are sensitive to the idea that foreign nationals should not be entering our nation's mainland and doing things that make our citizens upset." 

There has been speculation out there that NEA has not followed its prescribed process in making this recommendation.

Again, you can't be this obtuse. The speculation has not been that NEA is not following its process-- the speculation is that NEA's process sucks. I can think you can find plenty of members who have always thought it sucked, but generally felt that A) they couldn't do anything about it and B) it didn't usually involve really damaging decisions. This time is different. This time teachers are feeling really, really pushed into a corner and attacked, and meanwhile, their state and national unions aren't hearing them or addressing the problem. The proscribed process is one more way in which rank and file members are left feeling unheard and ignored by their own leadership. I don't care if you followed the letter of the rules that you set up for yourselves. The rules stink.


Additionally, as I've written before, this was exact;y the wrong for this type of political maneuvering. Democratic processes are under attack in this country, and to take action that makes a blatant lie out of Eskelsen-Garcia's "This is what democracy looks like" statement is exactly the wrong action at the wrong time. I don't care if it follows the policy and procedures manual or not.

Over the recent summer and into the fall, we have watched the presidential campaign kick into high gear. We have witnessed several candidates stake out positions of support for:
- Cutting funding to public schools;
- Linking educator pay to student test scores;
- Expanding private school vouchers;
- Weakening collective bargaining rights; and
- Testing kids even more than we do now.
Some candidates even want to go so far as to get rid of the U.S. Department of Education!

The subtext here is supposed to be, "But the GOP is really, really worse," I suppose, and I cannot begin to tell you how thoroughly deeply completely utterly sick I am of having my union leadership tell me, "Well, we have to support the guy who will cut off our legs, because the other guy will cut off our legs and arms both."  I have had this argument with your predecessors, because this is a stupid, self-defeating line of reasoning, and we have suffered decades of misery because of it. The appropriate response to two candidates who each want to hurt us severely is to support NONE of them.

But your list is particularly galling because some of the things listed here are much-beloved by Hillary Clinton. She is a long-time fan of vouchers and privatizing schools through charters. The Center for American Progress has close ties to the Clinton campaign (John Podesta, previous CAP head and current Clinton campaign chief is just one example), and CAP has pushed for charters, high stakes testing, Common Core, and teacher evaluations tied to testing.

Faced with these grim realities, the NEA board chose to support a candidate who can win this election and will be a champion for our schools, our students, and our professions.

Yes, ultimately, this is about NEA leadership's belief in Clinton's ability to win-- at least, if she's not too bloodied by Sanders. But for the love of God-- if there is a credible reason to believe that Clinton will be a champion for any of those things, you had better share it with us, because I'll be damned if I've seen it.

Look-- beyond individual candidates, PSEA and NEA need to take a good hard look at their relationship with the Democratic party. In Pennsylvania, Ed Rendell was a disaster for public education, and nationally, the Obama-Duncan administration has been the most destructive ever. Like good little soldiers, we teachers keep sending our support, and our elected Democratic politicians keep stabbing us in the back. I'm pretty sure I'm not the only teacher who's had enough, and now I see us pushing a Clinton Presidency, which for education looks like more Obama Presidency, which simply doubled down on the worst policy choices of the Bush Presidency.

Other states found the backbone to stand up to the national union on this. It would have been heartening to see Pennsylvania do the same. Instead, once again, here comes the union leadership with carefully crafted spin and PR designed to manage the members and get them to fall into line. Honestly, I suppose I'll stew over quitting the union for a while and ultimately not do it, for a variety of unrelated reasons. But it's a union affiliation that am now ashamed of. This was a bad choice, divisive and destructive of NEA bargaining strength and dismissive of members at a time when we really don't need to be ignored by one more national organization, as well as supportive of a politician who we have no reason to believe will have our backs, ever. All the more ironic when NEA leadership has admitted that another candidate is much more in tune with our concerns.

PSEA has been trying to address the questions of 1) how to get more young teachers into the union and 2) how to get better PACE participation. Pretty sure this will not help. "Hey, young teacher, please join PSEA. We would love to have your dues, and we will be happy to tell you what you think." Not a winner. Also, "Give us money to support politicians who will only stomp on you a little," is not a great sales pitch.

This was a bad call, and your attempt to justify it only made it sting worse. Feel free to contact me with any information that would help me better understand. In the meantime, I'm just going to continue being pissed off.

Sincerely,

Peter Greene

The Future for Duncan and King

Let me join the sixty gazillion people who will have something to say about the news from the US Education Department. I'll make my comments in the form of predictions.

Duncan will spend his last couple of months doing what he's been doing for years-- making statements that sound pretty good, connecting them to terrible policies, and generally being ignored by Congress.

Some people will be really excited that he is leaving. These will be the same people who felt certain that Duncan was some sort of rogue agent, sneaking off to implement terrible policies behind the President's back, instead of the President's Guy, a cabinet bro who did just what Obama wanted him to. There have been fewer and fewer of these people over the years, but they still exist, and they still think that when Duncan leaves USED, things will get better. These folks should try investing in Floridian swampland.

Everyone east of the Mississippi who has been paying attention will express horror that former New York state ed chief John King had been names Duncan's replacement. In terms of policy, these two are cut from the same cloth. King loves him some Big Standardized Tests, test-based teacher evaluations, Common Core, and a whole world of privatization via barely-regulated charters. King has a hell of a personal story, from which he has failed to learn some critical lessons (a teacher changed his life, and he would like to install a system that would strap such a teacher to a giant steamrolling lemon). King does not suffer from a great deal of humility (he once astonishingly invoked both Martin Luther King, Jr., and Abraham Lincoln as supporters of Common Core and other reformy nonsense).

John King will make people actually miss Arne Duncan. King has a deft touch with the public. His biggest PR coup was to set up a series of meetings to educate the public about the awesomeness of his programs, and when they refused to sit quietly and be hectored, but instead actually talked back to him, King simply canceled the meetings. King never met a corporate reformy idea he didn't like, or a person objecting to such ideas that he thought was worth listening to. Duncan at least knew how to pay lip service to the concerns of stakeholders in public education; King will be so dismissive and obnoxious that folks will beg for Duncan to come back and tell us all pretty, pretty lies.

John King will most likely take office. There has been a lot of hopeful speculation that since he never went before Congress to take his "special advisor" position with USED, he somehow won't be eligible to act as secretary of the department. I can't think of any reason that would be true. And since he is a huge proponent of the reformy ideas beloved by republicans and democrats alike, I can't imagine who's going to stop him. Yes, he'll be terrible at it, but you may have noticed that competence is not necessary for gummint work.

John King will give a boost to national opt out. Remember, this is the guy who sowed the seeds of New York's opt out movement by being an intransigent asshat. There's no reason to think he won't scale that behavior up to the national level, allowing NY's highly successful opt outers to point and say, "See? See what we're talking about??"

King will also help convince Congress that any finished version of ESEA should keep the USED Secretary No Touching Anything clause.

Arne Duncan will get a nice cushy job somewhere, most likely working as a lobbyist or consultant or corporate hood ornament. He'll occasionally make a speech or a post somewhere, but nobody will be heavily influenced. If he is patient enough long enough, his stock will rise steadily as people forget what a disaster he was while they simultaneously build a huge storehouse of anger in reaction to Secretary King.

Fun times ahead, indeed.

Dammit, NEA

The national leadership of the NEA today is one step closer to ramming the endorsement of Hillary Clinton down its members throats.

Steven Singer zeroed in on the maximum-irony quote from NEA president Lily Eskelsen-Garcia:

“We are what Democracy looks like.”

With those words, Lily Eskelsen Garcia took the reigns of the National Education Association (NEA) as President in 2014.

Well, no. This is not what democracy looks like. Not even if you squint real hard and tilt your head.

This is not what a recruitment drive looks like, as NEA continues to wish that it could take a stronger hold among younger members. In my corner of the world, the state called some local union leaders together to talk about the secret of getting more contributions for the political action committee. This does not look like how you do that, either.

You do not build a union by going to young professionals and saying, "Hey, we'd like you to give us some dues. In return, we'll tell you what we want you to think about national issues without ever checking with you to get your opinion." You do not build a union by telling your members, "Hey, we know what's best for you. Shut up and sit down."

Democracy is under attack in this country. In Alabama, where the system is now rigged to deny poor, black voters the ID now required to vote. In city after city where the model for running schools has become to silence local voters and taxpayers.

Now more than ever, the union should be standing up for democracy, modeling for the world how it works. But a union that acts like this simply throws away whatever moral authority it might have had. It turns the declaration "We are what democracy looks like" into a cynical joke.

And in return, we get what? Nothing. Not a damn thing. Has Clinton made even an empty politician's vow to come out in support of public education? Nope-- not a peep, not a sound, not a word. And we did not so much as demand that she court us, not even a little.

There is a special kind of anger that comes when you have tried to defend someone and they insist on doing something indefensible, and that's the anger I'm feeling right now. At a time when so many people are punching holes in the ship, NEA shows up with buckets of water.

On what principle will this early endorsement of Clinton be defended? We figured it might get us more power, maybe? It might earn us a seat at the fabled table? It can't be defended as an expression of the will of the members, because NEA's leaders have no idea what the membership wants. It can't be defended as a stand on principle. And it certainly can't be defended as a rush to stand up for a friend of public education, because Clinton, who has long ties with Broad and praise for Bush's work on education in Florida, has not shown us one reason to be trusted.

In fact, the Clinton campaign has tried to establish its bona fides by sticking to safe issues around the edges-- pre-school and college availability. On public K-12 education, where there is plenty of room to come thundering out in favor of traditional public ed, in favor of preserving the voice of communities, in favor of preserving one of basic pillars of democratic process, Clinton has offered nothing.

I am, as you might have guessed, pissed. Pissed like I was when Dennis Van Roekel implicity accepted the notion that we needed Common Core because America's teachers were falling down on the job.

What the hell does my national union stand for? At what point will my national union leaders actually demand something from politicians in the name of public education and the teachers who work there? And what is the point of belonging to an organization that does not appear to value any of the things that I value?

How many times can I enter a conversation with my non-union friends about the union where I have to wade in with a sheepish, "Well, yeah, but--" when I am running out of things to put after the"but"?