Welcome to the new Charter Choice Cafeteria! Can I help you?
Wow! It's so clean and shiny here. And is that.... is that steak??
Why yes. We believe that all students should have the chance to eat steak for lunch.
Well, that's great. My usual public cafeteria only has meatloaf every other day, and it's not so good. So I would really love steak for lunch. Can I just--
Just a second. Charter Choice Cafeteria is only open to a few students. You need your Lunchtime Strivers Club Card to eat in here.
Well, how do I get one of those?
Just put in your application for the CCC lottery. You fill out these six forms available between the hours of 9 and 10 at our downtown office. Then submit them at the proper address and later we'll hold a drawing-- you have someone who'll take care of all that for you, right? Here's a flier.
Um, I guess. You know, nobody on this flier really looks like me. Anyway, do you serve steak every day?
The steak is today's meal. We serve other things the rest of the week that are totally as good as any steak, at least as far as you know. Very steak-like.
Sure. Hey-- that rail seems awfully close to the serving counter. Don't students have a hard time squeezing through there?
We find that some students don't fit easily into the serving line that we have created for our meals here. We find that students with a certain background need that extra guidance; any students who find that they don't fit well in our serving line are certainly free to return to the regular public cafeteria if that's what they think is best.
My buddy just tweeted from that cafeteria. He says that they've stopped serving desert and condiments because they're budget has been cut to fund you guys!
It is shameful how the public cafeteria is not giving all students access to an excellent high-performing meal.
Can I talk to somebody about this?
Our cafeteria manager is located in offices at this number. But they are two time zones away, so make sure you check the time before you call. I'm sure you'll be able to leave a message with their office staff.
But our cafeteria manager is right there. When we want to complain we just holler and she comes out to talk to us.
Oh, we don't allow any of that here. Any students who break any of our rules for decorum and proper obedience are subject to strong and immediate disciplinary action.
So let me get this straight. If I can manage to fill out this application for the lottery and I am lucky enough to be selected and I fit in your serving line and I don't get thrown out for acting uppity then I might get to eat something that sort of resembles steak on some days-- and I can never complain to the management. Otherwise, I just have to go eat at the public cafeteria where they have even less to offer because they also have to pay for everything you're doing over here.
Exactly. Because every student deserves a chance to eat steak.
But just a chance?
Well, sure. You didn't think anyone was going to spend the money to make sure that every single student actually got to eat steak, did you? We can't waste money trying to actually help all students. You get a chance, and a few students actually get steak, or at least something kind of like it. What more do you want?
Pages
▼
Monday, October 31, 2016
Sunday, October 30, 2016
PA: State Rep Compares School Boards to Hitler
Brad Roae is running for re-election for the PA House of Representatives for District 6. First elected in 2006, Roae has had some interesting things to say about education in Pennsylvania.
Hitler blamed the Jews for everything that was wrong with the world and school boards blame charter schools.
This was on Facebook, in response to a question about the currently-off-the-table HB 530, a bill that was supposed to provide big fat early Christmas presents to the charter school industry in PA.
Roae's district is just up the road from me and just down the road from Erie, where the schools have made some headlines with their economic issues, to the point that their board was seriously considering closing all of its high schools. Erie is one of several school districts that highlight the economic troubles of school districts in Pennsylvania. It's a complex mess, but the basic problems boil down to this.
First, Pennsylvania ranks 45th in the country for level of state support for local districts. That means the bulk of school district funding comes from local taxpayers, and that means that as cities like Erie with a previously-industrial tax base have lost those big employers, local revenue has gone into freefall, opening up some of the largest gaps between rich and poor districts in the country.
Second, Pennsylvania's legislature (the largest full-time legislature in the country, one of the most highly paid, and one of the most impressively gerrymandered) decided in the early 2000s that they would let local districts skimp on payments to the pension fund because, hey, those investments will grow the fund like wildfire anyway. Then Wall Street tanked the economy, and now local districts are looking at spectacularly ballooning pension payments on the order of payments equal to as much as one third of their total budget.
Oh, and a side note-- the legislature also periodically goes into spectacular failure mode about the budget. Back in 2015 districts across the state had to borrow huge chunks of money just to function, because Harrisburg couldn't get their job done.
Third, Pennsylvania is home to what our own Auditor General calls the worst charter laws in the country. There are many reasons for that judgment, but for local districts the most difficult part is that charter school students take 100% of their per-capita cost with them.
So Erie City Schools, despite some emergency funding from the state, will run up as much as a $10 million deficit this year, with a full quarter of their spending going to charter and pension costs. Meanwhile, the legislature is trying to phase in a new funding formula (or, one might say, its first actual funding formula). This is going to be a painful process because, to even things out, it will have to involve giving some cities a far bigger injection of state tax dollars than richer communities will get. Politicians face the choice of either explaining this process and making a case for fairness and justice, or they can just play to the crowd and decry Harrisburg "stealing our tax dollars to send to Those People." Place your bets now on which way that wind will blow.
Oh, and that formula is supposed to get straightened out over the next twenty years!!
Meanwhile, guys like Roae want to blame teachers and school districts. You can't give teachers raises and benefits. If Erie (and school districts like it) want state aid, then they should cut costs and stop blaming charter schools. Meanwhile, Roae has been lauded by the PA cyber industry as a "champion of school choice."
Roae, who graduated from Gannon in 1990 with a business degree and worked in the insurance biz until starting his legislative career, ought to know better.
When hospitals throughout Northwest PA wanted to cut costs, they didn't open more hospitals. If you are having trouble meeting your household budget, you do not open a second home and move part of your family into it.
Education seems to be the only field in which people suggest that when you don't have enough money to fund one facility, you should open more facilities. Charters are in fact a huge drain on public schools in the state. If my district serves 1,000 students and 100 leave for a charter school, my operating costs do not decrease by 10% even if my student population does. In fact, depending on which 100 students leave, my costs may not decrease at all. On top of that, I have to maintain capacity to handle those students because if some or all come back (and many of them do) I have to be able to accommodate them.
And while you can argue that losing students to charters may allow me to reduce the number of teachers in my school, in effect "moving" those jobs to the charter, the charter will still have to duplicate administrative costs.
Roae need only look at the schools all around his district to see schools that are cutting programs, closing buildings, jamming more students into classrooms, and offering the taxpayers of the district less in their public school system. Charter school costs aren't responsible for all of that, but they are certainly responsible for a lot of it, and that is doubly frustrating for school boards who, unlike Hitler, feel some responsibility for watching over the tax dollars that they were elected to spend wisely. And yet, unlike Hitler, they have no say at all over how those dollars are spent by the charters and, under Pennsylvania's lousy charter laws, nobody really has oversight once those public tax dollars go into private charter operator hands.
It's like board members get ten dollars from a taxpayer for lunch
Taxpayer: Couldn't you get us a better lunch with the money we gave you?
Board member: Well, the state said I had to give three dollars to that guy.
Taxpayer: Well, did he at least buy lunch with it.
Board member: I have no idea.
We could discuss the widespread fraud and scandal of Pennsylvania charter schools (if you hate the idea of your NWPA tax dollars going to Philly, you'll really hate what happens when they go to Philly charters), but that's really beside the point. If PA legislators think charters are such a good idea, they could come up with a funding system that didn't bleed the public system dry in order to get charters running.
Meanwhile, voters in District 6 might try voting against someone who thinks that after you rob Peter to pay Paul, you yell at Peter for not being thrifty enough to withstand the theft, and then compare him to Hitler.
Hitler blamed the Jews for everything that was wrong with the world and school boards blame charter schools.
This was on Facebook, in response to a question about the currently-off-the-table HB 530, a bill that was supposed to provide big fat early Christmas presents to the charter school industry in PA.
Roae's district is just up the road from me and just down the road from Erie, where the schools have made some headlines with their economic issues, to the point that their board was seriously considering closing all of its high schools. Erie is one of several school districts that highlight the economic troubles of school districts in Pennsylvania. It's a complex mess, but the basic problems boil down to this.
First, Pennsylvania ranks 45th in the country for level of state support for local districts. That means the bulk of school district funding comes from local taxpayers, and that means that as cities like Erie with a previously-industrial tax base have lost those big employers, local revenue has gone into freefall, opening up some of the largest gaps between rich and poor districts in the country.
Second, Pennsylvania's legislature (the largest full-time legislature in the country, one of the most highly paid, and one of the most impressively gerrymandered) decided in the early 2000s that they would let local districts skimp on payments to the pension fund because, hey, those investments will grow the fund like wildfire anyway. Then Wall Street tanked the economy, and now local districts are looking at spectacularly ballooning pension payments on the order of payments equal to as much as one third of their total budget.
Oh, and a side note-- the legislature also periodically goes into spectacular failure mode about the budget. Back in 2015 districts across the state had to borrow huge chunks of money just to function, because Harrisburg couldn't get their job done.
Third, Pennsylvania is home to what our own Auditor General calls the worst charter laws in the country. There are many reasons for that judgment, but for local districts the most difficult part is that charter school students take 100% of their per-capita cost with them.
So Erie City Schools, despite some emergency funding from the state, will run up as much as a $10 million deficit this year, with a full quarter of their spending going to charter and pension costs. Meanwhile, the legislature is trying to phase in a new funding formula (or, one might say, its first actual funding formula). This is going to be a painful process because, to even things out, it will have to involve giving some cities a far bigger injection of state tax dollars than richer communities will get. Politicians face the choice of either explaining this process and making a case for fairness and justice, or they can just play to the crowd and decry Harrisburg "stealing our tax dollars to send to Those People." Place your bets now on which way that wind will blow.
Oh, and that formula is supposed to get straightened out over the next twenty years!!
Meanwhile, guys like Roae want to blame teachers and school districts. You can't give teachers raises and benefits. If Erie (and school districts like it) want state aid, then they should cut costs and stop blaming charter schools. Meanwhile, Roae has been lauded by the PA cyber industry as a "champion of school choice."
Roae, who graduated from Gannon in 1990 with a business degree and worked in the insurance biz until starting his legislative career, ought to know better.
When hospitals throughout Northwest PA wanted to cut costs, they didn't open more hospitals. If you are having trouble meeting your household budget, you do not open a second home and move part of your family into it.
Education seems to be the only field in which people suggest that when you don't have enough money to fund one facility, you should open more facilities. Charters are in fact a huge drain on public schools in the state. If my district serves 1,000 students and 100 leave for a charter school, my operating costs do not decrease by 10% even if my student population does. In fact, depending on which 100 students leave, my costs may not decrease at all. On top of that, I have to maintain capacity to handle those students because if some or all come back (and many of them do) I have to be able to accommodate them.
And while you can argue that losing students to charters may allow me to reduce the number of teachers in my school, in effect "moving" those jobs to the charter, the charter will still have to duplicate administrative costs.
Roae need only look at the schools all around his district to see schools that are cutting programs, closing buildings, jamming more students into classrooms, and offering the taxpayers of the district less in their public school system. Charter school costs aren't responsible for all of that, but they are certainly responsible for a lot of it, and that is doubly frustrating for school boards who, unlike Hitler, feel some responsibility for watching over the tax dollars that they were elected to spend wisely. And yet, unlike Hitler, they have no say at all over how those dollars are spent by the charters and, under Pennsylvania's lousy charter laws, nobody really has oversight once those public tax dollars go into private charter operator hands.
It's like board members get ten dollars from a taxpayer for lunch
Taxpayer: Couldn't you get us a better lunch with the money we gave you?
Board member: Well, the state said I had to give three dollars to that guy.
Taxpayer: Well, did he at least buy lunch with it.
Board member: I have no idea.
We could discuss the widespread fraud and scandal of Pennsylvania charter schools (if you hate the idea of your NWPA tax dollars going to Philly, you'll really hate what happens when they go to Philly charters), but that's really beside the point. If PA legislators think charters are such a good idea, they could come up with a funding system that didn't bleed the public system dry in order to get charters running.
Meanwhile, voters in District 6 might try voting against someone who thinks that after you rob Peter to pay Paul, you yell at Peter for not being thrifty enough to withstand the theft, and then compare him to Hitler.
ICYMI: Catching Up with Reading (10/30)
I've been home for about a week and I am just about back up to speed. There's a lot to read this time around. As always, I encourage you to share wildly whatever you like here.
What Are the Main Reasons Teachers Call It Quits
NPR takes a look at why some folks are getting out of the teaching biz. No surprises here, but nice to see NPR catching on
LA Unified Takes a Hard Look at Charter Schools
Charters have taken it on the chin in LA, and there's a definite shift in attitude there.
A Public Education
Friend of this blog Phyllis Bush ran an op-ed this week that gets to the heart of what does and does not make a public education.
State-Run Kids: Suleika's Story
Here's a moving story of what the charter mess in New Jersey looks like to the families and children of the city.
Black Children Deserve the Stability That Neighborhood Schools Offer
Andre Perry absolutely nails it in discussing one of the worst effects of charter schools-- the loss of a stabilizing institutions for a community
What I Miss
Friend of this blog Mary Holden (it's nice to have all these friends) has been writing an honest and personal account of her departure from the classroom. Here's her look back at what she misses.
King of the Castle
Jennifer Berkshire (Edushyster) takes a look at the infamous Massachusetts charter that makes its teachers pay to leave.
Wall Street Firms Make Money from Pension Funds, Spend It On Charters
Actual reporter (you know-- the old fashioned type who actually goes out and finds thing out) David Sirota reports the maddening but predictable news that public teacher pension funds are helping fund the attack on public education.
The Vivisection of Literature
Another examination of how the study of literature has been beaten up in the rush to High Standards
The Absurd Defense of Standards Post-Common Core
Jane Robbins takes a quick look at how some folks are in a Kentucky spitting match over the Core
NAACP President: Why We Should Pause the Expansion of Charter Schools
Since they made the charteristas all sad, the NAACP has had lots of folks trying to tell them what happened, why it happened, and what they should really do. Here's the president of NAACP to explain what they did and why they did it.
What Are the Main Reasons Teachers Call It Quits
NPR takes a look at why some folks are getting out of the teaching biz. No surprises here, but nice to see NPR catching on
LA Unified Takes a Hard Look at Charter Schools
Charters have taken it on the chin in LA, and there's a definite shift in attitude there.
A Public Education
Friend of this blog Phyllis Bush ran an op-ed this week that gets to the heart of what does and does not make a public education.
State-Run Kids: Suleika's Story
Here's a moving story of what the charter mess in New Jersey looks like to the families and children of the city.
Black Children Deserve the Stability That Neighborhood Schools Offer
Andre Perry absolutely nails it in discussing one of the worst effects of charter schools-- the loss of a stabilizing institutions for a community
What I Miss
Friend of this blog Mary Holden (it's nice to have all these friends) has been writing an honest and personal account of her departure from the classroom. Here's her look back at what she misses.
King of the Castle
Jennifer Berkshire (Edushyster) takes a look at the infamous Massachusetts charter that makes its teachers pay to leave.
Wall Street Firms Make Money from Pension Funds, Spend It On Charters
Actual reporter (you know-- the old fashioned type who actually goes out and finds thing out) David Sirota reports the maddening but predictable news that public teacher pension funds are helping fund the attack on public education.
The Vivisection of Literature
Another examination of how the study of literature has been beaten up in the rush to High Standards
The Absurd Defense of Standards Post-Common Core
Jane Robbins takes a quick look at how some folks are in a Kentucky spitting match over the Core
NAACP President: Why We Should Pause the Expansion of Charter Schools
Since they made the charteristas all sad, the NAACP has had lots of folks trying to tell them what happened, why it happened, and what they should really do. Here's the president of NAACP to explain what they did and why they did it.
Saturday, October 29, 2016
How (Not) To Grade Schools
Bellwether Education Partners is a right-tilted thinky tank from the same basic neighborhood as the Fordham Institute. Chad Aldeman is one of their big guns, and this month he's out with Grading Schools: How States Should Define “School Quality” Under the Every Student Succeeds Act. It's a pretty thing with thirty-two pages of thoughts about how to implement school accountability under ESSA, and I've read the whole thing so that you don't have to. Let's take a look under the hood.
Introduction
Aldeman offers a few thoughts to start that give a hint about where he might be headed. School evaluation has been too rigid and rule-bound. We've focused too much on student test scores instead of student growth. But the window is now open for a "new conversation," which kind of presumes that there was an old conversation, and I suppose for people in the thinky tank world it might seem as if there were a conversation, but from out here the actual education field, school accountability has been imposed from the top down with deliberate efforts to silence any attempts at conversation.
In other words, the news that school accountability has been too rigid and rules-bound is only news to people who have steadfastly ignored the voices of actual teachers, who called that one from the very first moment that No Child Left Behind raised its rigid, inflexible, and not-very-smart head.
So to have this "new conversation," policy folks should brace themselves for a certain amount of "Told you so" or "No kidding" or even "No shit, Sherlock." Or alternately, as this new conversation is probably going to resemble the old one insofar as actual teacher voices will be once again excluded, something along the lines of, "Remember what happened the last time you ignored us?"
What Is Accountability and Why Does It Matter?
Alderman acknowledges that accountability covers a wide range of functions, from transparency for the general public on one end to rewards and punishments by government on the other end. He posits that somewhere in the middle that "accountability can act as a tool for improvement through goal-setting, performance benchmarking, and re-evaluation." And he also notes that accountability measures are state government's way of signalling what it values.
So accountability can be very many things. Who is it for?
Well, teachers and school leaders, who are supposed to be able to use the data to do a better job. And parents, too. And also the political leaders who are responsible for the oversight of public tax dollars. And on top of that, ESSA requires states to grade schools in order to stack rank and target some for some manner of fixing, including targeting the bottom five percent.
Aldfeman barrels on, pretending that meeting that last set of ESSA mandated stack-ranking, school-grading requirements will meet all the various versions of accountability that he has listed. He suggests in passing that we're really talking about different degrees of transparency for different groups of accountability viewers, but that's not really true either.
Neither Aldeman or, for that matter, the feds have seriously or realistically addressed the problems that come when you try to create an instrument that measures all things for all audiences. This is bananas, and it's why the entire accountability system continues to be built on a foundation of sand and silly putty. The instrument that tells a parent how their child is doing is not the same as the instrument that tells a teacher how to tweak instruction, and neither is the same as the instrument that tells the state and federal government if the school is doing a good job, and none of those are the same as an instrument used to stack ran all the schools in the state (and, it should also be noted, none of those functions are best done by a Big Standardized Test, and yet policymakers seem unable to let go of the assumption that the BS Tests are good for anything).
It's like weighing the entrees at a restaurant as a way of determining customer satisfaction, chef quality and efficiency, how well the restaurant is managed, compliance with health code regulations, reviews for the AAA guide, and the stability of the building in which the restaurant is housed. It's simply nuts.
Aldeman cites assorted research that is all based on the assumption that narrow poorly-written standardized math and reading tests are actually measuring something useful. They are not. Virtually all of the data generated by these tests is junk, and as their use becomes more widespread and students become more weary of them, the data becomes junkier and junkier.
Bottom line-- real accountability requires a wide range of instruments for a wide range of audiences, and we have not remotely solved that challenge. Not, let me note, that it isn't a challenge worth solving. But as long as we base the whole system on the BS Tests, we will not be remotely in the right neighborhood.
How Should States Select Accountability Measures
Again, Aldeman is working from some bad assumptions about what the system is for. Can you spot the key word in this sentence?
The trick, then, is to design accountability systems in which schools are competing on measures that truly matter
A competition system is not a measuring system. If I tell you that Chris is the tallest kid in class and Pat is the shortest, you still have no idea of Chris's or Pat's actual height.
Aldeman gets his next point right-- an accountability system should be simple, clear and fair. Well, partly right. His idea of "fair" is that the system only measures things that schools actually have control over. So he's skipped one other key attribute-- the accountability system needs to be accurate and measure what it actually says it measures. So, for instance, we should stop saying "student achievement" when we actually mean "student score on a single narrow standardized math and reading test that has never really passed tests for validity and reliability."
Aldeman notes the four required elements per ESSA:
1) "Achievement rates" aka "test scores."
2) Some other "valid and reliable" academic indicator. The word "other" assumes facts not in evidence.
3) Progress in achieving English language proficiency
4) Some other indicator of school quality or success
Aldeman offers a chart in which some possible elements are judged against qualities like simplicity, fairness, disagregatability, and giving guidance to the school. So measuring grit or other personal qualities is iffy because measuring and teaching it are iffy. Teacher and student surveys get a thumbs up for measuring stuff, but thumbs down for being actionable, though I think a good student or staff survey would provide a school with very specific issues to address.
Aldeman says to avoid redundant measures and reminds us that ESSA doesn't put a maximum limit on measures to be used.
How Can States Design School Ratings Systems That Are Simple, Clear, and Fair?
A fake subheading that simply covers an introduction that says, "And now I will tell you how." It does include a fun sidebar about how K-2 should be included in the accountability system. Aldeman notes that leaving them out previously was because of things like the unsolved challenge of how to assess the littles; he does not offer any new insights about that issue that have turned up since NCLB days, and in fact subjecting the littles to any kind of formal or standardized assessment is a truly, deeply indefensible policy notion, and serves as nothing more than a clear-cut example of putting the desires of policy-makers and data-grubbers over the needs of small children.
Incorporating Student Achievement
Of course, by "student achievement," we just mean "test scores." Aldeman recommends we start out with a simple performance scale index for points. He suggests five performance levels, with emphasis on proficiency because "proficiency is, after all, a benchmark for future success in college and careers." Which-- no, no it's not. There isn't an iota of data anywhere to connect a proficiency level on the BS Tests with college and career success, particularly because the proficiency rating is a normed ranking, so it moves every year depending on the mass of scores and the cut scores set annually by state testocrats.
So we're talking about using the test scores, which are junk, after they have been run through a normed scale, which adds more junk.
Using Growth as the "Other" Academic Indicator
Aldeman pays tribute to the "growth mindset" as a worthy stance for schools, though we are once again talking only about growth as it applies to standardized test scores. If the student grew in some other way, nobody cares.
The problem with coming up with a measure of student growth is, of course, that nobody has successfully done it yet. Aldeman mentions several models.
* Without using the words "value-added," Aldeman nods to the model that uses obtuse, opaque, and unproven mumbo-jumbo to make the claim that student performance can be statistically stripped from other characteristics. Aldeman suggests this is disqualified because it is neither simple nor understandable; he might also mention that it is baloney that has been debunked by all manner of authorities.
* Aldeman mentions the student percentiles model, a stack-ranking competitive model that compares a student's test score to the score of other students who had a similar score last year. Like all such normed models, this one involves goal posts that move every year, and like all percentile-based models, it guarantees the exact same distribution year after year. No amount of school quality will raise all students to the top 25%.
* Aldeman favors a transitional matrix, judging schools on how many students move from one group to another (say, below basic to basic). This is also a bad idea. Aldeman has elsewhere shown sensitivity to the unintended consequences of some of these policy choices, so I'm not sure how he misses the obvious implications here. A school's best strategy will be to invest its energy on students who are near a threshhold and not those for whom there's no real hope of enough improvement.
Creating an Overall Index and Incorporating Subgroup Results
Aldeman wants to use the two indicators we've got so far and average them for an overall index, and this is the score by which we'll "flag" the bottom 5%. These indexes would also be computed for subgroups so that schools can also be flagged for failing to close their achievement gaps.
To be clear, this approach assumes that identifying schools for improvement is an important lever at the state’s disposal. That’s intentional, because there are positive effects associated with the mere act of notifying schools that they need to improve. That’s especially true for accountability systems bearing consequences for schools, but it’s even true in systems relying purely on information and transparency.
In other words, threats work. At least, they work on raising test scores (and he's got some research from reformster research rock star Eric Hanushek to back it up). This is a deeply irresponsible policy idea, ignoring completely the question of what schools give up and get rid of in order to raise their test scores. Cutting recess, phys ed, art, music, etc. In my own district I have seen schools strip student schedules so that middle school students with low test scores spent their entire day in English and math class, with no history, art, science or other non-tested subjects.
This is the test-centered school at its worst. This is a lousy idea.
Incorporating Other Measures of School Success Into Final School Ratings
Here Aldeman brings out the English model of school inspections, in which trained and experienced educators visit the school for an extended inspection, both detailed and holistic, of how the school works, how well it ticks, how well it serves students, and how well it matches the notion of what a good school should be.
This is a good idea.
Though I can imagine that for schools that have been "flagged" because of test scores, the inspection visit might be a bit harrowing.
I would offer one editing suggestion to Aldeman for his system. Keep the school inspection system and get rid of everything else.
Yes, yes, ESSA has kept us beholden to the BS Testing system. But any sensible, realistic, useful accountability system is going to shrink the use of the BS Test down to the absolute minimum the feds will let the state get away with. Making the test scores the foundation of the rest of the accountability is the absolute wrong way to go.
Conclusion
Aldeman notes that ESSA somehow focuses less attention on punishing "failing" schools than on actually helping them, which, maybe, depending on how you read it. It would be worth it for the feds and states to back away from that, since they have shown absolutely no aptitude for turning around failing schools.
There is one other huge hole in Aldeman's plan, and that is the space where we should find the voice of the community in which the school is located. He has dodged one of the big accountability questions, which is this-- if the community in which a school is located is happy with their school, exactly what reason is there for the state and federal bureaucrats to get involved? I remain puzzled that the right-leaning policy folks continue to remain uninterested in local control of schools.
Introduction
Aldeman offers a few thoughts to start that give a hint about where he might be headed. School evaluation has been too rigid and rule-bound. We've focused too much on student test scores instead of student growth. But the window is now open for a "new conversation," which kind of presumes that there was an old conversation, and I suppose for people in the thinky tank world it might seem as if there were a conversation, but from out here the actual education field, school accountability has been imposed from the top down with deliberate efforts to silence any attempts at conversation.
In other words, the news that school accountability has been too rigid and rules-bound is only news to people who have steadfastly ignored the voices of actual teachers, who called that one from the very first moment that No Child Left Behind raised its rigid, inflexible, and not-very-smart head.
So to have this "new conversation," policy folks should brace themselves for a certain amount of "Told you so" or "No kidding" or even "No shit, Sherlock." Or alternately, as this new conversation is probably going to resemble the old one insofar as actual teacher voices will be once again excluded, something along the lines of, "Remember what happened the last time you ignored us?"
What Is Accountability and Why Does It Matter?
Alderman acknowledges that accountability covers a wide range of functions, from transparency for the general public on one end to rewards and punishments by government on the other end. He posits that somewhere in the middle that "accountability can act as a tool for improvement through goal-setting, performance benchmarking, and re-evaluation." And he also notes that accountability measures are state government's way of signalling what it values.
So accountability can be very many things. Who is it for?
Well, teachers and school leaders, who are supposed to be able to use the data to do a better job. And parents, too. And also the political leaders who are responsible for the oversight of public tax dollars. And on top of that, ESSA requires states to grade schools in order to stack rank and target some for some manner of fixing, including targeting the bottom five percent.
Aldfeman barrels on, pretending that meeting that last set of ESSA mandated stack-ranking, school-grading requirements will meet all the various versions of accountability that he has listed. He suggests in passing that we're really talking about different degrees of transparency for different groups of accountability viewers, but that's not really true either.
Neither Aldeman or, for that matter, the feds have seriously or realistically addressed the problems that come when you try to create an instrument that measures all things for all audiences. This is bananas, and it's why the entire accountability system continues to be built on a foundation of sand and silly putty. The instrument that tells a parent how their child is doing is not the same as the instrument that tells a teacher how to tweak instruction, and neither is the same as the instrument that tells the state and federal government if the school is doing a good job, and none of those are the same as an instrument used to stack ran all the schools in the state (and, it should also be noted, none of those functions are best done by a Big Standardized Test, and yet policymakers seem unable to let go of the assumption that the BS Tests are good for anything).
It's like weighing the entrees at a restaurant as a way of determining customer satisfaction, chef quality and efficiency, how well the restaurant is managed, compliance with health code regulations, reviews for the AAA guide, and the stability of the building in which the restaurant is housed. It's simply nuts.
Aldeman cites assorted research that is all based on the assumption that narrow poorly-written standardized math and reading tests are actually measuring something useful. They are not. Virtually all of the data generated by these tests is junk, and as their use becomes more widespread and students become more weary of them, the data becomes junkier and junkier.
Bottom line-- real accountability requires a wide range of instruments for a wide range of audiences, and we have not remotely solved that challenge. Not, let me note, that it isn't a challenge worth solving. But as long as we base the whole system on the BS Tests, we will not be remotely in the right neighborhood.
How Should States Select Accountability Measures
Again, Aldeman is working from some bad assumptions about what the system is for. Can you spot the key word in this sentence?
The trick, then, is to design accountability systems in which schools are competing on measures that truly matter
A competition system is not a measuring system. If I tell you that Chris is the tallest kid in class and Pat is the shortest, you still have no idea of Chris's or Pat's actual height.
Aldeman gets his next point right-- an accountability system should be simple, clear and fair. Well, partly right. His idea of "fair" is that the system only measures things that schools actually have control over. So he's skipped one other key attribute-- the accountability system needs to be accurate and measure what it actually says it measures. So, for instance, we should stop saying "student achievement" when we actually mean "student score on a single narrow standardized math and reading test that has never really passed tests for validity and reliability."
Aldeman notes the four required elements per ESSA:
1) "Achievement rates" aka "test scores."
2) Some other "valid and reliable" academic indicator. The word "other" assumes facts not in evidence.
3) Progress in achieving English language proficiency
4) Some other indicator of school quality or success
Aldeman offers a chart in which some possible elements are judged against qualities like simplicity, fairness, disagregatability, and giving guidance to the school. So measuring grit or other personal qualities is iffy because measuring and teaching it are iffy. Teacher and student surveys get a thumbs up for measuring stuff, but thumbs down for being actionable, though I think a good student or staff survey would provide a school with very specific issues to address.
Aldeman says to avoid redundant measures and reminds us that ESSA doesn't put a maximum limit on measures to be used.
How Can States Design School Ratings Systems That Are Simple, Clear, and Fair?
A fake subheading that simply covers an introduction that says, "And now I will tell you how." It does include a fun sidebar about how K-2 should be included in the accountability system. Aldeman notes that leaving them out previously was because of things like the unsolved challenge of how to assess the littles; he does not offer any new insights about that issue that have turned up since NCLB days, and in fact subjecting the littles to any kind of formal or standardized assessment is a truly, deeply indefensible policy notion, and serves as nothing more than a clear-cut example of putting the desires of policy-makers and data-grubbers over the needs of small children.
Incorporating Student Achievement
Of course, by "student achievement," we just mean "test scores." Aldeman recommends we start out with a simple performance scale index for points. He suggests five performance levels, with emphasis on proficiency because "proficiency is, after all, a benchmark for future success in college and careers." Which-- no, no it's not. There isn't an iota of data anywhere to connect a proficiency level on the BS Tests with college and career success, particularly because the proficiency rating is a normed ranking, so it moves every year depending on the mass of scores and the cut scores set annually by state testocrats.
So we're talking about using the test scores, which are junk, after they have been run through a normed scale, which adds more junk.
Using Growth as the "Other" Academic Indicator
Aldeman pays tribute to the "growth mindset" as a worthy stance for schools, though we are once again talking only about growth as it applies to standardized test scores. If the student grew in some other way, nobody cares.
The problem with coming up with a measure of student growth is, of course, that nobody has successfully done it yet. Aldeman mentions several models.
* Without using the words "value-added," Aldeman nods to the model that uses obtuse, opaque, and unproven mumbo-jumbo to make the claim that student performance can be statistically stripped from other characteristics. Aldeman suggests this is disqualified because it is neither simple nor understandable; he might also mention that it is baloney that has been debunked by all manner of authorities.
* Aldeman mentions the student percentiles model, a stack-ranking competitive model that compares a student's test score to the score of other students who had a similar score last year. Like all such normed models, this one involves goal posts that move every year, and like all percentile-based models, it guarantees the exact same distribution year after year. No amount of school quality will raise all students to the top 25%.
* Aldeman favors a transitional matrix, judging schools on how many students move from one group to another (say, below basic to basic). This is also a bad idea. Aldeman has elsewhere shown sensitivity to the unintended consequences of some of these policy choices, so I'm not sure how he misses the obvious implications here. A school's best strategy will be to invest its energy on students who are near a threshhold and not those for whom there's no real hope of enough improvement.
Creating an Overall Index and Incorporating Subgroup Results
Aldeman wants to use the two indicators we've got so far and average them for an overall index, and this is the score by which we'll "flag" the bottom 5%. These indexes would also be computed for subgroups so that schools can also be flagged for failing to close their achievement gaps.
To be clear, this approach assumes that identifying schools for improvement is an important lever at the state’s disposal. That’s intentional, because there are positive effects associated with the mere act of notifying schools that they need to improve. That’s especially true for accountability systems bearing consequences for schools, but it’s even true in systems relying purely on information and transparency.
In other words, threats work. At least, they work on raising test scores (and he's got some research from reformster research rock star Eric Hanushek to back it up). This is a deeply irresponsible policy idea, ignoring completely the question of what schools give up and get rid of in order to raise their test scores. Cutting recess, phys ed, art, music, etc. In my own district I have seen schools strip student schedules so that middle school students with low test scores spent their entire day in English and math class, with no history, art, science or other non-tested subjects.
This is the test-centered school at its worst. This is a lousy idea.
Incorporating Other Measures of School Success Into Final School Ratings
Here Aldeman brings out the English model of school inspections, in which trained and experienced educators visit the school for an extended inspection, both detailed and holistic, of how the school works, how well it ticks, how well it serves students, and how well it matches the notion of what a good school should be.
This is a good idea.
Though I can imagine that for schools that have been "flagged" because of test scores, the inspection visit might be a bit harrowing.
I would offer one editing suggestion to Aldeman for his system. Keep the school inspection system and get rid of everything else.
Yes, yes, ESSA has kept us beholden to the BS Testing system. But any sensible, realistic, useful accountability system is going to shrink the use of the BS Test down to the absolute minimum the feds will let the state get away with. Making the test scores the foundation of the rest of the accountability is the absolute wrong way to go.
Conclusion
Aldeman notes that ESSA somehow focuses less attention on punishing "failing" schools than on actually helping them, which, maybe, depending on how you read it. It would be worth it for the feds and states to back away from that, since they have shown absolutely no aptitude for turning around failing schools.
There is one other huge hole in Aldeman's plan, and that is the space where we should find the voice of the community in which the school is located. He has dodged one of the big accountability questions, which is this-- if the community in which a school is located is happy with their school, exactly what reason is there for the state and federal bureaucrats to get involved? I remain puzzled that the right-leaning policy folks continue to remain uninterested in local control of schools.
Friday, October 28, 2016
GA: Ed Consultant Slams Takeover Amendment
In Georgia, reformsters are pushing hard for Amendment 1, a constitutional amendment that would institute a state-level takeover district, modeled after the pioneering Achievement School District in Tennessee.
Dr. David K. Lerch is a Georgia resident and ran his own educational consulting firm for over three decades. He has worked all over the country, writing grants and overseeing programs (e.g. Pueblo hired him to evaluate their STEM programs).
Lerch has presumably seen plenty in the ed field; he earned his Master's Degree in Public School Administration from the University of Virginia back in 1967. By 1984 he was forming the National Association of Magnet School Development and was touting magnets as a path to desegregation and what we now call educational equity. He was also saying the kinds of things that charter fans would chime in on decades later:
Parents want neighborhood schools until they find a program they support and then they will send a child halfway across the county if the education program is attractive.
Lerch now works for the Juliana Group, Inc, a Savannah-based business that specializes in selling furniture for Montessori schools.
In short, Lerch is not a long-time hard-core supporter for traditional public education. However, when a letter-writer to the Savannah Morning News wrote to warn against Amendment 1, Lerch felt moved to back her up.
I can add some first-hand experiences validating her timely concern about what will happen with the loss of local control of schools and the resulting loss of millions of state and federal revenue.
I served as a consultant to school districts in two states, Louisiana and Michigan, where the governors set up takeover districts identical to that proposed by Gov. Nathan Deal.
The legislative amendment in Louisiana’s constitution (Recovery School District) provided for the same type of state control. While I was working with East Baton Rouge Parish School District, the state took over Istrouma High School, operated it for five years and returned it to the district without students showing any measurable academic success.
Then the school board had to spend over a $21 million of local funds to repair the facility.
I also worked with Michigan’s Education Achievement Authority (EAA), which was set up by Governor Snyder as a model of Louisiana’s Recovery School District.
I helped them obtain a $35 million federal grant for teacher training and support in 15 of 60 schools that were scheduled for operation by EAA. After only four years of state control, and massive evidence of EAA’s failure cited by education experts and the federal government monitoring their grant, Governor Snyder decided to shut down the agency and turn EAA’s schools into charter schools.
Those two failures are, of course, on top of the failure of Tennessee's ASD. (see here, here and here). And just in case you have doubts:
Why anyone would duplicate a state controlled takeover district that has proven to be a failure in two states is beyond belief. If you don’t believe the controversy caused by the takeover districts similar to OSD, read the February 2016 document “State Takeover of Low-Performing Schools – A Record of Academic Failure, Financial Mismanagement & Student Harm.”
It is available on the Internet and will shake you to the core about Amendment 1.
He's correct. That report is available on the internet, and it is yet more evidence that state-run takeover turnaround districts have failed-- and not just marginally, but spectacularly and totally-- every time they have been attempted. Georgia has ample evidence and ample warning. Here's hoping that Georgia voters get the message.
Dr. David K. Lerch is a Georgia resident and ran his own educational consulting firm for over three decades. He has worked all over the country, writing grants and overseeing programs (e.g. Pueblo hired him to evaluate their STEM programs).
Lerch has presumably seen plenty in the ed field; he earned his Master's Degree in Public School Administration from the University of Virginia back in 1967. By 1984 he was forming the National Association of Magnet School Development and was touting magnets as a path to desegregation and what we now call educational equity. He was also saying the kinds of things that charter fans would chime in on decades later:
Parents want neighborhood schools until they find a program they support and then they will send a child halfway across the county if the education program is attractive.
Lerch now works for the Juliana Group, Inc, a Savannah-based business that specializes in selling furniture for Montessori schools.
In short, Lerch is not a long-time hard-core supporter for traditional public education. However, when a letter-writer to the Savannah Morning News wrote to warn against Amendment 1, Lerch felt moved to back her up.
I can add some first-hand experiences validating her timely concern about what will happen with the loss of local control of schools and the resulting loss of millions of state and federal revenue.
I served as a consultant to school districts in two states, Louisiana and Michigan, where the governors set up takeover districts identical to that proposed by Gov. Nathan Deal.
The legislative amendment in Louisiana’s constitution (Recovery School District) provided for the same type of state control. While I was working with East Baton Rouge Parish School District, the state took over Istrouma High School, operated it for five years and returned it to the district without students showing any measurable academic success.
Then the school board had to spend over a $21 million of local funds to repair the facility.
I also worked with Michigan’s Education Achievement Authority (EAA), which was set up by Governor Snyder as a model of Louisiana’s Recovery School District.
I helped them obtain a $35 million federal grant for teacher training and support in 15 of 60 schools that were scheduled for operation by EAA. After only four years of state control, and massive evidence of EAA’s failure cited by education experts and the federal government monitoring their grant, Governor Snyder decided to shut down the agency and turn EAA’s schools into charter schools.
Those two failures are, of course, on top of the failure of Tennessee's ASD. (see here, here and here). And just in case you have doubts:
Why anyone would duplicate a state controlled takeover district that has proven to be a failure in two states is beyond belief. If you don’t believe the controversy caused by the takeover districts similar to OSD, read the February 2016 document “State Takeover of Low-Performing Schools – A Record of Academic Failure, Financial Mismanagement & Student Harm.”
It is available on the Internet and will shake you to the core about Amendment 1.
He's correct. That report is available on the internet, and it is yet more evidence that state-run takeover turnaround districts have failed-- and not just marginally, but spectacularly and totally-- every time they have been attempted. Georgia has ample evidence and ample warning. Here's hoping that Georgia voters get the message.
CA: Is the Fox Guarding the Henhouse?
The Los Angeles Unified School District put away their charter rubber stamp, and it has touched off a wave of hand wringing and baloney shoveling.
Earlier this month, the LAUSD board pulled the plug on five charters. Three of them were Magnolia schools, part of the Gulen charter web of schools allegedly tied to the reclusive cleric who is also an exiled political leader from Turkey allegedly tied to this year's coup attempt. The Magnolia chain has been accused of significant financial shenanigans, The other two were Celerity schools, a chain that has such a spotted record that even reformy John Deasy has cast a wary eye in their direction. Oversight and transparency, two important qualities that charter schools generally do very badly, were cited as issues with the five.
But the unexpected move by the board to hold any charters accountable for anything ever has stirred some folks up.
Here's a charter-friendly look at the "issue" from KPCC, the Southern California Public Radio station, that opens with the exactly wrong question:
Is the Los Angeles Unified School District able to give a fair shake to the charter schools it authorizes and oversees even though the district loses money every time a student leaves to attend a charter?
And follows it up with this mis-statement of the issue:
On Tuesday, board members addressed the underlying concern the California Charter Schools Association and others have raised in the wake of their vote: that letting L.A. Unified review such requests from charter schools — especially in an environment where the district and charters compete for funding — is letting the fox guard the henhouse.
Emphasis mine. Because I wouldn't frame the situation by suggesting that the school board is somehow out to steal money it's not entitled to.
Instead of "letting the fox guard te henhouse," let's say "requiring the elected representatives of the taxpayers to oversee how those taxpayers' dollars are used."
Some members of the board expressed frustration that the California system allows unhappy charters to next ask the county to authorize them. Board member Richard Vladovic noted that the district would save a lot of money if charters were authorized and supervised by the state. He neglected to add having the charters financed by the state as well.
Other board members clearly get the backwardness of the system:
Charter school petitioners “who are turned down will always have a complaint,” said school board vice president George McKenna. “Their opinion will always be that they were wronged, that we weren’t fair, that the burden is on [L.A. Unified] to prove their guilt, not on them to prove their innocence.”
Yes, in California (and several other states), we've got a system in which charters feel entitled to open and stay open, drawing on public tax dollars as long as they're inclined.
There really isn't anything like this. If I want to pave the driveway to my private business, I can't demand state highway tax dollars to finance the driveway and expect to get those dollars unless someone can prove I've done something really terrible. If I want to start my own private security force, I can't bill the Department of Defense and expect them to shoulder the burden of proving why I shouldn't be paid public tax dollars.
But somehow California charters feel entitled to public tax dollars and will hold onto them until someone can pry the pursestrings out of their chartery fingers. This is not the fox guarding the henhouse; this the fox moving into the henhouse and getting indignant when the farmer shows up with an eviction notice.
Earlier this month, the LAUSD board pulled the plug on five charters. Three of them were Magnolia schools, part of the Gulen charter web of schools allegedly tied to the reclusive cleric who is also an exiled political leader from Turkey allegedly tied to this year's coup attempt. The Magnolia chain has been accused of significant financial shenanigans, The other two were Celerity schools, a chain that has such a spotted record that even reformy John Deasy has cast a wary eye in their direction. Oversight and transparency, two important qualities that charter schools generally do very badly, were cited as issues with the five.
But the unexpected move by the board to hold any charters accountable for anything ever has stirred some folks up.
Here's a charter-friendly look at the "issue" from KPCC, the Southern California Public Radio station, that opens with the exactly wrong question:
Is the Los Angeles Unified School District able to give a fair shake to the charter schools it authorizes and oversees even though the district loses money every time a student leaves to attend a charter?
And follows it up with this mis-statement of the issue:
On Tuesday, board members addressed the underlying concern the California Charter Schools Association and others have raised in the wake of their vote: that letting L.A. Unified review such requests from charter schools — especially in an environment where the district and charters compete for funding — is letting the fox guard the henhouse.
Emphasis mine. Because I wouldn't frame the situation by suggesting that the school board is somehow out to steal money it's not entitled to.
Instead of "letting the fox guard te henhouse," let's say "requiring the elected representatives of the taxpayers to oversee how those taxpayers' dollars are used."
Some members of the board expressed frustration that the California system allows unhappy charters to next ask the county to authorize them. Board member Richard Vladovic noted that the district would save a lot of money if charters were authorized and supervised by the state. He neglected to add having the charters financed by the state as well.
Other board members clearly get the backwardness of the system:
Charter school petitioners “who are turned down will always have a complaint,” said school board vice president George McKenna. “Their opinion will always be that they were wronged, that we weren’t fair, that the burden is on [L.A. Unified] to prove their guilt, not on them to prove their innocence.”
Yes, in California (and several other states), we've got a system in which charters feel entitled to open and stay open, drawing on public tax dollars as long as they're inclined.
There really isn't anything like this. If I want to pave the driveway to my private business, I can't demand state highway tax dollars to finance the driveway and expect to get those dollars unless someone can prove I've done something really terrible. If I want to start my own private security force, I can't bill the Department of Defense and expect them to shoulder the burden of proving why I shouldn't be paid public tax dollars.
But somehow California charters feel entitled to public tax dollars and will hold onto them until someone can pry the pursestrings out of their chartery fingers. This is not the fox guarding the henhouse; this the fox moving into the henhouse and getting indignant when the farmer shows up with an eviction notice.
Thursday, October 27, 2016
Reflect now. Now!! NOW!!!
One of the fully screwed-up features of modern standardized assessments is the time frame.
A standardized test is the only place where students are told, "Starting from scratch, read this, reflect on it, answer questions about it, and do it all in the next fifteen minutes." We accept the accelerated time line as a normal feature of assessment, but why?
Never ever in a college course was a student handed a book for the first time and told, "Read this book and write an intelligent, thoughtful paper about the text. Hand it in sixty minutes from now."
Reflective, thoughtful, deep, even close reading, the kind of reading that reformsters insist they want, takes time. The text has to be read and considered carefully. Theories about the ideas, the themes, the characters, the author's use of language, the thoughtful consideration of the various elements of the writing-- those all need time to percolate, to simmer, to be mulled by the reader. Those of us who teach literature and reading in high school never have to tell our students, "Hurry up and zip through that faster." Most commonly we have to find ways to encourage our students to slow down, pay attention, really think about what they're reading instead of trying to race to the end.
A reader's relationship with a text, like any good relationship, takes time. It may start with a certain slow grudging acquaintance of necessity, or it may start with an instant spark of attraction, but either way, if the relationship is going to have any depth or quality, time and care will have to be invested. Standardized tests are the "hit it and quit it" of the reading world.
The reasons that we test this way are obvious. Test manufacturers want a short, closed test period so that no test items can "leak," though, of course, some of the best reflection on reading comes through discussion and sharing. English teachers have adopted reading circles for a reason. Test manufacturers also want to keep the testing experience uniform, which means a relatively short, set time (the longer the test lasts, the more variables creep in). But it's important to note that none of the reasons that we test this way have anything to do with more effectively testing the skills we say we want to test.
There's a whole other discussion to be had about trying to treat reading skills as discrete abilities that exist and can be measured in a vacuum without any concern about the content being read. They can't, but even if they could, none of the skills we say we want in readers are tested by the instant quicky test method. We say we want critical thinking, deep reading, and reflection beyond simple recall and fact-spitting, but none of that fits with the cold-reading and instant analysis method used in tests. We test as if we want to train students to cold read and draw conclusions quickly, in an isolated brief period.
This is nuts. It is a skill set that pretty much nobody is looking for, an ability favored by no-one, and yet, it is a fundamental part of the Big Standardized Test. No-- I take that back. This is a set of skills that is useful if you want to train a bunch of people to read and follow directions quickly and compliantly. That's about it.
Real reading takes time. Real reflection takes time. Both are best served by a rich environment that includes other thoughtful readers and resources to enrich the experience. To write any sort of thoughtful, deep, or thorough reflection on that reading also takes time.
If policymakers were serious about building critical thinking, deep reading skills, and thoughtful responses to the text, they would not consider BS Tests like the PARCC for even five minutes. It is one more area where stated intent and actual actions are completely out of alignment.
A standardized test is the only place where students are told, "Starting from scratch, read this, reflect on it, answer questions about it, and do it all in the next fifteen minutes." We accept the accelerated time line as a normal feature of assessment, but why?
Never ever in a college course was a student handed a book for the first time and told, "Read this book and write an intelligent, thoughtful paper about the text. Hand it in sixty minutes from now."
Reflective, thoughtful, deep, even close reading, the kind of reading that reformsters insist they want, takes time. The text has to be read and considered carefully. Theories about the ideas, the themes, the characters, the author's use of language, the thoughtful consideration of the various elements of the writing-- those all need time to percolate, to simmer, to be mulled by the reader. Those of us who teach literature and reading in high school never have to tell our students, "Hurry up and zip through that faster." Most commonly we have to find ways to encourage our students to slow down, pay attention, really think about what they're reading instead of trying to race to the end.
A reader's relationship with a text, like any good relationship, takes time. It may start with a certain slow grudging acquaintance of necessity, or it may start with an instant spark of attraction, but either way, if the relationship is going to have any depth or quality, time and care will have to be invested. Standardized tests are the "hit it and quit it" of the reading world.
The reasons that we test this way are obvious. Test manufacturers want a short, closed test period so that no test items can "leak," though, of course, some of the best reflection on reading comes through discussion and sharing. English teachers have adopted reading circles for a reason. Test manufacturers also want to keep the testing experience uniform, which means a relatively short, set time (the longer the test lasts, the more variables creep in). But it's important to note that none of the reasons that we test this way have anything to do with more effectively testing the skills we say we want to test.
There's a whole other discussion to be had about trying to treat reading skills as discrete abilities that exist and can be measured in a vacuum without any concern about the content being read. They can't, but even if they could, none of the skills we say we want in readers are tested by the instant quicky test method. We say we want critical thinking, deep reading, and reflection beyond simple recall and fact-spitting, but none of that fits with the cold-reading and instant analysis method used in tests. We test as if we want to train students to cold read and draw conclusions quickly, in an isolated brief period.
This is nuts. It is a skill set that pretty much nobody is looking for, an ability favored by no-one, and yet, it is a fundamental part of the Big Standardized Test. No-- I take that back. This is a set of skills that is useful if you want to train a bunch of people to read and follow directions quickly and compliantly. That's about it.
Real reading takes time. Real reflection takes time. Both are best served by a rich environment that includes other thoughtful readers and resources to enrich the experience. To write any sort of thoughtful, deep, or thorough reflection on that reading also takes time.
If policymakers were serious about building critical thinking, deep reading skills, and thoughtful responses to the text, they would not consider BS Tests like the PARCC for even five minutes. It is one more area where stated intent and actual actions are completely out of alignment.
The Death of Testing Fantasies
It is one of the least surprising research findings ever, confirmed now by at least two studies-- students would do better on the Big Standardized Test if they actually cared about the results.
One of the great fantasies of the testocrats is their belief that the Big Standardized Tests provide useful data. That fantasy is predicated on another fantasy-- that students actually try to do their best on the BS Test. Maybe it's a kind of confirmation bias. Maybe it's a kind of Staring Into Their Own Navels For Too Long bias. But test manufacturers and the policy wonks who love them have so convinced themselves that these tests are super-important and deeply valuable that they tend to believe that students think so, too.
Somehow they imagine a roomful of fourteen-year-olds, faced with a long, tedious standardized test, saying, "Well, this test has absolutely no bearing on any part of my life, but it's really important to me that bureaucrats and policy mavens at the state and federal level have the very best data to work from, so I am going to concentrate hard and give my sincere and heartfelt all to this boring, confusing test that will have no effect on my life whatsoever." Right.
This is not what happens. I often think that we would get some serious BS Test reform in this country if testocrats and bureaucrats and test manufacturers had to sit in the room with the students for the duration of the BS Tests. As I once wrote, if the students don't care, the data aren't there.
There are times when testocrats seem to sense this, though their response is often silly. For instance, back when Pennsylvania was using the PSSA test as our BS Test, state officials decided that students would take the test more seriously if a good score won them a shiny gold sticker on their diploma.
The research suggests that something more than a sticker may be needed. Some British research suggests that cash rewards for good test performance can raise test scores in poor, underperforming students. And then we've got this new, unpublished working paper from researchers John List (University of Chicago), Jeffrey Livngston (Bentley University) and Susan Neckermann (University of Chicago) which asks via title the key question-- "Do Students Show What They Know on Standardized Tests?" Here's the abstract, in all its stilted academic-languaged glory:
Standardized tests are widely used to evaluate teacher performance. It is thus crucial that they accurately measure a student’s academic achievement. We conduct a field experiment where students, parents and tutors are incentivized based partially on the results of standardized tests that we constructed. These tests were designed to measure the same skills as the official state
standardized tests; however, performance on the official tests was not incented. We find substantial improvement on the incented tests but no improvement on the official tests, calling into question whether students show what they know when they have no incentive to do so.
I skimmed through the full paper, though I admit I just didn't feel incented to examine it carefully because this paper is destined to be published in the Journal of Blindingly Obvious Conclusions. Basically, the researchers paid students to try harder on one pointless test, but found that this did not inspire the students to try harder on other pointless tests for free.
A comparable experiment would be for a parent to pay their teenage daughter to clean up her room, then wait to see if she decided to clean the living room, too. There is some useful information here (finding out if she actually knows how to clean a room), but what we already know about motivation (via both science and common sense) tells us that paying her to clean her room actually makes it less likely that she will clean the living room for free.
And my analogy is not perfect because she actually lives in her room and uses the living room, so she has some connection to the cleaning task. Perhaps it would improve my analogy to make it about two rooms in some stranger's home.
The study played with the results of different rewards for the student lab rats, again, with unsurprising results ("The effects are eliminated completely however when the reward amount is small or
payment is delayed by a month").
More problematically, the study authors do not seem to have fully understood what they were doing as witnessed by what they believed was their experimental design--
The experiment is designed to evaluate whether these incentives successfully encourage
knowledge acquisition, then measure whether this acquisition results in higher ISAT scores.
Using a system developed by Discovery Education, the organization which creates the ISAT, we
created “probe” tests which are designed to assess the same skills and knowledge that the official
standardized tests examine.
No. The experiment was designed, whether you grokked it or not, to determine if students could be bribed to try harder on the tests, thereby getting better scores.
The answer is, yes, yes they can, and that result underlines one of the central flaws of test-driven accountability-- if you give students a test that is a pointless exercise in answer-clicking, many will not make any effort to try, and your results are useless crap. The fantasy that BS Tests produce meaningful data is a fantasy deserves to die.
As for the secondary question raised by these studies-- should we start paying students for test performance-- we already know a thousand reasons that such extrinsic rewarding for performance tasks is a Very Bad Idea. So let me leave you with one of the most-linked pieces of work on this blog, Daniel Pink's "Drive"
One of the great fantasies of the testocrats is their belief that the Big Standardized Tests provide useful data. That fantasy is predicated on another fantasy-- that students actually try to do their best on the BS Test. Maybe it's a kind of confirmation bias. Maybe it's a kind of Staring Into Their Own Navels For Too Long bias. But test manufacturers and the policy wonks who love them have so convinced themselves that these tests are super-important and deeply valuable that they tend to believe that students think so, too.
Somehow they imagine a roomful of fourteen-year-olds, faced with a long, tedious standardized test, saying, "Well, this test has absolutely no bearing on any part of my life, but it's really important to me that bureaucrats and policy mavens at the state and federal level have the very best data to work from, so I am going to concentrate hard and give my sincere and heartfelt all to this boring, confusing test that will have no effect on my life whatsoever." Right.
This is not what happens. I often think that we would get some serious BS Test reform in this country if testocrats and bureaucrats and test manufacturers had to sit in the room with the students for the duration of the BS Tests. As I once wrote, if the students don't care, the data aren't there.
There are times when testocrats seem to sense this, though their response is often silly. For instance, back when Pennsylvania was using the PSSA test as our BS Test, state officials decided that students would take the test more seriously if a good score won them a shiny gold sticker on their diploma.
The research suggests that something more than a sticker may be needed. Some British research suggests that cash rewards for good test performance can raise test scores in poor, underperforming students. And then we've got this new, unpublished working paper from researchers John List (University of Chicago), Jeffrey Livngston (Bentley University) and Susan Neckermann (University of Chicago) which asks via title the key question-- "Do Students Show What They Know on Standardized Tests?" Here's the abstract, in all its stilted academic-languaged glory:
Standardized tests are widely used to evaluate teacher performance. It is thus crucial that they accurately measure a student’s academic achievement. We conduct a field experiment where students, parents and tutors are incentivized based partially on the results of standardized tests that we constructed. These tests were designed to measure the same skills as the official state
standardized tests; however, performance on the official tests was not incented. We find substantial improvement on the incented tests but no improvement on the official tests, calling into question whether students show what they know when they have no incentive to do so.
I skimmed through the full paper, though I admit I just didn't feel incented to examine it carefully because this paper is destined to be published in the Journal of Blindingly Obvious Conclusions. Basically, the researchers paid students to try harder on one pointless test, but found that this did not inspire the students to try harder on other pointless tests for free.
A comparable experiment would be for a parent to pay their teenage daughter to clean up her room, then wait to see if she decided to clean the living room, too. There is some useful information here (finding out if she actually knows how to clean a room), but what we already know about motivation (via both science and common sense) tells us that paying her to clean her room actually makes it less likely that she will clean the living room for free.
And my analogy is not perfect because she actually lives in her room and uses the living room, so she has some connection to the cleaning task. Perhaps it would improve my analogy to make it about two rooms in some stranger's home.
The study played with the results of different rewards for the student lab rats, again, with unsurprising results ("The effects are eliminated completely however when the reward amount is small or
payment is delayed by a month").
More problematically, the study authors do not seem to have fully understood what they were doing as witnessed by what they believed was their experimental design--
The experiment is designed to evaluate whether these incentives successfully encourage
knowledge acquisition, then measure whether this acquisition results in higher ISAT scores.
Using a system developed by Discovery Education, the organization which creates the ISAT, we
created “probe” tests which are designed to assess the same skills and knowledge that the official
standardized tests examine.
No. The experiment was designed, whether you grokked it or not, to determine if students could be bribed to try harder on the tests, thereby getting better scores.
The answer is, yes, yes they can, and that result underlines one of the central flaws of test-driven accountability-- if you give students a test that is a pointless exercise in answer-clicking, many will not make any effort to try, and your results are useless crap. The fantasy that BS Tests produce meaningful data is a fantasy deserves to die.
As for the secondary question raised by these studies-- should we start paying students for test performance-- we already know a thousand reasons that such extrinsic rewarding for performance tasks is a Very Bad Idea. So let me leave you with one of the most-linked pieces of work on this blog, Daniel Pink's "Drive"
Wednesday, October 26, 2016
More Bogus Research from Rocketship
When you need some "research" to pump up your ad copy, what better than to just hire it yourself. In fact, why not hire the guys who have been pumping it out reliably for you for years.
Rocketship Academy is patting itself on the back over a new "research" report that it commissioned from good, old SRI International, a group that just happens to be heavily invested in technology-based education. SRI used to stand for Stanford Research Institute, but separated from Stanford in 1970 and changed its name in 1977 (which suggests the split was plenty amicable). That was long before Preston Smith tacked his Teach for America credits onto his resume and looked to enter the lucrative world of edubusiness, but since then, SRI has teamed up with Rocketship to show the awesomeness of the Rocketship Academy product.
Way back in 2011, SRI did a super-duper study of the K-5 Rocketship to show that the Dreambox program (Dreambox because it's what you bury dreams in?) raised scores on the NWEA MAP test by a couple of points. Dreambox is a company that Reed "Elected School Boards Stink" Hastings (Netflix) bought for the Charter School Growth Fund, a fund that also invests in Success Academy and Rocketship Academies. We could play "Follow The Incestuous Privatization Ties" all day, but we'll pass for the moment. We'll also let the usefulness of NWEA results ride for a moment (though you may recognize the name from the successful boycott of the test by Seattle teachers). SRI was back with another glowing report in 2014 and were touting further awesomeness in 2015.
It's almost as if SRI was a corporate partner of Rocketship rather than an "independent nonprofit research center," and indeed, way back in 2010 we find Rocketship then-CEO John Danner (now CEO of Zeal-- more in a second) explaining in an interview that they are teaming up with SRI
Next year, with SRI’s help we’re going to instrument Rocketship to be a test lab where we can measure the effectiveness of every online curricula for elementary schools.
Funny side note. Zeal started out as one more adaptive instructional software start-up, with money from NewSchools Venture fund and other folks interested in a good ROI on their edubiz dollars.Visit their site today and you'll find them headlining "live, on-demand coaching" from "real coaches." So I guess Danner has lost a little of his faith in soft-ware based education.
Fine. Whatever. What about that New Research?
So Rocketship's use of SRI as an independent evaluator is about as fishy as the "volunteer" pulled out of the audience by the bad magician at the company picnic. How about the actual research? Does it look plausible anyway?
Short answer: not so much.
What the research claims is that Rocketship middle schoolers have gained a full year of learning over their peers.
After controlling for demographic differences and other key variables, SRI found that Rocketship alumni outperformed their classmates on NWEA MAP, a nationally norm referenced assessment, by approximately one year of academic growth in both math and reading. This independent evidence from a world-renowned research institute is a powerful validation of our pioneering personalized learning model. Our incredibly dedicated teachers, deeply engaged parents and purposeful approach to technology are delivering significant gains for our Rocketeers that extend into middle school.
Let me pause a moment to nod at my usual ridicule of the notion of measuring learning growth in years, and especially in parts of years. Which part of the year? Did the student gain an April or a September? The notion that a year of learning represents some sort of homogenous tofu-like spread of steady increase only make sense to someone who doesn't know many human beings. It's a statistical construct, like saying that one car is stolen every fifteen minutes, as if car theft is a regular ticking event by which we can measure time. "How long have we been trapped in this dungeon, Chris?" "I'm not sure, but I think Pat's learning has grown about this much, so I figure around three months."
But let's look at the experimental design.
The study population included Rocketship alumni enrolled in one of seven participating charter middle schools in the San Jose area between the 2012–13 and 2015–16 school years, totaling 625 students. The comparison group of non-Rocketship peers was made up of students who did not attend any Rocketship elementary school and who attended the participating charter middle schools in the same grade level and in the same academic years as the Rocketship alumni in the study, totaling 1,294 students.
There's some other baloney about controlling for demographic characteristics "using propensity score weighting and multiple linear regression" (I just hope the reset the turginator on the framistan before they did all that), but the bottom line here is that they took a bunch of Rocketship middle schoolers and compared the ones who had attended Rosketship elementary school to the ones that hadn't.
From this we can conclude that students who attended Rocketship Academy elementary schools were better prepared for Rocketship Academy middle schools than students who did not.
Other research also indicates that at most Rocketship Acdemies, the sun rose to the East of the school. Early working papers also suggest that water is wet.
Also, grit!
Just in case you think Rocketship is only focused on the quality of their test prep for a standardized math and reading test, they'd like you to know that they are also looking into the hearts of Rocketshippiteers.
Of course, test scores are only one indicator of future success. That is why we embed our core core values, socio-emotional curriculum, and positive behavior interventions and supports throughout our Rocketeers’ learning experience. And while these skills are harder to measure, they still matter tremendously, especially for the student population we serve.
Emphasis mine. And the emphasis is just in bolding because I don't have an emoticon for a dropping jaw. "Especially for the student population we serve"??!! You mean poor minority deficient defective kids? Good behavior, grit, socio-emotional qualities-- the rich white kids don't need that? On the one hand, there's a bit of a point here. We have living proof that in our society, a person born into privilege can display every socially undesirable quality known to humankind and still have a shot at being both successful and also Presidential material. On the other hand, it kind of looks like Rocketship needs to work on its savior complex and its deficit thinking model for approaching non-white, non-wealthy students. But then, wealthy kids are not the Rocketship market, nor are the privileged crowd lined up for a chance to partake of this educational awesomeness. The whole model reeks of a low-cost edu-product that's good enough for Those People. Every once in a while they just forget to avoid saying so out loud.
Also, where Smith wrote "hard to measure," I think that's a typo. "Impossible to measure" is undoubtedly what he actually meant.
Bottom Line?
It's more charter marketeering fluffed up with science-flavored PR filler. It's dishonest and not very useful in adding to a real conversation about meeting educational needs or evaluating the actual impact of charters in general and the Rocketship blended plunk-kidsd-in-front-of-computers model in particular. I'm sure we'll see this thing passed around in the weeks ahead. Do not be fooled.
Rocketship Academy is patting itself on the back over a new "research" report that it commissioned from good, old SRI International, a group that just happens to be heavily invested in technology-based education. SRI used to stand for Stanford Research Institute, but separated from Stanford in 1970 and changed its name in 1977 (which suggests the split was plenty amicable). That was long before Preston Smith tacked his Teach for America credits onto his resume and looked to enter the lucrative world of edubusiness, but since then, SRI has teamed up with Rocketship to show the awesomeness of the Rocketship Academy product.
Way back in 2011, SRI did a super-duper study of the K-5 Rocketship to show that the Dreambox program (Dreambox because it's what you bury dreams in?) raised scores on the NWEA MAP test by a couple of points. Dreambox is a company that Reed "Elected School Boards Stink" Hastings (Netflix) bought for the Charter School Growth Fund, a fund that also invests in Success Academy and Rocketship Academies. We could play "Follow The Incestuous Privatization Ties" all day, but we'll pass for the moment. We'll also let the usefulness of NWEA results ride for a moment (though you may recognize the name from the successful boycott of the test by Seattle teachers). SRI was back with another glowing report in 2014 and were touting further awesomeness in 2015.
It's almost as if SRI was a corporate partner of Rocketship rather than an "independent nonprofit research center," and indeed, way back in 2010 we find Rocketship then-CEO John Danner (now CEO of Zeal-- more in a second) explaining in an interview that they are teaming up with SRI
Next year, with SRI’s help we’re going to instrument Rocketship to be a test lab where we can measure the effectiveness of every online curricula for elementary schools.
Funny side note. Zeal started out as one more adaptive instructional software start-up, with money from NewSchools Venture fund and other folks interested in a good ROI on their edubiz dollars.Visit their site today and you'll find them headlining "live, on-demand coaching" from "real coaches." So I guess Danner has lost a little of his faith in soft-ware based education.
Fine. Whatever. What about that New Research?
So Rocketship's use of SRI as an independent evaluator is about as fishy as the "volunteer" pulled out of the audience by the bad magician at the company picnic. How about the actual research? Does it look plausible anyway?
Short answer: not so much.
What the research claims is that Rocketship middle schoolers have gained a full year of learning over their peers.
After controlling for demographic differences and other key variables, SRI found that Rocketship alumni outperformed their classmates on NWEA MAP, a nationally norm referenced assessment, by approximately one year of academic growth in both math and reading. This independent evidence from a world-renowned research institute is a powerful validation of our pioneering personalized learning model. Our incredibly dedicated teachers, deeply engaged parents and purposeful approach to technology are delivering significant gains for our Rocketeers that extend into middle school.
Let me pause a moment to nod at my usual ridicule of the notion of measuring learning growth in years, and especially in parts of years. Which part of the year? Did the student gain an April or a September? The notion that a year of learning represents some sort of homogenous tofu-like spread of steady increase only make sense to someone who doesn't know many human beings. It's a statistical construct, like saying that one car is stolen every fifteen minutes, as if car theft is a regular ticking event by which we can measure time. "How long have we been trapped in this dungeon, Chris?" "I'm not sure, but I think Pat's learning has grown about this much, so I figure around three months."
But let's look at the experimental design.
The study population included Rocketship alumni enrolled in one of seven participating charter middle schools in the San Jose area between the 2012–13 and 2015–16 school years, totaling 625 students. The comparison group of non-Rocketship peers was made up of students who did not attend any Rocketship elementary school and who attended the participating charter middle schools in the same grade level and in the same academic years as the Rocketship alumni in the study, totaling 1,294 students.
There's some other baloney about controlling for demographic characteristics "using propensity score weighting and multiple linear regression" (I just hope the reset the turginator on the framistan before they did all that), but the bottom line here is that they took a bunch of Rocketship middle schoolers and compared the ones who had attended Rosketship elementary school to the ones that hadn't.
From this we can conclude that students who attended Rocketship Academy elementary schools were better prepared for Rocketship Academy middle schools than students who did not.
Other research also indicates that at most Rocketship Acdemies, the sun rose to the East of the school. Early working papers also suggest that water is wet.
Also, grit!
Just in case you think Rocketship is only focused on the quality of their test prep for a standardized math and reading test, they'd like you to know that they are also looking into the hearts of Rocketshippiteers.
Of course, test scores are only one indicator of future success. That is why we embed our core core values, socio-emotional curriculum, and positive behavior interventions and supports throughout our Rocketeers’ learning experience. And while these skills are harder to measure, they still matter tremendously, especially for the student population we serve.
Emphasis mine. And the emphasis is just in bolding because I don't have an emoticon for a dropping jaw. "Especially for the student population we serve"??!! You mean poor minority deficient defective kids? Good behavior, grit, socio-emotional qualities-- the rich white kids don't need that? On the one hand, there's a bit of a point here. We have living proof that in our society, a person born into privilege can display every socially undesirable quality known to humankind and still have a shot at being both successful and also Presidential material. On the other hand, it kind of looks like Rocketship needs to work on its savior complex and its deficit thinking model for approaching non-white, non-wealthy students. But then, wealthy kids are not the Rocketship market, nor are the privileged crowd lined up for a chance to partake of this educational awesomeness. The whole model reeks of a low-cost edu-product that's good enough for Those People. Every once in a while they just forget to avoid saying so out loud.
Also, where Smith wrote "hard to measure," I think that's a typo. "Impossible to measure" is undoubtedly what he actually meant.
Bottom Line?
It's more charter marketeering fluffed up with science-flavored PR filler. It's dishonest and not very useful in adding to a real conversation about meeting educational needs or evaluating the actual impact of charters in general and the Rocketship blended plunk-kidsd-in-front-of-computers model in particular. I'm sure we'll see this thing passed around in the weeks ahead. Do not be fooled.
Tuesday, October 25, 2016
Rot and The Yearbook Biz
Being a yearbook photographer was one of the great joys of my high school career, so I naturally gravitated toward yearbook advising as a teacher. I've been at it for about sixteen years, and I've watched the business change.
Some of the changes are not what folks might have predicted. Technology has been about to destroy yearbooks for years, but there has always been a problem with that-- when I first started, our publisher was trying hard to sell us the newest innovation, and that was a "digital yearbook" housed entirely on a cd-rom. Many schools jumped on that bandwagon, and I'm sure those cd-roms now make excellent coasters and scarecrow decorations. Meanwhile, I can go into our archive and look at fifty, sixty, even seventy year old books and still find that the operating software (my hands and eyeballs) is still working just fine.
Meanwhile, students still love pictures. We have to provide something a little cooler than the selfies and snaps that they share daily on their phones, but as a percentage of the student body, our yearbook sales have stayed pretty solid.
Most of the significant changes are changes the yearbook reader can't see. We produce the book digitally on-line at tremendous savings (when I started, we were spending thousands of dollars annually on photo printing), and my students get practice and training in layout and design. I am one of the fortunate few to have yearbook as a scheduled class during the day, and we build our book up from scratch-- no templates or clip art or anything not produced by the students themselves. The computerizing of all this has created a hugely rich and rewarding educational experience for my students.
But other changes in the biz are not so rich and rewarding-- not for students, anyway. I'm doing one of my every-so-often look at the market to see whether it's time to change companies again, and I notice a common thread among the big companies. Let's see if you can spot it, too!
Taylor Publishing
Founded in 1938 by the three Taylor brothers, who had become interested in yearbook publishing as boys, bought an engraving company, explored some different printing techniques, and finally settled on lithography books which helped them to explode into one of the major players in the business. At times, they were bringing in customers faster than they could serve them. By the seventies the now-public company was suffering from growth without modernization. In the eighties they struggled with Chapter 11, but by the nineties had emerged from that with a strong footing in computer-aided printing and were doing quite well.
Then in 1999 the company was scarfed up by Castle Harlan Partners III, a division of Castle Harlan, Inc, a merchant bank. Castle Harlan is a "Global Leading Private Equity Firm" which is "distinguished by disciplined focus on investing in control positions in middle market private companies."
So Taylor Publishing is no longer in the yearbook business, but is instead in the making money for investors business.
Herff Jones
Back in 1920, Harry Herff and Randall Jones started an insignia company that specialized in school hardware like letters and bars and trophies. They didn't get into the yearbook biz until the 1960s and were perhaps a little late to the digital production party, but they were an employee-owned company, which gave them a nice homegrown feel. They did try an on-line digital version of the book, but that was not such a hit (who knew that people don't like to pay for online content--oh, no, wait-- everybody!). They rebranded themselves a Varsity Brands, with the yearbook as a division of that conglomerate. In 2014, under a new CEO, they sold off several divisions (Lifetouch photography, the ubiquitous church directory outfit, is a Herff-Jones offspring).
Then in December of 2014, the whole company was bought by Charlesbank Capital Partners.
Charlesbank Capital Partners is an investment group with a "broad focus," and they have "A consistent track record of creating value in middle-market companies."
So Herff Jones is no longer in the yearbook business, but is instead in the making money for investors business.
Jostens
Jostens is the 800 pound gorilla of the yearbook biz. In the late 1800s, Otto Josten looked to expand his jewelry business into school memorabilia, and by 1906 he was in the class ring business, which grew into a hugely lucrative business of Otto. In 1950 they set up the American Yearbook Company which quickly stomped all over the marketplace. The next few decades paralleled the whole industry's story-- trouble with management succession in the seventies, financial problems in late eighties-early nineties, expansion into computer stuff. Bought by Bain Capital, then soon bought themselves back and tried to make Wall Street happy, with mixed results.
Then in 2015, Jarden bought up Jostens. Jarden was a "spin-out" company of Ball Corporation that owned a variety of companies including Yankee Candle, Rival and Mr. Coffee. Jarden itself has just completed a merger/absorption by Newell Rubbermaid, putting Jostens in the same pen as Sharpie markers. When contemplating the move, Bloomberg offered this quote:
“This deal is about creating scale,” Chris Ferrara, an analyst for Wells Fargo & Co., said Monday in a research note. “Jarden is a portfolio to which Newell can apply its blueprint of integration, cost savings and selective reinvestment for growth.”
So Jostens is not in the yearbook business, but in the making money for investors business.
So what?
The effects of all these corporate moves have been visible on the local level. My school has dealt with all three of these companies at one point or another over the past few decades, and all three have been marked by a steady decrease in quality of service accompanied by an increase in cost. We dropped Taylor years ago because they increasingly made mistakes with our book. Jostens was at first a nice fit with a production plant right in State College, but that facility has been shut down. All companies have billing and business departments that appear to have no working relationship with the sales or production departments at all.
Meanwhile, longtime sales reps are pushed to get out of the field to make room for newer, younger and presumably cheaper replacements, all of whom are pushed to sell, sell, sell. None of this helps the rep's relationship with the local yearbook adviser, which is rough on everybody because those reps are the point of contact between the company and the customer. How much can you trust someone to look out for your interests when they are busy trying to protect their own back?
Choices?
There are other smaller companies out there, but without naming names (because that would just be mean) some of them just don't create a very good product or offer very good tools. Jostens, for all its corporate faults, has a design product co-developed with Adobe, meaning students can get real preparation for digital publishing in the real world. That's not nothing. But beneath all of it is a sense that there is roiling and mess beneath the surface. I've had two seasoned reps pushed into retirement. I've met reps who just finished jumping from one company to another. My last rep quit suddenly this fall after just two years after leaving his previous company. Now my new rep is a guy whose LinkedIn profile tells me that he is also a recent refuge from another company.
The bottom line is that the more I read, the more I understand that none of the Big Three are as interested in helping me produce a school yearbook as they are in using me to get their numbers up. Maybe that dose of the real world is a useful lesson for me and for my students, but it's a little depressing and it makes the choosing that much more difficult.
It's also a reminder that there's free market and there's free market. All three always existed in the open market, subject to competition and market forces, but braving that sea with ships built of quality yearbooks. Now they're just the patches on ships that are sailing the free market in search of quick profits and ROI. Surviving the free market by serving customers is way different from serving investors. It's a reminder that it's not just the idea of free market schools that is problematic, but the idea of free market schools that are meant to serve investors and hedge fundies.
It's an ugly, destructive rot, and it's spreading everywhere.
Some of the changes are not what folks might have predicted. Technology has been about to destroy yearbooks for years, but there has always been a problem with that-- when I first started, our publisher was trying hard to sell us the newest innovation, and that was a "digital yearbook" housed entirely on a cd-rom. Many schools jumped on that bandwagon, and I'm sure those cd-roms now make excellent coasters and scarecrow decorations. Meanwhile, I can go into our archive and look at fifty, sixty, even seventy year old books and still find that the operating software (my hands and eyeballs) is still working just fine.
Meanwhile, students still love pictures. We have to provide something a little cooler than the selfies and snaps that they share daily on their phones, but as a percentage of the student body, our yearbook sales have stayed pretty solid.
Most of the significant changes are changes the yearbook reader can't see. We produce the book digitally on-line at tremendous savings (when I started, we were spending thousands of dollars annually on photo printing), and my students get practice and training in layout and design. I am one of the fortunate few to have yearbook as a scheduled class during the day, and we build our book up from scratch-- no templates or clip art or anything not produced by the students themselves. The computerizing of all this has created a hugely rich and rewarding educational experience for my students.
But other changes in the biz are not so rich and rewarding-- not for students, anyway. I'm doing one of my every-so-often look at the market to see whether it's time to change companies again, and I notice a common thread among the big companies. Let's see if you can spot it, too!
Taylor Publishing
Founded in 1938 by the three Taylor brothers, who had become interested in yearbook publishing as boys, bought an engraving company, explored some different printing techniques, and finally settled on lithography books which helped them to explode into one of the major players in the business. At times, they were bringing in customers faster than they could serve them. By the seventies the now-public company was suffering from growth without modernization. In the eighties they struggled with Chapter 11, but by the nineties had emerged from that with a strong footing in computer-aided printing and were doing quite well.
Then in 1999 the company was scarfed up by Castle Harlan Partners III, a division of Castle Harlan, Inc, a merchant bank. Castle Harlan is a "Global Leading Private Equity Firm" which is "distinguished by disciplined focus on investing in control positions in middle market private companies."
So Taylor Publishing is no longer in the yearbook business, but is instead in the making money for investors business.
Herff Jones
Back in 1920, Harry Herff and Randall Jones started an insignia company that specialized in school hardware like letters and bars and trophies. They didn't get into the yearbook biz until the 1960s and were perhaps a little late to the digital production party, but they were an employee-owned company, which gave them a nice homegrown feel. They did try an on-line digital version of the book, but that was not such a hit (who knew that people don't like to pay for online content--oh, no, wait-- everybody!). They rebranded themselves a Varsity Brands, with the yearbook as a division of that conglomerate. In 2014, under a new CEO, they sold off several divisions (Lifetouch photography, the ubiquitous church directory outfit, is a Herff-Jones offspring).
Then in December of 2014, the whole company was bought by Charlesbank Capital Partners.
Charlesbank Capital Partners is an investment group with a "broad focus," and they have "A consistent track record of creating value in middle-market companies."
So Herff Jones is no longer in the yearbook business, but is instead in the making money for investors business.
Jostens
Jostens is the 800 pound gorilla of the yearbook biz. In the late 1800s, Otto Josten looked to expand his jewelry business into school memorabilia, and by 1906 he was in the class ring business, which grew into a hugely lucrative business of Otto. In 1950 they set up the American Yearbook Company which quickly stomped all over the marketplace. The next few decades paralleled the whole industry's story-- trouble with management succession in the seventies, financial problems in late eighties-early nineties, expansion into computer stuff. Bought by Bain Capital, then soon bought themselves back and tried to make Wall Street happy, with mixed results.
Then in 2015, Jarden bought up Jostens. Jarden was a "spin-out" company of Ball Corporation that owned a variety of companies including Yankee Candle, Rival and Mr. Coffee. Jarden itself has just completed a merger/absorption by Newell Rubbermaid, putting Jostens in the same pen as Sharpie markers. When contemplating the move, Bloomberg offered this quote:
“This deal is about creating scale,” Chris Ferrara, an analyst for Wells Fargo & Co., said Monday in a research note. “Jarden is a portfolio to which Newell can apply its blueprint of integration, cost savings and selective reinvestment for growth.”
So Jostens is not in the yearbook business, but in the making money for investors business.
So what?
The effects of all these corporate moves have been visible on the local level. My school has dealt with all three of these companies at one point or another over the past few decades, and all three have been marked by a steady decrease in quality of service accompanied by an increase in cost. We dropped Taylor years ago because they increasingly made mistakes with our book. Jostens was at first a nice fit with a production plant right in State College, but that facility has been shut down. All companies have billing and business departments that appear to have no working relationship with the sales or production departments at all.
Meanwhile, longtime sales reps are pushed to get out of the field to make room for newer, younger and presumably cheaper replacements, all of whom are pushed to sell, sell, sell. None of this helps the rep's relationship with the local yearbook adviser, which is rough on everybody because those reps are the point of contact between the company and the customer. How much can you trust someone to look out for your interests when they are busy trying to protect their own back?
Choices?
There are other smaller companies out there, but without naming names (because that would just be mean) some of them just don't create a very good product or offer very good tools. Jostens, for all its corporate faults, has a design product co-developed with Adobe, meaning students can get real preparation for digital publishing in the real world. That's not nothing. But beneath all of it is a sense that there is roiling and mess beneath the surface. I've had two seasoned reps pushed into retirement. I've met reps who just finished jumping from one company to another. My last rep quit suddenly this fall after just two years after leaving his previous company. Now my new rep is a guy whose LinkedIn profile tells me that he is also a recent refuge from another company.
The bottom line is that the more I read, the more I understand that none of the Big Three are as interested in helping me produce a school yearbook as they are in using me to get their numbers up. Maybe that dose of the real world is a useful lesson for me and for my students, but it's a little depressing and it makes the choosing that much more difficult.
It's also a reminder that there's free market and there's free market. All three always existed in the open market, subject to competition and market forces, but braving that sea with ships built of quality yearbooks. Now they're just the patches on ships that are sailing the free market in search of quick profits and ROI. Surviving the free market by serving customers is way different from serving investors. It's a reminder that it's not just the idea of free market schools that is problematic, but the idea of free market schools that are meant to serve investors and hedge fundies.
It's an ugly, destructive rot, and it's spreading everywhere.
Monday, October 24, 2016
Familiar Miracles and Ed Tech
I started typing this a few thousand feet in the air somewhere over Illinois. I worked all day the day before to get things ready to go, got up well before dawn, drove myself to Pittsburgh, where I boarded the first of two planes that will transport my to Seattle for the next several days. Now, after spending many days on the other side of the country, I am finishing up this post thousands of feet in the air, connected to the internet on a computer-based device.
Every step in that process is a small miracle. I could travel to the airport in about 90 minutes because of the big, smooth interstate highway system, as well as the technological marvel that is my car. Now I'm in an airplane, off to try to help out with my daughter's family, which recently upgraded from one nearly two-year-old to one nearly-two-year old and a new-born infant. I am going to miss my wife a great deal, but we can exchange pictures and words any time we like because smartphones. And I'm writing about this on an airplane that has Wi-Fi. Before we took off, I used a pocket-sized, wireless network enabled, computer to tell my family in several parts of the country how my travels were progressing.
We live in an age of miracles, but then, human beings usually do. Henry David Thoreau's family was in the pencil, a technological marvel that allowed people to write or draw with a handheld piece of wood that had ben implanted with just the right amount of material to make easily controlled marks. Thoreau thought the world was already moving too fast for people to properly appreciate it. And it was Thoreau's friend Emerson who noted that if the stars only came out every few hundred years or so, they would strike humans with overwhelming wonder and awe (and, added Isaac Asimov, panic and hysteria).
Miracles are a relative thing. On this plane I'm surrounded by people reading cheap, readily available books. The fact that a million identical copies of a book can be printed and bound so that the very same reading experience can be available to just about anybody-- that's miraculous, but it's also been around for centuries, so very few people feel very much excitement contemplating a book.
Modern familiar miracles give us many responsibilities as teachers. For one, I consider it part of my job to remind my students that they live with miracles, many of which are relatively new. My eighth grade social studies teacher, Mr. Confer, frequently regaled us with stories of his boyhood, and we thought we were getting out of class every time he opened his mouth. Only later in life did I realize he had taught a hugely difficult lesson-- the world was not always the way it is now, and it will not always be this way.
It's also part of our modern gig to remind students that new miracles mean new possibilities. When miracles become familiar, they become transparent, and we forget to look and see and consider what else they can do. I-79 runs down to Pittsburgh, but it runs many other places as well, and if I think of it as just "the road to Pittsburgh,": I miss that. My students carry their own pocket computers, but for some these miraculous devices are simply their Snapchat machine, or their Instagram Access Device. They absolutely forget that other uses are possible.
Reminding these students what miracles can be achieved can also help us remember that devices are only as interesting as their uses. Familiar miracles don't look like miracles at all, and boy do we forget that. Since the first computer landed in a classroom right up until today, this minute, there have been folks who believed that a banal, bland, boring worksheet becomes an explosion of educational awesomeness if we run it through a computer.
It does not. Imagining that your stupid educational idea (let's have a really huge bank of worksheets and hand each student a specific individual one based on how she did on the last one we gave her) will be an awesome idea because you're doing it On A Computer -- that's dumb. Even dumber than saying, "This lesson will be super-engaging if we print it in a book!"
Tools are conduits, no better or worse than the pedagogy we send through them. A speedy delivery system can deliver crap as quickly as it delivers gold, but it doesn't transform one into the other. The fact that we find modern computer tools miraculous in and of themselves really means one thing-- we are old. Each generation experiences its own miracles, and each generation gets to see its miracles become dull, commonplace, familiar.
Yes, we should work hard to preserve and share a sense of wonder, an appreciation for miracles. But in the classroom we must recognize that our favorite miracles may retain little power to amaze in and of themselves (well, unless you're teaching the little ones-- then everything is new and amazing). That's the challenge-- to remain familiar but not jaded, amazed but not foolish. And now I could say more, but here comes the food cart. I will see you all later, back on earth.
Every step in that process is a small miracle. I could travel to the airport in about 90 minutes because of the big, smooth interstate highway system, as well as the technological marvel that is my car. Now I'm in an airplane, off to try to help out with my daughter's family, which recently upgraded from one nearly two-year-old to one nearly-two-year old and a new-born infant. I am going to miss my wife a great deal, but we can exchange pictures and words any time we like because smartphones. And I'm writing about this on an airplane that has Wi-Fi. Before we took off, I used a pocket-sized, wireless network enabled, computer to tell my family in several parts of the country how my travels were progressing.
We live in an age of miracles, but then, human beings usually do. Henry David Thoreau's family was in the pencil, a technological marvel that allowed people to write or draw with a handheld piece of wood that had ben implanted with just the right amount of material to make easily controlled marks. Thoreau thought the world was already moving too fast for people to properly appreciate it. And it was Thoreau's friend Emerson who noted that if the stars only came out every few hundred years or so, they would strike humans with overwhelming wonder and awe (and, added Isaac Asimov, panic and hysteria).
Miracles are a relative thing. On this plane I'm surrounded by people reading cheap, readily available books. The fact that a million identical copies of a book can be printed and bound so that the very same reading experience can be available to just about anybody-- that's miraculous, but it's also been around for centuries, so very few people feel very much excitement contemplating a book.
Modern familiar miracles give us many responsibilities as teachers. For one, I consider it part of my job to remind my students that they live with miracles, many of which are relatively new. My eighth grade social studies teacher, Mr. Confer, frequently regaled us with stories of his boyhood, and we thought we were getting out of class every time he opened his mouth. Only later in life did I realize he had taught a hugely difficult lesson-- the world was not always the way it is now, and it will not always be this way.
It's also part of our modern gig to remind students that new miracles mean new possibilities. When miracles become familiar, they become transparent, and we forget to look and see and consider what else they can do. I-79 runs down to Pittsburgh, but it runs many other places as well, and if I think of it as just "the road to Pittsburgh,": I miss that. My students carry their own pocket computers, but for some these miraculous devices are simply their Snapchat machine, or their Instagram Access Device. They absolutely forget that other uses are possible.
Reminding these students what miracles can be achieved can also help us remember that devices are only as interesting as their uses. Familiar miracles don't look like miracles at all, and boy do we forget that. Since the first computer landed in a classroom right up until today, this minute, there have been folks who believed that a banal, bland, boring worksheet becomes an explosion of educational awesomeness if we run it through a computer.
It does not. Imagining that your stupid educational idea (let's have a really huge bank of worksheets and hand each student a specific individual one based on how she did on the last one we gave her) will be an awesome idea because you're doing it On A Computer -- that's dumb. Even dumber than saying, "This lesson will be super-engaging if we print it in a book!"
Tools are conduits, no better or worse than the pedagogy we send through them. A speedy delivery system can deliver crap as quickly as it delivers gold, but it doesn't transform one into the other. The fact that we find modern computer tools miraculous in and of themselves really means one thing-- we are old. Each generation experiences its own miracles, and each generation gets to see its miracles become dull, commonplace, familiar.
Yes, we should work hard to preserve and share a sense of wonder, an appreciation for miracles. But in the classroom we must recognize that our favorite miracles may retain little power to amaze in and of themselves (well, unless you're teaching the little ones-- then everything is new and amazing). That's the challenge-- to remain familiar but not jaded, amazed but not foolish. And now I could say more, but here comes the food cart. I will see you all later, back on earth.
NY: Commissioner Struggles with Teacher Shortage
Maryellen Elia was brought in to the Empire State to help clean up the mess that former education honcho John King left behind.
Her results have been mixed. She invested a lot of energy in defending the Big Standardized Test, even issuing a handy propaganda kit to help push down those opt out numbers. It didn't work.
Last Friday, Elia talked to the NYS Association of Teacher Educators and the NY Association of Colleges for Teacher Education about some of the issues tied to the looming (or possibly not, or possibly already here, depending on who's talking) teacher shortage. As reported by the Times-Union, some of what she had to say was on point, and some of it indicates that she still doesn't understand the situation.
On the plus side, Elia is one policy leader who understands that more than decade of beating up on teachers and the teaching profession has not exactly made the field attractive to folks.
But folks at the meting also talked about a side-effect of test-driven accountability that has been long-predicted and now visibly concerning. Placing student teachers is becoming increasingly difficult in a world where high stakes testing season runs the schools. How can I possibly turn my class over to a pre-rookie when so much about the future of my school and my own professional career is riding on what's happening in my classroom. And the focus at this meeting was on student teachers who are hard to place at al-- that's before we even get to student teachers who get a placement, but who are not allowed to do much of anything because of concerns about the BS Tests.
Elia said that the department is working on "several efforts to boost district cooperation." But the department also made efforts to boost district cooperation and participation with the BS Test-- and they failed.
Elia addressed the exceptionally confusing mess in NY regarding subject area tests, which are transitioning except when they aren't, and which allow students to take one of a couple of options, their choice best based on judicious use of a Ouija board, since even education professors have thrown up their hands in confusion. There is no report on whether anyone specifically called Elia out on New York's use of the giant baloneyfest that is the expensive-but-useless EdTPA tests.
Elia was also called on the defend the new system by which out-of-state teachers can waltz right into a NY classroom with no additional tests or paperwork, meaning that the absolute best path for someone who wants to teach in NY is to go get certification in some other state.
Teacher education schools are understandably grumpy-- this is a direct assault on their market base. Elia defended the practice by pointing to the teacher shortage, and in her defense, I should point out that NY's solution is still far better than the many states flirting with or embracing the practice of simply issuing teaching certificates to any upright hominid with a pulse. That's what we've come to in education-- any discussion of bad policy can eventually lead us to observe that it could be worse and, in fact, somewhere (probably Florida or North Carolina) it is.
However, "not the worst policy around" is a low hurdle to clear. While from the standpoint of a Pennsylvanian who knows that our teacher ed programs are having trouble rounding up students, I think New York's policies are just swell. But as someone who cares about US public education, I'm not so impressed. Maybe as a veteran of some giant Sunshine Stare disasters, Elia can't help but cast a rosy gaze on New York policies, but I would encourage her to shoot for a more ambitious slogan than "New York education-- At Least We're Not Florida." That is not how to recruit folks for the profession.
Her results have been mixed. She invested a lot of energy in defending the Big Standardized Test, even issuing a handy propaganda kit to help push down those opt out numbers. It didn't work.
Last Friday, Elia talked to the NYS Association of Teacher Educators and the NY Association of Colleges for Teacher Education about some of the issues tied to the looming (or possibly not, or possibly already here, depending on who's talking) teacher shortage. As reported by the Times-Union, some of what she had to say was on point, and some of it indicates that she still doesn't understand the situation.
On the plus side, Elia is one policy leader who understands that more than decade of beating up on teachers and the teaching profession has not exactly made the field attractive to folks.
But folks at the meting also talked about a side-effect of test-driven accountability that has been long-predicted and now visibly concerning. Placing student teachers is becoming increasingly difficult in a world where high stakes testing season runs the schools. How can I possibly turn my class over to a pre-rookie when so much about the future of my school and my own professional career is riding on what's happening in my classroom. And the focus at this meeting was on student teachers who are hard to place at al-- that's before we even get to student teachers who get a placement, but who are not allowed to do much of anything because of concerns about the BS Tests.
Elia said that the department is working on "several efforts to boost district cooperation." But the department also made efforts to boost district cooperation and participation with the BS Test-- and they failed.
Elia addressed the exceptionally confusing mess in NY regarding subject area tests, which are transitioning except when they aren't, and which allow students to take one of a couple of options, their choice best based on judicious use of a Ouija board, since even education professors have thrown up their hands in confusion. There is no report on whether anyone specifically called Elia out on New York's use of the giant baloneyfest that is the expensive-but-useless EdTPA tests.
Elia was also called on the defend the new system by which out-of-state teachers can waltz right into a NY classroom with no additional tests or paperwork, meaning that the absolute best path for someone who wants to teach in NY is to go get certification in some other state.
Teacher education schools are understandably grumpy-- this is a direct assault on their market base. Elia defended the practice by pointing to the teacher shortage, and in her defense, I should point out that NY's solution is still far better than the many states flirting with or embracing the practice of simply issuing teaching certificates to any upright hominid with a pulse. That's what we've come to in education-- any discussion of bad policy can eventually lead us to observe that it could be worse and, in fact, somewhere (probably Florida or North Carolina) it is.
However, "not the worst policy around" is a low hurdle to clear. While from the standpoint of a Pennsylvanian who knows that our teacher ed programs are having trouble rounding up students, I think New York's policies are just swell. But as someone who cares about US public education, I'm not so impressed. Maybe as a veteran of some giant Sunshine Stare disasters, Elia can't help but cast a rosy gaze on New York policies, but I would encourage her to shoot for a more ambitious slogan than "New York education-- At Least We're Not Florida." That is not how to recruit folks for the profession.
Sunday, October 23, 2016
ICYMI: Reading in the Seattle Dew (10/23)
I'm far from home, but there's still plenty to read on the interwebs.
Our Stupid Questions
Teacher Tom with a striking twist on the old "no such thing as a dumb question" shtick.
How Joy Became the New Grit
Jennifer Berkshire gives us a look at how the world of grit is giving way to something even worse
Talk about Passion
Kristen Perkins guest-blogs at Blue Cereal Education with a piece about passion in teaching.
Reflections from a Nasty Woman
Nancy Flanagan with a brave and honest post about the price women pay for coming forward in matters of sexual harassment. If you're only going to read one item on this list, this is the one.
What New Challenges To The Charter School Industry Reveal
These were supposed to be the salad days for the charter business. Jeff Bryant takes a broad look at why that's not happening.
The Value-Added of Teacher Preparation Programs
Audrey Amrein-Beardsley takes a look about what new research tells us when it comes to using VAM to evaluate teacher education programs
You Can't Teach Writing from the Side of the Pool
Did we just talk about this last week? Sure we did, but it's a point worthy of repeating. Russ Walsh hosts a guest post from Cynthia Mershon.
More About Attrition Rates in Boston
Mark Weber (Jersey Jazzman) is back to once again cut through the smoke and mirrors of the Massachusetts charter debate.
Our Stupid Questions
Teacher Tom with a striking twist on the old "no such thing as a dumb question" shtick.
How Joy Became the New Grit
Jennifer Berkshire gives us a look at how the world of grit is giving way to something even worse
Talk about Passion
Kristen Perkins guest-blogs at Blue Cereal Education with a piece about passion in teaching.
Reflections from a Nasty Woman
Nancy Flanagan with a brave and honest post about the price women pay for coming forward in matters of sexual harassment. If you're only going to read one item on this list, this is the one.
What New Challenges To The Charter School Industry Reveal
These were supposed to be the salad days for the charter business. Jeff Bryant takes a broad look at why that's not happening.
The Value-Added of Teacher Preparation Programs
Audrey Amrein-Beardsley takes a look about what new research tells us when it comes to using VAM to evaluate teacher education programs
You Can't Teach Writing from the Side of the Pool
Did we just talk about this last week? Sure we did, but it's a point worthy of repeating. Russ Walsh hosts a guest post from Cynthia Mershon.
More About Attrition Rates in Boston
Mark Weber (Jersey Jazzman) is back to once again cut through the smoke and mirrors of the Massachusetts charter debate.
The Best-Laid Plans of Grown-Ups
This after noon we took the grandsons to a playground. It's a lovely playground, one of many, many lovely playgrounds available in Seattle. Here's a look at just some of the cool playground stuff available there.
And here is how my oldest grandson spent a good chunk of his time.
It's a well-flogged truism that children will throw away the toy and play with the box, that they will reject the finest plastic construction that the toy industry can muster in order to play with ordinary household objects. I suppose that somebody could have forced my grandson to drop the stick and play "properly" but why, unless they were intent on imposing adult will and plans on a child. "I planned on you playing on that jungle gym over there. Now put down that stick and go have fun, dammit, or else."
The bottom line is that children have instincts and interests and involvement of their own. Adults can go nuts trying to direct that, and they can twist children's brains up by hammering them withy messages about what they are "supposed" to do.
It is certainly true that there is room for adult direction and guidance. My grandson played with some of the equipment and played with his father, who did not try to tell my grandson what to do, but joined wholeheartedly in helping my grandson tap into his transcendent joy over swinging.
But if you go to the playground armed with an adult agenda that allows no room for the voice of the children, you are on the wrong path. The damage is evident by the time students land in my eleventh grade classroom and have trouble writing well because they are more concerned about what they are supposed to write-- what they are supposed to do to meet the requirements of the grown-ups' agenda-- instead of tying to get in touch withy what they actually think.
It is easy as parents or teachers to get caught up in the desire to see the tiny humans make the safest, wisest, best decisions. But that process has to include their own voice, their own aims, their own intentions and inclinations. That's not just how you honor their existent as thinking, feeling, sentient, individual human beings-- it's how you create future entrepreneurs, leaders, creators, makers, employees, employers, and people who are not inclined to elect raging tyrants out of desire to have "strong" leaders who will tell them just what they are supposed to do.
Yes, the world needs a certain amount of order and sense, and I am not advocating unleashing wild anarchic chaos on the universe (not today, anyway). But attempting to impose adult best-laid plans on every minute of children's lives is both evil and foolish. Evil, because every human's voice is a precious thing no matter how young. Foolish because-- well, I will give my grandson the last word with his ideas about how to use carrot slices.
Saturday, October 22, 2016
Nobel-Winning Evaluation Advice
Sometimes it takes a Nobel Prize-winning economist to confirm what many of us in education have been saying for years-- reformsters are doing teacher and school evaluation all wrong.
The writer pointing this out if Derek Neal (University of Chicago), over at Education Next of all places, The prize-winner he's talking about is Bengt Holmstrom, a Finnish economist at MIT.
Holmstrom has done pretty much no work on education. What he has done is work with contract theory, particularly with contracted with incentives in areas "where worker actions are hard to observe and worker output is difficult to quantify." So, totally education.
Neal focuses on two major elements of Holmstrom's work, providing two insights into how an incentive pay system can go wrong.
1) Good incentive systems use all the data that provides more clarity and detail about employee performance. So, not just some sliver of data, and not data that doesn't really improve the picture of job performance.
2) There must be alignment between the performance task and the actual desired task.
Both which tell us that the test-based evaluation that has been favored since NCLB was a pup is seriously off the mark.
So, if out of the full range of teacher behaviors, you collect only data about how students do on a narrow reading-and-math test (which is also measures a mess of data unrelated to teacher performance), you cannot build a good incentive system on that data.
And if, for instance, you are not measuring how well students read, but rather how well they answer multiple-choice questions, your data is not suitable for creating performance incentives.
Basically, Holmstrom provides a fancy explanation of why Campbell's Law is a thing and how it works. When you measure the wrong thing and/or measure the right thing incompletely, you incentivize the wrong behavior, and you get lousy results.
Holmstrom also notes that in some settings, collecting enough of the right data can be really prohibitively expensive. So what to do instead?
In these settings, the best approach may be to adopt hiring procedures that identify workers who will perform well in order to satisfy their own personal norms and the norms espoused by the organization, and then pay these workers a fixed salary.
What?? Hire competent, self-directed people with solid training and then just pay them well?! That's crazy talk, you Nobel-winning loon.
Neal also notes that reformsters are more focused on doing evaluation than on doing it correctly, but that the data from the past decade or three shows the systems put in place are not working so well.
Many voices in current education policy debates are advocating an end to all forms of assessment-based incentives. These reactions are understandable given the evidence gathered over the last two decades or more, but we do not yet know how educators would respond to well-designed incentive schemes that incorporate the theoretical insights of Holmstrom and others, as well as the empirical insights produced by decades of research on the use of incentive systems in schools, government agencies, and businesses. We do know that, if policymakers continue to ignore these insights, those who oppose all forms of assessment-based incentives will continue to gather evidence that lends support to their cause.
It's not rocket surgery. If policymakers continue to design incentive systems based on measuring the wrong things badly, on measuring based on tasks that are not aligned withy the results we actually want, then these systems will continue to stink.
The writer pointing this out if Derek Neal (University of Chicago), over at Education Next of all places, The prize-winner he's talking about is Bengt Holmstrom, a Finnish economist at MIT.
This guy has more Nobel Prizes than you |
Holmstrom has done pretty much no work on education. What he has done is work with contract theory, particularly with contracted with incentives in areas "where worker actions are hard to observe and worker output is difficult to quantify." So, totally education.
Neal focuses on two major elements of Holmstrom's work, providing two insights into how an incentive pay system can go wrong.
1) Good incentive systems use all the data that provides more clarity and detail about employee performance. So, not just some sliver of data, and not data that doesn't really improve the picture of job performance.
2) There must be alignment between the performance task and the actual desired task.
Both which tell us that the test-based evaluation that has been favored since NCLB was a pup is seriously off the mark.
So, if out of the full range of teacher behaviors, you collect only data about how students do on a narrow reading-and-math test (which is also measures a mess of data unrelated to teacher performance), you cannot build a good incentive system on that data.
And if, for instance, you are not measuring how well students read, but rather how well they answer multiple-choice questions, your data is not suitable for creating performance incentives.
Basically, Holmstrom provides a fancy explanation of why Campbell's Law is a thing and how it works. When you measure the wrong thing and/or measure the right thing incompletely, you incentivize the wrong behavior, and you get lousy results.
Holmstrom also notes that in some settings, collecting enough of the right data can be really prohibitively expensive. So what to do instead?
In these settings, the best approach may be to adopt hiring procedures that identify workers who will perform well in order to satisfy their own personal norms and the norms espoused by the organization, and then pay these workers a fixed salary.
What?? Hire competent, self-directed people with solid training and then just pay them well?! That's crazy talk, you Nobel-winning loon.
Neal also notes that reformsters are more focused on doing evaluation than on doing it correctly, but that the data from the past decade or three shows the systems put in place are not working so well.
Many voices in current education policy debates are advocating an end to all forms of assessment-based incentives. These reactions are understandable given the evidence gathered over the last two decades or more, but we do not yet know how educators would respond to well-designed incentive schemes that incorporate the theoretical insights of Holmstrom and others, as well as the empirical insights produced by decades of research on the use of incentive systems in schools, government agencies, and businesses. We do know that, if policymakers continue to ignore these insights, those who oppose all forms of assessment-based incentives will continue to gather evidence that lends support to their cause.
It's not rocket surgery. If policymakers continue to design incentive systems based on measuring the wrong things badly, on measuring based on tasks that are not aligned withy the results we actually want, then these systems will continue to stink.