Chad Aldeman at Bellwether Ed Partners has a new report out. Mind the Gap: Re-Imagining the Way States Judge High School Quality gets a few critical questions right, but the answers still need work.
First, the big news
Keep in mind that Bellwether is a reliable member of the reformster thinky tank pantheon, often sipping free market ambrosia while sharing a cloud with the Fordham Institute. I want you to keep that in mind when you read the next quote, which would be unremarkable anywhere else:
State and federal policies on high schools typically reward schools that perform well on measures like test scores and graduation rates while forcing changes on those that don’t. When these two measures alone serve as proxies for a quality high school they paint an incomplete picture of success, one that can reflect more on the school’s demographics than its success in educating students and preparing them for the future. And instead of focusing on higher-order skills,challenging coursework, and annual progress toward college and career readiness, schools are encouraged to focus on lower-level skills and push all students through to a diploma, regardless of what they learn.
Yup. Turns out that the Big Standardized Tests doesn't give a very complete picture of student achievement. Not only that, but because the BS Tests come with such high stakes, schools tend to focus on preparing students for the test and not on giving students a great complete education.And also, this:
Accountability systems reflect these choices. Ideally they would reflect what society values, not
what’s easiest to measure.
Look, I don't want to minimize the fact that these guys finally arrived here. But at the same time, if I'm honest, my most visceral reaction is, "No shit, Sherlock" or "Duhhh" or "In other news, the sun rose in the East this morning."
One Remaining Problem
There is still a big hitch in Aldeman's gitalong, and it shows up in the assumption that a high school's job is to get a student ready for college or a career. Unstated is the idea that education might be for life or even something as old-fashioned as citizenship. That will come back to bite us in the butt, later.
Blah Blah Blah
The first segments of the paper involve pointing out that people should breathe air and drink water. No, okay, the first sections are used to re-make the arguments that some of us have been making for years now-- like what gets measured gets managed, so if you're measuring the wrong things (like narrowly targeted items on BS Tests) then those become the focus of your school and the school stops doing the work that society set it up to do in the first place. Also, if you jam your finger into your eye, your eyeball will start to hurt.
Yes, it would be nice to include a paragraph saying, "Many critics of education reform have been pointing this out for years now and I've come around to thinking they may have a point." But hey-- that's just selfishness talking. I'm glad to find Belwether on this page, finally.
Some Data Points
Because what would a "paper" be without them. Aldeman takes a moment to note that our college drop-out rate is way up there. We're going to interpret this to mean (sigh) that these dropouts were lied to by their schools and told they were ready for college when they weren't. I don't want to side track this too much, but I will offer some alternative theories for why Chris left college.
1) Chris's high school said, "You are not ready for college, at least not THAT college," but Chris went anyway.
2) Wassamatta University accepted Chris because WU needs money, not because they thought Chris was a great prospect.
3) Chris dropped out because WU's costs turned out to be unmanageable.
The real goal is for Aldenman to lay out an alternative program for measuring high school successfulness. Aldeman has used the SCORE model from Tennessee as his template. I will spoil the suspense by observing that his solution needs work.
This would be where Aldeman drops the truth-grenade that test scores have a lot to do with poverty rates. Again, the news not so much in someone saying what we've already known for years, but in who is saying it.
While additional alignment and enrigorfication may help, Aldeman believes that some other factors have "predictive power" in telling us who is really ready for college. And this is where he starts to head into the weeds, mostly because he's still stuck in the notion that high school is basically vocational training, or pre-vocational training for people who will get their vocational training in college. And that still-narrow too-limited view of the purpose of education will lead Aldeman to some cramped solutions to the accountability puzzle.
For his accountability model, Aldeman will keep test results plus advanced course passing rate as 40% of the school's score. The course passage rate strikes me as extremely game-able, but mostly this part is the same old same old.
Next, Aldeman tries some sleight-of-hand. Let's look at actual progression and graduation rates as compared to a reasonable prediction of how the students would have done-- hey, wait a minute!! That's VAM! Aldeman wants to use the magic of VAM sauce to measure the whole school. For 20%. Well, that's just a crock. And the fact that he studiously avoids the usual language of VAM suggests that Aldeman knows he's trying to sneak that wolf into the shepherd's party dressed in a sheep suit.
Aldeman throws in 5% for school environment measures, which is a fair idea, though I'm a little surprised to find a modern reformster measuring inputs instead of outputs. I thought inputs were so fifteen years ago.
His final piece is intriguing. Aldeman proposes measuring post-high school outcomes-- college attendance rates (actual compared, sigh, to predicted, which means it will eventually top out and the school's rating will tank), college credit accumulation, employment, employment earnings!!
On the one hand, I think an after-graduation follow-up has merit. My colleagues and I regularly do informal follow-up with our grads-- how did we do preparing you, what helped, what did you need, that sort of thing. On the other hand, Aldeman is suggesting that we stop relying on data that is highly influenced by the students' socio-economic background and start including some other factors that are highly influenced by the students's socio-economic background. We know that social mobility is limited, and that poor students from poor communities mostly don't end up in the top 5% of income sets. In other words, I'm not sure that Aldeman's system doesn't continue the business of punishing poor schools in poor neighborhoods for having poor students.
The Best Sentence
So Aldeman's paper has a great deal of work to do yet. Good job of identifying the problem; not such a great job of designing a solution. But I will give Aldeman a bonus point or two for this sentence:
Public school accountability systems should measure what society values out of its education system.
In this respect, we are in perfect agreement. And if we can agree that society does not most value students getting scores on Big Standardized Tests, we have a basis to move forward. We might even be able to talk about higher aspirations than simply educating to get a job.