Chad Aldeman at Bellwether Ed Partners has a new report out. Mind the Gap: Re-Imagining the Way States Judge High School Quality gets a few critical questions right, but the answers still need work.
First, the big news
Keep in mind that Bellwether is a reliable member of the reformster thinky tank pantheon, often sipping free market ambrosia while sharing a cloud with the Fordham Institute. I want you to keep that in mind when you read the next quote, which would be unremarkable anywhere else:
State and federal policies on high schools typically reward schools that perform well on measures like test scores and graduation rates while forcing changes on those that don’t. When these two measures alone serve as proxies for a quality high school they paint an incomplete picture of success, one that can reflect more on the school’s demographics than its success in educating students and preparing them for the future. And instead of focusing on higher-order skills,challenging coursework, and annual progress toward college and career readiness, schools are encouraged to focus on lower-level skills and push all students through to a diploma, regardless of what they learn.
Yup. Turns out that the Big Standardized Tests doesn't give a very complete picture of student achievement. Not only that, but because the BS Tests come with such high stakes, schools tend to focus on preparing students for the test and not on giving students a great complete education.And also, this:
Accountability systems reflect these choices. Ideally they would reflect what society values, not
what’s easiest to measure.
Look, I don't want to minimize the fact that these guys finally arrived here. But at the same time, if I'm honest, my most visceral reaction is, "No shit, Sherlock" or "Duhhh" or "In other news, the sun rose in the East this morning."
One Remaining Problem
There is still a big hitch in Aldeman's gitalong, and it shows up in the assumption that a high school's job is to get a student ready for college or a career. Unstated is the idea that education might be for life or even something as old-fashioned as citizenship. That will come back to bite us in the butt, later.
Blah Blah Blah
The first segments of the paper involve pointing out that people should breathe air and drink water. No, okay, the first sections are used to re-make the arguments that some of us have been making for years now-- like what gets measured gets managed, so if you're measuring the wrong things (like narrowly targeted items on BS Tests) then those become the focus of your school and the school stops doing the work that society set it up to do in the first place. Also, if you jam your finger into your eye, your eyeball will start to hurt.
Yes, it would be nice to include a paragraph saying, "Many critics of education reform have been pointing this out for years now and I've come around to thinking they may have a point." But hey-- that's just selfishness talking. I'm glad to find Belwether on this page, finally.
Some Data Points
Because what would a "paper" be without them. Aldeman takes a moment to note that our college drop-out rate is way up there. We're going to interpret this to mean (sigh) that these dropouts were lied to by their schools and told they were ready for college when they weren't. I don't want to side track this too much, but I will offer some alternative theories for why Chris left college.
1) Chris's high school said, "You are not ready for college, at least not THAT college," but Chris went anyway.
2) Wassamatta University accepted Chris because WU needs money, not because they thought Chris was a great prospect.
3) Chris dropped out because WU's costs turned out to be unmanageable.
The Alternative
The real goal is for Aldenman to lay out an alternative program for measuring high school successfulness. Aldeman has used the SCORE model from Tennessee as his template. I will spoil the suspense by observing that his solution needs work.
This would be where Aldeman drops the truth-grenade that test scores have a lot to do with poverty rates. Again, the news not so much in someone saying what we've already known for years, but in who is saying it.
While additional alignment and enrigorfication may help, Aldeman believes that some other factors have "predictive power" in telling us who is really ready for college. And this is where he starts to head into the weeds, mostly because he's still stuck in the notion that high school is basically vocational training, or pre-vocational training for people who will get their vocational training in college. And that still-narrow too-limited view of the purpose of education will lead Aldeman to some cramped solutions to the accountability puzzle.
For his accountability model, Aldeman will keep test results plus advanced course passing rate as 40% of the school's score. The course passage rate strikes me as extremely game-able, but mostly this part is the same old same old.
Next, Aldeman tries some sleight-of-hand. Let's look at actual progression and graduation rates as compared to a reasonable prediction of how the students would have done-- hey, wait a minute!! That's VAM! Aldeman wants to use the magic of VAM sauce to measure the whole school. For 20%. Well, that's just a crock. And the fact that he studiously avoids the usual language of VAM suggests that Aldeman knows he's trying to sneak that wolf into the shepherd's party dressed in a sheep suit.
Aldeman throws in 5% for school environment measures, which is a fair idea, though I'm a little surprised to find a modern reformster measuring inputs instead of outputs. I thought inputs were so fifteen years ago.
His final piece is intriguing. Aldeman proposes measuring post-high school outcomes-- college attendance rates (actual compared, sigh, to predicted, which means it will eventually top out and the school's rating will tank), college credit accumulation, employment, employment earnings!!
On the one hand, I think an after-graduation follow-up has merit. My colleagues and I regularly do informal follow-up with our grads-- how did we do preparing you, what helped, what did you need, that sort of thing. On the other hand, Aldeman is suggesting that we stop relying on data that is highly influenced by the students' socio-economic background and start including some other factors that are highly influenced by the students's socio-economic background. We know that social mobility is limited, and that poor students from poor communities mostly don't end up in the top 5% of income sets. In other words, I'm not sure that Aldeman's system doesn't continue the business of punishing poor schools in poor neighborhoods for having poor students.
The Best Sentence
So Aldeman's paper has a great deal of work to do yet. Good job of identifying the problem; not such a great job of designing a solution. But I will give Aldeman a bonus point or two for this sentence:
Public school accountability systems should measure what society values out of its education system.
In this respect, we are in perfect agreement. And if we can agree that society does not most value students getting scores on Big Standardized Tests, we have a basis to move forward. We might even be able to talk about higher aspirations than simply educating to get a job.
I can help a little with the story of Chris.
ReplyDeleteAt my institution, about 20 out of every 100 first year students will not return for their second year. Of those 20 who leave, 13 were asked to leave by the University because of poor academic performance. All 13 were judged by their high schools to be adequately prepared for my institution (there is automatic admission for any student with an average high school GPA of 2.0 across a set of academic classes and the state high schools are well aware of that), but it may be that the students were not actually prepared academically, it may be that the students were not able to deal with the temptations from living away from home, or it may be that the students were not able to both work to pay tuition (in state it is about $4,500 a semester) and keep up with classes. Some of those students will return after they show evidence of improved academic performance, typically by successfully taking classes a community college.
The remaining 7 who leave are in good academic standing when they leave. They might have left because the costs became unmanageable or they might have left because they transferred to another college or university. They might decide that they don't want to go to college after all.
No one has any idea about what happens to those 20 students, but there are efforts now to try and find out how many of them eventually get a college degree or not after leaving the first institution they attend.
I do not think that nearly enough attention is paid to the number of students who are over zealous in their leisure activities when they first leave for college. I attended college in the 80s, a school that was considered a good commuter college. There were many students starting in the second year because of problems with living in a dorm. And, there were many students that did not stay passed the first or second year because of partying, or the need to work. College is not for everyone and education reform will not change that.
ReplyDeleteAldeman says accountability systems should "reflect what society values, not what's easiest to measure", but doesn't that mean we have to determine what society actually values before we can set up an accountability system? And what exactly does "society" mean? Parents? Students? Corporations? I'll go with parents and students.
ReplyDeleteBesides talking about being ready for college or a career, Aldeman mentions "soft skills that are essential to success in life," which for him are "perseverance, grit, and higher order thinking skills." To measure higher order thinking skills, he suggests placement into and passing rates on AP tests and career/technical programs and certifications, but he has no suggestion for measuring grit or perseverance. Rather than grit or perseverance (I want to scream every time I see the word "grit"), I would suggest focusing on what they call the "non-cognitive" skills: emotional and social skills. Of course, they're hard to measure.
Aldeman says student surveys on "engagement" and safety have been shown to predict student learning growth, which I think is more than what can be said about a lot of other things in his system, so I would certainly weight them more than 5%. And certainly it would be good to figure out how to better align K-12 with post-secondary, and also to fund career-technical programs and make sure there are, as Bernie Sanders says, "effective, attainable pathways for young people to pursue middle class careers."
By all means, collect data. But data only tell you where the problems lie, not why they exist or how to solve them. Collect data, but don't put them in a stupid algorithm to rank anything. Closing schools, firing teachers and hiring new ones isn't going to change a thing if you still don't know the whys and hows, and data and algorithms won't tell you. Figuring out why the problem exists so you can figure out specifically what might work to solve it is a lot harder than collecting data, but can certainly be done if we actually try to do it instead of stopping at data collection. Aldeman thinks schools just need "incentives," stupid sticks and carrots, to solve problems. No, they don't. The problem isn't motivation. The problem is figuring out what works.