Learning Loss and the Big Standardized Test

On Twitter yesterday I posted a thread that seemed to touch a few nerves, so I'm sharing it here as well.

You want to talk about learning loss? Then talk about all the learning lost to high stakes testing. 1/

— Peter Greene (@palan57) March 1, 2022

Not just the test itself, but the time spent in test prep and just plain learning how to take that kind of test and speak the test manufacturers' language. Plus the pre-test tests (to see how well we're on track for the test) 2/

— Peter Greene (@palan57) March 1, 2022

People who have not been in the classroom for the last couple of decades have no idea how much education time has been eroded by high stakes testing. 3/

— Peter Greene (@palan57) March 1, 2022

Want to address learning loss? Never mind tutoring and summer school. In most schools. you could add 6-10 weeks of instruction just by completely scrapping the Big Standardized Test. 4/

— Peter Greene (@palan57) March 1, 2022

And it wouldn't cost the taxpayers a cent. But it would make test manufacturers and the Data Gods sad. "But we need the data to address the problem," say testocrats. 5/

— Peter Greene (@palan57) March 1, 2022

It's a Kafkaesque bad office joke--let's make people spend more time drawing up reports on how the job is going so that they spend less and less time actually doing the job. 6/

— Peter Greene (@palan57) March 1, 2022

Note also that we have the research telling us the data collected doesn't help improve test scores https://t.co/OG4TdXbMRQ 7/

— Peter Greene (@palan57) March 1, 2022

And test data isn't any good at improving teacher evaluation, either.https://t.co/Fs3pO6Kj2S 8/

— Peter Greene (@palan57) March 1, 2022

It is amazing to me that twenty-some years on, we are still here. Responders included a huge number of teachers telling stories of the time lost to testing and test prep, plus a much smaller assortment of people saying things like "But how will we know how students are doing" and "If you just teach 'em good, the test scores will take care of themselves." 

How can we be a whole generation down the road and still be wasting time on the Big Standardized Test as if it were a valuable data-gathering tool that told us Many Useful Things about education? How can the backbone of our educational accountability systems be a bad test that, in its high-stakes threats, brings more damage than help? 

 

Catalyze Challenge: Throwing Money At Edupreneurs

It's another one of those things that helps you realize that some reformsters are operating in just another world entirely.

Meet the Catalyze Challenge. It's a grant competition that "supports innovation in career-connected learning that meaningfully bridge education and career." Its analysis of the need is the same old same old-- "Ourt education system promises economic opportunity for all--but right now, too many students graduate high school and college without the resources, skills, and support to thrive in their careers."

Press about the initiative says it's about how "to help foster new learning models." The idea is to fund "education entrepreneurs" aka education flavored businesses. And if we look at the funders--well, it's a familiar roster.

Sponsoring partners are the Joyce Foundation, the Walton Family Foundation, the Charkes Koch Foundation, and American Student Assistance. Koch has loaned Brennan Brown, their director of Education Partnerships, as an advisor. Brown's previous education experience is teaching at Northwood University for a decade. Northwood is located in Midland, Michigan; they're home to the DeVos Graduate School of Management and annually award the Richard DeVos Young Entrepreneur Award to honor the Amway founder who was "a true friend of Northwood University."

The challenge has two tracks. For Accelerate Award you can score up to half a million dollars; the Ignite Award goes up to $50K. There's a lot of talk about unlocking career success for students, equity, scalability, career identity development, post-high school pathways, and being groundbreaking (at one point the term "moonshot" crops up.

The implementing partners include good old NewVentureFund, a kind of venture philanthropy/lobbying/consulting supposedly-left-tilted outfit, connected to The Bill and Melinda Gates Foundation and considered by some to be a dark money group, and they appear to indulge in plenty of the same sort of faux group and astro-turfy sponsorship we know and love from other sources, and they themselves come under the umbrella of DC-base Arabella Advisors.

But the lead operational group appears to be Common Group, a "social impact consulting firm." They partner with groups including the Charter School Growth Fund, The Walton Foundation, Education Quality Outcomes, Christensen Institute, and, of course, the Charles Koch Foundation. Their team rings all the familiar bells. Alicia Bolton, the Program Manager is a product of the Broad Academy and worked in the Obama/Duncan Ed Department before becoming the Director of Caree Education in DC schools.  The Principal, Colleen Miner,  spent three years at McKinsey. Senior Advisor Rachel Wexler lists herself on LinkedIn as a "social impact consultant, change agent, and executive coach; she's worked for Penn Foster Education and Pearson. CEO George Vinton started out as a Client Director for Unipart Expert Practices (consulting firm) before getting a Masters in Education at Harvard and starting Common Group in September of 2020, in San Francisco.

There are a thousand rabbit holes to go down here, but the pattern is already there. The usual reformster social engineering philanthropy along with entrepreneurs, venture capitalists, professional consultants and people who think that someone can be unironically called a Change Agent.

This is the second round of these awards. School districts are allowed to apply, but in fifteen previous winners there are no public schools in sight. There are three initiatives for using digital remote materials to rural schools including the Crowder College initiative to "reshape the narrative about career opportunities in rural America to include a flourishing digital workforce"), a couple of IT programs (one provides BlPOC students a chance to "get real-world experience doing web development and social media marketing for mission-driven organizations), some tech-based interny things, and lots of tech-based distance programs. Lots and lots of computer technology based stuff, much of it connected to a particular geographic area, and some of it looking suspiciously like elevated language wrapped around fairly pedestrian and redundant programs; a couple of these sound suspiciously like "We'll give them some software that lets them look up different careers." 

But then, that's the kind of stuff when you have a pile of money driven by people who don't actually know much at all about K-12 education, and/or are most interested in pushing ideas of their own, and who harbor the underlying bias the public education needs to be hit by "break things and move fast" rather than reinforced and supported, that the solution for educational progress must somehow be wrapped in the entrepreneurial spirit. 

This is just a sample of what's out there, and probably not the most harmful thing that anyone is pushing, but a couple of million dollars represents a pretty large opportunity cost. After all these years, you'd think they would get better at throwing money around, but the signs are not good. 

Burnout and Moral Injury

I never had a name for it. But there was a frustration, something like anger and a kick in the gut, that I was being required to do things as a teacher that I knew were bad practice, not just useless in achieving success, but detrimental to my students.

I first encountered the term "moral injury" in writing about the medical field (in an article no longer accessible on line). But I recognized it immediately from the last many years of my teaching career, and I think it's a useful concept to add to the whole "Is it low morale, or burnout, or what?" It has roots in work with soldiers and veterans and is sometimes connected to PTSD.

Here's a useful definition from Psychology Today:

Moral injury is the social, psychological, and spiritual harm that arises from a betrayal of one’s core values, such as justice, fairness, and loyalty.

Or this one from Syracuse University's Moral Injury Project:

Moral injury is the damage done to one’s conscience or moral compass when that person perpetrates, witnesses, or fails to prevent acts that transgress one’s own moral beliefs, values, or ethical codes of conduct.

There's no question that teachers do not deal with the kind of intense moral stress experienced by a soldier bearing arms in the field of battle. And I am naturally dubious about attempts to give Things That Happen In Life a fancy nametag and scholarly attention. But I still think the concept of moral injury is a useful model for teachers (and others) in understanding why the work they took on with such high hopes and great dreams has begun to make them feel bad.

I used to make my students a promise every fall--that I would never knowingly waste their time and that everything I asked them to do would have a useful purpose (and that I was prepared to explain that purpose). Then came the Big Standardized Test, and I was told that I should emphasize how very important these tests were --except, of course, they weren't. Certainly they weren't important for the students' learning. I was required to break my promise--to waste their time--and expected to lie to them about it, or, alternately, act counter to the directives and leadership of my bosses. Either way, I was going to run counter to core values about my professional role.

With time, there was more. "Here are some 'coaching' books," they said. And there they were--pages and pages of short article excerpts paired with multiple questions. Workbooks that served no purpose except test prep. Days of teaching lost to test prep tests--tests that had no purpose except to target students who might fall on the line between proficient and not, so that we could hammer them with more test prep and maybe get our numbers up. 

And all of this wasn't just a waste of their time (and mine), but it crowded out things that would have been, in my considered professional opinion, useful and worthwhile. Most of my career I worked to become more efficient each year and thereby squeeze just a little more educating into my 178 days, but in the last years in the classroom, I switched to damage control, to minimizing the erosion, no longer asking what I could add, but what I could cut with the least damage (while simultaneously trying to gauge just how insubordinate I could afford to be). 

Moral injury is said to have multiple effects, including depression and "serious distress." My last few years on the job, I was just angry most of the time (just go back and read this blog from the years before I retired and see how permanently pissed off I was--not that I'm all sunshine and cream now, but back then it was on another level). 

We can call it burnout or low morale or moral injury. Burnout suggests that something inside the teacher just couldn't take the pressure, without looking at where the pressure comes from. Low morale suggests that administrators aren't having enough pep rallies. Both of those terms nod to the unique stress these days that come from a nation of people and politicians yammering about how awful teachers are. And all that is before we pile on the issues related to teaching in a pandemic.

But moral injury points to another source of stress--teachers being asked to operate contrary to their professional values. About the only attempt to address that has been to try to get the teaching profession to change its values, to try to crank out new teachers who believe that aligning to the standards so that you can more effectively teach to the test--that's what good teaching is. I'm not sure how effective they've been in selling this new model of teaching values, but it certainly doesn't seem to be a model with great recruiting power, as witnessed by the steady drying up of the teacher pipeline. 

Teaching is not simply some kind of technical job; it has a strong moral component that requires teachers to bring their values to work. When the job, either on the national or building level, requires them not just to leave those values outside the classroom, but to actively violate them, then you get the kind of massive job dissatisfaction we're seeing these days. 

None of this may fix anything, but it may be useful to give it a name.

Study: Test Data Does Not Help Students Raise Test Scores

Today in "Things Teachers Have Been Saying For Twenty Years But Are Now Being Verified By Research," we present Heather Hill, a professor of education at the Harvard Graduate School of Education. 

Hill has recently publicized some of her recent study which somehow combines the obvious with dubious conclusions. Here she was at EdWeek back in 2020:

Question: What activity is done by most teachers in the United States, but has almost no evidence of effectiveness in raising student test scores?

Answer: Analyzing student assessment data.

This practice arose from a simple logic: To improve student outcomes, teachers should study students’ prior test performance, learn what students struggle with, and then adjust the curriculum or offer students remediation where necessary. By addressing the weaknesses revealed by the test results, overall student achievement would improve.

Oh, look--the sun is rising in the East

Well, the "simple logic" was never simple nor logical. Or rather, this is what teachers already do with their own testing. What was actually proposed was that students take a poorly designed test, followed by providing teachers with very little data, much of it bad, in order to raise test scores. Teachers knew pretty much immediately that A) this was not going to work and B) wasn't even that great of a goal.

Hill has some thoughts about why using test data hasn't improved anything. They are not great thoughts, and we get the hint in the very next sentence.

Yet understanding students’ weaknesses is only useful if it changes practice.

Hill is not the only researcher to note the "problem" and mis-diagnose the "solution." Hechinger recently talked to Hill and two other researchers who "explained" that "while data is helpful in pinpointing students’ weaknesses, mistakes and gaps, it doesn’t tell teachers what to do about them."

So here are the issues that these researchers have missed.

1) Raising test scores is a lousy educational goal. There is no research to suggest that raising a student's score will improve their life outcomes. Nor is there any research to suggest that the tests are actual measures of educational quality or actual student achievement. This is a good time to recommend, yet again, Daniel Koretz's The Testing Charade. Testing data continues to be exemplified by that story of the drunk searching for his lost car keys not where they were lost, but under a streetlamp because the light is better there. 

2) The tests yield little useful data. Testocrats love to talk about these tests as if they yield all sorts of rich data. They don't. Their validity--aka their ability to measure what they claim they can measure--is unproven. And multiple choice questions are great for machine correction, but not great for measuring any level of deep understanding.

More importantly from the classroom teacher standpoint, the tests are a black box. Teachers are forbidden to see the questions or the answers, and so the data is just a score. In my own classroom, with my own tests, I would operate much as Hill describes-- give the test, then break down the wrong answers to see exactly what kinds of mistakes students are making. None of that is possible with the Big Standardized Test--from those I would get things like a single score on "Reading Nonfiction." Test manufacturers have whipped all sorts of pretty graphs and colored charts, but the data is still meager and thin.

3) I can't just walk by one of the assumptions of this whole approach is that teachers either can't or won't do their jobs, and so some system of carrots and sticks must be devised to get them to do the work that they signed up for. Hill suggests a picture of teachers who just keep doing the same thing over and over, as if teachers are not motivated or capable of searching out other techniques and approaches. 

Hill is correct in noting that the infamous Data Meetings imposed on teachers by all sorts of data-loving administrators aren't helping. Again, not news. But when you've got bad, thin data that you're supposed to apply toward a pointless goal, what can you expect.

Data-loving testocrats have all along insisted that those darn teachers just don't want to use data properly. But teachers collect, crunch and act on data on a daily basis (though they don't always turn it into numbers and charts). What testocrats seem unwilling to admit, accept, or even see, is that the BS Tests offer little useful data for the process. 

Likewise, the whole "someone should show teachers the better way to teach these things they aren't teaching" always seems to break down when it's time for edu-amateurs to show teachers how to do their jobs better. Hill says that "teachers need to change their approach to address student misunderstandings," as if all teachers use one approach, though she names neither the approach they use or the one she thinks they should use. 

Nevertheless, it appears this earthbound equine will continue to be reflogged. Hill's appearance in Hechinger was prompted by a presentation at the "newly formed" Research Partnership for Professional Learning, yet another group dedicated to fixing teachers. Members include Teaching Lab and TNTP, and sponsors include The Walton Family Foundation and the Bill and Melinda Gates Foundation. So expect what we've had for decades--attempts to explain the failure of beloved high-stakes standardized testing data-driven eduication, blaming everything except the fundamental flaws in the approach.

IN: Barring Teachers From School Boards

On the national level, it really has become death by a thousand tiny cuts for the teaching profession. This proposed amendment in Indiana is by no means the most egregious, but it is just an unnecessary swipe at teachers and school employees.

HB 1130 is mostly about making sure that school boards get a full hammering from the public-- each public member in attendance must be given at least three minutes to speak, while adding restrictions for non-physical meetings--but Senator Jeff Raatz has decided to add another wrinkle.

A section of Indiana law says that an employee of a school district cannot serve on the school board for that district, which is a sensible and common rule. Raatz proposes to add just a few words to that

Where the law now ends with "may not be a member of the governing board of the school corporation," Raatz proposes to add the words "or any other school corporation."

So no teacher (or any other school employee) could serve on any school board anywhere in the state. The reasoning behind this is... well, I don't know. School district employees might know too much about how school districts work? The voters can't be trusted to make the right choice? Teacher board members will be too sympathetic to teachers and their damned unions? Or just one more legislative opportunity to give public schools and the people who work there a big slap in the face.

ICYMI: Goodbye, Rose Edition (2/26)

 My wife's grandmother passed away last week, a ripe and well-aged 90 years old. She was a feisty old broad in the best sense of the word. Salty, sassy and a constant reminder to live your damn life. Her memory will be a blessing.

And now for this week's reading.

Jargon may have turned parents against social and emotional learning

Javeria Salman at Hechinger with a theory about why there's such a disconnect between what people want and the SEL that delivers it.

Who is writing the model bills against CRT?

Jan Resseger traces the roots of the many, many nearly identical gag bills being floated coast to coast

Choosing to end Public Education

Thomas Ultican takes a look inside the new anthology of essays about public education

I Got a Voucher Only to Find No Private School Wanted My Son

If you have not yet checked out the website Public Voices for Public Schools, here's a good post to start with, reminding us that school choice is too often school's choice.

Except nothing breaks itself

When your body is telling you you're carrying too much stress, listen. From Eduhonesty.

Freedom's Just Another Word

Nancy Flanagan looks at how poor "freedom" has been put through the mill.

What’s It Like to Be the Target of A Book Banning Effort? School Librarian Martha Hickson Tells Her Story.

I was a little late coming across this gem, but here's a tale from the front lines of the current book banning debates.

Alabama educators oppose bill that would ban "divisive" concepts from classroom

The Alabama version of this stuff, advertised as a "compromise"

Supreme Court case could open door to religious charter schools

We've looked at this before, but it deserves to be revisited regularly. Matt Barnum, the Chalkbeat reporter we most appreciate here at the Institute, takes a look at what's at stake and what some of the outcomes could be. 

Parenting while teaching: How the classroom stress is coming home with educators

Megan Megansky reporting in Harrsiburg, PA, covers one of the less-covered aspects of the pandemess--teachers who are also parents. Several great quotes, but I'll give you this one

“We really have to get to a point where people in charge stop telling us to take care of ourselves and instead take some of this off of our plates," middle school band director Shanna Danielson said.

Recalls Rebuffed

Sarah Darer Littman with the story of how to fight an attack on board members.

Thank you sir, may I have another?

TC Weber has a good look at the various shenanigans involved in Tennessee Governor Lee's plan to change up how schools are funded (or not).

More students attend Colorado charter schools, but access still isn't equal.

One more story (this time from Erica Meltzer at Chalkbeat) to remind us that the big secret for charter success remains carefully curating your student body. 

Every once in a while I write something and feel as if I've nailed an important point, and then nobody much looks at it. That happened this week, when I pointed out the evidence that some choicers really aren't choicers at all, and they're saying the quiet part pretty loudly these days. Here's the piece.

Tax Credit Scholarships and Education Savings Accounts: A Primer

Tax credit scholarships and education savings accounts often travel together and end up as two sides of the same policy. It's easy to get them kind of confused (here at the Institute we might have suffered from that confusion ourselves on occasion). But here's a quick, simple explainer to help you figure out what particular policy is being pushed in your state.

Education Savings Accounts

ESAs are school vouchers on steroids; call them super-vouchers or neo-vouchers. Where a school voucher is basically a ticket good for admission (or partial admission) to the school of your choice (if they'll accept you), an ESA can be spent on a wide variety of educational services. 

Each state with ESAs has a list of Things You Can Spend The Money On, usually including tuition, hardware and software, books, tests, even transportation. Rather than a ticket to a school, an ESA is a debit card loaded with money that can be spent on educational services. The ESA is typically managed by some third party company responsible for accepting parent applications and dispersing the money.

That third-party company is an important feature, because it means the government is not directly handing parents money to spend on things like, say, private religious schooling. It's a piece of financial juggling familiar to every underage teen who gave an older sibling money to buy them some beer.

The money that funds the ESA most typically comes from the state, which hands over the money that they would have given to the student's home public school. 

But an ESA can also be funded through other sources.

Tax Credit Scholarships

If ESAs are about giving parents choices, TCSs are about giving wealthy taxpayers a chance to duck taxes and fund their favorite private school. TCS typically have a cap, because every dollar given into this program is a tax dollar (or part of a tax dollar, depending on how much credit is given) that the state doesn't get to collect. Corporations or wealthy fund the program instead of paying taxes, so the state will put a limit on just how much tax liability they will allow contributors to duck.

The tax credit money is given to a third party company, which once again gives the state deniability--they didn't funnel the tax dollars to private religious schools because they never actually had the tax dollars in their own government hands. 

Those Third Party Groups

You can see where ESAs and TCSs could meet--right there in the middle. Set up your TSC program to fund your ESA program, and the whole thing can take wings. 

But that's not what often happens. For ESAs, states seem to prefer just one or two large groups managing all the program money. But with TSCs, you may see a plethora of scholarship groups (Pennsylvania has over 200), many of which are attached to one particular school, or one particular "brand" (say, Catholic schools). With a TSC, contributors can often be pretty specific in offering "scholarships" to specific schools or types of schools. 

Cost to taxpayers

Both types of program take from the state's use of taxpayer money to support public education. An ESA typically redirects tax dollars intended for public schools into private hands. Fans of TCS like to argue that it doesn't take any state money, but it blows a hole in state revenue by letting corporations or wealthy individuals pay into the program instead of the state treasury. If the TSC program has a $50 million cap, that represents $50 million in tax dollars that the state will not get, and that has to be made up somehow, either through tax increases or service cuts.

Accountability

Accountability remains an issue for both programs. Some ESA programs don't even require annual audits. Arizona notably found that $700,000 in ESA money had been spent on beauty supplies and clothing. Some have mechanisms for investigating fraud--but only if somebody reports it. Meanwhile, TSC programs like Pennsylvania's EITC program may have no accountability or oversight at all. Where did the money go? What was it spent on? Were the education flavored businesses required to follow any particular rules? You may never know the answers to any of these questions (okay, the answer to the third one is probably "no").

That's the basic basics. Both programs can come under a variety of different names, often trying to work in "freedom" and "scholarship," because "voucher" and "tax dodge" are not very popular with the general public.