How Khan Academy (And Others) Fudged Their Research

Computer tutoring is the hot thing, and the big players have all sorts of sexy research numbers to back them up. Are the numbers bunk? They sure are. 

I'll warn you--this is spun from an article by Laurence Holt, a guy who has worked with NewSchools Venture Fund, Amplify, and, currently, XQ. But most of my readers don't also read Education Next, where the piece appeared in April. But his point is too important to ignore. 

Thanks to COVID, computer-delivered instruction has experienced a boost, from microschool to catch-up interventions. Programs include Khan Academy, i-Ready, Dreambox--but here's the question--

Do they work? In August 2022, three researchers at Khan Academy, a popular math practice website, published the results of a massive, 99-district study of students. It showed an effect size of 0.26 standard deviations (SD)—equivalent to several months of additional schooling—for students who used the program as recommended.

A 2016 Harvard study of DreamBox, a competing mathematics platform, though without the benefit of Sal Khan’s satin voiceover, found an effect size of 0.20 SD for students who used the program as recommended. A 2019 study of i-Ready, a similar program, reported an effect size in math of 0.22 SD—again for students who used the program as recommended. And in 2023 IXL, yet another online mathematics program, reported an effect size of 0.14 SD for students who used the program as designed.

Did you notice a key phrase?

"For students who used the program as recommended."

So how many students is that. Well, Holt checked the footnotes on the Khan Academy study and found the answer--

4.7%

Not a typo. The study threw out over 95% of the results. Holt says that the other programs report similar numbers. 

I suppose the takeaway could be that folks should be trying harder to follow the program as recommended. Of course, it could also be that students who rea motivated to follow the program as recommended are the most ready-to-learn ones. 

But if you hand me a tool that has been made so difficult or unappealing to use that 95% of the "users" say, "No, thanks," I'm going to blame your tool design. 

It's a problem eerily similar to that of ed tech itself, where the pitch to teachers is so often, "If you just change what you do and how you try to do it, this tool will be awesome." When the main problem with your piece of education technology is that it's not designed in such a way that your end users find it actually useful, that is on you. 

In the meantime, schools might want to be a little more careful about how they select these programs. Ed tech companies are interested in marketing, in selling units, and if they have to massage the data to do it--well, the free market. As I've said many times before, the free market does not foster superior quality; the free market fosters superior marketing. And nothing markets in the ed sector like Scientific Evidence Supported by Hard Data.

Always check the data. Always. 

Houston and the End of an Era

Maybe you missed it because VP choices and the assassination attempt sucked up so much oxygen, or maybe you are not a fan of the author, but in mid-July Robert Pondiscio put up a piece about the Houston school takeover, and you should read it.

In "The Last Hurrah," Pondiscio frames the Mike Miles takeover as the last gasp of the big urban takeover model of reform, a model, he suggests, never really worked. 

There are two things that have always set Pondiscio (AEI Fellow and former Fordham guy) apart from the rest of the reformster crowd-- five years spent in an actual classroom, and real experience in journalism. Like his book about Success Academy, this piece clearly reflects a bunch of legwork and some level reportage that will annoy people on all sides.

There's history here. Pondiscio locates Houston in particular and Texas in general as the "Bethlehem" of ed reform. Teach for America. The Texas Miracle that Bush II used to sell No Child Left Behind. Rod Paige. The first Broad Prize for Urban Education, given in 2002, went to Houston. Then in 2013, Houston won it again. Then they stopped handing it out because the judges couldn't find a worthy district. 

That, Pondiscio shows, was reflective of a "gloomy pattern" in the world of "wholesale systemic reform." He looks back at the big marquee names in the field (Michelle Rhee, Joel Klein, et al) and notes that "promising initial gains prove evanescent or quickly plateau" and years later have vanished.

His profile of Mike Miles emphasizes the reformster's "hard-charging" super-self-confident attitude, including the belief that he will succeed where all these others have failed because they just didn't reform hard enough. He also notes, at length, and throughout the piece, the many, many, many objections to Miles both in terms of substance and style.

In particular Pondiscio notes Miles's military approach to management and the micro-management that comes with it. There's a goal here to grab the less capable teachers and force them into a very specific model that, Miles believes, will get results. 

The problem with totalitarian-style management is, as always, that if what the boss says goes and what the worker bees do is comply, then what the boss says had better be right, all the time, and Pondiscio points to some Mile ideas that are "more speculative and stand on shakier empirical ground." And Pondiscio is not sugar-coating the pushback:

A group calling itself Community Voices for Public Education organized protests, petitions, and testimonials from parents and teachers decrying what they saw as the “tired old script from 2012,” and asserting that NES was leaving children “overwhelmed, crying, and complaining.”

This year, “they’re not relating to us at all,” said one student. “This is not fun,” said another. “I feel like I’m in prison.” A former Houston ISD principal said Miles is instilling a “culture of fear.” The district’s largest teachers union mounted a picket to protest the reforms. At a September board meeting, members of the audience set alarms on their phones to go off every four minutes to mock the NES requirement that teachers stop every four minutes to do a multiple response strategy, which conjured up images of timers ringing on a fast-food deep fryer to goad a Pavlovian response from low-skill McTeachers. Nor did it help that the takeover was marked by what one former Houston ISD board member described as a series of unforced errors. Early on, district-made curriculum units were riddled with errors, and poor communications led to national news stories erroneously claiming that Miles was turning school libraries into detention centers for misbehaving students.

For Pondiscio, Miles's problems are a sign of how the context for full-scale reform has changed. "Ed reform embodied youthful energy and do-gooder earnestness" he says, having elsewhere noted more than once that Miles is--well, he's not young (he's my age). Back in the day, reformsters like Rhee were on magazine covers, and Waiting for Superman was a hot ticket, even shown at the Democratic Convention. 

As for Miles himself and his reaction to the criticism of his work-- "I'm old and I don't care."

It's not quite that simple, and Pondiscio crams a lot of nuance into a small space. But he notes that the clock is ticking for Miles, for a variety of reasons, including the super-voucher love of Governor Abbott, Miles's main patron. In one sentence, Pondiscio captures the current drift of reformsterism:

Among many red state Republicans, who often view traditional public schools as irredeemable cauldrons of “woke” indoctrination, ESAs have become the preferred remedy for public education.

For reformsters, particularly the reformsters of the past twenty years, the article may sting with its elegy for a style of reform whose time has passed, and a lack of optimism for Miles's prospect for success in Houston. For defenders of public education, there will be irritation with what Pondiscio has left out: details like how Miles did in Dallas, and details in the presented history that invite debate, particularly in the pictures from reformy days gone by. Regular readers of this space will have repeated urges to say, "Hey, yeah, but, wait--", including a longer litany of Miles's missteps.

But Houston's takeover and the history of big urban school takeovers would require a book. What Pondiscio has condensed into a tiny space shows many serious flaws in the Houston takeover, put in the context of a change in the reformy world. The piece suggests that Miles is doomed to fail, not just because of his plan's faults, but because it is a reform model that has always failed, and if it had ever had a time at all, that time has passed. 

Pondiscio's observation is that support and patience for Houston-style reforms has gone. 

If parents, politicians, philanthropists, and the news media have grown impatient with urban public-school reform, not even waiting for measurable outcomes before pronouncing the entire enterprise a failure—too disruptive, too disrespectful of teachers, too stressful for children—who is the constituency left for big-city reform? Who is left to champion change for the vast majority of children who, even in an emerging era of increasing choice, are likely to remain in urban public schools and struggle to read or do math at a reasonable standard, limiting their future opportunities and life prospects?

There are unexamined questions here--how is it that big city reform lost all of its supposed constituencies? And is it possible to champion change for those students without championing the Superstar CEO Takeover model for improving big city schools?  

Maybe being a) powerful and b) sure that you're right is not the recipe for successful leadership, and maybe that is doubly true when that particular management model has never produced any significant, lasting success. Maybe this idea has lost support because it's a bad idea. And maybe putting a hard-charging high-powered person in control of a school district is not the only way to lift up those students who are falling behind. Maybe this was a tree that was never going to bear fruit, and reform fans should have been cultivating something else entirely.

It's a piece that deserves some attention and discussion, well-crafted, with something for everyone to object to. It's an interesting picture of what's going on in Houston combined with an encapsuled history of one slice of reformster history. You may find plenty to jeer, but I recommend reading it anyway.

OK: State Rescinds Approval For Christian Charter School--For Now

One small addition to the story of St. Isidore, the Catholic cyber charter that was angling to be the nation's first religious charter school.

St. Isidore was approved a little over a year ago, despite the opinion of Republican attorney general Gentner Drummond that it was a Very Bad Idea and also Probably Illegal. There was nothing particularly sneaky about it--the Catholic Diocese of Tulsa was very clear that they intended it to be a full-on Catholic charter school, just as explicitly religious as any parochial school.

The supporters of the school (including, of course Governor Stitt and education dudebro-in-chief Ryan Walters) were banking on the Carson v. Makin decision paved the way for this new move. Meanwhile, GOP opponents like Drummond feared that it would open the door for all manner of religious charter schools (The Satanic Temple was ready to roll), and charter world opponents like Nina Rees of the National Alliance of Public Charter Schools objected because it would challenge the notion that charter schools are public schools, a definition that charteristas have historically preferred to bring up only when it suited them.

But the most immediate result of the state's board overseeing the approval of St. Isidore was, of course, a lawsuit.

Which the Catholic school lost. 

The court was pretty clear. The state's charter act says that charter schools are public schools. The state constitution says that the state must support public schools and may not spend taxpayer money on religious institutions.

Therefor the Establishment Clause and the Oklahoma Constitution apply, and the Free Exercise does not (because, says the court, St. Isidore is not a private entity). Wrote the court:

The State’s establishment of a religious charter school violates Oklahoma statutes Oklahoma Constitution, and the Establishment Clause. St. Isidore cannot justify existence by invoking Free Exercise rights as religious entity. St. Isidore came into existence through its charter with the State and will function as a component of the state’s public school system. The case turns on the State’s contracted-for religious teachings and activities through a new public charter school, not the State’s exclusion of a religious entity.

In other words, charters can’t invoke the rights of a private organization to Free Exercise, because they are not private organizations, but part of the state. Rescind the contract, ordered the court.

That was earlier this summer. So this month, the state's Charter School Board rescinded the contract with St. Isidore as directed by the court. 

However, there is a large "but" with that action.

The vote to rescind (8-0) comes with a condition-- if the state court or the Supreme Court reverses the court's decision, then St. Isidore's contract will be reinstated.

The board had previously passed up two opportunities to rescind the contract. Murray Evans in The Oklahoman (which has been all over this) reports that the delay was to give St. Isidore a chance to procure a stay. The appeal was denied last Monday, so the board took this action to rescind. 

But St. Isidore has indicated that they intend to ask SCOTUS to hear the case, and just at the end of July, the state board voted to join in that appeal. 

Would the current version of the Supreme Court welcome the chance to rule that a Catholic school should have a chance to hoover up taxpayer dollars? Ten years ago, that would have been a ridiculous proposition. Right now, St. Isidore and the Catholic Church think they have a shot. This story isn't over yet.

ICYMI: Teacher Start Up Edition (8/18)

We are back at the home office, and getting ready to roll because this coming week is the beginning of the teacher year in these parts. Will our local schools hit that magical balance between PD sessions and time to work in the room? Cross your fingers and hope. Meanwhile, here's some reading from the week.

Skiatook HS pulls assignment on Christianity after Osage family protests

Chelsea Hicks reports for Osage News about the completely predictable result of Oklahoma's attempt to inject Christianity into classrooms. Bonus: the teacher in question is on an expired emergency certificate and was disqualified from running for sheriff because he embezzled money from a Taco Bueno. Oklahoma, the Florida of the West.

From the Frontlines of the MAGA War on Higher Education: The Ms. Q&A With New College of Florida Professor Amy Reid

It's always a moment when non-education media pick up an education story. Carrie Baker talks to a faculty member about Florida's attempt to create a Hillsdale of the South, for Ms. magazine. Illuminating and alarming.

Sasse’s spending spree: Former UF president channeled millions to GOP allies, secretive contracts

The Independent Alligator broke this story of shamelessly spectacular grift. 

Ghost candidates and Closed Primaries: Another Great Reason to Vote NO on Amendment 1.

Some Florida are worried that school board elections aren't political enough yet. Sue Kingery Woltanski explains their solution, and why it's a bad one.

Let’s Take a Peek Behind the Curtain

Tennessee has always been one of those places where the many threads of reformsterdom come together, and nobody plays connect the dots like TC Weber. What does Penny Schwinn have to do with Florida? He knows. 

Rally opposing proposed Penn Manor Independence Law Center contract planned for Monday

The ILC is Pennsylvania's legal clearing house for creating repressive anti-reading, anti-LGBTQ policies for PA schools. Here's one place with pushback. I recommend clicking through to Lancaster Online just to look at the signs in the photo accompanying the article.

Schools have made slow progress on record absenteeism, with millions of kids still skipping class

The Associated Press is working on a "package" about student absenteeism; this installment includes some interesting reporting on some of what is being tried.

Elmbrook Schools narrowly votes to keep 2 books from being removed

Some good reporting by Rebeccas Klopf of WTMJ from a small district in Wisconsin that really encapsulates the stances in these debates.

Can Vice-Presidential Pick Tim Walz Make Democrats the Education Party Again?

Jeff Bryant looks at Tim Walz in the context of the Democratic Party's less-than-stellar record with public education in the recent past.

The new and radical school voucher push is quietly unwinding two centuries of U.S. education tradition

Douglas Harris, writing for Brookings, breaks down the three major traditions that vouchers threaten-- separation of church and state, anti-discrimination, and public accountability.

Fintech bullies stole your kid's lunch money

Cory Doctorow looks at three players in the school lunch payment racket, who, he says, take as much as sixty cents on the dollar.

Why Are Schools Eliminating Recess, and What Are the Impacts?

Did you think we were done with this issue? Steve Nuzum looks at how this plays out on the ground, and why we need a course correction

Religious Charter School's Controversial Move: Seeking Public Funds to Relocate to Valley Village

Hans Johnson at City Watch LA reports on one more attempt to get public tax dollars to help finance private religious education. 

After six years of low scores for students learning English, Texas educators say it’s the test’s fault

Keaton Peters for the Texas Tribune. Shocking news that a standardized test might not be a perfect objective measure of students.

Charter School Transparency Hearings Starting In Senate Next Month

Betsy DeVos's home state is also home to some spectacular charter school abuses. Now some hearings are going to try to hold some of those shenanigans up to sunlight.

Private school tuition hikes have surged since Oklahoma tax credit began

Ruby Topalian at the Oklahoman with completely unsurprising news. As has been the case in other states, vouchers are a windfall for tuition-hiking schools. Choice? Not so much.

A-F grades for Texas schools blocked again by a judge

Texas reformsters really want their letter grades for public schools, but schools have sued, pointing out that the grades are based on the deeply flawed STAAR test. So far, the court keeps agreeing.

Hysterical Women

Nancy Flanagan points out that education policy is largely informed by the fact that most educators are women, and she suggests that maybe women are getting a chance to have major political influence.

Why has education reform always failed?

Paul Thomas takes a look at Johnathan Haidt's The Anxious Generation and connects that to education reform and the endless search for straightforward solutions to complicated problems.

Check Out New, Short, Informative Guide to Project 2025’s Education Policies

If you're not sick of reading about Project 2025 yet, Jan Resseger has a nice little resource for you.

Yes this matters

Benjamin Riley and the battle against knowledge nihilism.

Bear wanders into California teacher's classroom

Somebody may owe Betsy DeVos an apology.

At Forbes, I have a thought about how to help your child's teachers start the new year. Yes, I'm doing it. 

Join me on substack. It's easy and free forever. 

Petrilli: Can Conservatives Back Both Public And Choice Schools

In the National Review, Michael Petrilli, Thomas Fordham Institute honcho and long-time reformster, poses the argument that folks on the right don't need to choose "between expanding parental options and improving traditional public schools." Instead, he asserts, they "can and should do both."

On the one hand, it's a welcome argument these days when the culture panic crowd has settled on a scorched earth option for public schools. As Kevin Roberts, Heritage Foundation president, put it in his now-delayed-until-after-it-can't-hurt-Trump-election-prospects book, "We don’t merely seek an exit from the system; we are coming for the curriculums and classrooms of the remaining public schools, too." For many on the right, the education policy goal is to obliterate public schools and/or force them to closely resemble the private christianist schools that culture panickers favor. 

Pertrilli is sympathetic to the "let's just give parents the money and be done with it" crowd. 

We’ve inherited a “system” that is 150 years old and is saddled with layers upon layers of previous reforms, regulations, overlapping and calcified bureaucracies, and a massive power imbalance between employees and constituents, thanks to the almighty teachers unions.

Sigh. Reforms and regulations, sure, though it would be nice for Petrilli to acknowledge that for the last forty-ish years, those have mostly come from his own reformster crowd. And I am deeply tired of the old "almighty teachers unions" trope, which is some serious baloney. But his audience thinks it's true, so let's move on. 

Petrilli's point is that conservatives should not be focusing on "school choice" alone, but should embrace an "all of the above" approach. Petrilli dismisses Democrats as "none of the above" because of their "fealty to the unions," which is, again, baloney. Democrats have spent a couple of decades as willing collaborators with the GOP ; if they are "none of the above" it's because they've lost both the ability and authority to pretend to be public education supporters. The nomination of Tim Walz has given them a chance to get on the public education team, but let's wait and see--there's no ball that the Democratic Party can't drop.

Petrilli sits on a practical point here (one that Robert Pondiscio has made repeatedly over the years)-- public schools are a) beloved by many voters, b) not going away, and c) still educate the vast, vast majority of U.S. students. Therefore, folks should care about the quality of public education.

Petrilli then floats some ideas, all while missing the major obstacle to his idea. There are, he claims, many reforms that haven't been tried yet, "including in red states where the teachers unions don't have veto power." I believe the actual number of states where the union doesn't have veto power is fifty. But I do appreciate his backhanded acknowledgement that many states have dis-empowered their teachers unions and still haven't accomplished diddly or squat. It's almost as if the unions are not the real obstacle to progress.

His ideas? Well, there's ending teacher tenure, a dog that will neither hunt nor lie down and die. First of all, there is no teacher tenure. What there is is policy that requires school districts to follow a procedure to get rid of bad teachers. Behind every teacher who shouldn't still have a job is an administrator who isn't doing theirs. 

Tenure and LIFO (Last In First Out) interfere with the reformster model of Genius CEO school management, in which the Genius CEO should be able to fire anyone he wants to for any reason he conceives of, including having become too expensive or so experienced they start getting uppity. 

The theory behind much of education reform has been that all educational shortfalls have been caused by Bad Teachers, and so the focus has been on catching them (with value-added processing of Big Standardized Test scores), firing them, and replacing them with super-duper teachers from the magical super-duper teacher tree. Meanwhile, other teachers would find this new threatening environment inspirational, and they would suddenly unleash the secrets of student achievement that they always had tucked away in their file cabinet, but simply hadn't implemented.

This is a bad model, a non-sensical model, a model that has had a few decades to prove itself, and has not. Nor has Petrilli's other idea-- merit pay has been tried, and there are few signs that it even sort of works, particularly since schools can't do a true merit pay system and also it's often meant as a cost-saving technique (Let's lower base pay and let teachers battle each other to win "merit" bonuses that will make up the difference).

Petrilli also argues against increased pay for teacher masters degrees because those degrees "add no value in terms of quality of teaching and learning" aka they don't make BS Test scores go up. He suggests moving that extra money to create incentives for teachers to move to the toughest schools. 

Petrilli gets well into weeds in his big finish, in which he cites the "wisdom of former Florida governor Jeb Bush" and the golden of state of Florida as if it's a model for all-of-the-above reform and not a state that has steadily degraded and undercut public schools in order to boost charter and private operations, with results that only look great if you squint hard and ignore certain parts (Look at 4th grade scores, but be sure to ignore 8th and 12th grade results). And if you believe that test results are the only true measure of educational excellence.

So, in sum, Petrilli's notion that GOP state leaders should support public education is a good point. What is working against it?

One is that his list is lacking. Part of the reform movement's trouble at this point is that many of its original ideas were aimed primarily at discrediting public education. The remaining core-- use standardized tests to identify and remove bad teachers-- is weak sauce. Even if you believe (wrongly) that the core problem of public education is bad teaching, this is no way to address that issue. 

Beyond bad teachers, the modern reform movement hasn't had a new idea to offer for a couple of decades. 

Petrilli also overlooks a major challenge in the "all of the above approach," a challenge that reformsters and choicers have steadfastly ignored for decades.

You cannot run multiple parallel school systems for the same cost as a single system. 

If you want to pay for public schools and charter schools and vouchers, it is going to cost more money. "School choice" is a misnomer, because school choice has always been available. Choicers are not arguing for school choice--they're arguing for taxpayer funded school choice. That will require more taxpayer funds. 

You can't have six school systems for the price of one. So legislators have been left with a choice. On the one hand, they can tell taxpayers "We think school choice is so important that we are going to raise your taxes to pay for it." On the other hand, they can drain money from the public system to pay for charters and vouchers all while making noises about how the public system is totes overfunded and can spare the money easy peasy. 

I can offer a suggestion for conservatives who want to help public schools improve.

Get over your anti-union selves.

I have a close friend who worked in management in the manufacturing sector for his whole adult life. He liked the union, because he found they made excellent partners in solving problems in the business. They brought a perspective to the problems that was helpful, and while it was sometimes still a challenge to hammer out agreements, the resulting policies were solid and useful. 

Union leaders can be excellent points of contact for management looking to communicate with the staff and coming up with a meaningful list of problems and actionable solutions. School districts can work like that, too. I know that places like New York City and Chicago are tangled political messes, but there are over 13,000 school districts in the country. I often suspect that much over-heated rhetoric about schools and unions is the direct result of most pundits being located in New York and DC. 

It's baffling that so much rhetoric about teachers and unions seems based on the assumption that teachers and their union leaders got into education because they hate children and want to ruin their lives. Conservatives sincerely interested in making schools work a little better in this country would do well to assume that teachers are there on the ground in classrooms because they are committed to education. Conservatives interested in using schools as political fodder are never going to be "all of the above."

Let's Ban Cellphones (But...)

Getting cellphones under control in the classroom is a fabulous idea. But there are a few real obstacles to overcome on that path.

First and foremost will be parental objections. Nobody likes having students carry cellphones more than their parents.

For parents who balk at ever letting their children out of their sight or touch or control, cellphones are just the thing to make sure that contact is maintained and that you can check up in at any moment. In all fairness, it's not just the helicopter parents. Families with three jobs and one car, families with big logistical challenges, families dealing with emergent situations--they all benefit from having the child just a click away.

And nowadays, the specter of various school crises are part of the picture. What if there's a school shooter and my child doesn't have a phone? What if Something Is Going On and the school doesn't contact me quickly enough?

Parents are likely to push back hard--harder than the students--against cellphone restrictions.

At the other end of the issue is the actual specific issue of implementing the policy in the classroom.

There are lots of great ideas out there for collecting and managing all the cellphones, but there is still going to come that moment--

You are teaching class. You see that a student still has their phone. You ask them to hand it in (put it in the box, whatever). They say no. Now what do you do?

Stop class to have a battle of wills? Shrug and let it go? Send the student out of the room? Call the office? Call home later? This is different from a simple classroom management issue of focus, of trying to get the student attention directed at what they are supposed to do rather than what they are not. It's not simply a behavior thing--it's a moment in which you are trying to confiscate (not just put away) an object (a very expensive object) that they have in their possession.

Like so many other policies, this one dies the moment building administration says "Well, what do you want us to do about it?"

There is a huge amount of room, a Grand Canyon's amount, to argue about just how dangerous or destabilizing cellphones actually are for the mental health of young people. But I think you'd find general agreement that teaching would be a bit easier if students were not attached to their phones all day.

But cellphone bans run the risk of being one more of those moments when leaders say, "Well, we will solve this social issue by passing a law or policy and then just let teachers take care of it." Done poorly, it's one more unfunded mandate for education, except instead of sucking up money, it will suck up time and attention. 

Can it be done? Sure. Should it be done? I say yes. By all means, pass the policies/laws/rules/edicts. Just stick around long enough to help the people who are going to have to implement. It's a benefit to teachers to have cellphone restrictions as an institutional policy instead of their own personal preference ("This is not just my rule; this is the school's rule"), but only if the institution caries the weight of making the policy. Otherwise it's just one more damn thing that teachers have to take care of.

How To Help Students Write About Theme

In high school ELA class, the theme essay remains one of the great staples of the field. And yet, students are too often so bad at it. They're supposed to be exercises in analysis and critical thinking, and yet they often turn out to be mind-numbingly dull. Let me share one simple shift in approach that helped me help students be better.

Tell them that a theme is a statement, not a word. 

Here's the problem with one-word "themes." Let's say Pat decides to write about the theme of death in Hamlet. Pat then collects a list of quotes that mention or allude to death. Then Pat turns that list into an essay, but it's an essay that is a walking tour of the play. "Over here, we see the word 'death,' and on your left, you'll see 'shuffle off this mortal coil' which is also a reference to death."

The one word theme essay too easily descends into a sentence-ified list, a catalog of references that does not actually say anything other than "here's that word." 

This kind of essay can pretend to be about something by offering some analysis in the listing, such as pointing out that when Hamlet accuses his friends of trying to play him like a pipe, he is really talking about death, a point that would require enough pretzel logic that it would give the appearance of the student author really Doing Something. 

But a list is not an analysis of a theme in a literary work. And, "In Hamlet, William Shakespeare talks a whole lot about death" is not a useful thesis statement for a student essay. In other words, do not mistake a topic for a theme.

In my class, a theme requires a sentence. Lord knows, there's a still a wide range of possible quality. A Hamlet theme paper could be built around "In the course of Shakespeare's Hamlet, the main character moves from anger and fear of death to acceptance" or "Death sucks." A legit theme can talk about the writer's technique or it can talk about the idea embedded in the work or it can wrestle with an observation about how the world works.

But what a theme gives the writer that a topic does not is something to prove, an idea for which one must marshal evidence beyond simply "proving" that the topic is included in the work. So many "theme" papers turn into aimless slogs because the student has centered on a topic rather than a theme. While my explanation of theme is very reductive, it also was a big help in giving my students a quick, simple way to determine whether or not they were on the right track or to diagnose why their essay felt boring and pointless to write.