Russ Walsh is an expert in reading instruction, a blogger, and (as near as I can tell) a gentleman. A recent post of his is, for my money, one of his most important ones because it collects some research and clear thinking to remind us of one of the great truths of both reading instruction in particular and education itself in general.
The post sets out to take a more nuanced look at text complexity, leaning particularly on the work of Lauren Anderson and Jamy Stillman, (Over)Simplifying Complexity: Interrogating the Press for a more Complex Text.
Both Russ's post and the original article are well worth reading in their entirety; I'm going to oversimplify them here because that's how I roll.
First, Anderson and Stillman re-support what teachers and other humans with common sense already know-- that giving a student a text above her frustration level does not actually help anything, at all. But there's more than that. Writes Walsh
They were increasingly aware that they needed to revise their definition of text complexity to include the context of the reading situation, the background knowledge and skills of the students and the reading instruction goals.
In other words, the level of challenge in any text is not something that exists as a discrete quality, separate from all others. Text difficulty (or complexity or level or whatever other name tag you want to put on these various measures) is not an objective immutable quality. How challenging a text is depends on context, on whose hands are holding it, on what purpose has been attached to it.
Instruction-- the directions and pedagogy that a teacher attaches to the text-- can change the level of challenge. If I hand first graders a copy of War and Peace and tell them to tell me how many pages or how many chapters are in the book, there is only a little bit of challenge there. If I hand seniors a copy of Green Eggs and Ham, and assign a paper using the book as basis for an analysis of social pressures on the individual as experienced in a post-agrarian society resisting the imperialism of other oppressive cultures, it is now a highly challenging text.
I have been a voracious reader for most of my life (miraculous, considering my parents failed to give me the benefit of high quality pre-pre-K when I was three). Early on, I fell in love with dinosaurs and devoured everything by Roy Chapman Andrews. When I had run out of kid books about dinosaurs, I moved on to grown-up books that were, technically, way above my reading level. But at that point I knew an awful lot about dinosaurs, so between the background knowledge I already had and my high degree of interest and motivation, I managed. On the other hand, the first book I was ever unable to finish was the classic Black Beauty, a stirring tale of some horse who does something or other and then zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz. Horses-- I neither knew nor cared about anything horsey. I returned to that book as late as 8th or 9th grade but still found that I was easily distracted from it by, well, watching grass grow would do it. It was like Black Beauty was surrounded by a special impenetrable anti-reading forcefield that would push my eyes off in any direction. Still one of the biggest textual challenges I have ever faced, and I've read Moby Dick.
We can play this anecdotal game all day. Whenever I teach seniors, I always teach them Macbeth, no matter what level students they are. But how I teach, what I teach about it, what I expect them to get out of it, what I assess them for-- that varies widely depending on the students.
Bottom line-- I cannot assess the challenge level of any reading material as a specific, objective quality in and of itself. I can do broad strokes (I feel comfortable saying that Macbeth is more challenging than Green Eggs and Ham), but the real classroom challenge of a work comes down to the relationship between the specific work, the specific students, and the pedagogical approach and techniques of the teacher.
The notion of reading difficulty as some static objective isolated quality is a common mental mistake of the reformsters, and it completely misses the importance of relationships. Current education policy is so off track that it qualifies as both necessary and radical to say that relationships matter.
Yet policy is built on ignoring relationships. Teachers are evaluated in a manner that suggests that a teacher's quality and effectiveness are somehow static, absolute, objective, isolated qualities that exist outside of any context, background or purpose. It's like insisting that if a man is a Good Husband, he will be a Good Husband for any woman selected at random from any place, age or location in the world.
Context matters. Background matters. Purpose matters. And relationships matter most of all. Relationships between students and text, students and teachers, students and each other. The fact that we don't have a handy lexile score or quality index or piece of inanely-generated "data" to measure relationships does not mean they aren't important. That's true for reading and for everything else in education as well.