When PARCC wrote a press release earlier this month addressing some testing concerns, most commentators focused on this quote:
“High quality assessments go hand-in-hand with high quality instruction based, on high quality standards,” said Laura Slover, the Chief Executive Officer of the PARCC nonprofit. “You cannot have one without the other. The PARCC states see quality assessments as a part of instruction, not a break from instruction.”
It's a noteworthy quote, given its baldfaced admission that testing, curriculum, and the standards are all of a piece. The connection between all the parts is not a surprise, but reformsters rarely divert from the standard story of separate lives for their reformy wares.
But plenty of folks have hit that point, and I don't want you to miss this quote that appears a bit further down the page.
“The PARCC assessment system is a new way of testing that reduces time spent on ‘test prep,’ because the only way to prepare for these more sophisticated assessments is through good teaching and learning all year long," Slover said.
That, in one short sentence, attempts the bank shot of changing the definitions of both test prep and good teaching by throwing them into a big cauldron in which they can smush together.
"Test prep" is just "good teaching," and "good teaching" is that which gets students ready for the test. Which is of course what we usually call "test prep."
Wednesday, August 27, 2014
Law Professor Says Duncan's Actions Un-Constitutional
An upcoming article in the Vanderbilt Law Review argues that the administration's waiver program is both illegal and a very, very bad precedent. University of South Carolina law professor Derek W. Black has written articles about the intersection of federal power and school law before, but none quite as feisty as "Federalizing Education By Waiver." And folks have questioned the legality of Duncan's waivers all along, but this takes that game to a whole new level.
Black opens with one of the most concise summaries of the current reformster wave you'll ever see
Two of the most significant events in the history of public education occurred over the last year. First, after two centuries of local control and variation, states adopted a national curriculum. Second, states changed the way they would evaluate and retain teachers, significantly altering teachers’ most revered right, tenure. Not all states adopted these changes of their own free will. The changes were the result of the United States Secretary of Education exercising unprecedented agency power in the midst of an educational crisis: the impending failure of almost all of the nation’s schools under the
No Child Left Behind Act (NCLB). The Secretary invoked the power to impose new conditions on states in exchange for waiving their obligations under NCLB....As a practical matter, he federalized
education in just a few short months.
This allows the kibbitzing to start immediately in response. Black does not distinguish at all between Common Core Standards and a national curriculum, a distinction without a difference that reformsters have fought hard to maintain. Nor will reformsters care for the assertion that states did not all adopt reform measures of their own free will. But all of that background in the first paragraph of the article is simply setting the stage for Black's main point.
This unilateral action is remarkable not only for education, but from a constitutional balance-of-power perspective. ... Yet, as efficacious as unilateral action through statutory waiver might be, it is unconstitutional absent carefully crafted legislative authority. Secretary Duncan lacked that authority. Thus, the federalization of education through conditional waivers was momentous, but unconstitutional. [emphasis mine]
I should note that all of his material comes heavily laden with footnotes.
There follows a more detailed recap of the Tale of NCLB and Creeping Federalism.
Once upon a time, Congress created NCLB which kept the line on states' rights by making the states accountable for educational results. Scholars called it "cooperative federalism" and it was a new role for the feds, but a limited one. But NCLB was flawed, and as early as 2008, Congress and the President were looking to stop the train. The President proposed a fix in 2010, but Congress was not having it. However, NCLB came with its own magic beans-- the Sec of Ed had the power to waive noncompliance consequences for the states.
The Sec of Ed broke out the magic beans, but he said they will come at a price-- a price, it turns out, remarkably identical to the 2010 proposal ideas. And here's the thing about that 2010 blueprint-- it was proposed as a way to take education in a completely different direction, away from NCLB.
That means that the waiver requirements were decidedly NOT an outgrowth of the underlying legislation, and were in fact meant to bury it, not to save it. That means that waivers took us into a magical new land of Not Actually Legal.
Specifically, it assaults the magical fairies of the Spending Clause in these two ways. 1) You can't use federal money to change the rules that the money is attached to and 2) Congress can't use federally funded programs to coerce states into adopting federal policies. And so it's time to break out the magic wands and zap some naughty federal fairies.
There's not a lot of scholarship about this web of fine legal detail, and so Black sets out to fill in the gap with four very erudite and legally sections of the article. I am going to summarize them in very non-legally ways, and any damage I do to Black's arguments is on me, not him.
PART I: No Changing the Rules
When the feds pass a law, they have to lay out all the rules that do and will apply to that law. You can't pass a law, start folks working under it, and then years later announce, "Oh, yeah, and by the way, we've changed this law about making cheese sandwiches so that it also covers sloppy joes, and also, if you don't go along with us on this, we get to take your car." Also, you can't suddenly say, "We've given my brother-in-law the power to judge your sloppy joes."
Conditions for receiving federal fund must be "unambiguous" and non-coercive. Also, you can't suddenly delegate Congressional authority to an agency of the Executive branch.
There's not a lot of constitutional case law related to waivers, but Black is pretty sure that insufficient notice of waiver conditions as well as "leverage and surprise at the point of waiver" (wasn't that an Alan Parson's Project album?) are problematic. Even more problematic is the issue of an agency of the federal government using waiver conditions to rewrite laws passed by Congress. And then he takes a few pages to explain how these issues should be navigated.
PART II: Using NCLB Waiver To Impose New Policy
If you're going to understand why this was bad, it helps to understand how it happened. The smoking gun for Black is the President's Blueprint for Reauthorizing the Elementary and Secondary Education Act. He is painstakingly specific in this (reading this 51 page article has helped me remember why I'm not a lawyer), but the upshot seems to be this:
The President said quite plainly that his blueprint was meant to erase, replace, and supersede No Child Left Behind. So when the same requirements appear in the waivers, that makes it hard to argue that the waivers are meant to conform with and help preserve NCLB. Put another way, a waiver cannot legitimately be based on replacing the waived law with some other law entirely. It's like those movies where federal agents offer a criminal release from jail only if he'll steal something for them-- it may be cool drama, but it is in fact coercive and ultimately illegal.
PART III: The Constitutional Flaws of NCLB Waivers
The Constitution does not give agencies (executive branch) the power to rewrite laws (legislative branch). They have some limited legal power to do waivery things, but only to the extent that the waivery things are described in the original law. NCLB does not contain any waiver descriptions that match what Arne Duncan has been doing. Duncan has no authority to offer these waivers under the conditions he's set.
PART IV: That Would Be Extraordinarily Bad
If the NCLB waivers are ruled as Constitutional, then we've just extended to an agency of the executive branch the authority to create new laws. This would be bad. Really, unprecedentedly bad.
Yes, regulatory agencies like the EPA often have to make judgments that seem tantamount to creating policy and law, but they still have to make those judgments based on facts and in ways that fit the original regulations. Agencies like the FCC have very broad legislative mandates, but other language and explication actually narrowed their scope considerably. What the ed waivers have done is create a whole new version of ESEA without the country's actual lawmakers ever touching a bit of it.
CONCLUSION
With no more power than the authority to waive noncompliance with NCLB, Secretary Arne Duncan achieved a goal that educational equality advocates had long sought, but never secured: the federalization of public education. His path to the “holy grail” of education, however, was fundamentally flawed. He only reached it by imposing waiver conditions that were neither explicitly nor implicitly authorized by the text of NCLB. Thus, he exceeded his statutory authority and violated the Constitution’s clear notice requirements regarding conditions on federal funds.
States only acceded to these new and unforeseeable terms because their impending non-compliance with NCLB put so much at stake financially, practically, and politically. By the time Secretary Duncan announced the conditions, states were out of options and left in a position where the Secretary could compel them to accept terms that, under most any other circumstances, they would reject. The administration took the states’ vulnerability as an opportunity to unilaterally impose policy that had already failed in Congress. In doing so, the administration unconstitutionally coerced states.
This is fifty-one pages of detailed argument with a mountain of footnotes and a heck of a lot of Constitutional lawyerese. But it is a thorough argument about how the current reformy wave of waiverism is not merely bad policy, but illegal. It is going to be really interesting to see what fuss is kicked up once this article hits the fan. Plenty of folks have been calling the waivers illegal since waivering first began, but now they've got a heavy-duty law professor in a professional journal to back them up. Who would like to start the countdown to lawsuit?
(h/t to Love Light for passing this article along to me)
Black opens with one of the most concise summaries of the current reformster wave you'll ever see
Two of the most significant events in the history of public education occurred over the last year. First, after two centuries of local control and variation, states adopted a national curriculum. Second, states changed the way they would evaluate and retain teachers, significantly altering teachers’ most revered right, tenure. Not all states adopted these changes of their own free will. The changes were the result of the United States Secretary of Education exercising unprecedented agency power in the midst of an educational crisis: the impending failure of almost all of the nation’s schools under the
No Child Left Behind Act (NCLB). The Secretary invoked the power to impose new conditions on states in exchange for waiving their obligations under NCLB....As a practical matter, he federalized
education in just a few short months.
This allows the kibbitzing to start immediately in response. Black does not distinguish at all between Common Core Standards and a national curriculum, a distinction without a difference that reformsters have fought hard to maintain. Nor will reformsters care for the assertion that states did not all adopt reform measures of their own free will. But all of that background in the first paragraph of the article is simply setting the stage for Black's main point.
This unilateral action is remarkable not only for education, but from a constitutional balance-of-power perspective. ... Yet, as efficacious as unilateral action through statutory waiver might be, it is unconstitutional absent carefully crafted legislative authority. Secretary Duncan lacked that authority. Thus, the federalization of education through conditional waivers was momentous, but unconstitutional. [emphasis mine]
I should note that all of his material comes heavily laden with footnotes.
There follows a more detailed recap of the Tale of NCLB and Creeping Federalism.
Once upon a time, Congress created NCLB which kept the line on states' rights by making the states accountable for educational results. Scholars called it "cooperative federalism" and it was a new role for the feds, but a limited one. But NCLB was flawed, and as early as 2008, Congress and the President were looking to stop the train. The President proposed a fix in 2010, but Congress was not having it. However, NCLB came with its own magic beans-- the Sec of Ed had the power to waive noncompliance consequences for the states.
The Sec of Ed broke out the magic beans, but he said they will come at a price-- a price, it turns out, remarkably identical to the 2010 proposal ideas. And here's the thing about that 2010 blueprint-- it was proposed as a way to take education in a completely different direction, away from NCLB.
That means that the waiver requirements were decidedly NOT an outgrowth of the underlying legislation, and were in fact meant to bury it, not to save it. That means that waivers took us into a magical new land of Not Actually Legal.
Specifically, it assaults the magical fairies of the Spending Clause in these two ways. 1) You can't use federal money to change the rules that the money is attached to and 2) Congress can't use federally funded programs to coerce states into adopting federal policies. And so it's time to break out the magic wands and zap some naughty federal fairies.
There's not a lot of scholarship about this web of fine legal detail, and so Black sets out to fill in the gap with four very erudite and legally sections of the article. I am going to summarize them in very non-legally ways, and any damage I do to Black's arguments is on me, not him.
PART I: No Changing the Rules
When the feds pass a law, they have to lay out all the rules that do and will apply to that law. You can't pass a law, start folks working under it, and then years later announce, "Oh, yeah, and by the way, we've changed this law about making cheese sandwiches so that it also covers sloppy joes, and also, if you don't go along with us on this, we get to take your car." Also, you can't suddenly say, "We've given my brother-in-law the power to judge your sloppy joes."
Conditions for receiving federal fund must be "unambiguous" and non-coercive. Also, you can't suddenly delegate Congressional authority to an agency of the Executive branch.
There's not a lot of constitutional case law related to waivers, but Black is pretty sure that insufficient notice of waiver conditions as well as "leverage and surprise at the point of waiver" (wasn't that an Alan Parson's Project album?) are problematic. Even more problematic is the issue of an agency of the federal government using waiver conditions to rewrite laws passed by Congress. And then he takes a few pages to explain how these issues should be navigated.
PART II: Using NCLB Waiver To Impose New Policy
If you're going to understand why this was bad, it helps to understand how it happened. The smoking gun for Black is the President's Blueprint for Reauthorizing the Elementary and Secondary Education Act. He is painstakingly specific in this (reading this 51 page article has helped me remember why I'm not a lawyer), but the upshot seems to be this:
The President said quite plainly that his blueprint was meant to erase, replace, and supersede No Child Left Behind. So when the same requirements appear in the waivers, that makes it hard to argue that the waivers are meant to conform with and help preserve NCLB. Put another way, a waiver cannot legitimately be based on replacing the waived law with some other law entirely. It's like those movies where federal agents offer a criminal release from jail only if he'll steal something for them-- it may be cool drama, but it is in fact coercive and ultimately illegal.
PART III: The Constitutional Flaws of NCLB Waivers
The Constitution does not give agencies (executive branch) the power to rewrite laws (legislative branch). They have some limited legal power to do waivery things, but only to the extent that the waivery things are described in the original law. NCLB does not contain any waiver descriptions that match what Arne Duncan has been doing. Duncan has no authority to offer these waivers under the conditions he's set.
PART IV: That Would Be Extraordinarily Bad
If the NCLB waivers are ruled as Constitutional, then we've just extended to an agency of the executive branch the authority to create new laws. This would be bad. Really, unprecedentedly bad.
Yes, regulatory agencies like the EPA often have to make judgments that seem tantamount to creating policy and law, but they still have to make those judgments based on facts and in ways that fit the original regulations. Agencies like the FCC have very broad legislative mandates, but other language and explication actually narrowed their scope considerably. What the ed waivers have done is create a whole new version of ESEA without the country's actual lawmakers ever touching a bit of it.
CONCLUSION
With no more power than the authority to waive noncompliance with NCLB, Secretary Arne Duncan achieved a goal that educational equality advocates had long sought, but never secured: the federalization of public education. His path to the “holy grail” of education, however, was fundamentally flawed. He only reached it by imposing waiver conditions that were neither explicitly nor implicitly authorized by the text of NCLB. Thus, he exceeded his statutory authority and violated the Constitution’s clear notice requirements regarding conditions on federal funds.
States only acceded to these new and unforeseeable terms because their impending non-compliance with NCLB put so much at stake financially, practically, and politically. By the time Secretary Duncan announced the conditions, states were out of options and left in a position where the Secretary could compel them to accept terms that, under most any other circumstances, they would reject. The administration took the states’ vulnerability as an opportunity to unilaterally impose policy that had already failed in Congress. In doing so, the administration unconstitutionally coerced states.
This is fifty-one pages of detailed argument with a mountain of footnotes and a heck of a lot of Constitutional lawyerese. But it is a thorough argument about how the current reformy wave of waiverism is not merely bad policy, but illegal. It is going to be really interesting to see what fuss is kicked up once this article hits the fan. Plenty of folks have been calling the waivers illegal since waivering first began, but now they've got a heavy-duty law professor in a professional journal to back them up. Who would like to start the countdown to lawsuit?
(h/t to Love Light for passing this article along to me)
Tuesday, August 26, 2014
Four Reformy Problems in a Single Tweet
@NeeravKingsland @CtrEdEcon @smarick @amandaripley How do we measure schools' progress without annual tests?
— Michael Petrilli (@MichaelPetrilli) August 24, 2014
This is the kind of statement that brings me up short. It's neither notably nefarious, nor is it larded with some deliberate obfuscation, and yet it is loaded with so many assumptions.
Measure schools' progress
Is "schools' progress" really a thing? It sounds more sciency than "is the school getting better"? But it has the same problem. Progress is a giant tribble of a word, so fuzzy that one cannot really make out a shape within the furry expanse. Progress from where to where? Become a better connector of the community? Become safer? Develop a stronger arts program, or more successful sports program? Do a better job of creating students who are able to function as self-directed learners? Graduate more students who get into college or get good jobs?
There are so many ways in which a school can progress (and most schools are tying to move in several of them at once. How exactly will standardized test results measure all of those many and varied forms of progress?
Measure schools' progress
Annual standardized tests will only measure one sort of progress-- how well the school is doing at housing students who do well on standardized tests. That is a standard for progress that can be met many ways, including paying close attention to which kind of students you're housing. This truly does measure the school's progress-- but not the progress of students, which would be a nice goal for a school. But we've moved from measuring student achievement to measuring student progress to measuring schools' progress.
Why? Why is measuring school progress useful? Why is it the end goal? It seems a little like checking the oven temperature as a way of determining if the turkey is cooked. Why not just check the turkey?
And that gets us right back to progress. When we talk about great schools, schools or students that are making progress-- what do we mean?
Without annual tests
Is that what we mean? Progress will be defined as "whatever we can measure with a standardized test?" Thomas Newkirk gets to the heart of this:
It all comes down to the parable of the drunk and his keys, an old joke that goes like this: A drunk is fumbling along under a streetlight when a policeman comes up and asks him what he doing. The drunk explains he is looking for his keys. “Do you think you lost them there?” the policeman asks.
“No.But the light is better here.”
Neither of the necessary questions is being asked-- what do we really want to measure, and what would be the best way to measure it?
We
What do you mean "we"? Who is this we? And why do they need to have a measure of schools' progress?
Is it parents? I don't think so-- why would parents care about any plural schools beyond their own singular one? Who is it that needs a national scale everyschool rundown of how things are progressing? When exactly did the US appoint a Grand High Overseer to whom all schools must answer, and whom did we appoint? Is this the US DOE talking? A representative council elected by all the American taxpayers?
I know it's picky as shit to peel apart a simple tweet, but I want to highlight how many unexamined assumptions find their way into the education discussion. If we could start examining them more often, maybe we could start talking about the real concerns that should be on the table.
The Cult of Order
Many, many, many reformsters are members of the Cult of Order.
The Cult of Order believes in blind, unthinking devotion to Order. Everything must be in its proper place. Everything must go according to plan. Everything must be under control.
It is not new to find cult members in education. We all work with a least a couple. Desks must be just so. The surface of the teacher desk must be pristine and orderly enough that bacteria will avoid it and others will either stay back in awe (or experience a near-uncontrollable urge to violate it). Students lose a letter grade for putting their name in the wrong corner of a paper. In high schools, they believe that even seniors would benefit from going class to class in neat and orderly lines.
But reformster members take the Cult of Order to new levels.
They were bothered by the chaos of the crazy-quilt state standards, each different from the rest. They are alarmed at the possibility that individual teachers might be teaching differently from other teachers. Order, predictability, uniformity-- these are qualities to be pursued, not because they are a path to better outcomes of some sort, but because they are in and of themselves desirable outcomes. Standardization and a national curriculum that gets every student in every classroom on the same page at the same time-- this vision is good. Don't ask "Good for what?" To the Cult of Order that's a nonsense question, like asking about the utility of a kiss. For them, controlled, orderly standardization is as beautiful as a sunrise.
The Cult of Order is all about fear-- fear that some sort of dark, menacing chaos lurks just beyond the borders. There's a horrible monster waiting just on the other side of that white picket fence, and the only way to keep it at bay is to make the fence just as neat and orderly as possible.
And yet, we know this is not how the world of human beings works. Human relationships are messy, wobbly, unpredictable, hard to plan. At first flutter of your heart, you cannot know how that story will end. Friendship may grow or wither, and no amount of orderly control can change it. And on the large scale, throughout human history, the dream of perfect order always travels hand in hand with aspirations of totalitarian despots.
It is only in the modern age that a true dream of perfect order seems attainable. The Romans maintained centuries of empire precisely because they developed a system that did not depend on perfect order and standardization, but left the many varied local governments in place. The Romans knew that complete order and control was unattainable. The modern Cult of Order believes it is.
Science also tells us that the Cult of Order is wrong. Chaos theory and information theory and quantum mechanics all tell the same story, The dream of simple linear Newtonian order, where insisting on A always gets you to B-- that dream is unattainable, a failed model that does not reflect how the world works. There is a kind of order in the chaos, but it is more rich and complex than we have ever imagined. More importantly, it is an order that does not respond to nor allow for planning and control.
Reformsters keep asking, "How can we precisely control the aspects of education in order to get the exact results that we want?" This is a oxymoronic nonsense question, like asking "How can I best kick that cute girl in the face in order to get her to love me?"
To try to exert that kind of exacting control over other human beings is not just futile-- it's damaging and destructive. To remodel American public education into the model preferred the Cult of Order is destructive and wrong.
The Cult of Order believes in blind, unthinking devotion to Order. Everything must be in its proper place. Everything must go according to plan. Everything must be under control.
It is not new to find cult members in education. We all work with a least a couple. Desks must be just so. The surface of the teacher desk must be pristine and orderly enough that bacteria will avoid it and others will either stay back in awe (or experience a near-uncontrollable urge to violate it). Students lose a letter grade for putting their name in the wrong corner of a paper. In high schools, they believe that even seniors would benefit from going class to class in neat and orderly lines.
But reformster members take the Cult of Order to new levels.
They were bothered by the chaos of the crazy-quilt state standards, each different from the rest. They are alarmed at the possibility that individual teachers might be teaching differently from other teachers. Order, predictability, uniformity-- these are qualities to be pursued, not because they are a path to better outcomes of some sort, but because they are in and of themselves desirable outcomes. Standardization and a national curriculum that gets every student in every classroom on the same page at the same time-- this vision is good. Don't ask "Good for what?" To the Cult of Order that's a nonsense question, like asking about the utility of a kiss. For them, controlled, orderly standardization is as beautiful as a sunrise.
The Cult of Order is all about fear-- fear that some sort of dark, menacing chaos lurks just beyond the borders. There's a horrible monster waiting just on the other side of that white picket fence, and the only way to keep it at bay is to make the fence just as neat and orderly as possible.
And yet, we know this is not how the world of human beings works. Human relationships are messy, wobbly, unpredictable, hard to plan. At first flutter of your heart, you cannot know how that story will end. Friendship may grow or wither, and no amount of orderly control can change it. And on the large scale, throughout human history, the dream of perfect order always travels hand in hand with aspirations of totalitarian despots.
It is only in the modern age that a true dream of perfect order seems attainable. The Romans maintained centuries of empire precisely because they developed a system that did not depend on perfect order and standardization, but left the many varied local governments in place. The Romans knew that complete order and control was unattainable. The modern Cult of Order believes it is.
Science also tells us that the Cult of Order is wrong. Chaos theory and information theory and quantum mechanics all tell the same story, The dream of simple linear Newtonian order, where insisting on A always gets you to B-- that dream is unattainable, a failed model that does not reflect how the world works. There is a kind of order in the chaos, but it is more rich and complex than we have ever imagined. More importantly, it is an order that does not respond to nor allow for planning and control.
Reformsters keep asking, "How can we precisely control the aspects of education in order to get the exact results that we want?" This is a oxymoronic nonsense question, like asking "How can I best kick that cute girl in the face in order to get her to love me?"
To try to exert that kind of exacting control over other human beings is not just futile-- it's damaging and destructive. To remodel American public education into the model preferred the Cult of Order is destructive and wrong.
Monday, August 25, 2014
ACT Report Finds ACT Really Important
Okay kids. Here's today's lesson in critical thinking (which, as you may have heard, is built right into the heart and sinew of the Common Core).
When a business releases a study showing the importance and effectiveness of that business's product, is there a possibility that the study might be aimed at something other than Telling the Whole Truth?
The folks at ACT must be really sweating these days. Their competitors at the College Board have scored a couple of coups, including A) hiring the well-connected author of the CCSS ELA standards, David Coleman and B) getting state and federal governments to adopt their line of AP products as the official education product of the US government. So the ACT people are in need of some realmPR work.
So they are here to report on the national level of college readiness.
This report provides the ACT folks with a great chance to let us know what they've been up to as theydesperately try to grab some market share work to make US education sweller.
They have, for instance, released ACT Aspire, a test package for assessing student college readiness for grades 3 through 10. It aligns with the ACT College Readiness Standards, so take that, new multi-grade PSAT.
The ACT itself has been "enhanced" by the addition of stats and probability math questions, new questions that require students to "integrate knowledge and ideas across multiple texts, a STEM score, a new writing section enhanced with God-knows-what, and, of course, a computerfied version to take on line, because, computers. They will start throwing in ACT Profile, which appears to be career-suggestion software, so I guess the ACT is now an aptitude test. They also want you to know that they are committed to making sure that the test is continuously monitored for validity and evidence, which may seem obvious, but since David "An Education Is What I Say It Is" Coleman seems inclined to move SATs away from validity and evidence, it's worth the ACT's time to toot their own trumpet of obviousness.
Now, if you think any of that qualifies as a "report" and not "advertising copy," you are not exceeding expectations in your applying of critical thinking. But that's okay, because even Caralee Adams writing about the report at EdWeek, fails critical thinking regarding the meat of this report.
She takes away exactly what she was supposed to-- students who take lots of ACT courses do better in college, therefor, the ACT courses must have prepared them super-well for college.
Why is it that education "research" is so riddled with an inability or unwillingness to distinguish between correlation and causation?
If we were going to design an experiment to determine the effectiveness of a product (say, New Shiny ACT Creme) we would need to first start with subjects who were close to identical. So, let's start with 200 fifteen-year-olds who have similar amounts of acne, give 80 of them New Shiny ACT Creme, 80 of them nothing, and 40 of them mayonnaise. After several months, we'll check to see how the acne looks.
That is not what this "study" has done.
Instead, this study says, "We want the teens who think they have the best complexion to use ACT Creme. We'll let them try it, and in a few months, we'll check to see how they look."
What the ACT report seems to show is not terribly remarkable. Apparently the same high-achieving, hard-working, top-ability, ambitious students who do well in college also seek out the challenging courses in high school. Put another way, the students who value college success and take steps to achieve it in college, also take steps to achieve it in high school. In other news, the sun is expected to rise in the East tomorrow.
I'm not suggesting that the ACT is a scam and terrible waste of time. Like AP courses and the SAT, the ACT clearly has some benefits for some students, and that's a Good Thing.
But we really need to stop looking at these groups as if they are philanthropic organizations trying to make the education world a better place out of the goodness of their highly objective hearts. The ACT is a business, and any "report" they offer about their product is most reasonably viewed as advertising copy, not scientific research.
News organizations do not run stories with headlines like "Researchers Discover Frosted Flakes Are Grrrreat" or "Study Shows Men Who Wear AXE Get Laid More Often." At least not yet. In the meantime, let's keep those critical thinking caps firmly attached.
When a business releases a study showing the importance and effectiveness of that business's product, is there a possibility that the study might be aimed at something other than Telling the Whole Truth?
The folks at ACT must be really sweating these days. Their competitors at the College Board have scored a couple of coups, including A) hiring the well-connected author of the CCSS ELA standards, David Coleman and B) getting state and federal governments to adopt their line of AP products as the official education product of the US government. So the ACT people are in need of some realmPR work.
So they are here to report on the national level of college readiness.
This report provides the ACT folks with a great chance to let us know what they've been up to as they
They have, for instance, released ACT Aspire, a test package for assessing student college readiness for grades 3 through 10. It aligns with the ACT College Readiness Standards, so take that, new multi-grade PSAT.
The ACT itself has been "enhanced" by the addition of stats and probability math questions, new questions that require students to "integrate knowledge and ideas across multiple texts, a STEM score, a new writing section enhanced with God-knows-what, and, of course, a computerfied version to take on line, because, computers. They will start throwing in ACT Profile, which appears to be career-suggestion software, so I guess the ACT is now an aptitude test. They also want you to know that they are committed to making sure that the test is continuously monitored for validity and evidence, which may seem obvious, but since David "An Education Is What I Say It Is" Coleman seems inclined to move SATs away from validity and evidence, it's worth the ACT's time to toot their own trumpet of obviousness.
Now, if you think any of that qualifies as a "report" and not "advertising copy," you are not exceeding expectations in your applying of critical thinking. But that's okay, because even Caralee Adams writing about the report at EdWeek, fails critical thinking regarding the meat of this report.
She takes away exactly what she was supposed to-- students who take lots of ACT courses do better in college, therefor, the ACT courses must have prepared them super-well for college.
Why is it that education "research" is so riddled with an inability or unwillingness to distinguish between correlation and causation?
If we were going to design an experiment to determine the effectiveness of a product (say, New Shiny ACT Creme) we would need to first start with subjects who were close to identical. So, let's start with 200 fifteen-year-olds who have similar amounts of acne, give 80 of them New Shiny ACT Creme, 80 of them nothing, and 40 of them mayonnaise. After several months, we'll check to see how the acne looks.
That is not what this "study" has done.
Instead, this study says, "We want the teens who think they have the best complexion to use ACT Creme. We'll let them try it, and in a few months, we'll check to see how they look."
What the ACT report seems to show is not terribly remarkable. Apparently the same high-achieving, hard-working, top-ability, ambitious students who do well in college also seek out the challenging courses in high school. Put another way, the students who value college success and take steps to achieve it in college, also take steps to achieve it in high school. In other news, the sun is expected to rise in the East tomorrow.
I'm not suggesting that the ACT is a scam and terrible waste of time. Like AP courses and the SAT, the ACT clearly has some benefits for some students, and that's a Good Thing.
But we really need to stop looking at these groups as if they are philanthropic organizations trying to make the education world a better place out of the goodness of their highly objective hearts. The ACT is a business, and any "report" they offer about their product is most reasonably viewed as advertising copy, not scientific research.
News organizations do not run stories with headlines like "Researchers Discover Frosted Flakes Are Grrrreat" or "Study Shows Men Who Wear AXE Get Laid More Often." At least not yet. In the meantime, let's keep those critical thinking caps firmly attached.
Sunday, August 24, 2014
Objectivity Is a Unicorn
Objectivity is a unicorn. It's inspiring to believe in, pretty to create pictures of. Some folks love the idea of objectivity so much that they dress up horses or try their hand at photoshop. But at the end of the day, it doesn't actually exist.
Nevertheless, we continue to enshrine the idea of objectivity in places where it does not exist. Writers have repeatedly reminded us, for instance, that internet search engines are not objective. This article from Michael Kassner at Tech Republic provides a good summary of the basic arguments. When you google something, you do not pull up some objective summary of what the internet contains, but a list that has been weighted by programmers who judge that certain factors should be considered (including a cyber-reading of you and your own proclivities). Google results are just as subjective as if they were compiled by some guy and his buddy making their best guess about what you want and what you should see.
But we really like to believe in Objective Facts, and in particular the notion that certain instruments provide such facts for us.
Some simple instruments do provide some rudimentary objective data. A measuring stick will provide an objective indication of length of an object. But the stick might be measuring meters or feet, and it might include hatchmarks for very small units or not, and if you're using to measure something that's not exactly straight, you will have to make a judgment about how do that. Even the simplest measurement includes subjective judgments. And that's working with objects, not humans.
Complex measures are mostly subjective judgments, whether we are measuring all the articles on the internet and how well they match your search terms, or we are measuring how "college and career ready" your eight year old child might be.
We keep talking about standardized tests as if they are objective measures of... well... anything. They are not. A standardized test is the product of the individual personal judgment of the test writers, who have their ideas about which specific bits of knowledge and skill should be tested and who make their own judgments about what exact tasks would measure those bits. They may claim that research backs up some of their choices, but research is itself the result of individual subjective judgments and choices made by the researchers deciding what to measure, how to measure it, and how to interpret the data they generate. In some cases, such ads David Coleman's reconfiguration of SAT vocabulary, they simply baldly say, "I think we should do this, not that."
Track the elements of standardized tests in any direction you wish; you will soon arrive at human beings making personal subjective judgments about how the test should work.
Reformsters keep talking about the use of testing and data as if it will result in replacing the varied subjective judgments of a teacher with the pure objective results of the testing. No such thing is true. What they seek to do (whether they understand it or not) is replace the judgment of the teacher in the classroom with the subjective judgment of the persons who make the test.
When someone claims "this test is an objective measure of a students language use ability," they are wrong. The test is, at best, a pretty good subjective measure of some tasks that some test-writer guys believe probably indicate language use skill. It is no different from having some person come into the classroom and say, "I'm going to sit and talk to Pat for a couple of hours, and then I'll tell you how good a reader I think he is."
It is not humanly possible to remove subjective judgment from education (or, arguably, anything at all, but let's narrow our focus for now). Never even mind the question of whether or not we should-- it cannot be done.
How do we deal with the inherent subjectivity?
The problem with subjectivity is that it solves problems within a personal framework. We operate in our own little bubble or silo and we define problems and search for solutions based on what we see and know inside that little framework. The trick is to temper subjectivity by expanding that framework. We need to see and know more.
We expand the framework with professional knowledge. We train teachers to understand pedagogy and subject matter so that their instructional judgment is not based on considerations like "What kind of mood am I in" or "Do I like this kid."
We expand the framework with personal relationships. One of the terrible lies of the cult of objectivity is that we can make good decisions about students without knowing a thing about them. Management schools recommend that managers not live in the same communities as their employees, so that relationships are not part of their framework.
Baloney. One of the marks of a good decision is that you can talk about it out loud with the people who are affected by it right there in the room with you. When our framework is expanded to include the people who are part of the choices and the results, our subjective view of the situation is more complete, more useful to people beyond ourselves.
Reformsters have done their damnedest to keep teachers, students, and much of accumulated wisdom of American public ed outside of their framework. This leads them to say naive things (Hey, did you guys know that water is wet) and stupid things (Hey, if you tried hard, I bet you could build a house out of water). It doesn't ever lead them to say objective things.
They claim that their goal is to inject objectivity into the education world, when in fact they're simply trying to impose their own subjective judgments in place of others'. When someone wearing a white hat rides into town on a unicorn, you'd better check to see how the horn is attached.
Nevertheless, we continue to enshrine the idea of objectivity in places where it does not exist. Writers have repeatedly reminded us, for instance, that internet search engines are not objective. This article from Michael Kassner at Tech Republic provides a good summary of the basic arguments. When you google something, you do not pull up some objective summary of what the internet contains, but a list that has been weighted by programmers who judge that certain factors should be considered (including a cyber-reading of you and your own proclivities). Google results are just as subjective as if they were compiled by some guy and his buddy making their best guess about what you want and what you should see.
But we really like to believe in Objective Facts, and in particular the notion that certain instruments provide such facts for us.
Some simple instruments do provide some rudimentary objective data. A measuring stick will provide an objective indication of length of an object. But the stick might be measuring meters or feet, and it might include hatchmarks for very small units or not, and if you're using to measure something that's not exactly straight, you will have to make a judgment about how do that. Even the simplest measurement includes subjective judgments. And that's working with objects, not humans.
Complex measures are mostly subjective judgments, whether we are measuring all the articles on the internet and how well they match your search terms, or we are measuring how "college and career ready" your eight year old child might be.
We keep talking about standardized tests as if they are objective measures of... well... anything. They are not. A standardized test is the product of the individual personal judgment of the test writers, who have their ideas about which specific bits of knowledge and skill should be tested and who make their own judgments about what exact tasks would measure those bits. They may claim that research backs up some of their choices, but research is itself the result of individual subjective judgments and choices made by the researchers deciding what to measure, how to measure it, and how to interpret the data they generate. In some cases, such ads David Coleman's reconfiguration of SAT vocabulary, they simply baldly say, "I think we should do this, not that."
Track the elements of standardized tests in any direction you wish; you will soon arrive at human beings making personal subjective judgments about how the test should work.
Reformsters keep talking about the use of testing and data as if it will result in replacing the varied subjective judgments of a teacher with the pure objective results of the testing. No such thing is true. What they seek to do (whether they understand it or not) is replace the judgment of the teacher in the classroom with the subjective judgment of the persons who make the test.
When someone claims "this test is an objective measure of a students language use ability," they are wrong. The test is, at best, a pretty good subjective measure of some tasks that some test-writer guys believe probably indicate language use skill. It is no different from having some person come into the classroom and say, "I'm going to sit and talk to Pat for a couple of hours, and then I'll tell you how good a reader I think he is."
It is not humanly possible to remove subjective judgment from education (or, arguably, anything at all, but let's narrow our focus for now). Never even mind the question of whether or not we should-- it cannot be done.
How do we deal with the inherent subjectivity?
The problem with subjectivity is that it solves problems within a personal framework. We operate in our own little bubble or silo and we define problems and search for solutions based on what we see and know inside that little framework. The trick is to temper subjectivity by expanding that framework. We need to see and know more.
We expand the framework with professional knowledge. We train teachers to understand pedagogy and subject matter so that their instructional judgment is not based on considerations like "What kind of mood am I in" or "Do I like this kid."
We expand the framework with personal relationships. One of the terrible lies of the cult of objectivity is that we can make good decisions about students without knowing a thing about them. Management schools recommend that managers not live in the same communities as their employees, so that relationships are not part of their framework.
Baloney. One of the marks of a good decision is that you can talk about it out loud with the people who are affected by it right there in the room with you. When our framework is expanded to include the people who are part of the choices and the results, our subjective view of the situation is more complete, more useful to people beyond ourselves.
Reformsters have done their damnedest to keep teachers, students, and much of accumulated wisdom of American public ed outside of their framework. This leads them to say naive things (Hey, did you guys know that water is wet) and stupid things (Hey, if you tried hard, I bet you could build a house out of water). It doesn't ever lead them to say objective things.
They claim that their goal is to inject objectivity into the education world, when in fact they're simply trying to impose their own subjective judgments in place of others'. When someone wearing a white hat rides into town on a unicorn, you'd better check to see how the horn is attached.
Saturday, August 23, 2014
I've Launched at EdWeek
As noted a while ago, I have taken on blogging for Education Week, and the new blog launched a few days ago.
The EdWeek blog will be called "View from the Cheap Seats," and I'll be posting there at least weekly. I even get a cool graphicky thing for the header!I am excited about the potential for reaching a new audience. The first post is about the absence of the teacher voice from education debates. Feel free to stop over, read it, and shower them with demands that I be paid a zillion dollars. Or just enjoy it.
The EdWeek blog will be called "View from the Cheap Seats," and I'll be posting there at least weekly. I even get a cool graphicky thing for the header!I am excited about the potential for reaching a new audience. The first post is about the absence of the teacher voice from education debates. Feel free to stop over, read it, and shower them with demands that I be paid a zillion dollars. Or just enjoy it.
Subscribe to:
Posts (Atom)