My blog-colleague Daniel Katz reported a curious follow-up to his recent post about the Pearson-Tracx-Social media monitoring flap-- a piece of spambot advertising for Tracx that popped up in his comments. "Hilarious," I thought. "A company spams a blog that is actually holding it up as an example of bad behavior."
But lo and behold, this afternoon I find that I have two of the identical message on one of my own posts about Pearson's Big Brotherly behavior.
Tracx offers a unified, enterprise-scale, social media management platform. We help brands and organizations from around the world listen and learn about issues related to their products and services so that they can provide a better customer experience and reach new audiences. To learn more about Tracx visit http://www.tracx.com #customerexperience #betterservice #bettersupport #betterproducts #engagingnewaudiences
The comment was "signed" by Benjamin Foley. Now put on your water-wings, boys and girls, as we head down the Tracx rabbit hole.
Turns out that Ben Foley is a person. In fact, he's the person who writes for the blog Making Tracx, all about the awesomeness of Tracx as a means of stalking customers on the interwebs. Come on! Let's learn more!
Meet Ben Foley
Foley's LinkedIn account announces that he is "a creative, results-oriented, energetic and highly motivated marketing leader. " Now, Ben is a young fella-- he's only been at his current post at Tracx since July of 2014, where he does cool things like leading "content creation activities (blog posts, social posts, thought leadership pieces/whitepapers)." Before that he spent nine months as a Tracx sales associate. Before that he was an intern at the Concord, MA, district courthouse (4 months). Before that four months at Vector Marketing, before that four months at the Nauset surf shop (summer of 2010), and before that, a little over two years as a grocery manager at Chatham Village Market in Chatham, MA.
Ben is a 2013 graduate of St. Lawrence University where-- well, I'll just let him tell you in his special prose style:
His relentless analytical interest in the forces that drive people’s emotions and behaviors, both at the individual and group level, resulted in his obtainment of a B.S. in Psychology from St. Lawrence University. Since then Ben has applied his passion for people, their motivations, and their interests to the social media analytics industry by influencing strategic vision and executing multifaceted marketing campaigns.
I don't want to indulge in too much mockation of Young Ben; I remember the days after my obtainment of my degreeification, and the deep pleasure I took in spouting college level gibberish. But this is a man who has clearly found a home in an industry that rewards garblizationizing. I mean, here's another one of his achievements as marketing coordinator:
Drives multichannel lead generation activities which have yielded significant YOY increase in growth, coupled with a YOY increase in marketing generated pipeline.
This is the guy that Tracx has pushing Tracx out into the world and representing their consummate social mediazation skills. I'm feeling a little YOY myself.
What does he say about Tracx?
I happened to land on this blog post--Out of The Dark Ages: The Rise of Social Media Sentiment Analysis, Part 2-- The Renaissance.
Major irony alert: This is all about how media monitoring software can successfully read the sentiment of a post. In other words, the bots can tell whether your post is filled with love or deeply infusified with hatred and anger-- admittedly useful for companies to know. The post is from 2014, and so one would think that this wonderful software would be able to tell the difference between a blog post that says, "Shame on you, you terrible stalking big brothery corporate stooges" and one that says, "I love being stalked on line. Please tell me more about how I can be stalked more effectively." But apparently that feature is not yet available.
The latest post on the blog reports breathlessly that Edison Partners has featured Tracx CEO Eran Gilad because he's awesome. Eran first met the Tracx team when a friend asked him to validate their business plan, which I guess is how you "meet cute" in the tech world.
Eran has some business background beyond grocery management.
He also spent five years in the military and worked with "the Nordics." His recommended reading for every executive is Siddhartha. And -- make of this what you will-- if given the chance to have a super-power, he would choose the power to cancel everyone else's super power.
The blog also trumpets that Tracx ("the global leader in social listening and engagement platforms for Fortune 1000 companies") has integrated image and text analytics, the better to figure out what we're all up to. For this post, Young Ben throws in another description of the Tracx brand:
Tuesday, March 17, 2015
Monday, March 16, 2015
Spellings Remains Steadfastly Wrong
US News ran another of its nifty Debate Club features on Monday, this time tackling the question, "Does No Child Left Behind's Testing Regime Work?"
Arguing "no" were the presidents of America's two large teachers' unions. Sticking up for NCLB's test-and-punish regime were Cheryl Oldham from the US Chamber of Commerce, and Margaret Spellings, George W. Bush's secretary of education.
Spellings uses her time at the debate podium to demonstrate that she remains steadfastly devoted to the same bad policy ideas that she promoted back in the day, still without support. But her piece lets her tick off all the standard bad arguments for keeping NCLB on its same doomed path of educational destruction.
The success of every student in reading and doing math on grade level is vital to the future success of our nation.
That's the very first sentence, and while it's a piece of the NCLB canon, after all these years, there still isn't a lick of proof that it's true. It certainly wasn't true in the past, where large chunks of the US population could not do either of those things and yet the country still did pretty well. She may well want to argue that times have changed, and I wouldn't disagree-- but there's still no proof that there's any linkage between eight-year-olds who read on grade level and national success (and what do we mean by "success" anyway).
Results from the National Assessment of Educational Progress, known as the Nation's Report Card, show that No Child Left Behind has added to the focus on poor and minority students and resulted in increases in their achievement.
Lots of smart people have looked at the same data and failed to see the improvement that Spellings likes to tout.
Prior to the federal requirement for annual assessments that was instituted in 2002, few states had assessments in place. That means that few students had the necessary information to truly meet the needs of each and every child.
Because....? Teachers were incapable of evaluating student progress? No educational measure known to man could possibly be as awesome as a big fat federal test?
Annual, comparable, valid, reliable statewide assessments give educators and policymakers the ability to focus resources on problem areas, find strategies that work and reward results.
After an over a decade, educators and policymakers have not managed to do any of those things (unless, by "focus resources" you mean "unleash privatizers upon"). I have a standing offer for any reformster to name a single pedagogical strategy that has been discovered by the testing program and then scaled up into schools across the nation. The closest thing we have is fairly widespread adoption of test prep techniques, but all those do is teach students how to do better on tests. Is test-taking our nation's educational aspiration?
Spellings also likes the tale of how, pre-NCLB, nobody knew anything about how schools that served poor and minority students were under-served, under-resourced, and under-funded. Once test results came out, minority children could no longer fall through the cracks. Perhaps you remember that year that state and federal government unleashed a wave of financial support, delivering on the promise that our poorest schools would be funded just as well as our richest ones. Oh, wait. That never actually happened.
Did (and do) some of our nation's schools do a terrible job of serving poor and minority students? Absolutely. But NCLB 1) did not find troubled schools that nobody ever knew were troubled nor 2) lead to a redirection of resources to those schools.
According to the Nation’s Report Card, Hispanic and African-American nine-year-olds grew by two grade levels in reading between 1999 and 2008.
Did they? What was the rate of improvement prior to 1999? And why are we counting from 1999 when NCLB didn't take effect until 2003? How much of the wonderful gainage she cites came before NCLB even had a chance to affect student results?
Our main concern must be students. Timely and transparent reporting of data is the only way to keep the focus where it belongs, on increasing student achievement. How else will we know whether they are prepared for college or a good job after high school? How else will we ensure that they aren’t being pushed through the system by those who were elected to ensure they received the best possible education and opportunity for the future?
"Increased student achievement" simply means "higher standardized test scores." And all those "How else" questions? Standardized test scores don't provide answers to any of them. This is like declaring "We must keep polishing our bicycles. How else will we know if our vest still has no sleeves?"
The Texas Senate Education Committee recently unanimously approved legislation that would let high school kids receive their diploma without having demonstrated that they have the basic skills and knowledge to be successful in attending college, work or in life.
Did anybody know how to measure those things with a test? Any proof that such a test exists? Because I'd bet that if any such test exists, it is stapled to a Yeti riding on the back of a centaur.
The argument for it is that nearly 30,000 seniors will be unable to pass the exams and therefore will be limited in their future opportunities. But when we eliminate the requirement that assesses their college and job readiness, how successful can we expect their future opportunities to be? It is a cruel trick to suggest they are ready for life when the data show otherwise.
It is also a cruel trick to insist that you know whether they will be successful or not when there is in fact no proof, evidence, research or tea leaf reading to support the idea that the single narrow test of some math and reading skills is an accurate predictor of a child's future.
Also, what was the plan for those 30K seniors. They were going to flunk high school and then.... what? Not take the GED (nobody is passing the new Pearsonized GED these days). Take an entire senior year over again just to take a single test?
The question we should be asking is why the same student who cannot pass a ninth or tenth grade level test is receiving passing grades in the classroom.
Orrrr.... we could ask why a test claims a passing student is actually failing. If a large number of students fail a test in my class, particularly if all other indicators show they're doing well with the material, then I don't look at the students-- I look at the test.
We must use this opportunity to move education forward and not dilute the progress that has been made.
The "we can't turn back and waste our accomplishments so far" argument is special because it is an argument used to oppose NCLB back in the day and Common Core more recently. But somehow back then the reformsters thought that new and awesome things were worth a little chaos and disorder. Now suddenly they are huge fans of inertia. It should not be news to anybody that when you are doing something that doesn't work, you should think about not doing it any more.
Look, some of these would be great things to say if they represented reality. But the standardized test does not become an accurate measure of a student's entire life prospects just because you say so, and while it would be nice if the test results were used to improve education for underserved students, we've been at this for over a decade and it hasn't happened yet.
Spelling's paean to NCLB testing is a news broadcast from an alternate universe. Read the pieces by Garcia and Weingarten instead.
Arguing "no" were the presidents of America's two large teachers' unions. Sticking up for NCLB's test-and-punish regime were Cheryl Oldham from the US Chamber of Commerce, and Margaret Spellings, George W. Bush's secretary of education.
Spellings uses her time at the debate podium to demonstrate that she remains steadfastly devoted to the same bad policy ideas that she promoted back in the day, still without support. But her piece lets her tick off all the standard bad arguments for keeping NCLB on its same doomed path of educational destruction.
The success of every student in reading and doing math on grade level is vital to the future success of our nation.
That's the very first sentence, and while it's a piece of the NCLB canon, after all these years, there still isn't a lick of proof that it's true. It certainly wasn't true in the past, where large chunks of the US population could not do either of those things and yet the country still did pretty well. She may well want to argue that times have changed, and I wouldn't disagree-- but there's still no proof that there's any linkage between eight-year-olds who read on grade level and national success (and what do we mean by "success" anyway).
Results from the National Assessment of Educational Progress, known as the Nation's Report Card, show that No Child Left Behind has added to the focus on poor and minority students and resulted in increases in their achievement.
Lots of smart people have looked at the same data and failed to see the improvement that Spellings likes to tout.
Prior to the federal requirement for annual assessments that was instituted in 2002, few states had assessments in place. That means that few students had the necessary information to truly meet the needs of each and every child.
Because....? Teachers were incapable of evaluating student progress? No educational measure known to man could possibly be as awesome as a big fat federal test?
Annual, comparable, valid, reliable statewide assessments give educators and policymakers the ability to focus resources on problem areas, find strategies that work and reward results.
After an over a decade, educators and policymakers have not managed to do any of those things (unless, by "focus resources" you mean "unleash privatizers upon"). I have a standing offer for any reformster to name a single pedagogical strategy that has been discovered by the testing program and then scaled up into schools across the nation. The closest thing we have is fairly widespread adoption of test prep techniques, but all those do is teach students how to do better on tests. Is test-taking our nation's educational aspiration?
Spellings also likes the tale of how, pre-NCLB, nobody knew anything about how schools that served poor and minority students were under-served, under-resourced, and under-funded. Once test results came out, minority children could no longer fall through the cracks. Perhaps you remember that year that state and federal government unleashed a wave of financial support, delivering on the promise that our poorest schools would be funded just as well as our richest ones. Oh, wait. That never actually happened.
Did (and do) some of our nation's schools do a terrible job of serving poor and minority students? Absolutely. But NCLB 1) did not find troubled schools that nobody ever knew were troubled nor 2) lead to a redirection of resources to those schools.
According to the Nation’s Report Card, Hispanic and African-American nine-year-olds grew by two grade levels in reading between 1999 and 2008.
Did they? What was the rate of improvement prior to 1999? And why are we counting from 1999 when NCLB didn't take effect until 2003? How much of the wonderful gainage she cites came before NCLB even had a chance to affect student results?
Our main concern must be students. Timely and transparent reporting of data is the only way to keep the focus where it belongs, on increasing student achievement. How else will we know whether they are prepared for college or a good job after high school? How else will we ensure that they aren’t being pushed through the system by those who were elected to ensure they received the best possible education and opportunity for the future?
"Increased student achievement" simply means "higher standardized test scores." And all those "How else" questions? Standardized test scores don't provide answers to any of them. This is like declaring "We must keep polishing our bicycles. How else will we know if our vest still has no sleeves?"
The Texas Senate Education Committee recently unanimously approved legislation that would let high school kids receive their diploma without having demonstrated that they have the basic skills and knowledge to be successful in attending college, work or in life.
Did anybody know how to measure those things with a test? Any proof that such a test exists? Because I'd bet that if any such test exists, it is stapled to a Yeti riding on the back of a centaur.
The argument for it is that nearly 30,000 seniors will be unable to pass the exams and therefore will be limited in their future opportunities. But when we eliminate the requirement that assesses their college and job readiness, how successful can we expect their future opportunities to be? It is a cruel trick to suggest they are ready for life when the data show otherwise.
It is also a cruel trick to insist that you know whether they will be successful or not when there is in fact no proof, evidence, research or tea leaf reading to support the idea that the single narrow test of some math and reading skills is an accurate predictor of a child's future.
Also, what was the plan for those 30K seniors. They were going to flunk high school and then.... what? Not take the GED (nobody is passing the new Pearsonized GED these days). Take an entire senior year over again just to take a single test?
The question we should be asking is why the same student who cannot pass a ninth or tenth grade level test is receiving passing grades in the classroom.
Orrrr.... we could ask why a test claims a passing student is actually failing. If a large number of students fail a test in my class, particularly if all other indicators show they're doing well with the material, then I don't look at the students-- I look at the test.
We must use this opportunity to move education forward and not dilute the progress that has been made.
The "we can't turn back and waste our accomplishments so far" argument is special because it is an argument used to oppose NCLB back in the day and Common Core more recently. But somehow back then the reformsters thought that new and awesome things were worth a little chaos and disorder. Now suddenly they are huge fans of inertia. It should not be news to anybody that when you are doing something that doesn't work, you should think about not doing it any more.
Look, some of these would be great things to say if they represented reality. But the standardized test does not become an accurate measure of a student's entire life prospects just because you say so, and while it would be nice if the test results were used to improve education for underserved students, we've been at this for over a decade and it hasn't happened yet.
Spelling's paean to NCLB testing is a news broadcast from an alternate universe. Read the pieces by Garcia and Weingarten instead.
Arne's Dumb Expectations
Andrea Mitchell, always a reliable amplifier of administration PR, gave Arne Duncan yet another opportunity to try to peddle his wares recently.
Arne wanted to stand up for Common Core in the face of the GOP pre-pre-primary beat-em-up on the standards. As reported by Ben Kamisar at The Hill, Duncan is framing his pitch with one of his favorite spins-- it's all about the expectations.
“When you dumb down expectations to make politicians look good, that's one of the most insidious things that happens.”
Of course, another insidious thing that happens is when politicians raise expectations to make themselves look good, but don't actually deliver any of the support needed to meet those expectations.
It's also insidious when politicians raise expectations and back them up by holding other people responsible for meeting them. Imagine how different education would reform would play out if we just changed half of the following sentence. Instead of
Where we find failing schools and students, we must hold teachers and school districts responsible for their failure to properly teach those students
we could instead say
Where we find failing schools and students, we must hold politicians responsible for their failure to properly support those schools with needed resources.
Duncan's belief in the magic of expectations is well-documented. It just isn't well-founded in reality. High expectations are great-- when realistic. High expectations are great-- when they are applied to the people and government agencies that are supposed to provide the resources needed to meet those expectations.
But among the many things that Duncan fails to grasp is the result not of high expectation, but the placement of blame and punishment when those expectations are not met.
When a child fails to meet expectations that were set unreasonably high by politicians, who should be held responsible?
When a child struggles to meet expectations because her school and community are starved for the resources needed to help her achieve, who should be held responsible?
Duncan continues to fail to understand the system that he has perpetuated. Politicians did not lower expectations "to look good" under NCLB-- they did it to avoid losing badly-needed money for poor, struggling schools. Duncan continues to ask as if public education has twisted itself into a test-obsessed mis-directed pretzel on some sort of random whim, and not as a predictable and not-irrational response to the policy of test-and-punish pursued with gusto by this administration.
Arne wanted to stand up for Common Core in the face of the GOP pre-pre-primary beat-em-up on the standards. As reported by Ben Kamisar at The Hill, Duncan is framing his pitch with one of his favorite spins-- it's all about the expectations.
“When you dumb down expectations to make politicians look good, that's one of the most insidious things that happens.”
Of course, another insidious thing that happens is when politicians raise expectations to make themselves look good, but don't actually deliver any of the support needed to meet those expectations.
It's also insidious when politicians raise expectations and back them up by holding other people responsible for meeting them. Imagine how different education would reform would play out if we just changed half of the following sentence. Instead of
Where we find failing schools and students, we must hold teachers and school districts responsible for their failure to properly teach those students
we could instead say
Where we find failing schools and students, we must hold politicians responsible for their failure to properly support those schools with needed resources.
Duncan's belief in the magic of expectations is well-documented. It just isn't well-founded in reality. High expectations are great-- when realistic. High expectations are great-- when they are applied to the people and government agencies that are supposed to provide the resources needed to meet those expectations.
But among the many things that Duncan fails to grasp is the result not of high expectation, but the placement of blame and punishment when those expectations are not met.
When a child fails to meet expectations that were set unreasonably high by politicians, who should be held responsible?
When a child struggles to meet expectations because her school and community are starved for the resources needed to help her achieve, who should be held responsible?
Duncan continues to fail to understand the system that he has perpetuated. Politicians did not lower expectations "to look good" under NCLB-- they did it to avoid losing badly-needed money for poor, struggling schools. Duncan continues to ask as if public education has twisted itself into a test-obsessed mis-directed pretzel on some sort of random whim, and not as a predictable and not-irrational response to the policy of test-and-punish pursued with gusto by this administration.
NM: Defending the Test
Reader Terry Ward drew my attention to a part of the New Mexico Administrative Code. It's just one more example of how far the government protections of the Big Standardized Test goes.
Section 6.10.7.11 of the NMAC deals with staff responsibilities regarding testing, and it includes a list of "prohibitive practices"-- things that staff are forbidden to do. At the end of the list, that it shall be prohibitive practice for the staff member
disparage or diminish the significance, importance or use of the standardized tests.
Other no-no's include photocopying the test, teaching from a copy of the test, copying copyrighted test prep materials, giving students reviews of the test questions, leaving test materials alone in an unlocked room, coaching students during the test, and taking standardized test materials off campus. All fairly standard stuff. Only the "don't say mean things about the test" rule surprised me.
The implications seem, well, chilling. Can teachers who look at blog pieces that disparage the significance, importance or use of standardized tests (I think I may have written one or twelve of those) be disciplined? Am I now an outlaw in New Mexico?
Penalties relating to these particular misbehaviors are unclear, but further along in section 6.10.7 we have section 6.10.7.12 which includes very stern language about flauting the non-disclosure of test materials rules (because "given the proprietary nature of any assessment which is part of the NMSAP, under no circumstance shall a standardized test which is part of the NMSAP be released").
Section 6.10.7.14 gets to the penalties for "testing irregularities" which can include anything from directing the "named individual" to cease and desist as well as barring them from ever administering a test again (boy, that hurts) all the way up to suspending or revoking their professional license (okay, that actually hurts). Whether Saying Mean Things about testing may or may not rise to the level of a testing irregularity.
So remember, New Mexico educators-- never speak ill of the beloved standardized test in all its resplendent swellness. I look forward to the New Mexico blog embargo that will somehow silence the rest of us.
Section 6.10.7.11 of the NMAC deals with staff responsibilities regarding testing, and it includes a list of "prohibitive practices"-- things that staff are forbidden to do. At the end of the list, that it shall be prohibitive practice for the staff member
disparage or diminish the significance, importance or use of the standardized tests.
Other no-no's include photocopying the test, teaching from a copy of the test, copying copyrighted test prep materials, giving students reviews of the test questions, leaving test materials alone in an unlocked room, coaching students during the test, and taking standardized test materials off campus. All fairly standard stuff. Only the "don't say mean things about the test" rule surprised me.
The implications seem, well, chilling. Can teachers who look at blog pieces that disparage the significance, importance or use of standardized tests (I think I may have written one or twelve of those) be disciplined? Am I now an outlaw in New Mexico?
Penalties relating to these particular misbehaviors are unclear, but further along in section 6.10.7 we have section 6.10.7.12 which includes very stern language about flauting the non-disclosure of test materials rules (because "given the proprietary nature of any assessment which is part of the NMSAP, under no circumstance shall a standardized test which is part of the NMSAP be released").
Section 6.10.7.14 gets to the penalties for "testing irregularities" which can include anything from directing the "named individual" to cease and desist as well as barring them from ever administering a test again (boy, that hurts) all the way up to suspending or revoking their professional license (okay, that actually hurts). Whether Saying Mean Things about testing may or may not rise to the level of a testing irregularity.
So remember, New Mexico educators-- never speak ill of the beloved standardized test in all its resplendent swellness. I look forward to the New Mexico blog embargo that will somehow silence the rest of us.
Big Security
ICYMI-- one of the best summaries of the Pearson surveillance flap is by Anthony Cody over at Living in Dialogue.
In addition to covering the various types of surveillance in play, and he includes a reference to the column by Cynthia Liu at K-12 News which advances and articulates an argument that several edu-bloggers have raised. It's not just the surveillance (which odes not really reach the level of spying) , but the enforcement-- that Pearson et al have turned state Boards of Education into agencies tasked with preserving corporate intellectual property rights. What we're being treated to is the spectacle of a system whose first priority is watching out for the business interests of a corporation-- the rights and education of students is a lesser priority. Why doe the corporation's concerns come first?
Daniel Katz has raised the question "Why is Pearson's intellectual property a thing?" It's a good question; why are the states that joined together in consortia in order to hire a corporation to produce a test for them-- why do these states not take possession at the end of the process?
I think Cody's column answers all of these questions. He writes this:
Any system that imparts heavy consequences for success or failure must have intense security.
That's correct. If the stakes are huge, that means that the people on the receiving end of potential punishments or rewards are highly motivated to make their numbers any possible way they can. If you put all the food in the castle and tell the villagers that only those who get inside the walls get to eat, you'd better believe that the villagers will be highly motivated to get past those walls any way they can.
The testing system requires Big Security because it is a big system. It's spread all over the country. Back when gold mattered, we put it all in a couple of forts because that was easier to defend. If we had put one gold bar in every city hall in America, defending it would have been a nightmare because there would be a million points of vulnerability.
The Big Standardized Test has a million points of vulnerability. BS Tests face an inherent contradiction-- security is maintained by letting as few people as possible actually see the product, and yet the product can't be used without being viewed. This means (in a relationship dynamic repeated throughout the world of education reform) that the clients are also the enemy.
All across the nation, millions of pairs of eyeballs are seeing what must not be seen.
As Cody notes, the credibility of the entire reform program rests on those tests. Everything in the reformster program depends on those tests being a fair and accurate measure of the (many, many) things they purport to measure. So the state education departments, the Data Overlords, the reformsters entrenched in various offices across the nation-- they need for the test to at least look secure and valid. This means security must be tight because 1) it's a lousy test whose gotcha questions must be sprung as a surprise and 2) the more responsible grown-ups see the test, the more criticism of the test gains traction.
So both state reformsters and corporations need tight super-security, and only the corporation has the resources. The state will be willing to pitch in on security because they have a stake in it, and they'll be willing to let Pearson et al cruise social media and deploy test police because the corporations have those kind of resources. And of course Pearson won't actually hand over the test to the states because the system has given the states a huge stake in the security of Pearson's "intellectual property."
The issue of test security is welded to test validity, and both are bonded tightly to the issues of ed reform itself. The BS Tests have simply advanced the smooshing together of state and corporate interests. The only interests not represented in all this-- the interests of students and of public education.
In addition to covering the various types of surveillance in play, and he includes a reference to the column by Cynthia Liu at K-12 News which advances and articulates an argument that several edu-bloggers have raised. It's not just the surveillance (which odes not really reach the level of spying) , but the enforcement-- that Pearson et al have turned state Boards of Education into agencies tasked with preserving corporate intellectual property rights. What we're being treated to is the spectacle of a system whose first priority is watching out for the business interests of a corporation-- the rights and education of students is a lesser priority. Why doe the corporation's concerns come first?
Daniel Katz has raised the question "Why is Pearson's intellectual property a thing?" It's a good question; why are the states that joined together in consortia in order to hire a corporation to produce a test for them-- why do these states not take possession at the end of the process?
I think Cody's column answers all of these questions. He writes this:
Any system that imparts heavy consequences for success or failure must have intense security.
That's correct. If the stakes are huge, that means that the people on the receiving end of potential punishments or rewards are highly motivated to make their numbers any possible way they can. If you put all the food in the castle and tell the villagers that only those who get inside the walls get to eat, you'd better believe that the villagers will be highly motivated to get past those walls any way they can.
The testing system requires Big Security because it is a big system. It's spread all over the country. Back when gold mattered, we put it all in a couple of forts because that was easier to defend. If we had put one gold bar in every city hall in America, defending it would have been a nightmare because there would be a million points of vulnerability.
The Big Standardized Test has a million points of vulnerability. BS Tests face an inherent contradiction-- security is maintained by letting as few people as possible actually see the product, and yet the product can't be used without being viewed. This means (in a relationship dynamic repeated throughout the world of education reform) that the clients are also the enemy.
All across the nation, millions of pairs of eyeballs are seeing what must not be seen.
As Cody notes, the credibility of the entire reform program rests on those tests. Everything in the reformster program depends on those tests being a fair and accurate measure of the (many, many) things they purport to measure. So the state education departments, the Data Overlords, the reformsters entrenched in various offices across the nation-- they need for the test to at least look secure and valid. This means security must be tight because 1) it's a lousy test whose gotcha questions must be sprung as a surprise and 2) the more responsible grown-ups see the test, the more criticism of the test gains traction.
So both state reformsters and corporations need tight super-security, and only the corporation has the resources. The state will be willing to pitch in on security because they have a stake in it, and they'll be willing to let Pearson et al cruise social media and deploy test police because the corporations have those kind of resources. And of course Pearson won't actually hand over the test to the states because the system has given the states a huge stake in the security of Pearson's "intellectual property."
The issue of test security is welded to test validity, and both are bonded tightly to the issues of ed reform itself. The BS Tests have simply advanced the smooshing together of state and corporate interests. The only interests not represented in all this-- the interests of students and of public education.
Sunday, March 15, 2015
Why Critical Thinking Won't Be on The Test
Critical thinking is one of the Great White Whales of education. Every new education reform promises to foster it, and every new generation of Big Standardized Tests promises to measure it.
Everybody working in education has some idea of what it is, and yet it can be hard to put into a few words. There are entire websites devoted to it, and organizations and foundations dedicated to it. Here, for example, is the website of the Foundation for Critical Thinking. They've got a definition of critical thinking from the 1987 National Council for Excellence in Critical Thinking that goes on for five paragraphs. One of the shortest definitions I can pull out of their site is this one:
The intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action.
Bottom line-- critical thinking is complicated.
So can we believe test manufacturers when they say that their test measures critical thinking skills? Can a series of questions that can be delivered and scored on a national scale be designed that would actually measure the critical thinking skills of the test takers?
I think the obstacles to creating such a standardized test are huge. Here are the hurdles that test manufacturers would have to leap.
Critical thinking takes time.
Certainly there are people who can make rapid leaps to a conclusion, who can see patterns and structure of ideas quickly and clearly (though we could argue that's more intuitive thinking than critical thinking, but then, intuition might just be critical thinking that runs below the level of clear consciousness, so, again, complicated). But mostly the kind of analyses and evaluation that we associate with critical thinking takes time.
There's a reason that English teachers rarely give the assignment, "The instant you finish reading the last page of the assigned novel, immediately start writing the assigned paper and complete it within a half hour." Critical thinking is most often applied to complex constructions, and for most people it takes a while to examine, reflect, re-examine and pull apart the pieces of the matter.
If you are asking a question that must be answered right now, this second, you are at the very best asking a question that measures how quickly the student can critically think-- but you're probably not measuring critical thinking at all.
Critical thinking takes place in a personal context.
We do not do our critical thinking in a vacuum. We are all standing in a particular spot in space and time, and that vantage point gives us a particular perspective. What we bring to the problem in terms of prior understanding, background, and our own mental constructs, profoundly influences how we critically think about any problem.
We tend to make sense out of unfamiliar things by looking for familiar structures and patterns within them, and so our thinking is influenced by what we already know. I've been an amateur musician my whole life, so I can readily spot structures and patterns that mimic the sorts of things I know form the world of music. However, I am to athletics what Justin Bieber is to quantum physics, and my mental default is not to look at things in athletic terms. Think about your favorite teachers and explainers-- they are people who took something you couldn't understand and put it in terms you could understand. They connected what you didn't know to what you did know.
None of this is a revolutionary new insight, but we have to remember that it means every individual human beings brings a different set of tools to each critical thinking problem. That means it is impossible to design a critical thinking question that is a level playing field for all test takers. Impossible.
Critical thinking is social.
How many big critical thinking problems of the world were solved single-handedly by a single, isolated human being?
Our sciences have a finely-tuned carefully-structured method for both carrying on and acknowledging dialogue with the critical thinkers of the past. If a scientist popped up claiming to have written a groundbreaking paper for which he needed no citations nor footnotes because he had done it all himself, he would be lucky to be taken seriously for five minutes. The Einsteins of history worked in constant dialogue with other scientists; quantum theories were hammered out in part by dialogue by a disbelieving Einstein ("God does not play dice") and the wave of scientists building on the implications of his work.
On the less grand scale, we find our own students who want to talk about the test, want to compare answers, want to (and sometimes love to) argue about the finer points of every thinking assignment.
Look at our own field. We've all been working on a big final test question-- "What is the best way to take American education forward?"-- and almost everyone on every side of the question is involved in a huge sprawling debate that sees most of us pushing forward by trying to articulate our own perspective and thoughts while in dialogue with hundreds of other thinkers in varying degrees of agreement and disagreement. One of the reasons I trust and believe David Coleman far less than other reformsters is that he almost never acknowledges the value of any other thinker in his development of Common Core. To watch Coleman talk, you would think he developed the entire thing single-handedly in his own head. That is not the mark of a serious person.
Do people occasionally single-handedly solve critical thinking problems on their own, in isolation, like a keep-your-eyes-on-your-own-paper test? It's certainly not unheard of-- but it's not the norm. If your goal is to make the student answer the question in an isolation chamber, you are not testing critical thinking.
Critical thinking is divergent.
Let's go back to that critical thinking problem about how to best move forward with public education. You may have noticed that people have arrived a wide variety of conclusions about what the answer might be. There are two possible explanations for the wide variety of answers.
The first explanation is the childish one, and folks from both sides indulge in it-- people who have reached a conclusion other than mine are some combination of stupid, uninformed, and evil.
The more likely explanation is that, given a wide variety of different perspectives, different histories, and different values, intelligent people will use critical thinking skills and arrive at different conclusions.
Critical thinking is NOT starting with the conclusion that you want to reach and then constructing a bridge of arguments specifically designed to get you there, and yet this is perilously close to the kind of thinking a standardized test requires.
But here's a good rule of thumb for anyone trying to test critical thinking skills-- if you are designing your assessment and thinking, "Okay, any student who is really using critical thinking skills must come up with answer B," you are not testing critical thinking skills. No-- I take that back. Oddly enough this is a sort of critical thinking question, but the actual question is, "Given what you know about the people giving you the test and the clues they have left for you, what answer do you think the testmakers want you to select?" But that is probably not the question that you thought you were asking. As soon as you ask a question with one right answer (even if the one right answer is to select both correct answers), you are not testing critical thinking.
Critical thinking must be assessed by critical thinking.
How do you assess the answer to your critical thinking question? Again, I direct you to the education debates, where we "grade" each others' work all the time, checking and analyzing, probing for logical fallacies, mis-presentation of data, mis-reading of other peoples' writing, honesty of logic, etc etc etc.
To assess how well someone has answered a critical thinking question, you need to be knowledgeable about the answerer, the subject matter, and whatever background knowledge they have brought to the table (if I answer a question using a music analogy and you know nothing about music, will you know if my analogy holds up). On top of all that, you need some critical thinking skills of your own. And that means all of the issues listed above come back into play.
What are the odds that you can get all that in a cadre of minimum-wage test-scorers who can wade through a nation's worth of tests quickly, efficiently, and accurately?
Can it be done?
When I look at all those hurdles and try to imagine a nationally scaled test that gets deals with all of them, I'm stumped. Heck, it's a challenge to come up with good measure for my own classroom, and that's because critical thinking is more of a tool than an end in itself. Testing for critical thinking skills is kind of like testing for hammering skills-- it can be done, but it will be an artificial situation and not as compelling and useful and telling as having the student actually build something.
So I try to come up with assessments that require critical thinking as a tool for completion of the assignment. Then I try to come up with the time to grade them. Could I come up with something for the entire nation? Practically speaking, no. Even if I get past the first few hurdles, when I reach the point that I need a couple million teachers to score it, I'm stumped. Plus, standardized test fans are not going to like the lack of standardization in my test.
No, I think that standardized testing and critical thinking are permanently at odds and we'd be further ahead trying to develop a test to compare the flammability of the water from different rivers.
Critical thinking is not on the BS Tests. It will not be on the new generations of the BS Tests. It will never be on the BS Tests. Test manufacturers should stop promising what they cannot hope to deliver.
Everybody working in education has some idea of what it is, and yet it can be hard to put into a few words. There are entire websites devoted to it, and organizations and foundations dedicated to it. Here, for example, is the website of the Foundation for Critical Thinking. They've got a definition of critical thinking from the 1987 National Council for Excellence in Critical Thinking that goes on for five paragraphs. One of the shortest definitions I can pull out of their site is this one:
The intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action.
Bottom line-- critical thinking is complicated.
So can we believe test manufacturers when they say that their test measures critical thinking skills? Can a series of questions that can be delivered and scored on a national scale be designed that would actually measure the critical thinking skills of the test takers?
I think the obstacles to creating such a standardized test are huge. Here are the hurdles that test manufacturers would have to leap.
Critical thinking takes time.
Certainly there are people who can make rapid leaps to a conclusion, who can see patterns and structure of ideas quickly and clearly (though we could argue that's more intuitive thinking than critical thinking, but then, intuition might just be critical thinking that runs below the level of clear consciousness, so, again, complicated). But mostly the kind of analyses and evaluation that we associate with critical thinking takes time.
There's a reason that English teachers rarely give the assignment, "The instant you finish reading the last page of the assigned novel, immediately start writing the assigned paper and complete it within a half hour." Critical thinking is most often applied to complex constructions, and for most people it takes a while to examine, reflect, re-examine and pull apart the pieces of the matter.
If you are asking a question that must be answered right now, this second, you are at the very best asking a question that measures how quickly the student can critically think-- but you're probably not measuring critical thinking at all.
Critical thinking takes place in a personal context.
We do not do our critical thinking in a vacuum. We are all standing in a particular spot in space and time, and that vantage point gives us a particular perspective. What we bring to the problem in terms of prior understanding, background, and our own mental constructs, profoundly influences how we critically think about any problem.
We tend to make sense out of unfamiliar things by looking for familiar structures and patterns within them, and so our thinking is influenced by what we already know. I've been an amateur musician my whole life, so I can readily spot structures and patterns that mimic the sorts of things I know form the world of music. However, I am to athletics what Justin Bieber is to quantum physics, and my mental default is not to look at things in athletic terms. Think about your favorite teachers and explainers-- they are people who took something you couldn't understand and put it in terms you could understand. They connected what you didn't know to what you did know.
None of this is a revolutionary new insight, but we have to remember that it means every individual human beings brings a different set of tools to each critical thinking problem. That means it is impossible to design a critical thinking question that is a level playing field for all test takers. Impossible.
Critical thinking is social.
How many big critical thinking problems of the world were solved single-handedly by a single, isolated human being?
Our sciences have a finely-tuned carefully-structured method for both carrying on and acknowledging dialogue with the critical thinkers of the past. If a scientist popped up claiming to have written a groundbreaking paper for which he needed no citations nor footnotes because he had done it all himself, he would be lucky to be taken seriously for five minutes. The Einsteins of history worked in constant dialogue with other scientists; quantum theories were hammered out in part by dialogue by a disbelieving Einstein ("God does not play dice") and the wave of scientists building on the implications of his work.
On the less grand scale, we find our own students who want to talk about the test, want to compare answers, want to (and sometimes love to) argue about the finer points of every thinking assignment.
Look at our own field. We've all been working on a big final test question-- "What is the best way to take American education forward?"-- and almost everyone on every side of the question is involved in a huge sprawling debate that sees most of us pushing forward by trying to articulate our own perspective and thoughts while in dialogue with hundreds of other thinkers in varying degrees of agreement and disagreement. One of the reasons I trust and believe David Coleman far less than other reformsters is that he almost never acknowledges the value of any other thinker in his development of Common Core. To watch Coleman talk, you would think he developed the entire thing single-handedly in his own head. That is not the mark of a serious person.
Do people occasionally single-handedly solve critical thinking problems on their own, in isolation, like a keep-your-eyes-on-your-own-paper test? It's certainly not unheard of-- but it's not the norm. If your goal is to make the student answer the question in an isolation chamber, you are not testing critical thinking.
Critical thinking is divergent.
Let's go back to that critical thinking problem about how to best move forward with public education. You may have noticed that people have arrived a wide variety of conclusions about what the answer might be. There are two possible explanations for the wide variety of answers.
The first explanation is the childish one, and folks from both sides indulge in it-- people who have reached a conclusion other than mine are some combination of stupid, uninformed, and evil.
The more likely explanation is that, given a wide variety of different perspectives, different histories, and different values, intelligent people will use critical thinking skills and arrive at different conclusions.
Critical thinking is NOT starting with the conclusion that you want to reach and then constructing a bridge of arguments specifically designed to get you there, and yet this is perilously close to the kind of thinking a standardized test requires.
But here's a good rule of thumb for anyone trying to test critical thinking skills-- if you are designing your assessment and thinking, "Okay, any student who is really using critical thinking skills must come up with answer B," you are not testing critical thinking skills. No-- I take that back. Oddly enough this is a sort of critical thinking question, but the actual question is, "Given what you know about the people giving you the test and the clues they have left for you, what answer do you think the testmakers want you to select?" But that is probably not the question that you thought you were asking. As soon as you ask a question with one right answer (even if the one right answer is to select both correct answers), you are not testing critical thinking.
Critical thinking must be assessed by critical thinking.
How do you assess the answer to your critical thinking question? Again, I direct you to the education debates, where we "grade" each others' work all the time, checking and analyzing, probing for logical fallacies, mis-presentation of data, mis-reading of other peoples' writing, honesty of logic, etc etc etc.
To assess how well someone has answered a critical thinking question, you need to be knowledgeable about the answerer, the subject matter, and whatever background knowledge they have brought to the table (if I answer a question using a music analogy and you know nothing about music, will you know if my analogy holds up). On top of all that, you need some critical thinking skills of your own. And that means all of the issues listed above come back into play.
What are the odds that you can get all that in a cadre of minimum-wage test-scorers who can wade through a nation's worth of tests quickly, efficiently, and accurately?
Can it be done?
When I look at all those hurdles and try to imagine a nationally scaled test that gets deals with all of them, I'm stumped. Heck, it's a challenge to come up with good measure for my own classroom, and that's because critical thinking is more of a tool than an end in itself. Testing for critical thinking skills is kind of like testing for hammering skills-- it can be done, but it will be an artificial situation and not as compelling and useful and telling as having the student actually build something.
So I try to come up with assessments that require critical thinking as a tool for completion of the assignment. Then I try to come up with the time to grade them. Could I come up with something for the entire nation? Practically speaking, no. Even if I get past the first few hurdles, when I reach the point that I need a couple million teachers to score it, I'm stumped. Plus, standardized test fans are not going to like the lack of standardization in my test.
No, I think that standardized testing and critical thinking are permanently at odds and we'd be further ahead trying to develop a test to compare the flammability of the water from different rivers.
Critical thinking is not on the BS Tests. It will not be on the new generations of the BS Tests. It will never be on the BS Tests. Test manufacturers should stop promising what they cannot hope to deliver.
Saturday, March 14, 2015
Canaries, Schools & Poverty
Let's step back from public education itself for a moment. Look at the bigger picture.
The economic engine of the US is messed up. Call it conspiracy, policy, oligarchy, or just a bad turn-- the poor are being left further and further behind, and the rich are consolidating their own piece of the pie. The shared energy and mission of the country are fragmenting, and more and more people and communities are sinking into poverty. But this is a long, slow process, and it wouldn't show up everywhere at once.
What might be a leading indicator of the growing corrosive and destructive power of poverty? How about schools-- the common good that is supposed to be provided by all citizens for all citizens.
Public education is the canary in the coal mine, an early notable indicator that something is wrong, that something toxic and damaging is in the air. And of the public schools, those that are already weak and poor, least able to stand the shock and the strain, that are most bowed under the weight of poverty will start to falter first.
Now, when the canary starts to falter and fall, that's a sign that something is wrong, that we need to get the people out of the mine or more fresh air into it.
But suppose instead we had a bunch of people who said, "No, what we need to do is work on resuscitating the canary! We need to hire canary doctors and develop new canary breathing programs." Those would be the reformers. They are not wrong about the distress of the canary, or the need to do something before the canary dies, but it's a huge mistake to ignore the conditions that are killing the canary in the first place. All the respiratory therapy in the world will not save the canary if we don't get it some oxygen and get rid of the bad gas poisoning its system.
The Data Overlords want to run extensive tests on the canary. "Let's measure its oxygen intake every five minutes. If we keep measuring, it should start breathing more freely." When questioned on that point, they simply reply, "Look, this is the same oxygen intake test we use for those canaries up on the surface in the special gilded cages. Why shouldn't these mine canaries get to take the same test?"
Charter operators just want to bring in other canaries. "Your canary is weak and stupid and has a bad attitude," they say. "What we need are these fresh new alternative canaries. Once we get those canaries in there, they will breathe so much better than your dumb canary."
Meanwhile, the profiteers are in talks with the mine operators. "If you would just unleash the power of the free market, we could make a delicious and profitable canary stew."
When some folks try to push the idea that pumping oxygen into the mine could help revive the canary, some reformers cry foul. "What's the matter with you? Don't you believe this canary can breathe? Do you think this canary isn't good enough to survive!"
Meanwhile, the canaries continue to die and air in the mine becomes more and more toxic, until not a canary or an eagle or a full grown human could hope to survive there. The canaries absolutely deserve attention and assistance, and we absolutely have an imperative to keep them alive. But if we don't find a way to replace the bad air with good, to sweep out the lung-clenching methane of poverty and bring in some oxygen, we'll just be stuck in endless cycle of canary rescue, complete with arguments about how to rescue the canary, who should rescue the canary, and whether or not anyone can profit from rescuing the canary. All the while the bad air spreads.
Subscribe to:
Posts (Atom)