Don't think of them as standards. Think of them as tags.
Think of them as the pedagogical equivalent of people's names on facebook, the tags you attach to each and every photo that you upload.
We know from our friends at Knewton what the Grand Design is-- a system in which student progress is mapped down to the atomic level. Atomic level (a term that Knewton lervs deeply) means test by test, assignment by assignment, sentence by sentence, item by item. We want to enter every single thing a student does into the Big Data Bank.
But that will only work if we're all using the same set of tags.
We've been saying that CCSS are limited because the standards were written around what can be tested. That's not exactly correct. The standards have been written around what can be tracked.
The standards aren't just about defining what should be taught. They're about cataloging what students have done.
Remember when Facebook introduced emoticons. This was not a public service. Facebook wanted to up its data gathering capabilities by tracking the emotional states of users. But if users just defined their own emotions, the data would be too noisy, too hard to crunch. But if the user had to pick from the facebook standard set of user emotions-- then facebook would have manageable data.
Ditto for CCSS. If we all just taught to our own local standards, the data noise would be too great. The Data Overlords need us all to be standardized, to be using the same set of tags. That is also why no deviation can be allowed. Okay, we'll let you have 15% over and above the standards. The system can probably tolerate that much noise. But under no circumstances can you change the standards-- because that would be changing the national student data tagging system, and THAT we can't tolerate.
This is why the "aligning" process inevitably involves all that marking of standards onto everything we do. It's not instructional. It's not even about accountability.
It's about having us sit and tag every instructional thing we do so that student results can be entered and tracked in the Big Data Bank.
And that is why CCSS can never, ever be decoupled from anything. Why would facebook keep a face tagging system and then forbid users to upload photos?
The Test does not exist to prove that we're following the standards. The standards exist to let us tag the results from the Test. And ultimately, not just the Test, but everything that's done in a classroom. Standards-ready material is material that has already been bagged and tagged for Data Overlord use.
Oddly enough, this understanding of the CCSS system also reveals more reasons why the system sucks.
Facebook's photo tagging system is active and robust. Anybody can add tags, and so the system grows because it is useful. On the other hand, their emoticon system, which requires users to feel only the standardized facebook emotions, is rigid and dying on the vine because it's not useful and it can't adapt.
The CCSS are lousy standards precisely because they are too specific in some areas, too vague in others, and completely missing other aspects of teaching entirely. We all know how the aligning works-- you take what you already do and find a standard that it more or less fits with and tag it.
Because the pedagogical fantasy delineated by the CCSS does not match the teacher reality in a classroom, the tags are applied in inexact and not-really-true ways. In effect, we've been given color tags that only cover one side of the color wheel, but we've been told to tag everything, so we end up tagging purple green. When a tagging system doesn't represent the full range of reality, and it isn't flexible enough to adapt, you end up with crappy tagging. And that's the CCSS.
It's true that in a massive tagging system like this, a Big Test could be rendered unnecessary-- just use all the data that's pouring in from everywhere else. Two reasons that won't happen:
1) While our Data Overlord's eyes were on the data prize, their need for tagged and connected data opened the door for profiteering, and once that stream is flowing, no Pearsonesque group will stand for interfering with it.
2) High stakes tests are necessary to force cooperation. To get people to fork over this much data, they must be motivated. We've seen that evolution in PA, as the folks in charge have realized that nothing less than the highest stakes will get students to stop writing the pledge to the flag on their tests and teachers to stop laughing when they do.
Decoupling? Not going to happen. You can't have a data system without tagging, and you can't have a tagging system with nothing to tag. Education and teaching are just collateral damage in all this, and not really the main thing at all.
PS: Note Diane Ravitch's morning post which displays how badly the standards fail at being standards by all standard standards standards. Why did they do such a bad job of writing standards? Because they weren't trying to write standards-- they were writing data tags!
Monday, March 24, 2014
Sunday, March 23, 2014
The Coming Teacher Shortage
Friday I sat down for coffee with the president of a local university (in my other incarnation as a local newspaper columnist, I get the occasional request to chat). Among other things, she confirmed what I have been hearing for a while-- enrollment in state school teacher programs is plummeting, down in my region almost 50% from better times.
Part of the problem is that, at least in my part of Pennsylvania, the college-age demographic sector is shrinking, and so all college enrollment is shrinking. But as we look at the shrinking interest in joining our profession, I think we have a couple of factors to consider.
The obvious
Teachers have been getting slammed for a couple of decades now. Today's eighteen-year-olds have heard a lifetime of noise about how teachers are screwing up education, standing in the way of progress, failing in all meaningful ways. They have heard the backhanded attack fallacy that a teacher is the most important factor in a school, and schools are failing, so what do you suppose that means about teachers? They have heard that teachers suck, school suck, that US education is just a giant suckfest.
They have even heard (and I feel sad to have to admit it, because I think it's wrong) veteran teachers tell them, "Don't do it. Don't pursue teaching."
They have also heard that you don't really need to study teaching to be a teacher. Pittsburgh beat back an attempted incursion by Teach for America, but while that looked like a done deal, what message did it send to students entering their final year of a teacher prep program? "Don't bother. We're going to hire some business majors with five weeks of training. Those credentials you just spent four years acquiring don't mean jack to us."
I should also acknowledge that in some areas, massive staff cuts and school closings have created a situation of local teacher surplus. In places like Chicago this only underlines my last point, because certified teachers have in effect been replaced with untrained TFA and TNTP bodies. This all adds further to the "why bother" view of applying to college teaching programs.
These factors would be enough to drive down interest in the profession, but I don't think they're the whole story.
The less obvious
Most of us developed our idea of what school is about during our own years in school. "School" almost always means, especially at the start of a career, "the kind of school I grew up in."
As many of us have said before, high stakes test-driven accountability is not "reform"-- it's the status quo. After over a decade of test prep and test taking and practice testing and test result obsession, we have now created a generation of students who don't know anything else. For today's high school senior, all this test obsession is not some new thing that is threatening education-- it's what education is. For this generation, school is the place you go to get ready for big important tests.
For many of this generation (depending on the lucky or unlucky draw of a local district), a teacher is someone who helps students get ready for big tests. A teacher is someone who delivers a prepared program-in-a-box; they don't develop units or create material or do anything except open the box and unpack what's inside.
This makes the teaching profession hugely less appealing. "If I can help just one kid figure out the right bubble to fill in on his test, I will feel like I've made the world a better place," said no young person ever. The inspiring, exciting image of teaching-- the independence, the intellectual searching, the firing of imaginations, the sparking of young minds, the nurturing of fragile young souls, the passing on of vibrant living knowledge, the participation in the miracle of growth, the guiding on a path to being fully human-- all those things that fired us up about teaching-- we got that bug from our own teachers and our own school experience. But far more of today's young people associate school with the drudgery of clerical work, the autonomy of assembly line workers.
The Masters of Reforming Our Nation's Schools have done their best to refashion teaching into work as uninspiring as minimum wage work in a fast food chain. (Ironically, the one place we still find teaching described in such inspirational terms is in TFA propaganda.) We dislike what the MoRONS are trying to do to teaching so much that we fight; what in that sad new world of teaching would attract somebody.
Which is perhaps the more sobering implication. Because some people do still enter teacher prep programs. Some of them have had the fortune to encounter inspirational old-school teaching, to become fired up like the rest of us did. But some have encountered the new fast-food clerical status quo, and they're okay with it. "Teaching's not hard. Do what I'm told, prep them for a test. Easy peasy!" And those future clerks will do just well in college programs that spend less time on "How To Inspire Your Students" and more on "How To Use Aligned Standards To Raise Test Scores."
See, the coming teacher shortage is not just about having fewer people who call themselves teachers. It's also about new young people who will claim the name of teacher and won't really have any idea what they're talking about.
Part of the problem is that, at least in my part of Pennsylvania, the college-age demographic sector is shrinking, and so all college enrollment is shrinking. But as we look at the shrinking interest in joining our profession, I think we have a couple of factors to consider.
The obvious
Teachers have been getting slammed for a couple of decades now. Today's eighteen-year-olds have heard a lifetime of noise about how teachers are screwing up education, standing in the way of progress, failing in all meaningful ways. They have heard the backhanded attack fallacy that a teacher is the most important factor in a school, and schools are failing, so what do you suppose that means about teachers? They have heard that teachers suck, school suck, that US education is just a giant suckfest.
They have even heard (and I feel sad to have to admit it, because I think it's wrong) veteran teachers tell them, "Don't do it. Don't pursue teaching."
They have also heard that you don't really need to study teaching to be a teacher. Pittsburgh beat back an attempted incursion by Teach for America, but while that looked like a done deal, what message did it send to students entering their final year of a teacher prep program? "Don't bother. We're going to hire some business majors with five weeks of training. Those credentials you just spent four years acquiring don't mean jack to us."
I should also acknowledge that in some areas, massive staff cuts and school closings have created a situation of local teacher surplus. In places like Chicago this only underlines my last point, because certified teachers have in effect been replaced with untrained TFA and TNTP bodies. This all adds further to the "why bother" view of applying to college teaching programs.
These factors would be enough to drive down interest in the profession, but I don't think they're the whole story.
The less obvious
Most of us developed our idea of what school is about during our own years in school. "School" almost always means, especially at the start of a career, "the kind of school I grew up in."
As many of us have said before, high stakes test-driven accountability is not "reform"-- it's the status quo. After over a decade of test prep and test taking and practice testing and test result obsession, we have now created a generation of students who don't know anything else. For today's high school senior, all this test obsession is not some new thing that is threatening education-- it's what education is. For this generation, school is the place you go to get ready for big important tests.
For many of this generation (depending on the lucky or unlucky draw of a local district), a teacher is someone who helps students get ready for big tests. A teacher is someone who delivers a prepared program-in-a-box; they don't develop units or create material or do anything except open the box and unpack what's inside.
This makes the teaching profession hugely less appealing. "If I can help just one kid figure out the right bubble to fill in on his test, I will feel like I've made the world a better place," said no young person ever. The inspiring, exciting image of teaching-- the independence, the intellectual searching, the firing of imaginations, the sparking of young minds, the nurturing of fragile young souls, the passing on of vibrant living knowledge, the participation in the miracle of growth, the guiding on a path to being fully human-- all those things that fired us up about teaching-- we got that bug from our own teachers and our own school experience. But far more of today's young people associate school with the drudgery of clerical work, the autonomy of assembly line workers.
The Masters of Reforming Our Nation's Schools have done their best to refashion teaching into work as uninspiring as minimum wage work in a fast food chain. (Ironically, the one place we still find teaching described in such inspirational terms is in TFA propaganda.) We dislike what the MoRONS are trying to do to teaching so much that we fight; what in that sad new world of teaching would attract somebody.
Which is perhaps the more sobering implication. Because some people do still enter teacher prep programs. Some of them have had the fortune to encounter inspirational old-school teaching, to become fired up like the rest of us did. But some have encountered the new fast-food clerical status quo, and they're okay with it. "Teaching's not hard. Do what I'm told, prep them for a test. Easy peasy!" And those future clerks will do just well in college programs that spend less time on "How To Inspire Your Students" and more on "How To Use Aligned Standards To Raise Test Scores."
See, the coming teacher shortage is not just about having fewer people who call themselves teachers. It's also about new young people who will claim the name of teacher and won't really have any idea what they're talking about.
News from Institute of Grittology
Here at the Institute of Grittology, we're committed to helping monetize the work of our research partners, The Research Institute for the Study of Obvious Conclusions ("Working hard to recycle conventional wisdom as proprietary programing").
Our speakers bureau has determined that statements such as "treating children with support and kindness helps them do better in life" do not enhance the revenue stream. However, folks will fork over good money to hear "it is our collective responsibility to strive at all levels of our educational communities to provide environments for students that are calm, supportive, encouraging, thoughtful, and planned, which provide opportunities for students to ignite their latent capabilities to be resilient."
In addition to repackaging such insights as "people who don't quit tend to finish more stuff," we have found that Grittology also provides good cover for traditional management insights such as, "If people think you're abusive, they just need to suck it up and grow a pair." If people find a situation difficult, challenging, upsetting or oppressive, they should understand that it is because they lack sufficient grit. Moving forward, locating and identifying grittacious individuals will become increasingly important for employers who don't want to feel pressure to make their work environment more human-friendly.
Of course, in today's educational marketplace, to really sell grit we're going to need to collect some data in order to quantify the objectively measurable aspects of grit. We hope to be part of the great cradle-to-grave data trail because this will allow prospective employers to better assess the continued employable of individual human resource units vis-a-vis more efficacious application of task performance potential productivity growth ROI workplace retention growth. Also, we expect to make a shitload of money.
We have developed some testing tools for assessing an individual's Grit Or Resiliency Proficiency. The GORP score can be generated for school students. Here are some ample items.
For small children
Have the child sit in a small room and with a cute puppy. Once the child has had the opportunity to bond, enter the room, take the puppy, and tell the child, "This is your fault. You don't deserve nice things." and storm out. Observe child's reaction.
Below basic: Cries like some sort of baby.
Basic: Sniffles and sulks
Proficient: Calls parents to buy a new puppy
Advanced: Builds a puppy with materials in examination room
For older children
Tell the subject that his/her parents have been killed in a terrible car crash and the student will now have to go live in an orphanage
Below basic: Cries like some sort of baby
Basic: Curls up quietly in fetal position
Proficient: Runs away
Advanced: Plans to use estate to attend nice private school
For teens
Put teen in room with person they would find attractive who flirts with student for short period before abruptly announcing that the student "is too gross for anybody to ever love."
Below basic: Cries like some sort of baby
Basic: Whines and asks "Why don't you like me?"
Proficient: Says "Well, I know you are, but what am I"
Advanced: Offers to have parents buy attractive person a car
Human resources departments in school districts have also expressed an interest in using GORP scores as part of the hiring process. Intense research has demonstrated that people who tend to stick with their commitments tend to stick with their commitments, and as school working conditions become worse and worse, identifying employees who can put up with those conditions for a full teaching year is becoming cost-effective. We suggest GORP scoring be part of the hiring process. Here are some sample items for pre-employment GORP testing.
Sample GORP test item 1:
Lock applicant in room without food for forty-eight hours
Below basic: Dies
Basic: Becomes gravely ill
Proficient: Remains healthy but thin
Advanced: Calls his lawyer and arranges release
Sample GORP test item 2:
Punch applicant in the face
Below basic: Falls down
Basic: Falls down but gets up slowly
Proficient: Punches examiner back
Advanced: Calls lawyer and has district sued
Important note for school districts
Studies of grit suggest that grit is often associated with independent thought and inability to follow orders blindly. Too much grit in your teaching staff and before you know it you have test boycotts and union activity and teachers asking annoying questions in staff meetings. District human resource departments should ideally hire candidates whose GORP scores are only basic or proficient, as teachers with advanced GORP scores might not be willing to just quietly sit and take it.
Remember-- having grit is valuable and important, but not as important as being compliant and within all standard acceptable ranges of behavior. We need people in the workplace who can take abuse, but not people who will actually fight back.
Our speakers bureau has determined that statements such as "treating children with support and kindness helps them do better in life" do not enhance the revenue stream. However, folks will fork over good money to hear "it is our collective responsibility to strive at all levels of our educational communities to provide environments for students that are calm, supportive, encouraging, thoughtful, and planned, which provide opportunities for students to ignite their latent capabilities to be resilient."
In addition to repackaging such insights as "people who don't quit tend to finish more stuff," we have found that Grittology also provides good cover for traditional management insights such as, "If people think you're abusive, they just need to suck it up and grow a pair." If people find a situation difficult, challenging, upsetting or oppressive, they should understand that it is because they lack sufficient grit. Moving forward, locating and identifying grittacious individuals will become increasingly important for employers who don't want to feel pressure to make their work environment more human-friendly.
Of course, in today's educational marketplace, to really sell grit we're going to need to collect some data in order to quantify the objectively measurable aspects of grit. We hope to be part of the great cradle-to-grave data trail because this will allow prospective employers to better assess the continued employable of individual human resource units vis-a-vis more efficacious application of task performance potential productivity growth ROI workplace retention growth. Also, we expect to make a shitload of money.
We have developed some testing tools for assessing an individual's Grit Or Resiliency Proficiency. The GORP score can be generated for school students. Here are some ample items.
For small children
Have the child sit in a small room and with a cute puppy. Once the child has had the opportunity to bond, enter the room, take the puppy, and tell the child, "This is your fault. You don't deserve nice things." and storm out. Observe child's reaction.
Below basic: Cries like some sort of baby.
Basic: Sniffles and sulks
Proficient: Calls parents to buy a new puppy
Advanced: Builds a puppy with materials in examination room
For older children
Tell the subject that his/her parents have been killed in a terrible car crash and the student will now have to go live in an orphanage
Below basic: Cries like some sort of baby
Basic: Curls up quietly in fetal position
Proficient: Runs away
Advanced: Plans to use estate to attend nice private school
For teens
Put teen in room with person they would find attractive who flirts with student for short period before abruptly announcing that the student "is too gross for anybody to ever love."
Below basic: Cries like some sort of baby
Basic: Whines and asks "Why don't you like me?"
Proficient: Says "Well, I know you are, but what am I"
Advanced: Offers to have parents buy attractive person a car
Human resources departments in school districts have also expressed an interest in using GORP scores as part of the hiring process. Intense research has demonstrated that people who tend to stick with their commitments tend to stick with their commitments, and as school working conditions become worse and worse, identifying employees who can put up with those conditions for a full teaching year is becoming cost-effective. We suggest GORP scoring be part of the hiring process. Here are some sample items for pre-employment GORP testing.
Sample GORP test item 1:
Lock applicant in room without food for forty-eight hours
Below basic: Dies
Basic: Becomes gravely ill
Proficient: Remains healthy but thin
Advanced: Calls his lawyer and arranges release
Sample GORP test item 2:
Punch applicant in the face
Below basic: Falls down
Basic: Falls down but gets up slowly
Proficient: Punches examiner back
Advanced: Calls lawyer and has district sued
Important note for school districts
Studies of grit suggest that grit is often associated with independent thought and inability to follow orders blindly. Too much grit in your teaching staff and before you know it you have test boycotts and union activity and teachers asking annoying questions in staff meetings. District human resource departments should ideally hire candidates whose GORP scores are only basic or proficient, as teachers with advanced GORP scores might not be willing to just quietly sit and take it.
Remember-- having grit is valuable and important, but not as important as being compliant and within all standard acceptable ranges of behavior. We need people in the workplace who can take abuse, but not people who will actually fight back.
Saturday, March 22, 2014
Cloudy with a Chance of Data
There are so many reasons to be opposed to the business of mining and crunching data. We like to rail about how the data miners are oppressive and Big Brothery and overreaching. But there's another point worth making about our Data Overlords:
Data miners are not very good at their job.
My first wife and I divorced about twenty years ago. We have both since remarried and moved multiple times. And yet, I still get occasional pieces of mail for her here at my current home. The last time I looked at my credit report, it included me living at an address that she used after we split. I could try to get it changed but A) she is a responsible woman who I'm sure has excellent credit and B) have you ever tried to get info on your credit report changed?
As I work on this, several other browser windows are showing ads for K12. I cruised to some sites maybe two weeks ago doing research for some pieces about cyber charters, but now my browser and adsense are sure I'm in the market for cyberschool. It is tempting to click the ads repeatedly in order to drain k12's ad budget of another wasted 25 cents, but I would have to live with the consequences.
My brother and I have an old game we sometimes play. When pollsters call us, we answer opposite of our actual beliefs in order to feed the pollster false info. Because who says we can't or shouldn't?
Before anything of use can happen in the data cloud, two things must be true:
1) The data must be good.
The tools for collection must be accurate. Designing good data collection tools is hard. The Data Overlords are trying to convert all the tools of instruction and assessment into tools for data gathering, but that's not what they're generally designed to do. Most fundamentally, I collect data about a student to create a picture of that student, not to turn that student into one data point among millions.
But beyond the accuracy of the tool, there is the willingness of the data generators. I suspect this is a blind spot for Data Overlords-- they are so convinced of the importance of data collection that they don't necessarily understand that most of us feel no compelling reason to cooperate.
There is no moral imperative to help the Data Overlords gather accurate data.
2) The program for crunching it must be good.
In the late seventies I was studying BASIC programming language and our professor was reminding us repeatedly that computers are stupid machines that happen to possess speed and long attention spans. If we tell them to do stupid things, they will-- but really, really fast! A computer is not one whit "smarter" than the person who programmed it.
If the person writing the software believes that knowing "2 + 2 = 5" means you're ready for calculus, the program will find many six-year-olds are prepared for math courses.
Put another way, a computer doesn't know how to predict anything that no human being knows how to predict, and it particularly doesn't know how to predict anything involving a series of complicated data points that the software writer failed to anticipate. So a human being could easily figure out that my ex-wife doesn't live here, but the software lacks the complexity to pull together the right data. And a human being could figure out that I used some of my brother's airline points to get a magazine subscription, but the software thinks he might live here, too.
The software can't figure out how to put every single person together with his/her perfect romantic match. It can't figure out exactly what movie you want to watch right this minute. And it doesn't know that I hope K12 dies a permanent death.
It's as simple as GIGO-- bad data processed poorly yields no useful results. Waving your laser pointer and intoning, "Look! Compuuuuters! Data! Data made out of numbers!! It's magical!!" will not convince me to cheerfully welcome my New Data Overlords.
Data miners are not very good at their job.
My first wife and I divorced about twenty years ago. We have both since remarried and moved multiple times. And yet, I still get occasional pieces of mail for her here at my current home. The last time I looked at my credit report, it included me living at an address that she used after we split. I could try to get it changed but A) she is a responsible woman who I'm sure has excellent credit and B) have you ever tried to get info on your credit report changed?
As I work on this, several other browser windows are showing ads for K12. I cruised to some sites maybe two weeks ago doing research for some pieces about cyber charters, but now my browser and adsense are sure I'm in the market for cyberschool. It is tempting to click the ads repeatedly in order to drain k12's ad budget of another wasted 25 cents, but I would have to live with the consequences.
My brother and I have an old game we sometimes play. When pollsters call us, we answer opposite of our actual beliefs in order to feed the pollster false info. Because who says we can't or shouldn't?
Before anything of use can happen in the data cloud, two things must be true:
1) The data must be good.
The tools for collection must be accurate. Designing good data collection tools is hard. The Data Overlords are trying to convert all the tools of instruction and assessment into tools for data gathering, but that's not what they're generally designed to do. Most fundamentally, I collect data about a student to create a picture of that student, not to turn that student into one data point among millions.
But beyond the accuracy of the tool, there is the willingness of the data generators. I suspect this is a blind spot for Data Overlords-- they are so convinced of the importance of data collection that they don't necessarily understand that most of us feel no compelling reason to cooperate.
There is no moral imperative to help the Data Overlords gather accurate data.
2) The program for crunching it must be good.
In the late seventies I was studying BASIC programming language and our professor was reminding us repeatedly that computers are stupid machines that happen to possess speed and long attention spans. If we tell them to do stupid things, they will-- but really, really fast! A computer is not one whit "smarter" than the person who programmed it.
If the person writing the software believes that knowing "2 + 2 = 5" means you're ready for calculus, the program will find many six-year-olds are prepared for math courses.
Put another way, a computer doesn't know how to predict anything that no human being knows how to predict, and it particularly doesn't know how to predict anything involving a series of complicated data points that the software writer failed to anticipate. So a human being could easily figure out that my ex-wife doesn't live here, but the software lacks the complexity to pull together the right data. And a human being could figure out that I used some of my brother's airline points to get a magazine subscription, but the software thinks he might live here, too.
The software can't figure out how to put every single person together with his/her perfect romantic match. It can't figure out exactly what movie you want to watch right this minute. And it doesn't know that I hope K12 dies a permanent death.
It's as simple as GIGO-- bad data processed poorly yields no useful results. Waving your laser pointer and intoning, "Look! Compuuuuters! Data! Data made out of numbers!! It's magical!!" will not convince me to cheerfully welcome my New Data Overlords.
Who Puts the Scary in Pearson? Meet Knewton.
Behind the data generating-and-collecting behemoth that is Pearson is a company called Knewton. And here's a video from the November 2012 Education Datapallooza (a name that I did NOT make up, but was officially given the event by the Dept of Education, because they are so hip. I believe they also listen to the rap music). In just under ten minutes, Jose Ferreira, Knewton CEO, delivers the clearest picture I've ever seen of the intentions of the Acolytes of Data. (H/T to Anne Patrick.)
He opens with the notion that in the next few decades, we will become a totally data mined world. There are plenty of reasons to be concerned about that, but that's another post. He may well be right. He believes that has big implications for education, because while everybody is just collecting data in dribs and drabs, education is the Great River O'Data.
Knewton is now (and remember-- "now" is 2012) collecting millions of data points per day per student. And they can do that because these are students who are plugged into Pearson, and Pearson has tagged every damn thing. And it was this point at which I had my first light bulb moment.
All that aligning we've been doing, all that work to mark our units and assignments and, in some places, every single work sheet and assignment so that we can show at a glance that these five sentences are tied to specific standards-- all those PD afternoons we spent marking Worksheet #3 as Standard LA.12.B.3.17-- that's not, as some of us have assumed, just the government's hamfisted way of making sure we've toed the line.
It's to generate data.
Worksheet #3 is tagged LA.12.B.3.17, so that when Pat does the sheet his score goes into the Big Data Cloud as part of the data picture of Pat's work. (If you'd already figured this out, forgive me-- I was never the fastest kid in class).
Knewton will generate this giant data picture. Ferreira presents this the same way you'd say, "Once we get milk and bread at the store," when I suspect it's really more on the order of "Once we cure cancer by using our anti-gravity skateboards," but never mind. Once the data maps are up and running, Knewton will start operating like a giant educational match.com, connecting Pat with a perfect educational match so that Pat's teacher in Iowa can use the technique that some other teacher used with some other kid in Minnesota. Because students are just data-generating widgets.
Ferreira is also impressed that the data was able to tell him that some students in a class are slow and struggling, while another student could take the final on Day 14 and get an A, and for the five billionth time I want to ask this Purveyor of Educational Revolution, "Just how stupid do you think teachers are?? Do you think we are actually incapable of figuring those sorts of things out on our own?"
But don't be insulted-- it's not just teachers who are stupid, but the students themselves. Knewton imagines a day when they can tell students how they best learn and under what conditions. Will you do best watching videos or reading? "We should be able to tell you what you should have for breakfast [to do well on a test]"
Because human beings are simple linear systems and if you measure all the inputs, you can predict all the outputs? That seems to be our assumption, and even I, a high school English teacher for crying out loud, know enough about chaos theory and the systems of complex systems to know that that is a fool's game. (If you want to read more about it, and you should, I highly recommend Chaos by James Gleick)
Beyond the privacy implications and the human-beings-as-widgets implications and the necessity to tag every damn sentence of every damn assignment so our data overlords may drink their fill-- beyond all that, there are implications for what an education means.
One aspect of becoming an educated person is getting to know yourself, to understand your strengths and weaknesses, your abilities and deficits, defining your own character, and making choices about how to be in the world as a your particular version of a human being.
How, I wonder, do we adjust to software that attempts to do most of that for you? How do you get to know who you are when you've got a software program looking over your shoulder and telling you all about who you are with implacable inhuman data-driven assurance? It's a huge question and one that I feel unsure of how to answer. I wish the guys at Knewton shared a little bit of my fear and unsureness.
UPDATE: Twitter user Barmak Nassirian directed my attention to this article, which provides an even more complete view of exactly how Knewton thinks they can accomplish their goals. It confirms the impression that these are guys who know a lot more about data systems than about carbon based life forms. It's long-- but it's interesting reading.
"The New Intelligence" by Steve Kolowich. Inside Higher Ed.
He opens with the notion that in the next few decades, we will become a totally data mined world. There are plenty of reasons to be concerned about that, but that's another post. He may well be right. He believes that has big implications for education, because while everybody is just collecting data in dribs and drabs, education is the Great River O'Data.
Knewton is now (and remember-- "now" is 2012) collecting millions of data points per day per student. And they can do that because these are students who are plugged into Pearson, and Pearson has tagged every damn thing. And it was this point at which I had my first light bulb moment.
All that aligning we've been doing, all that work to mark our units and assignments and, in some places, every single work sheet and assignment so that we can show at a glance that these five sentences are tied to specific standards-- all those PD afternoons we spent marking Worksheet #3 as Standard LA.12.B.3.17-- that's not, as some of us have assumed, just the government's hamfisted way of making sure we've toed the line.
It's to generate data.
Worksheet #3 is tagged LA.12.B.3.17, so that when Pat does the sheet his score goes into the Big Data Cloud as part of the data picture of Pat's work. (If you'd already figured this out, forgive me-- I was never the fastest kid in class).
Knewton will generate this giant data picture. Ferreira presents this the same way you'd say, "Once we get milk and bread at the store," when I suspect it's really more on the order of "Once we cure cancer by using our anti-gravity skateboards," but never mind. Once the data maps are up and running, Knewton will start operating like a giant educational match.com, connecting Pat with a perfect educational match so that Pat's teacher in Iowa can use the technique that some other teacher used with some other kid in Minnesota. Because students are just data-generating widgets.
Ferreira is also impressed that the data was able to tell him that some students in a class are slow and struggling, while another student could take the final on Day 14 and get an A, and for the five billionth time I want to ask this Purveyor of Educational Revolution, "Just how stupid do you think teachers are?? Do you think we are actually incapable of figuring those sorts of things out on our own?"
But don't be insulted-- it's not just teachers who are stupid, but the students themselves. Knewton imagines a day when they can tell students how they best learn and under what conditions. Will you do best watching videos or reading? "We should be able to tell you what you should have for breakfast [to do well on a test]"
Because human beings are simple linear systems and if you measure all the inputs, you can predict all the outputs? That seems to be our assumption, and even I, a high school English teacher for crying out loud, know enough about chaos theory and the systems of complex systems to know that that is a fool's game. (If you want to read more about it, and you should, I highly recommend Chaos by James Gleick)
Beyond the privacy implications and the human-beings-as-widgets implications and the necessity to tag every damn sentence of every damn assignment so our data overlords may drink their fill-- beyond all that, there are implications for what an education means.
One aspect of becoming an educated person is getting to know yourself, to understand your strengths and weaknesses, your abilities and deficits, defining your own character, and making choices about how to be in the world as a your particular version of a human being.
How, I wonder, do we adjust to software that attempts to do most of that for you? How do you get to know who you are when you've got a software program looking over your shoulder and telling you all about who you are with implacable inhuman data-driven assurance? It's a huge question and one that I feel unsure of how to answer. I wish the guys at Knewton shared a little bit of my fear and unsureness.
UPDATE: Twitter user Barmak Nassirian directed my attention to this article, which provides an even more complete view of exactly how Knewton thinks they can accomplish their goals. It confirms the impression that these are guys who know a lot more about data systems than about carbon based life forms. It's long-- but it's interesting reading.
"The New Intelligence" by Steve Kolowich. Inside Higher Ed.
Friday, March 21, 2014
In Praise of Non-Standardization
It is hard for me to argue with fans of national standards, because we hold fundamentally different values.
I'm opposed to CCSS, but unlike many other CCSS opponents, I'm opposed to any national standards at all. But it's hard to have that conversation because it comes down to this not-very-helpful exchange:
Standards fan: But if we had national standards, everyone would be on the same page. The system would be standardized. That's a good thing.
Me: No, it's not.
I'm not advocating the destruction of all rules and order. I'm not calling for the Land of Do-As-You-Please. But let me speak in praise of non-standardization.
Standardization is safe. It's predictable. We can walk into any McDonald's in the country and it will be just like any other and we will know exactly what we will get. I am not excited about that prospect. Let me plop you into the center of any mall in the country and defy you to guess where you are. That's not a good thing.
Complete organization and standardization is complete boredom. A canvas painted by Monet is interesting precisely because it is disorganized. There's more of some paint over here, less of the other paint over there. A wall painted by Bob's House Painting is perfectly orderly and organized. It's also flat and featureless and nobody particularly wants to look at it; in fact, once it has dried, the homeowners will break up its monotony by hanging photos or decorations or a print of a Monet painting.
Take a glass of water and drop one drop of food coloring into it. At first it will be a group of stark swirls against a clear background. It will be disorganized, disorderly. It will also be cool, interesting. After a while, it will be completely organized and orderly. And boring and uniform.
Chaos and information theories tell us that disorder and entropy are not necessarily best buds, that in fact achieving order and increasing entropy actually go hand in hand. Progress and creation arise out of chaos.
We don't have to be all philosophysicsy about this. Look at the arts. Watch the following process repeat over and over and over again:
1) The prevailing standard has become moribund and stultifying.
2) A large group of alternatives suddenly arise, almost simultaneously providing a whole host of exciting alternatives
3) Eventually one or two emerge as the "winners."
4) The winners cement their status as the new standard by becoming more orderly, more formalized, more organized (but less energetic)
5) See step 1. Rinse and repeat.
This covers everything from the French Impressionist movement to the rise of varied forms of Rock and Roll and Pop in response to the easy listening of the fifties. Or the arc of the computer software and app industry.
It is not just that the non-standard makes the world beautiful and interesting. It is the non-standard that is necessary for human beings to rise and advance. It is the non-standard that allows us to be our best selves, to express whatever unique blend of human qualities that birth and circumstances bring to us.
The goal of standardization is the exact opposite of what is, I would argue, the business of human life. We exist as human beings to make our mark, to make a difference, to be agents of change, to put our unique fingerprints on the things we touch. The goal of the standardized human is to not make a difference, to not leave a mark, to interact in the world in such a way that it would not have made the slightest difference if some other standardized human had been there in our place.
Some loose standardization greases the wheels of society, gives us a common foundation to develop our individual differences. But to imagine that standardization is in and of itself a high and desirable virtue is to imagine that a foundation is the only thing we need in a house. So no, I don't see some sort of national standard as a worthy goal.
I'm opposed to CCSS, but unlike many other CCSS opponents, I'm opposed to any national standards at all. But it's hard to have that conversation because it comes down to this not-very-helpful exchange:
Standards fan: But if we had national standards, everyone would be on the same page. The system would be standardized. That's a good thing.
Me: No, it's not.
I'm not advocating the destruction of all rules and order. I'm not calling for the Land of Do-As-You-Please. But let me speak in praise of non-standardization.
Standardization is safe. It's predictable. We can walk into any McDonald's in the country and it will be just like any other and we will know exactly what we will get. I am not excited about that prospect. Let me plop you into the center of any mall in the country and defy you to guess where you are. That's not a good thing.
Complete organization and standardization is complete boredom. A canvas painted by Monet is interesting precisely because it is disorganized. There's more of some paint over here, less of the other paint over there. A wall painted by Bob's House Painting is perfectly orderly and organized. It's also flat and featureless and nobody particularly wants to look at it; in fact, once it has dried, the homeowners will break up its monotony by hanging photos or decorations or a print of a Monet painting.
Take a glass of water and drop one drop of food coloring into it. At first it will be a group of stark swirls against a clear background. It will be disorganized, disorderly. It will also be cool, interesting. After a while, it will be completely organized and orderly. And boring and uniform.
Chaos and information theories tell us that disorder and entropy are not necessarily best buds, that in fact achieving order and increasing entropy actually go hand in hand. Progress and creation arise out of chaos.
We don't have to be all philosophysicsy about this. Look at the arts. Watch the following process repeat over and over and over again:
1) The prevailing standard has become moribund and stultifying.
2) A large group of alternatives suddenly arise, almost simultaneously providing a whole host of exciting alternatives
3) Eventually one or two emerge as the "winners."
4) The winners cement their status as the new standard by becoming more orderly, more formalized, more organized (but less energetic)
5) See step 1. Rinse and repeat.
This covers everything from the French Impressionist movement to the rise of varied forms of Rock and Roll and Pop in response to the easy listening of the fifties. Or the arc of the computer software and app industry.
It is not just that the non-standard makes the world beautiful and interesting. It is the non-standard that is necessary for human beings to rise and advance. It is the non-standard that allows us to be our best selves, to express whatever unique blend of human qualities that birth and circumstances bring to us.
The goal of standardization is the exact opposite of what is, I would argue, the business of human life. We exist as human beings to make our mark, to make a difference, to be agents of change, to put our unique fingerprints on the things we touch. The goal of the standardized human is to not make a difference, to not leave a mark, to interact in the world in such a way that it would not have made the slightest difference if some other standardized human had been there in our place.
Some loose standardization greases the wheels of society, gives us a common foundation to develop our individual differences. But to imagine that standardization is in and of itself a high and desirable virtue is to imagine that a foundation is the only thing we need in a house. So no, I don't see some sort of national standard as a worthy goal.
Standardized Tests Tell Nothing
Testy stuff experts could discuss all of the following in scholarly type terms, and God bless them for that. But let me try to explain in more ordinary English why standardized tests must fail, have failed, will always fail. There's one simple truth that the masters of test-driven accountability must wrestle with, and yet fail to even acknowledge:
It is not possible to know what is in another person's head.
We cannot know, with a perfect degree of certainty, what another person knows. Here's why.
Knowledge is not a block of amber.
First, what we call knowledge is plastic and elastic.
Last night I could not for the life of me come up with the name of a guy I went to school with. This morning I know it.
Forty years ago, I "knew" Spanish (although probably not well enough to converse with a native speaker). Today I can read a bunch, understand a little, speak barely any.
I know more when I am rested, excited and interested. I know less when I am tired, frustrated, angry or bored. This is also more true by a factor of several hundred if we are talking about any one of my various skill sets.
In short, my "knowledge" is not a block of immutable amber sitting in constant and unvarying form just waiting for someone to whip out their tape measure and measure it. Measuring knowledge is a little more like trying to measure a cloud with a t-square.
We aren't measuring what we're measuring.
We cannot literally measure what is going on in a student's head (at least, not yet). We can only measure how well the student completes certain tasks. The trick-- and it is a huge, huge, immensely difficult trick-- is to design tasks that could only be completed by somebody with the desired piece of knowledge.
A task is as simple as a multiple choice question or an in-depth paper. Same rules apply. I must design a task that could only be completed by somebody who knows the difference between red and blue. Or I must design a task that could only be completed by somebody who actually read and understood all of The Sun Also Rises.
We get this wrong all the time. All. The. Time. We ask a question to check for understanding in class, but we ask it in such a tone of voice that students with a good ear can tell what the answer is supposed to be. We think we have measured knowledge of the concept. We have actually measured the ability to come up with the correct answer for the question.
All we can ever measure, EVER, is how well the student completed the task.
Performance tasks are complicated as hell.
I have been a jazz trombonist my whole adult life. You could say that I "know"many songs-- let's pick "All of Me." Can we measure how well I know the song by listening to me perform it?
Let's see. I'm a trombone guy, so I rarely play the melody, though I probably could. But I'm a jazz guy, so I won't play it straight. And how I play it will depend on a variety of factors. How are the other guys in the band playing tonight? Do I have a good thing going with the drummer tonight, or are our heads in different places? Is the crowd attentive and responsive? Did I have a good day? Am I rested? Have I played this song a lot lately, or not so much? Have I ever played with this band before-- do I know their particular arrangement of the song? Is this a more modern group, because I'm a traditional (dixie) jazz player and if you start getting all Miles on me, I'll be lost. Is my horn in good shape, or is the slide sticking?
I could go on for another fifty questions, but you get the idea. My performance of a relatively simple task that you intended to use to measure my knowledge of "All of Me" is contingent on a zillion other things above and beyond my knowledge of "All of Me."
And you know what else? Because I'm a half-decent player, if all those other factors are going my way, I'll be able to make you think I know the song even if I've never heard it before in my life.
If you sit there with a note-by-note rubric of how you think I'm supposed to play the song, or a rubric given to you to use, because even though you're tone-deaf and rhythm-impaired, with rubric in hand you should be able to make an objective assessment-- it's hopeless. Your attempt to read the song library in my head is a miserable failure. You could have found out just as much by flipping a coin. You need to be knowledgeably yourself-- you need to know music, the song, the style, in order to make a judgment about whether I know what I'm doing or not.
You can't slice up a brain.
Recognizing that performance tasks are complicated and bubble tests aren't, standardized test seemed designed to rule out as many factors as possible.
In PA, we're big fans of questions that ask students to define a word based on context alone. For these questions, we provide a selection that uses an obscure meaning of an otherwise familiar word, so that we can test students' context clue skills by making all other sources of knowledge counter-productive.
Standardized tests are loaded with "trick" questions, which I of course am forbidden to reveal, because part of the artificial nature of these tasks is that they must be handled with no preparation and within a short timespan.But here's a hypothetical that I think comes close.
We'll show a small child three pictures (since they are taken from the National Bad Test Clip Art directory, there's yet another hurdle to get over). We show a picture of a house, a tent and a cave. We ask the child which is a picture of a dirt home. But only the picture of the house has a sign that says, "Home Sweet Home" over the door. Want to guess which picture a six-year-old will pick? We're going to say the child who picked the cave failed to show understanding of the word "dirt." I'd say the test writers failed to design an assessment that will tell them whether the child knows the meaning of the word "dirt" or not.
Likewise, reading selections for standardized tests are usually chosen from The Grand Collection of Boring Material That No Live Human Being Would Ever Choose To Read. I can only assume that the reasoning here is that we want to see how well students read when they are not engaged at all. If you're reading something profoundly boring, then only your reading skills are involved, and no factors related to actual human engagement.
These are performance task strategies that require the student to only use one slice of brain while ignoring all other slices, an approach to problem solving that is used nowhere, ever, by actual real human beings.
False Positives, Too
The smartest students learn to game the system, which invariably means figuring out how to complete the task without worrying about what the task pretends to measure. For instance, for many performance tasks for a reading unit, Sparknotes will provide just as much info as the students need. Do you pull worksheets and unit quizzes from the internet? Then your students know the real task at hand is "Find Mr. Bogswaller's internet source for answer keys."
Students learn how to read teachers, how to divine expectations, what tricks to expect and how to generally beat the system by providing the answers to the test without possessing the knowledge that the test is supposed to test for.
The Mother of all Measure
Tasks, whether bubble tests or complex papers, may assess for any number of things from students's cleverness to how well-rested they are. But they almost always test one thing above all others-
Is the student any good at thinking like the person who designed the task?
Our students do Study Island (an internet-based tutorial program) in math classes here. They may or may not learn much math on the island, but they definitely learn to think the same way the program writers think.
When we talk about factors like the colossal cultural bias of the SAT, we're talking about the fact that the well-off children of college-educated parents have an edge in thinking along the same lines as the well-off college-educated writers of the test.
You can be an idiot, but still be good at following the thoughty paths of People in Charge. You can be enormously knowledgeable and fail miserably at thinking like the person who's testing you.
And the Father of all Measure
Do I care to bother? When you try to measure me, do I feel even the slightest urge to co-operate?
Standardized tests are a joke
For all these reasons, standardized tests are a waste of everybody's time. They cannot measure the things they claim to measure any better than tea leaves or rice thrown on the floor.
People in the testing industry have spent so much time convincing themselves that aspects of human intelligence can be measured (and then using their own measurements of measurement to create self-justifying prophecies) that they've lost fact of that simple fact:
You cannot know what's in another person's head
What goes on in my head is the last boundary I have that you cannot cross. I can lie to you. I can fake it. I can use one skill to substitute for another (like that kid in class who can barely read but remembers every word you say). Or I may not be up to the task for any number of reasons.
Standardized test fans are like people who measure the circumference of a branch from the end of a tree limb and declare they now have an exact picture of the whole forest. There are many questions I want to ask (in a very loud voice that might somewhat resemble screaming) of testmakers, but the most fundamental one is, "How can you possibly imagine that we are learning anything at all useful from the results of this test?"
It is not possible to know what is in another person's head.
We cannot know, with a perfect degree of certainty, what another person knows. Here's why.
Knowledge is not a block of amber.
First, what we call knowledge is plastic and elastic.
Last night I could not for the life of me come up with the name of a guy I went to school with. This morning I know it.
Forty years ago, I "knew" Spanish (although probably not well enough to converse with a native speaker). Today I can read a bunch, understand a little, speak barely any.
I know more when I am rested, excited and interested. I know less when I am tired, frustrated, angry or bored. This is also more true by a factor of several hundred if we are talking about any one of my various skill sets.
In short, my "knowledge" is not a block of immutable amber sitting in constant and unvarying form just waiting for someone to whip out their tape measure and measure it. Measuring knowledge is a little more like trying to measure a cloud with a t-square.
We aren't measuring what we're measuring.
We cannot literally measure what is going on in a student's head (at least, not yet). We can only measure how well the student completes certain tasks. The trick-- and it is a huge, huge, immensely difficult trick-- is to design tasks that could only be completed by somebody with the desired piece of knowledge.
A task is as simple as a multiple choice question or an in-depth paper. Same rules apply. I must design a task that could only be completed by somebody who knows the difference between red and blue. Or I must design a task that could only be completed by somebody who actually read and understood all of The Sun Also Rises.
We get this wrong all the time. All. The. Time. We ask a question to check for understanding in class, but we ask it in such a tone of voice that students with a good ear can tell what the answer is supposed to be. We think we have measured knowledge of the concept. We have actually measured the ability to come up with the correct answer for the question.
All we can ever measure, EVER, is how well the student completed the task.
Performance tasks are complicated as hell.
I have been a jazz trombonist my whole adult life. You could say that I "know"many songs-- let's pick "All of Me." Can we measure how well I know the song by listening to me perform it?
Let's see. I'm a trombone guy, so I rarely play the melody, though I probably could. But I'm a jazz guy, so I won't play it straight. And how I play it will depend on a variety of factors. How are the other guys in the band playing tonight? Do I have a good thing going with the drummer tonight, or are our heads in different places? Is the crowd attentive and responsive? Did I have a good day? Am I rested? Have I played this song a lot lately, or not so much? Have I ever played with this band before-- do I know their particular arrangement of the song? Is this a more modern group, because I'm a traditional (dixie) jazz player and if you start getting all Miles on me, I'll be lost. Is my horn in good shape, or is the slide sticking?
I could go on for another fifty questions, but you get the idea. My performance of a relatively simple task that you intended to use to measure my knowledge of "All of Me" is contingent on a zillion other things above and beyond my knowledge of "All of Me."
And you know what else? Because I'm a half-decent player, if all those other factors are going my way, I'll be able to make you think I know the song even if I've never heard it before in my life.
If you sit there with a note-by-note rubric of how you think I'm supposed to play the song, or a rubric given to you to use, because even though you're tone-deaf and rhythm-impaired, with rubric in hand you should be able to make an objective assessment-- it's hopeless. Your attempt to read the song library in my head is a miserable failure. You could have found out just as much by flipping a coin. You need to be knowledgeably yourself-- you need to know music, the song, the style, in order to make a judgment about whether I know what I'm doing or not.
You can't slice up a brain.
Recognizing that performance tasks are complicated and bubble tests aren't, standardized test seemed designed to rule out as many factors as possible.
In PA, we're big fans of questions that ask students to define a word based on context alone. For these questions, we provide a selection that uses an obscure meaning of an otherwise familiar word, so that we can test students' context clue skills by making all other sources of knowledge counter-productive.
Standardized tests are loaded with "trick" questions, which I of course am forbidden to reveal, because part of the artificial nature of these tasks is that they must be handled with no preparation and within a short timespan.But here's a hypothetical that I think comes close.
We'll show a small child three pictures (since they are taken from the National Bad Test Clip Art directory, there's yet another hurdle to get over). We show a picture of a house, a tent and a cave. We ask the child which is a picture of a dirt home. But only the picture of the house has a sign that says, "Home Sweet Home" over the door. Want to guess which picture a six-year-old will pick? We're going to say the child who picked the cave failed to show understanding of the word "dirt." I'd say the test writers failed to design an assessment that will tell them whether the child knows the meaning of the word "dirt" or not.
Likewise, reading selections for standardized tests are usually chosen from The Grand Collection of Boring Material That No Live Human Being Would Ever Choose To Read. I can only assume that the reasoning here is that we want to see how well students read when they are not engaged at all. If you're reading something profoundly boring, then only your reading skills are involved, and no factors related to actual human engagement.
These are performance task strategies that require the student to only use one slice of brain while ignoring all other slices, an approach to problem solving that is used nowhere, ever, by actual real human beings.
False Positives, Too
The smartest students learn to game the system, which invariably means figuring out how to complete the task without worrying about what the task pretends to measure. For instance, for many performance tasks for a reading unit, Sparknotes will provide just as much info as the students need. Do you pull worksheets and unit quizzes from the internet? Then your students know the real task at hand is "Find Mr. Bogswaller's internet source for answer keys."
Students learn how to read teachers, how to divine expectations, what tricks to expect and how to generally beat the system by providing the answers to the test without possessing the knowledge that the test is supposed to test for.
The Mother of all Measure
Tasks, whether bubble tests or complex papers, may assess for any number of things from students's cleverness to how well-rested they are. But they almost always test one thing above all others-
Is the student any good at thinking like the person who designed the task?
Our students do Study Island (an internet-based tutorial program) in math classes here. They may or may not learn much math on the island, but they definitely learn to think the same way the program writers think.
When we talk about factors like the colossal cultural bias of the SAT, we're talking about the fact that the well-off children of college-educated parents have an edge in thinking along the same lines as the well-off college-educated writers of the test.
You can be an idiot, but still be good at following the thoughty paths of People in Charge. You can be enormously knowledgeable and fail miserably at thinking like the person who's testing you.
And the Father of all Measure
Do I care to bother? When you try to measure me, do I feel even the slightest urge to co-operate?
Standardized tests are a joke
For all these reasons, standardized tests are a waste of everybody's time. They cannot measure the things they claim to measure any better than tea leaves or rice thrown on the floor.
People in the testing industry have spent so much time convincing themselves that aspects of human intelligence can be measured (and then using their own measurements of measurement to create self-justifying prophecies) that they've lost fact of that simple fact:
You cannot know what's in another person's head
What goes on in my head is the last boundary I have that you cannot cross. I can lie to you. I can fake it. I can use one skill to substitute for another (like that kid in class who can barely read but remembers every word you say). Or I may not be up to the task for any number of reasons.
Standardized test fans are like people who measure the circumference of a branch from the end of a tree limb and declare they now have an exact picture of the whole forest. There are many questions I want to ask (in a very loud voice that might somewhat resemble screaming) of testmakers, but the most fundamental one is, "How can you possibly imagine that we are learning anything at all useful from the results of this test?"
Subscribe to:
Posts (Atom)