New pet peeve: "Relatively unique." You can no more be "relatively unique" than relatively pregnant.— Robert Pondiscio (@rpondiscio) June 29, 2016
I totally owe this post to Pondiscio's tweet.
First, I share his pet peeve (even though I'm not prepared to bet anything valuable that I've never violated it). You are unique or you're not. You are pregnant or you're not. There is no relatively unique, relatively pregnant.
Put another way, when you go to the doctor to see if you're pregnant or not, the doctor does not say, "Well, let me draw some blood. As soon as I've collected samples from a few hundred other women, I'll be able to decide whether you're relatively pregnant or not." No, there is no "relatively pregnant."
You take the test. You find out whether or not you're pregnant. You are just as pregnant when you take the test alone as you are when you take it at the same time as thousand so of other women.
Yes, this is yet another way to look at the difference between standards-referenced and norm-referenced testing.
If the Big Standardized Test (PARCC, SBA, PSSA, MOUSE, ETC) were truly a standards-referenced test, a student could take the test and know if she were proficient or not. And that is the implied promise of the BS Tests-- that they will measure proficiency against a standard.
But if that were true, we would not need thousands of students to take the test. One student, alone, could take the test and be told whether she was proficient or basic or whatever. No test manufacturer, no state would have to say, "Okay, as soon as we have the test results from thousands of other students, we'll be able to tell you if you're proficient."
One student, alone, could take the test and be given the results. We would not need the state government, the federal government, and the testocrats to say, "We can't have opt out because every child must take the test because if every child doesn't take the test, we won't get meaningful results."
We have been promised a test that tells us whether or not a student is proficient in reading and math. We were promised a test that would tell us whether or not a student is, in absolute terms, proficient, giving us, as Arne Duncan put it, the power to look an eight year old in the face and tell her whether or not she's on track for college.
What we have instead is a test that tells us if a student is relatively proficient. Which makes no more sense than a test to determine if a woman is relatively pregnant.
One of my favorite quotes has long been from John Scalzi: "Being an expert/pro doesn’t mean you’re right about everything in your field. It does mean you likely know when others are wrong about it."
ReplyDeleteAnd therein lies the challenge with posts like this: you're relatively right. In theory, one child should be able to take the test and know how they're doing against the standards. The challenge, is that's not how it works.
These tests are not designed for one child - they are designed for hundreds, even thousands, of children. They were designed - using protocols and structures with a solid research base - in response to a bi-partisan law. The reasons for the delay between administration and score release are boring, technical, and not interesting. They're just part of the completely unsexy work that is large-scale test design. The often overlooked truth of those who design these "BS" tests is that they are the first to advocate for multiple measures and not treating test scores as anything more than a piece of data. You don't have to believe me - check out the AERA or NCME websites and read their policy statements or research studies.
Posts like these are frustrating as they seem to want to set up test designers as teachers'antagonists. And my frustration is second-hand; I live in the world of authentic assessment design and help teachers eliminate multiple choice tests.
All of which is to say: what's the point of a post like this? What's gained by offering information to your readers that is relatively correct?
The point of writing a post like this is that I, like a huge number of teachers in the field, have the sense that there is something profoundly wrong with these tests and how they are used, and we all struggle to find ways to express what we sense.
DeleteI'm inclined not to think of test designers as teacher antagonists. Test marketing companies and policy makers, on the other hand, through their sometimes confused, sometimes deliberate misuse of test, or marketing tests by suggesting they have qualities they don't have-- those folks I have a problem with.
Ultimately, as you know, I would prefer to see the BS Tests go away. They waste time, money, resources, cause a lot of trouble, and give virtually nothing back in return.
When it comes to test designers, I feel like I keep looping through versions of this conversation:
Teacher: The nuclear power source you developed is being used to poison our town and is giving our residents cancer. In the next state over, it has been used to bomb a city into rubble.
Test designers: We designed it to provide safe, clean energy.
Teacher: That's not how it's being used. It's being used to destroy buildings and human beings.
Test designers: Well, that's not what it is designed for. That's not how it's supposed to be used. It's a safe, clean energy source.
Teachers: Dude, that is not how they are being used.
Test designers: When we handed them over to our bosses, we handed them clear instructions. I'm sure they understood what we were telling them. They're just a clean, safe energy source. You should try to relax.
I recently took and passed the test for the motorcycle endorsement on my drivers license. As I answered each question, the machine told me whether my answer was right or wrong. It works the same way in my college math classes -- answers are either right or wrong, and are completely unaffected by how anyone else does on the test. The question I would ask Jennifer is -- if you can't even convince a couple of teachers like Peter and myself (and yes, I understand pretty high level math), how would you convince the general public ? And if you can't convince the public, then what good are the tests themselves ?
ReplyDeleteI have been asking non-math types (parents) about this for years -- I believe, and parents in general believe, that if your test requires you to know what other students did before you can know whether this test taker is proficient (or any other level), then you're doing it wrong. Most parents don't support these tests anyway, according to national polls, but for those that have -- all you need to do is explain this one little fact to them -- and their support is quite likely to vanish. In other words, the public believes that tests ought to work just like the motorcycle endorsement test -- you know the material, or not. What other folks do doesn't matter.
I had some exposure to what I consider to be the mathematically indefensible thinking of the psychometrician community when I challenged the purportedly correct answer to one of the sixth grade SBAC math problems. The math director of SBAC informed me that after the field testing, they would be able to determine whether the answer I told her was mathematically correct was actually correct and should be accepted. The problem with this answer is that you didn't need field testing to know that their answer was mathematically incorrect, period. Psychometricians have simply lost their grip on reality if they think otherwise.
"...if you can't even convince a couple of teachers like Peter and myself (and yes, I understand pretty high level math), how would you convince the general public?"
DeleteI suspect it's easier to convince an uninformed public. :-( The less information they have, the easier a job convincing them.
You're right, in the sense that the public actually ASSUMES that these school tests are like the driver tests....once they find out otherwise, there'll be no convincing them of the usefulness of these tests at all.
DeleteTo your analogy, Peter - test designers are absolutely not telling teachers to "relax." Read the NCME and AERA policy statements. Look at the conference session titles at their annual conferences.
ReplyDeleteAnd Julie, I am not trying to convince you to like the tests. Trust me on this one. I've written at length about alternatives to large scale multiple choice tests. We can do better. We must do better. My goal is speaking up is to offer that psychometrics is a highly specific area of training. To say that "psychometricians have lost their grip on reality" because one did something you disagree with is like walking into an operating room and telling a brain surgeon they're messing it up because you read a book on meditation. Large-scale test design is inaccessible to the public in the same way brain surgery is. Either we trust the professionals doing the work... or we don't. And given with PARCC, the vast majority of those professionals are classroom teachers, there's cognitive dissonance writ large.
Here's the sticky wicket of this conversation and to the point of Peter's first paragraph: Large-scale tests go through more rigorous reviews, rounds of feedback, and field testing any teacher-created test. Point to a student who failed a course because they got a 64 on their teacher-created final exam, and you've identified a student who likely would have passed with a 66 had the test been reviewed for measurement error and threats to reliability. There is an incredibly painful tension when there is pushback against a test that has 19 steps of design at the same time teachers are making the news for giving kids a final exam that asks about "gangbangers" and kilos of coke.
Don't like how the results large-scale tests are used? Great. Let's rail together against bad policy. Don't like how bias in large-scale tests can potentially reinforce the inherent racism and unfairness of our public ed system? Join the psychometricians who are working to ensure that doesn't happen.
All of the psychomagians on Earth and all of the BS standardized test jargon you want to use cannot change the fact that everything you do and say and believe about these Common Core tests is dependent on one very false assumption: that students care about them and try their best. Sorry, not happening.
DeleteAnd no you cannot create reliable and valid alternatives to the MC format for testing tens of millions of students.
"Large-scale test design is inaccessible to the public in the same way brain surgery is. Either we trust the professionals doing the work... or we don't. And given with PARCC, the vast majority of those professionals are classroom teachers, there's cognitive dissonance writ large."
DeleteWait, I agree that the vast majority of people implementing Common Core, which is tested by PARCC (among other tests), are classroom teachers, but I hope you're not trying to convince me that the "vast majority of professionals" involved in creating PARCC were classroom teachers. Relatively (see what I did there?) few were involved even in the review stage, and certainly no "vast majority" in the design stage.
And Jenn, apologies if I'm outright misunderstanding your quote. I'm only on my first cup of coffee. :-)
DeleteI meant the former - that is the vast majority of eyeballs that looked at PARCC in the design process belonged to teachers. It's one of the things that makes PARCC so different than previous NCLB-mandated tests.
DeleteCan't hyperlink but I pulled together a bunch of examples here: https://jennbbinis.com/2016/05/18/vegetarian-butchers-unite/
Count the actual classroom teachers, then: http://blogs.edweek.org/teachers/living-in-dialogue/2009/07/national_standards_process_ign.html
DeleteAnd yes, that was Common Core, not PARCC....(hit the wrong button)...but the link you provided was not...substantial. A link to K-12 educator *reviewers* - not test designers, a link to one ELA teacher about CCSS & PARCC, more about test *review* (not creation)...Sorry, but can't take that seriously as "the vast majority of eyeballs that looked at PARCC in the design process belong[ing] to teachers."
DeleteNot sure what else to say here, Crunch. I could give the phone number of a friend of mine who worked on the design committee. I could share pictures taken of the rooms filled with teachers during the design process. I could continue to hunt down links to show what made PARCC design so different but... one can only go around the merry-go-round so many times. Have a good one! :)
DeleteSuppose I want to know whether a kid can find angles from a triangle, using the law of cosines. I give them a triangle with three sides known, and see if they can calculate the angles using the law of cosines. It's simple and they can either do it or not. One kid at a time -- either they can or they can't, or maybe they can set it up partially but make some sort of mistake, etc. Now, this is the way we do it in college classes -- in college classes everywhere, no need at all for standardized testing (I've never even heard anyone offer to create such things, let alone for teachers to want them). This is also the way it is and should be done in high school. The standardized tests my kids take test an extremely small percentage of what they know and learn, at best. And regardless, how other kids do is simply irrelevant to the question of whether a kid knows the material he is held accountable to know. ALL of math is like this -- the SBAC question I referred to had exactly one right answer -- the one I gave, not the one that the psychometricians gave. The psychometricians' plan, though, was to count it right for their particular chosen wrong answer, and to count my answer wrong.
ReplyDeleteIt is true that I am not a psychometrician -- and true that they are entitled to their own expertise, such as it is -- however, they are most emphatically not entitled to create their own math, a math that is different from the generally accepted math that is used, for example, in school and in science and engineering. So, no, I do not accept that. In your surgery analogy, even I could have seen if they were supposed to be operating on a person's leg and they were in reality cutting open their head. Or if they were supposed to be operating on David, and instead they were operating on Maureen. And I would hope that the other professionals in the room -- nurses, assistants, anesthetists, etc. would all chime in on this as well. And all of us that are math-savvy see how math and statistics are used to misinform and confuse people all the time -- why would we "trust" that our need for a product "standardized tests" exists because some others say we need it ? We wouldn't, and shouldn't.
Beyond that, I would say that this work with tests is not necessary. It certainly is not helpful. And to use your argument -- we either trust the teachers to be able to tell if their students learned what they were supposed to learn, or we don't. In my case, no, I do not trust the SBAC designers -- clearly, they do not understand the math (or, in the case of the question I challenged, simple directions in English) -- I simply cannot trust the results of their test, because I know, for example, that I would have gotten that sixth grade question "wrong" -- yet, I was right and they were wrong. It's dead simple, and there's no reason to behave as if it's just too complicated for ordinary people to understand.
Standardized tests are not used in college classes, and they should not be used anywhere else either. Everyone who pays attention to this sort of thing realizes that teacher assigned grades are more predictive of students' future success than are standardized test scores. This fact strongly supports the claim that teachers' assessments are more predictive (though grades do reflect some of those soft skills, also important in future success) than the standardized test scores. So, I'm sorry, Jennifer -- thank you for your response, but all of this test design is unnecessary and unhelpful. I look forward to the time it goes the way of phrenology, taking the last gasps of the rest of the eugenics movement with it.
Julie - Every time a classroom teacher gives a multiple choice test, they are using the tools of the psychometric field. Every time a teacher puts a problem on test to figure out if a student understands trig, they are using the fruit of the psychometric tree.
ReplyDeleteStandardized tests are used in college classes - give a task to more than one student? That's standardized format. Score all students' work using the same rubric or scoring schema? That's standardized.
I respect your claim that test design is unnecessary. I would invite you to go first on a plane with a pilot who hasn't passed her pilot license exam.
Teachers have been giving math tests to students for millenia. One doesn't need the fruit of the psychometric tree to give a math test.
DeleteThe field of psychometrics was, in effect, discovered during the Sing Dynasty when the emperor designed a series of tasks for potential government employees to complete.
DeleteAll measures of human learning are fruits of that tree.
From Jenn...
Delete"Standardized tests are used in college classes - give a task to more than one student? That's standardized format. Score all students' work using the same rubric or scoring schema? That's standardized."
I think we all here know the difference between The Tests Professor Jenn gives all her Euclidean Geometry sections and the (hypothetical) BS tests that would be given to all Euclidean Geometry college classes nationwide.
There's "standardized" and there's "Standardized" with a Capital S. you know, the BST, not just what Professor Jenn made for her own classes.
I can't believe we have to make that distinction here, of all places.
Hello again, Cruch! On the issue of standardization - Peter and I have exchanged a few posts on the topic. The last one: https://jennbbinis.com/2015/10/30/what-exactly-is-standardization-in-assessment-design/
DeleteHow in the world could math tests given in ancient Greece, say at Plato's academy, be fruits of a tree that supposedly started in the Sing (Song?) Dynasty? That is absurd. You are confusing the creation of a test with the study of testing and human measurement. Two different things. In fact, you are so loose with your terms and statements, that you don't seem to have any serious concern with the truth.
DeleteEric - you sweet talker, you!
DeleteI wasn't confusing them - I was speaking to the issue of scale. What happened at the Academies was intimate, ad hoc and often informal. The Shang (not Sing - autocorrect got me there) Dynasty exams were about taking it to scale - about sampling a skill set in order to make inferences about a test takers abilities. I'm fully aware that the field of pyschometrics wouldn't emerge until much later.
In other news, be sure to check out my podcast, Ed History 101 - in the episode on grades, I go into the topic in more depth :)
Mr. Greene, thank you for this piece. As is typical of your work, it gives good voice to thoughts many of us have but can't express as eloquently.
ReplyDeleteMs. Binis, your insight into BS testing is helpful. My question to you is, if the psyshometricians are doing such a great job, why are so many tests so full of errors? It's profoundly difficult to trust in the people who assume teachers and parents can't understand the boring unsexy aspects of standardized testing when it's clear to us that the testing folks can't produce questions which have correct answers choices. And I'd like to point out that your reference to the pilot who didn't pass the licensing exam would be a standards-based testing situation, not a norm-referenced one, which rather proves Mr. Greene's point.
I don't give multiple choice tests, ever. But the question is whether we need tests on which it matters what OTHER people do in order to say whether this somebody is "proficient" or "has passed". I don't know about pilot testing -- but the motorcycle qualification is determined by a type of test on which it does NOT matter what others do. The road test (is it called "air test" for pilots ?) certainly won't be like that -- you'll know immediately if you are considered "proficient". That what Peter's topic was and also my point about the SBAC test which therefore causes me to believe that test -- and all other tests designed using those principles -- are neither legitimate nor helpful.
ReplyDeleteThe only tests in education where a child's performance is truly compared to how other students', is in special education. WISC-R, IQ tests, HELP, etc.
DeleteLet's say, hypothetically, that all students got almost every single item/task correct on an NCLB-mandated test (PARCC, etc.). All students would be proficient. The students who got 99% correct wouldn't "fail" or be deemed "not proficient" even though some students scored higher than them.
Well, that certainly throws the validity of the "But how will we know if my kid in MD is getting a better education (aka "scoring higher") than some kid in CA?" into some doubt.
DeleteHow so? And feel free to shout at my over on Twitter so we don't blow up Peter's comments.
DeleteNah, closing out here, trying to spend less time on long Twitter threads these days.
DeleteYou said (paraphrasing) that only special ed tests compare students, while test proponents too often insist that it's important to use BST's to be able to compare how students do in different schools, cities, states.
Which is it?
There are different ways of comparing students. In a norm-referenced test (i.e the tests used to identify children with disabilities), a child's performance is compared to his/her peers to determine his/her scores. For large scale tests, the goal is compare students' performance across subgroups (the so-called "achievement gap"), schools, districts, or states. So ... it's both.
DeleteMs. Wilkinson - If I may offer two quick clarifications. Tests that were designed and implemented for NCLB - which Mr. Greene calls "BS" are standards-based. Norm-referenced tests are primarily used education to identify students with disabilities. Please see Mr. Greene's previous post on standards and norms for more about that. Second, my claim isn't that they're doing a great job per se. Rather offering that the job that critiquing them as akin to shooting the profession in the foot. Every single day, millions of teachers use psychometric theories and practices in their classroom.
ReplyDelete"So full of errors" is a hard one to unpack. So, I would be curious what you have in mind when you say "error." Granted, there are printing snafus that can happen. That's one of the reasons there's a delay between administration and score release.
For example, in 2004 (I believe), there was a printing error on the NYS 4th grade math test resulting in a line being off on one problems. Students got the item wrong at a rate that far exceeded what the test designers expected (i.e. they thought it was an easy item, students responded as if it was hard). The item was removed from students' scores.
I'm sure that was a comfort to all the 9YO's who were smacking their heads on desks over the problem at the time....oh, wait, they'll never know.
DeleteIt's likely not. No moreso than it's a comfort to the child who gets a 64 on a teacher-created test to hear "sorry, nothing I can do. You got a 64."
DeleteIn the comment above, NY Teacher criticizes the exams because "... everything you do and say and believe about these Common Core tests is dependent on one very false assumption: that students care about them and try their best. Sorry, not happening."
DeleteWhy would a student smack their heads on desks over a problem in an exam that they do not care about?
TE, I wasn't responding to NY Teacher. I was responding to Jenn. But thanks for playing.
DeleteAnd Jenn, a real-life flesh-and-blood teacher not only knows what the student got wrong but can show that student which question they got wrong, do it in a timely fashion, AND can show the student how to do it correctly, NONE of which the BS tests are designed or able to do. There's no transparency, there's no timeliness, and there's no opportunity for remediation. For that matter, the real-life teacher who created the test has the option to re-test one or many or all of the children to check for mastery before moving on, and/or to amend the grade accordingly (or not) - again, not a feature of the BST.
Jennifer
DeleteIf a student got a 64 on a teacher-created test, depending on what the test was for, the teacher could see where the problem areas are, tutor the student, and re-give the test. That cannot be done with the BS tests because we can't even see them.
So the director of SBAC is not a good person to speak for the field of psychometrics?
So you're saying that any test a teacher gives is using psychometrics, yet psychometrics is so complicated it's like rocket science, and the reason for a standards-based test to not be graded immediately like the motorcycle exam is impossible to explain? A good teacher can explain anything they understand to anyone so that the other person can also understand it.
Why are there so many errors? Why are the ELA standards and associated tests developmentally inappropriate? Maybe because the author, David Coleman, is not an expert in anything except perhaps "close reading."
Your definition of "standardized test" is any test given to more than one person. My definition of "standardized test" is a test not created by the teacher who did the actual teaching.
Neither of your analogies, the one about the brain surgeon/meditation book, or that of the pilot license, makes any sense. I hope you're not putting analogies on the tests you design.
Designing assessments is only one small part of the learning process and one small part of what we do as teachers. It's by far the easiest part. Designing how to teach the lesson is much harder. Creating a community in the classroom is much harder. The assessment design comes from the design of the teaching.
You say you're not a fan of the way the BS tests are being misused. Then you shouldn't be upset when Peter criticizes this. If I were a psychometrician, and my tests were being used in ways they were not designed for and were causing harm, which these tests are, I would be the most vocal voice against them, join with others who also realized this, and refuse to make them if they're going to be misused. Otherwise it is like Peter's analogy of nuclear power. Or, if this isn't too much over the top, maybe it's analogous to the way psychologists were complicit in CIA torture. I certainly would refuse to be knowingly involved in causing the real harm that these tests are causing unless I had a metaphorical gun to my head.
The authors, designers, etc. of large-scale tests never claimed to do any of those things. Different tests have different purposes.
DeleteRebecca - there's a reason I referenced "teacher-created final exams." Those are the tests that often have the highest stakes for students.
DeleteTo the error issue, I don't know to what the other commentator was referring so there's little to be said there. Coleman is a strawman.
In truth, I'm not sure what point you're trying to make. If it's that psychometrics is indefensible, then we need to have a conversation about why so many teachers like multiple choice. If your point is that testing is the least important part of teaching and learning, you're choir preaching.
Ms. Binis, I'm not certain how you arrived at the conclusion that BS tests are standard-based. Mr. Greene clarified that point pretty clearly. As for examples of the rampant number of errors in the tests, there have been hundreds of blog post & articles describing these, so you might want to spend some time reviewing that literature. These errors are clearly far more than occasional printing snafus, and range from questions with confusing wording to questions with no correct answer. I also find your thinking odd about how critiquing BS testing is shooting the teaching profession in the foot, as it appears that the testing is a far larger problem than the critiquing. At this point, the theory behind these tests matters far less than the truth that they are pointless, detrimental, and expensive.
DeleteMs Wilkinson - Allow me to offer another possibility - that perhaps Mr. Greene is mistaken. That the tests are, in fact, standards based.
DeleteThe tests may be all of those things. They may also be helpful to researchers and policymakers. IMO, what matters more is what assessment looks like the other 176 days of the year.
"Full of errors" -- a good example is the problem I raised with the SBAC people. They literally claimed that to know the correct answer one has to have field tests....but this is not the reality as we teach it in mathematics. We have correct answers, and incorrect answers. In the case of the SBAC test, the psychometricians who support it clearly believe that it does not matter which answer a mathematician would consider correct, and that only field testing can resolve the question of whether a mathematician knows math. This argument is clearly ridiculous. This is the kind of error that -- once the psychometricians know about it and deliberately refuse to correct it by in fact simply denying that it is correct -- causes teachers to have "no faith at all" in the results of these tests. I could add that I've also known a lot of kids who can "pass" multiple choice "standardized" tests but who do not, in fact, understand the material at all. So, if I am assessing students on their proficiency, one method of assessing I would never choose is a test written or designed by a psychometrician. If that were the only test available to me, I would simply have to resign myself to not knowing!
ReplyDeleteHum... I'm not sure how to reconcile the statement a "I would never chose is test written or designed by a psychometrician" as you took one when you took your motorcycle test, and your teacher cert test, etc. etc.
DeleteMy hunch is you ran into a member of the SBAC staff that thought you were asking something different than what you were - someone who doesn't speak for the entire profession or field.
I think you will also find many students who understand mathematics deeply and do much better on standardized exams than would be predicted by teacher assigned grades. That was certainly the case for my middle son.
DeleteJen states, “Let's say, hypothetically, that all students got almost every single item/task correct on an NCLB-mandated test (PARCC, etc.). All students would be proficient. The students who got 99% correct wouldn't "fail" or be deemed "not proficient" even though some students scored higher than them.”
ReplyDeleteThis is an absurd suggestion. The type of standardized tests (PARRC, SBAC, SAT, ACT, etc.) we are discussing here are developed using a wide range of pre-tested items that makes this IMPOSSIBLE. The sole intent of standardized test design is to create a bell curve into which an arbitrary, “pass-fail” cut score is placed. Your insistence that true Standardized tests are criterion referenced is ridiculous. Conflating the development and use of true Standardized tests and common, every-day teacher-made classroom quizzes or tests is weak attempt at distraction. You talk in circles about simple straight forward ideas and cannot provide simple yes or no answers to objective questions. When I asked you in the last thread about using the MC format for subjective test items (in ELA) you were unwilling to refute this obvious form of test design malpractice was very revealing.
Jen, you have spent far too much time in the assessment bubble because you show no sign of understanding the overriding point here: the BS tests have NO educational value – they cannot inform instruction, they do not promote authentic teaching and learning, they waste valuable time and money, and their misuse has undermined healthy classroom dynamics and best teaching practices. All the testo-babble in the world will never change that. And based on most of your commentary here, no, we are not preaching to the choir; we are preaching to just another testocrat.
NY teacher - it's Jenn. Two N's.
ReplyDeleteI'm not sure what else to say here. I will offer that, while I enjoy being snarked at as much as the next person, at no point have I said the tests promote authentic teaching and learning, or have I defended their mis-use.
My goal is speaking up on this post was to offer some clarity around a giant albatross in the education system. As an advocate for and defender of authentic and meaningful assessment, I believe we're better served by understanding the albatross, calling it what it is, and moving on.
I have no idea what a testocrat is. Though I do admire the linguistic creativity. And to the previous thread - I suspect you missed my follow-up question to you about MC questions.
When I asked you in the last thread about using the MC format for subjective test items (in ELA) you were unwilling to refute this obvious form of test design malpractice was very revealing.
DeleteWill you refute it here? And now? And in clear terms?
I did not coin the term testocrat, but I will take credit (linguistic creativity?) for the the very descriptive word "psychomagician" on whose side you clearly stand.
And "what", exactly, is this "albatross"? Because I have no clarity on it from what you've said.
DeleteNY Teacher - if you're asking me if I think Multiple Choice is a good format for literacy - No. No, I do not think MC is ideal when it comes to literature and poetry. No one asked me though, when they wrote NCLB and now ESSA. FWIW, when English teachers ask me about writing MC questions for her English class in order to "prepare students" for state tests (a not-infrequent question), I offer alternatives.
DeleteRebecca - The albatross is federally-mandated large scale testing. It's been around for decades. Before NCLB, states had some form of state-wide testing. After NCLB, all states have them. ESSA kept the requirement. They're not going away. My pragmatic take is we figure out a way to live with them and work around them to make school a place children want to be 180 days of the year.
Jenn
DeleteBefore NCLB, state-wide testing was not a problem because they weren't Big Stakes. Nobody didn't graduate because of them. Nobody didn't get promoted to the next grade just because of them. Schools didn't get closed because of them. Other classes weren't cut to make room for test prep. Learning how to use the stupid testing format and how to game it didn't take up real instruction time. Schools didn't have to spend millions of dollars when they're underfunded to begin with on expensive computer systems to take tests, systems that often ended up not working on test day. School personnel didn't feel they had to cheat because the tests and how they're used to close schools were unfair. Teachers weren't fired because of the use of invalid VAM scores.
The real albatross isn't the federally-mandated tests, it's the Big Stakes that are still attached to them. There's no way to work around them and the damage they're still doing.
I don't disagree with the content of your post, but I do disagree with your disagreement with the phrase "relatively unique". Taken independently, you can be relatively unique. "There's nothing new under the sun." There are very few truly original thoughts and ideas to be had, very few patterns of behavior that haven't been exhibited before. *grabs random books* As a writer, if you compare Inkheart (Cornelia Funke) to Fangirl (Rainbow Rowell), you'll discover that they are extremely unique. But if you compare Inkheart to The Neverending Story, you find "young child enters fantasy world, cross-pollination between reality and fantasy world, young child saves fantasy world using RL knowledge". And if you compare Fangirl to Harry Potter, you discover that Fangirl is basically a description of what happens when us HP addicts grow up and have to go to college. Or compare it to An Abundance of Katherines (John Green) and get the same theme of quirky romance overcoming obsessions. Now, I'm not one to bag on authors and I believe each of these books has great individual value and is unique because each author handles the subject matter differently, but still. I do think that "relative uniqueness" is a thing that can really exist, depending on whether you're comparing similar things or unlike things.
ReplyDeleteNone of this has anything to do with test scores or your actual point. I simply object to the initial metaphor.
"Unique" comes from the word "one" and is supposed to mean the only one of its kind, that there is nothing else in the world exactly like it. (Regarding education, I would say that every child is unique, even though there are often some similarities in attributes among different children.) However, nowadays the way people use it, it ends up meaning simply "unusual".
DeleteBut if everyone is unique, then it would never make sense to say that anyone is unique. It would not distinguish anyone from anyone else. It would not increase our information about anyone. We already know that they are unique. We are all unique. So, what is meant by calling someone "unique"? This is why some dictionaries have unusual as one of the meanings of "unique." I think that there are two senses or meaning to the word, and I don't see anything wrong with that. Many words start out meaning one thing and are transformed or expanded to mean other things.
DeleteI think it's important to state that each child is unique (not exactly like anyone else), because I think too many people don't realize this about children (or people in general), so they label them and put them in categories and try to standardize instruction.
DeleteIt's true that words change meaning in any spoken language (Latin doesn't change any more because it's a "dead" language because nobody speaks it.) There are always people who say the word isn't being used "correctly", but it's considered "correct" when a majority of educated speakers use it the new way. Tiffany is using "relatively unique" to mean "having some original qualities", and I think another way "unique" is used now is to mean that something or someone stands out from the norm.