Along with Anthony Colella (Seton Hall), Christian Angelillo (Boonton Township SD), Meredith Fox (Nanuet Union SD), Kevin McCahill (George W. Miller Elementary) and Adam Wolfe (Peoria Unified SD), Tienken has once again answered the question-- what do the Big Standardized Tests actually measure?
Put another way, Tienken et. al. have demonstrated that we do not need to actually give the Big Standardized Test in order to generate the "student achievement" data, because we can generate the same data by looking at demographic information all by itself.
Tienken and his team used just three pieces of demographic data--
1) percentage of families in the community with income over $200K
2) percentage of people in the community in poverty
3) percentage of people in community with bachelor's degrees
Using that data alone, Tienken was able to predict school district test results accurately in most cases. In New Jersey 300 or so middle schools, the team could predict middle school math and language arts test scores for well over two thirds of the schools.
I suppose some folks could see this as good news ("Cancel the PARCC test and don't pay them a cent! We can just fudge our test results by plugging in demographic data!") but I'd characterize it more as frightening, given that ESSA continues to demand that teachers and administrators and schools be judged based on test scores (generally under the euphemism "student achievement") and if those test scores can be fudged based on data having nothing to do with what actually goes on inside the school, then a whole bunch of careers and funding are riding on things that have nothing to do with schools.
This is also one more reason that any future teacher (there are, I hear, still one or two out there) who is paying attention should know better than to take a job in a poor neighborhood, where anything from her professional standing to her future career is liable to be trashed by the demographics of her neighborhood.
There are other conclusions to be drawn here, not the least of which is that you are in one of those A-F school rating states, the best way to change your school's grade is to change your demographics (aka turn into a charter and recruit students from outside your old neighborhood).
Make sure to read this report and pass it on. It has been peer reviewed, it is legitimate research, and it does raise huge red-flaggy questions about the validity or usefulness of the BS Tests. At the very least you can be asking your state and national policy leaders, "If we can generate the same data by just analyzing demographics, why are we wasting time and money on these tests?"
In the meantime, here's an oldie but a goodie from Tienken, in case you like your explanations more video style.
If Tienken had added one more criteria to his study, I'll bet he could have predicted test scores at 80+% accuracy:
ReplyDeleted) percentage of students living in a two (biological) parent household.
What does biological have to do with anything? Are adoptive parents inferior? Gay/lesbian parents? There is absolutely no evidence to support that.
DeleteOf course economists are the ones that first argued this was true and that is why we try very very hard to control for student demographics in our analysis of test scores. For evidence of this, look at any paper published published by an economist analyzing test scores.
ReplyDeleteChetty Freedman and Rockoff, for example, work very hard to control for individual student characteristics to show that good teaching has a lifelong impact on students independent of student background. I am baffled that teachers often reject this finding.
Well, yes. We reject it because, while they may have tried very hard, there is zero evidence that they succeeded,
DeleteNor can all the adjustment in the world matter when you start with junk data
DeleteInteresting that you allow people to read this post, but not my earlier post about delaying enrollment in kindergarten. One of the mysteries about fake news is constructed I guess.
ReplyDeleteHave you actually read the papers in question? I have.
Lets suppose your right. Is there any evidence at all that good teaching matters to students? Your paper suggests not, it is only community characteristics that matter. No reason to waste any resources training teachers if they have just as much impact on student learning as sleeping with the textbook under the pillow.
As always, you are so intent on showing your erudition that you either miss the point or choose not to see it. The paper suggests no such thing-- what the paper suggests is that the standardized tests do not measure student learning at all. And if you are concerned that people are being deprived of your insights, I can offer you a tutorial on how to set up a blog of your own.
Delete"I can offer you a tutorial on how to set up a blog of your own." Please do.
DeleteBig Standardized ELA Test scores ultimately reflect a lifetime of language acquisition and use. This is why there is such a strong correlation with parental income, education level, and family functionality. Of all the disciplines, ELA is probably the most difficult for a teacher to influence. The Common Core/RTTT emphasis on teaching empty reading and writing skill sets of informational texts was doomed from the get-go.
ReplyDeleteDue to the PC nature of our culture, the mention of innate intelligence (genetics) seems to be a taboo topic. The notion that all children can learn challenging material simply isn't true.
I think the idea of innate intelligence is relative. My area is foreign language. Some people would say that it's ridiculous for special education students to take foreign language. I have taught foreign language to students whose IQ was around 80. They mostly have trouble with reading and writing, but the way I teach they do fine with speaking and listening. Memory is also a problem, but again, the way I teach they do okay, and they even get by with reading and writing if they have good study skills. There are people who have trouble with terminology, but can understand any concept, no matter how difficult. I had a student who, according to standardized testing, had a third grade reading comprehension level, but his vocabulary knowledge and math reasoning were very good, and he did fine. He was one of my best students because he always asked questions until he understood. He had more trouble with third year because there's more difficult reading involved, but he got through. Motivation is very key; students who are interested in a subject can do much better than you might expect. I've also had students who memorized easily and did well on tests, but remembered nothing afterwards and never understood the concepts. There are people who are brilliant in one area and have no understanding of anything else (Ben Carson, for example). I get what you're saying as far as standardized testing does not show everything a student can do. There are students who aren't academically inclined, yet have natural ability or skill in more hands-on fields, or who have innate people skills. We can't expect everybody to have the same ease or motivation in learning the same things at the same speed. I had Honors math through pre-calc, but I never felt I understood it in the least. I'm convinced, however, that with the right teacher and enough time - lots and lots of time - I could understand it. But I don't like blanket statements saying some kids are just smarter than others. It's much more complex than that.
ReplyDeleteAre you OK with idea that some children are more gifted athletes or musicians, regardless of practice time? Take it from a musical idiot who wasted endless hours trying to learn how to play the guitar.
DeleteIn some cases, severe cognitive disabilities can be very limiting. Denying this would be foolish, yet I have been held "accountable" for students who could not count from one to 20 due to their unfortunate disabilities.