The National Center for Analysis of Longitudinal Data in Education Research (CALDER) is a program of the American Institute for Research (AIR) along with an assortment of universities. They've just released a working paper version of some research that some people think is pretty significant. I've read it so you don't have to. Let me explain what they found, and why nobody needs to get excited about it.
AIR is better known for their work in test manufacturing and marketing, which gives them a dog in this particular hunt, but I'm not sure that really need concern us here.
"Assessing the Accuracy of Elementary School Test Scores as Predictors of Students' High School Outcomes" was written by Dan Goldhaber (AIR/CALDER/University of Washington), Malcolm Wolff (University of Washington), and Timothy Daily (EdNavigator). EdNavigator is a business positioned to help parents navigate a complicated education-flavored-products market--again, I'm not sure we need to be concerned here, other than to note that many of the involved parties are not exactly disinterested observers of the Big Standardized Test biz. Oh yeah--the Carnegie Foundation did some of the funding for this.
There's a lot of fancy math in this paper, but the outcome is pretty straightforward--studying student test results in North Carolina, Massachusetts, and Washington, the researchers found that third grade test scores correlate with high school outcomes, namely, high school test scores, advanced course-taking in high school, and high school graduation.
The result is that third grade tests correlate with those factors about as well as eighth grade tests do. They also found that poverty and race/ethnicity correlate with a drop of those high school outcomes.
There is nothing surprising here. They have been painstaking in their figuring and crunching and mathing the crap out of their data. But ultimately, they have asked the wrong question.
Because--say it with me--correlation is not causation.
We know, via many many many pieces of research, that Big Standardized Test scores are directly tied to socioeconomic status. Both third grade and high school tests.
We know that socioeconomic status correlates with other life outcomes, like graduation and jobs etc. We can even talk about how it correlates with baloney like the Success Sequence (which just puts things ass-backwards, declaring that if you wear large clothes, that's what caused you to get bigger).
We know all about these correlations, and they point pretty clearly to SES as a cause. So research like this, while not a total waste of time, because I suppose if third grade scores were a terrible predictor of high school stuff, we'd know something was definitely cattywumpus somewhere in the system-- research like this isn't helpful because it's asking the wrong question.
What still remains unproven is this-- if you take a student who would have scored 60 on the third grade test and somehow get them to score 80 or 90, would that improve the student's later outcomes?
We have (and have had for years) evidence that the answer is no, that raising student test scores does not improve student outcomes.
We have been subjected to a multi-decade parade of reformsters asserting and assuming that raising student test scores would unlock a host of benefits for our students, our economy, our entire nation. But instead of proof, we've just had tautological research proving that students who do well in school do well in school., or evidence that your socioeconomic background is well-measured by the Big Standardized Test no matter what grade you're in.
What we really need is research that asks the right question. This paper is not that.
I would like to see the data that shows the correlation between family structure and school success. The elephant in the poverty v. affluence discussion is the proven failure of broken family units. With nearly 80% of inner city children living in single mother homes, it is irresponsible to ignore this nearly insurmountable obstacle.
ReplyDeleteI would also argue that affluence is not the direct, causal factor in school success. It is highly correlated because of the variables that contribute to higher incomes: intact family units and academically successful parents. This produces a catch-22 situation for the millions enmeshed in generational poverty and dependence and probably explains why those test scores just don't budge.
You'd have to convince me that intact families are a cause of affluence and not an effect, or that they're not both the effect of a separate cause. There are correlations all over this landscape, but precious little research to show what cause needs to be addressed in order to lift up everything else.
DeleteAre you saying that as a matter of science, now and for all times, in all places, that in general, people who are richer do better on standardized tests than poorer ones?
ReplyDeleteOr are you saying this is true for American students right now in our current era? Or since the beginning of public education in the US, or for the last 30 years, or some other qualifier? Or since a certain crowd of people started developing the BS tests?
Is this a universal statement or only an American statement? Personally, I am not a believer in standardized tests, and never have been. This post would be much more effective if you would clarify your comment on the role of socioeconomic status and student standardized test performance.
Standardized tests are not helpful to students themselves. The question is, who benefits from them? Only the companies who profited from selling them to a school district. You believe the data results are predictable making the test unnecessary. But your argument begs further clarification.