At EdWeek, Larry Ferlazzo ran a guest post by Rene Grimes that focused on the "science of math" and the programs that are supposed to be based on it. Between the EdWeek part and the "math" part, some folks may have missed the really good set of questions that Grimes proposes at the end of the piece:
*What question did the original research actually investigate?
*Which students, settings, and outcomes were studied?
*Does the evidence directly support this recommendation, or is the recommendation partly an interpretation?
*What limitations or unanswered questions should shape its use?
*What evidence will show whether this approach is working for our students?
What a great batch of questions.
Imagine, for instance, if anyone backing Common Core had tried to answer these questions. Blanks all around.
What question did the original research actually investigate?
So much education "research" asks only one question-- if we do this thing, will students get a higher score on the Big Standardized Test this year? Lots of folks really want to use that as a proxy for asking questions like "Will students have a higher level of reading comprehension" or "Will students be able to have an increased ability to solve complex problems." But those aren't the same question at all. Sloppy and inexact questions lead to sloppy conclusions.
Which students, settings, and outcomes were studied?
I remember the days when it seemed like every time you peeled back a piece of education research that said "this is how students learn" you found a "study" performed on a dozen college sophomores at a private university. This has always been a problem for education because educators are correctly reluctant to use students as guinea pigs.
Some private and charter schools make claims about their effectiveness, but these are once again bits of research done on a very select group of students. That research isn't nothing, but it doesn't apply to a general student population.
Does the evidence support this sales pitch recommendation, or is the recommendation partly an interpretation?
Boy is this one an issue. Every single piece of evidence based on Big Standardized Test scores involves interpreting those test scores to mean all sorts of things that they don't necessarily mean.
What limitations or unanswered questions should shape its use?
It is always a red flag when neither limitations nor unanswered questions are even mentioned. One of the Common Core red flags was that there was no mechanism at all in place for talking about limitations or questions arising from classroom deployment.
There is no bit of legitimate science or evidence-gathering that declares "This question is now definitively solved and we don't need to look at it or question it ever again." And yet how rare is it that a new program is launched in a school and the Powers That Be say, "Now, here are some issues we want you to keep an eye on, and some questions that we are still looking to answer." In education, it's far more likely that every great new program is meant to be the end of the conversation, and uppity teachers who want to ask questions get a big round of administrative stinkeye.
What evidence will show that this is working for our students?
Again, the all-too-common default is that evidence will not be collected because it might not provide the evidence the Powers That Be want to find. No administrator wants to hear, "We've been trying your new program for a year and we have no evidence that it is working for our students." Should any teacher make that comment to admin, or any educator make that comment to a policy maker who mandated the Great New Thing, and they will get that old familiar refrain-- "You're implementing it wrong."
I love these five questions, and I hope more educators will use them more often-- including for every damned AI doodad that comes down the pike.
No comments:
Post a Comment