ACT Report Finds ACT Really Important

Okay kids. Here's today's lesson in critical thinking (which, as you may have heard, is built right into the heart and sinew of the Common Core).

When a business releases a study showing the importance and effectiveness of that business's product, is there a possibility that the study might be aimed at something other than Telling the Whole Truth?

The folks at ACT must be really sweating these days. Their competitors at the College Board have scored a couple of coups, including A) hiring the well-connected author of the CCSS ELA standards, David Coleman and B) getting state and federal governments to adopt their line of AP products as the official education product of the US government. So the ACT people are in need of some realmPR work.

So they are here to report on the national level of college readiness.

This report provides the ACT folks with a great chance to let us know what they've been up to as they desperately try to grab some market share work to make US education sweller.

They have, for instance, released ACT Aspire, a test package for assessing student college readiness for grades 3 through 10. It aligns with the ACT College Readiness Standards, so take that, new multi-grade PSAT.

The ACT itself has been "enhanced" by the addition of stats and probability math questions, new questions that require students to "integrate knowledge and ideas across multiple texts, a STEM score, a new writing section enhanced with God-knows-what, and, of course, a computerfied version to take on line, because, computers. They will start throwing in ACT Profile, which appears to be career-suggestion software, so I guess the ACT is now an aptitude test. They also want you to know that they are committed to making sure that the test is continuously monitored for validity and evidence, which may seem obvious, but since David "An Education Is What I Say It Is" Coleman seems inclined to move SATs away from validity and evidence, it's worth the ACT's time to toot their own trumpet of obviousness.

Now, if you think any of that qualifies as a "report" and not "advertising copy," you are not exceeding expectations in your applying of critical thinking. But that's okay, because even Caralee Adams writing about the report at EdWeek, fails critical thinking regarding the meat of this report.

She takes away exactly what she was supposed to-- students who take lots of ACT courses do better in college, therefor, the ACT courses must have prepared them super-well for college.

Why is it that education "research" is so riddled with an inability or unwillingness to distinguish between correlation and causation?

If we were going to design an experiment to determine the effectiveness of a product (say, New Shiny ACT Creme) we would need to first start with subjects who were close to identical. So, let's start with 200 fifteen-year-olds who have similar amounts of acne, give 80 of them New Shiny ACT Creme, 80 of them nothing, and 40 of them mayonnaise. After several months, we'll check to see how the acne looks.

That is not what this "study" has done.

Instead, this study says, "We want the teens who think they have the best complexion to use ACT Creme. We'll let them try it, and in a few months, we'll check to see how they look."

What the ACT report seems to show is not terribly remarkable. Apparently the same high-achieving, hard-working, top-ability, ambitious students who do well in college also seek out the challenging courses in high school. Put another way, the students who value college success and take steps to achieve it in college, also take steps to achieve it in high school. In other news, the sun is expected to rise in the East tomorrow.

I'm not suggesting that the ACT is a scam and terrible waste of time. Like AP courses and the SAT, the ACT clearly has some benefits for some students, and that's a Good Thing.

But we really need to stop looking at these groups as if they are philanthropic organizations trying to make the education world a better place out of the goodness of their highly objective hearts. The ACT is a business, and any "report" they offer about their product is most reasonably viewed as advertising copy, not scientific research.

News organizations do not run stories with headlines like "Researchers Discover Frosted Flakes Are Grrrreat" or "Study Shows Men Who Wear AXE Get Laid More Often." At least not yet. In the meantime, let's keep those critical thinking caps firmly attached.