Nobody squawked much when it was announced that Pearson had won the bid to develop the framework for the 2018 PISA test. The PISA, you will recall, is administered by the Organization for Economic Co-operative and Development every three years, leading directly to a festival of handwringing and pearl-clutching as various politicians and bureaucrats scramble to squeeze statistical blood from the big fat turnip of test results.
And yes, Pearson just won the right to design the 2018 edition. Given that back in 2011 Pearson won the contract to develop the 2015 PISA, the new contract is not a shocker. Given that Pearson is marching toward becoming the Corporation In Control Of Universal Testing, this barely qualifies as a blip. They have the GED. They have the PARCC. They have dreams of managing via computer every test, testlet, and testicle that exists.
There are many problems with that, but one of the fundamental issues is the one raised by this post's title.
When one person with one ruler does all the measuring, how are we to know if he's correct?
If we want to confirm the accuracy of our Pearson measuring tool so we check it against our Pearson standards device and make sure those results line up with the Pearson Master Assessment-- well, at the end of all that, what do we really know?
If Pearson tells us that our six-inch long baby pig weighs 500 pounds, how are we to discover that it's a lie? If Pearson weighs our bag of gold and tells us it's worth $1.98, and they own all the scales, how do we know if we're being cheated?
It doesn't matter whether the people who make the rulers are devious or incompetent-- if there is no one left to check their work, how do we know the true dimensions of anything? If Pearson makes all the tests and keeps assuring us, "Yessiree, this test lines up with our other test and fits in with the main test, so we can assure you that this absolutely measures true learning or complete education or intelligence or character or what matters in a human brain or the strength of a nation's education program," how do we check to prove whether that is true or not?
Who watches the watchmen? Who measures the rulers? To whom does Pearson answer, other than stockholders? I'm hoping we don't wake up some morning to discover the answer is "nobody."