Voices in Education

Randomized Controlled Trials and the Blob
William Bennett, US education secretary in the 1980s, came up with the term “the Blob” to describe education’s bureaucracy. Recently, in the United Kingdom, Michael Gove, the secretary of state for education until 2014, resurrected the term, apparently to apply to all who work in the field—teachers, officials, academics—everyone. All of these people, presumably out of a hypnosis-induced allegiance to some kind of Trotskyist cult, would infuriatingly, reject “reform” initiatives out of hand.

The “reformers,” that is to say, those who believe that change can be imposed from above on teachers, are continually exasperated by the Blob’s resistance to their new schemes. So they devise ever-better means of trying to convince the Blob of the rightness of reforming ways. The latest is science. Science is good; science says do it this way; ergo, this way is the right way.

I’ve got nothing against science. Science is wonderful. But I object to science being requisitioned to bolster arguments for silly initiatives in education. The latest import from science is the randomized controlled trial (RCT). For those who haven’t come across it, the RCT is an experiment—a special kind of experiment where allocation of people to experimental and control groups is made randomly. This group gets this treatment; that group gets that treatment, while the one over there gets nothing at all. At the end of the experiment, we can tell if the treatments have had any effect.

That’s the theory, anyway. It’s excellent for testing pharmaceuticals, and I wouldn’t want any drugs prescribed to me that hadn’t been tested thoroughly using RCTs. The problem comes in migrating the logic of RCTs to a social environment. The point I have tried to make in my article in the Harvard Educational Review is that RCTs rarely provide useful information in the educational landscape. For a start, a crucial element of the RCT used in pharma—namely, double-blinding, where neither recipient nor researcher knows who is in which group—is difficult or impossible to manage effectively in the scenarios of education research.

Even more importantly, though, social situations are like games: people and situations change as the game develops. Strategies are adopted; strengths are deployed to best advantage. And this is why RCTs are never used to test game strategies—in soccer, chess, or any other game. They would be of no use since the adaptations, compensations, and guile people use in real situations conspire to undermine the best-laid plans. In short, interventions in the social world work nothing like pharmaceuticals.

We’ve had nearly two decades now since we were told that these scientific evaluation instruments borrowed from pharma and plant science would transform education by guiding us to the right way of working. At the beginning of the twenty-first century, No Child Left Behind instructed teachers to use scientifically proven practices in their classrooms. But there has been surprisingly little, if any, consequence from the policy, if we look at Programme for International Student Assessment (PISA) scores, for example. What’s going wrong? Isn’t it about time that there were some critical evaluations of the process and product of RCT methodology? Or am I being too Blobbish about all of this?

About the Author: Gary Thomas is professor of inclusion and diversity at the University of Birmingham in the United Kingdom.