People Aren't Stupid; Science Is Just Hard -- Why We're Wrong to Quickly Dismiss Ed Research
RCEd Commentary
As someone who spends most of their time thinking about the application of scientific findings to education, I encounter plenty of opinions about the scientific status of such efforts. Almost always, a comparison is made between the rigor of “hard” sciences and education. What varies is the accompanying judgment: derision for education researchers or despondency about the difficulty of the enterprise. In a recent article, physicist Carl Wieman offers a different perspective on the issue, suggesting that the difference between research in education and in physics is smaller than you might guess.
In education, Wieman is best known for refining and popularizing techniques to have college students better engage in large lecture courses. He started his career as a physicist, producing work that culminated in a Nobel prize. So he has some credentials in talking about the work of “hard” scientists.
Wieman begins by making clear what he takes to be the outcome of good science: predictive power. Can you use the results of your research to predict with some accuracy what will happen in a new situation? A common mistake is to believe that in education, one ought to be able to predict outcomes for individual students; not necessarily so, any more than a physicist must be able to predict the behavior of each atom. Prediction in aggregate — a liter of gas or a school of children — is still an advance.
Wieman’s other points follow from his strong emphasis on prediction.
First, it follows that “rigorous” methods are any that contribute to better prediction. You don’t state in the absolute that randomized controlled trials are better than qualitative research. They provide different types of information in a larger effort to allow prediction.
Second, the emphasis on prediction frames the way one thinks about the messiness of research. Education research is often portrayed as inherently messy because there are so many variables at play. Physics research, in contrast, is portrayed as better controlled and more precise because there are many fewer variables that matter. Wieman argues this view is a misconception.
Physics seems tidy because you’ve probably only studied physics in textbooks, where everything is worked out: in other words, where no extraneous variables are discussed as possibly mattering. When the work that you study was first being conducted, it was plenty messy: false leads were pursued, ideas that, in retrospect are self-contradictory, were taken seriously, and so on. The same is true today: the frontiers of physics research is messy. “Messy” means that you don’t have a very good idea of which variables are important in gaining the predictive power that characterizes good science.
Third, Wieman suggests that bad research is the same in physics and education. Research is bad when the researcher has failed to account for factors that, based on prior research, he or she should have known to include. There is plenty of bad research in the hard sciences. People aren’t stupid; it’s just that science is hard.
I agree with Wieman that differences in “hardness” are mostly illusory. (That’s why I’ve been putting the term in quotation marks.) The fundamentals of the scientific method don’t differ much wherever they are applied. I also agree that people -- usually people uninvolved in research -- are too quick to conclude that, compared to other fields, a higher proportion of education research is low-quality. Come to a meeting of the Society for Neuroscience and I’ll show you plenty of studies that were poorly conceived, poorly controlled, or were simply wheel-spinning and will be ignored.
Wieman does ignore a difference between physics and education that I take to have important consequences: physics (and other basic sciences) strive to describe the world as it is, and so strive to be value-neutral. Education is an applied science; it is in the business of changing the world, making it more like it ought to be. As such, education is inevitably saturated with values.
Education policy would, I think, benefit, with a greater focus on the true differences between education and other varieties or research, and a reduced focus on the phantom differences in rigor.
Reference:
Wieman, C. E. (2014). The similarities between research in education and research in the hard sciences. Educational Researcher, 43, 12-14.
