Forum

The Current Environment in Education Research

I am curious: The author (Steven Nelson, “The Promise of Scientific Evidence in Education,” summer 2004) frequently tells us that the only research that is “good” is that which is empirical and objective, and follows proof-of-reason based on the theory and evidence. This effectively excludes ethnography, qualitative studies, and case studies. He writes, “So we generally recognize that true experiments are less equivocal than quasi-experiments, and those are better than correlational and case studies, which in turn supersede expert opinion, which trumps anecdotal evidence.”

My question is: With what scientifically based research does the author support this claim? It seems to me that this statement is a matter of “expert opinion,” which is clearly not considered to be of high enough quality to demonstrate the usefulness of this statement, much less the article as a whole. How is this article, which seems to be largely a matter of opinion rather than rigorous, scientifically based research, supposed to help me as a teacher?

A.M. Will
Instructor
Alpena Community College
Alpena, Michigan

The author responds:
Thank you for your comments and concerns regarding the article in Northwest Education magazine on scientific research. You might be surprised to learn that I agree with your observations about the merits of various research methods. The claims that I use to support the assertions in the article are based upon federal policy: the No Child Left Behind Act and the Education Sciences Reform Act. Indeed, goal four of the U.S. Department of Education’s strategic plan is to “transform education into an evidence-based field.” Their performance indicators include: “Of the new research and evaluation projects funded by the IES (Institute for Education Sciences) that address causal questions, the percentage of projects that employ randomized experimental designs [will increase].” The hierarchy I cite is theirs, not mine. But it is the same hierarchy of research designs used for testing the efficacy of pharmaceuticals and the environmental quality of land-use plans.

The poster child of IES is the What Works Clearinghouse, which employs criteria so rigorous in its search for high-quality research that, for example, they only found four acceptable studies out of 800 reviewed in the area of middle school mathematics. These are the facts of the public policy environment within which both you and I are operating, whether we agree with them or not. As a trained sociologist, I also struggle with the true experiment as the gold standard. You will note that I referenced the National Research Council’s Scientific Research in Education (The National Academies Press, 2002), which I encourage you to read. It provides a broader perspective on the nature of evidence and scientific inquiry both in purpose and approach. So I will conclude with a congratulatory touché, that indeed expert opinion doesn’t constitute scientifically based research. Federal law, however, still stands. Thank you for your readership and comments.

Steven R. Nelson
Director
Office of Planning and Service Coordination
Northwest Regional Educational Laboratory

We want to hear from you! Send your letters to the editor, article ideas, and tips on places where good things are happening to nwedufeedback@nwrel.org. Letters may be edited for length or clarity.