Is there something wrong with the scientific method?


by Jonah LehrerDECEMBER 13, 2010

Many results that are rigorously proved and accepted start shrinking in later studies.

Many results that are rigorously proved and accepted start shrinking in later studies.

On September 18, 2007, a few dozen neuroscientists, psychiatrists, and drug-company executives gathered in a hotel conference room in Brussels to hear some startling news. It had to do with a class of drugs known as atypical or second-generation antipsychotics, which came on the market in the early nineties. The drugs, sold under brand names such as Abilify, Seroquel, and Zyprexa, had been tested on schizophrenics in several large clinical trials, all of which had demonstrated a dramatic decrease in the subjects’ psychiatric symptoms. As a result, second-generation antipsychotics had become one of the fastest-growing and most profitable pharmaceutical classes. By 2001, Eli Lilly’s Zyprexa was generating more revenue than Prozac. It remains the company’s top-selling drug.

But the data presented at the Brussels meeting made it clear that something strange was happening: the therapeutic power of the drugs appeared to be steadily waning. A recent study showed an effect that was less than half of that documented in the first trials, in the early nineteen-nineties. Many researchers began to argue that the expensive pharmaceuticals weren’t any better than first-generation antipsychotics, which have been in use since the fifties. “In fact, sometimes they now look even worse,” John Davis, a professor of psychiatry at the University of Illinois at Chicago, told me.

Before the effectiveness of a drug can be confirmed, it must be tested and tested again. Different scientists in different labs need to repeat the protocols and publish their results. The test of replicability, as it’s known, is the foundation of modern research. Replicability is how the community enforces itself. It’s a safeguard for the creep of subjectivity. Most of the time, scientists know what results they want, and that can influence the results they get. The premise of replicability is that the scientific community can correct for these flaws.



About carlos

I'm a curious person, of reasonable intellect, "on the beach" (retired) and enjoying my interest in anthropology, language, civil rights, and a few other areas. I've been a hippie/student/aerospace tech writer in the '60s, a witness to the Portuguese revolution in the ‘70s, a defense test engineer and witness to the Guatemalan genocide in the '80s, and a network engineer for an ISP in the '90s. Now I’m a student and commentator until my time is up. I've spent time under the spell of the Mesoamerican pyramids and the sweet sound of the Portuguese language. I've lived in Europe, traveled in Brazil, Central America, Iceland, New Zealand, and other places. My preferred mode of travel is with a backpack and I eat (almost) anything local. Somehow, many of the countries I have been to have had civil unrest (for which I was not responsible). I'm open to correspond with anyone who might share my liberal, humanist interests. I live in San Buenaventura, California.
This entry was posted in Anthropology and tagged , , , . Bookmark the permalink.