The Novella Principle

Steven NovellaThe skeptical movement has a lot of high-profile people who specialise in skepticism as it relates to medicine, CAM (complimentary-and-alternative-medicine), medical scams, medical research, health claims and so on. In my opinion, the skeptic that stands out in this sub-field (which has many excellent people in it) is Steven Novella.

First, a bit about him. Novella is a clinical neurologist and neurology researcher at the Yale University School of Medicine, the medical advisor of Quackwatch, a James Randi Educational Foundation fellow and is the host and/or cofounder of the Skeptics’ Guide to the Universe podcast (a weekly podcast on science and skepticism), Science Based Medicine (a group blog that Novella co-founded, promoting science-based-medicine, as the name implies) and the Neurologica blog (with Novella as sole author, focuses on his medical specialty of neuroscience).

In the hundreds of hours of the SGU podcast I’ve listened to over the years, there’s one principle that Novella has explained that’s been particularly helpful to my understanding of alternative medicine. So much so that I’m dubbing it the Novella Principle — and I think it’s one of those important things everyone should be aware of.

Academic research is very messy, especially in a field like medicine which has thousands of confounding factors and studies one of the messiest animals on the planet. This combined with the sheer number of studies being conducted (plus publication bias means that you can find a positive study on pretty much anything. That is, even if the therapy is (say) foot massage to cure tuberculosis, if there is enough interest there will be enough trials to give at least a few with positive results. Some of those may even be statistically significant. All of them will tend to be cherry-picked by proponents of said foot massage.

The Novella Principle is a way of considering the entire landscape of evidence for a given therapy. It describes the situation that should obtain for a treatment that does nothing. Here it is paraphrased by me:

The trials that are most positive and show the greatest effect are usually small, preliminary, poorly controlled and with inadequate blinding. When any of these problems are improved on, the better the study the smaller the effect it finds. Finally, in the best (ie. most comprehensive) studies, the effect disappears entirely to a level that’s indistinguishable from placebo.

It might seem obvious in retrospect but then again everything is. Plus it’s a staple of CAM proponents to promote treatments that follow this pattern. Meaning that it’s worth having a good grasp of this principle and being able to apply it. The principle’s wording is specific to medical claims but with a few tweaks (eg. replacing “placebo” with “no effect”) it can be fully generalised.

The interesting thing was, once I realised this was an important effect, I’ve seen many other fields where it could be applied. For instance, Cordelia Fine has appeared on several skeptic podcasts to talk about her book Delusions Of Gender: How Our Minds, Society, and Neurosexism Create Difference. The book’s thesis is that the studies used to promote sex differences (especially the kinds of studies reported in the media, often based on brain scans) aren’t good science. Now I don’t have the expertise to do a full evaluation of something like this, and I haven’t read the book so I’m not necessarily endorsing the viewpoint in its details. However in her appearance on Skeptically Speaking, Fine spelled out a landscape that was identical to the Novella Principle.

Novella’s main example of this is homeopathy and ESP research, however I believe he also argued for the effect in acupuncture research. On ESP, you can really get into the nitty gritty in this episode of SGU where they interviewed paranormal proponent Alex Tsakiris, where they get very deep into statistical analysis, Richard Wiseman and Sheldrake’s experiments into dogs that allegedly know when their owners are coming home.

Are there any other fields that fit the pattern based on your own reading? Of course someone claiming a pattern is not enough but it would be good to start an initial list.

avatar

About this blog

The thinly-veiled identity of lives and rants in Sydney. Views not his own, provided by hivemind. All my original work on this blog is licensed under a CC BY-NC License.

Subscribe to Fail Blue Dot


OR