Bad Science: Quacks, Hacks, and Big Pharma Flacks

by Ben Goldacre

At school you were taught about chemicals in test tubes, equations to describe motion, and maybe something on photosynthesis—about which more later—but in all likelihood you were taught nothing about death, risk, statistics, and the science of what will kill or cure you. The hole in our culture is gaping: evidence-based medicine, the ultimate applied science, contains some of the cleverest ideas from the past two centuries; it has saved millions of lives, but there has never once been a single exhibit on the subject in London’s Science Museum.

You cannot reason people out of positions they didn’t reason themselves into.

What do they mean by toxins? I’ve asked the manufacturers of many detox products this question time and again, but they demur. They wave their hands, they talk about stressful modern lifestyles, they talk about pollution, they talk about junk food, but they will not tell me the name of a single chemical that I can measure.

Because it has no scientific meaning, detox is much better understood as a cultural product. Like the best pseudoscientific inventions, it deliberately blends useful common sense with outlandish, medicalized fantasy.

On one thing we must be absolutely clear, because this is a recurring theme throughout the world of bad science: there is nothing wrong with the notion of eating healthily and abstaining from various risk factors for ill health like excessive alcohol use. But that is not what detox is about; these are quick-fix health drives, constructed from the outset as short term, while lifestyle risk factors for ill health have their impact over a lifetime. But I am even willing to agree that some people might try a five-day detox and remember (or even learn) what it’s like to eat vegetables, and that gets no criticism from me. What’s wrong is to pretend that these rituals are based in science or even that they are new. Almost every religion and culture have some form of purification or abstinence ritual, with fasting, a change in diet, bathing, or any number of other interventions, most of which are dressed up in mumbo jumbo.

Like so much of the nonsense in bad science, “detox” pseudoscience isn’t something done to us, by venal and exploitative outsiders; it is a cultural product, a recurring theme, and we do it to ourselves.

But Brain Gym perfectly illustrates two more recurring themes from the industry of pseudoscience. The first is this: you can use hocus pocus—or what Plato euphemistically called a noble myth—to make people do something fairly sensible like drink some water and have an exercise break.

The second theme is perhaps more interesting: the proprietorialization of common sense. You can take a perfectly sensible intervention, like a glass of water and an exercise break, but add nonsense, make it sound more technical, and make yourself sound clever. This will enhance the placebo effect, but you might also wonder whether the primary goal is something much more cynical and lucrative: to make common sense copyrightable, unique, patented, and owned.

Most people know what constitutes a healthy diet already. If you want to make money out of it, you have to make a space for yourself in the market, and to do this, you must overcomplicate it, attach your own dubious stamp.

In general, you don’t absorb things very well through your skin, because its purpose is to be relatively impermeable. When you sit in a bath of baked beans for charity, you do not get fat, nor do you start farting.

I’m not on a consumer crusade. Just like the lottery, the cosmetics industry is playing on people’s dreams, and people are free to waste their money. I can very happily view fancy cosmetics—and other forms of quackery—as a special, self-administered, voluntary tax on people who don’t understand science properly.

We know that two sugar pills are a more effective treatment than one sugar pill, for example, and we know that saltwater injections are a more effective treatment for pain than sugar pills, not because saltwater injections have any biological action on the body, but because an injection feels like a more dramatic intervention. We know that the color of pills, their packaging, how much you pay for them, and even the beliefs of the people handing the pills over are all important factors. We know that placebo operations can be effective for knee pain and even for chest pain. The placebo effect works on animals and children.

As Voltaire said, “The art of medicine consists in amusing the patient while nature cures the disease.”

We cannot simply decide such things on the basis of one individual’s experiences, for the reasons described above: they might be mistaking the placebo effect for a real effect or mistaking a chance finding for a real one. Even if we had one genuine, unambiguous, and astonishing case of a person’s getting better from terminal cancer, we’d still be careful about using that one person’s experience, because sometimes, entirely by chance, miracles really do happen. Sometimes, but not very often.

it would be fair to say that all research has some flaws, simply because every trial will involve a compromise between what would be ideal and what is practical or cheap.

When doctors and scientists say that a study was methodologically flawed and unreliable, it’s not because they’re being mean, or trying to maintain the “hegemony,” or to keep the backhanders coming from the pharmaceutical industry; it’s because the study was poorly performed—it costs nothing to blind properly—and simply wasn’t a fair test.

as a general rule it’s always worth worrying when people don’t give you sufficient details about their methods and results.

There is a moral and financial issue here too: randomizing your patients properly doesn’t cost money. Blinding your patients to whether they had the active treatment or the placebo doesn’t cost money. Overall, doing research robustly and fairly does not necessarily require more money; it simply requires that you think before you start.

A meta-analysis is a very simple thing to do, in some respects: you just collect all the results from all the trials on a given subject, bung them into one big spreadsheet, and do the math on that, instead of relying on your own gestalt intuition about all the results from each of your little trials. It’s particularly useful when there have been lots of trials, each too small to give a conclusive answer, but all looking at the same topic.

In most studies we don’t have a “no treatment” group to compare both the placebo and the drug with, and for a very good ethical reason: if your patients are ill, you shouldn’t be leaving them untreated simply because of your own mawkish interest in the placebo effect. In fact, in most cases today it is considered wrong even to use a placebo in a trial; whenever possible you should compare your new treatment with the best preexisting, current treatment.

The answer is that the placebo effect is about far more than just the pill; it is about the cultural meaning of the treatment. Pills don’t simply manifest themselves in your stomach; they are given in particular ways, they take varying forms, and they are swallowed with expectations, all of which have an impact on a person’s beliefs about his own health and, in turn, on outcome. Homeopathy is, for example, a perfect example of the value in ceremony.

a survey of the color of pills currently on the market found that stimulant medication tends to come in red, orange, or yellow tablets, while antidepressants and tranquilizers are generally blue, green, or purple.

This raises the specter of something way beyond the placebo effect, and cuts even further into the work of alternative therapists, because we should remember that alternative therapists don’t just give placebo treatments; they also give what we might call placebo explanations or placebo diagnoses: ungrounded, unevidenced, often fantastical assertions about the nature of the patient’s disease, involving magical properties, or energy, or supposed vitamin deficiencies, or “imbalances,” which the therapist claims uniquely to understand.

And here it seems that this placebo explanation—even if grounded in sheer fantasy—can be beneficial to a patient, although interestingly, perhaps not without collateral damage, and it must be done delicately; assertively and authoritatively giving someone access to the sick role can also reinforce destructive illness beliefs and behaviors, unnecessarily medicalize symptoms like aching muscles (which for many people are everyday occurrences), and militate against people’s getting on with life and getting better. It’s a very tricky area.

A modern medic would struggle to find a form of words that would permit her to hand out a placebo, for example, and this is because of the difficulty in resolving two very different ethical principles: one is our obligation to heal our patients as effectively as we can; the other is our obligation not to tell them lies.

placebo responders have been found to be more extroverted but more neurotic, more well adjusted but more antagonistic, more socially skilled, more belligerent but more acquiescent, and so on. The placebo responder is everyman. You are a placebo responder. Your body plays tricks on your mind. You cannot be trusted.

To me, this is yet another example of a fascinating paradox in the philosophy of alternative therapists: when they claim that their treatments are having a specific and measurable effect on the body, through specific technical mechanisms rather than ritual, they are championing a very old-fashioned and naive form of biological reductionism, in which the mechanics of their interventions, rather than the relationship and the ceremony, have the positive effect on healing. Once again, it’s not just that they have no evidence for their claims about how their treatments work: it’s that their claims are mechanistic, intellectually disappointing, and simply less interesting than the reality.

Believing in things that have no evidence carries its own corrosive intellectual side effects, just as prescribing a pill in itself carries risks: it medicalizes problems, as we shall see, it can reinforce destructive beliefs about illness, and it can promote the idea that a pill is an appropriate response to a social problem, or a modest viral illness.

The newspapers sometimes seem to be engaged in a bizarre ongoing ontological project, diligently sifting through all the inanimate objects of the universe in order to categorize them as a cause of—or cure for—cancer. At the core of this whole project are a small number of repeated canards, basic misunderstandings of evidence that recur with phenomenal frequency. These intellectual crimes are ferried to you by journalists, celebrities, and, of course, “nutritionists,” members of a newly invented profession who must create a commercial space to justify their own existence. In order to do this, they must mystify and overcomplicate diet and foster your dependence upon them.

Fairy liquid will kill cells in a test tube, but you don’t take it to cure cancer. This is just another example of how nutritionism, despite the alternative medicine rhetoric and phrases like “holistic,” is actually a crude, unsophisticated, old-fashioned, and, above all, reductionist tradition.

The idea is to try and give all the information to help others to judge the value of your contribution; not just the information that leads to judgment in one particular direction or another. —Richard P. Feynman

There are few opinions so absurd that you couldn’t find at least one person with a Ph.D. somewhere in the world to endorse them for you; and similarly, there are few propositions in medicine so ridiculous that you couldn’t conjure up some kind of published experimental evidence somewhere to support them, if you didn’t mind its being a tenuous relationship and cherry-picked the literature, quoting only the studies that were in your favor.

today cherry-picking is one of the most common dubious practices in alternative therapies, particularly in nutritionism, where it seems to be accepted essentially as normal practice (it is this cherry-picking, in reality, that helps characterize what alternative therapists conceive of, rather grandly, as their alternative paradigm). It happens in mainstream medicine also, but with one crucial difference: there it is recognized as a major problem, and hard work has been done to derive a solution. That solution is a process called systematic review. Instead of just mooching around online and picking out your favorite papers to back up your prejudices and help you sell a product, in a systematic review you have an explicit search strategy for seeking out data (openly described in your paper, even including the search terms you used on databases of research papers), you tabulate the characteristics of each study you find, you measure—ideally blind to the results—the methodological quality of each one (to see how much of a “fair test” it is), you compare alternatives, and then finally you give a critical, weighted summary.

It’s interesting to note, while we’re here, that carrots were the source of one of the great disinformation coups of World War II, when the Germans couldn’t understand how our pilots could see their planes coming from huge distances, even in the dark. To stop them from trying to work out if we’d invented anything clever like radar (as we had), the British instead started an elaborate and entirely made-up nutritionist rumor. Carotenes in carrots, they explained, are transported to the eye and converted to retinal, which is the molecule that detects light in the eye (this is basically true and is a plausible mechanism, like those we’ve already dealt with), so, went the story, doubtless with much chortling behind their excellent RAF mustaches, we have been feeding our chaps huge plates of carrots, to jolly good effect.

Things that work in theory often do not work in practice, and in such cases we need to revise our theories, even if it is painful. Hormone replacement therapy seemed like a good idea for many decades, until the follow-up studies revealed the problems with it, so we changed our views. And calcium supplements once looked like a good idea for osteoporosis, but now it turns out that they probably increase the risk of heart attacks in older women, so we change our view.

The food supplement industry has engineered itself a beneficent public image, but this is not borne out by the facts. First, there is essentially no difference between the vitamin industry and the pharmaceutical and biotech industries (that is one message of this book, after all: the tricks of the trade are the same the world over).

“Doubt is our product,” he wrote, “since it is the best means of competing with the ‘body of fact’ that exists in the minds of the general public. It is also the means of establishing a controversy.”

More than anything it illustrates how this atomized, overcomplicated view of diet can be used to mislead and oversell. I don’t think it’s melodramatic to speak of people disempowered and paralyzed by confusion, with all the unnecessarily complex and conflicting messages about food. If you’re really worried, you can buy Fruitella Plus, which are nasty, chewy sweets but with added vitamins A, C, E, and calcium. In the last five years, even chocolate companies have jumped on the bandwagon, in the ultimate expression of how nutritionism has perverted and distorted our common sense about food.

If I were writing a lifestyle book, it would have the same advice on every page, and you’d know it all already. Eat lots of fruit and vegetables, and live your whole life in every way as well as you can: exercise regularly as part of your daily routine, avoid obesity, don’t drink too much, don’t smoke, and don’t get distracted from the real, basic, simple causes of ill health.

I’ve got no great beef with the organic food movement (even if its claims are a little unrealistic), but it’s still interesting to note that Graham’s health food store—in 1837—heavily promoted its food as being grown according to “physiological principles” on “virgin unvitiated soil.” By the retro-fetishism of the time, this was soil that had not been “subjected” to “overstimulation”…by manure.

Let’s look at the evidence. Diet has been studied very extensively, and there are some things that we know with a fair degree of certainty: there is reasonably convincing evidence that having a diet rich in fresh fruit and vegetables, with natural sources of dietary fiber, avoiding obesity, moderating one’s intake of alcohol, cutting out cigarettes, and taking physical exercise are protective against such things as cancer and heart disease. Nutritionists don’t stop there, because they can’t; they have to manufacture complication, to justify the existence of their profession.

The remedies she advocated for AIDS are beetroot, garlic, lemons, and African potatoes. A fairly typical quote, from the health minister in a country where eight hundred people die every day from AIDS, was this: “Raw garlic and a skin of the lemon—not only do they give you a beautiful face and skin but they also protect you from disease.”

Big pharma is evil; I would agree with that premise. But because people don’t understand exactly how big pharma is evil, their anger gets diverted away from valid criticisms—its role in distorting data, for example, or withholding lifesaving AIDS drugs from the developing world—and channeled into infantile fantasies. “Big pharma is evil,” goes the line of reasoning; “therefore homeopathy works and the MMR vaccine causes autism.” This is probably not helpful.

you do Phase I, or “first in man,” studies on a small number of brave, healthy young men who need money, first to see if it kills them and also to measure basic things like how fast the drug is excreted from the body (this is the phase that went horribly wrong in the TGN1412 tests in 2006, when several young men were seriously injured). If this works, you move to a Phase II trial, in a couple of hundred people with the relevant illness, as a “proof of concept,” to work out the dose and to get an idea if it is effective or not. A lot of drugs fail at this point, which is a shame; bringing a drug to market costs around five hundred million dollars in total. Then you do a Phase III trial, in hundreds or thousands of patients, randomized, blinded, comparing your drug against placebo or a comparable treatment, and collect much more data on efficacy and safety. You might need to do a few of these, and then you can apply for a license to sell your drug. After it goes to market, you should be doing more trials, and other people will probably do trials and other studies on your drug too, and we hope everyone will keep their eyes open for any previously unnoticed side effects, ideally reporting them to their country’s drug regulator (anyone can report an adverse event to the FDA’s MedWatch system online).

But drug trials are expensive, so an astonishing 90 percent of clinical drug trials, and 70 percent of trials reported in major medical journals, are conducted or commissioned by the pharmaceutical industry. A key feature of science is that findings should be replicated, but if only one organization is doing the funding, then this feature is lost.

The issue has been studied so frequently that in 2003 a systematic review found thirty separate studies looking at whether funding in various groups of trials affected the findings. Overall, studies funded by a pharmaceutical company were found to be four times more likely to give results that were favorable to the company than were independent studies.

Publication bias is a very interesting and very human phenomenon. For a number of reasons, positive trials are more likely to get published than negative ones. It’s easy enough to understand, if you put yourself in the shoes of the researcher.

Vioxx was taken off the market in 2004, but analysts from the FDA estimated that it had caused between 88,000 and 139,000 heart attacks, 30 to 40 percent of which were probably fatal, in its five years on the market. It’s hard to be sure if that figure is reliable, but when you look at the pattern of how the information came out, it’s certainly felt, very widely, that both Merck and the FDA could have done much more to mitigate the damage done over the many years of this drug’s life span, after the concerns were apparent to them. Data in medicine is important; it means lives. Merck has not admitted liability and has proposed a $4.85 billion settlement in the United States.

What’s truly extraordinary is that almost all these problems—the suppression of negative results, data dredging, hiding unhelpful data, and more—could largely be solved with one very simple intervention that would cost almost nothing: a clinical trials register, public, open, and properly enforced. This is how it would work. You’re a drug company. Before you even start your study, you publish the protocol for it, the methods section of the paper, somewhere public. This means that everyone can see what you’re going to do in your trial, what you’re going to measure, how, in how many people, and so on, before you start.

It’s worth noting that drug adverts aimed directly at the public are legally allowed only in the United States and New Zealand, as pretty much everywhere else in the developed world has banned them, for the simple reason that they work. Patients are so much more easily misled by drug company advertising than doctors that the budget for direct-to-consumer advertising in America has risen twice as fast as the budget for addressing doctors directly. These ads have been closely studied by medical academic researchers and have been repeatedly shown to increase patients’ requests for the advertised drugs, as well as doctors’ prescriptions for them.