Bad Science

by Ben Goldacre

Children are routinely being taught—by their own teachers, in thousands of British state schools—that if they wiggle their head up and down it will increase blood flow to the frontal lobes, thus improving concentration; that rubbing their fingers together in a special sciencey way will improve ‘energy flow’ through the body; that there is no water in processed food; and that holding water on their tongue will hydrate the brain directly through the roof of the mouth, all as part of a special exercise programme called ‘Brain Gym’.

I will show you evidence that a vanguard of startling wrongness is entering British universities, alongside genuine academic research into nutrition.

You cannot reason people out of positions they didn’t reason themselves into.

To these people ‘science’ is a monolith, a mystery, and an authority, rather than a method.

what do they mean by toxin? I’ve asked the manufacturers of many detox products this question time and again, but they demur.

In terms of basic human biochemistry, detox is a meaningless concept.

we seek redemption and purification from the more extreme forms of our material indulgence: we fill our faces with drugs, drink, bad food and other indulgences, we know it’s wrong, and we crave ritualistic protection from the consequences, a public ‘transitional ritual’ commemorating our return to healthier behavioural norms.

‘Brain buttons lie directly over and stimulate the carotid arteries.’

Brain Gym is promoted by local education authorities, funded by the government, and the training counts as continuing professional development for teachers.

As it happens, this very phenomenon has been studied in a fascinating set of experiments from the March 2008 edition of the Journal of Cognitive Neuroscience, which elegantly demonstrated that people will buy into bogus explanations much more readily when they are dressed up with a few technical words from the world of neuroscience.

This will enhance the placebo effect, but you might also wonder whether the primary goal is something much more cynical and lucrative: to make common sense copyrightable, unique, patented, and owned

This process of professionalising the obvious fosters a sense of mystery around science, and health advice, which is unnecessary and destructive. More than anything, more than the unnecessary ownership of the obvious, it is disempowering. All too often this spurious privatisation of common sense is happening in areas where we could be taking control, doing it ourselves, feeling our own potency and our ability to make sensible decisions; instead we are fostering our dependence on expensive outside systems and people.

Of course, forgetting that technical stuff, most of the ‘evidence’ quoted in cream adverts is from subjective reports, where ‘seven out of ten people who received free pots of cream were very pleased with the results’.

When the cream dries on your face, these long, soggy chains contract and tighten: the slightly unpleasant taut sensation you get on your face when you wear these creams is from the protein chains contracting all over your skin, which temporarily shrinks your finer wrinkles.

Homeopathy was devised by a German doctor named Samuel Hahnemann in the late eighteenth century.

The theory of ‘like cures like’ which he conjured up on that day is, in essence, the first principle of homeopathy.

If you go to Boots the Chemist’s website, www.bootslearningstore.co.uk, for example, and take their 16-plus teaching module for children on alternative therapies, you will see, amongst the other gobbledegook about homeopathic remedies, that they are teaching how Hahnemann’s provings were ‘clinical trials’. This is not true, as you can now see, and that is not uncommon.

It’s not about the dilution, he said: it’s the succussion.

that amazing things simply happen sometimes: people can survive, despite all the odds, for no apparent reason.

Archie Cochrane, one of the grandfathers of evidence-based medicine, once amusingly described how different groups of surgeons were each earnestly contending that their treatment for cancer was the most effective: it was transparently obvious to them all that their own treatment was the best.

Some of the biggest figures in evidence-based medicine got together and did a review of blinding in all kinds of trials of medical drugs, and found that trials with inadequate blinding exaggerated the benefits of the treatments being studied by 17 per cent.

Randomisation is another basic concept in clinical trials. We randomly assign patients to the placebo sugar pill group or the homeopathy sugar pill group, because otherwise there is a risk that the doctor or homeopath—consciously or unconsciously—will put patients who they think might do well into the homeopathy group, and the no-hopers into the placebo group, thus rigging the results.

Academics conventionally measure the quality of a study using standardised tools like the ‘Jadad score’, a seven-point tick list that includes things we’ve been talking about, like ‘Did they describe the method of randomisation?’ and ‘Was plenty of numerical information provided?’

collect all the results from all the trials on a given subject, bung them into one big spreadsheet, and do the maths on that, instead of relying on your own gestalt intuition about all the results from each of your little trials.

As I said, information alone can be life-saving, and one of the greatest institutional innovations of the past thirty years is undoubtedly the Cochrane Collaboration, an international not-for-profit organisation of academics, which produces systematic summaries of the research literature on healthcare research, including meta-analyses.

A landmark meta-analysis was published recently in the Lancet. It was accompanied by an editorial titled: ‘The End of Homeopathy?’ Shang et al. did a very thorough meta-analysis of a vast number of homeopathy trials, and they found, overall, adding them all up, that homeopathy performs no better than placebo.

the trial found that the more elaborate placebo ritual had a greater benefit.

one study found that more than half of all the homeopaths approached advised patients against the MMR vaccine for their children, acting irresponsibly on what will quite probably come to be known as the media’s MMR hoax.

‘confounding variables’: these are things which are related both to the outcome you’re measuring (wrinkles) and to the exposure you are measuring (food), but which you haven’t thought of yet. They can confuse an apparently causal relationship, and you have to think of ways to exclude or minimise confounding variables to get to the right answer, or at least be very wary that they are there.

This is just another example of how nutritionism, despite the ‘alternative medicine’ rhetoric and phrases like ‘holistic’, is actually a crude, unsophisticated, old fashioned, and above all reductionist tradition.

it needs to be a strong association, which is consistent, and specific to the thing you are studying, where the putative cause comes before the supposed effect in time; ideally there should be a biological gradient, such as a dose–response effect; it should be consistent, or at least not completely at odds with, what is already known (because extraordinary claims require extraordinary evidence); and it should be biologically plausible.

There are problems with using such surrogate outcomes. They are often only tenuously associated with the real disease, in a very abstract theoretical model, and often developed in the very idealised world of an experimental animal, genetically inbred, kept under conditions of tight physiological control.

Linus Pauling, the greatgrandfather of modern nutritionism, and his seminal work on vitamin C and the common cold.

This is what the Cochrane Collaboration does on all the healthcare topics that it can find. It even invites people to submit new clinical questions that need an answer. This careful sifting of information has revealed huge gaps in knowledge, it has revealed that ‘best practices’ were sometimes murderously flawed, and simply by sifting methodically through pre-existing data, it has saved more lives than you could possibly imagine.

whether it is theoretically plausible, whether it is backed up by what we know from observing diets and health, whether it is supported by ‘intervention trials’ where we give one group one diet and another group a different one, and whether those trials measured real-world outcomes, like ‘death’, or a surrogate outcome, like a blood test, which is only hypothetically related to a disease.

The theory underlying the view that antioxidants are good for you is the ‘free radical theory of ageing’. Free radicals are highly chemically reactive, as are many things in the body. Often this reactivity is put to very good use. For example, if you have an infection, and there are some harmful bacteria in your body, then a phagocytic cell from your immune system might come along, identify the bacteria as unwelcome, build a strong wall around as many of them as it can find, and blast them with destructive free radicals.

The theory that antioxidants are protective is separate to—but builds upon—the free radical theory of disease. If free radicals are dangerous, the argument goes, and antioxidants on the big diagrams are involved in neutralising them, then eating more antioxidants should be good for you, and reverse or slow ageing, and prevent disease.

Most recently, a Cochrane review looked at the number of deaths, from any cause, in all the placebo-controlled randomised trials on antioxidants which have ever been performed (many of which looked at quite high doses, but perfectly in line with what you can buy in health-food stores), describing the experiences of 230,000 people in total. This showed that overall, antioxidant vitamin pills do not reduce deaths, and in fact they may increase your chance of dying.

Graham crackers are a digestive biscuit invented in the nineteenth century by Sylvester Graham, the first great advocate of vegetarianism and nutritionism as we would know it, and proprietor of the world’s first health food shop.

The Multiple Risk Factor Intervention Trial was one of the largest medical research projects ever undertaken in the history of mankind, involving over 12,866 men at risk of cardiovascular events, who went through the trial over seven years. These people were subjected to a phenomenal palaver: questionnaires, twenty-four-hour dietary recall interviews, three-day food records, regular visits, and more. On top of this, there were hugely energetic interventions which were supposed to change the lives of individuals, but which by necessity required that whole families’ eating patterns were transformed: so there were weekly group information sessions for participants—and their wives—individual work, counselling, an intensive education programme, and more. The results, to everyone’s disappointment, showed no benefit over the control group (who were not told to change their diet).

When did you ever hear about elegant ideas like ‘walking school buses’, or were stories about their benefits crowded out by the latest urgent front-page food fad news?

No. The story was unanimously ignored by the British news media, despite their preoccupation with both antisocial behaviour and miracle cures, for one simple reason: the research was not about a pill. It was about a cheap, practical parenting programme.

At the same time, for over five years now, newspapers and television stations have tried to persuade us, with ‘science’, that fish-oil pills have been proven to improve children’s school performance, IQ, behaviour, attention, and more. In fact, nothing could be further from the truth.

Thirdly, children will do better just from being in a special group that is being studied, observed and closely attended to, since it seems that the simple fact of being in a trial improves your performance, or recovery from illness. This phenomenon is called the ‘Hawthorne effect’, not after a person, but after the factory where it was first observed.

I contacted Durham. They put me on to Madeleine Port-wood, the scientific brains behind this enormous and enduring operation. She appears regularly on telly talking about fish oils, using inappropriately technical terms like ‘limbic’ to a lay audience.

George Orwell first noted, the true genius in advertising is to sell you the solution and the problem.

eyes—but in big pharma the story goes like this: the low-hanging fruit of medical research has all been harvested, and the industry is rapidly running out of novel molecular entities. They registered fifty a year in the 1990s, but now it’s down to twenty a year, and a lot of those are just copies. They are in trouble.

any more than we want to talk about social inequality, the disintegration of local communities, the breakdown of the family, the impact of employment uncertainty, changing expectations and notions of personhood, or any of the other complex, difficult factors that play into the apparent rise of antisocial behaviour in schools.

reminiscent of the well-known free flow of employees in America between the pharmaceutical industry regulator and the boards of various pharmaceutical companies: in fact, at the risk of hammering a point home too hard, you will have noticed by now that I am telling the story of all the pill industries, using examples from mainstream media, and you couldn’t put a cigarette paper between them.)

evening primrose oil is not effective in eczema, and it has lost its medicines licence. The case is still cited by leading figures in evidence-based medicine, such as Sir Iain Chalmers, founder of the Cochrane Collaboration, as an example of a pharmaceutical company refusing to release information on clinical trials to academics wishing to examine their claims.