Ending Medical Reversal: Improving Outcomes, Saving Lives

by Vinayak K. Prasad

These four trials studied more than 900 patients randomized to either antibiotics (with surgery reserved for those who got worse) or surgery. The results showed that using antibiotics to treat appendicitis may be better than taking people right to surgery. More than 60 percent of the patients did fine with antibiotics alone and did not require subsequent surgery.

Strikingly, they found that 90 percent of the time, the large treatment effects got smaller when you looked at the other studies on the same question.

Across all the studies, only one intervention had a large effect on mortality: extracorporeal membrane oxygenation (ECMO), a method to oxygenate the blood of newborns who cannot adequately breathe on their own.* In other words: parachutes are uncommon in medicine.

Traditionally, therefore, we set error levels as follows: a standard trial will fail to recognize 20 percent of treatments that are actually beneficial (false negatives) and will mistakenly identify 5 percent of ineffective interventions as being effective (false positives).

It turns out that the likelihood that the positive study you read is a true positive, as opposed to a false positive, depends on how likely it was in the first place that the intervention you are studying would work.

The problem is that for most clinical investigations, there is less than a 50 percent chance that the treatment being evaluated is effective.

Table 10.3 describes the situation when there is only a 10 percent chance that the proposed treatment is beneficial. In this scenario, if there are 125 studies reporting positive outcome, only 80 of them, or 64 percent, are correct. Thus, almost one-third of the treatments that you would adopt would actually be ineffective. These treatments would eventually turn out to be medical reversals.

In the end he concluded that the likelihood that any positive study represents a true finding is probably far less than 50 percent and that even the best randomized trial is only true 85 percent of the time.

91 studies were terminated early, while 424 matching studies were completed. The researchers found that studies of an intervention that were stopped early claimed larger benefits of the treatment than those studies of the same intervention that ran to a preplanned conclusion.

If the meta-analysis was positive, only 68 percent of the RCTs were also positive; and if the meta-analysis was negative, 67

just one or two individual trials are positive and the meta-analysis is negative, we tend to have doubts that the treatment is effective.

Conversely, if an excellent and large randomized trial is negative but a meta-analysis is positive, we would be concerned that there is bias at play in the meta-analysis, most likely publication bias.

hand, when a practice is shown to work in a large randomized trial, although not everyone benefits, people do benefit, on average. This is not trivial. In human history, there are few things we have done that we know brought about net good.

It is hard to believe that just 15 years ago there was nearly universal enthusiasm for hormone-replacement therapy in postmenopausal women. Some practitioners’ current embrace of treating age-related testosterone decline in men suggests how completely we have forgotten our error.

with the rise of the influence of the biopharmaceutical industry, more and more often, promising reports come in what appear to be well-done randomized trials, whose flaws are revealed only upon careful scrutiny.

The authors found that industry-sponsored studies were more likely to reach positive conclusions regarding the benefits or cost-effectiveness of a therapy and more likely to test a new therapy against a placebo (as opposed to a real competitor).

When considering both original trials and meta-analyses, industry-sponsored studies are four times as likely to reach a positive conclusion.

industry-sponsored studies are less likely to be published or presented, or to have been published after a delay, than nonindustry research.

When the trial started, the two drugs were dosed equivalently, but if a patient required a dose reduction, the dose of the industry drug fell a bit, while the dose of the comparison drug fell a lot. This

suvorexant,

randomize medical schools to the placement of posters and fliers and follow the weights of a random sample of students, faculty, and staff before and after. This design is called a “cluster randomized trial.”

Whatever the reason, our tendency to believe that “newer is better” is real. Focus groups of health-care consumers reveal this bias.* In one study, not only did people believe that newer technology was usually better, but one-third also felt that medical treatments that work the best usually cost the most.

Tomorrow’s doctor probably needs to master three areas while in medical school: the doctor-patient relationship, systems-based practice, and practice-based learning. First, the doctor-patient relationship.

There is a quiet bias that it is better to be an innovator than a critic.

What if we said to the next 1,000 pneumonia patients, “There are many different ways to treat this infection. All of them work, but we do not know which is best. To study this, we are going to randomly pick one of these effective treatments for you, unless you want to opt out.”

Few people question colon-cancer screening, but it takes doing endoscopy on 191 people and following them for 11 years to prevent one case of colon cancer.

In these cases, when a patient’s condition is deteriorating, and when the cause is neither clear nor typical, it is reasonable to suspend the burden-of-proof principle and do what seems reasonable.

We are only talking about hypertension. If you have heart failure, you really need your carvedilol or metoprolol succinate!

In later chapters we’ll learn about other procedures that make people feel better but provide no real benefit. These examples will make you reconsider the whole “feeling better” issue.

* If you wonder why we are taking physical therapy for granted, you are catching on. Is physical therapy really better than just telling patients to exercise at home? Or maybe, do nothi...

Countries with nationalized health care frequently produce very successful cohort studies. It has been said that every time someone sneezes in Sweden, someone writes it down.

The other problem with the study is the use of priming. Priming means that people give you the answer you told them to give. At a simple level, it is the difference between asking, “What color was the car?” versus “The car was white, wasn’t it?” In the study by Kesselheim and colleagues, a close look at the methods reveals that three times, prior to the survey, doctors were reminded that the study they were participating in was “not associated with any pharmaceutical manufacturer.” This repeated mention of funding source arguably primed readers to be more vigilant about the funding source of research. In a way, Kesselheim and colleagues may have inadvertently sown the seeds for the results of their study in their repeated insistence that it was not industry funded.

Eszopiclone (Lunesta), one of these drugs, earned about $225 million in the fourth quarter of 2013 alone. A recent meta-analysis showed that, on average, these drugs get you to sleep only about 22 minutes faster.