How doctors decide
I hope to write regularly on doctors’ decision-making. This first installment concerns types of evidence-based studies. Later, I will talk about how and whether doctors actually use such evidence to make decisions.
There are a number of kinds of evidence-based advice which doctors can use to make decisions. The differences among them are instructive.
1. Individual journal articles. These can present findings from a single trial. The potential biases are multiple. Not all findings get published (publication bias – i.e. we never find out about the results which do not support the hypothesis). Many results are never reproduced. To quote one widely read study, “studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported.” Some studies have to be retracted later by the journal which publishes them.
2. Reviews. These can be of many kinds. A narrative review recounts in a summary form the findings of a number of articles. The summary is susceptible to bias from many factors: which articles are chosen, what emphases are given, which author is doing the review. For that reason, some people prefer a –
3. Systematic review. This combines research articles on a given topic according to strict definitions made a priori. Then the results from these individual articles are combined only if they are comparable, i.e. applying to the same population or disease. Some of the time individual results can be merged in a statistical way, called a meta-analysis. I work on a number of these; they have their own problems (the link is from an orthodontics journal but the issues are generally applicable).
4. Guidelines. Using the above reviews, a group of experts, often physicians, draft official recommendations for a given disease, group of diseases, or use of a given medication or procedure. But guidelines are only as good as the evidence they are based on, the applicability of the guidelines to the particular question that is being considered — is the patient from the population that was studied? is the clinical condition the same? — and any bias that might have affected the results.
To take one example of a well-done review, the American Academy of Neurology and the American Headache Society recently published Evidence-based guideline update: Pharmacologic treatment for episodic migraine prevention in adults. Even such a guideline, however, comes with weaknesses. For some of these we can quote the authors: while good-quality evidence does support the effectiveness of certain medications for treating migraines, “evidence is unavailable to help the practitioner choose one therapy over another.”
Another sort of weakness was also reported by these authors, but in a different way, in an often-ignored paragraph at the end of the article. For those that can’t read the small type, these are disclosures of possible conflicts of interest. Nearly all the authors have received honorariums from a number of pharmaceutical companies. I don’t think this completely obviates the recommendations, but it might if we look closely enough. This is why, to my way of thinking, any such guidelines need to be either free of conflict of interest or ensure balanced COIs within the writers.
Dr. Silberstein is on the advisory panel of and receives honoraria from AGA, Allergan, Amgen, Capnia, Coherex, Colucid, Cydex, GlaxoSmithKline, Lilly, MAP, Medtronic, Merck, Minster, Neuralieve, NINDS, NuPathe, Pfizer, St. Jude Medical, and Valeant. He is on the speakers’ bureau of and receives honoraria from Endo Pharmaceuticals, GlaxoSmithKline, and Merck. He serves as a consultant for and receives honoraria from Amgen and Novartis. His employer receives research support from AGA, Allergan, Boston Scientific, Capnia, Coherex, Endo Pharmaceuticals, GlaxoSmithKline, Lilly, MAP, Medtronic, Merck, NINDS, NuPathe, St. Jude Medical, and Valeant Pharmaceuticals. Dr. Holland (formerly Dr. Pearlman) receives consulting income from Map Pharmaceuticals and the American Headache Society and research support from Albert Einstein College of Medicine. Dr. Freitag has served on the scientific advisory boards of Zogenix Pharmaceuticals, Allergan Pharmaceuticals, Nautilus, MAP Pharmaceuticals, and Nupathe; has received travel expenses and or honoraria from GlaxoSmithKline, Zogenix, Merck, Nautilus, Allergan, Diamond Headache Clinic Research and Educational Foundation (not for profit), and the American Headache Society (travel). Dr. Freitag is a member of the Board of Directors of the National Headache Foundation. Dr. Dodick, within the past 3 years, serves on advisory boards and has consulted for Allergan, Alder, Pfizer, Merck, Coherex, Ferring, Neurocore, Neuralieve, Neuraxon, NuPathe Inc., MAP, SmithKlineBeecham, Boston Scientific, Medtronic, Inc., Nautilus, Eli Lilly & Company, Novartis, Colucid, GlaxoSmithKline, Autonomic Technologies, MAP Pharmaceuticals, Inc., Zogenix, Inc., Impax Laboratories, Inc., Bristol Myers Squibb, Nevro Corporation, Atlas, Arteaus, and Alder Pharmaceuticals. Within the past 3 years, Dr. Dodick has received funding for travel, speaking, or editorial activities from CogniMed, Scientiae, Intramed, SAGE Publishing, Lippincott Williams & Wilkins, Oxford University Press, Cambridge University Press, Miller Medical, Annenberg for Health Sciences; he serves as Editor-in-Chief and on the editorial boards of The Neurologist, Lancet Neurology, and Postgraduate Medicine; and has served as Editor-in-Chief of Headache Currents and as an Associate Editor of Headache; he receives publishing royalties for Wolff’s Headache, 8th edition (Oxford University Press, 2009) and Handbook of Headache (Cambridge University Press, 2010). Within the past 3 years, Dr. Dodick has received research grant support from Advanced Neurostimulation Systems, Boston Scientific, St Jude Medical, Inc., Medtronic, NINDS/NIH, Mayo Clinic. Dr. Argoff has served on a scientific advisory board for the Department of Defense and DSMB for the NIH; has received funding for travel and/or speaking and/or has served on a speakers’ bureau for Pfizer (King), Janssen (Pricara), Millennium Laboratories, Neurogesx, Forest Laboratories, Eli Lilly, Covidien, and Endo Pharmaceuticals; has received research support from Endo Pharmaceuticals, Forest Laboratories, Eli Lilly, Neurogesx, Pfizer, and SBRT funded by the NIH; and has received stock/stock options from Pfizer. Dr. Ashman is the Level of Evidence editor for Neurology and serves on the AAN Guideline Development Subcommittee. He reports no other disclosures. Full disclosures were provided at the time of Board approval. Go to Neurology.org for full disclosures.
