EDITORIAL MAKING EVIDENCE-BASED DECISIONS IN MEDICINE: (OR MORE IMPORTANTLY) USING EVIDENCE WHEN THE CASE DOESN’T QUITE FIT Ann Barry Flood, PhD* Center for the Evaluative Clinical Sciences at Dartmouth, Dartmouth Medical School, Hanover, New Hampshire Received 22 December 2003 In trying to establish evidence of an impact on health outcomes associated with a new treatment, a random- ized clinical trial (RCT) is undisputedly called the gold standard—the best evidence you can get from a single study. Better yet, a structured meta-analysis of the evidence from several RCTs evaluating the same ther- apy, such as those conducted under the aegis of the Cochrane groups, (cf The Cochrane Collaboration, 2003; O’Connor et al., 1999b; O’Connor et al., 2001; Olsen et al., 1998) provide the most widely respected scientific basis for estimating the likelihood of achiev- ing any impact on health outcomes (its potential to improve health or at least avoid the untoward conse- quences of untreated disease) as well as estimating harmful side effects of the treatment (its potential to harm). There are some standards to assess the helpfulness and harmfulness of a therapy that take into account the seriousness of the harms and benefits from the treatment as well as their likelihood, and compare it to alternative treatment choices—including no treat- ment. For example, the Federal Drug Agency has several explicit standards it uses to evaluate new drugs or to change the status of a drug’s availability from prescription to over the counter. Various expert panels, such as the Task Force on Preventive Medi- cine, professional organizations, and consensus groups also attempt to set criteria to help weigh the evidence to arrive at a recommended therapy or guidelines for their use. And yet—problems in sorting out what the evidence shows and how it applies to a particular case remain. Despite attempts to use a rational, scientific, neutral approach to assess the evidence base of medicine and to create evidence-based recommendations for the use of therapeutic technologies, the holy grail for clini- cians and their patients being able to know what to do remains elusive. There are of course many reasons why the right answer for a given patient is not always so clear. The controversies surrounding the evidence and therefore the appropriate guidelines for detection and treatment of breast cancer provide a particularly co- gent example of four reasons why there is a break- down in transforming scientific evidence— even that based on RCTs—into practice governing patient/cli- nician decisions. First, consider a brief review of the history of recent controversies in breast cancer. Breast cancer treatment and screening has had sev- eral scientific controversies involving the clinical im- plications of evidence in recent years. Starting in the 1980s, one controversy was whether or when to treat breast cancer with lumpectomy plus radiation versus mastectomy—and how extensive the removal of tissue needed to be even among mastectomy procedures. The primary sources of controversy were disagree- ments over the quality and implications of the evi- dence—including the peer-reviewed quality and rele- vance of studies conducted by and published in other countries like Italy, the United Kingdom, and Canada instead of by American investigators and whether and why it is important to remove most of the breast tissue versus creating only a safe, cancer-free margin around the tumor. * Address correspondence to Ann Barry Flood, Professor of Health Policy and Sociology, Chair, PhD Program, Center for the Evaluative Clinical Sciences at Dartmouth, Dartmouth Medical School, 316 Strasenburgh Hall, HB 7251, Hanover, NH 03755-3863. E-mail: Ann.B.Flood@Dartmouth.edu. Women’s Health Issues 14 (2004) 3– 6 Copyright © 2004 by the Jacobs Institute of Women’s Health. 1049-3867/04 $-See front matter. Published by Elsevier Inc. doi:10.1016/j.whi.2003.12.003