Journal of Clinical Epidemiology 55 (2002) 545–555 0895-4356/02/$ – see front matter © 2002 Elsevier Science Inc. All rights reserved. PII: S0895-4356(02)00395-5 Citation of randomized evidence in support of guidelines of therapeutic and preventive interventions Ioannis A. Giannakakis a , Anna-Bettina Haidich a , Despina G. Contopoulos-Ioannidis a,b , George N. Papanikolaou a , Maria S. Baltogianni a , John P.A. Ioannidis a,c, * a Clinical Trials and Evidence-Based Medicine Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45110, Greece b Department of Pediatrics, George Washington University School of Medicine and Health Sciences, Washington, DC, USA c Department of Medicine, Tufts University School of Medicine, Boston, MA., USA Abstract Guideline statements may be supported by evidence obtained from various study designs, but randomized trials are usually considered most im- portant for making recommendations about therapeutic and preventive interventions. This study evaluated the extent to which randomized trials are cited in guidelines published in major journals. The references of 191guidelines of therapeutic and/or preventive interventions published in Annals of Internal Medicine, BMJ, JAMA, Lancet, NEJM and Pediatrics in 1979, 1984, 1989, 1994, and 1999, were analyzed. The percentage of guidelines not citing any randomized controlled trials (RCTs) decreased gradually from 95% in 1979 to 53% in 1999. Among 4,853 references of the guidelines, there were 393 RCTs (8.1% of total), 19 systematic reviews (0.4%), and 23 meta-analyses of RCTs (0.5%). Among 19 guidelines published in 1999 or 1994 with 2 RCTs cited, in eight cases additional pertinent RCTs were identified that had not been cited by the guideline. There is a clear in- crease in the use of randomized evidence by guidelines over time. However, several guidelines in major journals still cite few or no RCTs. © 2002 Elsevier Science. Inc. All rights reserved. Keywords: Practice guidelines; Randomized controlled trial; Evidence; Citation 1. Introduction Guidelines are aimed to influence clinical decisions and clinical care [1]. Recently, there have been concerns about the rigorousness of the guideline development process [1–4]. It is recognized that these influential publications should reflect the best available evidence [5,6]. Useful evi- dence could be obtained from many different study designs. For example, basic research can provide insight into patho- physiology and biologic plausibility, and observational un- controlled studies may offer insight about prevalence issues. Nevertheless, for decisions pertaining to therapeutic and preventive interventions, controlled studies are usually con- sidered essential. Controlled studies include both random- ized controlled trials (RCTs) and nonrandomized observa- tional studies. The relative merits of these two large groups have been widely debated [7–11]. Although both are impor- tant, RCTs have conventionally been considered to provide a higher level of evidence for making recommendations in guideline development [12–16]. In this article, citations to randomized evidence were evaluated in a sample of guide- lines published in six major medical journals over the last 20 years. This database was used to assess whether the de- pendence of guidelines on randomized evidence is improv- ing over time, and whether there are parameters that influ- ence the use of randomized evidence by these influential publications. Furthermore, it was possible to clarify whether the lack of citation of RCTs in specific guidelines reflects mostly the fact that no RCTs are available or the fact that RCTs may be available, but are not cited. 2. Methods 2.1. Source of guidelines and eligibility criteria The study was based on six prestigious medical journals that are likely to publish guidelines with major impact on medical practice. Hand searching was performed on the An- nals of Internal Medicine, BMJ, JAMA, Lancet, NEJM, and Pediatrics for guidelines published in five incremental years, 5 years apart (1979, 1984, 1989, 1994, and 1999). The sampling approach allowed evaluation of trends in guidelines over a period of 20 years. Hand searching was preferred over computerized searches to minimize loss of retrievals due to inadequate indexing. * Corresponding author. Tel.: +30-651-097807; fax: +30-651-067881. E-mail address: jioannid@cc.uoi.gr (J.P.A. Ioannidis).