Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects Su Golder a, * , Yoon Loke b , Heather M. McIntosh c a Centre for Reviews and Dissemination (CRD), University of York, York, YO10 5DD, UK b University of East Anglia, Norwich NR4 7TJ, UK c Cochrane Infectious Diseases Group Accepted 5 June 2007 Abstract Objective: Systematic reviews incorporating adverse effects are assuming increasing importance as questions raised extend beyond clinical effectiveness to all effects (beneficial and harmful). The aim of this study was to survey the methods used to identify relevant studies for systematic reviews of adverse effects. Study Design and Setting: All records within the Database of Abstracts of Reviews of Effects and the Cochrane Database of System- atic Reviews were scanned for systematic reviews in which the primary outcomes were adverse effects. Two information professionals independently assessed the methods used to identify relevant research as reported in the 277 reviews that met the inclusion criteria. Results: A major weakness of the reviews was inadequate reporting of the search strategies used. In addition, of the reviews that did report a search strategy, few used the sensitive search strategies recommended for systematic reviews. The majority of reviews did not search more than one or two databases, and few other methods of identifying information were used. Conclusion: This investigation shows the variation in the searching element of systematic reviews of adverse effects and demonstrates that the reporting of the methods used to identify research in such reviews could be vastly improved. Ó 2008 Elsevier Inc. All rights reserved. Keywords: Systematic review; Literature review; Adverse effects; Information retrieval; Meta-analysis; Database 1. Introduction Systematic reviews aim to provide comprehensive, unbi- ased evaluations of the effects of health care interventions. This process of systematic assessment often requires au- thors to adhere to painstakingly thorough methods for iden- tifying relevant research. Most reviews have concentrated on identifying information regarding the effectiveness of interventions, but there is now a growing realization that adverse effects should be scrutinized in the same thorough manner [1]. Conducting systematic reviews of adverse effects brings about many new challenges to systematic review methodology, not least in the retrieval of adverse effects information. Searching for relevant studies to include in a systematic review is one of the most important steps in the systematic review process, and a thorough search is one of the key fac- tors that distinguishes a systematic review from other types of review. Missing relevant studies can result in bias for systematic reviews. A thorough search depends on the sen- sitivity of the search strategies as well as the variety of sources searched. Sensitive searches tend to use a mixture of text words and indexing as well as synonyms, and appro- priate truncation [2,3]. A number of researchers have reported on the difficul- ties faced when searching electronic databases for infor- mation on adverse effects [4e7]. Unlike the assessment of clinical effectiveness, the difficulties with identifying adverse effects information are compounded by poor re- porting and indexing of adverse effects terms [4e7]. The adverse effects of interest may also include new, previ- ously unrecognized effects, so it may not be known which particular adverse effects the searches should be designed to retrieve [1,4,7,8]. Moreover, there are a wide variety of study designs that are potentially useful in evaluating * Corresponding author. Tel.: þ44-1904-321095; fax: þ44-1904- 321041. E-mail address: spg3@york.ac.uk (S. Golder) or y.loke@uea.ac.uk (Y. Loke) or hmmci@tiscali.co.uk (H.M. McIntosh). 0895-4356/08/$ e see front matter Ó 2008 Elsevier Inc. All rights reserved. doi: 10.1016/j.jclinepi.2007.06.005 Journal of Clinical Epidemiology 61 (2008) 440e448