REVIEW ARTICLE Basic study design influences the results of orthodontic clinical investigations Spyridon N. Papageorgiou a,b,c, * , Guilherme M. Xavier d , Martyn T. Cobourne d a Department of Orthodontics, School of Dentistry, University of Bonn, Welschnonnenstr. 17, Bonn 53111, Germany b Department of Oral Technology, School of Dentistry, University of Bonn, Welschnonnenstr. 17, Bonn 53111, Germany c Clinical Research Unit 208, University of Bonn, Welschnonnenstr. 17, Bonn 53111, Germany d Department of Orthodontics, King’s College London Dental Institute, Floor 27, Guy’s Hospital, London SE1 9RT, UK Accepted 18 March 2015; Published online xxxx Abstract Objectives: Meta-analysis is the gold standard for synthesizing evidence on the effectiveness of health care interventions. However, its validity is dependent on the quality of included studies. Here, we investigated whether basic study design (i.e., randomization and timing of data collection) in orthodontic research influences the results of clinical trials. Study Design and Setting: This meta-epidemiologic study used unrestricted electronic and manual searching for meta-analyses in or- thodontics. Differences in standardized mean differences (DSMD) between interventions and their 95% confidence intervals (CIs) were calculated according to study design through random-effects meta-regression. Effects were then pooled with random-effects meta-analyses. Results: No difference was found between randomized and nonrandomized trials (25 meta-analyses; DSMD 5 0.07; 95% CI 5 0.21, 0.34; P 5 0.630). However, retrospective nonrandomized trials reported inflated treatment effects compared with prospective (40 meta- analyses; DSMD 5 0.30; 95% CI 5 0.53, 0.06; P 5 0.018). No difference was found between randomized trials with adequate and those with unclear/inadequate generation (25 meta-analyses; DSMD 5 0.01; 95% CI 5 0.25, 0.26; P 5 0.957). Finally, subgroup analyses indicated that the results of randomized and nonrandomized trials differed significantly according to scope of the trial (effective- ness or adverse effects; P 5 0.005). Conclusion: Caution is warranted when interpreting systematic reviews investigating clinical orthodontic interventions when non- randomized and especially retrospective nonrandomized studies are included in the meta-analysis. Ó 2015 Elsevier Inc. All rights reserved. Keywords: Orthodontics; Meta-analysis; Systematic review; Randomized controlled trial; Prospective clinical study; Retrospective clinical study 1. Introduction 1.1. Background Meta-analysis of clinical trials provides the best evi- dence for evaluating orthodontic interventions because of the increased statistical power and precision [1]. However, if the methodological quality of these studies is suboptimal, then the results will be biased, even if the meta-analysis is conducted to the highest standards (the so-called ‘‘garbage in-garbage out’’ concept) [2]. Ideally, meta-analyses assessing the efficacy of ortho- dontic interventions would include only well-conducted and appropriately reported randomized controlled trials (RCTs), which are seen as the epitome of clinical research [2]. Their principal advantage lies in the random allocation of patients to different interventions, which minimizes se- lection bias [3]. However, high-quality clinical trials do not always exist, and non-RCTs of interventions (non- RCTs) are often included in systematic reviews and meta- analyses in orthodontics [4,5], which can potentially affect their conclusions. Indeed, many widely used interventions in orthodontics are not adequately supported by clinical ev- idence possibly because orthodontics is a field where pa- tient lives are rarely put at risk [6]. Funding: S.N.P. received funds from Clinical Research Unit 208 ‘‘Eti- ology and. Sequelae of Periodontal Diseases - Genetic, Cell. Biological and Biomechanical Aspects’’ (University of Bonn, Bonn, Germany) but no relevance to the topic. G.M.X. and M.T.C. are supported by the Acad- emy of Medical Sciences (The Wellcome Trust, British Heart Foundation, Arthritis Research UK). * Corresponding author. Tel.: þ49-(0)228-287-22449; fax: þ49-(0) 228-287-22588. E-mail address: snpapage@gmail.com (S.N. Papageorgiou). http://dx.doi.org/10.1016/j.jclinepi.2015.03.008 0895-4356/Ó 2015 Elsevier Inc. All rights reserved. Journal of Clinical Epidemiology - (2015) -