Justifying the use of non-experimental methods and disqualifying the use of randomized controlled trials: challenging folklore in evaluation research in crime and justice David Weisburd Published online: 18 April 2010 # Springer Science+Business Media B.V. 2010 Abstract The key limitation of non-experimental evaluation methods is that they require an assumption that all confounding factors related to treatment are identified in the statistical models developed. The key advantage of randomized experiments is that this assumption can be relaxed. In this paper, I describe and explain why this assumption is so critical for non-experiments and why it can be ignored in randomized controlled trials (RCTs). I also challenge what I describe as “folklores” that are used to justify the use of non-randomized studies despite this statistical limitation, and to justify the failure of evaluation researchers in crime and justice to use randomized experiments despite their unique ability to overcome this limitation. I conclude by reinforcing what Joan McCord had argued after a life time of review of evaluations: “(W)henever possible” evaluation studies “should employ random assignment.” Keywords Randomized experiments . Randomized controlled trials (RCTs) . Non-experimental methods . Evaluation studies . Confounding variables There is wide agreement among evaluation researchers that nonrandomized studies present distinct limitations for drawing policy conclusions (Boruch 1997; Campbell and Russo 1999; Farrington et al. 2002; McCord 2003; Weisburd 2003). These limitations come from the fact that the assumptions underlying non-experimental research are very hard to meet in most research contexts. Because of the inherent J Exp Criminol (2010) 6:209–227 DOI 10.1007/s11292-010-9096-2 An earlier version of this paper was delivered as the Joan McCord Lecture at the American Society of Criminology Meeting in St. Louis in November of 2008. I would like to thank Breanne Cave, Lorraine Green Mazerolle, Dave McClure, Shomron Moyal, Anthony Petrosino, Cody Telep, Tal Yonaton, Gali Weissmann, Julie Willis, and David Wilson for their helpful comments on earlier drafts of this work. D. Weisburd (*) Hebrew University, Jerusalem, Israel e-mail: msefrat@mscc.huji.ac.il D. Weisburd George Mason University, Fairfax, VA, USA