Four different study designs to evaluate vaccine safety were equally validated with contrasting limitations Jason M. Glanz a, * , David L. McClure a , Stanley Xu a , Simon J. Hambidge a,b,c , Martin Lee d , Margarette S. Kolczak e , Ken Kleinman f , John P. Mullooly g , Eric K. France a a Clinical Research Unit, Kaiser Permanente, P. O. Box 378006, Colorado, Denver, CO, 80237-8066, USA b Community Health Services, Denver Health Medical Center, Denver, CO, USA c Department of Pediatrics, University of Colorado School of Medicine, Denver, CO, USA d UCLA Center for Vaccine Research, Torrance, CA, USA e Centers for Disease Control and Prevention, Atlanta, GA, USA f Department of Ambulatory Care and Prevention, Harvard Medical School and Harvard Pilgrim Health Care, Boston, MA, USA g Center for Health Research, Northwest Kaiser Permanente, Portland, OR, USA Accepted 16 November 2005 Abstract Objective: We conducted a simulation study to empirically compare four study designs [cohort, case–control, risk-interval, self- controlled case series (SCCS)] used to assess vaccine safety. Study Design and Methods: Using Vaccine Safety Datalink data (a Centers for Disease Control and Prevention-funded project), we simulated 250 case sets of an acute illness within a cohort of vaccinated and unvaccinated children. We constructed the other three study designs from the cohort at three different incident rate ratios (IRRs, 2.00, 3.00, and 4.00), 15 levels of decreasing disease incidence, and two confounding levels (20%, 40%) for both fixed and seasonal confounding. Each of the design-specific study samples was analyzed with a regression model. The design-specific ^ b estimates were compared. Results: The ^ b estimates of the case–control, risk-interval, and SCCS designs were within 5% of the true risk parameters or cohort estimates. However, the case–control’s estimates were less precise, less powerful, and biased by fixed confounding. The estimates of SCCS and risk-interval designs were biased by unadjusted seasonal confounding. Conclusions: All the methods were valid designs, with contrasting strengths and weaknesses. In particular, the SCCS method proved to be an efficient and valid alternative to the cohort method. Ó 2006 Elsevier Inc. All rights reserved. Keywords: Simulation study; Cohort; Case–control; Risk-interval; Self-controlled case series (SCCS); Bias (epidemiology); Confounding factors (epidemiology) 1. Introduction The most widely accepted methods for evaluating vac- cine safety have been study designs that compare distinct exposed and unexposed, or diseased and nondiseased pop- ulations. These study methods include prospective designs such as the cohort, and retrospective designs such as the case–control. This investigation evaluates these traditional study designs as well as two newer designs in a simulated analysis of a known, rare, and acute vaccine reaction: idio- pathic thrombocytopenic purpura (ITP) after measles- mumps-rubella (MMR) vaccination [1,2]. In a cohort study, a group of healthy vaccinated and unvac- cinated individuals are followed forward in time, and the in- cidence of illness in the two groups is compared. This design provides a direct estimate of effect (the incidence rate ratio, IRR), is well suited for rare exposures, and can be used to analyze multiple outcomes [3,4]. It can, however, be difficult and costly to implement when the disease is rare, and because vaccine safety studies typically involve populations with high vaccine coverage rates, there may be few unvaccinated controls available. The design is also susceptible to biases that can be introduced by comparing vaccinated and unvacci- nated populations, as these groups may differ by ethnicity, socioeconomic status, and underlying health states [5]. In nested case–control studies, individuals who experi- enced a particular event over a defined time period are iden- tified. This group of cases is then compared to a control group of event-free individuals from the same time period, who are often matched to the cases on variables such as gender, managed care organization (MCO), and age [1,6– 8]. This design is economical and well suited for rare ill- nesses. In addition, because the cases are typically matched * Corresponding author. Tel.: 303-636-3118; fax: 303-636-3109. E-mail address: jason.m.glanz@kp.org (J.M. Glanz). 0895-4356/06/$ – see front matter Ó 2006 Elsevier Inc. All rights reserved. doi: 10.1016/j.jclinepi.2005.11.012 Journal of Clinical Epidemiology 59 (2006) 808–818