Methods in Public Health Services and Systems Research A Systematic Review Jenine K. Harris, PhD, Kate E. Beatty, MPH, Colleen Barbero, MPPA, Alex F. Howard, MPH, Robin A. Cheskin, BA, Robert M. Shapiro II, MALS, Glen P. Mays, PhD, MPH Context: Public Health Services and Systems Research (PHSSR) is concerned with evaluating the organiza- tion, fınancing, and delivery of public health services and their impact on public health. The strength of the current PHSSR evidence is somewhat dependent on the methods used to examine the fıeld. Methods used in PHSSR articles, reports, and other documents were reviewed to assess their methodologic strengths and challenges in light of PHSSR goals. Evidence acquisition: A total of 364 documents from the PHSSR library met the inclusion criteria as empirical and based in the U.S. After additional exclusions, 327 of these were analyzed. Evidence synthesis: A detailed codebook was used to classify articles in terms of (1) study design; (2) sampling; (3) instrumentation; (4) data collection; (5) data analysis; and (6) study validity. Inter-coder reliability was assessed for the codebook; once it was found reliable, the available empirical documents were coded. Conclusions: Although there has been a dramatic increase in the amount of published PHSSR recently, methods used remain primarily cross-sectional and descriptive. Moreover, although appro- priate for exploratory and foundational work in a new fıeld, these approaches are limiting progress toward some PHSSR goals. Recommendations are given to advance and strengthen the methods used in PHSSR to better meet the goals and challenges facing the fıeld. (Am J Prev Med 2012;42(5S1):S42–S57) © 2012 American Journal of Preventive Medicine Context P ublic health services and systems research (PHSSR) is a multidisciplinary fıeld of study con- cerned with evaluating the “organization, fınanc- ing, and delivery of public health services and the impact of these services on public health.” 1,2 PHSSR brings to- gether a wide variety of research areas and theoretic and methodologic traditions. 1,3–5 Historically, the development of PHSSR has been slow, with progress not always apparent; however, develop- ment has begun to speed up over the last few decades. In 1988, the influential IOM Future of Public Health report 6 called for the development of the evidence base in PHSSR. In 1990, the DHHS decennial publication, Healthy People, set the goal that by 2000 at least 90% of the population would be served by a public health depart- ment that effectively carries out the IOM’s core func- tions. 2,7 In direct response, the CDC and the National As- sociation of County and City Health Offıcials (NACCHO) began to research strategies for guidelines and self-assessment tools to measure how well public health agencies carried out core functions. 2 Between 2001 and 2009, the U.S. government invested more than $10 billion in new funds to support public health activities. 2 In the midst of these 10 years of public health investment, CDC introduced its fırst PHSSR agenda (in 2003) and the Robert Wood Johnson Foundation (RWJF) began to support efforts to convene representatives from states working on PHSSR (in 2006). 2 In 2007, the RWJF awarded the University of Kentucky Research Foundation more than $2.8 million for “Creating a resource center for public health systems and services research.” During this time of growth, six overarching goals for PHSSR were defıned: (1) determine how public health From the George Warren Brown School of Social Work (Harris, Barbero, Cheskin), Washington University in St. Louis, the School of Public Health (Beatty), Saint Louis University, St. Louis, Missouri; the College of Public Health (Howard), the Medical Center Library (Shapiro), University of Kentucky, Lexington, Kentucky; and the University of Arkansas for Medi- cal Sciences (Mays), Little Rock, Arkansas Address correspondence to: Jenine K. Harris, PhD, Assistant Professor, George Warren Brown School of Social Work, Campus Box 1196, Wash- ington University in St. Louis, St. Louis MO 63130. E-mail: jharris@ brownschool.wustl.edu. 0749-3797/$36.00 doi: 10.1016/j.amepre.2012.01.028 S42 Am J Prev Med 2012;42(5S1):S42–S57 © 2012 American Journal of Preventive Medicine • Published by Elsevier Inc.