Reporting of Validity from School Health Promotion Studies Published in 12 Leading Journals, 1996-2000 zyx Paul Estabrooks, David A. Dzewaltowski, Russell E. Glasgow, Lisa M. Klesges ABSTRACT: zyxwvutsrqponml A targeted rrview was conducted of school-based, controlled intervention studies that promoted good nutrition,physical activity, or smoking cessationlprevention,and were published in one of 12 leading health behavior journals between I996 and 2000. The RE-AIM framework was used to evaluate the zyxwvutsrq extent to which each paper reported on elements of reach, efficacy, adoption, imple- mentation, and maintenance. Thirty-twopublications were reviewed. Reporting rates across the RE-AIM dimensions varied substan- tially: Reach = 59.3%; Efficacy = 100%; Adoption = 14.8%; Implementation = 37%; Maintenance = 25.9% for individuals, 0% for schools. Few studies reported zyxwvutsrqp if characteristics of the study sample were representative of those found in the broader population of students or schools. Among studies reporting on the RE-AIM dimensions, participation rates generally were high (median 82%}. adoption rates were moderate (median 72.5%),and reports of implementation were high (87%). To increase the potential to translate controlled research to "real-world practice conditions, a stronger emphasis should he placed on reporting the representativeness of the sample of students and schools. zyxwvutsrqp (J zyxwvutsrqp Sch Health. 2003;73(1):21-28) ifelong patterns of health behavior often are established during an individual's school-aged years and, unfortu- nately, current patterns indicate poor health in the nation's children.* Everyday in the United States thousands of young people start to smoke and one-third of those who start will become dependent,' and physical inactivity and obesity rates have reached epidemic proportion^.^ ' Only 44% of male adolescents and 27% of female adolescents meet the minimum average daily goal of at least five serv- ings of vegetables and fruits.6 Recent reviews document that well-controlled studies of school health promotion programs have positive effects.',* However, little evidence suggests that these intervention strategies translate into sustained programs or general prac- tice. This lack of reported dissemination may relate to researchers' tendency to emphasize issues of internal valid- ity over issues of external validity or generalizability of effects. Yet, studies that report strong external validity in terms of students, schools, and resources available are more suited to dissemination and could greatly facilitate the research to practice translation process. This paper exam- ined the extent to which researchers who conduct school- based health promotion studies report on issues of external validity, and lessons learned in these studies. Better knowl- edge of external validity factors can facilitate implementa- tion of proven and broadly applicable interventions.' If external validity is poor or unknown, this factor may require changes in the way school-based research is conducted and reported, and possibly in the types of inter- ventions evaluated. Given the large number of school-based studies, and the existence of comprehensive reviews on the efficacy of Paul Estabrooks, PhD, Assistant Professor, Dept. of Kinesiology, and office of Community Health; (twist@ksu.edu): and David A. Dzewaltowski, PhD, Professor, Head, Dept. of Kinesiology. Director of the Office of Community Health; (dadx@ksu.edu); Kansas State University, 8 Natatorium, OCH, KSU, Manhattan KS, 66506; Russell E. Glasgow, PhD, Senior Scientist, AMC Cancer Research Center, PO Box 349 Canon City, CO 81215: (russgeris-net); and Lisa M. Klesges, PhD, Associate Professor, Preventive Medicine and Pediatrics, Dept. of Preventive Medicine, University of Tennessee Health Science Center, 66 N. Pauline, Suite 633, Memphis, TN 38/63: (lklesges@utmem.edu). This research was funded by the National Institutes of Health (NICHD NINR ODP NIAID ODs ROlHD37367-02) and The Robert Wood Johnson Foundation. This article was submitted February 26, 2002, and revised and accepted for publication July 29,2002. health promotion strategies, the approach used was to supplement this knowledge by reviewing the external valid- ity issues in recent school-based research. To ensure that high-quality research was included in the review, controlled studies on dietary change, smoking cessation, and physical activity reported between 1996 and 2000 in 12 leading jour- nals, including Journal of School Health, were identified (Table 1). To evaluate these selected studies, it was necessary to identify important external validity elements to be reported. Hence, the RE-AIM evaluation framework was used to structure the review.'.'" This framework brings together important issues relevant for evaluating the impact of public health interventions. Created by Green, Abrams, Rogers, and Oldenburg, the framework places increased emphasis on external validity while still addressing internal validity issues. The framework identifies variables associ- ated with internal validity, but external validity issues were the primary focus of this paper."-'s Glasgow and associates propose that the public health impact of health promotion programs is best evaluated by examining five dimensions' of the RE-AIM model: a) Reach - proportion and representativeness of individuals willing to participate in a given initiative; b) Efficacy or Effectiveness - effect of an intervention on both positive and potential negative outcomes; c) Adoption - proportion and representativeness of settings and intervention agents willing to initiate a program; d) Implementation - interven- tion agents' fidelity to an intervention's protocol including consistency of delivery as intended; and e) Maintenance - extent to which a program or policy becomes institutional- ized or part of routine organizational practices and policies. Maintenance in the RE-AIM framework also has referents at the individual or student level. At the student level, main- tenance has been defined in relation to long-term effects of a program on outcomes six months or more following the most recent intervention contact.' METHODS This review provides a quantitative estimate of the extent to which recent school-based studies in 12 leading journals addressed RE-AIM issues, with an emphasis on external validity. It was hypothesized that studies report more frequently on issues of internal validity than on issues ~~ ~~ Journal of School Health January 2003, Vol. 73, No. 1 21