School improvement plans and student learning in Jamaica § Marlaine Lockheed a, *, Abigail Harris b,1 , Tamara Jayasundera c,2 a World Bank (retired), United States b School Psychology, Fordham University, United States c National Community Reinvestment Coalition, United States 1. Introduction Site-based interventions to improve student learning outcomes, based on needs assessments and school improvement plans, often called ‘‘comprehensive school reform’’ have been hallmarks of school reform in North America over the past decade (Bifulco et al., 2005; Cross, 2004; Ross et al., 2008). Many programs have been subject to large-scale evaluations. But recent longitudinal studies and meta-analyses have concluded that these reforms have only modest effects on students’ math and reading performance. For example, Gross et al. (2009) recently reported a highest effect size of .04 for a program covering 231 schools in Texas, and Borman et al. (2004) found that the average effect sizes across 232 studies of comprehensive school reform were .13 for reading and .15 for math. 3 Although larger effects were found for a few specific program models, virtually no reform attributes – such as specific curriculum materials, specific instructional practices, parent governance, goals and benchmarks or ongoing professional development – were systematically related to higher achievement, and the only methodological variable statistically related with student performance was that the evaluation was conducted by the program developer. In addition, the effectiveness of school improvement plans in boosting student learning suggest that if such plans are not owned by the school they will show little impact on student achievement (Borman et al., 2004). Although many donor agencies have both encouraged and financially supported comprehensive school reforms in developing countries, large-scale evaluations of such reforms in low- or middle-income countries are rare (Riddell, 2008). 4 Given the costs associated with such reforms and the rather modest impact they have shown in the U.S., research on comprehensive school reform in developing countries is essential for improved policy. This study adds to the literature by its independent evaluation of Jamaica’s primary school improvement program, the New Horizons for Primary Schools (NHP), which incorporated needs assessments and school improvement plans to develop site-specific interven- tions for program schools. The program was implemented by an International Journal of Educational Development 30 (2010) 54–66 ARTICLE INFO Keywords: Comparative education International education Development Educational policy Learning Propensity score matching HLM Comprehensive school reform ABSTRACT A school improvement program that provided support to poor-performing schools on the basis of needs identified in a school improvement plan was implemented in 72 government schools in Jamaica, from 1998 to 2005. In this independent evaluation of the program, we use propensity score matching to create, post hoc, a control group of schools that were similar to program schools in the baseline year. By the final year of the program, we find that program schools had received more inputs to improve literacy and numeracy than control schools, and that some inputs associated with the program were correlated with improvements school average achievement: supplementary reading materials, additional training for reading resource teachers, and functioning computers. At the student level, however, we find no evidence that students enrolled in program schools achieved higher reading or math scores than those in control schools. We suggest three possible reasons for this: (a) the lack of sensitivity of the learning measures to improvements at the lower end of the scales; (b) the availability of program-like inputs in non-program schools, provided by other programs and donors; and (c) the growth in student enrollment in the program schools, which may have diluted the program effect for incoming students in upper grades. Schools with school improvement plans did not outperform comparable schools that did not have these plans. ß 2009 Elsevier Ltd. All rights reserved. § This paper is based on a longer evaluation report prepared for USAID and funded through USAID Contract Number NE-I-000-00-0076-00 to the Academy of Educational Development. * Corresponding author at: 561 Lake Drive, Princeton, NJ 08540, United States. Tel.: +1 609 921 3930; fax: +1 609 921 3930. E-mail addresses: mlockheed@verizon.net, lockheed@princeton.edu (M. Lock- heed), harris@fordham.edu (A. Harris), tjayasundera@gmail.com (T. Jayasundera). 1 Tel.: +1 212 636 6466; fax: +1 212 636 6416. 2 Tel.: +1 240 499 4015. 3 The modal effect size, however, was close to .00. 4 McEwan and Patrick’s (2008) study of long-term reforms in Chile and various evaluations of the EDUCO program are exceptions. Contents lists available at ScienceDirect International Journal of Educational Development journal homepage: www.elsevier.com/locate/ijedudev 0738-0593/$ – see front matter ß 2009 Elsevier Ltd. All rights reserved. doi:10.1016/j.ijedudev.2009.06.007