11 Successful welfare-to-work programs: Were Riverside and Portland really that good? assembled specifically to synthesize findings from evalu- ations of welfare-to-work programs, we examine alterna- tive explanations for the large effects on earnings and welfare receipt found for the Riverside and Portland pro- grams. We ask if the exceptionally favorable results from the programs are based on superior design and implemen- tation—for example, the combination of services and sanctions put into effect—or are instead due to factors not under the control of program administrators, such as fa- vorable socioeconomic conditions or the characteristics of those in the recipient population. Finally, we examine the possibility that sampling errors account for some of the results. Because it is likely that each explanation is to some extent responsible, our goal in this research was to sort out the relative contributions of the program designs and of fac- tors outside the control of the program administrators. For our study, we assembled data on 24 mandatory wel- fare-to-work programs that were implemented between 1982 and 1996—that is, before the introduction of TANF, and while the main cash welfare program was Aid to Families with Dependent Children (AFDC). We in- cluded only programs evaluated using random-assign- ment designs, the “gold standard” of evaluation method- ology. 4 The 24 evaluations provide information about 64 welfare-to-work programs in over 50 sites. In each evalu- ation the sample population was composed entirely or almost entirely of single parents, well over 90 percent of whom were female. We inflated all the financial informa- tion that we used to year 2000 dollars, using the Con- sumer Price Index (CPI-U). From the evaluations themselves we gathered information on the characteristics of the program participants, mea- sures of program effects on earnings and welfare recipiency, and measures of program effects on sanction rates and on the receipt of different services (which we call “net program participation rates”), generally com- puted as differences between the program and the control groups. From official sources, mainly the U.S. Census Bureau and the Bureau of Labor Statistics, we extracted socioeconomic data for the relevant areas, mostly states and counties—site unemployment and poverty rates, the workforce in manufacturing, median household income, and the size of welfare benefits. Most of the evaluations in the study measured program effects for only two years or less; thus we chose to focus on the third and seventh quarters after random assignment. Robert Walker, David Greenberg, Karl Ashworth, and Andreas Cebulla Robert Walker is Professor of Social Policy at the Uni- versity of Nottingham, UK, and Research Fellow at the Institute of Fiscal Studies; David Greenberg is Professor Emeritus of Economics at the University of Maryland, Baltimore County, and an IRP affiliate; Karl Ashworth is Head of Statistical Resources and Andreas Cebulla is Assistant Director at the Centre for Research in Social Policy at Loughborough University, UK. Because work, not cash assistance, is now at the center of the social safety net, it is crucial that we understand which programs might be effective, and to what extent, in moving welfare recipients into stable jobs. But following the shift in primary welfare policymaking to state and local agencies, welfare-to-work programs of all shapes and styles have pro- liferated, making it more difficult to identify successful strategies. The federal Balanced Budget Act of 1997, for example, authorized $3 billion in grants to states and local communities to promote job opportunities and employment preparation for the hardest-to-employ recipients of Tempo- rary Assistance for Needy Families (TANF) and for noncus- todial parents of children on TANF. Under this law alone, by the year 2000, over 700 public and private organizations were running programs, varying widely in their goals, their implementation, and their scope. 1 And these programs con- stitute only a fraction of the many local and state welfare-to- work initiatives currently in place. Policymakers seeking better employment strategies have much previous experience to draw upon, but the lessons of that experience are not necessarily clear nor very encourag- ing. Most evaluations have concluded that welfare-to-work programs have only modest positive effects on participants’ earnings and welfare receipt. Two programs, however, one in Riverside, California, and the other in Portland, Oregon, have emerged as “clear winners” in comparative studies, producing unusually large earnings gains and taxpayer sav- ings and, in Portland, more stable employment and higher wages. 2 As a consequence, both have been considered wor- thy of imitation by welfare agencies throughout the United States and in some European countries. Indeed, the favor- able results from Riverside were influential in shaping the work emphases of the TANF legislation itself. The research reported here asks whether these programs should be emulated. 3 Using analytic tools from meta- analysis (see the box on p. 12) and a unique database Focus Vol. 22, No. 3, Summer 2003