The challenge of program evaluation: When increasing program participation decreases the relative well-being of participants Craig Gundersen a, * , Dean Jolliffe b , Laura Tiehen b a Department of Agricultural and Consumer Economics, University of Illinois, 324 Mumford Hall, 1301 West Gregory Dr., Urbana, IL 61801-3605, USA b Economic Research Service, US Department of Agriculture, 1800 M Street NW, Washington, DC 20036-5831, USA article info Article history: Received 3 January 2007 Received in revised form 25 February 2008 Accepted 17 December 2008 Keywords: Food insecurity Food Stamp Program Food assistance programs Poverty Survey of Income and Program Participation (SIPP) abstract Two goals of food assistance programs are to improve well-being and to increase participation among those in need. Progress in meeting the first goal can be measured by the difference in well-being between participants and eligible non-participants. This gap in well-being though can be affected by progress made toward the second goal of increasing participation rates. In particular, if those with the lowest lev- els of well-being disproportionately enter the program, the gap can increase. To examine whether this tension between goals exists empirically, we consider the case of the Food Stamp Program and its effect on food insecurity in the United States, using data from the Survey of Income and Program Participation (SIPP). In particular, we use variation in State policies (the change in administrative error rates and the combined value of Temporary Assistance for Needy Families (TANF) and food stamp benefits) to analyze whether increasing participation rates leads to increases in the difference in food insecurity rates. Con- trolling for other factors, we find evidence of the tension between these two program goals insofar as States with policies which encourage participation have higher differences in the food insecurity rates of participants and non-participants. Ó 2008 Elsevier Ltd. All rights reserved. Introduction Food assistance programs in countries across the world are de- signed to help address poverty-related nutrition challenges facing those most in need. These programs and their potential benefits will lead policymakers and program administrators to encourage participation. As an example, consider the case of the largest food assistance program in the United States, the Food Stamp Program. This program is ‘‘...the most critical component of the safety net against hunger because it provides basic protection for citizens of all ages and household status (US Department of Agriculture, Food and Nutrition Service, 1999, p. 7).” To be this critical component, the Food Stamp Program must reach households in need of assis- tance and, at the same time, provide these households with an effective means to reduce the probability of hunger. For the former goal, researchers have examined how successful the program is at reaching the intended recipients, especially the most needy of po- tential recipients (see, e.g., Cunnyngham, 2004). For the latter goal, researchers have examined the effect of food stamps on food inse- curity (i.e., the restricted access to enough food for active, healthy lives because of financial constraints) and hunger in the US (Gundersen and Oliveira, 2001; Borjas, 2004; Wilde and Nord, 2005; Gundersen and Kreider, 2008). The existence of the dual goals of improving well-being and encouraging participation can create a challenge for those examin- ing the success of a food assistance program. Consider again the case of food stamps and food insecurity in the United States. To gauge the effectiveness of food stamps in reducing food insecurity, one can compare the food insecurity status of recipients with eligi- ble non-recipients. In almost all comparisons of eligible house- holds, food stamp recipients have higher rates of food insecurity than eligible non-recipients. In 2006, for example, 52% of food stamp recipients were food insecure at some point in the previous year, while 25% of eligible non-recipients were food insecure (Nord et al., 2007, Table 11). This is not surprising insofar as that the households most at risk of being food insecure are also most likely to enter the Food Stamp Program. Given the goals of the Food Stamp Program, program administrators and policymakers are motivated to take actions to reduce this gap in food insecurity rates between participants and eligible non-participants. This motiva- tion is further encouraged by the closely followed reports of State-level food insecurity rates (e.g., Cook, 2007). At the same time, program administrators and policymakers are motivated to increase participation rates among eligible households. The success of States in achieving high participation rates is monitored by the annual compilation of participation rates by State (e.g., Castner 0306-9192/$ - see front matter Ó 2008 Elsevier Ltd. All rights reserved. doi:10.1016/j.foodpol.2008.12.001 * Corresponding author. Tel.: +1 217 333 2857. E-mail addresses: cggunder@illinois.edu (C. Gundersen), jolliffe@ers.usda.gov (D. Jolliffe), ltiehen@ers.usda.gov (L. Tiehen). Food Policy 34 (2009) 367–376 Contents lists available at ScienceDirect Food Policy journal homepage: www.elsevier.com/locate/foodpol