Validating a Sampling Revolution: Benchmarking Address Lists against Traditional Listing Colm O’Muircheartaigh 1 , Ned English 1 , Stephanie Eckman 1 , Heidi Upchurch 1 , Erika Garcia 1 , and James Lepkowski 2 National Opinion Research Center, 55. E Monroe St, Chicago, IL, 60603 1 University of Michigan, Ann Arbor, MI 2 Abstract The paper presents empirical findings from a comparison of two approaches for national area probability sampling. The National Opinion Research Center (NORC) at the University of Chicago and the Institute for Social Research (ISR) at the University of Michigan are collaborating to compare two national area-probability sampling frames for household surveys: (i) the frame produced by traditional listing, using survey field staff, and (ii) the list of postal addresses compiled by the United States Postal Service (USPS). We conducted this comparison in an ongoing survey operation which combines the current wave of the Health and Retirement Survey (HRS) with the first wave of the National Social Life, Health, and Aging Project (NSHAP). Since 2000, survey samplers have been exploring the potential of the United States Postal Service (USPS) address lists to serve as a sampling frame for probability samples from the general population. Though the early work has demonstrated the strengths of the USPS address lists, there has not been a comparison of the two methods on a national scale. We report the relative coverage properties of the two frames, as well as predictors of the coverage and performance of the USPS frame. The research provides insight into the coverage and cost/benefit trade-offs that researchers can expect from traditionally listed frames and USPS address databases. The results in this paper are not weighted to represent the population of the US as a whole. Keywords: USPS List Frames, GIS, Area Probability Sampling 1. Introduction While it has been generally assumed that field listing is the gold standard for generating area-probability samples in terms of coverage and accuracy, until now there has been no national evaluation of the process. NORC has, since 2001, been carrying out an examination of the alternative approach of using the United States Postal Service (USPS) list as a basis for frame construction for area probability surveys. In 2004, NORC and ISR embarked on a national benchmark comparison, whose goal was to provide a quantitative analysis describing the benefits and drawbacks of traditional listing (the “gold standard”) vs. the USPS list (O’Muircheartaigh et al. 2005). In our earlier report on this research, we have compared a traditionally-listed list frame to a USPS- based frame in the same set of areas (O’Muircheartaigh et al. 2005). When discrepancies arose between the two frames, however, it was not possible to determine the source of the error, or which frame, was more accurate. In order to address this question, we conducted additional field work in a set of areas that covered the major dimensions of variation identified earlier. This “best” frame provides a basis for determining the performance of the two approaches relative to this best frame, and to distinguish the circumstances under which each approach may be preferable. 2. Background and Problem The research was undertaken as a methodological supplement to the National Social Life and Health in Aging Project (NSHAP) using field listing and screening for the Health and retirement Survey (HRS), both NIH/NIA projects. 1 ISR at the University of Michigan conducted a listing in 2004 of 549 national segments for the HRS survey. NORC and ISR collaborated on the below research as a methodological supplement to evaluate the overall quality of the ISR listing. NORC then used the results of ISR screening as the sampling frame for the NSHAP survey. The two components of the comparison are as follows. ISR provided the whole frame of listings (and not just the sampled households) for this project. NORC licenses the national USPS delivery point database from ADVO, which contains a record for every 1 This research was supported by National Institute on Aging (NIA) grant R01-AG021487-02S1; L Waite, Principal Investigator. AAPOR - ASA Section on Survey Research Methods 4189