ARTICLES Interobserver Agreement, Intraobserver Reliability, and the Rorschach Comprehensive System Marvin W. Acklin, Claude J. McDowell II, Mark S. Verschell, and Darryl Chan Honolulu, Hawaii Interrater agreement and reliability for the Rorschach have recently come under in- creasing scrutiny. This is the second report examining methods of Comprehensive System reliability using principles derived from observational methodology and ap- plied behavioral analysis. This study examined a previous nonpatient sample of 20 protocols (N = 412 responses) and also examined a new clinical sample of 20 proto- cols (N = 374 responses) diagnosed with Research Diagnostic Criteria. Reliability was analyzed at multiple levels of Comprehensive System data, including response- level individual codes and coding decisions and ratios, percentages, and derivations from the Structural Summary. With a number of exceptions, most Comprehensive System codes, coding decisions, and summary scores yield acceptable, and in many instances excellent, levels of reliability. Limitations arising from the nature of Ror- schach data and Comprehensive System coding criteria are discussed. In this article, we report on the latest in a series of studies undertaken to examine and improve the performance of the Rorschach Inkblot Test as a clinical and re- search instrument (Acklin, McDowell, & Ornduff, 1992; McDowell & Acklin, 1996). The Rorschach relies on the application of a complex coding system to sam- ples of verbal behavior to yield data for clinical and research purposes. As such, the Rorschach is less a test than a behavioral observation methodology (McDowell & Acklin, 1996). The Comprehensive System was developed by Exner (1990, 1991, 1993, 1995; Exner & Weiner, 1994) to standardize Rorschach observation (i.e., ad- ministration) and coding procedures, and it remains the focus of ongoing investiga- tion and refinement. JOURNAL OF PERSONALITY ASSESSMENT, 74(1), 15–47 Copyright © 2000, Lawrence Erlbaum Associates, Inc.