Identification of High-Risk Residents Patricia C. Bergen, M.D.,* ,1 John H. Littlefield, Ph.D.,† Grant E. O’Keefe, M.D.,* Robert V. Rege, M.D.,* Thomas A. Anthony, M.D.,* Lawrence T. Kim, M.D.,* and Richard H. Turnage, M.D.* *Department of Surgery, University of Texas Southwestern Medical Center, 5323 Harry Hines Boulevard, Dallas, Texas 75390-9156; and †University of Texas Health Science Center at San Antonio, 7703 Floyd Curl Drive, San Antonio, Texas 78229-7895 Submitted for publication on December 7, 1999 Background. Identification of high-risk residents allows remediation and support for administrative action when necessary. This study characterizes dif- ferences in documentation of marginally performing residents in a general surgery residency. Methods. High-risk residents were identified by the former program director. Twenty-four of one hun- dred fifteen residents over a 10-year period had one to four problematic areas: cognitive, synthetic, family/health, and interpersonal skills. Outcomes in- cluded finished (18), voluntary withdrawal (1), and involuntary withdrawal (5). A case– control study matching controls to cases by date of entry into the training program was used. Records were reviewed for demographics, preentry qualifications, American Board of Surgery In-Training Exam (ABSITE) scores, letters of complaint or praise, events of coun- seling, and monthly ratings. The records of 48 resi- dents were reviewed. Ward evaluations were on eight categories with a 5-point Leikert scale (3— unacceptable to 7— outstanding). The evaluation score assigns points only to low ratings. High scores represent progressively poorer performance. A Wil- coxon signed ranks test was used to compare the cases and controls for continuous variables. The Mc- Nemar test was used in comparisons of categorical data with binary outcomes. Exact P values are re- ported. Results. Objective data were similar for both groups. Study residents tended to score higher on monthly evaluations at Year 2 and by Year 3 this achieved significance (0.026). Study residents were more likely to have negative faculty letters (0.016) and events of counseling by a faculty member (0.017) and the program director (0.005). Conclusions. Identification of residents at risk should begin as early as possible during training. A combination of faculty evaluations and evidence of letters of counseling can detect high-risk residents. Programs may use such indicators to support deci- sions regarding remedial work or administrative ac- tion. © 2000 Academic Press Key Words: surgical education; evaluation; perfor- mance. INTRODUCTION The identification of high-risk residents early enough in training to provide remediation or support administrative decision making is very difficult. The residency program and the individual resident make substantial investment in the satisfactory completion of the training process. Programs have a responsibility to assure the community that competent physicians are entering the practice. Resident performance ap- praisal is the process most training programs use to evaluate the progress of its trainees. These appraisals provide information for annual promotion, eligibility for board qualification, hospital privileging, etc. The evaluation form often serves as the cornerstone of res- ident feedback. But information comes from other sources, informal and formal. Yet, when programs are faced with underperforming residents who are per- forming poorly, it is typical for the formal rating form to lack documentation supporting the claim. Often res- idents are identified through informal means late in the training program. The formal rating record fails to provide critical information regarding the character of Presented at the Annual Meeting of the Association for Academic Surgery, Philadelphia, Pennsylvania, November 18 –20, 1999. 1 To whom correspondence should be addressed. Journal of Surgical Research 92, 239 –244 (2000) doi:10.1006/jsre.2000.5924, available online at http://www.idealibrary.com on 239 0022-4804/00 $35.00 Copyright © 2000 by Academic Press All rights of reproduction in any form reserved.