Assessing resident’s knowledge and communication skills using four different evaluation tools Jim Nuovo, Klea D Bertakis & Rahman Azari PURPOSE This study assesses the relationship between 4 Accreditation Council for Graduate Medical Education (ACGME) outcome project measures for interpersonal and communication skills and medical knowledge; specifically, monthly performance evaluations, objective structured clin- ical examinations (OSCEs), the American Board of Family Practice in-training examination (ABFP– ITE) and the Davis observation code (DOC) prac- tice style profiles. METHODS Based on previous work, we have DOC scoring for 29 residents from the University of California, Davis Department of Family and Com- munity Medicine. For all these residents we also had the results of monthly performance evalua- tions, 2 required OSCE exercises, and the results of 3 American Board of Family Medicine (ABFM) ITEs. Data for each of these measures were abstracted for each resident. The Pearson correla- tion coefficient was used to assess the presence or lack of correlation between each of these evaluation methods. RESULTS There is little correlation between various evaluation methods used to assess medical know- ledge, and there is also little correlation between various evaluation methods used to assess communi- cation skills. CONCLUSION The outcome project remains a Ôwork in progressÕ, with the need for larger studies to assess the value of different assessment measures of resident competence. It is unlikely that DOC will become a useful evaluation tool. KEYWORDS clinical competence ⁄ *standards; *communication; internship and residency ⁄ *stand- ards; physician–patient relations; United States Medical Education 2006; 40: 630–636 doi:10.1111/j.1365-2929.2006.02506.x INTRODUCTION The Accreditation Council for Graduate Medical Education (ACGME) has recently implemented the outcome project. 1 The goal of the project is to emphasise educational outcomes in the assessment of a resident’s performance. Assessments are centred around 6 general competencies (patient care, med- ical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism and systems-based practice). The outcome project has been described as a Ôwork in progressÕ that will lead to the identification and development of Ôacceptable toolsÕ. The outcome project’s website includes a ÔTable of Toolbox Meth- odsÕ, which provides examples of potentially useful assessment methods. Examples of these tools are: the Davis observation code (DOC), objective structured clinical examination (OSCE), 360-degree evaluation instrument, chart stimulated recall oral examination, standardised patient examination and an in-training examination (ITE). 2 Of these tools, one of the most commonly used is the OSCE. In an OSCE, standard- ised patients are trained to portray a scripted patient presentation. Assessment of a resident’s performance is carried out with a pre-determined checklist, often completed by the faculty observer and by the ÔpatientÕ. OSCEs are often used to develop formative assessments of resident’s communication and inter- personal skills. Some of the suggested tools have not been used previously for a performance assessment; an communication Department of Family and Community Medicine, University of California, Davis, Sacramento, CA, USA Correspondence: Jim Nuovo, Department of Family and Community Medicine, University of California, Davis, 4860 Y Street, Suite 2300, Sacramento, CA 95817, USA. Tel: +916 734 3248; Fax: +916 734 5461; E-mail: james.nuovo@ucdmc.ucdavis.edu Ó Blackwell Publishing Ltd 2006. MEDICAL EDUCATION 2006; 40: 630–636 630