Musculoskeletal Ultrasound Objective Structured Clinical Examination: An Assessment of the Test EUGENE Y. KISSIN, 1 PETER C. GRAYSON, 2 AMY C. CANNELLA, 3 PAUL J. DEMARCO, 4 AMY EVANGELISTO, 5 JANAK GOYAL, 6 RANY AL HAJ, 7 JAY HIGGS, 8 DANIEL G. MALONE, 9 MIDORI J. NISHIO, 10 DARREN TABECHIAN, 11 AND GURJIT S. KAELEY 12 Objective. To determine the reliability and validity of an objective structured clinical examination (OSCE) for muscu- loskeletal ultrasound (MSUS). Methods. A 9-station OSCE was administered to 35 rheumatology fellows trained in MSUS and to 3 expert faculty (controls). Participants were unaware of joint health (5 diseased/4 healthy). Faculty assessors (n  9) graded image quality with predefined checklists and a 0 –5 global rating, blinded to who performed the study. Interrater reliability, correlation between a written multiple choice question examination (MCQ) and OSCE performance, and comparison of fellow OSCE results with those of the faculty were measured to determine OSCE reliability, concurrent validity, and construct validity. Results. Assessors’ interrater reliability was good (intraclass correlation coefficient [ICC] 0.7). Score reliability was good in the normal wrist and ankle stations (ICC 0.7) and moderate in the abnormal wrist and ankle stations (ICC 0.4). MCQ grades significantly correlated with OSCE grades (r  0.52, P < 0.01). The fellows in the bottom quartile of the MCQ scored 3.07 on the OSCE, significantly worse than the top quartile fellows (3.32) and the faculty (3.29; P < 0.01). Scores also significantly discriminated bottom quartile fellows from faculty in the normal wrist and ankle stations (3.38 versus 3.78; P < 0.01), but not in the abnormal stations (3.37 versus 3.49; P  0.08). Conclusion. MSUS OSCE is a reliable and valid method for evaluation of MSUS skill. Normal joint assessment stations are more reliable than abnormal joint assessment stations and better discriminate poorly performing fellows from faculty. Therefore, MSUS OSCE with normal joints can be used for the assessment of MSUS skill competency. INTRODUCTION Utilization of musculoskeletal ultrasound (MSUS) has ex- panded greatly since its first use in 1958 (1). In addition to radiology, many specialties now employ MSUS for point- of-care imaging, including rheumatology, physiatry, podi- atry, emergency medicine, general internal medicine, and family practice, and this has led to a 316% increase in MSUS volume from 2000 to 2009 (2). The proliferation of MSUS has elicited questions about the qualifications of physicians performing MSUS examination. In response to these questions, certification examinations have been cre- ated by the American Registry for Diagnostic Medical The view(s) expressed herein are those of the author(s) and do not reflect the official policy or position of Brooke Army Medical Center, the U.S. Army Medical Department, the U.S. Army Office of the Surgeon General, the Depart- ment of the Army, the U.S. Air Force, the Department of Defense, or the U.S. Government. Supported by the Clinician Scholar Educator Award from the Rheumatology Research Foundation. 1 Eugene Y. Kissin, MD: Boston University School of Medicine, Boston, Massachusetts; 2 Peter C. Grayson, MD, MSc: Boston Medical Center, Boston, Massachusetts; 3 Amy C. Cannella, MD: University of Nebraska Medical Center, Omaha; 4 Paul J. DeMarco, MD, FACR: Arthritis and Rheumatism Associates PC, Wheaton, Maryland; 5 Amy Evangelisto, MD: Hospital of the University of Pennsylvania, Philadelphia; 6 Janak Goyal, MD: Raritan Bay Medical Cen- ter, Perth Amboy, New Jersey; 7 Rany al Haj, MD: Shore Arthritis & Rheumatism Associates, Ocean, New Jersey; 8 Jay Higgs, MD: Brooke Army Medical Center, Fort Sam Hous- ton, Texas; 9 Daniel G. Malone, MD, RMSK: Excel Ortho- pedics, Beaver Dam, Wisconsin; 10 Midori J. Nishio, MD: John Muir Health, Walnut Creek, California; 11 Darren Tabechian, MD: University of Rochester, Rochester, New York; 12 Gurjit S. Kaeley, MRCP: University of Florida, Jacksonville. Dr. Kissin receives royalties from Gulfcoast Ultrasound for the wrist ultrasound video. Dr. DeMarco has received consulting fees, speaking fees, and/or honoraria (less than $10,000 each) from Amgen and Auxilium and (more than $10,000) from Abbott/AbbVie. Dr. Malone has received hon- oraria (less than $10,000) from SonoSite. Dr. Nishio has received speaking fees (less than $10,000) from Abbott. Address correspondence to Eugene Y. Kissin, MD, Arthri- tis Center, Boston University School of Medicine, 72 East Concord Street, Evans-506, Boston, MA 02118. E-mail: eukissin@bu.edu. Submitted for publication March 13, 2013; accepted in revised form July 31, 2013. Arthritis Care & Research Vol. 66, No. 1, January 2014, pp 2– 6 DOI 10.1002/acr.22105 © 2014, American College of Rheumatology SPECIAL THEME ARTICLE: CLINICAL IMAGING AND THE RHEUMATIC DISEASES 2