Downloaded from https://journals.lww.com/anesthesia-analgesia by zzT40Ynnliv65HId/JvPVAR8flosc+yAnKBwxlaiHrYbUsrguuRtMSCwZIh68s+hpzfkwWx0TmS/mXt61aQ65JKsBVkGGSU6pkLWux/ZA+lRdIBLDBbMSnLVLcdncMbzswZh7tpxxEZkYIgLfZ8XfQ== on 11/14/2019 Copyright © 2019 International Anesthesia Research Society. Unauthorized reproduction of this article is prohibited. November 2019 • Volume 129 • Number 5 www.anesthesia-analgesia.org e159 DOI: 10.1213/ANE.0000000000004250 GLOSSARY ABA = American Board of Anesthesiology; CA-1 = clinical anesthesia year 1; CI = confdence interval; ITE = In-Training Examination T he American Board of Anesthesiology (ABA; Raleigh, NC) has transitioned to a staged examination system for initial board certifcation. In the former (“tradi- tional”) system, physicians were required to pass a written examination (Part 1) and an oral examination (Part 2) after completing a 4-year anesthesiology residency. In the new “staged” examination system, there are now 2 written examinations: the BASIC Examination (frst offered in 2014), typically taken at the end of clini- cal anesthesia year 1 (CA-1; the second year of residency), and the ADVANCED Examination (frst offered in 2016), taken after fnishing residency. Residents must pass the BASIC Examination before completing residency and then the ADVANCED Examination before having to pass the APPLIED Examination to achieve initial board certifcation. 1 The goal of introducing the BASIC Examination in the staged examination system was for residents to improve knowledge acquisition during training. The ABA offers an annual In-Training Examination (ITE) to assess a resident’s progress toward the mastery of knowledge eventually assessed in the ADVANCED Examination. A prior study suggested that the staged examination system accelerated the improvement of ITE performance during residency. 2 However, the method used to score and equate the ITE in those years did not allow evaluation of whether adding the BASIC Examination had improved resident knowledge once residents fnished training as measured by the written certifcation examination. The ABA determines the passing standard on written certifcation examinations through a standard-setting study performed by an expert group using the Hofstee Method, 3 a combination of criterion-referenced and norm-referenced approaches. Once a standard is set, test forms (the actual set of test questions delivered to candidates) used in subsequent administrations are equated analytically to the test form used to set the standard using test items (questions) common to both forms, referred to as “linking items.” When the Part 1 Examination in the traditional system was replaced by the ADVANCED Examination in the staged system in 2016, link- ing items were not explicitly included as the ADVANCED Examination was a new examination and therefore required its own standard-setting study. Nevertheless, the 2016 and 2017 ADVANCED Examinations did include some items pre- viously used in the Part 1 Examinations, making it possible to compare examination performance before and after 2016. The purpose of this study was to compare performance on the written certifcation examinations taken by anes- thesiology residency graduates before and after the ABA introduced the staged examination system by equating test scores using common test items. METHODS This study was deemed exempt from review by the Mayo Clinic Institutional Review Board (Rochester, MN). Study Population This study included the entering classes of CA-1 residents in 2011–2014 who graduated by September 30 of their expected This study compared anesthesiology residency graduates’ written certifcation examination per- formance before and after the American Board of Anesthesiology (ABA) introduced the staged examination system. After equating test scores using common test items, the frst 2 cohorts (2013, 2014) in the staged system scored 7.1 points and 8.3 points higher than the 2011 baseline cohort in the former examination system. The 2013 and 2014 cohorts’ pass rates (94.2% and 95.9%) were also higher than the 2011 and 2012 cohorts (91.9% and 92.6%) if a common standard had been applied. The staged examination system may be associated with improved knowledge of anesthesiology graduates. (Anesth Analg 2019;129:e159–e162) The American Board of Anesthesiology’s Staged Examination System and Performance on the Written Certification Examination After Residency Yan Zhou, PhD,* Huaping Sun, PhD,* Alex Macario, MD, MBA,† Donald E. Martin, MD,‡ James P . Rathmell, MD,§ and David O. Warner, MD║ See Editorial, p 1197 From the *American Board of Anesthesiology, Raleigh, North Carolina; †Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University, Stanford, California; ‡Department of Anesthesiology and Perioperative Medicine, Penn State College of Medicine, Hershey, Pennsylvania; §Department of Anesthesiology, Perioperative and Pain Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts; and ║Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, Minnesota. Accepted for publication April 24, 2019. Funding: Institutional and/or departmental. Conficts of Interest: See Disclosures at the end of the article. Listen to this Article of the Month podcast and more from OpenAnesthesia.org® by visiting http://journals.lww.com/anesthesia-analgesia/pages/default.aspx. Reprints will not be available from the authors. Address correspondence to Huaping Sun, PhD, The American Board of Anesthesiology, 4208 Six Forks Rd, Suite 1500, Raleigh, NC 27609. Address e-mail to huaping.sun@theaba.org. Copyright © 2019 International Anesthesia Research Society E BRIEF REPORT