Modeling relationships between traditional preadmission measures and clinical skills performance on a medical licensure examination William L. Roberts • Gina Pugliano • Erik Langenau • John R. Boulet Received: 18 April 2011 / Accepted: 21 August 2011 / Published online: 28 August 2011 Ó Springer Science+Business Media B.V. 2011 Abstract Medical schools employ a variety of preadmission measures to select students most likely to succeed in the program. The Medical College Admission Test (MCAT) and the undergraduate college grade point average (uGPA) are two academic measures typically used to select students in medical school. The assumption that presently used preadmission measures can predict clinical skill performance on a medical licensure examination was evaluated within a validity argument framework (Kane 1992). A hierarchical generalized linear model tested relationships between the log-odds of failing a high-stakes medical licensure performance examination and matriculant academic and non-academic pread- mission measures, controlling for student-and school-variables. Data includes 3,189 ma- triculants from 22 osteopathic medical schools tested in 2009–2010. Unconditional unit- specific model expected average log-odds of failing the examination across medical schools is -3.05 (se = 0.11) or 5%. Student-level estimated coefficients for MCAT Verbal Rea- soning scores (0.03), Physical Sciences scores (0.05), Biological Sciences scores (0.04), uGPA science (0.07), and uGPA non-science (0.26) lacked association with the log-odds of failing the COMLEX-USA Level 2-PE, controlling for all other predictors in the model. Evidence from this study shows that present preadmission measures of academic ability are not related to later clinical skill performance. Given that clinical skill performance is an important part of medical practice, selection measures should be developed to identify students who will be successful in communication and be able to demonstrate the ability to systematically collect a medical history, perform a physical examination, and synthesize this information to diagnose and manage patient conditions. W. L. Roberts (&)  E. Langenau National Board of Osteopathic Medical Examiners, Inc, 101 West Elm Street, Suite 150, Conshohocken, PA 19428-2004, USA e-mail: broberts@nbome.org G. Pugliano UPMC Cancer Centers Breast Program, Magee-Women’s Hospital of UPMC, 300 Halket Street Room I-410, Pittsburgh, PA 15213, USA J. R. Boulet Foundation for Advancement of International Medical Education and Research, 3624 Market Street, Philadelphia, PA 19104, USA 123 Adv in Health Sci Educ (2012) 17:403–417 DOI 10.1007/s10459-011-9321-4