Predictive validity of the multiple mini-interview for selecting medical trainees Kevin W Eva, 1 Harold I Reiter, 2 Kien Trinh, 3 Parveen Wasi, 4 Jack Rosenfeld 5 & Geoffrey R Norman 1 INTRODUCTION In this paper we report on further tests of the validity of the multiple mini- interview (MMI) selection process, comparing MMI scores with those achieved on a national high-stakes clinical skills examination. We also continue to explore the stability of candidate performance and the extent to which so-called ‘cognitive’ and ‘non-cognitive’ qualities should be deemed independent of one another. METHODS To examine predictive validity, MMI data were matched with licensing exami- nation data for both undergraduate (n = 34) and postgraduate (n = 22) samples of partici- pants. To assess the stability of candidate performance, reliability coefficients were generated for eight distinct samples. Finally, correlations were calculated between ‘cognitive’ and ‘non-cognitive’ measures of ability col- lected in the admissions procedure, on gradu- ation from medical school and 18 months into postgraduate training. RESULTS The median reliability of eight administrations of the MMI in various cohorts was 0.73 when 12 10-minute stations were used with one examiner per station. The correlation between performance on the MMI and number of stations passed on an objective structured clinical examination-based licensing examina- tion was r = 0.43 (P < 0.05) in a postgraduate sample and r = 0.35 (P < 0.05) in an under- graduate sample of subjects who sat the MMI 5 years prior to sitting the licensing examina- tion. The correlation between ‘cognitive’ and ‘non-cognitive’ assessment instruments increased with time in training (i.e. as the focus of the assessments became more tailored to the clinical practice of medicine). DISCUSSION Further evidence for the validity of the MMI approach to making admissions decisions has been provided. More generally, the reported findings cast further doubt on the extent to which performance can be captured with trait-based models of ability. Finally, although a complementary predictive relation- ship has consistently been observed between grade point average and MMI results, the extent to which cognitive and non-cognitive qualities are distinct appears to depend on the scope of practice within which the two classes of qualities are assessed. admissions Medical Education 2009: 43: 767–775 doi:10.1111/j.1365-2923.2009.03407.x 1 Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada 2 Department of Oncology, McMaster University, Hamilton, Ontario, Canada 3 Department of Family Medicine, McMaster University, Hamilton, Ontario, Canada 4 Department of Medicine, McMaster University, Hamilton, Ontario, Canada 5 Department of Pathology and Molecular Medicine, McMaster University, Hamilton, Ontario, Canada Correspondence: Kevin W Eva, Department of Clinical Epidemiology and Biostatistics, Program for Educational Research and Development, MDCL 3522, McMaster University, Hamilton, Ontario L8N 3Z5, Canada. Tel: 00 1 905 525 9140 (ext 27241); Fax: 00 1 905 572 7099; E-mail: evakw@mcmaster.ca ª Blackwell Publishing Ltd 2009. MEDICAL EDUCATION 2009; 43: 767–775 767