10.1177/0163278702250082 ARTICLE Evaluation & the Health Professions / March 2003 Violato et al. / EXPERT JUDGMENT AND CLUSTER ANALYSIS This study compares an expert judgment process—minimal performance levels (MPL) using the Nedelsky and Ebel procedures—for setting cutoff scores for pass/fail on licensure examinations with an empirical approach— cluster analysis. Data from all three compo- nents of the Canadian Standard Assessment in Optometry (CSAO) examinations (knowl- edge, clinical judgment, and clinical skills) from 243 candidates were obtained. Results indicate that for the written components of the exams employing the Nedelsky method of MPL setting, there was a mean agreement of pass/fail of 81% with the cluster analysis approach on pass/fail categorization. For the performance exams using the Ebel method, the mean agreement of pass/fail with the clus- ter analysis was 93%. Thus the subjective approaches to setting cutoff scores (i.e., expert judgment methods) converge with the objec- tive method (i.e., cluster analysis) of classify- ing test takers in the same categories. Keywords: high-stakes examinations; criterion- referenced testing; minimum per- formance levels; cluster analysis A VALIDITY STUDY OF EXPERT JUDGMENT PROCEDURES FOR SETTING CUTOFF SCORES ON HIGH-STAKES CREDENTIALING EXAMINATIONS USING CLUSTER ANALYSIS CLAUDIO VIOLATO University of Calgary and Edumetrics Ltd. ANTHONY MARINI University of Calgary and Martek CURTIS LEE Royal College of Physicians and Surgeons of Canada 59 EVALUATION & THE HEALTH PROFESSIONS, Vol. 26 No. 1, March 2003 59-72 DOI: 10.1177/0163278702250082 © 2003 Sage Publications