The rating of pre-clerkship examination questions by postgraduate medical students: an assessment of quality and relevancy to medical practice June A Harris, Helen C Heneghan & Donald W McKay Objective To determine whether postgraduate students are able to assess the quality of undergraduate medical examinations and to establish whether faculty can use their results to troubleshoot the curriculum in terms of its content and evaluation. Subjects First and second year family medicine post- graduate students. Materials A randomly generated sample of undergra- duate medical examination questions. Methods Postgraduate students were given two under- graduate examinations which included questions with an item difficulty (ID) > 0®60. The students answered and then rated each question on a scale of 1–7. Results The percentage of postgraduate students answering each question correctly correlated signi- ficantly with the average perceived relevance (Examination 1: r ¼ 0®372; P <0®05; Examination 2: r ¼ 0®458; P <0®05). Questions plotted for average postgraduate ⁄ undergraduate performance ratio versus the average perceived relevance were significantly correlated (Examination 1: r ¼ 0®462; P <0®01; Examination 2: r ¼ 0®458; P <0®05). Conclusions This study offers a method of validating question appropriateness prior to examination admin- istration. The design has the potential to be used as a model for determining the relevancy of a medical curriculum. Keywords education, medical, undergraduate ⁄ *standards; *educational measurement; curriculum; clinical clerkship ⁄ *standards. Medical Education 2003;37:105–109 Introduction Several studies indicate that knowledge is not retained unless it is reinforced by use. Knowledge which is not reinforced is soon lost and evidence shows that most longterm loss occurs shortly after final examinations in the subject. Results of several studies have shown losses of 18)20% of knowledge after 6–21 months. 1–5 Retention after 2 years was not significantly better than could have been obtained by chance. 6 Postgraduate medical students in their pre-clerkship years (first 2 years) have a certain level of knowledge, which is presumably reinforced during their clerkship years (second 2 years) and during their postgraduate training. Presumably, further learning and clinical experience later in the curriculum reinforces relevant information presented early in medical school. It is likely that any knowledge that is not applicable to the profession is lost rather quickly. In order for medical students to make competent clinical decisions based on sound scientific principles, they must retain knowledge from their early undergra- duate medical training. To prevent the extensive loss of this knowledge, the information must be relevant. However, students in the early stages of their under- graduate medical education are not in a position to judge the relevance of material presented to them by faculty. In this study, postgraduate medical students were asked to judge relevancy. Dawson-Saunders et al. 7 surveyed basic science and clinical faculty in order to identify the basic biomedical concepts important to the practice of medicine that medical students should learn, understand and apply. Few differences were found between basic science and clinical faculty members’ identification of the concepts and the importance accorded to them. Faculty of Medicine, Memorial University of Newfoundland, Canada Correspondence: Dr June A. Harris, Faculty of Medicine, Memorial University of Newfoundland, St. John’s, Newfoundland AlB 3V6, Canada. Tel.: 00 1 709 777 6747; Fax: 00 1 709 777 8379; E-mail: juneh@mun.ca Undergraduate education Ó Blackwell Publishing Ltd MEDICAL EDUCATION 2003;37:105–109 105