Using a diary to quantify learning activities Tim J Wilkinson 1 , J Elisabeth Wells 1 & John A Bushnell 2 BACKGROUND Diaries of actual learning activities can fill the gap between the planned curriculum and students’ opinions and outcomes. We report the development and validity of such a method, estimate sources of variation and model sampling strategies to determine efficient ways to obtain information about a curriculum or about individual students. METHODS Following development and piloting, the diary was administered to fourth- and fifth-year medical students. Each student was asked to complete a diary on 3 randomly selected days of the academic year. Sources of variance and generalisability were determined using variance components analysis. Validity was explored by comparing activities with what is known about the curriculum, assessment, timetables and the 2 classes of students. RESULTS Response rate was 83% (287 ⁄ 345). Learn- ing activities varied as expected with timing of assessments, and on weekdays compared with week- ends. For most activities, 14 days per student would be needed to obtain generalisable information about an individual student. The variation between days is greater than the variation between students, meaning that sampling for information on a curriculum should include all students and all days of the year but the number of diaries per student could be kept low depending on the desired power to detect any differences. CONCLUSION Such an evaluation method is feas- ible and can provide reliable and valid information about study activities. Reasons for good compliance are discussed. Sampling strategies should be tailored to the purpose of the study. KEYWORDS education, medical, undergraduate ⁄ *methods; educational measurement; curriculum; learning; attitude of health personnel. Medical Education 2005; 39: 657–664 doi:10.1111/j.1365-2929.2005.02196.x INTRODUCTION The curriculum that is planned and put into action is not always the same as the curriculum the students experience. Curricula are evaluated com- monly by the documented planned curriculum, the students’ opinions of the curriculum and, through assessments, some of the outcomes. However, there can be discrepancies between the outcomes at the end of a curriculum and what was actually intended at the beginning. 1 The curriculum of students’ experience is sometimes referred to as the Ôhidden curriculumÕ 2 and can be much more powerful in determining what students actually do than many teachers or curriculum planners imagine. 1 Having some way to quantify what the students are actually doing during the curriculum might shed light on this gap. Within the context of changes to our assessments and to our curriculum, we decided to develop a diary that students would complete to quantify what learning activities each student may or may not be doing at any particular time. For example, how much time do they really spend preparing for examinations? In their time for independent learning, how much is spent doing this rather than relaxing? How much time do they spend seeing patients? Quantifying the educa- tional impact of changes in a curriculum is a measure that should be incorporated alongside more tradi- tional quality measures such as student assessment. 3 undergraduate learning 1 Christchurch School of Medicine and Health Sciences, University of Otago, New Zealand 2 Wellington School of Medicine and Health Sciences, University of Otago, New Zealand Correspondence: Assoc Prof T J Wilkinson, Christchurch School of Medicine and Health Sciences, C ⁄ – The Princess Margaret Hospital, PO Box 800, Christchurch, New Zealand. Tel: (64) 33377899; Fax: (64) 33377975; E-mail: tim.wilkinson@chmeds.ac.nz Ó Blackwell Publishing Ltd 2005. MEDICAL EDUCATION 2005; 39: 657–664 657