Shef®eld Assessment Instrument for Letters (SAIL): performance assessment using outpatient letters J G M Crossley, 1 A Howe, 2 D Newble, 3 B Jolly 3 & H A Davies 1 Context Well-designed assessments of performance are urgently required for training doctors, and to provide indicators of the quality of practice. Objectives To design an assessment process that uses routine outpatient letters, and to evaluate its validity, feasibility, reliability (reproducibility and discrimin- ation) and potential educational impact. Participants All 26 paediatric registrars in North Trent attending annual assessment panel in 1999 participated. Study design An assessment instrument (SAIL) was developed from a consensus framework for good prac- tice in written communication. It comprises an 18-point checklist and a global rating scale. Three judges applied the instrument to 260 letters from the routine clinical practice of the 26 participants. Results We achieved consensus on good practice in written communication. This was in keeping with the published literature. All participants completed the assessment. Scoring took 3±6 min per judge per letter. The reliability coef®cient in this test situation is 0á72. Modelling predicts that a coef®cient of 0á8 (the threshold for high-stakes judgements about perform- ance) can be achieved with more judges or letters. The assessment results are well suited to formative feed- back. Conclusions SAIL uses letters as a face valid indicator of written communication performance. The instrument is feasible to use, and produces reliable results when applied to paediatric registrars to inform the annual Record of In-Training Assessment (RITA). Feedback from the assessment should help doctors to improve their written communication. Its use may extend to other specialities and settings including revalidation. Keywords Communication; family practice/*education; Great Britain; pilot study; *professional competence; reproducibility of results; writing. Medical Education 2001;35:1115±1124 Introduction The importance of assessment The assessment of medical practice has assumed a new level of importance for two reasons. First, progress in our understanding of learning has highlighted the pos- itive role of assessment in training. Its content and style strongly in¯uence training since students and trainees modify their approach to learning in order to pass. 1,2 In addition, feedback from the results of assessment exerts a powerful educational in¯uence both in general set- tings, 3 and in medical education. 4,5 Secondly, publicised failures in medical performance and conduct have reduced public con®dence in the medical profession and in professional self-regulation. 6 Robust regulation is needed. The General Medical Council plans to introduce consultant revalidation in 2003±2004 accompanied by rigorous procedures for the investigation of poor performance. (http:// www.gmc-uk.org). These are in addition to the clinical governance initiative, 7 and the annual assessment of specialist registrars (specialist postgraduate trainees), 8 which are already in place. All of these strategies hinge on good methods of evaluating clinical performance. Regulators cannot ethically justify making high- stakes judgements about professional performance based on invalid or unreliable assessments. Failure to evaluate or achieve either of these measurement char- acteristics exposes patients to the risk of continuing poor performance, and exposes doctors to the risk of 1 Department of Paediatrics, Shef®eld Children's Hospital NHS Trust, UK 2 Institute of General Practice and Primary Care, University of Shef- ®eld, UK 3 Department of Medical Education, University of Shef®eld, UK Correspondence: Helena Davies, Consultant in Medical Education/Late Effects, Shef®eld Children's Hospital NHS Trust, Western Bank, Shef®eld, S10 2TH, UK. Tel.: 0114 271 7108; Fax: 0114 271 7108; E-mail: h.davies@shef®eld.ac.uk Teaching methods Ó Blackwell Science Ltd MEDICAL EDUCATION 2001;35:1115±1124 1115