Objective assessment of surgical competence in gynaecological laparoscopy: development and validation of a procedure-specific rating scale CR Larsen, a T Grantcharov, b L Schouenborg, a C Ottosen, a JL Soerensen, a B Ottesen a a Department of Gynaecology and Obstetrics, The Juliane Marie Centre (for Children, Women and Reproduction), Copenhagen University Hospital Rigshospitalet, Copenhagen, Denmark b Department of Surgery, University of Toronto, St Michael’s Hospital, Toronto, Ontario, Canada Correspondence: Dr CR Larsen, Department of Gynaecology, The Juliane Marie Centre (for Children, Women and Reproduction), Copenhagen University Hospital, Rigshospitalet, Blegdamsvej 3, DK-2100 Ø, Copenhagen, Denmark. Email crl@rh.regionh.dk Accepted 27 February 2008. Objective The purpose of this study was to develop a global- and a procedure-specific rating scale based on a well-validated generic model (objective structured assessment of technical skills) for assessment of technical skills in laparoscopic gynaecology. Furthermore, we aimed to investigate the construct validity and the interrater agreement (IRA) of the rating scale. We investigated both the gamma coefficient (Kendall’s rank correlation), which is a measure of the strength of dependence between observations, and the kappa value for each of the ten individual items included in the rating scale. Design Prospective cohort, observer-blinded study. Setting Departments of Obstetrics and Gynaecology in Zealand, Denmark. Population Twenty one gynaecologists or gynaecological trainees. Material and methods Twenty-one video recordings of right side laparoscopic salpingectomies were collected prospectively, eight from novices (defined as <10 procedures), seven from intermediate experienced (20–50 procedures) and six from experts (>200 procedures). All operations were performed by the same operative principles and using a standardised technique. The recordings were analysed by two independent, blinded observers. Main outcome measures Construct validity of the rating scale based on operative performance (median of total score) and interrater reliability. Results There were significant differences between the three groups: median score of novices 24.00 versus intermediate 29.50 versus expert 39.50, P < 0.003) The IRA was 0.83 overall. The gamma correlation coefficient was 0.91. The kappa values varied from 0.510–0.933 for each of the individual items of the rating scale. Conclusions The procedure-specific rating scale for laparoscopic salpingectomy is a valid and reliable tool for assessment of technical skills in gynaecological laparoscopy. Keywords Assessment, construct validity, gynaecology, interrater reliability, laparoscopy, salpingectomy. Please cite this paper as: Larsen C, Grantcharov T, Schouenborg L, Ottosen C, Soerensen J, Ottesen B. Objective assessment of surgical competence in gynaecological laparoscopy: development and validation of a procedure-specific rating scale. BJOG 2008;115:908–916. Introduction Laparoscopy, training and assessment An increasing number of gynaecological surgical procedures are presently performed by laparoscopic technique. This leads to an increasing demand for evidence- and proficiency-based education, training and assessment of laparoscopic skills. Tra- ditionally, education has been based on the apprenticeship model leaving both training and assessment of knowledge as well as technical skills subjective and unstructured. 1 So far, little has been carried out to develop and integrate structured basic training in laparoscopy in the surgical curriculum, and no validated structured system for objective assessment of technical surgical skills in gynaecological laparoscopy is avail- able. Consequently, we need a feasible, structured and objec- tive system for assessment of both technical and procedural skills. In Denmark, 94% of all surgically treated ectopic pregnan- cies are managed by laparoscopy. 2 The laparoscopic salpin- gectomy was in the present study chosen as the operative procedure for developing a method for objective structured assessment of technical surgical skills based on human oper- ations. Laparoscopic salpingectomy is a basic laparoscopic procedure possessing the necessary complexity for skills 908 ª 2008 The Authors Journal compilation ª RCOG 2008 BJOG An International Journal of Obstetrics and Gynaecology DOI: 10.1111/j.1471-0528.2008.01732.x www.blackwellpublishing.com/bjog Gynaecological surgery