Measuring intra-operative decision-making during laparoscopic cholecystectomy: validity evidence for a novel interactive Web-based assessment tool Amin Madani 1,2 • Yusuke Watanabe 3 • Elif Bilgic 2 • Philip H. Pucher 4 • Melina C. Vassiliou 1,2 • Rajesh Aggarwal 1,2,5 • Gerald M. Fried 1,2,5 • Elliot J. Mitmaker 1 • Liane S. Feldman 1,2 Received: 22 December 2015 / Accepted: 5 July 2016 Ó Springer Science+Business Media New York 2016 Abstract Background Errors in judgment during laparoscopic cholecystectomy can lead to bile duct injuries and other complications. Despite correlations between outcomes, expertise and advanced cognitive skills, current methods to evaluate these skills remain subjective, rater- and situation- dependent and non-systematic. The purpose of this study was to develop objective metrics using a Web-based plat- form and to obtain validity evidence for their assessment of decision-making during laparoscopic cholecystectomy. Methods An interactive online learning platform was developed (www.thinklikeasurgeon.com). Trainees and surgeons from six institutions completed a 12-item assessment, developed based on a cognitive task analysis. Five items required subjects to draw their answer on the surgical field, and accuracy scores were calculated based on an algorithm derived from experts’ responses (‘‘visual concordance test’’, VCT). Test–retest reliability, internal consistency, and correlation with self-reported experience, Global Operative Assessment of Laparoscopic Skills (GOALS) score and Objective Performance Rating Scale (OPRS) score were calculated. Questionnaires were administered to evaluate the platform’s usability, feasibil- ity and educational value. Results Thirty-nine subjects (17 surgeons, 22 trainees) participated. There was high test–retest reliability (intra- class correlation coefficient = 0.95; n = 10) and internal consistency (Cronbach’s a = 0.87). The assessment Presented at the SAGES 2016 Annual Meeting, March 16–19, 2016, Boston, Massachusetts. & Amin Madani amin.madani@mail.mcgill.ca Yusuke Watanabe ywatanabe328@gmail.com Elif Bilgic elif.bilgic@mail.mcgill.ca Philip H. Pucher p.pucher@imperial.ac.uk Melina C. Vassiliou melina.vassiliou@mcgill.ca Rajesh Aggarwal rajesh.aggarwal@mcgill.ca Gerald M. Fried gerald.fried@mcgill.ca Elliot J. Mitmaker elliot.mitmaker@mcgill.ca Liane S. Feldman liane.feldman@mcgill.ca 1 Department of Surgery, McGill University, Montreal, Canada 2 Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, Rm D6-257, Montreal, QC H3G 1A4, Canada 3 Department of Gastroenterological Surgery II, Hokkaido University, Sapporo, Japan 4 Department of Surgery and Cancer, Imperial College London, London, UK 5 Faculty of Medicine, Steinberg Centre for Simulation and Interactive Learning, McGill University, Montreal, Canada 123 Surg Endosc DOI 10.1007/s00464-016-5091-7 and Other Interventional Techniques