A Graphical Tool for Parsing and Inspecting Surgical Robotic Datasets D´ avid El-Saig * , Ren´ ata Nagyn´ e Elek * and Tam´ as Haidegger *† * Antal Bejczy Center for Intelligent Robotics, University Research, Innovation and Service Center (EKIK) ´ Obuda University, Budapest, Hungary † Austrian Center for Medical Innovation and Technology (ACMIT), Wiener Neustadt, Austria Email: {david.elsaig, renata.elek, haidegger}@irob.uni-obuda.hu Abstract—Skill and practice of surgeons greatly affect the outcome of surgical procedures, thus surgical skill assessment is exceptionally important. In the clinical practice, even today, the standard is peer assessment of the capabilities. In the case of robotic surgery, the observable motion of the laparoscopic tools can hide the expertise level of the operator. JIGSAWS is a database containing kinematic and video data of surgeons training with the da Vinci Surgical System. The JIGSAWS can be an extremely powerful tool in surgical skill assessment research, but due to its data-storage it is not user-friendly. In this paper, we propose a graphical tool for the JIGSAWS, which can ease the usage of this annotated surgical dataset. Index Terms—surgical robotics, surgical skill assessment, JIG- SAWS, annotated surgical database, surgical data science I. I NTRODUCTION Surgical robotics in our days is a widely used technique in the field of Minimally Invasive Surgeries due to its ac- curacy, advanced vision system and ergonomics. With the da Vinci Surgical System (Intuitive Surgical Inc., Sunnyvale, CA)—which is currently the most successful surgical robot— clinicians perform nearly a million interventions annually [1]. While, surgical robotics has advantages for the patient and the surgeon as well, it requires training and extensive skills from the operator. “Human motion is stochastic in nature” [2], and many believe that skills are hidden in the motion, and somehow it can be determined based on the data analysis of a task execution. This task can be a surgical procedure: knot-tying, suturing, dissection, etc. Nowadays there are no standard objective assessment methods of surgical skills in the clinical practice. This would be crucial for quality assurance reasons, and for direct feedback to the clinician. Peer assessment (when an expert scores the clinician during the procedure based on a known metric) is relatively easy to implement, but it requires a senior clinician during the intervention, and it can be subjective [3]. Automated skill assessment techniques are harder to apply, yet they can provide an objective and universal solution to surgical skill estimation. For automated skill assessment, we have to examine annotated surgical data to construct a theory. Surgical robotics provides an exceptional opportunity to study human motion due to the recordable kinematic and video data. Robot-Assisted Minimally Invasive Surgery (RAMIS) data collection can be done with different tools. The da Vinci Research Kit (DVRK) is a hardware and software platform for the da Vinci providing complete read and write access to the robot’s arms [4]. Virtual reality simulators (dVSS, dV-Trainer, RoSS, etc.) can also be platforms for surgical data collection [5]. According to our knowledge, JIGSAWS (JHU–ISI Gesture and Skill Assessment Working Set) is the only publicly available annotated database for RAMIS skill assessment. JIGSAWS contains eight surgeons’ data, who have different levels of expertise and they are rated using a Global Rating Score derived from a modified version of the OSATS system. Kinematic and video data were captured during the execution of three surgical tasks [6] (Fig. 1): a suturing b knot-tying c needle-passing. The manual annotations of the surgical gestures and the expertise levels were also determined. However, JIGSAWS database is a unique tool to work with surgical data, but it is not easy to process the information in it due to the complicated metadata storage. In this paper, we propose a graphical interface tool for parsing and inspecting the JIGSAWS dataset. This software provides a user-friendly environment for processing the JIG- SAWS database. It can separate and save the different types of data, furthermore it visualizes the captured information. It can be downloaded for free from its GitHub page, with detailed instructions on its wiki: https://github.com/DAud-IcI/staej/. Our aim is to use our tool for examine surgical data, and automated skill assessment method development for RAMIS. In the near future we plan to extend our tool to handle not just reading out the information, but writing in and visualize DVRK data as well. Fig. 1. Surgical tasks captured in the JIGSAWS database (a) suturing, b) knot-tying, c) needle-passing) [6] CINTI 2018 • 18th IEEE International Symposium on Computational Intelligence and Informatics • Nov. 21-22, 2018, • Budapest, Hungary 978-1-7281-1117-9/18/$31.00 ©2018 IEEE 000131