Statistical Modeling and Recognition of Surgical Workflow Nicolas Padoy, Tobias Blum, Seyed-Ahmad Ahmadi, Hubertus Feussner, Marie-Odile Berger, and Nassir Navab Abstract In this paper, we contribute to the development of context-aware operating rooms by introducing a novel approach to modeling and monitoring the workflow of surgical interventions. We first propose a new representation of interventions in terms of multidimensional time-series formed by synchronized signals acquired over time. We then introduce methods based on Dynamic Time Warping and Hidden Markov Models to analyze and process this data. This results in work- flow models combining low-level signals with high-level information such as predefined phases, which can be used to detect actions and trigger an event. Two methods are presented to train these models, using either fully or partially labeled training surgeries. Results are given based on tool usage recordings from sixteen laparoscopic cholecystectomies performed by several surgeons. Key words: surgical workflow, context aware operating room, surgical assistance system, Hidden Markov Model, cholecystectomy 1. Introduction The operating room (OR) needs to be constantly adapted to the introduction of new technolo- gies and surgical procedures. A key element within this process is the analysis of the workflow inside the OR[7, 16]. It has impact on patient safety, working conditions of surgical staff and even overall throughput of the hospital. While new technologies add much complexity to the daily routine of OR staff, they also facilitate the design of assistance systems that can relieve the surgical staff from performing simple but time-consuming tasks and assist them in the tedious ones. In this paper, we focus on the design of a context-aware system that is able to recognize the surgical phase performed by the surgeon at each moment of the surgery. We believe that robust real-time action and workflow recognition systems will be a core component of the Operating Room of the Future. They will enable various applications, ranging from automation of simple tasks to detecting failures, suggesting modifications, documenting procedures and producing final reports. Our contribution is threefold: In the introduction we identify the generic need for an automatic recognition system and introduce a signal based modeling of the surgical actions to achieve such automation. We then propose two statistical models constructed from generic signals from the OR: the annotated average surgery and the annotated Hidden Markov Model, for off-line and on-line recognition of the surgical phases in a standard endoscopic surgery. The models are built based on a set of training surgeries where the phases have been labeled. We also propose a method only requiring partially labeled data. We finally demonstrate and evaluate the methods Preprint submitted to MIA - Special Issue on CAI March 15, 2011