Minimally Invasive Therapy. 2014;23:198–205 ORIGINAL ARTICLE Sensor-based surgical activity recognition in unconstrained environments CHRISTIAN MEIßNER 1 , JÜRGEN MEIXENSBERGER 1,2 , ANDREAS PRETSCHNER 3 & THOMAS NEUMUTH 1 1 Innovation Center Computer Assisted Surgery, Universität Leipzig, Faculty of Medicine, Leipzig, Germany, 2 University Hospital Leipzig, Faculty of Medicine, Department of Neurosurgery, Leipzig, Germany, and 3 Leipzig University of Applied Sciences, Faculty of Electrical Engineering and Information Technology, Department of Process Automation and Embedded Systems, Leipzig, Germany Abstract Introduction: Automatic surgical activity recognition in the operating room (OR) is mandatory to enable assistive surgical systems to manage the information presented to the surgical team. Therefore the purpose of our study was to develop and evaluate an activity recognition model. Material and methods: The system was conceived as a hierarchical recognition model which separated the recognition task into activity aspects. The concept used radio frequency identification (RFID) for instrument recognition and accelerometers to infer the performed surgical action. Activity recognition was done by combining intermediate results of the aspect recognition. A basic scheme of signal feature generation, clustering and sequence learning was replicated in all recognition subsystems. Hidden Markov models (HMM) were used to generate probability distributions over aspects and activities. Simulated functional endoscopic sinus surgeries (FESS) were used to evaluate the system. Results and discussion: The system was able to detect surgical activities with an accuracy of 95%. Instrument recognition performed best with 99% accuracy. Action recognition showed lower accuracies with 81% due to the high variability of surgical motions. All stages of the recognition scheme were evaluated. The model allows distinguishing several surgical activities in an unconstrained surgical environment. Future improvements could push activity recognition even further. Key words: Accelerometers, surgical activity recognition, computer assisted surgery, radio frequency identification, sensors, workflow Introduction In the last few years the number of medical devices in the operating room (OR) has increased. More infor- mation and communication technology (ICT) becomes available and is expected to support better, faster, and less cost-intensive surgical interventions in the future OR. Unfortunately not all information pro- vided by ICT systems is useful in every situation. The whole information stream has to be managed by an automatic system which is aware of the current phase and situation of the intervention. Example applications are context-dependent display of pre- and intra- operative images, intervention time prediction, workflow management or automatic surgical decision support. Therefore the system must include autono- mous detection and recognition of surgical activities or steps. The objective of this work is to propose and evaluate a system which renders such a recognition task in a surgical setting. In the literature an atomic surgical step is also known as surgical activity (1) or surgeme (2–4). It represents the most detailed infor- mation about a surgical task. A surgical activity can be decomposed into aspects which describe specific facets of the activity (5). In this paper we use a 5-tuple which incorporates the information about the actor, the used body part, the used surgical instrument, the surgical action and the treated structure. Other works also use Correspondence: C. Meißner, Innovation Center Computer Assisted Surgery (ICCAS), Universität Leipzig, Medical Faculty, Semmelweisstraße 14, D-04103 Leipzig, Germany. E-mail: christian.meissner@iccas.de ISSN 1364-5706 print/ISSN 1365-2931 online Ó 2014 Informa Healthcare DOI: 10.3109/13645706.2013.878363