Landmark-based Feature Tracking for Endoscopic Motion Analysis S. Friedl 1,2 , B. Morgus 1 , A. Kage 1 , C. Münzenmayer 1 , T. Wittenberg 1 , T. Bergen 1 1 Fraunhofer Institute for Integrated Circuits IIS, Erlangen, Germany 2 University Hospital Erlangen, Germany Contact: sven.friedl@iis.fraunhofer.de Abstract: Automated image analysis and interpretation within computer assisted minimally invasive surgery (MIS) most often de- pend and rely on manually defined landmarks, visible in endoscopic views. More specific, within many types of applica- tions, such landmarks must be tracked automatically during the intervention. Typical feature tracking approaches are able to track slightly changing landmarks over time, as they occur in endoscopic image sequences, but are originally most often designed to track automatically detected salient points. In this contribution an approach is presented, where the advantages of feature descriptors and corresponding matchers can be used to track manually defined landmarks. Based on such initiated landmark points, local feature detection and tracking utilizing SURF or KLT features as de- scriptors, is executed. Within the region of interest as a constraint, movements of the detected features can be used to approximate the original landmark movements. Keywords: Feature Tracking, Anatomical Landmarks, SURF, KLT 1 Problem In various clinical scenarios, as e.g. minimally invasive surgery (MIS), endoscopy-based diagnostics in the upper and lower GI tract or clinical motion analysis of organs, the view onto the site is usually restricted by endoscopic apertures. Nevertheless, computer based (surgical) assistance systems often rely on manually defined and initiated landmarks, which must be tracked during the intervention. A typical application of this type is for example the tracking of landmarks on the beating heart within open or minimal invasive heart surgery to enable the augmentation of the view with matched and warped pre-operative image data [1,2,3,4]. Also for clinical motion analysis as the determination of movements of heart valves, manually defined landmarks have to be tracked [5,6]. Nevertheless, due to the slight but constant change of the underlying anatomical structure as well as changes of endoscopic orientation and illumination, classic template mat- ching approaches such as correlation or SSD are neither stable nor sufficient [1,2] for such applications. Alternatively, feature tracking methods are used to determine movements in similar image data [7,8]. The literature dealing with track- ing of medical image data from monocular image sequences describes such approaches, which are designed to automati- cally detect newly appearing local features in the scene and are hence not designed to deal with externally provided landmarks. To solve this problem and to exploit the advantages of combining robust feature descriptors with manually defined landmark points, a local feature tracking approach for endoscopic motion analysis is presented and evaluated on various types of endoscopic imagery. 2 Methods Robust feature descriptors and the corresponding matching algorithms are well known and established methods to iden- tify and track prominent identical points in sequences of consecutive image frames. These methods have been optimized to track feature points with specific characteristics in monocular image sequences. In contrast, manually initiated land- marks for tracking are unlikely to coincide with optimal (in the sense of feature tracking) features. Thus, depending on the visibility of interesting anatomical structures or landmarks and their movement and velocity within an endoscopic image sequence, a region of interest (ROI) can be defined around a manually initiated landmark point. As one side con- dition the ROIs must be defined sufficiently small and in such a way that clearly visible landmarks and the interrelated anatomy lies within the region and independent movements of adjacent ROIs do not interfere with each other. Addition- ally, the ROIs have to be large enough to cover the possible displacements of consecutive image frames based on the oc- curring movements. Now, within each manually initiated region, local feature detection and tracking is applied using well-established feature tracking approaches, which are promising for clinical (real-time) applications: the Kanade- Lucas-Tomasi (KLT) feature tracker [9,10] and speeded up robust features (SURF) [11]. SURF tracking is realized by descriptor comparison within the ROI. 10. CURAC-Jahrestagung, 15. - 16. September 2011, Magdeburg 57