1558-1748 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JSEN.2018.2886418, IEEE Sensors Journal Real-Time 6DOF Pose Estimation of Endoscopic Instruments Using Printable Markers Aniket Gadwe 1,2 and Hongliang Ren 1∗ Abstract— To accurately navigate surgical instruments around in-vivo environments in minimally invasive surgeries through flexible endoscopes or rigid scopes, real-time tracking of surgical instruments is required. It is even challenging to track regular-shaped rigid bodies on a less-distorted general- purpose monocular vision, while most of the surgical instru- ments or devices used are cylindrical and slim imaged with endoscope distortion in nature. Thus more dedicated and ac- curate rigid-body tracking approach for cylindrical endoscopic instruments will be more helpful in intra-operative guidance. In this paper, we present an endoscopic instrument tracking approach with printable markers that consists of a green band and a square with four white and one black circle inscribed in it. The proposed approach can estimate the 6-DOF (Degrees of freedom) pose, i.e. X, Y, Z, Roll, Pitch and Yaw of the imaged instrument with respect to the endoscope. Detailed experiments justify that our proposed marker detection framework allows translation motions between 42 mm and 100 mm along depth direction and rotational motions around X and Y up to ±180 ◦ , with 90 % detection rate in dynamic condition, 99 % in static condition and accurate pose estimation with a mean translation error of 1.286 mm with standard deviation of 0.673 mm and mean rotational error of 1.497 ◦ with a standard deviation of 0.873 ◦ . Our proposed marker can be easily printed and wrapped to existing endoscopic instruments which potentially can assist in the precise guidance in minimally invasive surgeries without changing the existing workflow or incurring significant cost burdens. Index Terms— Minimally invasive surgeries (MIS), endo- scopic instruments, printable markers, laparoscopy, pose es- timation. I. I NTRODUCTION Minimally Invasive Surgeries (MIS) are emerging using the state-of-the-art navigation and tracking technologies [1] such as electromagnetic tracking [2], inertial tracking, and optical tracking, in order to reduce the positioning errors that happen during the human-operated surgeries. In MIS, keyhole-like incisions are made and tubular trocars are inserted through these incisions. Laparoscope or Endoscope camera which is mostly monocular at present is passed through one of these holes to monitor the inside situation. Advantages of MIS include reduced post-operation infec- tions, less bleeding, fewer complications, less scarring and * Corresponding author. This work is supported by the Singa- pore Academic Research Fund under Grants R-397-000-227-112 and R397000297114 awarded to Dr. Hongliang Ren. 1 Department of Biomedical Engineering, National University of Singapore, 21 Lower Kent Ridge Rd, Singapore 119077 gadwe.aniket@gmail.com ren@nus.edu.sg/hlren@ieee.org 2 Department of Chemical Engineering, Visvesvaraya National Institute of Technology, India, Nagpur 440010 gadwe.aniket@gmail.com shorter hospitalization time. There are also disadvantages of such type of surgeries like it requires high-end specialized devices, needs specialized surgeons, suffers from the line of sight constraints and poor depth perception from endoscope or laparoscope [3]. To track the target anatomical structures and instruments inside the body, medical imaging modalities like ultrasound can be incorporated. But surgeons have to interpret the data from ultrasound images and relate it with the endoscopic or laparoscopic images [4]. Feed from ultra- sound and laparoscope or endoscope are shown separately on different screens to reduce the complication for surgeons. In previously presented work [5], a navigation method is introduced in which registering of 2D ultrasound images has been achieved into a surgical scene. To get all the results of this approach on a single display screen, continuous tracking of a device is necessary. Major techniques that are used to track surgical devices since it involves optical tracking and electromagnetic tracking systems. This type of systems involves external reference stations such as multiple optical cameras or magnetic field generators, which occupies the valuable space in surgical theatre. But their performance can be affected by magnetic or occlusion interference’s [4]. Due to the line-of-sight constraints in MIS, a technique of instrument tracking like a Magnetic field, Angular Rate, and Gravity (MARG) can be incorporated as explained in [6]. Normally pose estimation is done with the help of fiducial markers explained in [7]. This fiducial marker includes cylindrical [3] or planar [8] markers which are mounted on the surface of the devices which have to be tracked. Both monocular camera and depth camera can be used for marker- based surgical tool tracking. In [9] and [10] depth and stereo camera are used for marker-based tool tracking. Comparative (a) Planar printable marker view before wrapping on the tool (b) A front view of the tool after wrapping with printable marker Fig. 1: Endoscopic view of marker before and after wrapping on the tool.