T. Jiang et al. (Eds.): MICCAI 2010, Part I, LNCS 6361, pp. 275–282, 2010. © Springer-Verlag Berlin Heidelberg 2010 Real-Time Stereo Reconstruction in Robotically Assisted Minimally Invasive Surgery Danail Stoyanov, Marco Visentini Scarzanella, Philip Pratt, and Guang-Zhong Yang Institute of Biomedical Engineering, Imperial College London, London SW7 2AZ, UK {danail.stoyanov,marco.visentini-scarzanella02, p.pratt,g.z.yang}@imperial.ac.uk http://vip.doc.ic.ac.uk Abstract. The recovery of 3D tissue structure and morphology during robotic assisted surgery is an important step towards accurate deployment of surgical guidance and control techniques in minimally invasive therapies. In this article, we present a novel stereo reconstruction algorithm that propagates disparity in- formation around a set of candidate feature matches. This has the advantage of avoiding problems with specular highlights, occlusions from instruments and view dependent illumination bias. Furthermore, the algorithm can be used with any feature matching strategy allowing the propagation of depth in very dispa- rate views. Validation is provided for a phantom model with known geometry and this data is available online in order to establish a structured validation scheme in the field. The practical value of the proposed method is further dem- onstrated by reconstructions on various in vivo images of robotic assisted pro- cedures, which are also available to the community. 1 Introduction In robotically assisted Minimally Invasive Surgery (MIS), recovering the underlying 3D structure of the operating field in vivo is important for registering pre-operative data to the surgical field-of-view for providing dynamic active constraints and motion compensation [1]. Tomographic intra-operative imaging modalities can potentially provide anatomically co-registered information about the 3D shape and morphology of the soft tissues but their deployment in operating theatres is a significant challenge [2]. Currently, the most practical method of recovering the 3D structure of the operat- ing site in situ is through optical techniques using a stereo laparoscope. This informa- tion can subsequently be used to align multimodal information within a global reference 3D coordinate system and enhance robotic instrument control. However, the recovery of 3D geometry from stereo in real-time during robotic procedures is diffi- cult due to tissue deformation, partial occlusion due to instrument movement, and specular inter-reflections. The recovery of 3D information from stereo images is one of the classic problems in computer vision. Given a calibrated stereo rig, the task is to identify the unique corre- spondence of image primitives across the stereo image pair. Recent review articles