Abstract—Endoscopic surgery, a typical example of minimally invasive surgery, has changed the traditional surgical procedure in the way of accessing the visual information available on the corporal site. Its development heavily relies on the improvement of the visualization technologies. Three-dimensional (3D) image system that is capable of providing more depth cues, therefore, has a potentially strong impact on the current 2D image guiding technology. Unlike most other systems that utilize stereoendoscopes with dual lens, we study the feasibility of 3D image construction from single-lens endoscopic video by exploring the theory of multiple view geometry. We coupled a regular endoscope to a position-measurement equipment to provide extrinsic parameters of the acquired images. With the available multiple view geometric algorithms, depth maps were constructed from two consecutively collected images. We present the experimental results and conclude that computational stereopsis has a great potential in building 3D endoscopic image systems for both surgical aiding and training. I. INTRODUCTION HE development of endoscopic surgery is centered at gaining sufficient view of the corporal site on operation. While the current image guidance systems satisfy the basic needs of endoscopic surgery, the 2D flat image does not provide sufficient depth cues. To compensate this shortcoming, systems that provide 3D visual information have found an increasing number of surgical applications [1]. In these systems, stereoendoscopes with two cameras have been employed. A major problem with this type of 3D endoscope is its physical limitation. At the same physical dimensions, a dual-lens endoscope comprises the luminance and resolution compared to a single-lens one. This loss may not be acceptable in some surgical work (e.g. expanded endonasal brain surgery developed in the University of Pittsburgh), which requires ultra small endoscope (4mm in bore diameter). For this reason, stereoendoscopic system based on a single camera is under active development [2]. While most researches seek for a hardware solution to building a 3D video system, there are few reports that focus on a software approach. We formulate this problem from a computational point of view. In theory, to determine the 3D position of a physical point in space, one only needs its images collected from two arbitrary viewpoints, as long as they are distinct. The video frames obtained by a moving This work was supported in part by National Institutes of Health grants No. R01EB002309, R01NS38494 endoscope provide the images of the operating site from distinct viewpoints. Therefore, if the surface structure does not change during the time interval when the video frames are acquired, its 3D locations can be reconstructed by applying computational stereo theories [3]. During the past two decades, computational stereo has reached a mature stage in many stands. The advances in computational stereo are seeking for real applications. We propose to employ the available techniques of computational stereo with the facilitation of an auxiliary equipment to construct 3D structure of endoscopic images. A tracking sensor is mounted on the endoscope to provide accurate measurements of its position and orientation. These measurements are utilized to rectify the images and thereafter the calculation of the depth of the scene. With the depth information, we can construct a 3D surface of the operation site. In addition to aiding surgery on site, this approach will benefit the courses of surgery review, teaching and training as well. II. SYSTEM SETUP We employed Polhemus (Colchester, VT ) Fastrak system to measure the pose of the endoscope in space. This system provides a six-degree-of-freedom output data, including a 3D position vector and three angles (Azimuth, Elevation and Roll attitude). The tracking sensor was mounted on the handle, as shown in Fig. 1, where a special mounting rack was utilized. The measurements from the Polhemus and the video from the endoscope camera were collected by a computer. 3D Construction of Endoscopic Images Based on Computational Stereo Qiang Liu 1 , Robert J. Sclabassi 1,2,3 , Ning Yao 2 and Mingui Sun 1,2,3 Laboratory for Computational Neuroscience Departments of 1 Neurological Surgery, 2 Electrical Engineering and 3 Biomedical Engineering University of Pittsburgh, Pittsburgh, PA 15261, USA T Fig. 1 Test platform that includes an endoscope with camera, a light source, a camera control box, a Polhemus system that is composed of an acquisition box, a transmitter and a receiver. Both the camera control box and the Polhemus acquisition box are connected to the PC. 0-7803-9564-6/06/$20.00 ©2006 IEEE 69