3D Scene Reconstruction from Reflection Images in a Spherical Mirror Masayuki Kanbara, Norimichi Ukita, Masatsugu Kidode and Naokazu Yokoya Graduate School of Information Science, Nara Institute of Science and Technology 8916-5 Takayama-cho, Ikoma-shi, Nara 630-0192, JAPAN {kanbara, ukita, kidode, yokoya } @is.naist.jp Abstract This paper proposes a method for reconstructing a 3D scene structure by using the images reflected in a spherical mirror. In our method, the mirror is moved freely within the field of view of a camera in order to observe a surrounding scene virtually from multiple viewpoints. The observation scheme, therefore, allows us to obtain the wide-angle multi- viewpoint images of a wide area. In addition, the following characteristics of this observation enable multi-view stereo with simple calibration of the geometric configuration be- tween the mirror and the camera; (1) the distance and direc- tion from the camera to the mirror can be estimated directly from the position and size of the mirror in the captured im- age and (2) the directions of detected points from each po- sition of the moving mirror can be also estimated based on reflection on a spherical surface. Some experimental results show the effectiveness of our 3D reconstruction method. 1. Introduction This paper proposes a method for 3D reconstruction of a wide area from an image sequence capturing a spherical mirror by a camera whose projection center is fixed. In the field of computer vision, 3D scene reconstruction using im- ages taken from different view points have attracted much attention [1, 2]. Especially, as computers and cameras have made remarkable progress in recent years, a large number of methods for reconstructing a 3D scene from multiple im- ages have been proposed[3]. One of the major approaches to 3D reconstruction from multiple images is to use a static stereo vision [4, 5]. To re- construct the whole 3D structure of a scene, a large number of cameras must be employed for wide observation. How- ever, conventional methods cannot employ a large number of images because it is difficult to calibrate a large number of cameras accurately. These methods, therefore, are not appropriate for reconstructing the whole 3D structure of a scene. One of other approaches is to use an image sequence taken by a moving camera, which is called shape-from- motion [6, 7]. The method can recover extrinsic camera parameters and 3-D positions of natural features simultane- ously by tracking the 2D positions of the natural features in multiple images of the sequence. Therefore, the method allows us to move a camera freely and widely in order to observe multidirectional images of a scene. A factorization algorithm [8] is one of the well known shape from motion methods that can estimate a rough 3D scene model stably and efficiently by assuming an affine camera model. How- ever, when the 3-D scene is not suitable for the affine cam- era model, the estimated results (i.e., both of extrinsic cam- era parameters and 3-D positions of feature points) are not reliable. Therefore, this method is not suitable for recon- structing a dense 3D structure in general. Although, some other methods of shape-from-motion are based on a pro- jective reconstruction method [9], most of these methods reconstruct only a limited scene from a small number of images and are not designed to obtain a dense structure. We can summarize the above discussion as follows: The meth- ods of 3D reconstruction from many images need an accu- rate calibration of a large number of cameras or undesired assumptions regarding camera/scene models, and thus these methods become usually complex and/or unstable. On the other hand, 3D reconstruction methods using re- flection images acquired by capturing a mirror surface by a camera have been also investigated [10, 11, 12]. Some of these methods reconstruct 3D information by using the re- lationship between real features, that are directly captured by a camera, and virtual features, that are points observed through a mirror surface [13, 14]. In these methods, both of the real and virtual features of a 3D point must be captured simultaneously by the camera in order to reconstruct its 3D position, and thus the reconstruction environment is limited to a local scene. This paper proposes a 3D reconstruction method using only a combination of a spherical mirror and a high res- The 18th International Conference on Pattern Recognition (ICPR'06) 0-7695-2521-0/06 $20.00 © 2006