Abstract-- We introduce a mobile system for high resolution panoramic image creation. In our system the user can rotate the camera arbitrarily and see the preview panorama in real-time. The system automatically captures high resolution images and generates a high-quality result with unlimited viewing angle. We employ a coarse-to-fine method for high-quality registration, and a seam-finding method to remove the ghosting effect due to mov- ing objects. The proposed system has been tested on several types of camera phones, and the tests reveal that the system can effi- ciently provide a high-quality panoramic image in spite of the low computational power and memory available in such devices. I. INTRODUCTION Panorama can provide a stunning wide-angle representation of the scene beyond a normal photograph. However, it is painstaking to capture a good panorama. Typically, user has to fix the camera on a tripod to capture the images. Those images are then stitched into a high resolution panorama via computa- tionally demanding PC software, such as Adobe Photoshop © . This two-step procedure makes it difficult to infer the appear- ance and quality of the final result during acquisition. This disadvantage can be overcome by mobile implementa- tions [1, 2]. However, designing a high-quality mobile pano- ramic system is challenging due to low computational re- sources available in such devices. Therefore, previous methods usually impose strong assumptions to the input data. For ex- ample, most systems can only capture images along a one- dimensional path, and require the user to manually align the images to simplify or bypass the registration step. Also, they all use fast methods to blend the captured images. However, they cannot handle moving objects and result in serious ghost- ing effects. II. THE PROPOSED SYSTEM We propose a complete solution for mobile panoramic im- aging which can handle unlimited angle of view (both horizon- tally and vertically) and moving objects, and generate results comparable to the PC software. The system flow is shown in Fig. 1. Our system allows the user to move the camera arbitrar- ily by tracking camera motion in real-time, and automatically determining when to take high-resolution images. The high- resolution images are then registered on the spherical manifold. After warping, all images are stitched together using a high- quality blending method that removes all artifacts. The user can examine the resulting panorama on the phone with interac- tive panning and zooming operations. A. Image capturing Camera motion is estimated by tracking consecutive low- resolution viewfinder frames captured at 30 frames per second, using the algorithm proposed in our previous work [3]. During capture, a low-resolution panorama generated from previously captured images is displayed, along with a moving box indicat- ing the position of the current frame. So the user can easily decide the size and shape of panorama (see Fig. 2). When the camera motion with respect to the previous image exceeds a threshold, the high-resolution images for the current position are automatically captured. B. Image registration and warping Image registration of high resolution captured image frames is essential for ensuring an accurate and seamless representa- tion of the panoramic scene. We designed a hybrid registration approach that follows a coarse-to-fine strategy. The algorithm starts by matching two images at coarse resolution in order to estimate large displacements, followed by a progressive re- finement of the registration parameter by matching salient im- age features (e.g., corners) at finer resolution levels. Both im- age and feature matching operations are carried out using simi- larity metrics that are invariant to illumination changes. In ad- dition, feature-based matching is performed in conjunction with RANSAC to achieve the necessary robustness to outliers, such as moving objects. We enable an unlimited angle of view by mapping the pano- rama onto a spherical manifold. In order to reduce the number of image warping operations, the proposed registration method acts directly in the final spherical manifold coordinates. More- Panoramic Imaging System for Camera Phones Kari Pulli 1 , Marius Tico 1 , Yingen Xiong 1 , Xianglin Wang 1 , and Chia-Kai Liang 1,2 1 Nokia Research Center 2 National Taiwan University Fig. 1. System flow of the proposed solution. The final 360° pano- rama here is generated from 18 input images. Fig. 2. Illustration of the capture interface. As the user moves the camera, the motion is tracked in real-time and shown as a red box. The next image is automatically captured when the motion is large enough. The preview panorama is immediately updated. 2.2-5 978-1-4244-4316-1/10/$25.00 ©2010 IEEE Authorized licensed use limited to: Nokia Information Services. Downloaded on July 02,2010 at 04:56:22 UTC from IEEE Xplore. Restrictions apply.