An image registration technique aimed at super resolution on mobile devices Mihail Georgiev, Ilian Todorov, Atanas Boev, Atanas Gotchev and Karen Egiazarian Institute of Signal Processing, Tampere University of Technology, P.O.Box 553, FIN-33101 Tampere, FINLAND, Email: Firstname.Lastname@tut.fi ABSTRACT We propose an image registration technique to be implemented on mobile devices equipped with cameras. We address the limited computational power and low-quality optics of such devices and aim at designing a registration algorithm, which is fast, robust with respect to noise, and allows for corrections of optical distortions. We favor a feature-based approach, consisting of feature extraction, feature filtering, feature matching, and transformation estimation. In our application, the transformation estimation is robust to local distortions, and is accurate enough to allow for a subsequent super-resolution on the registered images. The performance of the technique is demonstrated in fixed-point implementation on the TMS 320 C5510 DSP. Keywords: Image registration, Super Resolution, Mobile Device, DSP, OMAP 1. INTRODUCTION Modern portable electronic devices, such as mobile and smart phones and PDAs include digital camera to extend their functionality. Such devices generally have low-cost image acquisition subsystem, which inevitably results in visible optical distortions in the produced images and loss of high-frequency details. Images acquired by a typical camera phone suffer from aliasing, sensor noise, blur and geometrical distortions. Furthermore, portable devices usually have limited computational resources for image post-processing, because of battery, CPU and operating system limitations. The most commonly used solution being a simple post-processing with interpolation to higher-resolution grid is not good enough to produce clearer and sharper images. It does not cope with the optical limitations of the camera, or with the lost of high-frequency information in the image. Hence, the idea of fast and low-complexity image enhancement for mobile applications emerges. Super-resolution enhancement aims at restoring high-frequency information which is unavailable in original low- resolution image. Super-resolution works by combining several slightly spatially-different low-resolution observations into a single high-resolution image 1 . It is essential that the spatial displacements of the low-resolution images are precisely known. The image registration task aims at finding the optimal transformation that would match two images. When using a hand-held camera, a series of shots of the same scene would produce images with displacements and local image misalignments due to moving objects, intensity changes, etc. Together with the presence of rigid-body transformations, there are optical distortions introduced as well. Therefore, a good image registration algorithm for a mobile camera should handle noise, large displacements, local distortions, and to cope with lens distortions. Additionally, such an algorithm should allow for implementation on hardware with limited computational resources. There are various image registration techniques building transformation models on given non-aligned images (2, 3) . We will refer only to these that are potentially suitable for super-resolution, i.e. achieving sub-pixel accuracy level and exploiting globally linear camera motion model. Intensity-based methods are typically chosen to obtain accurate alignment of images. A thorough survey on optical-flow estimation is given in 4 . By this approach any linear transformation can be estimated with sub-pixel accuracy. Keren et al. proposed an improved approach 5 , based on optical-flow method and used as a basis to super-resolution 6 . The authors stated that the approach satisfies the accuracy requirements in the case of planar shifts and rotations. However, for large rotations they suggested that feature-based registration should be used. Intensity-based methods are also noise vulnerable. Intensity-based methods that rely on Fourier Transform are widely used in applications that require exact alignment. The main advantage of such approach is that rotation, scale, and shift can be easily expressed in Fourier domain. Such Multimedia on Mobile Devices 2008, edited by Reiner Creutzburg, Jarmo H. Takala Proc. of SPIE-IS&T Electronic Imaging, SPIE Vol. 6821, 68210A, © 2008 SPIE-IS&T · 0277-786X/08/$18 SPIE-IS&T Vol. 6821 68210A-1 2008 SPIE Digital Library -- Subscriber Archive Copy