- 1 - A study of some super resolution techniques in video sequence Christophe RIEDINGER * , Nadhir KHEMAKHEM * and Gérard CHOLLET * *TSI Department – Télécom-ParisTech: 37-39, rue Dareau, 75014, Paris - FRANCE crieding@telecom-paristech.fr khemaken@telecom-paristech.fr chollet@telecom-paristech.fr Abstract: This paper proposes a comparative study of various super resolution (SR) techniques: Spectral approach, Interpolation and fusion, Wiener filtering, Maximum A Posteriori (MAP), Eigenfaces reconstruction and Projection Onto Convex Sets (POCS). MAP, Eigenfaces, Wiener filtering and POCS have been implemented and experimented on gray level images. We compare the performances of the algorithms on standard test protocols. The POCS and MAP give similar results. Blind SR [4], hybrid methods POCS-MAP and also the improvement of algorithms so that they can operate in real time are future directions for research work. Key words: Eigenfaces, Frequential domain, Interpolation, MAP, POCS, Super resolution, Wiener INTRODUCTION Super resolution (SR) is the operation of obtaining a high-resolution (HR) image from a set of low resolution (LR) images. The HR image is of better quality than the LR images. It is not necessary nowadays to demonstrate the utility of SR reconstruction. Several military or civil uses of SR exist. SR techniques have been used in the context of the ANR CSOSG2007 KIVAOU project 1 . Intuitively, we can perform SR reconstruction by adding the LR zoomed images. The output image will exhibit more details thanks to the information that is brought by the various images. To do this reconstruction, the reconstructed object, must not have experienced strong changes in pose and illumination from one image to another. Occlusions are problems that have not really been solved up to date in SR. To add the SR images, we must before register them and perform the warping so that the shifts 1 This project aims at developing a demonstrator with innovative video analysis tools for solving 2 types of problems: 1) Mobile tool (valise) for identification and facial biometric indexation through real time video analysis. And 2) Multiple video analysis platform. These videos are recorded during an event. This tool uses video synchronization, people signatures extraction, and tracking. The goal is to makes possible or easier the a posteriori analysis of recorded data to perform investigation. They should provide us faces pictures of better quality than the pictures provided by surveillance cameras. In all the cases, we used video sequences as input. between LR images are compensated. This fundamental step is discussed previously to the discussion of reconstruction techniques. 1. Image registration The main existent techniques for registration and warping of the image for SR are correlation in the spatial domain, correlation in the frequency domain, phase correlation, optical flow calculation (Lucas Kanade algorithm). We chose to implement the correlation in the frequency domain method. This is only valid for translation movements between the frames of the video (global translational model). Correlation in the frequency domain, computes the correlation function between two images (say two consecutive frames of the video). The maximum of the correlation function indicates the horizontal and vertical shifts that exist between the two images. This shift has to be estimated at the accuracy of the high resolution pixels. Indeed, if we perform the correlation on the LR images, we will have a shift that is in LR image pixel unit. To have a shift that is in HR image pixel unit (a not integer shift in the LR image pixel unit), we zoomed in the LR images (interpolation) before performing the correlation. We interpolate at the HR scale using a polyphase filter implementation (the resample function of MATLAB). 2. Interpolation and fusion As previously said, the most intuitive way of performing SR is to simply add the registered and zoomed LR images and to divide the result by the number of LR images. Generally, this result is a