Extension of Phase Correlation to Subpixel Registration Hassan Foroosh (Shekarforoush), Josiane B. Zerubia, Senior Member, IEEE, and Marc Berthod Abstract—In this paper, we have derived analytic expressions for the phase correlation of downsampled images. We have shown that for downsampled images the signal power in the phase corre- lation is not concentrated in a single peak, but rather in several co- herent peaks mostly adjacent to each other. These coherent peaks correspond to the polyphase transform of a filtered unit impulse centered at the point of registration. The analytic results provide a closed-form solution to subpixel translation estimation, and are used for detailed error analysis. Excellent results have been ob- tained for subpixel translation estimation of images of different na- ture and across different spectral bands. Index Terms—Image alignment, phase correlation, subpixel reg- istration. I. INTRODUCTION A NALYSIS and fusion of data in an image sequence often require registration of the images. A wide variety of methods and techniques [4] can be found in the literature for solving this fundamental and challenging problem. Although in many applications pixel-level registration may be adequate, some important problems in remote sensing [13], [22], [25], [26] and biomedical imaging [5], [9] have introduced the requirement for subpixel registration. In this paper, we are interested in the refinement of coarsely registered images to subpixel accuracy, where by coarse regis- tration we imply pixel-level registration. We will assume that the error in registration at this refinement level is a translation within the bounds of the noise and the error of the imaging system. Therefore we confine our attention only to the problem of subpixel translation estimation. We are also interested in sub- pixel registration across different spectral bands. The most commonly used approach to subpixel registration is based on interpolation. Examples include correlation inter- polation [6], [28], intensity interpolation [28], phase correlation interpolation [21], [28] and the geometric methods [2], [8]. It is obvious that the accuracy of these methods depends highly on the quality of the interpolation algorithms. One popular approach for subpixel registration without in- terpolation is based on the differential properties of image se- quences [11]–[13], [17], [28], [30]. The main idea in this ap- Manuscript received June 8, 2000; revised November 13, 2001. The associate editor coordinating the review of this manuscript and approving it for publica- tion was Dr. Eric L. Miller. H. Foroosh (Shekarforoush) is with the Department of Electrical Engineering and Computer Science, University of California, Berkeley, CA 94720 USA (e-mail: hshekar@eecs.berkeley.edu). J. Zerubia and M. Berthod are with INRIA, 06902 Sophia Antipolis Cedex, France (e-mail: zerubia@sophia.inria.fr; berthod@sophia.inria.fr). Publisher Item Identifier S 1057-7149(02)00803-5. proach is to relate the temporal derivatives of the sequence to the spatial derivatives of the reference frame using the so called gra- dient constraint equation under intensity conservation assump- tion [10], [11], [17]. Due to the intensity conservation assump- tion, these methods often require that the inter-frame displace- ments to be relatively small compared to the intensity gradi- ents of the reference frame. Also, to reduce the sensitivity of the derivative operators to the noise process, in general, some regularization is required and the gradient constraint is com- bined within some local neighborhood of each pixel. However, the main pitfall of this approach is probably the so called aper- ture problem [10], where for some patterns such as a gradual curve, the available information in a local neighborhood of a pixel (small aperture) is not sufficient to disambiguate the true registration parameters. A second possible approach without interpolation is to for- mulate subpixel registration as an optimization problem. Ex- amples of this approach include [14], [27], [29]. In this ap- proach the problem is formulated as a cost function to be min- imized with respect to the registration parameters of the inter- frame transformation model. As in the gradient based methods these methods are highly dependent on image intensity con- servation assumption and would fail when applied across mul- tiple spectral bands or when considerable amount of luminance variations are present between frames. This shortcoming can probably be alleviated by an adequate normalization of var- ious spectral bands. Also note that these methods as well as the gradient-based methods implicitly include interpolation by ap- plying local smoothing and regularizing operators. There are also other techniques that are based on the local normalized correlation [1], polynomial regression [32], the dis- crete cosine transform [15], and the use of control points using a model-based approach [7]. This work is particularly motivated by some important fea- tures of the phase correlation method which make its use at- tractive for multispectral registration, and hence analytic results have been derived to demonstrate how the method can be ex- tended to subpixel accuracy. A condensed version of part of this work was presented in [23] and [24]. This paper is organized as follows. In the next section, we describe the phase correlation method and outline its impor- tant properties. In Section III, we present our extension of the method to subpixel registration. Section IV provides thorough analysis of various sources of error. Experimental results are then given in Section V for different modalities of images in- cluding satellite images across different spectral bands. Finally in Section VI some concluding remarks are provided. 1057–7149/02$17.00 © 2002 IEEE