3792 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 46, NO. 11, NOVEMBER 2008 Improving Piecewise Linear Registration of High-Resolution Satellite Images Through Mesh Optimization Vicente Arévalo and Javier González, Associate Member, IEEE Abstract—Piecewise linear transformation is a powerful tech- nique for coping with the registration of images affected by local geometric distortions, as it is usually the case of high-resolution satellite images. A key point when applying this technique is to divide the images to register according to a suitable common triangular mesh. This comprises two different aspects: where to place the mesh vertices (i.e., the mesh geometrical realization) and to set an appropriate topology upon these vertices (i.e., the mesh topological realization). This paper focuses on the latter and presents a novel method that improves the registration of two images by an iterative optimization process that modifies the mesh connectivity by swapping edges. For detecting if an edge needs to be swapped or not, we evaluate the registration improvement of that action on the two triangles connected by the edge. Another contribution of our proposal is the use of the mutual information for measuring the registration consistency within the optimization process, which provides more robustness to image changes than other well-known metrics such as normalized cross-correlation or sum of square differences. The proposed method has been successfully tested with different pairs of panchromatic QuickBird images (0.6 m/pixel of spatial resolution) of a variety of land covers (urban, residential, and rural) acquired under different lighting conditions and viewpoints. Index Terms—High-resolution satellite images, mesh optimiza- tion, piecewise linear (PWL) registration. I. I NTRODUCTION I MAGE registration is an essential step in many remote sens- ing applications like image fusion, change detection, 3-D scene reconstruction, etc. In this process, one image remains fixed (the reference image), whereas the other (the input or moving image), which is acquired on a different date, from a different viewpoint and/or using a different sensor, is spatially transformed until fitting with the first one. Traditionally, the registration process is dealt with in two stages. In the first one, the positions of a set of pairs of corresponding points are identified in the images, and in the second stage, this set of correspondence pairs is exploited to robustly estimate a map- ping function which is applied to transform all the pixels of the Manuscript received March 29, 2007; revised October 25, 2007. Current version published October 30, 2008. This work was supported in part by the Spanish Government under Research Contracts CICYT DPI-2005-01391 and AECI PCI-A-7286-06. The authors are with the Department of System Engineering and Automa- tion, University of Málaga, 29071 Málaga, Spain (e-mail: varevalo@ctima. uma.es; jgonzalez@ctima.uma.es). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TGRS.2008.924003 input image onto the reference one (some kind of interpolation is required in this step) [1]. A variety of mapping functions have been reported in the lit- erature for image registration, including polynomial [2], radial basis functions [3], piecewise linear (PWL) [4] or piecewise- cubic [5] functions, multiquadric functions [6], B-spline func- tions [7], etc. In remote sensing, global polynomial (POL) functions usually perform well with low and medium reso- lution images (National Oceanic and Atmospheric Adminis- tration Advanced Very High Resolution Radiometer, EarthSat, Landsat, Indian Remote Sensing, etc.) but may not be powerful enough to register high-resolution ones (QuickBird, Ikonos, or OrbView), where geometric distortions may become important to attain short revisit time, and high-resolution satellites pitch along their orbit to observe the scene from off nadir; thus, two (temporal) images of a certain scene may have been acquired from quite different angles, which entails large local geometric differences, particularly in urban scenarios and high- relief terrain. For registering such images, PWL functions are particularly suitable (as revealed in [8]), because they divide the images into a mesh of triangular patches, which are individually registered through linear transformations. In this registration method, it is of particular relevance—the case where the camera projection can be approximated by a paraperspective one. Under this assumption, provided that two corresponding image triangles come from the projection of a planar patch of the scene, they must perfectly overlap for a certain affine transformation [9] (see Fig. 1). It is clear that, in order to meet such desirable condition for all mesh triangles, the mesh cannot be an arbitrary one, but it must fulfil some geometrical and topological requirements. Current approaches for PWL registration (including those com- mercially available in packages such as ERDAS, ENVI, PCI, etc.) create the triangular meshes for the two images from a set of correspondence (conjugate) pairs, which are localized either automatically or by hand. Typically, they are generated by using the Delaunay’s triangulation method [10] (or other similar one), which produces triangles of balanced size and shape but are not optimal for covering as many planar patches as possible (see Fig. 2). The aim of this paper is to modify the topology (not the vertices) of a given initial mesh by iteratively swapping its edges in order to improve the global registration of a pair of images. This process can be seen as an optimization procedure that, at each step, focuses on a particular edge and improves the registration consistency of the quadrilateral formed by the 0196-2892/$25.00 © 2008 IEEE Authorized licensed use limited to: Universidad de Malaga. Downloaded on November 25, 2008 at 07:09 from IEEE Xplore. Restrictions apply.