1046 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 21, NO. 3, MARCH 2012 Image Registration Under Illumination Varia Using Region-Based Confidence Weighted M -Estimators Mohamed M. Fouad, Member, IEEE,Richard M. Dansereau, Senior Member, IEEE,and Anthony D. Whitehead, Member, IEEE Abstract—We present an image registration model for image sets with arbitrarily shaped local illumination variations between im- ages. Any nongeometric variations tend to degrade the geometric registration precision and impact subsequent processing. Tradi- tional image registration approaches do not typically account for changes and movement of light sources, which result in interimage illumination differences with arbitrary shape. In addition, these approaches typically use a least-square estimator that is sensitive to outliers, where interimage illumination variations are often large enough to act as outliers. In this paper, we propose an image reg- istration approach that compensates for arbitrarily shaped inter- image illumination variations, which are processed using robust -estimators tuned to that region. Each -estimator for each il- lumination region has a distinct cost function by which small and large interimage residuals are unevenly penalized. Since the seg- mentation of the interimage illumination variations may not be perfect, a segmentation confidence weighting is also imposed to re- duce the negative effect of mis-segmentation around illumination region boundaries. The proposed approach is cast in an iterative coarse-to-fine framework, which allows a convergence rate similar to competing intensity-based image registration approaches. The overall proposed approach is presented in a general framework, but experimental results use the bisquare -estimator with region segmentation confidence weighting. A nearly tenfold improvement in subpixel registration precision is seen with the proposed tech- nique when convergence is attained, as compared with competing techniques using both simulated and real data sets with interimage illumination variations. Index Terms—Image registration, local illumination changes, -estimation, segmentation confidence weighting. I. I NTRODUCTION G EOMETRIC image registration (GIR) is the process of determining the point-by-point correspondence between two views of a scene and the transformation to align these cor- respondences. The GIR of a set of images is a common pre- processing step in many applications [1], such as generating Manuscript received March 09, 2011; revised June 30, 2011; accepted Au- gust 17, 2011. Date of publication September 08, 2011; date of current version February 17, 2012. The associate editor coordinating the review of this manu- script and approving it for publication was Dr. Chares Creusere. M. M. Fouad is with the Department of Company Engineering, Military Tech- nical College, Cairo, Egypt (e-mail: mmafoad@ieee.org). R. M. Dansereau is with the Department of Systems and Company En- gineering, CarletonUniversity, Ottawa,ON K1S 5B6, Canada(e-mail: rdanse@sce.carleton.ca). A. D. Whitehead is with the School of Information Technology, Carleton Uni- versity, Ottawa, ON K1S 5B6, Canada (e-mail: awhitehe@connect.carleton.ca). Digital Object Identifier 10.1109/TIP.2011.2167344 panoramic image mosaics [2], performing super-resolution hancement [3], image stitching [4], change detection [5], m ical imaging [6], and remote-sensing applications [7]. For and similar applications, subpixel registration accuracy is essary for satisfactory postprocessing results. A primary problem in many image registration scenarios that, often, there also exists nongeometrical transform var within the image set. These variations manifest from many tors, including changes in lighting, motion/deformations o tures in the scene, changes in occluded and occluding obje and changes in exposure, aperture, focus, lens distortion o photometric factors. In addition, multimodal data sets cou clude larger variations in terms of imaging modality, resol and information content. Literature exists on handling ma these confounding factors for GIR, although arguably few proaches have been proposed to flexibly handle local chan in illumination, with the multimodal techniques being a no exception that can have good registration performance bu the level of fine subpixel precision that we seek. The focus this paper is hence to look at how interimage illumination tions in the scene can be compensated within an intensity- GIR framework, with particular attention to the arbitrary s typical for illumination variations resulting from shadows from one or more light sources. Note that the light source assumed to be far field from the scene. Interimage variations in illumination do indeed impact the GIR, as discussed in the following literature. Refs. [8] and present similar global-illumination models (GIMs) with sca gain and offset modeling in their GIR to relate the intensity levels of an image pair to improve the registration perform In [8],an additional smoothness constraint, with predefined neighborhoods, is also imposed. In [10], an affine illumina model(AIM) is proposed with an extended gain and offset model that also used triangular or quadrilateral region sup (not arbitrarily shaped) to allow an improved modeling of illumination variations. In [11],a dualinverse compositional algorithm is presented based on an assumption that the geo- metric and photometric transformations can be replaced, thus impeding the use of explicit local photometric transformat Although Bartoli [11] models the photometric changes ins of illumination variations, he does so in a fashion using sca gain and offset, when dealing with gray images, quite the as in the GIM. In this paper, we will refer to the GIM and t AIM as our comparator group. The approaches in [8]–[11] do not take into account mul- tiple shading levels with arbitrariness in the shape ofthe 1057-7149/$26.00 © 2011 IEEE