IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 57, NO. 10, OCTOBER 2009 3815 Registration Errors: Are They Always Bad for Super-Resolution? Guilherme Holsbach Costa, Member, IEEE, and José Carlos M. Bermudez, Senior Member, IEEE Abstract—The super-resolution reconstruction (SRR) of images is an ill posed problem. Traditionally, it is treated as a regularized minimization problem. Moreover, one of the major problems con- cerning SRR is its dependence on an accurate registration. In this paper, we show that a certain amount of registration error may, in fact, be beneficial for the performance of the least mean square SRR (LMS-SRR) adaptive algorithm. In these cases, the regular- ization term may be avoided, leading to reduction in computational cost that can be important in real-time SRR applications. Index Terms—Adaptive systems, image reconstruction, image registration, least mean square (LMS). I. INTRODUCTION A N approach for digital image quality improvement that has attracted large interest in the last decade uses super- resolution reconstruction (SRR). SRR consists basically of com- bining multiple low-resolution (LR) images of the same scene or object to form a higher resolution image. Reference [1] re- views several important results on SRR available in the litera- ture. These results have been successfully applied to areas such as digital photography [2], [3], surveillance [4]–[6], machine vision and biometry [7] among others. The SRR is tradition- ally formulated as a minimization problem. However, SRR fre- quently leads to ill-posed inverse problems. This is because of the influence of blurring operators, the information loss in the decimation (acquisition) process, and the availability of an in- sufficient number of LR images. For this reason, a regularization term is usually employed in the formulation of the minimization problem [1], [8]–[10]. In image processing, regularization usu- ally means the use of a priori information (restrictions) to deter- mine a single solution or to better condition the inverse problem. The simplicity of the least mean square (LMS-SRR) algo- rithm proposed in [9], [10] to solve the SRR problem makes it an interesting solution for real-time applications such as SRR of video sequences. The results in [9] suggest a performance improvement when LMS is regularized (leading to the R-LMS algorithm). However, a more detailed study of the registration error effects on the SRR performance is not pursued in [9]. Manuscript received October 05, 2008; revised April 22, 2009. First pub- lished May 19, 2009; current version published September 16, 2009. The as- sociate editor corrdinating and approving this manuscript for publication was Prof. William A. Sethares. This work was supported in part by CNPq by Grants 141456/2003-5, 307024/2006-7, and 470792/2006-0. G. H. Costa is with the Department of Mechanical Engineering, University of Caxias do Sul, 95070-560, Caxias do Sul, Brazil (e-mail: holsbach@ieee.org). J. C. M. Bermudez is with the Department of Electrical Engineering, Federal University of Santa Catarina, 88040-900, Florianópolis, Brazil (e-mail: j.bermudez@ieee.org). Digital Object Identifier 10.1109/TSP.2009.2023402 One of the major issues regarding SRR algorithms is their de- pendence on an accurate registration. Registration errors have always been regarded as detrimental to the SRR performance, and several works have proposed algorithms that are robust to the effects of such errors [8], [11], [12]. This robustness usually comes at the cost of an increase in computational complexity. More recently, a new algorithm has been proposed which is ro- bust to registration and admits a fast implementation for global translational image motions [13]. However, even this fast im- plementation cannot compete with the LMS-SRR algorithm in either computational or memory requirements [9], [10]. In this paper, we present a study 1 of the registration error effects on the regularization of the LMS-SRR algorithm. The study extends a previous work by the authors that presented a de- terministic model for the stochastic behavior of the LMS-SRR algorithm, emphasizing the influence of the registration errors on the reconstruction result [15]. Now, we propose a similar model for the regularized version of this algorithm (R-LMS- SRR), when registration is considered to be ideal (without er- rors). Using these two models, it is possible to isolate and com- pare the effect of the registration errors on the reconstruction re- sult of the LMS-SRR with the effect of the regularization term on the reconstruction result of the R-LMS-SRR. For mathemat- ical tractability, we analytically compare these effects for the case of a single iteration per time sample and equal step-sizes. The single iteration case is of practical interest, as it corresponds to the least computational cost implementation. Moreover, it has been shown in [16] that an appropriate choice of the step-size can reduce performance losses caused by a small number of it- erations per time sample. More information on the design of the LMS-SRR algorithm can be found in [16]. We show that, contrary to traditional wisdom, moderate levels of registration errors may improve the LMS-SRR algorithm performance. In this case, registration errors act like a form of regularization. Hence, regularization may be avoided, leading to computational savings that can be important in real-time SRR applications. These results introduce a new discussion about the behavior of the LMS-SRR in the presence of registration errors, illustrating how well they may work acting as regularization. Our analysis assumes whole-image translational movements. This is the simplest case to handle and is representative of sev- eral practical applications [13], [15]. Later on, we show through examples that the conclusions of this analysis may hold also for more general movements that can be approximated by the global translational model. In Section II, we briefly review the LMS-SRR algorithm and its regularized version (R-LMS-SRR) [9], [10]. In Section III 1 Intial results of this work have been presented in [14]. 1053-587X/$26.00 © 2009 IEEE