THE PERIODIC STEP GRADIENT DESCENT ALGORITHM - GENERAL ANALYSIS AND APPLICATION TO THE SUPER RESOLUTION RECONSTRUCTION PROBLEM Tamir Sagi 1 , Arie Feuer 2 and Michael Elad 3 1,2 Dept. of Electrical Engineering, Technion - Israel Institute of Technology. Haifa 32000, Israel. 3 Hewlett Packard Laboratories - Israel, Technion. Haifa 32000, Israel 1 e-mail: stamir@tx.technion.ac.il 2 e-mail: feuer@ee.technion.ac.il 3 e-mail: elad@hp.technion.ac.il ABSTRACT Solving image restoration problems, especially complex problems like Super Resolution reconstruction, is very demanding computationally. Iterative algorithms are the practical tool frequently used for this purpose. This paper reviews the Periodic Step Gradient Decent (PSGD) algorithm, suggested as a sub-optimal algorithm for solving restoration problems (with emphasis on Super Resolution reconstruction problems). The PSGD differs from well-known iterative algorithms in the way the data of the problem in hand is processed. Whereas iterative algorithms process the entire given data in order to update the result, the PSGD updates the result progressively. This paper provides an analysis of the PSGD. We show that the PSGD has an efficient implementation, easy to achieve convergence conditions and fast convergence speed when applied to a Super Resolution reconstruction problem. The performance of the PSGD when applied to a Super Resolution reconstruction problem, is demonstrated by simulations and compared to the performance of other well-known algorithms. 1 INTRODUCTION One of the problems widely discussed in the image processing literature, is the problem of image restoration [1]-[3]. Throughout the last decade as computer technology develops, there is a growing interest in restoration problems that are more demanding computationally. One such problem is the problem of Super Resolution reconstruction. In this problem a single improved resolution image is reconstructed from a set of geometrically warped, blurred, downsampled and noisy measured images. The recent work of Elad and Feuer [4], [5] presented a new approach toward the Super Resolution reconstruction problem. According to Elad and Feuer the Super Resolution reconstruction problem may be modeled using the well-known classical single image restoration model ([1]-[3]) (1) = + where is a known × vector of measurements, is a known × matrix represents a linear distortion operator, is the × additive noise vector (assumed to be a white Gaussian noise with zero mean and a known covariance matrix) and is a × vector of unknowns. (Note that the images are represented using a columnwise lexicographic ordering). That way, methods associated with solving restoration problems may be used for solving the more complex Super Resolution reconstruction problem. When facing a restoration problem our goal is to get an estimate of the unknown vector . The common solution associated with restoration problems, is the Least Squares (LS) solution [3], This solution is achieved by solving the quadratic minimization problem (2) - - The solution of the minimization problem presented in (2) is (3) ∧ = In order to actually solve the equations set (3), the inverse of the matrix (which for further reference will be denoted by ) must be calculated and stored in memory. The dimensions of in a practical restoration problem are very large, thus making the invertion task computationally impossible and storage very demanding. As a result motivation to investigate indirect methods to solve restoration problems arises. Iterative algorithms are usually suggested as the practical tool for solving restoration problems [3]. Typically, iterative algorithms refer to the given data (the matrix and the vector ) as a package. At each iteration this whole package is processed and a new estimate of the solution is calculated. In this paper we analyze an algorithm which processes the given data equation after equation and not as one package, we refer to this algorithm as the Periodic Step Gradient Decent (PSGD). The steady state solution of the PSGD is sub-optimal to the LS solution, however in this paper the PSGD is investigated as a stand-alone algorithm. We settle for a sub-optimal solution and investigate the PSGD as an algorithm for solving restoration problems. Using the PSGD algorithm for Super Resolution reconstruction brings out the main advantages of the algorithm. Simulations we performed show that the PSGD typically converges to the steady state solution faster and with low computational cost when compared to well-known algorithms such as Steepest Descent (SD), Normalized Steepest Descent (NSD), Jacoby (J), Gauss-Siedel (GS), Successive Over Relaxation (SOR) and Conjugate Gradient (CG) (those algorithms are reported in detail in references [6]-[12]). This paper is organized as follows: Section 2 presents the PSGD algorithm, the PSGD algorithm is analyzed and the main results are presented. Section 3 presents a comparison between the PSGD and other known algorithms, when applied to the Super Resolution reconstruction problem. Simulations results are presented in Section 4 and Section 5 concludes the paper. EUSIPCO 1998.