778 IEtE I K A N S A C IIONS Oh ACOUSTICS. SPI<F.CH. .ANI) SIGKAL PROCESSING. VOL. 3X. NO 5. MAY IYYO A Class of Iterative Signal Restoration Algorithms AGGELOS K. KATSAGGELOS, MEMBER, IEEE, AND SERAFIM N. EFSTRATIADIS, STUDENT MEMBER, IwE Absfmct-In this paper, a class of iterative signal restoration algo- rithms is derived based on a representation theorem for the general- ized inverse of a matrix. These algorithms exhibit a first or higher or- der of convergence, and some of them consist of an on-line and an off- line computational part. The conditions for convergence, the rate of convergence of these algorithms, and the computational load required to achieve the same restoration result5 are derived. A new iterative algorithm is also presented which exhibits a higher rate of convergence than the standard quadratic algorithm with no extra computational load. These algorithms can be applied to the restoration of signals of any dimensionalitj. Iterative restoration algorithms that have ap- peared in the literature represent special cases of the class of algo- rithms described here. Therefore, the approach presented here unifies a large number of iterative restoration algorithms. Furthermore, ba\ed on the convergence properties of these algorithms, combined algo- rithms are proposed that incorporate apriori knowledge about the w- lution in the form of constraints and converge faster than the previ- ously used algorithms. I. INTRODUCTION HE recovery or restoration of a signal that has been T distorted is one of the most important problems in sig- rial processing applications [I], [ 181. More specifically, the following degradation model is considered: 41 = Dx, (1) where the vectors y and x represent, respectively, lexi- cographically ordered blurred and original signals. The matrix D represents a linear deterministic distortion which may be space varying or space invariant. When y and x represent images, then the distortion may be due to mo- tion between the camera and the scene or due to atmo- spheric turbulence. The signal restoration problem is then to invert (1) or to find a signal as close as possible to the original one, subject to a suitable optimality criterion given y and D. Equation (I) also represents the more gen- eral degradation model where an additive noise term is considered. In this case, the restoration problem takes again the form of solving (1) for x, where D is replaced by a square well-conditioned matrix and y by D'y, where denotes the transpose of a matrix or vector. This case will be separately studied in Section 111, since computa- tionally simpler algorithms can be used. Iterative algorithms are used in our work in solving the Manuscript received August 2. 1988; revised June 16, 1989. This work was supported in part by the National Science Foundation under Grant MIP- 86142 17. The authors are with the Department 01' Electrical Engineering and Computer Science. Northwestern University. The Technological Institute. Evanaton. 1L 60208-3 I 18. IEEE Log Number 9034417. signal restoration problem. Iterative restoration algo- rithms have a number of advantages over direct or recur- sive restoration techniques, and they have been used ex- tensively in the literature [ 181. Most of these algorithms have a linear or first-order convergence rate. Singh et al. [ 191 derived an iterative restoration algorithm with a quadratic rate of convergence, when the matrix D in (I) is invertible. Morris et ul. [ 14]-[ 161 and Lagendijk et ul. [13] generalized this algorithm for higher orders of con- vergence. In their derivation, the matrix D in (I) was in- vertible. In 1141-[I61 it was further assumed that D rep- resents a convolution operator. In this paper, we extend the results in [ 131-[ 161 and [ 191 by showing that when D is singular, the higher order algorithms converge to the minimum norm solution of (l), provided that a solution exists. This is a very important result because for a large number of distortions of prac- tical interest (motion, out-of-focus), the matrix D is sin- gular. Furthermore, we derive iterative algorithms with linear and higher order convergence rates for the general case when D in (1) is a rectangular matrix. In this case, the limiting solution of these algorithms is the minimum norm least-squares (MNLS) solution of (1). The deriva- tion of these algorithms is based on a representation theo- rem for the generalized inverse D+ of the matrix D. Iter- ative restoration algorithms benefit a great deal from the use of constraints which incorporate properties of the so- lution into the restoration process. However, the direct use of constraints with the higher order algorithms may result in divergence or meaningless results. We propose techniques which allow us to effectively use constraints with a combination of linear and higher order iterative algorithms. The derivation of the linear and higher order algorithms obtaining the MNLS solution of (1) is presented in Sec- tion 11. Computationally simpler higher order algorithms solving for the minimum norm solution of (I), when D is a square, positive semidefinite matrix, are presented in Section 111. Such a situation may result, for example, when a noise term is added to (I). Then. after regular- ization, the restoration problem is again the solution of a set of linear equations analogous to (I), where D and are replaced by another matrix A and a vector 6, respec- tively. These algorithms extend the results reported in [13]-[I61 and 1191. In Section IV, the algorithms are compared with respect to their computational load. The incorporation of constraints are discussed in Section V, and a number of experimental results are presented in Sec- tion VI. Finally, conclusions are presented in Section VII. 0096-35 18/90/0500-0778$0 1 .OO O 1990 IEEE