Image restoration by sparse 3D transform-domain collaborative filtering Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian Department of Signal Processing, Tampere University of Technology P.O. Box 553, 33101 Tampere, Finland firstname.lastname@tut.fi ABSTRACT We propose an image restoration technique exploiting regularized inversion and the recent block-matching and 3D filtering (BM3D) denoising filter. The BM3D employs a non-local modeling of images by collecting similar image patches in 3D arrays. The so-called collaborative filtering applied on such a 3D array is realized by transform- domain shrinkage. In this work, we propose an extension of the BM3D filter for colored noise, which we use in a two-step deblurring algorithm to improve the regularization after inversion in discrete Fourier domain. The first step of the algorithm is a regularized inversion using BM3D with collaborative hard-thresholding and the seconds step is a regularized Wiener inversion using BM3D with collaborative Wiener filtering. The experimental results show that the proposed technique is competitive with and in most cases outperforms the current best image restoration methods in terms of improvement in signal-to-noise ratio. Keywords: image restoration, deconvolution, deblurring, block-matching, collaborative filtering 1. INTRODUCTION Image blurring is a common degradation in imaging. In many cases, the blurring can be assumed space-invariant and thus modeled as a convolution of the true image with a fixed point-spread function (PSF). Such a model is given by } ({)=(| ~ y)({)+  ({) , (1) where | is the true (non-degraded) image, y is a blur PSF,  is i.i.d. Gaussian noise with zero mean and variance  2 , and { 5 [ is a 2D coordinate in the image domain [. The inversion of the blurring is in general an ill-posed problem; thus, even noise with very small magnitude, such as truncation noise due to limited-precision arithmetic, can cause extreme degradations after naive inversion. Regularization is a well known and extensively studied approach to alleviate this problem. It imposes some regularity conditions (e.g., smoothness) on the obtained image estimate and/or on its derivatives. Numerous approaches that employ regularization have been proposed; an introduction can be found for example in the books. 1, 2 In particular, an image restoration scheme that comprises of regularized inversion followed by denoising has been a basis of the current best-performing restoration methods. 3, 4 Such denoising after the inversion can be considered as part of the regularization since it attenuates the noise in the obtained solution (i.e. the solution is smoothed). Various denoising methods can be employed to suppress the noise after the inversion. Filtering in multiresolu- tion transform domain (e.g., overcomplete wavelet and pyramid transforms) was shown 4—6 to be eective for this purpose. In particular, the SV-GSM, 4 which employs Gaussian scale mixtures in overcomplete directional and multiresolution pyramids, is among the current best image deblurring methods. Another denoising technique used after regularized inversion 3, 7, 8 is the LPA-ICI 9 which exploits a non-parametric local polynomial fit in anisotropic estimation neighborhoods. The best results of the methods based on LPA-ICI were achieved by the shape-adaptive discrete cosine transform (SA-DCT) deblurring 3 where the denoising is realized by shrinkage of the SA-DCT applied on local neighborhoods whose arbitrary shapes are defined by the LPA-ICI. This work was partly supported by the Academy of Finland, project No. 213462 (Finnish Centre of Excellence program [2006 - 2011]); the work of K. Dabov was supported by the Tampere Graduate School in Information Science and Engineering (TISE).