Edge-based Blur Kernel Estimation Using Patch Priors Libin Sun * Brown University lbsun@cs.brown.edu Sunghyun Cho Adobe Research scho@adobe.com Jue Wang Adobe Research juewang@adobe.com James Hays Brown University hays@cs.brown.edu Abstract Blind image deconvolution, i.e., estimating a blur kernel k and a latent image x from an input blurred image y, is a severely ill-posed problem. In this paper we introduce a new patch-based strategy for kernel estimation in blind de- convolution. Our approach estimates a “trusted” subset of x by imposing a patch prior specifically tailored towards modeling the appearance of image edge and corner primi- tives. To choose proper patch priors we examine both sta- tistical priors learned from a natural image dataset and a simple patch prior from synthetic structures. We show that our patch prior prefers sharp image content to blurry ones. Based on the patch priors, we iteratively recover the par- tial latent image x and the blur kernel k. A comprehensive evaluation shows that our approach achieves state-of-the- art results for uniformly blurred images. 1. Introduction Image blur caused by camera shake is a common prob- lem in consumer photography. A motion blurred image y capturing a static scene is often modeled as: y = k ∗ x + n, (1) where k is the blur kernel, x is the latent image, n is noise, and ∗ is the convolution operator. For blind deconvolution, the goal is to recover both x and k from y, which is an ill- posed problem. To overcome the ill-posedness of blind deconvolution, previous works place strong assumptions or prior knowl- edge on k and x. Regarding k, it is often assumed that k should be sparse [7, 16] and continuous [3]. For x, it is often assumed that image gradients are heavy-tailed [7, 16, 13, 14, 11]. However, we argue that this popular family of sparsity priors is not suitable for the task of kernel estima- tion for the following reasons. First, sparsity priors prefer blurry images to sharp ones [13], because blur reduces over- all gradient magnitude. Hence, sparsity priors have limited * Part of the work was done while the first author was an intern at Adobe Resarch. capacity to steer the latent image towards a sharp solution. Second, they fundamentally suffer from the fact that the unit of representation is extremely limited: gradient filters often consider two or three pixels, hence ignoring longer- range dependencies which give rise to the most salient im- age structures and geometry. This is also why state-of-the- art image restoration methods often involve larger neighbor- hoods or image patches [2, 15, 18, 20]. Another family of blind deconvolution algorithms ex- plicitly exploits edges for kernel estimation. Joshi et al. [8] and Cho et al. [5] directly restore sharp edges from blurry edges and rely on them to estimate the blur kernel. While these methods work well for small scale blur, they have dif- ficulty dealing with large blur kernels, as directly restoring sharp edges from a severely blurred image is non-trivial. To handle large blur kernels, Cho and Lee [4] introduce an edge-based approach, which alternates between restor- ing sharp edges and estimating the blur kernel in a coarse- to-fine fashion. This framework has been further extended by Xu and Jia [19] and has proven to be effective [9]. How- ever, these approaches heavily rely on heuristic image fil- ters such as shock and bilateral filtering for restoring sharp edges, which are often unstable, as we will show in Sec. 3.3. In this paper, we propose a new edge-based approach us- ing patch priors on edges of the latent image x. Patches can model image structures better than filter responses. In our approach, we estimate a “trusted” subset of x by imposing patch priors specifically tailored towards modeling the ap- pearance of image edge and corner primitives. We only re- store these primitives since other image regions, e.g. flat or highly-textured ones, do not carry much useful blur infor- mation for kernel estimation. Furthermore, restoring tex- tures often results in hallucinated high frequency content, which corrupts the subsequent kernel estimation steps. We illustrate how to incorporate the patch prior into an edge- based iterative blind deconvolution framework through an optimization process, where we iteratively recover the par- tial latent image x and the blur kernel k. Experimental results show that our approach achieves state-of-the-art re- sults (Sec. 5). The main question we address in this paper is what is 1