PERCEPTUAL QUALITY ASSESSMENT FOR COLOR IMAGE INPAINTING A. DANG Thanh Trung, B. Azeddine BEGHDADI, C. Chaker LARABI L2TI, Institut Galil´ ee, Universit´ e Paris 13, France A-B XLIM, D´ epartment SIC, Universit´ e de Poitiers, France C Email:{dang.thanhtrung, azeddine.beghdadi}@univ-paris13.fr, chaker.larabi@univ-poitiers.fr ABSTRACT A novel objective measure for assessing the quality of image in- painting is proposed. In contrast to standard image quality met- rics, the proposed one takes into account some constraints and characteristics related to the specific goals of inpainting tech- niques. The idea is to combine spatial low-level features and perceptual criteria in the design of the objective Image Inpaint- ing Quality Metric (IIQM). The used characteristics are the vi- sual coherence of the recovered regions and the visual saliency describing the visual importance of an area. Experimental results demonstrate the good performance of the proposed IIQM and its well adaptation to the evaluation of image inpainting results. Index Terms— Image inpainting, image completion, image quality assessment, inpainting quality assessment. 1. INTRODUCTION Image inpainting becomes a very active field of research for many real world applications such as digital cinema, com- putational photography, archaeological image restoration and archival documents restoration [1]. Most inpainting algorithms could be classified into two broad categories depending upon their scope and goals. First, geometry-oriented approaches are mainly designed for filling narrow or small holes because they are less suitable for synthesizing semantic textures or struc- tures [2–4]. Second, exemplar-oriented methods could deal with large holes and they could be further subdivided into two subgroups: greedy strategy [5–7] and global optimization strat- egy [8–10]. However, when it comes to the evaluation of in- painting results, visual inspection is often used because there is almost no dedicated objective assessment tool. Image quality assessment (IQA) plays a prominent role for many applications, including video streaming monitoring, med- ical imaging and lossy compression control among others. IQA in its broad sense refers to the problem of evaluating the level of perceptual quality of an image. Many interesting methods for predicting image quality have been proposed in the litera- ture [11–15]. After all, the subjective evaluation is still the most reliable approach. The latter is not sound for real-time applica- tions because it is tedious and time consuming. It is worth notic- ing that the goals of IQA in the case of image inpainting are sub- stantially different from the classical image quality evaluation. Indeed, in the case of inpainting, the intent is to evaluate the quality of the restored image. It is also worth understanding that inpainting could be considered as a specific image restoration. In both problems, inpainting and classical image restoration, the existing IQA metrics could not be directly applied because of the specificity of the aforementioned applications. For example, in the case of image inpainting, the recovered region is totally different from the original one (which is in the major cases un- known because of degradations or occlusions). This operation aims at restoring the missing parts or replacing some parts of the image in a visually plausible way. The intent of Image In- painting Quality Assessment (IIQA) metric is then to evaluate the visual quality of the inpainted regions in terms of spatial co- herence with the existing parts of the image. Results of image inpainting are very often evaluated subjec- tively or by using some objective metrics not well adapted to the specificities of their criteria. However, subjective evaluation ex- periments are known being time consuming, complex and some- times unpredictable due to uncontrolled human factors such as fatigue, visual discomfort, knowledge, etc. A few interesting works on objective IIQA have been pub- lished recently where the proposed approaches suffer from some shortcomings. For instance, an analysis of gaze patterns ob- tained thanks to eye-tracking experiments has been described in [16] for predicting inpainted image quality. This method has the same disadvantages as subjective evaluation. Authors of [17] proposed a full-reference quality metric for image inpainting based on a variant of SSIM composed of luminance, definition and gradient similarity. Actually, image inpainting is known as a blind image completion where the notion of ”reference” is nonexistent. Moreover, this index cannot be applied when inpainted areas are large because original and inpainted images may highly differ from each other. Two other quality indexes are introduced in [18] using saliency maps. The key idea is based on the variation of the saliency map before and after inpainting, ex- pressed by two metrics: average squared visual salience (ASVS) and degree of noticeability (DN). However, these indexes do not take into consideration the global visual appearance of the image that significantly affects to the restoration quality. The most contribution of this paper is to propose a global im- age inpainting quality index. The proposed metric is constructed 398 978-1-4799-2341-0/13/$31.00 ©2013 IEEE ICIP 2013