Do We Need a New Large-Scale Quality Assessment Database for Generative Inpainting Based 3D View Synthesis? (Student Abstract) Sadbhawna, Vinit Jakhetiya, Badri N. Subudhi, Harshit Shakya, Deebha Mumtaz Indian Institute of Technology Jammu, India {2018rcs0013, vinit.jakhetiya, subudhi.badri, 2020pcs2064, 2018rcs0011}@iitjammu.ac.in Abstract The advancement in Image-to-Image translation techniques using generative Deep Learning-based approaches has shown promising results for the challenging task of inpainting-based 3D view synthesis. At the same time, even the current 3D view synthesis methods often create distorted structures or blurry textures inconsistent with surrounding areas. We ana- lyzed the recently proposed algorithms for inpainting-based 3D view synthesis and observed that these algorithms no longer produce stretching and black holes. However, the ex- isting databases such as IETR, IVC, and IVY have 3D- generated views with these artifacts. This observation sug- gests that the existing 3D view synthesis quality assessment algorithms can not judge the quality of most recent 3D syn- thesized views. With this view, through this abstract, we an- alyze the need for a new large-scale database and a new perceptual quality metric oriented for 3D views using a test dataset. Introduction A good 3D-synthesized images/videos can provide con- sumers with a more engaging and better immersive ex- perience. Free Viewpoint Video (FVV), 3D-Television, 360°video, Virtual Reality (VR) are some of the applications of 3D-synthesis, famous because of their realistic and inter- active experience (Shih et al. 2020; Niklaus et al. 2019). Un- fortunately, the rendered 3D views, even using the contem- porary methods, cannot generate the perfect novel 3D view (Shih et al. 2020). These methods cannot perform efficiently on complex surfaces and produce some artifacts, as shown in Fig. 1. The artifacts in the 3D synthesized views are differ- ent from the conventional artifacts in regular natural images. With the advancement of efficient algorithms for generating 3D synthesized views, it is required to have an image quality assessment (IQA) algorithm which can automatically judge the perceptual quality of generated 3D synthesized views that match with the human visual system. The 3D IQA al- gorithms can judge the perceptual quality and are also help- ful in the fast development of Image Restoration (IR) (IR includes tasks such as super-resolution (SR), denoising, en- hancement, etc.) algorithms. With this view, through this ab- stract, we will address the research problem statement that is Copyright © 2022, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. Figure 1: (a). A synthesized view. (b). The failure (green arrows) of a random patch (red window) in a 3D synthesized view. Synthesized Using: (Shih et al. 2020) there a need for creating a large-scale 3D synthesized IQA database which is generated using the recently proposed 3D synthesized view generation algorithms? Related Work Novel View Synthesis(NVS): From proxy geometry based traditional methods to Generative Adversarial Networks (GANs) based deep learning methods, NVS has evolved with the advancement in computer vision techniques. (Shih et al. 2020; Niklaus et al. 2019) are some of the new view synthesis techniques. Synthesizing novel views requires stipulations such as comprehensive scene understandings, preserving structures observed in the input semantics, lowered baseline requirements, etc. Even the newest view synthesis methods in the literature lack in one or the other of these stipulations. Hence, much research is being done to make the view synthesis perfect. Figure 1 shows an example of the failure of such a contemporary 3D synthesized method. From this Figure, it is clear that view synthesis does not render a clear view in certain circumstances, such as complex surfaces. Image Quality Assessment(IQA): SSIM(Wang et al. 2004) is the most widely used IQA method as it introduced the structural similarity in comparing images as compared to the Mean Square Error (MSE) value, which are FR (Full- Reference) IQAs. There are various other popular NR (No- Reference) IQAs, such as BRISQUE, NIQE, etc., in the lit- erature. There are various new IQAs based on deep features The Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI-22) 13039