EUROGRAPHICS 2020/ F. Banterle and A. Wilkie Short Paper UV Completion with Self-referenced Discrimination Jiwoo Kang † , Seongmin Lee † and Sanghoon Lee ‡ Department of Electrical & Electronic Engineering, Yonsei University, Seoul, Korea Facial Alignment with Facial Model UV Texture with Holes UV Completion Facial Reconstruction Figure 1: A facial UV completion example for an occluded UV texture constructed by sampling an image using the fitted facial model. In this paper, a self-referenced discrimination method is presented to model the facial UV distribution without the complete UV ground-truth based on face symmetry, enabling the network to be trained to synthesize high-quality facial texture with a set of incomplete UVs. Abstract A facial UV map is used in many applications such as facial reconstruction, synthesis, recognition, and editing. However, it is difficult to collect a number of the UVs needed for accuracy using 3D scan device, or a multi-view capturing system should be required to construct the UV. An occluded facial UV with holes could be obtained by sampling an image after fitting a 3D facial model by recent alignment methods. In this paper, we introduce a facial UV completion framework to train the deep neural network with a set of incomplete UV textures. By using the fact that the facial texture distributions of the left and right half-sides are almost equal, we devise an adversarial network to model the complete UV distribution of the facial texture. Also, we propose the self-referenced discrimination scheme that uses the facial UV completed from the generator for training real distribution. It is demonstrated that the network can be trained to complete the facial texture with incomplete UVs comparably to when utilizing the ground-truth UVs. CCS Concepts • Computing methodologies → Image processing; Neural networks; 1. Introduction Significant progress of recent years in 3D face alignment enables us to obtain accurate and dense correspondence between a 3D face model and a 2D facial image [ZLL * 17]. A facial UV map generated by sampling textures over the fitted image using the correspondence has been widely used in many applications such as facial recon- † J. Kang and S. Lee contributed equally to this work. ‡ Corresponding author. struction, face recognition, and face editing [DCX * 18, TL19]. The facial UV has many missing pixels due to the self-occlusion of the face, i.e., the UV map is an image with hole regions. Fortunately, image inpainting methods recently proposed have demonstrated impressive completion capability of the hole regions on image [DCX * 18, LRS * 18, YLY * 19]. In particular, Deng et al. [DCX * 18] proposed a framework for Deep Convolutional Neural Network (DCNN) to complete the facial UV map with the self-occluded re- gion. However, the corresponding ground-truth images without the holes for training the hole completion networks are necessarily re- quired for the previous methods. Whereas lots of complete images c  2020 The Author(s) Eurographics Proceedings c  2020 The Eurographics Association. DOI: 10.2312/egs.20201018 https://diglib.eg.org https://www.eg.org