ON THE VULNERABILITY OF FACE RECOGNITION SYSTEMS TO SPOOFING MASK ATTACKS Neslihan Kose, Jean-Luc Dugelay Multimedia Department, EURECOM, Sophia-Antipolis, France {neslihan.kose, jean-luc.dugelay}@eurecom.fr ABSTRACT There are several types of spoofing attacks to face recognition systems such as photograph, video or mask attacks. To the best of our knowledge, the impact of mask spoofing on face recognition has not been analyzed yet. The reason for this delay is mainly due to the unavailability of public mask attacks databases. In this study, we use a 2D+3D mask database which was prepared for a research project in which the authors are all involved. This paper provides new results by demonstrating the impact of mask attacks on 2D, 2.5D and 3D face recognition systems. The results show that face recognition systems are vulnerable to mask attacks, thus countermeasures have to be developed to reduce the impact of mask attacks on face recognition. The results also show that 2D texture analysis provides more information than 3D face shape analysis in order to develop a countermeasure against high-quality mask attacks. Index Terms— spoofing; face mask; face recognition 1. INTRODUCTION In a spoofing attempt, a person tries to masquerade as another person and thereby, tries to gain access to the system. Based on the observations that 2D face recognition systems are vulnerable to spoofing attacks, researchers started to work on countermeasures to reduce the impact of spoofing attacks on face recognition performances. There have been studies on countermeasures to detect photograph and video spoofing, which are 2D face attacks [1 - 3]. Mask attacks to face recognition systems, which are 3D face attacks, is a considerably new subject. To the best our knowledge, the impact of mask attacks on face recognition has not been analyzed yet. The main reason for this delay is due to the unavailability of public mask attacks databases. In this paper, it is the first time that the impact of mask spoofing is analyzed on face recognition using the mask database which was prepared within the context of the European Union (EU) research project TABULA RASA [4]. The preparation of a mask attacks database is much more difficult and expensive than the preparation of photo or video attacks databases. Initially, to prepare a high quality mask, a 3D scanner is necessary to obtain the 3D model of the target person, which are generally high-cost devices. The procedure continues with manufacturing of the masks which Figure 1. Example sample for fabric mask. In the second column, the mask is worn on the face. The picture is taken from [5]. is also an expensive procedure. The mask attacks database which is used in this study was created by MORPHO [6]. Since the database includes many high-quality mask samples, it is possible to detect the performances of face recognition systems, accurately, under mask attacks. The mask database consists of both the 3D scans and the corresponding 2D texture images. Thanks to the nature of this database, in this paper, we are able to conduct the benchmark evaluations for each of 2D, 2.5D and 3D face recognition. The aim of this study is not to propose a new face recognition method, but instead to show the impact of mask attacks on existing face recognition methods. The paper is organized as follows: Section 2 gives brief information on the mask database which is used in this study. Section 3 explains the face recognition systems which are selected to test the performance of these systems under mask attacks. Section 4 shows the experiments and results. Finally, conclusions are provided in Section 5. 2. THE MASK DATABASE A mask is an object normally worn on the face, typically for protection, performance or entertainment. Additionally, masks can also be used for spoofing purposes. There are several ways of mask manufacturing. Mask of a person can be prepared even by using papers.The company ‘Thats My Face’ [5] provides colored masks (Fig. 1). For each ethnicity, the company has a standard 3D face model and masks are manufactured by mapping one frontal and one profile picture of the target person on this model. However, since the model is based on an ethnic shape, it does not show exact 3D face shape characteristic of the target person. The mask which is used for 3D face spoofing purposes has to show very similar 3D face shape characteristics of the target face to be considered as a successful attack. The mask database used in this study was prepared for this purpose. To obtain similar face shape characteristics of the target person,