(a) (b) (c) Countermeasure for the Protection of Face Recognition Systems Against Mask Attacks Neslihan Kose, Jean-Luc Dugelay Multimedia Department EURECOM Sophia-Antipolis, France {neslihan.kose, jean-luc.dugelay}@eurecom.fr Abstract— There are several types of spoofing attacks to face recognition systems such as photograph, video or mask attacks. Recent studies show that face recognition systems are vulnerable to these attacks. In this paper, a countermeasure technique is proposed to protect face recognition systems against mask attacks. To the best of our knowledge, this is the first time a countermeasure is proposed to detect mask attacks. The reason for this delay is mainly due to the unavailability of public mask attacks databases. In this study, a 2D+3D face mask attacks database is used which is prepared for a research project in which the authors are all involved. The performance of the countermeasure is evaluated on both the texture images and the depth maps, separately. The results show that the proposed countermeasure gives satisfactory results using both the texture images and the depth maps. The performance of the countermeasure is observed to be slight better when the technique is applied on texture images instead of depth maps, which proves that face texture provides more information than 3D face shape characteristics using the proposed approach. Keywords- face spoofing; mask attacks; countermeasure I. INTRODUCTION In a spoofing attempt, a person tries to masquerade as another person and thereby, tries to gain an access to the system. Based on the observations that face recognition systems are vulnerable to spoofing attacks, researchers started to work on countermeasures to reduce the impact of spoofing attacks on recognition performances. Recently, there have been studies on 2D face countermeasures to detect photograph and video spoofing [1 - 3]. However, the topic of mask spoofing attacks to face recognition systems is considerably new. The main reason for this delay is due to the unavailability of public mask attacks databases. This paper aims to fill this gap by proposing a countermeasure technique to protect face recognition systems against mask spoofing using the mask database which is prepared within the context of a European Union (EU) research project. The preparation of mask spoofing database is much more difficult and expensive than the preparation of photograph or video spoofing databases. This is why there is still a gap in the areas which analyze the impact of mask spoofing attacks on face recognition (FR) systems and the countermeasure Figure 1. Example from the mask attacks database created by [5] (a) Texture image (the default output of most existing 3D scanners) (b) the snapshot of 3D scan (the default output of 3D scanners) (c) the snapshot of the 3D scan with texture that is obtained when the texture image (a) is mapped on the 3D scan (b). techniques to reduce these impacts. In the present study, our aim is to reduce the impact of mask attacks on the performances of face recognition systems by applying a local binary patterns (LBP) based countermeasure technique. Photograph and video attacks are 2D face attacks whereas mask attack is a 3D face attack. Camera is used for 2D FR systems to capture the image of a person and scanner is used for 3D FR systems to obtain the 3D scan of a person. Since camera captures the image of a mask attack (2D face image), we can say that mask attacks can be used to spoof both 2D and 3D FR systems. Furthermore, most of the existing 3D scanners do not provide only 3D scan, they also capture texture image. Fig. 1 (a) & (b) shows an example for the two outputs of a scanner. Therefore, in case of no additional hardware (only one camera for 2D FR system and one scanner for 3D FR system), a countermeasure which is developed by using only texture images can be used to protect not only 2D but also 3D FR systems if the texture images are provided as default output of the scanner. On the other hand, depth maps are estimated from 3D scans, therefore countermeasure which is developed by using only the depth maps can be used to protect 3D FR systems since these scans can be obtained only by using 3D scanners. In this study, the proposed countermeasure do not need any extra hardware and user collaboration. The technique relies on a single image. The mask attacks database which is used in this