FACE RECOGNITION WITH BIOMETRIC ENCRYPTION FOR PRIVACY-ENHANCING SELF-EXCLUSION Haiping Lu, Karl Martin, Francis Bui, K. N. Plataniotis, Dimitris Hatzinakos The Edward S. Rogers Sr. Department of Electrical and Computer Engineering University of Toronto, M5S 3G4, Canada {haiping, kmartin, bui, kostas, dimitris}@comm.toronto.edu ABSTRACT Face recognition has been employed in various security- related applications such as surveillance, mugshot identi- fication, e-passport, and access control. Despite its recent advancements, privacy concern is one of several issues pre- venting its wider deployment. In this paper, we address the privacy concern for a self-exclusion scenario of face recog- nition, through combining face recognition with a simple biometric encryption scheme called helper data system. The combined system is described in detail with focus on the key binding procedure. Experiments are carried out on the CMU PIE face database. The experimental results demonstrate that in the proposed system, the biometric encryption module tends to significantly reduce the false acceptance rate while increasing the false rejection rate. Index Terms— Face recognition, biometric encryption, security, privacy, watch list. 1. INTRODUCTION Face recognition has a wide range of applications, such as surveillance, access control, e-passport, and human-computer interaction [1]. In particular, face recognition is one of the three identification methods used in e-passports. Further- more, facial features scored the highest compatibility among the six biometric attributes in a machine readable travel doc- uments (MRTD) system based on several evaluation fac- tors including enrollment, renewal, machine requirements, and public perception [2]. This is largely due to the fact that compared to other popular biometric technologies: face recognition is non-intrusive and easy to use [3]. The work presented in this paper has been partially supported by the Ontario Lottery and Gaming Corporation (OLG). The views, opinions, and findings contained in this paper are those of the authors and should not be construed as official positions, policies, or decisions of the OLG, unless so designated by other official documentation. The authors would like to thank Mr. Klaus Peltsch from the Ontario Lot- tery and Gaming Corporation, and Dr. Ann Cavoukian and Dr. Alex Stoianov from the Information and Privacy Commissioner of Ontario for many useful discussions. Although face recognition has made tremendous progress in the past two decades, there have been several concerns preventing its wider deployment, such as the effectiveness in field test, the performance under uncontrolled conditions, and privacy concern. Privacy concern arises when there are large centralized databases of biometric passwords and there are risks of identity theft and privacy leaks [4]. Consequently, biometric encryption has emerged to address this concern. The objective is to deploy biometrics in a privacy-enhancing way that minimizes the possibility of abuse, maximizes in- dividual control, and ensures full functionality of the sys- tems in which biometrics are used [5]. For face recognition with biometric encryption, rather than storing one’s facial im- age in a database, the facial image is used to encrypt (code) some other information such as a cryptographic key and only the biometrically-encrypted data is stored. This removes the need to collect and store actual biometric data in database and most privacy concerns associated with centralized databases are eliminated. There has been several works proposed to construct privacy-enhancing systems using biometric encryption for face biometrics. A fuzzy vault based cryptographic key gen- eration method was introduced by Wang et al. [6]. The helper data system (HDS) is applied to face recognition in 2005 [4], and the multi-bit quantization using likelihood ratio method is proposed for privacy-enhancing face recognition in 2007 [7]. In [8], we have investigated a biometric encryption system based on the quantization index modulation (QIM) approach [9, 10] for a self-exclusion scenario of face recog- nition. In this paper, we investigate a biometric encryption system based on the helper data system in [4] for the same self-exclusion scenario. In the HDS approach for face recognition presented in [4], the fiducial points are extracted from face images. Then, bi- narized features are constructed based on estimates of the re- liability statistics. In a general HDS construction, during en- rollment, these features are used to bind a cryptographic key, creating one of the helper data. The operation involved is the binary XOR. Here, the goal of the system is to reject, during the verification process, an unauthorized subject who does not