Face Recognition with Liveness Detection using Eye and Mouth Movement Avinash Kumar Singh, Piyush Joshi, G. C. Nandi Robotics and Artiicial Intelligence Laboratory Indian institute of Information Technology, Allahabad - 211012, UP, india {avinashkumarsingh 1986, piyushjoshi3839}@gmail.com, gcnandi@iiita.ac.in Abstract- The recent literature on face recognition technology discusses the issue of face spooing which can bypass the authentication s y stem by placing a photo/video/mask of the enrolled person in front of the camera. This problem could be minimized b y detecting the liveness of the person. Therefore, in this paper, we propose a robust liveness detection scheme based on challenge and response method. The liveness module is added as extra la y er of securit y before the face recognition module. The Iiveness module utilizes face macro features, especially eye and mouth movements in order to generate random challenges and observing the user's response on account of this. The reliabilit y of liveness module is tested b y placing diferent t y pes of spooing attacks with various means, like using photograph, videos, etc. In all, ive types of attacks have been taken care of and prevented by our system. Experimental results show that system is able to detect the Iiveness when subj ected to all these attacks except the eye & mouth imposter attack. This attack is able to bypass the liveness test but it creates massive changes in face structure. Therefore resultant unrecognized or misclassiied b y the face recognition module. An experimental test conducted on 65 persons on university of Essex face database conirms that removal of e y e and nose components results 75% misclassiication. Keywords- Face Recognition, Liveness Detection, Face Spooing, Face Macro Features Movement, HAAR Classiier, Principal Component Anal y sis. I. INTRODUCTION Biometric authentication is a way to authenticate persons that usually a human does in his life. Every human being has some physiological or behavioural characteristics like (face, ingerprint, voice, gait, etc.), which make them unique, thereby diferentiating them rom others. Among all these biometric traits, we restrict ourselves to face. The history of face recognition technology started roml960s. From the last 50 years, face recognition technology has experienced a rapid growth in itself [10]. Various methods have been proposed by various researchers so far, to recognize the person even in the bad illumination, different facial expressions, orientations, and even in partial occlusions [1]. The latest survey [2] shows that face recognition technology is the second most used biometric technology used by the marke/users. Face recognition is now used in diferent areas like access controls, human robot interaction, surveillance, etc. Recently several researchers quantiied the integrity of the face recognition system and found that these systems are also vulnerable to diferent kinds of attacks. Most requently used attack is the spooing attack [7][8][6]. Face spooing is an attack where attacker tries to bypass the face recognition system by placing a photo/video/mask of the enrolled persons in ront of the camera. The problem lies within the working principle of Face Recognition. The principal of face recognition doesn't care about who is submitting the credentials. It only concerns about the person is enrolled or not. Hence whatever effort we will do for making our classiication good and effort in using good resolution camera, nothing will help. Besides this, it helps attacker to perform their attack more accurately. This problem leads to the question where one can think about the signiicance of the accuracy and eiciency of the system, when the reliability is not assured. Liveness detection of the user could be a way to deal with this problem. Therefore Researchers observed the need of security mechanism and proposed various ways to deal with this problem. On the basis of literature we have grouped possible solution in three main categories (1) liveness detection by using challenge and response method (2) liveness detection by utilizing face texture (image quality), and (3) liveness detection by combining two or more biometrics (multi-modal approach). In challenge and response method system throws some challenge in terms of eyes and mouth movement which can only be performed by real user not by photo, and analyses there response in account of the given challenges.In this regard most of the researchers [3][4][15] have utilized eye blinking, while in type-II researcher's exploits texture information (smoothness/roughness, edges, etc.) to distinguish between real and imposter [5][17]. Multimodal approach mostly uses speech and face as the combination to deal with this attack [18]. These spooing techniques are going more complex day by day right rom simple photograph to painted contact lenses and polymeric face, ingers. Hence a list of modem approaches to deal with all these circumstances is mentioned by [6][7].He has suggested, in their report that for liveness detection, maximum utilization of face macro features is the best way; hence in this paper we utilized both the eye and mouth movement to detect the liveness with the constraint of challenge and response. Previous techniques discussed in the literature are mostly based on the eye blinking, which can be 978-1-4799-3140-8/14/$31.00 ©2014 IEEE 592