Face Spoofing Detection based on 3D Lighting Environment Analysis of Image Pair Xu Zhang, Xiyuan Hu * , Mingyang Ma, Chen Chen and Silong Peng Institute of Automation, Chinese Academy of Sciences, Beijing, China, 100190 Email: zhangxu2013@ia.ac.cn, xiyuan.hu@ia.ac.cn, mma8@mmm.com, chen.chen@ia.ac.cn, silong.peng@ia.ac.cn Abstract—In this paper, we present a novel face spoofing detection method based on 3D lighting environment analysis of an image pair collected before and after the lighting environment change. Our idea is inspired from the unimpressive fact that the illumination distributions of the internal spoof face stays stable under the protection of the photo and screen plane, while that of a exposed genuine face changes accordingly to different lighting environment due to a natural response of 3D structure. After estimating two sets of lighting environment coefficients of client’s face image pair with the hand of 3D Morphable Model (3DMM) and Sphere Harmonic Illumination Model (SHIM), robust liveness judgement is conducted by hypothesis tests. Experimental results show the effectiveness of proposed method on multiple kinds of face attacks including printed photo, screen photo, and video replay attack, and other advantages such as user cooperation free, loose using conditions, simple equipment demand, easy to camouflage and propitious to face recognition. I. I NTRODUCTION With the development of face recognition system, face spoofing detection becomes a major issue for urgent solution in the security field. Photo and replay video are often used as spoof face because of their easy access from the web site or surveillance equipment. Based on the different facial clues, the existing methods can be classified into three categories: texture based, motion based, and 3D structure based methods. Texture based methods [1], [2], [3] assume that texture information on the fake face has been changed due to the shadow, blur and highlight brought by print or screen. Thus these methods distinguish the attack from the genuine ac- cess by extracting texture feature representing micro-texture arrangement. However, high resolution camera is needed when capturing the micro-texture change. Motion based methods assume that the movement of planar faces differs greatly from genuine faces. The real face can blink eyes [4], move lips [5] and change the gaze casually [6]. Furthermore, a real face can response to the system instructions correctly [7]. The attacks can also be detected by analyzing the relative movement between face region and background [8]. However, these motion based methods fail against refusal to cooperate, tilt the papers or video replay attack. As the essential difference between real and fake face, 3D information provides the most effective protection against spoof attempts. According to the fact that a planar face photo gives a flat structure whereas genuine yields a quite different structure,3D structure based methods [9], [10] make use of structure and depth information to classify real and fake faces. Among existing face spoofing methods, [2], [11], [12] use the light reflection clue. Tan [2] treats the reflection difference between 2D spoof face and 3D genuine face, which caused by geometry structure and the surface roughness, as one of the features that create different image quality under the same imaging condition. But not all spoof faces have high frequency components, which is easily to be avoided by tilting the photo with a tiny angle, to make the photo looks more like a genuine face. Zhang [11] gives a distance robust and user friend- ly method by utilizing the difference of surface reflectance properties of skin and non-skin under multi-spectral light. However, the heavy equipment demand restricts its application in practice, especially on the consumer devices. Smith [12] presents a noninvasive anti-spoofing method used on consumer devices by computing the matching degree between the face reflections and the sequence of colors that were displayed on the screen, but it only works in a darkened environment. It has to be said that in addition to properties mentioned above, there is still an intrinsic but inconspicuous difference, which has been overlooked and under-utilized: under the umbrella of the plane of the photo or the screen, the internal spoof face exists separately from the external environment; on the contrary, the intensities of genuine face change with the lighting environment because it’s just exposed to the camera directly. The correlation of face image pair collected before and after the lighting environment changes, is apparently higher for spoof face than that for genuine face, especially when the dominant light direction seriously deviate far from original’s. Thus the change of lighting environment can be used as a powerful weapon to distinguish whether the face is genuine or not, and a novel face anti-spoofing method based on 3D lighting environment analysis of image pair is proposed in this paper with the hand of 3DMM and SHIM. Compared with the existing face spoofing detection meth- ods mentioned above, our work has the following advantages: 1) User cooperation free: the user aren’t required to move head, blink eye, smile or keep still deliberately. 2) Loose use condition: it’s competent for different light conditions, camera resolutions and skin colors. 3) Extensive application scope: it’s effective to tackle variable photo attacks and replay video attacks. 4) Simple equipment demand: only a few extra light sources are needed; change the screen brightness of consumer device also works in relatively dim light. 5) Easy to camouflage: the additional light is easily taken for granted to improve the light condition. 6) Propitious to face recognition: as illumination cones from different subjects are distinctive, face recogni- tion performance itself improved through the active lighting used to change illumination [13].