8D–THERMO CAM: Combination of Geometry with Physiological Information for Face Recognition I.A. Kakadiaris, G. Passalis, T. Theoharis, G. Toderici, I. Konstantinidis and N. Murtuza Visual Computing Lab, Dept. of Computer Science University of Houston Houston, TX 77204–3010, USA Fig. 1 OVERVIEW OF MULTI - MODAL FACE RECOGNITION. I. DESCRIPTION Biometrics-based technologies in the area of identity man- agement are gaining increasing importance, as a means of es- tablishing non-falsifiable credentials for end users. However, in the three-way tug-of-war between convenient, unobtrusive data collection (required for user acceptance), accuracy in results (required for justifying deployment), and speed (required for widespread use in practice), no single biometric to date has managed to hold the middle ground that would allow for its ready adoption. The overall goal of our project is to develop the theoretical framework and computational tools that will lead to the development of a practical, unobtrusive, and accurate face recognition system for convenient and effective access control. This framework will encompass 8D characteristics of the face (3D geometry + 2D visible texture + 2D infrared texture, over time). In this video, we present a novel multi-modal facial recognition approach that employs data from both visible spectrum and thermal infrared sensors (Fig 1). Data from multiple cameras are used to construct a 3D mesh representing the upper body and a thermal texture map. We have constructed a subdivision surface using anthropo- metric statistics, which serves as a parametric model of the human face. This model is aligned and fitted to the data using the elastically adaptive deformable model–based fitting framework [1]. From the fitted parametric model we extract two images corresponding to the subject’s face: • a three channel parametric deformation image encoding geometry (by recording the displacement of the corre- sponding face model point) and • a one channel parametric thermal image encoding tem- perature. We subsequently process these images to extract biometric signatures. Specifically, the deformation image is compressed using a wavelet transform and the vasculature graph is ex- tracted from the parametric thermal image. Recognition is accomplished by comparing the signatures obtained from: 1) the parametric deformation image, 2) the parametric thermal image, and 3) the visible spectrum texture map. The novelty of our work lies in the use of deformation images and physiological information as means for compar- ison [2]. By combining deformation images and vasculature graphs as metadata, our algorithms can overcome changes in pose or appearance from the time of enrollment, including facial expressions. We have performed extensive tests using the Face Recog- nition Grand Challenge datasets [3] and our own multimodal database with very encouraging results. The latest updates can be found at the following URL: www.vcl.uh.edu/UR8D II. ACKNOWLEDGEMENTS We are grateful to Paul Ellis, Nikos Karampatziakis, Ioannis Pavlidis, 3dMD and the Texas Learning and Computation Center (TLC 2 ) for their help in carrying out the work we present. We would also like to thank Angela D. Williams for narrating, David Vaughan for directing, and Jerome Crowder for producing this video. REFERENCES [1] D. Metaxas and I. Kakadiaris, “Elastically adaptive deformable models,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 24, no. 10, pp. 1310–1321, 2002. [2] I. A. Kakadiaris, G. Passalis, T. Theoharis, G. Toderici, I. Konstantinidis, and N. Murtuza, “Multimodal face recognition: combination of geometry with physiological information,” in Proc. Computer Vision and Pattern Recognition (CVPR), San Diego, CA, June 2005. [3] P. Phillips, P. Flynn, T. Scruggs, K. W. Bowyer, J. Chang, K. Hoffman, J. Marques, J. Min, and W. Worek, “Overview of the face recognition grand challenge,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), San Diego, CA, June 2005.