A 3D face recognition system using curvature-based detection and holistic multimodal classification Alessandro Colombo Claudio Cusano Raimondo Schettini DISCo (Dipartimento di Informatica, Sistemistica e Comunicazione) Universit` a degli Studi di Milano-Bicocca Via Bicocca degli Arcimboldi 8, 20126 Milano, Italy colomboal@disco.unimib.it, cusano@disco.unimib.it, schettini@disco.unimib.it Abstract We present here a fully automated system for face de- tection and recognition. In a scene acquired by a three- dimensional laser scanner the system can detect the pres- ence of human faces; the faces detected are registered in a canonical position and then recognized. Both the 3D and 2D (i.e. pictorial) information pro- vided by the scanner are exploited. The 3D representation is used to detect the presence of faces and normalize them. In the recognition step 3D and 2D information are indepen- dently analyzed, producing two different scores which are then combined to compute the output of the system. 1. Introduction Face detection and recognition have attracted the atten- tion of many research groups. Important applications in fields such as video surveillance, law enforcement, and human-computer interfaces require the detection and recog- nition of human faces. Most of the procedures discussed in the literature pro- vide for only one of the tasks involved in face recognition [2, 11, 12]. We present, instead, a complete system for the detection of the faces in a scene, normalization, via 3D reg- istration, of the faces found and recognition. The input scenes are acquired by a Minolta Vivid 900 laser range scanner. The scanner produces a high quality 3D representation of each scene, and a corresponding color image. Detection is feature-based: the system explores the scene to locate salient facial features, such as the nose and eyes. This is done by analyzing the curvature of the surfaces in the scene. A similar analysis is also performed to select relevant landmarks on the faces which are then used in the normalization step. Recognition, instead, is based on a holistic approach: the registered 3D and 2D images are projected into two specific eigenfaces spaces, where the classification is performed us- ing different combination strategies. We have considered two recognition scenarios: identity verification and identification. In verification the subject, whose face has been acquired, claims his identity and the system must ascertain whether or not the claimed identity corresponds to the truth. To be able to do so the subject must have been previously presented to the system through a procedure called enrollment. In the identification scenario the system must discern the identity of a subject by com- paring the input face with a gallery of previously enrolled faces. 2. Overview of the system As shown in Figure 1, the system is composed of three major components: a face detector, a normalization com- ponent, and a bimodal 3D/2D component for identification and verification. 3D Scanner Face detection Face normalization verification or identification Figure 1. The main components of the face recognition system. The face detector analyzes the curvature of the surfaces in the 3D representation of the scene acquired by the scan- ner. Salient facial features, such as the nose and eyes are easily characterized in terms of curvature statistics. After the features have been detected, they are grouped in triplets of two eyes and one nose. The geometric relationships be- tween the elements of each triplet are considered, and if these reveal the characteristics of human face the triplet is included in a set of facial candidates. The surface sur- rounded by each candidate is then processed by a PCA- based classifier trained to discriminate between faces and non-faces.