Rotated Profile Signatures for Robust 3D Feature Detection Timothy C. Faltemier, Kevin W. Bowyer, and Patrick J. Flynn Abstract— While recent years have seen progress in face recognition from 3D images, non-frontal head pose is still a challenge to existing techniques. We introduce a new system for 3D face recognition that is robust to facial pose variation. Large degrees of facial pose variation may lead to a significant fraction of the features visible in frontal images being occluded. High accuracy automatic feature and pose detection is performed by a new technique called Rotated Profile Signatures (RPS). Experiments are performed on the largest available database of 3D faces acquired under varying pose. This database contains over 7,300 total images of 406 unique subjects gathered at the University of Notre Dame. Experimental results show that the RPS detection algorithm is capable of performing nose detection with greater than 96.5% accuracy across the pose variation represented in the data set used. I. I NTRODUCTION Scenarios containing differences in expression, pose, and lighting must be addressed for 3D face recognition to become successful in less constrained environments. Many current 3D face recognition algorithms are able to automatically locate fiducial points based on the assumption that they will be provided a frontal image with a neutral expression. For example, one approach for performing nose detection is to assume that it is the point closest to the camera [4], [2]. Such a heuristic enables fast detection of the hypothesized nose tip. Under non-frontal pose, however, the heuristic can fail. Figure 1 shows examples where this assumption is valid (a,b) and when it is not (c,d). Another approach to automatic nose detection is to use the curvature information at each point on the face [5], [6], [7], [8], [9]. The use of curvature information solves many problems associated with changes in pitch (up/down). However, noise, holes, and changes in yaw (left/right) can be problematic. The Iterative Closest Point (ICP) matching algorithm used in many 3D face recognition systems has been shown to perform poorly when an accurate initial alignment is not provided. Many times, the initial alignment is provided by automatically detected feature points in the image. This suggests that an application’s recognition performance is often limited by the accuracy of the feature detection module. In this paper, we describe a system for 3D face recognition that is more robust to non-frontal face poses. It includes an algorithm that is able to locate the nose tip automatically in the presence of pose or expression variation and occlusion with a high level of accuracy using a technique we call Rotated Profile Signatures (RPS). This method rotates the T. Faltemier is with Progeny Systems Corporation, 9500 Innovation Drive, Manassas VA 20110, USA tfaltemier@progeny.net Kevin W. Bowyer and Patrick J. Flynn are with the Department of Computer Science and Engineering, University of Notre Dame Notre Dame IN 46545, USA kwb@cse.nd.edu and flynn@cse.nd.edu (a) Frontal 2D (b) Frontal 3D (c) Non-Frontal 2D (d) Non-Frontal 3D Fig. 1. Nose detection examples using frontal and non-frontal images (Subject 04385). The nose tip detected by the RPS algorithm is signified by the blue sphere, and the nose tip located by the Z-heuristic is signified by the green sphere. 3D face over a 180 ◦ interval in 5 ◦ increments, extracting the rightmost “profile” points on the image at each step. These profiles are then matched to a variety of nose models, as described in Section IV, resulting in a similarity score. As the nose is rotated into view, the similarity score reaches a minimum, indicating the correct nose location. The remainder of the paper is organized as follows. Section II gives an overview of related work in the area of 3D face recognition. Section III discusses experimental materials and methods. The RPS method for efficient and accurate nose detection is introduced and results attained on the NDOff2007 data set are discussed in Section IV. Finally, Section V provides conclusions and discussion. II. RELATED WORK A recent survey of research on face recognition using 3D data is given in [12]. This section focuses on selected prior work that is most closely related to 3D feature detection and recognition. Lu et al. [13] propose a method of feature extraction based on the directional maximum in a 3D image. A nose profile is represented by different subspaces and a nearest neighbor approach is used to select the best candidates for the nose tip. Of the nose candidates, the point that best fits the statistical feature location model (i.e. the nose should be below the 978-1-4244-2154-1/08/$25.00 ©2008 IEEE