MEASURING DIFFERENCES BETWEEN FACES Manuele Bicego, Andrea Lagorio, Enrico Grosso, Massimo Tistarelli DEIR / DAP - University of Sassari Computer Vision Laboratory via Sardegna 58 - 07100 Sassari piazza Duomo 6 - 07041 Alghero (SS) - Italy tista@uniss.it ABSTRACT This paper presents a novel approach for extracting charac- teristic parts of a face. Rather than finding a priori specified features such as nose, eyes, mouth or others, the proposed approach is aimed at extracting from a face the most dis- tinguishing or dissimilar parts with respect to another given face, i.e. at “finding differences” between faces. This is ac- complished by feeding a binary classifier by a set of image patches, randomly sampled from the two face images, and scoring the patches (or features) by their mutual distances. In order to deal with the multi-scale nature of natural facial features, a local space-variant sampling has been adopted. 1. INTRODUCTION Automatic face analysis is an active research area, whose interest has grown in the last years, for both scientific and practical reasons: on one side, the problem is still open, and surely represents a challenge for Pattern Recognition and Computer Vision scientists; on the other, the stringent secu- rity requirements derived from terroristic attacks have driven the research to the study and development of working sys- tems, able to increase the total security level in industrial and social environments. One of the most challenging and interesting issue in auto- matic facial analysis is the detection of the “facial features”, intended as characteristic parts of the face. As suggested by psychological studies, many face recognition systems are based on the analysis of facial features, often added to an holistic image analysis. The facial features can be either ex- tracted from the image and explicitly used to form a face representation, or implicitly recovered and used such as in the PCA/LDA decomposition or by applying a specific clas- sifier. Several approaches have been proposed for the extrac- tion of the facial features [1]. In general terms, all feature extraction methods are de- voted to the detection of a priori specified features or gray level patterns such as the nose, eyes, mouth, eyebrows or other, non anatomically referenced, fiducial points. Never- theless, for face recognition and authentication, it is neces- sary to also consider additional features, in particular those features that really characterize a given face. In other words, in order to distinguish the face of subject “A” from the face of subject “B”, it is necessary to extract from the face image of subject “A” all features that are significantly different or even not present in face “B”, rather than extract standard patterns. This paper presents a novel approach towards this direc- tion, aiming at “finding differences” between faces. This is accomplished by extracting from one face image the most distinguishing or dissimilar areas with respect to another face image, or to a population of faces. 2. FINDING DISTINGUISHING PATTERNS The amount of distinctive information in a subject’s face is not uniformly distributed within its face image. Consider, as an example, the amount of information conveyed by the image of an eye or a chin (both sampled at the same res- olution). For this reason, the performance of any classifier is greatly influenced by the uniqueness or degree of sim- ilarity of the features used, within the given population of samples. On one side, by selecting non-distinctive image ar- eas increases the required processing resources, on the other side, non-distinctive features may drift or bias the classifier’s response. This assert is also in accordance with the mechanisms found in the human visual system. Neurophysiological stud- ies from impaired people demonstrated that the face recog- nition process is heavily supported by a series of ocular sac- cades, performed to locate and process the most distinctive areas within a face [2, 3]. In principle, this feature selection process can be per- formed by extracting the areas, within a given subject’s face image, which are most dissimilar from the same areas in a “general” face. In practice, it is very difficult to define the appearance of a “general face”. This is an abstract concept, definitely present in the human visual system, but very dif- ficult to replicate in a computer system. A more viable and practical solution is to determine the face image areas which mostly differ from any other face image. This can be per- formed by feeding a binary classifier with a set of image patches, randomly sampled from two face images, and scor- ing the patches (or features) by their mutual distances, com- puted by the classifier. The resulting most distant features, in the “face space”, have the highest probability of being the most distinctive face areas for the given subjects. In more detail, the proposed algorithm extracts, from two face images, a set of sub-images centered at random points within the face image. The sampling process is driven to cover most of the face area 1 . The extracted image patches constitute two data sets of location-independent features, each one characterizing one of the two faces. A binary Sup- port Vector Machine (SVM) [7, 8] is trained to distinguish between patches of the two faces: the computed support vec- tors define a hyperplane separating the patches belonging to the two faces. Based on the distribution of the image patches 1 A similar image sampling model has been already used in other appli- cations such as image classification (the so called patch-based classification [4, 5]) or image characterization (the epitomic analysis proposed by Joijc and Frey in [6])