Validation of Detected Facial Components For an Accurate Face Recognition Jean V. Fonou Dombeu and Jules-R. Tapamo School of Computer Science, University of KwaZulu-Natal, Durban, 4000, South Africa {dombeujv,tapamoj}@ukzn.ac.za Abstract In the face recognition research domain, features-based ap- proach have been widely used in many works during the recent years. However, only few of these have used a validation step to assess whether the detected facial components were those appropriate for the recognition. In this paper, we present an ap- proach of detecting and validating facial components from gray scale images. We first binarize the image. Thereafter, the con- nected components of the resulting image are detected and la- belled. An iterative strategy is applied to remove the irrelevant components. The iteration terminates when the remaining com- ponents are the targeted components: eyes, nose and mouth. Afterward, we compute the centroids of the detected compo- nents. The convex hull of these centroids is computed and the validity of the detected components are further assessed, by ap- plying the k-means on the features extracted from the angles at the two lowest points of the convex hull. Our approach has the advantage that it is straightforward and fast, and there is no manual interaction in choosing and extracting face components. Experiments show that our approach provides promising results as it performs automatically without any assumption about the location of face components as well as in different orientations of face. Furthermore, our work is a great contribution in the features-based face recognition research domain as the earlier detection of the wrong detected set of facial components could increase the efficiency and the speed of the recognition consid- erably. Keywords—Binarization, Connected components, Face Recognition, Validation, Convex Hull, k-means, Nearest Neighbors. 1. Introduction Face recognition techniques are classified into three categories: holistic, feature-based and hybrid techniques. Many works have been done that show that feature-based approaches are faster and more robust against variation in face orientation and illumination than holistic techniques [1, 2, 3, 4]. However, one of the main problems remains the automatic extraction and validation of face components without human intervention as well as any assumptions on the location of facial components. In [5], a component based face detection system is pre- sented. It uses two level Support Vector Machines (SVM) to detect and validate facial components. Training face images are automatically extracted from 3-D head models that provide the expected positions of the components. These expected positions are used to match the detected components to the geometrical configuration of the face. Loulia and Veikko [6] propose a method for detecting facial landmarks. In their method, edge orientations are used to construct edge maps of the image. The estimation of the orientation of local edges is done by means of a kernel with maximum response. The local oriented edges are extracted and grouped into regions representing candidates for facial landmarks. The detected candidates are further classified manually into noise or facial landmark categories. In [7] and [8], facial components are used to detect the face in an image. The components are either assumed to be the holes in the detected facial regions, features computed in given color spaces, or the darkest region of the face. A geometrical technique is used by Zoltan and Tamas [9] to detect and extract facial components. They compute the facial symmetry axis and use it to deduce the nose region based on the assumption that the region of the nose is the most vertically detailed region on a face. Afterward, the positions of the eyes and mouth are estimated from the chosen nose region. A facial segmentation method based on dialation and erosion operations is presented in [10]. Facial symmetry and relative positions among the facial features, are used to locate the face contour, mouth, nostrils and eyes. Tian and Bolle [11] present a method of detecting a neutral face. In their approach, six facial points are chosen as being the most reliable that could be extracted from a face. Thereafter, the normalized distances be- tween them are computed and used as discriminating features. Two preprocessing operations named Skin Color Similarity Map (SCSM) and Hair Color Similarity Map (HCSM) are employed in [12] to compute the coordinates of face and head regions. The SCSM is projected onto the x-axis to determine the x-coordinate of the facial region. The y-coordinate of the face and head regions are determined by projecting the SCSM and HCSM on to the y-axis. Afterward, the positions and the sizes of the facial features are estimated based on the computed coordinates of the face. In [13], a method of detecting facial features such as eyebrows, eyes, nose, mouth, and ears is presented. Facial features are determined by searching for minima in the grey value relief of the segmented facial region based on the assumption that each facial feature generates a minimum in the projection of the grey value relief or pixel grey level and the expectation that eyebrows, eyes, nostrils, mouth, and chin are ranked respectively as the first, second, third, fourth, and fifth significant minimum on the horizontal relief. Furthermore, as the number of minima is usually greater than the number of features, a geometrical technique is employed to get clues about the relative positions of facials features. A method for automatically learning face components for detection and recognition is presented by Heisele et al. [14]. Initially, an object window of fixed size is slid over the input image. Afterward, 14 referenced points are manually selected in the object window based on their 3D correspondences from a morphable model. The learning algorithm then iteratively grew small rectangles around the manually preselected reference points. The detection of facial components is carried out by searching for maximum output within the rectangular