Application Notes 10 IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | MAY 2007 1556-603X/07/$25.00©2007IEEE Mohamed Abdel-Mottaleb and Mohammad H. Mahoor University of Miami, USA Algorithms for Assessing the Quality of Facial Images I. Introduction V ideo surveillance cameras are installed in public places of many cities such as Jersey City, New Orleans, Chicago, and Los Angles in the US and in many locations around the world. The City of Tampa, Florida, used face recognition technology during the 2001 Super Bowl. The State of Col- orado is scanning the photos of drivers’ licenses into a database to match against criminal mug shots on file nationwide. “Despite complaints by privacy advo- cates, the number of surveillance cam- eras is growing and proving increasingly valuable to police for catching criminals as well as protecting against terrorists” [13]. With the tremendous increase in the number of installed video surveil- lance cameras and the deployment of face-recognition software, the demand for high performance face recognition systems is obvious. In 2005, US-VISIT's [3] study showed that most of the poor quality fingerprints encountered from frequent US-VISIT travelers were not from indi- viduals with intrinsically poor finger- prints (nicknamed “goats”), but were instead due to collection problems. The quality of captured biometric data can be improved by better sensors, better user interfaces, or by compliance with standards [5]. In the past few years, researchers developed algorithms to measure the quality of fingerprint images [12], [7] and iris images [8]. The National Institute of Standards and Technology (NIST) addressed this problem in August 2004 when it pub- lished the NIST Fingerprint Image Quality algorithm, which was designed to be predictive of the per- formance of minutiae matchers [17]. Since then, NIST has been consider- ing how quality measures should be evaluated, develop- ing quality measures for other biometrics, and considering the wider use of such measures. Recently, NIST had a work- shop to present the state of the art in this field [1]. The importance of facial image quality and its effects on the performance of face recognition systems was also consid- ered by Face Recognition Vendor Test (FRVT) protocols [14], [2]. In face recognition systems, many factors such as blurring effect, facial expres- sions, lighting conditions, head pose, and facial hair could affect the quality of the facial images. These factors could affect both the Holistic and the Geometric based face recognition techniques. In this paper, we develop algorithms for assessing the quality of facial images with respect to the effects of blurring, lighting conditions, head pose, and facial expressions . These algorithms can be used in the Quality Assessment (Q.A.) module of a face recognition system (Figure 1). As shown in Figure 1, one role of Q.A. is to assess the quality of facial images to either reject or accept them for the recognition step. Quality assessment can also be used to assign weights to different face recognition algorithms in a fusion scheme. In order to develop algorithms for assessing the quality of facial images, the challenge is to measure the level or the intensity of the fac- tors that affect the quali- ty of the facial images. For example, a facial image could have an expression intensity ranging from neutral to maximum. Obviously, the recognition of a facial image with exaggerated expres- sions is more difficult than the recogni- tion of a facial image with a light expression. For blurring, lighting condi- tions, and head pose effects, measuring the level of these factors is possible. But, measuring the intensity of a face expres- sion is difficult because of the absence of a reference neutral face image. Considering the issues discussed above, we take two different strategies to assess the quality of facial images: one strategy for blurring, lighting conditions, and head pose effects and another strate- gy for facial expressions. In the first strategy, we define measures that corre- lates with the level of degradation of the captured facial images. Based on each measure, we define a polynomial func- tion for predicting the performance of the Eigenface [18] technique on a given image; and then by selecting a suitable threshold for the predicted recognition rate, we accept or reject the image for recognition. In the second strategy for © BRAND X PICTURES