IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 42, NO. 4, AUGUST 2012 1095 A New Biased Discriminant Analysis Using Composite Vectors for Eye Detection Chunghoon Kim, Member, IEEE, Sang-Il Choi, Member, IEEE, Matthew Turk, Senior Member, IEEE, and Chong-Ho Choi, Member, IEEE Abstract—We propose a new biased discriminant analysis (BDA) using composite vectors for eye detection. A composite vector consists of several pixels inside a window on an image. The covariance of composite vectors is obtained from their inner product and can be considered as a generalization of the covari- ance of pixels. The proposed composite BDA (C-BDA) method is a BDA using the covariance of composite vectors. We con- struct a hybrid cascade detector for eye detection, using Haar- like features in the earlier stages and composite features obtained from C-BDA in the later stages. The proposed detector runs in real time; its execution time is 5.5 ms on a typical PC. The experimental results for the CMU PIE database and our own real-world data set show that the proposed detector provides robust performance to several kinds of variations such as facial pose, illumination, eyeglasses, and partial occlusion. On the whole, the detection rate per pair of eyes is 98.0% for the 3604 face images of the CMU PIE database and 95.1% for the 2331 face images of the real- world data set. In particular, it provides a 99.7% detection rate for the 2120 CMU PIE images without glasses. Face recognition performance is also investigated using the eye coordinates from the proposed detector. The recognition results for the real-world data set show that the proposed detector gives similar performance to the method using manually located eye coordinates, showing that the accuracy of the proposed eye detector is comparable with that of the ground-truth data. Index Terms—Biased discriminant analysis (BDA), composite feature, composite vector, eye detection, face recognition. I. I NTRODUCTION R ECENTLY, SEVERAL studies have been performed on eye detection as a preprocessing step for face recognition [1]–[8]. After detecting faces in an image, it is necessary to align the faces for recognition. Face alignment is usually performed by warping the image so that the eye positions line up with predefined image coordinates, and the accuracy Manuscript received April 30, 2011; revised September 2, 2011 and December 28, 2011; accepted January 9, 2012. Date of publication March 6, 2012; date of current version July 13, 2012. This work was supported by the Korea Research Foundation Grant funded by the Korean Government (KRF-2007-357-D00161). C. Kim is with the the Qualcomm Research, Seoul 137-920, Korea. S.-I. Choi is with the Department of Applied Computer Engineer- ing, Dankook University, Gyeonggi-do 448-701, Korea (e-mail: csichoisi@ gmail.com). M. Turk is with the Computer Science Department, University of California, Santa Barbara, CA 93106 USA. C.-H. Choi is with the School of Electrical Engineering and Computer Science, Seoul National University, Seoul 151-744, Korea. Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TSMCB.2012.2186798 of the eye coordinates greatly affects the performance of a face recognition system [5], [9], [10]. The location of the eye is commonly measured at the iris or pupil center [11], [12]. According to recent results in the field of face recognition, state-of-the-art methods provide robust performance to several kinds of variations, such as facial expression and illumination, when using manually located eye coordinates [9], [13]. When the eye coordinates were shifted randomly, the face recognition rates degraded rapidly [9], [14], [10]. From these results, we can see that eye detection is very important in face recognition systems. However, as pointed out in [11], eye detection remains a very challenging task due to several variations. The shape and appearance of the eye vary with identity, race, viewing direction, illumination condition, eye motion, etc. Occlusions due to eyeglasses or glare on the glasses can also cause a severe problem [15]. In previous studies, appearance-based methods have been popularly used to discriminate between eyes and noneyes. The appearance-based methods use the photometric appearance as characterized by the pixel intensity distribution or filter re- sponses of the eye and its surroundings [11]. Pentland et al. used the eigeneyes, eigennoses, and eigenmouths, based on principal component analysis (PCA), to detect the eyes, nose, and mouth [4]. Huang and Wechsler used wavelet packets for eye representation and radial basis functions for subsequent classification of eyes and noneyes [2]. Ma et al. used Haar- like features to detect eyes [3]. Zhu and Ji used a support vector machine to discriminate eyes from noneyes [8]. Wang and Ji constructed a cascade detector for eye detection, using Haar-like features and features obtained from the recursive non- parametric discriminant analysis [6], [7]. Choi and Kim used a cascade detector for eye detection, using features obtained from the modified census transform (MCT) [16]. Song et al. also used a cascade detector, employing Haar-like features and rectangle features obtained from a visual-context pattern [17]. In object detection problems such as face and eye detection, a sliding window detection approach [18] is commonly used on an image pyramid because the location and size of the object are unknown. A large number of windows are used for detection, the vast majority of which do not contain target objects. In this case, a cascade detector is an efficient way to detect the object, where a simple classifier with a small number of features is used to reject the majority of detection windows at the first stage [19]–[22]. When constructing a cascade detector, Haar- like features are most frequently used in face and eye detection due to their computational efficiency. Haar-like features use binary rectangles on an image, computing feature values by 1083-4419/$31.00 © 2012 IEEE