On Channel Reliability Measure Training for Multi-Camera Face Recognition Binglong Xie, Visvanathan Ramesh, Ying Zhu Terry Boult Real-Time Vision and Modeling Dept. Department of Computer Science Siemens Corporate Research University of Colorado Princeton, NJ 08540 Colorado Springs, CO 80933 Abstract Single-camera face recognition has severe limitations when the subject is not cooperative, or there are pose changes and different illumination conditions. Face recognition us- ing multiple synchronized cameras is proposed to overcome the limitations. We introduce a reliability measure trained from examples to evaluate the inherent quality of channel recognition. The recognition from the channel predicted to be the most reliable is selected as the final recognition re- sults. In this paper, we enhance Adaboost to improve the component based face detector running in each channel as well as the channel reliability measure training. Effective features are designed to train the channel reliability mea- sure using data from both face detection and recognition. The recognition rate is far better than that of either single channel, and consistently better than common classifier fu- sion rules. 1. Introduction Traditionally face recognition was performed on 2D im- ages, often frontal or near-frontal view faces. There are statistical methods including PCA (Principal Component Analysis, or Eigenfaces) [1] and LDA (Linear Discriminant Analysis, or Fisherfaces) [2], Neural Network approaches, EBGM (Elastic Bunch Graph Matching) [3] and so on. In general 2D face recognition methods suffer from pose and illumination changes, because they rely on seen images while the same face can generate novel image instances by varying the pose and lighting conditions. 3D face recog- nition methods include range-based recognition [4], stereo reconstruction [5], SFS (Shape From Shading) [6], 3D mor- phable model [7], etc. The 3D reconstruction used in these methods is often either intrusive, slow, inaccurate, or re- quiring manual initialization, and is not appropriate for real- time applications. Currently face recognition still has some severe limitations in typical applications like surveillance and access control, for example, when the subject is not co- operative and turns away from the camera, the accuracy of face recognition can be marred significantly [8]. We propose a face recognition system using two cam- eras [9]. In each channel, component-based face detector that is trained with a modified AdaBoost algorithm detects faces with pose and illumination changes and LDA-based face recognition is performed to recognize the normalized faces. The recognitions from the two channels are fused to get the final results, using a selection scheme based on a channel reliability measure trained inherent to the individ- ual channel performance. In this paper, we mainly discuss a modified AdaBoost algorithm and effective weak classifier design for reliability measure training. In selecting a weak classifier for the cur- rent strong classifier, the modified AdaBoost tries to lower the empirical training error instead of merely minimizing the error bound as in AdaBoost. Our experiments show that it provides an alternative way to AdaBoost with better per- formance. We also design effective features from face de- tection and recognition to train the channel reliability mea- sure using the modified AdaBoost. The recognition rate of this approach is far better than that of either single channel, and consistently better than common classifier fusion rules. The paper is organized as follows. Section 2 revisits the basic AdaBoost algorithm, as this is referred to throughout the paper. Section 3 briefly introduces the system we use to perform multi-camera face recognition. Section 4 describes the modified AdaBoost algorithm. Section 5 shows how the features and rules are designed to train the reliability measure. In Section 6 we show the experiments and results of the modified AdaBoost, reliability measure training, and system performance. We then conclude in Section 7. 2. AdaBoost The AdaBoost learning algorithm was first introduced by Freund et al[10]. In AdaBoost, the rough base rules are called the weak classifiers, and they are combined into the strong classifier that is much more accurate than any of the individual weak classifiers. For two-class AdaBoost, the sample set {(x i ,y i )},i = 1, ..., m contains pairs of data x i and its label y i , where x i ∈ X , and y i ∈ Y = {-1, +1}. X is the sample data space,