Streaming Face Recognition using Multicamera Video Arrays zyx sec/uence zyxwvu of Clussificutiori Results zyxwvut Kohsia S. Huang and Mohan M. Trivedi Computer Vision and Robotics Research (CVRR) Laboratory University of California, San Diego zyxw Sequence of Feuture Vectors Abstract zyxwvutsrq In this paper we present face recognition schemes based on video streams: the majority decision rule and HMM maximum likelihood (ML) decision rules. PCA type of subspace feature analysis is first applied to the face images in a video segment of fixed number of frames. Majority decision rule is then applied to PCA recognition results in the video segment. Discrete HMM (DHMM) is also applied to the single-frame recognition sequences. Continuous density HMM (CDHMM) is applied directly to the sequence of PCA feature vectors for ML decision on the video segment in a delayed decision manner. Experimental results ure compared between these three schemes in terms of the number of states and Gaussian mixtures of the HMMs. CDHMM-based decision rule achieved a 99% correct recognition rate in average. A geometric interpretation of ML in the feature subspace well explains the observed pegormances. 1 Introduction Intelligent rooms are systems which automatically derive and continuously aware of the space, the composition, and activities taking place in them [5][8]. An important requirement for the systems is to let the humans do their activities naturally. This design guideline places many challenges on the computer vision algorithms, especially for face recognition algorithms. In intelligent room applications, single-frame based face recognition algorithms [3][6][9] are hardly robust enough under unconstrained situations such as free human motion, uneven illumination, different backgrounds, etc. Several efforts have devoted to loose the operation constraints [1][2][6], yet they only cope with limited situations. On the other hand, video based face recognition would improve performance by accumulating visual information over time. Existing methods are based on mutual subspace method zyxwvutsrq [ 1 I] and incremental decision tree [IO]. In this paper we propose a different approach by combining subspace feature analysis like PCA [9] and time series modeling like Hidden Markov Models URL of CVRR Lab. is hrrii://c\.rr.uCsd.edu/. The authors can be reached by email at khuaiie@ucsct.cdu and tri\,edi~:ece.ucsd.cdu. (HMMs) [7]. Features or recognition results of the face images in a video sequence are collected and classified by maximum likelihood (ML) rules. Then these streaming face recognition (SFR) decision rules are compared experimentally in our intelligent room testbed zyx [4] on omnidirectional video array. Finally we seek explanations to the experimental results. 2 Streaming face recognition (SFR) schemes Given a face image stream Str = vi, f2, zyx j,, . ..}, it is first partitioned into overlapping or non-overlapping segment sequences of fixed length L, S, = {fl, f, ,... , f,}, , S, zyxwvut c Str , i = 1,2,3 ,... . Also suppose the faces in Str belong to M individuals I ={1,2 ,...,AI}. The SFR schemes are shown in Figure 1. We use PCA-type single- frame subspace feature analysis. The collected segment sequences are classified by the majority decision rule and the HMM maximum likelihood (ML) rules. Face Video Stream Decision Decision Decision 2.1 Single-frame subspace feature analysis Our single-frame feature analysis is an alternation to the standard PCA or eigenface [6][9] method. The major differences are: (1) the eigenvector basis is generated by 1051-4651/02 $17.00 zyxwvutsrq 0 2002 IEEE 2 13