Machine Vision and Applications (2009) 20:35–46 DOI 10.1007/s00138-007-0103-1 ORIGINAL PAPER Complete discriminant evaluation and feature extraction in kernel space for face recognition Xudong Jiang · Bappaditya Mandal · Alex Kot Received: 20 December 2006 / Accepted: 21 July 2007 / Published online: 15 November 2007 © Springer-Verlag 2007 Abstract This work proposes a method to decompose the kernel within-class eigenspace into two subspaces: a reliable subspace spanned mainly by the facial variation and an unre- liable subspace due to limited number of training samples. A weighting function is proposed to circumvent undue scal- ing of eigenvectors corresponding to the unreliable small and zero eigenvalues. Eigenfeatures are then extracted by the dis- criminant evaluation in the whole kernel space. These efforts facilitate a discriminative and stable low-dimensional fea- ture representation of the face image. Experimental results on FERET, ORL and GT databases show that our approach consistently outperforms other kernel based face recognition methods. Keywords Face recognition · Kernel discriminant analysis · Feature extraction · Subspace methods 1 Introduction Face recognition has drawn considerable attention in bio- metric society because of its property of non-intrusiveness, which is recognizing a person from a distance. Numerous linear and nonlinear subspace based face recognition methods are proposed in the last two decades [25, 30, 35]. The most popular nonlinear methods are kernel principal X. Jiang (B ) · B. Mandal · A. Kot School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 639798, Singapore e-mail: exdjiang@ntu.edu.sg B. Mandal e-mail: bapp0001@ntu.edu.sg A. Kot e-mail: eackot@ntu.edu.sg component analysis (KPCA) [29] and kernel Fisher discrim- inant analysis (KFDA) [22]. These kernel based methods and their variations encode pattern information based on higher order dependencies and are tactful to the dependen- cies among pixels in the samples. The kernel mappings can capture the nonlinearities and complex relationships among the input data that exist due to the expression, illumination and pose variations. The basic idea of these kernel methods is to apply nonlin- ear mapping Φ : X ∈ R n → Φ( X ) ∈ H in the image space R n , followed by linear subspace methods like PCA and FDA in the mapped feature space H. Examples include KPCA [29] and KFDA [22, 23]. Since the feature space H can be very high or possibly infinite dimensional and the orthogonality needs to be characterized in such a space, it is reasonable to view H as a Hilbert space. It is difficult to compute the dot products in the high dimensional feature space H. Instead of mapping the data explicitly, the feature space can be com- puted by using the kernel trick, in which the inner products 〈Φ( X ij ),Φ( X st )〉 in H can be replaced with a kernel function K ( X ij , X st ), where K ( X ij , X st ) =〈Φ( X ij ),Φ( X st )〉 and X ij , X st are sample vectors in the image space R n . So, the nonlinear mapping Φ can be performed implicitly in image space R n [28, 31]. Numerous studies [8, 30, 34] demonstrate that these kernel based approaches are effective in some real-world applications. However, the basic subspace analy- sis has still outstanding challenging problems when applied to the face recognition due to the high dimensionality of the face image and the finite number of training samples in practice. Most of the kernel subspace based face recognition meth- ods perform dimensionality reduction or discard a subspace before the discriminant evaluation. A popular method called kernel Fisherface [34] applies PCA first for dimensional- ity reduction so as to make the within-class scatter matrix 123