Face Recognition Using Nearest Feature Space Embedding Ying-Nong Chen, Chin-Chuan Han, Member, IEEE, Cheng-Tzu Wang, and Kuo-Chin Fan Abstract—Face recognition algorithms often have to solve problems such as facial pose, illumination, and expression (PIE). To reduce the impacts, many researchers have been trying to find the best discriminant transformation in eigenspaces, either linear or nonlinear, to obtain better recognition results. Various researchers have also designed novel matching algorithms to reduce the PIE effects. In this study, a nearest feature space embedding (called NFS embedding) algorithm is proposed for face recognition. The distance between a point and the nearest feature line (NFL) or the NFS is embedded in the transformation through the discriminant analysis. Three factors, including class separability, neighborhood structure preservation, and NFS measurement, were considered to find the most effective and discriminating transformation in eigenspaces. The proposed method was evaluated by several benchmark databases and compared with several state-of-the-art algorithms. According to the compared results, the proposed method outperformed the other algorithms. Index Terms—Face recognition, nearest feature line, nearest feature space, Fisher criterion, Laplacianface. Ç 1 INTRODUCTION R ECENTLY, the manifold-based learning approach in face recognition has attracted a lot of researchers. He et al. [1] proposed an eigenspace method, called Laplacianface, to preserve the local structure of training samples. Using the locality preserving projection (LPP), the locality of a manifold structure is preserved by a nearest-neighbor graph (NN- graph) in face feature spaces. According to the consequences in [1], the local manifold structure preserved by the LPP method is more effective than the global euclidean structure (e.g., PCA [2] or LDA approach [3]). It is more suitable for classification in a number of applications. Moreover, an orthogonal locality preserving projection (OLPP) method enhanced by Cai et al. [4] could preserve more local information. The other nonlinear manifold structures could be generated by Isomap [5], [6], locally linear embedding (LLE) [7], [8], Laplacian Eigenmap [9], topology preserving nonnegative matrix factorization [10], and unsupervised discriminant projection (UDP) [11] approaches. In these unsupervised methods, the class information is not used in finding the transformation matrices. Similarly to the PCA approach, they work well for dimensionality reduction and sample reconstruction, but not for classification. Prior class labels were adopted to guide the procedure of discriminant analysis in many algorithms, such as the LDA [3], F-LDA [12], D-LDA [13], K-DDA [14], FD-LDA [15], and WLDA [16]. The Fisher criterion was optimized by maximizing intraclass compactness and interclass separ- ability. The modified within-class and between-class scat- ters were calculated based on LPP for preserving the local structures. The similarity matrices for describing the neighborhood relations between sample points were calcu- lated in the supervised LPP [1], the orthogonal neighbor- hood preserving discriminant analysis (ONPDA) [17], the kernel class-wise LPP [18], the marginal Fisher analysis (MFA) [19], etc. Yan et al. [19] proposed a general framework, called graph embedding, for providing a common perspective on various algorithms. Two graphs, an intrinsic graph (IG) and a penalty graph (PG), describe the intraclass point adjacency relationship and the interclass marginal point adjacency relationship in three functions: linearization, kernelization, and tensorization. All of these approaches preserve both the local intraclass and interclass neighborhood structures. In addition to the discriminant analysis in eigenspaces, other problems in face recognition have been discussed. First, a small sample size (S3) is also a problem. Many methods have been proposed to solve this problem, such as PCA plus LDA [20], null space [21], discriminative common vector(DCV) [22], and principal nonparametric subspace analysis(PNSA) [23]. Second, a limitation is assumed in the traditional LDA approach. The data distribution in each class is assumed to be of a Gaussian distribution. The performance usually degrades for non-Gaussian distribu- tions in many cases. Nonparametric discriminant analy- sis(NDA) [23], [24], MFA [19], and TMAF-SVM [31] algorithms use the features points located at the class boundaries to remove this assumption. Next, many re- searchers try to extract the regularity of face features and eliminate unwanted noises from eigenspaces. Jiang et al. [25] decomposed the within-class scatter into three subspaces: the reliable subspace, the unstable subspace, and the null IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 33, NO. 6, JUNE 2011 1073 . Y.N. Chen and K.-C. Fan are with the Department of Computer Science and Information Engineering, National Central University, No. 300, Jhongda Rd., Jhongli City, Taoyuan County 32001, Taiwan (R.O.C.). E-mail: 93542021@cc.ncu.edu.tw, kcfan@csie.ncu.edu.tw. . C.C. Han is with the Department of Computer Science and Information Engineering, National United University, No. 1, Lienda, Kung-ching Li, Miaoli City, 36003, Taiwan (R.O.C). E-mail: cchan@nuu.edu.tw. . C.T. Wang is with the Department of Computer Science, National Taipei University of Education, No. 134, Sec. 2, Heping E. Rd., Da-an District, Taipei City 106, Taiwan (R.O.C.). E-mail: ctwang@tea.ntue.edu.tw. Manuscript received 15 Nov. 2009; revised 2 May 2010; accepted 15 Sept. 2010; published online 9 Nov. 2010. Recommended for acceptance by S. Li. For information on obtaining reprints of this article, please send e-mail to: tpami@computer.org, and reference IEEECS Log Number TPAMI-2009-11-0763. Digital Object Identifier no. 10.1109/TPAMI.2010.197. 0162-8828/11/$26.00 ß 2011 IEEE Published by the IEEE Computer Society