Learning Neighborhood Discriminative Manifolds for Video-based Face Recognition John See 1 and Mohammad Faizal Ahmad Fauzi 2 1 Faculty of Information Technology, Multimedia University, Persiaran Multimedia, 63100 Cyberjaya, Selangor, Malaysia. 2 Faculty of Engineering, Multimedia University, Persiaran Multimedia, 63100 Cyberjaya, Selangor, Malaysia. {johnsee,faizal1}@mmu.edu.my Abstract. In this paper, we propose a new supervised Neighborhood Discriminative Manifold Projection (NDMP) method for feature extrac- tion in video-based face recognition. The abundance of data in videos often result in highly nonlinear appearance manifolds. In order to ex- tract good discriminative features, an optimal low-dimensional projec- tion is learned from selected face exemplars by solving a constrained least-squares objective function based on both local neighborhood geom- etry and global manifold structure. The discriminative ability is enhanced through the use of intra-class and inter-class neighborhood information. Experimental results on standard video databases and comparisons with state-of-art methods demonstrate the capability of NDMP in achieving high recognition accuracy. Keywords: Manifold learning, feature extraction, video-based face recog- nition 1 Introduction Recently, manifold learning has become an increasingly growing area of research in computer vision and pattern recognition. With the rapid development in imag- ing technology today, it plays an important role in many applications such as face recognition in video, human activity analysis and multimodal biometrics, where the abundance of data often demands better representation. Typically, an image can be represented as a point in a high-dimensional image space. However, it is common presumption that the perceptually mean- ingful structure of the data lies on or near a low-dimensional manifold space [1]. The mapping between high- and low-dimensional spaces is accomplished through dimensionality reduction. This remains a challenging problem for face data in video, where large complex variations between face images can be better represented by extracting good features in the low-dimensional space. In this paper, we propose a novel supervised manifold learning method called Neighborhood Discriminative Manifold Projection (NDMP) for feature extrac- tion in video-based face recognition. NDMP builds a discriminative eigenspace