Multi-view Discriminant Analysis Meina Kan 1 , Shiguang Shan 1 , Haihong Zhang 2 , Shihong Lao 2 , and Xilin Chen 1 1 Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS, Beijing, 100190, China 2 Omron Social Solutions Co., LTD., Kyoto, Japan {meina.kan,shiguang.shan,xilin.chen}@vipl.ict.ac.cn, angelazhang@ssb.kusatsu.omron.co.jp, lao@ari.ncl.omron.co.jp Abstract. The same object can be observed at different viewpoints or even by different sensors, thus generating multiple distinct even heteroge- neous samples. Nowadays, more and more applications need to recognize object from distinct views. Some seminal works have been proposed for object recognition across two views and applied to multiple views in some inefficient pairwise manner. In this paper, we propose a Multi-view Discriminant Analysis (MvDA) method, which seeks for a discriminant common space by jointly learning multiple view-specific linear trans- forms for robust object recognition from multiple views, in a non-pairwise manner. Specifically, our MvDA is formulated to jointly solve the multi- ple linear transforms by optimizing a generalized Rayleigh quotient, i.e., maximizing the between-class variations and minimizing the within-class variations of the low-dimensional embeddings from both intra-view and inter-view in the common space. By reformulating this problem as a ra- tio trace problem, an analytical solution can be achieved by using the generalized eigenvalue decomposition. The proposed method is applied to three multi-view face recognition problems: face recognition across poses, photo-sketch face recognition, and Visual (VIS) image vs. Near Infrared (NIR) image face recognition. Evaluations are conducted respectively on Multi-PIE, CUFSF and HFB databases. Intensive experiments show that MvDA can achieve a more discriminant common space, with up to 13% improvement compared with the best known results. Keywords: Multi-view Discriminant Analysis, Multi-view Face Recog- nition, Common space for Multi-view. 1 Introduction In many computer vision applications, the same object can be observed at dif- ferent viewpoints or even by different sensors, thus generating multiple distinct even heterogeneous samples. For example, given a face, photos can be taken from different viewpoints, resulting multi-pose face images [1]; a face can be also illu- minated by visible lighting or near infrared lighting to capture visual images or near infrared images respectively [2]. Recently, more and more applications need A. Fitzgibbon et al. (Eds.): ECCV 2012, Part I, LNCS 7572, pp. 808–821, 2012. c  Springer-Verlag Berlin Heidelberg 2012