Online local learning algorithms for linear discriminant analysis G.K. Demir a,b, * , K. Ozmehmet b a Department of Computer Science and Engineering, University of Minnesota, Minneapolis, MN 55455 USA b Department of Electrical and Electronics Engineering, Dokuz Eylul University, TR35160 Izmir, Turkey Available online 13 September 2004 Abstract Online local learning algorithms for a laterally-connected single-layer neural network for performing linear discri- minant analysis have been proposed. A convergence proof is provided for the algorithm based on Hebbian learning. The algorithms are simulated and applied to the face recognition problem. Ó 2004 Elsevier B.V. All rights reserved. Keywords: Linear discriminant analysis; Generalized eigenvalue problem; Online local learning 1. Introduction Linear discriminant analysis (LDA) is a well- known statistical method, first proposed by Fisher (1936), for extracting the features to classify the samples. In LDA, the data are projected into a lower dimensional subspace in a way increasing the separation of classes. For that purpose, the method enhances difference between the means rel- ative to some measure of standard deviation for each class (Fukunaga, 1990). This is accomplished by introducing the scatter matrices, which are the within-class scatter matrix S w , the between-class scatter matrix S b , and the total scatter matrix S m . In general, LDA makes use of certain criteria, composed of the scatter matrices and a transfor- mation matrix W, e.g. tr[W T S b W]/tr[W T S m W], det[W T S b W]/det[W T S w W] and tr[W T (S w + S b )W] whose optimal solution requires solving the gener- alized eigenvalue problem (GEP): S b W = KS m W, K being a diagonal matrix. Thus solving the LDA is equivalent to solving the GEP with real symmetric matrices. Although the numerical meth- ods for these problems have been well studied (Golub and Loan, 1993) there has remained still some need to online and real-time solutions for 0167-8655/$ - see front matter Ó 2004 Elsevier B.V. All rights reserved. doi:10.1016/j.patrec.2004.08.005 * Corresponding author. Address: Department of Electrical and Electronics Engineering, Dokuz Eylul University, TR35160 Izmir, Turkey. E-mail addresses: gdemir@cs.umn.edu (G.K. Demir), ozmehmet@eee.deu.edu.tr (K. Ozmehmet). Pattern Recognition Letters 26 (2005) 421–431 www.elsevier.com/locate/patrec