14 Subspace Image Representation for Facial Expression Analysis and Face Recognition and its Relation to the Human Visual System Ioan Buciu 1,2 and Ioannis Pitas 1 1 Department of Informatics, Aristotle University of Thessaloniki GR-541 24, Thessaloniki, Box 451, Greece. pitas@zeus.csd.auth.gr 2 Electronics Department, Faculty of Electrical Engineering and Information Technology, University of Oradea 410087, Universitatii 1, Romania. ibuciu@uoradea.ro Summary. Two main theories exist with respect to face encoding and representa- tion in the human visual system (HVS). The first one refers to the dense (holistic) representation of the face, where faces have “holon”-like appearance. The second one claims that a more appropriate face representation is given by a sparse code, where only a small fraction of the neural cells corresponding to face encoding is activated. Theoretical and experimental evidence suggest that the HVS performs face analysis (encoding, storing, face recognition, facial expression recognition) in a structured and hierarchical way, where both representations have their own contribution and goal. According to neuropsychological experiments, it seems that encoding for face recognition, relies on holistic image representation, while a sparse image represen- tation is used for facial expression analysis and classification. From the computer vision perspective, the techniques developed for automatic face and facial expres- sion recognition fall into the same two representation types. Like in Neuroscience, the techniques which perform better for face recognition yield a holistic image rep- resentation, while those techniques suitable for facial expression recognition use a sparse or local image representation. The proposed mathematical models of image formation and encoding try to simulate the efficient storing, organization and coding of data in the human cortex. This is equivalent with embedding constraints in the model design regarding dimensionality reduction, redundant information minimiza- tion, mutual information minimization, non-negativity constraints, class informa- tion, etc. The presented techniques are applied as a feature extraction step followed by a classification method, which also heavily influences the recognition results. Key words: Human Visual System; Dense, Sparse and Local Image Repre- sentation and Encoding, Face and Facial Expression Analysis and Recogni- tion. R.P. W¨ urtz (ed.), Organic Computing. Understanding Complex Systems, doi: 10.1007/978-3-540-77657-4 14, © Springer-Verlag Berlin Heidelberg 2008