Majlesi Journal of Electrical Engineering Vol. 6, No. 2, June 2012 14 Facial Expression Recognition Using Temporal Templates Mahdi Bejani 1 , Davood Gharavian 2 , Nasrollah Moghaddam Charkari 3 1- South Tehran Branch, Islamic Azad University, Tehran. Iran. Email: St_m_bejani@azad.ac.ir 2- Assistant Professor of EE Department, Shahid Abbaspour University, Tehran, Iran. Email: gharavian@pwut.ac.ir 3- Distributed Processing LAB, Tarbiat Modares University, Tehran, Iran. Email: charkari@modares.ac.ir Received: November 2011 Revised: April 2012 Accepted: May 2012 ABSTRACT: To make human–computer interaction (HCI) more natural and friendly, it would be beneficial to give computers the ability to recognize situations the same way a human does. Naturally, people use a spontaneous combination of face, body gesture and speech to express their feelings. In this paper, we simulate human perception of emotion with emotion related information from facial expression and facial expression recognition based upon ITMI and QIM, which can be seen as an extension to temporal templates. The system was tested on two different databases, the eNterface‘05 and the Cohn-Kanade face database and the recognition accuracy of our systems 71.8 % on Cohn- Kanade and 39.27% on eNterface’05, compared to the published results in the literature. KEYWORDS: Human computer interaction, Temporal templates, Emotion recognition. 1. INTRODUCTION Humans communicate with each other far more naturally than they do with machines. Human-computer interaction (HCI) cannot be same as face-to-face interaction until HCI designs, involve traditional interface devices such as the keyboard and mouse are constructed to emphasize the transmission of explicit messages while ignoring implicit information about the user, such as changes in the affective state. To make human–computer interaction more natural and friendly, it would be beneficial to give computers the ability to recognize situations the same way a human does. Naturally, People use a spontaneous combination of face, body gesture and speech to express their feelings. Depending on the environment in which the interaction takes place and the subjects themselves, this combination takes a wide variety of patterns [1]. A large number of studies in psychology and linguistics confirm the correlation between some affective displays and specific audio and visual signals [2, 3]. Affective computing is the art of enabling computers to understand human’s affective states and respond in the same way [4]. The goal of this paper is to simulate human perception of emotion with emotion related information from facial expression. In next work, we want to work on emotion recognition system based on speech, and then we will try to combine them in different ways. Several models for quantifying and measuring emotions have been proposed. The most popular example of this model is the prototypical (basic) emotion categories, which include happiness, sadness, fear, anger, disgust, and surprise. Since the model of the basic emotions is universal, it is easy to understand and quantify. The main advantage of a category representation is that people use this categorical scheme to describe observed emotional displays in daily life [5]. In video databases, one of the important methods for describing the video scene is utilization of space and time relation between objects in the scene. In this paper facial expression recognition based upon ITMI and QIM [6] is used, which can be seen as an extension to temporal templates. Temporal templates are 2D images, constructed from image sequences, which show motion history; that is, where and when motion in the image sequence has occurred. A drawback innate to temporal templates proposed originally by Bobick and Davis [7] is the problem of motion self occlusion due to overwriting. The remainder of this paper is organized as follows: Section 2 contains a review of the recent researches in the field. Section 3 covers the temporal templates and facial expression system and section 4 contain the experimental results. Finally, conclusions are drawn in Section 5. 2. RELATED WORK Because of the importance of face in emotion expression and perception, most of the vision-based