Recognising Human Emotions from Body Movement and Gesture Dynamics Ginevra Castellano 1 , Santiago D. Villalba 2 , and Antonio Camurri 1 1 Infomus Lab, DIST, University of Genoa ⋆ 2 MLG, School of Computer Science and Informatics, University College Dublin ginevra.castellano@unige.it, santiago.villalba@ucd.ie Abstract. We present an approach for the recognition of acted emo- tional states based on the analysis of body movement and gesture ex- pressivity. According to research showing that distinct emotions are often associated with different qualities of body movement, we use non- propositional movement qualities (e.g. amplitude, speed and fluidity of movement) to infer emotions, rather than trying to recognise different gesture shapes expressing specific emotions. We propose a method for the analysis of emotional behaviour based on both direct classification of time series and a model that provides indicators describing the dynamics of expressive motion cues. Finally we show and interpret the recognition rates for both proposals using different classification algorithms. 1 Introduction One critical aspect of human-computer interfaces is the ability to communicate with users in an expressive way [1]. Computers should be able to recognise and interpret users’ emotional states and to communicate expressive-emotional in- formation to them. Recently, there has been an increased interest in designing automated video analysis algorithms aiming to extract, describe and classify in- formation related to the emotional state of individuals. In this paper we focus on video analysis of movement and gesture as indicators of an underlying emotional process. Our research aims to investigate which are the motion cues indicating dif- ferences between emotions and to define a model to recognise emotions from video analysis of body movement and gesture dynamics. The possibility to use movement and gesture as indicators of the state of individuals provides a novel approach to quantitatively observe and evaluate the users in an ecological en- vironment and to respond adaptively to them. Our research is grounded in the Component Process Model of emotion (CPM) proposed by Scherer [2], and ex- plores the component of the model representing motor activation in emotion. Specifically, the CPM describes a relationship between the results of different event appraisals (such as novelty, intrinsic pleasantness, goal conduciveness etc.) ⋆ Part of this work was carried out in the context of the EU Project HUMAINE (Human-Machine Interaction Network on Emotion)(IST-2002-507422). A. Paiva, R. Prada, and R.W. Picard (Eds.): ACII 2007, LNCS 4738, pp. 71–82, 2007. c  Springer-Verlag Berlin Heidelberg 2007