Natural Interaction expressivity modeling and analysis * George Caridakis † Intelligent Systems, Content and Interaction Lab National Technical University of Athens gcari@image.ntua.gr Konstantinos Moutselos Dept. of Computer Science and Biomedical Informatics University of Central Greece kmouts@ucg.gr Ilias Maglogiannis Department of Digital Systems. University of Piraeus imaglo@unipi.gr ABSTRACT Behavior, including non verbal, expressiveness is an inte- gral part of the communication process since it can provide information on the emotional state and the user’s perfor- mance when the aim of the interaction is measurable. Long term temporal measurements can also assist in monitoring the user for either emergencies or long term mood instabili- ties. Current article presents research work on the computa- tional formalization and analysis of full body 3D expressiv- ity in Natural (bodily) Interaction within the framework of Pervasive Assistance. Expressivity dimensions are selected as the most complete approach to body expressivity mod- eling, since they cover the entire spectrum of expressivity parameters related to emotion and affect. In this study five expressivity parameters are computationally formalized, us- ing different approaches based on silhouette, limbs position and joints rotation, for each expressivity feature. These ap- proaches are then evaluated in terms of their effectiveness in modeling the expressivity aspect in question. The mod- eling effectiveness of each approach is assessed using Linear Discriminant Analysis (LDA) and its coefficients on the au- tomatically extracted parameters, defined in the computa- tional formalization, against an experimental dataset con- sisting of extreme expressions (positive and negative) of the investigated expressivity aspects. The The experimental re- * Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions@acm.org. PETRA ”13, May 29 - 31 2013, Island of Rhodes, Greece Copyright 2013 ACM 978-1-4503-1973-7/13/05 ˇ E$15.00. http://dx.doi.org/10.1145/2504335.2504378 † also affiliated with the Dept. of Cultural Technology, Uni- versity of the Aegean sults confirm that the proposed Fading Silhouette Motion Volumes (FMSV) approach, is the most effective in model- ing body expressivity. General Terms Affective Computing, Pervasive Assistance Keywords Natural Interaction, Body expressivity, Linear Discriminant Analysis, Behavioral Analysis Categories and Subject Descriptors J.3 [Life And Medical Sciences]: Medical information systemsPerformance, Design 1. INTRODUCTION An increasing number of studies from various disciplines have shown that body expressions are as powerful as fa- cial expressions in conveying emotions [24] [23]. According to Mehrabian and Friar [17] and Wallbott [26], changes in a person’s affective state are also reflected by changes in body posture. Recently, introduced and established technologies become more and more ubiquitous and the interaction with these technologies becomes natural. A typical example is that of the computer game industry where body movement is not only a means to control the interaction between us and the games, but also a way to capture and affect our own emotional and cognitive performances. In a wider per- spective, these emotional and cognitive performances could be monitored in Pervasive Assistive context in order to serve as behavior-driven assistance or rehabilitation and patients during therapy progress evaluation tools. Alternative to conventional means of interaction, Natural Interaction (NI), and bodily interaction specifically, is in- creasingly attracting the attention of researchers in related research areas. Within Natural Interaction context body actions, movement and postures, either intentional or not, convey important emotional content, enhanced with qualita- tive expressive cues. Body motion or posture qualitative as- pects (formulated using different approaches) communicate affective and emotional content and are embodied in the di- rect and natural emotional expression of body movement [6]. Accordingly, the application areas of Natural Interaction are constantly increasing both in numbers as well as in breadth, including pervasive and assistive environments.