Emotionally expressive dynamic physical behaviors in robots Mason Bretan a,n , Guy Hoffman b , Gil Weinberg a a Center for Music Technology, Georgia Institute of Technology, 840 McMillan St., Atlanta, GA 30332, USA b Media Innovation Lab, Interdisciplinary Center Herzliya,167 Herzliya 46150, Israel article info Article history: Received 2 April 2013 Received in revised form 22 January 2015 Accepted 23 January 2015 This paper was recommended for publication by E. Motta. Available online 4 February 2015 Keywords: Human robotic interaction Emotion Affective computing Expression Sentiment analysis abstract For social robots to respond to humans in an appropriate manner, they need to use apt affect displays, revealing underlying emotional intelligence. We present an artificial emotional intelligence system for robots, with both a generative and a perceptual aspect. On the generative side, we explore the expressive capabilities of an abstract, faceless, creature-like robot, with very few degrees of freedom, lacking both facial expressions and the complex humanoid design found often in emotionally expressive robots. We validate our system in a series of experiments: in one study, we find an advantage in classification for animated vs static affect expressions and advantages in valence and arousal estimation and personal preference ratings for both animated vs static and physical vs on-screen expressions. In a second experiment, we show that our parametrically generated expression variables correlate with the intended user affect perception. Combining the generative system with a perceptual component of natural language sentiment analysis, we show in a third experiment that our automatically generated affect responses cause participants to show signs of increased engagement and enjoyment compared with arbitrarily chosen comparable motion parameters. & 2015 Elsevier Ltd. All rights reserved. 1. Introduction The ability to express emotion through nonverbal means can be an effective tool for computational and mechanical systems which interact with people. Coulson's component process view of emotions is defined as the “affective significance of a series of evaluations” (Cañamero and Aylett, 2008). This relationship between cognition and evaluation often results in some physical behavior such as a smile or scream. These physical behaviors are usually an unconscious reaction and can be considered representations of one's internal states. There is a large body of evidence which supports facial expressions and prosodic cues as being indicative of a person's internal emotional state (Ekman, 1993; Fridlund et al., 1987; Schuller et al., 2006, 2011). However, the question of whether pose and body movements are reflections of internal emotional states has been subject to debate for many years. Some studies suggest emotional body language and physical expression are used primarily for social and communicative purposes rather than an unconscious expression of internal emotion (Fridlund, 1991; Kraut and Johnston, 1979). Though in more recent years evidence has been mounting which suggests the opposite is true. There is indeed a direct relationship between external physical behavior and emotional states (Inderbitzin et al., 2011; Walbott, 1998). In fact, Aviezer et al. (2012) declare that body language instead of facial expression better broadcasts what a person is experiencing especially in circum- stances of extreme positively or negatively valenced emotions. Other research also demonstrates that gesture is useful for conveying information other than affect and is a component of the speech planning process (Alibali et al., 2000; Kita et al., 2007). In essence, gesture aids speech generation beyond lexical retrieval by helping speakers to organize and conceptualize spatial informa- tion. Movement is also important for interactive scenarios. The timing of visual cues including movement illustrators and gaze plays an important role in the collaborative process of conversa- tion (Bavelas et al., 2002; Bavelas and Chovil, 2006). In fact, Bavelas and Chovil (2000) describe an integrated model of com- munication which unifies the visible and audible components of face-to-face dialogue. Though the debate concerning the true function of body pose and body movements remains, and it appears the use of gesture has many functions, it is largely accepted that, at some level, people are able to associate postures and movements with particular emotions (de Gelder, 2006; Nele Dael and Scherer, 2012; Coulson, 2004; Krauss et al., 1991; Kipp and Martin, 2009). There is even evidence of the brain processing emotional body language unconsciously and without reliance on the primary visual cortex (de Gelder and Hadjikhani, 2006). The human ability to recognize emotion through body lan- guage is an important trait and quite relevant to the field of affective computing. Additionally, the capacity for processing emotion through Contents lists available at ScienceDirect journal homepage: www.elsevier.com/locate/ijhcs Int. J. Human-Computer Studies http://dx.doi.org/10.1016/j.ijhcs.2015.01.006 1071-5819/& 2015 Elsevier Ltd. All rights reserved. n Corresponding author. Tel.: þ1 415 246 6475. E-mail address: masonbretan@gmail.com (M. Bretan). Int. J. Human-Computer Studies 78 (2015) 1–16