Affective Interaction with an Expressive Robot Nadia Magnenat Thalmann Zerrin Kasap Maher Ben Moussa MIRALab – CUI Robots that interact with humans just like other humans have forever t d th i i ti f i fi ti it d h Nadia Magnenat-Thalmann, Zerrin Kasap, Maher Ben Moussa As the emotion engine is suitable for any kind of artificial character we first www.miralab.unige.ch Prof. Nadia M-Thalmann, thalmann@miralab.unige.ch captured the imagination of science fiction writers and researchers. Interaction with robots via facial expressions and animation is important because humans intuitively give a lot of importance to facial expressions when they communicate with other humans. In our research, we introduce Socially Interactive Robot Eva which has the capability of doing human-like emotions and facial expressions (Fig1). Our research on affective modeling and facial animation on the robotic face is applied for a museum guide robot in the European research project INDIGO [1] There are three primary components in this research: As the emotion engine is suitable for any kind of artificial character, we first test it on virtual characters and then transfer it to the robotic character. Fig4 shows the interaction architecture integrating different software components. Facial Animation for the Robotic Head As a final output, the blender in the interaction architecture creates facial animation for virtual characters. The facial animation is described via MPEG-4 Facial Animation Parameters (FAPs) (Fig5a) The FAPs are then INDIGO [1]. There are three primary components in this research: 1. Mechanics of the face – the bones, muscles, skin that make the expressions on the face and the subtle movements and gestures physically possible. 2. The mind that processes the emotion, mood, personality and relationships of the robot and triggers the mechanics 3. Facial animation - which creates the animation for the virtual character and converts it to animation on the robotic head Figure 2: Virtual and robotic Eva with facial expressions Figure 1: A student interacting with Eva MPEG-4 Facial Animation Parameters (FAPs) (Fig5a). The FAPs are then mapped to the movements of the servo-motors (Fig5b) that control the facial expressions of the robotic face thus transferring the facial animation from virtual to the robot using a conversion algorithm (Fig5c) [3]. Fig2 shows various facial expressions applied on the virtual and robotic Eva. References [1] D. Vogiatzis, C. Spyropoulos, S. Konstantopoulos, V. Karkaletsis, Z. Kasap C Matheson and O Deroo An Affective Robot Guide to Museums Mechanics of the face For our research, Hanson Robotics created a custom robotic face with 32 degrees of freedom representing the major 48 muscle actions of the human face. The skin of the robot is distinctive, being made from a structured porosity elastomer material developed by Hanson Robotics, called Frubber. Affect Simulation Kasap, C. Matheson and O. Deroo. An Affective Robot Guide to Museums . Fourth International Workshop on Human-Computer Conversation, October 2008 [2] Z. Kasap, M. Benmoussa, P. Chaudhuri, N. Magnenat-Thalmann. Making Them Remember—Emotional Virtual Characters with Memory. IEEE Computer Graphics and Applications, 29, 2, pp. 20-29, March 2009 [3] N. Magnenat-Thalmann, Z. Kasap and M. Benmoussa. Communicating with a virtual human or a skin-based robot head. 8th IEEE International Conference on Automatic Face and Gesture Recognition, Amsterdam, Netherlands, 2008 Affect Simulation The second aspect that models the emotions, moods and personality of the robotic character are derived from an emotion engine that we have developed [2] (Fig3). An artificial character with this emotion engine has a personality much like a human being has and is also able to emotionally respond to the user during a conversation. The emotion engine has a memory model that uses the memory of past interactions to build long-term relationships between the artificial characters and human users. Figure 3: Virtual Eva with mood & relationship model Acknowledgement We would like to thank Nedjma-Cadi Yazli for creating the facial expression database and Marlène Aravelo-Poizat for preparing the poster. The work described in this poster is funded by the European research project INDIGO (IST-045388). Figure 4: Interaction Architecture Figure 5: a) MPEG-4 Facial Animation Parameters b) Servo-motors of the robotic head c) Conversion from MPEG-4 animation to robotic animation http://www.ics.forth.gr/indigo/