Representation of Expressivity for Embodied Conversational Agents Catherine Pelachaud LINC-Paragrape IUT of Montreuil University Paris VIII c.pelachaud@iut-univ.paris8.fr Vincent Maya LINC-Paragrape IUT of Montreuil University Paris VIII v.maya@iut-univ.paris8.fr Myriam Lamolle LINC-Paragrape IUT of Montreuil University Paris VIII m.lamolle@iut-univ.paris8.fr Abstract We aim at creating an Embodied Conversational Agent (ECA) that would exhibit not only a consistent behavior with her personality and contextual environment factors but also that would be defined as an individual and not as a generic agent. The behavior of an agent depends not only on factors defining her individuality (such as her culture, her social and professional role, her personality and her experi- ence), but also on a set of contextual (such as her interlocu- tor, the social conversation setting), and dynamic variables (belief, goal, emotion). We call these types of factors ‘influ- ences’ in the sense that they affect the behaviors to be dis- played. As humans vary greatly in their way in expressing a given meaning, there exist many manners to map a mean- ing into a set of signals. Moreover influences may act at dif- ferent levels: they may act not only on the selection of a non-verbal behavior to convey a meaning but also on its ex- pressivity. In this paper we present how we model influences working on the signals. We also describe our computational model of the agent’s expressivity. 1. Introduction We aim at creating an Embodied Conversational Agent (ECA) that would exhibit a consistent behavior with her personality and with contextual environment factors. The behavior of an agent depends not only on factors defining her individuality (such as her culture, her social and pro- fessional role, her personality and her experience), but also on a set of contextual (such as her interlocutor, the social conversation setting), and dynamic variables (belief, goal, emotion). We call these types of factors ‘influences’ in the sense that they affect the behaviors to be displayed. Influ- ences may act at different levels: they may act on what to say and when as well on how to say it and to express it. Thus influences may act not only on the selection of a non- verbal behavior to convey a meaning (i.e. on the choice of the signals) but also on its expressivity (e.g. on their inten- sity level), in order to qualify it or to accentuate it. In this paper we present how we model influences and the agent’s expressivity. We do not aim at modelling the dif- ferent factors (such as culture, personality, profession) that characterize an agent. Our work does not intend either to model how different agents would differ in their emotional reaction to an event, what culture or personality mean in their emotional reaction or to model where does a certain type of behavior come from. We are interested in modelling how a given communicative act would be expressed quan- titatively and qualitatively. Expressing a given communica- tive act may be done via discourse, choice of words, into- nation, attitude as well as particular choice of non verbal behaviors. Currently we are limiting our scope to the con- sideration of non-verbal signals. We are working at the sig- nal level and not at the discourse level and are concerned solely with external qualities of behaviors. In this paper we limit our scope at representing influences that would mod- ify the set of behaviors an agent will display to communi- cate, for e.g., a given meaning, goal within a specific con- text. We are aware that our work fully rely on the modelling of complex factors such as culture, role in a society and the like. We have elaborated a computational model of expres- sivity. We have left on the side the modelling of the what, why, how and where does come from expressivity. In the next section we describe our approach. In section 4 we present a taxonomy of influences. We then provide a general overview of an agent system. In section 6 we turn our attention to the modelling of expressivity. Finally we end the paper by presenting a state of the art. We then con- clude by describing what remains to be done. 2. Our approach Conversation is made of action (the act of speaking) and perception (the act of listening). Speaker and hearer adapt each other behaviors as the interaction evolves. Interaction involves not only speech but also non-verbal behaviors. A