Real-time Action Selection for ECA Listeners Etienne de Sevin, Elisabetta Bevacqua and Catherine Pelachaud CNRS - Telecom ParisTech 37/39, rue Dareau 75014 Paris, France {de-sevin, bevacqua, pelachaud}@telecom-paristech.fr Abstract We aim to build a real-time Embodied Conversational Agent able to generate automatically verbal and non verbal backchannels that a human interlocutor displays during an interaction. In this paper, we propose a backchannel selection algorithm working in real-time to choose the more appropriate backchannels to display among the possible conflicting ones according to the level of user’s interest perceived by the agent. Keywords: Backchannels, User’s interest level, Personality, Action selection, ECAs 1. Introduction To be believable, virtual agents have to deal with the problem of action selection which can be resumed to decide what to do next according to the internal and external variables of the agent [1]. This work is part of the EU SEMAINE project in which a real-time Embodied Conversational Agent (ECA) will be a Sensitive Artificial Listener (SAL) [2]. The SEMAINE project partners [3] provide us the interest level of the user by interpreting the audio (microphone). We implement an action selection to listener ECA in order to choose the right backchannel [4] to display according to the user behaviors. In this paper, we want to study the influence of the interest level of the user on the selection of backchannels. 3. Action selection algorithm Fig. 1. Schematic view of the Backchannel architecture including a BC selection module Backchannels can be potentially conflicting because only one signal can be displayed on one modality of the agent (i.e. head). To resolve this conflict, we use the interest level of the user. We integrate the value of the user interest level provided by the SEMAINE partner [3] into a temporal window in order to neglect the non significant variations. The selection algorithm has to choose between two types of backchannels (see figure 1). Mimicry is chosen preferentially when the agent perceives that the user is very interested in the interaction so that the agent can show its high engagement in the interaction [5]. When fully engaged in an interaction, mimicry of behaviors may happen [6]. Reactive Backchannels ECA Backchannel Selection Mimicry Backchannels Perceived User’s Level of Interest Triggering of backchannels User Agent’s Mental State