Towards Designing Enthusiastic AI Agents Carla Viegas Stella AI Carnegie Mellon University Pittsburgh, Pennsylvania, USA cviegas@cs.cmu.edu Malihe Alikhani University of Pittsburgh Pittsburgh, Pennsylvania, USA malihe@pitt.edu ABSTRACT Immersive virtual worlds are increasingly being used for education, training, and entertainment, and virtual humans that can inter- act with human users in these worlds play many important roles. Understating the emotional constructs of the user and generating multimodal forms of communications that are aligned with the user’s needs and input is key to designing AI agents. Most virtual agents and communicative systems lack the ability to understand enthusiasm or generate multimodal enthusiastic communicative presentations. In this work, we argue for the importance of in- cluding enthusiasm in the design of humanśAI collaboration and communication and review the existing datasets and models that can be used to bridge the gap in this area. CCS CONCEPTS · Human-centered computing → Human computer interac- tion (HCI). KEYWORDS datasets, enthusiasm, virtual agents, engagement, conversational expressions ACM Reference Format: Carla Viegas and Malihe Alikhani. 2021. Towards Designing Enthusiastic AI Agents. In 21th ACM International Conference on Intelligent Virtual Agents (IVA ’21), September 14ś17, 2021, Virtual Event, Japan. ACM, New York, NY, USA, 3 pages. https://doi.org/10.1145/3472306.3478366 1 OVERVIEW We have recently observed a growing interest in the development of embodied agents that can understand user’s emotional constructs. The fuent exchange of information and the display of cognitive and emotional states is essential for establishing and maintaining en- gagement. Studies have suggested that the six basic emotions [14] cannot best represent the emotional constructs that AI agents need to work with. Conversational expressions are more fne-grained and diverse. Examples of such expressions include clueless, annoyed, and interested [6, 7, 9]. In this work, we want to call attention to enthusiasm as a conversational expression. Enthusiasm is one of the most desired traits in employees, co-workers, mentors, leaders, and teachers [2, 5, 10, 27, 31, 39]. Enthusiastic people are not only Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for proft or commercial advantage and that copies bear this notice and the full citation on the frst page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). IVA ’21, September 14ś17, 2021, Virtual Event, Japan © 2021 Copyright held by the owner/author(s). ACM ISBN 978-1-4503-8619-7/21/09. https://doi.org/10.1145/3472306.3478366 motivated and excited about a topic, they can also spark this excite- ment in their listeners and even move their audience to action [12]. Given that the ability to speak enthusiastically provides a clear advantage in human interactions when the goal is to engage the interlocutor emotionally, we shall also aim to transfer it to virtual agents. Although enthusiasm has been studied extensively in psychol- ogy, showing that students clearly beneft from enthusiastic teach- ers [5, 44] as well as companies do with enthusiastic leaders [23, 34], it is still unclear what exactly makes a person to be perceived as enthusiastic [19]. Most of the work on detecting enthusiasm au- tomatically is based on using written human-to-human conver- sational dialogues [17, 38]. Limited work on enthusiastic virtual agents and robots exist. Liew et al. showed that virtual agents with enthusiastic voices improve learning outcomes in students [24ś26], however, they used prerecorded voices from actors. Despite the limited work on detecting and generating enthusiastic behavior, the recent release of the frst multimodal dataset on enthusiasm, called Entheos [40], is not only a chance to gain more understanding on enthusiasm, but also to create enthusiastic virtual agents. In the following we will describe the opportunities for enthusias- tic virtual agents as well as the challenges that need to be addressed in order to have an engaging conversation between humans and virtual agents. We will also present some preliminary analysis on the Entheos dataset 1 and discuss how explainable AI techniques such as SHAP [28] can be used to understand which features are important for enthusiasm classifcation. 2 OPPORTUNITIES FOR ENTHUSIASTIC VIRTUAL AGENTS Although several applications of virtual agents could beneft from understanding and generating enthusiastic behavior, we will focus on three specifc use cases: a) teaching, b) coaching, and c) sales. All three applications are not purely conversational, but focus on delivering a message in an engaging way, which we believe is a feasible frst step to improving existing virtual agents. Teaching: Enthusiasm has shown to improve students’ perfor- mance not only with human teachers [5, 19, 20, 44] but also with vir- tual teachers using prerecorded enthusiastic voices [24, 26]. Given the increasing importance of pedagogical agents and robot teachers to overcome the global shortage of teachers [29], it is essential to automatically generate enthusiastic behavior during teaching to ensure students’ performance and their engagement. Coaching: Several virtual coaches have been designed to pro- mote healthier behaviors targeting for ex. elderly people [11] or patients recovering from spinal cord injuries [36]. Virtual coaches 1 https://github.com/clviegas/Entheos-Dataset