MIT Media Laboratory Software Agents Group Technical Report. November 2002. Automatic Affective Feedback in an Email Browser Hugo Liu Software Agents Group MIT Media Laboratory Cambridge, MA 02139 +1 617 253 5334 hugo@media.mit.edu Henry Lieberman Software Agents Group MIT Media Laboratory Cambridge, MA 02139 +1 617 253 0315 lieber@media.mit.edu Ted Selker Context-Aware Computing Group MIT Media Laboratory Cambridge, MA 02139 +1 617 253 6968 selker@media.mit.edu ABSTRACT This paper demonstrates a new approach to recognizing and presenting the affect of text. The approach starts with a corpus of 400,000 responses to questions about everyday life in Open Mind Common Sense. This so-called commonsense knowledge is the basis of a textual affect sensing engine. The engine dynamically analyzes a user’s text and senses broad affective qualities of the story at the sentence level. This paper shows how a commonsense affect model was constructed and incorporated into Chernov face style feedback in an affectively responsive email browser called EmpathyBuddy. This experimental system reacts to sentences as they are typed. It is robust enough that it is being used to send email. The response of the few dozen people that have typed into it is dramatically enthusiastic. This paper debuts a new style of user interface technique for creating intelligent responses. Instead of relying on specialized handcrafted knowledge bases this approach relies on a generic commonsense repository. Instead of relying on linguistic or statistical analysis alone to “understand” the affect of text, it relies on a small society of approaches based on the commonsense repository. Keywords Emotion and Affective UI, Agents and Intelligent Systems, Context-Aware Computing, User and Cognitive models. INTRODUCTION One of the impressive triumphs of the computer revolution is that it has given us more effective tools for personal and social expression. Through emails, weblogs, instant messages, and web pages, we are able to share our experiences with friends, family, co-workers, or anyone else in the world who will listen. We use these mediums on a daily basis to share stories about our daily lives. However, as useful as these tools have become, they still lack the highly treasured social interactivity of an in-person conversation. Much as we desire to relate stories of experiences that have saddened, angered, frustrated, and delighted us, the text sits unmoved in cold, square boxes on the computer screen. Nass et al.’s study of human- computer social interaction reveals that people naturally expect their interactions with computers to be social and affective, just as with other people! [20],[21]. Sadly though, people have been so conditioned to expect so little from the user interfaces of today that we are not even bothered by their inability to affectively respond to us like a friend or family member might do. This shortcoming in current user interfaces hinders progress in the bigger picture too. If software is to transform successfully into intelligent software agents, a social-affective connection between the user and computer must be established because the capacity for affective interaction plays a vital role in making agents believable [2],[27]. Without it, it will be hard to build trust and credibility in the human-computer relationship. All of this gives rise to the question: Can a user interface react affectively with useful and believable responsiveness to a user engaged in a storytelling task like email or weblogging? We argue that the answer is yes! In this paper, we present a commonsense-based textual analysis technology for sensing the broad affective qualities of everyday stories, told line-by-line. We then demonstrate this technology in an affectively responsive email browser called EmpathyBuddy. EmpathyBuddy gives the user automatic affective feedback by putting on different Chernov-style emotion faces to match the affective context of the story being told through the user’s email. We evaluate the impact of the system’s interactive affective response on the user, and on the user’s perception of the system. Paper’s Organization This paper is structured as follows: First, we put our approach into perspective discussing existing approaches to textual affect sensing and other related work. Second we motivate our commonsense treatment of emotions with research from the cognitive psychology and artificial intelligence literature. Third, we discuss methods for constructing and applying an commonsense affect model. Fourth, we discuss how a textual affect sensing engine was incorporated into Chernov face style feedback in an affectively responsive email browser called EmpathyBuddy, and we examine a user scenario for our system. Sixth, we present the results of a user evaluation of EmpathyBuddy. The paper concludes with a summary of contributions, and plans for further research.