True emotion vs. Social Intentions in Nonverbal Communication: Towards a Synthesis for Embodied Conversational Agents Jonathan Gratch University of Southern California gratch AT ict.usc.edu Abstract. Does a facial expression convey privileged information about a per- son’s mental state or is it a communicative act, divorced from “true” beliefs, desires and intentions? This question is often cast as a sharp dichotomy be- tween competing theoretical accounts of nonverbal behavior -- a dichotomy that is often reflected in embodied conversational agents. On the one hand, theorists such as Ekman or Frank argue for the primacy of emotion as a determinant of nonverbal behavior: emotions “leak” through our behaviors and convey true in- formation about our current beliefs and feelings, which only indirectly serve so- cial ends. On the other hand, theorists such as Fridlund argue for the primacy of social ends as the determinant of nonverbal displays. This dichotomy has worked to divide virtual character research. Whereas there have been consider- able advances in modeling emotions and giving virtual characters the ability to derive situationally appropriate emotions, this work is often seen as irrelevant to the generation of communicative nonverbal behavior. In this chapter, I will re- view current findings on the interpersonal function of emotion and its potential role in communication. I’ll discuss some recent developments in Social Ap- praisal theory as a way to reconcile the true emotion and social motive views of nonverbal behavior. I will then discuss our attempts to model these functions within the context of embodied conversational agents. Keywords: 1 Introduction Do facial expressions convey privileged information about a person’s mental state or are they communicative acts, independent of a person’s actual beliefs, desires and intentions? This question is often cast as a dichotomy between competing theoretical accounts of nonverbal behavior. On the one hand, theorists such as Ekman [2] or Frank [3] argue for the primacy of emotion as a determinant of nonverbal behavior: emotions “leak” through our behaviors and convey true information about our current beliefs and feelings, which only indirectly serve social ends. On the other hand, social intentions theorists such as Fridlund [4] or Chovil [5] argue for the primacy of social ends as the determinant of nonverbal displays: organisms use displays strategically to To appear in Modeling Communication with Robots and Virtual Humans, Wachmuth and Knoblich (eds.), Springer