The Belfast Induced Natural Emotion Database Ian Sneddon, Margaret McRorie, Gary McKeown, and Jennifer Hanratty Abstract—For many years psychological research on facial expression of emotion has relied heavily on a recognition paradigm based on posed static photographs. There is growing evidence that there may be fundamental differences between the expressions depicted in such stimuli and the emotional expressions present in everyday life. Affective computing, with its pragmatic emphasis on realism, needs examples of natural emotion. This paper describes a unique database containing recordings of mild to moderate emotionally colored responses to a series of laboratory-based emotion induction tasks. The recordings are accompanied by information on self- report of emotion and intensity, continuous trace-style ratings of valence and intensity, the sex of the participant, the sex of the experimenter, the active or passive nature of the induction task, and it gives researchers the opportunity to compare expressions from people from more than one culture. Index Terms—Natural emotion, database, emotion induction, emotional corpora, affective annotation. Ç 1 INTRODUCTION T HEORIES about the capacity of humans to recognize emotions from the facial behavior of others are based largely on the study of highly selected still photographs of posed expressions—an experimental paradigm that has been called the “standard method” [1]. Theories of emo- tional expression are, of course, attempts to explain natural, not posed human behavior. Despite this fact, there has been relatively little research on natural dynamic examples of emotion (see [2] for a review). Although the lack of natural evidence in support of theories of emotional expression has been criticized [1], the reluctance to use examples of natural expression when studying emotion is at least partly under- standable. When using posed photographs the researcher typically presents participants with examples of prototypi- cal facial expressions based on the Facial Action Coding System (FACS) developed by Ekman and Friesen [3]. Thus, the recognition of the emotion can be judged against the “correct” expression adopted by the poser. The use of examples of natural emotion is much more problematic. In this case, the researcher is using a spontaneous facial expression with no means of knowing objectively what emotion is being experienced by the encoder or the extent to which the facial expression reflects that emotion. The problem can be conceived as a tradeoff between ecological validity and reliability, with the decision over which approach to favor being influenced by the nature of the research question and its sensitivity to each of these factors. If natural everyday emotional expressions showed only quantitative differences from the posed examples (perhaps occurring as a weaker version of the prototypical forms) then perhaps the overreliance on these examples would be of little importance, but recent research suggests that there may be fundamental differences between the two. A number of research studies have indicated that even actors do not show the prototypical patterns of facial behavior when attempting to convey emotion. It has been reported [4] that Hollywood actors only exhibit the prototypical patterns specified by Ekman and Friesen when portraying happiness, and more recent research [5] found none of the prototypical patterns of action units when experienced actors simulated strong emotions. Additionally, it has been suggested [6] that in speech, there may be important qualitative differences between posed and natural examples of emotion expression. Recently, the overreliance on static images has also been questioned. Evidence is mounting which demonstrates that important information is contained in the dynamic unfold- ing of facial expressions over time [7], [8] and that expressions of different emotional states may have specific temporal patterns as well as morphological ones. For example, it has been shown [8] that when dynamic expressions were presented to participants, anger is recog- nized best when played at a medium speed, sadness is best recognized when it unfolds slowly, and happiness and surprise are recognized most accurately when the expres- sions change/develop quickly. This mismatch between theory and reality is of crucial importance for researchers in the field of affective comput- ing. Arguably, it might be possible to construct systems that can deliver believable emotional performances based on the posed prototypical patterns—after all, humans appear to have the capacity to be emotionally engaged by cartoon characters, line drawings, and even inanimate objects. However, if the prototypical patterns do not reflect natural facial behavior, then automatic systems trained on such patterns could not conceivably detect with any degree of accuracy the everyday emotions expressed by real people. 32 IEEE TRANSACTIONS ON AFFECTIVE COMPUTING, VOL. 3, NO. 1, JANUARY-MARCH 2012 . The authors are with the School of Psychology, Queen’s University Belfast, David Keir Building, Northern Ireland, Belfast BT7 1NN, United Kingdom. E-mail: {I.Sneddon, m.mcrorie, g.mckeown, jhanratty02}@qub.ac.uk. Manuscript received 30 Nov. 2010; revised 12 May 2011; accepted 30 May 2011; published online 1 Aug. 2011. Recommended for acceptance by B. Schuller, E. Douglas-Cowie, and A. Batliner. For information on obtaining reprints of this article, please send e-mail to: taffc@computer.org, and reference IEEECS Log Number TAFFCSI-2010-11-0118. Digital Object Identifier no. 10.1109/T-AFFC.2011.26. 1949-3045/12/$31.00 ß 2012 IEEE Published by the IEEE Computer Society