Are You Really Smiling at Me? Spontaneous versus Posed Enjoyment Smiles Hamdi Dibeklio˘ glu ‡ , Albert Ali Salah § , and Theo Gevers ‡ ‡ Intelligent Systems Lab Amsterdam, University of Amsterdam, Amsterdam, The Netherlands {h.dibeklioglu,th.gevers}@uva.nl § Department of Computer Engineering, Bo ˘ gazic¸i University, Istanbul, Turkey salah@boun.edu.tr Abstract. Smiling is an indispensable element of nonverbal social interaction. Besides, automatic distinction between spontaneous and posed expressions is im- portant for visual analysis of social signals. Therefore, in this paper, we propose a method to distinguish between spontaneous and posed enjoyment smiles by us- ing the dynamics of eyelid, cheek, and lip corner movements. The discriminative power of these movements, and the effect of different fusion levels are inves- tigated on multiple databases. Our results improve the state-of-the-art. We also introduce the largest spontaneous/posed enjoyment smile database collected to date, and report new empirical and conceptual findings on smile dynamics. The collected database consists of 1240 samples of 400 subjects. Moreover, it has the unique property of having an age range from 8 to 76 years. Large scale experi- ments on the new database indicate that eyelid dynamics are highly relevant for smile classification, and there are age-related differences in smile dynamics. Key words: Face analysis, smile classification, affective computing 1 Introduction Human facial expressions are indispensable elements of non-verbal communication. Since faces can reveal the mood or the emotional feeling of a person, automatic under- standing and interpretation of facial expressions provide a natural way to interact with computers. In recent studies, analysis of spontaneous facial expressions have gained more interest. For social interaction analysis, it is necessary to distinguish spontaneous (felt) expressions from the posed (deliberate) ones. Spontaneous expressions can re- veal states of attention, agreement and interest, as well as deceit. The foremost facial expression for spontaneity analysis is the smile, as it is the most frequently performed expression, and used for signaling enjoyment, embarrassment, politeness, etc. [1]. It is also used to mask other emotional expressions, since it is the easiest emotional facial expression to pose voluntarily [2]. Several characteristics of spontaneous and posed smiles, such as symmetry, speed, and timing are analyzed in the literature [3], [4], [5]. Their findings suggest that dif- ferent facial regions contribute differently to the classification of smiles. In this paper, we assess the facial dynamics for different face regions, and demonstrate that the eye region contains the most useful information for this problem. Our method combines Note that this is not the final copy of the paper as published in LNCS. There might be changes in the final version. Please check Springer website for the final version.