Automatic Personality and Interaction Style Recognition from Facebook Proﬁle Pictures Fabio Celli University of Trento via sommarive 5 Trento, Italy fabio.celli@unitn.it Elia Bruni University of Bozen-Bolzano Dominikanerplatz 3 Bozen-Bolzano, Italy elia.bruni@unibz.it Bruno Lepri FBK via sommarive 18 Trento, Italy lepri@fbk.eu ABSTRACT In this paper, we address the issue of personality and interaction style recognition from proﬁle pictures in Facebook. We recruited volunteers among Facebook users and collected a dataset of pro- ﬁle pictures, labeled with gold standard self-assessed personality and interaction style labels. Then, we exploited a bag-of-visual- words technique to extract features from pictures. Finally, different machine learning approaches were used to test the effectiveness of these features in predicting personality and interaction style traits. Our good results show that this task is very promising, because pro- ﬁle pictures convey a lot of information about a user and are directly connected to impression formation and identity management. Categories and Subject Descriptors 5.5 [Emotional and Social Signals in Multimedia]: Novel meth- ods for the classiﬁcation and representation of interactive social and/or emotional signals General Terms Facebook proﬁling personality pictures algorithms Keywords personality recognition; Facebook; data mining; machine learning; feature extraction 1. INTRODUCTION People spend a considerable amount of effort in order to form and to manage impressions, especially in the initial stage of social interactions [10]. Nowadays, this fundamental process has been modiﬁed by the usage of new communication technologies. So- cial networking technologies, such as Facebook, offer new ways for self-presentations. Several studies reported that Facebook users engage in actively creating, maintaining and modifying an image of selves by adjusting their proﬁles, including descriptions and pic- tures, joining groups and displaying their likes and dislikes [12]. Hence, the Facebook proﬁle page can be considered as a medi- ated representation of the Facebook user. Although users may be Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for proﬁt or commercial advantage and that copies bear this notice and the full citation on the ﬁrst page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior speciﬁc permission and/or a fee. Request permissions from Permissions@acm.org. MM’14 November 03 - 07 2014, Orlando, FL, USA Copyright 2014 ACM 978-1-4503-3063-3/14/11 ...$15.00 http://dx.doi.org/10.1145/2647868.2654977. tempted to enhance their self-presentations, friends who are both ofﬂine and online keep Facebook users’ self-presentations in check. Indeed, misrepresentation on proﬁle pages can have serious ofﬂine consequences. Therefore, the online proﬁle usually reﬂect the of- ﬂine proﬁle, although slightly enhanced [33]. Moreover, social psychology research has also highlighted that personality plays an important role in the way people manage the images they convey in self-presentations [17]. For this reason, some recent studies re- ported that users can make accurate personality impressions from the information displayed in social network user proﬁles [33], and have investigated the speciﬁc features from user proﬁles and pho- tos that are more useful to create personality impressions [8]. For example, the results obtained by Hall et al. [13] indicate that ob- servers could accurately estimate extraversion, agreeableness and conscientiousness of unknown proﬁle owners. Interestingly, proﬁle pictures were useful for estimating extraversion and agreeableness. Again, Utz [31] associated user extraversion with photo expres- siveness. Finally, Van deer Heide et al. [32] have shown that in the context of Facebook, photos may have more impact on judg- ments of extraversion than textual self-disclosures. Based on these previous ﬁndings, we propose to automatically predict personal- ity traits and interaction styles from Facebook proﬁle images. Re- garding personality, we resort to the big ﬁve model [7], a multi- factorial approach which owes its name to the ﬁve traits it takes as constitutive of people’s personality: extraversion, emotional sta- bility/neuroticism, agreeableness, conscientiousness, and openness to experience. Moreover, we also target the prediction of people’s interaction traits. To this end, we exploited the interpersonal cir- cumplex, a model for organizing and assessing interpersonal traits [19]. Speciﬁcally, the interpersonal circumplex is deﬁned by two orthogonal axes, a vertical axis of dominance or agency and a hor- izontal axis of affect or warmth. The main contributions of the paper are as follows: 1) We collect a dataset of proﬁle images from 100 Facebook users and we anno- tate them with gold standard self-assessed personality and interac- tion style labels; 2) we exploit a bag-of-visual-word representation of images in order to extract relevant visual features; 3) we pro- pose and validate a supervised learning approach, based on Support Vector Machines and logistic regression, to the automatic recogni- tion of personality and interaction style traits from visual features extracted from proﬁle pictures. The the paper is structured as fol- lows: Section 2 describes the relevant previous work, with a partic- ular focus on personality recognition from social media; Section 3 describes the data used for our experiments and Section 4 provides the deﬁnition of the research task and the detailed information on the methodology (feature extraction, feature selection and classi- ﬁcation). In Section 5 we present and discuss the results of our experiments and ﬁnally, in Section 6 we draw our conclusions.