Automatic Creation of Photo Books from Stories in Social Media Mohamad Rabbath OFFIS - Institute for Information Technology Oldenburg, Germany rabbath@offis.de Philipp Sandhaus OFFIS - Institute for Information Technology Oldenburg, Germany sandhaus@offis.de Susanne Boll University of Oldenburg Oldenburg, Germany susanne.boll@uni- oldenburg.de ABSTRACT Photos are a special way to tell stories of our best memories. The representation of those photos in appealing physical photo books is highly appreciated by many people. Today, many photos are shared via social networking sites, where people upload their pho- tos and share their stories with their friends. The members of the social network interact with the photo by commenting, adding tags or descriptions and uploading photos for the same events to their albums. While the media of different personal events are available on the social network, there is no easy way to collect and bundle them into a story and print this story as a photo book. We propose an approach to automatically detect media elements that match a query (where, when, what, who) in the user’s social network and intelligently arrange and compose them into a printed photo book. We combine content analysis of text and images to automatically and semi-automatically select photos of a specific story. Photos that are highly probable to belong to the same event are clustered using an Expectation-Maximization algorithm to be able to easily retrieve them using online queries. People tags that are added to the photos are employed to select important photos by automati- cally calculating the individual social importance of friends. The selected content and derived semantics are employed to create an appealing layout for automatically created photo books. Categories and Subject Descriptors D.2.2 [Design Tools and Techniques]: User interfaces; H.3.3 [In- formation Search and Retrieval]: Search process; H.3.3 [Infor- mation Search and Retrieval]: Selection process; I.5.1 [Models]: Statistical; I.5.1 [Clustering]: Similarity measures; I.7.2 [Docu- ment Preparation]: Photocomposition/typesetting General Terms Algorithms, Design, Human Factors Keywords photo books, social media, social networks, Expectation-Maximiza- tion, multimedia retrieval, information retrieval, layout Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. WSM’10, October 25, 2010, Firenze, Italy. Copyright 2010 ACM 978-1-4503-0173-2/10/10 ...$10.00. 1. INTRODUCTION In recent years social community websites have become an im- portant means for many to stay in touch and share personal media. Albeit the more sophisticated facilities of dedicated photo sharing websites like Flickr or Panoramio, social community websites like Facebook offer a more direct way to share photos and annotations with others. Users can upload their photos to their albums, provide meaningful captions, share those photos, tag each other, and com- ment on each other’s photos. This has led to an enormous growth of highly valuable semantically rich personal media. Facebook for example constitutes the biggest photo sharing site on the web with more than 15 Billion photos in total, an average of 220 million new pictures posted each week 1 . However, despite this growing trend to share important moments online with friends, social community sites remain digital and thus do not provide the valuable experience of a physical souvenir of the documented stories. Physical photo products, such as photo books, have always been the preferred way to reliably preserve the memory of important moments for a long time in contrast to the in- herently dynamic nature of the social community sites: Events rep- resented in social media sites could potentially change over time, photos, annotations, and comments could be deleted or the social community site itself could vanish from the internet. In this paper we therefore propose a solution to combine the advantages of both social community web sites and printed photo books. We bridge the gap between the digital and physical world by providing a system to automatically transform a snapshot of the representation of social media stories (one or more events) in so- cial networks into a photo book. Our approach is implemented as a Facebook application. We are faced with a number of challenges: With a large amount of photos over the user’s social net, it is not an easy task to manually determine the photos related to a specific social media story. The manual selection of specific photos into a photo book editor, and processing them in order to create a final photo book is a tedious process and is the main reason for many people not to finish a started photo book 2 . In our paper we will contribute to make the process of creating a photo book of social media stories automatic. We introduce the concept of a social media story represented in social networks and provide a solution to determine such stories automatically. This involves the automatic selection of related photos, automatically ranking them based on semantic information present in the social network and compiling the selected photos into a visually pleasing designed photo book. We implement our approach for Facebook 1 http://mashable.com/2009/04/30/ Facebook-photo-sharing/ 2 http://www.docstoc.com/docs/17451788/ PMA-Photobook-Report-Summary/ 15