Visual Lifelogs Retrieval: State of the Art and Future Challenges Zenonas Theodosiou 1* and Andreas Lanitis 1, 2† 1 Research Centre on Interactive Media, Smart systems and Emerging Technologies (RISE), Nicosia, Cyprus 2 Dept. of Multimedia and Graphic Arts, Cyprus University of Technology, Limassol, Cyprus Email: * z.theodosiou@gmail.com, † andreas.lanitis@cut.ac.cy Abstract—The use of wearable cameras covers several areas of application nowadays, where the need for developing smart applications providing the sustainability and well-being of citizens it is more necessary than ever before. The tremendous amount of lifelogging data to extract valuable knowledge about the every day life of the wearers requires state of the art retrieval techniques to efficiently store, access, search and retrieve use- ful information. Several works have been proposed combining computer vision and machine learning techniques to analyze the content of the data captured from visual wearable devices on a daily basis. This paper presents an overview of the progress in visual lifelogging retrieval and indicates the current advances and future challenges, highlighting the prospects of incorporating visual lifelogging retrieval in social computing applications. Index Terms—wearable cameras, lifelogging, retrieval, digital memory I. I NTRODUCTION Within the general theme of ambient intelligence, wearable computing constitutes an important research and technological direction. Wearable devices are on the rise in recent years due to technological developments and play a significant role in Internet of Things (IoT) applications and in the highly promis- ing future developments of ubiquitous computing. Wearable devices are electronic components integrated on clothing or accessories which can be easily worn from the users. Thus, technology companies have shown great interest by investing a lot in the development of innovative small components with embedded advanced sensing technologies to easily collect and transmit data from the wearer’s environment. Usually wearable cameras are small and light devices which can be fastened at human body covering the point of view of the wearer. They provide the capability to seamlessly record visual data in a passive way, in a first-person persepctive, while the wearer is performing her/his activities. A lot of research has been carried out using wearable cameras the last years, including studies related to: cognition and social interaction between humans [1], navigation/assistive technologies for the blind [2], monitoring and assistance of physical environ- ments [3], automated life story creation [4], summarization [5], action (e.g. fall detection) or location recognition [6], security, safety and protection of citizens. Visual lifelogging is the seamless collection of images and/or videos through the use of wearable cameras and involves the continuous recording of the daily life of the wearer for a long periods of time [7]. The new field of the computer vision which deals with the content analysis of data collected by wearable devices, is called Egocentric Vision or First-person Vision. The analysis of such visual data can be successfully used to study everyday life and draw useful conclusions about human behavior, aiming to improve the quality of life and prevent people from mental disorders. Furthermore, visual lifelogging can be used as a digital memory to help elderly people suffering from memory disorders cope with the demands of modern lifestyle [8]. The enormous increase in the number of available visual lifelogging data requires the development of technologies for efficient archiving and access to visual content. Retrieving data from a digital collection can solve several problems in the field of Egocentric Vision, including: (1) searching for elements, (2) navigating, (3) understanding the environment, and (4) organizing huge amounts of data [7]. Due to the fact that the retrieval approaches which have already proposed for other type of data are not effective enough when applied for lifelogging tasks, there is a need for the development of beyond the state of the art techniques to successfully retrieve data from large scale lifelog databases. In this paper we restrict our study on the current state of the art of visual lifelloging retrieval and demonstrate the potential of using lifelogging in social computing. In section II, the state of the art of the visual lifelogging retrieval is discussed. In section III, we present the techniques of analyzing the lifelog visual content. In section IV, various retrieval systems are investigated. The applications of the lifelogging retrieval are discussed in section V, followed by discussion and future challenges in Section VI. Finally, section VII concludes the paper. II. STATE OF THE ART One of the main applications of visual lifelogging is to play the role of a visual memory where the lifelogs are saved in a database providing users with the opportunity to access the past memory anytime based on a query (Fig. 1). The successful management of huge amount of lifelogs needs 978-1-7281-3634-9/19/$31.00 ©2019 IEEE