A Content-Based Image Retrieval Scheme Allowing for Robust Automatic Personalization Sotirios Chatzis Electrical and Computer Engineering Department National Technical University of Athens 15772, Zografos, Athens, Greece stchat@telecom.ntua.gr Anastasios Doulamis Electrical and Computer Engineering Department National Technical University of Athens 15772, Zografos, Athens, Greece adoulam@cs.ntua.gr Theodora Varvarigou Electrical and Computer Engineering Department National Technical University of Athens 15772, Zografos, Athens, Greece dora@telecom.ntua.gr ABSTRACT The retrieval performance of content-based image retrieval (CBIR) systems is often disappointingly low, mainly due to the subjectivity of human perception. Relevance feed- back (RF) has been widely considered as a powerful tool to enhance CBIR systems by incorporating human perception subjectivity into the retrieval procedure. However, usually, the obtained feedback logs are scarce and contain lots of outliers, undermining the RF adaptation effectiveness. In this paper, we tackle these shortcomings exploiting the in- herent outlier downweighting capabilities mixtures of Stu- dent’s t distributions offer. Each semantic class is modeled by a mixture of t distributions fitted to data provided by the system operators. Further, the semantic class models get personalized by application of a novel, efficient RF al- gorithm allowing for the robust adaptation of the semantic class models to the accumulated feedback of each user. The efficacy of our approach is validated through a series of ex- periments using objective performance criteria. Categories and Subject Descriptors H.3.3 [Information Storage and Retrieval]: Informa- tion Search and Retrieval—clustering, relevance feedback, retrieval models General Terms Algorithms Keywords t distributions, mixture models, personalization 1. INTRODUCTION Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. CIVR’07, July 9–11, 2007, Amsterdam, The Netherlands Copyright 2007 ACM 978-1-59593-733-9/07/0007 ...$5.00. The unprecedented upsurge in multimedia databases has set off multimedia information retrieval as an important re- search topic for many computer science communities [17]. One of the key aspects of multimedia information retrieval is content-based image retrieval (CBIR). CBIR systems are faced with two key-challenges stemming from the high level- semantic nature and the subjectivity of the way humans per- ceive the content of images. The first one is the semantic gap between the low-level visual features and high-level hu- man perception [17]. Humans perceive the content of images based on high-level semantic concepts. Despite the exten- sive efforts, the formulation of techniques and mathematical models to effectively extract and represent this type of in- formation based on the visual attributes of image pixels is extremely laborious and a general scope approach has yet to be proposed [20]. The second major challenge concerns the subjectivity of human perception. The way humans de- termine the content of an image is a rather nebulous proce- dure. Characteristic of its ill-defined nature is the fact that the same individual might perceive the same semantic en- tities at different times in a different manner, let alone the case where different individuals are considered [14]. Hence, personalization is one of the most important functions in de- signing successful CBIR systems, providing the mechanisms to make the system adaptable to the individual perception of its users [3]. Relevance feedback [13] provides the feasible means to mitigate the semantic gap between low-level image features and high-level semantic concepts by exploiting user-provided information to create successful mappings of low-level image features to high-level semantic concepts. Furthermore, rele- vance feedback allows for the effective resolution of the hu- man perception subjectivity issue, allowing for the personal- ization of CBIR systems. Personalization of CBIR systems can be attained by adapting the retrieval models and crite- ria they use, individually to the feedback provided by each of their users. Relevance feedback techniques for CBIR sys- tems have evolved [7] from earlier heuristic weighting tech- niques [14], to optimal learning [5] and the more recent ma- chine learning techniques (e.g. [11, 19]). The majority of the proposed relevance feedback tech- niques for CBIR systems regard the problem as a strict two- class classification problem, with equal treatments on both positive and negative examples. Although it is reasonable to assume that positive examples of a semantic class follow a 1