Abstract—Nowadays, Content-Based Image Retrieval has been the mainstay of image retrieval both in fields of research and application. To attain optimal retrieval results, relevance feedback (RF) methods are incorporated into CBIR by taking user’s feedbacks into account. However, explicit RF methods rely heavily on active user engagement during search sessions, which is unrealistic in real applications. This paper presents an implicit RF method, Preference Estimation-based RF (PERF) for CBIR. PERF utilizes implicit user browsing histories to build a user preference model. The model will be refined iteratively and used to train a preference classifier for the user. In addition, an adaptive mechanism is adopted to realize the personalization of preference model. Our proposed method is tested and the experimental results reveal that PERF can achieve good retrieval precision with scarce explicit engagement from users. Index Terms—CBIR, relevance feedback, implicit, browsing behaviors, preference model, adaptive mechanism. I. INTRODUCTION Multimedia contents are increasing explosively and the need for multimedia retrieval grows quickly. Among researches in this field, image understanding is a hotspot. Multimedia mining becomes an important method of understanding images via extracting valuable knowledge from a large-scale multimedia repository [1]. In the development process of image retrieval system, traditional annotation based image retrieval faces more and more difficulties due to its heavy dependence on related captions [2], [3] and its two sufferings: high-priced manual annotation and inappropriate automated annotation. Content-Based Image Retrieval (CBIR) is a current active topic in image retrieval studies. The purpose of CBIR is to present image retrieval results which have similar contents to user’s query [4]. Most CBIR approaches are based on computation the similarities of low level features between user’s query and images via a query by example system [5]. Although some search strategies are very powerful, it is hard to obtain an optimal retrieval result within one query process. Thus relevance feedback (RF) [6]-[9] techniques are incorporated into CBIR by taking into account user’s Manuscript received April 13, 2013; revised July 5, 2013. This work was supported in part by Key Projects in the Science and Technology Pillar Program of Tianjin under Grant No.11ZCKFGX01200. Wei Dai, Wenbo Li, Zhipeng Mo, and Tianhao Zhao are with the Department of Computer Software Engineering, Tianjin University, No. 92, Weijin Road, Nankai District, Tianjin, P.R.C. (e-mail: shindaveee@gmail.com, wenboli@tju.edu.cn, zhipengandsky@163.com, zhaotianhao168@sohu.com). feedbacks to the results in the next retrieval process. In order to find user’s real intention for the image retrieval, conventional RF approaches require users to evaluate retrieval results in multiple iterations. It is apparently unrealistic to require too much users’ explicit involvement in retrieval iterations. To our best knowledge, there are two types of strategies to reduce users’ burden: reducing iterations and easing users’ burden in iterations. For the former strategy, Su et al. [1] proposed Navigation-Pattern-based Relevance Feedback (NPRF) to achieve high retrieval quality of CBIR with RF by using discovered navigation patterns. Their work aims at reducing the redundant browsing and reaching exploration convergence quickly by mining the user query log. They report their method can achieve accuracy of over 90% in 6 iterations. As all RF methods, the retrieval accuracy of NPRF relies on users’ active involvement. The faulty operations may lead to performance reduction. As for the latter strategy, the implicit RF methods are proposed. Different with explicit RF, the implicit RF technique [10], [11] is a kind of RF technique by gathering useful data indirectly by monitoring behaviors of the users during the search session instead of requiring much active user engagement. This kind of technique was first applied in retrieving documents and was brought into CBIR several years ago. A number of studies that employ implicit RF have been made to lower the users’ burden. Auer et al. [12] proposed a system which infers users’ intention from eye movements by using machine learning method. Then the system learns a similarity metric of common image features depending on the current interests of the user. Similarly, the system of [13] improves the performance of image retrieval by re-ranking the retrieved images according to color and texture features extracted from the regions where the users pay more attention. The users’ interests are found by gazing information collected from an unobtrusive eye tracker. Unfortunately, performances of current implicit RF methods are frustrating compared to those of explicit ones. To resolve the aforementioned problems, we propose a novel method, PERF to achieve the high retrieval quality of CBIR with implicit RF by monitoring user’s behaviors during search sessions, such as browsing time, download, zoom-in and scrolling. In iterations of retrieval, the user browses images according to his/her custom and meanwhile four foresaid data will be collected by the system implicitly. User’s preference values for his/her browsed images will be inferred automatically by the system. Then the preference values will be used to train an image classifier to improve the retrieved results for next iteration. Considering that browsing Wei Dai, Wenbo Li, Zhipeng Mo, and Tianhao Zhao Implicit Relevance Feedback for Content-Based Image Retrieval by Mining User Browsing Behaviors and Estimating Preference Lecture Notes on Software Engineering, Vol. 1, No. 4, November 2013 334 DOI: 10.7763/LNSE.2013.V1.72