A graph-based relevance feedback mechanism in content-based image retrieval q Malay Kumar Kundu a , Manish Chowdhury a,⇑ , Samuel Rota Bulò b a Machine Intelligence Unit, Indian Statistical Institute, 203 B.T. Road, Kolkata 108, India b FBK-irst, Via Sommarive, 18, I-38123 Trento, Italy article info Article history: Received 27 March 2014 Received in revised form 18 July 2014 Accepted 9 October 2014 Available online xxxx Keywords: NSCT Content based image retrieval Re-ranking Relevance feedback Feature evaluation index abstract Content-Based Image Retrieval (CBIR) is an important problem in the domain of digital data management. There is indeed a growing availability of images, but unfortunately the traditional metadata-based search systems are unable to properly exploit their visual information content. In this article we introduce a novel CBIR scheme that abstracts each image in the database in terms of statistical features computed using the Multi-scale Geometric Analysis (MGA) of Non-subsampled Contourlet Transform (NSCT). Noise resilience is one of the main advantages of this feature representation. To improve the retrieval performance and reduce the semantic gap, our system incorporates a Relevance Feedback (RF) mecha- nism that uses a graph-theoretic approach to rank the images in accordance with the user’s feedback. First, a graph of images is constructed with edges reflecting the similarity of pairs of images with respect to the proposed feature representation. Then, images are ranked at each feedback round in terms of the probability that a random walk on this graph reaches an image tagged as relevant by the user before hitting a non-relevant one. Experimental analyses on three different databases show the effectiveness of our algorithm compared to state-of-the-art approaches in particular when the images are corrupted with different types of noise. Ó 2014 Elsevier B.V. All rights reserved. 1. Introduction A Content Based Image Retrieval (CBIR) system enables a user to organize and retrieve images in a database by analyzing the char- acteristics of the visual content. The whole process is usually done by presenting a visual query to the system and by extracting a set of images from the database that have highest resemblance to the query image [1–3]. This query-by-example procedure compares the visual content of images in terms of low level features by computing a distance between the features of the query image and the possible target images in the database [4–6]. A modern interactive CBIR system consists of the following main parts: feature extraction, feature reduction, ranking and rel- evance feedback. The first two phases allow to obtain abstract, compact representations for the query and database images, which possibly summarize their most distinctive features. The ranking phase consists in sorting the database images based on their relevancy to the query image. Finally, the relevance feedback phase involves the user intervention to tag the images in the result set as relevant or irrelevant. This triggers a re-ranking of the database images which accounts for the new feedback information. Multiple feedback rounds can follow until user satisfaction is achieved. Various feature extraction and feature reduction schemes have been used in the literature to find the low-dimensional salient and significant features, which can be effectively used to represent the underlying image’s characteristics [7–9]. It has been found that feature extraction techniques working in the frequency domain are more effective in representing the significant and subtle details of the image than the conventional spatial domain schemes [10,11]. Among the various frequency domain methods, Wavelet Transform (WT) and its variants (like M-band wavelet, complex wavelet, wavelet packets etc.) have been extensively used in CBIR systems [12,13,10,14]. Low level features based on WT provide a unique representation of the image and they are highly suitable to characterize textures of the image [12,15,16]. However, the main problem of WT-based features is the inherent lack of support to directionality and anisotropy. To overcome these limitations, a recent theory called Multi-scale Geometric Analysis (MGA) for high-dimensional signals has been introduced and several MGA tools have been developed like Ripplet, Curvelet and Contourlet, http://dx.doi.org/10.1016/j.knosys.2014.10.009 0950-7051/Ó 2014 Elsevier B.V. All rights reserved. q This work is supported by internal academic project fund of Machine Intelli- gence Unit, Indian Statistical Institute, 203 B.T. Road, Kolkata 108. ⇑ Corresponding author. Tel.: +91 33 2575 3100; fax: +91 33 2578 3357. E-mail addresses: malay@isical.ac.in (M.K. Kundu), st.manishc@gmail.com (M. Chowdhury), rotabulo@fbk.eu (S. Rota Bulò). Knowledge-Based Systems xxx (2014) xxx–xxx Contents lists available at ScienceDirect Knowledge-Based Systems journal homepage: www.elsevier.com/locate/knosys Please cite this article in press as: M.K. Kundu et al., A graph-based relevance feedback mechanism in content-based image retrieval, Knowl. Based Syst. (2014), http://dx.doi.org/10.1016/j.knosys.2014.10.009