Copyright 2000 IEEE. Appeared in IEEE Conf. on Computer Vision and Pattern Recognition, vol. 2, pp:357–362, Hilton Head Island, South Ca Probabilistic vs. Geometric Similarity Measures for Image Retrieval Selim Aksoy and Robert M. Haralick Department of Electrical Engineering University of Washington Seattle, WA 98195-2500 aksoy,haralick @isl.ee.washington.edu Abstract Similarity between images in image retrieval is measured by computing distances between feature vectors. This pa- per presents a probabilistic approach and describes two likelihood-based similarity measures for image retrieval. Popular distance measures like the Euclidean distance im- plicitly assign more weighting to features with large ranges than those with small ranges. First, we discuss the ef- fects of five feature normalization methods on retrieval per- formance. Then, we show that the probabilistic methods perform significantly better than geometric approaches like the nearest neighbor rule with city-block or Euclidean dis- tances. They are also more robust to normalization effects and using better models for the features improves the re- trieval results compared to making only general assump- tions. Experiments on a database of approximately 10,000 images show that studying the feature distributions are im- portant and this information should be used in designing feature normalization methods and similarity measures. 1. Introduction Image database retrieval has become a very popular re- search area in recent years [15]. Initial work on content- based retrieval [8, 12, 10] focused on using low-level fea- tures like color and texture for image representation. After features are computed for all images in the database, simi- larity measures are used to find matches between images. Feature vectors usually exist in a very high dimensional space. Due to this high dimensionality, their parametric characterization is usually not studied. A commonly used assumption is that images that are close to each other in the feature space are also visually similar. In geometric similar- ity measures like the nearest neighbor rule, no assumption is made about the probability distribution of the features and similarity is based on the distances between feature vectors in the feature space. Given this, Euclidean distance has been the most widely used distance measure [8, 12, 9, 18], as well as the weighted Euclidean distance [4, 16], city-block ( ) distance [10, 18], the general Minkowsky distance [17] and the Mahalanobis distance [12, 18]. The distance was also used under the name “histogram intersection” [18]. Polynomial combinations of predefined distance measures were also used to create new distance measures [5]. This paper presents a probabilistic approach for image retrieval. We describe two likelihood-based similarity mea- sures that compute the likelihood of two images, one being the query image and the other one being an image in the database, being similar or dissimilar. First, we define two classes, the relevance class and the irrelevance class, and then the likelihood values are derived from a Bayesian clas- sifier. We use two different methods to estimate the condi- tional probabilities used in the classifier. The first method uses a multivariate Normal assumption and the second one uses independently fitted distributions for each feature. The performances of these two methods are compared to the per- formances of geometric approaches that use the city-block ( ) and Euclidean ( ) distances as similarity measures. An important step between feature extraction and dis- tance computation is feature normalization. Complex im- age database retrieval systems use features that are gener- ated by many different feature extraction algorithms and not all of these features have the same range. Popular distance measures, for example the Euclidean distance, implicitly assign more weighting to features with large ranges than those with small ranges. This paper discusses five normal- ization methods; linear scaling to unit range, linear scaling to unit variance, transformation to a Uniform[0,1] random variable, rank normalization and normalization by fitting distributions. Experiments are done on a database of ap- proximately 10,000 images and average precision is used to evaluate performances of both the normalization methods and the similarity measures. The rest of the paper is organized as follows. First, the features that we use in this study are summarized in Section 2. Then, the feature normalization methods are described in Section 3 and are followed by the similarity measures in Section 4. Experiments and results are discussed in Sec- tion 5. Finally, conclusions are given in Section 6. 2. Feature Extraction Textural features that were described in detail in [2, 3] are used for image representation in this paper. The first