Visual Image Query Kreˇ simir Matkovi´ c VRVis Research Center in Vienna, Austria, Matkovic@VRVis L´ aszl ´ o Neumann Maros u. 36 H-1122 Budapest, Hungary lneumann@cg.tuwien.ac.at Johannes Siglaer Institute for Design and Assessment of Technology, TU Vienna, Austria jsiglaer@pop.tuwien.ac.at Martin Kompast Institute for Design and Assessment of Technology, TU Vienna, Austria mkompast@pop.tuwien.ac.at Werner Purgathofer Institute of Computer Graphics and Algorithms, TU Vienna, Austria wp@cg.tuwien.ac.at ABSTRACT The explosion of storage media size and bandwidth has led to huge image databases. Methods are needed to find a particular image based on a crude description by the user. Keywording is not only tedious, but also subjective and therefore often incorrect. Avail- able visual query systems have different properties, and are mostly based on some image transformation. An alternative visual query system is introduced, which finds an image similar to a user drawn sketch, or to any other reference image. A descriptor is created for each image in the database, and for the query image. Descrip- tors are compared in order to find the best matches. Descriptors are computed by inserting a limited number of quasi-random rect- angles in the image, and computing the average colors of the rect- angles. Furthermore, a reduced color histogram is computed and stored in the descriptor. The difference between descriptors is cal- culated as the weighted average of CIE LUV differences between corresponding rectangles. Using the Contrast Sensitivity Function this average is adapted to the users perception. The metric used for comparing images operates in the original image space, which makes the whole algorithm intuitive and easy to understand, and enables the comparison of images sections, as well. Keywords Image Retrieval, Color Layout Query, Digital Image Matching, Hu- man Perception 1. INTRODUCTION The amount of electronic images an average user is confronted with has exploded in the last decade. This trend seems to continue with further bandwidth and storage media size expansion. As a re- sult of such an explosion huge image databases are quite common today. Unfortunately, as the size of the image database grows, the time required to find an image increases as well. When some crit- Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to distribute to lists, requires prior specific permission and/or fee. Int. Symp. on Smart Graphics, June 11-13, 2002, Hawthorne, NY, USA. Copyright 2002 ACM 1-58113-555-6/02/0600...$5.00 ical number of images is reached, it is no longer possible to find a specific image fast, by using thumbnail view only. Of course, it is possible to describe images using lists of keywords, but it seems that there are just a few of us willing to type keywords for each scanned or downloaded image. Furthermore assigning keywords is not only tedious, but the description via textual attributes is not very objective. Another way to find a specific image or a group of similar pictures in a large database consists in using a visual query system. Such a system tries to find an image similar to an image that is drawn by the user, or that is supplied in some other way (existing image, low-quality scan...) An ideal system would be ca- pable of finding a particular item, e.g. a company logo, or a human face, in a large database of arbitrary images. There is a lot more of research to be done before such a system is constructed. Existing systems can find an image with similar color-layout, or can recognize a person if predefined constraints are met. So a per- son can only be identified if the image shows a portrait of the person and is compared within a database of portraits, and not of arbitrar- ily images. There are a lot of image query systems, either commer- cially available or still in research phase. Only the best known will be mentioned here, and Eakins et al.[4] give an excellent overview of almost all existing methods. Probably the most well known systems are: QBIC[5] developed at IBM, VIR Image engine[1] from Virage, Inc. and Photobook Project[16] developed in the MIT Media Lab. Another work well known in the computer graphics community is Multi resolution Im- age Querying by Jacobs et al.[10]. This method was the first intro- duced to the community, and yields impressive results. Since our basic approach is similar to theirs, the differences to their method will be stressed out more explicitly. A new visual image query method is introduced. It is simple to implement and simple to understand, and gives satisfactory results. This simplicity is the main strength of our method. The main idea is to create image descriptors in a preprocess- ing phase, to store them in a database, and to use them for the query. When the user requests a query, a descriptor is computed for the new query image, and it is compared to the descriptors in the database. The results are sorted, and the best matches are shown to the user. That’s exactly the same as Jacobs et al.[10] do. What makes our method different is the way how the descriptors are cre- ated, and the metric that is used in comparing the query with the target images. Our work relies strongly on the metric proposed by Neumann et al.[15]. We will not describe it in detail, due to space limitations, and the interested reader should have a look at the orig- 116