Sketch4Match – Content-based Image Retrieval System Using Sketches B. Sz´ ant´ o, P. Pozsegovics, Z. V´ amossy, Sz. Sergy´ an ´ Obuda University/Institute of Software Technology, Budapest, Hungary {szanto90balazs, pozsee1st}@gmail.com, {vamossy.zoltan, sergyan.szabolcs}@nik.uni-obuda.hu Abstract— The content based image retrieval (CBIR) is one of the most popular, rising research areas of the digital image pro- cessing. Most of the available image search tools, such as Google Images and Yahoo! Image search, are based on textual annotation of images. In these tools, images are manually annotated with keywords and then retrieved using text-based search methods. The performances of these systems are not satisfactory. The goal of CBIR is to extract visual content of an image automatically, like color, texture, or shape. This paper aims to introduce the problems and challenges concerned with the design and the creation of CBIR systems, which is based on a free hand sketch (Sketch based image retrieval – SBIR). With the help of the existing methods, describe a possible solution how to design and implement a task spesic descriptor, which can handle the informational gap between a sketch and a colored image, making an opportunity for the efcient search hereby. The used descriptor is constructed after such special sequence of preprocessing steps that the transformed full color image and the sketch can be compared. We have studied EHD, HOG and SIFT. Experimental results on two sample databases showed good results. Overall, the results show that the sketch based system allows users an intuitive access to search-tools. The SBIR technology can be used in several applications such as digital libraries, crime prevention, photo sharing sites. Such a system has great value in apprehending suspects and indentifying victims in forensics and law enforcement. A possible application is matching a forensic sketch to a gallery of mug shot images. The area of retrieve images based on the visual content of the query picture intensied recently, which demands on the quite wide methodology spectrum on the area of the image processing. I. I NTRODUCTION Before the spreading of information technology a huge number of data had to be managed, processed and stored. It was also textual and visual information. Parallelly of the appearance and quick evolution of computers an increasing measure of data had to be managed. The growing of data storages and revolution of internet had changed the world. The efciency of searching in information set is a very important point of view. In case of texts we can search exibly using keywords, but if we use images, we cannot apply dynamic methods. Two questions can come up. The rst is who yields the keywords. And the second is an image can be well represented by keywords. In many cases if we want to search efciently some data have to be recalled. The human is able to recall visual information more easily using for example the shape of an object, or arrangement of colors and objects. Since the human is visual type, we look for images using other images, and follow this approach also at the categorizing. In this case we search using some features of images, and these features are the keywords. At this moment unfortunately there are not frequently used retrieval systems, which retrieve images using the non-textual information of a sample image. What can be the reason? One reason may be that the text is a human abstraction of the image. To give some unique and identiable information to a text is not too difcult. At the images the huge number of data and the management of those cause the problem. The processing space is enormous. Our purpose is to develop a content based image retrieval system, which can retrieve using sketches in frequently used databases. The user has a drawing area where he can draw those sketches, which are the base of the retrieval method. Using a sketch based system can be very important and efcient in many areas of the life. In some cases we can recall our minds with the help of gures or drawing. In the following paragraph some application possibilities are analyzed. The CBIR systems have a big signicance in the criminal investigation. The identicaton of unsubstantial images, tattoos and grafties can be supported by these systems. Similar applications are implemented in [9], [10], [11]. Another possible application area of sketch based informa- tion retrieval is the searching of analog circuit graphs from a big database [7]. The user has to make a sketch of the analog circuit, and the system can provide many similar circuits from the database. The Sketch-based image retrieval (SBIR) was introduced in QBIC [6] and VisualSEEK [17] systems. In these systems the user draws color sketches and blobs on the drawing area. The images were divided into grids, and the color and texture fea- tures were determined in these grids. The applications of grids were also used in other algorithms, for example in the edge histogram descriptor (EHD) method [4]. The disadvantage of these methods is that they are not invariant opposite rotation, scaling and translation. Lately the development of difcult and robust descriptors was emphasized. Another research approach is the application of fuzzy logic or neural networks. In these cases the purpose of the investment is the determination of suitable weights of image features [15]. II. OUR PROJECT In this section the goal and the global structure of our sys- tem is presented. The components and their communications SAMI 2011 • 9th IEEE International Symposium on Applied Machine Intelligence and Informatics • January 27-29, 2011 • Smolenice, Slovakia - 183 - 978-1-4244-7430-1/11/$26.00 ©2011 IEEE