APPROXIMATE NEAREST NEIGHBORS USING SPARSE REPRESENTATIONS Joaquin Zepeda INRIA Centre Rennes - Bretagne Atlantique Rennes, France Ewa Kijak Universit´ e de Rennes 1 IRISA Rennes, France Christine Guillemot INRIA, Centre Rennes - Bretagne Atlantique Rennes, France ABSTRACT A new method is introduced that makes use of sparse image repre- sentations to search for approximate nearest neighbors (ANN) under the normalized inner-product distance. The approach relies on the construction of a new sparse vector designed to approximate the nor- malized inner-product between underlying signal vectors. The re- sulting ANN search algorithm shows significant improvement com- pared to querying with the original sparse vectors. The system makes use of a proposed transform that succeeds in uniformly distributing the input dataset on the unit sphere while preserving relative angular distances. Index Terms— Sparse representations, indexing, data condi- tioning. 1. INTRODUCTION Local descriptors computed on affine normalized image regions have proven successful in computer vision applications requiring image matching and recognition [1]. The selected regions are affine nor- malized to be invariant under common transformations such as those resulting from camera perspective or illumination changes. Different descriptors y have been developed to describe the resulting normal- ized image regions. At query time, a nearest neighbors (NN) search is carried out between the query descriptor y q and the descriptor vec- tors y b computed on the database images. Yet since local descriptors are high-dimensional, they are subject to the curse of dimensionality [1], meaning that the NN search complexity is very high. Approximate nearest neighbor (ANN) searches based on vari- ous sparse representation schemes have been recently proposed to address the high computational complexity in local descriptor query systems [2, 3, 4]. Given some sparse representation x b of each y b , the search index will be the sparse matrix with columns x b . This sparse matrix index is stored in compact row-major format by group- ing all non-zero coefficients of any given row (and their column indices) to form a contiguous memory bin. This results in an im- plicit complexity-reducing pruning mechanism when using similar- ity measures based on the inner-product x t q x b , as only the bins cor- responding to the non-zero positions of x q need to be processed. The work carried out in the present paper uses sparse vectors x that are a dictionary-based sparse representation (DBSR) of the cor- responding y : Given an overcomplete matrix D (the dictionary), x will satisfy y = Dx + r . The chosen x produces an approximation Dx minimizing the distortion |r | under a constraint on the number of non-zero positions of x (a measure of rate). Using the sparse vec- tors x built following a rate-distortion criterion raises a new prob- lem: The residual transformations following the geometrical region normalization result in descriptors y that have DBSR x with unsta- ble support (positions of non-zero coefficients). This instability can severely impact the similarity score between regions and therefore the ranking performance of the ANN search task. In this paper, we address the problem of instabilities in the sup- port of DBSRs x by first modelling the instabilities and then con- structing, from x , a new vector which we call a reduced vector. The construction of the reduced vector is formulated as a minimization of the reference distance approximation error subject to a sparsity constraint as, for sparse matrix indices, sparsity is related to both memory and computational complexity. Computational complexity is further maximized for a uniform distribution of non-zero positions in x , which is in turn favored by a uniform distribution of y on the unit sphere. Thus we further introduce a data conditioning method which succeeds in approximately preserving the relative position of data points y on the unit sphere while making their distribution more uniform. One potential application of our proposed method is that of enhancing the performance/complexity tradeoff of bag-of-features (BOF) indices [3]. BOFs make use of vector quantization (VQ), which is a specific case of the more general DBSR framework. Thus VQ again uses a rate-distortion criterion that does not favor a sta- ble support of the resulting x , an issue addressed by our proposed method. A second potential application involves using x to design a low rate image/local descriptors package for the case when the normal- ized image regions are used directly as descriptors y . Transmitting the x thus obtained yields an initial image estimate at no extra rate penalty. Since the receiver requires x rather than y for querying or indexing, including x in the transmitted package further exempts the receiver from descriptor extraction and processing. The rest of this paper is organized as follows: we present the proposed reduced vector construction strategy in section 2. Our al- gorithm makes use of an adaptive sparse correlation matrix, and we explain how to obtain it in section 3. Our proposed data conditioning method is then presented in section 4, and evaluated along with our main approach in section 5. We provide concluding remarks in the last section. 2. FORMULATING SPARSE SUPPORT SELECTION AS AN OPTIMIZATION PROBLEM We now explain the method used to build the proposed reduced vec- tors from the sparse representations x . Reduced vectors enjoy a more stable support relative to x and are thus better suited for ANN searches based on sparse matrix indices. Assuming all y = Dx + r to be normalized and compress- ible (i.e., with negligible r ), we first expand hy q ,y b i as x t q CDx b = χ t q x b ,where CD = D t D and χ q = CDx q ; the operator h·, ·i de- notes correlation (normalized inner-product). Since computational