A FRAMEWORK FOR SOFT HASHING AND ITS APPLICATION TO ROBUST IMAGE HASHING E. McCarthy, F. Balado, G.C.M. Silvestre and N.J. Hurley University College Dublin, Belfield, Dublin 4 – Ireland ABSTRACT An increasing interest in the soft hashing problem has been witnessed in recent times. Techniques implementing soft hashing intend to mirror the behaviour of cryptographic hashing when the information to be hashed can be subject to different kinds of distortions. Many heuristic techniques for undertaking soft hashing of images and other multime- dia data have been devised. Except for some attempts, a framework giving solid guidelines to solve the problem is largely lacking. In this extended summary we provide one possible approach to undertake the modelling of robust soft hashing, detailing the basic problems involved. We show how some prior schemes partly fit inside our model, and we provide an example of soft image hashing following the given scheme. 1. INTRODUCTION Soft hashing, also known as robust hashing or perceptual hashing, consists of summarising multimedia data, so as to obtain a concise representation called a hash value (also, fingerprint, message digest, or label). The hash generation procedure should be such that perceptually similar data yield the same hash value. The soft hashing problem is interesting for a number of scenarios which in most cases involve indexing of multimedia databases and/or authenti- cation, where the hash provides a compact representation which can be used to identify the data efficiently. Different applications impose different requirements, but usually soft hashing should significantly reduce the dimen- sionality of the data. This has to be done without substan- tially increasing the probability of collision (or false positive rate), i.e., the probability of having the same hash value for any two perceptually different data objects. In the case of authentication applications, the dimensionality reduction should be made using a one-way key-dependent function. Last, a relevant requirement in indexing applications is the robustness to (usually unintentional) distortions which do not affect the perceptual similarity of the multimedia data. Up to now, many different robust hashing schemes have been proposed for image hashing (see for instance [1, 2, 3, 4]), not forgetting those for other kinds of multimedia sig- nals. Nevertheless, a more general approach allowing the problem to be addressed in a systematic way is largely lack- ing. Previous proposals by Johnson and Ramchandran [5] Enterprise Ireland is kindly acknowledged for supporting this work under the Informatics Research Initiative. and by Mıh¸cak and Venkatesan [6] have partially tried to fill this gap already. In this extended summary we propose a soft-hashing framework to gain insight into the main design lines of these types of systems. We emphasize its robustness as- pect, as required by their application to database index- ing. We identify the central blocks of the problem, which, although already hinted at by different researchers in one way or another, are presented here in a unified way. Last, we propose an application of the developed methodology to the problem of image hashing. 2. SOFT HASHING FOR DATABASE INDEXING In the following, we will consider a soft hashing system for database indexing. Due to this, key-related security issues will not be discussed, and we will focus our efforts on the design of distortion-resistant soft hashing methods. The multimedia signal to be hashed will be denoted without loss of generality by a continuous-valued n-dimensional vector x =(x1,...,xn). In the general case, this vector may un- dergo some possibly random distortion function that we can write as f (·): R n → R n . Our objective is to map the signal x to an index belonging to a finite set H, and as indepen- dent as possible of the distortion function applied. A work- ing hypothesis is that distortions have to be constrained so that the distorted signal ˜ x = f (x) is not too different from x under some perceptually meaningful criterion (see Sect. 3.1). As depicted in Fig. 1, it is possible to divide database indexing systems using soft hashing into three quite inde- pendent blocks, namely: • Synchronisation. As in communications problems it is necessary that the signal y =(y1,...,ym) pre- sented to the hashing function always matches the same indices, in spite of possible desynchronisations undergone by x. Distortions that affect synchronism can be of very different nature, such as warpings, croppings, rotations, etc. It is not possible to insert synchronisation pilots in x, as is common practice in communications systems. Most existing soft hashing systems try to solve this issue through the use of fea- ture mappings. These mappings are just functions s(·): R n → R m that exploit geometrical invariances of x, and that are therefore very dependent on the nature of the multimedia signal. Examples of such mappings for images may be moments of different or- ders, or the Fourier-Mellin transform [7], and hence