A Hierarchical Framework for Spectral Correspondence Marco Carcassoni and Edwin R. Hancock Department of Computer Science, University of York,York YO1 5DD, UK. {marco,erh}@cs.york.ac.uk Abstract. The modal correspondence method of Shapiro and Brady aims to match point-sets by comparing the eigenvectors of a pairwise point proximity matrix. Although elegant by means of its matrix repre- sentation, the method is notoriously susceptible to differences in the re- lational structure of the point-sets under consideration. In this paper we demonstratehowthemethodcanberenderedrobusttostructuraldiffer- encesbyadoptingahierarchicalapproach.Weplacethemodalmatching problem in a probabilistic setting in which the correspondences between pairwise clusters can be used to constrain the individual point corre- spondences. To meet this goal we commence by describing an iterative method which can be applied to the point proximity matrix to identify thelocationsofpairwisemodalclusters.Oncewehaveassignedpointsto clusters, we compute within-cluster and between-cluster proximity ma- trices. The modal co-efficients for these two sets of proximity matrices areusedtocomputeclustercorrespondenceandcluster-conditionalpoint correspondence probabilities. A sensitivity study on synthetic point-sets reveals that the method is considerably more robust than the conven- tional method to clutter or point-set contamination. 1 Introduction Eigendecomposition, or modal analysis, has proved to be an alluring yet elu- sive method for correspondence matching. Stated simply, the aim is to find the pattern of correspondence matches between two sets of objects using the eigen- vectors of an adjacency matrix or an attribute proximity matrix. The problem has much in common with spectral graph theory [1] and has been extensively studied for both the abstract problem of graph-matching [17,16], and for point pattern matching [14,12,11]. In the case of graph-matching the adjacency ma- trix represents either the weighted or unweighted edges of the relational structure under study. For point pattern matching, the proximity matrix represents the pairwise distance relationships. The method may be implemented in a number of ways. The simplest of these is to minimize the distance between the modal co- efficients. A more sophisticated approach is to use a factorization method such as singular value decomposition to find the permutation matrix which minimizes the differences between the adjacency structures. Unfortunately, the method in- variable fails when the sets of objects being matched are not of the same size due A. Heyden et al. (Eds.): ECCV 2002, LNCS 2350, pp. 266–281, 2002. c  Springer-Verlag Berlin Heidelberg 2002