Correspondences of Point Sets using Particle Filters Rolf Lakaemper, Shusha Li, Marc Sobel Temple University, Philadelphia, PA [lakamper,shusha,marc.sobel]@temple.edu Abstract The paper shows how Particle Filters can be used to establish visually consistent partial correspondences between similar features in unrestricted 2D point sets representing shapes. Given an update rule, the PF sys- tem has the advantage that global constraints can be learned. We motivate and define the update rule for the given task and show its superior performance in com- parison to a globally unrestricted approach. 1. Introduction For many years, finding correspondences between visual features of pairs of shapes has been the focus of computer vision research. Solutions involving shapes with closed boundaries have been particularly impres- sive, e.g., [1], [3], [2]. In [4], we introduced Particle Filtering (PF) to solve a minimization problem to find correspondences between features of shapes and shape parts, represented by polygonal boundaries. The most important feature of that approach was to distinguish between local and global constraints of the correspon- dence problem. This idea leads to elegant implementa- tion of the global constraints, since the iterative multi- ple hypothesis PF approach allows the system to learn these from the feedback obtained from current corre- spondence hypotheses. [4] describes the PF system as a general framework in which the importance of cor- respondences are defined via: (i) a matrix representing local constraints, and (ii) an update rule which itera- tively generate a second matrix characterizing global constraints. The update rule implements the aforemen- tioned feedback cycle. Figure 1 illustrates this idea. The main contribution of this paper is to demonstrate how the versatile PF tool can be utilized to find partial correspondences between unordered point sets, repre- senting e.g. partial shapes with inner structures. We carry out the new task by redefining the update rule for global constraints, hence only the constraints are ad- justed in comparison to an otherwise identical system Figure 1. Global and local constraints influence the PF process. In our approach, the current particle and the lo- cal constraints are fed back to update the global constraints: global constraints are learned during the PF process. to [4]. We will give a brief introduction to the PF sys- tem, motivate and define the update rule. We will also discuss the role of global constraints in the algorithms performance. 2 Related Work The dynamic programming (DP) approach to shape feature correspondence detection defines the correspon- dence problem as one of minimizing an energy or cost function depending only on the local feature constraint matrix, e.g. [6] Shape feature correspondence detection with Dynamic Programming. The global constraints, in e.g., [6] consist in order preservation which is im- plicit in the algorithms structure, and hence, not eas- ily changeable. In contrast to dynamic programming, which has fixed global constraints, our’s are learned in particle filter settings (PF). This makes DP a pre- ferred approach for relatively simple tasks, e.g. corre- spondence of closed boundaries, while the PF system shows its advantages in problems requiring more com- plex global constraints. The process of finding correct correspondences can be seen as a labeling process. The features of one shape correspond to the labels; the features of the other have 978-1-4244-2175-6/08/$25.00 ©2008 IEEE