International Journal of Computer Vision 59(3), 259–284, 2004 c  2004 Kluwer Academic Publishers. Manufactured in The Netherlands. SoftPOSIT: Simultaneous Pose and Correspondence Determination PHILIP DAVID University of Maryland Institute for Advanced Computer Studies, College Park, MD 20742, USA; Army Research Laboratory, 2800 Powder Mill Road, Adelphi, MD 20783-1197, USA DANIEL DEMENTHON, RAMANI DURAISWAMI AND HANAN SAMET University of Maryland Institute for Advanced Computer Studies, College Park, MD 20742, USA Received November 21, 2002; Revised September 3, 2003; Accepted November 18, 2003 Abstract. The problem of pose estimation arises in many areas of computer vision, including object recognition, object tracking, site inspection and updating, and autonomous navigation when scene models are available. We present a new algorithm, called SoftPOSIT, for determining the pose of a 3D object from a single 2D image when correspondences between object points and image points are not known. The algorithm combines the iterative softassign algorithm (Gold and Rangarajan, 1996; Gold et al., 1998) for computing correspondences and the iterative POSIT algorithm (DeMenthon and Davis, 1995) for computing object pose under a full-perspective camera model. Our algorithm, unlike most previous algorithms for pose determination, does not have to hypothesize small sets of matches and then verify the remaining image points. Instead, all possible matches are treated identically throughout the search for an optimal pose. The performance of the algorithm is extensively evaluated in Monte Carlo simulations on synthetic data under a variety of levels of clutter, occlusion, and image noise. These tests show that the algorithm performs well in a variety of difficult scenarios, and empirical evidence suggests that the algorithm has an asymptotic run-time complexity that is better than previous methods by a factor of the number of image points. The algorithm is being applied to a number of practical autonomous vehicle navigation problems including the registration of 3D architectural models of a city to images, and the docking of small robots onto larger robots. Keywords: object recognition, autonomous navigation, POSIT, softassign 1. Introduction This paper presents an algorithm for solving the model- to-image registration problem, which is the task of de- termining the position and orientation (the pose) of a three-dimensional object with respect to a camera co- ordinate system, given a model of the object consisting of 3D reference points and a single 2D image of these points. We assume that no additional information is available with which to constrain the pose of the object or to constrain the correspondence of object features to image features. This is also known as the simultaneous pose and correspondence problem. Automatic registration of 3D models to images is an important problem. Applications include object recog- nition, object tracking, site inspection and updating, and autonomous navigation when scene models are available. It is a difficult problem because it comprises two coupled problems, the correspondence problem and the pose problem, each easy to solve only if the other has been solved first: 1. Solving the pose (or exterior orientation) problem consists of finding the rotation and translation of the object with respect to the camera coordinate sys- tem. Given matching object and image features, one