Pattern Reco#nition, Vol. 26, No. 4, pp. 595 610, 1993 Printed in Great Britain 0031 3203/93 $6.00+.00 Pergamon Press Ltd Pattern Recognition Society MATCHING MOTION TRAJECTORIES USING SCALE-SPACE KRISHNAN RANGARAJAN, WILLIAM ALLEN and MUBARAKSHAH~" Computer ScienceDepartment, University of Central Florida, Orlando, FL 32816, U.S.A. (Received 22 January 1992; in revised form 8 July 1992; received for publication 18 Auqust 1992) A~traet--The goal is to design a recognition system which can distinguish between two objects with the same shape but different motion, or between two objects with the same motion but a different shape. The input to the system is a set of two-dimensional (2D) trajectories from an object tracked through a sequence of n frames. The structure and three-dimensional (3D) trajectories of each object in the domain are stored in the model. The problem is to match the information in the model with the input set of 2D trajectories and determine if they represent the same object. The simplest way to perform these steps is to match the input 2D trajectories with the 2D projections of the 3D model trajectories. First, a simple algorithm is presented which matches two single trajectories using only motion information. The 2D motion trajectories are converted into two one-dimensional (1D) signals based on their speed and direction components. The signals are then represented by scale-space images, both to simplifymatching and because the scale-space representations are translation and rotation invariant. The matching algorithm is extended to include spatial information and a second algorithm is proposed which matches multiple trajectories by combining motion and spatial match scores. Both algorithms are tested with real and synthetic data. Motion analysis Motion representation Recognition Scale-space Trajectory matching I. INTRODUCTION One of the goals of computer vision is to design object recognition systems that can identify a specific object of interest. Previous research in object recognition has relied exclusively on shape information. Shape is a very important attribute which defines the form and spatial arrangement of an object, and is invariant to certain transformations. Other information about an object exists, however, such as motion, specularity and texture, which may prove useful in recognition. Our research has been directed toward the use of motion information in object recognition. We believe that in many cases, where an object has a fixed and predefined motion, the trajectories of several points on the object may serve to uniquely identify the object. Therefore, it should be possible to recognize certain objects based on motion information obtained from the trajectories of representative points. We have developed a method for matching sets of trajectories which supplements motion information with knowl- edge about the spatial relationships between certain points on the object. The input to our system is a set of two-dimensional (2D) trajectories from an object tracked through a sequence of n frames. These trajectories are segmented by identifying each trajectory with a single object in the image, tu The structure and three-dimensional (3D) trajectories of each object in the domain are stored in the model. The problem is to match the information in the model with the input set of 2D trajectories. The f Author to whom correspondence should be addressed. matching process has two main steps: (1) establish a one-to-one, onto correspondence between the object model 3D trajectories and the input set of 2D trajec- tories, and (2) determine the position of the object in the model coordinate system such that the input 2D trajectories closely match the corresponding 2D pro- jections of the model's 3D trajectories. The simplest way to perform these steps is to match the input 2D trajectories with the 2D projections of the 3D model trajectories. In this paper, we propose a multi-scale approach for matching 2D trajectories. First, we pre- sent Algorithm A, which matches two single trajectories using only motion information. We then extend that algorithm to include spatial information and propose Algorithm B, which matches multiple trajectories. 2. RELATED WORK We will briefly survey recent'research in motion and recognition. A great deal of work has been done in the field of psychology to show that people can recognize objects from their trajectories. It has been theorized that humans can recognize an object based on the motion of several points on that object by inferring the 3D structure of the object from the transform- ations the 2D image undergoes. Todd 12) is interested in distinguishing between rigid and several types of non-rigid motion such as bending, stretching, twisting and flowing. By displaying the trajectories of either rigid or non-rigid objects, Todd shows that human observers are able to distinguish between the two. Cutting13} and Johansson ~4) discuss the relative motion of individual parts of an object and the common 595