2008 IEEE Swarm Intelligence Symposium St. Louis MO USA, September 21-23, 2008 978-1-4244-2705-5/08/$25.00 ©2008 IEEE Image-Based Tracking with Particle Swarms and Probabilistic Data Association Edward Kao, Peter VanMaasdam, John Sheppard The Johns Hopkins University {ekao3, pvanmaa1, jsheppa2}@jhu.edu Abstract - The process of automatically tracking people within video sequences is currently receiving a great deal of interest within the computer vision research community. In this paper we contrast the performance of the popular Mean-Shift algorithm’s gradient descent based search strategy with a more advanced swarm intelligence technique. Towards this end, we propose the use of a Particle Swarm Optimization (PSO) algorithm to replace the gradient descent search, and also combine the swarm based search strategy with a Probabilistic Data Association Filter (PDAF) state estimator to perform the track association and maintenance stages. Performance is shown against a variety of data sets, ranging from easy to complex. The PSO-PDAF approach is seen to outperform both the Mean-Shift + Kalman filter and the single- measurement PSO + Kalman filter approach. However, PSO’s robustness to low contrast and occlusion comes at the cost of higher computational requirements. I. INTRODUCTION Automated detection and tracking of people within video sequences is a topic that is currently receiving a great deal of interest within the computer vision research community. For many surveillance problems, such as border or other perimeter security applications, the region under video surveillance is simply too large for continuous human observation of the video streams. Thus, some form of automation is required to alert the human observers to the presence of objects (e.g. people or vehicles) within the surveillance region and to maintain track on these objects while the human decides what further action to take. Persistent tracking is an important objective under such an application. It allows for observations on the subject of interest over time that may be used to infer the intent and activities of the subject. In order to perform persistent tracking in a real world application, the tracking algorithm must be robust to difficult operating conditions such as low contrast (between the objects and competing clutter) and occlusions of the objects being tracked. Another practical constraint is the need for long-range acquisition and tracking, which usually results in a reduced number of pixels on target when compared to most image-based tracking experiments presented by the research community. At present, the most widely used method for image-based tracking is the Mean-Shift algorithm with a Kalman filter state estimator [4]. This algorithm has been very successful, but does have several limitations. The most prominent limitation is that, as a gradient descent based search strategy, the Mean-Shift algorithm is susceptible to converging to a local optimum instead of the global optimum, a problem that often occurs when objects with similar appearance surround the object being tracked. In addition, Mean-Shift tracking, even with the help of a Kalman filter prediction step, often fails to initialize the gradient descent search at a location close enough to the object in the new frame under challenging conditions such as occlusion and rapidly changing object velocity. This can lead to a loss of track because the gradient descent does not perform an adequate search to re-discover the object’s location in the new frame. In this paper, we present a swarm intelligence based tracking approach whereby Particle Swarm Optimization (PSO) replaces the Mean-Shift’s gradient descent search strategy, while maintaining the use of the Mean-Shift feature in calculating the fitness function. In addition, we replace the traditional Kalman filter state estimator with a Probabilistic Data Association Filter (PDAF). We hypothesize that this approach will adequately address the Mean-Shift issues stated above while avoiding an exhaustive search in the region of interest. Comparisons are made using color imagery from a single camera against a variety of scenarios, ranging from simple to complex. The remainder of the paper is organized as follows. Section II describes the basic formulation of an image-based tracking system. Section III describes the Mean-Shift feature and gradient descent search, as well as the alternative PSO search strategy. In Section IV, we propose a novel PSO-PDAF approach for image-based tracking. Section V consists of a description of the test sequences, with Section VI reporting the results of the experiments. Conclusions and future work are discussed in Section VII. II. IMAGE-BASED TRACKING The classical single camera image-based tracking problem can usually be described as accomplishing the following tasks. Detection is performed on the current image frame to generate Regions of Interest (ROIs). This detection can take the form of a simple double or triple window Constant False Alarm Rate (CFAR) detector, a matched filter, Moving Target Indication (MTI), etc [10]. For this study, the initial detection consists of a ground truth location at which to initiate tracking, which allows us to contrast algorithm performance free of any initial biases introduced by a detection algorithm. We assume that, in a practical implementation, a more autonomous detection component would replace the ground truth locator. Detections are then assigned to existing tracks in the Association phase. This step, within a correlation-based tracker, is often performed simultaneously with the detection phase, where a small region about each predicted track location in the image frame is searched for a peak correlation with the template. Many methods exist to perform this Authorized licensed use limited to: MONTANA STATE UNIV BOZEMAN. Downloaded on January 9, 2009 at 11:30 from IEEE Xplore. Restrictions apply.