AN ONLINE MOTION-BASED PARTICLE FILTER FOR HEAD TRACKING APPLICATIONS Nidhal Bouaynaya, Wei Qu and Dan Schonfeld University of Illinois at Chicago Department of Electrical and Computer Engineering nbouay1@uic.edu, wqu@ece.uic.edu, ds@ece.uic.edu ABSTRACT Particle filtering framework has revolutionized probabilistic tracking of objects in a video sequence. In this framework the proposal density can be any density as long as its sup- port includes that of the posterior. However, in practice, the number of samples is finite and consequently the choice of the proposal is crucial to the effectiveness of the tracking. The CONDENSATION filter uses the transition prior as the proposal density. We propose in this paper a motion-based proposal. We use Adaptive Block Matching (ABM) as the motion estimation technique. The benefits of this model are two fold. It increases the sampling efficiency and handles abrupt motion changes. Analytically, we derive a Kullback- Leibler (KL)-based performance measure and show that the motion proposal is superior to the proposal of the CON- DENSATION filter. Our experiments are applied to head tracking. Finally, we report promising tracking results in complex environments. 1. INTRODUCTION Reliable object tracking in complex environments is a chal- lenging task. Its applications include video surveillance [1], autonomous vehicle navigation [2], and virtual reality [3] among many others. Recently temporal Bayesian filtering [4], [5] has become very popular for object tracking. In this probabilistic framework, the goal is to estimate the system’s current state given its past and current observations. How- ever, except for the linear gaussian case (Kalman filter [5]) the problem does not admit an analytical solution. More- over, real world object tracking does not satisfy kalman fil- ter requirements: the system dynamics can be highly non- linear and the observation density is multimodel due to clut- ter. Particle filters can handle non-linear and non-Gaussian systems. The idea is to approximate the posterior density by its sample set. Since it is hard to sample directly from the posterior, Particle filter employs the Importance Sampling technique [6], [7], [8]. In Importance Sampling, a proposal density, also called importance function, is used to generate samples. Each sample is then assigned a proper weight to make up the difference between the posterior density and the proposal density. It can be shown that (i) the compen- sated sample set is a fair approximation of the posterior and (ii) if number of samples is sufficiently large, the sample ap- proximation of the posterior density can be made arbitrarily accurate [9], [10]. However, in practice the resources are fi- nite. To make the situation worse, if a good dynamic model is not available or if the state dimension of the tracked ob- ject is high, the number of required samples becomes even larger and Particle filter can be computationally prohibitive. Choosing the right proposal density is one of the most im- portant issues in particle filters’ design. In this paper we propose to use a motion-based proposal density. The benefits of this model are two fold. First the motion estimation allows efficient allocation of the samples. Second the tracker is adaptive to all kinds of motion. Partic- ularly it handles sudden and unexpected motion; e.g., a mo- tion that is not captured by the state transition model. We choose Adaptive Block Matching technique because of its simplicity in implementation [11]. However, our model can be generalized with any motion estimation algorithm. The rest of the paper is organized as follows. In Section 2, we construct the motion proposal using the motion vector esti- mated by the ABM. In Section 3, we set up an optimization problem based on the KL performance measure and prove that the motion proposal is superior to the proposal used in the CONDENSATION filter, i.e., the transition prior. In Section 4, we apply our algorithm to head tracking using challenging real-world video sequences. Concluding re- marks and future work are given in Section 5. 2. A MOTION-BASED PARTICLE FILTER We assume a Markovian discrete-time state space model. Let X k represent the target characteristics at discrete time k (position, velocity, shape, etc). The state space model is described by a state transition and measurement equations. The goal is to estimate the posterior density p(X) of the tar- get given its past and current observations. In what follows, II - 225 0-7803-8874-7/05/$20.00 ©2005 IEEE ICASSP 2005