Rao-Blackwellised Particle Filter with Adaptive System Noise and its Evaluation for Tracking in Surveillance Xinyu Xu, Baoxin Li Center for Cognitive Ubiquitous Computing Dept. of Computer Science and Engineering, Arizona State University 699 South Mill Avenue, Tempe, AZ, USA 85281 {xinyu.xu and baoxin.li}@asu.edu ABSTRACT In the visual tracking domain, Particle Filtering (PF) can become quite inefficient when being applied into high dimensional state space. Rao-Blackwellisation [1] has been shown to be an effective method to reduce the size of the state space by marginalizing out some of the variables analytically [2] . In this paper based on our previous work [3] we propose an RBPF tracking algorithm with adaptive system noise model. Experiments using both simulation data and real data show that the proposed RBPF algorithm with adaptive noise variance improves its performance significantly over conventional Particle Filter tracking algorithm. The improvements manifest in three aspects: increased estimation accuracy, reduced variance for estimates and reduced particle numbers are needed to achieve the same level of accuracy. The last two performance improvements are evaluated in this paper using simulation data. 1. INTRODUCTION Visual tracking is an important step in many practical applications including video-based surveillance. In recent years, particle-filter-based visual tracking has been extensively studied [4, 5, 6 and 7] . Particle filtering has been shown to offer improvements in performance over some conventional methods such as the Kalman filter, especially in non- linear/non-Gaussian environments [8] , but the large number of samples required to represent the posterior density prevent its use in high dimensional state-space. However, in some cases, the model may have “tractable structure” with some components having linear dynamics and so can be marginalized out and analytically estimated using exact filters conditional on certain other components. The exact filters could be the Kalman filter, the HMM filter, or any other finite dimensional optimal filters [1] . This technique is called Rao-Blackwellisation [2] . The resultant method is often called Rao-Blackwellised particle filter (RBPF). RBPF has been studied in the context of multi-target tracking [9] , fault diagnosis for robot and robot localization [10, 11] and signal processing [12] . In addition, in our previous work [3] , with video-based surveillance as a case study, we utilized the constraints imposed by typical camera-scene configuration to partition the original state space into two sub-spaces, and then we proposed a RBPF algorithm for surveillance tracking. However, in most existing tracking algorithms including both RBPF and PF, the noise variance in the system motion model is typically static which makes the tracker unstable and un-robust when tracking objects with dramatically and fast changing velocity. To overcome this drawback and based on our existing work, in this paper we present a RBPF algorithm with adaptive system noise model. Furthermore, a systematic evaluation of a RBPF algorithm over conventional particle filtering is not available in the visual tracking literature. Motivated by this fact, in this paper we evaluate the improvements of RBPF over conventional particle filter based on the RBPF algorithm with adaptive system noise model. Experimental results show that the improvements of RBPF over conventional PF manifest in the following three aspects: reduced variance for estimates, reduced number of particles needed to achieve the same level of accuracy, and increased estimation accuracy. Denote the state to be tracked as X t and observation as Z t with subscript t the time index. The key idea of RBPF for tracking is to partition the original state-space X t into two parts R t (root variables), and L t (leaf variables), such that p(L 1:t |R 1:t ,Z 1:t ) can be analytically updated using an exact filter, and thus the approximation to p(R 1:t |Z 1:t ) using a Monte Carlo method yields straightforwardly an approximation to joint posterior p(R 1:t ,L 1:t |Z 1:t ). The justification for this decomposition follows from the factorization of the probability: p(R 1 :t ,L 1 :t |Z 1 :t )=p(L 1 :t |R 1 :t ,Z 1 :t )p(R 1 :t |Z 1 :t ) (1)