AUTOMATIC MULTI-HEAD DETECTION AND TRACKING SYSTEM USING A NOVEL
DETECTION-BASED PARTICLE FILTER AND DATA FUSION
Wei Qu, Nidhal Bouaynaya and Dan Schonfeld
Multimedia Communications Laboratory,
ECE Department, University of Illinois at Chicago, Chicago, IL 60607.
E-mail: {wqu, nbouayna, ds}@ece.uic.edu
ABSTRACT
We present a novel automatic system integrating head de-
tection with particle filter for realtime multi-head tracking
(MHT) in video. Distinct with the conventional particle fil-
ter which gets particles from the prior density, we propose
a novel importance function based on the up to date detec-
tion and motion observation which makes the particles more
effective and helps us to achieve stable tracking by using
much less particles. We also propose a general likelihood
model in the context of MHT. Different information can be
fused in a principle manner to make the tracker more stable.
The proposed approach can handle not only the changes of
scale, lighting, zooming, and pose, but also fast motion, ap-
pearance, and hard multi-head occlusion.
1. INTRODUCTION
Head detection and tracking has been an intensive research
area due to its wide applications. However, because of lack-
ing effective representation and good scheme to handle oc-
clusion, robust and efficient head tracking especially MHT
in complex environment is still an open research problem.
Particle filter has received much attention in recent years.
Consider a dynamic system presented by the continuous-
time Hidden Markov Model. The tracking problem is to
estimate the posterior p(x
t
|Z
1:t
) by the Bayesian inference
p(x
t
|Z
1:t
) ∝ kp(z
t
|x
t
)p(x
t
|Z
1:t-1
) (1)
where k is the normalization constant and
p(x
t
|Z
1:t-1
)=
p(x
t
|x
t-1
)p(x
t-1
|Z
1:t-1
) dx
t-1
. (2)
Because the likelihood p(z
t
|x
t
) is usually nonlinear, non-
Gaussian which makes the integral unfeasible, the posterior
density can be approximated by properly weighted particles
sampled from any proposal distribution q (also called the
importance function) [1]. Since particles are sampled from
p(x
t
|Z
1:t-1
) and weights are only computed by likelihood,
the standard particle filter is vulnerable to the degeneracy
Importance
Function
Prior
System Dynamics
Likelihood
Motion Estimation
Skin&Hair Color Model
Color Histogram
Edge Information
Template Matching
Head Detector
Importance
Function
Prior
System
Dynamics
Likelihood
Tracker using
Mixed Particle Filters
Data
Fusion
Subtracker 1
Subtracker M
Fig. 1. The structure of MHT system.
problem [2]. [3] has proved that the optimal importance
function which minimizes the variance of weights is
q
opt
= p(x
t
|x
t-1
,Z
t
) (3)
where Z
t
is the newest observation. But no realization is
given in this paper and most discussion in this context limits
to single object tracking. In this paper, we propose a novel
optimal importance function suitable for MHT.
The paper is organized as follows. Section 2 presents the
system structure. Section 3 and 4 describe the detection and
motion estimation methods we used respectively. Section 5
discusses the detection and motion-based MHT framework
we proposed. Experiment results are given in Section 6. In
Section 7, we conclude this paper.
2. SYSTEM STRUCTURE
Fig. 1 shows the system structure. The tracker is composed
of mixed particle filters where each keeps tracking one head.
Up to date motion and detection information has been used
to make the importance function optimal. The detector is
also used for initialization. Different information has been
fused together to make the tracker more stable. To handle
multi-head occlusion where the particle filters are depen-
dent, the likelihood is associated with corresponding head
under the following assumptions: I. One head can produce
zero or one observation at one time. II. One observation can
II - 661 0-7803-8874-7/05/$20.00 ©2005 IEEE ICASSP 2005