Pedestrian tracking based on colour and spatial information Florian H. Seitner * and Brian C. Lovell † * Pattern Recognition and Image Processing Group, Vienna University of Technology, AUSTRIA † National ICT Australia and School of ITEE, The University of Queensland, AUSTRALIA {seitner, lovell}@itee.uq.edu.au Abstract This paper describes a tracking with appearance modelling system for pedestrians. A cascade of boosted classifiers and Haar-like rectangular fea- tures [6, 12] are used for the pedestrian detection. Statistical modelling in the HSV colour space is used for adaptive background modelling and subtraction, where the use of circular statistics for hue is proposed. By using the background model in combination with the detector, the system extracts a feature vector based on colour statistics and the spatial information. Circular [9] and linear statistics are applied on the extracted features to robustly track the pedestrians and other moving objects through the scene. An adaptive appearance model copes with partial or full occlusions and ad- dresses the problem of missing or wrong detections in single frames. Keywords: tracking, background segmentation, appearance model, HSV, circular statistic, Haar- like features 1. Introduction The proposed tracking system uses a background segmentation algorithm in combination with an ob- ject classifier to quickly find pedestrians in each video frame. After detection of a possible pedes- trian, the moving object is subdivided into three zones (head, upper body, and lower body) and the colour and spatial properties of each part which form the basic appearance model in this system are extracted. The colour information is analyzed in the HSV (Hue, Saturation and Value) colour space which provides a natural means of colour represent- ation. The HSV colour model describes each colour by one angular (Hue) and two linear values (Satur- ation and Value). Although HSV has been applied to a wide range of applications like motion analysis, background modelling, and image retrieval, often its mixed topological nature of linear and circular domains is not appropriately taken into account. For example, it is clear that the mean of angles 359 ◦ and 1 ◦ is not 180 ◦ like the arithmetic mean would yield — it should be 0 ◦ . Furthermore, twins born one minute before and one minute after mid- night are born only two minutes apart — not 23 hours and 58 minutes. Therefore, important defin- itions of circular statistics are given in Section 2 and used in this work to accurate process direc- tional hue data. Section 3 describes how color distributions can be approximated by parametric descriptions and how the adaptive background model distinguishes between foreground and background. The detector (Section 4) uses a cascade of boosted classifiers and Haar-like features to describe pedestrians in a highly efficient way. Section 5 describes how track- ing features can be extracted by using the obtained pedestrian detections and the foreground data. The structure of the adaptive appearance model is de- scribed in Section 6 and results and conclusions of this work are given in Section 7 and Section 8. 2. Applied circular statistics The algebraic structure of the line and the circle are different and therefore adequate methods of circular data analysis as discussed in [9] must be used when working with directional data. In contrast to the linear domain only one operation, the addition modulo 2π is available in the circular domain. Due to the fact that the circle is a closed