SWITCHING TEMPLATE FITTING METHODS DURING ARTICULATED OBJECT TRACKING Martin Tosas, Bai Li and Steven Mills School of Computer Science and IT, University of Nottingham Jubilee Campus, Nottingham, NG8 1BB, UK mtb@cs.nott.ac.uk Keywords: articulated object tracking, hand tracking, particle filters, deformable template fitting. Abstract This paper describes two methods of fitting deformable templates when tracking articulated objects using particle filters. One method fits a template to each of the links of an articulated object in a hierarchical way. The method first fits a template for the base of the articulated object and then fits a template for each of the links deeper in the hierarchy. The second method fits the whole articulated object as a rigid object, and then refines the fitting for each of the links of the articulated object in a hierarchical way, starting from the base. Advantages and disadvantages of each method are discussed and a way of combining the best of each method in a single tracker is presented. 1 Introduction Tracking of articulated objects is an area of great interest in computer vision. The most interesting examples of articulated tracking include tracking of the human body and human hands. Tracking of the human body can be used in applications such as surveillance, gait analysis and behaviour recognition. Tracking of human hands has applications in human computer interaction. The classical approach to articulated tracking uses kinematic models and deterministic algorithms to find the position of the various links of an articulated object [13]. However in recent years the use of particle filters together with deformable templates has become a popular way of dealing with various forms of tracking, including articulated objects [8, 10]. One advantage of particle filters in visual tracking, with respect to the classical approaches, is that they can handle tracking of objects against cluttered backgrounds. This is possible because particle filters can keep several hypotheses of target configurations at the same time. Some of these hypotheses can represent the target configuration with certain accuracy; others can be located on background features, not representing accurately the target, and therefore may later be rejected [8]. When tracking articulated objects using particle filters and deformable templates, a template can represent each link of the articulated object. Various template configurations for a given link would make up hypotheses of what the configuration of this link is. The process of fitting a template to a link involves finding a set of hypotheses that represent this link accurately. Then one of two methods can be used to find the configuration of an articulated object. In one method, the base, or root link is fitted first, then the links at the second level of the hierarchy 1 are fitted, then the links at the third level, etc. Each link is configured relative to the previous link. An alternative method for finding the configuration of an articulated object is to fit the whole articulated object as one rigid object, according to some previous configurations for each of its links. This initial match is then refined by adjusting the fitting for each link starting from the base of the articulated object and progressing to the links deeper in the hierarchy. In this paper we discuss the advantages and disadvantages of each of these two methods of fitting the links of an articulated object. Results are given for the case of an articulated hand tracker. We also propose a way of combining both methods in a single tracker. The paper is organized as follows: Section 2 is a brief literature review of related work; Section 3 describes the articulated hand tracker used for the experiments; Section 4 describes the two methods of fitting templates to the links of an articulated object, and shows results of the two methods, separately and in combination, for the articulated hand tracker; Finally Section 5 gives some conclusions, and future work directions. 2 Related work The most common applications for articulated object trackers are human body tracking and hand tracking. Articulated object trackers have evolved from using kinematics models and various deterministic approaches [13, 12, 17] to particle filters [2, 8, 10] and approaches that include an off-line calculated hierarchy of all the valid object shapes [7, 15]. 1 An articulated object inherently has a tree structure, the root of the tree would be the first link in the hierarchy, and the various levels of branches would make up the levels deeper in the hierarchy.