Tracking Multiple Mouse Contours (without Too Many Samples) Kristin Branson and Serge Belongie Dept. of Computer Science and Engineering UC San Diego La Jolla, CA 92093-0114 Abstract We present a particle filtering algorithm for robustly tracking the contours of multiple deformable objects through severe occlusions. Our algorithm combines a mul- tiple blob tracker with a contour tracker in a manner that keeps the required number of samples small. This is a natu- ral combination because both algorithms have complemen- tary strengths. The multiple blob tracker uses a natural mul- titarget model and searches a smaller and simpler space. On the other hand, contour tracking gives more fine-tuned results and relies on cues that are available during severe occlusions. Our choice of combination of these two algo- rithms accentuates the advantages of each. We demonstrate good performance on challenging video of three identical mice that contains multiple instances of severe occlusion. 1. Introduction We address the problem of tracking the contours of mul- tiple identical mice from video of the side of their cage; see Figure 3 for example frames. Although existing tracking al- gorithms may work well from an overhead view of the cage, the majority of vivaria are set up in a way that prohibits this view. A solution to the side view tracking problem would be very useful for medical researchers wishing to automati- cally monitor the health and behavior of lab animals [3]. This problem is also interesting and uniquely difficult from a computer vision standpoint. Because mice are highly deformable 3D objects with unconstrained motion, an accu- rate contour model is necessarily complex. Because mouse motion is erratic, the distribution of the current mouse po- sitions given their past trajectories has high variance. The biggest challenge to tracking mice from a side view is that the mice occlude one another severely and often. Tracking the mice independently would inevitably result in two track- ers following the same mouse. Instead, we need a multitar- get algorithm that tracks the mice in concert. As the num- ber of parameters that must be simultaneously estimated increases linearly with K, the number of mice, the search space size increases exponentially with K [13]. Thus, us- ing existing approaches to directly search the contour space for all mice at once is prohibitively expensive. In addition, tracking individual mouse identities is dif- ficult because the mice are indistinguishable. We cannot rely on object-specific identity models (e.g. [4, 9]) and must instead accurately track the mice during occlusions. This is challenging because mice have few if any trackable fea- tures, their behavior is erratic, and edges (particularly be- tween two mice) are hard to detect. Other features of the mouse tracking problem that make it difficult are clutter (the cage bedding, scratches on the cage, and the mice’s tails), inconsistent lighting throughout the cage, and moving re- flections and shadows cast by the mice. Our algorithm is of general interest to the tracking com- munity because the challenges to successful mouse track- ing are common to many real world tracking applications. While many video sequence testbeds are constructed to show off the novelty of an algorithm, our algorithm is con- structed to address the challenges of a specific tracking problem. Thus, our feature extraction algorithm must be powerful, our objects’ state representation must be detailed, and our algorithm must be able to search the complex pa- rameter space with a limited number of samples. We propose a solution that combines existing blob and contour tracking algorithms. However, just combining these algorithms in the obvious way does not effectively solve the difficulties discussed above. We propose a novel combina- tion of these algorithms which accentuates the strengths of each individual algorithm. In addition, we capitalize on the independence assumptions of our model to perform most of the search independently for each mouse. This reduces the size and complexity of the search space exponentially, and allows our Monte Carlo sampling algorithm to search the complex state parameter space with a reasonable number of samples. Our algorithm works with a detailed representa- tion of a mouse contour to achieve encouraging results on a video sequence of three mice exploring a cage. The paper is organized as follows. In Section 2, we de- scribe the algorithms we build off of: the Bayesian Multiple Blob (BraMBLe) tracker [7] and MacCormick et al.’s con- 0-7695-2372-2/05/$20.00 (c) 2005 IEEE