Hybrid Stochastic / Deterministic Optimization for Tracking Sports Players and Pedestrians ⋆ Robert T. Collins 1 and Peter Carr 2 1 The Pennsylvania State University, USA 2 Disney Research Pittsburgh, USA Abstract. Although ‘tracking-by-detection’ is a popular approach when reliable object detectors are available, missed detections remain a diffi- cult hurdle to overcome. We present a hybrid stochastic/deterministic optimization scheme that uses RJMCMC to perform stochastic search over the space of detection configurations, interleaved with deterministic computation of the optimal multi-frame data association for each pro- posed detection hypothesis. Since object trajectories do not need to be estimated directly by the sampler, our approach is more efficient than traditional MCMCDA techniques. Moreover, our holistic formulation is able to generate longer, more reliable trajectories than baseline tracking- by-detection approaches in challenging multi-target scenarios. 1 Introduction Multi-target tracking of pedestrians and sports players is difficult due to the presence of many similar-looking objects interacting in close proximity. For this reason there has been recent interest in sliding temporal window methods that recover tracking solutions by considering a batch of frames at a time. The mo- tivation is that people who are occluded or otherwise difficult to disambiguate in a few frames will be easier to find in others, and that propagating tempo- ral consistency constraints both backwards and forwards in time leads to better solutions than purely causal processing. It is also advantageous to solve for detections and data association jointly, rather than computing detections first and then linking them into trajectories. Despite the obvious benefits, this holistic approach has received considerably less attention because the complexity of the search space of data association in- creases exponentially with the number of candidate detections in each frame, and therefore committing to a small set of high-quality discrete detections makes the later association problem more manageable. However, not being able to recon- sider detection decisions puts a large burden on the data association algorithm to handle deficiencies such as missed detections and false positives. ⋆ Electronic supplementary material -Supplementary material is available in the online version of this chapter at http://dx.doi.org/10.1007/978-3-319-10605-2_20 . Videos can also be accessed at http://www.springerimages.com/videos/978-3- 319-10604-5. D. Fleet et al. (Eds.): ECCV 2014, Part II, LNCS 8690, pp. 298–313, 2014. c  Springer International Publishing Switzerland 2014