Estimation and Prediction of Multiple Flying Balls Using Probability Hypothesis Density Filtering Oliver Birbach Udo Frese Abstract— We describe a method for estimating position and velocity of multiple flying balls for the purpose of robotic ball catching. For this a multi-target recursive Bayes filter, the Gaussian Mixture Probability Hypothesis Density filter (GM- PHD), fed by a circle detector is used. This recently developed filter avoids the need to enumerate all possible data association decisions, making them computationally efficient. Over time, a mixture of Gaussians is propagated as tracks, predicted into the future and then sent to the robot. By learning a prior from training data we are focusing on detections that are likely to lead to a catchable trajectory which increases robustness. We evaluate the tracker’s performance by comparing it with ground truth data, assessing tracking performance as well as the prediction precision of single tracks. Reasonable prediction performance is acquired right from the start, leading to a good overall catching rate. I. I NTRODUCTION Target tracking from image sequences has the goal to estimate the states of an unknown number of targets by integrating detections. Often, this includes dealing with false- alarms, missed and noisy detections, and target birth and death. In a classical way, this problem is solved using Multiple Hypothesis Tracking (MHT) [1], [2], [3], which hypothesizes associations between measurements and targets and propagates a set of these, where the one with the highest posterior probability is considered to be the most probable association. Optimally, all hypotheses should be propagated but due to combinatorial intractability when the number of targets and measurements is increased, only the most probables are kept [4]. An emerging technique for multi-target tracking is the Random Finite Set approach presented in [5], [6]. Here, the fundamental idea is to model states and measurements as random variables that take random sets as values. This allows a direct Bayesian formulation of the multi-target tracking O. Birbach and U. Frese are with German Center for Artificial Intelligence (DFKI). 28359 Bremen, Germany oliver.birbach@dfki.de problem, instead to the explicit modeling of data associations between targets and measurements as in MHT. For most practical applications, the computational intractability of the multi-target integrals prohibit using the formulation directly. For this, a first moment approximation known as the Proba- bility Hypothesis Density (PHD) Filter was proposed which propagates a posterior intensity recursively. This intensity is similar to a distribution in state space, i.e. with peaks denoting targets, except that its integral is not 1 but the expected number of targets. Implementations of the PHD recursion exist modeling the intensity as particles, known as the Sequential Monte Carlo (SMC) PHD filter [5], or as a Gaussian Mixture (GM), known as the GM-PHD filter [7], [5]. These filters still enumerate between measurements and targets but do not consider different combinations of associat- ing these such as MHT does. This makes them computational attractive. In this paper, we make use of the GM-PHD filter to address the problem of estimating and predicting multiple balls pitched towards a humanoid robot (see image sequence from the robot’s cameras on top) with the goal to catch each ball with one arm. This delicate task is a challenging tracking problem: Due to the short flight time, the trajectory of pitched balls must be detected as a track as early as possible so the ensuing planning stage has enough time to find a valid arm posture. Also, tracking accuracy is crucial, especially at the end of the trajectory. A special problem are systematic false- alarms for instance created by people’s heads. The tracker initially creates a track from such a measurement, but soon discards it because it does not follow the parabolic flight of a ball. The paper presents three main contributions. First, we present the foundations to successfully track multiple balls pitched to the observer using the PHD filter with non-linear Gaussian models. In particular, we feed circle measurements to the filter with the need to handle birth and death of targets,