A Sufficient Comparison of Trackers David Bizup University of Virginia Department of Systems and Information Engineering P.O. Box 400747 151 Engineer's Way Charlottesville, VA 22904 Donald E. Brown University of Virginia Department of Systems and Information Engineering P.O. Box 400747 151 Engineer's Way Charlottesville, VA 22904 Abstract Tracking maneuvering targets with radar is a difficult problem, but making a fair comparison between two or more maneuvering target trackers may be even more difficult. At the very least, it is tried less often. In this paper we present a method for comparing trackers based on the probabilistic notion of sufficiency. The advantages of our approach are twofold. First, comparisons are made across tens of thousands of trajectories, not just a few. Second, if one tracker is sufficient for another then it is better no matter how better is defined. We demonstrate the sufficient comparison technique for two trackers; one sets noise levels adaptively based on a statistic of accelerations first introduced at the 7 th International Command and Control Research and Technology Symposium, the other is the well known and widely used Interacting Multiple Model. Introduction Many papers compare maneuvering target trackers for at most a few different measures of performance and trajectories. Bar-Shalom and Li [Bar-Shalom 1993], de Feo, Graziano, Migliolo, and Farina [deFeo 1997], and Kameda, Tsujimichi, and Kosuge [Kameda 2002] are typical examples. Some report one measure of performance: position accuracy. Some report position and speed accuracies. Averages and interval estimates are sometimes reported, but it is not unusual to see these statistics reported for a single trajectory and maybe even a single simulation run. Other measures of performance like heading and range rate accuracy are rarely seen. The reader wonders why the authors choose particular trajectories, simulation runs, and measure of performance. Or conversely, why they did not choose others. Were they chosen at random, because the results are typical, or because they favor one tracker over the other? Occasionally, different papers report results for the same tracker. This happens in [Schutz 1997], [Schutz 1999] and [Kirubarajan 1999]. [Schutz 1997] describes what the authors call a Combined Kalman Filter (CKF) tracker. It has one mode, switches process noise levels based on a statistical threshold test of the position measurement residual history, and is intended for a military airborne early warning and control system. [Kirubarajan 1999] compares that tracker to an interacting multiple model (IMM) intended for the same application. Both papers test their trackers against the same 120 target simulation. The papers have some common authors. In fact, the company that