Fusion of FLIR automatic target recognition algorithms Syed A. Rizvi a, * , Nasser M. Nasrabadi b a Department of Engineering Science and Physics, College of Staten Island, City University of New York, 2800 Victory Blvd., Staten Island, NY 10314, USA b US Army Research Laboratory, ATTN: AMSRL-SE-SE, 2800 Powder Mill Road, Adelphi, MD 20783, USA Received 15 November 2001; received in revised form 17 December 2002; accepted 4 April 2003 Abstract In this paper, we investigate several fusion techniques for designing a composite classifier to improve the performance (prob- ability of correct classification) of forward-looking infrared (FLIR) automatic target recognition (ATR). The motivation behind the fusion of ATR algorithms is that if each contributing technique in a fusion algorithm (composite classifier) emphasizes on learning at least some features of the targets that are not learned by other contributing techniques for making a classification decision, a fusion of ATR algorithms may improve overall probability of correct classification of the composite classifier. In this research, we propose to use four ATR algorithms for fusion. The individual performance of the four contributing algorithms ranges from 73.5% to about 77% of probability of correct classification on the testing set. The set of correctly classified targets by each contributing algorithm usually has a substantial overlap with the set of correctly identified targets by other algorithms (over 50% for the four algorithms being used in this research). There is also a significant part of the set of correctly identified targets that is not shared by all contributing algorithms. The size of this subset of correctly identified targets generally determines the extent of the potential im- provement that may result from the fusion of the ATR algorithms. In this research, we propose to use Bayes classifier, committee of experts, stacked-generalization, winner-takes-all, and ranking-based fusion techniques for designing the composite classifiers. The experimental results show an improvement of more than 6.5% over the best individual performance. Published by Elsevier B.V. 1. Introduction Automatic target recognition (ATR) systems gener- ally consist of three stages as shown in Fig. 1 [1]: (1) a preprocessing stage (target detection stage) that operates on the entire image and extracts regions containing potential targets, (2) a clutter 1 rejection stage that uses a sophisticated classification technique to identify true targets by discarding the clutter images (false alarms) from the potential target images provided by the de- tection stage, and (3) a classification stage that deter- mines the type of the target. ATR using forward-looking infrared (FLIR) imagery is an integral part of the ongoing research at US Army Research Laboratory (ARL) for digitization of the battlefield. The real-life FLIR imagery (for example the database available at the ARL, see Figs. 2 and 3) demonstrates a significantly high level of variability of target thermal signatures. The high variability of target thermal signatures is due to several reasons, including meteorological conditions, times of the day, locations, ranges, etc. This highly unpredictable nature of thermal signatures makes FLIR ATR a very challenging prob- lem. In recent years a number of FLIR ATR algorithms have been developed by the scientists at ARL as well as by the researchers in academia and industry working under ARL-sponsored research. These research activi- ties have used a common development set of FLIR data (17,318 target images). The performance of these inde- pendently developed algorithms is measured in terms of the probability of correct classification using a common testing FLIR data collected under relatively unfavorable conditions. The testing FLIR data is not used during the algorithm development. The performance of these al- gorithms seems to be topped off around 77% of proba- bility of correct classification. In this paper, we investigate several fusion techniques for designing a composite classifier to improve the per- formance (probability of correct classification) of FLIR * Corresponding author. Tel.: +1-718-982-3125; fax: +1-718-982- 2830. E-mail address: rizvi@postbox.csi.cuny.edu (S.A. Rizvi). 1 A clutter is anything that mimics a target but is not a real target. 1566-2535/$ - see front matter Published by Elsevier B.V. doi:10.1016/S1566-2535(03)00043-5 Information Fusion 4 (2003) 247–258 www.elsevier.com/locate/inffus