Comparative Image Fusion Analysais Firooz Sadjadi Lockheed Martin Corporation firooz.sadjadi@ieee.org Abstract Image fusion is and will be an integral part of many existing and future surveillance systems. However, little or no systematic attempt has been made up to now on studying the relative merits of various fusion techniques and their effectiveness on real multi-sensor imagery. In this paper we provide a method for evaluating the performance of image fusion algorithms. We define a set of measures of effectiveness for comparative performance analysis and then use them on the output of a number of fusion algorithms that have been applied to a set of real passive infrared (IR) and visible band imagery. 1. Introduction Image fusion is a process of combining images, obtained by sensors of different wavelengths simultaneously viewing of the same scene, to form a composite image. The composite image is formed to improve image content and to make it easier for the user to detect, recognize, and identify targets and increase his situational awareness. The research activities are mainly in the area of developing fusion algorithms that improves the information content of the composite imagery, and for making the system robust to the variations in the scene, such as dust or smoke, and environmental conditions, i.e. day or and night [1-31].This paper is structured in the following way: Section 2 provides details on several fusion algorithms. This consists of pyramid based algorithms that form a large number of image fusion techniques, biologically inspired fusion approaches, and total probability of error technique Section 3 defines a set of image fusion measures of effectiveness. Section 4 provides a comparative performance evaluation of the fusion techniques and the experimental fusion results, using real passive and active infrared and visible band imagery, for selected approaches. Finally, Section 5 provides a summary of the paper and its main conclusions. 2. Image Fusion Algorithms Image Pyramid Approaches- An image pyramid consists of a set of lowpass or bandpass copies of an image, each copy representing pattern information of a different scale [4-6]. Typically, in an image pyramid every level is a factor two smaller as its predecessor, and the higher levels will concentrate on the lower spatial frequencies. An image pyramid does contain all the information needed to reconstruct the original image. The Gaussian pyramid is a sequence of images in which each member of the sequence is a low pass filtered version of its predecessor [9]. Laplacian pyramid of an image is a set of bandpass images, in which each is a bandpass filtered copy of its predecessor. Bandpass copies can be obtained by calculating the difference between lowpass images at successive levels of a Gaussian pyramid [5]. Ratio of Low Pass Pyramid is another pyramid in which at every level the image is the ratio of two successive levels of the Gaussian pyramid [7]. Contrast Pyramid is similar to the ratio of Low Pass Pyramid approach. Contrast itself is defined as the ratio of the difference between luminance at a certain location in the image plane and local background luminance to the local background luminance. Luminance is defined as the quantitative measure of brightness and is the amo unt of visible light energy leaving a point on a surface in a given direction. Filter-Subtract-decimate (FSD) Pyramid technique is a more computationally efficient variation of the Gaussian Pyramid [27]. Morphological Pyramid-The multi-resolution techniques introduced by Burt and Adelson etc. typically use low or bandpass filters as part of the process. These filtering operations usually alter the details of shape and the exact location of the object sin the image. This problem has been addressed by using morphological filters to remove the image details without adverse effects [24]. Morphological filters, introduced by Serra are composed of a number of