Proceeding sof SPIE on Signal processing, Sensor Fusion, and target Recognition XI (Vol. 4729), IIvan Kadar, Editor, Orlando FL, 1-3 April 2002. 280 Cross-Sensor Image Fusion and Spectral Anomaly Detection Mark J. Carlotto (mark.carlotto@veridian.com) Veridian Systems Division 1400 Key Blvd., Suite 100 Arlington VA 22209 Abstract A nonlinear mean square estimation algorithm for cross-sensor image fusion and spectral anomaly detection is described. The algorithm can be used to enhance a low resolution image with a higher resolution coregistered multispectral image, and to detect anomalies between spectral bands (features in one spectral band that do not occur in other bands). Experimental results for Landsat data are presented illustrating the spatial enhancement of thermal imagery, the detection of thermal anomalies (heat sources), and the detection of smoke plumes. Introduction A variety of methods have been developed for fusing multisensor imagery for band sharpening, feature classification, and object/change detection. Examples include principal components (Li, Zhou, and Li, 1999), wavelet transforms (Gomez, Jazaeri, and Kafatos, 2001; Lemeshewsky, 1999), pseudoinverse estimation and fuzzy reasoning (Patterson, Bullock and Wada 1992), and linear prediction (Tom, Carlotto, and Scholten 1985) for band sharpening; spectral mixing models (Gross and Schott, 1996) and neural networks (Shaikh, Tian, Azimi-Sadjadi, Eis, and Vonder Haar, 1996) for classification; signal subspace processing (Soumekh 1998), neural networks (Brown, Derouin, Beck, and Archer, 1992), and nonlinear transforms (Carlotto 2000) for multisensor/multispectral object/change detection. This paper describes the application of a nonlinear mean-square estimation technique originally developed for multispectral haze reduction (Carlotto 1999) to the detection of manmade objects and activities using a cross-spectral anomaly detection approach. The anomaly detector operates under the assumption that over natural backgrounds there is a high degree of correlation between spectral bands. Manmade activities, on the other hand, alter selected portions of the spectrum and thus introduce spectral anomalies at certain wavelengths. We use a nonlinear mean-square estimator to segment one set of spectral bands into different background types, and predict the average response over each background type in another set of bands. Spectral anomalies (pixels whose value differs from other pixels of the same background type) are detected based on the difference between their actual and predicted values.