DOES MULTISPECTRAL / HYPERSPECTRAL PANSHARPENING IMPROVE THE PERFORMANCE OF ANOMALY DETECTION ? Ying Qu and Hairong Qi EECS Department The University of Tennessee, Knoxville, TN 37996 {yqu3, hqi}@utk.edu Bulent Ayhan, Chiman Kwan Applied Research LLC, Rockville, MD Richard Kidd Jet Propulsion Lab Pasadena, CA ABSTRACT Pansharpening refers to the fusion of a high spatial resolution panchromatic image with high spectral resolution multispec- tral or hyperspectral images (MSI or HSI) to yield high reso- lution data in both spectral and spatial domains. It has been widely adopted as a primary preprocessing step for numer- ous applications. In this paper, we perform a literature survey of various pansharpening algorithms including the most ad- vanced deep learning approaches for both multispectral and hyperspectral images. We further evaluate the effect of the resolution difference on anomaly detection. Synthetic mul- tispectral and hyperspectral images are generated to evalu- ate the performance of anomaly detection on high resolution images. Eight state-of-the-art MSI and HSI pansharpening methods are compared in this paper. Experimental results show that, performing anomaly detection on high resolution images improves the detection rate, and at the mean time sup- presses the false alarm rate. Index Terms— Hyperspectral images, multispectral im- ages, pansharpening, anomaly detection, deep learning 1. INTRODUCTION Multispectral images (MSI) and Hyperspectral images (HSI) can provide important spectral information that benefits re- mote sensing applications. However, due to the limitations of remote sensors, there is a trade-off between spatial resolution and spectral resolution. Pansharpening refers to the fusion of high spatial resolution panchromatic images (PAN) with high spectral resolution MSI or HSI to generate images with high resolution (HR) in both spectral and spatial domains. Recent technologies are able to acquire PAN with the corresponding MSI or HSI simultaneously using commercial satellites such as Google Earth and Bing Maps, that has made the pansharp- ening possible. Nowadays, pansharpening plays an increasingly impor- tant role for remote sensing, and is becoming one of the essen- tial preliminary processing procedures for many applications such as environmental monitoring, change detection, object recognition, and classification [1]. For these applications, high resolution images in both spatial and spectral domains are required to generate more promising results [2]. Anomaly detection is one of the remote sensing applications, which aims to identify anomaly pixels inside an image. It can be modeled as an unsupervised binary classification problem be- tween the background class and the anomaly class. In this paper, we evaluate the performance of HR MSI or HSI achieved by pansharpening through the application of anomaly detection and investigate if adopting pansharp- ening as a preprocessing step will influence the performance of anomaly detection. The detection method we choose for anomaly detection is Subspace-RX (SSRX) [3], which per- forms RX in the primary subspace. We choose SSRX be- cause it is a popularly used detection method that works ef- fectively especially on small anomalies [4], which has been the common scenario of anomalies appeared in MSI or HSI. The contribution of this work is two-fold. First, several state- of-the-art pansharpening methods for both multispectral and hyperspectral images are surveyed and investigated including the most advanced deep learning based approaches. Second, to the best of our knowledge, we make the first attempt to evaluate the pansharpening technique through anomaly de- tection and investigate if pansharpening can benefit the task of anomaly detection. The rest of the paper is organized as follows. Sec. 2 reviews the pansharpening methods for both MSI and HSI. Sec. 3 evaluates the state-of-the-art pansharpening algorithm through anomaly detection. Conclusion are drawn in Sec. 4. 2. PANSHARPENING TECHNIQUES 2.1. Pansharpening for Multispectral Images MSI pansharpening can be dated back to the 19th century, and has been well developed through decades of works [5]. Tradi- tional widely used pansharpening algorithms can be roughly classified into two groups, the component substitution (CS) and the multiresolution analysis (MRA) approaches. CS–based approaches [5] mainly project the low reso- lution (LR) MSI onto a predefined space, which separates the spectral information from spatial information.By substi- tuting the spatial components with histogram-matched PAN and converting back to the MSI space, the resolution of data is effectively improved. Band-dependent spatial detail(BDSD) 6130 978-1-5090-4951-6/17/$31.00 ©2017 IEEE IGARSS 2017