FUSION OF THEMIS AND TES FOR ACCURATE MARS SURFACE CHARACTERIZATION C. Kwan, B. Ayhan, and B. Budavari Signal Processing, Inc. ABSTRACT This paper presents a novel approach to fusing Thermal Emission Imaging System (THEMIS) and Thermal Emission Spectrometer (TES) satellite images, aiming to improve Mars surface characterization performance from orbit. Our approach includes proven registration and advanced pansharpening algorithms developed by us and others. Preliminary experiments show that the fusion approach is highly promising despite the extremely high resolution difference of THEMIS and TES (30 to 1). We also observed some potential issues that require further research. Index Terms— image fusion, pansharpening, THEMIS, TES, registration 1. INTRODUCTION Recently, NASA has expressed strong interest in improving surface characterization of Mars using orbital imagers. THEMIS [1] and TES [2] are orbital multispectral imagers of Mars. THEMIS has 10 spectral bands in the 6-13 micrometers region and a spatial resolution of 100 m. TES has 143 spectral bands in the 5-50 micrometers range, but with low spatial resolution of 3x3 km. Although both have been used to map out the surface composition of Mars, there are some limitations. First, THEMIS has low spectral resolution that may not provide accurate surface characterization. Second, TES has low spatial resolution that cannot provide fine spatial details of surface characteristics. Fig. 1 illustrates the spatial difference between THEMIS and TES images. Roughly speaking, each TES pixel contains about 900 THEMIS pixels. It is therefore very challenging to fuse the two data sets. There are 4 research questions that we would like to address: Q1: Image registration: In addition to aligning THEMIS and TES bands, it turns out that THEMIS bands also have large shifts. How can one compensate for the shifts observed in the THEMIS bands? Q2: Pansharpening performance: Normally, pansharpening is only done for fusing images with 3 to 4 times resolution differences. Will pansharpening algorithms still perform well despite the huge resolution difference (30:1) between THEMIS and TES? Q3: Data compatibility between THEMIS and TES: Although both THEMIS and TES are thermal imagers, do they have the same signal response? Q4: Surface characterization from orbit: What is the achievable and quantifiable performance using orbital imagers as compared to in situ and ground based instruments? Fig. 1: Spatial resolution difference between THEMIS and TES. Background shows one band of THEMIS. Each dark square shows a single TES pixel. In this paper, we present the first application of a novel approach to fusing THEMIS and TES images. The goal is to eventually improve Mars surface characterization from orbit. It will be a long way to get there. We believe the first step is to fuse THEMIS and TES to form a high spatial resolution and high spectral resolution image cube. After that, we can then apply advanced hyperspectral image processing algorithms to analyze the pixels in the data cube. We are currently focusing on answering Q1 and Q2 and have observed some issues for Q3. Q4 will be addressed in the future. The fusion consists of two steps. First, the THEMIS and TES images need to be aligned. Even within the THEMIS bands, we observed that the bands have serious misalignments. To align the THEMIS bands, we extracted Speeded Up Robust Features (SURF) [3] and then applied RANSAC (Random Sample Consensus) technique [4]. To align THEMIS and TES, we utilized the coordinate information stored in the metadata of the images. Second, we evaluated 10 pansharpening algorithms to the registered image cube using a synthetic data. Due to the huge resolution difference between THEMIS and TES, some algorithms worked and some did not. Both objective (4 performance metrics) and subjective evaluations (false color 3381 978-1-5090-4951-6/17/$31.00 ©2017 IEEE IGARSS 2017