EXPLORATORY VISUAL ANALYSIS OF MULTISPECTRAL EO IMAGES BASED ON DNN Iulia Neagoe(1), Daniela Faur(1), Corina Vaduva(1) and Mihai Datcu(1)(2) University Politehnica of Bucharest UPB (1), German Aerospace Centre DLR (2) ABSTRACT Exploratory visual analysis is often required to assist human operator to understand and interpret Earth Observation (EO) images. Optimal image representation offers cognitive support in discovering relevant facts about the scene with respect to a particular application. This is of crucial importance for training data sets selection in all Machine Learning tasks, particularly in the design of active learning tools for multispectral (MS) EO data. This paper proposes a deep neural network (DNN) based method to compress, learn and reveal the most significant information included in the spectral bands of EO data in support of relevant visualization for image content analysis. The advanced method uses a DNN to discover the most suggestive pseudo-color representation able to highlight the entire MS image content better than the particular 3 bands selection (R, G, B). We propose the use of information theory and the concept of mutual information to rank the spectral bands based on the amount of information contained, by applying the minimum-redundancy-maximum- relevance (mRMR) criterion on a the image so that we obtain the ranked bands. A DNN stacked autoencoder based paradigm is developed in order to extract and compress in three bands the overall information from the MS EO data. The developed method is demonstrated and validated for Sentinel 2 dataset. Index Terms— EO images, DNN, mRMR, information theory, mutual information 1. INTRODUCTION With the evolution of sensors, the complexity of remote sensing data increased, causing the processes of analyzing the information contained to become more challenging. The main reason for this is the fact that most of the tasks involved in the process are still manually performed by a human operator. Requiring accuracy, knowledge and attention, visual interpretation is the most relying part of data analysis. A more suggestive representation would improve data analysis and provide support for further human interpretation [1]. High resolution multispectral images contain information that can be useful in a wide range of applications (e.g. urban Fig. 1 Sentinel 2 images, Musura Golf, Sulina, Romania. Represents the comparison between two image representations in: visible domain (left) and infrared domain (right) of the electromagnetic specter. monitoring, assessment of forest deforestation). Each of the semantic classes (e.g. water, forest, urban) has a specific spectral signature, which may not include the visible spectral bands, so the visualization of the “true color” version of the image may hide important details. The need for visualization of the most relevant features comes from the fact that the Multi Spectral Instrument (MSI) measures the Earth’s reflected radiance in 13 spectral bands (as in the case of Sentinel 2 sensor) so the usual visualization of the RGB display (4, 3, and 2) lacks the information contained in the others 10 bands. Comparing the RGB representation (visible domain) with the IR pseudo-color representation (6, 7 and 8A bands of Sentinel-2), as in Fig. 1, we can observe that there are hidden details in the RGB display, but in the IR display these appear very clear. Also, in the IR representation we can define the water contour very accurate, in contrast with the RGB where, due to river deposits, it is hard to discriminate water from vegetation. The confusion is caused by the fact that they may seem to be color similar in the visibile part of the specter, yet they have different characteristics in the IR part of the electromagnetic specter. This paper aims to first demonstrate the performance of the mRMR criterion for the selection of spectral features that optimally describe a semantic class [2], and then to advance to the use of a deep learning approach and information theory concepts to collect the relevant, consistent information hidden in all the spectral bands imaging a scene. The authors of [2] exploit the potential of the mRMR criterion, first introduced by [3], to discover the optimal 2079 978-1-5386-7150-4/18/$31.00 ©2018 IEEE IGARSS 2018