Visual Analysis for Multi-Spectral Images Comparisons Guozheng Li 1 * Shuai Chen 1 Qiusheng Li 2 Zhibang Jiang 1 Yuening Shi 1 Qiangqiang Liu 1 Xi Liu 2 Xiaoru Yuan 1† 1) Key Laboratory of Machine Perception (Ministry of Education), and School of EECS, Peking University 2) Qihoo 360 Technology Co. Ltd. ABSTRACT The analysis for images helps people to gain insights by extract- ing the inner features and variances between them. However, it is hard to analyze the underlying events further without users partici- pation. We proposes a visual analytic system based on collaborative tagging techniques to allow users to identify features and changes from multi-spectral images. We evaluate our system with mini chal- lenge 3 of VAST Challenge 2017. The exploration results validate the efficiency and effectiveness of our system. 1 I NTRODUCTION Despite the abundance of image processing algorithms for images data, it is hard to make comparisons for different images and an- alyze the differences efficiently [2]. Taking the VAST Challenge 2017 mini challenge3 [1] as an example, the tasks is to identify fea- tures, more specifically, sub-area of image and detect changes over time. The dataset provided are multi-spectral images from satellite system, the analysis of which require much human experience, so how to combine the computation capabilities and human experience together is quite significant and challenging. Information visualization leverages the innate human visual pro- cessing capacity [3]. We propose an visual analysis pipeline (as shown in Figure 1). In Features Identification step, features of im- ages are interactively identified in Image Tagged View and recorded in Image Matrix View. In Change Identification step, changes based on the detected features are identified in Image Comparison View and also stored in Image Matrix View. Based on this, we develop the visual analytic system (as shown in Figure 2) to support these desirable functionalities. This system is based on collaborative tag- ging techniques. This work makes the following contributions: • An analysis pipeline to help people to analyze the features, changes and the underlying events efficiently. • A visualization system for image exploration based on the col- laborative tagging method. 2 BACKGROUND In VAST Challenge mini challenge3, the dataset contains 12 multi- spectral images of a preserve on different dates. Our tasks are to identify features in the preserve area as captured in the imagery and detect the features that change over time. The data files are images from a multispectral sensor with six different bands (B1 - B6). B1, B2 and B3 represent portions of the visible spectrum. B4, B5 and B6 represent longer wavelengths that are beyond human perception. Single and combinations of multiple bands could help users to detect specific features, for example the combination of * e-mail: guozheng.li@pku.edu.cn † e-mail: xiaoru.yuan@pku.edu.cn Features Selection Changes Identification Image Matrix View Image Comparison View Distribution Histogram View Event List View Changes Records Image Selection Features Identification Features Records Image Matrix View Image Tagged View Operation View Feature Identification Process 1 Change Identification Process 2 Image Matrix View Image Matrix View Figure 1: Analysis Pipeline. The operations on the top correspond to the views on the bottom respectively. channels B4, B3, B2 could be useful in seeing changes in plant health. The computation results could also be useful in some cases. 3 VISUAL ANALYTIC SYSTEM Our visual analytic system contains two processes: Feature Identifi- cation Process and Change Identification Process. These two com- ponents constitute the entire process of the image analysis. The second step (Change Identification) is based on the detect features from the first step (Feature Identification). 3.1 Feature Identification Process Feature Identification Process leverages two views, Image Matrix View for selecting interested images and recording the detected fea- tures, Image Tagged View for labelling features from the images and adding tags on them. In Image Matrix View (as shown in Figure Figure 2(a)), images are arranged horizontally according to the time sequence and each row contains images derived under different band or bands com- binations. For each image in the matrix, the detected features are placed adjacently to it. Image Tagged View allows users to lasso the interested area and add descriptions to the selection, the interface is shown in Fig- ure 2(b). After submitting the detected features, the features is added into Image Matrix View. 3.2 Change Identification Process To select and compare the features over time, Image Comparison Viewmainly exploits four views: Image Matrix View, Image Com- parison View, Distribution Histogram View and Event List View. In Change Identification Process, Image Matrix View also sup- ports features selection and recording of comparison results. In Change Identification Process, users select two features and the im- ages with tags will be displayed in the Image Comparison View (as shown in Figure 2(c)). At the same time, the selected features are highlighted using black border in the Image Matrix View. Image Comparison View enable users compare images and inter- actively define events, and Distribution Histogram View (as shown in Figure 2(d)) support the comparison from the statistics perspec- tive. Both Image Comparison View and Distribution Histogram View (as shown in Figure 2(d)) are designed to compare images from the and statistics perspectives. The Distribution Histogram View shows the pixel values distribution of the selected area and the height of 1