DIFFERENT MODALITY BASED REMOTE SENSING DATA FUSION APPROACH FOR EFFICIENT CLASSIFICATION OF AGRICULTURE AND URBAN SUBCLASSES S. N. Chaudhri 1 , N. S. Rajput 1 , K. P. Singh 1 , D. Singh 2 1 Indian Institute of Technology (BHU) Varanasi, India 2 Indian Institute of Technology Roorkee, India ABSTRACT Subclasses classification is one of the major challenges in remote sensing (RS) scene classification. The area under observation, in order to classify agriculture and urban subclasses, requires efficient classification algorithms. Among such algorithms, deep learning algorithm based on Convolutional Neural Network (CNN) architecture is one such promising candidate to obtain the classified map. In this work, performance of a CNN network has been demonstrated on the data obtained from National Ecological Observatory Network (NEON) field site Domain 17 by considering different modality data and its subsequent fusion using the proposed model of CNN as applied on (i) the Hyperspectral, (ii) the Light Detection and Ranging (LiDAR) and then (iii) fused data respectively. Both the Hyperspectral and the LiDAR data have been fused at pixel level. Using the proposed methodology, a classified map is obtained with an overall accuracy of 96 percent for fused data. Index Terms— Subclass classification, Convolutional Neural Network (CNN), Data Fusion 1. INTRODUCTION Remote sensing (RS) has wide areas of applications. In agriculture, crop type determination is carried out using RS data classification while in land use land cover (LULC) classification, change detection and classification analysis is given more importance. Change detection analysis requires temporal data of the geo-location under observation. On the other hand, land cover classification requires certain data which is captured on a single date [1-4]. The RS data of two modalities has been considered in this paper, viz. passive RS data and active RS data. The passive and active data are basically categorized on the basis of the way of sensing technology. The hyperspectral data is a passive RS data. It is captured by the radiance of the surface materials just because of solar radiation. Since, here the natural energy source i.e. the Sun is used instead of derived energy source; so it is called as passive. The second data used in this paper i.e. the Light Detection and Ranging (LiDAR) is an active RS data [5]. A scanner system is used to capture this data and a laser beam is used as an energy source. With the fact of using a derived energy source, this sensing of data named as active RS. The principle of LiDAR data scanning is same as used for target detection by the Radio Detection and ranging (RADAR). By using the RADAR principle, active data is used to measure the canopy of objects. The hyperspectral data has capability to identify the surface objects based on their material composition. Sometimes it fails; when two or more objects composed of same material have different heights. Having different modalities the LiDAR data will have capability to discriminate the objects of different heights. It is also fails; when two or more objects of same height composed of different material. The complementary nature of both the data is utilized to enhance the classification accuracy by fusing at pixel level. In this work, a Convolutional Neural Network (CNN) based algorithm has been proposed for efficient classification of objects for the sake of better urban planning. The complementary nature of data taken in this study used to increase the accuracy of classification. 2. DATASET Online available dataset provided by National Ecological Observatory Network (NEON) has been used in this work. The NEON program is funded by National Science Foundation (NSF) and operated by BATTELLE organization. The Hyperspectral and LiDAR dataset both are downloaded from online NEON data portal [6]. A subset of this dataset chosen and eight subclasses labelled as area of interest. Distribution of subclasses is shown in Table I; and the actual optical RGB (Red-Green-Blue) image of field site is shown in Fig. 1. 2.1. Hyperspectral Dataset The Hyperspectral data is a cube as it has 3-dimension. It is formed by the stacking of 2-dimensional images captured at 5710 978-1-5386-9154-0/19/$31.00 ©2019 IEEE IGARSS 2019 Authorized licensed use limited to: Indian Institute Of Technology (Banaras Hindu University) Varanasi. Downloaded on August 26,2022 at 08:52:18 UTC from IEEE Xplore. Restrictions apply.