634 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 5, MAY 2005 Landcover Classification in MRF Context Using Dempster–Shafer Fusion for Multisensor Imagery Anjan Sarkar, Anjan Banerjee, Nilanjan Banerjee, Siddhartha Brahma, B. Kartikeyan, Manab Chakraborty, and K. L. Majumder Abstract—This work deals with multisensor data fusion to ob- tain landcover classification. The role of feature-level fusion using the Dempster–Shafer rule and that of data-level fusion in the MRF context is studied in this paper to obtain an optimally segmented image. Subsequently, segments are validated and classification ac- curacy for the test data is evaluated. Two examples of data fusion of optical images and a synthetic aperture radar image are pre- sented, each set having been acquired on different dates. Classi- fication accuracies of the technique proposed are compared with those of some recent techniques in literature for the same image data. Index Terms—Dempster–Shafer theory, Fisher’s discriminant, Hotelling’s , Markov random field (MRF). I. INTRODUCTION I N THIS paper, we address the problem of landcover classifi- cation for multisensor images that are similar in nature. Im- ages of the same site acquired by different sensors are to be an- alyzed by combining the information available in them. Such a combination is necessary since data from individual sensors are insufficient to describe ground complexity. They may be par- tially complementary and partially redundant as sensors have different characteristics and physical interaction mechanisms. In principle, fusion of multisource data for a purpose provides significant improvements over a single source data. Moreover, fusion can be carried out at different stages, viz., data level, fea- ture level, decision level, etc. An excellent framework for dif- ferent stages of multisensor data fusion is available in Hall and Llinas [14]. Although it is advantageous to carry out fusion at one of the levels, yet there is a possibility of further improve- ment if we combine two different types of fusion. With this idea, the role of feature-level fusion using Dempster–Shafer (DS) rule and that of data-level fusion in the Markov random field Manuscript received February 18, 2004; revised October 3, 2004. This work was supported by the ISRO under the Grant for the Development of Landcover Classification Methodology with Fusion of Data from Different Sensors, Ref. 10/4/416, February 27, 2003. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Zoltan Kato. A. Sarkar is with the Department of Mathematics, Indian Institute of Tech- nology, Kharagpur 721302, India (e-mail: anjan@maths.iitkgp.ernet.in). A. Banerjee and S. Brahma are with the Indian Institute of Tech- nology, Kharagpur 721302, India (e-mail: anjanbanerji@yahoo.com; sbrahma2k3@yahoo.com). N. Banerjee is with the University of Massachusetts, Amherst, MA 01002 USA (e-mail: nilanb@cs.umass.edu). B. Kartikeyan, M. Chakraborty, and K. L. Majumder are with the Space Application Centre, Ahmedabad 380015, India (e-mail: bkartik@ipdpg.ipdpg.gov.in; manab@sac.isro.org; klm@ipdpg.ipdpg.gov.in). Digital Object Identifier 10.1109/TIP.2005.846032 (MRF) context is studied in this work to obtain an optimal seg- mented image. This segmented image is subsequently labeled with groundtruth classes by a cluster validation scheme to ob- tain the classified image. There are numerous reports available in the literature, such as [3], [17], [25], and [29], among others, which analyze data from different sensors or sources. An extensive review work is presented in Abidi and Gonzales [1]. A very brief survey is also available in Solberg et al. [29]. In the past many approaches for data fusion have been attempted. However, DS theory of evi- dence has created a lot of interest among researchers although its isolated pixel by pixel use has not shown encouraging re- sults. Among the statistical approaches, the simplest method adopted for fusion is to form an extended data vector for a pixel, comprising information from all the sources that are sim- ilar in nature. A statistical approach with similar sources has been investigated in [25] under the assumption of multivariate Gaussian distribution incorporating a source reliability factor. In [25], the authors also demonstrate the use of the mathemat- ical theory of evidence or DS rule for aggregating the recom- mendations of the two sources. Although the latter approach has certain advantages, the statistical approach demonstrates better performance in overall classification for the numerical data in- vestigated. DS theory at the pixel level [17] has also been used for unsupervised identification of landcover type with some de- gree of success. A methodological framework is presented in Solberg et al. [29] that considers the important element of spa- tial context as well as temporal context in the MRF model for multisource classification. The performance analysis of the pro- posed MRF fusion model was found to be quite encouraging. An interesting work of Bendjebbour et al. [3] demonstrates the use of DS theory in Markovian context. In this paper, the MRF was defined over pixel sites, and as such, the computation time for such an approach is expected to be very high when large number of groundtruth classes occur in a natural scene, which is usually the case. The MRF model-based image segmentation allows the spa- tial interactions between pixels and renders different Bayesian methods of segmentation very effective. Hence, such a model has been an area of interest for several researchers [5], [9], [10], [13]. However, its main drawback is that it is computationally intensive. Some works of MRF-based segmentation aiming at reducing computational time are also available in the litera- ture [6], [23], [28]. As shown in [28], a substantial reduction of computational time is possible if the MRF is defined on a set of preliminary segmented regions instead of defining it on image pixels. In this paper, we adopt a methodology similar to 1057-7149/$20.00 © 2005 IEEE