IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 43, NO. 8, AUGUST 2005 1901 Supervised Segmentation of Remote Sensing Images Based on a Tree-Structured MRF Model Giovanni Poggi, Giuseppe Scarpa, and Josiane B. Zerubia, Fellow, IEEE Abstract—Most remote sensing images exhibit a clear hierar- chical structure which can be taken into account by defining a suitable model for the unknown segmentation map. To this end, one can resort to the tree-structured Markov random field (MRF) model, which describes a -ary field by means of a sequence of binary MRFs, each one corresponding to a node in the tree. Here we propose to use the tree-structured MRF model for supervised segmentation. The prior knowledge on the number of classes and their statistical features allows us to generalize the model so that the binary MRFs associated with the nodes can be adapted freely, together with their local parameters, to better fit the data. In addi- tion, it allows us to define a suitable likelihood term to be coupled with the TS–MRF prior so as to obtain a precise global model of the image. Given the complete model, a recursive supervised segmenta- tion algorithm is easily defined. Experiments on a test SPOT image prove the superior performance of the proposed algorithm with re- spect to other comparable MRF-based or variational algorithms. Index Terms—Hierarchical fields, image classification, image segmentation, Markov random fields (MRFs), regression trees, structured images. I. INTRODUCTION S EGMENTATION is one of the most important low-level processing carried out on remote sensing imagery, espe- cially relevant for subsequent image classification and interpre- tation. Segmentation techniques based on clustering in the ob- servation space, like minimum distance, maximum likelihood, and the like [1], are well understood and exhibit a low com- plexity, but usually do not guarantee a satisfactory performance. To improve accuracy, one has to take into account prior infor- mation about the observed image, especially its spatial structure and the dependencies between neighboring pixels. In the statistical framework, this is done by defining a suitable joint probabilistic model for the unknown segmentation map and the data and by estimating the map ac- cording to some useful statistical criteria. A popular choice is the maximum a posteriori probability (MAP) criterion, where is selected as the map that maximizes the joint probability distri- bution. The key point becomes the selection of suitable models for the conditional data likelihood and the prior . Manuscript received May 25, 2004; revised April 7, 2005. G. Poggi is with the Dipartimento di Ingegneria Elettronica e delle Tele- comunicazioni, Università Federico II di Napoli, 80125 Napoli, Italy (e-mail: poggi@unina.it). G. Scarpa is with the Pattern Recognition Department, Institute of Informa- tion Theory and Automation, Academy of Sciences of the Czech Republic, 14131 Prague, Czech Republic. J. B. Zerubia is with the Unité de Recherche de Sophia Antipolis, l’Institut National de Recherche en Informatique et en Automatique, 06902 Sophia An- tipolis Cedex, France. Digital Object Identifier 10.1109/TGRS.2005.852163 Although data modeling is an important problem and the ob- ject of intense research, e.g., see [2]–[4], here we focus on the prior term and will therefore follow the common simplifying assumption that the data be conditionally independent given the class, and Gaussian distributed, leaving for future work the re- finement of this hypothesis. As for the prior , it should be meaningful and present a limited complexity for analytical and numerical treatment. The Markov random field (MRF) model [5] meets all these requirements because it is a relatively simple, yet effective, tool to encompass prior knowledge in the segmen- tation process. By modeling the segmentation map as a MRF, one assumes that each pixel depends statistically on the rest of the image only through a selected group of neighbors. This greatly simplifies the problem of assigning a meaningful prior, since only local characteristics of the image need be specified through the definition of local potential functions whose inte- gration gives the global energy of a Gibbs distribution [5]. Many MRF models have been proposed (see [6] and [7]) to describe the different features encountered in the images of in- terest, often with remarkable results in the applications. Very few models, however, try to capture the local variations of image statistics, which represent one of the most relevant phenomena to account for. Instead, it is implicitly assumed that interregion dependencies can be governed by a single set of parameters, to be estimated based on data from the whole image. However, im- ages are highly structured sources of information, and this over- simplification can lead to major errors. Consider for example the image in Fig. 1, where the data (a) were generated by adding Gaussian noise to a synthetic segmentation map (b). This image exhibits different statistical behaviors in different areas, with labels changing rapidly in some regions and much more slowly in others. A segmentation based on conventional nonadaptive models of the data provides very poor results, such as the segmentation map shown in (c), ob- tained with a second-order Potts model, where the two fine-grain classes are badly mixed because of an overregularization. The problem, here, is that the model parameters are estimated on the whole image irrespective of the different statistics encountered in different regions. Spatially adaptive models (e.g., [8]) could certainly improve upon this result but they are quite cumbersome and, more important, do not address the real problem, which is that a class-adaptive model is needed here. In particular, to fit well this image, a more complex model is required, such as the hi- erarchy of classes shown in Fig. 1(d): here a first set of parameters (just one parameter in the example) describes the relationship between the “black” class and the other two, which are indistin- guishable at this level, while a second set of parameters describes the relationship between the “white” and “gray” classes. 0196-2892/$20.00 © 2005 IEEE