120 IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 10, NO. 1, JANUARY 2013 A Spatially-Constrained Color–Texture Model for Hierarchical VHR Image Segmentation Zhongwen Hu, Zhaocong Wu, Qian Zhang, Qian Fan, and Jiahui Xu Abstract—This letter presents a novel spatially-constrained color–texture model for hierarchical segmentation of very high resolution images. The segmentation starts with an initial par- tition, where the image is partitioned into many homogeneous regions. Then, the regions are regarded as node sets of a region adjacency graph, in which the distances of each pair of adjacent regions are calculated combining color and textural features with spatial constraint. Finally, a stepwise optimized region merging process is applied to obtain hierarchical segmentation results. Experiments and comparisons by using different satellite images are carried out to demonstrate the encouraging performance as well as the high efficiency of the proposed method. Index Terms—Color–texture segmentation, hierarchical region merging, hierarchical stepwise optimization (HSWO), local binary patterns (LBPs), remote sensing images, spatial constraint. I. I NTRODUCTION I MAGE segmentation is a key step in object-oriented remote sensing applications [1], such as object-based classification and change detection. It is used with the expectation that it will divide the image into semantically significant regions, or objects, to be recognized by further processing steps [2]. This work has attracted thousands of researchers in the past decade but is still an intractable problem [3]. According to the features utilized in different methods, segmentation methods can be cat- aloged into three types: spectral-based methods, texture-based methods, and spectral–texture combined methods [3]. Although some spectral- or texture-based segmentation methods have been proved effective in segmenting remote sensing images [4]–[6], it is proved that color–texture combined methods pro- duce more reliable and accurate segmentation results among [3], [7], and [8]. Many spectral–texture segmentation methods proposed in the past few years had produced encouraging results; however, there still exist some limitations applied to remote sensing images. Manuscript received September 10, 2011; revised December 10, 2011 and February 21, 2012; accepted April 2, 2012. This work was supported in part by the National High Technology Research and Development Program of China under Grant 2007AA12Z143, by the National Natural Science Foundation under Grants 40201039 and 40771157, and by the Fundamental Research Funds for the Central Universities under Grants 20102130201000134, 201121302020003, and 2011QD03. Z. Hu, Z. Wu, Q. Zhang, and Q. Fan are with the School of Remote Sens- ing and Information Engineering, Wuhan University, Wuhan 430079, China (e-mail: zhongwenhu@163.com; zcwoo@whu.edu.cn; hangfanzq@163.com; fanqian86@gmail.com). J. Xu is with the School of Geographic and Oceanographic Sciences, Nanjing University, Nanjing 210093, China (e-mail: amy_8688@126.com). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/LGRS.2012.2194693 Some complex models have been developed for segmenting nature and synthetic images, such as the J-images segmentation (JSEG) [9], Gaussian mixture models [10], and Markov random fields [11]. However, these methods will meet innate theoretical limitations or poor efficiency when applied to remote sensing images, which are very complex and containing thousands of different objects. Moreover, some other methods combining different textural features with color features were proposed, such as the Gabor [3] and wavelet [12] textural features. These methods were proved effective in segmenting remote sensing images but suffered poor efficiency in extraction of textures. In recent years, a theoretically simple but effective texture descriptor named local binary patterns (LBPs) has been de- veloped for texture description [13], [14]. It was introduced to color–texture segmentation of nature and synthetical images by Chen and Chen [15] and further developed for color–texture segmentation of remote sensing images [7], [8]. However, this kind of methods is limited by the blocklike initial partition, which leads to the inability to combine spatial features (such as geometric features). In addition, the size of each block is limited to be no less than 16 × 16 pixels in order to keep the stability of color and textural features, but this is not applicable for segmenting narrow ribbonlike objects, whose width would be much smaller than the block size. In this letter, a novel spatially-constrained color–texture model is introduced to hierarchical segmentation of very high resolution (VHR) images. For a given image, the method starts with an initial partition, followed by a representation of the regions using a region adjacency graph (RAG) [16], in which a novel spatially-constrained color–texture model is used to measure the distances between adjacent regions. Finally, the RAG-based hierarchical stepwise optimized region merging process [17] is then applied to obtain hierarchical represen- tations, which are not only useful for efficient algorithmic implementation but also can give important information on the relations between the regions in the image. This letter is organized as follows: In Section II, the hier- archical image segmentation framework used in the proposed method is described; the spatially-constrained color–texture model is described in Section III, which is divided into three progressive sections; experimental results and discussions are presented in Section IV; the conclusion is drawn in Section V. II. HIERARCHICAL I MAGE SEGMENTATION In this section, we describe the hierarchical segmentation model utilized in the proposed method. In this approach, the hierarchical stepwise optimization (HSWO) approach proposed by Beaulieu and Goldberg [17] is adopted, which employs a sequence of optimization processes to produce hierarchical 1545-598X/$31.00 © 2012 IEEE