Image segmentation for complex natural scenes Albert P Choo, Anthony J Maeder and Binh Pham - zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA An algorithm suitable for segmenting complex textured images of natural scenes is described. There are three stages of processing: defining starting regions; merging similar regions; and refining the boundaries of regions. Starting and merged regions are constructed from fixed size subimages (blocks) to permit the use of local statistical properties of pixel intensities. The choice of block size is made automatically using a similarity test on the statistical properties for randomly chosen blocks of different sizes. The algorithm has performed favour- ably on typical images of natural scenes using only a few standard texture measures. It uses two simple texture measures, mean and standard deviation, and from the tests performed, these measures were sufficient in pick- ing out the required features. Another measure, contrast, was added to pick out other more obscure regions. Keywords: segmentation, region grow ing, texture - The process of image segmentation is central to image analysis problems which require machine understand- ing of image contents. Two common examples of such problems are object recognition in a static scene, and change detection in an image sequence. Segmenting an image consists of subdividing it into a number of non- overlapping regions, each of which is spatially con- nected and differs from its neighbouring regions in some meaningful properties. Regions are delimited by boundaries. Unsupervised, automated segmentation is difficult as it cannot rely on prior knowledge of the image contents or characteristics. Segmentation is often computationally expensive because a number of differ- ent stages of collecting and assessing information from the image are needed. Consequently, considerable effort has been spent on designing segmentation algor- ithms in recent yearslm3. Two different approaches are evident in segmenta- tion algorithms, aimed at boundary and region detection, respectively. The boundary approach attempts to find precise local features of the image which are typical of some physical feature of the scene (e.g. finding - Department of (‘omputer Science, Monash University, Clayton VA 3168. Australia intensity discontinuities which indicate object edges). When such image features have been located, they may need to be extended to improve the correspondence with a physical feature, or discarded to reduce insigini- ficant information. The segmented image is described in terms of these features, and it is assumed that the regions sought are determined by them in some way. For example. the edges of an object are boundaries for its component surfaces which thus correspond to regions of the resulting segmentation. The region approach tries to isolate areas with consistent intensity properties which are dissimilar to those of adjacent areas. The intensity properties need not be confined to uniform or smooth variations, but may include locally perturbed intensities typical of texture or noise. Candi- date areas may be grown, shrunk. merged, split, created or destroyed during the segmenting process4. The areas existing when the algorithm terminates define the regions of the segmented scene, e.g. areas of different uniform intensity in an image might corres- pond to the component surfaces of objects. The boundary approach is well suited for segmenting simple scenes containing man-made objects where edges are clear and predominant, such as the ‘blocks’ world of robotics. The region approach is better when boundaries are imprecise and the image content is complex, typical of natural scenes such as remotely sensed geographical imagery. The boundary approach often uses algorithms operating on single or few pixels to detect the desired features. The region approach normally considers subimages of many pixels, usually termed blocks. Consequently, the region approach tends to sacrifice resolution and detail in the image to gain a sample large enough for the calculation of useful statistics for local intensity properties. This can result in failure to detect clear boundaries when they do exist, and in a failure to distinguish regions that would be small in comparison with the block size. This paper describes a region-based segmentation method suitable for use on complex, highly textured natural scenes. It provides an automated choice of block size, which is often determined heuristically in current methods. The starting block size is determined by repeatedly sampling subimage blocks at random from the image at increasing resolutions. Throughout each cycle, the blocks chosen are tested for consistency 0262-8856/90/020155-09 0 1990 Butterworth & Co (Publishers) Ltd ~018 no 2 may 1990 155