1038 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 23, NO. 3, MARCH 2014 A New Iterative Triclass Thresholding Technique in Image Segmentation Hongmin Cai, Zhong Yang, Xinhua Cao, Member, IEEE, Weiming Xia, and Xiaoyin Xu, Member, IEEE Abstract— We present a new method in image segmentation that is based on Otsu’s method but iteratively searches for subregions of the image for segmentation, instead of treating the full image as a whole region for processing. The iterative method starts with Otsu’s threshold and computes the mean values of the two classes as separated by the threshold. Based on the Otsu’s threshold and the two mean values, the method separates the image into three classes instead of two as the standard Otsu’s method does. The first two classes are determined as the foreground and background and they will not be processed further. The third class is denoted as a to-be-determined (TBD) region that is processed at next iteration. At the succeeding iteration, Otsu’s method is applied on the TBD region to calculate a new threshold and two class means and the TBD region is again separated into three classes, namely, foreground, background, and a new TBD region, which by definition is smaller than the previous TBD regions. Then, the new TBD region is processed in the similar manner. The process stops when the Otsu’s thresholds calculated between two iterations is less than a preset threshold. Then, all the intermediate foreground and background regions are, respectively, combined to create the final segmentation result. Tests on synthetic and real images showed that the new iterative method can achieve better performance than the standard Otsu’s method in many challenging cases, such as identifying weak objects and revealing fine structures of complex objects while the added computational cost is minimal. Index Terms— Binarization, Otsu’s method, segmentation, threshold, triclass segmentation. Manuscript received July 10, 2013; revised October 25, 2013; accepted December 12, 2013. Date of publication January 9, 2014; date of current version January 23, 2014. The work of H. Cai was supported in part by the National Nature Science Foundation of China under Grant 61372141 and in part by the Fundamental Research Funds for the Central Universities under Award 2013ZM0079. The work of Z. Yang was supported by the National Natural Science Foundation of China under Award 31171148. The work of W. Xia was supported by the U.S. Department of Veterans Affairs Office of Research and Development Service. The work of X. Xu was supported by the National Science Foundation under Award 0958345. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Hitoshi Kiya. H. Cai is with the School of Computer Science and Engineering, South China University of Technology, Guangzhou 510641, China (e-mail: hongmin.cai@gmail.com). Z. Yang is with the Department of Clinical Hematology, The Third Military Medical University, Chongqing 400038, China (e-mail: zyang@tmmu.edu.cn). X. Cao is with the Department of Radiology, Children’s Hospital Boston, Boston, MA 02115 USA (e-mail: xinhua.cao@childrens.harvard.edu). W. Xia is with the Department of Medical Research and Development, VA Hospital, Bedford, MA 02132 USA (e-mail: weiming.xia@va.gov). X. Xu is with the Department of Radiology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA 02115 USA (e-mail: xxu@bwh.harvard.edu). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TIP.2014.2298981 I. I NTRODUCTION I N IMAGE processing, segmentation is often the first step to pre-process images to extract objects of interest for further analysis. Segmentation techniques can be generally catego- rized into two frameworks, edge-based [1]–[3] and region- based [4]–[6] approaches. As a segmentation technique, Otsu’s method is widely used in pattern recognition [7]–[9], document binarization [10]–[12], and computer vision [13]. In many cases Otsu’s method is used as a pre-processing technique to segment an image for further processing such as feature analy- sis and quantification. Otsu’s method searches for a threshold that minimizes the intra-class variances of the segmented image [14] and can achieve good results when the histogram of the original image has two distinct peaks, one belongs to the background, and the other belongs to the foreground or the signal. The Otsu’s threshold is found by searching across the whole range of the pixel values of the image until the intra-class variances reach their minimum. As it is defined, the threshold determined by Otsu’s method is more profoundly determined by the class that has the larger variance, be it the background or the foreground. As such, Otsu’s method may create suboptimal results when the histogram of the image has more than two peaks or if one of the classes has a large variance. Over the years, researchers have proposed many methods to improve the standard Otsu’s method. For example, Cheriet et al. proposed a recursive approach based on Otsu’s technique to focus on the brightest homogeneous object in an image [15]. A quad-tree approach was developed to segment images by combining a centroid clustering and boundary estimation methods but the approach only works under the assumption that the histogram consists of Gaussian distribu- tions only [16]. In [17], the authors added a weight term to force the resultant threshold value resides at the valley of the two peaks or at the bottom rim of a single peak. The standard bi-level thresholding techniques has been extended to multi- level thresholding in [18]–[20]. In the standard Otsu’s method 1D histogram is used for binization and methods have been proposed to expand the histogram to two dimensions (2D) by considering gray levels and average, albeit the 2D imple- mentation is more computational intensive. Theoretically, it has been shown in [21] that the objective function of Otsu’s method is equivalent to that of K-means method in multilevel thresholding [13]. In terms of speeding up computations, a fast search implementation of the threshold was proposed by Reddi et al [22].