IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 22, NO. 2, FEBRUARY 2003 155 Combining Low-, High-Level and Empirical Domain Knowledge for Automated Segmentation of Ultrasonic Breast Lesions Anant Madabhushi* and Dimitris N. Metaxas Abstract—Breast cancer is the most frequently diagnosed malignancy and the second leading cause of mortality in women [45]–[48]. In the last decade, ultrasound along with digital mammography has come to be regarded as the gold standard for breast cancer diagnosis [9], [10], [40]. Automatically detecting tumors and extracting lesion boundaries in ultrasound images is difficult due to their specular nature and the variance in shape and appearance of sonographic lesions. Past work on automated ultrasonic breast lesion segmentation has not addressed important issues such as shadowing artifacts or dealing with similar tumor like structures in the sonogram. Algorithms that claim to automat- ically classify ultrasonic breast lesions, rely on manual delineation of the tumor boundaries. In this paper, we present a novel tech- nique to automatically find lesion margins in ultrasound images, by combining intensity and texture with empirical domain specific knowledge along with directional gradient and a deformable shape-based model. The images are first filtered to remove speckle noise and then contrast enhanced to emphasize the tumor regions. For the first time, a mathematical formulation of the empirical rules used by radiologists in detecting ultrasonic breast lesions, popularly known as the “Stavros Criteria” is presented in this paper. We have applied this formulation to automatically deter- mine a seed point within the image. Probabilistic classification of image pixels based on intensity and texture is followed by region growing using the automatically determined seed point to obtain an initial segmentation of the lesion. Boundary points are found on the directional gradient of the image. Outliers are removed by a process of recursive refinement. These boundary points are then supplied as an initial estimate to a deformable model. Incorporating empirical domain specific knowledge along with low and high-level knowledge makes it possible to avoid shadowing artifacts and lowers the chance of confusing similar tumor like structures for the lesion. The system was validated on a database of breast sonograms for 42 patients. The average mean boundary error between manual and automated segmentation was 6.6 pixels and the normalized true positive area overlap was 75.1%. The algorithm was found to be robust to 1) variations in system parameters, 2) number of training samples used, and 3) the position of the seed point within the tumor. Running time for segmenting a single sonogram was 18 s on a 1.8-GHz Pentium machine. Index Terms—Automatic segmentation, boundary, breast, deformable model, directional gradient, intensity, medical, seed point, texture, tumor, ultrasound. Manuscript received July 3, 2002; revised October 25, 2002. The Associate Editor responsible for coordinating the review of this paper and recommending its publication was M. Viergever. Asterisk indicates corresponding author. *A. Madabhushi is with the Department of Bioengineering, University of Pennsylvania, 120 Hayden Hall, Philadelphia, PA 19104 USA (e-mail: anantm@seas.upenn.edu). D. N. Metaxas is with the Departments of Biomedical Engineering and Com- puter Science, Rutgers, The State University of New Jersey, Piscataway, NJ 08854 USA. Digital Object Identifier 10.1109/TMI.2002.808364 I. INTRODUCTION B REAST cancer is the one of the leading causes of death in women. Between 10%–30% of women who have breast cancer and undergo mammography have negative mam- mograms. In about two-thirds of these cases, the radiologist failed to detect the cancer that was evident retrospectively. Such misses have been attributed to the subtle nature of the visual findings, poor image quality, fatigue, or oversight by the radiologist. Mammography is the most effective method for early detection of breast cancer, and periodic screening of asymp- tomatic women reduces the mortality rate [15], [40], [45]–[47]. Typically, the first step in breast cancer detection is screening mammography. This is a low-dose X-ray examination on asymptomatic women. Diagnostic mammography is an X-ray examination done to evaluate a breast complaint or to inves- tigate an abnormality found during a physical examination or during screening mammography. Breast ultrasound is some- times used to evaluate breast abnormalities that are found at screening mammography, diagnostic mammography or on a physical exam [15], [40], [45]. Breast ultrasound is also used for screening women with dense breasts as this is difficult to do using routine mammography. If something suspicious is found on the ultrasound, the patient is referred to for a surgical biopsy or core needle biopsy. Ultrasound can diagnose cysts with an accuracy approaching 100% and, hence, could reduce the potential number of unnecessary biopsies. On an average, less than 30% of the masses referred for surgical breast biopsy are actually malignant [9], [10], [15], [45]. Computer-aided diagnosis (CAD) refers to the use of com- puters in helping doctors recognize abnormal areas in a medical image. One goal of CAD is to increase the efficiency and ef- fectiveness of breast cancer screening by using the computer as a second reader [1], [2]. Considering the traumatic nature and cost of biopsy, it is desirable to develop computer-based tech- niques to distinguish accurately between cysts and malignant tumors. However, on account of the extremely noisy nature of ultrasound images, CAD work with this modality has been lim- ited. The specular nature of sonograms makes image segmenta- tion and object delineation a formidable task. Further, the pres- ence of tumor like structures in the image, e.g., glandular tissue, coopers ligaments, and subcutaneous fat [15] makes it difficult to automatically determine the lesion area using conventional image processing and computer vision techniques alone. Hence, most of the breast cancer related CAD work has been done on digital or digitized mammograms [3], [5]–[7], [14], [41]. Dig- ital mammograms are acquired on a digital X-ray machine and a 0278-0062/03$17.00 © 2003 IEEE