Pergamon Pattern Recognition, Vol. 28, No. 4, pp. 475-491, 1995 Elsevier Science Ltd Copyright @ 1995 Pattern Recognition Society Printed in Great Britain. All rights reserved m-3203/95 $9.50 + a0 0031-3203(94)00124-3 KNOWLEDGE-BASED ORGAN IDENTIFICATION FROM CT IMAGES MASAHARU KOBASHI and LINDA G. SHAPIRO* Department of Computer Science and Engineering, University of Washington, Seattle, WA 98195, U.S.A. (Received 29 July 1993; in revised form 8 September 1994; receivedfor publication 23 September 1994) Abstract-This paper describes a new knowledge-based procedure for identifying and extracting organs from normal CT imagery. Our procedure differs from previous attempts in its use of a wide variety of knowledge about both the anatomy and the image processing operations. The system features the use of constraint-based dynamic thresholding, negative-shape constraints to rapidly rule out infeasible segmenta- tions, and progressive landmarking that takes advantage of the different degrees of certainty of successful identification of each organ. The results of a series of tests on training data of 100 images from five patients plus additional test data of 75 images from three more patients indicate that the knowledge-based approach is promising. Knowledge-based vision Dynamic thresholding Medical imaging CT images 1. INTRODUCTION Patients who are scheduled to receive radiation treat- ment for cancer undergo CT scans which produce a sequence of images, each representing one slice through the three-dimensional (3D) organs and vessels being scanned. From this sequence of 2D images, it is possible to estimate the 3D structures through which the slices were taken and, if the estimate is good, to determine their approximate locations and volumes. This extrac- tion of structures from parallel CT images of the patients is an important first step in the creation of patient- specific models that can be used by treatment planning programs to deliver maximum dosage to the tumor and minimum dosage to critical anatomical structures. Currently this step is performed manually by technic- ians called dosimetrists who use an interactive device, such as a mouse, to trace the contours of each organ of each image of a patient data set. Although many attempts have been made to automate the extraction of anatomy, no techniques have been successful enough to replace the current manual methods. Since these methods take up to half the planning time, there remains a great need to speed up, if not completely automate the process. boundaries are located in some images. Standard tech- niques such as absolute thresholds, edge-finding, or region growing acting blindly on the gray tones of an image are not powerful enough. Instead the system needs a knowledge-based control structure that can use standard techniques in a goal-directed and in- formed manner and can evaluate its own success or failure. Automatic segmentation of CT images is a challeng- ing problem in computer vision. While the positions of the organs within each slice can be predicted the organs are flexible and the 2D contours they exhibit can vary. Furthermore, the boundaries separating organs from their surroundings are not always clear; even humans have to guess where some portions of The goal of our work is to develop a knowledge- based recognition system that utilizes knowledge of anatomy, knowledge of the imaging process, and knowledge of the effects of various image processing operators to extract the organs from parallel CT images. To this effect we have developed an experimental system that locates the major organs in sets of images through the abdomen. The major features of our system include (1) dynamic thresholding controlled by feedback in- formation on various properties of image regions, (2) the use of negative shape constraints (constraints that rule out certain impossible shapes), and (3) progressive landmarking that extracts organs in order of predicted success and uses already-found organs to help locate other organs. This paper describes the knowledge- based system we have implemented. Section 2 describes the characteristics of the problem. Section 3 sum- marizes the related literature on the knowledge-based approach. Section 4 indicates the knowledge used by our system. Section 5 discusses the algorithms we developed, and Section 6 describes the performance of the system. 2. CHARACTERISTICS OF THE PROBLEM *All correspondence to: Prof. Linda G. Shapiro, Depart- ment of Computer Science and Engineering, FR-35, University of Washington, Seattle, Washington 98195, U.S.A. Our task of constructing an automated dosimetry system is equivalent to producing a definition in com- putable terms for each organ as a homogeneous region Object recognition 475