Integrated Segmentation and Nonrigid Registration for Application in Prostate Image-Guided Radiotherapy Chao Lu 1,⋆ , Sudhakar Chelikani 2 , Zhe Chen 2 , Xenophon Papademetris 2 , Lawrence H. Staib 1,2 , and James S. Duncan 1,2 1 Department of Electrical Engineering 2 Department of Diagnostic Radiology Yale University, New Haven, CT, USA chao.lu@yale.edu Abstract. Many current image-guided radiotherapy (IGRT) systems incorporate an in-room cone-beam CT (CBCT) with a radiotherapy lin- ear accelerator for treatment day imaging. Segmentation of key anatom- ical structures (prostate and surrounding organs) in 3DCBCT images as well as registration between planning and treatment images are es- sential for determining many important treatment parameters. Due to the image quality of CBCT, previous work typically uses manual seg- mentation of the soft tissues and then registers the images based on the manual segmentation. In this paper, an integrated automatic segmen- tation/constrained nonrigid registration is presented, which can achieve these two aims simultaneously. This method is tested using 24 sets of real patient data. Quantitative results show that the automatic segmentation produces results that have an accuracy comparable to manual segmenta- tion, while the registration part significantly outperforms both rigid and non-rigid registration. Clinical application also shows promising results. 1 Introduction Prostate cancer is the most commonly diagnosed cancer among men in the United States. For the majority, external beam radiotherapy (EBRT) is one of the primary treatment modalities for prostate cancer [1]. Recent advances in EBRT have led to three-dimensional conformal radiother- apy (3DCRT) and intensity modulated radiotherapy(IMRT). Prostate 3DCRT requires a precise delineation of the target volume and the adjacent critical or- gans in order to deliver an optimal dose to the prostate with minimal side effect on nearby normal tissues. Many current image-guided radiotherapy (IGRT) sys- tems integrate an in-room cone-beam CT (CBCT) with a radiotherapy linear accelerator for treatment day imaging. With both imaging and radiotherapy available on the same platform, daily CBCTs can now be acquired and used for patient positioning. ⋆ This work is supported by NIH/NIBIB Grant R01EB002164. T. Jiang et al. (Eds.): MICCAI 2010, Part I, LNCS 6361, pp. 53–60, 2010. c  Springer-Verlag Berlin Heidelberg 2010