A Novel Stochastic Combination of 3D Texture Features for Automated Segmentation of Prostatic Adenocarcinoma from High Resolution MRI Anant Madabhushi 1 , Michael Feldman 1 , Dimitris Metaxas 2 , Deborah Chute 1 , and John Tomaszewski 1 1 University of Pennsylvania, Philadelphia, PA 19104 {anantm@seas.upenn.edu} 2 Rutgers the State University of New Jersey, Piscataway, NJ 08854 {dnm@cs.rutgers.edu} Abstract. In this work, we present a new methodology for fully auto- mated segmentation of prostatic adenocarcinoma from high resolution MR by using a novel feature ensemble of 3D texture features. This work represents the first attempt to solve this difficult problem using high resolution MR. The difficulty of the problem stems from lack of shape and structure in the adenocarcinoma. Hence, in our methodology we compute statistical, gradient and Gabor filter features at multiple scales and orientations in 3D to capture the entire range of shape, size and orientation of the tumor. For an input scene, a classifier module gener- ates Likelihood Scenes for each of the 3D texture features independently. These are then combined using a weighted feature combination scheme. The ground truth for quantitative evaluation was generated by an expert pathologist who manually segmented the tumor on the MR using regis- tered histologic data. Our system was quantitatively compared against the performance of the individual texture features and against an ex- pert’s manual segmentation based solely on visual inspection of the 4T MR data. The automated system was found to be superior in terms of Sensitivity and Positive Predictive Value. 1 Introduction Prostatic adenocarcinoma is the most common malignancy of men with an esti- mated 189,000 new cases in the USA in 2002 and is the most frequently diagnosed cancer among men. Prostate cancer is most curable when detected early. Current screening for prostate cancer relies on digital rectal exam and serum prostate specific antigen (PSA) levels [2]. Definitive diagnosis of prostate carcinoma, how- ever, rests upon histologic tissue analysis, most often obtained via needle biopsy guided by transrectal ultrasound (TRUS). Magnetic resonance imaging of the prostate gland is a relatively new technique for staging prostate cancer and it has been shown to produce better tissue contrast between cancers in the pe- ripheral zone compared to ultrasound [2]. The 1.5T MR has been shown to be R.E. Ellis and T.M. Peters (Eds.): MICCAI 2003, LNCS 2878, pp. 581–591, 2003. c  Springer-Verlag Berlin Heidelberg 2003