Hierarchical Normalized Cuts: Unsupervised Segmentation of Vascular Biomarkers from Ovarian Cancer Tissue Microarrays Andrew Janowczyk 1 , Sharat Chandran 1 , Rajendra Singh 2 , Dimitra Sasaroli 3 , George Coukos 3 , Michael D. Feldman 4 , and Anant Madabhushi 5,⋆ 1 Dept of Computer Science & Engineering, Indian Institute of Technology Bombay {andrew,sharat}@cse.iitb.ac.in 2 Quest Diagnostics, Inc, USA 3 Ovarian Cancer Research Center, University of Pennsylvania, USA 4 Dept of Pathology and Lab Medicine, University of Pennsylvania, USA 5 Dept of Biomedical Engineering, Rutgers University, USA anantm@rci.rutgers.edu Abstract. Research has shown that tumor vascular markers (TVMs) may serve as potential OCa biomarkers for prognosis prediction. One such TVM is ESM-1, which can be visualized by staining ovarian Tissue Microarrays (TMA) with an antibody to ESM-1. The ability to quickly and quantitatively estimate vascular stained regions may yield an im- age based metric linked to disease survival and outcome. Automated segmentation of the vascular stained regions on the TMAs, however, is hindered by the presence of spuriously stained false positive regions. In this paper, we present a general, robust and efficient unsupervised seg- mentation algorithm, termed Hierarchical Normalized Cuts (HNCut), and show its application in precisely quantifying the presence and ex- tent of a TVM on OCa TMAs. The strength of HNCut is in the use of a hierarchically represented data structure that bridges the mean shift (MS) and the normalized cuts (NCut) algorithms. This allows HNCut to efficiently traverse a pyramid of the input image at various color reso- lutions, efficiently and accurately segmenting the object class of interest (in this case ESM-1 vascular stained regions) by simply annotating half a dozen pixels belonging to the target class. Quantitative and qualitative analysis of our results, using 100 pathologist annotated samples across multiple studies, prove the superiority of our method (sensitivity 81%, Positive predictive value (PPV), 80%) versus a popular supervised learn- ing technique, Probabilistic Boosting Trees (sensitivity, PPV of 76% and 66%). ⋆ This work was supported via grants from the New Jersey Commission on Cancer Research, the National Cancer Institute (R21CA127186-01, R03CA128081-01), Wal- lace H. Coulter Foundation (PI: Anant Madabhushi) and Ovarian Cancer SPORE Grant P50 CA083638 (PI: George Coukos). G.-Z. Yang et al. (Eds.): MICCAI 2009, Part I, LNCS 5761, pp. 230–238, 2009. c  Springer-Verlag Berlin Heidelberg 2009