Image Segmentation with Implicit Color Standardization Using Spatially Constrained Expectation Maximization: Detection of Nuclei J. Monaco 1 , J. Hipp 2 , D. Lucas 2 , U. Balis 2 , S. Smith 2 , A. Madabhushi 1* 1 Department of Biomedical Engineering, Rutgers University, USA. 2 Department of Pathology, University of Michigan, USA. Abstract. Color nonstandardness — the propensity for similar objects to exhibit different color properties across images — poses a significant problem in the computerized analysis of histopathology. Though many papers propose means for improving color constancy, the vast majority assume image formation via reflective light instead of light transmis- sion as in microscopy, and thus are inappropriate for histological anal- ysis. Previously, we presented a novel Bayesian color segmentation al- gorithm for histological images that is highly robust to color nonstan- dardness; this algorithm employed the expectation maximization (EM) algorithm to dynamically estimate — for each individual image — the probability density functions that describe the colors of salient objects. However, our approach, like most EM-based algorithms, ignored impor- tant spatial constraints, such as those modeled by Markov random field (MRFs). Addressing this deficiency, we now present spatially-constrained EM (SCEM), a novel approach for incorporating Markov priors into the EM framework. With respect to our segmentation system, we replace EM with SCEM and then assess its improved ability to segment nuclei in H&E stained histopathology. Segmentation performance is evaluated over seven (nearly) identical sections of gastrointestinal tissue stained us- ing different protocols (simulating severe color nonstandardness). Over this dataset, our system identifies nuclear regions with an area under the receiver operator characteristic curve (AUC) of 0.838. If we disregard spatial constraints, the AUC drops to 0.748. 1 Introduction Color nonstandardness, the propensity for similar objects (e.g. cells) to exhibit different color properties across images, poses a significant challenge in the anal- ysis of histopathology images. This nonstandardness typically results from vari- ations in tissue fixation, staining, and digitization. Though methods have been proposed for improving color constancy in images formed via reflective light (see [1] for a review) and for mitigating the analogous intensity drift in grayscale images (e.g. MRI [2]), these methods are not extensible to color images formed * This work was made possible through funding from the NCI (R01CA136535-01, R01CA140772-01A1, R03CA128081-01) and the Burroughs Wellcome Fund.