C. Barillot, D.R. Haynor, and P. Hellier (Eds.): MICCAI 2004, LNCS 3216, pp. 274–282, 2004. © Springer-Verlag Berlin Heidelberg 2004 Automatic Optimization of Segmentation Algorithms Through Simultaneous Truth and Performance Level Estimation (STAPLE) Mahnaz Maddah, Kelly H. Zou, William M. Wells, Ron Kikinis, and Simon K. Warfield Computational Radiology Laboratory, Surgical Planning Laboratory, Brigham and Women’s Hospital, Harvard Medical School, Boston MA 02115, USA. {mmaddah, zou, sw, kikinis, warfield}@bwh.harvard.edu http://spl.bwh.harvard.edu Abstract. The performance of automatic segmentation algorithms often de- pends critically upon a number of parameters intrinsic to the algorithm. Appro- priate setting of these parameters is a pre-requisite for successful segmentation, and yet may be difficult for users to achieve. We propose here a novel algo- rithm for the automatic selection of optimal parameters for medical image seg- mentation. Our algorithm makes use of STAPLE (Simultaneous Truth and Per- formance Level Estimation), a previously described and validated algorithm for automatically identifying a reference standard by which to assess segmentation generators. We execute a set of independent automated segmentation algo- rithms with initial parameter settings, on a set of images from any clinical ap- plication under consideration, estimate a reference standard from the segmenta- tion results using STAPLE, and then identify the parameter settings for each algorithm that maximizes the quality of the segmentation generator result with respect to the reference standard. The process of estimating a reference standard and estimating the optimal parameter settings is iterated to convergence. 1 Introduction The analysis of medical images is a critical process, enabling applications ranging from fundamental neuroscience, to objective evaluation of interventions and drug treatments, to monitoring, navigation and assessment of image guided therapy. Seg- mentation is the key process by which raw image acquisitions are interpreted. Inter- active segmentation is fraught with intra-rater and inter-rater variability which limits its accuracy, while also being costly and time-consuming. Automatic segmentation holds out the potential of dramatically increased precision, and reduction in time and expense. However, the performance of automatic segmentation algorithms often de- pends critically upon a number of parameters intrinsic to the algorithm. Such pa- rameters may control assumptions regarding tissue intensity characteristics, spatial homogeneity constraints, boundary smoothness or curvature characteristics or other prior information. Appropriate setting of these parameters by users is often a pre- requisite for successful segmentation, and yet may be difficult for users to achieve