Segmentation of Brain MR Images via Sparse Patch Representation Tong Tong 1 , Robin Wolz 1 , Joseph V. Hajnal 2 , and Daniel Rueckert 1 1 Department of Computing, Imperial College London, London, UK 2 MRC Clinical Sciences Center, Imperial College London, London, UK t.tong11@imperial.ac.uk Abstract. Recently, patch-based segmentation has been proposed for brain MR images. However, the segmentation accuracy of this method depends on similarities over small image patches, which may not be an optimal estimator. In this paper, we propose a new segmentation strat- egy based on patch reconstruction rather than patch similarity. In the proposed method, the training patch library is considered as a dictio- nary, and the target patch is modeled as a sparse linear combination of the atoms in the dictionary. The sparse representation is naturally dis- criminative, which presents an entirely data-driven approach to patch- selection and label definition. This Sparse Representation Classification (SRC) strategy produces segmentation results that compare favourably to existing approaches. In addition, a smoothing term is added to the cost function of the sparse coding technique, making the proposed method more robust. To the best of our knowledge, the sparse representation technique has never been used in brain segmentation. In a leave-one-out validation, the proposed method yields a median Dice coefficient of 0.871 for hippocampus on 202 ADNI images, which is competitive compared with state-of-the-art methods. 1 Introduction Magnetic resonance imaging (MRI) is the primary imaging modality for the analysis of brain structures. It enables us to describe how brain anatomy changes during aging or disease progression. For the extraction of biomarkers for diseases like Alzheimers Disease (AD) or schizophrenia, the accurate and robust segmen- tation of subcortical structures is an essential step. Since manual labeling by clinical experts is a highly laborious task, an automated technique is desirable to allow a routine analysis of brain MRIs in clinical use. However, it still re- mains a challenging task to develop fast and accurate automated segmentation methods due to the complexity of subcortical structures. Several automated methods have been reported to extract subcortical struc- tures. Among them, atlas-based methods have been shown to outperform other state-of-the-art algorithms [1]. To avoid bias by using a single atlas, several simi- lar atlases can be used to improve the segmentation performance [2,3]. However, the segmentation performance of multi-atlas techniques is directly affected by