IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 55, NO. 1, JANUARY 2008 147 Atlas-Based Indexing of Brain Sections via 2-D to 3-D Image Registration Smadar Gefen*, Member, IEEE, Nahum Kiryati, Senior Member, IEEE, and Jonathan Nissanov, Member, IEEE Abstract—A 2-D to 3-D nonlinear intensity-based registration method is proposed in which the alignment of histological brain sections with a volumetric brain atlas is performed. First, sparsely cut brain sections were linearly matched with an oblique slice auto- matically extracted from the atlas. Second, a planar-to-curved sur- face alignment was employed in order to match each section with its corresponding image overlaid on a curved-surface within the atlas. For the latter, a PDE-based registration technique was de- veloped that is driven by a local normalized-mutual-information similarity measure. We demonstrate the method and evaluate its performance with simulated and real data experiments. An atlas- guided segmentation of mouse brains’ hippocampal complex, re- trieved from the Mouse Brain Library (MBL) database, is demon- strated with the proposed algorithm. Index Terms—Normalized mutual information, PDE-based methods, 2-D to 3-D nonlinear registration. I. INTRODUCTION I N RECENT years, numerous approaches for 2-D to 2-D and 3-D to 3-D image registration have been studied. Several comprehensive surveys of image registration have been pub- lished summarizing the research on this important topic [1]–[8]. Many of these registration algorithms dealt with images that had the same resolution, dimension, and support. However, dealing with nonisotropic, unevenly sampled, not equal-dimensional, or incomplete datasets remains a challenge. This challenge is espe- cially pertinent to biomedicine where volumetric images from various modalities are often reconstructed from previously ac- quired sectional images. A. Background In the literature, 2-D to 3-D alignment methods are often proposed in the context of aligning 3-D data to their projec- tive images in order to solve a 3-D model pose problem. An example of such an alignment is the spatial mapping between Manuscript received March 21, 2006; revised March 19, 2007. This work was supported in part by the Human Brain Project under Grant P20 MH62009 and in part by the National Science Foundation under DBI Grant 0352421. Asterisk indicates corresponding author. *S. Gefen is with PVI Virtual Media Services LLC, 15 Princess Road, Lawrenceville, NJ 08648 USA (e-mail: sgefen@pvi.tv). N. Kiryati is with the Electrical Engineering School, Tel-Aviv University, Ramat Aviv 69978, Israel (e-mail: nk@eng.tau.ac.il). J. Nissanov is with the Laboratory for Bioimaging and Anatomical In- formatics, Department of Neurobiology and Anatomy, Drexel University College of Medicine, Philadelphia, PA 19129-1096 USA (e-mail: jonathan.nis- sanov@drexelmed.edu). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TBME.2007.899361 the 3-D model of an object and the coordinate system of a physical scene. In image-guided surgery, 2-D to 3-D registra- tion is used to map a preoperative segmented 3-D model of an organ to the operating room coordinate system. For instance, intracranial aneurysms are treated via endovascular coiling: A micro-catheter is image-guided through a small puncture in the femoral artery up into the cerebral artery in the location of the aneurysm to be treated. In this application 3-D magnetic resonance angiography (MRA) is registered with 2-D X-ray angiograms using either an intensity-based method [9]–[11] or feature-based method [12]–[14]. Another example is localizing the center of a tumor in the liver which is to be treated with radio frequency. In this case, tumor center localization is achieved via registration of a 3-D CT image of the liver with 2-D video images [15]. Most of the studies dealing with 2-D to 3-D registration tech- niques were confined to rigid-body transformation [16]–[19]. Kim et al. [16], for instance, performed motion correction by mapping each slice of functional magnetic resonance imaging (fMRI) image onto the volumetric MR image which was ac- quired in the same session by using a rigid-body transformation. However, anatomical changes over time, patients’ movements, or limitations of the imaging procedure may create nonlinear deformation and, therefore, require a method that will compen- sate for that. Nonrigid registration of postmortem brain slices to MRI volume was proposed by Kim et al. [20], where a polynomial transformation was used to parametrically represent the defor- mation field. Slice-to-volume nonlinear registration was also proposed by Liu et al. [21] for the application of ultrasound spa- tial compounding. In [21], cubic B-spline functions were used to approximate the deformation field based on given control points. In contrast, in this study, we propose an image-based registration method (no corresponding points are required) that restores a free-form deformation field relating histological sections from experimental dataset to histological volumetric atlas of mouse brains. This operation facilitates indexing of sections of interest by the Atlas. B. Atlas-Based Indexing of Mouse Brain Sections Histological images of mouse brains allow detailed structural analysis and have been critical to our present understanding of the nervous system. They are generated by cutting the brains into thin slices (sections) that are then stained to demarcate spe- cific tissue types or localized particular chemical moieties. The tissue slices are then imaged with a scanner or a microscope, yielding a set of 2-D ordered but unaligned images. Often, not every tissue section is collected and the data sets have a much 0018-9294/$25.00 © 2007 IEEE