IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 22, NO. 9, SEPTEMBER 2003 1111 Mutual Information-Based CT-MR Brain Image Registration Using Generalized Partial Volume Joint Histogram Estimation Hua-mei Chen and Pramod K. Varshney*, Fellow, IEEE Abstract—Mutual information (MI)-based image registration has been found to be quite effective in many medical imaging ap- plications. To determine the MI between two images, the joint his- togram of the two images is required. In the literature, linear in- terpolation and partial volume interpolation (PVI) are often used while estimating the joint histogram for registration purposes. It has been shown that joint histogram estimation through these two interpolation methods may introduce artifacts in the MI registra- tion function that hamper the optimization process and influence the registration accuracy. In this paper, we present a new joint his- togram estimation scheme called generalized partial volume esti- mation (GPVE). It turns out that the PVI method is a special case of the GPVE procedure. We have implemented our algorithm on the clinically obtained brain computed tomography and magnetic resonance image data furnished by Vanderbilt University. Our ex- perimental results show that, by properly choosing the kernel func- tions, the GPVE algorithm significantly reduces the interpolation- induced artifacts and, in cases that the artifacts clearly affect reg- istration accuracy, the registration accuracy is improved. Index Terms—Image registration, interpolation-induced arti- facts, joint histogram estimation, mutual information, registration of brain CT and MR images. I. INTRODUCTION M ULTIMODALITY image registration has become an important research topic because of its great value in a variety of applications. For medical image analysis, an image showing functional and metabolic activity—such as single photon emission computed tomography (SPECT), positron emission tomography (PET), and magnetic resonance spec- troscopy (MRS)—is often registered to an image which shows anatomical structures, such as magnetic resonance image (MRI), computed tomography (CT), and ultrasound. These registered multimodality images lead to improved diagnosis, better surgical planning, more accurate radiation therapy and countless other medical benefits [1]. Existing image registration techniques can be broadly classified into two categories: feature-based and Manuscript received October 25, 2001; revised March 3, 2002. This work was supported in part by the Air Force Research Laboratory, Air Force Materiel Command, USAF, under Grant F30602-95-1-0027 and in part by the Defense Advanced Research Project Agency (DARPA) under Grant N66001-99-1-8922. The Associate Editor responsible for coordinating the review of this paper and recommending its publication was C. Meyer. Asterisk indicates corresponding author. H.-M. Chen is with the Department of Computer Science and Engineering, University of Texas at Arlington, Arlington, TX 76019 USA (e-mail: hchen@cse.uta.edu). *P. K. Varshney is with the Department of Electrical Engineering and Computer Sciences, Syracuse University, Syracuse, NY 13244 USA (e-mail: varshney@syr.edu). Digital Object Identifier 10.1109/TMI.2003.816949 intensity-based methods [2]. A feature-based method requires the extraction of features common in both images. Obviously, a feature-based method is data dependent. Since different image data may have different features, the feature extraction algo- rithms adopted in a feature-based image registration algorithm are expected to be different for different registration tasks. In contrast, intensity-based image registration techniques are free from this limitation because they do not deal with the identification of geometrical landmarks. The general design criterion of an intensity-based image registration technique can be expressed as (1) where and are the images to be registered. is the transformation, characterized by the pose parameters , that will be applied to the coordinates of each grid point in . is an intensity-based similarity measure calculated over the region of overlap of the two images. The above criterion says that the two images and are registered through when optimizes the selected similarity measure . Among a variety of existing similarity measures, mutual information (MI) has received substantial attention recently because of its ability to measure the similarity between images from different modalities, especially in, but not limited to, medical imaging applications [3]–[8]. Many aspects of the use of MI as the similarity measure to be maximized have been studied. In [9]–[11], three variations of MI are proposed to provide an overlap-invariant measure. In [12], maximization of MI is found to be a maximum likelihood estimation problem under very minimal assumptions. In [13], a multiresolution optimization approach using an optimizer specifically designed for the MI measure is presented. In [14], a multivariate MI measure is proposed to increase the accuracy provided that at least two highly accurate pre-registered images are available. In [15], an upper bound is derived to provide useful insights about the use of MI as a similarity measure. Finally, a phenomenon called interpolation artifacts that may appear in MI-based registration functions is studied in [16]. In [16], it is pointed out that, under certain circumstances, dis- cussed in Section III, existing joint histogram estimation methods may result in different types of artifact patterns in a MI-based reg- istration function. In that study, two joint histogram estimation methods were examined: linear interpolation and partial volume interpolation (PVI) [4]. It was shown that both methods may re- sult in significant artifact patterns that introduce spurious local 0278-0062/03$17.00 © 2003 IEEE