Cross modality guided liver image enhancement of CT using MRI Rabia Naseem 1 , Faouzi Alaya Cheikh 1 , Azeddine Beghdadi 2 , Ole Jacob Elle 3 , Frank Lindseth 1 1 Department of Computer Science, Norwegian University of Science and Technology, Norway 2 L2TI, Institut Galil´ ee, University Paris 13, France, 3 Oslo University Hospital, University of Oslo, Norway {rabia.naseem, faouzi.cheikh, frankl}@ntnu.no, azeddine.beghdadi@univ-paris13.fr, oelle@ous-hf.no Abstract—Low contrast Computed Tomographic (CT) images often hamper the diagnosis of critical tumors found in various hu- man organs. Contrast enhancement schemes play significant role in improving the visualization of these structures. To achieve this objective, Crossmodality Guided Enhancement (CMGE) method is proposed in this paper. The idea is to exploit the diversity of the information extracted from one modality to enhance the important structures including vessels and tumors in another modality. Our method employs information from liver Magnetic Resonance Image (MRI) to generate an enhanced CT image. It entails applying two dimensional histogram specification to map 2D histogram of CT to that of MRI followed by application of top and bottom hat transformations. These morphological operations highlight areas brighter than their surroundings and suppress darker areas. The final image is obtained by combining the results of these operations. Our method is compared with other state of the art contrast enhancement methods both visually and in terms of quality assessment metrics IEM and EME. The results show that our method performs better than these methods. CMGE technique yields improved contrast in low contrast CT images of the human liver and highlights tumors and vessels. Index Terms—2D histogram specification, top hat, bottom hat transform, contrast enhancement I. I NTRODUCTION Image enhancement is widely used in the field of medical imaging [1], [7], [9], [11]. Undesired noise, poor contrast, illumination variations and other artifacts are often introduced while acquiring medical images including CT, MRI and US. Surgical outcomes are greatly enhanced after the integration of image guidance techniques with surgical procedures. The un- desired image artifacts limit the efficiency of image guidance techniques [3]. However, image enhancement techniques are extremely useful in streamlining the planning and navigation phases. They improve visualization of liver and its internal anatomy to help doctors better diagnose existence of liver tumor and plan intervention accordingly. Moreover, they make subsequent image-based navigation tasks such as registration, feature extraction and segmentation more robust [25]. Computed tomographic imaging is regarded as a primary tool in diagnosis of various human diseases. However, low contrast and imprecise visualization are the drawbacks that limit its utility. CT is frequently preferred over other modalities owing to its quick acquisition time, better ability to capture bony structures and low cost. Keeping human liver into consid- eration, few structures such as tumors can be better visualized in MR image, while certain vessels are clearly visible in CT. Better diagnosis can be done if information from multiple imaging modalities can be combined in a certain way to get an enhanced image. However, there are few published works on the combination of multiple modalities in the design of guided contrast enhancement methods [4], [5], [9], [10]. The paper is organized as follows. First, we present the review of the recent contrast enhancement approaches. Then, we present our proposed technique. Afterwards, we discuss our results and present comparison with few existing methods. Finally, conclusion is presented. II. RELATED WORK Several image enhancement methods have been proposed in the literature including spatial domain [2], [4], [6] and trans- form domain methods [1], [11]. Wavelet based approaches decompose the image into different scales. In transform based approach, the image is first decomposed into spatial frequency components. Then each component is processed in order to adapt the energy amplification to its spatial frequency content. Wavelet-based methods have been proved to provide better contrast enhancement than other transform based methods. Moreover, wavelet decomposition offers the possibility to denoise the signal and enhance selectively the contrast si- multaneously [30], [31]. Another scheme calculates enhance- ment parameters according to local dispersion of wavelet coefficients [11]. Recently, multimodal image enhancement techniques have been proposed [4], [5] in the context of natural images; these methods denoise an image using its clean counterpart. To propose a cross modality guided denoising scheme is challenging, however, contents in the natural images used under the abovementioned schemes are exactly same, and both images are perfectly registered. Moreover, a pixel level fusion scheme could be used in order to simplify the enhancement process [4], [21]. One of the earlier attempts to use Near Infra-Red (NIR) images for enhancing visible pho- tographs exploited their similar statistical characteristics [4]. The authors used histogram matching in gradient domain to transfer NIR contrast to target image and wavelet coefficients for passing texture information. Nevertheless, the method fails for low light images. Jiang et. al. used dark channel prior model to improve perceptual quality of videos taken at night [22]. Another scheme uses dark flashed infrared noise free image to denoise the corresponding color image. A scale map [9]