Copyright © 2018 Authors. This is an open access article distributed under the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. International Journal of Engineering & Technology, 7 (4.31) (2018) 322-325 International Journal of Engineering & Technology Website: www.sciencepubco.com/index.php/IJET Research paper HARIRAYA: a Novel Breast Cancer Pseudo-Color Feature for Multimodal Mammogram using Deep Learning Azree Shahrel Ahmad Nazri 1 *, Olalekan Agbolade 2 1 Department of Computer Science, FSKTM, UPM 2 Institute Bioscience, UPM *Corresponding author E-mail: azree@upm.edu.my Abstract Breast cancer is the leading cancer in the world. Mammogram is a gold standard for detecting breast cancer at earlier screening because of its sensitivity. Standard grayscale mammogram images are used by expert radiologists and Computer Aided-Diagnosis (CAD) systems. Yet, this original x-ray color provides little information to human radiologists and CAD systems to make decision. This binary color code thus affects sensitivity and specificity of prediction and subsequently affects accuracy. In order to enhance classifier models’ perfor- mance, this paper proposes a novel feature-level data integration method that combines features from grayscale mammogram and spec- trum mammogram based on a deep neural network (DNN), called HARIRAYA. Pseudo-color is generated using spectrum color code to produce Spectrum mammogram from grayscale mammogram. The DNN is trained with three layers: grayscale, false-color and joint fea- ture representation layers. Empirical results show that the multi-modal DNN model has a better performance in the prediction of malig- nant breast tissue than single-modal DNN using HARIRAYA features. Keywords: Breast Cancer; convolutional neural network; mammogram; multi-modal features; false color 1. Introduction Breast cancers are increasing in the alarming rate worldwide. This particular type of cancer is the leading cancer among women [1]. Typically breast tissues are characterized by background tissues: fatty tissues, fatty glandular tissues, and dense glandular tissues. As for breast cancer tissues, they are divided into two severities of abnormalities: benign and malignant [2]. Benign tumor cells are different only slightly in behavior and appearance from their tissue of origin while Malignant on the other hand describes tissues that grow rapidly and capable of metastasizing. Both abnormalities are categorized into six classes: Calcification, Spiculated masses, Ill- defined masses, Architectural distortion, Well- defined/circumscribed masses and Asymmetry. The major classes of abnormalities in breast cancer are calcification and masses [2]. Calcifications grow inside the soft breast tissue as a small lump of calcium deposits. Masses is typically difficulty to detect due to its poor exhibition of image contrast [3]. Architectural distortion and asymmetry are the most difficult tissue abnormalities to detect. These two abnormalities do not have clear features or characteris- tics as observables. These problems in detecting tissue abnormali- ties in both abnormal tumors are great challenge to radiologists and in medical image processing. The most operative and low- level ways have been developed by digital mammograms for the detection of abnormal tumors. Radiologists mainly use their eyes with the help of breast cancer diagnostic to discern cancer they screen the mammograms. Yet, cancer is not easily detected in many cases by the naked eyes as a result of poor imaging conditions. For instance, the appearance of unstable and subtle breast cancer on mammogram in their early stage which may cause the radiologists to misjudge the abnormali- ty if only diagnosis by experiences is used [2], making the perfor- mance to vary from 65% to 88%. These challenges faced by hu- man limitations can partially be overcome by using CAD system which typically serves as alternative opinion to radiologist. The major goal of CAD system is to detect and screen the breast tissue abnormalities with improved accuracy and reliability using com- puter vision and machine learning approaches. The typical system flow of CAD system is shown in Figure 1. Several variants of CAD system for detection of abnormalities have been proposed and developed. A CAD system that utilized Gabor filters and undecimated wavelet transform have been suggested [4], [5]. Fig. 1: Block diagram of a CAD system The extraction of multi-resolutional features which quantify the mass shapes was performed by the proposition of DWT modulus maxima technique with wavelet based sub-band image decompo-