(IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 14, No. 11, 2023 1063 | Page www.ijacsa.thesai.org Using Generative Adversarial Networks and Ensemble Learning for Multi-Modal Medical Image Fusion to Improve the Diagnosis of Rare Neurological Disorders Dr.Bhargavi Peddi Reddy 1 , Dr K Rangaswamy 2 , Doradla Bharadwaja 3 , Mani Mohan Dupaty 4 , Partha Sarkar 5 , Dr. Mohammed Saleh Al Ansari 6 Associate Professor, Dept of CSE, Associate Professor, Vasavi College of Engineering, Hyderabad, India 1 Associate Professor, Department of Computer Science and Engineering (Data Science), Rajeev Gandhi Memorial College of Engineering and Technology (Autonomous) Nandyal, Andhra Pradesh, India 2 Assistant Professor, Information Technology, Prasad V Potluri Siddhartha Institute of Technology, Vijayawada, India 3 Assistant Professor, Dept. of Computer Science & Engineering, Koneru Lakshmaiah Education Foundation, Vaddeswaram, Guntur District -522502, Andhra Pradesh, India 4 Department of Electronics and Communication Engineering, National Institute of Technology, Mahatma Gandhi Road, Durgapur, West Bengal, India 5 Associate Professor, College of Engineering-Department of Chemical Engineering, University of Bahrain, Bahrain 6 Abstract—The research suggests a unique ensemble learning approach for precise feature extraction and feature fusion from multi-modal medical pictures, which may be applied to the diagnosis of uncommon neurological illnesses. The proposed method makes use of the combined characteristics of Convolutional Neural Networks and Generative Adversarial Networks (CNN-GAN) to improve diagnostic accuracy and enable early identification. In order to do this, a diverse dataset of multi-modal patient medical records with rare neurological disorders was gathered. The multi-modal pictures are successfully combined using a GAN-based image-to-image translation technique to produce fake images that effectively gather crucial clinical data from different paradigms. To extract features from extensive clinical imaging databases, the research employs trained models using transfer learning approaches with CNN frameworks designed specifically for analyzing medical images. By compiling unique traits from each modality, a thorough grasp of the core pathophysiology is produced. By combining the strengths of several CNN algorithms using ensemble learning techniques including voting by majority, weight averaging, and layering, the forecasts were also integrated to arrive at the final diagnosis. In addition, the ensemble approach enhances the robustness and reliability of the assessment algorithm, resulting in increased effectiveness in identifying unusual neurological conditions. The analysis of the collected data shows that the proposed technique outperforms single-modal designs, demonstrating the importance of multi- modal fusion of pictures and feature extraction. The proposed method significantly outperforms existing methods, achieving an accuracy of 99.99%, as opposed to 85.69% for XGBoost and 96.12% for LSTM. The proposed method significantly outperforms existing methods, achieving an average increase in accuracy of approximately 13.3%. The proposed method was implemented using Python software. Keywords—Multi-modal medical images; ensemble learning; CNN; GAN; neurological disorders; image-to-image method; transfer learning; feature extraction I. INTRODUCTION A neurological condition called Alzheimer's disease (AD) causes diminished cognitive abilities as well as memory and mobility problems. As civilization gets older, this illness affects a growing number of elderly people. According to research, poorer nations have a significantly greater incidence of AD than advanced economies do [1]. MCI gradually develops into AD as the disorder progresses, and early mild cognitive impairment (EMCI) and late mild cognitive impairment (LMCI) are transitional states among healthy normal persons and those with Alzheimer's. Therefore, it is crucial to understand the best way to properly diagnose MCI and AD. The prevalence of brain illnesses has increased recently all across the world. One of the world's most prevalent neurological conditions is Alzheimer's disease (AD) that primarily manifests clinically as diminished memory and loss of mental abilities, along with difficulties with language and abnormalities of movement. AD is currently the fifth most common cause fatalities in the United States. The Alzheimer's Disease Association of USA published an article in 2018. The information provided by the Center for Health Statistics on the rate of change in death from a variety of hazardous illnesses in the US [2]. The number of fatalities from various risk conditions has increased. Moreover, 123% rise in AD prevalence has been reported. Another study found that in 2050, there will be around one million new instances of dementia caused by Alzheimer's, with a fresh case being identified every 33 seconds. One of the main illnesses endangering the well-being of elderly people and having an impact on social sustainability is AD. Presently, only a few medications have proven effective for the medical management