Contents lists available at ScienceDirect Physica Medica journal homepage: www.elsevier.com/locate/ejmp Original paper NF-RCNN: Heart localization and right ventricle wall motion abnormality detection in cardiac MRI Saeed Kermani a,1 , Mostafa Ghelich Oghli b,c, ⁎ ,1 , Ali Mohammadzadeh c , Raheleh Kafieh d a School of Advanced Technologies in Medicine, Isfahan University of Medical Sciences, Isfahan, Iran b Research and Development Department, Med Fanavarn Plus Co., Karaj, Iran c Rajaie Cardiovascular Medical and Research Center, Iran University of Medical Science, Tehran, Iran d Medical Image & Signal Processing (MISP) Research Center, School of Advanced Technologies in Medicine, Isfahan University of Medical Sciences, Isfahan, Iran ARTICLE INFO Keywords: Medical localization Cardiac magnetic resonance imaging Faster RCNN Convolutional networks ABSTRACT Convolutional neural networks (CNNs) are extensively used in cardiac image analysis. However, heart locali- zation has become a prerequisite to these networks since it decreases the size of input images. Accordingly, recent CNNs benefit from deeper architectures in gaining abstract semantic information. In the present study, a deep learning-based method was developed for heart localization in cardiac MR images. Further, Network in Network (NIN) was used as the region proposal network (RPN) of the faster R-CNN, and then NIN Faster-RCNN (NF-RCNN) was proposed. NIN architecture is formed based on “MLPCONV” layer, a combination of convolu- tional network and multilayer perceptron (MLP). Therefore, it could deal with the complicated structures of MR images. Furthermore, two sets of cardiac MRI dataset were used to evaluate the network, and all the evaluation metrics indicated an absolute superiority of the proposed network over all related networks. In addition, FROC curve, precision-recall (PR) analysis, and mean localization error were employed to evaluate the proposed network. In brief, the results included an AUC value of 0.98 for FROC curve, a mean average precision of 0.96 for precision-recall curve, and a mean localization error of 6.17 mm. Moreover, a deep learning-based approach for the right ventricle wall motion analysis (WMA) was performed on the first dataset and the effect of the heart localization on this algorithm was studied. The results revealed that NF-RCNN increased the speed and decreased the required memory significantly. 1. Introduction Cardiovascular diseases (CVDs), resulting in a great deal of mor- tality and morbidity, are the number one cause of death in the world. As reported by the World Health Organization, more than 23 million an- nual deaths will occur due to CVDs by 2030 [1]. The most important factor in controlling and treating these types of diseases is an early diagnosis, which is achievable by utilizing various diagnostic tools such as imaging systems. Among various imaging modalities available to visualize the heart, magnetic resonance imaging (MRI) has demonstrated a strong cap- ability for diagnosing CVDs and evaluating cardiac function. In recent decades, a wealth of approaches has been offered to extract clinically relevant information from Cardiac magnetic resonance (CMR) images [2,59] and echocardiography [60,61]. Determining a region of interest (ROI) centered on the heart decreases the computational load in all these analyses. Heart localization is extremely useful in fully auto- mated, and especially deep learning, applications although it is less likely to interest practicing clinicians. The main advantage of heart localization is increasing the efficiency of the forehead algorithms (such as segmentation or sequence processing etc.) by forcing the procedure to only face the heart and ignoring the organs nearby [3]. As shown in Fig. 1, localized heart can appear in different sizes and in the presence or absence of lung. Object detection and localization are two core tasks in computer vision as they are applied in many real-world applications, such as autonomous vehicles and robotics. Introducing region-based convolu- tional network (R-CNN) in 2014 made a major breakthrough in object detection [4]. R-CNN inspired several further methods such as fast R- CNN [5], faster R-CNN [6], YOLO [7], SSD [8] and R-FCN [9]. In ad- dition, R-CNN is considered a robust and accurate object detection technique in natural images [4]. It presents a simple and scalable https://doi.org/10.1016/j.ejmp.2020.01.011 Received 7 June 2019; Received in revised form 31 December 2019; Accepted 9 January 2020 ⁎ Corresponding author at: 10th St. Shahid Babaee Blvd. Payam Special Economic Zone, Karaj, Iran. E-mail address: m.g31.mesu@gmail.com (M. Ghelich Oghli). 1 Both authors contributed equally to this manuscript. Physica Medica 70 (2020) 65–74 1120-1797/ © 2020 Associazione Italiana di Fisica Medica. Published by Elsevier Ltd. All rights reserved. T