Medical Image Analysis 50 (2018) 82–94 Contents lists available at ScienceDirect Medical Image Analysis journal homepage: www.elsevier.com/locate/media Direct delineation of myocardial infarction without contrast agents using a joint motion feature learning architecture Chenchu Xu a,b , Lei Xu c , Zhifan Gao a , Shen Zhao a , Heye Zhang d,∗ , Yanping Zhang b , Xiuquan Du b , Shu Zhao b , Dhanjoo Ghista e , Huafeng Liu f , Shuo Li a a Department of Medical Imaging, Western University, London ON, Canada b School of Computer Science and Technology, Anhui University, Hefei, China c Department of Medical Imaging, Beijing Anzhen Hospital, Capital Medical University, Beijing, China d School of Biomedical Engineering, Sun Yat-Sen University, Shenzhen, China e University 2020 Foundation, MA,USA f Department of Optical Engineering, Zhejiang University, Hangzhou, China a r t i c l e i n f o Article history: Received 13 July 2017 Revised 25 August 2018 Accepted 5 September 2018 Available online 6 September 2018 Keywords: Myocardial infarction Deep learning Motion feature Optical flow a b s t r a c t Changes in mechanical properties of myocardium caused by a infarction can lead to kinematic abnormal- ities. This phenomenon has inspired us to develop this work for delineation of myocardial infarction area directly from non-contrast agents cardiac MR imaging sequences. The main contribution of this work is to develop a new joint motion feature learning architecture to efficiently establish direct correspondences between motion features and tissue properties. This architecture consists of three seamless connected function layers: the heart localization layers can automatically crop the region of interest (ROI) sequences involving the left ventricle from the cardiac MR imaging sequences; the motion feature extraction layers, using long short-term memory-recurrent neural networks, a) builds patch-based motion features through local intensity changes between fixed-size patch sequences (cropped from image sequences), and b) uses optical flow techniques to build image-based features through global intensity changes between adja- cent images to describe the motion of each pixel; the fully connected discriminative layers can combine two types of motion features together in each pixel and then build the correspondences between mo- tion features and tissue identities (that is, infarct or not) in each pixel. We validated the performance of our framework in 165 cine cardiac MR imaging datasets by comparing to the ground truths manually segmented from delayed Gadolinium-enhanced MR cardiac images by two radiologists with more than 10 years of experience. Our experimental results show that our proposed method has a high and sta- ble accuracy (pixel-level: 95.03%) and consistency (Kappa statistic: 0.91; Dice: 89.87%; RMSE: 0.72 mm; Hausdorff distance: 5.91 mm) compared to manual delineation results. Overall, the advantage of our framework is that it can determine the tissue identity in each pixel from its motion pattern captured by normal cine cardiac MR images, which makes it an attractive tool for the clinical diagnosis of infarction. © 2018 Elsevier B.V. All rights reserved. 1. Introduction Direct delineation of myocardial infarction (MI) area without contrast agents highly impacts early patient management and ther- apy planning. In routine clinical practice, delayed enhancement (DE) – cardiac magnetic resonance (CMR) imaging can be consid- ered as the current standard for the detection of infarction area be- cause it uses gadolinium contrast agent to provide highly accurate delineation of MI area during the imaging process (Jörg Barkhausen et al., 2002; Ingkanisorn et al., 2004). However, this imaging pro- ∗ Corresponding author. E-mail address: zhangheye@mail.sysu.edu.cn (H. Zhang). cess may be dangerous because the administration of gadolinium contrast agent is fatal to the patients with chronic end-stage kid- ney diseases. According to US renal data system, more than 40% patients with chronic kidney disease suffered cardiovascular dis- ease. Approximately 20% of acute MI patients accompany with chronic kidney disease (Fox et al., 2010; Kali et al., 2014). Moreover, recent studies have demonstrated that gadolinium might deposit into the skin, dentate nucleus, and globus pallidus of the patients with normal renal function (Stojanov et al., 2016). Thus, direct de- lineation of MI without a contrast agent is a great clinical advance- ment, not only for indicating the presence, location, and transmu- ral extent of acute and chronic MI (Choi et al., 2001; Kramer et al., https://doi.org/10.1016/j.media.2018.09.001 1361-8415/© 2018 Elsevier B.V. All rights reserved.