Deep learning classification of cardiomegaly using combined imaging and non-imaging ICU data Declan Grant 1 , Bartlomiej W. Papie˙ z 2 , Guy Parsons 3 , Lionel Tarassenko 1 , and Adam Mahdi 1,* 1 Department of Engineering Science, University of Oxford, UK declan.grant@exeter.ox.ac.uk, lionel.tarassenko,adam.mahdi@eng.ox.ac.uk 2 Big Data Institute, Li Ka Shing Centre for Health Information and Discovery, University of Oxford, Oxford, UK bartlomiej.papiez@bdi.ox.ac.uk 3 NIHR Academic Clinical Fellow at Oxford University, Kadoorie Centre and Intensive Care Registrar, Thames Valley Deanery, UK guy.parsons@nhs.net * Corresponding author: Adam Mahdi, adam.mahdi@eng.ox.ac.uk Abstract. In this paper, we investigate the classification of cardiomegaly using multimodal data, combining imaging data from chest radiography with routinely collected Intensive Care Unit (ICU) data comprising vi- tal sign values, laboratory measurements, and admission metadata. In practice a clinician would assess for the presence of cardiomegaly using a synthesis of multiple sources of data, however, prior machine learning approaches tothis task have focused on chest radiographs only. We show that non-imaging ICU data can be used for cardiomegaly classification and propose a novel multimodal network trained simultaneously on both chest radiographs and ICU data. We compare the predictive power of both single-mode approaches with the joint network. We use a subset of data from the publicly available MIMIC-CXR and MIMIC-IV datasets, which contain both chest radiographs and non-imaging ICU data for the same patients. The approach from non-imaging ICU data alone achieves an AUC of 0.684 and the standard chest radiography approach an AUC of 0.840. Our joint model achieves an AUC of 0.880. We conclude that non-imaging ICU data have predictive value for cardiomegaly, and that combining chest radiographs with non-imaging ICU data has the poten- tial to improve model performance for the same subset of patients, with further work required to demonstrate a significant improvement. Keywords: deep learning · chest X-ray · cardiomegaly · multimodal approach. 1 Introduction Cardiomegaly is an abnormal enlargement of the heart usually indicating an underlying pathology warranting further investigation. In clinical practice, car- diomegaly can be detected visually by examining the size of the heart on a