A Hierarchical Classification System for the Detection of Covid-19 from Chest X-Ray Images Meghna P Ayyar Jenny Benois-Pineau Akka Zemmari Univ. Bordeaux, CNRS, LaBRI, UMR 5800, F-33400 Talence, France meghna-parameswaran.ayyar@etu.u-bordeaux.fr {jenny.benois-pineau,akka.zemmari}@u-bordeaux.fr Abstract With the ever-increasing cases of the Covid-19 pan- demic, it is important to leverage deep learning methods to create tools that can aid in relieving the pressure that is put on the limited resources in most developing countries. In this work, we propose a hierarchical classification system for the classification of Covid-19 from Chest X-Ray (CXR) images following a recent proposal of massive use of this modality instead of CT. The system composed of multiple bi- nary classifiers outperforms a tailor-made multi-class clas- sifier COVID-Net. We also show that using well-known es- tablished deep learning frameworks combined with a global attention mechanism outperforms the baseline COVID-Net specifically designed for the classification of Covid-19 from CXR images. Our method shows approximately a 4% im- provement in the sensitivity to Covid-19 detection from 91% of COVID-Net to 96%. Using popular networks with the possibility of cross-domain transfer learning ensures that the designing and training times are reduced. Furthermore, well-established frameworks can be faster adapted into an application in clinical practice. 1. Introduction The novel COVID-19 or SARS-Cov-2 is an infectious disease that has been declared as a pandemic by WHO in March 2020 [36]. First reported in Wuhan, China at the end of 2019 [12], it has had devastating effects on human life and the economy. Many nations are still combating its pro- liferation and are facing a shortage of resources. The cru- cial step to control and stop Covid-19 is to detect infected patients effectively and impose immediate isolation. Currently, Reverse Transcription Polymerase Chain Re- action (RT-PCR) or gene sequencing for respiratory or blood specimens are the standard screening methods for COVID-19 [30]. However, these methods have a long turnaround time and are not sufficient when there are expo- nential increases in daily cases. The effectiveness of chest radiography imaging (X-Ray, Computed Tomography (CT) imaging) for the detection of infection[34, 15, 23] makes it a suitable alternative in the screening process and for the design of adapted therapy. A Chest X-Ray (CXR) image affected by COVID- 19, shows patchy or diffuse reticular-nodular opacities and consolidation, with basal, peripheral and bilateral predominance[8]. The major abnormalities observed in CT images are ground-glass opacity, consolidation and inter- lobular septal thickening in both lungs[32]. Fig.1 shows a sample CXR image that has been annotated by an expert ra- diologist to show the primary regions for diagnosis making. Though CT is a more precise modality for decision mak- ing by medical experts[25], CXR images are useful, specifi- cally in low-income countries as they are cheaper and more widely available, and XR examinations of patients can be done with a portable equipment. Furthermore, patients with acute respiratory syndrome are difficult to manipulate to be placed into CT scanners and is easier with XR equipment. Finally, CXR images enable rapid triaging as they can be completed much faster than a CT scan. With a high vol- ume of patients, this is essential to relive pressure on the available resources. In CXR, Covid-19 syndrome and atypical pneumonia ap- pear similar and experts cannot always make distinction just by observing the CXR image[25]. Hence the the automatic classification on images with Deep Neural Networks (DNN) is necessary not only for massive screening, but also for highlighting specific patterns on this modality. Recent medical studies [21, 22, 29] amongst others have attempted to develop specific Deep Neural architectures for the automatic detection of COVID-19 on CXR images and some open datasets have become available for research pur- poses [5, 17, 7]. The main contribution of our work are as follows: • We define a set of hierarchical rules to create a multi- level classification pipeline for the three classes - 519