Liver Lesion Localization using Deep Convolutional Neural Networks Sheun Aluko Department of Medicine Stanford University oluwa@stanford.edu Surya Narayanan Hari Department of Management Science and Engineering surya21@stanford.edu Vadim Piccini Yakubenko Department of Statistics Stanford Univeristy vadimp@stanford.edu Abstract Within the field of medicine, automated annotation of Computed Tomography (CT) scans is poised to revolutionize the practice of radiology and to improve patient care through increased efficiency and accuracy of diagnosis. In this work, we develop a deep learning algorithm designed for the automated detection of liver lesions on CT scans. We explore three distinct neural network architectures for the object detection task, with increasing complexity respectively. These consist of a baseline convolutional network, a transfer learning approach which incorporates VGG-16 features into the baseline model, and finally the Faster R-CNN network. We find that Faster R-CNN greatly outperforms the baseline and can achieve a sensitivity of 0.5 with one average false positive per image even when trained and evaluated on noisy labels. 1 Introduction Computed Tomography (CT) scans are a widely used imaging modality in global healthcare systems owing to their fast acquisition time and high resolution 3D rendering of the human body. These scans are read by radiologists who spend hours each day visually inspecting and interpreting the images, to provide a report that will be used to guide clinical care. Initially, lesion detectors will likely serve as an aid to the radiologist to highlight potential lesions that they may have missed. This will augment the radiologists ability to process large volumes of images and scale to the volumes required for our growing healthcare system, without a simultaneous decrease in accuracy. This work specifically explores the automated detection of liver lesions in CT images. The liver performs essential filtration of blood from the digestive tract as well as detoxifies and metabolizes harmful chemicals in the body. Because the liver is the largest internal organ by mass and in a healthy state appears relatively homogenous on CT Imaging, it is an appropriate baseline target for lesion detection. Lesions in the liver consist of benign cysts, collections of blood (hematoma), and hepatocellular carcinoma among other pathologies. Our detector does not aim to classify these lesions, but instead to localize any abnormality that is present. Specifically, we explore three neural network architectures for the task of lesion detection. The input to each model consists of 3 slices of CT image data of the liver (the slice with the lesion of interest and its two adjacent slices which are provided for volumetric context). The output of the model consists of 4 bounding box coordinates, which attempt to localize the lesion. The first model architecture consists of a baseline model with 4 convolutional and one fully connected layer. Next, we develop a model that computes VGG-16 features from the input and concatenates these with the input before it is passed into the baseline convolutional model. Finally, we implement the Faster R-CNN architecture. We discuss the performance of these models and compare them to a reference publication which utilizes the same dataset. CS230: Deep Learning, Fall 2019, Stanford University, CA. (LateX template borrowed from NIPS 2017.)