Intelligent Automation & Soft Computing DOI:10.32604/iasc.2022.xxxxxx Type: xxx Detection of Diabetic Retinopathy Using Custom CNN to Segment the Lesions Saleh Albahli 1,2, †,* and Ghulam Nabi Ahmad Hassan Yar 3,† 1 Department of Information Technology, College of Computer, Qassim University, Buraydah, Saudi Arabia 2 Department of Computer Science, Kent State University, Kent, OH, USA (salbahli@kent.edu) 3 Department of Electrical and Computer Engineering, Air University, Islamabad, Pakistan * Corresponding Author: Saleh Albahli. Email: salbahli@qu.edu.sa † These authors have contributed equally Received: XX Month 2021; Accepted: XX Month 2021 Abstract: Diabetic retinopathy is an eye deficiency that affects the retina as a result of the patient having Diabetes Mellitus caused by high sugar levels. This condition causes the blood vessels that nourish the retina to swell and become distorted and eventually become blocked. In recent times, images have played a vital role in using convolutional neural networks to automatically detect medical conditions, retinopathy takes this to another level because there is need not for just a system that could determine is a patient has retinopathy, but also a system that could tell the severity of the procession and if it would eventually lead to macular edema. In this paper, we designed three deep learning models that would detect the severity of diabetic retinopathy from images of the retina and also determine if it would lead to macular edema. Since our dataset was a small one, we employed three techniques for generating images from the ones we have, the techniques are Brightness, color and, contrast (BCC) enhancing, Color jitters (CJ), and Contrast Limited Adaptive Histogram Equalization (CLAHE). After the dataset was ready, we used it to train the ResNet50, VGG16, and VGG19 models both for determining the severity of the retinopathy and also the chances of macular edema. After validation, the models yielded very reasonable results. Keywords: Convolutional Neural Networks (CNN); Deep Learning; Diabetic Retinopathy; Diabetes Mellitus; ResNet50; VGG16; VGG19 1. Introduction Retinopathy is a disease of the retina. It occurs due to several reasons in a diabetic person. These reasons include hemorrhage, extrudes, and microaneurysms. Macular edema is fluid build-up in the macula (area in the center of the retina). This causes the macula to swell and hence causes blurry and distorted vision. This fluid buildup is caused by retinopathy, when hemorrhage occurs it also causes the fats and other fluids to leak along with blood. This is how the segmentation and disease grading task of the Indian Diabetic Retinopathy Image Dataset (IDRiD) dataset [1] given are linked. We will first find the severity of retinopathy and macular edema caused by hemorrhage, extrudes, and microaneurysms. Then we will segment areas affected by hemorrhage, extrudes, and microaneurysms. Fig. 1 shows the lesions in the eye and their respective masks [1]. Diabetic retinopathy (DR) has four stages of progression, which goes from having no DR to proliferative DR. The first stage is type 0, where there are no abnormalities observed meaning there is no DR. The second stage is type 1 called the mild non-proliferative retinopathy and it is characterized by observing micro-aneurysms, it is considered the earliest stage. The next stage, type 2 is moderate no