Biomedical Image Segmentation with Modified U-Net Umut Tatli 1 , Cafer Budak 2* 1 Department of Engineering, Biomedical Engineering, Dicle University, Diyarbakır 21000, Turkey 2 Department of Engineering, Electrical and Electronics Engineering, Dicle University, Diyarbakır 21000, Turkey Corresponding Author Email: cafer.budak@dicle.edu.tr https://doi.org/10.18280/ts.400211 ABSTRACT Received: 16 July 2022 Accepted: 8 March 2023 Image segmentation is an important field in image processing and computer vision, particularly in the development of methods to assist experts in the biomedical and medical fields. It plays a vital role in saving time and costs. One of the most successful and significant methods in image segmentation using deep learning is the U-Net model. In this paper, we propose U-Net11, a novel variant of U-Net that uses 11 convolutional layers and introduces some modifications to improve the segmentation performance. The classical U-Net model was developed and tested on three different datasets, outperforming the traditional U-Net approach. The U-Net11 model was evaluated for breast cancer segmentation, lung segmentation from CT images, and the nuclei segmentation dataset from the Data Science Bowl 2018 competition. These datasets are valuable due to their varying image quantities and the varying difficulty levels in segmentation tasks. The modified U-Net model has achieved Dice Similarity Coefficient scores of 69.09% on the breast cancer dataset, 95.02% on the lung segmentation dataset and 81.10% on the nuclei segmentation dataset, exceeding the performance of the classical U-Net model by 5%, 2% and 4% respectively. This difference in success rates is particularly significant for critical segmentation datasets. deep learning, Keywords: image segmentation, biomedical image, U-Net 1. INTRODUCTION In recent years, image segmentation has become a widely researched topic. It serves as an essential element in numerous visual applications. Image Segmentation is the process of classifying each pixel in an image as belonging to a certain class; therefore, it is considered a pixel classification method. The main goal of image segmentation is to divide an image into several meaningful and analyzable segments with similar or exactly the same features. Two types of segmentation methods exist: Semantic segmentation and Instance segmentation methods. Instance Segmentation aims to estimate class labels and pixel-level sample masks to accommodate the varying number of samples that appear in each image [1]. In other words, it classifies the sought-after objects individually. On the other hand, semantic segmentation aims to assign a categorical label to each pixel in an image, but this label is not different from the others [2]. In other words, if there are five different color cars in an image, instance segmentation determines five different color labels, while semantic segmentation determines one label. Semantic segmentation is used in areas related to health and medicine, while instance segmentation is mainly used in areas related to daily life. Today, semantic segmentation is frequently used in the biomedical field. In the biomedical field, cell segmentation is used for nucleus segmentation, cancer segmentation, tumor segmentation, or organ segmentation. Image segmentation has been used to detect cancerous tissue or the cells in this tissue in recent years. There exist various methods for image segmentation in the deep learning area. Mask R-CNN was proposed by He et al. in 2020 [3], InstanceCut with the Model was introduced by Kirillov et al. in 2017 [4], FCN was developed by Long et al. in 2015 [5], R-CNN was presented by Girshick et al. in 2014 [6], U-Net was designed by Ronneberger et al. in 2015 [7], Deeplab was implemented by Chen et al. in 2018 [8], INet was created by Weng and Zhu in 2021 [9], Superpixels and Clustering Methods were applied by Mendi and Budak in histopathological images in 2021 [10] and GCN was used by Peng et al. in 2017 [11] to provide semantic segmentations by using different deep learning algorithms in image segmentation. The U-Net Model is one of the most commonly used image segmentation methods with deep learning. Some of the studies that used this model are: separating concrete cracks from concrete [12], tumor detection from 3D brain images [13], cell segmentation for 2D and 3D images [14], glaucoma detection [15], vessel detection for Cerebrovascular disease [16], human placenta image detection [17], uterine region estimation from MR images [18], and lung lobe segmentation from 3D chest tomography [19]. In addition, the U-Net model has inspired many models. Zhou et al. proposed U-Net++ in 2018 [20], Diakogiannis et al. introduced ResUNet-a in 2020 [21], Zhang et al. developed ResUnet in 2018 [22], Alom et al. presented R2U-Net in 2018 [23], Oktay et al. designed Attention U-Net in 2018 [24], Ibtehaz et al. implemented MultiResUNet in 2020 [25], Zhuang created LadderNet in 2018 [26], Iglovikov et al. applied Ternausnet in 2018 [27], Stoller et al. used Wave-U- Net in 2018 [28], Meseguer-Brocal and Peeters exploited CU- Net in 2019 [29], Ma et al. leveraged Docunet in 2018 [30], Isensee et al. built nnU-Net in 2021 [31], Olimov et al. constructed FU-Net in 2021 [32], Wang et al. employed Non- Traitement du Signal Vol. 40, No. 2, April, 2023, pp. 523-531 Journal homepage: http://iieta.org/journals/ts 523