5th World Congress on Civil, Structural, and Environmental Engineering (CSEE'20) Lisbon, Portugal Virtual Conference – October 2020 Paper No. ICEPTP 140 DOI: 10.11159/iceptp20.140 ICEPTP 140-1 Identification and Instance Segmentation of Oil Spills Using Deep Neural Networks Zahra Ghorbani 1 , Amir H. Behzadan 1 1 Texas A&M University 3137 TAMU, College Station, TX, USA zahraghorabani@tamu.edu; abehzadan@tamu.edu Abstract - Oil spills impact natural and built environments, people and communities, food chain, and wildlife, and the road to full recovery is often long and costly to businesses, contractors, communities, and local governments. Existing oil spill detection methods including in-situ measurements and remote sensing primarily depend on involving skilled personnel in data collection, processing, and analysis, which could be expensive, slow, and subjective (influenced by prior experience, and judgment of problem parameters or solution space). In addition, oil pipelines and platforms can be located in remote and harsh areas, making it difficult and even hazardous for engineers to conduct timely inspections. Applying artificial intelligence (AI) can streamline this process and create more objective measures of oil spill and leakage detection. In this research, deep learning models, namely VGG-16 and mask R-CNN (mask region- based convolutional neural network) are employed to identify and locate oil spills. These models represent state-of-the-art object recognition algorithms in computer vision. Red-green-blue (RGB) training images are collected using semi-supervised learning (i.e., keyword search) from the web. This initial visual dataset consists of a diverse set of photos taken by unmanned aerial vehicles (UAVs) or first-person cameras from previous oil spill accidents. The methodology consists of model training and validation, image classification, object detection, and semantic segmentation. The VGG16 model is used for image classification (to predict the existence of oil spill in an image) and yields an accuracy of ~93%. The mask R-CNN model is used for instance segmentation (to detect oil spills and marking their boundaries at pixel-level) and yields average precision and recall of 61% and 70%, respectively. Results can create opportunities for advancing the current practice of integrating AI and data analytics into downstream and upstream operations in the oil and gas industry, as well as enabling non-intrusive techniques for detection of environmental pollutants. Keywords: Deep neural networks; oil spills; classification; instance segmentation; drones. 1. Introduction Recent advancements in data sensing and artificial intelligence (AI) have created new opportunities to tackle challenging problems in environmental monitoring such as air, solid waste, and wastewater pollution [1]. In the area of air pollution, past work has developed artificial neural networks (ANNs) to predict daily or hourly values of critical pollutants such as nitrogen dioxide (NO2), carbon monoxide (CO), and Ozone (O3) in the atmosphere [2]. In solid waste management, researchers have investigated waste generation and waste composition generation, and for instance, proposed AI algorithms based on support vector machine (SVM), k-nearest neighbour (KNN), ANN, and adaptive neuro-fuzzy inference system (ANFIS) to forecast waste generation in Queensland, Australia [3]. Other work includes the use of genetic algorithms to optimize the type of vehicle and length of waste collection route, and a fuzzy logic to represent customer satisfaction [4]. ANNs and fuzzy logic models have been also used to predict leachate penetration into groundwater and assess its environmental impacts [5]. In wastewater and water pollution control, researchers have used AI models to estimate water treatment processes and control pollutant flows. Multivariate adaptive regression splines (MARS), M5 model tree (M5Tree), and least square support vector machine (LSSVM) were used to model river water pollution and predict monthly chemical oxygen demand (COD) [6]. In another study, back propagation neural network models (BPNNs) and radial basis function neural network (RBFNN) were used to forecast water quality index (WQI) based on variables such as pH, dissolved oxygen (DO), total suspended solids (TSS), biological oxygen demand (BOD), and COD [7]. Despite previous work, there is still a dearth of research in using AI for monitoring oil spills, as a major source of environmental pollution. Oil spills can negatively affect plant growth [8] and soil nutrient levels [9], and lead to soil infertility [10] and contamination [11]. With increasing global oil consumption [12, 13] especially in the developing world, oil pollution is also expected to be on the rise [14]. Traditional in-situ methods of detecting oil pollution such as pressure-point-analysis