Proceedings of the ASME 2019 International Mechanical Engineering Congress and Exposition IMECE2019 November 11-14, 2019, Salt Lake City, UT, USA IMECE2019-11077 Robust weed recognition through color based image segmentation and convolution neural network based classification M. Nazmuzzaman Khan 1 Mechanical and Energy Engineering Department Indiana University-Purdue University Indianapolis Indianapolis, Indiana Sohel Anwar Mechanical and Energy Engineering Department Indiana University-Purdue University Indianapolis Indianapolis, Indiana ABSTRACT Current image classification techniques for weed detection (classic vision techniques and deep-neural net) provide encouraging results under controlled environment. But most of the algorithms are not robust enough for real-world application. Different lighting conditions and shadows directly impact vegetation color. Varying outdoor lighting conditions create different colors, noise levels, contrast and brightness. High component of illumination causes sensor (industrial camera) saturation. As a result, threshold-based classification algorithms usually fail. To overcome this shortfall, we used visible spectral- index based segmentation to segment the weeds from background. Mean, variance, kurtosis, and skewness are calculated for each input image and image quality (good or bad) is determined. Bad quality image is converted to good-quality image using contrast limited adaptive histogram equalization (CLAHE) before segmentation. A convolution neural network (CNN) based classifier is then trained to classify three different types of weed (Ragweed, Pigweed and Cocklebur) common in a corn field. The main objective of this work is to construct a robust classifier, capable of classifying between three weed species in the presence of occlusion, noise, illumination variation, and motion blurring. Proposed histogram statistics-based image enhancement process solved weed mis-segmentation under extreme lighting condition. CNN based classifier shows accurate, robust classification under low-to-mid level motion blurring and various levels of noise. Keywords: Image-segmentation, image-classification, precision-farming. 1 Contact author: mdkhan@iu.edu 1. INTRODUCTION With rapidly increasing global population, the demand for higher crop yield is also increasing rapidly. Weeds are one of the major culprits behind lower crop yield. They grow randomly in field and compete with crops for water, nutrients and sunlight. To eliminate weeds and reduce uncontrolled spray of herbicides, real-time detection of weeds with high accuracy using low cost sensors is needed. Among all the sensing techniques (machine vision, spectroscopy, fluorescence, LiDAR and ultrasonics) [1], this work will focus on RGB camera based machine vision. CNN based weed classification gained popularity in recent times due to their generalization capability and hardware acceleration. Dyrmann et al. [2] achieved 86.2% accuracy with 22 species and 10413 images with a CNN network build from scratch. McCool et al. [3] achieved grater than 90% accuracy with 1.07-1.82 frame per second using a combination of lightweight CNN models. But classification accuracy of CNN models are highly dependent on motion blur, noise, and contrast [4]. Any work on the effect of noise, blur and contrast for real- time weed detection, and steps to mitigate them have not been well investigated. Any real-time robust classification method is based on redundant system. In this case, we deploy visible spectral-index based image segmentation (weed segment from background) as fail-safe when CNN performance deteriorates. Both the segmented image of the weed and classified image of the weed can be feed into a decision-making system. Based on the performance of the CNN, the decision-making system can decide which herbicide to spray. The goal is to not miss any weed (or not spray wrong herbicide) even if CNN misclassified. The block diagram is showed in Fig 1. ExG [5], ExGR [6] and CIVE [7] are color based segmentation methods and they all perform badly when light is high or low. Guijarro et al. showed that _______________________________________________ This is the author's manuscript of the article published in final edited form as: Khan, M. N., & Anwar, S. (2020, January 21). Robust Weed Recognition Through Color Based Image Segmentation and Convolution Neural Network Based Classification. ASME 2019 International Mechanical Engineering Congress and Exposition. https://doi.org/10.1115/IMECE2019-11077