Towards practical object detection for weed spraying in precision agriculture* Adrian Salazar Gomez 1 Madeleine Darbyshire 2 Junfeng Gao 1 Elizabeth I Sklar 1 Simon Parsons 2 Abstract—The evolution of smaller, faster processors and cheaper digital storage mechanisms across the last 4-5 decades has vastly increased the opportunity to integrate intelligent technologies in a wide range of practical environments to address a broad spectrum of tasks. One exciting application domain for such technologies is precision agriculture, where the ability to integrate on-board machine vision with data-driven actuation means that farmers can make decisions about crop care and harvesting at the level of the individual plant rather than the whole field. This makes sense both economically and environmentally. However, the key driver for this capability is fast and robust machine vision—typically driven by machine learning (ML) solutions and dependent on accurate modelling. One critical challenge is that the bulk of ML-based vision research considers only metrics that evaluate the accuracy of object detection and do not assess practical factors. This paper introduces three metrics that highlight different aspects relevant for real-world deployment of precision weeding and demonstrates their utility through experimental results. Index Terms— precision agriculture, automated weeding, computer vision, object detection I. INTRODUCTION The current agricultural approach to weeding in arable crops is to spray an entire field with a selective herbicide that kills the weeds, but does not harm the crops. Such a broadcast spraying approach is easy to deliver—requiring only a sprayer to dispense the herbicide—but wasteful, since much of the area sprayed does not contain weeds, being ei- ther bare earth or crops (see Figure 3). Precision agriculture aims to use ideas from artificial intelligence (AI) and robotics to create agricultural solutions that can be delivered at the level of individual plants, instead of entire fields. An im- portant application within precision agriculture is automated weeding, which aims to detect and target individual weeds, resulting in the precise delivery of herbicide [1] on the weeds while avoiding wastage, using a laser [2], or a mechanical tool [1]. In the work presented here, we are concerned with precise delivery of herbicide in a real-world farm setting. Now, it is clear that any approach to automated weeding needs to identify the weeds, and there have been numerous *This work was partially supported by Lincoln Agri-Robotics as part of the Expanding Excellence in England (E3) Programme and by Ceres under the “AI Unleashed” project. 1 ASG, JG and ES are with the Lincoln Institute of Agri-food Technology, University of Lincoln, UK. {asalazargomez, jugao, esklar}@lincoln.ac.uk 2 MD and SP are with the Lincoln Centre for Autonomous Systems, University of Lincoln, UK. 25696989@students.lincoln.ac.uk, sparsons@lincoln.ac.uk *This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible attempts to use computer vision to do this (see Section II), many using AI methods based on machine learning (ML). However, most of this work has treated automated weeding as purely a computer vision problem: datasets are collected and annotated, object identifiers are trained, and the resulting models are optimised with respect to accuracy and/or mAP for classifying crops and weeds. We argue that while these metrics are important, there are additional measures that must be considered in assessing the feasibility of ML models for use in precision spraying. Here we focus on three of these. First, the weed coverage rate (WCR) is more important than conventional ML-based metrics. WCR identifies the proportion of the weeds that could be targeted by a sprayer triggered by the model. WCR depends on the accuracy of the model, but also takes into account the resolution of a spray. Second, we need to understand the area sprayed, that is the area covered in herbicide by the precision sprayer in order to understand the saving in herbicide compared with current practice. Third, it is important to know whether the approach is practical—because you can’t just pile more GPUs on-board a tractor operating in an open field, where issues like compute power and energy consumption come into play. Real-world automated weeding (see Figure 1) will need to process many images very rapidly, so here we use inference speed as a proxy for comparing the practicality of different ML approaches. The primary contribution of this paper is to assess the feasibility of precision spraying by comparing a selection of standard ML-based vision methods applied to weed detection using the additional metrics we have introduced (WCR, area sprayed and inference speed). Section II briefly describes the state-of-the-art in ML-based vision approaches applied to agriculture. Section III explains our methodology and Section IV details the experiments we conducted which form the basis of our comparison. Section V analyses these results and Section VI closes with conclusions. II. RELATED WORK Initial approaches to weed detection used machine learning algorithms with handcrafted features based on the differences in colour, shape or texture. [3] extracted local binary patterns with support vector machines for plant discrimination. This method generally requires a relatively small dataset for model development. However, it might fail to generalise under different field conditions. The deep learning-based methods for computer vision increasingly gain more pop- ularity, offering an end-to-end weed detection solution that deals with the issue of generalisation. An adjusted YOLOv3 arXiv:2109.11048v1 [cs.CV] 22 Sep 2021