Contents lists available at ScienceDirect Computers and Electronics in Agriculture journal homepage: www.elsevier.com/locate/compag Crop conditional Convolutional Neural Networks for massive multi-crop plant disease classification over cell phone acquired images taken on real field conditions Artzai Picon a, ⁎ , Maximiliam Seitz c,d , Aitor Alvarez-Gila a , Patrick Mohnke d , Amaia Ortiz-Barredo b , Jone Echazarra a a Computer Vision, TECNALIA, Parque Tecnolgico de Bizkaia, C/ Geldo. Edificio 700, E-48160 Derio, Bizkaia, Spain b NEIKER, Plant health Dp, Arkaute Agrifood Campus, E-01080 Vitoria-Gasteiz, Araba, Spain c Institute of Process Engineering in Plant Production, University of Hohenheim, 70599 Stuttgart, Germany d BASF SE, Speyererstrasse 2, 67117 Limburgerhof, Germany ARTICLE INFO Keywords: Convolutional neural network Deep learning Contextual meta-data Contextual meta-data conditional neural network Crop protection Multi-label classification Multi-crop classification Image processing Plant disease Early pest Disease identification Precision agriculture Phyto-pathology ABSTRACT Convolutional Neural Networks (CNN) have demonstrated their capabilities on the agronomical field, especially for plant visual symptoms assessment. As these models grow both in the number of training images and in the number of supported crops and diseases, there exist the dichotomy of (1) generating smaller models for specific crop or, (2) to generate a unique multi-crop model in a much more complex task (especially at early disease stages) but with the benefit of the entire multiple crop image dataset variability to enrich image feature de- scription learning. In this work we first introduce a challenging dataset of more than one hundred-thousand images taken by cell phone in real field wild conditions. This dataset contains almost equally distributed disease stages of seventeen diseases and five crops (wheat, barley, corn, rice and rape-seed) where several diseases can be present on the same picture. When applying existing state of the art deep neural network methods to validate the two hypothesised ap- proaches, we obtained a balanced accuracy ( = BAC 0.92) when generating the smaller crop specific models and a balanced accuracy ( = BAC 0.93) when generating a single multi-crop model. In this work, we propose three different CNN architectures that incorporate contextual non-image meta-data such as crop information onto an image based Convolutional Neural Network. This combines the advantages of simultaneously learning from the entire multi-crop dataset while reducing the complexity of the disease clas- sification tasks. The crop-conditional plant disease classification network that incorporates the contextual in- formation by concatenation at the embedding vector level obtains a balanced accuracy of 0.98 improving all previous methods and removing 71% of the miss-classifications of the former methods. 1. Introduction Plant pathogens; also causing fungal diseases; represent relevant biotic stress factors responsible for significant crop yield losses. Their damage potential is estimated among crops between 16 and 18% globally. The management of fungal diseases, which relays heavily on synthetic chemicals, can reduce pathogen caused potential yield losses by 32% (Oerke, 2006). To fully exploit this management potential, chemical crop protection treatments have to be applied based on in- festation situation and time, which also maps the requirements of in- tegrated crop protection. Therefore, continuous plant stock controls are required to identify disease symptoms in preferably early infestation stages to enable most efficient treatments. This is a time and cost in- tensive work (Kübler, 1994). In addition it can be assumed, that it is also challenging for common farmers as detailed knowledge of current pathogen species is necessary, especially for the visual identification of fungal disease symptoms at early infestation stages as similar stress symptoms are caused by different pathogens and abiotic factors (Oerke et al., 2010; Stafford, 2000). Over the last few years, extensive research has been done on RGB image-based plant disease classification methods. Since the late 90s, classical computer vision approaches have been widely used to address https://doi.org/10.1016/j.compag.2019.105093 Received 21 May 2019; Received in revised form 24 September 2019; Accepted 3 November 2019 ⁎ Corresponding author. E-mail address: artzai.picon@tecnalia.com (A. Picon). Computers and Electronics in Agriculture 167 (2019) 105093 Available online 19 November 2019 0168-1699/ © 2019 Elsevier B.V. All rights reserved. T