(IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 12, No. 8, 2021 736 | Page www.ijacsa.thesai.org The Effect of Adaptive Learning Rate on the Accuracy of Neural Networks Jennifer Jepkoech 1 *, David Muchangi Mugo 2 , Benson K. Kenduiywo 3 , Edna Chebet Too 4 University of Embu, P.O BOX 6 – 60100, Embu, Kenya 1, 2 Jomo Kenyatta University of Science and Technology, P.O. Box 62 000 – 00200, Nairobi, Kenya 3 Chuka University, P.O BOX 109-60400, Chuka, Kenya 4 Abstract—Learning rates in gradient descent algorithms have significant effects especially on the accuracy of a Capsule Neural Network (CNN). Choosing an appropriate learning rate is still an issue to date. Many developers still have a problem in selecting a learning rate for CNN leading to low accuracies in classification. This gap motivated this study to assess the effect of learning rate on the accuracy of a developed (CNN). There are no predefined learning rates in CNN and therefore it is hard for researchers to know what learning rate will give good results. This work, therefore, focused on assessing the effect of learning rate on the accuracy of a CNN by using different learning rates and observing the best performance. The contribution of this work is to give an appropriate learning rate for CNNs to improve accuracy during classification. This work has assessed the effect of different learning rates and came up with the most appropriate learning rate for CNN plant leaf disease classification. Part of the images used in this work was from the PlantVillage dataset while others were from the Nepal database. The images were pre-processed then subjected to the original CNN model for classification. When the learning rate was 0.0001, the best performance was 99.4% on testing and 100% on training. When the learning rate was 0.00001, the highest performance was 97% on testing and 99.9% on training. The lowest performance observed was 81% accuracy on testing and 99% on training when the learning rate was 0.001. This work observed that CNN was able to achieve the highest accuracy with a learning rate of 0.0001. The best Convolutional Neural Network accuracy observed was 98% on testing and 100% on training when the learning rate was 0.0001. Keywords—CNN; ConvNet; learning rate; gradient descent I. INTRODUCTION Deep learning has been used over time for plant leaf disease detection and classification. Some of the researchers who have used deep learning include [29,30,31,32,33,34,35,36,37,38,39]. Capsule neural networks (CNN) are a regularly used neural network structure that has significant effects on deep learning, particularly in computer vision studies. CNN's have attained superhuman levels in different computer task categories, for example, object detection, classification, incidence segmentation, semantic segmentation, and parsing. The learning rate is viewed as the absolute hyper-parameter to tune and remarkably influence model training with gradient descent algorithms [1, 2]. Studies have come up with several learning rate techniques including inverse square root decay, linear decay, exponential decay, and cosine decay [3, 4]. These learning rates have varying procedures that are based on an optimization problem. One of the limitations involves the selection of a suitable learning rate for a given application. Practically, researchers have adopted a trial-and-error method for various learning rates alongside diverse hyper- parameters, which is a very tedious process [5]. This paper utilizes a regulator that adapts three learning rate schedules of 0.001, 0.0001, and 0.00001. Existing learning rate schedules adopt predefined parametric learning rate changes, which are fixed regardless of prevailing training dynamics. The predefined parametric learning rate changes have a limited flexibility and may not be improved for the training dynamics of various high dimensional and non-convex advancement issues [6]. The context for this work provides adaptive meta- learned learning rates that dynamically adjust to current training. The process of training a neural network using an algorithm, for example, the error back-propagation [1, 2, 3, 4] is normally time-consuming, especially when working on complex problems. These types of algorithms naturally have a learning rate parameter that controls the extents by which the weights can change based on an observed error that was noted on the training set. Learning rate schedules can dramatically affect the accuracy of the results. Therefore, the process of choosing learning rates using training algorithms can be problematic especially when there is no guiding value for specific tasks. Various algorithms have been used to tune the learning rate parameters [6, 7, and 8], yet such strategies generally have failed to concentrate on refining the resulting accuracy. Most of the experts in neural networks use the highest learning rates that allow merging. However, when learning rates are set too high, it causes unwanted divergent behavior in the loss function. Hence when the highest learning rates are applied to complex and large problems, there is a negative effect on the training process and accuracy. On the other hand, when the learning rate is set too low, the training progress will be very slow because very small updates are made to the weights of the work [9]. So there is a need to balance and there is no better way to do that other than to test several learning rates and observe their performances. This work adopts the use of online training instead of batch training. This is because batch training needs more time compared to online training with no corresponding improvement inaccuracy [5]. This paper aims to investigate the effect of learning rate on the accuracy of CNN's as applied in plant disease detection. Since Tensor flow recommends a learning rate of 0.001, this works started by using that learning rate and observed a low percentage of 84% *Corresponding Author