Search Based Repair of Deep Neural Networks Jeongju Sohn School of Computing, KAIST Daejon, Republic of Korea kasio555@kaist.ac.kr Sungmin Kang School of Computing, KAIST Daejon, Republic of Korea stuatlittle@kaist.ac.kr Shin Yoo School of Computing, KAIST Daejon, Republic of Korea shin.yoo@kaist.ac.kr ABSTRACT Deep Neural Networks (DNNs) are being adopted in various do- mains, including safety critical ones. The wide-spread adoption also calls for ways to guide the testing of their accuracy and robustness, for which various test adequacy criteria and input generation meth- ods have been recently introduced. In this paper, we explore the natural subsequent step: given an input that reveals unexpected behaviour in a trained DNN, we propose to repair the DNN us- ing input-output pairs as a specifcation. This paper introduces Arachne, a novel program repair technique for DNNs. Arachne frst performs sensitivity based fault localisation to limit the number of neural weights it has to modify. Subsequently, Arachne uses Particle Swarm Optimisation (PSO) to directly optimise the localised neural weights until the behaviour is corrected. An empirical study using three diferent benchmark datasets shows that Arachne can reduce the instances of the most frequent misclassifcation type committed by a pre-trained CIFAR-10 classifer by 27.5%, without any need for additional training data. Patches generated by Arachne tend to be more focused on the targeted misbehaviour than DNN retraining, which is more disruptive to non-targeted behaviour. The overall results suggest the feasibility of patching DNNs using Arachne until they can be retrained properly. KEYWORDS Automated Program Repair, Deep Neural Network 1 INTRODUCTION Deep Neural Networks (DNNs) have been rapidly being adopted in many application areas [19], ranging from image recognition [17, 31], speech recognition [11], machine translation [13, 30], to safety critical domains such as autonomous driving [2, 3] and medical imaging [21]. With the widening application areas, there has been a growing concern that these DNNs should be tested, both in isolation and as a part of larger systems, to ensure dependable performance. The need for testing resulted in two major classes of techniques. First, to evaluate sets of test inputs, many test adequacy criteria have recently been proposed [15, 24, 32]. Second, ways to synthesise new inputs by applying small perturbations (such as emulation of diferent lighting or weather conditions) to given inputs have been introduced [22, 38]. Newly synthesised inputs can not only improve the proposed test adequacy due to the increased input diversity, but also actually reveal unexpected behaviour of DNNs under test. Compared to the traditional software developed by human en- gineers, however, the stages after the detection of unexpected be- haviour remain relatively unexplored for DNNs. This is due to the major diference between the way DNNs and code are devel- oped: one is trained by algorithms based on training data, while the other is written by human engineers based on specifcations. Consequently, existing eforts to repair the unexpected behaviour mostly focus on how to efciently and efectively retrain the DNNs. For example, Ma et al. [23] try to identify the features that are the most responsible for the unexpected behaviour, and synthesise inputs for retraining based on their relevance to these features. Given that DNNs are initially trained by algorithms, retraining as repair can be considered as a natural solution. One weakness of retraining as a repair technique, however, is its stochastic nature. Unlike human debugging, there is no guarantee that, at the end of the retraining process, the unexpected behaviour will be removed while all correct behaviour are retained. The retraining may, or may not, correct the misbehaviour, and it may do so at the expense of other, initially correct, behaviour. Even when compared to sto- chastic Automated Program Repair (APR) techniques [10, 27, 37], retraining is less focused on the misbehaviour as the loss function usually focuses only on the overall accuracy improvement. We ar- gue that a need for focused repairs can be realistic in real world use cases: for example, users may prioritise higher accuracy for a specifc label of a DNN classifer in safety critical applications, even if it requires a trade-of with accuracy for lower priority labels. This paper introduces Arachne, a search-based automated pro- gram repair technique for DNNs. Instead of retraining, Arachne directly manipulates the neural weights and searches the space of possible DNNs, guided by a specifcally designed ftness function fol- lowing Generate and Validate APR techniques [10, 27, 37]. Arachne resembles APR techniques for traditional code in many aspects: it adopts a fault localisation technique, and uses both positive and negative inputs to retain correct behaviour and to generate a patch, respectively. Its internal representation of a patch is even simpler than the representation for code, as a set of neural weights can be represented as a vector of real numbers. Consequently, Arachne uses Particle Swarm Optimisation (PSO) [14, 35] as its search al- gorithm. During a ftness evaluation, Arachne updates the chosen neural weights of the DNN under repair with values from the PSO candidate solution, executes the inputs, and computes the ftness value based on the outcomes. We have empirically evaluated Arachne on three image classifca- tion benchmark datasets and three corresponding DNN classifers: Fashion MNIST [36], CIFAR-10 [16], and German Trafc Sign Recog- nition Benchmark (GTSRB) [29]. Initially, to study the impact that fault localisation and a particular choice of misbehaviour has on the success rate of the repair, we evaluate the feasibility of Arachne against faults that are artifcially induced via under-training of DNNs. Subsequently, we also show that Arachne can repair a fully trained DNN as newly incoming inputs reveal previously unknown unexpected behaviour. This adaptive repair process does not re- quire any additional data, apart from the new inputs that revealed the unexpected behaviour. The technical contributions of this paper are as follows: arXiv:1912.12463v1 [cs.LG] 28 Dec 2019