EVOLVING NEURAL NETWORKS THAT SUFFER MINIMAL CATASTROPHIC FORGETTING TEBOGO SEIPONE JOHN A. BULLINARIA School of Computer Science, The University of Birmingham Birmingham, B15 2TT, UK {t.seipone, j.a.bullinaria}@cs.bham.ac.uk Catastrophic forgetting is a well-known failing of many neural network systems whereby training on new patterns causes them to forget previously learned patterns. Humans have evolved mechanisms to minimize this problem, and in this paper we present our preliminary attempts to use simulated evolution to generate neural networks that suffer significantly less from catastrophic forgetting than traditionally formulated networks. 1. Introduction Virtually all natural systems gradually forget what they have learned previously, particularly when they learn new information. However, with traditional artificial neural networks, the forgetting is much more catastrophic, and this proves to be a serious limitation for them (McCloskey & Cohen, 1989; Ratcliff, 1990). Some relatively complex systems, involving the interleaving of pseudo- patterns, and/or dual network architectures, have already been shown to deal with this problem quite successfully (French, 1999). However, we wish to explore the possibility of avoiding the problem within much simpler systems. Natural neural networks, such as human brains, have presumably evolved by natural selection to minimize the forgetting, and the aim of this paper is to present a preliminary investigation into using simulated evolution to see how far we can minimize the problem in artificial neural networks. The strategy employed starts by measuring how well traditional networks remember sets of input-output mappings after being trained on new items, and then explores systematically whether these can be evolved into better performing networks. 2. Evolving Neural Networks We simulate evolution by maintaining a population of individual neural networks, each specified by a number of ‘innate’ parameters, and using an appropriate fitness measure to determine which to discard and which to use to create the next generation by genetic cross-over and random mutation. Repeating this process should result in increasingly fit populations.