Efficient Evolution of Asymetric Recurrent Neural Networks Using a Two-dimensional Representation Jo˜ ao Carlos Figueira Pujol and Riccardo Poli School of Computer Science The University of Birmingham Birmingham B15 2TT, UK E-MAIL: J.Pujol,R.Poli @cs.bham.ac.uk Technical Report CSRP-97-31 December 12, 1997 Abstract Recurrent neural networks are particularly useful for processing time sequences and sim- ulating dynamical systems. However, methods for building recurrent architectures have been hindered by the fact that available training algorithms are considerably more complex than those for feedforward networks. In this paper, we present a new method to build recurrent neural networks based on evolutionary computation, which combines a linear chromosome with a two- dimensional representation inspired by Parallel Distributed Genetic Programming (a form of genetic programming for the evolution of graph-like programs) to evolve the architecture and the weights simultaneously. Our method can evolve general asymetric recurrent architectures as well as specialized recurrent architectures. This paper describes the method and reports on results of its application. 1 Introduction The ability to store temporal information makes recurrent neural networks (RNNs) ideal for time sequence processing and dynamical sytems simulation. However, building RNNs is far more difficult than building feedforward neural networks. Constructive and destructive algorithms, which combine training with structural modifications which change the complexity of the network, have been proposed for the design of recurrent networks [1, 2], but their application has been hindered by the fact that training algorithms for recurrent architectures are considerably more complex than their feedforward counterparts [3, 4, 5, 6, 7]. Recently, new promising approaches based on evolutionary algorithms, such as evolutionary programming (EP) [8] and genetic algorithms (GAs) [9], have been applied to the development of artificial neural networks (ANNs). Approaches based on EP operate on the neural network directly, and rely exclusively on mutation [10, 11, 12, 13] or combine mutation with training [14]. Methods based on genetic algorithms usually represent the structure and the weights of ANNs as a string of 1