Neural Networks 20 (2007) 1095–1108 www.elsevier.com/locate/neunet Analysis and test of efficient methods for building recursive deterministic perceptron neural networks David A. Elizondo a,∗ , Ralph Birkenhead a , Mario G ´ ongora a , Eric Taillard b , Patrick Luyima a a Centre for Computational Intelligence, School of Computing, Faculty of Computing Sciences and Engineering, De Montfort University, Leicester, UK b EIVD, University of Applied Sciences of Western Switzerland, Route de Cheseaux 1, Case postale, CH-1401 Yverdon, Switzerland Received 20 June 2006; accepted 12 July 2007 Abstract The Recursive Deterministic Perceptron (RDP) feed-forward multilayer neural network is a generalisation of the single layer perceptron topology. This model is capable of solving any two-class classification problem as opposed to the single layer perceptron which can only solve classification problems dealing with linearly separable sets. For all classification problems, the construction of an RDP is done automatically and convergence is always guaranteed. Three methods for constructing RDP neural networks exist: Batch, Incremental, and Modular. The Batch method has been extensively tested and it has been shown to produce results comparable with those obtained with other neural network methods such as Back Propagation, Cascade Correlation, Rulex, and Ruleneg. However, no testing has been done before on the Incremental and Modular methods. Contrary to the Batch method, the complexity of these two methods is not NP-Complete. For the first time, a study on the three methods is presented. This study will allow the highlighting of the main advantages and disadvantages of each of these methods by comparing the results obtained while building RDP neural networks with the three methods in terms of the convergence time, the level of generalisation, and the topology size. The networks were trained and tested using the following standard benchmark classification datasets: IRIS, SOYBEAN, and Wisconsin Breast Cancer. The results obtained show the effectiveness of the Incremental and the Modular methods which are as good as that of the NP-Complete Batch method but with a much lower complexity level. The results obtained with the RDP are comparable to those obtained with the backpropagation and the Cascade Correlation algorithms. c  2007 Elsevier Ltd. All rights reserved. Keywords: Recursive deterministic perceptron; Batch learning; Incremental learning; Modular learning; Performance sensitivity analysis; Convergence time; Generalisation; Topology 1. Introduction The single layer perceptron topology (SLPT), introduced by Rosenblatt (1962), was one of the first neural network models shown to be able to learn how to classify patterns. However, Minsky and Papert (1969) showed that this topology is only capable of learning linearly separable patterns. This is a big limitation since most classification problems are non-linearly separable (NLS). The Recursive Deterministic Perceptron feed- forward neural network (Elizondo, 1997; Tajine & Elizondo, 1997) is a multilayer generalisation of this topology, which provides a solution to any two-class classification problem ∗ Corresponding author. E-mail addresses: elizondo@dmu.ac.uk (D.A. Elizondo), rab@dmu.ac.uk (R. Birkenhead), mgongora@dmu.ac.uk (M. G ´ ongora), Eric.Taillard@heig-vd.ch (E. Taillard), pluyima@dmu.ac.uk (P. Luyima). (even if the two classes are NLS). The RDP neural network is guaranteed to converge, does not need any parameters from the user, does not suffer from catastrophic interference, and provides transparent extraction of knowledge as a set of rules (Elizondo & Gongora, 2005). These rules can be generated, using a computational geometry approach, as a finite union of open polyhedral sets. Although this study will focus on two- class classification problems, an m class (m > 2) generalisation of the RDP exists (Tajine, Elizondo, Fiesler, & Korczak, 1997). The approach taken by the RDP is to augment the affine dimension of the input vectors, by adding to these vectors the outputs of a sequence of intermediate neurons as new components. Each intermediate neuron (IN) is an SLPT that can linearly separate an LS subset of points from all the rest of the points in an NLS problem. This allows for additional degrees of freedom for transforming the original NLS problem into a 0893-6080/$ - see front matter c  2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.neunet.2007.07.009