Ensembles of Multilayer Perceptron and Modular Neural Networks for Fast and Accurate Learning R.M. Valdovinos Lab. Reconocimiento de Patrones Instituto Tecnol´ ogico de Toluca Av. Tecnol´ ogico s/n, 52140 Metepec (Mexico) li rmvr@hotmail.com J.S. S´ anchez Dept. Llenguatges i Sistemes Inform` atics Universitat Jaume I Av. Sos Baynat s/n, E-12071 Castell´ o de la Plana (Spain) sanchez@uji.es Abstract In this paper, we analyze the performance of a classifier ensemble consisting of small-sized neural networks for accurate and fast learning. To this end, three topologies of neural networks are evaluated: the multilayer perceptron with two different configurations in the hidden layer, and the modular neural network. The experiments here carried out illustrate the effectiveness of the ensembles of neural networks in terms of average predictive accuracy and processing time, as compared to the single classifiers. 1. Introduction Currently, the combination of multiple classifiers (also known as ensemble of classifiers, com- mittee of learners, mixture of experts, etc.) constitutes a well-established research field in Pattern Recognition and Machine Learning. It is recognized that in general, the use of multiple classifiers instead of a single classifier leads to an increase in the overall predictive accuracy [7,19]. Even in some cases, although the ensemble might not result better than the ”best” single classifier, it could diminish or eliminate the risk of picking an inadequate single classifier [19]. Let D = {D 1 ,...,D h } be a set of h classifiers. Each classifier D i (i =1,...,h) gets as input a feature vector x ∈ℜ n , and assigns it to one of the c problem classes. The output of an ensemble of classifiers is an h-dimensional vector [D 1 (x),...,D h (x)] T containing the decisions of each of the h individual classifiers. For combining the individual decisions, the most popular (and simplest) method is the majority voting rule [18], although there exist other more complex schemes (e.g., average, minority, medium, product of votes) [3, 14, 17]. In general, an ensemble is built in two steps, that is, training multiple individual classifiers and then combining their predictions. According to the styles of training the base classifiers, current ensemble algorithms can be roughly categorized into two groups: algorithms where base classifiers must be trained sequentially (e.g., AdaBoost [9], Arc-x4 [8], MultiBoost [23], LogitBoost [10]),