Pergamon CONTRIBUTED ARTICLE 0893-6080( 94 )E0049-Q Neural Networks, Vol. 7, No. 9, PP. 1441-1460, 1994 Copyright © 1994 Elsevier Science Ltd Printed in the USA. All rights reserved 0893-6080/94 $6.00 + .00 Growing Cell Structures A Self-Organizing Network for Unsupervised and Supervised Learning BERND FRITZKE Institut fiir Neuroinformatik (Received 25 May 1993; revised and accepted 23 March 1994) Alrstract-- We present a new self-organizing neural network model that has two variants. The first variant performs unsupervised learning and can be usedfor data visualization, clustering, and vector quantization. The main advantage over existing approaches ( e.g., the Kohonen feature map) is the ability of the model to automatically find a suitable network structure and size. This is achieved through a controlled growth process that also includes occasional removal of units. The second variant of the model is a supervised learning method that results from the combination of the above-mentioned self-organizing network with the radial basis function (RBF) approach. In this model it is possible--in contrast to earlier approaches--to perform the positioning of the RBF units and the supervised training of the weights in parallel. Therefore, the current classification error can be used to determine where to insert new RBF units. This leads to small networks that generalize very well. Results on the two-spirals benchmark and a vowel classification problem are presented that are better than any results previously published. Keywords--Self-organization, Incremental learning, Radial basis function, Clustering, Data visualization, Pattern classification, Two-spiral problem, Feature map. 1. INTRODUCTION Self-organizing neural network models, as proposed by Willshaw and vonder Malsburg (1976) and Kohonen (1982), generate mappings from high-dimensional sig- nal spaces to lower-dimensional topological structures. These mappings are able to preserve neighborhood re- lations in the input data and have the property to rep- resent regions of high signal density on correspondingly large parts of the topological structure. This makes them interesting for applications in various areas ranging from speech recognition (Kohonen, 1988) and data compression (Schweizer et al., 1991 ) to combinatorial optimization (Favata & Walker, 1991 ). The fact that similar mappings can be found at various places in the brains of humans and animals indicates that preser- This work was performedmainly during a stay of the author at the InternationalComputer ScienceInstitute at Berkeley. Acknowledgements:The author likesto thank Scott Fahlman for the permission to reproduce Figure20b and for maintainingthe CMU Benchmark Collection.Moreover, the two anonymousreviewers de- serve thanks for their most helpful comments. Requests for reprints shouldbe sent to BerndFritzke,Institut t'tir Neuroinformatik, Ruhr-Universi~t Bochum, 44780 Bochum, Ger- many; E-mail:fritzke@neuroinformatik.ruhr-uni-bochum.de vation of topology is an important principle at least in natural "signal processing systems." It has been noted that the predetermined structure and size of Kohonen's model imply limitations on the resulting mappings. Often one realizes only at the end of a simulation that a different shape or number of elements would have been more appropriate. On the other hand, there is in most cases no a priori infor- mation available that would allow to choose a suitable network size and shape in advance. A solution to this dilemma is to determine shape as well as size of the network during the simulation in an incremental fashion. This is the main principle of the model presented below. It has a flexible, problem-de- pendent structure, a variable number of elements, and a k-dimensional topology whereby k can be arbitrarily chosen. Recently it was demonstrated that the new model improves over Kohonen's feature map with re- spect to various important criteria (Fritzke, 1993a). We acknowledge, however, that the new model owes several ideas to Kohonen's approach and that it is an extension of his work rather than a completely different formalism. First we outline the network for unsupervised learn- ing and introduce later on the extension of the model to supervised learning. 1441