IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 31, NO. 6, DECEMBER 2001 881 Some Novel Classifiers Designed Using Prototypes Extracted by a New Scheme Based on Self-Organizing Feature Map Arijit Laha and Nikhil R. Pal, Senior Member, IEEE Abstract—We propose two new comprehensive schemes for designing prototype-based classifiers. The scheme addresses all major issues (number of prototypes, generation of prototypes, and utilization of the prototypes) involved in the design of a prototype-based classifier. First we use Kohonen’s self-organizing feature map (SOFM) algorithm to produce a minimum number (equal to the number of classes) of initial prototypes. Then we use a dynamic prototype generation and tuning algorithm (DYNAGEN) involving merging, splitting, deleting, and retraining of the prototypes to generate an adequate number of useful prototypes. These prototypes are used to design a “1 nearest multiple prototype (1-NMP)” classifier. Though the classifier performs quite well, it cannot reasonably deal with large variation of variance among the data from different classes. To overcome this deficiency we design a “1 most similar prototype (1-MSP)” classifier. We use the prototypes generated by the SOFM-based DYNAGEN algorithm and associate with each of them a zone of influence. A norm (Euclidean)-induced similarity measure is used for this. The prototypes and their zones of influence are fine-tuned by minimizing an error function. Both classifiers are trained and tested using several data sets, and a consistent improvement in performance of the latter over the former has been observed. We also compared our classifiers with some benchmark results available in the literature. Index Terms—Dynamic prototype generation, nearest neighbor (NN) classifier, prototype-based classifier, self-organizing feature map (SOFM). I. INTRODUCTION A CLASSIFIER designed from a data set can be defined as a function , where is the set of label vectors, is the number of features, and is the number of classes. If is a fuzzy classifier, then and . If is a crisp classifier, is a basis vector with components and ; consequently, here also . However, for a possibilistic classifier [3]. Designing a classifier involves finding a good . may be specified parametrically, e.g., Bayes classifier [1], or nonparametrically, e.g., nearest neighbor (NN) classifiers (crisp and fuzzy) [1], [3], nearest prototype (NP) classifiers (crisp or fuzzy) [1], [3], neural networks [4], etc. Manuscript received October 29, 1999; revised May 31, 2001. This paper was recommended by Associate Editor B. J. Oommen. A. Laha is with the National Institute of Management Calcutta, Alipore, Cal- cutta 700 027, India (e-mail: arijitl@yahoo.com). N. R. Pal is with the Electronics and Communication Sciences Unit, Indian Statistical Institute, Calcutta 700 035, India (e-mail: nikhil@isical.ac.in). Publisher Item Identifier S 1083-4419(01)08548-X. Although Bayes classifier is statistically optimal, it requires complete knowledge of prior probabilities ; and class densities , which is almost never possible in practical cases. Usually no knowledge of the underlying distribution is available except that it can be esti- mated from the samples. Among the nonparametric classification schemes the (NN) al- gorithms are the most straightforward. This family of algorithms is known as -NN algorithms. Given a set of labeled training samples the crisp -NN classifier assigns a sample as follows. Find the set of samples closest to . Assign to the class from which majority of closest neighbors has come. All crisp classifiers assign an absolute label to a sample tested. But in real world situation this is often disadvantageous. It would be better to have some measure of confidence available for different alternative decisions. If more than one decision have close confidence values then they may be examined more closely using additional information (if available) instead of immediately committing to a decision that might incur heavy penalty. This has motivated development of several variants of fuzzified -NN algorithms [2], [3]. Despite their simplicity, a straightforward implementation of an NN classifier may require large memory (the complete set of training samples has to be kept in memory) and may in- volve large computational overhead (the distance between and each of the training data points has to be computed). There have been several attempts to minimize the computational overhead of -NN algorithm [5], [6], some of which approximate the -NN scheme. However, even with most of them, the entire data set needs to be maintained. The prototype-based classifiers over- come these drawbacks. The training data set is represented by a set of prototypes , where . Each prototype can be thought of as a representative of a subset of . While designing a prototype-based classifier, one faces three fundamental issues. • How to generate the prototypes. • How many prototypes are to be generated. • How to use the prototypes for designing the classifier. Depending on the schemes adopted to address these ques- tions, there are a large number of classifiers. To illustrate the point we discuss the simplest of them, i.e., the NP classifiers 1083–4419/01$10.00 © 2001 IEEE