IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 29, NO. 5, OCTOBER 1999 619 Fuzzy Relational Classifier Trained by Fuzzy Clustering Magne Setnes, Student Member, IEEE, and Robert Babuˇ ska Abstract—A novel approach to nonlinear classification is pre- sented. In the training phase of the classifier, the training data is first clustered in an unsupervised way by fuzzy -means or a similar algorithm. The class labels are not used in this step. Then, a fuzzy relation between the clusters and the class identifiers is computed. This approach allows the number of prototypes to be independent of the number of actual classes. For the classification of unseen patterns, the membership degrees of the feature vector in the clusters are first computed by using the distance measure of the clustering algorithm. Then, the output fuzzy set is obtained by relational composition. This fuzzy set contains the membership degrees of the pattern in the given classes. A crisp decision is obtained by defuzzification, which gives either a single class or a “reject” decision, when a unique class cannot be selected based on the available information. The principle of the proposed method is demonstrated on an artificial data set and the applicability of the method is shown on the identification of live-stock from recorded sound sequences. The obtained results are compared with two other classifiers. Index Terms— Classification, fuzzy clustering, fuzzy relations, pattern recognition, recognition of sound sequences. I. INTRODUCTION T HE objective of pattern recognition is the identification of structures in data similar to known structures. In the statistical approach to numerical pattern recognition [1] the known structures are based on mathematical models, and the usefulness of such methods depends on the availability of sufficiently accurate models of the objects generating the data. Methods based on clustering and techniques recently devel- oped in the field of computational intelligence such as neural networks, fuzzy logic and genetic algorithms are becoming increasingly popular in the pattern recognition community [2]–[5]. Such methods offer an attractive alternative to sta- tistical approaches as they do not require a priori assumptions of statistical models. They are able to learn the mapping of functions and systems, and can perform classification from labeled training data as well as explore structures and classes in unlabeled data. This article presents a new approach to pattern classification which uses a fuzzy logic relation to establish the corre- spondence between structures in the feature space and the class identifiers (labels). This approach can effectively deal Manuscript received May 12, 1998; revised November 22, 1998. This work was supported in part by the Research Council of Norway. This paper was recommended by Associate Editor L. O. Hall. The authors are with the Faculty of Information Technology and Systems, Control Laboratory, Delft University of Technology, 2600 GA Delft, The Netherlands (e-mail: m.setnes@its.tudelft.nl). Publisher Item Identifier S 1083-4419(99)05270-X. with classes that cannot be described by a single construct in the feature space. This is especially useful for problems where one does not a priori know which features should be selected in order to yield well-separated classes. By using the fuzzy logic relation, one avoids the problem of labeling the prototypes which can be particularly difficult when classes are characterized by partially shared structures or when the train- ing data contains classification errors (typical for subjective classification). This partial sharing of structures among several classes is naturally captured by the fuzzy relation. Moreover, class labels may be fuzzy distributions as well. The fuzzy relation-based classification scheme represents a transparent alternative to conventional black-box techniques like artificial neural networks for complex nonlinear classification problems. The transparency of the relational classifier allows for the analysis of both the trained classifier and of the classification result for unseen patterns. In the training of the classifier, two steps are distinguished: 1) exploratory data analysis (unsupervised fuzzy cluster- ing); 2) construction of a logical relation between the structures found in the previous step and the class labels. In the exploratory step, the available data objects are clustered in groups by the fuzzy -means (FCM) or a similar algorithm. Clustering results in a fuzzy partition matrix, which specifies for each training sample a -tuple of membership degrees in the obtained clusters. In the second step, a fuzzy relation is computed, using the memberships obtained in the first step and the target membership of the pattern in the classes (which may be crisp or fuzzy). This relation is built by means of the -composition (a fuzzy implication) and conjunctive aggregation. It specifies the logical relationship between the cluster membership and the class membership. To classify new patterns, the membership of each pattern in the clusters (fuzzy prototypes) is computed from its distance to the cluster centers, giving a fuzzy set of prototype membership. Then, relational composition of this fuzzy set with the fuzzy relation is applied to compute an output fuzzy set. This set gives a fuzzy classification in terms of membership degrees of the pattern in the given classes. When a crisp decision is required, defuzzification has to be applied to this fuzzy set. Typically, the maximum defuzzification method is used. The rest of this paper is organized in three sections. First, the training of the classifier is explained in Section II. The classification of new patterns is described in Section III. A simple example is presented throughout these two sections in order to illustrate the individual steps. Section IV reports 1083–4419/99$10.00  1999 IEEE