IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 21, NO. zyxwvutsrqp 4, JULYIAUGUST 1991 735 Fuzzy Set Representation of Neural Network Classification Boundaries Norman P. Archer and Shouhong Wang Abstract-In neural network classification techniques, the un- certainty of a new observation belonging to a particular class is difficult to express in statistical terms. On the other hand, statistical classification techniques are also poor for supplying uncertainty information for new observations. However, the use of fuzzy sets is a promising approach to providing imprecise class membership information. The monotonic function neural network is a tool that can be used to develop fuzzy member- ship functions. This research suggests that a multiarchitecture monotonic function neural network can be used for fuzzy set representation of classification boundaries in monotonic pattern recognition, I. INTRODUCTION HERE HAS BEEN a recent upsurge in interest in neural T network applications in pattern recognition [2]. Like tra- ditional statistical classification methods [3], [4], most neural network classifiers set an ultimate objective of finding a clear cut-off classification boundary to divide the pattern space into two or more decision or classification regions based on some predefined criterion such as minimizing the misclassification rate. Since fuzzy set theory was suggested in the 1960s zyxwvut [5], pattern recognition problems have been intensively studied in the fuzzy set sense, especially when applying these concepts in the social context [6]. In fuzzy theory, class membership is not binary, but is represented by the value of a gradually changing function that can take on intermediate values between 0 and 1. In this way a pattern class need not have a sharp cut-off but may have a gradual fade-out [7]. The major attractions of fuzzy set theory in pattern recognition are threefold. First, it is difficult, if not impossible, to find a “true” or optimal clear cut- off classification boundary in a real problem. Second, decision makers often need information about classification uncertainty for particular real events. Third, considering pattern recogni- tion as a model for cognitive processes, the use of fuzzy sets is a promising approach to providing imprecise class membership information [6], [8], especially in the case where probability theory is difficult to apply directly. There have been several studies associating neural networks with fuzzy set theory [9]. Kosko [lo], for example, suggested combining fuzzy knowledge with neural networks in expert system reasoning. Shiue [ l l ] and Keller [12] used fuzzy set theory in designing learning algorithms for neural networks Manuscript received July 14, 1990; revised February 4, 1991. N. P. Archer is with the Faculty of Business, McMaster University, S. Wang is with the Division of Administration, University of New IEEE Log Number 9143737. Hamilton, ON, L8S 4M4 Canada. Brunswick, Saint John, NB, E2L 4L5 Canada. zyxwvutsrqp Linear boundarv zyxwvutsr I: Class 1 \ Sample population 1 zyxw 0 Sample population 2 Fig. 1. Linear discriminant analysis and perceptrons, respectively. However, research on repre- senting fuzzy membership in neural network classification problems is rare. Archer and Wang [ 11 developed a monotonic function neural network model, which is a modification of the standard back propagation neural network [13]. In that model, the neural net- work has monotonic constraints imposed during the learning process, to improve neural network performance for classifi- cation problems occurring in managerial and other situations, where the feature vector changes monotonically with the pattern vector. This research begins with that model, and de- velops a neural network model to represent fuzzy membership functions in two class monotonic pattern recognition problems. The remainder of the paper proceeds as follows. Section I1 describes the boundary representation problem in statistical classification. Section zyxwv I11 briefly reviews fuzzy theory con- cepts. Section IV describes how a fuzzy boundary relates to neural network classification, including some examples, and Section V is a general discussion of the suggested approach. 11. A PROBLEM IN STATISTICAL CLASSIFICATION In order to explore problems with statistical classification techniques, we begin with the two class linear discriminant analysis (LDA) classification method [14]. Suppose that a linear boundary y = zyxw E,”=, zyx bjxj separates the pattern space into two regions as shown in Fig. 1. The linear boundary is optimal only under the assumption that the sample data have multivariate normal distributions with common covariance It is worth noting two closely related characteristics of the LDA result. First, the linear boundary itself reveals nothing about the statistical behavior of the sample data distributions. For instance, the two very different sample populations in Fig. 1 can theoretically result in the same classification boundary. [141. 0018-9472/91/$01.00 zyxwvutsrq 0 1991 IEEE