2011 IEEE International Conference on Fuzzy Systems June 27-30, 2011, Taipei, Taiwan 978-1-4244-7317-5/11/$26.00 ©2011 IEEE Fuzzy C-Means Clustering Based Construction And Training For Second Order RBF Network Kanishka Tyagi Xun Cai Michael T.Manry Department of Electrical Engineering School of Computer Science and Technology Department of Electrical Engineering The University of Texas at Arlington Shandong University The University of Texas at Arlington Arlington, TX, USA, 76010 Jinan, Shandong, P.R.China, 25010 Arlington, TX, USA, 76010 kanishka.tyagi@mavs.uta.edu caixunzh@sdu.edu.cn manry@uta.edu Abstract—The paper presents a novel two-step approach for constructing and training of optimally weighted Euclidean distance based Radial-Basis Function (RBF) neural network. Unlike other RBF learning algorithms, the proposed paradigms use Fuzzy C-means for initial clustering and optimal learning factors to train the network parameters (i.e. spread parameter and mean vector). We also introduce an optimized weighted Distance Measure (DM) to calculate the activation function. Newton’s algorithm is proposed for obtaining multiple optimal learning factor for the network parameters (including weighted DM). Simulation results show that regardless of the input data dimension, the proposed algorithms are a significant improvement in terms of convergence speed, network size and generalization over conventional RBF models which use a single optimal learning factor. The generalization ability of the proposed algorithm is further substantiated by using k-fold validation. Keywords- Fuzzy-C means clustering, Hessian Matrix, Newton’s Method, Optimal Learning Factor, Orthogonal Least Square. I. INTRODUCTION The Radial Basis function (RBF) network is a three layer supervised feed-forward network [1] used in interpolation, probability density function estimation and approximation of smooth multivariate functions [2]-[7]. The RBF was first introduced as a solution to the real multivariate interpolation problem. The RBF model can approximate any multivariate continuous function arbitrarily well on a compact domain if a sufficient number of radial basis functions are given [4]. Fuzzy clustering has been used in RBF networks for many applications. The model presented in [8] uses it for Traffic status evaluation. Though it has been applied only to the simulated data and real traffic condition is still an “open” problem. Conditional Fuzzy Clustering in the design of RBF networks [9]. The idea in [9] is further extended by using supervised Fuzzy clustering to improve RBF performance in regression task [10]. In [11] Fuzzy Clustering is used for designing a RF network used in Modeling of Respiratory system. A hybrid learning procedure is proposed in [12] which employs unsupervised clustering algorithm like k-means for determining the center for radial basis function and supervised learning for updating output weights connecting the radial basis function unit (hidden unit) to the output unit. In [13] a gradient training algorithm for updating all the network parameters (mean vector, spread parameter and output weights) is presented. A novel space filling curves with genetically evolving parameters is proposed in [14]. In order to train the RBF network mostly first order methods are used. Gradient descent learning described in [15], [16] offers a balance between performance and training speed. These networks can be compared with sigmoid hidden unit based feed forward neural networks in [16]-[18]. The combination of steepest descent and Newton’s method would seem to be more promising for unconstrained optimization problems [19]. This method is convergent and has a high convergence rate. Since Newton’s method for the RBF often has non-positive definite or even singular Hessian, Levenberg- Marquardt (LM) or other methods are used instead. However second order methods do not scale well and suffer from heavy computational cost. Although first order methods scale better, they are sensitive to input means and gain factor [20]. Using other permissible radial basis function apart from Gaussian function has also been explored by constructing reformulated radial basis function neural networks trained with gradient descent algorithm [20]-[22]. But again the convergence and performance of the gradient descent algorithm is strongly affected by the spread parameter and the radial basis function mean vector. To solve these problems we introduce an interesting family of RBF networks based on Newton’s method. The paper is organized as follows: conventional RBF structure and parameter determining methods are reviewed in section 2; In Section 3, the theoretical and mathematical treatment for the proposed family of RBF models is presented. Section 4 discusses the learning parameters for the proposed model. In Section 5, learning algorithms for the proposed model are elaborated; a detailed algorithm for the proposed RBF models is presented in Section 6; the training performance of various RBF models are compared in Section 7, with single Optimal Learning Factor based RBF model on several bench mark and real life datasets. II. REVIEW OF RADIAL BASIS FUNCTION NETWORK Without the loss of generality, we restrict ourselves to a three layer fully connected RBF with non linear activation function 248