Extreme Learning for Evolving Hybrid Neural Networks Fernando Bordignon and Fernando Gomide School of Electrical and Computer Engineering University of Campinas Campinas, SP, Brazil E-mail: {bordi,gomide}@dca.fee.unicamp.br Abstract—This paper addresses a structure and introduces an evolving learning approach to train uninorm-based hybrid neural networks using extreme learning concepts. Evolving systems are high level adaptive systems able to simultaneously modify their structures and parameters from a stream of data, online. Learning from data streams is a contemporary and challenging issue due to the increasing rate of the size and temporal availability of data, turning traditional learn- ing methods impracticable. Uninorm-based neurons, rooted in triangular norms and conorms, generalize fuzzy neurons. Uninorms bring flexibility and generality to fuzzy neuron models as they can behave like triangular norms, triangular conorms, or in between by adjusting identity elements. This feature adds a form of plasticity in neural network modeling. An online clustering method is used to granulate the input space, and a scheme based on extreme learning is developed to train the neural network. Computational results show that the learning approach is competitive when compared with alternative evolving modeling methods. Keywords-hybrid neural networks; unineurons; evolving sys- tems; online learning; extreme learning. I. I NTRODUCTION Machine learning methods are being reevaluated over the last years as the need of online capabilities is made evident by the massive growth in numbers and accessibility of computing devices [1] [2]. Computers are connected in a unprecedented level making available a massive amount of data, generating interest to understand and forecast key economic, technical, and social variables. With this context in mind new methods have been con- ceived during recent years. A new class of machine learning approach with adaptive abilities to simultaneously learn a system structure and its parameters emerged, namely, the evolving system approach. The term evolving means online, gradual systems development and adaptation. Evolving sys- tems are an alternative and innovative way of adapt, learn and represent knowledge about changing environments [1]. Fuzzy set theory provides solid methods and efficient mechanisms to develop evolving systems as reported in early works on fuzzy rule-based modeling [3] and neural fuzzy inference DENFIS models [4]. More recently, more gen- eral evolving connectionist systems (ECOS) framework [5], flexible fuzzy inference systems (FLEXFIS) [6], evolving Takagi-Sugeno (eTS) modeling [7] and its improved versions [8] [9] have been developed. Current research focuses on granular computing and modeling extensions such as interval and fuzzy set-based evolving modeling (FBeM) systems [2] From the neural networks perspective, extensions of clas- sic neuron model using triangular norms and co-norms (t- norms and t-conorms) have been extensively studied in the literature lately e.g. [10], [11], [12], [13] and [14]. In particular, the notion of uninorm has been explored to generalize further both, the basic fuzzy neuron model, and neural architectures through hybridizations. Uninorms are interesting not only because they add flexibility in neuron and neural network design, but also because they naturally provide a way to add neuronal plasticity through a single parameter called the identity element. This is also very useful for evolving systems and neural structures. In this paper we address a hybrid neural fuzzy network whose structure has two main parts: a fuzzy neural system and a neural network. The multilayer structure of the net- work has membership functions in the input layer neurons, uninorm-based neurons in the second layer, and classic neurons in the third layer. Uninorms provide flexibility at the cost of an extra parameter (identity element) to learn for each unineuron. Interesting, however, the identity element can be either chosen by the designer if he has prior knowledge or needs a particular structure, or leave it to be learnt using data. Due to its inherent highly nonlinear nature, learning complexity can be reduced using extreme learning [15]. Basically, extreme learning consists in randomly assigning weights for the hidden layer of feed-forward neural networks and analytically determine the output weights. Extreme learning has shown to provide fast training, accurate results and good generalization performance [16]. The input layer of the hybrid neural fuzzy network ad- dressed in this paper uses Gaussian membership functions to granulate the input space. Centers of the membership functions are determined online via recursive clustering methods. The hidden layer unineurons are compositions of uninorms at local synaptic processing, with  t-conorm at global aggregation level. A typical neural network layer with sigmoid activation function forms the output layer. After this brief introduction, the paper proceeds as fol- lows. Section II gives detailed description of the learning steps including the evolving fuzzy neural network system approach. Next, computational results of the tests conducted 2012 Brazilian Symposium on Neural Networks 1522-4899/12 $26.00 © 2012 IEEE DOI 10.1109/SBRN.2012.14 196