This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 1 PANFIS: A Novel Incremental Learning Machine Mahardhika Pratama, Student Member, IEEE, Sreenatha. G. Anavatti, Plamen. P. Angelov, Senior Member, IEEE, and Edwin Lughofer Abstract— Most of the dynamics in real-world systems are compiled by shifts and drifts, which are uneasy to be overcome by omnipresent neuro-fuzzy systems. Nonetheless, learning in nonstationary environment entails a system owning high degree of flexibility capable of assembling its rule base autonomously according to the degree of nonlinearity contained in the system. In practice, the rule growing and pruning are carried out merely benefiting from a small snapshot of the complete training data to truncate the computational load and memory demand to the low level. An exposure of a novel algorithm, namely parsimonious network based on fuzzy inference system (PANFIS), is to this end presented herein. PANFIS can commence its learning process from scratch with an empty rule base. The fuzzy rules can be stitched up and expelled by virtue of statistical contributions of the fuzzy rules and injected datum afterward. Identical fuzzy sets may be alluded and blended to be one fuzzy set as a pursuit of a transparent rule base escalating human’s interpretability. The learning and modeling performances of the proposed PANFIS are numerically validated using several benchmark problems from real-world or synthetic datasets. The validation includes comparisons with state-of-the-art evolving neuro-fuzzy methods and showcases that our new method can compete and in some cases even outperform these approaches in terms of predictive fidelity and model complexity. Index Terms— Evolving neuro-fuzzy systems (ENFSs), incremental learning, sample-wise training. I. I NTRODUCTION A. Preliminary T HE salient motivation behind the use of fuzzy system is that it allows operations interpretable in a way akin to the human’s logical reasoning. Fuzzy logic system proposed by Zadeh [1] is intelligible using fuzzy linguistic rule and can realize approximate reasoning to deal with imprecision and uncertainty in a decision-making process. These traits are preferable when the system is too complex to be analyzed by a physics-based approach or the source of information can Manuscript received April 13, 2012; revised May 1, 2013; accepted June 22, 2013. This work was supported in part by the Austrian fund for promoting scientific research under Contract I328-N23, the IREFS, the research program with the Austrian Center of Competence in Mechatronics, and the COMET K2 program of the Austrian government. M. Pratama and S. G. Anavatti are with the School of Engineering and Information Technology, University of New South Wales, Australian Defence Force Academy, Canberra ACT 2600, Australia (e-mail: pratama@ieee.org; s.anavatti@adfa.edu.au). P. P. Angelov is with the School of Computing and Commu- nications, Lancaster university, Lancaster LA14WA, U.K. (e-mail: p.p.angelov@lancaster.ac.uk). E. Lughofer is with the Department of Knowledge-Based Mathematical Systems, Johannes Kepler University of Linz, Linz A-4040, Austria (e-mail: edwin.lughofer@jku.at). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TNNLS.2013.2271933 be merely interpreted qualitatively, inexactly, or uncertainly. Traditional approaches in designing the fuzzy system are unfortunately overdependent on expert knowledge [2] and usu- ally necessitate tedious manual interventions. This for brevity leads to a static rule base, which cannot be tuned once its initial setting to gain a better performance. The designers noticeably have to spend a laborious time to examine all input–output relationships of a complex system to elicit a representative rule base constraining its practicability in evolving dynamic and time critical environments. This issue has led to the development of neuro-fuzzy systems (NFSs) [42], a powerful hybrid modeling approach that assimilates the learning ability, parallelism, and robustness of neural networks with the human-like linguistic and approx- imate reasoning traits of the fuzzy logic systems. The complex and dynamic natures of real-world engineering problems are in general complicated by time-varying or regime shifting issues. Classical NFSs are in contrast trained completely from offline data and remain as static models, which are impractical for nonstationary environments. Recently, coping with nonstationary environments has drawn intensive research works for typical NFSs, which are able to adapt their parameters and to automatically expand their design contexts simultaneously. Evolving NFS (ENFS) based on the concept of incremental learning [3] accordingly opens a new unchartered territory as a plausible solution provider to settle time-varying cases that convey regime shifting and drifting properties, and enriched a landscape of fuzzy system for coping with time-variant systems in real- time fashions. The principal construct of the ENFS initiates a favorable cornerstone in handling time-varying system. That is, whenever a new characteristic of the system appears, the existing rules will automatically reorganize its structure or even split a supplementary rule to accommodate the uncovered data distributions. The creation of an extraneous rule in the following signifies the presence of the untouched data region, which could be a new characteristic of the process or a reaction to a new disturbance. This obviously produces a promising impetus to handle the nonstationary, shifting, and drifting effects of data streams [9] in real-time with immense oppor- tunities for future smart devices and algorithms embedded on them. B. Survey Over State-of-the-Art Works In the early development of the ENFSs, most of the mod- els are generally featureless as a complete dataset at hand a priori is indispensable for them to properly accomplish a given task dubbed as a batch learning scheme. A retraining 2162-237X/$31.00 © 2013 IEEE