Vol.:(0123456789) 1 3 International Journal of Machine Learning and Cybernetics https://doi.org/10.1007/s13042-018-0843-4 ORIGINAL ARTICLE A reinforced fuzzy ARTMAP model for data classifcation Farhad Pourpanah 1  · Chee Peng Lim 2  · Qi Hao 1 Received: 28 December 2017 / Accepted: 6 June 2018 © Springer-Verlag GmbH Germany, part of Springer Nature 2018 Abstract This paper presents a hybrid model consisting of fuzzy ARTMAP (FAM) and reinforcement learning (RL) for tackling data classifcation problems. RL is used as a feedback mechanism to reward the prototype nodes of data samples established by FAM. Specifcally, Q-learning is adopted to develop the hybrid model known as QFAM. A Q-value is assigned to each prototype node, which is updated incrementally based on the prediction accuracy of the node pertaining to each data sample. To evaluate the performance of the proposed QFAM model, a series of experiments with benchmark problems and a real- world case study, i.e., human motion recognition, are conducted. The bootstrap method is used to quantify the results with the 95% confdence interval estimates. The results are also compared with those from FAM as well as other models reported in the literature. The outcomes indicate the efectiveness of QFAM in tackling data classifcation tasks. Keywords Data classifcation · Fuzzy ARTMAP · Reinforcement learning · Q-learning 1 Introduction Artifcial neural networks (ANNs) [1] are popular data-based learning models for tackling classifcation problems. To this end, many ANN models such as multi-layered perceptron (MLP) [2], probabilistic neural network (PNN) [3], radial basis function (RBF) [4], and adaptive resonance theory (ART) [5], have been proposed. The main challenge of data- based learning models is to overcome the stability–plastic- ity dilemma [5], that means the learning model should be able to absorb useful information from new data samples incrementally without forgetting or corrupting previously learned information. However, many ANN models such as the traditional MLP and RBF [4] networks with batch learn- ing procedures are not able to overcome this dilemma. When a batch-learning network is given a set of new data samples after its training phase, the network executes an iterative training procedure using both new and previous data samples for learning, in an attempt to preserve its existing knowledge base (established based on previous data samples). This procedure often results in slow convergence to the optimal network weights and overftting problem when a large data set is used [6]. To alleviate these problems, online neural networks that are able to learn incrementally have been pro- posed such as PNN, ART, Fuzzy Min–Max network (FMM) [7], and related models [8, 9]. A review of online learning in supervised neural network models is presented in Ref. [10]. Fuzzy ARTMAP (FAM) [11] is a supervised neural net- work that combines the capability of ART in solving the stability–plasticity dilemma and the benefts of fuzzy logic. Instead of more complex fuzzy sets, e.g., type-2 fuzzy set or institutional fuzzy sets [12], FAM uses the conventional type-1 fuzzy set to handle vague and imprecise human linguistic information in a fast manner. FAM is an online learning model that operates by measuring the similar- ity between its prototype nodes and the new input sample against a threshold. If the similarity measure is not satis- fed, a new prototype node can be added to encode the new data sample. While new information can be absorbed by increasing the number of prototype nodes incrementally [5], previously learned information can still be preserved in its network structure; therefore overcoming the stability–plas- ticity dilemma. However, it is possible for classifcation algorithms to incorrectly classify input samples from diferent classes that are located along the decision boundary [13, 14]. In Ref. [15], an optimal hyperplane is generated to maximize the * Farhad Pourpanah farhad.086@gmail.com 1 School of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China 2 Institute for Intelligent Systems Research and Innovation, Deakin University, Waurn Ponds, Australia