1063-6706 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TFUZZ.2019.2935688, IEEE Transactions on Fuzzy Systems IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. XX, NO. X, XX 2018 1 Incremental Missing-Data Imputation for Evolving Fuzzy Granular Prediction Cristiano Garcia, Daniel Leite and Igor ˇ Skrjanc Abstract—Missing values are common in real-world data stream applications. This paper proposes a modified evolving granular fuzzy rule-based model for function approximation and time series prediction in an online context where values may be missing. The fuzzy model is equipped with an incremental learning algorithm that simultaneously imputes missing data and adapts model parameters and structure over time. The evolving Fuzzy Granular Predictor (eFGP) handles single and multiple missing values on data samples by developing reduced- term consequent polynomials and utilizing time-varying granules. Missing at random (MAR) and missing completely at random (MCAR) values in nonstationary data streams are approached. Experiments to predict monthly weather conditions, the number of bikes hired on a daily basis, and the sound pressure on an airfoil from incomplete data streams show the usefulness of eFGP models. Results were compared with those of state- of-the-art fuzzy and neuro-fuzzy evolving modeling methods. A statistical hypothesis test shows that eFGP outperforms other evolving intelligent methods in online MAR and MCAR settings, regardless of the application. Keywords—Evolving Intelligence, Fuzzy System, Data Stream, Incremental Learning, Missing-Data Imputation. I. I NTRODUCTION Knowledge discovery from data streams is helpful for many practical purposes. Detecting frequent patterns, trends, season- alities, nonstationarities may help human decision-making in a variety of situations, applications and endeavors. Data mining, machine learning and computational intelligence methods have been applied to the purpose of finding useful information in sets of data [1]. These methods generally fit data patterns into classification or prediction models. Data sources may be static or time-varying depending if their probability distributions change over time. In this con- text, static very often means that the source remains nearly unchanged so that data collected in a specific time interval are sufficient to represent reasonably well the behavior of the process or phenomenon in the future. Conversely, time- varying means that the data distribution is subject to variations (concept change), and online learning and model adaptation are necessary. Models to deal with time-varying data streams must take into consideration that: (i) samples cannot be permanently stored; (ii) data sets are potentially unbounded; (iii) processing Manuscript received on Sept. 8, 2018; revised on Mar. 29 and Aug. 7, 2019. Cristiano Garcia and Daniel Leite are with the Department of Automat- ics, Federal University of Lavras, Brazil. Igor ˇ Skrjanc is with the Faculty of Electrical Engineering, University of Ljubljana, Slovenia. E-mails: cris- tiano.garcia@ufla.br, daniel.leite@ufla.br, igor.skrjanc@fe.uni-lj.si. time should scale linearly (or at least polynomially) with the number of samples, attributes and model parameters; and (iv) the data distribution may change gradually (concept drift) or abruptly (concept shift) at any time, or new patterns may emerge [1]–[12]. A survey on evolving fuzzy and neuro-fuzzy systems that deal with these fundamental issues and with some additional challenges of online-modeling and learning scenarios was recently published in [13]. The vast majority of stream-oriented learning methods re- quire the values of all attributes to be available to work properly. However, missing values are common in real-world applications. Missing data arise due to incomplete obser- vations, data transfer problems, malfunction of sensors or devices, incomplete information obtained from experts or on public surveys, among others [14][15]. Three ways of treating missing data can be mentioned [16]: (i) discard samples, or even attributes, with many missing data; (ii) impute values by maximum likelihood and parameter estimation procedures; and (iii) identify relationships among attributes and from previous values of an attribute to estimate new values. Statistical and intelligent methods have been proposed to deal with missing data, especially in offline settings, where historical datasets are available [17][18]. General methods en- tail deleting samples that contain one or more missing values, deleting attributes with more than a predefined percentage of their values missing, or imputing zeros, mean, or median for an attribute. After imputation, the complete, approximated, dataset is used for learning, classification, or prediction [15][17][19]. Model-based imputation methods provide a nonlinear way of handling missing data. These methods very often outperform general and independent methods [16][20]. In [21], for exam- ple, several standalone imputation methods were overcome in a number of datasets and situations by imputation methods used in conjunction with a given predictive model. Evolving systems are intelligent systems that, differently from adaptive and machine-learning systems, learn their pa- rameters and structure simultaneously using a stream of data [1][13][22][23]. The structural elements of evolving systems can be artificial neurons, fuzzy rules, data clusters, data clouds, sub-trees, or information granules [3][13][23]. To deal with nonlinear and time-varying processes, evolving models should be updated through the use of online learning algorithms so that eventually-large data flows can be processed in real time. The use of offline methods in this kind of problem is infeasible due to the unavailability of data to offline training, and tight time and memory constraints [1][13]. In this paper, missing values in nonstationary data streams is for the first time considered by means of an evolving approach. Authorized licensed use limited to: UNIVERSITY OF LJUBLJANA. Downloaded on February 26,2020 at 09:26:34 UTC from IEEE Xplore. Restrictions apply.