The Incremental Fourier Classifier: Leveraging the Discrete Fourier Transform for Classifying High Speed Data Streams Chamari I. Kithulgoda a,∗ , Russel Pears a , M. Asif Naeem a a School of Engineering, Computer and Mathematical Sciences Auckland University of Technology, Auckland, New Zealand Abstract Two major performance bottlenecks with decision tree based classifiers in a data stream environment are the depth of the tree and the update overhead of maintaining leaf node statistics on an instance-wise basis to ensure that classification is consistent with the current state of the data stream. Previous research has shown that classifiers based on Fourier spectra derived from decision trees produce compact array structures that can be searched and maintained much more efficiently than deep tree based structures. However, the key issue of incrementally adapting the spectrum to changes has not been addressed. In this research we present a strategy for incremental maintenance of the Fourier spectrum to changes in concept that take place in data stream environments. Along with the incremental approach we also propose schemes for feature selection and synopsis generation that enable the coefficient array to be refreshed efficiently on a periodic basis. Our empirical evaluation on a number of widely used stream classifiers reveals that the Fourier classifier outperforms them, both in terms of classification accuracy as well as speed of classification. Keywords: Data Stream, Ensemble Classifier, Discrete Fourier Transform, Concept Drift, Fourier Spectrum, Feature Selection 1. Introduction The need for scaling up the process of mining high speed data streams is now paramount than ever before. However, greater throughput should not come at the price of prediction accuracy. Incremental learning techniques have been used extensively to address the data stream classification problem and to maintain a good balance be- tween accuracy and efficiency Mena-Torres & Aguilar-Ruiz (2014). In this research we adopt an incremental strategy based on the use of the Discrete Fourier Transform (DFT). We propose a novel approach for leveraging the DFT to scale up throughput while maintaining or improving classification accuracy over current state-of-the-art data stream classifiers. The Discrete Fourier Transform has long been a key tool in signal processing and has also been applied to data mining Park (2001), Kargupta & Park (2004), Kargupta et al. (2006). It has several attractive properties for cap- turing patterns that sets it apart from conventional mech- anisms such as decision trees and other types of classifiers. Firstly, it has been shown rigorously that spectra gen- erated from hierarchical classifiers such as decision trees can be represented in compact form thus speeding up the * Corresponding author Email addresses: ckithulg@aut.ac.nz (Chamari I. Kithulgoda), rpears@aut.ac.nz (Russel Pears), mnaeem@aut.ac.nz (M. Asif Naeem) classification process Sripirakas & Pears (2014). Secondly, Fourier spectra have the ability to embed several different patterns (concepts) into one entity unlike conventional en- semble classifier systems which maintain multiple models. This is due to the fact that Spectra can be represented in array form and hence spectra generated at several different points in time can be aggregated into one unifying spec- trum that embeds the properties of its constituent spectra. Thirdly, classification can be performed in Fourier space and Fourier spectra, once generated from conventional classifiers, can be used independently of them. Fourthly, the distributive nature of the inverse Fourier transform operation offers the possibility of exploiting parallelism in the classification process. Such properties have been exploited Sakthithasan et al. (2015), Kithulgoda & Pears (2016) in mining data streams but their use comes at a price. The application of the DFT on multivariate data to produce a spectrum is a non-trivial operation and has time complexity O(|X| 2 ), where |X| is the size of the feature space Kargupta & Park (2004). The size of the feature space |X| grows exponen- tially with the dimensionality of the data. In a highly dynamic data stream environment the time spent on re- peated application of the DFT at each concept detection point can quickly become prohibitive as our experimenta- tion in Section 6.9 shows. A much more effective strategy would be to incremen- tally maintain a spectrum in a fashion analogous to the incremental maintenance of a conventional classifier such Preprint submitted to Expert Systems with Applications December 10, 2017 Manuscript Click here to download Manuscript: Revised Article_11Dec2017_Expert Systems_KithulgodaPearsNaeem.pdf Click here to view linked References