A First Study on a Fuzzy Rule-Based Multiclassification System Framework Combining FURIA with Bagging and Feature Selection Krzysztof Trawi´ nski, Oscar Cord´ on, and Arnaud Quirin Abstract— In this work, we conduct a preliminary study considering a fuzzy rule-based multiclassification system de- sign framework based on Fuzzy Unordered Rule Induction Algorithm (FURIA). This advanced method serves as the fuzzy classification rule learning algorithm to derive the component classifiers considering bagging combined with feature selection. We develop a study on the use of both bagging and feature selection to design a final FURIA-based fuzzy multiclassifier applied to ten popular UCI datasets. The results obtained show that this approach provides promising results. I. I NTRODUCTION Multiclassification systems (MCSs) (also called multiclas- sifiers or classifier ensembles) have been shown as very promising tools to improve the performance of single clas- sifiers when dealing with complex, high dimensional clas- sification problems in the last few years [1]. This research topic has become especially active in the classical machine learning area, considering decision trees or neural networks to generate the component classifiers, but also some work has been done recently using different kinds of fuzzy classifiers [2], [3], [4], [5], [6], [7], [8]. Fuzzy Unordered Rule Induction Algorithm (FURIA) [9], [10] is a powerful fuzzy classification rule learning algorithm that can deal with a very common problem of fuzzy rule- based classification systems (FRBCSs), the so-called curse of dimensionality [11]. By combining advantages of the RIP- PER algorithm [12] with fuzzy logic, this algorithm is able to generate simple and compact sets of fuzzy classification rules, even when tackling datasets with a large amount of features. Apart from its ability to deal with high dimensional datasets, this approach has shown a performance advantage in comparison to classical machine learning methods such like RIPPER [12] and C4.5 [13]. There are several techniques in order to obtain diversity, which leads to a highly accurate ensemble [1], [14], among the classifiers. Bagging [15] and boosting [16] are the two most popular generic approaches to do so [17]. There are also other more recent proposals considering other ways to promote disagreement between the component classifiers, with feature selection being an extended strategy [18]. All in all, it turned out that a combination between bagging and feature selection is a generic approach leading to good MCS designs for any kind of classifier learning method [19]. The idea of this paper is a preliminary study of the performance of FURIA-based fuzzy MCSs. FURIA-based Krzysztof Trawi´ nski, Oscar Cord´ on, and Arnaud Quirin are with European Centre for Soft Computing. Mieres, Spain. (email: {krzysztof.trawinski, oscar.cordon, arnaud.quirin}@softcomputing.es). fuzzy MCSs are build using a combination of bagging and feature selection. We considered three different types of feature selection algorithms: random subspace [18], mutual information-based feature selection (MIFS) [20], and the random-greedy feature selection based on MIFS and the GRASP approach [21]. In order to test the accuracy of the proposed fuzzy MCSs, we conduct experiments with 10 datasets taken from the UCI machine learning repository and provide a study of the results obtained. Then, we compare them against single FURIA classifiers. This paper is structured as follows. The next section presents a state of the art about MCSs and fuzzy MCSs. In Sec. III the FURIA algorithm is described, while Sec. IV describes our approach for designing FURIA-based fuzzy MCSs. The experiments developed and their analysis are detailed in Sec. V. Finally, Sec. VI collects some concluding remarks and future research lines. II. BACKGROUND AND RELATED WORK This section explores the current literature related to the generation of fuzzy rule-based multiclassification systems (FRBMCSs). The techniques used to generate MCSs and fuzzy MCSs are described in Sec. II-A and II-B, respectively. A. Related work on MCSs A MCS is the result of the combination of the outputs of a group of individually trained classifiers in order to get a system that is usually more accurate than any of its single components [1]. These kinds of methods have gained a large acceptance in the machine learning community during the last two decades due to their high performance. Decision trees are the most common classifier structure considered and much work has been done in the topic [22], [23], although they can be used with any other type of classifiers (the use of neural networks is also very extended, see for example [24]). There are different ways to design a classifier ensemble. On the one hand, there is a classical group of approaches considering data resampling to obtain different training sets to derive each individual classifier. In bagging [15], they are independently learnt from resampled training sets (“bags”), which are randomly selected with replacement from the original training data set. Boosting methods [16] sequen- tially generate the individual classifiers (weak learners) by selecting the training set for each of them based on the performance of the previous classifier(s) in the series. Op- posed to bagging, the resampling process gives a higher