Tackling the problem of classification with noisy data using Multiple Classifier Systems: Analysis of the performance and robustness José A. Sáez a,⇑ , Mikel Galar b , Julián Luengo c , Francisco Herrera a a Department of Computer Science and Artificial Intelligence, University of Granada, CITIC-UGR, Granada 18071, Spain b Department of Automática y Computación, Universidad Pública de Navarra, Pamplona 31006, Spain c Department of Civil Engineering, LSI, University of Burgos, Burgos 09006, Spain article info Article history: Received 25 May 2012 Received in revised form 13 May 2013 Accepted 2 June 2013 Available online 13 June 2013 Keywords: Noisy data Class noise Attribute noise Multiple Classifier System Classification abstract Traditional classifier learning algorithms build a unique classifier from the training data. Noisy data may deteriorate the performance of this classifier depending on the degree of sensitiveness to data corruptions of the learning method. In the literature, it is widely claimed that building several classifiers from noisy training data and combining their pre- dictions is an interesting method of overcoming the individual problems produced by noise in each classifier. This statement is usually not supported by thorough empirical studies considering problems with different types and levels of noise. Furthermore, in noisy envi- ronments, the noise robustness of the methods can be more important than the perfor- mance results themselves and, therefore, it must be carefully studied. This paper aims to reach conclusions on such aspects focusing on the analysis of the behavior, in terms of per- formance and robustness, of several Multiple Classifier Systems against their individual classifiers when these are trained with noisy data. In order to accomplish this study, sev- eral classification algorithms, of varying noise robustness, will be chosen and compared with respect to their combination on a large collection of noisy datasets. The results obtained show that the success of the Multiple Classifier Systems trained with noisy data depends on the individual classifiers chosen, the decisions combination method and the type and level of noise present in the dataset, but also on the way of creating diversity to build the final system. In most of the cases, they are able to outperform all their single classification algorithms in terms of global performance, even though their robustness results will depend on the way of introducing diversity into the Multiple Classifier System. Ó 2013 Elsevier Inc. All rights reserved. 1. Introduction Classifier learning algorithms aim to extract the knowledge from a problem from the available set of labeled examples (training set) in order to predict the class for new, previously unobserved, examples [8]. Classic learning algorithms [36,4] build a unique model, called a classifier, which attempts to generalize the peculiarities of the training set. Therefore, the suc- cess of these methods, that is, their ability to classify new examples, highly depends on the usage of a concrete feature descriptor and a particular inference procedure, and directly on the training data. 0020-0255/$ - see front matter Ó 2013 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.ins.2013.06.002 ⇑ Corresponding author. Tel.: +34 958 240598; fax: +34 958 243317. E-mail addresses: smja@decsai.ugr.es (J.A. Sáez), mikel.galar@unavarra.es (M. Galar), jluengo@ubu.es (J. Luengo), herrera@decsai.ugr.es (F. Herrera). Information Sciences 247 (2013) 1–20 Contents lists available at SciVerse ScienceDirect Information Sciences journal homepage: www.elsevier.com/locate/ins