Dynamic selection of classifiers—A comprehensive review Alceu S. Britto Jr. a,b,n , Robert Sabourin c , Luiz E.S. Oliveira d a Pontifícia Universidade Católica do Paraná (PUCPR), Curitiba, PR, Brazil b Universidade Estadual de Ponta Grossa (UEPG), Ponta Grossa, PR, Brazil c École de technologie supérieure (ÉTS), Université du Québec, Montreal, QC, Canada d Universidade Federal do Paraná (UFPR), Curitiba, PR, Brazil article info Article history: Received 28 August 2013 Received in revised form 3 May 2014 Accepted 7 May 2014 Keywords: Ensemble of classifiers Dynamic selection of classifiers Data complexity abstract This work presents a literature review of multiple classifier systems based on the dynamic selection of classifiers. First, it briefly reviews some basic concepts and definitions related to such a classification approach and then it presents the state of the art organized according to a proposed taxonomy. In addition, a two-step analysis is applied to the results of the main methods reported in the literature, considering different classification problems. The first step is based on statistical analyses of the significance of these results. The idea is to figure out the problems for which a significant contribution can be observed in terms of classification performance by using a dynamic selection approach. The second step, based on data complexity measures, is used to investigate whether or not a relation exists between the possible performance contribution and the complexity of the classification problem. From this comprehensive study, we observed that, for some classification problems, the performance contribution of the dynamic selection approach is statistically significant when compared to that of a single-based classifier. In addition, we found evidence of a relation between the observed performance contribution and the complexity of the classification problem. These observations allow us to suggest, from the classification problem complexity, that further work should be done to predict whether or not to use a dynamic selection approach. & 2014 Elsevier Ltd. All rights reserved. 1. Introduction Classification is a fundamental task in Pattern Recognition, which is the main reason why the past few decades have seen a vast number of research projects devoted to classification methods applied to different fields of the human activity. Although the methods available in the literature may differ in many respects, the latest research results lead to a common conclusion; creating a monolithic classifier to cover all the variability inherent to most pattern recognition problems is somewhat unfeasible. With this in mind, many researchers have focused on Multiple Classifier Systems (MCSs), and consequently, many new solutions have been dedicated to each of the three possible MCS phases: (a) generation, (b) selection, and (c) integration, which are represented in Fig. 1. In the first phase, a pool of classifiers is generated; in the second phase, one or a subset of these classifiers is selected, while in the last phase, a final decision is made based on the prediction(s) of the selected classifier(s). It is worth noting that such a representation is not unique, since the selection and integration phases may be facultative. For instance, one may find MCS where the whole pool of classifiers is used without any selection or systems where just one classifier is selected from the pool, making the integration phase unnecessary. In a nutshell, recent contributions with respect to the first phase indicate that the most promising direction is to generate a pool of accurate and diverse classifiers. The authors in [1] state that a necessary and sufficient condition for an ensemble of classifiers to be more accurate than any of its individual members is for the classifiers to be accurate and diverse. Dietterich [2] explains that an accurate classifier has an error rate lower than the random guessing on new samples, while two classifiers are diverse if they make different errors on new samples. The rationale behind this is that the individual accurate classifiers in the pool may compete each other by making different and perhaps complemen- tary errors. As for the selection phase, interesting results have been obtained by selecting specific classifiers for each test pattern, which characterizes a dynamic selection of classifiers, instead of using the same classifier for all of them (static selection). More- over, additional contributions have been observed when ensem- bles are selected instead of just one single classifier. In such a case, Contents lists available at ScienceDirect journal homepage: www.elsevier.com/locate/pr Pattern Recognition http://dx.doi.org/10.1016/j.patcog.2014.05.003 0031-3203/& 2014 Elsevier Ltd. All rights reserved. n Corresponding author at: Post-graduate Program in Informatics (PPGIa), Ponti- fical Catholic University of Parana Rua Imaculada Conceição, 1155, Curitiba (PR), 80215-901, Brazil. Tel.: +55 41 3271 1669; fax: +55 41 3271 2121. E-mail addresses: alceu@ppgia.pucpr.br (A.S. Britto Jr.), robert.sabourin@etsmtl.ca (R. Sabourin), lesoliveira@inf.ufpr.br (L.E.S. Oliveira). Please cite this article as: A.S. Britto Jr. et al., Dynamic selection of classifiers—A comprehensive review, Pattern Recognition (2014), http://dx.doi.org/10.1016/j.patcog.2014.05.003i Pattern Recognition ∎ (∎∎∎∎) ∎∎∎–∎∎∎