An approach to the automatic design of multiple classi®er systems Giorgio Giacinto, Fabio Roli * Department of Electrical and Electronic Engineering, University of Cagliari, Italy Piazza D'Armi, 09123, Cagliari, Italy Abstract Multiple classi®er systems (MCSs) based on the combination of outputs of a set of dierent classi®ers have been proposed in the ®eld of pattern recognition as a method for the development of high performance classi®cation systems. Previous work clearly showed that multiple classi®er systems are eective only if the classi®ers forming them are ac- curate and make dierent errors. Therefore, the fundamental need for methods aimed to design ``accurate and diverse'' classi®ers is currently acknowledged. In this paper, an approach to the automatic design of multiple classi®er systems is proposed. Given an initial large set of classi®ers, our approach is aimed at selecting the subset made up of the most accurate and diverse classi®ers. A proof of the optimality of the proposed design approach is given. Reported results on the classi®cation of multisensor remote sensing images show that this approach allows the design of eective multiple classi®er systems. Ó 2001 Elsevier Science B.V. All rights reserved. Keywords: Combination of classi®ers; Design of multiple classi®er systems; Accuracy and error diversity in classi®er ensembles; Image classi®cation; Remote sensing 1. Introduction Multiple classi®er systems (MCSs) based on the combination of outputs of a set of dierent clas- si®ers have been proposed in the ®eld of pattern recognition as a method of developing high per- formance classi®cation systems (Xu et al., 1992; Kittler et al., 1998; Kittler and Roli, 2000; Giacinto et al., 2000a,b). Typically, classi®ers are combined by means of voting rules, statistical techniques, belief functions, Dempster Shafer evidence theory, and other fusion schemes (Xu et al., 1992). Theo- retical and experimental results reported in the literature clearly showed that classi®er combina- tion is eective only when the individual classi®ers are ``accurate'' and ``diverse'', that is, if they ex- hibit low error rates and make dierent errors (Hansen and Salamon, 1990; Sharkey, 1999; Tumer and Ghosh, 1999; Kuncheva et al., 2000). In particular, it was shown that the combination of ``weak'' classi®ers (namely classi®ers exhibiting accuracies only slightly in excess of 50%) can oer dramatic improvements of performances if such classi®ers make dierent errors (Hansen and Sal- amon, 1990; Ji and Ma, 1997; Kuncheva et al., 2000). Unfortunately, the reported experimental and theoretical results also indicated that the cre- ation of accurate and diverse classi®ers is a very dicult task (Partridge, 1996; Partridge and Yates, www.elsevier.nl/locate/patrec Pattern Recognition Letters 22 (2001) 25±33 * Corresponding author. Tel.: +39-070-675-5874; fax: +39- 070-675-5900. E-mail addresses: giacinto@diee.unica.it (G. Giacinto), roli@diee.unica.it (F. Roli). 0167-8655/00/$ - see front matter Ó 2001 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 7 - 8 6 5 5 ( 0 0 ) 0 0 0 9 6 - 9