Evolving Classiﬁers Ensembles with Heterogeneous Predictors Pier Luca Lanzi 1, 2 , Daniele Loiacono 1 , and Matteo Zanini 1 1 Artiﬁcial Intelligence and Robotics Laboratory (AIRLab), Politecnico di Milano. P.za L. da Vinci 32, I-20133, Milano, Italy 2 Illinois Genetic Algorithm Laboratory (IlliGAL), University of Illinois at Urbana Champaign, Urbana, IL 61801, USA lanzi@elet.polimi.it, loiacono@elet.polimi.it, mat.zanini@tiscali.it Abstract. XCS with computed prediction, namely XCSF, extends XCS by replacing the classiﬁer prediction with a parametrized prediction func- tion. Although several types of prediction functions have been intro- duced, so far XCSF models are still limited to evolving classiﬁers with the same prediction function. In this paper, we introduce XCSF with heterogeneous predictors, XCSFHP, which allows the evolution of classi- ﬁers with diﬀerent types of prediction function within the same popula- tion. We compared XCSFHP to XCSF on several problems. Our results suggest that XCSFHP generally performs as XCSF with the most appro- priate prediction function for the given problem. In particular, XCSFHP seems able to evolve, in each problem subspace, the most adequate type of prediction function. 1 Introduction XCS with computed prediction [22], namely XCSF, extends the typical idea of classiﬁers by replacing the prediction parameter with a prediction function p(s t , w). The prediction function, typically deﬁned as a linear combination of s and w, is used to compute the classiﬁer prediction on the basis of the cur- rent state s t and a parameter vector w associated to each classiﬁer. Since its introduction, XCSF has been extended with several techniques for computing the classiﬁer prediction, ranging from polynomial functions [11] to Neural Net- works [10] and Support Vector Machines [18]. However the experimental results reported so far [12,10,18] suggest that none of such techniques outperforms the others in every respect. In particular, as discussed in [10], a powerful predictor may solve complex problems but may learn more slowly or be unnecessarily ex- pensive in the simple ones. Therefore, the choice of the predictor to use in each problem still requires human expertise and good knowledge of the problem. In this paper we introduce XCSF with heterogeneous predictor, dubbed XCSFHP that extends XCSF by evolving an ensemble of classiﬁers with diﬀerent types of prediction function. In XCSFHP, the predictors are not speciﬁed at design time, instead the system is in charge of evolving the most adequate predictor for each problem subspace. J. Bacardit et al. (Eds.): IWLCS 2006/2007, LNAI 4998, pp. 218–234, 2008. c  Springer-Verlag Berlin Heidelberg 2008