Ensemble Learning with Local Diversity ? Ricardo ˜ Nanculef 1 , Carlos Valle 1 , H´ ector Allende 1 , and Claudio Moraga 2,3 1 Universidad T´ ecnica Federico Santa Mar´ ıa, Departamento de Inform´atica, CP 110-V Valpara´ ıso, Chile {jnancu, cvalle, hallende}@inf.utfsm.cl 2 European Centre for Soft Computing 33600 Mieres, Asturias, Spain 3 Dortmund University, 44221 Dortmund, Germany claudio.moraga@udo.edu Abstract. The concept of Diversity is now recognized as a key characte- ristic of successful ensembles of predictors. In this paper we investigate an algorithm to generate diversity locally in regression ensembles of neural networks, which is based on the idea of imposing a neighborhood rela- tion over the set of learners. In this algorithm each predictor iteratively improves its state considering only information about the performance of the neighbors to generate a sort of local negative correlation. We will assess our technique on two real data sets and compare this with Negative Correlation Learning, an effective technique to get diverse ensembles. We will demonstrate that the local approach exhibits better or comparable results than this global one. 1 Introduction Ensemble methods offer a simple and flexible way to build powerful learning ma- chines for a great variety of problems including classification, regression and clus- tering [7] [3]. An ensemble algorithm to learn a set of examples D = {(x i ,y i ); i = 1,...,m} selects a set of predictors S = {h 0 ,h 2 ,...,h n-1 } from some base hy- pothesis space H and builds a decision function f as a composition f = L S, where L is an aggregation operator such as voting for categorical outputs or a linear combination for continuous outputs. To be useful, the set S has to have some degree of heterogeneity or diversity that allows the group compensate individual errors and reach a better expected performance. The characterization of methods to generate diversity has matured in the last years [5] [6] and the concept is now recognised as a central element to get significant performance improvements with the ensemble. Negative Correla- tion Learning [9] [11] for example has been proved to be an effective method to get diversity in regression ensembles. ? This work was supported in part by Research Grant Fondecyt (Chile) 1040365 and 7050205, and in part by Research Grant DGIP-UTFSM (Chile). Partial support was also received from Research Grant BMBF (Germany) CHL 03-Z13.