Multi-Label Classification Methods for Multi-Target Regression Eleftherios Spyromitros-Xioufis 1 , Grigorios Tsoumakas 1 , William Groves 2 , and Ioannis Vlahavas 1 1 Department of Informatics, Aristotle University of Thessaloniki, Greece {espyromi,greg,vlahavas}@csd.auth.gr 2 Department of Computer Science and Engineering, University of Minnesota, USA groves@cs.umn.edu Abstract. Real world prediction problems often involve the simultane- ous prediction of multiple target variables using the same set of predic- tive variables. When the target variables are binary, the prediction task is called multi-label classification while when the target variables are real- valued the task is called multi-target regression. Although multi-target regression attracted the attention of the research community prior to multi-label classification, the recent advances in this field motivate a study of whether newer state-of-the-art algorithms developed for multi- label classification are applicable and equally successful in the domain of multi-target regression. In this paper we introduce two new multi- target regression algorithms: multi-target stacking (MTS) and ensemble of regressor chains (ERC), inspired by two popular multi-label classi- fication approaches that are based on a single-target decomposition of the multi-target problem and the idea of treating the other prediction targets as additional input variables that augment the input space. Fur- thermore, we detect an important shortcoming on both methods related to the methodology used to create the additional input variables and de- velop modified versions of the algorithms (MTSC and ERCC) to tackle it. All methods are empirically evaluated on 12 real-world multi-target regression datasets, 8 of which are first introduced in this paper and are made publicly available for future benchmarks. The experimental results show that ERCC performs significantly better than both a strong base- line that learns a single model for each target using bagging of regression trees and the state-of-the-art multi-objective random forest approach. Also, the proposed modification results in significant performance gains for both MTS and ERC. Keywords: multi-target regression, multi-output regression, multivari- ate regression, multi-label classification, regressor chains, stacking 1 Introduction Learning from multi-label data has recently received increased attention by re- searchers working on machine learning and data mining for two main reasons. arXiv:1211.6581v4 [cs.LG] 17 Jun 2014