Vector Field Learning via Spectral Filtering Luca Baldassarre 1 , Lorenzo Rosasco 2,3 , Annalisa Barla 1 , and Alessandro Verri 1 1 Universit`a degli Studi di Genova - DISI Via Dodecaneso 35, Genova, Italy {baldassarre,barla,verri}@disi.unige.it 2 Istituto Italiano di Tecnologia, Via Morego, 30 16163 Genova, Italy 3 CBCL, Massachusetts Institute of Technology Cambridge, MA 02139 - USA lrosasco@mit.edu Abstract. In this paper we present and study a new class of regularized kernel methods for learning vector fields, which are based on filtering the spectrum of the kernel matrix. These methods include Tikhonov regu- larization as a special case, as well as interesting alternatives such as vector valued extensions of L2-Boosting. Our theoretical and experimen- tal analysis shows that spectral filters that yield iterative algorithms, such as L2-Boosting, are much faster than Tikhonov regularization and attain the same prediction performances. Finite sample bounds for the different filters can be derived in a common framework and highlight dif- ferent theoretical properties of the methods. The theory of vector valued reproducing kernel Hilbert space is a key tool in our study. Keywords: Vector-valued Functions; Multi-task; Regularization; Spec- tral Filtering; Kernels. 1 Introduction In this paper we study theoretical and computational properties of a class of kernel methods for learning a vector valued function. These methods are based on filtering the spectrum of the kernel matrix rather than empirical risk min- imization. The idea of using kernel methods for vector field learning has been considered in [1] where the framework of vector valued reproducing kernel Hilbert spaces was adopted and the representer theorem for Tikhonov regularization was generalized to the vector valued setting. Our work can be seen as an extension of the work in [1] aimed in particular at: 1) investigating the application of spectral filtering schemes [2] to learning vector fields; 2) establishing consistency and finite sample bounds for Tikhonov regularization as well as for all spec- tral filters in the setting of vector valued learning. One of the main outcomes of our study is that iterative algorithms based on spectral filtering outperform Tikhonov regularization from the computational perspective, while preserving the good prediction performances. J.L. Balc´azar et al. (Eds.): ECML PKDD 2010, Part I, LNAI 6321, pp. 56–71, 2010. c  Springer-Verlag Berlin Heidelberg 2010