A hybrid meta-learning architecture for multi-objective optimization of SVM parameters Péricles B.C. Miranda a,b,n , Ricardo B.C. Prudêncio a , André P.L.F. de Carvalho c , Carlos Soares d a Centro de Informática, Universidade Federal de Pernambuco, Recife, Brazil b Departamento de Estatística e Informática, Universidade Federal Rural de Pernambuco, Recife, Brazil c Depto. de Ciências da Computação, Universidade de São Paulo, São Carlos, Brazil d INESC TEC, Faculdade de Engenharia, Universidade do Porto, Porto, Portugal article info Article history: Received 5 February 2014 Received in revised form 29 May 2014 Accepted 8 June 2014 Communicated by V. Palade Available online 18 June 2014 Keywords: Multi-objective optimization Particles swarm optimization Meta-learning Support vector machines Parameter selection abstract Support Vector Machines (SVMs) have achieved a considerable attention due to their theoretical foundations and good empirical performance when compared to other learning algorithms in different applications. However, the SVM performance strongly depends on the adequate calibration of its parameters. In this work we proposed a hybrid multi-objective architecture which combines meta- learning (ML) with multi-objective particle swarm optimization algorithms for the SVM parameter selection problem. Given an input problem, the proposed architecture uses a ML technique to suggest an initial Pareto front of SVM configurations based on previous similar learning problems; the suggested Pareto front is then refined by a multi-objective optimization algorithm. In this combination, solutions provided by ML are possibly located in good regions in the search space. Hence, using a reduced number of successful candidates, the search process would converge faster and be less expensive. In the performed experiments, the proposed solution was compared to traditional multi-objective algorithms with random initialization, obtaining Pareto fronts with higher quality on a set of 100 classification problems. & 2014 Elsevier B.V. All rights reserved. 1. Introduction Support Vector Machines (SVMs) represent a class of very competitive algorithms that have been successfully applied by the Machine Learning community to different learning problems. Despite the potential good results that can be yielded, the SVM performance strongly depends on the adequate choice of its parameters, and an exhaustive trial-and-error procedure for selecting good values of parameters is not practical for computational reasons [1]. The selection of SVM parameters is treated by different authors as an optimization task in which a search technique is used to find adequate configurations of parameters for the given learning pro- blem at hand. Different optimization techniques have been applied in literature to SVM parameter selection, including for instance Evolutionary Algorithms (EA) [2], Particle Swarm Optimization (PSO) [3] and Tabu Search [4]. Previous works commonly used single objective techniques for SVM parameter selection, however this is not totally adequate since this task is inherently a Multi-Objective Optimization (MOO) problem [5]. In this context, we can mention the use of Multi-Objective EA (MOEA) [5,6], Multi-Objective PSO (MOPSO) [7] and Gradient-Based techniques [1], which considered multiple objectives in the parameter selection process. Although the application of search techniques represents an automatic solution to select SVM parameters, this approach can be very expensive, since a large number of candidate configurations of parameters are often evaluated during the search process [8]. An alternative approach to SVM parameter selection is the use of Meta-Learning (ML), which treats parameter selection as a supervised learning task [8,9]. Each training example for ML (i.e. each meta-example) stores the characteristics of a past learning problem (i.e., its number of training instances, its class entropy, etc.) and the performance obtained by a set of candidate config- urations of parameters on the problem. By receiving a set of such meta-examples, a meta-learner can predict configurations of parameters for a new problem based on its characteristics. ML can provide good or intermediate results with a relatively small number of suggested configurations of parameters [11]. Optimization algorithms in turn can produce better results com- pared to ML, but with high computational costs. Although ML can be a lower cost alternative, it is dependent on the chosen learning problems in the meta-base and the characteristics adopted to Contents lists available at ScienceDirect journal homepage: www.elsevier.com/locate/neucom Neurocomputing http://dx.doi.org/10.1016/j.neucom.2014.06.026 0925-2312/& 2014 Elsevier B.V. All rights reserved. n Corresponding author. E-mail addresses: pbcm@cin.ufpe.br (P.B.C. Miranda), rbcp@cin.ufpe.br (R.B.C. Prudêncio), andre@icmc.usp.br (A.P.L.F. de Carvalho), csoares@fep.up.pt (C. Soares). Neurocomputing 143 (2014) 27–43