Exploratory basis pursuit classification M. Brown a, * , N.P. Costen b a Control Systems Centre, School of Electrical and Electronics Engineering, University of Manchester, P.O. Box 88, Manchester M60 1QD, UK b Department Computing and Mathematics, Manchester Metropolitan University, Manchester, UK Available online 23 May 2005 Abstract Feature selection is a fundamental process in many classifier design problems. However, it is NP-complete and approximate approaches often require requires extensive exploration and evaluation. This paper describes a novel approach that represents feature selection as a continuous regularization problem which has a single, global minimum, where the modelÕs complexity is measured using a 1-norm on the parameter vector. A new exploratory design process is also described that allows the designer to efficiently construct the complete locus of sparse, kernel-based classifiers. It allows the designer to investigate the optimal parametersÕ trajectories as the regularization parameter is altered and look for effects, such as SimpsonÕs paradox, that occur in many multivariate data analysis problems. The approach is dem- onstrated on the well-known Australian Credit data set. Ó 2005 Published by Elsevier B.V. Keywords: Feature selection; Sparse classification; Regularization 1. Introduction Feature selection is a fundamental process with- in many classification algorithms, as many classifi- cation problems are not well understood and a large dictionary of potential, flexible features ex- ists, from which it is necessary to select a relevant subset (Weston et al., 2001). This is typical in the vision and biometrics fields, where new measure- ment or transformation techniques are explored in order to boost the classifierÕs performance. The aims of feature selection include: • improved, more robust parameter estimation and • improved insight into the decision making process. Feature selection is generally an empirical pro- cess that is performed prior to, or jointly with, the parameter estimation process. Training data is used to assess different model combinations 0167-8655/$ - see front matter Ó 2005 Published by Elsevier B.V. doi:10.1016/j.patrec.2005.03.012 * Corresponding author. E-mail address: martin.brown@manchester.ac.uk (M. Brown). Pattern Recognition Letters 26 (2005) 1907–1915 www.elsevier.com/locate/patrec