Classification Algorithms Based on Linear Combinations of Features Dominik ´ Sl¸ ezak and Jakub Wr´ oblewski Institute of Mathematics, Warsaw University, ul. Banacha 2, 02-097 Warsaw, Poland. slezak@alfa.mimuw.edu.pl jakubw@mimuw.edu.pl, http://alfa.mimuw.edu.pl/˜jakubw Abstract. We provide theoretical and algorithmic tools for finding new features which enable better classification of new cases. Such features are proposed to be searched for as linear combinations of continuously val- ued conditions. Regardless of the choice of classification algorithm itself, such an approach provides the compression of information concerning dependencies between conditional and decision features. Presented re- sults show that properly derived combinations of attributes, treated as new elements of the conditions’ set, may significantly improve the perfor- mance of well known classification algorithms, such as k-NN and rough set based approaches. 1 Introduction Classification is the problem of forecasting the decision for new cases, basing on their conditional features, by comparison with already known instances. An exemplar classification technique is the nearest neighborhood approach [3]. Given some arbitrarily fixed distance measure ρ, defined over the Cartesian product of conditional features treated as real valued dimensions, we can find for a new ex- ample kρ-nearest known cases u 1 , ..., u k and classify it as belonging to the same decision class as that most supported by them. The efficiency of this approach depends obviously on the choice of distance type and the choice of conditions over which we define ρ. Namely, it turns out that sometimes it is even better to consider smaller subset of conditions, to obtain better classification results (see e.g. [1]). Appropriate selection of conditions is the very important task with respect to practical applications, where it is more effective to base on smaller (or easier to be analyzed) groups of features. In the above k-NN approach such a selection is concerned just in view of the classification performance. There are, however, ap- proaches where it is regarded as the main paradigm, enabling to focus not on the classification only, but also on the representation of the dependencies between conditions and decisions. One of them is the decision rules based method, devel- oped within rough sets theory (see Section 2 for details and, e.g., [5] for further references). Although designed originally for discrete data, it can be applied to continuous conditions as well, by using discretization (see e.g. [4]) or tolerance J.M. Zytkow and J. Rauch (Eds.): PKDD’99, LNAI 1704, pp. 548-553, 1999. •  Springer-Verlag Berlin Heidelberg 1999