On the evolutionary optimization of k-NN by label-dependent feature weighting Daniel Mateos-García , Jorge García-Gutiérrez, José C. Riquelme-Santos Keywords: Feature weighting Evolutionary computation Label dependency abstract Different approaches of feature weighting and k-value selection to improve the nearest neighbour tech- nique can be found in the literature. In this work, we show an evolutionary approach called k-Label Dependent Evolutionary Distance Weighting (kLDEDW) which calculates a set of local weights depending on each class besides an optimal k value. Thus, we attempt to carry out two improvements simulta- neously: we locally transform the feature space to improve the accuracy of the k-nearest-neighbour rule whilst we search for the best value for k from the training data. Rigorous statistical tests demonstrate that our approach improves the general k-nearest-neighbour rule and several approaches based on local weighting. The use of weight-based models is common in machine learn- ing, and specifically, in classification problems (Wettschereck et al., 1997). The proper adjustment of weights during the training phase improves the model prediction rate. Weighted neural net- works are perhaps the most used example (Li et al., 2012; Park, 2009), but support vector machines (Shen et al., 2012) and near- est-neighbour (Chen et al., 2009) methods can also use weights for a better fit. All the weighting proposals have the search for opti- mal values (in training) in common, attempting to avoid overfit- ting. This optimization can be carried out by analytical methods (e.g., RELIEF method (Sun, 2007)) or by heuristics such as evolu- tionary computation (AlSukker et al., 2010) or tabu search (Tahir et al., 2007). A survey of the research in the literature on weight-based mod- els shows that most of them use a set of weights for all instances. For example, in the weighted nearest neighbour method, a x i value is wanted for each feature f i and represents the influence of f i in the calculation of every distance which is modified according to the influences of every feature. A weight x i is therefore a measure of the importance of f i . Although most approaches search for a global set of weights for all instances, there are also other local proposals which use weights mainly applied to prototypes (region) (Fernandez and Isasi, 2008) or class labels (AlSukker et al., 2010; Chen et al., 2011). In this work, the latter option has been analyzed, taking into account that optimizing a model with a set of weights for each prototype may prove to be too hard (in terms of execution time) for evolutionary computation. We explore the hypothesis that feature influence is not the same for every label. For example, if we classify patients according to three labels representing variants of a disease, the feature age could be more important for one variant than for another, and therefore, it could be more appropriate to have different weights depending on the variant. The difficulty of such a model is obvious: if we want to classify a new example with an unknown class and we have different weights for each class, which set of weights should we apply? Hence, we propose a heuristic based on an evo- lutionary algorithm that firstly searches for a set of different weights per class or label in the training phase and later selects the best set of weights for each instance in the testing phase to optimize a nearest neighbour classifier. Moreover, the selection of the number of neighbours (k parameter) for the nearest neigh- bour classifier is essential for its best performance. For this reason, we also use the evolutionary search to simultaneously attempt to find the best k. In the next sections, we show how the combination of weight matrices and a proper number of neighbours results in an objective good performance of the nearest neighbour rule. The rest of the pa- per is organized as follows. Section 2 presents related works. Sec- tion 3 describes the general process for obtaining and using the weight matrix in the nearest neighbour method. The results achieved are shown in Section 4. Finally, Section 5 presents a sum- mary of the conclusions and future lines of work. 2. Related work There are many works in the literature which refer to how to obtain weights to improve machine learning techniques. These