A New Disagreement Measure for Characterization of Classification Problems Yulia Ledeneva, 1 Alexander Gelbukh, 1 René A. García-Hernández, 2 José Fco. Martínez- Trinidad, 3 J. Ariel Carrasco-Ochoa 3 1 Instituto Politécnico Nacional, Centro de Investigación en Computación, México yledeneva@yahoo.com, gelbukh@gelbukh.com 2 Instituto Tecnológico de Toluca, México renearnulfo@hotmail.com 3 Instituto Nacional de Astrofísica, Óptica y Electrónica, México fmartine @inaoep.mx, ariel@inaoep.mx Abstract Robert P.W. Duin, Elzbieta Pekalska and David M.J. Tax proposed the characterization of classification problems by classifier disagreement. They showed that it is possible to use a standard set of supervised classification problems for constructing a rule that allows deciding about the similarity of new problems to the existing ones. The classifier disagreement could be used to group classification problems in a way which could help to select the appropriate tools for solving new problems. Duin et al proposed a dissimilarity measure between two problems taking into account only the full disagreement matrices. They used a measure of the disagreement based on the coincidence of the classifier output however the correctness was not considered. In this work, we propose a new measure of disagreement which takes into account the correctness of classification result. To calculate the disagreement each object is analyzed to verify if it was classified correctly or incorrectly by the classifiers. We use this new disagreement measure to calculate the dissimilarity between two problems. Some experiments were done and the results were compared against Duin’s et al results. 1. Introduction In [1] the characterization of classification problems by classifier disagreement was proposed. The authors of the article used the problem characteristics to find the appropriate tools for solving it. They calculated the differences in classification results between pairs of individual classifiers and the result was defined as a measure of disagreement. The investigation of the authors of the paper showed that it is possible to use a standard set of supervised classification problems for constructing a tool that allows deciding about the similarity of new problems to the existing ones. The performance of these tools gives a first indication about how to solve the problem, as they tell whether the chosen classifiers are appropriate. The disagreement between a set of classifiers could be used to group classification problems in a way which could help to select the appropriate tools for solving new problems. In turn the disagreement patterns point towards different types of classification problems and indicate the usefulness of a classifier with respect to a set of classification problems and classifiers. We propose a new measure of disagreement which takes into account the correctness of classification result. The disagreement will be calculated analyzing each object to verify if it was classified correctly or incorrectly by the classifiers. We use this new disagreement measure to calculate the dissimilarity between two problems. We describe some experiments and the results are compared with previous results of Duin’s et al. In section 2, the set of classifiers, problems and the measure of disagreement proposed by Duin’s et al are described. A new disagreement measure for characterization of classification problems is proposed in section 3. Some experiments are presented in section 4. Conclusions and future work are presented in section 5.