Abstract—Two algorithms for building classification trees, based on Tsallis and Rényi entropy, are proposed and applied to customer churn problem. The dataset for modeling represents highly unbalanced proportion of two classes, which is often found in real world applications, and may cause negative effects on classification performance of the algorithms. The quality measures for obtained trees are compared for different values of α parameter. I. INTRODUCTION ECISION trees are powerful and very popular tools for different classification tasks [1]-[3]. The attractiveness of this technique is due to the fact that they create rules that can be easily interpreted. Decision trees use some statistical property called information gain to measure the classification power of the input attributes on classification problem as the difference between the entropy before and after a decision. Entropy computation is used to generate simple decision trees, in terms of the structure, with effective classification, since tree size reduction depends on the attribute selection. For this purpose, usually Shannon entropy is used, but other entropy formulas, such as Rényi [4] and Tsallis [5] entropy, can also be applied. Here, a comparative study based on Rényi and Tsallis entropy is described taking into account the issue of imbalance in the class distribution. We used data from telecommunication industry to predict loss of customers to competitors what is known as customer churn. In this dynamical and liberal market customers can choose among cellular service providers and actively migrate from one service provider to another. This problem is especially interesting due to the fact that the portion of churning customers in business practice is low, between 1% and 5%, depending on the country and type of the telecommunication service. D The comparison of the trees is carried out by taking into account different values of α parameter and set of the following measures: classification accuracy, area under the ROC curve, lift, and number of leaves in a tree as complexity measure. In the next section properties of Rényi and Tsallis entropies are described. The data used in this study are described in the third section. The empirical analysis and comparison of the entropies is shown in fourth section. This type of analysis is especially interesting for decision trees because of the high dimensionality of telecommunication data. Conclusions are given in the last section. II.THEORETICAL FRAMEWORK In this paper we assume that observations may belong to two given classes and for the classification we use a modified algorithm similar to C4.5 [6] to construct a binary tree in R environment [7]. As a general measure of diversity of objects, a Shannon entropy is often used which is defined as [8]: 1 log , n s i i i H p p = =- ∑ (1) where i p is the probability of occurrence of an event i x being an element of the event X that can take values ,..., i n x x . The value of the entropy depends on two parameters: (1) disorder (uncertainty) and is maximum when the probability i p for every i x is equal; (2) the value of n. Shannon entropy assumes a tradeoff between contributions from the main mass of the distribution and the tail. To control both parameters two generalizations were proposed by Rényi [4] and Tsallis [5]. The Rényi entropy is defined as: 1 1 log , 1 n R i i H p α α =   =  ÷ -   ∑ (2) where parameter α is used to adjust the measure depending on the shape of probability distributions. The Tsallis entropy is defined as: 1 1 1 . 1 n R i i H p α α =   = -  ÷ -   ∑ (3) With Shannon entropy, events with high or low probability have equal weights in the entropy computation. However, using Tsallis entropy, for 1 α > , events with high probability contribute more than low probabilities for the entropy value [9]. Therefore, the higher is the value of α , the higher is the contribution of high probability events in the final result. Furthermore, increasing α parameter ( ) α →∞ makes the Rényi entropy determined by events Comparison of Decision Trees with Rényi and Tsallis Entropy Applied for Imbalanced Churn Dataset Krzysztof Gajowniczek, Tomasz Ząbkowski, Arkadiusz Orłowski Department of Informatics, Warsaw University of Life Sciences, Nowoursynowska 159, 02-776Warsaw, Poland Email: krzysztof_gajowniczek@sggw.pl, tomasz_zabkowski@sggw.pl, arkadiusz_orlowski@sggw.pl Proceedings of the Federated Conference on Computer Science and Information Systems pp. 39–44 DOI: 10.15439/2015F121 ACSIS, Vol. 5 978-83-60810-66-8/$25.00 c 2015, IEEE 39