Pruning Method of Belief Decision Trees Salsabil Trabelsi Salsabilt@voila.fr Zied Elouedi zied.elouedi@gmx.fr Khaled Mellouli khaled.mellouli@ihec.rnu.tn LARODEC, Institut Sup´ erieur de Gestion de Tunis, 41 Avenue de la Libert´ e, 2000 Le Bardo, Tunisie Abstract — The belief decision tree (BDT) approach is a decision tree in an uncertain environment where the uncertainty is represented through the Transferable Belief Model (TBM), one interpretation of the belief function theory. The uncertainty can appear either in the actual class of training objects or attribute values of objects to classify. In this paper, we develop a post-pruning method of belief decision trees in order to reduce size and improve classification accuracy on unseen cases. The pruning of decision tree has a considerable intention in the areas of machine learning. Keywords: machine learning, uncertainty, belief function theory, belief decision tree, pruning. I. INTRODUCTION Decision trees are a simple yet successful technique for supervised classification learning. The visual presentation makes the decision tree model very easy to understand. It has also good classification accuracy compared to other classification techniques. However, the standard decision trees do not well perform their classification task in an environment characterized by uncertainty in data. In order to overcome this limitation, many researches have been done to adapt standard decision tree to this kind of environment. The idea was to introduce theories that could represent uncertainty. Several kinds of decision trees were developed: probabilistic decision trees [11], fuzzy decision trees [17], belief decision trees [2],[3] and possibilistic decision trees [7],[6]. In our work, we will focus on belief decision trees. The belief decision tree approach is a decision tree technique adapted in order to handle uncertainty about the actual class of the objects in the training set and also to classify objects characterized by uncertain attributes. The uncertainty is represented by the Transferable Belief Model (TBM), one interpretation of the belief function theory. The theory of belief functions is considered as a useful theory for representing and managing uncertain knowledge. It allows to express partial beliefs in a flexible way. Besides, it permits to handle partial or total ignorance concerning classification parametres. When a belief decision tree is built from real world databases, many of branches will reflect noise in the training data due to uncertainty. The results are many of undesirable nodes and difficulty to interpret the tree. Our aim is to overcome this problem of overfitting in belief decision tree. In order to reduce the size of the tree and improve classification accuracy. Pruning is a way to cope with this problem. So, our objective in this work is to prune belief decision tree. ”How does tree pruning work?” there are two common approaches to tree pruning. Methods that can control the growth of a decision tree during its development are called pre-pruning methods, the others are called post-pruning methods. In post-pruning approach, grow the full tree, allow it overfit the data and then post-prune it. It requires more computation than pre-pruning, yet generally leads to a more reliable tree. In this work, we focused on post-pruning approach. Pre- pruning in belief decision tree has developed in [5] by improv- ing the stopping criteria concerning the value of the selection measure. So, we suggest to develop post-pruning method to simplify the belief decision trees in order to reduce the size and the complexity. This paper is organized as follows: Section 2 provides a brief description of basics of belief function theory. In section 3, we describe the BDT approach. Then, in Section 4, we present the description of our pruning belief decision tree method. Finally in Section 5, we carry simulations to compare BDT without pruning, after pre-pruning and after our post-pruning method. II. BELIEF FUNCTION THEORY In this section, we briefly review the main concepts un- derlying the belief function theory. This theory is appropriate to handle uncertainty in classification problems especially within the decision tree technique. In belief decision trees the uncertainty is represented through the Transferable Belief Model (TBM), one interpretation of the belief function theory. A. Definitions The TBM is a model to represent quantified belief functions [15]. Let Θ be a finite set of elementary events to a given problem, called the frame of discernment [14]. All the subsets of Θ belong to the power set of Θ, denoted by 2 Θ . The impact of a piece of evidence on the different subsets of the frame of discernment Θ is represented by a basic belief assignment (bba). PROCEEDINGS OF WORLD ACADEMY OF SCIENCE, ENGINEERING AND TECHNOLOGY VOLUME 14 AUGUST 2006 ISSN 1307-6884 PWASET VOLUME 14 AUGUST 2006 ISSN 1307-6884 424 © 2006 WASET.ORG