Combinatorial Effects of Local Structures and Scoring Metrics in Bayesian Optimization Algorithm Hossein Karshenas Iran University of Science and Technology Narmak, Tehran, Iran ho_karshenas @comp.iust.ac.ir Amin Nikanjam Iran University of Science and Technology Narmak, Tehran, Iran nikanjam@iust.ac.ir B. Hoda Helmi Iran University of Science and Technology Narmak, Tehran, Iran helmi@iust.ac.ir Adel T. Rahmani Iran University of Science and Technology Narmak, Tehran, Iran rahmani@iust.ac.ir ABSTRACT Bayesian Optimization Algorithm (BOA) has been used with different local structures to represent more complex models and a variety of scoring metrics to evaluate Bayesian network. But the combinatorial effects of these elements on the performance of BOA have not been investigated yet. In this paper the performance of BOA is studied using two criteria: Number of fitness evaluations and structural accuracy of the model. It is shown that simple exact local structures like CPT in conjunction with complexity penalizing BIC metric outperforms others in terms of model accuracy. But considering number of fitness evaluations (efficiency) of the algorithm, CPT with other complexity penalizing metric K2P performs better. Categories and Subject Descriptors G.1.6 [Numerical Analysis]: Optimization; I.2.6 [Artificial Intelligence]: Learning; I.2.8 [Artificial Intelligence]: Problem Solving, Control Methods, and Search. General Terms Algorithms, Performance, Experimentation. Keywords Estimation of Distribution Algorithms, Bayesian Optimization Algorithm, Scoring Metrics, Local Structures. 1. INTRODUCTION Estimation of distribution algorithms (EDAs) [8] are types of evolutionary algorithms that replace typical genetic operators with building a probabilistic model of promising solutions which encodes the interactions among the variables of the problem as dependencies. This information about variables dependencies along with their corresponding parameters are later used to sample new solutions to be inserted into the population. In this way, this class of algorithms is able to overcome the linkage learning problem of normal genetic algorithms where the building-blocks are required to be in a tight conjunction. By building-block, we mean partial solutions that are contained in the optimum solution. Bayesian optimization algorithm (BOA) [14, 15] is one of the successful EDAs that uses Bayesian network as its probabilistic model. Bayesian networks are capable of showing multi-variable dependencies and this makes them a proper model of choice for domains where high order dependencies exist between problem variables. In BOA a greedy algorithm is used for building the probabilistic model of promising solutions which in turn employs a scoring metric. The scoring metric evaluates a network and assigns a score which shows how well this network represents the underlying data (set of promising solutions). An important problem arise here is that among different metrics that are available for network evaluation which of them should be used and what are the consequences of using each of these metrics. There are very few works in the literature that have considered this problem, but none of them has discussed it directly, they only have studied the combinatorial effect of metrics and other research issues. In [15] the authors tried to address this problem by comparing different metrics that are used in BOA for network evaluation. However they introduced decision graphs to the algorithm and mainly discussed its influence on BOA while only integrating one of the metrics with this version of BOA. Also, they did not include any discussion and analysis about the quality of the built models. Another important issue is the local structures which can be used to store network parameters. Bayesian network structure is represented by a directed acyclic graph (DAG) which encodes conditional dependencies and independencies between problem variables [2, 13]. But there are different local structures that can be used to store the network parameters (corresponding conditional probabilities): simple conditional probability table, default table, decision tree and decision graph. Again few works are reported which have studied these local structures and their effect on algorithm performance. This paper attempts to concentrate solely on the effect of popular local structures and different scoring metrics used in BOA. First of all a brief introduction to BOA and Bayesian networks are given in the next section. The local structures that are going to be used in our experiments are discussed in section 3. The scoring metrics are mentioned in Section 4. In section 5 the results obtained for model accuracy and algorithm performance are presented and analyzed and finally in section 6 conclusion remarks are presented. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. GEC’09, June 12–14, 2009, Shanghai, China. Copyright 2009 ACM 978-1-60558-326-6/09/06...$5.00. 263