IMPORTANCE SAMPLING: APPROACHES AND MISCONCEPTIONS José R. Gallardo * , Dimitrios Makrakis * , Luis Orozco-Barbosa ** note * Advanced Communications Engineering Centre – ACEC. Department of Electrical and Computer Engineering. The University of Western Ontario. London, Ontario, Canada N6A 5B9 Tel: (519) 679-2111 ext. 8335, 8243. Fax: (519) 661-3488. E-mail: gallajo, dimitris @engga.uwo.ca ** Department of Electrical Engineering. University of Ottawa. Ottawa, Ontario, Canada K1N 6N5 Tel: (613) 562-5800 ext. 6224. Fax: (613) 562-5175. E-mail: lbarbosa@uottawa.ca Additional author information --- J.R.G.: Also Ph.D. Candidate at the Electrical Engineering Department, School of Engineering and Applied Science, The George Washington University, Ashburn, VA, USA 20147. ABSTRACT A brief introduction to the widely spread theory of Importance Sampling is given. Several approaches to the optimization of the twisted density function are described, such as: i) using the ideal (zero-variance) estimator as a guideline, ii) increasing the probability of the event to be analyzed, iii) splitting the random process trajectory, iv) sequentially adapting the twisted density, and v) using uniformly bounded likelihood ratios. A brief description of the regenerative approach to simulations is given as well. Some details that should be taken into consideration when applying these techniques, in order to avoid a poor performance of the estimator, are also explained. 1. INTRODUCTION Monte Carlo methods were originally proposed by nuclear physics researchers during the 40’s as techniques to evaluate integrals that were very difficult to deal with both analytically and numerically. These methods rely on the fact that sample mean values approximate theoretical expectations when a sufficiently large number of samples are taken. As the systems being analyzed in several fields of knowledge (including economics, civil engineering, and telecommunications, to mention just a few) grew in complexity, Monte Carlo methods proved to be very effective to estimate reliability parameters. These systems, however, tend to be more complex every day and the probabilities of the events that need to be analyzed keep moving towards values that are too small to be dealt with properly using traditional Monte Carlo simulations. Importance Sampling techniques are then introduced, aimed at increasing the efficiency of Monte Carlo simulations and reducing processing time. The application of Importance Sampling techniques to speed up simulations has been an area of active research for a number of years now. The fact that there is still work being done in the area tells us about the difficulty to find an approach that is general enough to be applied in a variety of environments giving acceptable performance. The goal of this discussion is to summarize a number of concepts that have not been considered with the emphasis they deserve in the literature and which are essential to achieve a satisfactory estimator performance. The examples that will be given below are related to the telecommunications field, but otherwise, the ideas are equally applicable to any area where Importance Sampling techniques are implemented. For instance, we talk about buffer overflows in communications switches or the existence of ATM cells with excessive delays; we could just as well have mentioned the breakdown of a highly reliable production line or a defective product being detected during a quality test. 2. IMPORTANCE SAMPLING Importance sampling is one of the classical techniques for increasing the efficiency of Monte Carlo simulations [2] [7]. The basic idea is to modify the system under study by replacing one of the stochastic processes involved with a new one in order to reduce the variance of the estimator. That is usually achieved by increasing in an intelligent way the probability of occurrence of the events of interest. The estimated statistics that result from the simulation are then transformed (unbiased) to make them correspond to the original system. To be more specific, assume that we have a system whose behaviour depends on the stochastic process W and we want to estimate the expected value of a certain random variable X(W). The process W can represent the random input traffic to an ATM switch and X(W) can be the cell loss ratio or the proportion of cells with excessive delay. Then: ( 29 [ ] ( 29 ( 29 w w w W W U W W d f X X E ⋅ ⋅ = ∫ (1)