Simulation of Markovian models using Bootstrap method Ricardo M. Czekster, Paulo Fernandes, Afonso Sales, Dione Taschetto and Thais Webber Faculdade de Inform´ atica – Pontif´ ıcia Universidade Cat´ olica do Rio Grande do Sul Av. Ipiranga, 6681, Pr´ edio 32 – 90619-900 – Porto Alegre – RS – Brasil {ricardo.czekster, paulo.fernandes, afonso.sales, dione.taschetto, thais.webber}@pucrs.br Keywords: Markovian models, Discrete event simulation, Bootstrap method Abstract Simulation is an interesting alternative to solve Markovian models. However, when compared to analytical and numeri- cal solutions it suffers from a lack of precision in the results due to the very nature of simulation, which is the choice of samples through pseudorandom generation. This paper pro- poses a different way to simulate Markovian models by using a Bootstrap-based statistical method to minimize the effect of sample choices. The effectiveness of the proposed method, called Bootstrap simulation, is compared to the numerical so- lution results for a set of examples described using Stochastic Automata Networks modeling formalism. 1. INTRODUCTION Modeling formalisms are usually employed to describe real systems, capturing their behavior. An example of a well- known modeling formalism used for such purpose is Markov Chains (MC) [20]. MC models use simple primitives such as states and labeled transitions to exemplify system’s evolution and operational semantics. Besides Computer Science abun- dant applications, Markov modeling examples are present in bioinformatics, economics, engineering and chemistry, to name a few domains [20]. However, the major limitation of MC is the state space ex- plosion problem, observed when the amount of possible con- figurations is massively huge, rendering solution intractable and easily depleting computational resources [20]. In order to mitigate this problem, structured formalisms are proposed such as Stochastic Automata Networks (SAN) [9], General- ized Stochastic Petri Nets [6] or Stochastic Process Algebras (for example, Extended Markovian Process Algebra [3] or Performance Evaluation Process Algebra [12]), among sev- eral other approaches. Every structured formalism has a cor- respondent underlying MC and, in a particular way, SAN presents an inner representation based on tensor operations, producing a highly memory efficient fashion to map its under- lying transition system. SAN also enables complex modeling due to sophisticated primitives (such as events) to define the firing of transitions, locally or synchronously, through con- stant or functional rates (rates dependent on the other compo- nent states). Beyond model mapping, its numerical solution plays an equally important role when examining complex systems. There are distinct ways to solve any given model, e.g., us- ing numerical methods or simulation. Numerical methods often rely on iterative mathematical techniques such as the Power Method [20], Arnoldi [1] or GMRES [16] to calcu- late measures of interest. On the other hand, simulation tech- niques consider the model events, firing transitions according to pseudorandom numbers generation. Simulation precision is strongly related to the number of samples that are produced. Normally, a simulation study comprehends large amounts of time in order to obtain significant performance indices, usu- ally in function of the model state space size. Although numerical solution produces reliable results, it is bounded by the number of states of the model. Current nu- merical methods are directed to research on the acceleration of iterative methods [9]. However, methods based on Perfect Samplings for SAN have been used with satisfactory results, generating unbiased samples [10]. Another alternative is to use pure traditional simulation methods for Markovian mod- els [11]. Such methods are based on simulating sets of trajec- tories and counting the amount of visits for each state, where every step produces a sample (saved for further analysis). When using traditional simulation methods one must be concerned about result quality, i.e., precision. Since every simulation trajectory only approximates the numerical solu- tion, a thoroughly data inspection must be conducted, veri- fying results and searching for imprecise results. Succinctly, traditional simulation starts from an arbitrarily chosen initial state and randomly walks on a predefined state space during a fixed quantity of steps (trajectory length). Ideal length for a trajectory can be estimated using confidence intervals, among other approaches. Since every visited state generates a sam- ple, dividing the number of times a state has been visited by the trajectory length gives an approximation of the mean per- manence probability for each state. Bootstrap methods were firstly proposed by Efron [8] to perform estimations applied to different areas (for instance, machine learning algorithms [2]). The main idea is to reduce the “noise” that were observed from the samples. In the con- text of simulation, this noise corresponds to the noticed error in relation to the numerical solution. The method consists on resamplings, where each sample maps not only one state (the common approach in the traditional simulation) but a set of