Proceedings of the 2010 Winter Simulation Conference B. Johansson, S. Jain, J. Montoya-Torres, J. Hugan, and E. Yücesan, eds. SIMULATION EXPERIMENT DESIGN Russell R. Barton The Pennsylvania State University Department of Supply Chain and Information Systems University Park, PA U.S.A. ABSTRACT So you have built and validated a simulation model  how are you going to gain insight about the asso- ciated real system in order to make decisions? This introductory tutorial gives an overview of experiment design techniques for planning a series of simulation runs. These techniques make efficient use of simula- tion runs to uncover the impact of system design parameters on simulation output performance. The tu- torial highlights graphical methods for planning the experiment and displaying the results. 1 INTRODUCTION Discrete-event simulation modeling is a popular method for predicting the performance of complex sys- tems, particularly systems that include random phenomena. Simulation projects can fall short of their in- tended goals, however, unless the simulation model is exercised intelligently to gain useful understanding of the likely performance of the real system. This is where the design of simulation experiments plays a key role. Usually, simulation projects are conducted within time and budget limits. Often the bulk of time and resources are spent on building and validating the model, with little time or budget in the schedule to exercise the model for decision-making insight. This is risky, since poorly planned simulation runs can result in a significant loss of information, or worse, provide misleading results. Further, the kinds of decisions the simulation model will aid should be decided up front, since model construction, verification and validation depend on this information (Sargent 2009). This tutorial presents a five-step process for the design of a simulation experiment. Graphical me- thods are emphasized for the first four steps, drawing largely from Barton (1999). A hypothetical simula- tion project for a die-making machine shop will help to illustrate each step. The tutorial is an updated ver- sion of that in Barton (2002; 2004) with additional discussion on screening and optimization. Introduction to the design of simulation experiments is presented from different perspectives in the WSC papers by Sanchez and Wan (2009) and Kleijnen (2008b). The next section describes the limits of the topics cov- ered, defines the five-step process, and describes the machine shop scenario. Sections 3-7 describe each step in more detail and illustrate the activities for the machine shop simulation. The next section shows how to use the graphical framework to present results. In some cases the graphical display provides more insight than an ANOVA table or regression analysis. Section 9 covers some remaining issues. 75 978-1-4244-9864-2/10/$26.00 ©2010 IEEE