Proceedings of the 1998 Winter Simulation Conference D.J. Medeiros, E.F. Watson, J.S. Carson and M.S. Manivannan, eds. IDENTIFYING IMPORTANT FACTORS IN DETERMINISTIC INVESTMENT PROBLEMS USING DESIGN OF EXPERIMENTS Willem J.H. Van Groenendaal Jack P.C. Kleijnen Department of Information Systems and Auditing/Center for Economic Research (CentER) School of Management and Economics Tilburg University 5000 LE Tilburg, THE NETHERLANDS ABSTRACT For large investment projects sensitivity analysis is an important tool to determine which factors need further analysis and/or can jeopardize the future of a project. In practice reliable information on the joint probability distribution of factors affecting the investment is mostly lacking, so a stochastic analysis is not possible. This paper analyzes how and to what extend statistical design of experiments in combination with regression meta modeling can be helpful in finding important factors in deterministic models. Information that is useful to decision makers. 1 INTRODUCTION In practice, deciding on investment in infrastructure uses the Net Present Value (NPV); that is, a necessary condition to accept an investment proposal is that the NPV be not negative. In developing countries this criterion is used for investments financed by development aiding agencies (World Bank, Asian Development Bank). In this paper we address the problem of uncertainty in the model s inputs and parameters, further referred to as factors. In practice most models used to analyze investments are deterministic, because no or only limited information is available on the (joint) distribution of the factors. An additional question is: Which factors can make a project go wrong ; that is, which factors may cause NPV < 0 Decision makers ask for this type of information to support their decision making process; see Van Groenendaal (1998b). Note that information on which factors affect the NPV is useful also to evaluate implementation progress after the decision to proceed has been taken. In applied work sensitivity analysis is limited to one factor at a time in combination with a few scenarios. For this three data points per factor are required: the base case value, and a minimum and maximum value. The resulting information is, however, insufficient to meet the decision makers needs. Van Groenendaal and Kleijnen (1997) and Van Groenendaal (1998a) suggest to apply the statistical theory on design of experiments in combination with regression meta-modeling (further referred to as DOE) for sensitivity analysis of deterministic models. This approach requires the same information on factors as the currently used methods. DOE is typically applied in a constructive way; that is, one starts with a simple design and estimates a simple meta-model. For example, first use a design to identify important (main) effects and to see if there are possible interactions. Only if the estimation results indicate other effects, a more complicated design is introduced. This approach is chosen to minimize the amount of work required. In this paper we explain the different steps of DOE for deterministic models and discuss some of the hazards. To keep our analysis manageable we use a rather simple deterministic investment model, based on work done for the Asian Development Bank (ADB, 1996), instead of a complicated one. The remainder of this paper is organized as follows. Section 2 discusses DOE in more detail. Section 3 reviews the NPV model used as a test case. Section 4 applies DOE. Section 5 contains conclusions. 2 DESIGN OF EXPERIMENTS As argued by Van Groenendaal (1998b), the NPV-analysis of an investment problem has a typical form. Many inputs need not to be analyzed separately, but can be combined. The way they are introduced in the NPV-analysis acts as a funnel. An example is the analysis of investment cost, which in most cases is based on many inputs. In the calculation of the NPV the aggregated cost is used. It is not necessary to vary all separate inputs affecting the investment cost, the variation in the total cost suffices. If 713