Proceedings of the 2005 Winter Simulation Conference M. E. Kuhl, N. M. Steiger, F. B. Armstrong, and J. A. Joines, eds. STOCHASTIC OPTIMIZATION USING MODEL REFERENCE ADAPTIVE SEARCH Jiaqiao Hu Michael C. Fu Steven I. Marcus Institute for Systems Research University of Maryland College Park, MD 20854, U.S.A. ABSTRACT We apply the recently introduced approach of model refer- ence adaptive search to the setting of stochastic optimization and report on simulation results. 1 INTRODUCTION We consider an optimization problem of the form: x ∗ ∈ arg max x∈X E ψ [H(x,ψ)], x ∈ X ⊆ℜ n , (1) where X is the solution space, which can be either continuous or discrete, H(·, ·) is a deterministic, real-valued function, and ψ is a random variable (possibly depending on x ) representing the stochastic effects of the system. We let h(x) := E ψ [H(x,ψ)], and assume h(x) cannot be obtained easily, but instead, only the random variable H(x,ψ) can be observed, e.g., via simulation, which makes (1) much more difficult to solve than its deterministic counterpart (cf. Fu 2002). We assume (1) has a unique global optimal solution. In general, depending on the structure of the underlying solution space, the techniques for stochastic optimization problems could be quite different. When the solution space is continuous, there is a well-known class of methods called stochastic approximation (SA) for solving such problems. These methods rely on the estimation of the gradient of the objective function with respect to the decision variables; thus they generally find local optimal solutions. In terms of the different gradient estimation techniques used, the SA methods can be further characterized into different cate- gories. Detailed reviews can be found in e.g., Fu (1994, 2005). For problems with discrete decision variables, various randomized search methods have been suggested, including the stochastic ruler method (Yan and Mukai 1992; Alrefaei and Andradòttir 2001), random search methods (Andradòttir 1995, 1996; Hong and Nelson 2005), simulated annealing (Alrefaei and Andradòttir 1999), and nested partitions (Shi and Òlafsson 2000). Andradòttir (2005) and Òlafsson (2005) contain recent reviews of these techniques. This paper presents a new unified approach called Stochastic Model Reference Adaptive Search (SMARS) for solving simulation-based optimization problems with either continuous or discrete solution spaces. SMARS is the gener- alization of the Model Reference Adaptive Search (MRAS) method for deterministic optimization introduced in Hu, Fu, and Marcus (2005a) with some appropriated modifi- cations and extensions required for the stochastic setting. The method works with a parameterized probability distri- bution on the solution space and generates at each iteration a group of candidate solutions. These candidate solutions are then used to update the parameters associated with the distribution so that the future search will be biased toward the region containing high quality solutions. For complete technical developments of the approach and convergence proofs, the interested readers are referred to Hu, Fu, and Marcus (2005b). The rest of the paper is structured as follows. In Section 2, we describe the proposed SMRAS method. In Section 3, we present the global convergence property of SMRAS. Illustrative numerical examples on both continu- ous and discrete domains are given in Section 4. Finally Section 5 contains some concluding remarks. 2 ALGORITHM The algorithm is summarized in Figure 1. For a specified pa- rameterized family of distributions {f(·,θ),θ ∈ }, where  is the parameter space, the main body of the algorithm consists of the following steps: (1) generate candidate solutions according the current distribution, say f(·,θ k ); 811