10 Annealing Stochastic Approximation Monte Carlo for Global Optimization Faming Liang Department of Statistics Texas A&M University USA 1. Introduction During the past several decades, simulated annealing (Kirkpatrick et al., 1983) and the genetic algorithm (Holland, 1975; Goldberg, 1989) have been applied successfully by many authors to highly complex optimization problems in different fields of sciences and engineering. In spite of their successes, both algorithms suffer from some difficulties in convergence to the global optima. Suppose that we are interested in minimizing the function U(x) over a given space X. Throughout this article, U(x) is called the energy function in terms of physics. Simulated annealing works by simulating a sequence of distributions defined as where τ k is called the temperature. The temperatures form a decreasing ladder τ 1 >…> τ k > … with τ 1 being reasonably large such that the Metropolis-Hastings (MH) moves (Metropolis et al., 1953; Hastings, 1970) have a high acceptance rate at this level and lim k→∞ τ k = 0. It has been shown by Geman and Geman (1984) that the global minima of U(x) can be reached by simulated annealing with probability 1 if the temperature decreases at a logarithmic rate. In practice, this cooling schedule is too slow; that is, CPU times can be too long to be affordable in challenging problems. Most frequently, people use a linearly or geometrically decreasing cooling schedule, which can no longer guarantee the global minima to be reached. The genetic algorithm solves the minimization problem by mimicking the natural evolutionary process. A population of candidate solutions (also known as individuals), generated at random, are tested and evaluated for their energy values; the best of them are then bred through mutation and crossover operations; the process repeated over many generations, until an individual of satisfactory performance is found. The mutation operation is modeled by random perturbations of the individuals. The crossover operation is modeled by random perturbations of the couples formed by two individuals selected according to some procedure, e.g., a roulette wheel selection or a random selection. Through the crossover operation, the solution information distributed across the population can be effectively used in the minimization process. Schmitt (2001) showed that under certain conditions, the genetic algorithm can converge asymptotically to the global minima at a logarithmic rate in analogy to simulated annealing. Source: Simulated Annealing, Book edited by: Cher Ming Tan, ISBN 978-953-7619-07-7, pp. 420, February 2008, I-Tech Education and Publishing, Vienna, Austria Open Access Database www.i-techonline.com www.intechopen.com