Evolutionary Optimization for Computationally expensive problems using Gaussian Processes Mohammed A. El-Beltagy Andy J. Keane Computational Engineering and Design Centre Computational Engineering and Design Centre University of Southampton University of Southampton Highfield, Southampton SO17 1BJ Highfield, Southampton SO17 1BJ United Kingdom United Kingdom mohammed@computer.org Andy.Keane@soton.ac.uk Abstract The use of statistical models to approximate detailed analysis codes for evolutionary optimization has attracted some attention [1-3]. However, those early methodologies do suffer from some limitations, the most serious of which being the extra tuning parameter introduceds. Also the question of when to include more data points to the approximation model during the search remains unresolved. Those limitations might seriously impede their successful application. We present here an approach that makes use of the extra information provided by a Gaussian processes (GP) approximation model to guide the crucial model update step. We present here the advantages of using GP over other neural-net biologically inspired approaches. Results are presented for a real world-engineering problem involving the structural optimization of a satellite boom. Keywords: Evolutionally Computation, Optimization, Gaussian Processes, Computationally expensive problems. 1 Introduction The optimization of complex high dimensional, multimodal problems often requires a relatively high number of function evaluations. In many real world problems, this computational burden cannot be afforded. Examples of such problems include large- scale finite element analysis (FEA) or computational fluid dynamics (CFD) simulations. In such problems, the cost of a single function evaluation is in the order of hours of supercomputer time. In is been proven useful to build approximate models of the expensive analysis code and use it for the purpose of carrying out optimization [4]. These approximate models are orders of magnitude cheaper to run than the full analysis codes. Many regression and interpolation tools could be used to construct such an approximation (e.g. least square regression, back propagating artificial neural net, response surface models, etc.). In the multidisciplinary optimization (MDO) community the main focus has been on using response surface analysis and polynomial fitting techniques to build the approximate models. These methods have been proven to work well when single point traditional gradient-based optimization methods are used. However, they cannot cope with large dimensional multimodal problems since they generally carry out approximation using simple quadratic models. [5-11]. During the optimization, it is only feasible that the expensive analysis be used sparingly. This requirement clashes with that of good approximate model building, where as many sampled points are needed to obtain a good approximation. There is hence a need to develop a framework that balances those two requirements. Alexandrov et al [12] presented an approximation management framework for conducting gradient-based search. In our earlier work [3], we presented a framework for approximate model update based on fitness and design of experiments (DOE) criteria. Unlike Ratle’s work [1, 2], where only the most recently evaluated points were used to construct the approximation, in our approach the approximation model was continuously expanding; retaining all useful information provided by the evolutionary search. To limit the cost of model reconstruction, Ratle discarded most of the accurately evaluated points while updating his approximate model. It is our view that for problems of sufficient complexity and computational expense, the risk involved is discarding potentially useful points from the approximate model far outweigh the computational savings gained by discarding them. Like Ratle, a major drawback of our approach is that there were up to five extra tuning parameters to adjust for model construction and update. In this paper a present a far more elegant and flexible approach for carrying approximate model update. We use here a Gaussian process approximation model, since it has several attractive features. Most important of which is the ability to provide an error bar for each prediction. An algorithm for