IEEE TRANSACTIONS ON MAGNETICS, VOL. 36, NO. 4,JULY 2000 1061 Hybrid Optimization in Electromagnetics Using Sensitivity Information from a Neuro-Fuzzy Model Kashif Rashid, Jaime A. Ramírez, and Ernest M. Freeman Abstract—The use of sensitivity information from a neuro-fuzzy model for the purpose of optimization is investigated in this paper. This approach permits the application of classic deterministic or hybrid optimization methods in establishing the global minimum of any approximated objective function using neuro-fuzzy mod- eling. For nondifferentiable functions this approach is of great ben- efit. An analytical problem and the TEAM 22 benchmark problem are investigated. Results using the genetic algorithm method and the sequential quadratic programming method in sequence show the usefulness of the formulation. Index Terms—Artificial intelligence, fuzzy neural networks, op- timization methods, electromagnetic analysis. I. INTRODUCTION I N engineering the objective function of complex multi-objective problems is commonly nonlinear, con- tinuous, multivariate and constrained, which often is not readily differentiable. Such problems are typically optimized using stochastic methods (simulated annealing, etc.) [1]–[3], and deterministic methods (conjugate gradient methods, etc.) if at all applicable, tend to become trapped in local minima [4]. Stochastic methods however, have the disadvantage of requiring a much higher number of function evaluations in order to identify the global minimum. A means to overcome this high computational burden is to provide an approximation of the objective function such that future evaluations can be established at reduced cost [5]–[9]. These global response surface methods (GRSM’s) are particularly suited for tackling multi-minima problems [5]. Although, the search is undertaken faster at a lower computational cost, the global minimum is often not exactly found and the result is not always reproducible in the same number of finite steps due to the very nature of the random search process. This has led to the development of hybrid strategies, where the optimization process is started with a stochastic method and after some iterations is switched to a deterministic method. This paper examines the development of hybrid optimization strategies using sensitivity information derived from the fuzzy Manuscript received October 25, 1999. This work was supported by EPSRC, UK and in part by CNPq (Grant nos. 300353/97–9 and 451689/99) and by PUC- Minas, Brazil. K. Rashid and E. M. Freeman are with the Department of Electrical & Elec- tronic Engineering, Imperial College of Science, Technology & Medicine, Ex- hibition Road, London SW7 2BT, UK. J. A. Ramírez is with PUC-Minas, Programa de Pós-Graduação em Eng. Elétrica, Av. Dom José Gaspar, 500, 30535-610 Belo Horizonte, MG, Brazil. Publisher Item Identifier S 0018-9464(00)06699-1. rule set of an approximated objective function using a neuro- fuzzy model [10], [11]. II. THE OPTIMIZATION MODEL A trained neuro-fuzzy model, one which can predict the output of the model for any combination of input parameters, can be used to as an alternative to future experimental or finite element based simulations [12]. This empirical model provides an approximation to the real objective function and can be optimized using either stochastic or deterministic methods. Fig. 1 shows the optimization model in which a neuro-fuzzy model is designed and optimized iteratively. Initially the system under investigation is sampled in param- eter-space at iteration . The location and number of samples is dictated by the sampling strategy; grid based, random, ge- netic or dimensional sampling, along with the desired number of points. The sampling matrix is used to initiate a function call to the function generator, which can be a computational source, a physical device or a combination of the two. The sam- pled matrix, describing the input–output relationship between model variables, is used to train a neuro-fuzzy model. Gener- ally, the sampled matrix is partitioned into two sets; training and checking data. The neuro-fuzzy model is designed using training data and validated using checking error to ensure the model is not trained to over fit the model data. This empirical model then effectively replaces the actual function generator in the optimization process, obviating the need to run further com- putationally expensive simulations. Thus, stochastic methods can be applied with relatively little cost since approximated functions evaluations are made at a fraction of the cost in comparison to the real function generator. Moreover, by coupling a deterministic method, the start point of which is the locality of the global minimum identified by the stochastic method, a hybrid approach can be implemented. An efficient and accurate search results since the time spent run- ning a stochastic search is minimized and the pitfalls associated with poorly assigned starting points in a deterministic method are also overcome. The process is repeated with further function calls made to refine the area of interest until the convergence cri- terion is met. The final result is the perceived optimum. Note that model sampling incurs the greatest time and compu- tational cost which could be alleviated with the use of parallel processors or overcome if data were available directly from a design knowledge base. Also note, this paper deals with evenly partitioned neuro-fuzzy models in which the number and type of membership functions assigned to each input can be pre-defined by the user. 0018–9464/00$10.00 © 2000 IEEE