JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS: Vol. 112, No. 2, pp. 403–439, February 2002 (2002) Approximating Networks and Extended Ritz Method for the Solution of Functional Optimization Problems 1 R. ZOPPOLI, 2 M. SANGUINETI, 3 AND T. PARISINI 4 Communicated by Y. C. Ho Abstract. Functional optimization problems can be solved analytically only if special assumptions are verified; otherwise, approximations are needed. The approximate method that we propose is based on two steps. First, the decision functions are constrained to take on the struc- ture of linear combinations of basis functions containing free param- eters to be optimized (hence, this step can be considered as an extension to the Ritz method, for which fixed basis functions are used). Then, the functional optimization problem can be approximated by nonlinear programming problems. Linear combinations of basis functions are called approximating networks when they benefit from suitable density properties. We term such networks nonlinear (linear) approximating networks if their basis functions contain (do not contain) free param- eters. For certain classes of d-variable functions to be approximated, nonlinear approximating networks may require a number of parameters increasing moderately with d, whereas linear approximating networks may be ruled out by the curse of dimensionality. Since the cost functions of the resulting nonlinear programming problems include complex aver- aging operations, we minimize such functions by stochastic approxi- mation algorithms. As important special cases, we consider stochastic optimal control and estimation problems. Numerical examples show the effectiveness of the method in solving optimization problems stated in 1 This work was supported in part by the MURST Project on Identification and Control of Industrial Systems. The authors are indebted to Angelo Alessandri, Angela Di Febbraro, and Simona Sacone for the assistance in developing the simulation examples. They thank P. C. Kainen and V. Ku ˚ rkova ´ for helpful discussions. 2 Professor, Department of Communications, Computer, and System Sciences, University of Genova, Genova, Italy. 3 Research Associate, Department of Communications, Computer, and System Sciences, University of Genova, Genova, Italy. 4 Professor, Department of Electrical, Electronic Engineering and Computer Engineering, DEEI-University of Trieste, Trieste, Italy. 403 0022-3239020200-04030  2002 Plenum Publishing Corporation