MINIMUM MEAN SQUARE ESTIMATION AND NEURAL NETWORKS Michael T. Manry 1 , Steven J. Apollo 2 , and Qiang Yu 3 1 Department of Electrical Engineering, University of Texas at Arlington, Arlington, Texas 76019 2 Lockheed Fort Worth Company, Mail Zone 2615 P.O. Box 748 Fort Worth, Texas 76101 3 Worldcom, Inc. 1 William Center Tulsa, Oklahoma 74101-0949 Abstract Neural networks for estimation, such as the multilayer perceptron (MLP) and functional link net (FLN), are shown to approximate the minimum mean square estimator rather than the maximum likelihood estimator or others. Cramer-Rao maximum a posteriori lower bounds on estimation error can therefore be used to approximately bound network training error, when a statistical signal model is available for its inputs and the desired outputs are Gaussian. The bounds help the user to determine when to stop training, and to determine how close to optimal the neural net’s performance is. When a linear preprocessor is sought to compress raw data, before it is input into a neural network, the bounds can be used to determine the relative optimality of several candidate linear preprocessors or transforms. A method is proposed for re-ordering the rows of the preprocessor’s transform matrix. It is shown that a single linear transformation can be used, even when more than one parameter is estimated by the network. Published in : Neurocomputing, vol. 13, September 1996, pp. 59-74.