3426 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 48, NO. 12, DECEMBER 2000 The Stability of Nonlinear Least Squares Problems and the Cramér–Rao Bound Samit Basu, Member, IEEE, and Yoram Bresler, Fellow, IEEE Abstract—A number of problems of interest in signal processing can be reduced to nonlinear parameter estimation problems. The traditional approach to studying the stability of these estimation problems is to demonstrate finiteness of the Cramér–Rao bound (CRB) for a given noise distribution. We review an alternate, deter- minstic notion of stability for the associated nonlinear least squares (NLS) problem from the realm of nonlinear programming (i.e., that the global minimizer of the least squares problem exists and varies smoothly with the noise). Furthermore, we show that under mild conditions, identifiability of the parameters along with a finite CRB for the case of Gaussian noise is equivalent to deterministic stability of the NLS problem. Finally, we demonstrate the applica- tion of our result, which is general, to the problems of multichannel blind deconvolution and sinusoid retrieval to generate new stability results for these problems with little additional effort. Index Terms—Cramér–Rao bound, identifiability, nonlinear least squares, parameter estimation, stability, uniqueness. I. INTRODUCTION O FTEN, problems in signal processing involve the solu- tion of a nonlinear parameter estimation problem. Sensor array processing, fitting of splines to observational data, and filter design, are among a host of applications that can lead to nonlinear estimation problems. When confronted with such a problem, it is natural to ask questions such as “are the parame- ters uniquely determined by the data, i.e., identifiable,” and “is the problem stable in some sense?” Answering these questions is generally nontrivial, and requires careful study of the under- lying estimation problem. Furthermore, it may not seem obvious that these questions are independent, or even what questions need to be asked. To illustrate, we will examine two standard signal processing problems, multichannel blind deconvolution (MBD), and sinusoid retrieval (SR), and pose the relevant ques- tions in terms of these problems. The question of stability is often addressed in the signal pro- cessing literature from a stochastic standpoint. In particular, a nonlinear parameter estimation problem is considered stable if Manuscript received January 27, 1999; revised August 25, 2000. This work was supported in part by DARPA under Contract F49620-98-1-0498 adminis- tered by AFSOR and by a Fellowship from the Joint Services Electronics Pro- gram. The associate editor coordinating the review of this paper and approving it for publication was Dr. Alex B. Gershman. S. Basu is with the General Electric Corporate Research and Development Center, Niskayuna, NY 12309 USA (e-mail: basu@crd.ge.com). Y. Bresler is with the Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, Urbana, IL 61801 USA (e-mail: ybresler@uiuc.edu). Publisher Item Identifier S 1053-587X(00)10165-5. the CRB for an assumed noise distribution is finite. 1 For many problems of interest, this is a useful notion of stability, espe- cially since estimators like the MLE can asymptotically achieve the CRB under sufficient regularity conditions [1]. However, in the nonlinear programming literature, a different approach to stability is generally taken [2]. There, stability is ex- amined in a deterministic setting, and a problem is stable if there exists a sufficiently smooth map from the noisy observations to the global minimum of some fixed cost function. This notion of stability is also useful for estimation problems in which a tractable statistical noise model is not available. For example, in signal processing problems, signal dependent noises, such as quantization noise, modeling errors, and systematic errors can be difficult to handle under the stochastic notion of stability. In a deterministic setting, however, stability is guaranteed with re- spect to such errors, provided they are “small enough.” We review the concept of determinstic stability for the spe- cial case of a least squares cost function, and demonstrate that this notion of stability is equivalent (under mild regularity con- ditions) to the stochastic notion of stability for a Gaussian noise distribution. Thus, an identifiable nonlinear parameter estima- tion problem is stochastically stable in the presence of additive Gaussian noise if and only if the corresponding NLS problem is deterministically stable. When applied to the MBD and SR problems, we find that existing results on the stochastic stability and identifiability of both problems yield results in deterministic stability of their NLS formulations. A third form of stability that we do not consider is the notion of consistency of an estimator. This is another form of stochastic stability, but requires the amount of data to grow to infinity. Engineering applications with finite data do not fit well within this notion of stability. Instead, one generally has a model of the noise as “small” or bounded. Such a model is more appropriate for the two forms of stability considered in this paper. Good references on consistency and other asymptotic properties of estimators can be found in [1], [3]. II. SAMPLE APPLICATIONS As mentioned previously, we will use the problems of MBD and SR to motivate the study and illustrate the general behavior of NLS problems. In this section, we define the two problems as considered in this paper, and then pose questions that we would like to be able to answer about both problems. Answers to these 1 We will restrict our attention to parameter estimation problems that are suffi- ciently smooth to guarantee that the Fisher Information Matrix exists. Although stochastic stability through the CRB can be extended to less smooth problems, the extension is not straighforward, particularly when considering the joint es- timation of multiple parameters. 1053–587X/00$10.00 © 2000 IEEE