 This paper was not presented at any IFAC meeting. This paper was recommended for publication in revised form by Associate Editor S. Sagara under the direction of Editor Torsten So K derstro K m. * Corresponding author. Tel.: #32-10-472-382; fax: #32-10-472- 180. E-mail address: david@auto.ucl.ac.be (B. David). Automatica 37 (2001) 99}106 Brief Paper An estimator of the inverse covariance matrix and its application to ML parameter estimation in dynamical systems  B. David*, G. Bastin Center for Systems Engineering and Applied Mechanics, Universite & catholique de Louvain, Av. G. Lemaitre 4, B1348 Louvain-La-Neuve, Belgium Received 19 November 1998; revised 4 February 2000; received in "nal form 29 May 2000 Abstract An exact formula of the inverse covariance matrix of an autoregressive stochastic process is obtained using the Gohberg}Semencul explicit inverse of the Toeplitz matrix. This formula is used to build an estimator of the inverse covariance matrix of a stochastic process based on a single realization. In this paper, we show that this estimator can be conveniently applied to maximum likelihood parameter estimation in nonlinear dynamical system with correlated measurement noise. The e$ciency of the estimation scheme is illustrated via Monte-Carlo simulations. It is shown that the statistical properties of the estimated parameters are largely improved using the proposed inverse covariance matrix estimator in comparison to the classical variance estimator.  2000 Elsevier Science Ltd. All rights reserved. Keywords: Parameter estimation; Maximum likelihood; Covariance matrix; Correlated residuals; Nonlinear system 1. Introduction This paper deals with parameter estimation in a dynamical system. The system under consideration is described by a phenomenological model which is based on the prior knowledge of the physical phenomena that are supposed to take place in the system. The model is written under the form of a di!erential parametric deter- ministic state-space representation: x  "f (x, , u), x3, u3, 3, (1) where x"[x  , 2 , x  ] is the state vector, u" [u  , 2 , u  ] the input vector and "[  , 2 ,   ] the vector of parameters. The parameter estimation problem is to estimate the parameter values from input and state data in the presence of additive correlated noise on the state measurements. During the last 30 years, a number of publications were devoted to maximum likelihood (ML) estimation of system parameters. We should especially mention the work of Schoukens, Pintelon and coworkers who have deeply analyzed many aspects of ML estimation and in particular the role of the noise model. Most of their works deal with linear systems, represented by their transfer function, or particular forms of nonlinear sys- tems, typically represented by Volterra series (Schoukens, Pintelon & Renneboog 1988). For instance, the in#uence of estimating the covariance matrix of the noise by its sample value is studied in Schoukens, Pintelon, Vander- steen and Guillaume (1997) and it has been recently shown (Schoukens, Pintelon & Rolain, 1999) that in certain case, the full covariance matrix can be replaced by its main diagonal. Even though the frequency domain approach allows an elegant treatment of the noise model in ML parameter estimation, it is obviously not applic- able to estimate the parameters of a model that does not have a parametrized frequency domain representation which is the case of (1). In this paper, we address the problem of estimating the parameters of (1) tacking into account the correlation that the output measurement noise may exhibit, using the ML framework in the time domain. The problem is stated as follows. We assume that an experiment has been performed with a known input signal (often it is a piecewise constant signal) and that 0005-1098/01/$ - see front matter  2000 Elsevier Science Ltd. All rights reserved. PII: S 0 0 0 5 - 1 0 9 8 ( 0 0 ) 0 0 1 2 7 - 8