IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 45, NO. 4, APRIL 2000 653 Information States in Stochastic Control and Filtering: A Lie Algebraic Theoretic Approach Charalambos D. Charalambous and Robert J. Elliott Abstract—The purpose of this paper is twofold: i) to introduce the sufficient statistic algebra which is responsible for propagating the sufficient statistics, or information state, in the optimal con- trol of stochastic systems and ii) to apply certain Lie algebraic methods and gauge transformations, widely used in nonlinear con- trol theory and quantum physics, to derive new results concerning finite-dimensional controllers. This enhances our understanding of the role played by the sufficient statistics. The sufficient statistic al- gebra enables us to determine a priori whether there exist finite-di- mensional controllers; it also enables us to classify all finite-dimen- sional controllers. Relations to minimax dynamic games are delin- eated. Index Terms—Feynman–Kac, filtering, finite-dimensional, gauge transformations, Lie algebras, minimax, partial observa- tions, stochastic control, sufficient statistic. I. INTRODUCTION T HE PRACTICAL utility of the Duncan–Mortensen–Zakai (DMZ) equation is greatly appreciated in both nonlinear filtering and stochastic control problems with partial infor- mation. The DMZ equation of nonlinear filtering of diffusion processes is a linear, stochastic, partial differential equation (PDE) which describes in a recursive manner the evolution of the unnormalized conditional distribution of the state process, , given the observations, . If this distribution has a density function, say, , then [1] (1.1) Consequently, evolves forward in time with initial condition . Here, is a certain second-order differential operator related to the drift and diffusion coefficients of the state process, the Kolmogorov forward operator, and is a zero-order differential operator related to the signal in the observations. Manuscript received May 2, 1997; revised July 2, 1998, February 2, 1999, and May 6, 1999. Recommended by Associate Editor, G. G. Yin. This work was supported by the Natural Science and Engineering Research Council of Canada under Grant OGP0183720. C. D. Charalambous was with the ECE Department, McGill University, Mon- treal, Canada. He is now with the School of Information Technology and Engi- neering, University of Ottawa, Ottawa, ON, K1N 6N5 Canada. R. J. Elliott is with the Department of Mathematical Sciences, University of Alberta, Edmonton, Alberta, T6G 2G1 Canada. Publisher Item Identifier S 0018-9286(00)03237-2. In filtering problems one is concerned with conditional ex- pectations (1.2) where is the normalized con- ditional density. Therefore, it is of interest to classify problems for which (1.1) can be solved explicitly, as in the Kalman–Bucy case, in terms of finite-dimensional systems. Motivated by an analogous affine control system, namely, ,where is the control input, Brockett [2], Brockett and Clark [3], and Mitter [4] proposed that the Lie algebra methods widely used to analyze such systems might also be of important interest in classifying finite-dimensional filtering problems as well. In particular, they proposed that such progress relies on the finite-dimen- sionality of the Lie algebra generated by the operators . Moreover, Ocone [5] noted that if the Lie algebra generated by the operators , is finite-dimensional, then (under certain conditions) the Wei–Norman method may be used to derive the structure of the recursive filters. Ocone’s observation is made rigors by Tam et al. [6]. Subsequent generalizations are addressed by Yau [7]. An alternative approach to the Wei–Norman method is the Kallianpur–Striebel function space integration approach. For an earlier reference on these methods see Benes [8]. Some important early references addressing these issues are [5], [8], and [9]. Later references achieving additional generalizations of the Benes filter are [10], [6], [11], and [12]. In all these references the observations are linear in the state process. Recently, an important class of filtering problems with nonlinear observations is identified in [13]. The importance of nonlinear filtering in stochastic control of partially observed systems was elegantly broached by Mortensen [14]. Mortensen reformulated this problem as one of complete information, in which the control is a functional of an information state; the information state satisfies a controlled version of the DMZ equation. Using Mortensen’s formulation, Charalambous et al. [15] noted that finite-dimensional con- trollers can be constructed for the control analog of the Benes filter. Additional references addressing finite-dimensional controllers are [16]–[19]. One of the interesting aspects of [17] and [19] is that these nonlinear control problems are, within a gauge transformation, equivalent to linear-quadratic Gaussian 0018–9286/00$10.00 © 2000 IEEE