PII: S0005 – 1098(98)00044 – 2 Automatica, Vol. 34, No. 8, pp. 1031— 1034, 1998 1998 Elsevier Science Ltd. All rights reserved Printed in Great Britain 0005-1098/98 $19.00#0.00 Technical Communique Discrete-Time Optimal Control with Control-Dependent Noise and Generalized Riccati Difference Equations* ALESSANDRO BEGHI- and DOMENICO D’ALESSANDRO‡ Key Words—Discrete-time optimal control; control-dependent noise; generalized Riccati equations. Abstract—The optimal control law is derived for discrete-time linear stochastic systems with quadratic performance criterion and control-dependent noise. The analysis includes the study of a generalized Riccati difference equation and of the asymptotic behavior of its solutions. 1998 Elsevier Science Ltd. All rights reserved. 1. Introduction The analysis of systems with state and/or control-dependent noise is a classical problem in stochastic control theory. In recent years, there has been a renewed interest in this subject. In particular, by letting the statistical description of the noise not be known a priori but depend on the control and state evolution, robustness of the overall control system can be improved (see the work of Skelton and coworkers, e.g. Skelton and Shi, 1995). In several situations, this framework is more realistic than the classical one and the control law derived in the standard LQG framework is no more optimal; its use may lead to a serious degrading of the performances of the system, which may even become unstable (Ruan and Choudhury, 1993; Kro´likowski, 1997). With this motivation, we study, in this communique´, the linear quadratic optimal control problem for discrete-time sys- tems with control-dependent noise. A fundamental role in the synthesis of the control law will be played by a generalized Riccati difference equation (GRDE) (De Souza and Fragoso, 1990). We will investigate the asymptotic behavior of the solu- tions of such equation and derive results on the existence of an equilibrium solution and its attractiveness properties. Connec- tions between the results presented here and related problems dealt with in the literature are also reported. 2. Derivation of the optimal feedback control law We consider the optimal control problem of possibly time- varying systems on a finite interval I"[0, N], described by a state-space model such as the following x (t#1)"A(t) x (t)#B(t) u (t) # u (t) G (t) v (t)#D (t) w(t) (1) with x(0)N(0, (0)), x ( ) ) 3, u ( ) ) 3, v ( ) ) 3 , w( ) ) 3 , u denotes the ith component of the control vector u, and the A( ) ), B( ) ), G( ) ), D ( ) ) matrices have the appropriate dimensions. In equation (1), v (t) and w(t) are zero-mean white * Received 10 October 1995; revised 14 August 1997; receivd in final form 27 February 1998. This paper was recommended for publication in revised form by Editor Peter Dorato. Corres- ponding author Alessandro Beghi. Tel. 00-39-09-8277626; Fax 00-39-09-8277699; E-mail beghi@dei.unipd.it. - Dipartimeto di Elettronica e Informatica, Universita` di Padova, via Gradenigo 6/A, 35131 Padova, Italy. ‡ Department of Mechanical and Environmental Engineer- ing, University of California, Santa Barbara, CA 93106, USA. Gaussian noise (WGN) processes independent of x (0) and of each other, with intensities »(t) and ¼(t), respectively. The noise dependence on the control is modeled as in Whonam (1967, 1968) and McLane (1971), although the approach used in this paper could be easily adapted to cope with different choices, such as the one proposed in Skelton and Shi (1995) (see Beghi and D’Alessandro, 1997). The problem we consider is then stated as follows. Problem 1. For the system described by equation (1), find the state feedback law u (t)"!F(t) x (t) such that the performance index J" 1 2 E (x (t) Q(t) x (t)#u (t) R(t) u (t))#x (N) Mx(N) (2) is minimized. In equation (2), M50 and Q(t)50, R(t)'0, t"0, 2 , N!1. As a first step, we write the performance index J in equa- tion (2) as J" 1 2 Tr (Q(t)#F (t) R(t) F(t)) (t)#M(N) , (3) where (t) is the state variance of the controlled process. A re- cursion for (t) can be obtained by computing (t#1) as E [E[x (t#1) x (t#1) x (t)]]. We find that (t#1)"(A(t)!B(t) F(t)) (t)(A(t)!B(t) F(t)) # (F(t) (t) F(t)) G (t) »(t) G(t) #D(t) ¼(t) D(t), 04t4N!1 (4) with initial condition (0) , where with (X) we denote the ijth element of the matrix X. The solution of Problem 1 is equivalent to the search for the sequence of matrices F(t), t"0, 2 , N!1 which minimizes equation (3), subject to the constraint (4) on the controlled process variance. This can be seen as a deterministic optimiza- tion problem, with the elements of (t) as state variables, the recursion (4) as nonlinear state dynamics, the elements of F(t) as control variables, and equation (3) as the index to be minimized. In the following theorem, we solve such problem by using a matrix version of the maximum principle (Athans and Tse, 1967). ¹heorem 1. The feedback control law solving Problem 1 is given by u (t)"!F(t) x (t) where F(t)"[R(t)#B (t) P(t#1) B(t)#(P (t#1))] B (t) P(t#1) A(t) (5a) with ((P (t#1))) "Tr[P(t#1) G (t) »(t) G (t)] (5b) 1031