Received: 1 March 2017 Revised: 13 January 2018 Accepted: 8 April 2018 DOI: 10.1002/oca.2441 RESEARCH ARTICLE Inverse optimal control for asymptotic trajectory tracking of discrete-time stochastic nonlinear systems in block controllable form Santiago Elvira-Ceja 1 Edgar N. Sanchez 2 1 ITESO - Universidad Jesuita de Guadalajara, Tlaquepaque, Mexico 2 CINVESTAV-IPN Unidad Guadalajara, Zapopan, Mexico Correspondence Edgar N. Sanchez, CINVESTAV-IPN Unidad Guadalajara, Av del Bosque 1145, Colonia El Bajio, 45019 Zapopan, Jalisco, Mexico. Email: sanchez@gdl.cinvestav.mx Funding information CONACYT Mexico, Grant/Award Number: 131678Y and 257200 Summary This paper concerns an inverse optimal control–based trajectory tracking of discrete-time stochastic nonlinear systems. It is assumed that the nonlinear system can be transformed to the so called nonlinear block controllable form. Additionally, the synthesized control law minimizes a cost functional, which is posteriori determined. In contrast to the optimal control technique, this scheme avoids to solve the stochastic Hamilton-Jacobi-Bellman equation, which is not an easy task. Based on a discrete-time stochastic control Lyapunov function, the proposed optimal controller is analyzed. The proposed approach is applied successfully to the two degrees-of-freedom helicopter with uncertainties in real time. KEYWORDS inverse optimal control, stochastic nonlinear systems, trajectory tracking 1 INTRODUCTION Optimal stochastic control is related to determining a control law for a given stochastic system, such that the a pri- ori cost functional is minimized; the major drawback is solving the associated stochastic Hamilton-Jacobi-Bellman equation, 1 which is difficult or even impossible to solve. However, in the work of Zhang et al, 2 the optimal control of discrete-time nonlinear stochastic systems is solved based on discrete martingale theory. To overcome the solution of the stochastic Hamilton-Jacobi-Bellman equation, Crandall and Lions 3 introduced a weak solution named as viscosity solution, whereas Krsti ´ c et al 4 used the inverse optimal control approach in continuous-time nonlinear systems with uncertainties for stabilization, regulation, and tracking 3,4 ; consider the continuous-time case. On the other hand, the neu- ral network-based approximation methods have been used to solve the optimal control problem via adaptive dynamic programming, such as in the works of Luy, 5,6 where the cooperative control, for stabilization and tracking respectively, of multiple multiple-input–multiple-output nonlinear systems in strict feedback with no knowledge of internal system dynamics and affected by external disturbances is addressed. This paper proposes an inverse optimal control law for asymptotic stability in probability, along a desired trajectory, of discrete-time stochastic nonlinear systems in nonlinear block controllable (NBC) form. Using this approach, a feed- back control law, based on the a priori knowledge of a discrete-time stochastic control Lyapunov function (DSCLF), is synthesized first, and then it is established that this control law optimizes a cost functional. The inverse optimal control approach of deterministic systems is presented for the continuous 7-11 and discrete-time 12-14 case, respectively. Although the inverse optimal control for continuous-time stochastic nonlinear systems has been Optim Control Appl Meth. 2018;1–14. wileyonlinelibrary.com/journal/oca Copyright © 2018 John Wiley & Sons, Ltd. 1