IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 52, NO. 9, SEPTEMBER 2007 1587 On the Hybrid Optimal Control Problem: Theory and Algorithms M. Shahid Shaikh, Member, IEEE, and Peter E. Caines, Fellow, IEEE Abstract—A class of hybrid optimal control problems (HOCP) for systems with controlled and autonomous location transitions is formulated and a set of necessary conditions for hybrid system trajectory optimality is presented which together constitute gen- eralizations of the standard Maximum Principle; these are given for the cases of open bounded control value sets and compact con- trol value sets. The derivations in the paper employ: (i) classical variational and needle variation techniques; and (ii) a local con- trollability condition which is used to establish the adjoint and Hamiltonian jump conditions in the autonomous switching case. Employing the hybrid minimum principle (HMP) necessary con- ditions, a class of general HMP based algorithms for hybrid sys- tems optimization are presented and analyzed for the autonomous switchings case and the controlled switchings case. Using results from the theory of penalty function methods and Ekeland’s varia- tional principle the convergence of these algorithms is established under reasonable assumptions. The efficacy of the proposed algo- rithms is illustrated via computational examples. Index Terms—Controllability, hybrid systems, maximum prin- ciple, nonlinear systems. I. INTRODUCTION O VER the last few years the notion of a hybrid control system with continuous and discrete states and dynamics has fully crystallized and various classes of optimal control problems for such systems have been formalized (see, for example, [1]–[12]). In particular, generalizing the standard maximum (or minimum) principle (MP), Sussmann [13] and Riedinger et al. [14], [15], among other authors, have given versions of the hybrid minimum principle (HMP) with indica- tions of proof methods. An explicit theory for the two stage controlled switching optimal control problem was originally given by Tomiyama in [16] and a complete treatment of the HMP is given in [17] and [18] in the general multiprocess case; further, [19] treats the case of a priori fixed discrete state sequences and controlled switching times with switching costs. In this paper, we formulate a class of optimal control prob- lems for general hybrid systems with nonlinear dynamics in each location and with autonomous and controlled switchings. We obtain a set of necessary conditions for hybrid system opti- mality in two cases: 1) that where the control takes values in a Manuscript received November 9, 2005; revised November 30, 2006. Recommended by Associate Editor A. Bemporad. This work was supported by the NSERC under Grant 1329-00. M. S. Shaikh is with the National University of Computer and Emerging Sci- ences, Karachi, Pakistan (e-mail: msshaikh@cim.mcgill.ca). P. E. Caines is with the Department of Electrical and Computer Engineering and the Centre for Intelligent Machines, McGill University, Montréal, QC H3A 2A7, Canada (e-mail: peterc@cim.mcgill.ca). Digital Object Identifier 10.1109/TAC.2007.904451 compact set; and 2) that where the optimal control value set is an open bounded set. In particular, these conditions give infor- mation about the behaviour of the Hamiltonian and the adjoint trajectory at the switching times and are referred to as “jump conditions.” Jump conditions have been studied at least since the time of Weierstrass and in the calculus of variations they are known as Weierstrass-Erdmann corner conditions [20], corners being the points of nondifferentiability of extremals. In the context of optimal control theory similar conditions arose in the study of problems with bounded state constraint sets; these problems were studied, for example, by Pontryagin et al. [21] and Berkovitz [22]. The necessary conditions derived in this paper in the case of autonomous switchings are similar to those given in [21, p. 311] and [22] at the state and time where an optimal trajectory passes from the interior to the boundary of a state constraint set. In [21], Pontryagin et al. also give nec- essary conditions satisfied by the adjoint trajectory at an au- tonomous switching time; they use the terms “junction point” and “junction time” for switching state and switching time, re- spectively. Furthermore, in [23] Witsenhausen gives an outline of the proof of the necessity of jump conditions using a system of needle variations and geometric arguments similar to those in [21], while Bryson and Ho [24] obtain jump conditions using classical variational methods; the latter authors mention that a controllability condition is required for the derivation. In this paper, we give accessible proofs and analyses of the HMP which use the classical smooth variation approach, as well as needle variation techniques following the approach of de la Barrière [25] and Zabczyk [26]; the latter authors obtained the Maximum Principle necessary conditions by use of a single needle variation rather than a complex system of variations and geometric arguments as in [21]. In particular, we introduce into the analysis a controllability condition that is employed to es- tablish the adjoint and Hamiltonian jump conditions in the au- tonomous switching case Employing the HMP necessary conditions developed in this paper, we propose and analyze a new class of so-called HMP algorithms for the solution of hybrid optimal control problems (HOCP). We provide convergence results for the optimization algorithm called HMPMAS which treats HOCPs where multiple autonomous switchings (MAS) occur, that is to say, where the location switches whenever the continuous state passes through a specified sequence of switching manifolds but this sequence is assumed fixed. The convergence proofs are based on results from the theory of penalty function methods [27] and Ekeland’s variational principle [28]. The HMPMAS algorithm extends directly to multiple controlled switchings (MCSs) when the location sequence is fixed; for this case, we 0018-9286/$25.00 © 2007 IEEE