IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 55, NO. 1, JANUARY 2010 195 Almost Sure Stabilization of Uncertain Continuous-Time Markov Jump Linear Systems Mara Tanelli, Member, IEEE, Bruno Picasso, Paolo Bolzern, Member, IEEE, and Patrizio Colaneri, Senior Member, IEEE Abstract—This work deals with the robust Almost Sure (AS) stabiliza- tion problem for continuous-time Markov Jump Linear Systems (MJLS). Norm-bounded uncertainties affecting the system state and input matrices are considered. A deterministically testable sufficient condition for robust AS stability is provided which relies on a bound on the 2-norm of the system transition matrix. Such a condition can be profitably employed also to de- sign a robust feedback stabilization strategy. Such feedback design is based on a formulation of the sufficient condition for robust AS stability in terms of an equivalent LMI problem. Index Terms—Almost sure (AS) stability, Markov jump linear systems (MJLS), Markov processes, robust control, uncertain systems. I. INTRODUCTION Markov jump linear systems (MJLS) are a class of stochastic hybrid systems, where the switching between linear systems is governed by a finite state Markovian process. These systems are well suited to model physical systems subject to abrupt changes (e.g., components failures or environmental disturbances) which cause the modification of their dynamics, [1]–[3]). One of the main topics in the study of MJLS is investigating the notion of stability. Several definitions of stability have been given, which differ in conservativeness and ease of testability. The most significant stability notions for the analysis of such systems are: Moment stability, Mean-Square (MS) stability and Almost Sure (AS) stability, [4]. In this work, we focus on AS stability, which requires that almost all realizations of the sample path converge to zero. A necessary and sufficient condition for Exponential AS (EAS) stability has been given in terms of the sign of the top Lyapunov exponent, [5]. Nevertheless, testing such a condition is often practically unfeasible. Hence, suffi- cient conditions have been proposed whose testability relies on ran- domized algorithms, [6]. It has been shown that -moment stability implies EAS stability and that the two notions become equivalent for sufficiently small , see [7] and references therein. Up to present, how- ever, no easily testable necessary and sufficient conditions for checking -moment stability are known, at least for . On the contrary, simple necessary and sufficient conditions are available to check 2-mo- ment stability, i.e.,, MS stability, see [3], [4]. As far as stabilization is concerned, some conditions have been given in [7] for MS stabilization, which rely on the solution of an equivalent optimal control problem. Further, [8] addresses the AS stabilization problem in the discrete-time case. Recently, robust MS stability and stabilization of MJLS have been investigated, see e.g., [9]–[11]. For Manuscript received March 25, 2009; revised June 30, 2009. First published November 24, 2009; current version published January 13, 2010. This work was supported in part by MIUR project “Advanced Methodologies for Control of Hybrid Systems,” MIUR project “New methods for Identification and Adaptive Control for Industrial Systems.” Recommended by Associate Editor Z. Wang. The authors are with the Dipartimento di Elettronica e Informazione, Po- litecnico di Milano, Milano 20133, Italy (e-mail: tanelli@elet.polimi.it; picasso. bruno@gmail.com; bolzern@elet.polimi.it; colaneri@elet.polimi.it). Color versions of one or more of the figures in this technical note are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TAC.2009.2033844 MJLS with uncertain system matrices, robust MS stability can be as- sessed by checking the existence of the solution of a system of coupled Algebraic Riccati Equations, [11]. In [10] the problem of robust sta- bilizability is studied both in the case of uncertainties in the system matrices and in the jump parameters. Within this framework, two dif- ferent descriptions have been given for characterizing the jump uncer- tainty: [9] gives a polytopic description, while in [10] the elements of the transition rate matrix are supposed to be measurable with known error bounds. Finally, in [12] the authors tackle the problem of robust MS stabilization in the case of structured perturbations acting on both system matrices and jump probabilities and propose an LMI-based cri- terion for assessing the robust stability and for designing the stabilizing controller. Further recent results in analysis and synthesis problems of MJLS with uncertain transition probabilities are given in [13], [14]. However, little attention has been paid up to now to investigate robust AS stability and stabilization of MJLS. Recently, some results on AS stabilization of stochastic linear systems have been given in [15], where a state feedback controller is designed, both in the nominal and in the uncertain setting, by solving appropriate LMI problems. In this work, we develop a sufficient condition for robust AS stabiliz- ability of continuous-time MJLS, which leads to a practical algorithm for designing a stabilizing feedback. We consider norm-bounded uncer- tainties affecting the system state and input matrices, as they are com- putationally tractable and well suited to be analyzed within an set- ting. Notably, the proposed approach allows a seamless extension from the nominal to the uncertain case. Specifically, we start by proving a deterministically testable sufficient condition for robust AS stability of a MJLS, which relies on a bound on the 2-norm of the system transition matrix. Further, we show how to practically compute such a bound in the uncertain case via the solution of an appropriate Riccati equation coupled with an ad hoc algorithmic procedure. The proposed sufficient condition is then employed to design a robust AS feedback stabiliza- tion strategy, based on a formulation of such a condition in terms of an equivalent LMI problem. The proposed method is grounded on the preliminary results presented in [16]. The technical note is organized as follows. Section II recalls the basic properties of continuous-time MJLS. In Section III a sufficient condition for robust AS stability is worked out and the computation of a 2-norm bound on the matrix exponential is presented. Section IV addresses the stabilization problem and illustrates the equivalent LMI formulation, and an algorithmic strategy for the stabilizing feedback design. A numerical example is given in Section V. Notation: For and , and stand for the Euclidean norm of and the corresponding induced matrix norm of , respectively. is the expectation operator with respect to the probability P. II. PRELIMINARIES Consider a continuous time MJLS of the form (1) where , is a finite state, time homogeneous Markovian stochastic process taking values in a finite set and, for , , where , and are given matrices and the constants are the given perturbation radii. Associated with a realization , let and be the discontinuity points of (i.e., the switching in- stants). For , the matrix is assumed to be con- stant. On the other hand, given two time instants 0018-9286/$26.00 © 2009 IEEE