OPTIMAL DISTURBANCE ATTENUATION FOR ... - Semantic Scholar

SIAM J. CONTROL OPTIM. Vol. 45, No. 4, pp. 1329–1358

c 2006 Society for Industrial and Applied Mathematics 

OPTIMAL DISTURBANCE ATTENUATION FOR DISCRETE-TIME SWITCHED AND MARKOVIAN JUMP LINEAR SYSTEMS∗ JI-WOONG LEE† AND GEIR E. DULLERUD‡ Abstract. An exact condition for uniform stabilization and disturbance attenuation for switched linear systems is given in the discrete-time domain via the union of an increasing family of linear matrix inequality conditions. Associated with each Markovian jump linear system is a switched linear system, so we obtain a necessary and sufficient condition for almost sure uniform stabilization and disturbance attenuation for Markovian jump linear systems as well. The results lead to semidefinite programming–based controller synthesis techniques, from which optimal finite-path-dependent linear dynamic output feedback controllers arise naturally. In particular, under the notion of path-by-path optimal disturbance attenuation, finite-path-dependent controllers can outperform the usual modedependent ones. Key words. discrete linear inclusions, dynamic output feedback, H ∞ control, linear matrix inequalities, linear time-varying systems, semidefinite programming AMS subject classifications. 93B36, 93B12, 90C22, 93C55 DOI. 10.1137/050627538

1. Introduction. The switched linear system is defined as a family of linear time-varying systems whose parameters vary within a single finite set, and it serves as an abstraction of hybrid systems, where continuous system dynamics and discrete events coexist and depend on each other in complex ways [31]. On the other hand, the Markovian jump linear system is a linear system whose parameters jump according to the state transitions of a finite-state Markov chain, and it typically arises in the context of networked control systems, where the feedback loop is subject to random delays [32]. The parameters of these systems are indexed, where the indices are called the modes of the systems. This paper focuses on the discrete-time domain and considers the problems of uniform disturbance attenuation for switched linear systems and almost sure uniform disturbance attenuation for Markovian jump linear systems, where the uniformity refers to the stability and 2 -gain of the sequences of modes, called switching sequences, that are admissible. We develop semidefinite programming formulations [41] for the solutions to these problems, and their generalizations, without any assumption on the parameters or admissible switching sequences. The most fundamental problem of discrete-time switched linear systems is determining the asymptotic stability of the discrete linear inclusion (i.e., the family of linear time-varying systems over all infinite sequences of parameters in a given finite set) [12, 21]. This problem basically involves checking the stability of all possible (uncountably many) infinite matrix products due to the fact that the finiteness conjecture (i.e., the conjecture that it suffices to check the stability of periodic products) is generally false [8]. The problem is hence considered undecidable [7]; more precisely, ∗ Received by the editors March 24, 2005; accepted for publication (in revised form) April 4, 2006; published electronically October 3, 2006. This work was supported by AFOSR MURI grant F4962002-1-0325 and by AFOSR URI grant F49620-01-1-0365. http://www.siam.org/journals/sicon/45-4/62753.html † Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL 32611 (jiwoong@ufl.edu). ‡ Department of Mechanical and Industrial Engineering, University of Illinois at UrbanaChampaign, Urbana, IL 61801 ([email protected]).

1329

1330

JI-WOONG LEE AND GEIR E. DULLERUD

it is considered semidecidable (i.e., there exists an algorithm that is guaranteed to correctly determine, after a finite amount of computation, the stability of switched linear systems that are indeed stable) [6]. Recently, the authors gave a restatement of the result of [6] and introduced an increasing family of linear matrix inequality conditions whose union characterizes the uniform stability and stabilizability of discrete-time switched linear systems [30]. This result generalizes the multiple Lyapunov function approaches that have provided the most useful tools for stability analysis [9]. This paper complements the previous work on uniform stabilization described above and extends it to disturbance attenuation. The problem of disturbance attenuation reduces to that of H∞ control of linear time-invariant systems [16, 26] if the given set of system parameters is a singleton. On the other hand, if the family of admissible switching sequences is a singleton, the problem reduces to that of 2 induced norm minimization for linear time-varying systems [36, 4], where the set of possible parameter values is finite. Therefore, our work draws on the results in the H∞ control and 2 -gain minimization literature. In particular, from the linear operator inequality approach for analyzing linear time-varying systems [17], we deduce that the Riccati difference inequality associated with any stable and contractive linear time-varying system admits a uniformly stabilizing solution that has a finite memory of past parameters. This finite-memory property leads to our analysis results. On the other hand, our controller synthesis results are based on a straightforward extension of the linear matrix inequality approach originally developed for linear time-invariant systems [33, 20]. We relax the standard restriction to mode-dependent controllers (i.e., controllers that perfectly observe the present mode of the system but do not recall past modes) and consider finite-path-dependent controllers (i.e., controllers that not only perfectly observe the present mode but also have a finite memory of past modes). This relaxation, along with the aforementioned finite-memory property, enables us to derive a complete characterization of the existence of finite-path-dependent linear dynamic output feedback controllers that stabilize the switched linear system and achieve a uniform disturbance attenuation level. This characterization is given in terms of the “union” of an increasing family of systems of linear matrix inequality conditions. Although our result inherits the semidecidable nature of the underlying problem, this limitation is not likely to pose difficulties in practice, as examples show that it usually suffices to check the feasibility of the first few systems of linear matrix inequalities from an increasing family. Moreover, the result is amenable to the standard linear matrix inequality–based controller synthesis technique, from which admissible finitepath-dependent controller syntheses arise naturally. The notion of finite-path-dependent controllers not only serves as a relaxation to achieve an exact condition for disturbance attenuation but also is required for optimality under the notion of path-by-path disturbance attenuation. Roughly speaking, the path-by-path performance is defined as a finite family of disturbance attenuation levels over all admissible switching paths of a given length. This notion of performance is a natural extension of that of uniform disturbance attenuation and can be used to improve upon the performance of a controller synthesis that achieves a given uniform disturbance attenuation level. As long as the path-by-path optimality is concerned, examples show that finite-path-dependent controllers can outperform mode-dependent controllers. The result described above carries over to the almost sure uniform disturbance attenuation for Markovian jump linear systems because the uniformity requirement for stability and performance enables one to consider each Markovian jump linear

SWITCHED AND MARKOVIAN JUMP LINEAR SYSTEMS

1331

system as a switched linear system, where the underlying Markov chain of the former defines the switching path constraint of the latter. An immediate consequence is that, as long as almost sure uniform disturbance attenuation is concerned, the performance of Markovian jump linear systems is intrinsically robust against variations that preserve the directed graph of the underlying Markov chain (i.e., against sparsity pattern–preserving variations from the given transition probability matrix and initial distribution). On the other hand, under the notion of path-by-path disturbance attenuation, one can reduce the conservatism associated with this robustness property while still maintaining almost sure uniform stability; two Markov chains with the same directed graph can result in widely different path-by-path performances, depending on the actual values of initial and transition probabilities. A key contribution of this paper to the control of switched linear systems and Markovian jump linear systems is that exact “control-oriented” conditions for (almost sure) stabilization and disturbance attenuation are provided. These conditions are control oriented in the sense that they lead to semidefinite programming–based techniques, which render optimal controllers very efficiently. There has been little work done on the control of disturbance attenuation performances of switched linear systems, other than that some partial analysis results in the continuous-time domain exist [44, 23]. On the other hand, the usual approach in the literature to disturbance attenuation for Markovian jump linear systems has been based on the notion of stochastic stability in both continuous time [28, 34, 13] and discrete time [11, 19, 10, 37]. These results, however, though either partly satisfactory or exact, are not well-suited for efficient optimal controller synthesis. For instance, in the case of independent and identically distributed switching, an exact linear matrix inequality– based synthesis condition for discrete-time jump systems has been obtained in [38]; for general Markov switching, this condition is only sufficient. Moreover, even though stochastically stable Markovian jump linear systems are almost surely stable [29], the usual approach does not guarantee almost sure disturbance attenuation. The paper is organized as follows. Section 2 analyzes the performance of general linear time-varying systems. Then, based on this analysis, main results on uniform disturbance attenuation for switched linear systems and Markovian jump linear systems are derived in sections 3 and 4, respectively. Section 5 introduces the notion of path-by-path disturbance attenuation and provides some examples including optimal disturbance attenuation for a planar robot, called the Pendubot, subject to random delays in the feedback loop. Finally, concluding remarks are given in section 6. Notation. If X ∈ Rm×n , the range (or image) of X is denoted by Im X, the null space (or kernel) of X by Ker X, and the rank of X by rank X; denoted by N(X) is any particular full-rank matrix such that Im N(X) = Ker X. The indicator matrix 1(X) = (μij ) ∈ {0, 1}m×n of an X = (xij ) ∈ Rm×n is such that μij = 1 if xij = 0 and μij = 0 if xij = 0. The matrices whose entries are all 0 (resp., all 1) are denoted by 0 (resp., 1) whenever m and n are understood. If X, Y ∈ Rn×n are symmetric and X − Y is positive definite (resp., nonnegative definite), we write X > Y (resp., X ≥ Y). The identity matrix is denoted by I with n understood. √ For x ∈ Rn , denoted by x is the Euclidean vector norm x = xT x of x. If X ∈ Rm×n , the√ Euclidean vector norm induces the spectral norm X of X given by X = sup{ λ : λ is an eigenvalue of XT X}. Given a symmetric positive √definite matrix X ∈ Rn×n ,  · X is the Hilbert norm of x ∈ Rn defined by xX = xT X x. If x = (x(0), x(1), . . . ) is a sequence in Rn , then we write x ∈ 2 (Rn ) whenever the ∞ 2 2  norm of x, defined by x = s=0 x(s) , is finite.

1332

JI-WOONG LEE AND GEIR E. DULLERUD

2. Analysis of linear time-varying systems. In this section, we analyze the disturbance attenuation performance of the discrete-time linear time-varying system and the asymptotic property of the associated Riccati difference equation. We choose a notation that is compatible with the latter sections of this paper. Let a subset G of Rn×n × Rn×m × Rl×n × Rl×m and a sequence θ = (θ(0), θ(1), . . . ) in {0, 1, . . . } be as follows: (2.1)

G = {(A0 , B0 , C0 , D0 ), (A1 , B1 , C1 , D1 ), . . . },

θ = (0, 1, . . . ).

With (2.1), we have (Aθ(t) , Bθ(t) , Cθ(t) , Dθ(t) ) = (At , Bt , Ct , Dt ), and the pair (G, θ) is identified with the discrete-time linear time-varying system x(t + 1) = At x(t) + Bt w(t),

(2.2)

z(t) = Ct x(t) + Dt w(t),

where G defines an indexed family of parameter quadruples, and the sequence θ chooses one quadruple among G for each t ≥ 0. Given the initial state x(0) and disturbance sequence w = (w(0), w(1), . . . ), (2.2) determines the state sequence x = (x(0), x(1), . . . ) and output sequence z = (z(0), z(1), . . . ). If Bi , Ci , Di are all zero matrices, we write (A, θ) for (G, θ), where A = {A0 , A1 , . . . }.

(2.3)

Definition 2.1. The system (G, θ), and hence (A, θ), is said to be uniformly (exponentially) stable if there exist c ≥ 1 and λ ∈ (0, 1) such that, whenever w = 0, x(t) ≤ c λt−t0 x(t0 )

(2.4)

for t ≥ t0 ≥ 0 and for x(t0 ) ∈ Rn . Definition 2.2. The system (G, θ) is said to be uniformly (strictly) contractive if there exists a γ ∈ (0, 1) such that, whenever x(t0 ) = 0, t t (2.5) z(s)2 ≤ γ 2 w(s)2 s=t0

s=t0

for t ≥ t0 ≥ 0 and for w ∈ 2 (Rm ). Remark 1. It is clear that the uniform contractiveness is equivalent to the condition that the 2 -induced gain from w to z be less than one; that is, for some γ ∈ (0, 1), z ≤ γw whenever x(0) = 0 and w ∈ 2 (Rm ). The infimum of all γ > 0 that satisfy (2.5) is called the 2 -induced norm of the system (G, θ). Lemma 2.3. Let G and θ be as in (2.1); let G be bounded. The following are equivalent: (a) The system (G, θ) is uniformly exponentially stable and uniformly strictly contractive. (b) There exist α1 , β1 > 0, and Xt ∈ Rn×n , t = 0, 1, . . . , such that, for all t, (2.6a) (2.6b)

 At Ct

α1 I ≤ Xt ≤ β1 I; T     Xt+1 0 At Bt Xt Bt − 0 I Ct Dt 0 Dt

 0 ≤ −α1 I. I

(c) There exist α2 , β2 > 0, and Yt ∈ Rn×n , t = 0, 1, . . . , such that, for all t, (2.7a) (2.7b)

 At Ct

 Bt Y t 0 Dt

α2 I ≤ Yt ≤ β2 I; T   0 A t Bt Yt+1 − I Ct Dt 0

 0 ≤ −α2 I. I

SWITCHED AND MARKOVIAN JUMP LINEAR SYSTEMS

1333

Moreover, if either (b) or (c) holds, one may take Xt = Yt−1

(2.8)

for all t. Proof. The equivalence of (a) and (b) is shown in [17, Thm. 11] via an operator theoretic approach. A simple Schur complement argument, together with the matrix inversion formula, shows that (b) and (c) are equivalent via relation (2.8). Proposition 2.4. Let A be as in (2.3) and bounded; let θ be as in (2.1). Then the system (A, θ) is uniformly exponentially stable if and only if there exist α1 , β1 > 0 and Xt ∈ Rn×n (resp., α2 , β2 > 0 and Yt ∈ Rn×n ), t = 0, 1, . . . , such that α1 I ≤ Xt ≤ β1 I;

AT t Xt+1 At − Xt ≤ −α1 I

(resp., α2 I ≤ Yt ≤ β2 I;

At Yt AT t − Yt+1 ≤ −α2 I )

for all t ≥ 0. Proof. Set the matrices Bt , Ct , Dt , t ≥ 0, to zero in Lemma 2.3. Lemma 2.3 is a time-varying version of the classical Kalman–Yacubovitch–Popov (KYP) lemma (see, e.g., [35, 18]). Inequality (2.6b) is called the (extended) KYP inequality, and (2.7b) is its dual form. The solutions of these inequalities are obtained by solving the associated Riccati difference equations. Let S be the set of all symmetric matrices in Rn×n . For i = 0, 1, . . . , let Xi be the set of symmetric matrices X ∈ Rn×n such that T Wi (X) = I − DT i Di − Bi XBi

is invertible. Define Si : Xi → S, i = 0, 1, . . . , by T T T −1 T Si (X) = AT (BT i XAi + Ci Ci + (Ai XBi + Ci Di )Wi (X) i XAi + Di Ci )

for X ∈ Xi . Similarly, for i = 0, 1, . . . , let Yi be the set of symmetric matrices Y ∈ Rn×n such that T Vi (Y) = I − Di DT i − Ci YCi

is invertible, and define Ri : Yi → S, i = 0, 1, . . . , by T T T −1 T Ri (Y) = Ai YAT (Ci YAT i + Bi Bi + (Ai YCi + Bi Di )Vi (Y) i + Di Bi )

for Y ∈ Yi . Lemma 2.5. Let G and θ be as in (2.1); let G be bounded. The following are equivalent: (a) The system (G, θ) is uniformly exponentially stable and uniformly strictly contractive. (b) There exist ε1 , δ1 , η1 > 0 such that for all T = 0, 1, . . . , and ε ∈ [0, ε1 ], the equation  (ε,T )  (ε,T ) Xt = St Xt+1 + εI, (2.9a) (ε,T )

with the terminal condition XT +1 = εI, satisfies for t = 0, . . . , T that (2.9b)

 (ε,T )  Wt Xt+1 ≥ η1 I;

(ε,T )

εI ≤ Xt

(ε,T +1)

≤ Xt

≤ δ1 I.

1334

JI-WOONG LEE AND GEIR E. DULLERUD

(c) There exist ε2 , δ2 , η2 > 0 such that for all t0 = 0, 1, . . . , and ε ∈ [0, ε2 ], the equation  (ε,t )  (ε,t ) Yt+10 = Rt Yt 0 + εI,

(2.10a)

(ε,t0 )

with the initial condition Yt0

= εI, satisfies for t = t0 , t0 + 1, . . . that

 (ε,t )  Vt Yt 0 ≥ η2 I;

(2.10b)

(ε,t +1)

εI ≤ Yt+10

(ε,t )

≤ Yt+10 ≤ δ2 I.

(ε,T )

Moreover, if (b) holds, then one may take Xt = limT →∞ Xt in (2.6); if (c) holds, (ε,0) then one may take Yt = Yt in (2.7). Equation (2.9a) is the (generalized) Riccati difference equation associated with the KYP inequality, and (2.10a) is its dual form. The latter evolves forward in time and so provides an explicit recursive expression for computing the solution to the KYP inequalities (2.6)–(2.7). Similar, but less explicit, results from operator theoretic points of view exist [22, 27]. The proof of Lemma 2.5 is based on the following standard lemma. For Y ∈ Yi , let (2.11)

T −1 Ci . Ai (Y) = Ai + (Ai YCT i + Bi Di )Vi (Y)

Lemma 2.6. Let Y(1) , Y(2) ∈ Yi ; let Δ(12) = Y(1) − Y(2) . Then        T Ri Y(1) − Ri Y(2) = Ai Y(2) Δ(12) Ai Y(2)    (1) −1  T (2.12a) + Ai Y(2) Δ(12) CT Ci Δ(12) Ai Y(2) i Vi Y    T = Ai Y(1) Δ(12) Ai Y(2) . (2.12b) Proof. It follows from [22, Lem. 11, p. 77] that, for Y ∈ Yi , we may write (2.13)

T

T

−1 Ci YAi Ri (Y) = Ai YAi + Qi + Ai YCT i Vi (Y)

with T −1 Ai = Ai + Bi DT Ci , i (I − Di Di ) T T −1 Qi = Bi BT Di BT i + Bi Di (I − Di Di ) i .

Then [15, Lem. 3.1] leads to (2.12a). Since  −1  −1  −1  (1) −1 Vi Y(1) − Vi Y(2) − Vi Y(2) Ci Δ(12) CT = 0, i Vi Y it is easily seen that      (2) −1  Ci , Ai Y(2) = Ai Y(1) I − Δ(12) CT i Vi Y   (2) −1   (1) −1  (12) T (12) T I − Δ Ci Vi Y Ci I + Δ Ci Vi Y Ci = I. These equalities and (2.12a) yield (2.12b). Lemma 2.6 is useful in proving asymptotic properties of the Riccati difference  equation. In particular, an immediate consequence of (2.12a) is that Ri Y(1) ≥  (2)  Ri Y whenever Y(1) , Y(2) ∈ Yi , and Y(1) ≥ Y(2) .

SWITCHED AND MARKOVIAN JUMP LINEAR SYSTEMS

1335

Proof of Lemma 2.5. To first show that (c) is equivalent to (a), suppose that (c) holds. Let ε = ε2 . Since G is bounded, (2.10) implies that (ε,t0 )

At Yt

(ε,t )

0 T AT t + Bt Bt − Yt+1 + αI −1    (ε,t0 )   (ε,t ) (ε,t ) T T − αI Ct Yt 0 AT ≤0 + At Yt 0 CT t + Bt Dt Vt Yt t + Dt Bt

for some small α > 0. Then the Schur complement formula yields  At Ct

Bt Dt

  (ε,t0 ) Yt 0

 0 At I Ct

Bt Dt

T

 (ε,t0 ) Yt+1 − 0

 0 ≤ −αI. I

(ε,0)

Putting α2 = min{α, ε}, β2 = δ2 , and Yt = Yt , we obtain condition (c) of Lemma 2.3, which in turn leads to the uniform stability and contractiveness of (G, θ). Conversely, suppose that (G, θ) is uniformly stable and contractive so that condiT tion (c) of Lemma 2.3 holds. Taking the Schur complement of Ct Yt CT t + Dt Dt − I from (2.7b), we obtain Vt (Yt ) ≥ α2 I;

Rt (Yt ) + α2 I ≤ Yt+1

for all t. Choose an arbitrary ε ∈ [0, α2 ]. For all t0 ≥ 0 and all t ≥ t0 , define  (ε,t )  (ε,t ) (ε,t ) Yt0 0 = εI and Yt+10 = Rt Yt 0 + εI. Since Yt ≥ α2 I for all t, we have (ε,t ) Yt0 ≥ Yt0 0 ; by induction, together with (2.12a), we obtain  (ε,t )  Vt Yt 0 ≥ α2 I;

(ε,t0 )

εI ≤ Yt

≤ Yt ≤ β2 I

(ε,t )

(ε,t +1)

for t ≥ t0 ≥ 0. Moreover, Yt 0 ≥ εI for all t, and so we have Yt0 +10 from (2.12a); by induction, we obtain that (ε,t +1)

Yt+10

(ε,t )

≤ Yt0 +10

 (ε,t )  (ε,t ) ≤ Rt Yt 0 + εI = Yt+10

for t ≥ t0 ≥ 0. Putting ε2 = α2 , δ2 = β2 , and η2 = α2 , we obtain condition (c). This concludes the proof of the equivalence of (a) and (c). The proof that (a) and (b) are equivalent is analogous, and so is omitted. Theorem 2.7. Let G and θ be as in (2.1); let G be bounded. Suppose that the system (G, θ) is uniformly exponentially stable and uniformly strictly contractive so (ε,t ) that condition (c) of Lemma 2.5 holds. For ε ∈ (0, ε2 ) and t0 ≥ 0, let Yi 0 and (ε,t ) Ai (·), i = t0 , t0 + 1, . . . , be as in (2.10a) and (2.11), respectively, where Yt0 0 = εI. Then the following hold: (a) For each ε ∈ (0, ε2 ) and t0 ≥ 0, define  (ε,t ) 

A(ε,t0 ) = Ai Yi 0 : i = t0 , t0 + 1, . . . ,

θ (t0 ) = (t0 , t0 + 1, . . . ).

  Then each system A(ε,t0 ) , θ (t0 ) is uniformly exponentially stable. Moreover, for each ε ∈ (0, ε2 ), there exist cε ≥ 1 and λε ∈ (0, 1) such that (2.14) for t > t0 ≥ 0.

     0 At−1 Y(ε,t0 ) · · · At Yt(ε,t0 )  ≤ cε λt−t ε 0 t−1 0

1336

JI-WOONG LEE AND GEIR E. DULLERUD

(b) For each ε ∈ (0, ε2 ), there exist a nonnegative integer M and α2 , β2 > 0 such that (2.7) is satisfied with (ε,0) for t < M ; Yt Yt = (2.15) (ε,t−M ) Yt for t ≥ M for t ≥ 0. (t ) (ε ,t ) Proof. Fix ε, ε ∈ (0, ε2 ) such that ε < ε . For t ≥ t0 ≥ 0, let Yt 0 = Yt 0 − (ε,t ) Yt 0 . Then (2.12a) yields that  (ε,t )  (t )  (ε,t ) T (t0 ) Yt+1 ≥ At Y t 0 Y t 0 A t Y t 0 + (ε − ε)I for all t ≥ t0 . Since ε − ε > 0 and (ε − ε)I ≤ Yt 0 ≤ (δ2 − )I for t ≥ t0 ≥ 0, we  (t ) −1 (t ) have that, with Xt 0 = Yt 0 , there exist α1 , β1 > 0 such that (t )

(t0 )

α1 I ≤ Xt

≤ β1 I;

 (ε,t ) T (t0 )  (ε,t0 )  (t ) At Yt 0 Xt+1 − Xt 0 ≤ −α1 I At Yt

  for all t. The uniform stability of the systems A(ε,t0 ) , θ (t0 ) follows from Proposi tion 2.4. Due  to [30, Lem. 4], inequality (2.14) holds for t > t0 ≥ 0 with cε = β1 /α1 and λε = 1 − α1 /β1 . Thus assertion (a) holds true. To prove (b), pick a nonnegative integer M such that c2ε λ2M < ε/(δ2 − ε). Then, ε using (2.12b), we deduce that there exists an ε ∈ (0, ε) such that  (ε,t−M )   (ε,t−M +1)  Rt Yt − Rt Yt   (ε,t−M )   (ε,t−M )  = At Yt · · · At−M +1 Yt−M +1 Rt−M +1 (εI) − εI  (ε,t−M +1) T  (ε,t−M +1) T × At−M +1 Yt−M +1 · · · At Yt ≤ (δ2 − ε)c2ε λ2M ε I  ≤ε I for t ≥ M ; or equivalently,  (ε,t−M )  (ε,t−M +1) − Yt+1 Rt Yt ≤ −(ε − ε )I for t ≥ M . On the other hand,  (ε,0)  (ε,0) − Yt+1 = −εI ≤ −(ε − ε )I. Rt Yt Therefore, Yt defined by (2.15) satisfies Rt (Yt ) − Yt+1 ≤ −(ε − ε )I for all t ≥ 0. Using the Schur complement formula, we have that there exists an α > 0 such that  T     Yt+1 0 A t Bt Y t 0 A t Bt ≤ −αI − 0 I Ct Dt 0 I Ct Dt holds for t ≥ 0. Putting α2 = min{α, } and β2 = δ2 , we see that Yt satisfies (2.7) for all t ≥ 0, and hence assertion (b) holds.

SWITCHED AND MARKOVIAN JUMP LINEAR SYSTEMS

1337

Remark 2. Using (2.13), we may write (ε,t )

(ε,t0 )

Yt+10 = At Yt

T

(ε,t0 )

At + (Qt + εI) + At Yt

 (ε,t0 ) −1 (ε,t ) T CT Ct Yt 0 At t Vt Yt

for ε ∈ (0, ε2 ) and t ≥ t0 ≥ 0. Let G  = {(A0 , Q0 + εI, C0 , 0), (A1 , Q1 + εI, C1 , 0), . . . }. If (G, θ) is uniformly stable, it is easy to verify that (G  , θ) is uniformly detectable and   uniformly stabilizable. Then, by [1, 14], the system A(ε,t0 ) , θ (t0 ) is uniformly stable (ε,t ) for each t0 ≥ 0; it follows from (2.12b) that, for each t0 ≥ 0, Yt 0 converges to the unique “moving equilibrium” (i.e., the maximal solution) of the Riccati difference equation (2.10a) as t − t0 → ∞ [14]. However, part (a) of Theorem 2.7 says that this   convergence is uniform in (t, t0 ), as the uniform stability of A(ε,t0 ) , θ (t0 ) is again uniform in t0 . It is known that, if a given linear time-varying system is uniformly stable, the corresponding Lyapunov inequality admits a solution that has a finite memory of past parameters (see, e.g., [30]). Part (b) of Theorem 2.7 is an extension of this and says that the KYP inequality (or equivalently, the Riccati inequality) associated with a uniformly stable and contractive linear time-varying system has a solution that has a finite memory of past parameters. 3. Control of switched linear systems. The switched linear system is a family of linear time-varying systems whose parameters vary within a single finite set. Fix a positive integer N and define (3.1)

G = {(A1 , B1 , C1 , D1 ), . . . , (AN , BN , CN , DN )},

where Ai ∈ Rn×n , Bi ∈ Rn×m , Ci ∈ Rl×n , Di ∈ Rl×m for i = 1, . . . , N . Let Ω be the set of all infinite sequences in {1, . . . , N }; each member of Ω is called a switching sequence. If Θ is a nonempty subset of Ω, then the pair (G, Θ) defines the switched linear system, where Θ is the set of admissible switching sequences: for initial states x(0), disturbance sequences w, switching sequences θ ∈ Θ, and t ≥ 0, the system (G, Θ) has the state-space representation (3.2)

x(t + 1) = Aθ(t) x(t) + Bθ(t) w(t), z(t) = Cθ(t) x(t) + Dθ(t) w(t).

If θ(t) = i, then the system is said to be in mode i at time t, and its parameters at time t are given by the quadruple (Ai , Bi , Ci , Di ). When the set Θ is equal to the entire set Ω, the pair (G, Ω) defines the discrete linear inclusion, which is the switched linear system without a switching path constraint; on the other hand, if Θ = {θ} is a singleton, then the pair (G, Θ) is nothing but the linear time-varying system (G, θ). We require that the stability and contractiveness of the system (G, Θ) be uniform over all switching sequences in Θ. Definition 3.1. The system (G, Θ) is said to be uniformly (exponentially) stable if there exist c ≥ 1 and λ ∈ (0, 1) such that, whenever w = 0, inequality (2.4) holds for t ≥ t0 ≥ 0, for x(t0 ) ∈ Rn , and for θ ∈ Θ. Definition 3.2. The system (G, Θ) is said to be uniformly (strictly) contractive if there exists a γ ∈ (0, 1) such that, whenever x(t0 ) = 0, inequality (2.5) holds for t ≥ t0 ≥ 0, for w ∈ 2 (Rm ), and for θ ∈ Θ.

1338

JI-WOONG LEE AND GEIR E. DULLERUD

For the sake of convenience, we introduce a dummy mode 0 and think of each θ ∈ Θ as a two-sided sequence (. . . , θ(−1), θ(0), θ(1), . . . ) by putting θ(t) = 0 for t < 0. Fix a nonnegative integer L. Any finite sequence in {0, . . . , N } will be called a (finite) switching path; in particular, elements of the set {0, . . . , N }L+1 are switching paths of length L and are called L-paths. For θ ∈ Θ and t ≥ 0, let θL (t) = (θ(t − L), . . . , θ(t)). An L-path (i0 , . . . , iL ) is said to occur in Θ if θL (t) = (i0 , . . . , iL ) for some θ ∈ Θ and some t ≥ 0. Each θ ∈ Θ generates an L-path switching sequence θ L defined by θ L = (θL (0), θL (1), . . . ). Denote the set of L-paths occurring in Θ by LL (Θ) so that LL (Θ) = {θL (t) : θ ∈ Θ, t ≥ 0}. If (i0 , . . . , iL ) ∈ LL (Θ), then write (i0 , . . . , iL )− = (i0 , . . . , iL−1 ),

(i0 , . . . , iL )+ = (i1 , . . . , iL )

for L > 0, and write (i0 , . . . , iL )− = (i0 , . . . , iL )+ = 0 for L = 0. If L > 0, define ML (Θ) to be the smallest subset of LL (Θ) such that the following hold: θL (t) ∈ ML (Θ) for all t ≥ L and for all θ ∈ Θ, and, for each j ∈ L0 (Θ), there exists a switching path ij0 , . . . , ijL−1 ∈ {0, . . . , N }L such that (3.3)



θ(0)

i0

  θ(0)  θ(0) θ(0) , . . . , iL−1 , θ(0) , i1 , . . . , iL−1 , θ(0), θ(1) ,   θ(0) . . . , iL−1 , θ(0), . . . , θ(L − 1) ∈ ML (Θ)

for all θ ∈ Θ. If L = 0, then let M0 (Θ) = L0 (Θ). The sets ML (Θ), L = 0, 1, . . . , are unique and so are well defined. Let M− L (Θ) = {i− : i ∈ ML (Θ)},

L− L (Θ) = {i− : i ∈ LL (Θ)}.

In general, we have LL (Θ) ∩ {1, . . . , N }L+1 ⊂ ML (Θ) ⊂ LL (Θ)  {0, . . . , N }L+1 , − ML (Θ) ⊂ M− L+1 (Θ) ⊂ LL+1 (Θ).

Example 1. Let Θ consist of three periodic sequences (1, 1, . . . ), (1, 2, 1, 2, . . . ), and (2, 2, . . . ). To simplify notation, write i0 · · · iL for (i0 , . . . , iL ). Then we have L0 (Θ) = M0 (Θ) = {1, 2},

L1 (Θ) = {01, 02, 11, 12, 21, 22},

L2 (Θ) = {001, 002, 011, 012, 022, 111, 121, 212, 222}, and so on. For L = 1, we can replace the one-path (0, 1) in L1 (Θ) with (1, 1) or (2, 1); similarly, the one-path (0, 2) can be replaced with either (1, 2) or (2, 2). Hence, we have M1 (Θ) = {11, 12, 21, 22}. However, for L = 2, if we choose any (i0 , i1 ) = (0, 0), then we have that at least one of the two-paths (i0 , i1 , 1), (i1 , 1, 1), (i1 , 1, 2) does not belong to L2 (Θ) and so (0, 0, 1)

SWITCHED AND MARKOVIAN JUMP LINEAR SYSTEMS

1339

cannot be replaced with any of (0, 1, 1), (1, 1, 1), (1, 2, 1) ∈ L2 (Θ); on the other hand, both the two-paths (0, 0, 2) and (0, 2, 2) can be replaced with (2, 2, 2) ∈ L2 (Θ). Hence, M2 (Θ) = {001, 011, 012, 111, 121, 212, 222}. This Θ will be used later in Examples 4 and 7. Example 2. For each positive integer k, let Θ(k) consist of a single sequence θ (k) = (1, . . . , 1, 2, . . . , 2, 1, . . . , 1, 2, . . . , 2, . . . ),

        k

k

k

k

(k)

where k 1’s and k 2’s alternate in θ . Then     L0 Θ(k) = M0 Θ(k) = {1, 2} for all k;  (k)   (k)  {01, 12, 21}, k = 1; {12, 21}, k = 1; L1 Θ = M1 Θ = {01, 11, 12, 21, 22}, k ≥ 2; {11, 12, 21, 22}, k ≥ 2; ⎧ ⎪{001, 012, 121, 212}, k = 1;  (k)  ⎨ L2 Θ = {001, 011, 112, 122, 211, 221}, k = 2; ⎪ ⎩ {001, 011, 111, 112, 122, 211, 221, 222}, k ≥ 3; ⎧ ⎪{121, 212}, k = 1;  (k)  ⎨ M2 Θ = {112, 122, 211, 221}, k = 2; ⎪ ⎩ {111, 112, 122, 211, 221, 222} k ≥ 3, and so on. These sets will be used in Example 3. The two-path switching sequence generated by θ (2) , for instance, is (2)

θ 2 = (001, 011, 112, 122, 221, 211, 112, 122, 221, 211, . . . ). Theorem 3.3. Let G be as in (3.1); let Θ ⊂ Ω be nonempty. The system (G, Θ) is uniformly exponentially stable and uniformly strictly contractive if and only if there exist a nonnegative integer M and an indexed family {Xj : j ∈ M− M (Θ)} of symmetric positive definite matrices Xj ∈ Rn×n such that  T      A i M Bi M Xi+ 0 AiM BiM Xi− 0 − (3.4) 0 without one can choose ij0 , . . . , ijM −1 ∈ {0, . . . , N }M , j ∈ {θ(0) : θ ∈ Θ}, such that (3.3), with L replaced by M , holds for all θ ∈ Θ. Put ⎧   for t = 0; ⎪ θ(0) ⎪X iθ(0) ,...,iM −1 ⎨ 0  for 0 < t < M ; Xt = X θ(0) θ(0) it ,...,iM −1 ,θ(0),...,θ(t−1) ⎪ ⎪ ⎩ for t ≥ M . X(θ(t−M ),...,θ(t−1)) Then, since MM (Θ) is finite, one can choose α, β > 0 independently of θ ∈ Θ such that, in particular, αI ≤ Xt ≤ βI;

AT θ(t) Xt+1 Aθ(t) − Xt < −αI

1340

JI-WOONG LEE AND GEIR E. DULLERUD

for t ≥ 0; hence it follows from Proposition 2.4 and the first half of the proof of Theorem 2.7 that the system (G, Θ) is uniformly stable. On the other hand, the finiteness of MM (Θ) implies that there exists an η ∈ (0, 1), which is independent of θ, such that T   Aθ(t) x(t) Cθ(t) w(t)



Bθ(t) Dθ(t)

T  Xt+1 0

 Aθ(t) Cθ(t)

0 I

Bθ(t) Dθ(t)

 Xt − 0



  0 0 +η I 0

  0 x(t) ≤0 I w(t)

for all x(t) ∈ Rn and w(t) ∈ Rm so that z(t)2 + x(t + 1)2Xt+1 − x(t)2Xt ≤ (1 − η)w(t)2 for t ≥ 0. If t ≥ t0 ≥ 0 and x(t0 ) = 0, then this inequality, as well as the fact that Xi > 0 for all i ∈ M− M (Θ), leads to t s=t0

z(s)2 ≤ (1 − η)

t s=t0

w(s)2 .

√ Putting γ = 1 − η yields (2.5) for t ≥ t0 ≥ 0. Since γ is independent of θ, this inequality holds for all θ ∈ Θ, too. Hence the system (G, Θ) is uniformly contractive as well as uniformly stable. This proves sufficiency. To show necessity, suppose that (G, Θ) is uniformly stable and contractive. Consider the augmented disturbance signal w(t) ˜ = [w(t)T v(t)T ]T with v(t) ∈ Rn , t ≥ 0,  (ε)  and the perturbed system G , Θ , where

(ε)

Bi



 (ε) (ε)  (ε) (ε) 

A1 , B1 , C1 , D1 , . . . , AN , BN , CN , DN , √ (ε) = [Bi εI] ∈ Rn×(m+n) , Di = [Di 0] ∈ Rl×(m+n) .

G (ε) =

Then there exists a sufficiently small ε2 > 0, dependent on γ in (2.5) but independent of Θ, such that (G (ε) , Θ) is uniformly stable and contractive for all ε ∈ (0, ε2 ). If we fix an ε ∈ (0, ε2 ), then by Lemma 2.5, there exist δ2 , η2 > 0 such that the (dual)  (ε,t )  (ε,t ) (ε,t ) Riccati equation Yt+10 = Rθ(t) Yt 0 + εI, with the initial condition Yt0 0 = εI, satisfies  (ε,t )  (ε,t ) Vθ(t) Yt 0 ≥ η2 I; εI ≤ Yt+10 ≤ δ2 I for t ≥ t0 ≥ 0 and for θ ∈ Θ. Part (b) of Theorem 2.7 and its proof then imply that there exists a nonnegative integer M , which depends only on ε, δ2 , η2 , and G, such that for some α2 , β2 > 0, Yt given by (2.15), with At = Aθ(t) , Bt = Bθ(t) , Ct = Cθ(t) , and Dt = Dθ(t) , satisfies (2.7) for θ ∈ Θ and t ≥ 0. Then Lemmas 2.3 and 2.5 imply that the symmetric positive definite matrices Xt , t ≥ 0, satisfying (2.6) can be taken to be of the form Xt = f (θ(t − M ), . . . , θ(t − 1)) for some function f : {0, . . . , N }M → S, where θ(s) = 0 for s < 0. Putting X(θ(t−M ),...,θ(t−1)) = f (θ(t − M ), . . . , θ(t − 1))

SWITCHED AND MARKOVIAN JUMP LINEAR SYSTEMS

1341

leads to (3.4) for i = (i0 , . . . , iM ) ∈ LM (Θ). Since MM (Θ) ⊂ LM (Θ), and since the path length M is independent of θ, we obtain the desired result. Remark 3. In Theorem 3.3 the number of linear matrix inequalities (3.4) to solve simultaneously, given M , is equal to the cardinality of MM (Θ) and bounded above M +1 by k=1 N k ; the number of matrix variables to solve for is equal to the cardinality of M− M (Θ). In particular, if either Θ = Ω or θ ∈ Θ for some θ which is recurrent with respect to {1, . . . , N }, so that every finite switching path in {1, . . . , N } occurs infinitely many times in θ, then the cardinality of MM (Θ) is precisely N M +1 . Remark 4. If N = 1, then we have Θ = {(1, 1, . . . )}, and the set MM (Θ) is a singleton for each M , and so Theorem 3.3 reduces to the classical KYP lemma. If inequalities (3.4), with Xj > 0 for j ∈ M− M (θ), are feasible for some M , then it is also feasible when M is replaced with any integer greater than M . Hence Theorem 3.3 characterizes the performance of switched linear systems via the countably infinite union of an increasing family of systems of linear matrix inequalities. For uniform stability and contractiveness, not only is each member of this family sufficient, but also the union of the family is necessary. The condition in Theorem 3.3 simplifies if we focus on the uniform stability only. For nonnegative integers L and Θ ⊂ Ω, let N L (Θ) be the largest subset of ML (Θ) satisfying the following: For each (i0 , . . . , iL ) ∈ N L (Θ), there exist an integer M > L and a switching path (i0 , . . . , iM ) such that (iM −L , . . . , iM ) = (i0 , . . . , iL ) and (it , . . . , it+L ) ∈ N L (Θ) for 0 ≤ t ≤ M − L. Then we have N L (Θ) ⊂ ML (Θ) ∩ {1, . . . , N }M +1 . For example, if Θ = {(1, 2, 2, 2, 2, . . . ), (1, 1, 2, 2, 2, . . . ), (1, 1, 1, 2, 2, . . . ), . . . }, then N 0 (Θ) = {1, 2}, N 1 (Θ) = {11, 22}, N 2 (Θ) = {111, 222}, and so on. Corollary 3.4. Let G be as in (3.1); let Θ ⊂ Ω be nonempty. The system (G, Θ) is uniformly exponentially stable if and only if there exist a nonnegative integer M and matrices Xj > 0 such that (3.5)

AT iM Xi+ AiM − Xi− < 0

for all i = (i0 , . . . , iM ) ∈ N M (Θ). Proof. Set the matrices Bi , Ci , Di , i = 1, . . . , N , to zero in (3.4) to obtain (3.5). Since N M (Θ) ⊂ MM (Θ), it suffices to show sufficiency. If (3.5) holds for all i = (i0 , . . . , iM ) ∈ N M (Θ), perform the following algorithm:  M (Θ) = N M (Θ). 0. Set M  1. If MM (Θ) = MM (Θ), then stop; otherwise, choose an i = (i0 , . . . , iM ) ∈  M (Θ) such that (i1 , . . . , iM , iM +1 ) ∈ M  M (Θ) for some iM +1 ∈ MM (Θ) \ M {1, . . . , N }. (By definition of MM (Θ), such an M -path i exists.)  M (Θ)}, then choose an α > 0 such that AT Xi Ai − / {ˆı− : ˆı ∈ M 2. If i− ∈ + M iM αI < 0, put Xi− = αI, and go to step 4. 3. Choose an α > 0 such that AT iM Xi+ AiM − αXi− < 0, and substitute Xi− with αXi− . (By definition of N M (Θ), i− is not equal to i+ .) Whenever there exists an integer L > 0 and a switching path (i−L , . . . , i−1 ) such that  M (Θ)} for all 0 ≤ t ≤ L, then substitute (it−L , . . . , it+M −L )− ∈ {ˆı− : ˆı ∈ M X(i−L ,...,iM −L )− with αX(i−L ,...,iM −L )− , too. (Again, by definition of N M (Θ), such a path (i−L , . . . , iM −L )− cannot be equal to i+ .)

1342

JI-WOONG LEE AND GEIR E. DULLERUD

 M (Θ) with M  M (Θ) ∪ {i} and go to step 1. 4. Substitute M Since the cardinality of MM (Θ) is finite, we will have reconstructed an entire family {Xj : j ∈ M− M (Θ)} of matrices Xj > 0 at the termination of this algorithm so that (3.5) holds for all i = (i0 , . . . , iM ) ∈ MM (Θ). Remark 5. If Θ is defined via a strongly connected directed graph, then Corollary 3.4 reverts to the stability analysis result in [30]. Now, consider the set (3.6)

T = {(Ai , B1,i , B2,i , C1,i , C2,i , D11,i , D12,i , D21,i ) : i = 1, . . . , N }

with Ai ∈ Rn×n , B1,i ∈ Rn×m1 , B2,i ∈ Rn×m2 , C1,i ∈ Rl1 ×n , C2,i ∈ Rl2 ×n , D11,i ∈ Rl1 ×m1 , D12,i ∈ Rl1 ×m2 , D21,i ∈ Rl2 ×m1 for i = 1, . . . , N . If Θ ⊂ Ω and is nonempty, then the pair (T , Θ) defines the controlled switched linear system represented by x(t + 1) = Aθ(t) x(t) + B1,θ(t) w(t) + B2,θ(t) u(t), (3.7)

z(t) = C1,θ(t) x(t) + D11,θ(t) w(t) + D12,θ(t) u(t), y(t) = C2,θ(t) x(t) + D21,θ(t) w(t).

Given the initial state x(0), disturbance sequence w = (w(t)), control sequence u = (u(t)), and switching sequence θ ∈ Θ, this system of equations defines the evolution of the state x(t), controlled output z(t), and measured output y(t) for t ≥ 0. Based on the analysis result given by Theorem 3.3, we will be deriving a necessary and sufficient condition for controller synthesis. We make the standard assumption that the mode θ(t) is perfectly observed at each time instant t; however, relaxing the standard restriction to mode-dependent controllers (i.e., controllers that do not recall past modes), we consider all controllers that have a finite memory of past modes as well as a perfect observation of the current mode. Fix a nonnegative integer L. Let ΘL = {θ L : θ ∈ Θ} be the set of L-path switching sequences generated by Θ; let (3.8)

K = {(AK,i , BK,i , CK,i , DK,i ) : i ∈ LL (Θ)}

with AK,i ∈ RnK ×nK , BK,i ∈ RnK ×l2 , CK,i ∈ Rm2 ×nK , DK,i ∈ Rm2 ×l2 for i ∈ LL (Θ). Then the pair (K, ΘL ) defines the L-path-dependent (linear output feedback) controller (of order nK ), which determines the control sequence u according to (3.9)

xK (t + 1) = AK,θL (t) xK (t) + BK,θL (t) y(t), u(t) = CK,θL (t) xK (t) + DK,θL (t) y(t)

given the initial controller state xK (0) and L-path switching sequence θ L ∈ ΘL . Controllers that are L-path-dependent for some nonnegative integer L shall be said to be finite-path-dependent; zero-path-dependent controllers are called mode-dependent. The dependence of these controllers on the past measurements y(0), . . . , y(t) at each time instant t is encoded in the partition   AK,i BK,i Ki = ∈ R(nK +m2 )×(nK +l2 ) , i ∈ LL (Θ). CK,i DK,i

SWITCHED AND MARKOVIAN JUMP LINEAR SYSTEMS

1343

Given a finite-path-dependent controller (K, ΘL ), where L is the path length, let i = A i + B  2,i Ki C  2,i , A L L L (3.10)

i = B  1,i + B  2,i Ki D  21,i , B L L L

 1,i + D  12,i Ki C  i = D11,i + D  12,i Ki D i = C  2,i ,D  21,i C L L L L L L

for i = (i0 , . . . , iL ) ∈ LL (Θ), with    i = Ai 0 ∈ R(n+nK )×(n+nK ) , A 0 0    1,i = B1,i ∈ R(n+nK )×m1 , B 0   1,i = C1,i C   12,i = 0 D

 0 ∈ Rl1 ×(n+nK ) ,

 2,i = C



  21,i = D12,i ∈ Rl1 ×(nK +m2 ) , D

for i ∈ {1, . . . , N }. Let TK=



 B2,i ∈ R(n+nK )×(nK +m2 ) , 0

  2,i = 0 B I

0 C2,i

 I ∈ R(nK +l2 )×(n+nK ) , 0

 0 ∈ R(nK +l2 )×m1 D21,i





 i, C  i, D  i : i ∈ LL (Θ) .  i, B A

If we define the closed-loop state by  T x ˜(t) = x(t)T xK (t)T ∈ Rn+nK , then the closed-loop system (T K , ΘL ) has the representation (3.11)

 θ (t) x  θ (t) w(t), x ˜(t + 1) = A ˜(t) + B L L   ˜(t) + Dθ (t) w(t) z(t) = Cθ (t) x L

L

for each L-path switching sequence θ L ∈ ΘL . Let NL be the cardinality of LL (Θ). If we label the elements of LL (Θ) from 1 to NL , then each L-path switching sequence θ L = (θL (0), θL (1), . . . ) ∈ ΘL can be considered a closed-loop switching sequence in {1, . . . , NL }; letting θL (t) = 0 for t < 0, the closed-loop switching path (θL (t − M ), . . . , θL (t)) can be identified with the switching path (θ(t − L − M ), . . . , θ(t)) for each triple (t, L, M ) of nonnegative integers. This leads to the following identities for all integers L > 0 and M ≥ 0: (3.12)

MM (ΘL ) = LM (ΘL ) = LM +L (Θ).

Hence, even if L > 0, the closed-loop system (T K , ΘL ) is a switched linear system, where the closed-loop modes are the L-paths in LL (Θ), and the closed-loop M -paths are the (M + L)-paths in LM +L (Θ) for each nonnegative integer M . Lemma 3.5. Let T be as in (3.6); let Θ ⊂ Ω be nonempty. Suppose that K is finite-path-dependent as in (3.8) with some nonnegative integer L. Then the closed-loop system (T K , ΘL ) is uniformly exponentially stable and uniformly strictly contractive if and only if there exist an integer M ≥ L and an indexed family {Xj : j ∈ (n+nK )×(n+nK ) L− such that M (Θ)} of symmetric positive definite matrices Xj ∈ R (3.13)

T T H(i0 ,...,iM ) + GT iM K(iM −L ,...,iM ) FiM + FiM K(iM −L ,...,iM ) GiM < 0

1344

JI-WOONG LEE AND GEIR E. DULLERUD

for all M -paths (i0 , . . . , iM ) ∈ LM (Θ), where ⎤ ⎡ i  1,i −X−1 0 A B M M i+ ⎢ T T ⎥ −Xi− 0 C ⎢A 1,iM ⎥ ∈ R(2n+2nK +m1 +l1 )×(2n+2nK +m1 +l1 ) Hi = ⎢ TiM ⎥ T  ⎦ ⎣B 0 −I D 1,iM 11,iM  0 C1,iM D11,iM −I for i = (i0 , . . . , iM ) ∈ LM (Θ) and where #   " T T  0 0 D B Fi 2,i 12,i = ∈ R(2nK +m2 +l2 )×(2n+2nK +m1 +l1 )  2,i D  21,i Gi 0 C 0 for i ∈ {1, . . . , N }. Proof. Let us first assume that L > 0. It follows from Theorem 3.3 and (3.12) that (T K , ΘL ) is uniformly stable and contractive if and only if there exists a nonnegative integer M satisfying the following: either M = 0 and there is a single X0 > 0 such that " #T  #  "  i i i B i B X0 0 X0 0 A A − (3.14a) 0 and there are X(j1 ,...,jM +L ) > 0 such that " (3.14b)

 (i ,...,i A M M +L )  (i ,...,i C ) M

M +L

#T   (i ,...,i B Xi+ ) M M +L  (i ,...,i 0 D M M +L ) "  A ×  (iM ,...,iM +L ) C(i ,...,i ) M

M +L

for i = (i0 , . . . , iM +L ) ∈ LM +L (Θ). If M = all L-paths (j1 , . . . , jL ) and write (3.14a) as " #T  " i i B i Xi+ 0 A A (3.14c)   i 0 I Ci Di C



0 I

#   (i ,...,i B Xi− M M +L ) −  0 D(iM ,...,iM +L )

 0 0, then Lemma 2.3 and the finiteness of LL (Θ) imply that (T K , ΘL ) is uniformly stable and contractive. Now, as in [20, p. 431], rewrite (3.14b) and (3.14c) (the former for M > 0 and the latter for M = 0) as inequalities of the form (3.13) using the decompositions (3.10) along with the Schur complement formula and an appropriate congruence transformation. Replacing M with M − L then yields the desired result. If L = 0, then MM (ΘL ) = MM (Θ), which is not equal to LM +L (Θ) = LM (Θ) in general. However, the proof of Theorem 3.3 shows that the existence of Xj > 0, j ∈ M− M (Θ), such that (3.4) holds for all i = (i0 , . . . , iM ) ∈ MM (Θ), suffices for the existence of Xj > 0, j ∈ L− M (Θ), such that (3.4) holds for all i = (i0 , . . . , iM ) ∈ LM (Θ). Therefore, the proof of the result for L = 0 is identical to the case of L > 0. Note that Lemma 3.5 is stated in terms of the closed-loop (M − L)-paths in LM (Θ), M ≥ L, rather than the closed-loop M -paths in MM (ΘL ), because the former are easier to deal with. Inequality (3.13) is amenable to the standard linear matrix

1345

SWITCHED AND MARKOVIAN JUMP LINEAR SYSTEMS

inequality embedding technique, originally developed for linear time-invariant systems [33, 20]. Finite-path-dependent controllers arise naturally from this technique. Definition 3.6. The controller (K, ΘL ) is said to be an admissible (L-pathdependent) synthesis (of order nK ) for the system (T , Θ) if the closed-loop system (T K , ΘL ) is uniformly exponentially stable and uniformly strictly contractive. Theorem 3.7. Let T be as in (3.6); let Θ ⊂ Ω be nonempty. Suppose that nK ≥ n. There exists an admissible finite-path-dependent synthesis of order nK for the system (T , Θ) if and only if there exist a nonnegative integer M and an indexed family {(Rj , Sj ) : j ∈ M− M (Θ)} of pairs of symmetric positive definite matrices Rj , Sj ∈ Rn×n such that (3.15a)   A iM NT F,iM C1,iM

B1,iM D11,iM

(3.15b)   A iM T NG,iM C1,iM

B1,iM D11,iM

(3.15c)

 Ri− 0



0 I

A iM C1,iM

B1,iM D11,iM

T

 Ri+ − 0

T     Si+ 0 S A iM B1,iM − i− 0 I C1,iM D11,iM 0   Ri− I ≥0 I Si−



0 I

NF,iM < 0,



0 I

NG,iM < 0,

for all M -paths i = (i0 , . . . , iM ) ∈ MM (Θ), where     T , NG,i = N C2,i D21,i NF,i = N BT 2,i D12,i for i ∈ {1, . . . , N }. Moreover, if (3.15) holds for all i = (i0 , . . . , iM ) ∈ MM (Θ), then there exist a nonnegative integer L ≤ M and matrices K(j0 ,...,jL ) ∈ R(nK +m2 )×(nK +l2 ) such that (3.13) holds for all (i0 , . . . , iM ) ∈ MM (Θ) with (3.16)

"

Xj =

1

Sj 1

Vj2 UT j

Uj Vj2 Vj

#

" =

−1

#−1

− 12

Rj −Vj 2 UT j Rj

−1

−Rj Uj Vj

− 12

Vj 2 (I + UT j Rj Uj )Vj

>0

n×nK for j ∈ M− , Vj ∈ RnK ×nK are any matrices such that M (Θ), where Uj ∈ R −1 T Uj Uj = Sj − Rj and Vj > 0; in particular, one may take L = M . Proof. It follows from [20, Lem. 3.1] that, with L = M , inequality (3.13) in K(iM −L ,...,iM ) = K(i0 ,...,iM ) is feasible if and only if

N(FiM )T H(i0 ,...,iM ) N(FiM ) < 0;

N(GiM )T H(i0 ,...,iM ) N(GiM ) < 0.

These inequalities are equivalent to (3.15) due to the Schur complement arguments in [20, sect. 5] together with the matrix inverse completion result in [33, Lem. 6.2]. The proof of the latter shows that Xj , j ∈ M− M (Θ), can be reconstructed from Rj and Sj through (3.16). Now, since MM (Θ) ⊂ LM (Θ), the existence of matrices K(i0 ,...,iM ) , such that (3.13) holds for (i0 , . . . , iM ) ∈ LM (Θ), implies the feasibility of (3.15) for (i0 , . . . , iM ) ∈ MM (Θ). To show the converse, suppose that there are a nonnegative integer M and matrices Rj , Sj > 0, j ∈ M− M (Θ), such that (3.15) holds for i = (i0 , . . . , iM ) ∈ MM (Θ). Assume M > 0 without loss of generality. Reconstruct Xj > 0, j ∈ M− M (Θ), through

1346

JI-WOONG LEE AND GEIR E. DULLERUD

(3.16). Then there exist matrices K(i0 ,...,iM ) , (i0 , . . . , iM ) ∈ MM (Θ), such that (3.13), with L = M , holds for (i0 , . . . , iM ) ∈ MM (Θ). By the definition of MM (Θ),  for each j ∈ L0 (Θ), one can choose a switching path ij0 , . . . , ijM −1 ∈ {0, . . . , N }M such that (3.3), with L = M , holds for all θ ∈ Θ. If we put ⎧ ⎨X θ(0)

θ(0)

,

i0

,...,iM −1

i0

,...,iM −1

it

,...,iM −1 ,θ(0),...,θ(t)

t = 0;

XθM (t) =

, 0 < t < M ; ⎩X θ(0) θ(0) it ,...,iM −1 ,θ(0),...,θ(t) ⎧ , ⎨K θ(0) t = 0; θ(0)

KθM (t) =

⎩K θ(0)

θ(0)

 , 0 < t < M,

for all θ M ∈ ΘM and t ≥ 0 such that θM (t) ∈ / MM (Θ), then we recover (3.13) for all (i0 , . . . , iM ) ∈ LM (Θ). The result then follows from Lemma 3.5. Remark 6. Given a nonnegative integer M , the number of systems of linear matrix inequalities (3.15) to solve simultaneously is equal to the cardinality of MM (Θ), and the feasibility of inequalities (3.15) is sufficient for the existence of an admissible Lpath-dependent synthesis for some L ≤ M . However, given a nonnegative integer L, the feasibility of (3.15) for some M ≤ L is sufficient but not necessary for the existence of an admissible L-path-dependent synthesis. In fact, even if the existence of an admissible L-path-dependent controller guarantees that inequalities (3.15) are feasible for some finite M , there is no upper bound on such an M . See Example 3. Remark 7. In the case of reduced order controllers with nK < n, the matrices Rj , Sj , j ∈ M− M (Θ), must satisfy rank

 Ri− I

 I ≤ n + nK , Si−

in addition to (3.15), for all i = (i0 , . . . , iM ) ∈ MM (Θ). Corollary 3.8. Let T be as in (3.6); let Θ ⊂ Ω be nonempty. Suppose that nK ≥ n. There exists a finite-path-dependent linear output feedback controller of order nK that uniformly stabilizes the system (T , Θ) if and only if there exist a nonnegative integer M and matrices Rj , Sj > 0 such that (3.17a)

T T T N(BT iM ) (AiM Ri− AiM − Ri+ )N(BiM ) < 0,

(3.17b)

N(CiM )T (AT iM Si+ AiM − Si− )N(CiM ) < 0,   Ri− I ≥0 I Si−

(3.17c)

for all i = (i0 , . . . , iM ) ∈ N M (Θ). Proof. In (3.15), set the matrices B1,i , C1,i , D11,i , D12,i , and D21,i to zero for all i = 1, . . . , N , and obtain (3.17). The rest of the proof proceeds similarly to that of Corollary 3.4. Remark 8. Corollary 3.8 reverts to the stabilization result in [30] if a strongly connected directed graph defines Θ. Suppose that a set of matrices Ki , i ∈ ML (Θ), is obtained by solving (3.13) via Theorem 3.7 for some nonnegative integer L ≤ M . If L = 0, then it follows from M0 (Θ) = L0 (Θ) that we have all the matrices Ki , i ∈ L0 (Θ), that define an admissible zero-path-dependent controller synthesis. If L > 0, on the other hand,

SWITCHED AND MARKOVIAN JUMP LINEAR SYSTEMS

1347

  then choose switching paths ij0 , . . . , ijL−1 ∈ {0, . . . , N }L , j ∈ L0 (Θ), such that (3.3) holds for all θ ∈ Θ, and put KθL (t) = K θ(0) it

θ(0)

,...,iL−1 ,θ(0),...,θ(t)



whenever θ L ∈ ΘL , t < L, and θL (t) ∈ / ML (Θ); then we recover all matrices Ki , i ∈ LL (Θ), that define an admissible L-path-dependent controller synthesis. Example 3. Let θ (k) be as in Example 2 for positive integers k. Let

and Θ(k)

T = {(1, −1, 1, −1, 1, 1, −1, −1), (1, 0, 0, 0, 0, 0, 0, 0)}

  = θ (k) so that the controlled system T , Θ(k) has the representation

x(t + 1) = x(t) − w(t) + u(t), z(t) = −x(t) + w(t) − u(t), y(t) = x(t) − w(t) in mode 1 (i.e., when θ(k) (t) = 1), and x(t + 1) = x(t),

z(t) = 0,

y(t) = 0

in mode 2 (i.e., when θ(k) (t) = 2). Let M be a nonnegative integer. If M < k, then,   because mode 2 is not stabilizable and because MM Θ(k) contains the switching path (2, . . . , 2) that consists of mode 2 only, inequality (3.15) cannot be satisfied for     all i = (i0 , . . . , iM ) ∈ MM Θ(k) . On the other hand, the cardinality of MM Θ(k)  (k)  for all M > k. Hence, to design an admissible synthesis is equal to that of Mk Θ   (k) , it suffices to consider the single path length M = k in (3.15). It is for T , Θ readily verified that there indeed is an admissible finite-path-dependent controller for   T , Θ(k) for each k. This shows that there does not exist a general upper bound on the path length M in Theorem 3.7.  (k)  In this particular example, it is easy to find solutions Rj , Sj > 0, j ∈ M− , M Θ to (3.15) with M = k, such that L = 0 (which leads to mode-dependent controllers) suffices for (3.13) to be feasible. This indicates that the existence of an admissible L-path controller synthesis does not necessarily lead to the feasibility of (3.15) with M = L. Definition 3.9. Let γ > 0. The system (G, Θ) is said to satisfy uniform disturbance attenuation level γ if there exists a γ˜ ∈ (0, γ) such that, whenever x(t0 ) = 0, t t z(s)2 ≤ γ˜ 2 w(s)2 s=t0

s=t0

for t ≥ t0 ≥ 0, for w ∈  (R ), and for θ ∈ Θ. Definition 3.10. Let γ > 0. The controller (K, ΘL ) is said to be a γ-admissible (L-path-dependent) synthesis (of order nK ), or to achieve uniform disturbance attenuation level γ, for the system (T , Θ) if the closed-loop system (T K , ΘL ) is uniformly exponentially stable and satisfies uniform disturbance attenuation level γ. Given a γ > 0, let

G (γ) = (Ai , γ −1/2 Bi , γ −1/2 Ci , γ −1 Di ) : i = 1, . . . , N .   Then (G, Θ) satisfies uniform disturbance attenuation level γ if and only if G (γ) , Θ is uniformly strictly contractive. Using this fact and Theorem 3.7, and applying the Schur complement formula to (3.15a) and (3.15b), we obtain the following. 2

m

1348

JI-WOONG LEE AND GEIR E. DULLERUD

Corollary 3.11. Let T be as in (3.6), and let Θ ⊂ Ω be nonempty. Let γ > 0. Suppose that nK ≥ n. There exists a γ-admissible finite-path-dependent synthesis of order nK for the system (T , Θ) if and only if there exist a nonnegative integer M and an indexed family {(Rj , Sj ) : j ∈ M− M (Θ)} of pairs of symmetric positive definite matrices Rj , Sj ∈ Rn×n such that

(3.18a)

(3.18b)

 NF,iM 0

⎡ T AiM Ri− AT iM − Ri+ 0 ⎣ C1,iM Ri− AT iM I BT 1,iM

 NG,iM 0

⎡ T T AiM Si+ AiM − Si− 0 ⎢ ⎣ BT 1,iM Si+ AiM I C1,iM  Ri− I

(3.18c)

⎤ AiM Ri− CT B1,iM 1,iM ⎦ C1,iM Ri− CT 1,iM − γI D11,iM DT −γI 11,iM   NF,iM 0 × < 0, 0 I ⎤ AT CT iM Si+ B1,iM 1,iM ⎥ T BT 1,iM Si+ B1,iM − γI D11,iM ⎦ D11,iM

 I ≥0 Si−

−γI   NG,iM 0 × < 0, 0 I

for all i = (i0 , . . . , iM ) ∈ MM (Θ). Moreover, if (3.18) holds for all i = (i0 , . . . , iM ) ∈ MM (Θ), then there exist a nonnegative integer L ≤ M and matrices K(iM −L ,...,iM ) ∈ R(nK +m2 )×(nK +l2 ) such that (3.13) holds for (i0 , . . . , iM ) ∈ MM (Θ) with ⎡ ⎤ i  1,i 0 −X−1 A B M M (i0 ,...,iM )+ ⎢ T ⎥ T −X(i0 ,...,iM )− 0 C A ⎢ iM 1,iM ⎥ , H(i0 ,...,iM ) = ⎢ (3.19) ⎥ T T ⎣ ⎦ 0 −γI D B 1,iM 11,iM  1,i 0 C D11,i −γI M

M

where the matrices Xj are reconstructed via (3.16) for j ∈ M− M (Θ); in particular, one may take L = M . Example 4. Let T = {(0.3, 0, 0, 1, 1, 0, 1, 0), (3, 0.5, 1, 1, 1, 0, 1, 0)}; let Θ be as in Example 1. Then the controlled system (T , Θ) has the representation x(t + 1) = 0.3x(t),

z(t) = x(t) + u(t),

y(t) = x(t)

in mode 1, and x(t + 1) = 3x(t) + 0.5w(t) + u(t),

z(t) = x(t) + u(t),

y(t) = x(t)

in mode 2. With M = 0, the system of linear matrix inequalities (3.18) is feasible for any γ > 1; it is easy to see that, if γ = 1, then inequalities (3.18) are not feasible for any M ≥ 0. Solving the semidefinite program of minimizing γ subject to (3.18), and applying Corollary 3.11, we obtain a mode-dependent controller (K, Θ0 ) with     0 0 0 0 , K2 = . K1 = 0 −1 0 −3

SWITCHED AND MARKOVIAN JUMP LINEAR SYSTEMS

1349

The resulting controller (K, Θ0 ) has the representation −y(t) if θ(t) = 1, u(t) = −3y(t) if θ(t) = 2. This controller is optimal in the sense that it achieves any uniform disturbance attenuation level greater than one, and that no finite-path-dependent linear dynamic output feedback controller achieves the uniform disturbance attenuation level equal to one. (The optimal controller is static in this case because C2,1 = C2,2 = 1 and D21,1 = D21,2 = 0 lead to perfect observation of the state.) The controller, however, is not optimal under the notion of path-by-path disturbance attenuation—see Example 7. 4. Control of Markovian jump linear systems. In Markovian jump linear systems, the switching sequence is modeled as a finite-state homogeneous Markov chain. Let G be as in (3.1). Let p = (pi ) ∈ R1×N be a row vector whose entries are nonnegative and sum to one; let P = (pij ) ∈ RN ×N be a (row) stochastic matrix so that each row of P has nonnegative entries that sum to one. Then the discrete-time Markovian jump linear system, defined by the triple (G, P, p), has the representation (3.2). Here, the switching sequence θ is a realization of the Markov chain defined by the pair (P, p), where P is the transition probability matrix and p the initial distribution. The state θ(t) of the chain (P, p) at time t defines the mode of (G, P, p) at time t; the distribution of the mode at time t is given by pPt . As in the previous section, let Ω be the space of all infinite sequences in {1, . . . , N }. Let P be the unique consistent probability measure [39] on Ω such that P { θ(t + 1) = j | θ(t) = i } = pij ,

P { θ(0) = i } = pi

for all i, j, and t. Definition 4.1. The system (G, P, p) is said to be almost surely uniformly (exponentially) stable if there exists a set Θ ⊂ Ω with P (Θ) = 1 such that the system (G, Θ) is uniformly exponentially stable. Definition 4.2. The system (G, P, p) is said to be almost surely uniformly (strictly) contractive if there exists a set Θ ⊂ Ω with P (Θ) = 1 such that the system (G, Θ) is uniformly strictly contractive. A switching sequence θ in {1, . . . , N } is said to be admissible with respect to (P, p) if pθ(0) > 0 and pθ(t)θ(t+1) > 0 for t ≥ 0. If we define Θ(P, p) = {θ ∈ Ω : θ is admissible with respect to (P, p)} and let ML (P, p) = ML (Θ(P, p)),

− M− L (P, p) = ML (Θ(P, p))

for nonnegative integers L, then we have P (Θ(P, p)) = 1; on the other hand, whenever (i0 , . . . , iL ) ∈ ML (P, p), we have that P {θ ∈ Ω : (i0 , . . . , iL ) ∈ LL ({θ})} > 0 so that ML (P, p) ⊂ ML (Θ) whenever Θ ⊂ Ω and P (Θ) = 1. Example 5. Let N = 3, and let (P, p) be a Markov chain with ⎡ ⎤ 1 1 1 1(P) = ⎣0 0 1⎦ . 0 0 1

1350

JI-WOONG LEE AND GEIR E. DULLERUD

If 1(p) = [1 0 0], then L0 (P, p) = M0 (P, p) = {1, 2, 3}; L1 (P, p) = {01, 11, 12, 13, 23, 33}, M1 (P, p) = {11, 12, 13, 23, 33}, and so on. However, if 1(p) = [0 1 1], then L0 (P, p) = M0 (P, p) = {2, 3}; L1 (P, p) = {02, 03, 23, 33}, M1 (P, p) = {02, 23, 33}, and so on. Theorem 4.3. Let G be as in (3.1); let (P, p) be a Markov chain. The system (G, P, p) is almost surely uniformly exponentially stable and almost surely uniformly strictly contractive if and only if there exist a nonnegative integer M and an indexed n×n family {Xj : j ∈ M− such M (P, p)} of symmetric positive definite matrices Xj ∈ R that (3.4) holds for all M -paths i = (i0 , . . . , iM ) ∈ MM (P, p). Proof. The result is immediate from Theorem 3.3: sufficiency follows from P (Θ(P, p)) = 1, and necessity from the fact that Θ ⊂ Ω and P (Θ) = 1 implies ML (P, p) ⊂ ML (Θ). Remark 9. Define  nj (L) ni (0) = 1, ni (L + 1) = {j : pij >0}

for i ∈ {1, . . . , N } and for L = 0, 1, . . . . Given a Markov chain (P, p), let S(p) = {j : pj > 0},



(k) T (P, p) = j : pi pij > 0 for some (i, k) ,

 (k)  where Pk = pij is the k-step transition probability matrix [24]. Then, in Theorem 4.3, the number of linear matrix inequalities (3.4) to solve simultaneously, with a fixed M , is equal to the cardinality of MM (P, p), which is precisely given by M −1

 j∈S(p)\T (P,p)

k=0

nj (k) +

 j∈S(p)∪T (P,p)

nj (M ).

In particular, if P is irreducible (i.e., if the directed graph of P is strongly connected— N see, e.g., [25]), then the cardinality of MM (P, p) is equal to j=1 nj (M ). Remark 10. Theorem 4.3 implies that the Markovian jump linear system (G, P, p) is almost surely uniformly stable and contractive if and only if the switched linear system (G, Θ(P, p)) is uniformly stable and contractive. Therefore, Markovian jump linear systems can be treated as if they are switched linear systems. Moreover, since the set Θ(P, p) depends only on the sparsity patterns of P and p, the almost sure uniform stability and contractiveness of (G, P, p) is robust against sparsity pattern– preserving deviations from P and p. Remark 11. When only the almost sure uniform stability is considered, it is immediate from Corollary 3.4 that it suffices to consider the “irreducible parts” of the Markov chain (P, p); see [30]. Let T be as in (3.6), and let (P, p) be a Markov chain. Then the triple (T , P, p) defines the controlled Markovian jump linear system described by (3.7), where θ is a realization of (P, p). As in the previous section, we make the standard assumption that the state θ(t) of the chain (P, p) is perfectly observed at each time instant t; we consider all finite-path-dependent controllers.

SWITCHED AND MARKOVIAN JUMP LINEAR SYSTEMS

1351

Fix a nonnegative integer L. Let K = {(AK,i , BK,i , CK,i , DK,i ) : i ∈ LL (Θ(P, p))}. Then the pair (K, Θ(P, p)L ) defines an L-path-dependent controller, whose representation is given by (3.9). Label the L-paths in LL (Θ(P, p)) in dictionary order from 1 to NL , where NL is the cardinality of LL (Θ(P, p)). Define P(L) = (qij ) ∈ RNL ×NL as follows: Whenever (i0 , . . . , iL ) and (j0 , . . . , jL ) are L-paths labeled i and j, respectively, set qij = piL jL if (i0 , . . . , iL )+ = (j0 , . . . , jL )− ; otherwise, set qij = 0. Also, define a row vector p(L) = (qi ) ∈ RNL as follows: Whenever (i0 , . . . , iL ) is an L-path labeled i, set qi = piL if (i0 , . . . , iL )− = (0, . . . , 0); otherwise, set qi = 0. Then the pair (P(L) , p(L) ) defines the L-path Markov chain generated by (P, p). Consequently, the closed-loop system, given by the triple (T K , P(L) , p(L) ) with TK=





 i, C  i, D  i : i ∈ LL (Θ(P, p)) ,  i, B A

is a Markovian jump linear system whose representation is of the form (3.11) for each realization θ L of (P(L) , p(L) ). Definition 4.4. Let γ > 0. The system (G, P, p) is said to satisfy almost sure uniform disturbance attenuation level γ if there exists a set Θ ⊂ Ω with P (Θ) = 1 such that the system (G, Θ) satisfies uniform disturbance attenuation level γ. Definition 4.5. Let γ > 0. The controller (K, Θ(P, p)L ) is said to be a γadmissible (L-path-dependent) synthesis (of order nK ), or to achieve almost sure uniform disturbance attenuation level γ, for the system (T , P, p) if the closed-loop system (T K , P(L) , p(L) ) satisfies almost sure uniform disturbance attenuation level γ. Theorem 4.6. Let T be as in (3.6); let (P, p) be a Markov chain. Let γ > 0. Suppose that nK ≥ n. There exists a γ-admissible finite-path-dependent synthesis of order nK for the system (T , P, p) if and only if there exist a nonnegative integer M and an indexed family {(Rj , Sj ) : j ∈ M− M (P, p)} of pairs of symmetric positive definite matrices Rj , Sj ∈ Rn×n such that (3.18) holds for all M -paths i = (i0 , . . . , iM ) ∈ MM (P, p). Moreover, if (3.18) holds for all i = (i0 , . . . , iM ) ∈ MM (P, p), then there exist a nonnegative integer L ≤ M and matrices K(iM −L ,...,iM ) ∈ R(nK +m2 )×(nK +l2 ) such that (3.13), with (3.19), holds for (i0 , . . . , iM ) ∈ MM (P, p), where the matrices Xj are given by (3.16) for j ∈ M− M (P, p); in particular, one may take L = M . Proof. The result immediately follows from Corollary 3.11. Remark 12. The existence of finite-path-dependent controller syntheses achieving an almost sure uniform disturbance attenuation level for the Markovian jump linear system (T , P, p) is robust against sparsity pattern–preserving deviations from P and p. The conservatism associated with this robustness property can be reduced via path-by-path disturbance attenuation—see the examples in the next section. Example 6. Let the Markov chain (P, p) have ⎡ ⎤ 0 1 1   (4.1) 1(P) = ⎣0 0 1⎦ , 1(p) = 1 1 1 . 1 0 0 Then we have M0 (P, p) = {1, 2, 3},

M1 (P, p) = {12, 13, 23, 31},

M2 (P, p) = {123, 131, 231, 312, 313},

1352

JI-WOONG LEE AND GEIR E. DULLERUD

and so on. Let T = {(0.5, 0, 0, 0, 0, 0, 0, 0), (3, 0, 0, 0, 0, 0, 0, 0), (2, 1, 1, 1, 1, 1, 1, 1)} so that the controlled Markovian jump linear system (T , P, p) has the representation x(t + 1) = 0.5x(t), z(t) = 0, y(t) = 0; x(t + 1) = 3x(t), z(t) = 0, y(t) = 0; or x(t + 1) = 2x(t) + w(t) + u(t), z(t) = x(t) + w(t) + u(t), y(t) = x(t) + w(t), depending on whether the mode at time t is 1, 2, or 3, respectively. With M = 0, the system of linear matrix inequalities (3.18), over i0 ∈ M0 (P, p), is not feasible for any γ > 0. However, with M = 1, the semidefinite program of minimizing γ subject to (3.18) over (i0 , i1 ) ∈ M1 (P, p) leads to γ = 0.834; the same is true for all M > 1. Setting L = M = 1 in (3.13) with (3.19), we obtain       0 0 0 0 0 0.0000 , K(1,3) = , K(2,3) = (4.2) K(1,2) = K(3,1) = . 0 0 0 −1 0 −1.5555 The resulting one-path-dependent controller is optimal (up to the third digit below the decimal point) in the sense that no finite-path-dependent linear dynamic output feedback controller achieves the disturbance attenuation level 0.833. This controller is applied as follows: For t = 0, use K(3,1) if θ(0) = 1, use K(1,2) if θ(0) = 2, and use either K(1,3) or K(2,3) if θ(0) = 3; for t > 0, use K(θ(t−1),θ(t)) . It turns out that setting L = 0 and M = 1 in (3.13) with (3.19) also works and results in a mode-dependent optimal controller with     0 0 0 0.0000 K1 = K 2 = (4.3) , K3 = . 0 0 0 −1.5555 This mode-dependent controller, however, is not optimal under a path-by-path disturbance attenuation criterion—see Example 8. 5. Path-by-path disturbance attenuation. In this section, we formulate a refined disturbance attenuation problem for switched linear systems. The result holds for Markovian jump linear systems as well. We introduce the notion of path-by-path disturbance attenuation that improves upon the uniform disturbance attenuation performance presented in previous sections. It turns out that, under the notion of pathby-path disturbance attenuation, finite-path-dependent controllers can outperform mode-dependent ones—see Examples 7 and 8. We shall use the same notation as in section 3. Definition 5.1. If there exist a nonnegative integer M , a positive integer nK , an indexed family Γ = {γi : i ∈ MM (Θ)} of positive numbers γi , and an indexed family {(Rj , Sj ) : j ∈ M− M (Θ)} of pairs of symmetric positive definite matrices Rj , Sj ∈ Rn×n such that

1353

SWITCHED AND MARKOVIAN JUMP LINEAR SYSTEMS

(5.1a)

(5.1b)

 NF,iM 0

 NG,iM 0

" (5.1c)

⎡ T T AiM Ri− AiM − Ri+ 0 ⎢ ⎣ C1,iM Ri− AT iM I BT 1,iM

⎡ T T AiM Si+ AiM − Si− 0 ⎢ ⎣ BT 1,iM Si+ AiM I C1,iM

Ri−

I

I

Si−

#

AiM Ri− CT 1,iM C1,iM Ri− CT 1,iM − γi I

⎤

⎥ D11,iM ⎦

−γi I   NF,iM 0 × < 0, 0 I ⎤ AT CT iM Si+ B1,iM 1,iM ⎥ T BT 1,iM Si+ B1,iM − γi I D11,iM ⎦ DT 11,iM

D11,iM

$ ≥0

B1,iM

and, if nK < n,

rank

 Ri− I

−γi I   NG,iM 0 × < 0, 0 I  % I ≤ n + nK Si−

for all M -paths i = (i0 , . . . , iM ) ∈ MM (Θ), then an L-path-dependent controller (K, ΘL ) (of order nK ) is said to achieve path-by-path disturbance attenuation level Γ for the system (T , Θ) whenever it is constructed in the following manner: Choose a nonnegative integer L ≤ M , symmetric positive definite matrices X(i0 ,...,iM )− ∈ R(n+nK )×(n+nK ) , and matrices K(iM −L ,...,iM ) ∈ R(nK +m2 )×(nK +l2 ) such that (3.13) holds for all M -paths (i0 , . . . , iM ) ∈ MM (Θ) with

(5.2)

H(i0 ,...,iM )

⎡ −X−1 (i0 ,...,iM )+ ⎢ T A ⎢ iM =⎢ T ⎣ B 1,iM

0

i A M

 1,i B M

−X(i0 ,...,iM )− 0  1,i C

0

M

−γ(i0 ,...,iM ) I D11,iM

0

⎤

⎥ T C ⎥ 1,iM ⎥. T D11,iM ⎦ −γ(i0 ,...,iM ) I

The only difference between the synthesis conditions for the uniform disturbance attenuation and path-by-path disturbance attenuation is that, while a single variable γ is used in (3.18) and (3.19) for the former, multiple variables γ(i0 ,...,iM ) are used in (5.1) and (5.2) for the latter. Suppose that there exists a finite-path-dependent controller synthesis of order nK = n for (T , Θ) that achieves a uniform disturbance attenuation level γ0 > 0, and that the system of inequalities (3.18) is feasible with M = M0 . Then one can improve the closed-loop performance further by obtaining a path-bypath optimal controller synthesis through semidefinite programming as follows. Algorithm 5.2. Step 0. Set nK =  n. Choose γ ≥ γ0 , M ≥ M0 , and λi > 0, i ∈ MM (Θ). Step 1. Minimize i∈MM (Θ) λi γi subject to (5.1), with the additional constraint γi ≤ γ, for i = (i0 , . . . , iM ) ∈ MM (Θ). Step 2. Reconstruct Xj from (Rj , Sj ) via (3.16) for all j ∈ M− M (Θ). Step 3. Solve (3.13) with (5.2) for all (i0 , . . . , iM ) ∈ MM (Θ), and obtain a nonnegative integer L ≤ M and matrices K(iM −L ,...,iM ) . The path-by-path optimal controller (K, ΘL ) resulting from Algorithm 5.2 shall be said to be M -path Pareto optimal because of the following property: If (K, ΘL ) achieves path-by-path disturbance attenuation level {γi : i ∈ MM (Θ)}, then no M -path-dependent controller can achieve a disturbance attenuation level {˜ γi : i ∈ MM (Θ)} such that γ˜i ≤ γi for all i ∈ MM (Θ) and such that γ˜i < γi for some

1354

JI-WOONG LEE AND GEIR E. DULLERUD

i ∈ MM (Θ). In general, different sets of weights λi , i ∈ MM (Θ), result in different interpretations of optimality—see Examples 7 and 8—and hence different Pareto optimal path-by-path disturbance attenuation levels—see Example 9. Example 7. Let us revisit the controlled switched linear system (T , Θ) considered in Example 4. In this example, we set γ = 103 and λi = 1, i ∈ MM (Θ), for Algorithm 5.2. Running Algorithm 5.2 with M = 0, it turns out that the mode-dependent controller obtained in Example 4 achieves any path-by-path disturbance attenuation level {γi } satisfying γ1 > 0 and γ2 > 1. In particular, this mode-dependent controller is γ1 -admissible for (T , (1, 1, . . . )) and γ2 -admissible for (T , {(1, 2, 1, 2, . . . ), (2, 2, . . . )}) whenever γ1 > 0 and γ2 > 1. Running Algorithm 5.2 with M = 1 leads to γ(1,1) = γ(2,1) = 0 and γ(1,2) = 0.795, but γ(2,2) = 1.16, which is greater than one. However, running Algorithm 5.2 with M = 2 leads to a one-path-dependent controller (L = 1) with K(0,1) = K(1,1) = K(2,1) =

 0 0

 0 , −1

K(1,2) =

 0 0

 0 , −1

K(2,2) =

 0 0

 0 . −3

This controller is two-path Pareto optimal and achieves any path-by-path disturbance attenuation level {γi : i ∈ M2 (Θ)} such that γi > 1 for i = (2, 2, 2), and γi > 0 otherwise. In particular, this one-path-dependent controller is γ1 -admissible for (T , {(1, 1, . . . ), (1, 2, 1, 2, . . . )}) and γ2 -admissible for (T , (2, 2, . . . )) whenever γ1 > 0 and γ2 > 1. Clearly, this is an improvement over the path-by-path performance of the mode-dependent controller obtained in Example 4 with M = 0. In fact, it is not difficult to see that, in this example, no mode-dependent controller can achieve a pathby-path disturbance attenuation level {γi : i ∈ M2 (Θ)} such that, say, γi = 0.01 for i = (2, 2, 2). Example 8. This example revisits the controlled Markovian jump linear system (T , P, p) considered in Example 6. Since 1(P5 ) = 1, the Markov chain (P, p) is irreducible and aperiodic [25]. Let π = [π1 π2 π3 ] be the unique steady-state distribution of the chain. For each nonnegative integer M , let (5.3)

λ(i0 ,...,iM ) = πi0 pi0 i1 · · · piM −1 iM ,

(i0 , . . . , iM ) ∈ MM (P, p).

Then Algorithm 5.2 minimizes the “average steady-state disturbance attenuation level over M -paths,” where each M -path (i0 , . . . , iM ) is interpreted as the mode iM preceded by the path (i0 , . . . , iM −1 ). With γ = ∞ and M = 1, we obtain the one-pathdependent controller (L = 1) given by (4.2). This controller achieves the path-bypath disturbance attenuation level {γi : i ∈ M1 (P, p)} where, e.g., γ(1,2) = γ(1,3) = γ(3,1) = 0.001 and γ(2,3) = 0.834; the weighted sum of these disturbance attenuation levels γi is 0.001 + 0.833π2 . On the other hand, the mode-dependent controller given by (4.3) achieves the path-by-path disturbance attenuation level {˜ γi : i ∈ M1 (P, p)} where, e.g., γ˜(1,2) = γ˜(3,1) = 0.001 and γ˜(1,3) = γ˜(2,3) = 0.834. Clearly, the one-pathdependent controller performs better than the mode-dependent controller in terms of average steady-state performance. In fact, it is not difficult to see that, for some chain (P, p) with (4.1), there does not exist a mode-dependent controller that achieves the average steady-state performance level 0.001 + 0.833π2 . Example 9. We are to balance the Pendubot [40], a two-link planar robot with revolute joints and actuation at the shoulder, subject to random but bounded delays in the feedback loop from the relative angular position sensor to the actuator. We

1355

SWITCHED AND MARKOVIAN JUMP LINEAR SYSTEMS

use the following linearized model (of order 4) borrowed from [43]: x(t + 1) = Ax(t) + B1 w(t) + B2 u(t), z(t) = C1 x(t) + D11 w(t) + D12 u(t), y(t) = C2 x(t) + D21 w(t); ⎤ ⎡ ⎤ ⎡ ⎤ 0.0050 0.0003 0.0000 −0.0001 0.0006 ⎢ ⎥ ⎢ ⎥ 0.9992 0.1242 0.0003⎥ ⎥ , B1 = ⎢−0.0232⎥ , B2 = ⎢ 0.2243⎥ , ⎦ ⎣ ⎦ ⎣ 0.0000 1.0007 0.0050 0.0012 −0.0001⎦ 0.0008 0.2786 1.0007 0.4742 −0.0232 ⎤ ⎡ ⎤ ⎡ ⎤     0 0 0 1 0 0 0 0 ⎦ ⎣ ⎦ ⎣ ⎦ 0 , D11 = 0 , D12 = 0 , C2 = , D21 = . 0 0 1 0 0 0 0 1

⎡

0.9992 ⎢−0.3369 A=⎢ ⎣ 0.0008 0.3263 ⎡ 1 0 0 C1 = ⎣0 0 1 0 0 0

The random delays in the feedback loop are modeled as a Markov chain such that the amount of delay at time t is given by θ(t) − 1 if θ(t) is the state of the chain at time t. If the maximum possible amount of delay is N − 1, then the augmented state  T x ˆ(t) = x(t)T y(t − 1)T · · · y(t − N + 1)T and delayed measurement yˆ(t) yield the Markovian jump linear system (of order 14) ˆ(t) + B1,θ(t) w(t) + B2,θ(t) u(t), x ˆ(t + 1) = Aθ(t) x z(t) = C1,θ(t) x ˆ(t) + D11,θ(t) w(t) + D12,θ(t) u(t), yˆ(t) = C2,θ(t) x ˆ(t) + D21,θ(t) w(t), where

⎡

A ⎢C2 ⎢ ⎢ Ai = ⎢ 0 ⎢ .. ⎣ . C1,i

0 ··· 0 ··· I ··· .. . . . . 0 ···

0  = C1

0 0

for i = 1, . . . , N , and ⎤ ⎡ ⎡ C2 C2,1 ⎢ C2,2 ⎥ ⎢ 0 ⎥ ⎢ ⎢ ⎢ .. ⎥ = ⎢ .. ⎣ . ⎦ ⎣ . C2,N 0

⎤ 0 0 0 0⎥ ⎥ 0 0⎥ ⎥, .. .. ⎥ . .⎦ I 0 ···

⎡

B1,i

B2,i

⎡ ⎤ B2 ⎢0⎥ ⎢ ⎥ ⎢ ⎥ = ⎢ 0 ⎥, ⎢ .. ⎥ ⎣ . ⎦ 0

0



0 ,

0 ··· I ··· .. . . . . 0 ···

⎤ B1 ⎢D21 ⎥ ⎢ ⎥ ⎢ ⎥ = ⎢ 0 ⎥, ⎢ .. ⎥ ⎣ . ⎦

D11,i = D11 , ⎤ 0 0⎥ ⎥ .. ⎥ , .⎦ I

D12,i = D12

⎤ ⎡ ⎤ D21 D21,1 ⎢ D21,2 ⎥ ⎢ 0 ⎥ ⎥ ⎢ ⎥ ⎢ ⎢ .. ⎥ = ⎢ .. ⎥ . ⎣ . ⎦ ⎣ . ⎦ ⎡

D21,N

0

The measurements are time-stamped, so the controller perfectly observes θ(t) at each time t. Let N = 6. Let the transition probability matrix P of the Markov chain be ⎡ ⎤ 0.3 0.7 0 0 0 0 ⎢0.2 0.2 0.6 0 0 0⎥ ⎢ ⎥ ⎢0.2 0.2 0.2 0.4 0 0⎥ ⎢ ⎥, P=⎢ 0⎥ ⎢0.2 0.2 0.2 0.2 0.2 ⎥ ⎣0.1 0.1 0.1 0.1 0.3 0.3⎦ 0.1 0.1 0.1 0.1 0.3 0.3

1356

JI-WOONG LEE AND GEIR E. DULLERUD

where the fact that the controller can always use the most recent measurement is accounted for [42]; the initial distribution p is assumed to be equal to the unique steady-state distribution   π = 0.2142 0.2999 0.2699 0.1440 0.0504 0.0216 . The semidefinite program minimizing the uniform disturbance attenuation level γ subject to (3.18) over i = (i0 , . . . , iM ) ∈ MM (P, p) yields approximately γ = 11.1 for M = 0 and γ = 10.6 for M ≥ 1. Now running Algorithm  5.2 with γ = 100, M = 1, and λi as in (5.3) yields γi , i ∈ M1 (P, p), such that i∈M1 (P,p) λi γi = 9.96, where, if Λ = (λ(i,j) ) and Γ = (γ(i,j) ), then ⎡ 0.0643 0.1499 0 ⎢0.0600 0.0600 0.1799 ⎢ ⎢0.0540 0.0540 0.0540 Λ=⎢ ⎢0.0228 0.0228 0.0228 ⎢ ⎣0.0050 0.0050 0.0050 0.0022 0.0022 0.0022 ⎡ 9.52 9.56 0 ⎢9.48 9.52 9.39 ⎢ ⎢9.61 9.65 9.53 Γ=⎢ ⎢10.1 9.99 9.87 ⎢ ⎣17.7 17.7 17.6 27.9 28.0 27.7

⎤ 0 0 0 0 0 0⎥ ⎥ 0.1080 0 0⎥ ⎥, 0.0228 0.0228 0⎥ ⎥ 0.0050 0.0151 0.0151⎦ 0.0022 0.0065 0.0065 ⎤ 0 0 0 0 0 0⎥ ⎥ 9.24 0 0⎥ ⎥. 9.57 10.1 0⎥ ⎥ 17.1 10.4 10.9⎦ 27.2 18.9 11.5

On the other hand, running Algorithm 5.2 with γ = 100, M = 1, and λi = 1, i ∈ M1 (P, p), yields the disturbance attenuation levels γ˜i , i ∈ M1 (P, p), such that 1 ˜i = 10.3, where N1 is the cardinality of M1 (P, p) and equals 26. If i∈M1 (P,p) γ N1  Γ = (˜ γ(i,j) ), then ⎡ 13.3 ⎢9.90 ⎢ ⎢  = ⎢7.05 Γ ⎢5.74 ⎢ ⎣5.11 5.32

16.9 12.9 8.86 6.75 5.80 5.68

⎤ 0 0 0 0 17.7 0 0 0⎥ ⎥ 12.4 16.9 0 0⎥ ⎥. 8.84 12.0 16.0 0⎥ ⎥ 7.05 9.09 11.9 13.5⎦ 6.75 8.61 11.2 12.1

 are very different The two sets of disturbance attenuation levels given by Γ and Γ from each other, yet they are both one-path Pareto optimal. 6. Conclusion. This paper dealt with switched linear systems and Markovian jump linear systems in the discrete-time domain and developed complete conditions for (almost sure) uniform disturbance attenuation and (almost sure) path-by-path disturbance attenuation. These conditions naturally give rise to finite-path-dependent controllers and admit semidefinite programming algorithms for optimal dynamic output feedback controller synthesis. Limitations of these algorithms include that, in the worst case, the computational complexity grows exponentially in the number M of past modes that the optimal controller recalls, and that nonexistence of an admissible controller synthesis is not guaranteed to be correctly determined after a finite amount of computation. These limitations are due to the problem’s nature and considered

SWITCHED AND MARKOVIAN JUMP LINEAR SYSTEMS

1357

unavoidable; nevertheless, they will not pose difficulties in most cases because M is usually very small. There is at least one conceptually important question unanswered in this paper. This question is whether mode-dependent controllers perform as well as finitepath-dependent controllers as long as uniform disturbance attenuation is concerned; a more specific question is whether, given a Markovian jump linear system, there exists a stabilizing mode-dependent controller whenever a finite-path-dependent controller can stabilize the system. Under the notion of path-by-path disturbance attenuation, however, we showed that finite-path-dependent controllers can outperform mode-dependent ones. The optimal disturbance attenuation example in Example 9 showed that our results provide a new contribution to the study of networked control systems. The proposed controller synthesis techniques are expected to be applicable to other timedelay systems. Related to switched and Markovian jump linear systems are linear parameter-varying systems, where the common design approaches for gain scheduling [5, 2, 3] are similar to those for mode-dependent controllers. A possible future research direction is to investigate if the approach of finite-path-dependent controller synthesis applies to the control of linear parameter-varying systems. REFERENCES [1] B. D. O. Anderson and J. B. Moore, Detectability and stabilizability of time-varying discretetime linear systems, SIAM J. Control Optim., 19 (1981), pp. 20–32. [2] P. Apkarian and P. Gahinet, A convex characterization of gain-scheduled H∞ controllers, IEEE Trans. Automat. Control, 40 (1995), pp. 853–864. [3] P. Apkarian, P. Gahinet, and G. Becker, Self-scheduled H∞ control of linear parametervarying systems: A design example, Automatica J. IFAC, 31 (1995), pp. 1251–1261. [4] T. Bas¸ar, A dynamic games approach to controller design: Disturbance rejection in discretetime, IEEE Trans. Automat. Control, 36 (1991), pp. 936–952. [5] G. Becker and A. Packard, Robust performance of linear parametrically varying systems using parametrically-dependent linear feedback, Systems Control Lett., 23 (1994), pp. 205– 215. [6] P.-A. Bliman and G. Ferrari-Trecate, Stability analysis of discrete-time switched systems through Lyapunov functions with nonminimal state, in Proceedings of the IFAC Conference on the Analysis and Design of Hybrid Systems, Elsevier, Oxford, UK, 2003, pp. 325–330. [7] V. D. Blondel and A. Megretski, eds., Unsolved Problems in Mathematical Systems and Control Theory, Princeton University Press, Princeton, NJ, 2004. [8] V. D. Blondel, J. Theys, and A. A. Vladimirov, An elementary counterexample to the finiteness conjecture, SIAM J. Matrix Anal. Appl., 24 (2003), pp. 963–970. [9] M. S. Branicky, Multiple Lyapunov functions and other analysis tools for switched and hybrid systems, IEEE Trans. Automat. Control, 43 (1998), pp. 475–482. [10] Y.-Y. Cao and J. Lam, Stochastic stabilizability and H∞ control for discrete-time jump linear systems with time delay, J. Franklin Inst., 336 (1999), pp. 1263–1281. [11] O. L. V. Costa and M. D. Fragoso, Stability results on discrete-time linear systems with Markovian jumping parameters, J. Math. Anal. Appl., 179 (1993), pp. 154–178. [12] I. Daubechies and J. C. Lagarias, Sets of matrices all infinite products of which converge, Linear Algebra Appl., 161 (1992), pp. 227–263. [13] D. P. de Farias, J. C. Geromel, J. B. R. do Val, and O. L. V. Costa, Output feedback control of Markov jump linear systems in continuous-time, IEEE Trans. Automat. Control, 45 (2000), pp. 944–949. [14] G. de Nicolao, On the time-varying Riccati difference equation of optimal filtering, SIAM J. Control Optim., 30 (1992), pp. 1251–1269. [15] C. E. de Souza, On stabilizing properties of solutions of the Riccati difference equation, IEEE Trans. Automat. Control, 34 (1989), pp. 1313–1316. [16] J. C. Doyle, K. Glover, P. P. Khargonekar, and B. A. Francis, State-space solutions to standard H2 and H∞ control problems, IEEE Trans. Automat. Control, 34 (1989), pp. 831–847.

1358

JI-WOONG LEE AND GEIR E. DULLERUD

[17] G. E. Dullerud and S. Lall, A new approach for analysis and synthesis of time-varying systems, IEEE Trans. Automat. Control, 44 (1999), pp. 1486–1497. [18] G. E. Dullerud and F. Paganini, A Course in Robust Control Theory: A Convex Approach, Springer-Verlag, New York, 2000. [19] M. D. Fragoso, J. B. Ribeiro do Val, and D. L. Pinto, Jr., Jump linear H ∞ control: The discrete-time case, Control Theory Adv. Tech., 10 (1995), pp. 1459–1474. [20] P. Gahinet and P. Apkarian, A linear matrix inequality approach to H∞ control, Internat. J. Robust Nonlinear Control, 4 (1994), pp. 421–448. [21] L. Gurvits, Stability of discrete linear inclusion, Linear Algebra Appl., 231 (1995), pp. 47–85. [22] A. Halanay and V. Ionescu, Time-Varying Discrete Linear Systems, Oper. Theory Adv. Appl. 68, Birkh¨ auser, Basel, Switzerland, 1994. [23] J. Hespanha, Root-mean-square gains of switched linear systems, IEEE Trans. Automat. Control, 48 (2003), pp. 2040–2045. [24] P. G. Hoel, S. C. Port, and C. J. Stone, Introduction to Stochastic Processes, Waveland Press, Long Grove, IL, 1987. [25] R. A. Horn and C. R. Johnson, Matrix Analysis, Cambridge University Press, Cambridge, UK, 1985. [26] P. A. Iglesias and K. Glover, State-space approach to discrete-time H∞ control, Internat. J. Control, 54 (1991), pp. 1031–1073. [27] P. A. Iglesias and M. A. Peters, On the induced norms of discrete-time and hybrid timevarying systems, Internat. J. Robust Nonlinear Control, 7 (1997), pp. 811–833. [28] Y. Ji and H. J. Chizeck, Controllability, stability, and continuous-time Markovian jump linear quadratic control, IEEE Trans. Automat. Control, 35 (1990), pp. 777–788. [29] Y. Ji, H. J. Chizeck, X. Feng, and K. A. Loparo, Stability and control of discrete-time jump linear systems, Control Theory Adv. Tech., 7 (1991), pp. 247–270. [30] J.-W. Lee and G. E. Dullerud, Uniform stabilization of discrete-time switched and Markovian jump linear systems, Automatica J. IFAC, 42 (2006), pp. 205–218. [31] D. Liberzon, Switching in Systems and Control, Birkh¨ auser Boston, Boston, MA, 2003. [32] M. Mariton, Jump Linear Systems in Automatic Control, Marcel Dekker, New York, 1990. [33] A. Packard, Gain scheduling via linear fractional transformations, Systems Control Lett., 22 (1994), pp. 79–92. [34] Z. Pan and T. Bas¸ar, H ∞ -control of Markovian jump systems and solutions to associated piecewise-deterministic differential games, in New Trends in Dynamic Games and Applications, G. J. Olsder, ed., Birkh¨ auser Boston, Boston, MA, 1995, pp. 61–94. [35] A. Rantzer, On the Kalman–Yakubovich–Popov lemma, Systems Control Lett., 28 (1996), pp. 7–10. [36] R. Ravi, K. M. Nagpal, and P. P. Khargonekar, H ∞ control of linear time-varying systems: A state-space approach, SIAM J. Control Optim., 29 (1991), pp. 1394–1413. [37] P. Seiler and R. Sengupta, A bounded real lemma for jump systems, IEEE Trans. Automat. Control, 48 (2003), pp. 1651–1654. [38] P. Seiler and R. Sengupta, An H∞ approach to networked control, IEEE Trans. Automat. Control, 50 (2005), pp. 356–364. [39] A. N. Shiryayev, Probability, Springer-Verlag, New York, 1996. [40] M. W. Spong and D. J. Block, The Pendubot: A mechatronic system for control research and education, in Proceedings of the 34th IEEE Conference on Decision and Control, Vol. 1, IEEE, Piscataway, NJ, 1995, pp. 555–556. [41] L. Vandenberghe and S. Boyd, Semidefinite programming, SIAM Rev., 38 (1996), pp. 49–95. [42] L. Xiao, A. Hassibi, and J. P. How, Control with random communication delays via a discrete-time jump system approach, in Proceedings of the American Control Conference, Vol. 3, IEEE, Los Alamitos, CA, 2000, pp. 2199–2204. [43] B. Yoo, Real-Time Control of Dynamic Systems Over Communication Networks, Master’s thesis, University of Illinois at Urbana-Champaign, Urbana, IL, 2003. [44] G. Zhai, B. Hu, K. Yasuda, and A. N. Michel, Disturbance attenuation properties of timecontrolled switched systems, J. Franklin Inst., 338 (2001), pp. 765–779.