Stabilizability and positiveness of solutions of the jump linear ...
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 50, NO. 5, MAY 2005
691
Stabilizability and Positiveness of Solutions of the Jump Linear Quadratic Problem and the Coupled Algebraic Riccati Equation
W-detectability is the weakest condition that associates finite cost and mean square stability. A characterization developed here relates the zero cost solution of the JLQ problem with the kernel of the observability matrices introduced in [4], and [9]. The result allows us to show that, under the assumption that a solution exists, a system is W-detectable if and only if the solution is stabilizing. This provides a simple way to show that W-observability is a necessary and sufficient condition for the solution to be positive definite, in the sense of Definitions 5 and 6(ii). The characterization are directly extended to the associated CARE, insofar as its minimal positive semidefinite solution is identified with the solution of the JLQ problem. In addition, we mention that W-detectability provides uniqueness of positive semidefinite solutions of the CARE (see [3] and [4]) due to the stabilizing property mentioned earlier. Those properties suggest that W-detectability is the condition that provide the adequate stability framework for the JLQ problem and the associated CARE, and any further generalization would induce the loss of stabilizability of solutions. It is also a noteworthy feature that, contrary to linear deterministic systems, the adequate detectability and stabilizability concepts are not dual; see Remark 1. This subtlety arises due to the more complex setting pursued here. This note is organized as follows. In Section II, several stochastic detectability concepts are collected from the literature and they are compared. In Section III, we study the role that those conditions play in the JLQ problem and in the associated CARE. Illustrative examples are presented in Section V.
João B. R. do Val and Eduardo F. Costa
Abstract—This note addresses the jump linear quadratic problem of Markov jump linear systems and the associated algebraic Riccati equation. Necessary and sufficient conditions for stability of the optimal control and positiveness of Riccati solutions are developed. We show that the concept of weak detectability is not only a sufficient condition for the finiteness of cost functional to imply stablity of the associated trajectory, but also a necessary one. This, together with a characterization developed here for the kernel of the Riccati solution, allows us to show that the control solution stabilizes the system if and only if the system is weakly detectable, and that the Riccati solution is positive–definite if and only if the system is weakly observable. The connection between the algebraic Riccati equation and the control problem is made, as far as the minimal positive–semidefinite solution for the algebraic Riccati equation is identified with the optimal solution of the linear quadratic problem. Illustrative numerical examples and comparisons are included. Index Terms—Detectability, Markov jump linear systems (MJLSs), observability, quadratic control problem.
I. INTRODUCTION Basic concepts such as stabilizability and detectability play an important role in the jump linear quadratic (JLQ) problem as they provide a priori characterization of fundamental aspects of solutions, like existence and stabilizability. In the scenario of the Markov jump linear systems (MJLSs), it was shown in [14] that mean square (MS) stabilizability is a necessary and sufficient condition for existence of solutions for the associate JLQ problem and the coupled algebraic Riccati equations (CARE). As regards to detectability concepts, it was shown that the weak (W-)detectability notion, which is introduced in [4], generalizes the previous concepts of W-observability and MS-detectability (a dual of MS-stabilizability) that appear , e.g., in [8], [12], and [16] as well as earlier concepts appearing, e.g., in [15]. The W-detectability concept is a sufficient condition for the solution of the JLQ problem to stabilize the system, see also [9]. However, it is not known up to this date if the W-detectability concept is also a necessary condition for stabilizability of JLQ solutions. Another open question to be investigated is that of conditions for positiveness of CARE solutions, which is important for the performance of the control system. In this note, we address the aforementioned questions. We start showing that W-detectability is not only a sufficient condition for a finite cost functional to imply stable trajectories, but also a necessary one; see Lemma 1. This implies that any stabilizing control provides finite cost and conversely, from which results the important fact that
Manuscript received April 28, 2004. Recommended by Associate Editor L. Glielmo. This work was supported in part by FAPESP under Grant 03/06736-7, by CNPq under Grant 300721/86, by PRONEX under Grant 015/98 “Control of Dynamical Systems,” and by the IM-AGIMB Grant. J. B. R. do Val is with UNICAMP-Faculdade de Engenharia Elétrica e de Computação, Departamento de Telemática, 13081-970 Campinas, SP, Brazil (e-mail: [email protected]). E. F. Costa is with USP-Instituto de Ciências Matemáticas e de Computação, Departamento de Ciências de Computação e Estatística, 13560-970, São Carlos, SP, Brazil (e-mail: [email protected]). Digital Object Identifier 10.1109/TAC.2005.846600
II. STABILIZABILITY AND DETECTABILITY CONCEPTS Let n be the nth-dimensional Euclidean space. Let Rn;q (respectively, Rn ) represent the normed linear space formed by all n 2 q real matrices (respectively, n 2 n) and Rn0 (Rn+ ) the closed convex cone fU 2 Rn : U = U 0 0g (the open cone fU 2 Rn : U = U 0 > 0g) where U 0 denotes the transpose of U ; U V (U > V ) signifies that U 0 V is positive–semidefinite (definite). For U 2 Rn;q , NfU g represents the kernel of U . Let Mn;q denote the linear space formed by collection of N matrices, such that Mn;q = fU = (U1 ; . . . ; UN ) : Ui 2 Rn;q ; i = 1; . . . ; N g; also, Mn Mn;n . We denote by Mn0(Mn+ ) the set Mn when it is made up by Ui 2 Rn0 (Ui 2 Rn+ ) for all i = 1; . . . ; N . For the collections of matrices U , V 2 Mn , U V means that Ui 0 Vi 0, i = 1; . . . ; N , and similarly for other mathematical relations. For instance, U = V means that Ui = Vi , i = 1; . . . ; N and for 2 , U stands for the collection of matrices U = ( U1 ; U2 ; . . . ; UN ); notice that U 2 Mn;q if U does. The MJLS considered in this paper is described in a probabilistic space ( , =, f=t g, ) by
8 : x_ (t) = A t x(t) + B t u(t) x(0) = x ; (0) = y ( t) = C t x ( t) (1) where the input u(1) and the output y (1) are =t -measurable and the jump variable (t) is the state of an underlying continuous-time homogeneous Markov chain 2 = f(t); t 0g having S = f1; . . . ; N g as state–space and 3 = [ij ], i, j = 1; . . . ; N as the transition rate matrix. The state of the system 8 is the compound variable (x, ) with initial condition x 2 n and 2 S . At each time instant t 0,
0018-9286/$20.00 © 2005 IEEE
( )
( )
( )
0
0
0
0
692
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 50, NO. 5, MAY 2005
A(t) = Ai whenever (t) = i; the matrices Ai , i = 1; . . . ; N , belong to a given collection of matrices A = (A1 ; . . . ; AN ) 2 Mn , and similarly for B 2 Mn;r and C 2 Mp;n . The MS-stability concept and the associated concept of stabilizability are as follows. Definition 1 (MS-Stability): We say that (A, 3) is MS-stable when limt!1 E fkx(t)k2 g = 0, for each x0 2 n and 0 2 S . Definition 2 (MS-Stabilizability): Consider system 8. We say that (A, B , 3) is MS-stabilizable when there exist G 2 Mr;n such that (A + BG, 3) is MS-stable. Let us introduce the functionals W t (x 0 ; 0 ) :
t
=E
x( )0 C0 ( ) C( ) x( )d jx(0) = x0 ; (0) = 0 :
0
Let us also denote W (x0 ; 0 ) := limt!1 W t (x0 ; 0 ). For W-observability to hold, one only requires that there exists t1 > 0 for which W t (1) is positive, see [4]. For a similar concept in the time-varying context see [19], in the discrete-time case see [2] or [3], and in the deterministic context see [1] or [13]. Definition 3 (W-Observability): Consider system 8. We say that (A, C , 3) is W-observable if there exist scalars t1 , > 0 such that, for each initial condition x0 and 0
W t (x 0 ; 0 ) kx 0 k 2 :
Next, a relaxation of the W-observability provides a detectability concept; see [3] and [4] or [6] in the context of countably many Markov states. We show later in this note that the concept connects the simultaneous convergence in the MS-sense of the output and the state, see Corollary 1. Definition 4 (W-Detectability): Consider system 8. We say that (A, C , 3) is W-detectable if there exist scalars t1 , > 0, t2 0 and 0 < 1, such that, for each initial condition x0 and 0
W t (x 0 ; 0 ) kx 0 k 2 ;
whenever E fkx(t2 )kg kx0 k:
Remark 1: From a duality relation with MS-stabilizability, one says that (A, C , 3) is MS-detectable when there exist G 2 Mn;q for which (A + GC , 3) is MS-stable. The concept appears, e.g., in [8], [10], [12], and [19] as a sufficient condition for stabilizability of solutions, and it is stricter than W-detectability; see [4]. The linear operator E : Mn;q ! Mn;q , E (U ) = (E1 (U ); . . . ; EN (U )) is defined as
E i (U ) =
N
for i = 1; . . . ; N:
ij Uj ;
j =1
The next proposition summarizes some of the most important results concerning W-observability and W-detectability, which are drawn from [4]. For each i = 1; . . . ; N and k = 1; . . . ; n2 N , let Oi (k) be defined as Oi (k) := Ci0 Ci for k = 1 and recursively as
Oi (k) := A0i Oi (k 0 1) + Oi (k 0 1)Ai + Ei (O(k 0 1)) :
The set of observability matrices defined as
Oi :=
(2)
O 2 Mn n N ;n of the MJLS is
Oi (1) Oi (2)
(
)
. . . O i (n 2 N ) 0
(3)
for each i = 1; . . . ; N . Proposition 1: (A, C , 3) is W-observable if and only if Oi is full rank, for each i = 1; . . . ; N . Proposition 2: The following assertions are equivalent. i) (A, C , 3) is W-detectable. ii) limt!1 E fkx(t)k2 g = 0 whenever W (x0 ; 0 ) = 0.
iii)
There exists a solution for the following set of linear matrix inequalities in the unknowns X 2 Mn0 and L 2 Mn
A0i Xi + Xi Ai + O0 OLi0 + Li O0 O + Ei (X ) < 0;
i = 1; . . . ; N: (4) Note from the definition that if each pair (Ai , Ci ) is observable, the matrices Oi are of full rank, and observability of each pair implies that W-observability holds. Similarly, the detectability of each pair (Ai , Ci ) is sufficient for W-detectability to hold. III. CONDITIONS FOR STABILIZABILITY AND POSITIVENESS OF THE JLQ SOLUTIONS The infinite-time JLQ problem consists of the following optimization problem:
J (x 0 ; 0 ) =
inf E u(1)
1
x( )0 C0 ( ) C( ) x( )
0
+u( )0 R( ) u( )
d jx(0) = x0 ; (0) = 0
(5)
for each initial condition x0 , 0 , where u(t) = g (s; x(s); (s)), s t, is Ft -adapted, and R 2 Mr+ . In what follows, we indicate the expectation E f1jx0 ; 0 g by Ex ; f1g or simply omit the reference to the initial condition when not ambiguous. Remark 2: Note that stability is not a direct issue in the minimization problem in (5), and the resulting stability behavior is left to depend on the dynamics and on the data of the JLQ problem. As we shall see in Theorem 1, the LQ controlled system is stable if and only if (A, C , 3) is W-detectable. In this section, the role that the W-observability and W-detectability concepts play in the JLQ problem is clarified and, in the main result, we show that the solution of the JLQ problem is stabilizing if and only if the system is W-detectable. This means that W-detectability not only generalizes previous concepts, as shown in the previous section, but it is also the weakest concept that ensures stabilizability of solutions. We start the section by presenting some basic results concerning the form of the optimal solution of the JLQ problem. The result is quoted from [12] or [14], and it allows us to identify in the next definition, the JLQ solution with an element of the space Mn0 . Proposition 3: Assume that there exists a solution to the JLQ problem. Then, there exists P 2 Mn0 for which J (x0 ; 0 ) = x00 P x0 for each initial condition x0 and 0 . Moreover, the optimal control is in the linear state feedback form u(t) = 0R0(1t) B0 (t) P(t) x(t). Definition 5: We say that P 2 Mn0 is the unique solution of the JLQ problem when J (x0 ; 0 ) = x00 P x0 for each initial condition x0 and 0 . Definition 6: We say that the JLQ solution P is i) stabilizing if (A + BG, 3) is MS-stable, where G 2 Mr2n is defined as Gi = 0Ri01 Bi0 Pi for each i; ii) positive–definite (positive–semidefinite) if P 2 Mn+ (P 2 Mn0). Proposition 3 allows us to consider, with no loss of generality, the linear state feedback form u(t) = G(t) x(t) for some G 2 Mr2n and the associated closed-loop version (A + BG, 3). In connection, we define T JG
(x0 ; 0 ) = Ex ;
T
0
x( )0 C0 ( ) C( )
+ G0( ) R( ) G( ) x( )d
(6)
and
J G (x 0 ; 0 ) =
T lim !1 JG (x0 ; 0 )
T
(7)
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 50, NO. 5, MAY 2005
possibly, JG = +1. In the next lemma, we show that W-detectability is a necessary and sufficient condition for a finite cost functional to imply converging trajectories in the mean square sense. We need the preliminary result in the sequel, which follows from the equivalence between MS-stability and MS-exponential stability, e.g., see [11]. Proposition 4: There exist 0 < 1 and > 0 such that E fkx(t)k2 g exp(t)kx0 k2 whenever limt!1 E fkx(t)k2 g 6= 0. Lemma 1: Let G 2 Mr2n . The following assertions are equivalent: i) (A + BG, (C 0 C + G0 RG)1=2 , 3) is W-detectable; ii) if x0 and 0 are such that JG (x0 ; 0 ) < 1, then limt!1 E fkx(t)k2 g = 0. Proof: ii) ) i): It follows immediately from the assertion of the lemma that E fkx(t)k2 g ! 0, as t ! 1 whenever JG (x0 ; 0 ) = 0; Proposition 2ii) completes the proof, by setting W JG . = 1 whenever i) ) ii): we show that JG (x0 ; 0 ) limt!1 E fkx(t)k2 g 6= 0, provided that (A + BG, (C 0 C + G0 RG)1=2 ) is W-detectable. From the W-detectability of (A + BG, (C 0 C + G0 RG)1=2 ), one has from the definition that there exist t2 0, t1 , > 0 and 0 < 1 such that t+t
E
x( )0 C0 ( ) C( ) + G0( ) R( ) G( ) x( )d
t
E kx(t)k
2
(8)
whenever E fkx(t + t2 )k2 g E fkx(t)k2 g. On the other hand, since we assume that limt!1 E fkx(t)k2 g 6= 0, from Proposition 4, we have that
E
kx(t)k exp(t)kx k 2
0
2
(9)
for some 0 < 1 and > 0. Let us define the sequence K = fk ; k ; . . .g where k = 0 and each km , m = 1; 2; . . . is the smallest 0
1
0
integer such that km > km01 and
kx ((km + 1)t )k E kx(km t )k
E
2
2
2
2
(10)
holds. It is easy to check that, if the number of elements of K is finite, then 2 lim !1 E kx(mt2 )k = 0
which contradicts the initial hypothesis and we conclude that K has infinitely many elements. Hence, we can take a subsequence with infinitely many elements K0 = fkm ; km ; . . .g where km = m0 = 0 and each m` , ` = 1; 2; . . ., is the smallest integer such that km km + maxf1; (t1 =t2 )g and we write
JGT (x0 ; 0 )
=E
x( )0 C0 ( ) C( ) + G0( ) R( ) G( ) x( )d
0
E
`
k
`=0 k
t +t
t
x( )0 C0 ( ) C( ) + G0( ) R( ) G( )
2x( )d g
where `0 is the largest integer for which km t2 + t1 (8), (10), and (11), we evaluate
JGT
(x 0 ; 0 )
` `=0
k
t +t
E k
t
(11)
T . Finally, from
x( )0 C0 ( ) C( ) + G0( )
2 R G ( )
`
E
`=0 `
`=0
kx (km t )k 2
2
exp (km t2 ) kx0 k2
exp (km t ) (`0 + 1)kx k : (12) 0 Since ` ! 1 as T ! 1, we conclude that JG (x ; ) = limT !1 JGT (x ; ) = 1, and the proof is completed. 2
0
2
0
0
0
0
Note from Lemma 1ii) that the concept of W-detectability relates the convergence of the state and output trajectories. The next corollary formalizes the result. Corollary 1: Let G 2 Mr2n . Then, (A + BG, (C 0 C + G0 RG)1=2 , 3) is W-detectable if and only if 2 2 lim !1 E ky(t)k = 0: !1 E kx(t)k = 0 whenever tlim
t
In the sequel, we present an interesting result concerning the kernel of the solution of the JLQ problem and the kernel of the observability matrices. The next preliminary result is drawn from [4]. Proposition 5: The following assertions hold. i) [4, Cor. 9]: W (x0 ; 0 ) = 0 if and only if x0 2 NfO g. ii) [4, Cor. 15]: If x(t) 2 NfO(t) g, then x(s) 2 NfO(s) g almost surely (a.s.) for each s t. Lemma 2: Assume that P is the solution of the JLQ problem. Then, NfPi g = NfOi g, for all i 2 S . Proof: The fact that x0 2 NfP g is equivalent to J (x0 ; 0 ) = x00 P x0 = 0, for P the solution of the JLQ problem. When J (x0 ; 0 ) = 0, (5) implies that the optimal trajectory x(1) defined by an optimal control u(1) according to Proposition 3 satisfies R1(=12) u(1) = R1(=12) G(1) x(1) = 0 a.s.. Since R 2 Mr+ we have ~(1) a.s, where x~(1) is the that G(1) x(1) = 0 a.s. and that x(1) = x open-loop trajectory (u(1) 0) of system 8 with initial conditions x0 and 0 . Now, we can evaluate
J (x 0 ; 0 )
E
1
x( )0 C0 ( ) C( ) x( )d jx(0) = x0 ; (0) = 0
0
=E
1
x~( )0 C0 ( ) C( ) x~( )d jx~(0) = x0 ; (0) = 0
0
m
T
693
( )
x( )d
= W (x 0 ; 0 ) and conclude that 0 = J (x0 ; 0 ) = W (x0 ; 0 ) and from Proposition
5(i) one has that x0 2 NfO g. Now, we provide a necessary and sufficient condition for stabilizing and definite positive solutions of the JLQ problem in terms of W-detectability and W-observability respectively, thus clarifying the role that these concepts play in the JLQ problem. We need the following result adapted from [3]. Proposition 6: (A, C , 3) is W-detectable if and only if (A + BG, (C 0 C + G0 RG)1=2 , 3) is W-detectable for each G 2 Mr2n . Theorem 1: Assume that P is the solution of the JLQ problem. Then, the following assertions hold. i) P is a stabilizing solution if and only if (A, C , 3) is W-detectable. ii) P is positive definite (P 2 Mn+ ) if and only if (A, C , 3) is W-observable. Proof: i) Sufficiency: Let G = (G1 ; . . . ; GN ) be defined as Gi = 0Ri01 Bi0 Pi for each i. Proposition 3 provides that JG (x0 ; 0 ) = J (x0 ; 0 ) = x00 P x0 < 1 for each initial condition x0 , 0 . Proposition 6 that (A + BG, (C 0 C + G0 RG)1=2 , 3) is W-detectable and Lemma 1 leads to limt!1 E fkx(t)k2 g = 0 for each initial condition x0 , 0 , which means that (A + BG, 3) is MS-stable.
694
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 50, NO. 5, MAY 2005
TABLE I CHARACTERIZATION OF CARE SOLUTIONS VIA THE CONCEPTS OF W-DETECTABILITY AND MS-STABILIZABILITY
Necessity: We show that P is not a stabilizing solution provided the system is not W-detectable. First, let us consider the open loop version of system. From Proposition 2, one has that there exists an initial condition x0 , 0 such that W (x0 ; 0 ) = 0 and 2 lim !1 E kx(t)k 6= 0:
t
(13)
Now, we consider the closed-loop version of the system with G
Mr;n defined as Gi = 0Ri0 Bi0 Pi . From optimality J (x ; ) W (x ; ) = 0 1
0
0
0
0
2
(14)
and from (14), Proposition 5, and Lemma 2, we have that x(t) 2 NfO t g = NfP t g a.s. for t 0. Then, we have that u(t) = 0R0 t B0 t P t x(t) = 0 a.s. which means that (13) holds for the ( ) 1 ( ) ( )
( )
( )
closed-loop version and P is not a stabilizing solution. ii) The result follows immediately from Proposition 1 and Lemma 2. Remark 3: Theorem 1 generalizes previous results in which observability and detectability conditions appear as sufficient conditions for positivity and stability of solutions, respectively; see, e.g., [3], [4], [8]–[10], [12], and [14]–[20]. Remark 4: Notice from Theorem 1i) that if the system is not W-detectable and MS-stability is required, then this condition has to be taken explicitly as an additional constraint in the JLQ problem. For instance, this is the situation in [7], where a method that seeks the solution of the JLQ within the set of MS-stabilizing solutions is presented.
equivalently, to adopt a restricted class of stabilizing controllers, then the solution to the JLQ problem would be associated with the stabilizing/maximal solution of the CARE, see for instance [12, Prop. 6.11] or, in the deterministic scenario, the enlightening paper [17]. The role that the concepts of W-detectability and W-observability play in the CARE context is clarified by Proposition 7: The properties are immediately inherited from Theorem 1. Lemma 3: The following assertions hold: i) the minimal solution X 2 Mn0 of the CARE is stabilizing if and only if (A, C , 3) is W-detectable; ii) each solution of the CARE in Mn0 is positive definite (in the sense of Definition 6) if and only if (A, C , 3) is W-observable. The next properties complete the scenario of the relation between the CARE and the concept of W-detectability. The following result is drawn from [4]. Proposition 8: Each positive semidefinite solution to the CARE is stabilizing provided (A, C , 3) is W-detectable. As regards uniqueness of solutions, we mention that set of stabilizing solutions is known to be a singleton, see [7] and [20]. This fact, in addition to Lemma 3i), leads to the following property. Proposition 9: The positive semidefinite solution to the CARE is unique provided (A, C , 3) is W-detectable. We also mention that MS-stabilizability ensures existence of solutions of the CARE in the set Mn0 ; see, for instance, [12]. Table I summarizes the results.
IV. UNIQUENESS, STABILIZABILITY AND POSITIVENESS OF THE CARE SOLUTIONS
0
Consider the following CARE in the unknown X :
A i Xi
+ Xi Ai + Ei (X ) 0 Xi Bi Ri01Bi0 Xi + Ci0 Ci = 0; n0 i = 1; . . . ; N; with X 2 M :
(15)
The relation between the solution of the JLQ problem and the CARE is presented in the next proposition, which is drawn from [5, Th. 3]. We need the definition that follows, in a parallel with Definition 6. Definition 7: We say that a solution X 2 Mn0 of the CARE in (15) is i) minimal if X Y for any solution Y 2 Mn0 of the CARE; ii) stabilizing if (A + BG, 3) is MS-stable, where G 2 Mr2n is defined as Gi = 0Ri01 Bi0 Xi for each i; iii) positive–definite if X 2 Mn+ . Proposition 7: Assume that X 2 Mn0 is the minimal solution of the CARE and P 2 Mn0 is the solution of the JLQ problem. Then, X = P. Remark 5: If one considers in the JLQ problem (5) the additional constraint that limt!1 E fkx(t)k2 g = 0 for each initial condition or,
V. EXAMPLES In this section, we give numerical examples connected with the results in this paper. The examples possess certain parameters and " which are changed in order to obtain systems that are non-W-detectable, W-detectable, and W-observable, respectively, and we study the behavior of the solution of the JLQ problem and the CARE. Example 1: Consider the MJLS described as
=
"
1 2
1
1
3
= 0 24 B1 = 22 B2 = 22 1 01 1 1 C1 = C2 = 0 (16) 4 R1 = R2 = 1 3 = 5 05 with = 1 and " = 0. From Proposition 2iii), one can check that (A, C , 3) is not W-detectable. This is an interesting setting for studying A1
A2
the behavior of the solution of the JLQ problem and the CARE and to illustrate that the minimal solution differs from the stabilizing one when the system is MS-stabilizable but not W-detectable; see Table I. We use the method of [7] to show that X1
35 08:977 = 0538::977 2:685
X2
010:00 = 02210:73 :00 6:820
(17)
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 50, NO. 5, MAY 2005
695
is a stabilizing solution of the CARE. However, it is not the minimal one, since one can check by inspection that P1
0 = 00 0:6580
P2
0 = 00 0:8240
(18)
is also a solution of the CARE, and Xi > Pi . Indeed, P is the minimal solution of the CARE and thus it is the solution of the JLQ problem. However, one can check that the solution P is not stabilizing,1 thus verifying the result of Theorem 1i). It is interesting to notice from the solution P that the optimal control is null when the state is in the vector subspace generated by the vector [1 0]0 . The reason is that the cost functional is null for x in and, thus, no control effort is necessary in this space. One can check, in Pi ; see also Lemma 2. Along connection, that = Oi = this line, it is also interesting to check that the solution depends only ; in fact, if we solve the on the parameters related with subspace = JLQ problem associated with the reduced system A = A(2; 2), B = C (1; 2), R, and 3, we get the solution P = P (2; 2). B (2; 1), C Example 2: Consider the MJLS described in (16) of Example 1 now with < 0. With this modification we stabilize nonobserved trajectories. In this situation, one can check that (A, C , 3) is strictly W-detectable (strictly in the sense that it is not W-observable nor MS-detectable, see Remark 1) and P as in (18) is a stabilizing solution of the JLQ problem, see Theorem 1. Notice that the solution of the JLQ problem did not vary with , in connection with the comments of Example 1. Example 3: Consider the MJLS described in (16) of Example 1, now with " = 0. The parameter " is chosen in such a manner that (A, C , 3) is W-observable. The solution of the JLQ problem is stabilizing, positive definite and it approaches X in (17) when " 0. However, the solution of the JLQ problem P (") at " = 0 is as in (18); it is no longer stabilizing, and in view of (17) and (18) we conclude that " P (") is a lower semicontinuous function at zero. The fact is that the system is W-observable provided that " = 0 and thus the solution to the CARE is stabilizing and unique. By contrast, when " is set to zero W-detectability is lost and the CARE may have multiple solutions; in this situation, the minimal solution is not stabilizing (see Lemma 3 and Table I), as is the case in the present example. If stability is required without W-detectability, this condition should appear explicitly as an extra constraint in the minimization problem, and here one gets (17) as the solution, see Remark 5. Clearly, the stabilizing solution will incur in a greater cost. As result, a separation in multiple solutions of the CARE occurs, creating a lattice that has the JLQ solution as the minimal extreme. This comes as a natural extension of previous studies of the LQ problem of deterministic systems, such as [17] or [18, Sec. 6].
S
S
S Nf g Nf g S?
6
!
! 6
VI. CONCLUSION This paper clarifies the role that the concepts of W-detectability and W-observability play as necessary and sufficient conditions for stabilizability and positiveness of solutions of the JLQ problem, respectively. It is shown in Lemma 1 that W-detectability of (A, C , 3) is a sufficient and also a necessary condition for a finite cost to imply mean square stable state trajectories. This means that W-detectability is the weakest condition that relates a bounded cost with MS-stability of the system; if a weaker condition is taken into account, one has that limt E x(t) 2 = 0 for some initial condition x0 , 0 for which J (x 0 ; 0 ) < . Pi = Another characterization presented here is that i , n0 n N;n is the solution of the JLQ problem and where P
!1
1For B R
fk k g 6 1 2M
Nf g NfO g O2M
~, 3), with A~ this purpose, we can test the MS-stability of (A ~ = 0, and C~ = I , i = 1; . . . ; N . B P , B
=
A
0
is the set of observability matrices of the MJLS. The results allows us to present, in Theorem 1, the role that the concepts play in the JLQ problem. — W-detectability is a necessary and sufficient condition for the solution of the JLQ problem to be stabilizing. — W-observability is a necessary and sufficient condition for the solution of the JLQ problem to be positive–definite. These properties are immediately inherited by the CARE, since the minimal solution of the former is identified with the solution of the latter. The complete scenario is presented in Table I. ACKNOWLEDGMENT The author would like to thank two anonymous reviewers whose careful reading of the manuscript helped to improve this note. REFERENCES [1] B. D. O. Anderson and J. B. Moore, “Detectability and stabilizability of time-varying discrete-time linear systems,” SIAM J. Control Optim., vol. 19, no. 1, pp. 20–32, 1981. [2] E. F. Costa and J. B. R. do Val, “Weak detectability and the linear quadratic control problem of discrete-time Markov jump linear systems,” Int. J. Control, vol. 75, pp. 1282–1292, 2002. [3] , “On the detectability and observability of discrete-time Markov jump linear systems,” Syst. Control Lett., vol. 44, pp. 135–145, 2001. [4] , “On the detectability and observability of continuous-time Markov jump linear systems,” SIAM J. Control Optim., vol. 41, pp. 1295–1314, 2002. [5] [6] E. F. Costa, J. B. R. do Val, and M. D. Fragoso, “On a detectability concept of discrete-time infinite Markov jump linear systems,” Stoch. Anal. Appl., vol. 23, no. 1, pp. 1–5, 2005. [7] O. L. V. Costa, J. B. R. do Val, and J. C. Geromel, “Continuous-time state-feedback H -control of Markovian jump linear systems via convex analysis,” Automatica, vol. 35, pp. 259–268, 1999. [8] O. L. V. Costa and M. D. Fragoso, “Discrete-time LQ-optimal control problems for infinite Markov jump parameter systems,” IEEE Trans. Autom. Control, vol. 40, no. 12, pp. 2076–2088, Dec. 1995. [9] J. B. R. do Val and E. F. Costa, “Numerical solution for the linearquadratic control problem of Markov jump linear systems and a weak detectability concept,” J. Optim. Theory Appl., vol. 114, no. 1, pp. 69–96, 2002. [10] J. B. R. do Val, J. C. Geromel, and O. L. V. Costa, “Solutions for the linear quadratic control problem of Markov jump linear systems,” J. Optim. Theory Appl., vol. 103, pp. 283–311, 1999. [11] X. Feng, K. A. Loparo, Y. Ji, and H. J. Chizeck, “Stochastic stability properties of jump linear systems,” IEEE Trans. Autom. Control, vol. 37, no. 1, pp. 38–53, Jan. 1992. [12] M. Fragoso and J. Baczynski, “Optimal control for continuous time LQ problems with infinite Markov jump parameters,” SIAM J. Control Optim., vol. 40, no. 1, pp. 270–297, 2001. [13] W. W. Hager and L. L. Horowitz, “Convergence and stability properties of the discrete Riccati operator equation and the associated optimal control and filtering problems,” SIAM J. Control Optim., vol. 14, no. 2, pp. 295–312, 1976. [14] Y. Ji and H. J. Chizeck, “Controlability, stabilizability and continuous time Markovian jump linear quadratic control,” IEEE Trans. Autom. Control, vol. 35, no. 7, pp. 777–788, Jul. 1990. , “Jump linear quadratic Gaussian control: steady-state solution and [15] testable conditions,” Control Theory Adv. Technol., vol. 6, no. 3, pp. 289–319, 1990. [16] , “Jump linear quadratic gaussian control in continuous time,” IEEE Trans. Autom. Control, vol. 37, no. 12, pp. 1884–1892, Dec. 1992. [17] V. Kucera, “On nonnegative definite solutions to matrix quadratic equations,” Automatica, vol. 8, pp. 413–423, 1972. [18] B. P. Molinari, “The time-invariant linear-quadratic optimal control problem,” Automatica, vol. 13, pp. 347–357, 1977. [19] T. Morozan, “Stability and control for linear systems with jump Markov perturbations,” Stoch. Anal. Appl., vol. 13, no. 1, pp. 91–110, 1995. [20] M. A. Rami and L. E. Ghaoui, “LMI optimization for nonstandard Riccati equations arising in stochastic control,” IEEE Trans. Autom. Control, vol. 41, no. 11, pp. 1666–1671, Nov. 1996.
691
Stabilizability and Positiveness of Solutions of the Jump Linear Quadratic Problem and the Coupled Algebraic Riccati Equation
W-detectability is the weakest condition that associates finite cost and mean square stability. A characterization developed here relates the zero cost solution of the JLQ problem with the kernel of the observability matrices introduced in [4], and [9]. The result allows us to show that, under the assumption that a solution exists, a system is W-detectable if and only if the solution is stabilizing. This provides a simple way to show that W-observability is a necessary and sufficient condition for the solution to be positive definite, in the sense of Definitions 5 and 6(ii). The characterization are directly extended to the associated CARE, insofar as its minimal positive semidefinite solution is identified with the solution of the JLQ problem. In addition, we mention that W-detectability provides uniqueness of positive semidefinite solutions of the CARE (see [3] and [4]) due to the stabilizing property mentioned earlier. Those properties suggest that W-detectability is the condition that provide the adequate stability framework for the JLQ problem and the associated CARE, and any further generalization would induce the loss of stabilizability of solutions. It is also a noteworthy feature that, contrary to linear deterministic systems, the adequate detectability and stabilizability concepts are not dual; see Remark 1. This subtlety arises due to the more complex setting pursued here. This note is organized as follows. In Section II, several stochastic detectability concepts are collected from the literature and they are compared. In Section III, we study the role that those conditions play in the JLQ problem and in the associated CARE. Illustrative examples are presented in Section V.
João B. R. do Val and Eduardo F. Costa
Abstract—This note addresses the jump linear quadratic problem of Markov jump linear systems and the associated algebraic Riccati equation. Necessary and sufficient conditions for stability of the optimal control and positiveness of Riccati solutions are developed. We show that the concept of weak detectability is not only a sufficient condition for the finiteness of cost functional to imply stablity of the associated trajectory, but also a necessary one. This, together with a characterization developed here for the kernel of the Riccati solution, allows us to show that the control solution stabilizes the system if and only if the system is weakly detectable, and that the Riccati solution is positive–definite if and only if the system is weakly observable. The connection between the algebraic Riccati equation and the control problem is made, as far as the minimal positive–semidefinite solution for the algebraic Riccati equation is identified with the optimal solution of the linear quadratic problem. Illustrative numerical examples and comparisons are included. Index Terms—Detectability, Markov jump linear systems (MJLSs), observability, quadratic control problem.
I. INTRODUCTION Basic concepts such as stabilizability and detectability play an important role in the jump linear quadratic (JLQ) problem as they provide a priori characterization of fundamental aspects of solutions, like existence and stabilizability. In the scenario of the Markov jump linear systems (MJLSs), it was shown in [14] that mean square (MS) stabilizability is a necessary and sufficient condition for existence of solutions for the associate JLQ problem and the coupled algebraic Riccati equations (CARE). As regards to detectability concepts, it was shown that the weak (W-)detectability notion, which is introduced in [4], generalizes the previous concepts of W-observability and MS-detectability (a dual of MS-stabilizability) that appear , e.g., in [8], [12], and [16] as well as earlier concepts appearing, e.g., in [15]. The W-detectability concept is a sufficient condition for the solution of the JLQ problem to stabilize the system, see also [9]. However, it is not known up to this date if the W-detectability concept is also a necessary condition for stabilizability of JLQ solutions. Another open question to be investigated is that of conditions for positiveness of CARE solutions, which is important for the performance of the control system. In this note, we address the aforementioned questions. We start showing that W-detectability is not only a sufficient condition for a finite cost functional to imply stable trajectories, but also a necessary one; see Lemma 1. This implies that any stabilizing control provides finite cost and conversely, from which results the important fact that
Manuscript received April 28, 2004. Recommended by Associate Editor L. Glielmo. This work was supported in part by FAPESP under Grant 03/06736-7, by CNPq under Grant 300721/86, by PRONEX under Grant 015/98 “Control of Dynamical Systems,” and by the IM-AGIMB Grant. J. B. R. do Val is with UNICAMP-Faculdade de Engenharia Elétrica e de Computação, Departamento de Telemática, 13081-970 Campinas, SP, Brazil (e-mail: [email protected]). E. F. Costa is with USP-Instituto de Ciências Matemáticas e de Computação, Departamento de Ciências de Computação e Estatística, 13560-970, São Carlos, SP, Brazil (e-mail: [email protected]). Digital Object Identifier 10.1109/TAC.2005.846600
II. STABILIZABILITY AND DETECTABILITY CONCEPTS Let n be the nth-dimensional Euclidean space. Let Rn;q (respectively, Rn ) represent the normed linear space formed by all n 2 q real matrices (respectively, n 2 n) and Rn0 (Rn+ ) the closed convex cone fU 2 Rn : U = U 0 0g (the open cone fU 2 Rn : U = U 0 > 0g) where U 0 denotes the transpose of U ; U V (U > V ) signifies that U 0 V is positive–semidefinite (definite). For U 2 Rn;q , NfU g represents the kernel of U . Let Mn;q denote the linear space formed by collection of N matrices, such that Mn;q = fU = (U1 ; . . . ; UN ) : Ui 2 Rn;q ; i = 1; . . . ; N g; also, Mn Mn;n . We denote by Mn0(Mn+ ) the set Mn when it is made up by Ui 2 Rn0 (Ui 2 Rn+ ) for all i = 1; . . . ; N . For the collections of matrices U , V 2 Mn , U V means that Ui 0 Vi 0, i = 1; . . . ; N , and similarly for other mathematical relations. For instance, U = V means that Ui = Vi , i = 1; . . . ; N and for 2 , U stands for the collection of matrices U = ( U1 ; U2 ; . . . ; UN ); notice that U 2 Mn;q if U does. The MJLS considered in this paper is described in a probabilistic space ( , =, f=t g, ) by
8 : x_ (t) = A t x(t) + B t u(t) x(0) = x ; (0) = y ( t) = C t x ( t) (1) where the input u(1) and the output y (1) are =t -measurable and the jump variable (t) is the state of an underlying continuous-time homogeneous Markov chain 2 = f(t); t 0g having S = f1; . . . ; N g as state–space and 3 = [ij ], i, j = 1; . . . ; N as the transition rate matrix. The state of the system 8 is the compound variable (x, ) with initial condition x 2 n and 2 S . At each time instant t 0,
0018-9286/$20.00 © 2005 IEEE
( )
( )
( )
0
0
0
0
692
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 50, NO. 5, MAY 2005
A(t) = Ai whenever (t) = i; the matrices Ai , i = 1; . . . ; N , belong to a given collection of matrices A = (A1 ; . . . ; AN ) 2 Mn , and similarly for B 2 Mn;r and C 2 Mp;n . The MS-stability concept and the associated concept of stabilizability are as follows. Definition 1 (MS-Stability): We say that (A, 3) is MS-stable when limt!1 E fkx(t)k2 g = 0, for each x0 2 n and 0 2 S . Definition 2 (MS-Stabilizability): Consider system 8. We say that (A, B , 3) is MS-stabilizable when there exist G 2 Mr;n such that (A + BG, 3) is MS-stable. Let us introduce the functionals W t (x 0 ; 0 ) :
t
=E
x( )0 C0 ( ) C( ) x( )d jx(0) = x0 ; (0) = 0 :
0
Let us also denote W (x0 ; 0 ) := limt!1 W t (x0 ; 0 ). For W-observability to hold, one only requires that there exists t1 > 0 for which W t (1) is positive, see [4]. For a similar concept in the time-varying context see [19], in the discrete-time case see [2] or [3], and in the deterministic context see [1] or [13]. Definition 3 (W-Observability): Consider system 8. We say that (A, C , 3) is W-observable if there exist scalars t1 , > 0 such that, for each initial condition x0 and 0
W t (x 0 ; 0 ) kx 0 k 2 :
Next, a relaxation of the W-observability provides a detectability concept; see [3] and [4] or [6] in the context of countably many Markov states. We show later in this note that the concept connects the simultaneous convergence in the MS-sense of the output and the state, see Corollary 1. Definition 4 (W-Detectability): Consider system 8. We say that (A, C , 3) is W-detectable if there exist scalars t1 , > 0, t2 0 and 0 < 1, such that, for each initial condition x0 and 0
W t (x 0 ; 0 ) kx 0 k 2 ;
whenever E fkx(t2 )kg kx0 k:
Remark 1: From a duality relation with MS-stabilizability, one says that (A, C , 3) is MS-detectable when there exist G 2 Mn;q for which (A + GC , 3) is MS-stable. The concept appears, e.g., in [8], [10], [12], and [19] as a sufficient condition for stabilizability of solutions, and it is stricter than W-detectability; see [4]. The linear operator E : Mn;q ! Mn;q , E (U ) = (E1 (U ); . . . ; EN (U )) is defined as
E i (U ) =
N
for i = 1; . . . ; N:
ij Uj ;
j =1
The next proposition summarizes some of the most important results concerning W-observability and W-detectability, which are drawn from [4]. For each i = 1; . . . ; N and k = 1; . . . ; n2 N , let Oi (k) be defined as Oi (k) := Ci0 Ci for k = 1 and recursively as
Oi (k) := A0i Oi (k 0 1) + Oi (k 0 1)Ai + Ei (O(k 0 1)) :
The set of observability matrices defined as
Oi :=
(2)
O 2 Mn n N ;n of the MJLS is
Oi (1) Oi (2)
(
)
. . . O i (n 2 N ) 0
(3)
for each i = 1; . . . ; N . Proposition 1: (A, C , 3) is W-observable if and only if Oi is full rank, for each i = 1; . . . ; N . Proposition 2: The following assertions are equivalent. i) (A, C , 3) is W-detectable. ii) limt!1 E fkx(t)k2 g = 0 whenever W (x0 ; 0 ) = 0.
iii)
There exists a solution for the following set of linear matrix inequalities in the unknowns X 2 Mn0 and L 2 Mn
A0i Xi + Xi Ai + O0 OLi0 + Li O0 O + Ei (X ) < 0;
i = 1; . . . ; N: (4) Note from the definition that if each pair (Ai , Ci ) is observable, the matrices Oi are of full rank, and observability of each pair implies that W-observability holds. Similarly, the detectability of each pair (Ai , Ci ) is sufficient for W-detectability to hold. III. CONDITIONS FOR STABILIZABILITY AND POSITIVENESS OF THE JLQ SOLUTIONS The infinite-time JLQ problem consists of the following optimization problem:
J (x 0 ; 0 ) =
inf E u(1)
1
x( )0 C0 ( ) C( ) x( )
0
+u( )0 R( ) u( )
d jx(0) = x0 ; (0) = 0
(5)
for each initial condition x0 , 0 , where u(t) = g (s; x(s); (s)), s t, is Ft -adapted, and R 2 Mr+ . In what follows, we indicate the expectation E f1jx0 ; 0 g by Ex ; f1g or simply omit the reference to the initial condition when not ambiguous. Remark 2: Note that stability is not a direct issue in the minimization problem in (5), and the resulting stability behavior is left to depend on the dynamics and on the data of the JLQ problem. As we shall see in Theorem 1, the LQ controlled system is stable if and only if (A, C , 3) is W-detectable. In this section, the role that the W-observability and W-detectability concepts play in the JLQ problem is clarified and, in the main result, we show that the solution of the JLQ problem is stabilizing if and only if the system is W-detectable. This means that W-detectability not only generalizes previous concepts, as shown in the previous section, but it is also the weakest concept that ensures stabilizability of solutions. We start the section by presenting some basic results concerning the form of the optimal solution of the JLQ problem. The result is quoted from [12] or [14], and it allows us to identify in the next definition, the JLQ solution with an element of the space Mn0 . Proposition 3: Assume that there exists a solution to the JLQ problem. Then, there exists P 2 Mn0 for which J (x0 ; 0 ) = x00 P x0 for each initial condition x0 and 0 . Moreover, the optimal control is in the linear state feedback form u(t) = 0R0(1t) B0 (t) P(t) x(t). Definition 5: We say that P 2 Mn0 is the unique solution of the JLQ problem when J (x0 ; 0 ) = x00 P x0 for each initial condition x0 and 0 . Definition 6: We say that the JLQ solution P is i) stabilizing if (A + BG, 3) is MS-stable, where G 2 Mr2n is defined as Gi = 0Ri01 Bi0 Pi for each i; ii) positive–definite (positive–semidefinite) if P 2 Mn+ (P 2 Mn0). Proposition 3 allows us to consider, with no loss of generality, the linear state feedback form u(t) = G(t) x(t) for some G 2 Mr2n and the associated closed-loop version (A + BG, 3). In connection, we define T JG
(x0 ; 0 ) = Ex ;
T
0
x( )0 C0 ( ) C( )
+ G0( ) R( ) G( ) x( )d
(6)
and
J G (x 0 ; 0 ) =
T lim !1 JG (x0 ; 0 )
T
(7)
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 50, NO. 5, MAY 2005
possibly, JG = +1. In the next lemma, we show that W-detectability is a necessary and sufficient condition for a finite cost functional to imply converging trajectories in the mean square sense. We need the preliminary result in the sequel, which follows from the equivalence between MS-stability and MS-exponential stability, e.g., see [11]. Proposition 4: There exist 0 < 1 and > 0 such that E fkx(t)k2 g exp(t)kx0 k2 whenever limt!1 E fkx(t)k2 g 6= 0. Lemma 1: Let G 2 Mr2n . The following assertions are equivalent: i) (A + BG, (C 0 C + G0 RG)1=2 , 3) is W-detectable; ii) if x0 and 0 are such that JG (x0 ; 0 ) < 1, then limt!1 E fkx(t)k2 g = 0. Proof: ii) ) i): It follows immediately from the assertion of the lemma that E fkx(t)k2 g ! 0, as t ! 1 whenever JG (x0 ; 0 ) = 0; Proposition 2ii) completes the proof, by setting W JG . = 1 whenever i) ) ii): we show that JG (x0 ; 0 ) limt!1 E fkx(t)k2 g 6= 0, provided that (A + BG, (C 0 C + G0 RG)1=2 ) is W-detectable. From the W-detectability of (A + BG, (C 0 C + G0 RG)1=2 ), one has from the definition that there exist t2 0, t1 , > 0 and 0 < 1 such that t+t
E
x( )0 C0 ( ) C( ) + G0( ) R( ) G( ) x( )d
t
E kx(t)k
2
(8)
whenever E fkx(t + t2 )k2 g E fkx(t)k2 g. On the other hand, since we assume that limt!1 E fkx(t)k2 g 6= 0, from Proposition 4, we have that
E
kx(t)k exp(t)kx k 2
0
2
(9)
for some 0 < 1 and > 0. Let us define the sequence K = fk ; k ; . . .g where k = 0 and each km , m = 1; 2; . . . is the smallest 0
1
0
integer such that km > km01 and
kx ((km + 1)t )k E kx(km t )k
E
2
2
2
2
(10)
holds. It is easy to check that, if the number of elements of K is finite, then 2 lim !1 E kx(mt2 )k = 0
which contradicts the initial hypothesis and we conclude that K has infinitely many elements. Hence, we can take a subsequence with infinitely many elements K0 = fkm ; km ; . . .g where km = m0 = 0 and each m` , ` = 1; 2; . . ., is the smallest integer such that km km + maxf1; (t1 =t2 )g and we write
JGT (x0 ; 0 )
=E
x( )0 C0 ( ) C( ) + G0( ) R( ) G( ) x( )d
0
E
`
k
`=0 k
t +t
t
x( )0 C0 ( ) C( ) + G0( ) R( ) G( )
2x( )d g
where `0 is the largest integer for which km t2 + t1 (8), (10), and (11), we evaluate
JGT
(x 0 ; 0 )
` `=0
k
t +t
E k
t
(11)
T . Finally, from
x( )0 C0 ( ) C( ) + G0( )
2 R G ( )
`
E
`=0 `
`=0
kx (km t )k 2
2
exp (km t2 ) kx0 k2
exp (km t ) (`0 + 1)kx k : (12) 0 Since ` ! 1 as T ! 1, we conclude that JG (x ; ) = limT !1 JGT (x ; ) = 1, and the proof is completed. 2
0
2
0
0
0
0
Note from Lemma 1ii) that the concept of W-detectability relates the convergence of the state and output trajectories. The next corollary formalizes the result. Corollary 1: Let G 2 Mr2n . Then, (A + BG, (C 0 C + G0 RG)1=2 , 3) is W-detectable if and only if 2 2 lim !1 E ky(t)k = 0: !1 E kx(t)k = 0 whenever tlim
t
In the sequel, we present an interesting result concerning the kernel of the solution of the JLQ problem and the kernel of the observability matrices. The next preliminary result is drawn from [4]. Proposition 5: The following assertions hold. i) [4, Cor. 9]: W (x0 ; 0 ) = 0 if and only if x0 2 NfO g. ii) [4, Cor. 15]: If x(t) 2 NfO(t) g, then x(s) 2 NfO(s) g almost surely (a.s.) for each s t. Lemma 2: Assume that P is the solution of the JLQ problem. Then, NfPi g = NfOi g, for all i 2 S . Proof: The fact that x0 2 NfP g is equivalent to J (x0 ; 0 ) = x00 P x0 = 0, for P the solution of the JLQ problem. When J (x0 ; 0 ) = 0, (5) implies that the optimal trajectory x(1) defined by an optimal control u(1) according to Proposition 3 satisfies R1(=12) u(1) = R1(=12) G(1) x(1) = 0 a.s.. Since R 2 Mr+ we have ~(1) a.s, where x~(1) is the that G(1) x(1) = 0 a.s. and that x(1) = x open-loop trajectory (u(1) 0) of system 8 with initial conditions x0 and 0 . Now, we can evaluate
J (x 0 ; 0 )
E
1
x( )0 C0 ( ) C( ) x( )d jx(0) = x0 ; (0) = 0
0
=E
1
x~( )0 C0 ( ) C( ) x~( )d jx~(0) = x0 ; (0) = 0
0
m
T
693
( )
x( )d
= W (x 0 ; 0 ) and conclude that 0 = J (x0 ; 0 ) = W (x0 ; 0 ) and from Proposition
5(i) one has that x0 2 NfO g. Now, we provide a necessary and sufficient condition for stabilizing and definite positive solutions of the JLQ problem in terms of W-detectability and W-observability respectively, thus clarifying the role that these concepts play in the JLQ problem. We need the following result adapted from [3]. Proposition 6: (A, C , 3) is W-detectable if and only if (A + BG, (C 0 C + G0 RG)1=2 , 3) is W-detectable for each G 2 Mr2n . Theorem 1: Assume that P is the solution of the JLQ problem. Then, the following assertions hold. i) P is a stabilizing solution if and only if (A, C , 3) is W-detectable. ii) P is positive definite (P 2 Mn+ ) if and only if (A, C , 3) is W-observable. Proof: i) Sufficiency: Let G = (G1 ; . . . ; GN ) be defined as Gi = 0Ri01 Bi0 Pi for each i. Proposition 3 provides that JG (x0 ; 0 ) = J (x0 ; 0 ) = x00 P x0 < 1 for each initial condition x0 , 0 . Proposition 6 that (A + BG, (C 0 C + G0 RG)1=2 , 3) is W-detectable and Lemma 1 leads to limt!1 E fkx(t)k2 g = 0 for each initial condition x0 , 0 , which means that (A + BG, 3) is MS-stable.
694
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 50, NO. 5, MAY 2005
TABLE I CHARACTERIZATION OF CARE SOLUTIONS VIA THE CONCEPTS OF W-DETECTABILITY AND MS-STABILIZABILITY
Necessity: We show that P is not a stabilizing solution provided the system is not W-detectable. First, let us consider the open loop version of system. From Proposition 2, one has that there exists an initial condition x0 , 0 such that W (x0 ; 0 ) = 0 and 2 lim !1 E kx(t)k 6= 0:
t
(13)
Now, we consider the closed-loop version of the system with G
Mr;n defined as Gi = 0Ri0 Bi0 Pi . From optimality J (x ; ) W (x ; ) = 0 1
0
0
0
0
2
(14)
and from (14), Proposition 5, and Lemma 2, we have that x(t) 2 NfO t g = NfP t g a.s. for t 0. Then, we have that u(t) = 0R0 t B0 t P t x(t) = 0 a.s. which means that (13) holds for the ( ) 1 ( ) ( )
( )
( )
closed-loop version and P is not a stabilizing solution. ii) The result follows immediately from Proposition 1 and Lemma 2. Remark 3: Theorem 1 generalizes previous results in which observability and detectability conditions appear as sufficient conditions for positivity and stability of solutions, respectively; see, e.g., [3], [4], [8]–[10], [12], and [14]–[20]. Remark 4: Notice from Theorem 1i) that if the system is not W-detectable and MS-stability is required, then this condition has to be taken explicitly as an additional constraint in the JLQ problem. For instance, this is the situation in [7], where a method that seeks the solution of the JLQ within the set of MS-stabilizing solutions is presented.
equivalently, to adopt a restricted class of stabilizing controllers, then the solution to the JLQ problem would be associated with the stabilizing/maximal solution of the CARE, see for instance [12, Prop. 6.11] or, in the deterministic scenario, the enlightening paper [17]. The role that the concepts of W-detectability and W-observability play in the CARE context is clarified by Proposition 7: The properties are immediately inherited from Theorem 1. Lemma 3: The following assertions hold: i) the minimal solution X 2 Mn0 of the CARE is stabilizing if and only if (A, C , 3) is W-detectable; ii) each solution of the CARE in Mn0 is positive definite (in the sense of Definition 6) if and only if (A, C , 3) is W-observable. The next properties complete the scenario of the relation between the CARE and the concept of W-detectability. The following result is drawn from [4]. Proposition 8: Each positive semidefinite solution to the CARE is stabilizing provided (A, C , 3) is W-detectable. As regards uniqueness of solutions, we mention that set of stabilizing solutions is known to be a singleton, see [7] and [20]. This fact, in addition to Lemma 3i), leads to the following property. Proposition 9: The positive semidefinite solution to the CARE is unique provided (A, C , 3) is W-detectable. We also mention that MS-stabilizability ensures existence of solutions of the CARE in the set Mn0 ; see, for instance, [12]. Table I summarizes the results.
IV. UNIQUENESS, STABILIZABILITY AND POSITIVENESS OF THE CARE SOLUTIONS
0
Consider the following CARE in the unknown X :
A i Xi
+ Xi Ai + Ei (X ) 0 Xi Bi Ri01Bi0 Xi + Ci0 Ci = 0; n0 i = 1; . . . ; N; with X 2 M :
(15)
The relation between the solution of the JLQ problem and the CARE is presented in the next proposition, which is drawn from [5, Th. 3]. We need the definition that follows, in a parallel with Definition 6. Definition 7: We say that a solution X 2 Mn0 of the CARE in (15) is i) minimal if X Y for any solution Y 2 Mn0 of the CARE; ii) stabilizing if (A + BG, 3) is MS-stable, where G 2 Mr2n is defined as Gi = 0Ri01 Bi0 Xi for each i; iii) positive–definite if X 2 Mn+ . Proposition 7: Assume that X 2 Mn0 is the minimal solution of the CARE and P 2 Mn0 is the solution of the JLQ problem. Then, X = P. Remark 5: If one considers in the JLQ problem (5) the additional constraint that limt!1 E fkx(t)k2 g = 0 for each initial condition or,
V. EXAMPLES In this section, we give numerical examples connected with the results in this paper. The examples possess certain parameters and " which are changed in order to obtain systems that are non-W-detectable, W-detectable, and W-observable, respectively, and we study the behavior of the solution of the JLQ problem and the CARE. Example 1: Consider the MJLS described as
=
"
1 2
1
1
3
= 0 24 B1 = 22 B2 = 22 1 01 1 1 C1 = C2 = 0 (16) 4 R1 = R2 = 1 3 = 5 05 with = 1 and " = 0. From Proposition 2iii), one can check that (A, C , 3) is not W-detectable. This is an interesting setting for studying A1
A2
the behavior of the solution of the JLQ problem and the CARE and to illustrate that the minimal solution differs from the stabilizing one when the system is MS-stabilizable but not W-detectable; see Table I. We use the method of [7] to show that X1
35 08:977 = 0538::977 2:685
X2
010:00 = 02210:73 :00 6:820
(17)
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 50, NO. 5, MAY 2005
695
is a stabilizing solution of the CARE. However, it is not the minimal one, since one can check by inspection that P1
0 = 00 0:6580
P2
0 = 00 0:8240
(18)
is also a solution of the CARE, and Xi > Pi . Indeed, P is the minimal solution of the CARE and thus it is the solution of the JLQ problem. However, one can check that the solution P is not stabilizing,1 thus verifying the result of Theorem 1i). It is interesting to notice from the solution P that the optimal control is null when the state is in the vector subspace generated by the vector [1 0]0 . The reason is that the cost functional is null for x in and, thus, no control effort is necessary in this space. One can check, in Pi ; see also Lemma 2. Along connection, that = Oi = this line, it is also interesting to check that the solution depends only ; in fact, if we solve the on the parameters related with subspace = JLQ problem associated with the reduced system A = A(2; 2), B = C (1; 2), R, and 3, we get the solution P = P (2; 2). B (2; 1), C Example 2: Consider the MJLS described in (16) of Example 1 now with < 0. With this modification we stabilize nonobserved trajectories. In this situation, one can check that (A, C , 3) is strictly W-detectable (strictly in the sense that it is not W-observable nor MS-detectable, see Remark 1) and P as in (18) is a stabilizing solution of the JLQ problem, see Theorem 1. Notice that the solution of the JLQ problem did not vary with , in connection with the comments of Example 1. Example 3: Consider the MJLS described in (16) of Example 1, now with " = 0. The parameter " is chosen in such a manner that (A, C , 3) is W-observable. The solution of the JLQ problem is stabilizing, positive definite and it approaches X in (17) when " 0. However, the solution of the JLQ problem P (") at " = 0 is as in (18); it is no longer stabilizing, and in view of (17) and (18) we conclude that " P (") is a lower semicontinuous function at zero. The fact is that the system is W-observable provided that " = 0 and thus the solution to the CARE is stabilizing and unique. By contrast, when " is set to zero W-detectability is lost and the CARE may have multiple solutions; in this situation, the minimal solution is not stabilizing (see Lemma 3 and Table I), as is the case in the present example. If stability is required without W-detectability, this condition should appear explicitly as an extra constraint in the minimization problem, and here one gets (17) as the solution, see Remark 5. Clearly, the stabilizing solution will incur in a greater cost. As result, a separation in multiple solutions of the CARE occurs, creating a lattice that has the JLQ solution as the minimal extreme. This comes as a natural extension of previous studies of the LQ problem of deterministic systems, such as [17] or [18, Sec. 6].
S
S
S Nf g Nf g S?
6
!
! 6
VI. CONCLUSION This paper clarifies the role that the concepts of W-detectability and W-observability play as necessary and sufficient conditions for stabilizability and positiveness of solutions of the JLQ problem, respectively. It is shown in Lemma 1 that W-detectability of (A, C , 3) is a sufficient and also a necessary condition for a finite cost to imply mean square stable state trajectories. This means that W-detectability is the weakest condition that relates a bounded cost with MS-stability of the system; if a weaker condition is taken into account, one has that limt E x(t) 2 = 0 for some initial condition x0 , 0 for which J (x 0 ; 0 ) < . Pi = Another characterization presented here is that i , n0 n N;n is the solution of the JLQ problem and where P
!1
1For B R
fk k g 6 1 2M
Nf g NfO g O2M
~, 3), with A~ this purpose, we can test the MS-stability of (A ~ = 0, and C~ = I , i = 1; . . . ; N . B P , B
=
A
0
is the set of observability matrices of the MJLS. The results allows us to present, in Theorem 1, the role that the concepts play in the JLQ problem. — W-detectability is a necessary and sufficient condition for the solution of the JLQ problem to be stabilizing. — W-observability is a necessary and sufficient condition for the solution of the JLQ problem to be positive–definite. These properties are immediately inherited by the CARE, since the minimal solution of the former is identified with the solution of the latter. The complete scenario is presented in Table I. ACKNOWLEDGMENT The author would like to thank two anonymous reviewers whose careful reading of the manuscript helped to improve this note. REFERENCES [1] B. D. O. Anderson and J. B. Moore, “Detectability and stabilizability of time-varying discrete-time linear systems,” SIAM J. Control Optim., vol. 19, no. 1, pp. 20–32, 1981. [2] E. F. Costa and J. B. R. do Val, “Weak detectability and the linear quadratic control problem of discrete-time Markov jump linear systems,” Int. J. Control, vol. 75, pp. 1282–1292, 2002. [3] , “On the detectability and observability of discrete-time Markov jump linear systems,” Syst. Control Lett., vol. 44, pp. 135–145, 2001. [4] , “On the detectability and observability of continuous-time Markov jump linear systems,” SIAM J. Control Optim., vol. 41, pp. 1295–1314, 2002. [5] [6] E. F. Costa, J. B. R. do Val, and M. D. Fragoso, “On a detectability concept of discrete-time infinite Markov jump linear systems,” Stoch. Anal. Appl., vol. 23, no. 1, pp. 1–5, 2005. [7] O. L. V. Costa, J. B. R. do Val, and J. C. Geromel, “Continuous-time state-feedback H -control of Markovian jump linear systems via convex analysis,” Automatica, vol. 35, pp. 259–268, 1999. [8] O. L. V. Costa and M. D. Fragoso, “Discrete-time LQ-optimal control problems for infinite Markov jump parameter systems,” IEEE Trans. Autom. Control, vol. 40, no. 12, pp. 2076–2088, Dec. 1995. [9] J. B. R. do Val and E. F. Costa, “Numerical solution for the linearquadratic control problem of Markov jump linear systems and a weak detectability concept,” J. Optim. Theory Appl., vol. 114, no. 1, pp. 69–96, 2002. [10] J. B. R. do Val, J. C. Geromel, and O. L. V. Costa, “Solutions for the linear quadratic control problem of Markov jump linear systems,” J. Optim. Theory Appl., vol. 103, pp. 283–311, 1999. [11] X. Feng, K. A. Loparo, Y. Ji, and H. J. Chizeck, “Stochastic stability properties of jump linear systems,” IEEE Trans. Autom. Control, vol. 37, no. 1, pp. 38–53, Jan. 1992. [12] M. Fragoso and J. Baczynski, “Optimal control for continuous time LQ problems with infinite Markov jump parameters,” SIAM J. Control Optim., vol. 40, no. 1, pp. 270–297, 2001. [13] W. W. Hager and L. L. Horowitz, “Convergence and stability properties of the discrete Riccati operator equation and the associated optimal control and filtering problems,” SIAM J. Control Optim., vol. 14, no. 2, pp. 295–312, 1976. [14] Y. Ji and H. J. Chizeck, “Controlability, stabilizability and continuous time Markovian jump linear quadratic control,” IEEE Trans. Autom. Control, vol. 35, no. 7, pp. 777–788, Jul. 1990. , “Jump linear quadratic Gaussian control: steady-state solution and [15] testable conditions,” Control Theory Adv. Technol., vol. 6, no. 3, pp. 289–319, 1990. [16] , “Jump linear quadratic gaussian control in continuous time,” IEEE Trans. Autom. Control, vol. 37, no. 12, pp. 1884–1892, Dec. 1992. [17] V. Kucera, “On nonnegative definite solutions to matrix quadratic equations,” Automatica, vol. 8, pp. 413–423, 1972. [18] B. P. Molinari, “The time-invariant linear-quadratic optimal control problem,” Automatica, vol. 13, pp. 347–357, 1977. [19] T. Morozan, “Stability and control for linear systems with jump Markov perturbations,” Stoch. Anal. Appl., vol. 13, no. 1, pp. 91–110, 1995. [20] M. A. Rami and L. E. Ghaoui, “LMI optimization for nonstandard Riccati equations arising in stochastic control,” IEEE Trans. Autom. Control, vol. 41, no. 11, pp. 1666–1671, Nov. 1996.
