Feedback Control of Switched Stochastic Systems Using Randomly ...
Feedback Control of Switched Stochastic Systems Using Randomly Available Active Mode Information ⋆ Ahmet Cetinkaya a , Tomohisa Hayakawa
arXiv:1409.2578v1 [cs.SY] 9 Sep 2014
a
a,1
Department of Mechanical and Environmental Informatics, Tokyo Institute of Technology, Tokyo 152-8552, Japan
Abstract Almost sure asymptotic stabilization of a discrete-time switched stochastic system is investigated. Information on the active operation mode of the switched system is assumed to be available for control purposes only at random time instants. We propose a stabilizing feedback control framework that utilizes the information obtained through mode observations. We first consider the case where stochastic properties of mode observation instants are fully known. We obtain sufficient asymptotic stabilization conditions for the closed-loop switched stochastic system under our proposed control law. We then explore the case where exact knowledge of the stochastic properties of mode observation instants is not available. We present a set of alternative stabilization conditions for this case. The results for both cases are predicated on the analysis of a sequence-valued process that encapsulates the stochastic nature of the evolution of active operation mode between mode observation instants. Finally, we demonstrate the efficacy of our results with numerical examples. Key words: Switched stochastic systems; almost sure stabilization; random mode observations; missing mode observations; countable-state Markov processes; renewal processes
1 Introduction The framework developed for switched stochastic systems provides accurate characterization of numerous complex real life processes from physics and engineering fields that are subject to randomly occurring incidents such as sudden environmental variations or sharp dynamical changes (Cassandras and Lygeros, 2006; Yin and Zhu, 2010). Stabilization problem for switched stochastic systems has been investigated in many studies (e.g., Ghaoui and Rami (1996), de Farias et al. (2000), Fang and Loparo (2002), Costa et al. (2004), Sathanantan et al. (2008), Geromel et al. (2009) and the references therein). Control frameworks developed for switched stochastic systems often require the availability of information on the ⋆ This research was supported in part by JSPS Grant-in-Aid for Scientific Research (A) 26249062 and (C) 25420431, the Aihara Innovative Mathematical Modelling Project (JSPS) under FIRST program initiated by CSTP, and Japan Science and Technology Agency under CREST program. The material in this paper was partially presented at the 52nd IEEE Conference on Decision and Control, 2013, Firenze, Italy. Email addresses: [email protected] (Ahmet Cetinkaya), [email protected] (Tomohisa Hayakawa ). 1 Tel. : +81 3 5734 2762; Fax: +81 3 5734 2762
Preprint submitted to Automatica
active operation mode at all times. Note that for numerous applications the active mode describes the operating conditions of a physical process and is driven by external incidents of stochastic nature. The active mode, hence, may not be directly measurable and it may not be available for control purposes at all time instants during the course of operation. When the controller does not have access to any mode information, for achieving stabilization one can resort to adaptive control frameworks (Nassiri-Toussi and Caines, 1991; Caines and Zhang, 1992; Bercu et al., 2009) or mode-independent control laws (Vargas et al., 2006; Boukas, 2006). On the other hand, if mode information can be observed at certain time instants (even if rarely), this information can be utilized in the control framework. In our earlier work (Cetinkaya and Hayakawa, 2012; Cetinkaya and Hayakawa, 2013b), we investigated stabilization of switched stochastic systems for the case where only sampled mode information is available for control purposes. Under the assumption that the active mode is periodically observed, we proposed a stabilizing feedback control framework that utilizes the available mode information. In practical applications, it would be ideal if the mode information of a switched system is available for control purposes at all time instants or at least periodically. However, there are cases where mode information is obtained at random time instants. This situation occurs for example when the mode is sampled at all time instants; however, some of
September 10, 2014
setting, we present alternative sufficient stabilization conditions which can be used for verifying stability even if the distribution is not exactly known.
the mode samples are randomly lost during communication between mode sampling mechanism and the controller. On the other hand, in some applications, the mode has to be detected, but the detected mode information may not always be accurate. In this case each mode detection has a confidence level. Mode information with low confidence is discarded. As a result, depending on the confidence level of detection, the controller may or may not receive the mode information at a particular mode detection instant. In addition, we may also take advantage of random sampling for certain cases and observe the mode intentionally at random instants, as for such cases control under random sampling provides better results compared to periodic sampling. Note that random sampling has also been used for problems such as signal reconstruction and has been shown to have advantages over regular periodic sampling (see Boyle et al. (2007), Carlen and Mendes (2009)).
The paper is organized as follows. We provide the notation and a review of key results concerning renewal processes in Section 2. In Section 3, we propose our feedback control framework for stabilizing discrete-time switched stochastic systems under randomly available mode information. Then in Section 4, we present sufficient conditions under which our proposed control law guarantees almost sure asymptotic stabilization. In Section 5, we demonstrate the efficacy of our results with two illustrative numerical examples. Finally, in Section 6 we conclude our paper. 2 Mathematical Preliminaries
In this paper our goal is to explore the feedback stabilization problem for the case where the active operation mode, which is modeled as a finite-state Markov chain, is observed at random time instants. We provide an extended discussion based on our preliminary report (Cetinkaya and Hayakawa, 2013a). Specifically, we assume that the length of intervals between consecutive mode observation instants are identically distributed independent random variables. We employ a renewal process to characterize the occurrences of random mode observations. This characterization allows us to also explore periodic mode observations (Cetinkaya and Hayakawa, 2012; Cetinkaya and Hayakawa, 2013b) as a special case.
In this section, we provide notation and several definitions concerning discrete-time stochastic processes. Specifically, we denote positive and nonnegative integers by N and N0 , respectively. Moreover, R denotes the set of real numbers, Rn denotes the set of n × 1 real column vectors, and Rn×m denotes the set of n × m real matrices. We write (·)T for transpose, k · k for the Euclidean vector norm. We use λmin (H) (resp., λmax (H)) for the minimum (resp., maximum) eigenvalue of the Hermitian matrix H. A function V : Rn → R is called positive definite if V (x) > 0, x 6= 0, and V (0) = 0. We represent a finite-length sequence of ordered elements q1 , q2 , . . . , qn by q = (q1 , q2 , . . . , qn ). The length (number of elements) of the sequence q is denoted by |q|. The notations P[·] and E[·] respectively denote the probability and expectation on a probability space (Ω, F , P) with filtration {Fk }k∈N0 . Furthermore, we write 1[G] : Ω → {0, 1} for the indicator of the set G ∈ F , that is, 1[G] (ω) = 1, ω ∈ G, and 1[G] (ω) = 0, ω ∈ / G.
We propose a linear feedback control law with a piecewiseconstant gain matrix that is switched depending on the value of a randomly sampled version of the mode signal. In order to investigate the evolution of the active mode together with its randomly sampled version, we construct a stochastic process that represents sequences of values the mode takes between random mode observation instants. This sequencevalued stochastic process turns out to be a countable-state Markov chain defined over a set that is composed of all possible mode sequences of finite length. We first analyze the probabilistic dynamics of this sequence-valued Markov chain. Then based on our analysis, we obtain sufficient stabilization conditions for the closed-loop switched stochastic system under our proposed control framework. These stabilization conditions let us assess whether the closed-loop system is stable for a given probability distribution for the length of intervals between consecutive mode observation instants. As this probability distribution is not assumed to have a certain structure, the result presented in this paper can also be considered as a generalization of the result provided in Cetinkaya and Hayakawa (2011), where stabilization problem is discussed in continuous time and the random intervals between mode sampling instants are specifically assumed to be exponentially distributed. In this paper we also explore the case where perfect information regarding the probability distribution for the length of intervals between consecutive mode observation instants is not available. For this problem
2.1 Discrete-Time Renewal Processes A discrete-time renewal process {N (k) ∈ N0 }k∈N0 with initial value N (0) = 0 is an Fk -adapted stochastic counting P process defined by N (k) , i∈N 1[ti ≤k] , where ti ∈ N0 , i ∈ N0 , are random time instants such that t0 = 0 and τi , ti − ti−1 ∈ N, i ∈ N, are identically distributed independent random variables with finite expectation (i.e., E[τi ] < ∞, i ∈ N). Note that τi , i ∈ N, denote the lengths of intervals between time instants ti , i ∈ N0 . Furthermore, we use µ : N → [0, 1] to denote the common distribution of the random variables τi , i ∈ N, such that P[τi = τ ] = µτ ,
τ ∈ N,
i ∈ N,
(1)
P where µτ ∈ [0, 1]. Note that τ ∈N µτ = 1. Now, let P τˆ , τ ∈N τ µτ = E[τ1 ](= E[τi ], i ∈ N). It follows as a consequence of strong law of large numbers for renewal processes (see Serfozo (2009)) that limk→∞ Nk(k) = τ1ˆ .
2
Note that in Section 3, we employ a renewal process to characterize the occurrences of random mode observations. Figure 1. Mode transition diagram for {r(k) ∈ M , {1, 2}}k∈N0
2.2 Almost Sure Asymptotic Stability using a transition diagram, which shows possible transitions between the operation modes of the switched system. Mode transition diagram for a switched system with two modes is shown in Figure 1.
The zero solution x(k) ≡ 0 of a stochastic system is almost surely stable if, for all ǫ > 0 and ρ > 0, there exists δ = δ(ǫ, ρ) > 0 such that if kx(0)k < δ, then P[ sup kx(k)k > ǫ] < ρ.
In this paper, we assume that the mode signal is an aperiodic, irreducible Markov chain and has the invariant distribution π : M → [0, 1].
(2)
k∈N0
Furthermore, the zero solution x(k) ≡ 0 of a stochastic system is asymptotically stable almost surely if it is almost surely stable and P[ lim kx(k)k = 0] = 1.
3.1 Feedback Control Under Randomly Observed Mode Information
(3)
k→∞
In this paper, active mode of the switched stochastic system (4) is assumed to be observed only at random time instants, which we denote by ti ∈ N0 , i ∈ N0 . We assume that t0 = 0 and τi , ti − ti−1 ∈ N, i ∈ N, are independent random variables that are distributed according to a common distribution µ : N → [0, 1] for all i ∈ N such that τˆ , P τ ∈N τ µτ < ∞. In this problem setting, the initial mode information r0 is assumed to be available to the controller, and a renewal process {N (k) ∈ N0 }k∈N0 is employed for counting the number of mode observations that are obtained after the initial time. We assume that the renewal process {N (k) ∈ N0 }k∈N0 and the mode signal {r(k) ∈ M}k∈N0 are mutually independent.
In Sections 3 and 4, we investigate almost sure asymptotic stabilization of a switched stochastic system. 3 Stabilizing Switched Stochastic Systems with Randomly Available Mode Information In this section, we propose a feedback control framework for stabilizing a switched stochastic system by using only the randomly available mode information. Specifically, we consider the discrete-time switched linear stochastic system with M ∈ N number of modes given by x(k + 1) = Ar(k) x(k) + Br(k) u(k),
k ∈ N0 ,
Following our approach in Cetinkaya and Hayakawa (2011), Cetinkaya and Hayakawa (2012), Cetinkaya and Hayakawa (2013b), we employ a linear feedback control law with a ‘piecewise-constant’ feedback gain matrix that depends only on the obtained mode information. Specifically, we consider the control law
(4)
with the initial conditions x(0) = x0 , r(0) = r0 ∈ M , {1, 2, . . . , M }, where x(k) ∈ Rn and u(k) ∈ Rm respectively denote the state vector and the control input; furthermore, Ai ∈ Rn×n , Bi ∈ Rn×m , i ∈ M, are the subsystem matrices. The mode signal {r(k) ∈ M}k∈N0 is assumed to be an Fk -adapted, M -state discrete-time Markov chain with the initial distribution denoted by ν : M → [0, 1] such that νr0 = 1 and νi = 0, i 6= r0 .
u(k) = Kσ(k) x(k),
σ(k) , r(tN (k) ),
(l)
(l)
that pi,j ∈ [0, 1] is in fact the l-step transition probability from mode i to mode j, that is,
(0)
k ∈ N0 .
(7)
Note that the sampled mode signal {σ(k) ∈ M}k∈N0 acts as a switching mechanism for the linear feedback gain, which remains constant between two consecutive mode observation instants, that is, Kσ(k) = Kr(ti ) for k ∈ [ti , ti+1 ).
we use pi,j to denote (i, j)th entry of the matrix P l . Note
(l)
(6)
where {σ(k) ∈ M}k∈N0 is the sampled version of the mode signal defined by
We use the matrix P ∈ RM×M to characterize probability of transitions between the modes of the switched system. Specifically, pi,j ∈ [0, 1], which is the (i, j)th entry of the matrix P , denotes theP probability of a transition from mode i to mode j. Note that j∈M pi,j = 1, i ∈ M. Furthermore,
pi,j , P[r(k + l) = j|r(k) = i], l ∈ N0 , i, j ∈ M,
k ∈ N0 ,
Between two consecutive mode observation instants, the feedback gain Kσ(·) stays constant, whereas the active mode r(·) of the dynamical system (4) may change its value. Stabilization performance under the control law (6) hence depends not only on the length of the intervals between random mode observation instants, but also on how the active mode switches during the intervals.
(5)
(0)
with pi,i = 1, i ∈ M, pi,j = 0, i 6= j. Furthermore, (1)
pi,j = pi,j , i, j ∈ M. The mode signal can be represented
3
Figure 3. Transition diagram of the sequence– valued discrete-time countable-state Markov chain {s(i) ∈ S , {(1), (2), (1, 1), . . .}}i∈N0 over the set of mode sequences of variable length
Figure 2. Actual mode r(k) and its sampled version σ(k)
In Figure 2, we show sample paths of the active mode signal r(·) and its sampled version σ(·) for a switched stochastic system with M = 2 modes. In this example, active mode is observed at time instants t0 = 0, t1 = 2, t2 = 5, t3 = 6, t4 = 8, . . .. Note that at mode observation instants actual mode signal r(·) and its sampled version σ(·) have the same value. However, at the other time instants, sampled mode signal may differ from the actual mode, since between mode observation instants, system mode may switch.
Figure 4. Transition diagram of the sequence-valued discrete-time Markov chain {s(i) ∈ S , {(1, 1), (1, 2), (2, 1), (2, 2)}}i∈N0
closed-loop switched stochastic control system (4), (6). 3.2 Probabilistic Dynamics of Mode Sequences
In order to investigate the evolution of the active mode between consecutive mode observation instants, we construct a new stochastic process {s(i)}i∈N0 that takes values from a countable set of mode sequences of variable length. Specifically, we define {s(i)}i∈N0 by
The possible values of sequence that the stochastic process {s(i)}i∈N0 may take are characterized by the set
(8)
S , {(q1 , q2 , . . . , qτ ) : pqn ,qn+1 > 0, n ∈ {1, . . . , τ − 1}; qn ∈ M, n ∈ {1, . . . , τ }; µτ > 0}. (9)
with ti , i ∈ N0 , being the random mode observation instants. By the definition given in (8), s(i) represents the sequence of values that the active mode r(·) takes between the mode observation instants ti and ti+1 . Hence, sn (i), which denotes the nth element of the sequence s(i), represents the value of the active mode r(·) at time ti + n − 1. Furthermore, the value of the sampled mode signal σ(·) between time instants ti and ti+1 is represented by s1 (i) = r(ti ). Note that the active mode is observed and becomes available for control purposes only at time instants ti , i ∈ N0 . Thus, the controller has access only to the observed mode data σ(ti ) = r(ti ), i ∈ N0 , which correspond to the first elements of the sequences s(i), i ∈ N0 .
Note that the sequence-valued stochastic process {s(i)}i∈N0 is a discrete-time Markov chain on the countable state space represented by S, which contains all possible mode sequences for all possible lengths of intervals between consecutive mode observation instants. For example, consider the case where the switched system (4) has two modes. Furthermore, suppose that µτ > 0 for all τ ∈ N. In other words, lengths of intervals between mode observation instants may take any positive integer value. In this case, the state space S = {(1), (2), (1, 1), (1, 2), . . .} contains all finite-length mode sequences composed of elements from M = {1, 2}. See Figure 3 for the transition diagram of countable-state Markov chain {s(i) ∈ S}i∈N0 of this example.
For the sample paths of active mode signal r(·) and its sampled version σ(·) shown in Figure 2, mode sequences between mode observation instants t0 = 0, t1 = 2, t2 = 5, t3 = 6, t4 = 8, are given as s(0) = (1, 2), s(1) = (2, 1, 2), s(2) = (2), s(3) = (2, 1). The key property of the stochastic process {s(i)}i∈N0 is that, a given mode sequence s(i) indicates full information of the active mode as well as the information the controller has during the time interval between consecutive mode observation instants ti and ti+1 .
It is important to note that if the set {τ ∈ N : µτ > 0} has finite number of elements, then set S will also contain finite number of sequences. In other words, if the lengths of intervals between mode observation instants have finite number of possible values, then the number of possible sequences is also finite. For example, consider the case where the operation mode of the switched system, which takes values from the index set M = {1, 2}, is observed periodically with period 2, that is, µ2 = 1. In this case, S = {(1, 1), (1, 2), (2, 1), (2, 2)} (see Figure 4).
s(i) , r(ti ), r(ti + 1), . . . , r(ti+1 − 1) , i ∈ N0 ,
In what follows, we explain the probabilistic dynamics of the stochastic process {s(i)}i∈N0 and provide key results that we will use in Section 4 for analyzing stability of the
We now characterize the initial distribution and the statetransition probabilities of the discrete-time Markov chain
4
{s(i) ∈ S}i∈N0 as functions of the initial distribution and the state-transition probabilities of the mode signal {r(k) ∈ M}k∈N0 . Specifically, the initial distribution λ : S → [0, 1] of the Markov chain {s(i) ∈ S}i∈N0 is given by λq = P[s(0) = q] = P[t1 = |q|, r(0) = q1 , . . . , r(|q| − 1) = q|q| ] = P[t1 = |q| r(0) = q1 , . . . , r(|q| − 1) = q|q| ] · P[r(0) = q1 , . . . , r(|q| − 1) = q|q| ], q ∈ S.
by the last element of sequence q, to the mode represented by the first element of the sequence q¯. Furthermore, the exQ|¯q|−1 pression n=1 pq¯n ,¯qn+1 denotes the joint probability that the active mode takes the values denoted by the elements of the sequence q¯ until the next mode observation instant. Since the mode signal {r(k) ∈ M}k∈N0 is aperiodic and irreducible, mode sequences may start with any of the possible modes indicated by the index set M = {1, . . . , M }. Furthermore, it is possible to reach from any mode sequence to another mode sequence in a finite number of mode observations. Hence, the discrete-time Markov chain {s(i) ∈ S}i∈N0 is irreducible. In Lemma 3.1 below, we provide the invariant distribution for the countable-state discrete-time Markov chain {s(i) ∈ S}i∈N0 . Note that the distribution φ : S → [0, 1] : j 7→ φj is called invariantPdistribution of the Markov chain {s(i) ∈ S}i∈N0 if φj = i∈S φi ρi,j , j ∈ S. The invariant distribution for the case where S contains only sequences of fixed length T ∈ N is provided in Serfozo (2009). In Lemma 3.1, we consider the more general case where S may contain countably infinite number of sequences of all possible lengths.
(10)
Since the mode signal {r(k) ∈ M}k∈N0 and the mode observation counting process {N (k) ∈ N0 }k∈N0 are mutually independent, mode transitions and mode observations occur independently. Hence, t1 = τ1 is independent of r(n) for every n ∈ N0 . As a consequence, λq = P[t1 = |q|] P[r(0) = q1 , . . . , r(|q| − 1) = q|q| ] = P[t1 = |q|] P[r(0) = q1 ] |q|−1
·
Y
P[r(n) = qn+1 |r(n − 1) = qn ]
n=1
=
(
µ|q| 0,
Q|q|−1 n=1
pqn ,qn+1 ,
if q1 = r0 , q ∈ S, otherwise.
Lemma 3.1. Discrete-time Markov chain {s(i) ∈ S}i∈N0 has invariant distribution φ : S → [0, 1] : q 7→ φq given by
(11)
|q|−1
Note that s1 (0), which is the first element of the first mode sequence s(0), is equal to the initial mode r0 .
φq , πq1 µ|q|
r(ti ) = q1 , . . . , r(ti + |q| − 1) = q|q| ,
(12)
|q|−1
X
φq ρq,¯q =
n=1
q∈S
|¯ q|−1
Y
pqn ,qn+1 pq|q| ,¯q1
pq¯n ,¯qn+1 ,
q¯ ∈ S.
(15)
Now let Sτ , {q ∈ S : |q| = τ }, τ ∈ N. Note that the set Sτ contains all mode sequences of length τ . We rewrite the sum in (15) to obtain
P[r(ti+1 + n) = q¯n+1 |r(ti+1 + n − 1) = q¯n ]
|q|−1
X
|¯ q|−1
i ∈ N0 .
Y
n=1
n=1
pq¯n ,¯qn+1 ,
πq1 µ|q|
· µ|¯q|
|¯ q|−1
Y
X
q∈S
= P[r(ti+1 ) = q¯1 | r(ti + |q| − 1) = q|q| ]P[τi+1 = |¯ q |]
= pq|q| ,¯q1 µ|¯q|
(14)
Proof. We prove this result by showing that φq¯ = P ¯ ∈ S. First, by (13) and (14) q , for all q q∈S φq ρq,¯
ρq,¯q = P τi+1 = |¯ q |, r(ti+1 ) = q¯1 , . . . , r(ti+1 + |¯ q | − 1) = q¯|¯q| r(ti + |q| − 1) = q|q| Y
q ∈ S,
where π : M → [0, 1] and pi,j , i, j ∈ M, respectively denote the invariant distribution and transition probabilities of the finite-state Markov chain {r(k) ∈ M}k∈N0 .
for i ∈ N0 . Note that τi+1 is independent of the random variables r(n), n ∈ N0 , and τi . Furthermore, given r(ti +τi −1), the random variable r(ti+1 ) is conditionally independent of r(ti ), . . . , r(ti + τi − 2), and τi . It follows that
·
pqn ,qn+1 ,
n=1
Probability of a transition from a mode sequence q ∈ S to another mode sequence q¯ ∈ S is given by ρq,¯q = P[s(i + 1) = q¯|s(i) = q], = P τi+1 = |¯ q |, r(ti+1 ) = q¯1 , . . . , r(ti+1 + |¯ q | − 1) = q¯|¯q| τi = |q|,
Y
πq1 µ|q|
n=1
q∈S
(13)
n=1
=
X
µτ
X
µτ
τ ∈N
Note that µ|¯q | in (13) represents the probability that length of the interval between two mode observation instants is equal to the length of the sequence q¯, whereas pq|q| ,¯q1 ∈ [0, 1] represents the transition probability from the mode represented
=
τ ∈N
5
Y
X
pqn ,qn+1 pq|q| ,¯q1
πq1
n=1
q∈Sτ
X
qτ ∈M
τY −1
···
X
pqn ,qn+1 pqτ ,¯q1
q1 ∈M
πq1
τY −1
n=1
pqn ,qn+1 pqτ ,¯q1 .
(16)
Note that since π : M → [0, 1] is the invariant distribution of the finite-state Markov chain {r(k) ∈ M}k∈N0 , it P follows that π p i i,j = πj , i, j ∈ M. Thus, we have i∈M P = πqn+1 , n ∈ {1, . . . , τ − 1}, and p π Pqn ∈M qn qn ,qn+1 q1 = πq¯1 . As a result, from (16) we obtain qτ ∈M πqτ pqτ ,¯
then the control law (6) with the feedback gain matrix ˜ −1 , Kσ(k) = Lσ(k) R
guarantees that the zero solution x(k) ≡ 0 of the closed-loop system (4) and (6) is asymptotically stable almost surely.
|q|−1
X
πq1 µ|q|
Y
n=1
q∈S
X pqn ,qn+1 pq|q| ,¯q1 = µτ πq¯1
˜ −1 . It Proof. First, we define V (x) , xT Rx, where R , R follows from (4) and (6) that for k ∈ N0 ,
τ ∈N
= πq¯1 .
(17)
Finally, substituting (17) into (15) yields
V (x(k + 1)) = xT (k)(Ar(k) + Br(k) Kσ(k) )T R · (Ar(k) + Br(k) Kσ(k) )x(k).
|¯ q|−1
X
φq ρq,¯q = πq¯1 µ|¯q |
Y
pq¯n ,¯qn+1 = φq¯,
q¯ ∈ S,
(22)
(18) We set Lj = Kj R−1 , j ∈ M, and use (19) and (22) to obtain
n=1
q∈S
(21)
which completes the proof. V (x(k + 1)) ≤ ζr(k),σ(k) V (x(k)) ≤ η(k)V (x(0)), We have now established that the countable-state Markov chain {s(k) ∈ S}k∈N0 is irreducible and has the invariant distribution φ : S → [0, 1] presented in Lemma 3.1. Note that the strong law of large numbers (also called ergodic theorem; see Norris (2009), Serfozo (2009), Durrett (2010)) for discrete-time P Markov chains states that Pn−1 P[limn→∞ n1 k=0 ξs(k) P= i∈S φi ξi ] = 1, for any ξi ∈ R, i ∈ S, such that i∈S φi |ξi | < ∞. This result for the countable-state Markov chain {s(k) ∈ S}k∈N0 is crucial to obtain the main results of Section 4 below. Specifically, in our stability analysis we utilize the ergodic theorem for Markov chains. In the literature, for the stability analysis of finite-mode (Bolzern et al., 2004) and infinite-mode (Li et al., 2012) discrete-time switched stochastic systems, researchers employed ergodic theorem for the Markov chain that characterizes the mode signal. In the next section, we use ergodic theorem for the Markov chain that characterizes the sequence of mode values between consecutive mode observation instants.
Qk for k ∈ N0 , where η(k) , n=0 ζr(n),σ(n) , k ∈ N. We will first show that η(k) → 0 almost surely as k → ∞. Note that η(k) > 0, k ∈ N0 . Then, it follows that ln η(k) =
τ ∈N
µτ
(l−1)
πi pi,j
i, j ∈ M,
ln ζj,i < 0,
(24)
tN (k) −1
ln η(k) =
X
ln ζr(n),σ(n) +
X
ξs(i) +
N (k)−1
=
i=0
where ξq ,
P|q|
k X
ln ζr(n),σ(n)
n=tN (k)
n=0
n=1
k X
ln ζr(n),σ(n) ,
(25)
n=tN (k)
ln ζqn ,q1 , q ∈ S.
Next, in order to evaluate limk→∞ k1 ln η(k), note that Pk limk→∞ k1 n=tN (k) ln ζr(n),σ(n) = 0. Consequently, N (k)−1 1 X 1 ξs(i) ln η(k) = lim k→∞ k k→∞ k i=0
lim
Theorem 4.1. Consider the switched linear stochastic sys˜ > 0, Li ∈ Rm×n , i ∈ M, tem (4). If there exist matrices R and scalars ζi,j ∈ (0, ∞), i, j ∈ M, such that
τ X X
ln ζr(n),σ(n) .
By using the definitions of stochastic processes {N (k) ∈ N0 }k∈N0 and {s(i) ∈ S}i∈N0 , we obtain
In this section, we employ the results presented in Section 3 to obtain sufficient conditions for almost sure asymptotic stabilization of the closed-loop system (4) under the control law (6).
X
k X
n=0
4 Sufficient Conditions for Almost Sure Asymptotic Stabilization
˜ + Bi Lj )T R ˜ −1 0 ≥ (Ai R ˜ ˜ + Bi Lj ) − ζi,j R, · (Ai R
(23)
N (k)−1 X N (k) 1 = lim ξs(i) . k→∞ k N (k) i=0
(19)
(26)
It follows from strong law of large numbers for reN (k) 1 newal processes P (Section 2.1) that limk→∞ k = τˆ , where τˆ = τ ∈N τ µτ . Furthermore, by the ergodic theorem for countable-state Markov chains, it follows that
(20)
l=1 i,j∈M
6
Pn−1 P limn→∞ n1 i=0 ξs(i) = q∈S φq ξq . Using the invariant distribution φ : S → [0, 1] given by (14), we get 1 ln η(k) k→∞ k |q| |q|−1 Y X 1X = πq1 µ|q| ln ζqm ,q1 . pqn ,qn+1 τˆ m=1 n=1
such that P[supk≥n η(k) > ǫ2 ] < ρ for n ≥ N (ǫ, ρ). Equivalently, P[sup η(k − 1) > ǫ2 ] < ρ,
lim
Let Sτ , {q ∈ S : |q| = τ }, τ ∈ N. Note that Sτ contains all mode sequences of length τ . It follows from (27) that
P[sup kx(k)k > ǫ
1 ln η(k) k→∞ k |q|−1 |q| Y X 1XX = πq1 µ|q| pqn ,qn+1 ln ζqm ,q1 τˆ n=1 m=1 lim
k≥n
=
q∈Sτ
λmin (R) kx(k)k2 > ǫ2 ] 2 k≥n λmax (R) kx(0)k
≤ P[sup η(k − 1) > ǫ2 ] < ρ,
Let δ1 ,
q∈Sτ
τY −1
πq1 (
=
X
q∈Sτ
=
τ,l
πi (ln ζj,i )
X
πi (ln ζj,i )pi,j
pqn ,qn+1 ) ln ζql ,q1
i,j n=1 q∈Sτ,l
(l−1)
.
s
λmax (R) kx(0)k] λmin (R)
n ≥ N (ǫ, ρ) + 1.
(33)
Now let ζ¯ , max{1, maxi,j∈M ζi,j }. It follows from (23) that V (x(k)) ≤ ζ¯k−1 V (x(0)) ≤ ζ¯N (ǫ,ρ)−1 V (x(0)) for all k ∈ {0, 1, . . . , N (ǫ, ρ)}. Therefore, kx(k)k2 ≤ (R) ζ¯N (ǫ,ρ)−1 λλmax kx(0)k2 , and hence, we have kx(k)k ≤ min (R) q (R) ζ¯N (ǫ,ρ)−1 λλmax kx(0)k, for all k ∈ {0, 1, . . . , N (ǫ, ρ)}. min (R) q min (R) Furthermore, let δ2 , ǫ ζ¯−N (ǫ,ρ)+1 λλmax (R) . Consequently, if kx(0)k ≤ δ2 , then kx(k)k ≤ ǫ, k ∈ {0, 1, . . . , N (ǫ, ρ)}, which implies
−1 X τY ( pqn ,qn+1 )
X
i,j∈M
=
k≥n
n=1
i,j∈M q∈S i,j
If kx(0)k ≤ δ1 , then
≤ P[sup kx(k)k > ǫ
pqn ,qn+1 ) ln ζql ,q1 τY −1
λmin (R) λmax (R) .
k≥n
n=1
πq1 (
q
P[sup kx(k)k > ǫ]
< ρ,
X
n ≥ N (ǫ, ρ) + 1. (32)
k≥n
i,j Furthermore, let Sτ,l , {q ∈ Sτ : q1 = i, ql = j}, i, j ∈ i,j M, l ∈ {1, 2, . . . , τ − 1}. The set Sτ,l contains all mode sequences of length τ that have i ∈ M and j ∈ M as the 1st and the lth elements, respectively. We use (5) to obtain
X
λmax (R) kx(0)k2 ] λmin (R)
= P[sup
τ X
τ τY −1 X X 1X µτ pqn ,qn+1 ) ln ζqm ,q1 . (28) πq1 ( τˆ m=1 n=1 τ ∈N
λmax (R) kx(0)k] λmin (R)
k≥n
X 1X µτ ln ζqm ,q1 pqn ,qn+1 ) πq1 ( τˆ m=1 n=1 τ ∈N
s
= P[sup kx(k)k2 > ǫ2
τ ∈N q∈Sτ
=
(31)
By the definition of V (·) and (23), we obtain η(k − 1) ≥ V (x(k)) λmin (R) kx(k)k2 V (x(0)) ≥ λmax (R) kx(0)k2 for all k ∈ N. Hence, it follows from (31) that, for all ǫ > 0 and ρ > 0, there exists a positive integer N (ǫ, ρ) such that
(27)
q∈S
τY −1
n ≥ N (ǫ, ρ) + 1.
k≥n
(29)
i,j∈M
max
P[ Substituting (29) into (28) yields
k∈{0,1,...,N (ǫ,ρ)}
τ 1X X X 1 (l−1) ln η(k) = µτ πi pi,j ln ζj,i . (30) k→∞ k τˆ
(34)
It follows from (33) and (34) that for all ǫ > 0, ρ > 0,
lim
τ ∈N
kx(k)k > ǫ] = 0.
l=1 i,j∈M
P[ sup kx(k)k > ǫ] = P[{ k∈N0
P
Now, since τˆ = τ ∈N τ µτ < ∞, as a result of (20), we have limk→∞ k1 ln η(k) < 0. Thus, limk→∞ ln η(k) = −∞ almost surely; furthermore, P[limk→∞ η(k) = 0] = 1. In the following, we first show that the zero solution is almost surely stable. To this end first note that for all ǫ > 0, limn→∞ P[supk≥n η(k) > ǫ2 ] = 0, which implies that for all ǫ > 0 and ρ > 0, there exists a positive integer N (ǫ, ρ)
max
k∈{0,1,...,N (ǫ,ρ)}
∪{
sup
kx(k)k > ǫ}
kx(k)k > ǫ}]
k≥N (ǫ,ρ)+1
≤ P[
max
k∈{0,1,...,N (ǫ,ρ)}
+ P[
sup
kx(k)k > ǫ]
kx(k)k > ǫ]
k≥N (ǫ,ρ)+1
< ρ,
7
(35)
whenever kx(0)k < δ , min(δ1 , δ2 ), which implies almost sure stability. As a final step of proving almost sure asymptotic stability of the zero solution, we now show (3). First, note that by (23), we have V (x(k + 1)) ≤ η(k)V (x(0)), k ∈ N. Now, since P[limk→∞ η(k) = 0] = 1, it follows that P[limk→∞ V (x(k)) = 0] = 1, which implies (3), and hence the zero solution of the closed-loop system (4), (6) is asymptotically stable almost surely.
i, j ∈ M, that satisfy (20) and at each iteration we look for feasible solutions to the linear matrix inequalities (36). In Section 5 below, we employ this method and find values ˜ ∈ Rn×n , Li ∈ Rm×n , i ∈ M, and scalars for matrices R ζi,j ∈ (0, ∞), i, j ∈ M, that satisfy (19), (20) for a given discrete-time switched linear system. It is important to note that the scalars ζi,j ∈ (0, ∞), i, j ∈ M, that satisfy (20) form an unbounded set. Note that this2 set is smaller than the entire nonnegative orthant in RM . However, we still need to reduce the search space of ζi,j , i, j ∈ M. To this end, first note that it is harder to find feasible solutions to linear matrix inequalities given by (36) when the scalars ζi,j , i, j ∈ M, are close to zero. Note also that if there exist a feasible solution to (36) for certain values of ζi,j , i, j ∈ M, then it is guaranteed that feasible solutions to (36) exist also for larger values of ζi,j , i, j ∈ M. Therefore, we can restrict our search space and iterate over large values of ζi,j , i, j ∈ M, that satisfy (20), and check feasible solutions to (36). Specifically, we only iterate over ζi,j , i, j ∈ M, that is close to the search space’s boundary identified by Pτ P P (l−1) ln ζj,i = 0. Now note that l=1 i,j∈M πi pi,j τ ∈N µτ in order for (20) to be satisfied, there must exist at least a pair i, j ∈ M such that ζi,j < 1. Since the scalar ζi,j represents the stability/instability margin for the dynamics characterized by the ith subsystem and the jth feedback gain, we expect ζi,i < 1 for stabilizable modes i ∈ M. This further reduces the search space for our numerical method.
Theorem 4.1 provides sufficient conditions for almost sure asymptotic stability of the closed-loop system (4) and (6). Conditions (19) and (20) of Theorem 4.1 indicate dependence of stabilization performance on subsystem dynamics, mode transition probabilities, and random mode observations. The effect of mode transitions on the stabilization is reflected in (19) through the limiting distribution π : M → [0, 1] as well as l-step transition probabilities (l) pi,j , i, j ∈ M. Furthermore, the effect of random mode observations is indicated in condition (19) by µ : N → [0, 1], which represents the distribution of the lengths of intervals between consecutive mode observation instants. Remark 4.2. We investigate the stability of the closed-loop system through the Lyapunov-like function V (x) , xT Rx ˜ −1 , where R ˜ is a positive-definite matrix that with R = R satisfy (19). The scalar ζi,j ∈ (0, ∞) in (19) characterizes an upper bound on the growth of the Lyapunov-like function, when the switched system evolves according to dynamics of the ith subsystem and the jth feedback gain. Note that if ζi,j ∈ (0, 1) for all i, j ∈ M, it is guaranteed that the Lyapunov-like function will decrease at each time step. However, we do not require ζi,j ∈ (0, 1) for all i, j ∈ M. There may be pairs i, j ∈ M such that ζi,j > 1, hence Lyapunov-like function V (·) may grow when ith subsystem and the jth feedback gain is active. As long as ζi,j , i, j ∈ M, satisfy (20) the Lyapunov-like is guaranteed to converge to zero in the long-run (even if it may grow at certain instants). Note that even though the conditions (19), (20) allow unstable subsystem-feedback gain pairs, some conservativeness may still arise due the characterization with single Lyapunov-like function. This conservatism may be reduced with an alternative approach with multiple Lyapunov-like functions assigned for each subsystem-feedback gain pairs.
Remark 4.4. Note that conditions (19) and (20) presented in Theorem 4.1 can also be used for determining almost sure asymptotic stability of the switched stochastic control system (4), (6) with periodically observed mode information. The renewal process characterization presented in this paper in fact encompasses periodic mode observations (explored previously in Cetinkaya and Hayakawa (2012) and Cetinkaya and Hayakawa (2013b)) as a special case. Specifically, suppose that the mode observation instants are given by ti = iT , i ∈ N0 , where T ∈ N denotes the mode observation period. Our present framework allows us to characterize periodic mode observations by setting the distribution µ : N → [0, 1] such that µT = 1 and µτ = 0, τ 6= T . Note that condition (20) of Theorem 4.1 for this PT P (l−1) case reduces to l=1 i,j∈M πi pi,j ln ζj,i < 0. Furthermore, if the controller has perfect mode information at all time instants (T = 1, hence σ(k) = r(k), k ∈ N0 ), condition (20) takes even a simpler form given by the inequality P i∈M πi ln ζi,i < 0.
Remark 4.3. In order to verify conditions (19) and (20) of Theorem 4.1, we take an approach similar to the one presented in Cetinkaya and Hayakawa (2013b). Specifically, we use Schur complements (see Bernstein (2009)) to transform condition (19) into the matrix inequalities "
˜ AˆT ζi,j R i,j 0≤ ˜ Aˆi,j R
#
,
i, j ∈ M,
Remark 4.5. Condition (20) of Theorem 4.1 has a simpler form also for the case where the length of intervals between consecutive mode observation instants are uniformly distributed over the set {τL , τL + 1, . . . , τH } with τL , τH ∈ N such that τL ≤ τH . In this case the distribution µ : N → [0, 1] is given by
(36)
˜ + Bi Lj ), i, j ∈ M. Note that the inwhere Aˆi,j , (Ai R ˜ and Li , i ∈ M. In our nuequalities (36) are linear in R merical method, we iterate over a set of the values of ζi,j ,
µτ ,
8
(
1 τH −τL +1 ,
0,
if τ ∈ {τL , τL + 1, . . . , τH }, otherwise.
(37)
Figure 5. Uniform distribution given by (37) with τL = 2 and τH = 5 for the length of intervals between consecutive mode observation instants
Figure 6. Distribution given by (38) with θ = 0.3 for the length of intervals between consecutive mode observation instants
(1 − θ)l−1 . Therefore,
Figure 5 shows the distribution (37) for an example case with τL = 2 and τH = 5.
X
τ ∈N
With (37), condition (20) of Theorem 4.1 reduces to the Pτ Pτ P (l−1) inequality τH=τL l=1 i,j∈M πi pi,j ln ζj,i < 0.
=
τ ∈ N.
(38)
It turns out that for µτ : N → [0, 1] given by (38), the lefthand side of condition (20) has a closed-form expression. Note that by changing the order of summations and using (38), we can rewrite the left-hand side of (20) as
τ ∈N
= =
µτ
τ X X
(l−1)
πi pi,j
πi (ln ζj,i )
i,j∈M
τ ∈N
X
∞ X
πi (ln ζj,i )
i,j∈M
=
ln ζj,i
µτ
τ X
X
πi (ln ζj,i )
X
πi (ln ζj,i )
∞ X
(l−1)
pi,j
(l−1) pi,j
∞ X
µτ
−
i,j∈M
Note that 1 −
(l−1)
pi,j
1−
µτ )
τ −1 θ τ =1 (1 − θ)
l−1 X
(1 − θ)τ −1 θ .
τ =1
l=1
Pl−1
l−1 X
τ =1
l=1
∞ X
πi (ln ζj,i )
l−1
= 1 − θ 1−(1−θ) 1−(1−θ)
∞ X
(l−1)
pi,j
(1 − θ)l−1 .
(40)
l=1
i ∈ N0 ,
(41)
where τ¯ ∈ N is a known constant. In this case time instants of consecutive mode observations are assumed to be at most τ¯ ∈ N steps apart. In other words, if (41) is satisfied, it is guaranteed that the length of intervals between consecutive mode observation instants cannot be larger than τ¯ ∈ N. It is important to note that (41) characterizes a requirement on the intervals between mode observation instants and it is not related to mode switches.
τ =l
(l−1) pi,j (1
X
P[ti+1 − ti ≤ τ¯] = 1,
l=1
l=1
i,j∈M
=
X
ln ζj,i
l=1 i,j∈M
Remark 4.7. Note that in order to check condition (20) of Theorem 4.1, one needs to have perfect information regarding the distribution µ : N → [0, 1], according to which the lengths of intervals between consecutive mode observation instants are distributed. In Theorem 4.8 below, we present alternative sufficient stabilization conditions, which do not require exact knowledge of µ : N → [0, 1]. Specifically, we consider the case where the mode observation instants ti , i ∈ N0 , satisfy
l=1 i,j∈M
X
(l−1)
πi pi,j
P∞ l−1 Let Z , (1 − θ)l−1 , where P ∈ RM×M l=1 P denotes the transition probability matrix for the mode signal {r(k) ∈ M}k∈N0 . Note that the infinite sum in the definition of Z converges, because the eigenvalues of the matrix (1 − θ)P are strictly inside the unit circle of the complex plane. By using the formula for geometric series of matrices (Bernstein, 2009), we obtain Z = I − (1 − θ)P )−1 . Furthermore, it follows P P P (l−1) µτ τl=1 i,j∈M πi pi,j ln ζj,i = from (40) that τ ∈N P i,j∈M πi (ln ζj,i )zi,j , and therefore, when µ : N → [0, P 1] is given by (38), condition (20) takes the form i,j∈M πi (ln ζj,i )zi,j < 0, where zi,j is the (i, j)th entry of the matrix Z.
Figure 6 shows the distribution (38) with θ = 0.3.
X
τ X X
i,j∈M
Remark 4.6. Note that our probabilistic characterization of mode observation instants also allows us to explore the feedback control problem under missing mode samples. Specifically, consider the case where the mode is sampled at all time instants; however, some of the mode samples are lost during communication between mode sampling mechanism and the controller. Suppose that the controller receives a sampled mode data at each time step k ∈ N with probability θ ∈ (0, 1). In other words, the mode data is lost with probability 1 − θ. We investigate this problem by setting µτ , (1 − θ)τ −1 θ,
µτ
(39)
Theorem 4.8. Consider the switched linear stochastic system (4). Suppose that the mode-transition probability matrix P ∈ RM×M possesses only positive real eigenvalues.
=
9
˜ > 0, Li ∈ Rm×n , i ∈ M, and If there exist matrices R scalars τ¯ ∈ N, ζi,j ∈ (0, ∞), i, j ∈ M, such that (19), (41), 0 ≤ ζj,i − ζi,i , i, j ∈ M, τ¯ X X (l−1) πi pi,j ln ζj,i < 0,
As a consequence, for all τ ≤ τ¯ it follows that
(42) τ τ¯ 1 X (l−1) 1 X (l−1) pi,j ≤ pi,j , τ τ¯
(43)
l=1 i,j∈M
l=1
hold, then the control law (6) with the feedback gain matrix (21) guarantees that the zero solution x(k) ≡ 0 of the closed-loop system is asymptotically stable almost surely.
(l)
l→∞
l→∞
τ 1X X (l−1) πi pi,j ln ζj,i τ
(44)
l=1 i,j∈M
(l) pi
∈ [0, 1]1×M ,i ∈ M, denote the row vector Now, let with the jth element given by the l-step transition probability (·) (l) pi,j . Note that pi is the unique solution of the difference equation (l+1)
pi
(l)
= pi P,
l ∈ N0 ,
=
(l)
(l+1)
≤ pi,i ,
(l+1) pi,j
(l) pi,j ,
≥
(46)
i 6= j, i, j ∈ M, l ∈ N0 .
(47)
l=1
l=1
1 τ¯
X
(l−1)
πi pi,j
ln ζj,i .
(51)
l=1 i,j∈M
i 6= j, i, j ∈ M.
(52)
(48) Now, since
l=1
X
l=1
τ X 1 (τ ) (l−1) = pi,j + pi,j τ +1
1 τ +1
i∈M j∈M,j6=i
τ¯ X
l=1
τ τ τ X 1 X (τ ) 1 1 X (l−1) (l−1) pi,j + pi,j ≤ pi,j τ τ +1 τ
=
τ¯
πi (ln ζj,i )κi,j τ,¯ τ +
i,j (ln ζj,i )κi,j τ, τ,¯ τ ≤ (ln ζi,i )κτ,¯
≤ pτi,j , l ∈ {1, 2, . . . , τ }, i, j ∈ M, By (47), we have i 6= j. Hence, it follows from (48) that
l=1 τ +1 X
l=1
Note that by (50), we have κi,j ¯, i 6= j. It follows τ,¯ τ ≤ 0, τ ≤ τ from (42) that, for τ ≤ τ¯,
(l−1) pi,j
l=1
X
+
Now note that for all i, j ∈ M, and τ ∈ N, τ τ τ X 1 X (l−1) 1 X (l−1) 1 (l−1) pi,j + pi,j = pi,j . τ τ +1 τ
τ 1 X (l−1) pi,j τ
i∈M
(0)
i ∈ M, l ∈ N0 ,
πi ln ζj,i
1X X (l−1) πi pi,j ln ζj,i τ¯ i,j∈M l=1 i,j∈M X X X i,i = πi (ln ζi,i )κτ,¯τ + πi (ln ζj,i )κi,j τ,¯ τ =
(45) (0)
X
i,j∈M
with the initial condition pi,i = 1 and pi,j = 0, i 6= j, j ∈ M. Since all the eigenvalues of the mode-transition probability matrix P ∈ RM×M are positive real numbers, (·) the solution pi of the difference equation (45) does not comprise any oscillatory components, and l-step transition (l) probabilities pi,j , i, j ∈ M, converge towards their limiting values monotonically, that is, pi,i
(50)
Next, we show that (41)–(43) together with (50) imply (20). Pτ Pτ¯ (l−1) (l−1) 1 − τ1¯ l=1 pi,j , i, j ∈ M. First, let κi,j τ,¯ τ , τ l=1 pi,j It follows that
Proof. The mode signal {r(k) ∈ M}k∈N0 is an irreducible and aperiodic Markov chain; therefore, the invariant distribution π : M → [0, 1] is also the limiting distribution (Norris, 2009). Thus, for all i, j ∈ M and k ∈ N0 , lim pi,j = lim P[r(k + l) = j|r(k) = i] = πj .
i 6= j, i, j ∈ M.
l=1
(l−1)
pi,j
,
τ ∈ N, i 6= j.
P
κi,j τ,¯ τ =
j∈M
=
(l)
j∈M
pi,j = 1, l ∈ N0 , i ∈ M, we have
τ τ¯ X 1X X 1X (l−1) (l−1) pi,j − pi,j τ τ¯
j∈M l=1 τ X X
1 τ
l=1 j∈M
τ¯ τ = − = 0, τ τ¯
(49)
l=1
10
(l−1)
pi,j
−
j∈M l=1 τ¯ X X
1 τ¯
i ∈ M.
(l−1)
pi,j
l=1 j∈M
(53)
X
(l)
πi pi,j ln ζj,i
x1 (k)
1 τ
τ X
1.0 0.5 0.0 −0.5 −1.0
x2 (k)
We use (51)–(53) to obtain
0.2 0.0 −0.2 −0.4 −0.6 −0.8 −1.0
l=1 i,j∈M
≤
X
πi (ln ζi,i )κi,j τ,¯ τ
X
+
X
πi (ln ζi,i )κi,j τ,¯ τ
i∈M j∈M,j6=i τ¯ X (l−1) pi,j l=1
i∈M
1 τ¯ i,j∈M X X i,j = πi (ln ζi,i ) κτ,¯τ X
+
πi (ln ζj,i )
i∈M
=
i,j∈M τ¯ X X
1 τ¯
1 X (l−1) pi,j τ¯
(l−1)
ln ζj,i ,
τ ≤ τ¯.
(54)
l=1 i,j∈M
Finally, it follows from (41) and (54) that X
µτ
τ ∈N
=
τ X X
ln ζj,i
X
µτ τ
X
µτ τ
τ ∈N
τ 1X X (l−1) πi pi,j ln ζj,i τ
1 τ¯
l=1 i,j∈M τ¯ X X
(l−1)
πi pi,j
l=1 i,j∈M
ln ζj,i .
˜= R (55)
Example 5.1. Consider the switched stochastic system (4) with M = 2 modes described by the subsystems matrices 0
1
1.6 −0.3
,
A2 =
30
70
40
50
60
70
0
1
−0.5 1.4
#
"
3.0143 −0.1485 −0.1485 1.5280
#
,
(56)
(57) (58)
Sample paths of the state x(k) and the control input u(k) T (obtained with initial conditions x(0) = [1, −1] and r(0) = 1) are shown in Figures 7 and 8. Furthermore, Figure 9 shows a sample path of the actual mode signal r(k) and its sampled version σ(k). Figures 7–9 indicate that our proposed control framework guarantees stabilization even for the case where operation mode of the switched system is observed only at random time instants.
In this section we provide numerical examples to demonstrate the results presented in this paper.
A1 =
20
60
guarantees almost sure asymptotic stability of the closedloop switched stochastic system (4), (6).
5 Illustrative Numerical Examples
"
10
50
˜ −1 = [−1.1465 0.5174] , K1 = L1 R ˜ −1 = [−0.9718 1.1021] , K2 = L2 R
Conditions of Theorem 4.8 can be utilized for assessing stability of a switched stochastic control system, even if exact knowledge of the distribution µ : N → [0, 1] is not available. Note that the requirement on the knowledge of µ : N → [0, 1] is relaxed in Theorem 4.8 by imposing other conditions on the mode-transition probability matrix P ∈ RM×M and the scalars ζi,j ∈ (0, ∞), i, j ∈ M.
#
0
40
L1 = [−3.5326 0.9608], L2 = [−3.0029 1.8284], and the scalars ζ1,1 = 0.7, ζ1,2 = 1.8, ζ2,1 = 2, and ζ2,2 = 0.8 satisfy (19) and (20). Now, it follows from Theorem 4.1 that the proposed control law (6) with feedback gain matrices
Note that (43) and (55) imply (20). Hence, the result follows from Theorem 4.1.
"
30
Note that
l=1 i,j∈M
τ ∈N
≤
(l−1)
πi pi,j
20
be an aperiodic and irreducible Markov chain characterized by the transition probabilities p1,2 = p2,1 = 0.3 and p1,1 = p2,2 = 0.7. The invariant distribution for {r(k) ∈ M , {1, 2}}k∈N0 is given by π1 = π2 = 0.5. Moreover, µ : N → [0, 1], according to which the lengths of intervals between consecutive mode observation instants are distributed, is assumed to be given by µτ = (1−θ)τ −1 θ, τ ∈ N, with θ = 0.3. In this case, at each time step k ∈ N, the mode may be observed with probability θ = 0.3 (see Remark 4.6).
l=1
πi pi,j
10
Figure 7. State trajectory versus time
j∈M
πi (ln ζj,i )
0
Time [k]
τ¯
X
+
Mode switching instants
The control law (6) with feedback gain matrices (57) and (58) guarantee stabilization of the closed-loop system with random mode observations characterized by distribution µτ = (1 − θ)τ −1 θ with θ = 0.3. Note that for each time step, θ represents the probability of mode information being available for control purposes. In order to investigate conservativeness of our results, we search all values of parameter θ for which the control law (6) with feedback gains (57)
,
B1 = [0, 1]T , and B2 = [0, −1]T . The mode signal {r(k) ∈ M , {1, 2}}k∈N0 of the switched system is assumed to
11
0
10
20
30
40
50
60
70
Time [k]
x1 (k)
Mode switching instants Mode observation instants
1.0 0.5 0.0 −0.5 −1.0
x2 (k)
u(k)
1.5 1.0 0.5 0.0 −0.5 −1.0 −1.5 −2.0
0.4 0.2 0.0 −0.2 −0.4 −0.6 −0.8 −1.0
Figure 8. Control input versus time
Mode switching instants
0
5
10
0
5
10
r(k)
25
15
20
25
Figure 10. State trajectory versus time 1 0
10
20
30
40
50
60
Note that for this example the mode observation instants ti , i ∈ N0 , satisfy (41) with τ¯ = 5.
70
2
σ(k)
20
Time [k]
2
In this example, we will utilize Theorem 4.8 for the case where the upper-bounding constant τ¯ = 5 is known, but the exact knowledge of the distribution µ : N → [0, 1] is not available (see Remark 4.8). Specifically, note that
1 0
10
20
30
40
50
60
70
Time [k]
Figure 9. Actual mode r(k) and sampled mode σ(k)
˜= R
and (58) achieve stabilization. To this end, first, we search values of θ such that there exist a positive-definite matrix ˜ and scalars ζi,j , i, j ∈ M that satisfy conditions (19) R, ˜ ˜ and L2 = K2 R, and (20) of Theorem 4.1 with L1 = K1 R where K1 and K2 are given by (57) and (58). We find that for parameter values θ ∈ [0.2, 1], conditions (19) and (20) are satisfied. Hence Theorem 4.1 guarantees stabilization for the case where parameter θ is inside the range [0.2, 1]. On the other hand, through repetitive numerical simulations we observe that the states of the closed-loop system converge to the origin in fact for a larger range of parameter values (θ ∈ [0.12, 1]), which indicate some conservativeness in the conditions of Theorem 4.1 (see Remark 4.2).
"
0
1
1.5 0.5
#
, A2 =
"
0 1 1 0.5
#
, A3 =
"
0 −1 1.1 1.2
#
"
2.6465 −0.7851 −0.7851 1.2568
#
,
(59)
L1 = [−3.5858 0.1413], L2 = [−4.7066 − 0.3329], L3 = [−3.2532 − 0.3601], and the scalars ζ1,1 = 0.6, ζ1,2 = 1.7, ζ1,3 = 1.5, ζ2,1 = 1.6, ζ2,2 = 0.7, ζ2,3 = 2, ζ3,1 = 2, ζ3,2 = 2, and ζ3,3 = 0.5 satisfy (19), (42), and (43). Therefore, it follows from Theorem 4.8 that the proposed control law (6) with feedback ˜ −1 = [−1.6222 − 0.9009], gain matrices K1 = L1 R −1 ˜ −1 = ˜ = [−2.2794 − 1.6888], K3 = L3 R K2 = L2 R [−1.6132 − 1.2942] , guarantees almost sure asymptotic stability of the closed-loop system (4), (6). Figures 10 and 11 respectively show sample paths of the state x(k) and the control input u(k) obtained with initial conditions x(0) = [1, −1]T and r(0) = 1. Furthermore, a sample path of the actual mode signal r(k) and its sampled version σ(k) are shown in Figure 12. As it is indicated in Figures 10– 12, the proposed control framework (6) achieves asymptotic stabilization of the zero solution. It is important to note that the feedback gains K1 , K2 , and K3 are designed by utilizing Theorem 4.8 without using information on the distribution µ : N → [0, 1]. Note that Theorem 4.8 requires only the knowledge of an upper-bounding constant τ¯ ∈ N for the length of intervals between consecutive mode observation instants, instead of the exact knowledge of µ : N → [0, 1].
Example 5.2. Consider the switched stochastic system (4) with M = 3 modes described by the subsystems matrices A1 =
15
,
B1 = [0, 1]T , B2 = [0, 0.2]T , and B3 = [0, 0.7]T . The mode signal {r(k) ∈ M , {1, 2, 3}}k∈N0 of the switched system is assumed to be an aperiodic and irreducible Markov chain characterized by the transition matrix P with entries pi,i = 0.6, i ∈ M, and pi,j = 0.2, i 6= j, i, j ∈ M. The invariant distribution for {r(k) ∈ M , {1, 2, 3}}k∈N0 is given by π1 = π2 = π3 = 13 . Furthermore, note that the transition matrix P possesses positive real eigenvalues 0.4 (with algebraic multiplicity 2) and 1. The lengths of intervals between consecutive mode observation instants are assumed to be uniformly distributed over the set {2, 3, 4, 5} (see Remark 4.5). In other words, the distribution µ : N → [0, 1] is assumed to be given by (37) with τL = 2 and τH = 5.
6 Conclusion We proposed a feedback control framework for stabilization of switched linear stochastic systems under randomly available mode information. In this problem setting, information on the active operation mode of the switched system is assumed to be available for control purposes only at random
12
u(k)
1.5 1.0 0.5 0.0 −0.5
Cassandras, C. G. and Lygeros, J. (Eds.) (2006). Stochastic Hybrid Systems. CRC Press. Boca Raton.
Mode switching instants Mode observation instants
Cetinkaya, A. and T. Hayakawa (2011). Stabilization of switched linear stochastic dynamical systems under limited mode information. In ‘Proc. IEEE Conf. Dec. Contr.’. Orlando, FL. pp. 8032–8037. 0
5
10
15
20
25
Cetinkaya, A. and T. Hayakawa (2012). Feedback control of switched stochastic systems using uniformly sampled mode information. In ‘Proc. Amer. Contr. Conf.’. Montreal, Canada. pp. 3778–3783.
Time [k]
Figure 11. Control input versus time
Cetinkaya, A. and T. Hayakawa (2013a). Discrete-time switched stochastic control systems with randomly observed operation mode. In ‘Proc. IEEE Conf. Dec. Contr.’. Firenze, Italy. pp. 85–90. Cetinkaya, A. and T. Hayakawa (2013b). Stabilizing discrete-time switched linear stochastic systems using periodically available imprecise mode information. In ‘Proc. Amer. Contr. Conf.’. Watshington, DC, USA. pp. 3266–3271.
r(k)
3 2 1 0
5
10
15
20
25
σ(k)
3
Costa, O. L. V., M. D. Fragoso and R. P. Marques (2004). Discrete-Time Markov Jump Linear Systems. Springer.
2 1 0
5
10
15
20
de Farias, D. P., J. C. Geromel, J. B. R. do Val and O. L. V. Costa (2000). ‘Output feedback control of Markov jump linear systems in continuous-time’. IEEE Trans. Autom. Contr. 45, 944–949.
25
Time [k]
Durrett, R. (2010). Probability: Theory and Examples. Cambridge University Press: New York.
Figure 12. Actual mode r(k) and sampled mode σ(k)
Fang, Y. and K. A. Loparo (2002). ‘Stabilization of continous-time jump linear systems’. IEEE Trans. Autom. Contr. 47, 1590–1602.
time instants. We presented a probabilistic analysis concerning a sequence-valued stochastic process that captures the evolution of active operation mode between mode observation instants. We then used the results of this analysis to obtain sufficient almost sure asymptotic stability conditions for the zero solution of the closed-loop system.
Geromel, J. C., A. P. C. Goncalves and A. R. Fioravanti (2009). ‘Dynamic output feedback control of discrete-time Markov jump linear systems through linear matrix inequalities’. SIAM J. Contr. Optm. 48(2), 573– 593. Ghaoui, L. E. and M. A. Rami (1996). ‘Robust state-feedback stabilization of jump linear systems via LMIs’. Int. J. Robust Nonl. Contr. 6, 1015– 1022.
References
Li, C., M. Z. Q. Chen, J. Lam and X. Mao (2012). ‘On exponential almost sure stability of random jump systems’. IEEE Trans. Autom. Contr. 57(12), 3064–3077.
Bercu, B., F. Dufour and G. G. Yin (2009). ‘Almost sure stabilization for feedback controls of regime-switching linear systems with a hidden Markov chain’. IEEE Trans. Autom. Contr. 54, 2114–2125.
Nassiri-Toussi, K. and P. E. Caines (1991). On the adaptive stabilization and ergodic behaviour of stochastic jump-Markov systems via nonlinear filtering. In ‘Proc. IEEE Conf. Dec. Contr.’. Brighton, England. pp. 1784–1785.
Bernstein, D. (2009). Matrix mathematics: Theory, Facts, and Formulas. Princeton University Press: Princeton. Bolzern, P., P. Colaneri and G. De Nicolao (2004). On almost sure stability of discrete-time Markov jump linear systems. In ‘IEEE Conf. Dec. Contr.’. Nassau, Bahamas. pp. 3204–3208.
Norris, J. (2009). Markov Chains. Cambridge University Press: New York. Sathanantan, S., O.Adetona, C. Beane and L. H. Keel (2008). ‘Feedback stabilization of Markov jump linear system with time-varying delay’. Stoc. Anal. App. 26, 577–594.
Boukas, E. K. (2006). ‘Static output feedback control for stochastic hybrid systems: LMI approach’. Automatica 42, 183–188.
Serfozo, R. (2009). Basics of Applied Stochastic Processes. SpringerVerlag: New York.
Boyle, F. A., J. Haupt, G. L. Fudge and C. C. A. Yeh (2007). Detecting signal structure from randomly-sampled data. In ‘IEEE 14th Workshop on Stat. Sign. Proc.’. IEEE. pp. 326–330. Caines, P. E. and J. F. Zhang (1992). Adaptive control for jump parameter systems via non-linear filtering. In ‘Proc. IEEE Conf. Dec. Contr.’. Tucson, AZ. pp. 699–704.
Vargas, A. N., W. Furloni and J. B. R. do Val (2006). Constrained model predictive control of jump linear systems with noise and nonobserved Markov state. In ‘Proc. Amer. Contr. Conf.’. Minneapolis, MN. pp. 929–934.
Carlen, E. and R. V. Mendes (2009). ‘Signal reconstruction by random sampling in chirp space’. Nonl. Dyn. 56(3), 223–229.
Yin, G. and C. Zhu (2010). Hybrid Switching Diffusions: Properties and Applications. Springer-Verlag: New York.
13
arXiv:1409.2578v1 [cs.SY] 9 Sep 2014
a
a,1
Department of Mechanical and Environmental Informatics, Tokyo Institute of Technology, Tokyo 152-8552, Japan
Abstract Almost sure asymptotic stabilization of a discrete-time switched stochastic system is investigated. Information on the active operation mode of the switched system is assumed to be available for control purposes only at random time instants. We propose a stabilizing feedback control framework that utilizes the information obtained through mode observations. We first consider the case where stochastic properties of mode observation instants are fully known. We obtain sufficient asymptotic stabilization conditions for the closed-loop switched stochastic system under our proposed control law. We then explore the case where exact knowledge of the stochastic properties of mode observation instants is not available. We present a set of alternative stabilization conditions for this case. The results for both cases are predicated on the analysis of a sequence-valued process that encapsulates the stochastic nature of the evolution of active operation mode between mode observation instants. Finally, we demonstrate the efficacy of our results with numerical examples. Key words: Switched stochastic systems; almost sure stabilization; random mode observations; missing mode observations; countable-state Markov processes; renewal processes
1 Introduction The framework developed for switched stochastic systems provides accurate characterization of numerous complex real life processes from physics and engineering fields that are subject to randomly occurring incidents such as sudden environmental variations or sharp dynamical changes (Cassandras and Lygeros, 2006; Yin and Zhu, 2010). Stabilization problem for switched stochastic systems has been investigated in many studies (e.g., Ghaoui and Rami (1996), de Farias et al. (2000), Fang and Loparo (2002), Costa et al. (2004), Sathanantan et al. (2008), Geromel et al. (2009) and the references therein). Control frameworks developed for switched stochastic systems often require the availability of information on the ⋆ This research was supported in part by JSPS Grant-in-Aid for Scientific Research (A) 26249062 and (C) 25420431, the Aihara Innovative Mathematical Modelling Project (JSPS) under FIRST program initiated by CSTP, and Japan Science and Technology Agency under CREST program. The material in this paper was partially presented at the 52nd IEEE Conference on Decision and Control, 2013, Firenze, Italy. Email addresses: [email protected] (Ahmet Cetinkaya), [email protected] (Tomohisa Hayakawa ). 1 Tel. : +81 3 5734 2762; Fax: +81 3 5734 2762
Preprint submitted to Automatica
active operation mode at all times. Note that for numerous applications the active mode describes the operating conditions of a physical process and is driven by external incidents of stochastic nature. The active mode, hence, may not be directly measurable and it may not be available for control purposes at all time instants during the course of operation. When the controller does not have access to any mode information, for achieving stabilization one can resort to adaptive control frameworks (Nassiri-Toussi and Caines, 1991; Caines and Zhang, 1992; Bercu et al., 2009) or mode-independent control laws (Vargas et al., 2006; Boukas, 2006). On the other hand, if mode information can be observed at certain time instants (even if rarely), this information can be utilized in the control framework. In our earlier work (Cetinkaya and Hayakawa, 2012; Cetinkaya and Hayakawa, 2013b), we investigated stabilization of switched stochastic systems for the case where only sampled mode information is available for control purposes. Under the assumption that the active mode is periodically observed, we proposed a stabilizing feedback control framework that utilizes the available mode information. In practical applications, it would be ideal if the mode information of a switched system is available for control purposes at all time instants or at least periodically. However, there are cases where mode information is obtained at random time instants. This situation occurs for example when the mode is sampled at all time instants; however, some of
September 10, 2014
setting, we present alternative sufficient stabilization conditions which can be used for verifying stability even if the distribution is not exactly known.
the mode samples are randomly lost during communication between mode sampling mechanism and the controller. On the other hand, in some applications, the mode has to be detected, but the detected mode information may not always be accurate. In this case each mode detection has a confidence level. Mode information with low confidence is discarded. As a result, depending on the confidence level of detection, the controller may or may not receive the mode information at a particular mode detection instant. In addition, we may also take advantage of random sampling for certain cases and observe the mode intentionally at random instants, as for such cases control under random sampling provides better results compared to periodic sampling. Note that random sampling has also been used for problems such as signal reconstruction and has been shown to have advantages over regular periodic sampling (see Boyle et al. (2007), Carlen and Mendes (2009)).
The paper is organized as follows. We provide the notation and a review of key results concerning renewal processes in Section 2. In Section 3, we propose our feedback control framework for stabilizing discrete-time switched stochastic systems under randomly available mode information. Then in Section 4, we present sufficient conditions under which our proposed control law guarantees almost sure asymptotic stabilization. In Section 5, we demonstrate the efficacy of our results with two illustrative numerical examples. Finally, in Section 6 we conclude our paper. 2 Mathematical Preliminaries
In this paper our goal is to explore the feedback stabilization problem for the case where the active operation mode, which is modeled as a finite-state Markov chain, is observed at random time instants. We provide an extended discussion based on our preliminary report (Cetinkaya and Hayakawa, 2013a). Specifically, we assume that the length of intervals between consecutive mode observation instants are identically distributed independent random variables. We employ a renewal process to characterize the occurrences of random mode observations. This characterization allows us to also explore periodic mode observations (Cetinkaya and Hayakawa, 2012; Cetinkaya and Hayakawa, 2013b) as a special case.
In this section, we provide notation and several definitions concerning discrete-time stochastic processes. Specifically, we denote positive and nonnegative integers by N and N0 , respectively. Moreover, R denotes the set of real numbers, Rn denotes the set of n × 1 real column vectors, and Rn×m denotes the set of n × m real matrices. We write (·)T for transpose, k · k for the Euclidean vector norm. We use λmin (H) (resp., λmax (H)) for the minimum (resp., maximum) eigenvalue of the Hermitian matrix H. A function V : Rn → R is called positive definite if V (x) > 0, x 6= 0, and V (0) = 0. We represent a finite-length sequence of ordered elements q1 , q2 , . . . , qn by q = (q1 , q2 , . . . , qn ). The length (number of elements) of the sequence q is denoted by |q|. The notations P[·] and E[·] respectively denote the probability and expectation on a probability space (Ω, F , P) with filtration {Fk }k∈N0 . Furthermore, we write 1[G] : Ω → {0, 1} for the indicator of the set G ∈ F , that is, 1[G] (ω) = 1, ω ∈ G, and 1[G] (ω) = 0, ω ∈ / G.
We propose a linear feedback control law with a piecewiseconstant gain matrix that is switched depending on the value of a randomly sampled version of the mode signal. In order to investigate the evolution of the active mode together with its randomly sampled version, we construct a stochastic process that represents sequences of values the mode takes between random mode observation instants. This sequencevalued stochastic process turns out to be a countable-state Markov chain defined over a set that is composed of all possible mode sequences of finite length. We first analyze the probabilistic dynamics of this sequence-valued Markov chain. Then based on our analysis, we obtain sufficient stabilization conditions for the closed-loop switched stochastic system under our proposed control framework. These stabilization conditions let us assess whether the closed-loop system is stable for a given probability distribution for the length of intervals between consecutive mode observation instants. As this probability distribution is not assumed to have a certain structure, the result presented in this paper can also be considered as a generalization of the result provided in Cetinkaya and Hayakawa (2011), where stabilization problem is discussed in continuous time and the random intervals between mode sampling instants are specifically assumed to be exponentially distributed. In this paper we also explore the case where perfect information regarding the probability distribution for the length of intervals between consecutive mode observation instants is not available. For this problem
2.1 Discrete-Time Renewal Processes A discrete-time renewal process {N (k) ∈ N0 }k∈N0 with initial value N (0) = 0 is an Fk -adapted stochastic counting P process defined by N (k) , i∈N 1[ti ≤k] , where ti ∈ N0 , i ∈ N0 , are random time instants such that t0 = 0 and τi , ti − ti−1 ∈ N, i ∈ N, are identically distributed independent random variables with finite expectation (i.e., E[τi ] < ∞, i ∈ N). Note that τi , i ∈ N, denote the lengths of intervals between time instants ti , i ∈ N0 . Furthermore, we use µ : N → [0, 1] to denote the common distribution of the random variables τi , i ∈ N, such that P[τi = τ ] = µτ ,
τ ∈ N,
i ∈ N,
(1)
P where µτ ∈ [0, 1]. Note that τ ∈N µτ = 1. Now, let P τˆ , τ ∈N τ µτ = E[τ1 ](= E[τi ], i ∈ N). It follows as a consequence of strong law of large numbers for renewal processes (see Serfozo (2009)) that limk→∞ Nk(k) = τ1ˆ .
2
Note that in Section 3, we employ a renewal process to characterize the occurrences of random mode observations. Figure 1. Mode transition diagram for {r(k) ∈ M , {1, 2}}k∈N0
2.2 Almost Sure Asymptotic Stability using a transition diagram, which shows possible transitions between the operation modes of the switched system. Mode transition diagram for a switched system with two modes is shown in Figure 1.
The zero solution x(k) ≡ 0 of a stochastic system is almost surely stable if, for all ǫ > 0 and ρ > 0, there exists δ = δ(ǫ, ρ) > 0 such that if kx(0)k < δ, then P[ sup kx(k)k > ǫ] < ρ.
In this paper, we assume that the mode signal is an aperiodic, irreducible Markov chain and has the invariant distribution π : M → [0, 1].
(2)
k∈N0
Furthermore, the zero solution x(k) ≡ 0 of a stochastic system is asymptotically stable almost surely if it is almost surely stable and P[ lim kx(k)k = 0] = 1.
3.1 Feedback Control Under Randomly Observed Mode Information
(3)
k→∞
In this paper, active mode of the switched stochastic system (4) is assumed to be observed only at random time instants, which we denote by ti ∈ N0 , i ∈ N0 . We assume that t0 = 0 and τi , ti − ti−1 ∈ N, i ∈ N, are independent random variables that are distributed according to a common distribution µ : N → [0, 1] for all i ∈ N such that τˆ , P τ ∈N τ µτ < ∞. In this problem setting, the initial mode information r0 is assumed to be available to the controller, and a renewal process {N (k) ∈ N0 }k∈N0 is employed for counting the number of mode observations that are obtained after the initial time. We assume that the renewal process {N (k) ∈ N0 }k∈N0 and the mode signal {r(k) ∈ M}k∈N0 are mutually independent.
In Sections 3 and 4, we investigate almost sure asymptotic stabilization of a switched stochastic system. 3 Stabilizing Switched Stochastic Systems with Randomly Available Mode Information In this section, we propose a feedback control framework for stabilizing a switched stochastic system by using only the randomly available mode information. Specifically, we consider the discrete-time switched linear stochastic system with M ∈ N number of modes given by x(k + 1) = Ar(k) x(k) + Br(k) u(k),
k ∈ N0 ,
Following our approach in Cetinkaya and Hayakawa (2011), Cetinkaya and Hayakawa (2012), Cetinkaya and Hayakawa (2013b), we employ a linear feedback control law with a ‘piecewise-constant’ feedback gain matrix that depends only on the obtained mode information. Specifically, we consider the control law
(4)
with the initial conditions x(0) = x0 , r(0) = r0 ∈ M , {1, 2, . . . , M }, where x(k) ∈ Rn and u(k) ∈ Rm respectively denote the state vector and the control input; furthermore, Ai ∈ Rn×n , Bi ∈ Rn×m , i ∈ M, are the subsystem matrices. The mode signal {r(k) ∈ M}k∈N0 is assumed to be an Fk -adapted, M -state discrete-time Markov chain with the initial distribution denoted by ν : M → [0, 1] such that νr0 = 1 and νi = 0, i 6= r0 .
u(k) = Kσ(k) x(k),
σ(k) , r(tN (k) ),
(l)
(l)
that pi,j ∈ [0, 1] is in fact the l-step transition probability from mode i to mode j, that is,
(0)
k ∈ N0 .
(7)
Note that the sampled mode signal {σ(k) ∈ M}k∈N0 acts as a switching mechanism for the linear feedback gain, which remains constant between two consecutive mode observation instants, that is, Kσ(k) = Kr(ti ) for k ∈ [ti , ti+1 ).
we use pi,j to denote (i, j)th entry of the matrix P l . Note
(l)
(6)
where {σ(k) ∈ M}k∈N0 is the sampled version of the mode signal defined by
We use the matrix P ∈ RM×M to characterize probability of transitions between the modes of the switched system. Specifically, pi,j ∈ [0, 1], which is the (i, j)th entry of the matrix P , denotes theP probability of a transition from mode i to mode j. Note that j∈M pi,j = 1, i ∈ M. Furthermore,
pi,j , P[r(k + l) = j|r(k) = i], l ∈ N0 , i, j ∈ M,
k ∈ N0 ,
Between two consecutive mode observation instants, the feedback gain Kσ(·) stays constant, whereas the active mode r(·) of the dynamical system (4) may change its value. Stabilization performance under the control law (6) hence depends not only on the length of the intervals between random mode observation instants, but also on how the active mode switches during the intervals.
(5)
(0)
with pi,i = 1, i ∈ M, pi,j = 0, i 6= j. Furthermore, (1)
pi,j = pi,j , i, j ∈ M. The mode signal can be represented
3
Figure 3. Transition diagram of the sequence– valued discrete-time countable-state Markov chain {s(i) ∈ S , {(1), (2), (1, 1), . . .}}i∈N0 over the set of mode sequences of variable length
Figure 2. Actual mode r(k) and its sampled version σ(k)
In Figure 2, we show sample paths of the active mode signal r(·) and its sampled version σ(·) for a switched stochastic system with M = 2 modes. In this example, active mode is observed at time instants t0 = 0, t1 = 2, t2 = 5, t3 = 6, t4 = 8, . . .. Note that at mode observation instants actual mode signal r(·) and its sampled version σ(·) have the same value. However, at the other time instants, sampled mode signal may differ from the actual mode, since between mode observation instants, system mode may switch.
Figure 4. Transition diagram of the sequence-valued discrete-time Markov chain {s(i) ∈ S , {(1, 1), (1, 2), (2, 1), (2, 2)}}i∈N0
closed-loop switched stochastic control system (4), (6). 3.2 Probabilistic Dynamics of Mode Sequences
In order to investigate the evolution of the active mode between consecutive mode observation instants, we construct a new stochastic process {s(i)}i∈N0 that takes values from a countable set of mode sequences of variable length. Specifically, we define {s(i)}i∈N0 by
The possible values of sequence that the stochastic process {s(i)}i∈N0 may take are characterized by the set
(8)
S , {(q1 , q2 , . . . , qτ ) : pqn ,qn+1 > 0, n ∈ {1, . . . , τ − 1}; qn ∈ M, n ∈ {1, . . . , τ }; µτ > 0}. (9)
with ti , i ∈ N0 , being the random mode observation instants. By the definition given in (8), s(i) represents the sequence of values that the active mode r(·) takes between the mode observation instants ti and ti+1 . Hence, sn (i), which denotes the nth element of the sequence s(i), represents the value of the active mode r(·) at time ti + n − 1. Furthermore, the value of the sampled mode signal σ(·) between time instants ti and ti+1 is represented by s1 (i) = r(ti ). Note that the active mode is observed and becomes available for control purposes only at time instants ti , i ∈ N0 . Thus, the controller has access only to the observed mode data σ(ti ) = r(ti ), i ∈ N0 , which correspond to the first elements of the sequences s(i), i ∈ N0 .
Note that the sequence-valued stochastic process {s(i)}i∈N0 is a discrete-time Markov chain on the countable state space represented by S, which contains all possible mode sequences for all possible lengths of intervals between consecutive mode observation instants. For example, consider the case where the switched system (4) has two modes. Furthermore, suppose that µτ > 0 for all τ ∈ N. In other words, lengths of intervals between mode observation instants may take any positive integer value. In this case, the state space S = {(1), (2), (1, 1), (1, 2), . . .} contains all finite-length mode sequences composed of elements from M = {1, 2}. See Figure 3 for the transition diagram of countable-state Markov chain {s(i) ∈ S}i∈N0 of this example.
For the sample paths of active mode signal r(·) and its sampled version σ(·) shown in Figure 2, mode sequences between mode observation instants t0 = 0, t1 = 2, t2 = 5, t3 = 6, t4 = 8, are given as s(0) = (1, 2), s(1) = (2, 1, 2), s(2) = (2), s(3) = (2, 1). The key property of the stochastic process {s(i)}i∈N0 is that, a given mode sequence s(i) indicates full information of the active mode as well as the information the controller has during the time interval between consecutive mode observation instants ti and ti+1 .
It is important to note that if the set {τ ∈ N : µτ > 0} has finite number of elements, then set S will also contain finite number of sequences. In other words, if the lengths of intervals between mode observation instants have finite number of possible values, then the number of possible sequences is also finite. For example, consider the case where the operation mode of the switched system, which takes values from the index set M = {1, 2}, is observed periodically with period 2, that is, µ2 = 1. In this case, S = {(1, 1), (1, 2), (2, 1), (2, 2)} (see Figure 4).
s(i) , r(ti ), r(ti + 1), . . . , r(ti+1 − 1) , i ∈ N0 ,
In what follows, we explain the probabilistic dynamics of the stochastic process {s(i)}i∈N0 and provide key results that we will use in Section 4 for analyzing stability of the
We now characterize the initial distribution and the statetransition probabilities of the discrete-time Markov chain
4
{s(i) ∈ S}i∈N0 as functions of the initial distribution and the state-transition probabilities of the mode signal {r(k) ∈ M}k∈N0 . Specifically, the initial distribution λ : S → [0, 1] of the Markov chain {s(i) ∈ S}i∈N0 is given by λq = P[s(0) = q] = P[t1 = |q|, r(0) = q1 , . . . , r(|q| − 1) = q|q| ] = P[t1 = |q| r(0) = q1 , . . . , r(|q| − 1) = q|q| ] · P[r(0) = q1 , . . . , r(|q| − 1) = q|q| ], q ∈ S.
by the last element of sequence q, to the mode represented by the first element of the sequence q¯. Furthermore, the exQ|¯q|−1 pression n=1 pq¯n ,¯qn+1 denotes the joint probability that the active mode takes the values denoted by the elements of the sequence q¯ until the next mode observation instant. Since the mode signal {r(k) ∈ M}k∈N0 is aperiodic and irreducible, mode sequences may start with any of the possible modes indicated by the index set M = {1, . . . , M }. Furthermore, it is possible to reach from any mode sequence to another mode sequence in a finite number of mode observations. Hence, the discrete-time Markov chain {s(i) ∈ S}i∈N0 is irreducible. In Lemma 3.1 below, we provide the invariant distribution for the countable-state discrete-time Markov chain {s(i) ∈ S}i∈N0 . Note that the distribution φ : S → [0, 1] : j 7→ φj is called invariantPdistribution of the Markov chain {s(i) ∈ S}i∈N0 if φj = i∈S φi ρi,j , j ∈ S. The invariant distribution for the case where S contains only sequences of fixed length T ∈ N is provided in Serfozo (2009). In Lemma 3.1, we consider the more general case where S may contain countably infinite number of sequences of all possible lengths.
(10)
Since the mode signal {r(k) ∈ M}k∈N0 and the mode observation counting process {N (k) ∈ N0 }k∈N0 are mutually independent, mode transitions and mode observations occur independently. Hence, t1 = τ1 is independent of r(n) for every n ∈ N0 . As a consequence, λq = P[t1 = |q|] P[r(0) = q1 , . . . , r(|q| − 1) = q|q| ] = P[t1 = |q|] P[r(0) = q1 ] |q|−1
·
Y
P[r(n) = qn+1 |r(n − 1) = qn ]
n=1
=
(
µ|q| 0,
Q|q|−1 n=1
pqn ,qn+1 ,
if q1 = r0 , q ∈ S, otherwise.
Lemma 3.1. Discrete-time Markov chain {s(i) ∈ S}i∈N0 has invariant distribution φ : S → [0, 1] : q 7→ φq given by
(11)
|q|−1
Note that s1 (0), which is the first element of the first mode sequence s(0), is equal to the initial mode r0 .
φq , πq1 µ|q|
r(ti ) = q1 , . . . , r(ti + |q| − 1) = q|q| ,
(12)
|q|−1
X
φq ρq,¯q =
n=1
q∈S
|¯ q|−1
Y
pqn ,qn+1 pq|q| ,¯q1
pq¯n ,¯qn+1 ,
q¯ ∈ S.
(15)
Now let Sτ , {q ∈ S : |q| = τ }, τ ∈ N. Note that the set Sτ contains all mode sequences of length τ . We rewrite the sum in (15) to obtain
P[r(ti+1 + n) = q¯n+1 |r(ti+1 + n − 1) = q¯n ]
|q|−1
X
|¯ q|−1
i ∈ N0 .
Y
n=1
n=1
pq¯n ,¯qn+1 ,
πq1 µ|q|
· µ|¯q|
|¯ q|−1
Y
X
q∈S
= P[r(ti+1 ) = q¯1 | r(ti + |q| − 1) = q|q| ]P[τi+1 = |¯ q |]
= pq|q| ,¯q1 µ|¯q|
(14)
Proof. We prove this result by showing that φq¯ = P ¯ ∈ S. First, by (13) and (14) q , for all q q∈S φq ρq,¯
ρq,¯q = P τi+1 = |¯ q |, r(ti+1 ) = q¯1 , . . . , r(ti+1 + |¯ q | − 1) = q¯|¯q| r(ti + |q| − 1) = q|q| Y
q ∈ S,
where π : M → [0, 1] and pi,j , i, j ∈ M, respectively denote the invariant distribution and transition probabilities of the finite-state Markov chain {r(k) ∈ M}k∈N0 .
for i ∈ N0 . Note that τi+1 is independent of the random variables r(n), n ∈ N0 , and τi . Furthermore, given r(ti +τi −1), the random variable r(ti+1 ) is conditionally independent of r(ti ), . . . , r(ti + τi − 2), and τi . It follows that
·
pqn ,qn+1 ,
n=1
Probability of a transition from a mode sequence q ∈ S to another mode sequence q¯ ∈ S is given by ρq,¯q = P[s(i + 1) = q¯|s(i) = q], = P τi+1 = |¯ q |, r(ti+1 ) = q¯1 , . . . , r(ti+1 + |¯ q | − 1) = q¯|¯q| τi = |q|,
Y
πq1 µ|q|
n=1
q∈S
(13)
n=1
=
X
µτ
X
µτ
τ ∈N
Note that µ|¯q | in (13) represents the probability that length of the interval between two mode observation instants is equal to the length of the sequence q¯, whereas pq|q| ,¯q1 ∈ [0, 1] represents the transition probability from the mode represented
=
τ ∈N
5
Y
X
pqn ,qn+1 pq|q| ,¯q1
πq1
n=1
q∈Sτ
X
qτ ∈M
τY −1
···
X
pqn ,qn+1 pqτ ,¯q1
q1 ∈M
πq1
τY −1
n=1
pqn ,qn+1 pqτ ,¯q1 .
(16)
Note that since π : M → [0, 1] is the invariant distribution of the finite-state Markov chain {r(k) ∈ M}k∈N0 , it P follows that π p i i,j = πj , i, j ∈ M. Thus, we have i∈M P = πqn+1 , n ∈ {1, . . . , τ − 1}, and p π Pqn ∈M qn qn ,qn+1 q1 = πq¯1 . As a result, from (16) we obtain qτ ∈M πqτ pqτ ,¯
then the control law (6) with the feedback gain matrix ˜ −1 , Kσ(k) = Lσ(k) R
guarantees that the zero solution x(k) ≡ 0 of the closed-loop system (4) and (6) is asymptotically stable almost surely.
|q|−1
X
πq1 µ|q|
Y
n=1
q∈S
X pqn ,qn+1 pq|q| ,¯q1 = µτ πq¯1
˜ −1 . It Proof. First, we define V (x) , xT Rx, where R , R follows from (4) and (6) that for k ∈ N0 ,
τ ∈N
= πq¯1 .
(17)
Finally, substituting (17) into (15) yields
V (x(k + 1)) = xT (k)(Ar(k) + Br(k) Kσ(k) )T R · (Ar(k) + Br(k) Kσ(k) )x(k).
|¯ q|−1
X
φq ρq,¯q = πq¯1 µ|¯q |
Y
pq¯n ,¯qn+1 = φq¯,
q¯ ∈ S,
(22)
(18) We set Lj = Kj R−1 , j ∈ M, and use (19) and (22) to obtain
n=1
q∈S
(21)
which completes the proof. V (x(k + 1)) ≤ ζr(k),σ(k) V (x(k)) ≤ η(k)V (x(0)), We have now established that the countable-state Markov chain {s(k) ∈ S}k∈N0 is irreducible and has the invariant distribution φ : S → [0, 1] presented in Lemma 3.1. Note that the strong law of large numbers (also called ergodic theorem; see Norris (2009), Serfozo (2009), Durrett (2010)) for discrete-time P Markov chains states that Pn−1 P[limn→∞ n1 k=0 ξs(k) P= i∈S φi ξi ] = 1, for any ξi ∈ R, i ∈ S, such that i∈S φi |ξi | < ∞. This result for the countable-state Markov chain {s(k) ∈ S}k∈N0 is crucial to obtain the main results of Section 4 below. Specifically, in our stability analysis we utilize the ergodic theorem for Markov chains. In the literature, for the stability analysis of finite-mode (Bolzern et al., 2004) and infinite-mode (Li et al., 2012) discrete-time switched stochastic systems, researchers employed ergodic theorem for the Markov chain that characterizes the mode signal. In the next section, we use ergodic theorem for the Markov chain that characterizes the sequence of mode values between consecutive mode observation instants.
Qk for k ∈ N0 , where η(k) , n=0 ζr(n),σ(n) , k ∈ N. We will first show that η(k) → 0 almost surely as k → ∞. Note that η(k) > 0, k ∈ N0 . Then, it follows that ln η(k) =
τ ∈N
µτ
(l−1)
πi pi,j
i, j ∈ M,
ln ζj,i < 0,
(24)
tN (k) −1
ln η(k) =
X
ln ζr(n),σ(n) +
X
ξs(i) +
N (k)−1
=
i=0
where ξq ,
P|q|
k X
ln ζr(n),σ(n)
n=tN (k)
n=0
n=1
k X
ln ζr(n),σ(n) ,
(25)
n=tN (k)
ln ζqn ,q1 , q ∈ S.
Next, in order to evaluate limk→∞ k1 ln η(k), note that Pk limk→∞ k1 n=tN (k) ln ζr(n),σ(n) = 0. Consequently, N (k)−1 1 X 1 ξs(i) ln η(k) = lim k→∞ k k→∞ k i=0
lim
Theorem 4.1. Consider the switched linear stochastic sys˜ > 0, Li ∈ Rm×n , i ∈ M, tem (4). If there exist matrices R and scalars ζi,j ∈ (0, ∞), i, j ∈ M, such that
τ X X
ln ζr(n),σ(n) .
By using the definitions of stochastic processes {N (k) ∈ N0 }k∈N0 and {s(i) ∈ S}i∈N0 , we obtain
In this section, we employ the results presented in Section 3 to obtain sufficient conditions for almost sure asymptotic stabilization of the closed-loop system (4) under the control law (6).
X
k X
n=0
4 Sufficient Conditions for Almost Sure Asymptotic Stabilization
˜ + Bi Lj )T R ˜ −1 0 ≥ (Ai R ˜ ˜ + Bi Lj ) − ζi,j R, · (Ai R
(23)
N (k)−1 X N (k) 1 = lim ξs(i) . k→∞ k N (k) i=0
(19)
(26)
It follows from strong law of large numbers for reN (k) 1 newal processes P (Section 2.1) that limk→∞ k = τˆ , where τˆ = τ ∈N τ µτ . Furthermore, by the ergodic theorem for countable-state Markov chains, it follows that
(20)
l=1 i,j∈M
6
Pn−1 P limn→∞ n1 i=0 ξs(i) = q∈S φq ξq . Using the invariant distribution φ : S → [0, 1] given by (14), we get 1 ln η(k) k→∞ k |q| |q|−1 Y X 1X = πq1 µ|q| ln ζqm ,q1 . pqn ,qn+1 τˆ m=1 n=1
such that P[supk≥n η(k) > ǫ2 ] < ρ for n ≥ N (ǫ, ρ). Equivalently, P[sup η(k − 1) > ǫ2 ] < ρ,
lim
Let Sτ , {q ∈ S : |q| = τ }, τ ∈ N. Note that Sτ contains all mode sequences of length τ . It follows from (27) that
P[sup kx(k)k > ǫ
1 ln η(k) k→∞ k |q|−1 |q| Y X 1XX = πq1 µ|q| pqn ,qn+1 ln ζqm ,q1 τˆ n=1 m=1 lim
k≥n
=
q∈Sτ
λmin (R) kx(k)k2 > ǫ2 ] 2 k≥n λmax (R) kx(0)k
≤ P[sup η(k − 1) > ǫ2 ] < ρ,
Let δ1 ,
q∈Sτ
τY −1
πq1 (
=
X
q∈Sτ
=
τ,l
πi (ln ζj,i )
X
πi (ln ζj,i )pi,j
pqn ,qn+1 ) ln ζql ,q1
i,j n=1 q∈Sτ,l
(l−1)
.
s
λmax (R) kx(0)k] λmin (R)
n ≥ N (ǫ, ρ) + 1.
(33)
Now let ζ¯ , max{1, maxi,j∈M ζi,j }. It follows from (23) that V (x(k)) ≤ ζ¯k−1 V (x(0)) ≤ ζ¯N (ǫ,ρ)−1 V (x(0)) for all k ∈ {0, 1, . . . , N (ǫ, ρ)}. Therefore, kx(k)k2 ≤ (R) ζ¯N (ǫ,ρ)−1 λλmax kx(0)k2 , and hence, we have kx(k)k ≤ min (R) q (R) ζ¯N (ǫ,ρ)−1 λλmax kx(0)k, for all k ∈ {0, 1, . . . , N (ǫ, ρ)}. min (R) q min (R) Furthermore, let δ2 , ǫ ζ¯−N (ǫ,ρ)+1 λλmax (R) . Consequently, if kx(0)k ≤ δ2 , then kx(k)k ≤ ǫ, k ∈ {0, 1, . . . , N (ǫ, ρ)}, which implies
−1 X τY ( pqn ,qn+1 )
X
i,j∈M
=
k≥n
n=1
i,j∈M q∈S i,j
If kx(0)k ≤ δ1 , then
≤ P[sup kx(k)k > ǫ
pqn ,qn+1 ) ln ζql ,q1 τY −1
λmin (R) λmax (R) .
k≥n
n=1
πq1 (
q
P[sup kx(k)k > ǫ]
< ρ,
X
n ≥ N (ǫ, ρ) + 1. (32)
k≥n
i,j Furthermore, let Sτ,l , {q ∈ Sτ : q1 = i, ql = j}, i, j ∈ i,j M, l ∈ {1, 2, . . . , τ − 1}. The set Sτ,l contains all mode sequences of length τ that have i ∈ M and j ∈ M as the 1st and the lth elements, respectively. We use (5) to obtain
X
λmax (R) kx(0)k2 ] λmin (R)
= P[sup
τ X
τ τY −1 X X 1X µτ pqn ,qn+1 ) ln ζqm ,q1 . (28) πq1 ( τˆ m=1 n=1 τ ∈N
λmax (R) kx(0)k] λmin (R)
k≥n
X 1X µτ ln ζqm ,q1 pqn ,qn+1 ) πq1 ( τˆ m=1 n=1 τ ∈N
s
= P[sup kx(k)k2 > ǫ2
τ ∈N q∈Sτ
=
(31)
By the definition of V (·) and (23), we obtain η(k − 1) ≥ V (x(k)) λmin (R) kx(k)k2 V (x(0)) ≥ λmax (R) kx(0)k2 for all k ∈ N. Hence, it follows from (31) that, for all ǫ > 0 and ρ > 0, there exists a positive integer N (ǫ, ρ) such that
(27)
q∈S
τY −1
n ≥ N (ǫ, ρ) + 1.
k≥n
(29)
i,j∈M
max
P[ Substituting (29) into (28) yields
k∈{0,1,...,N (ǫ,ρ)}
τ 1X X X 1 (l−1) ln η(k) = µτ πi pi,j ln ζj,i . (30) k→∞ k τˆ
(34)
It follows from (33) and (34) that for all ǫ > 0, ρ > 0,
lim
τ ∈N
kx(k)k > ǫ] = 0.
l=1 i,j∈M
P[ sup kx(k)k > ǫ] = P[{ k∈N0
P
Now, since τˆ = τ ∈N τ µτ < ∞, as a result of (20), we have limk→∞ k1 ln η(k) < 0. Thus, limk→∞ ln η(k) = −∞ almost surely; furthermore, P[limk→∞ η(k) = 0] = 1. In the following, we first show that the zero solution is almost surely stable. To this end first note that for all ǫ > 0, limn→∞ P[supk≥n η(k) > ǫ2 ] = 0, which implies that for all ǫ > 0 and ρ > 0, there exists a positive integer N (ǫ, ρ)
max
k∈{0,1,...,N (ǫ,ρ)}
∪{
sup
kx(k)k > ǫ}
kx(k)k > ǫ}]
k≥N (ǫ,ρ)+1
≤ P[
max
k∈{0,1,...,N (ǫ,ρ)}
+ P[
sup
kx(k)k > ǫ]
kx(k)k > ǫ]
k≥N (ǫ,ρ)+1
< ρ,
7
(35)
whenever kx(0)k < δ , min(δ1 , δ2 ), which implies almost sure stability. As a final step of proving almost sure asymptotic stability of the zero solution, we now show (3). First, note that by (23), we have V (x(k + 1)) ≤ η(k)V (x(0)), k ∈ N. Now, since P[limk→∞ η(k) = 0] = 1, it follows that P[limk→∞ V (x(k)) = 0] = 1, which implies (3), and hence the zero solution of the closed-loop system (4), (6) is asymptotically stable almost surely.
i, j ∈ M, that satisfy (20) and at each iteration we look for feasible solutions to the linear matrix inequalities (36). In Section 5 below, we employ this method and find values ˜ ∈ Rn×n , Li ∈ Rm×n , i ∈ M, and scalars for matrices R ζi,j ∈ (0, ∞), i, j ∈ M, that satisfy (19), (20) for a given discrete-time switched linear system. It is important to note that the scalars ζi,j ∈ (0, ∞), i, j ∈ M, that satisfy (20) form an unbounded set. Note that this2 set is smaller than the entire nonnegative orthant in RM . However, we still need to reduce the search space of ζi,j , i, j ∈ M. To this end, first note that it is harder to find feasible solutions to linear matrix inequalities given by (36) when the scalars ζi,j , i, j ∈ M, are close to zero. Note also that if there exist a feasible solution to (36) for certain values of ζi,j , i, j ∈ M, then it is guaranteed that feasible solutions to (36) exist also for larger values of ζi,j , i, j ∈ M. Therefore, we can restrict our search space and iterate over large values of ζi,j , i, j ∈ M, that satisfy (20), and check feasible solutions to (36). Specifically, we only iterate over ζi,j , i, j ∈ M, that is close to the search space’s boundary identified by Pτ P P (l−1) ln ζj,i = 0. Now note that l=1 i,j∈M πi pi,j τ ∈N µτ in order for (20) to be satisfied, there must exist at least a pair i, j ∈ M such that ζi,j < 1. Since the scalar ζi,j represents the stability/instability margin for the dynamics characterized by the ith subsystem and the jth feedback gain, we expect ζi,i < 1 for stabilizable modes i ∈ M. This further reduces the search space for our numerical method.
Theorem 4.1 provides sufficient conditions for almost sure asymptotic stability of the closed-loop system (4) and (6). Conditions (19) and (20) of Theorem 4.1 indicate dependence of stabilization performance on subsystem dynamics, mode transition probabilities, and random mode observations. The effect of mode transitions on the stabilization is reflected in (19) through the limiting distribution π : M → [0, 1] as well as l-step transition probabilities (l) pi,j , i, j ∈ M. Furthermore, the effect of random mode observations is indicated in condition (19) by µ : N → [0, 1], which represents the distribution of the lengths of intervals between consecutive mode observation instants. Remark 4.2. We investigate the stability of the closed-loop system through the Lyapunov-like function V (x) , xT Rx ˜ −1 , where R ˜ is a positive-definite matrix that with R = R satisfy (19). The scalar ζi,j ∈ (0, ∞) in (19) characterizes an upper bound on the growth of the Lyapunov-like function, when the switched system evolves according to dynamics of the ith subsystem and the jth feedback gain. Note that if ζi,j ∈ (0, 1) for all i, j ∈ M, it is guaranteed that the Lyapunov-like function will decrease at each time step. However, we do not require ζi,j ∈ (0, 1) for all i, j ∈ M. There may be pairs i, j ∈ M such that ζi,j > 1, hence Lyapunov-like function V (·) may grow when ith subsystem and the jth feedback gain is active. As long as ζi,j , i, j ∈ M, satisfy (20) the Lyapunov-like is guaranteed to converge to zero in the long-run (even if it may grow at certain instants). Note that even though the conditions (19), (20) allow unstable subsystem-feedback gain pairs, some conservativeness may still arise due the characterization with single Lyapunov-like function. This conservatism may be reduced with an alternative approach with multiple Lyapunov-like functions assigned for each subsystem-feedback gain pairs.
Remark 4.4. Note that conditions (19) and (20) presented in Theorem 4.1 can also be used for determining almost sure asymptotic stability of the switched stochastic control system (4), (6) with periodically observed mode information. The renewal process characterization presented in this paper in fact encompasses periodic mode observations (explored previously in Cetinkaya and Hayakawa (2012) and Cetinkaya and Hayakawa (2013b)) as a special case. Specifically, suppose that the mode observation instants are given by ti = iT , i ∈ N0 , where T ∈ N denotes the mode observation period. Our present framework allows us to characterize periodic mode observations by setting the distribution µ : N → [0, 1] such that µT = 1 and µτ = 0, τ 6= T . Note that condition (20) of Theorem 4.1 for this PT P (l−1) case reduces to l=1 i,j∈M πi pi,j ln ζj,i < 0. Furthermore, if the controller has perfect mode information at all time instants (T = 1, hence σ(k) = r(k), k ∈ N0 ), condition (20) takes even a simpler form given by the inequality P i∈M πi ln ζi,i < 0.
Remark 4.3. In order to verify conditions (19) and (20) of Theorem 4.1, we take an approach similar to the one presented in Cetinkaya and Hayakawa (2013b). Specifically, we use Schur complements (see Bernstein (2009)) to transform condition (19) into the matrix inequalities "
˜ AˆT ζi,j R i,j 0≤ ˜ Aˆi,j R
#
,
i, j ∈ M,
Remark 4.5. Condition (20) of Theorem 4.1 has a simpler form also for the case where the length of intervals between consecutive mode observation instants are uniformly distributed over the set {τL , τL + 1, . . . , τH } with τL , τH ∈ N such that τL ≤ τH . In this case the distribution µ : N → [0, 1] is given by
(36)
˜ + Bi Lj ), i, j ∈ M. Note that the inwhere Aˆi,j , (Ai R ˜ and Li , i ∈ M. In our nuequalities (36) are linear in R merical method, we iterate over a set of the values of ζi,j ,
µτ ,
8
(
1 τH −τL +1 ,
0,
if τ ∈ {τL , τL + 1, . . . , τH }, otherwise.
(37)
Figure 5. Uniform distribution given by (37) with τL = 2 and τH = 5 for the length of intervals between consecutive mode observation instants
Figure 6. Distribution given by (38) with θ = 0.3 for the length of intervals between consecutive mode observation instants
(1 − θ)l−1 . Therefore,
Figure 5 shows the distribution (37) for an example case with τL = 2 and τH = 5.
X
τ ∈N
With (37), condition (20) of Theorem 4.1 reduces to the Pτ Pτ P (l−1) inequality τH=τL l=1 i,j∈M πi pi,j ln ζj,i < 0.
=
τ ∈ N.
(38)
It turns out that for µτ : N → [0, 1] given by (38), the lefthand side of condition (20) has a closed-form expression. Note that by changing the order of summations and using (38), we can rewrite the left-hand side of (20) as
τ ∈N
= =
µτ
τ X X
(l−1)
πi pi,j
πi (ln ζj,i )
i,j∈M
τ ∈N
X
∞ X
πi (ln ζj,i )
i,j∈M
=
ln ζj,i
µτ
τ X
X
πi (ln ζj,i )
X
πi (ln ζj,i )
∞ X
(l−1)
pi,j
(l−1) pi,j
∞ X
µτ
−
i,j∈M
Note that 1 −
(l−1)
pi,j
1−
µτ )
τ −1 θ τ =1 (1 − θ)
l−1 X
(1 − θ)τ −1 θ .
τ =1
l=1
Pl−1
l−1 X
τ =1
l=1
∞ X
πi (ln ζj,i )
l−1
= 1 − θ 1−(1−θ) 1−(1−θ)
∞ X
(l−1)
pi,j
(1 − θ)l−1 .
(40)
l=1
i ∈ N0 ,
(41)
where τ¯ ∈ N is a known constant. In this case time instants of consecutive mode observations are assumed to be at most τ¯ ∈ N steps apart. In other words, if (41) is satisfied, it is guaranteed that the length of intervals between consecutive mode observation instants cannot be larger than τ¯ ∈ N. It is important to note that (41) characterizes a requirement on the intervals between mode observation instants and it is not related to mode switches.
τ =l
(l−1) pi,j (1
X
P[ti+1 − ti ≤ τ¯] = 1,
l=1
l=1
i,j∈M
=
X
ln ζj,i
l=1 i,j∈M
Remark 4.7. Note that in order to check condition (20) of Theorem 4.1, one needs to have perfect information regarding the distribution µ : N → [0, 1], according to which the lengths of intervals between consecutive mode observation instants are distributed. In Theorem 4.8 below, we present alternative sufficient stabilization conditions, which do not require exact knowledge of µ : N → [0, 1]. Specifically, we consider the case where the mode observation instants ti , i ∈ N0 , satisfy
l=1 i,j∈M
X
(l−1)
πi pi,j
P∞ l−1 Let Z , (1 − θ)l−1 , where P ∈ RM×M l=1 P denotes the transition probability matrix for the mode signal {r(k) ∈ M}k∈N0 . Note that the infinite sum in the definition of Z converges, because the eigenvalues of the matrix (1 − θ)P are strictly inside the unit circle of the complex plane. By using the formula for geometric series of matrices (Bernstein, 2009), we obtain Z = I − (1 − θ)P )−1 . Furthermore, it follows P P P (l−1) µτ τl=1 i,j∈M πi pi,j ln ζj,i = from (40) that τ ∈N P i,j∈M πi (ln ζj,i )zi,j , and therefore, when µ : N → [0, P 1] is given by (38), condition (20) takes the form i,j∈M πi (ln ζj,i )zi,j < 0, where zi,j is the (i, j)th entry of the matrix Z.
Figure 6 shows the distribution (38) with θ = 0.3.
X
τ X X
i,j∈M
Remark 4.6. Note that our probabilistic characterization of mode observation instants also allows us to explore the feedback control problem under missing mode samples. Specifically, consider the case where the mode is sampled at all time instants; however, some of the mode samples are lost during communication between mode sampling mechanism and the controller. Suppose that the controller receives a sampled mode data at each time step k ∈ N with probability θ ∈ (0, 1). In other words, the mode data is lost with probability 1 − θ. We investigate this problem by setting µτ , (1 − θ)τ −1 θ,
µτ
(39)
Theorem 4.8. Consider the switched linear stochastic system (4). Suppose that the mode-transition probability matrix P ∈ RM×M possesses only positive real eigenvalues.
=
9
˜ > 0, Li ∈ Rm×n , i ∈ M, and If there exist matrices R scalars τ¯ ∈ N, ζi,j ∈ (0, ∞), i, j ∈ M, such that (19), (41), 0 ≤ ζj,i − ζi,i , i, j ∈ M, τ¯ X X (l−1) πi pi,j ln ζj,i < 0,
As a consequence, for all τ ≤ τ¯ it follows that
(42) τ τ¯ 1 X (l−1) 1 X (l−1) pi,j ≤ pi,j , τ τ¯
(43)
l=1 i,j∈M
l=1
hold, then the control law (6) with the feedback gain matrix (21) guarantees that the zero solution x(k) ≡ 0 of the closed-loop system is asymptotically stable almost surely.
(l)
l→∞
l→∞
τ 1X X (l−1) πi pi,j ln ζj,i τ
(44)
l=1 i,j∈M
(l) pi
∈ [0, 1]1×M ,i ∈ M, denote the row vector Now, let with the jth element given by the l-step transition probability (·) (l) pi,j . Note that pi is the unique solution of the difference equation (l+1)
pi
(l)
= pi P,
l ∈ N0 ,
=
(l)
(l+1)
≤ pi,i ,
(l+1) pi,j
(l) pi,j ,
≥
(46)
i 6= j, i, j ∈ M, l ∈ N0 .
(47)
l=1
l=1
1 τ¯
X
(l−1)
πi pi,j
ln ζj,i .
(51)
l=1 i,j∈M
i 6= j, i, j ∈ M.
(52)
(48) Now, since
l=1
X
l=1
τ X 1 (τ ) (l−1) = pi,j + pi,j τ +1
1 τ +1
i∈M j∈M,j6=i
τ¯ X
l=1
τ τ τ X 1 X (τ ) 1 1 X (l−1) (l−1) pi,j + pi,j ≤ pi,j τ τ +1 τ
=
τ¯
πi (ln ζj,i )κi,j τ,¯ τ +
i,j (ln ζj,i )κi,j τ, τ,¯ τ ≤ (ln ζi,i )κτ,¯
≤ pτi,j , l ∈ {1, 2, . . . , τ }, i, j ∈ M, By (47), we have i 6= j. Hence, it follows from (48) that
l=1 τ +1 X
l=1
Note that by (50), we have κi,j ¯, i 6= j. It follows τ,¯ τ ≤ 0, τ ≤ τ from (42) that, for τ ≤ τ¯,
(l−1) pi,j
l=1
X
+
Now note that for all i, j ∈ M, and τ ∈ N, τ τ τ X 1 X (l−1) 1 X (l−1) 1 (l−1) pi,j + pi,j = pi,j . τ τ +1 τ
τ 1 X (l−1) pi,j τ
i∈M
(0)
i ∈ M, l ∈ N0 ,
πi ln ζj,i
1X X (l−1) πi pi,j ln ζj,i τ¯ i,j∈M l=1 i,j∈M X X X i,i = πi (ln ζi,i )κτ,¯τ + πi (ln ζj,i )κi,j τ,¯ τ =
(45) (0)
X
i,j∈M
with the initial condition pi,i = 1 and pi,j = 0, i 6= j, j ∈ M. Since all the eigenvalues of the mode-transition probability matrix P ∈ RM×M are positive real numbers, (·) the solution pi of the difference equation (45) does not comprise any oscillatory components, and l-step transition (l) probabilities pi,j , i, j ∈ M, converge towards their limiting values monotonically, that is, pi,i
(50)
Next, we show that (41)–(43) together with (50) imply (20). Pτ Pτ¯ (l−1) (l−1) 1 − τ1¯ l=1 pi,j , i, j ∈ M. First, let κi,j τ,¯ τ , τ l=1 pi,j It follows that
Proof. The mode signal {r(k) ∈ M}k∈N0 is an irreducible and aperiodic Markov chain; therefore, the invariant distribution π : M → [0, 1] is also the limiting distribution (Norris, 2009). Thus, for all i, j ∈ M and k ∈ N0 , lim pi,j = lim P[r(k + l) = j|r(k) = i] = πj .
i 6= j, i, j ∈ M.
l=1
(l−1)
pi,j
,
τ ∈ N, i 6= j.
P
κi,j τ,¯ τ =
j∈M
=
(l)
j∈M
pi,j = 1, l ∈ N0 , i ∈ M, we have
τ τ¯ X 1X X 1X (l−1) (l−1) pi,j − pi,j τ τ¯
j∈M l=1 τ X X
1 τ
l=1 j∈M
τ¯ τ = − = 0, τ τ¯
(49)
l=1
10
(l−1)
pi,j
−
j∈M l=1 τ¯ X X
1 τ¯
i ∈ M.
(l−1)
pi,j
l=1 j∈M
(53)
X
(l)
πi pi,j ln ζj,i
x1 (k)
1 τ
τ X
1.0 0.5 0.0 −0.5 −1.0
x2 (k)
We use (51)–(53) to obtain
0.2 0.0 −0.2 −0.4 −0.6 −0.8 −1.0
l=1 i,j∈M
≤
X
πi (ln ζi,i )κi,j τ,¯ τ
X
+
X
πi (ln ζi,i )κi,j τ,¯ τ
i∈M j∈M,j6=i τ¯ X (l−1) pi,j l=1
i∈M
1 τ¯ i,j∈M X X i,j = πi (ln ζi,i ) κτ,¯τ X
+
πi (ln ζj,i )
i∈M
=
i,j∈M τ¯ X X
1 τ¯
1 X (l−1) pi,j τ¯
(l−1)
ln ζj,i ,
τ ≤ τ¯.
(54)
l=1 i,j∈M
Finally, it follows from (41) and (54) that X
µτ
τ ∈N
=
τ X X
ln ζj,i
X
µτ τ
X
µτ τ
τ ∈N
τ 1X X (l−1) πi pi,j ln ζj,i τ
1 τ¯
l=1 i,j∈M τ¯ X X
(l−1)
πi pi,j
l=1 i,j∈M
ln ζj,i .
˜= R (55)
Example 5.1. Consider the switched stochastic system (4) with M = 2 modes described by the subsystems matrices 0
1
1.6 −0.3
,
A2 =
30
70
40
50
60
70
0
1
−0.5 1.4
#
"
3.0143 −0.1485 −0.1485 1.5280
#
,
(56)
(57) (58)
Sample paths of the state x(k) and the control input u(k) T (obtained with initial conditions x(0) = [1, −1] and r(0) = 1) are shown in Figures 7 and 8. Furthermore, Figure 9 shows a sample path of the actual mode signal r(k) and its sampled version σ(k). Figures 7–9 indicate that our proposed control framework guarantees stabilization even for the case where operation mode of the switched system is observed only at random time instants.
In this section we provide numerical examples to demonstrate the results presented in this paper.
A1 =
20
60
guarantees almost sure asymptotic stability of the closedloop switched stochastic system (4), (6).
5 Illustrative Numerical Examples
"
10
50
˜ −1 = [−1.1465 0.5174] , K1 = L1 R ˜ −1 = [−0.9718 1.1021] , K2 = L2 R
Conditions of Theorem 4.8 can be utilized for assessing stability of a switched stochastic control system, even if exact knowledge of the distribution µ : N → [0, 1] is not available. Note that the requirement on the knowledge of µ : N → [0, 1] is relaxed in Theorem 4.8 by imposing other conditions on the mode-transition probability matrix P ∈ RM×M and the scalars ζi,j ∈ (0, ∞), i, j ∈ M.
#
0
40
L1 = [−3.5326 0.9608], L2 = [−3.0029 1.8284], and the scalars ζ1,1 = 0.7, ζ1,2 = 1.8, ζ2,1 = 2, and ζ2,2 = 0.8 satisfy (19) and (20). Now, it follows from Theorem 4.1 that the proposed control law (6) with feedback gain matrices
Note that (43) and (55) imply (20). Hence, the result follows from Theorem 4.1.
"
30
Note that
l=1 i,j∈M
τ ∈N
≤
(l−1)
πi pi,j
20
be an aperiodic and irreducible Markov chain characterized by the transition probabilities p1,2 = p2,1 = 0.3 and p1,1 = p2,2 = 0.7. The invariant distribution for {r(k) ∈ M , {1, 2}}k∈N0 is given by π1 = π2 = 0.5. Moreover, µ : N → [0, 1], according to which the lengths of intervals between consecutive mode observation instants are distributed, is assumed to be given by µτ = (1−θ)τ −1 θ, τ ∈ N, with θ = 0.3. In this case, at each time step k ∈ N, the mode may be observed with probability θ = 0.3 (see Remark 4.6).
l=1
πi pi,j
10
Figure 7. State trajectory versus time
j∈M
πi (ln ζj,i )
0
Time [k]
τ¯
X
+
Mode switching instants
The control law (6) with feedback gain matrices (57) and (58) guarantee stabilization of the closed-loop system with random mode observations characterized by distribution µτ = (1 − θ)τ −1 θ with θ = 0.3. Note that for each time step, θ represents the probability of mode information being available for control purposes. In order to investigate conservativeness of our results, we search all values of parameter θ for which the control law (6) with feedback gains (57)
,
B1 = [0, 1]T , and B2 = [0, −1]T . The mode signal {r(k) ∈ M , {1, 2}}k∈N0 of the switched system is assumed to
11
0
10
20
30
40
50
60
70
Time [k]
x1 (k)
Mode switching instants Mode observation instants
1.0 0.5 0.0 −0.5 −1.0
x2 (k)
u(k)
1.5 1.0 0.5 0.0 −0.5 −1.0 −1.5 −2.0
0.4 0.2 0.0 −0.2 −0.4 −0.6 −0.8 −1.0
Figure 8. Control input versus time
Mode switching instants
0
5
10
0
5
10
r(k)
25
15
20
25
Figure 10. State trajectory versus time 1 0
10
20
30
40
50
60
Note that for this example the mode observation instants ti , i ∈ N0 , satisfy (41) with τ¯ = 5.
70
2
σ(k)
20
Time [k]
2
In this example, we will utilize Theorem 4.8 for the case where the upper-bounding constant τ¯ = 5 is known, but the exact knowledge of the distribution µ : N → [0, 1] is not available (see Remark 4.8). Specifically, note that
1 0
10
20
30
40
50
60
70
Time [k]
Figure 9. Actual mode r(k) and sampled mode σ(k)
˜= R
and (58) achieve stabilization. To this end, first, we search values of θ such that there exist a positive-definite matrix ˜ and scalars ζi,j , i, j ∈ M that satisfy conditions (19) R, ˜ ˜ and L2 = K2 R, and (20) of Theorem 4.1 with L1 = K1 R where K1 and K2 are given by (57) and (58). We find that for parameter values θ ∈ [0.2, 1], conditions (19) and (20) are satisfied. Hence Theorem 4.1 guarantees stabilization for the case where parameter θ is inside the range [0.2, 1]. On the other hand, through repetitive numerical simulations we observe that the states of the closed-loop system converge to the origin in fact for a larger range of parameter values (θ ∈ [0.12, 1]), which indicate some conservativeness in the conditions of Theorem 4.1 (see Remark 4.2).
"
0
1
1.5 0.5
#
, A2 =
"
0 1 1 0.5
#
, A3 =
"
0 −1 1.1 1.2
#
"
2.6465 −0.7851 −0.7851 1.2568
#
,
(59)
L1 = [−3.5858 0.1413], L2 = [−4.7066 − 0.3329], L3 = [−3.2532 − 0.3601], and the scalars ζ1,1 = 0.6, ζ1,2 = 1.7, ζ1,3 = 1.5, ζ2,1 = 1.6, ζ2,2 = 0.7, ζ2,3 = 2, ζ3,1 = 2, ζ3,2 = 2, and ζ3,3 = 0.5 satisfy (19), (42), and (43). Therefore, it follows from Theorem 4.8 that the proposed control law (6) with feedback ˜ −1 = [−1.6222 − 0.9009], gain matrices K1 = L1 R −1 ˜ −1 = ˜ = [−2.2794 − 1.6888], K3 = L3 R K2 = L2 R [−1.6132 − 1.2942] , guarantees almost sure asymptotic stability of the closed-loop system (4), (6). Figures 10 and 11 respectively show sample paths of the state x(k) and the control input u(k) obtained with initial conditions x(0) = [1, −1]T and r(0) = 1. Furthermore, a sample path of the actual mode signal r(k) and its sampled version σ(k) are shown in Figure 12. As it is indicated in Figures 10– 12, the proposed control framework (6) achieves asymptotic stabilization of the zero solution. It is important to note that the feedback gains K1 , K2 , and K3 are designed by utilizing Theorem 4.8 without using information on the distribution µ : N → [0, 1]. Note that Theorem 4.8 requires only the knowledge of an upper-bounding constant τ¯ ∈ N for the length of intervals between consecutive mode observation instants, instead of the exact knowledge of µ : N → [0, 1].
Example 5.2. Consider the switched stochastic system (4) with M = 3 modes described by the subsystems matrices A1 =
15
,
B1 = [0, 1]T , B2 = [0, 0.2]T , and B3 = [0, 0.7]T . The mode signal {r(k) ∈ M , {1, 2, 3}}k∈N0 of the switched system is assumed to be an aperiodic and irreducible Markov chain characterized by the transition matrix P with entries pi,i = 0.6, i ∈ M, and pi,j = 0.2, i 6= j, i, j ∈ M. The invariant distribution for {r(k) ∈ M , {1, 2, 3}}k∈N0 is given by π1 = π2 = π3 = 13 . Furthermore, note that the transition matrix P possesses positive real eigenvalues 0.4 (with algebraic multiplicity 2) and 1. The lengths of intervals between consecutive mode observation instants are assumed to be uniformly distributed over the set {2, 3, 4, 5} (see Remark 4.5). In other words, the distribution µ : N → [0, 1] is assumed to be given by (37) with τL = 2 and τH = 5.
6 Conclusion We proposed a feedback control framework for stabilization of switched linear stochastic systems under randomly available mode information. In this problem setting, information on the active operation mode of the switched system is assumed to be available for control purposes only at random
12
u(k)
1.5 1.0 0.5 0.0 −0.5
Cassandras, C. G. and Lygeros, J. (Eds.) (2006). Stochastic Hybrid Systems. CRC Press. Boca Raton.
Mode switching instants Mode observation instants
Cetinkaya, A. and T. Hayakawa (2011). Stabilization of switched linear stochastic dynamical systems under limited mode information. In ‘Proc. IEEE Conf. Dec. Contr.’. Orlando, FL. pp. 8032–8037. 0
5
10
15
20
25
Cetinkaya, A. and T. Hayakawa (2012). Feedback control of switched stochastic systems using uniformly sampled mode information. In ‘Proc. Amer. Contr. Conf.’. Montreal, Canada. pp. 3778–3783.
Time [k]
Figure 11. Control input versus time
Cetinkaya, A. and T. Hayakawa (2013a). Discrete-time switched stochastic control systems with randomly observed operation mode. In ‘Proc. IEEE Conf. Dec. Contr.’. Firenze, Italy. pp. 85–90. Cetinkaya, A. and T. Hayakawa (2013b). Stabilizing discrete-time switched linear stochastic systems using periodically available imprecise mode information. In ‘Proc. Amer. Contr. Conf.’. Watshington, DC, USA. pp. 3266–3271.
r(k)
3 2 1 0
5
10
15
20
25
σ(k)
3
Costa, O. L. V., M. D. Fragoso and R. P. Marques (2004). Discrete-Time Markov Jump Linear Systems. Springer.
2 1 0
5
10
15
20
de Farias, D. P., J. C. Geromel, J. B. R. do Val and O. L. V. Costa (2000). ‘Output feedback control of Markov jump linear systems in continuous-time’. IEEE Trans. Autom. Contr. 45, 944–949.
25
Time [k]
Durrett, R. (2010). Probability: Theory and Examples. Cambridge University Press: New York.
Figure 12. Actual mode r(k) and sampled mode σ(k)
Fang, Y. and K. A. Loparo (2002). ‘Stabilization of continous-time jump linear systems’. IEEE Trans. Autom. Contr. 47, 1590–1602.
time instants. We presented a probabilistic analysis concerning a sequence-valued stochastic process that captures the evolution of active operation mode between mode observation instants. We then used the results of this analysis to obtain sufficient almost sure asymptotic stability conditions for the zero solution of the closed-loop system.
Geromel, J. C., A. P. C. Goncalves and A. R. Fioravanti (2009). ‘Dynamic output feedback control of discrete-time Markov jump linear systems through linear matrix inequalities’. SIAM J. Contr. Optm. 48(2), 573– 593. Ghaoui, L. E. and M. A. Rami (1996). ‘Robust state-feedback stabilization of jump linear systems via LMIs’. Int. J. Robust Nonl. Contr. 6, 1015– 1022.
References
Li, C., M. Z. Q. Chen, J. Lam and X. Mao (2012). ‘On exponential almost sure stability of random jump systems’. IEEE Trans. Autom. Contr. 57(12), 3064–3077.
Bercu, B., F. Dufour and G. G. Yin (2009). ‘Almost sure stabilization for feedback controls of regime-switching linear systems with a hidden Markov chain’. IEEE Trans. Autom. Contr. 54, 2114–2125.
Nassiri-Toussi, K. and P. E. Caines (1991). On the adaptive stabilization and ergodic behaviour of stochastic jump-Markov systems via nonlinear filtering. In ‘Proc. IEEE Conf. Dec. Contr.’. Brighton, England. pp. 1784–1785.
Bernstein, D. (2009). Matrix mathematics: Theory, Facts, and Formulas. Princeton University Press: Princeton. Bolzern, P., P. Colaneri and G. De Nicolao (2004). On almost sure stability of discrete-time Markov jump linear systems. In ‘IEEE Conf. Dec. Contr.’. Nassau, Bahamas. pp. 3204–3208.
Norris, J. (2009). Markov Chains. Cambridge University Press: New York. Sathanantan, S., O.Adetona, C. Beane and L. H. Keel (2008). ‘Feedback stabilization of Markov jump linear system with time-varying delay’. Stoc. Anal. App. 26, 577–594.
Boukas, E. K. (2006). ‘Static output feedback control for stochastic hybrid systems: LMI approach’. Automatica 42, 183–188.
Serfozo, R. (2009). Basics of Applied Stochastic Processes. SpringerVerlag: New York.
Boyle, F. A., J. Haupt, G. L. Fudge and C. C. A. Yeh (2007). Detecting signal structure from randomly-sampled data. In ‘IEEE 14th Workshop on Stat. Sign. Proc.’. IEEE. pp. 326–330. Caines, P. E. and J. F. Zhang (1992). Adaptive control for jump parameter systems via non-linear filtering. In ‘Proc. IEEE Conf. Dec. Contr.’. Tucson, AZ. pp. 699–704.
Vargas, A. N., W. Furloni and J. B. R. do Val (2006). Constrained model predictive control of jump linear systems with noise and nonobserved Markov state. In ‘Proc. Amer. Contr. Conf.’. Minneapolis, MN. pp. 929–934.
Carlen, E. and R. V. Mendes (2009). ‘Signal reconstruction by random sampling in chirp space’. Nonl. Dyn. 56(3), 223–229.
Yin, G. and C. Zhu (2010). Hybrid Switching Diffusions: Properties and Applications. Springer-Verlag: New York.
13
