Conditional densities for continuous-time ... - Semantic Scholar
2164
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 11, NOVEMBER 1999
Conditional Densities for Continuous-Time Nonlinear Hybrid Systems with Applications to Fault Detection Joseph L. Hibey and Charalambos D. Charalambous
Abstract—Continuous-time nonlinear stochastic differential state and measurement equations, all of which have coefficients capable of abrupt changes at a random time, are considered; finite-state jump Markov chains are used to model the changes. Conditional probability densities, which are essential in obtaining filtered estimates for these hybrid systems, are then derived. They are governed by a coupled system of stochastic partial differential equations. When the matrix of the Markov chain is either lower or upper diagonal, it is shown that the system of conditional density equations is finite-dimensional computable. These findings are then applied to a fault detection problem to compute state estimates that include the failure time.
Our paper is outlined as follows. In Section II, we give the mathematical model for our hybrid system. In Section III, we derive the reference probability model using Girsanov’s theorem, and arrive at the DMZ-type equation for the unnormalized conditional density. A gauge transformation then allows us to conclude the existence and uniqueness of the density function for an associated measure-valued process. Then, in Section IV, we give an application to a problem in fault detection in which explicit expressions for the conditional expectations of the continuous-time states and the state of the Markov chain are obtained; the latter corresponds to the distribution of the failure time.
Q
Index Terms— Fault detection, hybrid systems, measure transformations, nonlinear filtering, unnormalized densities.
I. INTRODUCTION We address the problem of obtaining filtering estimates for hybrid, continuous-time, nonlinear dynamic systems. We are motivated by an earlier paper of Hibey and Charalambous [1] that considered a version of the changepoint problem (i.e., a random time of change of probability distributions) and the attendant error probabilities used to characterize its performance. In that work, suboptimal filters of the extended Kalman–Bucy type were used. Here, attention is focused on deriving the unnormalized conditional probability density that is shown to satisfy a Duncan–Mortensen–Zakai (DMZ) type equation (see Wong and Hajek [2]) with an additional input term. The result reveals a coupling among various stochastic partial differential equations because of the finite-state jump Markov process used to model random, abrupt changes in the system model’s coefficients. Our findings complement those of Elliott et al. [3] which discusses similar modeling via jump Markov processes in discrete time. Also, the papers by Rishel [4] and Benes et al. [5] use models that can be included in our methodology. The procedure we use involves Girsanov-type measure transformations, measure-valued processes, and gauge transformations of the type used in, for example, [2] and [6]. Furthermore, as is well known, the unnormalized conditional probability density, unlike its normalized version, satisfies a linear equation, and thus holds the prospect of leading to finite-dimensional filters for the nonlinear systems we consider here; see, for example, Charalambous and Elliott [7], [8] for similar models. The use of the jump Markov processes to model the abrupt changes, however, results in the linear equation mentioned above being driven by an input term that shows a coupling of effects in the hybrid system.
II. MATHEMATICAL MODEL
Let ( ; F ; P ) be a probability space on which we have a right continuous, complete filtration fFt gt0 of sub- fields of F . In the sequel, E [1] will denote the mathematical expectation with respect to the probability P . On ( ; F ; P ), we assume the existence of the following processes. 1) A continuous-time Markov chain fXt gt0 adapted to fFt gt0 , whose state space is the finite set S = fe1 , e2 , 1 1 1, eN g (for some N 2 N ) of canonical unit vectors in R N . 2) A continuous-time jump diffusion process fxt gt0 , adapted to fFt gt0, with state space Rn . 3) A continuous-time jump observation process fyt gt0 , with state space Rd , which generates the complete filtration fFty gt0 (Fty is a sub- field of Ft ). Write pt for the probability distribution of fXt gt0 , that is, pit = 1 N 0 P (Xt = ei ); 1 i N , and pt = (pt ; 1 1 1 ; pt ) satisfies the equation dpt =dt = 5pt . Here, 5 = (ij ) is an N 2 N matrix, the so-called Q matrix of the process. Since 5 is a Q matrix, we have ii = 0 j 6=i ji . Then fXt gt0 is a corlol process which satisfies the following stochastic differential equation [9]: dXt
(2.1)
where fMt gt0 is an (fFt gt0 ; P ), corlol martingale such that sup0tT E [jMt j2 ] < 1. The continuous-time jump diffusion process fxt gt0 satisfies the stochastic differential equation dxt
= f (Xt ; xt ) dt + (Xt ; xt ) dwt :
(2.2)
For all ej S; f (ej ; x); (ej ; x) satisfy a global Lipshitz and linear growth condition in x; fwt gt0 is an (fFt gt0 ; P ) standard Brownian motion independent of fMt gt0 , and the initial state x0 has probability distribution Px (dx). The processes fXt gt0 ; fxt gt0 are observed by the continuous-time jump observation process fyt gt0 which satisfies the stochastic differential equation dyt
Manuscript received January 15, 1998. Recommended by Associate Editor, T. E. Duncan. The work of C. D. Charalambous was supported by the Natural Science and Engineering Research Council of Canada under Grant OGP0183720. J. L. Hibey is with the Department of Electrical Engineering, University of Colorado at Denver, Denver, CO 80217 USA. C. D. Charalambous is with the Department of Electrical and Computer Engineering and the Centre for Intelligent Machines, McGill University, Montr´eal, P.Q. H3A 2A7 Canada (e-mail: [email protected]). Publisher Item Identifier S 0018-9286(99)08585-2.
= 5Xt dt + dMt
2 ( )( (fF g 0 f g 0
= h(Xt ; xt ) dt + N 1=2 dbt :
( ) ) ( 2 2) (
(2.3)
For all ej S; h ej ; x satisfies a linear growth condition x; @=@x h ej ; x ; @ =@x h ej ; x are continuous, bt t is t t ; P ) standard Brownian motion which is independent = = > . Mt ; wt t , and
N =1 N 1 2 N 1 2
f g 0
)
0
in an of
The contribution of this paper is the derivation of the conditional density and distribution of (Xt ; xt ) given Fty , and the derivation of ^t = E [Xt jFty ]; x^t = explicit, finite-dimensional filtered estimates X y E [xt jFt ] for jump linear systems when the Q matrices are either
0018–9286/99$10.00 1999 IEEE
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 11, NOVEMBER 1999
upper or lower diagonal. The application of these results to fault detection problems is then delineated.
2165
Note that summing both sides of (3.6) from 1 to N with 8 = 1 ~ [3t jFty ] = Nj=1 RR qj (z; t) dz:Moreover, if i (z; t) leads to E denotes the normalized conditional density, i.e.,
III. REFERENCE PROBABILITY MODEL We model system (2.1)–(2.3) by supposing that, initially, we have a probability space ( ; F ; P~ ) such that, under P~ : 1) fwt gt0 is an m-dimensional Brownian motion and fxt gt0 is defined by (2.2); 2) fMt gt0 is an N -dimensional martingale such that supt2[0; T ] E [jMt j] < 1 and fXt gt0 is defined by (2.1); 3) fyt gt0 is a d-dimensional Brownian motion independent of fwt gt0 ; fMt gt0 ; x0 , and having quadratic variation hy; yit =1 N . Consider the exponential
3 = exp t
t
0
h0 (Xs ; xs )N 01 dys
0 21 h0 (X ; x )N 01 h(X ; x ) ds : 0 s
s
s
d3t = 3t h0 (Xt ; xt )N 01 dyt (3.2) ~ [3t ] = 1, where E~ denotes expectation with respect to and E probability measure P~ . We define a probability measure P in terms of P~ by setting (dP=dP~ )jF = 3t . Then Girsanov’s theorem implies that, under P; fbt gt0 is a standard d-dimensional Brownian motion if we define
bt =
0
N 01 2 (dy 0 h(X ; x ) ds); =
s
s
b0 = 0:
s
y ~ E [hXt ; ei i8(xt )jFty ] = E [3t hX~t ; ei i8(y xt )jFt ] : E [3t jFt ]
Write
N
j
qti (8) = E~ [3t hXt ; ei i8(xt )jFty ]
(3.3)
(3.4)
which is the unnormalized conditional expectation in the numerator of (3.3). Then, because N hXt ; ei i = 1, i=1 i E [hXt ; ei i8(xt )jFty ] = Nqt (8) qti (1) i=1
(3.5)
and q (8) is a measure-valued process. Suppose q (8) has a density qi (x; t). Then i t
E~ [3t hXt ; ei i8(xt )jFty ] =
i t
8(z)q (z; t) dz; i
R
1 i N: (3.6)
R
q (z; t) dz
1 i N:
;
(3.7)
j
=1
R
From (3.3) and (3.7), we can determine the conditional probability distributions P~ (Xt = ei jFty ) and P (Xt = ei jFty ) as follows: N
X^ t = E [Xt jFty ] = j
=1
ej P (Xt = ej jFty )
(3.8)
where
P (Xt = ei jFty ) = hX^ t ; ei i = E [hXt ; ei ijFty ] =
R R
1 i i (z; t)dz = t (3.9)
and
E~ [3t hXt ; ei ijFty ] = E~ [3t jFty ]
qi (z; t) dz
R N
j
That is, under measure P , the observations are described by (2.3). Notice that the distributions of the processes fXt gt0 and fxt gt0 are invariant with respect to P and P~ , and that these satisfy (2.1) and (2.2), respectively. Let 8: Rn ! R be a test function (i.e., measurable, C 2 , and with compact support), and let h1; 1i be the inner product in RN . The following problem posed earlier is concerned with estimates of the form E [8(xt )jFty ] and E [hXt ; ej ijFty ]. Using Bayes’ theorem [2], we have
i
qi (z; t)
i (z; t) =
(3.1)
Then
8(z) (z; t) dz
then from (3.3) [or (3.5)] we have
t
s
t
E [hXt ; ei i8(xt )jFty ] =
=1
1 : = i t
N
qj (z; t) dz
R R
j
=1
(3.10)
jt
Finally, N N
X^ t = j
=1
tj ej = j =1N j
ej jt
=1
:
(3.11)
j t
Next, we derive the Duncan–Mortensen–Zakai equation for q i (x; t). Theorem 3.1: Suppose the measure-valued process qti (8) = ~ E [3t hXt ; ei i8(xt )jFty ] has a density qi (x; t); 1 i N . Then
qi (x; t) = qi (x; 0) + 1
0 0 +
t
i
t
0
i
N
f 0 (ei ; x)rqi (x; s) ds + j
t
q (x; s)h0 (e ; x)N 01 dy i
i
i
i
0
0
2 0 2 0 Tr(r (q (x; s)(e ; x) (e ; x))) ds Tr(rf (e ; x))q (x; s) ds t
i
s
=1 0
t
ij qj (x; s) ds (3.12)
where r8; r2 8 denote the gradient and Hessian of 8: R n ! R , respectively. Proof: Using Ito’s formula to deduce an equation for 8(xt ), we apply the product rule to 8(xt ) and 3t of (3.2). Then, from this
2166
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 11, NOVEMBER 1999
result and (2.1), we obtain
hX ; e i3 8(x ) = hX0 ; e i8(x0 ) + t
i
t
t
t
i
t
+
0
+
0
+
Remark 3.2: The coupled system of stochastic partial differential equations (3.12) is the unnormalized conditional density of hXt; ei ixt given Fty ; 1 i N . When ij = 0; 8 i 6= j , this system is decoupled, and reduces to the well-known Duncan–Mortensen–Zakai equation of xt given Fty , for each 1 i N . Let q~(x; t) = [q 1 ; 1 1 1 ; q N ]0 (x; t). Then (3.12) is written in vector form as
i
s
i
s
s
s
s
s
s
i
s
s
s
s
~(
+ 21 hX ; e i3 Tr(r2 8(x )(X ; x )0 (X 0 + hX ; e i3 8(x )h0 (X ; x )N 01 dy : s
i
s
s
L1
s
s
s
; xs
)) ds
t
s
0
i
s
s
s
s
(3.13)
s
Taking conditional expectation with respect to F of each side of the above equation while using the mutual independence of fx0 ; wt ; yt; Mt gt0 (see [6]) and a version of Fubini’s theorem (see [2]), we obtain y t
t
s
0
+ E~
t
hX
s
0
+ ~
t
1E 2
i
s
s
s
s
i
s
+ E~
s
0
i
t
i
qs
0
+ 21 +
) jF
; xs ds
Since
~
N j =1 N
E
s
y t
s
s
s
; ei
s
s
s
y t
i3 8(x )jF ] ds s
s
y s
i
Tr(r2 8(e ; x)0 (e ; x))
i
qs
i
ds
i
8h0 (e ; x)N 01
i
qs
i
s
dys :
(3.14)
i
i
i
N
y s
s
j
2) For
( 8):
qs ij j =1
Substituting the last equality in (3.14), q
t
t
+
i s
N
j =1
i s
t
0
i
i
j
( 8) ds +
qs ij
^(
) ^ = q (x; 0);
j
j =1
i
ij q x; t ; q0
ds
i
t
0
i
qs
8h0 (e ; x)N 01 i
i
1iN
(3.18)
(
;
) = (L + )q (z; t; x; s) ds + q (z; t; x; s)h0 (e ; x)N 01 dy i
ii
i
i
s
:
(3.19)
8: R ! R, which is continuous with compact support, n
lim #
1
01
i
(
;
)8(x) dx = 8(z);
q z; t x; s
(3.20)
that is, limt#s q i (z; t; x; s) = (z 0 x) is a Dirac delta function. Lemma 3.4: Suppose qti (8) has a density function q i (x; t); 1 i N . Then
((r8)0 f (e ; x)) ds
(8) = q0 (8) + q 0 1 + 2 q Tr(r2 8(e ; x)0 (e ; x)) 0 i
N
i
i
s
(3.17)
1iN
i
t
) = L^ q^ (x; t) +
t s
i t
(3.16)
i
i
dq z; t x; s
hX ; e ih5X ; e i3 8(x )jF = s
NN
) = exp(0h0 (e ; x)N 01 y )q (x; t);
j
j
+
^ i is a second-order differential operator. The resulting system where L (3.18) is a nonstochastic system of second-order partial differential equations, with y 2 C ([0; T ]; Rd ) entering parametrically through its coefficients. Therefore, by expressing (3.18) in vector form (3.16), we can obtain the existence and uniqueness of classical solutions 2; 1 Cx; t (Rn 2 [0; T ]) (see [10]) due to the assumptions on the coefficients f; ; h; the existence of a density function q i (x; t) for the measure-valued process qti (8); 1 i N , will then follow. Definition 3.3: A fundamental solution of (3.12) is an Fty measurable function q i (z; t; x; s) with (z; x) 2 R n 2 R n ; 0 s t T such that the following hold. 1) For fixed (s; x) 2 (0; t) 2 R n ; q i (1; t; x; s) 2 C 2 (Rn ), q i (1; 1; x; s) satisfies
hX ; e i = 1, then s
j =1
s
((r8)0 f (e ; x)) ds
0
0
E
0
t
t
s
~ [h5X
t
= q0 (8) + +
i
LN
Moreover, the gauge transformation
y t
hX ; e i3 8(x )h0 (X ; x )N 01 dy jF
t
)
i
i
^(
1 Tr(r2 8(x )(X ; x )0 (X ; x )) dsjF s
N
i
@ i q x; t @t
s
s
~(
changes system (3.12) into
0 ; e i3 (r8(x )) f (X i
N
N
q x; t dt
.
+ 1 L2 + 2 1 1 1 + q~(x; t)h0 N 01 dy
+ 1
8 = 12 Tr r2 (8(e ; 1)0 (e ; 1)) 0Tr(rf (e ; 1))8 0 f 0 (e ; 1)r8: i
hX ; e i3
0
i
L
y t
s
..
q x; t
y t
i
t
.. .
LN
where
y t
t
+ 12 1 1 1 + 22
L2 L2
t
^(
~ [h i3 8(x )jF ] ~ = E [hX0 ; e i8(x0 )jF ] + E~ h5X ; e i3 8(x ) dsjF
E Xt ; ei
)=
s
t
+ 11 + 21
L1 L1
dq x; t
hX ; e i3 (r8(x ))0(X ; x ) dw s
0
s
hX ; e i3 (r8(x ))0 f (X ; x ) ds s
t
i
hdM ; e i3 8(x ) ds s
t
h5X ; e i3 8(x ) ds s
0
dys :
(3.15) Finally, integrating each term of (3.15) by parts [2] yields (3.12).
i
(
)=
N
q z; t
t
ij
0
j =1 j =i
6
+
i
(
j
;
q z; t x; R R
(
) ( i
;
)
q x; s q z; t x; s dx ds R R
0)q (x; 0) dx; i
1 i N: (3.21)
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 11, NOVEMBER 1999
Proof: Let q~i (z; t) denote the right-hand side of (3.21). By Ito’s rule,
dq~i (z; t)
=
N
j =1 j 6=i
+ =
j =1 j 6=i
0
+
t N
R
qj (x;
R
+ h0 (ei ; x)N 01 j =1 j 6=i
j =1 j 6=i
ij
+
qi (z;
j =1 j 6=i
qj (x; s)dqi (z; t; x; s)dx ds
R
dxt = F xt dt + G dwt ; x0 2 R dyt = Hxt dt + N 1=2 dbt ; y0 = 0 2 R:
t; x; s) dx ds dt t
0
R
t; x;
qj (x; s)qi (z; t; x; s) dx ds dys
0)qi (x;
t; x;
0) dx dt
0)qi (x;
IV. APPLICATION OF FINITE-DIMENSIONAL FILTERING TO FAULT DETECTION Consider (without loss of generality) the one-dimensional state and observation processes fxt ; yt gt0 of (2.2) and (2.3), respectively, described by
ij (Li + ii )
qi (z;
R
R
0
R
qi (z; t; x; 0)qi (x; 0) dx dt N j =1 j 6=i
ij
R
t
0
R
zt =
ij q~j (z; t) dt + (Li + ii )~ qi (z; t) dt
+ q~i (z;
t)h0 (ei ; x)N 01 dyt :
Thus, q~i (1; 1) satisfies (3.12) for (z; t) 2 Rn 2 [0; T ) and, for t = 0; q~i (z; 0) = qi (z; 0), which implies the validity of (3.21). Theorem 3.5: Consider the jump linear system
dXt = 5Xt dt + dMt ; dxt = A(Xt )xt dt + G(Xt )dwt ; dyt = H (Xt )xt dt + N 1=2 dbt ; x(0) 2 Rn ; y(0) = 0 2 Rd : If the Q matrix 5 is either lower or upper diagonal (e.g., correspond^ t and x^t are finite-dimensional ing to a counting process), then X computable. Proof: If 5 is lower diagonal, then q 1 (z; t) is unnormalized Gaussian. Moreover, the equation for q 2 (z; t) is coupled with that
(4.2)
0; 1;
if t < T if t T .
(4.3)
Then the faults can be expressed as
F = F1 (1 0 zt ) + F2 zt G = G1 (1 0 zt ) + G2 zt H = H1 (1 0 zt ) + H2 zt :
(4.4)
Furthermore, assume that T is an exponential random variable with constant parameter 0, independent of x0 ; fwt ; bt gt0 . As mentioned in [11], we can view fzt gt0 as being a Poisson process with rate stopped at its first jump time T , and having an associated martingale Mt = zt 0 min(t; T ). Then fzt gt0 satisfies the equation
dzt = (1 0 zt ) dt + dMt :
Also, for x
qj (x; s)qi (z; t; x; s) dx ds
qi (z; t; x; 0)qi (x; 0) dx dyt
(4.1)
Here, x0 is a Gaussian random variable. Assume that a fault occurs at time T , at which time F; G; H change from F1 to F2 , G1 to G2 , H1 to H2 , respectively. Following [11], let
0) dx dyt
qj (x; s)qi (z; t; x; s) dx ds
+ N
j =1 j 6=i
s)qi (z;
t
+ h0 (ei ; x)N 01
=
0
of q 1 (z; t) and, consequently, by Lemma 3.4, q 2 (z; t) is also finite-dimensional computable. A similar argument holds for the remaining densities. The case of upper diagonal matrices is treated similarly.
ij qj (z; t) dt + (Li + ii ) N
1
t
ij
N
ij h0 (ei ; x)N 01
j =1 j 6=i
N
j =1 j 6=i
ij qj (z; t) dt +
+ (Li + ii ) =
N
dqi (z; t; x; 0)qi (x; 0) dx
R
N
1
ij qj (z; t)dt +
2167
= 0 only (because zt stops at its first jump), P (T t) = 1 0 e0t 1 P (z = yjz = x) Pxy (t) = t 0 0t ; e x = 1 0 e0t ; ifif yy = = 1 0 x.
(4.5)
(4.6)
Define the infinitesimal generator of fzt gt0 by
1 [E ( (zt )jz0 = x) 0 (x)]: A (x) =1 lim t#0 t
Then
1 A (x) = lim (y)P (zt = yjz0 = x) 0 (x) t#0 t y = lim 1 [ (x)e0t + (1 0 x)(1 0 e0t ) 0 (x)]: t#0
t
Thus,
A (x) = [ (1 0 x) 0 (x)]:
(4.7)
fzt gt0 is a pure birth process; therefore, Pxy (t) = < x; 8 t. The transition probabilities can be computed from
Notice that
0 8y
the backward equation
dPxy (t) = APxy (t) = [P(10x)y (t) 0 Pxy (t)] (4.8) dt with P00 (0) = 1; P01 (0) = 0; P10 (t) = 0 8 t; P11 (t) = 1 8 t, or
from (4.6) directly, namely,
dPxy (0+ ) dt
= xy ;
(x; y) 2 f0; 1g:
(4.9)
2168
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 11, NOVEMBER 1999
Hence, the probability rates are given by the matrix 00 3 = 10
01 11
= 00 0
:
From Lemma 3.4, we obtain t (4.10)
Next, we find an equivalent representation for fzt gt0 in terms of the process fXt gt0 . Introduce the bijective mapping 0 70! e1 ; 1 70! e2 , e.g., Xt = (Xt1; Xt2 )0 = (1 0 zt ; zt )0 . Then
dXt =
0 0
Xt dt +
01
(4.11) 1 dMt that is, the Q matrix 5 is given by 5 = 30 . Moreover, (4.1) and
0
(4.2) are expressed as
dxt dyt
x0 2 R y0 = 0 2 R (4.12)
= F (Xt )xt dt + G(Xt ) dwt ; = H (Xt )xt dt + N 1=2 dbt ;
where
F (ei ) = Fi ;
G(ei ) = Gi ;
H (ei ) = Hi ;
i = 1; 2: (4.13)
From Theorem 3.1, the unnormalized conditional densities are obtained from the equations
q2 (z; t) =
(4.14)
where
(4.19)
3^ s;2 t (x) = s;2 t exp 0 21 (xSs;2 t x 0 2x2s; t )
where
1 exp 2t=
s;
where x ^2 :
(4.20)
[0; T ] 2 2 R ! R, P 2 : [0; T ] ! R, 3^ 2 : [0; T ] 2
2 R ! R are given by dx^2t (x) = F (e2 )^ x2t (x) dt + Pt2 H (e2 )N 01 1 (dyt 0 H (e2 )^x2t (x)dt); x^20 = x P_t2 = F (e2 )Pt2 + Pt2 F (e2 ) 0 Pt2 H (e2 )N 01 1 H (e2 )Pt2 + G(e2 )G(e2 ); P02 = 0 t H (e2 )^ x2s (x)N 01 dys 3^ 02; t (x) = exp 0 t 0 21 N 01 (H (e2 )^x2s (x))2 ds : 0
(4.21) (4.22)
s
t s t s
(4.26)
2 H (e2 )N 01 dy s;
2 H (e2 )N 01 H (e2 ) s;2 d s;
t
82s; H (e2 )N 01 H (e2 )8s;2 d
t
82s; H (e2 )N 01 [dy 0 H (e2 ) s;2 d ]:
Notice that 2 ; 2 ; S 2 , and 2 are independent of x. Also, anticipating the need to compute R zq 2 (z; t) dz in the numerator of (4.29)
1
below, we introduce the vector v = [x z ]0 in (4.24) and complete the square in v . Thus, after some algebra, we obtain t 1 ^ 01; s s;2 t q2 (z; t) = e0s 3 0 2(P01; s Ps;2 t )1=2
1 exp 0 12 0 14 Tr(A)(A01B)0 A01 B + f ds 1 exp 0 12 Tr A v + 12 A01 B 0 (v + 12 A01 B) R
+ where
R
q2 (z; t; x;
0)q2 (x; 0) dx
a e=2 e=2 b 1 c B= d
1 A=
1 (P 1 )01 + (P 2 )01 (82 )2 + S 2 a= 0; s s; t s; t s; t 1 (P 2 )01 b= s; t 1 c=
1 d= (4.23)
s
1 2s; t =
0)
= (2)1=21(P 2 )1=2 exp 0 12 (Pt2 )01 (z 0 x^2t (x))2 3^ 02; t (x) t
0 12
1 2t= Ss;
The fundamental solution of (4.15) is
q2 (z; t; x;
(4.25)
In addition, the solution to (4.23) can be written as
q1 (x; t) =
0
+
8_ 2s; t = [F (e2 ) 0 Ps;2 t H (e2 )N 01 H (e2 )]8s;2 t 2 t = F (e2 ) s; t dt + Ps;2 t H (e2 )N 01 d s; 1 (dyt 0 H (e2 ) s;2 t dt); s;2 s = 0:
The explicit solution of (4.14) is
0
q1 (x; s)q2 (z; t; x; s) dx ds
2t x^2s; t (x) = 82s; t x + s;
(4.15)
1 1 1 01 12 (2)1=2 (Pt1 )1=2 exp 0 2 (Pt ) (x 0 x^t ) 1 exp(0t)3^ 01; t (4.16) where x ^1 : [0; T ] 2 ! R, P 1 : [0; T ] ! R, P 1 0, 3^ 1 : [0; T ] 2
! R are given by dx^1t = F (e1 )^ x1t dt + Pt1 H (e1 )N 01 (dyt 0 H (e1 )^ x1t dt); x^10 (4.17) P_ t1 = F (e1 )Pt1 + Pt1 F (e1 ) 0 Pt1 H (e1 )N 01 H (e1 )Pt1 + G(e1 )G(e1 ); P01 (4.18) t t H (e1 )^ x1s N 01 dys 0 12 N 01 (H (e1 )^x1s )2 ds : 3^ 01; t = exp
R
q2 (z; t; x; 0)q2 (x; 0) dx (4.24) R where q 2 (x; 0) is Gaussian and, consequently, the second integral in (4.24) results in a Gaussian density given by (4.16) with = 0 and by x ^1t ; Pt1 ; 3^ t1 replaced, respectively, with x^2t ; Pt2 ; 3^ t2 which satisfy (4.21)–(4.23), with x = x ^2 (0) and P02 = P 2 (0); note that this integral could also be computed by completing the square in x. Notice next that the integration with respect to the spatial variable x in the double integral on the right side of (4.24) can be computed by completing the square in x. Toward this end, we proceed as follows. We note from (4.20) that q 2 (z; t; x; s) depends on x ^2s; t (x) and 3^ s;2 t (x) for the nonzero initial time s. Therefore, the solution to (4.21) can be written as
1 @ @2 dq1 (x; t) = G(e1 )2 2 q1 (x; t) dt 0 (F (e1 )xq1 (x; t)) dt 2 @x @x 0 q1 (x; t) dt + H (e1 )N 01 q1 (x; t) dyt; q1 (x; 0) 1 @ @2 dq2 (x; t) = G(e2 )2 2 q2 (x; t) dt 0 (F (e2 )xq2 (x; t)) dt 2 @x @x + q1 (x; t) dt + H (e2 )N 01 q2 (x; t) dyt ; q2 (x; 0):
0
1 e=
0 2(P01; s )01x^10; s + 2(Ps;2 t )01 8s;2 t s;2 t 0 22s; t 0 2(Ps;2 t )01 s;2 t 0 2(Ps;2 t )01 8s;2 t
1 (P 1 )01 (^x1 )2 + (P 2 )01 ( 2 )2 : f= 0; s 0; s 0; s s; t
dx (4.27)
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 11, NOVEMBER 1999
Writing (4.27) symbolically as t
q2 (z; t) = (tv)ds (sv)dx + 0 R 1 1 where tv = time variables and sv =
1
R
q2 (z; t; x; 0)q2 (x; 0)dx
space variables given by
0s 1 2 tv = 2 )1=2 e 3^ 0; s s; t 2(P01;s Ps;t 1 exp 0 12 0 14 Tr(A)(A01B)0 A01 B + f sv = exp 0 12 Tr A v + 12 A01 B 0 v + 12 A01 B
we then deduce that
t
q2 (z; t) =
0
;
1 exp 0 12 Tr(A) z + 12 (A01 B)2 2 ds
q2 (z; t; x; 0)q2 (x; 0) dx (4.28) R where (A01 B )2 denotes the second component of A01 B . Hence, the representation for q 1 (z; t); q 2 (z; t) can be computed explicitly. The filtered estimates are obtained from
+
x^t = j =12
R
zqj (z; t) dz
j =1 R
2
j =1
X^ t = 2
(4.29)
qj (z; t) dz
ej jt
j =1
it
jt
qi (z; t) dz; i = 1; 2: (4.30) R The first term in the numerator of (4.29) follows from (4.16) and is given by
=
1
1
1
zq (z; t) dz = x^t exp( t)3^ 0; t : R The second term follows from (4.24) and the discussion that follows it; thus, we find that R which implies R
0
z (sv) dx dz = 0(2=Tr(A))(A01 B )2 =2
zq2 (z; t) dz
= 0
t
0
(tv)(2=Tr(A))(A01 B )2 )=2 ds + x^2t 3^ 02; t :
Finally, for the it terms in the denominator of (4.29) and ^ 01; t follows from (4.16), and 2t = (4.30), 1t = exp(0t)3 (2=Tr(A)) 0t (tv) ds + 3^ 02; t follows from R (sv) dx dz = 2=Tr(A) and the discussion after (4.24) that implies
3^ 02; t =
q2 (z; t; x; 0)q2 (x; 0) dx dz:
models of this type where Xt can have more than two states and xt is nonscalar, no new concepts are needed. For example, the Q matrix associated with a pure birth three-state process Xt would involve at most five nonzero ij ’s, a four-state process Xt would involve at most 11 nonzero ij ’s, etc. Because of the nonscalar xt , the Gaussian densities involve vectors and matrices, but normalizations and the computation of mean values are still the only evaluations required. Extensions of this work to control diffusions should be investigated in the context of optimal control. REFERENCES
(tv)(2=Tr(A))1=2
2
2169
R In view of the above, both x ^t and X^ t are explicitly computed. As one can see, the above manipulations are no more complicated than using normalization and computing mean values for one- and two-dimensional Gaussian densities. Furthermore, for more general
[1] J. Hibey and C. Charalambous, “Performance analysis for a changepoint problem,” IEEE Trans. Automat. Contr., vol. 44, pp. 1628–1632, Aug. 1999. [2] E. Wong and B. Hajek, Stochastic Processes in Engineering Systems. New York: Springer-Verlag, 1985. [3] R. Elliott, F. Dufour, and D. Sworder, “Exact hybrid filters in discrete time,” IEEE Trans. Automat. Contr., vol. 41, pp. 1807–1810, 1996. [4] R. Rishel, “Tracking the output of a poorly known Markov process,” Stochastics and Stochastics Rep., pp. 147–156, 1995. [5] V. Benes, K. Helmes, and R. Rishel, “Pursuing a manuevering target which uses a random process for its control,” IEEE Trans. Automat. Contr., vol. 40, pp. 307–311, Feb. 1995. [6] H. Kunita, Stochastic Flows and Stochastic Differential Equations. New York: Cambridge Univ. Press, 1990. [7] C. Charalambous and R. Elliott, “Certain nonlinear stochastic optimal control problems with explicit control laws equivalent to LEQG/LQG problems,” IEEE Trans. Automat. Contr., vol. 42, pp. 482–497, Apr. 1997. [8] C. Charalambous, “Partially observable nonlinear risk-sensitive control problems: Dynamic programming and verification theorems,” IEEE Trans. Automat. Contr., vol. 42, pp. 1130–1138, Aug. 1997. [9] R. Elliott, L. Aggoun, and J. Moore, Hidden Markov Models Estimation and Control. New York: Springer-Verlag, 1995. [10] V. E. Benes and I. Karatzas, “On the relation of Zakai’s and Mortensen’s equations,” SIAM J. Contr. Optimiz., vol. 21, no. 3, pp. 472–489, 1983. [11] M. Davis, “The application of nonlinear filtering to fault detection in linear systems,” IEEE Trans. Automat. Contr., pp. 257–259, 1975.
Asymptotic Convergence from
Stability
Andrew R. Teel Abstract— This note recalls that an absolutely continuous function having a uniformly locally integrable (not necessarily essentially bounded) derivative is uniformly continuous on the semi-infinite interval. This observation, in conjunction with Barbalat’s lemma, allows concluding asymptotic convergence to zero of an output function for a general class of nonlinear systems with Lp (not necessarily L1 ) disturbances. Index Terms—Asymptotic convergence, Barbalat’s lemma.
I. INTRODUCTION A tool often used for concluding that signals converge to zero in time-varying nonlinear control systems is Barbalat’s lemma (see [8, Manuscript received April 7, 1998. Recommended by Associate Editor, A. Rantzer. This work was supported in part by the National Science Foundation under Grant ECS-9896140 and by the Air Force Office of Scientific Research under Grant F49620-98-1-0087. The author is with the Department of Electrical and Computer Engineering, University of California at Santa Barbara, Santa Barbara, CA 93106 USA (email: [email protected]). Publisher Item Identifier S 0018-9286(99)08586-4.
0018–9286/99$10.00 1999 IEEE
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 11, NOVEMBER 1999
Conditional Densities for Continuous-Time Nonlinear Hybrid Systems with Applications to Fault Detection Joseph L. Hibey and Charalambos D. Charalambous
Abstract—Continuous-time nonlinear stochastic differential state and measurement equations, all of which have coefficients capable of abrupt changes at a random time, are considered; finite-state jump Markov chains are used to model the changes. Conditional probability densities, which are essential in obtaining filtered estimates for these hybrid systems, are then derived. They are governed by a coupled system of stochastic partial differential equations. When the matrix of the Markov chain is either lower or upper diagonal, it is shown that the system of conditional density equations is finite-dimensional computable. These findings are then applied to a fault detection problem to compute state estimates that include the failure time.
Our paper is outlined as follows. In Section II, we give the mathematical model for our hybrid system. In Section III, we derive the reference probability model using Girsanov’s theorem, and arrive at the DMZ-type equation for the unnormalized conditional density. A gauge transformation then allows us to conclude the existence and uniqueness of the density function for an associated measure-valued process. Then, in Section IV, we give an application to a problem in fault detection in which explicit expressions for the conditional expectations of the continuous-time states and the state of the Markov chain are obtained; the latter corresponds to the distribution of the failure time.
Q
Index Terms— Fault detection, hybrid systems, measure transformations, nonlinear filtering, unnormalized densities.
I. INTRODUCTION We address the problem of obtaining filtering estimates for hybrid, continuous-time, nonlinear dynamic systems. We are motivated by an earlier paper of Hibey and Charalambous [1] that considered a version of the changepoint problem (i.e., a random time of change of probability distributions) and the attendant error probabilities used to characterize its performance. In that work, suboptimal filters of the extended Kalman–Bucy type were used. Here, attention is focused on deriving the unnormalized conditional probability density that is shown to satisfy a Duncan–Mortensen–Zakai (DMZ) type equation (see Wong and Hajek [2]) with an additional input term. The result reveals a coupling among various stochastic partial differential equations because of the finite-state jump Markov process used to model random, abrupt changes in the system model’s coefficients. Our findings complement those of Elliott et al. [3] which discusses similar modeling via jump Markov processes in discrete time. Also, the papers by Rishel [4] and Benes et al. [5] use models that can be included in our methodology. The procedure we use involves Girsanov-type measure transformations, measure-valued processes, and gauge transformations of the type used in, for example, [2] and [6]. Furthermore, as is well known, the unnormalized conditional probability density, unlike its normalized version, satisfies a linear equation, and thus holds the prospect of leading to finite-dimensional filters for the nonlinear systems we consider here; see, for example, Charalambous and Elliott [7], [8] for similar models. The use of the jump Markov processes to model the abrupt changes, however, results in the linear equation mentioned above being driven by an input term that shows a coupling of effects in the hybrid system.
II. MATHEMATICAL MODEL
Let ( ; F ; P ) be a probability space on which we have a right continuous, complete filtration fFt gt0 of sub- fields of F . In the sequel, E [1] will denote the mathematical expectation with respect to the probability P . On ( ; F ; P ), we assume the existence of the following processes. 1) A continuous-time Markov chain fXt gt0 adapted to fFt gt0 , whose state space is the finite set S = fe1 , e2 , 1 1 1, eN g (for some N 2 N ) of canonical unit vectors in R N . 2) A continuous-time jump diffusion process fxt gt0 , adapted to fFt gt0, with state space Rn . 3) A continuous-time jump observation process fyt gt0 , with state space Rd , which generates the complete filtration fFty gt0 (Fty is a sub- field of Ft ). Write pt for the probability distribution of fXt gt0 , that is, pit = 1 N 0 P (Xt = ei ); 1 i N , and pt = (pt ; 1 1 1 ; pt ) satisfies the equation dpt =dt = 5pt . Here, 5 = (ij ) is an N 2 N matrix, the so-called Q matrix of the process. Since 5 is a Q matrix, we have ii = 0 j 6=i ji . Then fXt gt0 is a corlol process which satisfies the following stochastic differential equation [9]: dXt
(2.1)
where fMt gt0 is an (fFt gt0 ; P ), corlol martingale such that sup0tT E [jMt j2 ] < 1. The continuous-time jump diffusion process fxt gt0 satisfies the stochastic differential equation dxt
= f (Xt ; xt ) dt + (Xt ; xt ) dwt :
(2.2)
For all ej S; f (ej ; x); (ej ; x) satisfy a global Lipshitz and linear growth condition in x; fwt gt0 is an (fFt gt0 ; P ) standard Brownian motion independent of fMt gt0 , and the initial state x0 has probability distribution Px (dx). The processes fXt gt0 ; fxt gt0 are observed by the continuous-time jump observation process fyt gt0 which satisfies the stochastic differential equation dyt
Manuscript received January 15, 1998. Recommended by Associate Editor, T. E. Duncan. The work of C. D. Charalambous was supported by the Natural Science and Engineering Research Council of Canada under Grant OGP0183720. J. L. Hibey is with the Department of Electrical Engineering, University of Colorado at Denver, Denver, CO 80217 USA. C. D. Charalambous is with the Department of Electrical and Computer Engineering and the Centre for Intelligent Machines, McGill University, Montr´eal, P.Q. H3A 2A7 Canada (e-mail: [email protected]). Publisher Item Identifier S 0018-9286(99)08585-2.
= 5Xt dt + dMt
2 ( )( (fF g 0 f g 0
= h(Xt ; xt ) dt + N 1=2 dbt :
( ) ) ( 2 2) (
(2.3)
For all ej S; h ej ; x satisfies a linear growth condition x; @=@x h ej ; x ; @ =@x h ej ; x are continuous, bt t is t t ; P ) standard Brownian motion which is independent = = > . Mt ; wt t , and
N =1 N 1 2 N 1 2
f g 0
)
0
in an of
The contribution of this paper is the derivation of the conditional density and distribution of (Xt ; xt ) given Fty , and the derivation of ^t = E [Xt jFty ]; x^t = explicit, finite-dimensional filtered estimates X y E [xt jFt ] for jump linear systems when the Q matrices are either
0018–9286/99$10.00 1999 IEEE
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 11, NOVEMBER 1999
upper or lower diagonal. The application of these results to fault detection problems is then delineated.
2165
Note that summing both sides of (3.6) from 1 to N with 8 = 1 ~ [3t jFty ] = Nj=1 RR qj (z; t) dz:Moreover, if i (z; t) leads to E denotes the normalized conditional density, i.e.,
III. REFERENCE PROBABILITY MODEL We model system (2.1)–(2.3) by supposing that, initially, we have a probability space ( ; F ; P~ ) such that, under P~ : 1) fwt gt0 is an m-dimensional Brownian motion and fxt gt0 is defined by (2.2); 2) fMt gt0 is an N -dimensional martingale such that supt2[0; T ] E [jMt j] < 1 and fXt gt0 is defined by (2.1); 3) fyt gt0 is a d-dimensional Brownian motion independent of fwt gt0 ; fMt gt0 ; x0 , and having quadratic variation hy; yit =1 N . Consider the exponential
3 = exp t
t
0
h0 (Xs ; xs )N 01 dys
0 21 h0 (X ; x )N 01 h(X ; x ) ds : 0 s
s
s
d3t = 3t h0 (Xt ; xt )N 01 dyt (3.2) ~ [3t ] = 1, where E~ denotes expectation with respect to and E probability measure P~ . We define a probability measure P in terms of P~ by setting (dP=dP~ )jF = 3t . Then Girsanov’s theorem implies that, under P; fbt gt0 is a standard d-dimensional Brownian motion if we define
bt =
0
N 01 2 (dy 0 h(X ; x ) ds); =
s
s
b0 = 0:
s
y ~ E [hXt ; ei i8(xt )jFty ] = E [3t hX~t ; ei i8(y xt )jFt ] : E [3t jFt ]
Write
N
j
qti (8) = E~ [3t hXt ; ei i8(xt )jFty ]
(3.3)
(3.4)
which is the unnormalized conditional expectation in the numerator of (3.3). Then, because N hXt ; ei i = 1, i=1 i E [hXt ; ei i8(xt )jFty ] = Nqt (8) qti (1) i=1
(3.5)
and q (8) is a measure-valued process. Suppose q (8) has a density qi (x; t). Then i t
E~ [3t hXt ; ei i8(xt )jFty ] =
i t
8(z)q (z; t) dz; i
R
1 i N: (3.6)
R
q (z; t) dz
1 i N:
;
(3.7)
j
=1
R
From (3.3) and (3.7), we can determine the conditional probability distributions P~ (Xt = ei jFty ) and P (Xt = ei jFty ) as follows: N
X^ t = E [Xt jFty ] = j
=1
ej P (Xt = ej jFty )
(3.8)
where
P (Xt = ei jFty ) = hX^ t ; ei i = E [hXt ; ei ijFty ] =
R R
1 i i (z; t)dz = t (3.9)
and
E~ [3t hXt ; ei ijFty ] = E~ [3t jFty ]
qi (z; t) dz
R N
j
That is, under measure P , the observations are described by (2.3). Notice that the distributions of the processes fXt gt0 and fxt gt0 are invariant with respect to P and P~ , and that these satisfy (2.1) and (2.2), respectively. Let 8: Rn ! R be a test function (i.e., measurable, C 2 , and with compact support), and let h1; 1i be the inner product in RN . The following problem posed earlier is concerned with estimates of the form E [8(xt )jFty ] and E [hXt ; ej ijFty ]. Using Bayes’ theorem [2], we have
i
qi (z; t)
i (z; t) =
(3.1)
Then
8(z) (z; t) dz
then from (3.3) [or (3.5)] we have
t
s
t
E [hXt ; ei i8(xt )jFty ] =
=1
1 : = i t
N
qj (z; t) dz
R R
j
=1
(3.10)
jt
Finally, N N
X^ t = j
=1
tj ej = j =1N j
ej jt
=1
:
(3.11)
j t
Next, we derive the Duncan–Mortensen–Zakai equation for q i (x; t). Theorem 3.1: Suppose the measure-valued process qti (8) = ~ E [3t hXt ; ei i8(xt )jFty ] has a density qi (x; t); 1 i N . Then
qi (x; t) = qi (x; 0) + 1
0 0 +
t
i
t
0
i
N
f 0 (ei ; x)rqi (x; s) ds + j
t
q (x; s)h0 (e ; x)N 01 dy i
i
i
i
0
0
2 0 2 0 Tr(r (q (x; s)(e ; x) (e ; x))) ds Tr(rf (e ; x))q (x; s) ds t
i
s
=1 0
t
ij qj (x; s) ds (3.12)
where r8; r2 8 denote the gradient and Hessian of 8: R n ! R , respectively. Proof: Using Ito’s formula to deduce an equation for 8(xt ), we apply the product rule to 8(xt ) and 3t of (3.2). Then, from this
2166
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 11, NOVEMBER 1999
result and (2.1), we obtain
hX ; e i3 8(x ) = hX0 ; e i8(x0 ) + t
i
t
t
t
i
t
+
0
+
0
+
Remark 3.2: The coupled system of stochastic partial differential equations (3.12) is the unnormalized conditional density of hXt; ei ixt given Fty ; 1 i N . When ij = 0; 8 i 6= j , this system is decoupled, and reduces to the well-known Duncan–Mortensen–Zakai equation of xt given Fty , for each 1 i N . Let q~(x; t) = [q 1 ; 1 1 1 ; q N ]0 (x; t). Then (3.12) is written in vector form as
i
s
i
s
s
s
s
s
s
i
s
s
s
s
~(
+ 21 hX ; e i3 Tr(r2 8(x )(X ; x )0 (X 0 + hX ; e i3 8(x )h0 (X ; x )N 01 dy : s
i
s
s
L1
s
s
s
; xs
)) ds
t
s
0
i
s
s
s
s
(3.13)
s
Taking conditional expectation with respect to F of each side of the above equation while using the mutual independence of fx0 ; wt ; yt; Mt gt0 (see [6]) and a version of Fubini’s theorem (see [2]), we obtain y t
t
s
0
+ E~
t
hX
s
0
+ ~
t
1E 2
i
s
s
s
s
i
s
+ E~
s
0
i
t
i
qs
0
+ 21 +
) jF
; xs ds
Since
~
N j =1 N
E
s
y t
s
s
s
; ei
s
s
s
y t
i3 8(x )jF ] ds s
s
y s
i
Tr(r2 8(e ; x)0 (e ; x))
i
qs
i
ds
i
8h0 (e ; x)N 01
i
qs
i
s
dys :
(3.14)
i
i
i
N
y s
s
j
2) For
( 8):
qs ij j =1
Substituting the last equality in (3.14), q
t
t
+
i s
N
j =1
i s
t
0
i
i
j
( 8) ds +
qs ij
^(
) ^ = q (x; 0);
j
j =1
i
ij q x; t ; q0
ds
i
t
0
i
qs
8h0 (e ; x)N 01 i
i
1iN
(3.18)
(
;
) = (L + )q (z; t; x; s) ds + q (z; t; x; s)h0 (e ; x)N 01 dy i
ii
i
i
s
:
(3.19)
8: R ! R, which is continuous with compact support, n
lim #
1
01
i
(
;
)8(x) dx = 8(z);
q z; t x; s
(3.20)
that is, limt#s q i (z; t; x; s) = (z 0 x) is a Dirac delta function. Lemma 3.4: Suppose qti (8) has a density function q i (x; t); 1 i N . Then
((r8)0 f (e ; x)) ds
(8) = q0 (8) + q 0 1 + 2 q Tr(r2 8(e ; x)0 (e ; x)) 0 i
N
i
i
s
(3.17)
1iN
i
t
) = L^ q^ (x; t) +
t s
i t
(3.16)
i
i
dq z; t x; s
hX ; e ih5X ; e i3 8(x )jF = s
NN
) = exp(0h0 (e ; x)N 01 y )q (x; t);
j
j
+
^ i is a second-order differential operator. The resulting system where L (3.18) is a nonstochastic system of second-order partial differential equations, with y 2 C ([0; T ]; Rd ) entering parametrically through its coefficients. Therefore, by expressing (3.18) in vector form (3.16), we can obtain the existence and uniqueness of classical solutions 2; 1 Cx; t (Rn 2 [0; T ]) (see [10]) due to the assumptions on the coefficients f; ; h; the existence of a density function q i (x; t) for the measure-valued process qti (8); 1 i N , will then follow. Definition 3.3: A fundamental solution of (3.12) is an Fty measurable function q i (z; t; x; s) with (z; x) 2 R n 2 R n ; 0 s t T such that the following hold. 1) For fixed (s; x) 2 (0; t) 2 R n ; q i (1; t; x; s) 2 C 2 (Rn ), q i (1; 1; x; s) satisfies
hX ; e i = 1, then s
j =1
s
((r8)0 f (e ; x)) ds
0
0
E
0
t
t
s
~ [h5X
t
= q0 (8) + +
i
LN
Moreover, the gauge transformation
y t
hX ; e i3 8(x )h0 (X ; x )N 01 dy jF
t
)
i
i
^(
1 Tr(r2 8(x )(X ; x )0 (X ; x )) dsjF s
N
i
@ i q x; t @t
s
s
~(
changes system (3.12) into
0 ; e i3 (r8(x )) f (X i
N
N
q x; t dt
.
+ 1 L2 + 2 1 1 1 + q~(x; t)h0 N 01 dy
+ 1
8 = 12 Tr r2 (8(e ; 1)0 (e ; 1)) 0Tr(rf (e ; 1))8 0 f 0 (e ; 1)r8: i
hX ; e i3
0
i
L
y t
s
..
q x; t
y t
i
t
.. .
LN
where
y t
t
+ 12 1 1 1 + 22
L2 L2
t
^(
~ [h i3 8(x )jF ] ~ = E [hX0 ; e i8(x0 )jF ] + E~ h5X ; e i3 8(x ) dsjF
E Xt ; ei
)=
s
t
+ 11 + 21
L1 L1
dq x; t
hX ; e i3 (r8(x ))0(X ; x ) dw s
0
s
hX ; e i3 (r8(x ))0 f (X ; x ) ds s
t
i
hdM ; e i3 8(x ) ds s
t
h5X ; e i3 8(x ) ds s
0
dys :
(3.15) Finally, integrating each term of (3.15) by parts [2] yields (3.12).
i
(
)=
N
q z; t
t
ij
0
j =1 j =i
6
+
i
(
j
;
q z; t x; R R
(
) ( i
;
)
q x; s q z; t x; s dx ds R R
0)q (x; 0) dx; i
1 i N: (3.21)
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 11, NOVEMBER 1999
Proof: Let q~i (z; t) denote the right-hand side of (3.21). By Ito’s rule,
dq~i (z; t)
=
N
j =1 j 6=i
+ =
j =1 j 6=i
0
+
t N
R
qj (x;
R
+ h0 (ei ; x)N 01 j =1 j 6=i
j =1 j 6=i
ij
+
qi (z;
j =1 j 6=i
qj (x; s)dqi (z; t; x; s)dx ds
R
dxt = F xt dt + G dwt ; x0 2 R dyt = Hxt dt + N 1=2 dbt ; y0 = 0 2 R:
t; x; s) dx ds dt t
0
R
t; x;
qj (x; s)qi (z; t; x; s) dx ds dys
0)qi (x;
t; x;
0) dx dt
0)qi (x;
IV. APPLICATION OF FINITE-DIMENSIONAL FILTERING TO FAULT DETECTION Consider (without loss of generality) the one-dimensional state and observation processes fxt ; yt gt0 of (2.2) and (2.3), respectively, described by
ij (Li + ii )
qi (z;
R
R
0
R
qi (z; t; x; 0)qi (x; 0) dx dt N j =1 j 6=i
ij
R
t
0
R
zt =
ij q~j (z; t) dt + (Li + ii )~ qi (z; t) dt
+ q~i (z;
t)h0 (ei ; x)N 01 dyt :
Thus, q~i (1; 1) satisfies (3.12) for (z; t) 2 Rn 2 [0; T ) and, for t = 0; q~i (z; 0) = qi (z; 0), which implies the validity of (3.21). Theorem 3.5: Consider the jump linear system
dXt = 5Xt dt + dMt ; dxt = A(Xt )xt dt + G(Xt )dwt ; dyt = H (Xt )xt dt + N 1=2 dbt ; x(0) 2 Rn ; y(0) = 0 2 Rd : If the Q matrix 5 is either lower or upper diagonal (e.g., correspond^ t and x^t are finite-dimensional ing to a counting process), then X computable. Proof: If 5 is lower diagonal, then q 1 (z; t) is unnormalized Gaussian. Moreover, the equation for q 2 (z; t) is coupled with that
(4.2)
0; 1;
if t < T if t T .
(4.3)
Then the faults can be expressed as
F = F1 (1 0 zt ) + F2 zt G = G1 (1 0 zt ) + G2 zt H = H1 (1 0 zt ) + H2 zt :
(4.4)
Furthermore, assume that T is an exponential random variable with constant parameter 0, independent of x0 ; fwt ; bt gt0 . As mentioned in [11], we can view fzt gt0 as being a Poisson process with rate stopped at its first jump time T , and having an associated martingale Mt = zt 0 min(t; T ). Then fzt gt0 satisfies the equation
dzt = (1 0 zt ) dt + dMt :
Also, for x
qj (x; s)qi (z; t; x; s) dx ds
qi (z; t; x; 0)qi (x; 0) dx dyt
(4.1)
Here, x0 is a Gaussian random variable. Assume that a fault occurs at time T , at which time F; G; H change from F1 to F2 , G1 to G2 , H1 to H2 , respectively. Following [11], let
0) dx dyt
qj (x; s)qi (z; t; x; s) dx ds
+ N
j =1 j 6=i
s)qi (z;
t
+ h0 (ei ; x)N 01
=
0
of q 1 (z; t) and, consequently, by Lemma 3.4, q 2 (z; t) is also finite-dimensional computable. A similar argument holds for the remaining densities. The case of upper diagonal matrices is treated similarly.
ij qj (z; t) dt + (Li + ii ) N
1
t
ij
N
ij h0 (ei ; x)N 01
j =1 j 6=i
N
j =1 j 6=i
ij qj (z; t) dt +
+ (Li + ii ) =
N
dqi (z; t; x; 0)qi (x; 0) dx
R
N
1
ij qj (z; t)dt +
2167
= 0 only (because zt stops at its first jump), P (T t) = 1 0 e0t 1 P (z = yjz = x) Pxy (t) = t 0 0t ; e x = 1 0 e0t ; ifif yy = = 1 0 x.
(4.5)
(4.6)
Define the infinitesimal generator of fzt gt0 by
1 [E ( (zt )jz0 = x) 0 (x)]: A (x) =1 lim t#0 t
Then
1 A (x) = lim (y)P (zt = yjz0 = x) 0 (x) t#0 t y = lim 1 [ (x)e0t + (1 0 x)(1 0 e0t ) 0 (x)]: t#0
t
Thus,
A (x) = [ (1 0 x) 0 (x)]:
(4.7)
fzt gt0 is a pure birth process; therefore, Pxy (t) = < x; 8 t. The transition probabilities can be computed from
Notice that
0 8y
the backward equation
dPxy (t) = APxy (t) = [P(10x)y (t) 0 Pxy (t)] (4.8) dt with P00 (0) = 1; P01 (0) = 0; P10 (t) = 0 8 t; P11 (t) = 1 8 t, or
from (4.6) directly, namely,
dPxy (0+ ) dt
= xy ;
(x; y) 2 f0; 1g:
(4.9)
2168
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 11, NOVEMBER 1999
Hence, the probability rates are given by the matrix 00 3 = 10
01 11
= 00 0
:
From Lemma 3.4, we obtain t (4.10)
Next, we find an equivalent representation for fzt gt0 in terms of the process fXt gt0 . Introduce the bijective mapping 0 70! e1 ; 1 70! e2 , e.g., Xt = (Xt1; Xt2 )0 = (1 0 zt ; zt )0 . Then
dXt =
0 0
Xt dt +
01
(4.11) 1 dMt that is, the Q matrix 5 is given by 5 = 30 . Moreover, (4.1) and
0
(4.2) are expressed as
dxt dyt
x0 2 R y0 = 0 2 R (4.12)
= F (Xt )xt dt + G(Xt ) dwt ; = H (Xt )xt dt + N 1=2 dbt ;
where
F (ei ) = Fi ;
G(ei ) = Gi ;
H (ei ) = Hi ;
i = 1; 2: (4.13)
From Theorem 3.1, the unnormalized conditional densities are obtained from the equations
q2 (z; t) =
(4.14)
where
(4.19)
3^ s;2 t (x) = s;2 t exp 0 21 (xSs;2 t x 0 2x2s; t )
where
1 exp 2t=
s;
where x ^2 :
(4.20)
[0; T ] 2 2 R ! R, P 2 : [0; T ] ! R, 3^ 2 : [0; T ] 2
2 R ! R are given by dx^2t (x) = F (e2 )^ x2t (x) dt + Pt2 H (e2 )N 01 1 (dyt 0 H (e2 )^x2t (x)dt); x^20 = x P_t2 = F (e2 )Pt2 + Pt2 F (e2 ) 0 Pt2 H (e2 )N 01 1 H (e2 )Pt2 + G(e2 )G(e2 ); P02 = 0 t H (e2 )^ x2s (x)N 01 dys 3^ 02; t (x) = exp 0 t 0 21 N 01 (H (e2 )^x2s (x))2 ds : 0
(4.21) (4.22)
s
t s t s
(4.26)
2 H (e2 )N 01 dy s;
2 H (e2 )N 01 H (e2 ) s;2 d s;
t
82s; H (e2 )N 01 H (e2 )8s;2 d
t
82s; H (e2 )N 01 [dy 0 H (e2 ) s;2 d ]:
Notice that 2 ; 2 ; S 2 , and 2 are independent of x. Also, anticipating the need to compute R zq 2 (z; t) dz in the numerator of (4.29)
1
below, we introduce the vector v = [x z ]0 in (4.24) and complete the square in v . Thus, after some algebra, we obtain t 1 ^ 01; s s;2 t q2 (z; t) = e0s 3 0 2(P01; s Ps;2 t )1=2
1 exp 0 12 0 14 Tr(A)(A01B)0 A01 B + f ds 1 exp 0 12 Tr A v + 12 A01 B 0 (v + 12 A01 B) R
+ where
R
q2 (z; t; x;
0)q2 (x; 0) dx
a e=2 e=2 b 1 c B= d
1 A=
1 (P 1 )01 + (P 2 )01 (82 )2 + S 2 a= 0; s s; t s; t s; t 1 (P 2 )01 b= s; t 1 c=
1 d= (4.23)
s
1 2s; t =
0)
= (2)1=21(P 2 )1=2 exp 0 12 (Pt2 )01 (z 0 x^2t (x))2 3^ 02; t (x) t
0 12
1 2t= Ss;
The fundamental solution of (4.15) is
q2 (z; t; x;
(4.25)
In addition, the solution to (4.23) can be written as
q1 (x; t) =
0
+
8_ 2s; t = [F (e2 ) 0 Ps;2 t H (e2 )N 01 H (e2 )]8s;2 t 2 t = F (e2 ) s; t dt + Ps;2 t H (e2 )N 01 d s; 1 (dyt 0 H (e2 ) s;2 t dt); s;2 s = 0:
The explicit solution of (4.14) is
0
q1 (x; s)q2 (z; t; x; s) dx ds
2t x^2s; t (x) = 82s; t x + s;
(4.15)
1 1 1 01 12 (2)1=2 (Pt1 )1=2 exp 0 2 (Pt ) (x 0 x^t ) 1 exp(0t)3^ 01; t (4.16) where x ^1 : [0; T ] 2 ! R, P 1 : [0; T ] ! R, P 1 0, 3^ 1 : [0; T ] 2
! R are given by dx^1t = F (e1 )^ x1t dt + Pt1 H (e1 )N 01 (dyt 0 H (e1 )^ x1t dt); x^10 (4.17) P_ t1 = F (e1 )Pt1 + Pt1 F (e1 ) 0 Pt1 H (e1 )N 01 H (e1 )Pt1 + G(e1 )G(e1 ); P01 (4.18) t t H (e1 )^ x1s N 01 dys 0 12 N 01 (H (e1 )^x1s )2 ds : 3^ 01; t = exp
R
q2 (z; t; x; 0)q2 (x; 0) dx (4.24) R where q 2 (x; 0) is Gaussian and, consequently, the second integral in (4.24) results in a Gaussian density given by (4.16) with = 0 and by x ^1t ; Pt1 ; 3^ t1 replaced, respectively, with x^2t ; Pt2 ; 3^ t2 which satisfy (4.21)–(4.23), with x = x ^2 (0) and P02 = P 2 (0); note that this integral could also be computed by completing the square in x. Notice next that the integration with respect to the spatial variable x in the double integral on the right side of (4.24) can be computed by completing the square in x. Toward this end, we proceed as follows. We note from (4.20) that q 2 (z; t; x; s) depends on x ^2s; t (x) and 3^ s;2 t (x) for the nonzero initial time s. Therefore, the solution to (4.21) can be written as
1 @ @2 dq1 (x; t) = G(e1 )2 2 q1 (x; t) dt 0 (F (e1 )xq1 (x; t)) dt 2 @x @x 0 q1 (x; t) dt + H (e1 )N 01 q1 (x; t) dyt; q1 (x; 0) 1 @ @2 dq2 (x; t) = G(e2 )2 2 q2 (x; t) dt 0 (F (e2 )xq2 (x; t)) dt 2 @x @x + q1 (x; t) dt + H (e2 )N 01 q2 (x; t) dyt ; q2 (x; 0):
0
1 e=
0 2(P01; s )01x^10; s + 2(Ps;2 t )01 8s;2 t s;2 t 0 22s; t 0 2(Ps;2 t )01 s;2 t 0 2(Ps;2 t )01 8s;2 t
1 (P 1 )01 (^x1 )2 + (P 2 )01 ( 2 )2 : f= 0; s 0; s 0; s s; t
dx (4.27)
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 11, NOVEMBER 1999
Writing (4.27) symbolically as t
q2 (z; t) = (tv)ds (sv)dx + 0 R 1 1 where tv = time variables and sv =
1
R
q2 (z; t; x; 0)q2 (x; 0)dx
space variables given by
0s 1 2 tv = 2 )1=2 e 3^ 0; s s; t 2(P01;s Ps;t 1 exp 0 12 0 14 Tr(A)(A01B)0 A01 B + f sv = exp 0 12 Tr A v + 12 A01 B 0 v + 12 A01 B
we then deduce that
t
q2 (z; t) =
0
;
1 exp 0 12 Tr(A) z + 12 (A01 B)2 2 ds
q2 (z; t; x; 0)q2 (x; 0) dx (4.28) R where (A01 B )2 denotes the second component of A01 B . Hence, the representation for q 1 (z; t); q 2 (z; t) can be computed explicitly. The filtered estimates are obtained from
+
x^t = j =12
R
zqj (z; t) dz
j =1 R
2
j =1
X^ t = 2
(4.29)
qj (z; t) dz
ej jt
j =1
it
jt
qi (z; t) dz; i = 1; 2: (4.30) R The first term in the numerator of (4.29) follows from (4.16) and is given by
=
1
1
1
zq (z; t) dz = x^t exp( t)3^ 0; t : R The second term follows from (4.24) and the discussion that follows it; thus, we find that R which implies R
0
z (sv) dx dz = 0(2=Tr(A))(A01 B )2 =2
zq2 (z; t) dz
= 0
t
0
(tv)(2=Tr(A))(A01 B )2 )=2 ds + x^2t 3^ 02; t :
Finally, for the it terms in the denominator of (4.29) and ^ 01; t follows from (4.16), and 2t = (4.30), 1t = exp(0t)3 (2=Tr(A)) 0t (tv) ds + 3^ 02; t follows from R (sv) dx dz = 2=Tr(A) and the discussion after (4.24) that implies
3^ 02; t =
q2 (z; t; x; 0)q2 (x; 0) dx dz:
models of this type where Xt can have more than two states and xt is nonscalar, no new concepts are needed. For example, the Q matrix associated with a pure birth three-state process Xt would involve at most five nonzero ij ’s, a four-state process Xt would involve at most 11 nonzero ij ’s, etc. Because of the nonscalar xt , the Gaussian densities involve vectors and matrices, but normalizations and the computation of mean values are still the only evaluations required. Extensions of this work to control diffusions should be investigated in the context of optimal control. REFERENCES
(tv)(2=Tr(A))1=2
2
2169
R In view of the above, both x ^t and X^ t are explicitly computed. As one can see, the above manipulations are no more complicated than using normalization and computing mean values for one- and two-dimensional Gaussian densities. Furthermore, for more general
[1] J. Hibey and C. Charalambous, “Performance analysis for a changepoint problem,” IEEE Trans. Automat. Contr., vol. 44, pp. 1628–1632, Aug. 1999. [2] E. Wong and B. Hajek, Stochastic Processes in Engineering Systems. New York: Springer-Verlag, 1985. [3] R. Elliott, F. Dufour, and D. Sworder, “Exact hybrid filters in discrete time,” IEEE Trans. Automat. Contr., vol. 41, pp. 1807–1810, 1996. [4] R. Rishel, “Tracking the output of a poorly known Markov process,” Stochastics and Stochastics Rep., pp. 147–156, 1995. [5] V. Benes, K. Helmes, and R. Rishel, “Pursuing a manuevering target which uses a random process for its control,” IEEE Trans. Automat. Contr., vol. 40, pp. 307–311, Feb. 1995. [6] H. Kunita, Stochastic Flows and Stochastic Differential Equations. New York: Cambridge Univ. Press, 1990. [7] C. Charalambous and R. Elliott, “Certain nonlinear stochastic optimal control problems with explicit control laws equivalent to LEQG/LQG problems,” IEEE Trans. Automat. Contr., vol. 42, pp. 482–497, Apr. 1997. [8] C. Charalambous, “Partially observable nonlinear risk-sensitive control problems: Dynamic programming and verification theorems,” IEEE Trans. Automat. Contr., vol. 42, pp. 1130–1138, Aug. 1997. [9] R. Elliott, L. Aggoun, and J. Moore, Hidden Markov Models Estimation and Control. New York: Springer-Verlag, 1995. [10] V. E. Benes and I. Karatzas, “On the relation of Zakai’s and Mortensen’s equations,” SIAM J. Contr. Optimiz., vol. 21, no. 3, pp. 472–489, 1983. [11] M. Davis, “The application of nonlinear filtering to fault detection in linear systems,” IEEE Trans. Automat. Contr., pp. 257–259, 1975.
Asymptotic Convergence from
Stability
Andrew R. Teel Abstract— This note recalls that an absolutely continuous function having a uniformly locally integrable (not necessarily essentially bounded) derivative is uniformly continuous on the semi-infinite interval. This observation, in conjunction with Barbalat’s lemma, allows concluding asymptotic convergence to zero of an output function for a general class of nonlinear systems with Lp (not necessarily L1 ) disturbances. Index Terms—Asymptotic convergence, Barbalat’s lemma.
I. INTRODUCTION A tool often used for concluding that signals converge to zero in time-varying nonlinear control systems is Barbalat’s lemma (see [8, Manuscript received April 7, 1998. Recommended by Associate Editor, A. Rantzer. This work was supported in part by the National Science Foundation under Grant ECS-9896140 and by the Air Force Office of Scientific Research under Grant F49620-98-1-0087. The author is with the Department of Electrical and Computer Engineering, University of California at Santa Barbara, Santa Barbara, CA 93106 USA (email: [email protected]). Publisher Item Identifier S 0018-9286(99)08586-4.
0018–9286/99$10.00 1999 IEEE
