Output-Feedback Stochastic Nonlinear Stabilization - Miroslav Krstic
328
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 2, FEBRUARY 1999
Output-Feedback Stochastic Nonlinear Stabilization Hua Deng and Miroslav Krsti´c
Abstract—The authors present the first result on global output-feedback stabilization (in probability) for stochastic nonlinear continuous-time systems. The class of systems that they consider is a stochastic counterpart of the broadest class of deterministic systems for which globally stabilizing controllers are currently available. Their controllers are “inverse optimal” and possess an infinite gain margin. A reader of the paper needs no prior familiarity with techniques of stochastic control.
where 2 IRn is the state, w is an r-dimensional independent standard Wiener process, and f : IRn ! IRn and g : IRn ! IRn2r are locally Lipschitz and satisfy f (0) = 0, g (0) = 0. Definition 1.1: The equilibrium = 0 of (1) is said to be globally asymptotically stable in probability if for any t0 0 and > 0, lim (t )!0 P fsuptt j (t)j > g = 0, and for any initial condition (t0 ), P flimt!1 (t) = 0g = 1. Theorem 1.1—Khas’minskii [15], Kushner [17], and Mao [18]: Consider system (1) and suppose there exists a positive definite, radially unbounded, twice continuously differentiable function V ( ) such that the infinitesimal generator
LV ( ) = @V f + 1 Tr gT @ V g @ 2 @ 2
Index Terms—Backstepping, control Lyapunov functions, inverse optimality, stochastic nonlinear output-feedback systems, stochastic stabilization.
2
is negative definite. Then the equilibrium asymptotically stable in probability.
I. INTRODUCTION Despite huge popularity of the linear-quadratic-Gaussian control problem, the stabilization problem for nonlinear stochastic systems has been receiving relatively little attention until recently. Efforts toward (global) stabilization of stochastic nonlinear systems have been initiated in the work of Florchinger [4]–[6] who, among other things, extended the concept of control Lyapunov functions and Sontag’s stabilization formula [25] to the stochastic setting. A breakthrough toward arriving at constructive methods for stabilization of broader classes of stochastic nonlinear systems came with the result of Pan and Basar [22] who derived a backstepping design for strict-feedback systems motivated by a risk-sensitive cost criterion [1], [11], [20], [24] (for other types of optimal control problems, see, e.g., [9] and [10]). In [2] and [3], we designed simpler inverse optimal control laws for strict-feedback systems which guarantee global asymptotic stability in probability and whose algorithms can be directly coded in symbolic software. In this paper, we address the output-feedback global stabilization problem for stochastic nonlinear systems. The output-feedback problem has received considerable attention in the recent robust and adaptive nonlinear control literature [12], [14], [16], [19], [23], [26]. The present paper is the first to address the output-feedback problem in the stochastic setting. We present two results. First, in Section II, we design an outputfeedback (observer-based) backstepping control law which guarantees global asymptotic stability in probability. Second, in Section III, based on a theorem derived in [3], we design stabilizing control laws which are also optimal with respect to meaningful cost functionals. The class of systems that we consider is the stochastic version of the output-feedback form, which is the broadest class for which global output-feedback controllers currently exist in the deterministic setting. Finally, in Section IV, we give a second-order simulation example.
(1)
Manuscript received February 21, 1997; revised June 23, 1997. Recommended by Associate Editor, G. G. Yin. This work was supported in part by the National Science Foundation under Grant ECS-951011-8461 and in part by the Air Force Office of Scientific Research under Grant F496209610223. The authors are with the Department of AMES, University of California at San Diego, La Jolla, CA 92093-0411 USA (e-mail: [email protected]). Publisher Item Identifier S 0018-9286(99)01284-2.
of (1) is globally
II. OUTPUT-FEEDBACK STABILIZATION IN PROBABILITY
where 'i (y ) are r-vector-valued smooth functions with 'i (0) = 0, and w is an independent r-dimensional standard Wiener process. Since the states x2 ; 1 1 1 ; xn are not measured, we first design an observer which would provide exponentially convergent estimates of the unmeasured states in the absence of noise. The observer is designed as
x^_ i = x^i+1 + ki (y 0 x^1 ); i = 1; 1 1 1 ; n where x ^n+1 = u. The observation errors x ~ = x0x ^ satisfy 0k1 I dx~ = ... x~ dt + '(y)T dw 0kn 0 1 1 1 0 T = A0 x ~ dt + '(y ) dw where A0 is designed to be asymptotically stable. Now, the
(4)
(5) entire
system can be expressed as
dx~ = A0 x~ dt + '(y)T dw dy = (^x2 + x~2 ) dt + '1 (y)T dw dx^2 = [^x3 + k2 (y 0 x^1 )] dt .. .
dx^n = [u + kn (y 0 x^1 )] dt:
Consider the nonlinear stochastic system
= 0
In this section we deal with nonlinear output-feedback systems driven by white noise. This class of systems is given by the following nonlinear stochastic differential equations: i = 1; 1 1 1 ; n 0 1 dxi = xi+1 dt + 'i (y)T dw; dxn = u dt + 'n (y)T dw (3) y = x1
A. Preliminaries on Stability in Probability
d = f ( ) dt + g( ) dw
(2)
(6)
Our output-feedback design will consist of applying a backstepping procedure to the system (y; x ^2 ; 1 1 1 ; x ^n ), which also takes care of the feedback connection through the x ~ system. In the standard backstepping method for deterministic systems [7] (where dw=dt would be a bounded deterministic disturbance), a sequence of stabilizing functions i (^ xi ; y), where x^i = [^x2 ; 1 1 1 ; x^i ]T , is constructed recursively to build a Lyapunov function of the form n 1 2 V= z + x~T P x~ (7) 2 i i=1
0018–9286/99$10.00 1999 IEEE
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 2, FEBRUARY 1999
where P is a positive definite matrix which satisfies 0I , and the error variables zi are given by
z1 = y zi = x^i 0 i01 (^xi01 ; y );
329
AT0 P + P A0 =
i = 2; 1 1 1 ; n:
(8) (9)
The Lyapunov design for stochastic systems cannot be performed using the quadratic Lyapunov function (7) because of the term 1 TrfgT (@ 2 V=@ 2 )gg in (2). We instead employ quartic (fourth2 order) Lyapunov functions
V
n =
i=1
T 2 1 4 4 zi + (~x P x~) :
(10)
Our presentation of the backstepping procedure here is very concise: instead of introducing the stabilizing functions i in a stepby-step fashion, we derive them simultaneously. A reader who is a novice to the technique of backstepping is referred to [16]. We start by an important preparatory comment. Since 'i (0) = 0, the i ’s will vanish at x ^i = 0, y = 0, as well as at z i = 0, where z i = [z1 ; 1 1 1 ; zi ]T . Thus, by the mean value theorem, i (^xi ; y) and '(y) can be expressed, respectively, as i
i (^xi ; y) =
zl il (^xi ; l=1 '(y) = y (y)
y)
(11)
Fig. 1. Feedback structure of the system (6).
=
(13)
2 0 @ @yi01 (^x2 + x~2 ) 0 21 @ @y i201 '1 (y)T '1 (y) dt (14) 0 @ @yi01 '1 (y)T dw; i = 2; 1 1 1 ; n:
As we announced previously, we employ a Lyapunov function of a quartic form
V (z; x~) = 1 y4 + 1 4
4
n i=2
zi4 + b (~xT P x~)2 2
i=2
zi3 x^i+1 + ki x~1 0
i01
@ i01 (^x + k x~ ) @ x^l l+1 l 1 l=2
2 i01 T 0 @ @yi01 (^x2 + x~2 ) 0 21 @ @y 2 '1 (y) '1 (y)
2 zi2 @ i01 '1 (y)T '1 (y) 0 bx~T P x~jx~j2 2 @y i=2 T T T + 2b Trf'(y )(2P x ~x ~ P +x ~ Px ~P )'(y ) g +
3
n
+y
(15)
LV =y3 (^x2 + x~2 ) + 32 y2 '1 (y)T '1 (y) n
3 1 + 3 1 (y)T 1 (y)y + 3 4=3 y + 3 4=3 y 2 4 1 4 1 +
where b is a positive constant. This form of the Lyapunov function clearly indicates that we view the system as a feedback connection in Fig. 1. The first two terms in (15) constitute a Lyapunov function for the (y; x ^2 ; 1 1 1 ; x ^n )-system, while the third term in (15) is a Lyapunov function for the x ~-system. Even though not obvious from the calculations that follow, we achieve a nonlinear small-gain global stabilization (in probability) in the style of [13]. xi ; y) to Now we start the process of selecting the functions i (^ make LV negative definite. Along the solutions of (5), (13), and (14), we have
+
3 y2 '1 (y)T '1 (y) 2 i01 n zi3 i + zi+1 + ki x~1 0 @ i01 (^xl+1 + kl x~1 ) + @ x^l i=2 l=2 2 i01 T 0 @ @yi01 (^x2 + x~2 ) 0 21 @ @y 2 '1 (y) '1 (y) n 2 3 + zi2 @ i01 '1 (y)T '1 (y) 2 @y i=2 n 0 b 0 3bnpn22 jP j4 0 41 14 0 414 jx~j4 1 i=2 i 3
~2 ) + + y ( 1 + z2 + x
(12)
where il (^ xi ; y) and (y) are smooth functions. Now, we are ready to start the backstepping design procedure. According to Itˆo’s differentiation rule [21], we have
dz1 = (^x2 + x~2 ) dt + '1 (y)T dw i01 @ i01 (^x + k x~ ) dzi = x^i+1 + ki x~1 0 @ x^l l+1 l 1 l=2
0 bx~T P x~jx~j2 + 2b Trf'(y)(2P x~x~T P + x~T P x~P )'(y)T g
n01
3 4
n i=2
p
i2 ( 1 (y)T 1 (y))2 y + 3bn2 n j (y)j4 y i01
2
@ i01 (^x + k x~ ) @ x^l l+1 l 1 i=2 l=2 2 0 @ @yi01 x^2 0 21 @ @y i201 '1 (y)T '1 (y) + 34 i4=3 zi 4=3 1 3 4=3 @ i01 + z + zi i i 4i401 4 @y 3 @ i01 4 z + 2 i 4i @y n01 @ n01 (^x + k x~ ) 3 + zn u + kn x ~1 0 l+1 l 1 @ x^l l=2 2 n01 1 T 0 @ @yn01 x^2 0 21 @ @y 2 '1 (y) '1 (y) + 4n4 01 zn 4=3 4 3 4=3 @ n01 + zn + 32 @ n01 zn (16) n 4 @y 4n @y +
zi3 i + ki x~1 0
where > 0 is the smallest eigenvalue of P . The second equality comes from substituting x ^i = zi + i01 , and the inequality comes from Young’s inequalities in Appendix A. At this point, we can see that all the terms can be cancelled by u and i . If we choose 1 ,
330
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 2, FEBRUARY 1999
2 ,
and i to satisfy
Theorem 3.1 [3]: Consider the control law
p b 0 3bn n2 jP j4 0 1 2
and
i
and
u
4
n i=2
1
i4
0 44 = p > 0 1
1
(17)
as
1 = 0c1 y 0 32 1 (y)T 1 (y)y 0 34 14=3 y 0 34 41=3 y 0 34 p n 1 i2 ( 1 (y)T 1 (y))2 y 0 3bn2 n j (y)j4y (18) 2 i=2 i01 @ i01 (^x + k x~ ) i = 0ci zi 0 ki x~1 + @ x^l l+1 l 1 l=2 @ i01 x^ + 1 @ 2 i01 ' (y)T ' (y) 0 3 4=3 z + 1 i i @y 2 2 @y2 1 4 4=3 4 0 414 zi 0 43 i4=3 @ @yi01 zi 0 432 @ @yi01 zi i01 i (19)
n01
@ n01 (^x + k x~ ) l+1 l 1 @ x^l l=2 @ n01 x^ + 1 @ 2 n01 ' (y)T ' (y) 0 1 z + 1 4 01 n @y 2 2 @y2 1 4n 4=3 4 0 43 n4=3 @ @yn01 zn 0 432 @ @yn01 zn n
u = 0cn zn 0 kn x~1 +
n i=1
ci zi4 0 pjx~j4 :
(20)
(21)
This section first reviews some definitions and theorems established in [3], which are then used in the design of an inverse optimal stabilizing control law. Consider the system (22)
where f (0) = 0, g1 (0) = 0, and u 2 Definition 3.1 [3]: The problem of inverse optimal stabilization in probability for system (22) is solvable if there exist a class K1 function1 2 whose derivative 20 is also a class K1 function, a matrix-valued function R2 ( ) such that R2 ( ) = R2 ( )T > 0 for all , a positive definite radially unbounded function l( ), and a feedback control law u = ( ) continuous away from the origin with (0) = 0, which guarantees global asymptotic stability in probability of the equilibrium = 0 and minimizes the cost functional m IR .
J (u) = E
0
!
1=2 uj)] d : [l( ) + 2 (jR2 ( )
function : IR+ IR+ is said to be in class strictly increasing, and limr!1 (r ) = . 1A
1
0 01:
( 2 )
(25)
If the control law (24) achieves global asymptotic stability in probability for the system (22) with respect to V ( ), then the control law
u3 = 3 ( ) R01 (L V )T ( 20 )01 (jLg V R201=2 j) ; = 0 g 2 2 jLg V R201=2j
2
(26)
solves the problem of inverse optimal stabilization in probability for the system (22) by minimizing the cost functional
1
0
l( ) + 2 2
2
jR21=2 uj d
2 l( ) = 2 ` 2 (jLg V R201=2j) 0 Lf V 0 1 Tr g1T @ V2 g1 2 @ 0 1=2 + ( 0 2)` 2 (jLg V R j): 2
III. INVERSE OPTIMAL OUTPUT-FEEDBACK STABILIZATION
1
` 2 =
(27)
where2
With (21), we have the following stability result. Theorem 2.1: The equilibrium at the origin of the closed-loop stochastic system (6), (20) is globally asymptotically stable in probability.
d = f ( ) dt + g1 ( ) dw + g2 ( )u dt
the Legandre–Fenchel transform defined as
J (u) = E
where ci > 0, then the infinitesimal generator of the closed-loop system (5), (13), (14), and (20) is negative definite
LV 0
01=2 u = ( ) = 0R201 (Lg V )T ` 2 (jLg V 0R12=2 2 j) (24) jLg V R2 j where V ( ) is a Lyapunov function candidate, 2 is a class K1 function whose derivative is also a class K1 function, R2 ( ) is a matrix-valued function such that R2 ( ) = R2 ( )T > 0, and ` 2 is
(23)
K1 if it is continuous,
(28)
Now we return to the output-feedback system (3) and redesign the control law (20) to make it inverse optimal. The following result is instrumental. Corollary 3.1 [3]: If there exists a continuous positive function M (y; x^) such that the control law
u = (y; x^) = 0M (y; x^)zn
(29)
globally asymptotically stabilizes the system (6) in probability with respect to the Lyapunov function (15), then the control law
u3 = 3 (y; x^) = (y; x^);
4 3
(30)
solves the problem of inverse optimal stabilization in probability. From Corollary 3.1, we know that if we can design a stabilizing control law that has zn as a factor, we can easily find another control law which solves the problem of inverse optimal stabilization in probability. If we consider carefully the last bracket of (16), every term except the second, the third, the fourth, and the fifth has zn as a factor. With the help of Young’s inequalities in Appendix B, we have
LV 0 b 0 3bnpn22 jP j4 0 41
n
1
4 i=2 i
0 414 0 414 0 414 kn4 jx~j4 1 4 3 +y
3
+
3 4
3 4=3 3 4=3 1 (y )y + 4 1 y + 4 1 y
T 1 (y )
1 + 32 n
i=2
1 + 447
i2 (
y+
pn
3bn T 2 1 (y ) 1 (y )) y + 22
1 845
j (y)j4y
y
2 The function l( ) is positive definite because, by assumption of the theorem, the bracketed term is positive definite, ` 2 (1) is in class K1 , and 2.
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 2, FEBRUARY 1999
n01
i01
@ i01 (^x + k x~ ) @ x^l l+1 l 1 i=2 l=2 2 i01 ' (y)T ' (y)+ 3 4=3 z 0 @ @yi01 x^2 0 12 @ @y 1 1 i i 2 4 4=3 1 + zi + 3 i4=3 @ i01 zi 4i401 4 @y 3 @ i01 4 z + 1 z + 1 z + i i i 4i2 @y 446 447 n01 4=3 3 3 + zn u + 6 @ n01 zn 4 @ x^l l=2 +
zi3 i + ki x~1 0
4=3
6 @ n01 zn + 14 zn 4 @y 46 4=3 n01 n01 3 @ n01 + 7 zn 4 @ x^l lk k=1 l=k +
+
+
+
3
3 8 3 4
4=3 2 5 @ n201 1 (y)T 1 (y) zn @y 4=3 n01 @ n01 4 kl zn + 3 43=3 zn + 41 zn @ x ^l 4 4n01 l=2 4=3
n4=3 @ n01 4 @y
3
zn +
3
4n 2
@ n01 @y
4
331
Theorem 3.2: The control law
u3 = 0 M (y; x^)zn ;
1 , 2 , 3 , 4 , 5 , 6 , 7 , and i
J (u) = E
1
0
p b 0 3bn n jP j 0 1 4 2 2
n
4
i4
i=2
1
4 1
1
4 4
4 3
4
4 7
1
446
where
c1
+ +
1
8
4 5
1
447
=
x^_ 1 = x^2 + k1 (y 0 x^1 ) x^_ 2 = u + k2 (y 0 x^1 ): The virtual control
c1
(33)
2
(34)
2
(35)
n01
4=3
3 6 @ n01 + 6 @ n01 @ x ^l 4 @y l=2 4 =3 n01 n01 1 3 @ n01 + + 7 446 4 @ x^l lk k=1 l=k n01 4=3 2 3 3 @ n01 k + 5 @ n201 + 4 8 @y 4 @ x^l l l=2
M (y; x^) = cn + 3
4=3
4
+ +
3 4
43=3 +
3
2 4n
1
4n01 4
@ n01 @y
+
n4=3 @ n01 4 @y 3
4=3
1 2
(36)
i=1
ci zi4 0 pjx~j4 :
and control
u
are
p 0 c y 0 83 y 0 43 = y 0 34 = y 0 643 y 0 6162b y 1
4 3 1
3
4 3 1
2 5 2
2 2
5
(42)
2 u = 0 c2 z2 0 k2 x~1 + @ 1 x^2 + 1 @ 21 y4 0 14 z2 @y 8 @y 41 4=3 4 1 0 43 24=3 @ z2 0 32 @ 1 z2 : (43) @y 42 @y We choose k1 = 3, k2 = 4:5, c1 = 0:01, c2 = 0:1, 1 = 0:1, 2 = 0:8, 1 = 0:01, 2 = 0:1, b = 0:1, 2 = 50, and set the initial condition at x1 (0) = 1:3, x2 (0) = 0, x ^1 (0) = 0, x ^2 (0) = 1 (0),
the states and control of the system are shown in Fig. 2. From Fig. 2, we can see that the output converges to zero. It is also interesting to note how the solutions become less noisy as they approach zero—a consequence of the fact that the noise vector field vanishes at zero. APPENDIX A
4
n
1 =
1
(41)
4=3
with (18), (19), and (35), we get
LV 0
(40)
For this system, the estimator is
ci =
0 M (y; x^)zn
(39)
dx1 = x2 dt + 12 x21 dw dx2 = u dt y = x1 :
4
and ci are those in (18) and (19), and
u=
M (y; x^)03 u4 d
27
16 2
IV. EXAMPLE
(32) 1
l(x; x~) +
We give a second-order example to illustrate Theorem 2.1. Consider the system
0 4 0 4 0 4 kn = p > 0 1
1
(38)
for some positive definite radially unbounded function l(x; x ~) parameterized by . Proof: Let 2 (r) = 14 r4 , R2 = ( 43 M )0(3=2) . Applying Theorem 3.1, the result follows readily. Remark 3.1: The inverse optimal control law has infinite upper gain margin and lower gain margin of 75% because u = ku3 is globally asymptotically stabilizing for k 2 [ 43 ; 1). The function l(x; x~) is lower bounded by 2 ni=1 ci zi4 + 4pjx~j4 (which means that it is a positive definite and radially unbounded function of x and x~).
zn :
are chosen to satisfy
4 3
guarantees that the equilibrium at the origin of the system (3), (5) is globally asymptotically stable in probability and also minimizes the cost functional
(31) If
(37)
Thus, according to Corollary 3.1, we achieve not only global asymptotic stability in probability, but also inverse optimality.
In this and the following Appendix, we use Young’s inequality [8, Th. 156] p xy jxjp + 1q jyjq (A.1) p q where > 0, the constants p > 1 and q > 1 satisfy (p01)(q 01) = 1; and (x; y ) 2 IR2 . Applying these inequalities leads to
y3 z2 3 14=3 y4 + 4
1
414
z24
(A.2)
332
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 2, FEBRUARY 1999
(a)
(b) Fig. 2. The states and control effort of the output-feedback system.
y 3 x~2 3 41=3 y4 + 14 x~42 4 41 3 4=3 4 4 1 y + 414 jx~j4 (A.3) 1 n01 n n 1 4 zi3 zi+1 3 i4=3 zi4 + 1 (A.4) 4i01 zi 4 4 i=2 i=2 i=3 n n n 4=3 0 zi3 @ @yi01 x~2 34 i4=3 @ @yi01 zi4 + 41 14 x~42 i=2 i=2 i=2 i
34
n 4=3 1 4 i4=3 @ i01 zi4 + 1 4i jx~j @y 4 i=2 i=2 n
(A.5)
2 zi2 @ i01 '1 (y)T '1 (y) 2 @y i=2 n n 4 34 12 @ @yi01 zi4 + 43 i2 ('1 (y)T '1 (y))2 i=2 i i=2 T T T 2b Trf'(y )(2P x ~x ~ P +x ~ Px ~P )'(y ) g 2bnj'(y)(2P x~x~T P + x~T P x~P )'(y)T j1 2bnpnj'(y)(2P x~x~T P + x~T P x~P )'(y)T j 6bnppny2 j (y)j2 jP j2 jx~j2 (cf. (12)) 3bn2 n y4 j (y)j4 + 3bnpn22 jP j4jx~j4 2 3
n
(A.6)
(A.7)
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 2, FEBRUARY 1999
where the chosen.
’s, ’s, ’s,
and
’s
are positive constants to be
[3] [4]
APPENDIX B
[5]
Similar to Appendix A, in the following inequalities, ’s are constants to be chosen:
[6]
zn3 kn x~1 3 43=3 zn4 + 14 kn4 x~41 3 43=3 zn4 + 14 kn4 jx~j4 4 43 4 43 n01 0zn3 @ @nx^0l 1 kl x~1 l=2 4=3 n01 34 4 @ @nx^0l 1 kl zn4 + 414 x~41 4 l=2 4 = 3 n01 34 4 @ @nx^0l 1 kl zn4 + 414 jx~j4 4 l=2 2 1 3 @ n01 T 0 2 zn @y2 '1 (y) '1 (y) 2 1 3 @ n01 T = 0 zn (y ) 1 (y)y2 2 @y2 1 4=3 2 n01 38 5 @ @y zn4 + 14 y4 1 (y)T 1 (y) 2 85 n01 0zn3 @ @yn01 x^2 0 zn3 @ @nx^0l 1 x^l+1 l=2 n01 @ n01 z 0 z 3 @ n01 z 3 = 0zn @ x^l l+1 n @y 2 l=2 n01 l @ n01 z 0 zn3 (cf. (9), (11)) @ x^l k lk l=1 k=1 n01 =0 zn3 @ n01 zl+1 0 zn3 @ n01 z2 @ x^l @y l=2 n01 n01 @ n01 z 3 z 0 @ x^l lk n k k=1 l=k n01 4=3 3 6 @ n01 zn4 + 14 zl4+1 4 @ x ^l 46 l=2 n01 4=3 3 + 6 @ n01 zn4 + 14 z24 + 4 @y 46 k=1 4 =3 n01 1 34 7 @ @nx^0l 1 lk zn4 + 414 zk4 7 l=k n01 4 =3 4=3 3 4 3 = zn 6 @ n01 + 6 @ n01 4 @ x^l 4 @y l=2 4=3 n01 n01 1 3 @ n01 + + 7 446 4 @ x^l lk k=1 l=k n01 n01 1 1 + zi4 + zi4 + 14 y4 : 4 46 447 47 i=2 i=2
(B.1)
[8] [9] [10] [11]
(B.2)
[12] [13] [14]
(B.3)
[15] [16] [17] [18] [19] [20] [21] [22] [23] [24]
[25] [26]
(B.4)
REFERENCES [1] T. Ba¸sar and P. Bernhard, 1 -Optimal Control and Related Minimax Design Problems: A Dynamic Game Approach, 2nd ed. Boston, MA: Birkh¨auser, 1995. [2] H. Deng and M. Krsti´c, “Stochastic nonlinear stabilization—Part I: A backstepping design,” Syst. Contr. Lett., vol. 32, pp. 143–150, 1997.
H
[7]
333
, “Stochastic nonlinear stabilization—Part II: Inverse optimality,” Syst. Contr. Lett., vol. 32, pp. 151–159, 1997. P. Florchinger, “Lyapunov-like techniques for stochastic stability,” SIAM J. Contr. Optim., vol. 33, pp. 1151–1169, 1995. , “Global stabilization of cascade stochastic systems,” in Proc. 34th Conf. Decision and Control, New Orleans, LA, 1995, pp. 2185–2186. , “A universal formula for the stabilization of control stochastic differential equations,” Stochastic Analysis and Appl., vol. 11, pp. 155–162, 1993. R. A. Freeman and P. V. Kokotovi´c, Robust Nonlinear Control Design: State-Space and Lyapunov Techniques. Boston, MA: Birkh¨auser, 1996. G. Hardy, J. E. Littlewood, and G. Polya, Inequalities, 2nd ed. Cambridge, U.K.: Cambridge Univ. Press, 1989. U. G. Haussmann and W. Suo, “Singular optimal stochastic controls—I: Existence,” SIAM J. Contr. Optim., vol. 33, pp. 916–936, 1995. , “Singular optimal stochastic controls—II: Dynamic programming,” SIAM J. Contr. Optim., vol. 33, pp. 937–959, 1995. M. R. James, J. Baras, and R. J. Elliott, “Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems,” IEEE Trans. Automat. Contr., vol. 39, pp. 780–792, 1994. M. Jankovic, “Adaptive nonlinear output feedback tracking with a partial high-gain observer and backstepping,” IEEE Trans. Automat. Contr., vol. 42, pp. 106–113, Jan. 1997. Z. P. Jiang, A. R. Teel, and L. Praly, “Small-gain theorem for ISS systems and applications,” Math. Contr., Signals, Syst., vol. 7, pp. 95–120, 1995. H. K. Khalil, “Adaptive output feedback control of nonlinear systems represented by input–output models,” IEEE Trans. Automat. Contr., vol. 41, pp. 177–188, Feb. 1996. R. Z. Khas’minskii, Stochastic Stability of Differential Equations. Rockville, MD: S & N, 1980. M. Krsti´c, I. Kanellakopoulos, and P. V. Kokotovi´c, Nonlinear and Adaptive Control Design. New York: Wiley, 1995. H. J. Kushner, Stochastic Stability and Control. New York: Academic, 1967. X. Mao, Stability of Stochastic Differential Equations with Respect to Semimartingales. Longman, 1991. R. Marino and P. Tomei, Nonlinear Control Design: Geometric, Adaptive, and Robust. Englewood Cliffs, NJ: Prentice-Hall, 1995. H. Nagai, “Bellman equations of risk-sensitive control,” SIAM J. Contr. Optim., vol. 34, pp. 74–101, 1996. B. Øksendal, Stochastic Differential Equations—An Introduction with Applications. New York: Springer-Verlag, 1995. Z. Pan and T. Ba¸sar, “Backstepping controller design for nonlinear stochastic systems under a risk-sensitive cost criterion,” SIAM J. Contr. Optim., to be published. L. Praly and Z. P. Jiang, “Stabilization by output feedback for systems with ISS inverse dynamics,” Syst. Contr. Lett., vol. 21, pp. 19–33, July 1993. T. Runolfsson, “The equivalence between infinite horizon control of stochastic systems with exponential-of-integral performance index and stochastic differential games,” IEEE Trans. Automat. Contr., vol. 39, pp. 1551–1563, 1994. E. D. Sontag, “A ‘universal’ construction of Artstein’s theorem on nonlinear stabilization,” Syst. Contr. Lett., vol. 13, pp. 117–123, 1989. A. Teel and L. Praly, “Tools for semiglobal stabilization by partial state and output feedback,” SIAM J. Contr. Optim., vol. 33, pp. 1443–1488, Sept. 1995.
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 2, FEBRUARY 1999
Output-Feedback Stochastic Nonlinear Stabilization Hua Deng and Miroslav Krsti´c
Abstract—The authors present the first result on global output-feedback stabilization (in probability) for stochastic nonlinear continuous-time systems. The class of systems that they consider is a stochastic counterpart of the broadest class of deterministic systems for which globally stabilizing controllers are currently available. Their controllers are “inverse optimal” and possess an infinite gain margin. A reader of the paper needs no prior familiarity with techniques of stochastic control.
where 2 IRn is the state, w is an r-dimensional independent standard Wiener process, and f : IRn ! IRn and g : IRn ! IRn2r are locally Lipschitz and satisfy f (0) = 0, g (0) = 0. Definition 1.1: The equilibrium = 0 of (1) is said to be globally asymptotically stable in probability if for any t0 0 and > 0, lim (t )!0 P fsuptt j (t)j > g = 0, and for any initial condition (t0 ), P flimt!1 (t) = 0g = 1. Theorem 1.1—Khas’minskii [15], Kushner [17], and Mao [18]: Consider system (1) and suppose there exists a positive definite, radially unbounded, twice continuously differentiable function V ( ) such that the infinitesimal generator
LV ( ) = @V f + 1 Tr gT @ V g @ 2 @ 2
Index Terms—Backstepping, control Lyapunov functions, inverse optimality, stochastic nonlinear output-feedback systems, stochastic stabilization.
2
is negative definite. Then the equilibrium asymptotically stable in probability.
I. INTRODUCTION Despite huge popularity of the linear-quadratic-Gaussian control problem, the stabilization problem for nonlinear stochastic systems has been receiving relatively little attention until recently. Efforts toward (global) stabilization of stochastic nonlinear systems have been initiated in the work of Florchinger [4]–[6] who, among other things, extended the concept of control Lyapunov functions and Sontag’s stabilization formula [25] to the stochastic setting. A breakthrough toward arriving at constructive methods for stabilization of broader classes of stochastic nonlinear systems came with the result of Pan and Basar [22] who derived a backstepping design for strict-feedback systems motivated by a risk-sensitive cost criterion [1], [11], [20], [24] (for other types of optimal control problems, see, e.g., [9] and [10]). In [2] and [3], we designed simpler inverse optimal control laws for strict-feedback systems which guarantee global asymptotic stability in probability and whose algorithms can be directly coded in symbolic software. In this paper, we address the output-feedback global stabilization problem for stochastic nonlinear systems. The output-feedback problem has received considerable attention in the recent robust and adaptive nonlinear control literature [12], [14], [16], [19], [23], [26]. The present paper is the first to address the output-feedback problem in the stochastic setting. We present two results. First, in Section II, we design an outputfeedback (observer-based) backstepping control law which guarantees global asymptotic stability in probability. Second, in Section III, based on a theorem derived in [3], we design stabilizing control laws which are also optimal with respect to meaningful cost functionals. The class of systems that we consider is the stochastic version of the output-feedback form, which is the broadest class for which global output-feedback controllers currently exist in the deterministic setting. Finally, in Section IV, we give a second-order simulation example.
(1)
Manuscript received February 21, 1997; revised June 23, 1997. Recommended by Associate Editor, G. G. Yin. This work was supported in part by the National Science Foundation under Grant ECS-951011-8461 and in part by the Air Force Office of Scientific Research under Grant F496209610223. The authors are with the Department of AMES, University of California at San Diego, La Jolla, CA 92093-0411 USA (e-mail: [email protected]). Publisher Item Identifier S 0018-9286(99)01284-2.
of (1) is globally
II. OUTPUT-FEEDBACK STABILIZATION IN PROBABILITY
where 'i (y ) are r-vector-valued smooth functions with 'i (0) = 0, and w is an independent r-dimensional standard Wiener process. Since the states x2 ; 1 1 1 ; xn are not measured, we first design an observer which would provide exponentially convergent estimates of the unmeasured states in the absence of noise. The observer is designed as
x^_ i = x^i+1 + ki (y 0 x^1 ); i = 1; 1 1 1 ; n where x ^n+1 = u. The observation errors x ~ = x0x ^ satisfy 0k1 I dx~ = ... x~ dt + '(y)T dw 0kn 0 1 1 1 0 T = A0 x ~ dt + '(y ) dw where A0 is designed to be asymptotically stable. Now, the
(4)
(5) entire
system can be expressed as
dx~ = A0 x~ dt + '(y)T dw dy = (^x2 + x~2 ) dt + '1 (y)T dw dx^2 = [^x3 + k2 (y 0 x^1 )] dt .. .
dx^n = [u + kn (y 0 x^1 )] dt:
Consider the nonlinear stochastic system
= 0
In this section we deal with nonlinear output-feedback systems driven by white noise. This class of systems is given by the following nonlinear stochastic differential equations: i = 1; 1 1 1 ; n 0 1 dxi = xi+1 dt + 'i (y)T dw; dxn = u dt + 'n (y)T dw (3) y = x1
A. Preliminaries on Stability in Probability
d = f ( ) dt + g( ) dw
(2)
(6)
Our output-feedback design will consist of applying a backstepping procedure to the system (y; x ^2 ; 1 1 1 ; x ^n ), which also takes care of the feedback connection through the x ~ system. In the standard backstepping method for deterministic systems [7] (where dw=dt would be a bounded deterministic disturbance), a sequence of stabilizing functions i (^ xi ; y), where x^i = [^x2 ; 1 1 1 ; x^i ]T , is constructed recursively to build a Lyapunov function of the form n 1 2 V= z + x~T P x~ (7) 2 i i=1
0018–9286/99$10.00 1999 IEEE
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 2, FEBRUARY 1999
where P is a positive definite matrix which satisfies 0I , and the error variables zi are given by
z1 = y zi = x^i 0 i01 (^xi01 ; y );
329
AT0 P + P A0 =
i = 2; 1 1 1 ; n:
(8) (9)
The Lyapunov design for stochastic systems cannot be performed using the quadratic Lyapunov function (7) because of the term 1 TrfgT (@ 2 V=@ 2 )gg in (2). We instead employ quartic (fourth2 order) Lyapunov functions
V
n =
i=1
T 2 1 4 4 zi + (~x P x~) :
(10)
Our presentation of the backstepping procedure here is very concise: instead of introducing the stabilizing functions i in a stepby-step fashion, we derive them simultaneously. A reader who is a novice to the technique of backstepping is referred to [16]. We start by an important preparatory comment. Since 'i (0) = 0, the i ’s will vanish at x ^i = 0, y = 0, as well as at z i = 0, where z i = [z1 ; 1 1 1 ; zi ]T . Thus, by the mean value theorem, i (^xi ; y) and '(y) can be expressed, respectively, as i
i (^xi ; y) =
zl il (^xi ; l=1 '(y) = y (y)
y)
(11)
Fig. 1. Feedback structure of the system (6).
=
(13)
2 0 @ @yi01 (^x2 + x~2 ) 0 21 @ @y i201 '1 (y)T '1 (y) dt (14) 0 @ @yi01 '1 (y)T dw; i = 2; 1 1 1 ; n:
As we announced previously, we employ a Lyapunov function of a quartic form
V (z; x~) = 1 y4 + 1 4
4
n i=2
zi4 + b (~xT P x~)2 2
i=2
zi3 x^i+1 + ki x~1 0
i01
@ i01 (^x + k x~ ) @ x^l l+1 l 1 l=2
2 i01 T 0 @ @yi01 (^x2 + x~2 ) 0 21 @ @y 2 '1 (y) '1 (y)
2 zi2 @ i01 '1 (y)T '1 (y) 0 bx~T P x~jx~j2 2 @y i=2 T T T + 2b Trf'(y )(2P x ~x ~ P +x ~ Px ~P )'(y ) g +
3
n
+y
(15)
LV =y3 (^x2 + x~2 ) + 32 y2 '1 (y)T '1 (y) n
3 1 + 3 1 (y)T 1 (y)y + 3 4=3 y + 3 4=3 y 2 4 1 4 1 +
where b is a positive constant. This form of the Lyapunov function clearly indicates that we view the system as a feedback connection in Fig. 1. The first two terms in (15) constitute a Lyapunov function for the (y; x ^2 ; 1 1 1 ; x ^n )-system, while the third term in (15) is a Lyapunov function for the x ~-system. Even though not obvious from the calculations that follow, we achieve a nonlinear small-gain global stabilization (in probability) in the style of [13]. xi ; y) to Now we start the process of selecting the functions i (^ make LV negative definite. Along the solutions of (5), (13), and (14), we have
+
3 y2 '1 (y)T '1 (y) 2 i01 n zi3 i + zi+1 + ki x~1 0 @ i01 (^xl+1 + kl x~1 ) + @ x^l i=2 l=2 2 i01 T 0 @ @yi01 (^x2 + x~2 ) 0 21 @ @y 2 '1 (y) '1 (y) n 2 3 + zi2 @ i01 '1 (y)T '1 (y) 2 @y i=2 n 0 b 0 3bnpn22 jP j4 0 41 14 0 414 jx~j4 1 i=2 i 3
~2 ) + + y ( 1 + z2 + x
(12)
where il (^ xi ; y) and (y) are smooth functions. Now, we are ready to start the backstepping design procedure. According to Itˆo’s differentiation rule [21], we have
dz1 = (^x2 + x~2 ) dt + '1 (y)T dw i01 @ i01 (^x + k x~ ) dzi = x^i+1 + ki x~1 0 @ x^l l+1 l 1 l=2
0 bx~T P x~jx~j2 + 2b Trf'(y)(2P x~x~T P + x~T P x~P )'(y)T g
n01
3 4
n i=2
p
i2 ( 1 (y)T 1 (y))2 y + 3bn2 n j (y)j4 y i01
2
@ i01 (^x + k x~ ) @ x^l l+1 l 1 i=2 l=2 2 0 @ @yi01 x^2 0 21 @ @y i201 '1 (y)T '1 (y) + 34 i4=3 zi 4=3 1 3 4=3 @ i01 + z + zi i i 4i401 4 @y 3 @ i01 4 z + 2 i 4i @y n01 @ n01 (^x + k x~ ) 3 + zn u + kn x ~1 0 l+1 l 1 @ x^l l=2 2 n01 1 T 0 @ @yn01 x^2 0 21 @ @y 2 '1 (y) '1 (y) + 4n4 01 zn 4=3 4 3 4=3 @ n01 + zn + 32 @ n01 zn (16) n 4 @y 4n @y +
zi3 i + ki x~1 0
where > 0 is the smallest eigenvalue of P . The second equality comes from substituting x ^i = zi + i01 , and the inequality comes from Young’s inequalities in Appendix A. At this point, we can see that all the terms can be cancelled by u and i . If we choose 1 ,
330
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 2, FEBRUARY 1999
2 ,
and i to satisfy
Theorem 3.1 [3]: Consider the control law
p b 0 3bn n2 jP j4 0 1 2
and
i
and
u
4
n i=2
1
i4
0 44 = p > 0 1
1
(17)
as
1 = 0c1 y 0 32 1 (y)T 1 (y)y 0 34 14=3 y 0 34 41=3 y 0 34 p n 1 i2 ( 1 (y)T 1 (y))2 y 0 3bn2 n j (y)j4y (18) 2 i=2 i01 @ i01 (^x + k x~ ) i = 0ci zi 0 ki x~1 + @ x^l l+1 l 1 l=2 @ i01 x^ + 1 @ 2 i01 ' (y)T ' (y) 0 3 4=3 z + 1 i i @y 2 2 @y2 1 4 4=3 4 0 414 zi 0 43 i4=3 @ @yi01 zi 0 432 @ @yi01 zi i01 i (19)
n01
@ n01 (^x + k x~ ) l+1 l 1 @ x^l l=2 @ n01 x^ + 1 @ 2 n01 ' (y)T ' (y) 0 1 z + 1 4 01 n @y 2 2 @y2 1 4n 4=3 4 0 43 n4=3 @ @yn01 zn 0 432 @ @yn01 zn n
u = 0cn zn 0 kn x~1 +
n i=1
ci zi4 0 pjx~j4 :
(20)
(21)
This section first reviews some definitions and theorems established in [3], which are then used in the design of an inverse optimal stabilizing control law. Consider the system (22)
where f (0) = 0, g1 (0) = 0, and u 2 Definition 3.1 [3]: The problem of inverse optimal stabilization in probability for system (22) is solvable if there exist a class K1 function1 2 whose derivative 20 is also a class K1 function, a matrix-valued function R2 ( ) such that R2 ( ) = R2 ( )T > 0 for all , a positive definite radially unbounded function l( ), and a feedback control law u = ( ) continuous away from the origin with (0) = 0, which guarantees global asymptotic stability in probability of the equilibrium = 0 and minimizes the cost functional m IR .
J (u) = E
0
!
1=2 uj)] d : [l( ) + 2 (jR2 ( )
function : IR+ IR+ is said to be in class strictly increasing, and limr!1 (r ) = . 1A
1
0 01:
( 2 )
(25)
If the control law (24) achieves global asymptotic stability in probability for the system (22) with respect to V ( ), then the control law
u3 = 3 ( ) R01 (L V )T ( 20 )01 (jLg V R201=2 j) ; = 0 g 2 2 jLg V R201=2j
2
(26)
solves the problem of inverse optimal stabilization in probability for the system (22) by minimizing the cost functional
1
0
l( ) + 2 2
2
jR21=2 uj d
2 l( ) = 2 ` 2 (jLg V R201=2j) 0 Lf V 0 1 Tr g1T @ V2 g1 2 @ 0 1=2 + ( 0 2)` 2 (jLg V R j): 2
III. INVERSE OPTIMAL OUTPUT-FEEDBACK STABILIZATION
1
` 2 =
(27)
where2
With (21), we have the following stability result. Theorem 2.1: The equilibrium at the origin of the closed-loop stochastic system (6), (20) is globally asymptotically stable in probability.
d = f ( ) dt + g1 ( ) dw + g2 ( )u dt
the Legandre–Fenchel transform defined as
J (u) = E
where ci > 0, then the infinitesimal generator of the closed-loop system (5), (13), (14), and (20) is negative definite
LV 0
01=2 u = ( ) = 0R201 (Lg V )T ` 2 (jLg V 0R12=2 2 j) (24) jLg V R2 j where V ( ) is a Lyapunov function candidate, 2 is a class K1 function whose derivative is also a class K1 function, R2 ( ) is a matrix-valued function such that R2 ( ) = R2 ( )T > 0, and ` 2 is
(23)
K1 if it is continuous,
(28)
Now we return to the output-feedback system (3) and redesign the control law (20) to make it inverse optimal. The following result is instrumental. Corollary 3.1 [3]: If there exists a continuous positive function M (y; x^) such that the control law
u = (y; x^) = 0M (y; x^)zn
(29)
globally asymptotically stabilizes the system (6) in probability with respect to the Lyapunov function (15), then the control law
u3 = 3 (y; x^) = (y; x^);
4 3
(30)
solves the problem of inverse optimal stabilization in probability. From Corollary 3.1, we know that if we can design a stabilizing control law that has zn as a factor, we can easily find another control law which solves the problem of inverse optimal stabilization in probability. If we consider carefully the last bracket of (16), every term except the second, the third, the fourth, and the fifth has zn as a factor. With the help of Young’s inequalities in Appendix B, we have
LV 0 b 0 3bnpn22 jP j4 0 41
n
1
4 i=2 i
0 414 0 414 0 414 kn4 jx~j4 1 4 3 +y
3
+
3 4
3 4=3 3 4=3 1 (y )y + 4 1 y + 4 1 y
T 1 (y )
1 + 32 n
i=2
1 + 447
i2 (
y+
pn
3bn T 2 1 (y ) 1 (y )) y + 22
1 845
j (y)j4y
y
2 The function l( ) is positive definite because, by assumption of the theorem, the bracketed term is positive definite, ` 2 (1) is in class K1 , and 2.
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 2, FEBRUARY 1999
n01
i01
@ i01 (^x + k x~ ) @ x^l l+1 l 1 i=2 l=2 2 i01 ' (y)T ' (y)+ 3 4=3 z 0 @ @yi01 x^2 0 12 @ @y 1 1 i i 2 4 4=3 1 + zi + 3 i4=3 @ i01 zi 4i401 4 @y 3 @ i01 4 z + 1 z + 1 z + i i i 4i2 @y 446 447 n01 4=3 3 3 + zn u + 6 @ n01 zn 4 @ x^l l=2 +
zi3 i + ki x~1 0
4=3
6 @ n01 zn + 14 zn 4 @y 46 4=3 n01 n01 3 @ n01 + 7 zn 4 @ x^l lk k=1 l=k +
+
+
+
3
3 8 3 4
4=3 2 5 @ n201 1 (y)T 1 (y) zn @y 4=3 n01 @ n01 4 kl zn + 3 43=3 zn + 41 zn @ x ^l 4 4n01 l=2 4=3
n4=3 @ n01 4 @y
3
zn +
3
4n 2
@ n01 @y
4
331
Theorem 3.2: The control law
u3 = 0 M (y; x^)zn ;
1 , 2 , 3 , 4 , 5 , 6 , 7 , and i
J (u) = E
1
0
p b 0 3bn n jP j 0 1 4 2 2
n
4
i4
i=2
1
4 1
1
4 4
4 3
4
4 7
1
446
where
c1
+ +
1
8
4 5
1
447
=
x^_ 1 = x^2 + k1 (y 0 x^1 ) x^_ 2 = u + k2 (y 0 x^1 ): The virtual control
c1
(33)
2
(34)
2
(35)
n01
4=3
3 6 @ n01 + 6 @ n01 @ x ^l 4 @y l=2 4 =3 n01 n01 1 3 @ n01 + + 7 446 4 @ x^l lk k=1 l=k n01 4=3 2 3 3 @ n01 k + 5 @ n201 + 4 8 @y 4 @ x^l l l=2
M (y; x^) = cn + 3
4=3
4
+ +
3 4
43=3 +
3
2 4n
1
4n01 4
@ n01 @y
+
n4=3 @ n01 4 @y 3
4=3
1 2
(36)
i=1
ci zi4 0 pjx~j4 :
and control
u
are
p 0 c y 0 83 y 0 43 = y 0 34 = y 0 643 y 0 6162b y 1
4 3 1
3
4 3 1
2 5 2
2 2
5
(42)
2 u = 0 c2 z2 0 k2 x~1 + @ 1 x^2 + 1 @ 21 y4 0 14 z2 @y 8 @y 41 4=3 4 1 0 43 24=3 @ z2 0 32 @ 1 z2 : (43) @y 42 @y We choose k1 = 3, k2 = 4:5, c1 = 0:01, c2 = 0:1, 1 = 0:1, 2 = 0:8, 1 = 0:01, 2 = 0:1, b = 0:1, 2 = 50, and set the initial condition at x1 (0) = 1:3, x2 (0) = 0, x ^1 (0) = 0, x ^2 (0) = 1 (0),
the states and control of the system are shown in Fig. 2. From Fig. 2, we can see that the output converges to zero. It is also interesting to note how the solutions become less noisy as they approach zero—a consequence of the fact that the noise vector field vanishes at zero. APPENDIX A
4
n
1 =
1
(41)
4=3
with (18), (19), and (35), we get
LV 0
(40)
For this system, the estimator is
ci =
0 M (y; x^)zn
(39)
dx1 = x2 dt + 12 x21 dw dx2 = u dt y = x1 :
4
and ci are those in (18) and (19), and
u=
M (y; x^)03 u4 d
27
16 2
IV. EXAMPLE
(32) 1
l(x; x~) +
We give a second-order example to illustrate Theorem 2.1. Consider the system
0 4 0 4 0 4 kn = p > 0 1
1
(38)
for some positive definite radially unbounded function l(x; x ~) parameterized by . Proof: Let 2 (r) = 14 r4 , R2 = ( 43 M )0(3=2) . Applying Theorem 3.1, the result follows readily. Remark 3.1: The inverse optimal control law has infinite upper gain margin and lower gain margin of 75% because u = ku3 is globally asymptotically stabilizing for k 2 [ 43 ; 1). The function l(x; x~) is lower bounded by 2 ni=1 ci zi4 + 4pjx~j4 (which means that it is a positive definite and radially unbounded function of x and x~).
zn :
are chosen to satisfy
4 3
guarantees that the equilibrium at the origin of the system (3), (5) is globally asymptotically stable in probability and also minimizes the cost functional
(31) If
(37)
Thus, according to Corollary 3.1, we achieve not only global asymptotic stability in probability, but also inverse optimality.
In this and the following Appendix, we use Young’s inequality [8, Th. 156] p xy jxjp + 1q jyjq (A.1) p q where > 0, the constants p > 1 and q > 1 satisfy (p01)(q 01) = 1; and (x; y ) 2 IR2 . Applying these inequalities leads to
y3 z2 3 14=3 y4 + 4
1
414
z24
(A.2)
332
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 2, FEBRUARY 1999
(a)
(b) Fig. 2. The states and control effort of the output-feedback system.
y 3 x~2 3 41=3 y4 + 14 x~42 4 41 3 4=3 4 4 1 y + 414 jx~j4 (A.3) 1 n01 n n 1 4 zi3 zi+1 3 i4=3 zi4 + 1 (A.4) 4i01 zi 4 4 i=2 i=2 i=3 n n n 4=3 0 zi3 @ @yi01 x~2 34 i4=3 @ @yi01 zi4 + 41 14 x~42 i=2 i=2 i=2 i
34
n 4=3 1 4 i4=3 @ i01 zi4 + 1 4i jx~j @y 4 i=2 i=2 n
(A.5)
2 zi2 @ i01 '1 (y)T '1 (y) 2 @y i=2 n n 4 34 12 @ @yi01 zi4 + 43 i2 ('1 (y)T '1 (y))2 i=2 i i=2 T T T 2b Trf'(y )(2P x ~x ~ P +x ~ Px ~P )'(y ) g 2bnj'(y)(2P x~x~T P + x~T P x~P )'(y)T j1 2bnpnj'(y)(2P x~x~T P + x~T P x~P )'(y)T j 6bnppny2 j (y)j2 jP j2 jx~j2 (cf. (12)) 3bn2 n y4 j (y)j4 + 3bnpn22 jP j4jx~j4 2 3
n
(A.6)
(A.7)
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 44, NO. 2, FEBRUARY 1999
where the chosen.
’s, ’s, ’s,
and
’s
are positive constants to be
[3] [4]
APPENDIX B
[5]
Similar to Appendix A, in the following inequalities, ’s are constants to be chosen:
[6]
zn3 kn x~1 3 43=3 zn4 + 14 kn4 x~41 3 43=3 zn4 + 14 kn4 jx~j4 4 43 4 43 n01 0zn3 @ @nx^0l 1 kl x~1 l=2 4=3 n01 34 4 @ @nx^0l 1 kl zn4 + 414 x~41 4 l=2 4 = 3 n01 34 4 @ @nx^0l 1 kl zn4 + 414 jx~j4 4 l=2 2 1 3 @ n01 T 0 2 zn @y2 '1 (y) '1 (y) 2 1 3 @ n01 T = 0 zn (y ) 1 (y)y2 2 @y2 1 4=3 2 n01 38 5 @ @y zn4 + 14 y4 1 (y)T 1 (y) 2 85 n01 0zn3 @ @yn01 x^2 0 zn3 @ @nx^0l 1 x^l+1 l=2 n01 @ n01 z 0 z 3 @ n01 z 3 = 0zn @ x^l l+1 n @y 2 l=2 n01 l @ n01 z 0 zn3 (cf. (9), (11)) @ x^l k lk l=1 k=1 n01 =0 zn3 @ n01 zl+1 0 zn3 @ n01 z2 @ x^l @y l=2 n01 n01 @ n01 z 3 z 0 @ x^l lk n k k=1 l=k n01 4=3 3 6 @ n01 zn4 + 14 zl4+1 4 @ x ^l 46 l=2 n01 4=3 3 + 6 @ n01 zn4 + 14 z24 + 4 @y 46 k=1 4 =3 n01 1 34 7 @ @nx^0l 1 lk zn4 + 414 zk4 7 l=k n01 4 =3 4=3 3 4 3 = zn 6 @ n01 + 6 @ n01 4 @ x^l 4 @y l=2 4=3 n01 n01 1 3 @ n01 + + 7 446 4 @ x^l lk k=1 l=k n01 n01 1 1 + zi4 + zi4 + 14 y4 : 4 46 447 47 i=2 i=2
(B.1)
[8] [9] [10] [11]
(B.2)
[12] [13] [14]
(B.3)
[15] [16] [17] [18] [19] [20] [21] [22] [23] [24]
[25] [26]
(B.4)
REFERENCES [1] T. Ba¸sar and P. Bernhard, 1 -Optimal Control and Related Minimax Design Problems: A Dynamic Game Approach, 2nd ed. Boston, MA: Birkh¨auser, 1995. [2] H. Deng and M. Krsti´c, “Stochastic nonlinear stabilization—Part I: A backstepping design,” Syst. Contr. Lett., vol. 32, pp. 143–150, 1997.
H
[7]
333
, “Stochastic nonlinear stabilization—Part II: Inverse optimality,” Syst. Contr. Lett., vol. 32, pp. 151–159, 1997. P. Florchinger, “Lyapunov-like techniques for stochastic stability,” SIAM J. Contr. Optim., vol. 33, pp. 1151–1169, 1995. , “Global stabilization of cascade stochastic systems,” in Proc. 34th Conf. Decision and Control, New Orleans, LA, 1995, pp. 2185–2186. , “A universal formula for the stabilization of control stochastic differential equations,” Stochastic Analysis and Appl., vol. 11, pp. 155–162, 1993. R. A. Freeman and P. V. Kokotovi´c, Robust Nonlinear Control Design: State-Space and Lyapunov Techniques. Boston, MA: Birkh¨auser, 1996. G. Hardy, J. E. Littlewood, and G. Polya, Inequalities, 2nd ed. Cambridge, U.K.: Cambridge Univ. Press, 1989. U. G. Haussmann and W. Suo, “Singular optimal stochastic controls—I: Existence,” SIAM J. Contr. Optim., vol. 33, pp. 916–936, 1995. , “Singular optimal stochastic controls—II: Dynamic programming,” SIAM J. Contr. Optim., vol. 33, pp. 937–959, 1995. M. R. James, J. Baras, and R. J. Elliott, “Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems,” IEEE Trans. Automat. Contr., vol. 39, pp. 780–792, 1994. M. Jankovic, “Adaptive nonlinear output feedback tracking with a partial high-gain observer and backstepping,” IEEE Trans. Automat. Contr., vol. 42, pp. 106–113, Jan. 1997. Z. P. Jiang, A. R. Teel, and L. Praly, “Small-gain theorem for ISS systems and applications,” Math. Contr., Signals, Syst., vol. 7, pp. 95–120, 1995. H. K. Khalil, “Adaptive output feedback control of nonlinear systems represented by input–output models,” IEEE Trans. Automat. Contr., vol. 41, pp. 177–188, Feb. 1996. R. Z. Khas’minskii, Stochastic Stability of Differential Equations. Rockville, MD: S & N, 1980. M. Krsti´c, I. Kanellakopoulos, and P. V. Kokotovi´c, Nonlinear and Adaptive Control Design. New York: Wiley, 1995. H. J. Kushner, Stochastic Stability and Control. New York: Academic, 1967. X. Mao, Stability of Stochastic Differential Equations with Respect to Semimartingales. Longman, 1991. R. Marino and P. Tomei, Nonlinear Control Design: Geometric, Adaptive, and Robust. Englewood Cliffs, NJ: Prentice-Hall, 1995. H. Nagai, “Bellman equations of risk-sensitive control,” SIAM J. Contr. Optim., vol. 34, pp. 74–101, 1996. B. Øksendal, Stochastic Differential Equations—An Introduction with Applications. New York: Springer-Verlag, 1995. Z. Pan and T. Ba¸sar, “Backstepping controller design for nonlinear stochastic systems under a risk-sensitive cost criterion,” SIAM J. Contr. Optim., to be published. L. Praly and Z. P. Jiang, “Stabilization by output feedback for systems with ISS inverse dynamics,” Syst. Contr. Lett., vol. 21, pp. 19–33, July 1993. T. Runolfsson, “The equivalence between infinite horizon control of stochastic systems with exponential-of-integral performance index and stochastic differential games,” IEEE Trans. Automat. Contr., vol. 39, pp. 1551–1563, 1994. E. D. Sontag, “A ‘universal’ construction of Artstein’s theorem on nonlinear stabilization,” Syst. Contr. Lett., vol. 13, pp. 117–123, 1989. A. Teel and L. Praly, “Tools for semiglobal stabilization by partial state and output feedback,” SIAM J. Contr. Optim., vol. 33, pp. 1443–1488, Sept. 1995.
