Viscosity solutions methods for singular perturbations in deterministic ...

Viscosity solutions methods for singular perturbations in deterministic and stochastic control  Olivier Alvarez Upresa 60-85, Universite de Rouen 76821 Mont-Saint Aignan cedex, France e-mail: [email protected] Martino Bardi y Dipartimento di Matematica P. e A., Universita di Padova, via Belzoni 7, 35131 Padova, Italy. e-mail: [email protected] February 2, 2000 Abstract

Viscosity solutions methods are used to pass to the limit in some penalization problems for rst order and second order, degenerate parabolic, Hamilton-Jacobi-Bellman equations. This characterizes the limit of the value functions of singularly perturbed optimal control problems for nonlinear deterministic systems and controlled degenerate diusions, respectively. The results cover also cases where the usual order reduction method does not give the correct limit, and diusion processes with fast state variables depending nonlinearly on the control. Some connections with ergodic control and periodic homogenization are discussed. Key words. singular perturbations, deterministic optimal control, stochastic optimal control, nonlinear systems, order reduction, viscosity solutions, Hamilton-Jacobi-Bellman equations, penalization, periodic homogenization, ergodic control, state constraints AMS subject classi cation. 49L25, 35B25, 93C73, 93E20, 35B27

Introduction In this paper we study three penalization problems for fully nonlinear partial dierential equations motivated by the optimal control theory for systems with dierent time scales. In all the problems the limit PDE is of lower dimension, and the limit operator is not obvious to guess. Problems of this kind were rst studied by R. Jensen and P.-L. Lions [28] for classical solutions of quasilinear uniformly elliptic PDEs. Here we study a rst order Hamilton-Jacobi equation and a degenerate parabolic, fully nonlinear, Hamilton-Jacobi-Bellman equation in the framework of viscosity solutions. The rst problem we consider is the limit as " ! 0+ of for the Hamiltonian

?@t u" + H (x; y; Dx u" ; Dy"u" ) = 0 in (0; T )  Rn  Y;

(1)

H (x; y; p; q) = max f?(p; f (x; y; )) ? (q; g(x; y; )) ? l(x; y; )g; 2A  This research was done within the TMR Project \Viscosity solutions and their applications" of the European Community. y Partially supported by M.U.R.S.T., project \Analisi e controllo di equazioni di evoluzione deterministiche e stocastiche."

1

where (
;
) denotes the scalar product and Y  Rm is open, bounded, connected, and smooth. This is the Hamilton-Jacobi-Bellman equation associated via Dynamic Programming to the minimization of the functional

J (t; x; y; 
) :=

Z

T

l(xs ; ys ; s ) ds + h(xT ; yT )

t

on the trajectories of the system

y_s = 1" g(xs ; ys ; s );

x_ s = f (xs ; ys ; s )

(2)

with xt = x, yt = y, where 
is the control function subject to s 2 A. Singular perturbation problems for deterministic controlled systems were studied by many authors, see e.g. the books by Kokotovic, Khalil & O'Reilly [31], Bensoussan [12], and Dontchev & Zolezzi [18], the recent articles by Artstein and Gaitsgory [22, 4, 3, 5], Veliov [38], Subbotina [37], the second author and Bagagiolo [6], and the references therein. We recall that the theory of singular perturbations has many important applications, in particular to the order reduction of large scale systems. As in [6] (see also [9, 7]) we assume the Hamiltonian H be coercive in the q = Dy u variables, which amounts to the complete controllability of the fast variables y of the system, and consider the boundary condition on @Y corresponding to the state-space constraint on the fast variables

ys 2 Y

for all t  s  T:

In [6] a separability assumptions on the controls acting on the fast and the slow variables yielded a simple explicit formula for the Hamiltonian H of the limit PDE. Here we show that in general the limit Hamiltonian H = H (x; p) is the unique constant such that the boundary value problem

H (x; y; p; Dy )  H in Y , H (x; y; p; Dy )  H in Y , has a viscosity solution  = (y) for xed (x; p). The existence and uniqueness of the eective Hamiltonian H was proved by Capuzzo-Dolcetta & Lions [16] in connection with ergodic control problems. We prove that the viscosity solution u" (t; x; y) of (1) with constrained boundary conditions on @Y and the terminal condition u" (T; x; y) = h(x; y) converges uniformly as " ! 0+ to the viscosity solution u = u(t; x) of ?@t u + H (x; Du) = 0 in (0; T )  Rn and u(T; x) = h(x) := infy h(x; y) for x 2 Rn : The eective Hamiltonian H admits a representation as the long time limit of the value function of a control problem in Y  Rm , see [16, 9]. This formula shows the strong connection between our result and the recent work of Artstein & Gaitsgory [5], even if they do not consider state constraints, make somewhat dierent assumptions, and use completely dierent methods. We also give a new representation of H as the Bellman Hamiltonian associated to a suitable set of \limiting" relaxed controls. This provides an interpretation of the limit u as the value function of an optimal control problem with n-dimensional state space, which is therefore the appropriate limit of the previous problem for the (n + m)-dimensional system (2) as " ! 0+. One might guess from (2) that in the limit the fast variables satisfy g(xs ; ys ; s )  0 and the Hamiltonian becomes

H0 (x; p) :=

sup

f(;y): g(x;y;)=0g

f?(p; f (x; y; ) ? l(x; y; )g:

This is indeed the case in many classical problems [31, 12] and we give some examples where

H = H0 . In general, however, H0 (x; p)  H (x; p) and the inequality can be strict when the

fast variables oscillate very rapidly in the limit. In this case we can pass to the limit because an averaging phenomenon occurs; this was studied for instance in [22, 4, 5, 39], see also the references therein for the earlier literature on averaging in ordinary dierential equations. Our main new contribution is a PDE approach to the problem, where the theory of viscosity solutions provides 2

many useful tools: the characterization of the eective Hamiltonian, the perturbed test function method of L. C. Evans [19, 20], and the relaxed semi-limits of Barles and Perthame [11, 9] that we slightly modify here. The viscosity solutions methods allow to treat in a very similar way our second and third problem. They are the limits as " ! 0+ of

u" n ?@t u" + H (x; y; Dxu" ; Dy"u" ; Dxxu" ; D"yy u" ; Dxy

" ) = 0 in (0; T )  R  Y; 2

2

2

2

for = 21 and = 1, respectively, where H (x; y; p; q; X; Y; Z ) = sup f? 21 [tr(T X ) ? tr( T Y ) ? tr(T Z ) ? tr(ZT )] 2A ?(p; f (x; y; )) ? (q; g(x; y; )) ? l(x; y; )g; and the coecients ; ; f; g; l are functions of (x; y; ). This is the fully nonlinear, degenerate parabolic, H-J-B equation arising in the minimization of the expectation E J (t; x; y; ) for the singularly perturbed controlled degenerate diusion process dx = f (x ; y ;  ) ds + (x ; y ;  ) dW ; dy = 1 g(x ; y ;  ) ds + 1  (x ; y ;  ) dW : (3) s

s s s

s s s

s

s

"

s s s

"

s s s

s

Problems of this nature with = 1=2 can be found in the book of Bensoussan [12] for dispersion matrices ;  that are not degenerate and independent of , and in Kushner's book [32] for possibly degenerate diusions but still independent of , and for uncontrolled fast drift g. The more recent papers by Kabanov & Runggaldier [30] and Kabanov & Pergamenshchikov [29], for < 1=2 and

= 1=2, respectively, allow the fast drift to depend linearly on the control whereas both dispersion matrices are uncontrolled. Here we allow all the terms of the fast dynamics to depend nonlinearly on the control. On the other hand we limit ourselves to the case of fast variables constrained on an m-dimensional torus, that is, Y = [0; 1]m, all the data are 1-periodic in the y variable, and periodic boundary conditions are imposed on @Y . In this case the existence, uniqueness, and representation of the eective (2nd order) Hamiltonian can be taken from a recent paper of Arisawa & Lions [2], and the simplicity of the boundary conditions reduces the technical diculties of the proof. Moreover we do not try here to represent the solution of the limit problem as a value function, nor to prove the convergence of nearly optimal controls. Among other technical conditions, we suppose that the terminal cost h is independent of y and make some mild restrictions on the slow dynamics. Concerning the assumptions on the fast dynamics, we shall make the one introduced by Arisawa and Lions [2] that guarantee some averaging behavior (ergodicity) in the fast dynamics. In the periodic setting, our results include the following three cases as well as several combinations. { The diusions (in the fast and slow variables) are uniformly non degenerate. { The problem is deterministic and the dynamical system in the fast variable is controllable. { The system in the fast variable is independent of x and y and satisfy the non-resonance condition of [2] (see Section 2 for a precise statement). The second case is the one covered in Section 1 in the state constraint setting. We remark that we have to make some nontrivial modi cations to the perturbed test function method of Evans [20] in order to avoid the crucial assumption in [20] that the Hamiltonian be uniformly continuous in all variables. For control problems this corresponds to the restrictive condition that the dynamics do not depend on the state variables. Section 2 is devoted to the classical scaling of (3) corresponding to = 1=2. Section 3 discusses the less usual case = 1. Our motivation to study this scaling is that, in the special case of f = g and  =  , it is equivalent to the problem of periodic homogenization. Therefore Section 3 extends in various ways previous results on periodic homogenization for uniformly elliptic, nondivergence form, quasilinear equations by Bensoussan, Boccardo, & Murat [13] and Evans [19, 20], and for rst order Hamilton-Jacobi equations by P.-L. Lions, Papanicolaou, & Varadhan [34] and Evans [20], see also the additional references on the viscosity solutions approach to homogenization in Section 3. 3

The main goal of this paper is to illustrate a uni ed PDE approach to singular perturbations for deterministic and stochastic systems and we do not pursue the minimal assumptions. We believe our method works for several other problems such as, for instance, deterministic systems under weaker controllability assumptions or with state constraints on the slow variables x as well, and stochastic systems with fast variables subject to more general state constraints or governed by a diusion re ected on @Y (giving rise to Neumann boundary conditions). We will come back to some of these problems in future papers. The rst application of viscosity solutions methods to singular perturbation problems in control goes back to P.-L. Lions' book [33] and more references can be found in [6]. To our knowledge the present paper is the rst using these methods for the 2nd order PDEs associated to controlled diusion processes.

1 Deterministic control with state constraints on the fast variables 1.1

The "?problem

Let T > 0 be xed. For every " > 0, we consider the control problem in (0; T ]  Rn  Rm with dynamics x_ s = f (xs ; ys ; s ) y_s = 1" g(xs ; ys ; s ); for s  t, with xt = x, yt = y, and the constraint on the fast variables ys 2 Y for all t  s  T; where Y  Rm is a given compact set. The control functions are measurable  : (0; T ] ! A, where A is a compact set, such that the corresponding trajectory satis es the state constraint; we denote this set with A(x;y). The value function is de ned on (0; T ]  Rn  Y by Z

u" (t; x; y) = 2Ainf f x;y)

(

t

T

l(xs ; ys ; s ) ds + h(xT ; yT )g:

Next we list our hypotheses. | A is a compact metric space. | Y  Rm is a bounded connected open set with Lipschitz boundary in the following sense: there exist  : Y ! Rm bounded and uniformly continuous and (4) c > 0 such that B (x + t(x); ct)  Y for all x 2 Y ; 0 < t  c: | The functions f , g, l, and h are continuous and bounded. | The functions f et g are Lipschitz continuous in (x; y) uniformly in ; the functions l and h are uniformly continuous in (x; y) uniformly in . | The problem is controllable in y, i.e., there exists r > 0 such that B (0; r)  convfg(x; y; ) j  2 Ag: We set

h(x; y): h(x) = inf y It is easy to see that under the preceding hypotheses h is uniformly continuous and bounded.

The last assumption is | A(x;y) 6= ; for all y 2 Y and u" is continuous in (0; T ]  Rn  Y . Remark The last assumption is not a consequence of the previous hypotheses on the data if the the boundary of Y has corners and g(x; y; A) is not convex, as it is easy to see on simple examples. However, it is automatically satis ed if the boundary is smooth, say C 2 , by a result of Soner (see [36] or Section IV.5 of [9]) , based on the \interior eld condition" min g(x; y; a)
n(y) < 0 for all y 2 @Y; x 2 Rn ; a2A 4

where n(y) is the exterior normal to Y at y, which holds in our case because of the controllability assumption on the fast variables. A more general sucient condition for the continuity of u" that allows for piecewise smooth @Y is the following: Y = fy 2 Rm j gi (y)  0 8i = 1; : : : ; pg (5) for some gi 2 C 1;1 (Rm ) with jDgi j > 0, i = 1; : : : ; p, and min max g(x; y; a)
Dgi (y) < 0 8y 2 @Y; x 2 Rn : (6) a2A fijgi (y)=0g

This is proved in Thm. A.1 of [6]. Note that (6) is automatically satis ed if g(x; y; A) is convex, in addition to the controllability assumption. In the general case of merely Lipschitz @Y a suitable formulation of the interior eld condition can be found in the paper of Ishii and Koike [26], where the continuity of the value function is proved for the in nite horizon problem.

Theorem 1 The value function u" is the unique viscosity solution in BC ((0; T ]  Rn  Y ) of the

terminal-boundary value problem 8 Dy u" n > < ?@t u" + H (x; y; Dx u" ; " )  0 in (0; T )  R  Y ; D u y " ?@t u" + H (x; y; Dx u" ; " )  0 in (0; T )  Rn  Y; > : u(T;
) = h in Rn  Y ; for the Hamiltonian H (x; y; p; q) = max f?(p; f (x; y; )) ? (q; g(x; y; )) ? l(x; y; )g: 2A

(HJ" )

The fact that a continuous value function satis es the appropriate H-J-B equation in viscosity sense is a standard consequence of the Dynamic Programming Principle, for the boundary condition on @Y due to the state constraint see [36] or [9]. The uniqueness can be proved by combining the proof of Thm. III.3.7 in [9] with Soner's argument on @Y , see [36] or Section IV.5 of [9]. 2

Proof

The eective Hamiltonian and the limit problem Theorem 2 For xed (x; p) there exists a unique constant  = H (x; p) such that the problem H (x; y; p; Dy )   in Y , H (x; y; p; Dy )   in Y , has a Lipschitz continuous solution. Moreover, Lip()  r? (H (x; p) + jjljj1 + jpjjjf jj1 ), where r 1.2

1

is the radius appearing in the controllability assumption. This result follows from the proof of Thm. VII.1.1 in [9]; the Lipschitz estimate can be done as in Prop. III.2.3 of [9] (see also section 2). Remark The eective Hamiltonian ?H (x; p) is the average cost of an ergodic control problem in the y variable, i.e., it satis es the formula Z t ?
(p; f (x; y ;  )) + l(x; y ;  ) ds j y_ = g(x; y ;  ); y = y; y 2 Y g H (x; p) = sup lim supf? 1

t

2A t!+1

0

s s

s s

s

s s

s

0

(7) for all y 2 Y , where A denotes the set of measurable controls  : (0; T ] ! A, by Prop. VII.1.3 in [9]. It is also the rescaled limit of the value functions of nite horizon problems as the horizon goes to in nity: Z t ?
(p; f (x; y ;  )) + l(x; y ;  ) ds j y_ = g(x; y ;  ); y = y; y 2 Y g; H (x; p) = lim sup f? 1 t!+1 2A

t

0

s s

s s

s

s s

0

s

(8)

for all y 2 Y , and the convergence is uniform in y as t ! +1, see Exercise VII.1.1 in [9]. Next result gives the regularity of the eective Hamiltonian. 5

Proposition 3 The eective Hamiltonian H has the following properties. | For all x; p inf inf H (x; y; p; q)  H (x; p)  sup H (x; y; p; 0); y q y | H is convex in p; | for all x; p; p0

jH (x; p) ? H (x; p0 )j  jjf jj1 jp ? p0 j;

| for all x; x0 ; p

jH (x0 ; p) ? H (x; p)j  Lip(f )jpjjx0 ? xj + !l (jx0 ? xj) + 2Lip(g) jjljj1 +rjpjjjf jj1 jx0 ? xj;

where !l is the modulus of continuity of l with respect to x; | H is uniformly continuous on Rn  B (0; R) for all R > 0. The proof is essentially the same as that of Prop. 3 in [1]. The bound for the Lipschitz continuity in x follows easily from the estimate H (x; p)  jjljj1 + jpjjjf jj1 and the bound on Lip() of Theorem 2. The last result of this subsection gives the solution of the limit problem. Proposition 4 There exists a unique viscosity solution in BC ((0; T ]  Rn ) of

?@t u + H (x; Du) = 0 in (0; T )  Rn and u(T;
) = h on Rn : Proof

(HJ)

In view of the regularity of H , in particular of

jH (x0 ; p) ? H (x; p)j  C jpjjx0 ? xj + !(jx0 ? xj);

where ! is a modulus, a comparison theorem holds for (HJ), see e.g. Thm. III.3.7 and Exercise V.1.5 in [9]. The existence can be proved by the Perron-Ishii method (see e.g. Section V.2.2 of [9]), or by representing the eective Hamiltonian H as the Bellman Hamiltonian of a control problem and then proving that the value function of such a problem solves (HJ). 2

An example: separated controls In the paper [6] the controls acting on the slow and the fast variables are separated, in the sense that  = ( ; ) 2 B  C with

1.3

f = f (x; y; ); l = l(x; y; ); g = g(x; y; ): In this case the Hamiltonian of the "?problem is H (x; y; p; q) = H1 (x; y; p) + H2 (x; y; q); H1 (x; y; p) := max f?(p; f (x; y; )) ? l(x; y; )g; H2 (x; y; q) = sup f?(q; g(x; y; ))g; 2B

2C

and we expect that the eective Hamiltonian be

H (x; p) = max H1 (x; y; p): y2Y

(9)

In fact, from the representation (8) of H we get, for all y 2 Y ,

H (x; p) = t!lim supf 1t +1

:

Z 0

t

H1 (x; ys ; p) ds j y_s = g(x; ys ; s ); y0 = y; ys 2 Y g:

(10)

This gives immediately H (x; p)  maxy2Y H1 (x; y; p). For the opposite inequality we x y such that H1 (x; y; p) = maxy H1 (x; y; p). If there is a control such that g(x; y; ) = 0, we choose y = y in the right hand side of (10) and the sup is attained by = because the average cost is H1 (x; y; p). If such a control does not exist, but y 2 Y , we deduce from the controllability assumption on the 6

fast variables that for any " > 0 there is a control such that y0 = y implies jys ? yj  " for all s > 0. The average cost associated with this control is bounded below by inf B(y;")\Y H1 (x; y; p). By taking the limit as " ! 0, the continuity of H1 gives H (x; p)  H1 (x; y; p). In the remaining case of y 2 @Y we need to assume (5) and (6). Then the system can move along a direction pointing inward Y : this is not hard to show by the calculations in the proof of Lemma A.2 in [6]. If we do this for a short time and reach ye 2 Y \ B (y; "=2), then we can use the controllability assumption as before and keep the trajectory in B (y; ") forever. Therefore we reach the desired inequality as in the previous case. In conclusion, the representation (9) of the eective Hamiltonian is proved under the assumptions (5) and (6), and the convergence theorem proved later in this section contains as a special case the main result of [6]. Remark In this case the limit problem (HJ) has a simple control interpretation. In fact its solution is the value function of the problem of minimizing the functional

J (t; x; 
; y
) :=

T

Z

t

l(xs ; ys ; s ) ds + h(xT )

(11)

on the trajectories of the system

x_ s = f (xs ; ys ; s ) xt = x; with measurable controls 
: (0; T ] ! A and y
: (0; T ] ! Y . Therefore the fast variables become

controls in the limit problem. 1.4

Connections with the order reduction method

If we try to follow the classical Levinson-Tichonov approach to singularly perturbed ordinary dierential equations, we have to set formally " = 0 in the dynamical system and get g(xs ; ys ; s )  0. This leads to conjecture that the limit dynamics is governed by the dierential inclusion (ys ; s ) 2 Z (xs );

x_ s = f (xs ; ys ; s );

(12)

where

Z (x) := f(y; ) 2 Y  Aj g(x; y; ) = 0g; and of course (12) makes sense if Z (xs ) 6= ; for almost every s. The conjecture turns out to be

true in many important problems that can be put in the reduced order form, see e.g. [31, 12] and the references therein. In this case the limit Hamiltonian is

H0 (x; p) :=

sup

y;)2Z (x)

(

F (x; y; ; p);

where

F (x; y; ; p) := ?(p; f (x; y; )) ? l(x; y; ): Lemma 5 Assume in addition Z (x) 6= ;. Then H0 (x; p)  H (x; p) for all x; p 2 Rn . Proof

Fix (x; p) and (; y) 2 Z (x). Since ys  y solves y_s = g(x; ys ; s ) for s  , Z t ?
1 F (x; y; ; p)  sup f? t (p; f (x; ys ; s )) + l(x; ys ; s ) dsg

2A

0

for all t > 0 and all solutions of y_s = g(x; ys ; s ), y0 = y, ys 2 Y . Then (8) implies F (x; y; ; p)  H (x; p), and we conclude by the arbitrariness of (y; ). 2 If the multifunction Z (
) is regular enough, say Lipschitz continuous in the Hausdor metrics, then the value function v(t; x) of the problem with dynamics (12) and cost functional J de ned by (11) is the viscosity solution of

?@t v + H (x; Dv) = 0 in (0; T )  Rn and v(T;
) = h on Rn ; 0

7

see, e.g., [15]. Then a comparison theorem gives v  u, where u is the solution of the limit problem (HJ). Next we give three examples where H0 = H , and therefore v =Ru. In the rst two we make assumptions on the m-dimensional control problem of maximizing 0t F (x; ys ; s ; p) ds for xed (x; p), that is connected to H by the formulas (7) and (8). Example 1: the ane-convex case. Suppose that A and Y are convex and, for all xed (x; p), f and g are ane and l is convex with respect to (y; a). We de ne

G(x; ; p) := lim sup 1t

t

Z

t!+1

0

F (x; ys ; s ; p) ds;

 2 A;

(13)

where y_s = g(x; ys ; s ) and y0 is xed. By the representation formula (7),

H (x; p) = sup G(x; ; p) 2A

for any choice of y0 . We x  2 A and choose tn ! +1 such that 1 G(x; ; p) = lim n t

Z tn

n

F (x; ys ; s ; p) ds:

0

(14)

By the convexity and compactness of A and Y we can extract a subsequence such that 1 lim n t

n

Z tn 0

1 lim n t

ys ds = y 2 Y ;

n

Z tn 0

s ds =  2 A:

The assumptions of this example imply the concavity of F (x;
;
; p) for all xed (x; p), so Z tn

G(x; ; p)  lim F (x; t1 n

n

Moreover

g(x; 1t

t

Z 0

ys ds; 1t

Z

t

0

0

ys ds; t1

n t

Z

s ds) = 1t

0

Z tn 0

s ds; p) = F (x; y; ; p):

g(x; ys ; s ) ds = 1t (yt ? y0 ):

Here we set t = tn and pass to the limit to get, by the boundedness of Y , g(x; y; ) = 0. Then

G(x; ; p) 

sup

y;)2Z (x)

(

F (x; y; ; p) = H0 (x; p):

Since  2 A is arbitrary we obtain H (x; p)  H0 (x; p), and we can conclude that H = H0 by the preceding Lemma. Example 2: the case of an asymptotically stable optimal trajectory. Here we follow Example 7.4 in [5] and suppose for all (x; p) that for some y0 there is a control  2 A such that t

Z 0

F (x; ys ; s ; p) ds = sup

where y_s = g(x; ys ; s ), y0 = y0 , and

2A

t

Z 0

F (x; ys ; s ; p) ds

lim  =  s!+1 s

8t > 0;

lim y = y s!+1 s

for some  =  (x; p) 2 A, y = y (x; p) 2 Y . Then g(x; y (x; p);  (x; p)) = 0 and Z t 1 F (x; ys ; s ; p) ds = F (x; y (x; p);  (x; p); p): lim t!+1 t 0

By the representation formula (8) we get

H (x; p) = F (x; y (x; p);  (x; p); p)  8

sup

y;)2Z (x)

(

F (x; y; ; p) = H0 (x; p);

and the equality H = H0 follows from the preceding Lemma. Example 3: separated controls. In the case of Subsection 1.3 formula (9) implies H  H0, and therefore H = H0 , if for all x and y there exists  2 C such that g(x; y; ) = 0. Note that this condition follows from the controllability assumption on the fast variables if in addition g(x; y; C ) is a convex set for all x; y, and this is the case, for instance, if one uses relaxed controls : . It is obvious that the equality H0 = H cannot hold at points where Z (x) = ;, but it is known that the equality may also fail at points where Z (x) 6= ;, see e.g. [4]. We end this subsection with a simple example that exhibits this phenomenon and satis es our assumptions. Example 4: ?1 < H0 < H . Consider A = [?1; 1], Y =] ? 1=2; 1=2[, g(y; ) =  ? y, l(x; y; ) = l1 (x) + jyj2 ? jj2 , with l1 continuous and bounded, and any f such that the assumptions of Subsection 1.1 are satis ed. Then H0 (x; 0) = ?l1(x). On the other hand, by switching fast enough from  = 1 to R = ?1 we can keep the solution ys of y_s = g(x; ys ; s ), y0 = 0, in any neighborhood of 0, so sup 1t 0t F (x; ys ; s ; 0) ds = ?l1 (x) + 1 and then H (x; 0) = ?l1(x) + 1 by (8).

A control interpretation for the limit problem Now we construct an optimal control problem whose Hamiltonian is H . Let (Y  A)r be the set of Radon probability measures on Y  A, and extend ' = f; l; g to functions f r ; lr ; gr de ned on Rn  (Y  A)r as it is usually done for relaxed controls, namely,

1.5

'r (x; ) :=

Z

Y A

 2 (Y  A)r :

'(x; y; a)d(y; a);

We call a limiting relaxed control a measure  2 (Y  A)r such that, for some  2 A, tn ! +1, and y0 , 1 Z tn  ds !  weak star,

tn

0

ys ;s )

(

where (y;) is the Dirac's mass at (y; ) and y_s = g(x; ys ; s ). We denote with Zl (x) the set of limiting relaxed control. The reason for the notation is that

Zl (x)  Z r (x) := f 2 (Y  A)r j gr (x; ) = 0g: In fact

gr (x; t1 n

Z tn

(ys ;s ) ds) = t1

Z tn

Z tn

g(x; ys ; s )ds = t1 (ytn ? y0 );

n 0 0 r and the limit as n ! +1 gives g (x; ) = 0 by de nition of weak star convergence. 0

n

gr (x; (ys ;s ) )ds = t1 n

Now we de ne

Hlr (x; p) := sup F r (x; ; p);

F r (x; ; p) := ?(p; f r (x; )) ? lr (x; ):

2Zl (x)

Theorem 6 For all x; p 2 Rn , H (x; p) = Hlr (x; p). Proof The proof is similar to the ane-convex example of the previous subsection and we use the same notations. Let  2 Zl (x) be generated by : ; y: , and the sequence tn ! +1. Then

1

tn

Z tn 0

F (x; ys ; s ; p)ds = t1

n

Z tn 0

F r (x; (ys ;s ) ; p)ds = F r (x; t1

n

Z tn 0

(ys;s ) ds; p)

and the right hand side converges to F r (x; ; p) as n ! +1 by de nition of weak star convergence. This proves G(x; ; p)  F r (x; ; p); where G is de ned by (13). By taking the sup over  2 Zl (x) we get Hlr (x; p)  sup G(x; ; p) = H (x; p): 2A

To prove the opposite inequality we x , y0 , and tn ! +1 such that Z tn 1 F (x; y ;  ; p)ds: G(x; ; p) = lim n tn

0

9

s s

ByRthe compactness of (Y  A)r we can extract a subsequence, that we do not relabel, such that tn 1 r tn r 0 (ys ;s ) ds converges weak star to some , and  2 Zl (x). Hence, G(x; ; p) = F (x; ; p)  Hl (x; p). By taking the sup over  2 A, we then get, by using again (8),

H (x; p)  Hlr (x; p);

2

which completes the proof.

The control problem associated with the Hamiltonian Hlr and the terminal cost h is the minimization of Z T J r (t; x; ) := lr (xs ; s ) ds + h(xT ) Remark

t

for the system

x_ s = f r (xs ; s ) s 2 Zl (xs ) xt = x; and measurable control functions  : [0; T ] ! (Y  A)r . If the multifunction Zl (
) is regular

enough, say it takes compact values and is Lipschitz continuous with respect to the Hausdor metrics [15], then the value function of this control problem is continuous and it is the solution of the limit problem (HJ). We postpone to a future paper the investigation of the properties of Zl and the connections with Artstein's invariant measures [4] and the related limit control problems of Vigodner [39]. Remark

In connection with the reduced order method we note that

H0  H = Hlr  H0r := supr F r (x; ; p): 2Z (x)

Convergence Theorem 7 As " ! 0+ the functions fu"g converge uniformly on compact subsets of (0; T )  Rn  Y to the unique solution u of (HJ); if h does not depend on y the convergence is uniform on compact subsets of (0; T ]  Rn  Y . 1.6

Proof

We de ne the weak limits in viscosity sense, or relaxed semi-limits

u (t; x; y) := "!0;lim inf inf u (t0 ; x0 ; y) u(t; x) := lim inf * inf y " t0 !t; x0 !x y " "!0

and u = lim sup * supy u" . We rede ne u at t = T by setting

ue(T; x) := 0 lim sup0 u(t0 ; x0 ) and ue(t; x) := u(t; x) for 0 < t < T: t !T? ; x !x

We will show that u is a supersolution of (HJ) and ue is a subsolution of (HJ). By comparison this gives u = u = u in (0; T )  Rn and implies the convergence of fu" g to u uniformly on compact subsets of (0; T )  Rn  Rm . To prove that ue is a subsolution of the limit H-J equation we consider a strict maximum point (t; x) of u ? ' with 0 < t < T and ' smooth. We want to show that

?@t '(t; x) + H (x; D'(t; x))  0; and suppose by contradiction that

?@t '(t; x) +  > 0; for  = H (x; p); p = D'(t; x): Let  be the solution of the cell problem at (x; p) and de ne the perturbed test function

'" (t; x; y) = '(t; x) + "(y): 10

(15)

We claim that for some r > 0 '" is a viscosity supersolution of

?@t '" + H (x; y; Dx '" ; Dy"'" )  0 in Ir  B (x; r)  Y ;

(16)

where Ir = (t ? r; t + r). To prove the claim we take a smooth such that '" ? attain its minimum over Ir  B (x; r)  Y at (et; xe; ye), and ('" ? )(et; xe; ye) = 0: Then the function y 7! (y) ? "?1 (et; xe; y) has a minimum at ye, so the de nition of  gives

H (x; ye; p; D"y (et; xe; ye))  :

Since  is Lipschitz continuous it is easy to check that j"?1 Dy (et; xe; ye))j  Lip(): Now we set := (?@t '(t; x) + )=2 and use the continuity of H in (x; p), uniformly for y 2 Y and j"?1 Dy j  Lip(), to nd  such that

H (x; ye; p; D"y (et; xe; ye))   ?

for jx ? xj <  and jp ? pj < . Now choose 0 < r   such that

jDx '(t; x) ? Dx'(et; xe)j <  and j@t '(t; x) ? @t '(et; xe)j < : Note that the choice of r is independent of . Since Dx (et; xe; ye) = Dx'(et; xe) and @t (et; xe; ye) =

@t '(et; xe) we get

[?@t + H (
;
; Dx ; D"y )](et; xe; ye)  ?@t '(t; x) ? +  ? = 0;

which completes the proof of the claim. In view of (16), we can use a Comparison Principle for the mixed boundary value problem with prescribed data on @ (Ir  B (x; r)) and state constrained condition at @Y (see, e.g., Thm. IX.1 in [16]) to obtain sup

Ir B (x;r)Y

(u" ? '" ) 

sup

@ (Ir B (x;r))Y

(u" ? '" ):

It is not hard to deduce from this and the de nitions of u and '" that (u ? ')(t; x) 

sup

@ (Ir B (x;r))

(u ? ');

and this is a contradiction with the fact that (t; x) is a strict maximum point of u ? '. This completes the proof of (15). Next we show that u is a supersolution of the limit H-J equation. Now (t; x) is a strict minimum point of u ? ' and we assume by contradiction that ?@t '(t; x) +  < 0; where  = H (x; D'(t; x)) as before. We also de ne  and '" as before, and now claim that '" is a viscosity subsolution of ?@t '" + H (x; y; Dx '" ; Dy"'" )  0 in Ir  B (x; r)  Y: The proof is essentially the same as the proof of (16). Now we exploit the fact that u" is a (constrained) supersolution of the same PDE in Ir  B (x; r)  Y and the Comparison Principle for the mixed Dirichlet-constrained boundary value problem (see, e.g., Thm. IX.1 in [16]) to get inf

Ir B (x;r)Y

This implies

(u" ? '" ) 

inf

@ (Ir B (x;r))Y

(u" ? '" ):

(u ? ')(t; x)  @ (I inf (u ? '); B (x;r)) r

a contradiction with the choice of (t; x). Therefore

?@t '(t; x) + H (x; D'(t; x))  0; 11

and so u is a supersolution of the limit H-J equation. Finally we check the terminal condition. The hypotheses on f and l imply easily the estimate

u" (t; x; y)  ?(T ? t)jjljj1 + inf fh(x0 ) : jx0 ? xj  jjf jj1 (T ? t)g; for all " > 0: Since h is continuous, in the limit we obtain u(T; x)  h(x). For ue, we note that u" (t; x; y)  (T ? t)jjljj1 + 2Ainf h(x; yTt ) + !h (jx ? xtT j); x;y)

(

where (xtT ; yTt ) denote the position of the system at time T if the position at time t is (x; y), and !h is the modulus of continuity of h. Since jx ? xtT j  (T ? t)jjf jj1 , the rst and third term on the right hand side of the preceding estimate tend to 0 as t ! T ?. To reach the conclusion we are going to prove that

lim0 sup0

0

sup 2Ainf 0 h(x0 ; yTt )  h(x)

"!0; t !t; x !x y

x ;y)

(

for all t < T . Without loss of generality we can assume that

B (0; r)  fg(x; y; ) j  2 Ag: In fact, if we take relaxed controls or use Catheodory's theorem (as in Lemma 2.7 of [6]) to convexify g(x; y; A), the value function u" does not change because the Hamiltonian H is the same. Then it is easy to see by means of a standard selection lemma (e.g., as in Lemma 2.8 of [6]) that any polygonal  Y is the trajectory of a solution yt of the system, with speed r=". Therefore, by the elementary Lemma 8 below, for some constant M the system can reach any point y 2 Y from any y 2 Y within the time M"=r. Then, for any initial position y 2 Y of the system, 0 ; yt ) = h(x0 ) if t0  t + M" : inf h ( x T 2A 0 r 0

x ;y)

(

For any t < T we can restrict " < r(T ? t)=M and get lim0 sup0

0

sup 2Ainf 0 h(x0 ; yTt ) = lim0 sup h(x0 ) = h(x);

"!0; t !t; x !x y

x ;y)

(

x !x

where in the last equality we used the continuity of h. Therefore ue(T; x)  h(x) and the proof of the rst statement is complete. In the case of h = h(x) we have, for 0  (T ? t)jjf jj1  1, jx ? x0 j  1, and all y,

ju" (t; x0 ; y) ? h(x)j  (T ? t)jjljj1 + !(jx ? x0 j + (T ? t)jjf jj1 );

where ! is the modulus of continuity of h in B (x; 2). Therefore u(T; x) = u(T; x) = ue(T; x) = h(x), and the convergence of u" is uniform on compact subsets up to time t = T . 2 We end this section with the property of sets with Lipschitz boundary that we used in the preceding proof. Lemma 8 If Y is a bounded connected open set with Lipschitz boundary, i.e., (4) holds, then there exists M > 0 such that all y; y 2 Y can be joined by a polygonal  Y with length( )  M . Proof

We de ne

d(y; y) := inf flength( )j  Y polygonal with endpoints y and yg: Since Y is open and connected with Lipschitz boundary, d is nite for all y; y 2 Y , and it is easy to see that it is a metric on Y . We claim that d is locally uniformly equivalent to the Euclidean metric, i.e., there exist r; C > 0 such that for all z 2 Y jy ? yj  d(y; y)  C jy ? yj 8y; y 2 Y \ B (z; r): 12

Then the topology induced by d is equivalent to the usual one, so Y is compact for this topology and therefore it is bounded for the metric d, which gives the desired conclusion. To prove the claim we rst consider the case z 2 @Y . By Proposition A.2 and Remark A.3 in [10] there exist r0 ; L > 0 independent of z such that in B (z; 2r0 ) Y is the epigraph of a Lipschitz function de ned on the hyperplane orthogonal to a vector  and with Lipschitz constant bounded by L. Then there is k independent of z such that

8 0 < t < k; y 2 Y \ B (z; r0 ): If we set v = y ? y=jy ? yj, the segments fy + t ? tkvj 0 < t < kg and fy + t + tkvj 0 < t < kg lie in Y and they intersect for t = jy ? yj=2k, which is acceptable for jy ? yj < 3k. Thus, for r = r0 ^ 3k, two points y; y 2 Y \ B (z; r) can be joined by a polygonal in Y of length jy ? yj=k. This proves B (y + t; kt)  Y

the claim in a neighborhood of @Y . In the complement of this neighborhood the claim is trivial and the proof is complete. 2

2 Stochastic control with periodic fast variables The "-control problem For " > 0 xed, we now consider the following nite horizon stochastic control problem in (0; T ]  Rn  Rm . Let ( ; F ; P ) be a probability space, endowed with a right-continuous ltration (Ft ) tT and a r-dimensional adapted Brownian motion Wt . Given a progressively measurable

2.1

0

 with values in a compact set A, the stochastic dierential equation

dxs = f (xs ; ys ; s ) ds + (xs ; ys ; s ) dWs ; dys = "?1 g(xs ; ys ; s ) ds + "?1=2  (xs ; ys ; s ) dWs for s  t, starting from xt = x 2 Rn , yt = y 2 Rm , has a pathwise unique adapted strong solution when the functions f , g, ,  are Lipschitz continuous in (x; y) uniformly in . The variable x is called the slow variable and y the fast variable. We refer to Fleming, Soner [21] for a presentation of the theory of stochastic control and its relationship with the theory of viscosity solutions. We shall always assume that all the functions are Zm-periodic in the fast variable y. The associated value function with running cost l and terminal cost h is given by

u" (t; x; y) = inf Ef 

Z

t

T

l(xs ; ys ; s ) ds + h(xT )g:

Under the assumptions we recall below, it is continuous and bounded on (0; T ]  Rn  Rm uniformly in ". It is also periodic in the fast variable y. We shall make throughout the following set of assumptions that are classical in the theory of stochastic control. { The control set A is a compact metric space. { The functions f , g, ,  and l are bounded continuous functions in Rn  Rm  A with values resp. in Rn , Rm , M n;r (the set of the n  r real matrices), M m;r and R. They are Zm-periodic in the fast variable y. { The drift vectors f and g and the dispersion matrices  and  are Lipschitz continuous in (x; y), uniformly in . { The running cost l is uniformly continuous in (x; y), uniformly in . { The terminal cost h : Rn ! R is bounded uniformly continuous. 2.2

The Hamilton-Jacobi-Bellman equation Consider the diusion matrices

T

T

T

a = 2 ; b = 2 ; c = 2 13

and associate the Hamiltonian H (x; y; p; q; X; Y; Z ) = sup f? tr(a(x; y; )X ) ? tr(b(x; y; )Y ) ? tr(c(x; y; )Z ) ? tr(Zc(x; y; )) 2A

?(p; f (x; y; )) ? (q; g(x; y; )) ? l(x; y; )g:

It is de ned on Rn  Rm  Rn  Rm  Sn  Sm  M n;m , where Sn designates the set of the symmetric n  n matrices. Given a function u(t; x; y) de ned on (0; T ]  Rn  Rm , we consider the 2 2 partial gradients @t u, Dx u and Dy u. We also de ne the partial Hessian matrices Dxx u, Dyy u and
? 2 2 Dxy u = @xi ;yj u 1in;1jm , so that the full Hessian matrix of u with respect to (x; y) is 



2 2 xy u : D u = (DD2xxuu)T D 2 Dyy u xy By the dynamic programming principle, the value function u" is a viscosity solution of the second2

order degenerate parabolic Hamilton-Jacobi-Bellman equation 8
0, the value function u" is the unique bounded continuous viscosity solution of (HJ" ) in (0; T ]  Rn  Rm . For further use, we recall that the uniqueness statement in the theorem takes the form of the following comparison principle. If u is a bounded u.s.c. viscosity subsolution of (HJ" ) and v is a bounded l.s.c. viscosity supersolution, then we have u  v in (0; T ]  Rn  Rm . We refer to Crandall, Ishii, Lions [17] and to [21] for the precise de nitions of a subsolution and a supersolution and for the proof of the comparison principle. 2.3

The ergodic control problem in the fast variable and the eective Hamiltonian

We shall make one of the following three assumptions to guarantee some averaging properties of the fast dynamics. (i) The diusions in the fast variable are uniformly non degenerate, i.e., there is a constant  > 0 such that b(x; y; )  Im ; for all (x; y; ); where Im denotes the m-dimensional identity matrix. Moreover the running cost l(x;
; ) is Holder continuous for some exponent 0 <  1, uniformly on (x; ). (ii) The diusions in the fast variable are independent of x (b  b(y; )) and there is a deterministic controllable subsystem in the fast variable, i.e., there is some r > 0 and some A0  A so that b(y; ) = 0 for all  2 A0 and B (0; r)  convfg(x; y; ) j  2 A0 g; for all (x; y): (iii) The drifts and diusions in the fast variable do not depend on x, y (g  g() and b  b()) and satisfy the non-resonance condition for every k 2 Zm, there is  2 A such that (g(); k) 6= 0 or b()k 6= 0:

In terms of the Hamiltonian, case (i) corresponds to the uniform ellipticity of H in Y H (x; y; p; q; X; Y + W; Z )  H (x; y; p; q; X; Y; Z ) ?  tr W; W  0; while (ii) corresponds to the coercivity with respect to q H (x; y; p; q; X; Y; Z )  rjqj ? C (1 + jpj + jX j): 14

The preceding assumptions are of two dierent natures. Part of them demand that some quantities in the fast dynamics are independent of the slow variable. They are needed to ensure enough regularity of the averaged quantities with respect to the slow variables. It is an open question whether they can be dispensed with. The second kind of assumptions is more fundamental. They guarantee the solvability of the ergodic control problem in the fast variable. These assumptions correspond to some of the cases studied by Arisawa, Lions [2]. In our context, their results read as follows. Theorem 10 Assume that either (i) or (ii) or (iii) holds. Let (x; p; X ) be xed. For every  > 0, let w denote the unique viscosity solution of the stationary problem in the fast variable 2 w + H (x; y; p; Dy w ; X; Dyy w ; 0) = 0 in Rm ;

w periodic:

(17)

Then, as  ! 0+, the family fw g converges to a constant ?H (x; p; X ) uniformly with respect to

y.

When one looks at the solution w of the Hamilton-Jacobi-Bellman equation (17) as the value function of a discounted control problem in the fast variable, the theorem asserts that 1

Z

?




e?s ? tr(a(x; ys ; s )X ) ? (p; f (x; ys ; s )) ? l(x; ys ; s ) ds (18)  0 j dys = g(x; ys ; s ) ds +  (x; ys ; s ) dWs ; y0 = yg; the convergence being uniform in y. H (x; p; X ) = lim supfE !0

In cases (i) and (ii), one can characterize the eective Hamiltonian in the more convenient form of Section 1. It is the unique constant H for which the cell problem

2 H (x; y; p; Dy ; X; Dyy ; 0) = H in Rm ;  periodic; has a continuous solution . However, such a characterization is not available in case (iii), for it

may happen that the cell problem has no solution. We refer to Arisawa, Lions [2] for an explicit example. The above assumptions are three of the ve cases studied by Arisawa, Lions [2]. Among the remaining two cases, the one-dimensional one in the fast variable (m = 1) can be handled in a similar way as (ii); we omit it for simplicity. On the other hand, for the viscosity solution techniques to apply, the uniform convergence of fw g is essential. This is why we do not consider the fth case that assumes (roughly) that at least one diusion is uniformly non degenerate (and not all, as in (i)), because the convergence of fw g may not be uniform (but in Lp for every 1  p < 1). An example when this happens is given in Arisawa, Lions [2].

Examples for the eective Hamiltonian Example 1: the coercive and separated case. The rst example, as in Section 1, is the case 2.4

of separated controls. We assume that the fast variable is controlled independently of the slow variable and that there is a controllable deterministic subsystem (case (ii)). The controls are of the form  = ( ; ) and f = f (x; y; ),  = (x; y; ), g = g(x; y; ) and  =  (x; y; ). We also assume that l = l(x; y; ). Under these assumptions, the representation formula reads

H (x; p; X ) = lim sup fE !0 ; )

(

= lim supfE !0

for

1

Z

0

Z 0

1

?




e?s ? tr(a(x; ys ; s )X ) ? (p; f (x; ys ; s )) ? l(x; ys ; s ) ds

j dys = g(x; ys ; s ) ds +  (x; ys ; s ) dWs g; e?s H (x; ys ; p; X ) ds j dys = g(x; ys ; s ) ds +  (x; ys ; s ) dWs g; 1

H1 (x; y; p; X ) = supf? tr(a(x; y; )X ) ? (p; f (x; y; )) ? l(x; y; )g:

Arguing as in Section 1, we deduce from the controllability assumption that

H (x; p; X ) = sup H1 (x; y; p; X ): y

15

Thus, the eective Hamiltonian corresponds to the original control problem where the fast variable plays the role of an additional control. Example 2: uncontrolled and nondegenerate diusion of the fast variables. The second example assumes that the fast variable is an uncontrolled uniformly non degenerate diusion. Since we are in case (i), we know that the eective Hamiltonian is characterized by the solvability of the linear cell problem 2 H1 (x; y; p; X ) ? tr(b(x; y)Dyy ) ? (Dy ; g(x; y)) = H (x; p; X ) in Rm ;

 periodic

(where H1 is given above, with  instead of ). Assuming that the functions b and g are smooth in y, there is a unique solution x (the invariant measure) of the adjoint equation

@ 2 (b (x; y) ) + X @ (g (x; y) ) = 0 in Rm ;  periodic ij x i x x i @yi i;j @yi @yj R with mean (0;1)m x (y) dy = 1. This follows from the Fredholm alternative (see, for instance, [14]

?

X

or [13]). A necessary and sucient condition for the cell problem to have a solution is that

H (x; p; X ) =

Z

; m

(0 1)

H1 (x; y; p; X )x(y) dy:

If in addition b and g are independent of y (and more generally when gi =

@bij ), we have j @yj

X

x  1. The eective Hamiltonian is therefore the average H (x; p; X ) =

Z

; m

(0 1)

H1 (x; y; p; X ) dy:

This example is a variant of results of Jensen, Lions [28] and Evans [19].

Regularity properties of the eective Hamiltonian Proposition 11 The eective Hamiltonian H is degenerate elliptic in X and convex in (p; X ). 2.5

Moreover, we have the bounds

inf H (x; y; p; 0; X; 0; 0)  H (x; p; X )  sup H (x; y; p; 0; X; 0; 0): y y

Proof

(19)

The bounds for H are a consequence of the observation that the constant functions

?? sup H (x; y; p; 0; X; 0; 0); y

?? infy H (x; y; p; 0; X; 0; 0)

1

1

are respectively a subsolution and a supersolution of (17). Therefore, by the comparison principle, we get ? sup H (x; y; p; 0; X; 0; 0)  w  ? infy H (x; y; p; 0; X; 0; 0): y

Sending  ! 0 yields (19). Degenerate ellipticity and convexity can be derived by analytical means as in [1]. They are also simple consequences of the representation formula (18). Indeed, for every  xed and every control s , the function

E

Z

0

1

?




e?s ? tr(a(x; ys ; s )X ) ? (p; f (x; ys ; s )) ? l(x; ys ; s ) ds

is linear in (p; X ) and non-increasing in X . Taking the supremum over the controls yields a function that is convex in (p; X ) and non-increasing in X . And so is the limit as  ! 0. 2 The continuity of the eective Hamiltonian is a consequence of the following result. 16

Proposition 12 There are a constant C > 0 and a modulus ! such that jH (x; p0 ; X 0) ? H (x; p; X )j  C (jp0 ? pj + jX 0 ? X j) for all (x; p; p0 ; X; X 0)

(20)

and

jH (x0 ; p; X ) ? H (x; p; X )j  C jx0 ? xj(1 + jpj + jX j) + !(jx0 ? xj)

for all (x; x0 ; p; X ): (21)

The rst inequality follows at once from the representation formula (18) by taking the constant C = max(kf kL1 ; kakL1 ). The second inequality is more delicate. When the drift and diusion in the fast variable are independent of ?x (case (iii)),
the inequality follows from the representation formula for the constant C = max Lip(f ); Lip(a) and for the modulus ! = !l . We give a second proof of this elementary result, which we shall modify in the other two cases. Since the drift and diusion for the fast variable are independent of x, the Hamiltonian H satis es

Proof

H (x0 ; y; p; q; X; Y; 0)  H (x; y; p; q; X; Y; 0) + C jx0 ? xj(1 + jpj + jX j) + !(jx0 ? xj): Therefore, the function w (
; x; p; X ) is a subsolution of 2 w + H (x0 ; y; p; Dy w ; X; Dyy w ; 0)  C jx0 ? xj(1 + jpj + jX j) + !(jx0 ? xj):

(22)

By the comparison principle, we obtain the uniform bound

w (
; x; p; X )  w (
; x0 ; p; X ) + C jx0 ? xj(1 + jpj + jX j) + !(jx0 ? xj): Sending  ! 0 yields

H (x; p; X )  H (x0 ; p; X ) ? C jx0 ? xj(1 + jpj + jX j) ? !(jx0 ? xj): We get (21) after exchanging x and x0 . We now assume (ii). As g may now depend on x (but not b), we have to replace (22) by ?
H (x0 ; y; p; q; X; Y; 0)  H (x; y; p; q; X; Y; 0) + C jx0 ? xj 1 + jpj + jqj + jX j + !(jx0 ? xj): (23) The controllability assumption gives the coercivity of H in q uniformly in Y H (x; y; p; q; X; Y; 0)  rjqj ? C (1 + jpj + jX j): Since

jH (x; y; p; 0; X; 0)j  C (1 + jpj + jX j); kw kL1  sup y

we deduce that the solution of (17) is Lipschitz continuous with the bound


?

(24) kDy w (
; x; p; X )kL1  r? kw kL1 + C (1 + jpj + jX j)  C (1 + jpj + jX j): We deduce from (23) that w (
; x; p; X ) is a subsolution of w + H (x0 ; y; p; Dy w ; X; Dyy w ; 0)  C jx0 ? xj(1 + jpj + jX j + kDy w kL1 ) + !(jx0 ? xj)  C jx0 ? xj(1 + jpj + jX j) + !(jx0 ? xj): 1

2

The inequality for H is deduced as before from the comparison principle. We nally consider case (i). As g and b now depend on x, the inequality for H reads ?




H (x0 ; y; p; q; X; Y; 0)  H (x; y; p; q; X; Y; 0) + C jx0 ? xj 1 + jpj + jqj + jX j + jY j + !(jx0 ? xj):

(25)

We claim that the solution w of (17) is in C 2; for some exponent 0 <  with

kw (
; x; p; X ) ? w (0; x; p; X )kC ; Rm  C (1 + jpj + jX j): 2

17

(

)

(26)

Admitting this temporarily, we deduce that w (
; x; p; X ) is a subsolution of 2 w + H (x0 ; y; p; Dy w ; X; Dyy w ; 0) 2 0 w kL1 ) + !(jx0 ? xj)  C jx ? xj(1 + jpj + jX j + kDy w kL1 + kDyy  C jx0 ? xj(1 + jpj + jX j) + !(jx0 ? xj): Inequality (21) for H follows as before by comparison. The proof of (26) relies on the regularity theory for uniformly elliptic Hamilton-JacobiBellman equations (see Gilbarg-Trudinger [23] and Safonov [35]). Our argument is patterned after the one of Arisawa, Lions [2]. We give a sketch of it to exhibit the linear growth in (p; X ) of the bound. The rst step is to establish the uniform bound kw (
; x; p; X ) ? w (0; x; p; X )kL1(Rm)  C (1 + jpj + jX j) for all (x; p; X ); 0 <  <  (27) for some constant C and some  > 0. Suppose that (27) is false. Then there is a sequence ( ; xk ; pk ; Xk ) with k ! 0 for which the solution wk = wk (
; xk ; pk ; Xk ) of (17) satis es kwk ? wk (0)kL1  k(1 + jpk j + jXk j): ? 1 We set k = kwk ? wk (0)kL1 and wek = k (wk ? wk (0)). Then, wek (0) = 0, kwek kL1 = 1 and wek is a solution of k wek + k k wk (0) + sup f? tr(b(xk ; y; )D2 wek ) ? (Dwek ; g(xk ; y; )) ? k L(y; ; xk ; pk ; Xk )g = 0 2A

for

L(y; ; x; p; X ) = tr(a(x; y; )X ) + (p; f (x; y; )) + l(x; y; ):

Since

kw kL1  C (1 + jpj + jX j)

and

kL(
; ; x; p; X )kC ;  C (1 + jpj + jX j); 0

we have

jk k wk (0)j + kk L(
; ; xk ; pk ; Xk )kC ;  Ck (1 + jpk j + jXk j)  Ck : 0

The regularity theory for uniformly elliptic Hamilton-Jacobi-Bellman equations therefore yields the uniform boundedness of wek in C 2; for some , depending only on m, the ellipticity constant  and . Moreover, for 0 xed, the families fb(xk ;
; 0 )g and fg(xk ;
; 0 )g are equi-bounded and equicontinuous. Therefore, along a subsequence, the functions wek and their derivatives of order  2, resp. b(xk ;
; 0 ), resp. g(xk ;
; 0 ) converge uniformly to some function we in C 2; and its derivatives of order  2, resp. to b, resp. to g. The function b is clearly  Ip , while we is a periodic function in C 2; such that we(0) = 0 and kwekL1 = 1. Since k wek , k k wk (0) and k L(
; ; xk ; pk ; Xk ) converge to 0 uniformly, we deduce from the stability results for viscosity solutions that we is a classical subsolution of the uniformly elliptic linear equation ? tr(b(y)D2 we) ? (Dw; e g (y ))  0: Since we is periodic, it achieves its maximum at some point. By the strong maximum principle, it must be constant. This is impossible, for we must have we(0) = 0 and kwekL1 = 1. Recalling that the running cost L(
; ; x; p; X ) is Holder continuous with C 0; norm growing linearly in p and X , we deduce from the bound (27) and from the regularity theory for uniformly elliptic Hamilton-Jacobi-Bellman equations that there is some 0 <  , depending only on m,  and , such that
? kw ? w (0)kC ;  C kw ? w (0)kL1 + jw (0)j + sup kL(
; ; x; p; X )kC ;  C (1 + jpj + jX j): 2



This is (26).

0

2

In order to solve the limit equation with Hamiltonian H , one needs to strengthen slightly the regularity of H of Proposition 12. One of the following properties is sucient to invoke results from the theory of viscosity solutions. 18

{ The eective Hamiltonian is uniformly elliptic and satis es (21). { The eective Hamiltonian is of rst order and satis es (21). { The eective Hamiltonian is degenerate elliptic and satis es the following structure condition (see [17]) : there is a modulus ! such that, for every  > 0 and every x; x0 2 Rn , X; X 0 2 Sn so that       I 0 X 0 I ? I ?3 0 I  0 ?X 0  3 ?I I ; we have

H (x0 ; (x ? x0 ); X 0 )  H (x; (x ? x0 ); X ) + !(jx0 ? xj2 + jx0 ? xj):

(28)

Condition (28) is the most general but it is an open question whether it is true in general. We can only prove it when the drift and diusion in the fast variable do not depend on x. This leads us to make one of the following assumptions on the dynamics. (iv) The diusions in the slow variable are uniformly non degenerate, i.e., there is a constant  > 0 such that a(x; y; )  In ; for all (x; y; ): (v) The problem in the slow variable is deterministic (a  0). (vi) The drifts and diusions in the fast variable are independent of x (g  g(y; ) and b  b(y; )). >From the representation formula (18), it is obvious that H is uniformly elliptic with constant  in case (iv) and that it is of rst-order in case (v). (28) also follows in case (vi) from the representation formula, since the fast dynamics is independent of x (and because we have classically ? tr(a(x0 ; y; )X 0 )  ? tr(a(x; y; )X ) + Cjx0 ? xj2 when the matrices X and X 0 satisfy the inequality in (28)). We can now invoke the theory of viscosity solutions to obtain the solvability of the limit equation. We refer to the User's guide [17] as well as to Ishii, Lions [27] for the results and proofs.

Proposition 13 Assume either (i) or (ii) or (iii), and either (iv) or (v) or (vi). Then, there is a unique bounded continuous viscosity solution of the limit equation

?@t u + H (x; Du; D u) = 0 in (0; T )  Rn ; 2

u(T;
) = h on Rn :

(HJ)

Moreover, if u is a bounded u.s.c. subsolution and v is a bounded l.s.c. supersolution, then u  v on (0; T ]  Rn .

Convergence Theorem 14 Assume either (i) or (ii) or (iii), and either (iv) or (v) or (vi). Then, as " ! 0+, the collection fu" g converges uniformly on the compact subsets of (0; T ]  Rn  Rm to the unique

2.6

viscosity solution u of (HJ).

The functions u" are bounded in (0; T ]  Rn  Rm uniformly in ". We can therefore de ne the half-relaxed limits on (0; T ]  Rn

Proof

u(t; x) = "!0;lim inf inf u (t0 ; x0 ; y); t0 !t; x0 !x y "

u(t; x) =

lim sup

sup u" (t0 ; x0 ; y):

"!0; t0 !t; x0 !x y

As in the rst section, we shall prove that u is a supersolution of (HJ) and that u is a subsolution of (HJ). By the comparison principle, we shall get u = u = u in (0; T ]  Rn . This gives classically the uniform convergence on the compact subsets of (0; T ]  Rn  Rm of fu"g to u. We only check that u is a subsolution of (HJ), the proof u is a supersolution being analogous. Let w(t; x) be the continuous viscosity solution of the equation

?@t w + infy H (x; y; Dxw; 0; Dxxw; 0; 0) = 0 in (0; T )  Rn ; 2

19

w(T;
) = h on Rn :

It is clearly a viscosity supersolution of (HJ" ). By the comparison principle, we have u" (t; x; y)  w(t; x) for all " > 0, 0 < t  T , x, y. Taking the semi-limit, we deduce that u(T;
)  h on Rn . This proves that u is a subsolution at the terminal boundary. We next prove that u is a subsolution in (0; T )  Rn . Let (t; x) 2 (0; T )  Rn be a strict maximum point of u(t; x) ? '(t; x) with u(t; x) = '(t; x). We argue by contradiction assuming that

?@t '(t; x) + H (x; D'(t; x); D '(t; x)) > 0: 2

Put H = H (x; D'(t; x); D2 '(t; x)). Let v" be the periodic solution of 2 v" ; 0) = H in Rm : "v" + H (x; y; D'(t; x); Dy v" ; D2 '(t; x); Dyy

?

By Theorem 10 and the de nition of the eective Hamiltonian, we know that " v" ?"?1 H ) converges uniformly to ?H . Therefore, "v" converges uniformly to 0. For " > 0, we consider the perturbed test function " (t; x; y ) = '(t; x) + "v" (y ): We will show that there is a small r 2 (0; t ^ (T ? t)) so that " is a supersolution of (HJ" ) in Qr = (t ? r; t + r)  B (x; r)  Rm for " small. We suppose this has been proved and reach a contradiction. Since f " g converges uniformly to ' on Qr , we have lim sup

sup(u" ? " ) = u(t; x) ? '(t; x):

"!0; t0 !t; x0 !x y

But (t; x) is a strict maximum point of u ? ', so the above relaxed upper limit is < 0 on @Qr . By compactness (recall that u" and " are periodic in y), one can nd  > 0 so that u" ? "  ? on @Qr for " small, i.e., "  u" +  on @Qr . Since " is a supersolution of (HJ" ) in Qr , we deduce from the comparison principle that "  u" +  in Qr for " small. Taking the upper semi-limit, we get '  u +  in (t ? r; t + r)  B (x; r). This is impossible, for '(t; x) = u(t; x). We have to show that " is a supersolution of (HJ" ) in Qr for r small, for all " small. For every (t; x; y) 2 Qr , we have

?@t " + H (x; y; Dx " ; Dy" " ; Dxx " ; Dyy" " ; Dpxy" " ) = ?@t '(t; x) + H (x; y; D'(t; x); Dy v" (y); D '(t; x); Dyy v" (y); 0): 2

2

2

2

2

(29)

When g and b are independent of x (case (iii)), the Hamiltonian satis es

H (x; y; p; q; X; Y; 0) ?
 H (x; y; p; q; X; Y; 0) ? C jx ? xj 1 + jpj + jX j ? !(jx ? xj) ? C jp ? pj ? C jX ? X j for p = D'(t; x) and X = D2 '(t; x). Therefore, the quantity in (29) is bounded from below by

?@t '(t; x) + H (x; y; D'(t; x); Dy v" (y); D '(t; x); Dyy v" (y); 0) ? o(1) (30) = ?@t '(t; x) ? "v" + H ? o(1); where o(1) goes to 0 as (t; x) ! (t; x) uniformly in ". Since "v" converges uniformly to 0 and since ?@t '(t; x) + H > 0, we can nd r > 0 so that the quantity is  0 in Qr for " small. We conclude 2

that

2

?@t " + H (x; y; Dx " ; Dy" " ; Dxx " ; Dyy" " )  0 in Qr : 2

2

The inequality was derived is a bit formally. Using the smoothness of ', it is an easy exercise to check that the inequality holds in the viscosity sense (see section 1). The modi cations for the cases (i) and (ii) are analogous to those performed in Proposition 12. We only sketch them. When b is independent of x (case (ii)), the Hamiltonian now satis es (23) where the additional q term appear. In (30), there is therefore the extra term jx ? xj jDy v" j. By the coercivity of H , we know that jDy v" j is bounded uniformly in y and " by C (1 + jpj + jX j) (see (24)). So the extra term converges uniformly on " and y to 0 as x ! x. The above argument 20

therefore applies and guarantees the existence of a small r > 0 so that " is a supersolution in Qr for " small. When both g and b may depend on x (case (i)), we must use the inequality (25) for the 2 Hamiltonian. But one now controls Dy v" and Dyy v" uniformly on " (see (26)). Thus, the extra 2 term jx ? xj(jDy v" j + jDyy v" j) converges uniformly to 0 as x ! x. And the argument still works.

2

3 Homogenization and stochastic control Homogenization

3.1

In the case of a periodic fast variable, a special singular perturbation problem is homogenization. We brie y illustrate this and refer to [19, 20, 8, 24, 25, 1] for recent developments in the theory of homogenization of Hamilton-Jacobi equations, which was introduced by Lions, Papanicolaou, Varadhan [34]. For an optimal control problem, homogenization corresponds to dynamics of the form dxs = f (xs ; x"s ; s ) ds + (xs ; x"s ; s ) dWs ; and the value function

v" (t; x) = inf Ef 

Z

T

t

l(xs ; x"s ; s ) ds + h(xT ) j xt = xg:

All the functions are of course assumed to be periodic in the second variable. Adding the new variable ys = xs =", the dynamical system becomes dxs = f (xs ; ys ; s ) ds + (xs ; ys ; s ) dWs ; dys = "?1 f (xs ; ys ; s ) ds + "?1 (xs ; ys ; s ) dWs (31) with starting point xt = x and yt = x=". When the problem is deterministic (  0) or when there is no drift (f  0), the value function v" can be expressed in terms of the value function u" of the singular perturbation problem of the preceding section (with g  f and   ) as follows

v" (t; x) = u" (t; x; x" )

v" (t; x) = u" (t; x; x" )

and

2

respectively:

The convergence of u" to the solution of the limit equation (which will be uniform in y by periodicity) of course implies the convergence of v" to the same limit. In general, however, the scaling in (31) diers from the one in (3). We explain brie y how the results can be adapted.

The associated singular perturbation problem

3.2

For " > 0 xed, we therefore consider a nite horizon stochastic control problem in (0; T ] 

Rn  Rm similar to the one of the preceding section but with the dynamics

dys = "?1 g(xs ; ys ; s ) ds + "?1  (xs ; ys ; s ) dWs :

dxs = f (xs ; ys ; s ) ds + (xs ; ys ; s ) dWs ; The value function

u" (t; x; y) = inf Ef 

Z

T

t

l(xs ; ys ; s ) ds + h(xT )g

is now the unique bounded continuous viscosity solution of the Hamilton-Jacobi-Bellman equation 8
0 and for " small, so that " is a supersolution. This completes the proof.

2

Acknowledgements. We are grateful to Z. Artstein for sending us the 1st and 2nd draft of the

preprint [5] that stimulated this research, and to V. Gaitsgory for a useful talk on the singular perturbation literature.

References [1] O. Alvarez, Homogenization of Hamilton-Jacobi equations in perforated sets, to appear in J. Dierential Equations. 22

[2] M. Arisawa and P.-L. Lions, On ergodic stochastic control, Comm. Partial Dierential Equations 23 (1998), 2187{2217. [3] Z. Artstein, Invariant measures of dierential inclusions applied to singular perturbations, J. Dierential Equations 152 (1999), 289{307. [4] Z. Artstein and V. Gaitsgory, Tracking fast trajectories along a slow dynamics: a singular perturbations approach, SIAM J. Control Optim. 35 (1997), 1487{1507. , The value function of singularly perturbed control systems, preprint. [5] [6] F. Bagagiolo and M. Bardi, Singular perturbation of a nite horizon problem with state-space constraints, SIAM J. Control Optim. 36 (1998), 2040{2060. [7] F. Bagagiolo, M. Bardi and I. Capuzzo Dolcetta: A viscosity solutions approach to some asymptotic problems in optimal control, in \Partial dierential equations methods in control and shape analysis", G. Da Prato and J.-P. Zolesio eds., 27{37, Marcel Dekker, New York 1997. [8] M. Bardi, Homogenization of quasilinear elliptic equations with possibly superquadratic growth, in \Nonlinear variational problems and partial dierential equations (Isola d'Elba, 1990)", A. Marino and M.K.V. Murthy eds., 44{56, Pitman Res. Notes Math. 320, Longman , Harlow 1995. [9] M. Bardi and I. Capuzzo-Dolcetta, Optimal control and viscosity solutions of Hamilton-JacobiBellman equations, Birkhauser, Boston, 1997. [10] M. Bardi and P. Soravia, A comparison result for Hamilton-Jacobi equations and applications to dierential games lacking controllability, Funkcial. Ekvac. 37 (1994), 19{43. [11] G. Barles: Solutions de viscosite des equations de Hamilton-Jacobi, Mathematiques et Applications 17, Springer-Verlag, Paris 1994. [12] A. Bensoussan, Perturbation methods in optimal control, Wiley/Gauthier-Villars, Chichester 1988. [13] A. Bensoussan, L. Boccardo, F. Murat, Homogenization of elliptic equations with principal part not in divergence form and Hamiltonian with quadratic growth, Comm. Pure Appl. Math. 39 (1986), 769{805. [14] A. Bensoussan, J.-L. Lions, and G. Papanicolaou, Asymptotic analysis for periodic structures, North-Holland, Amsterdam, 1978. [15] S. Bortoletto, The Bellman equation for constrained deterministic optimal control problems, Dierential Integral Equations 6 (1993), 905{924. [16] I. Capuzzo-Dolcetta and P.-L. Lions, Hamilton-Jacobi equations with state constraints, Trans. Amer. Math. Soc. 318 (1990), 643{683. [17] M. G. Crandall, H. Ishii, and P.-L. Lions, User's guide to viscosity solutions of second order partial dierential equations, Bull. Amer. Math. Soc. (N.S.) 27 (1992), 1{67. [18] A. L. Dontchev, T. Zolezzi, Well-posed optimization problems, Lecture Notes in Mathematics, 1543. Springer-Verlag, Berlin, 1993. [19] L. Evans, The perturbed test function method for viscosity solutions of nonlinear P.D.E., Proc. Roy. Soc. Edinburgh Sect. A 111 (1989), 359{375. [20] , Periodic homogenisation of certain fully nonlinear partial dierential equations, Proc. Roy. Soc. Edinburgh Sect. A 120 (1992), 245{265. [21] W. H. Fleming and H. M. Soner, Controlled markov processes and viscosity solutions, SpringerVerlag, Berlin, 1993. 23

[22] V. Gaitsgory, Suboptimization of singularly perturbed control systems, SIAM J. Control Optim. 30 (1992), 1228{1249. [23] D. Gilbarg and N. S. Trudinger, Elliptic partial dierential equations of second order, 2nd ed., Springer-Verlag, New York, 1983. [24] K. Horie and H. Ishii, Homogenization of Hamilton-Jacobi equations on domains with small scale periodic structure, Indiana Univ. Math. J. 47 (1998), 1011{1058. [25] H. Ishii, Homogenization of the Cauchy problem for Hamilton-Jacobi equations, in \Stochastic analysis, control, optimization and applications", W.M. McEneaney, G.G. Yin and Q. Zhang eds., 305{324, Birkhauser, Boston 1999. [26] H. Ishii and S. Koike, A new formulation of state constraints problems for rst-order PDEs, SIAM J. Control Optim. 36 (1996), 554{571. [27] H. Ishii and P.-L. Lions, Viscosity solutions of fully nonlinear second-order elliptic partial dierential equations, J. Dierential Equations 83 (1990), 26{78. [28] R. Jensen and P.-L. Lions, Some asymptotic problems in fully nonlinear elliptic equations and stochastic control, Ann. Scuola Norm. Sup. Pisa Cl. Sci. (4) 11 (1984), 129{176. [29] Y. Kabanov, S. Pergamenshchikov, On convergence of attainability sets for controlled two-scale stochastic linear systems, SIAM J. Control Optim. 35 (1997), 134{159. [30] Y. Kabanov, W. Runggaldier, On control of two-scale stochastic systems with linear dynamics in the fast variables, Math. Control Signals Systems 9 (1996), 107{122. [31] P.V. Kokotovic, H.K. Khalil and J. O'Reilly, Singular perturbation methods in control: analysis and design, Academic Press, London 1986. [32] H. J. Kushner, Weak convergence methods and singularly perturbed stochastic control and ltering problems, Birkhauser, Boston 1990. [33] P.-L. Lions, Generalized solutions of Hamilton-Jacobi equations, Pitman, Boston 1982. [34] P.-L. Lions, G. Papanicolaou, S.R.S. Varadhan, Homogeneization of Hamilton-Jacobi equations, Unpublished (1986). [35] M. V. Safonov, On the classical solution of nonlinear elliptic equations of second-order, Math. USSR-Izv. 33 (1989), 597{612, Engl. transl. of Izv. Akad. Nauk SSSR Ser. Mat. 52 (1988). [36] H. M. Soner, Optimal control with state-space constraint I, SIAM J. Control Optim. 24 (1986), 552{561. [37] N. N. Subbotina, Asymptotic properties of minimax solutions of Isaacs-Bellman equations in dierential games with fast and slow motions, J. Appl. Math. Mech. 60 (1996), 883{390. [38] V. Veliov A generalization of the Tikhonov theorem for singularly perturbed dierential inclusions, J. Dynamical Control Systems 3 (1997), 291{319. [39] A. Vigodner Limits of singularly perturbed control problems with statistical dynamics of fast motions, SIAM J. Control Optim. 35 (1997), 1{28.

24