Smoothing Newton and Quasi-Newton Methods for Mixed ...
Smoothing Newton and Quasi-Newton Methods for Mixed Complementarity Problems
Donghui Li 1 Department of Applied Mathematics Hunan University Changsha, China 410082 e-mail: [email protected] Masao Fukushima Department of Applied Mathematics and Physics Graduate School of Informatics Kyoto University Kyoto 606-8501, Japan e-mail: [email protected] February 3, 1999 Abstract The mixed complementarity problem can be reformulated as a nonsmooth equation by using the median operator. In this paper, we rst study some useful properties of this reformulation and then derive the Chen-Harker-Kanzow-Smale smoothing function for the mixed complementarity problem. On the basis of this smoothing function, we present a smoothing Newton method for solving the mixed complementarity problem. The smoothing Newton method converges globally if the problem involves a di erentiable P0 function. Under suitable conditions, the method exhibits a quadratic convergence property. We also present a smoothing Broyden-like method based on the same smoothing function. Under appropriate conditions, the method converges globally and superlinearly.
Key words:
Newton method
Mixed complementarity problem, smoothing function, Newton's method, quasi-
1 Present address (available until October, 1999): Department of Applied Mathematics and Physics, Graduate School of Informatics, Kyoto University, Kyoto 606-8501, Japan, e-mail: [email protected]
1.
Introduction
n
Let F = (F1 ; F2 ; : : : ; Fn ) be a mapping from Rn into itself and X = [li; ui ] be a box in Rn , i=1 where 01 li < ui 1, i = 1; 2; : : : ; n. The variational inequality problem of nding an x 2 X such that F (x)T (y 0 x) 0; 8y 2 X (1.1) is particularly called the mixed complementarity problem [4] and is simply denoted MCP(F ). The mixed complementarity problem is also called the box constrained variational inequality problem (BVIP) [1, 25, 29, 31]. MCP(F ) contains, as a special case, the nonlinear complementarity problem, NCP(F ), of nding an x 2 Rn such that x 0; F (x) 0; xT F (x) = 0; (1.2) which corresponds to the case where li = 0 and ui = +1, i = 1; 2; : : : ; n. Another extreme case of MCP(F ) is the problem with li = 01, i = 1; 2; : : : ; n and ui = +1, i = 1; 2; : : : ; n. In this case, MCP(F ) reduces to the nonlinear equation F (x) = 0: (1.3) Q
The study on iterative methods for solving NCP(F ) and MCP(F ) has been rapidly developed in the last decade. One of the most popular approaches is to reformulate NCP(F ) or MCP(F ) as an equivalent nonsmooth equation so that generalized Newton-type methods can be applied in a way similar to those for smooth equations. For example, consider the KKT system for the MCP(F ): F (x) 0 + = 0; (1.4) 0; x 0 l 0; T (x 0 l) = 0; 0; x 0 u 0; T (x 0 u) = 0: It is easy to see that the KKT system (1.4) can be rewritten as the equation F (x) 0 + (1.5) min(; x 0 l) = 0 min(; 0(x 0 u)) or equivalently min(F (x) + ; x 0 l) = 0: (1.6) min(; 0(x 0 u)) 8 > > < > > :
0
1
B B @
C C A
!
Another nonsmooth equation reformulation of MCP(F ) is the equation midfx 0 l; F (x); x 0 ug = 0: 1
(1.7)
Here and throughout the paper, the operator midfa; b; cg stands for the median of three scalars a; b; c 2 R [f61g. For vectors x; y; z 2 (R [f61g)n, the mid operation is done elementwise. We will focus on the reformulation (1.7) because it does not increase the dimension of the problem. For NCP(F ), (1.7) reduces to the following well known nonsmooth equation: minfx; F (x)g = 0: (1.8) There are a variety of nonsmooth Newton methods based on (1.5) { (1.8) and other reformulations for NCP and MCP. The interested readers may nd details in recent papers such as [8, 9, 11, 15, 22, 23, 27] and the references therein. In this paper, we pay our attention to another approach to NCP and MCP, called smoothing methods. A smoothing method tries to nd a solution of the nonsmooth equation (1.7) by solving a sequence of smooth equations that tend to the original equation (1.7). Smoothing methods for NCP and MCP have also received much attention in recent years [3, 4, 6, 14, 18, 28, 29]. In particular, Chen et al. [6] studied the Jacobian smoothing Newton method and established superlinear convergence of a smoothing Newton method. However, the merit function used in [6] may have an unbounded level set even if F is a uniform P function. Recently, Chen, Chen and Kanzow [1] proposed a new NCP-function by penalizing the Fisher-Burmeister function. The natural merit function of this new NCP-function has all nice features of KanzowYamashita-Fukushima merit function [19] for global convergence. In particular, it provides a bounded level set. By means of this merit function, a smoothing Newton method for solving NCP with global and superlinear convergence was proposed in [1]. Merit functions that possess similar properties have also been proposed recently in [32, 33]. Another interesting smoothing Newton method was proposed by Qi, Sun and Zhou [29] for solving MCP(F ) based on a 2ndimensional semismooth equation reformulation. An advantage of this method is that it only requires the function F to be de ned on the box X . Moreover, though the reformulation is 2n-dimensional, subproblems in the smoothing Newton method reduce to n-dimensional linear equations. However, it is not clear whether it is possible to develop a related quasi-Newton method in which subproblems still remain to be n-dimensional linear equations. In this paper, we present a new smoothing Newton method based on Chen-Harker-Kanzow-Smale smoothing function. The proposed smoothing Newton method retains global and superlinear convergence under the same assumptions as used by Chen et al. [6] and by Qi et al. [29], and the subproblems are n-dimensional linear equations. Moreover, we develop a smoothing quasi-Newton method where subproblems are also n-dimensional linear equations. We show that if F is a di erentiable P0 function with Lipschitz continuous Jacobian, then the smoothing quasi-Newton method converges globally. If in addition, the strict complementarity holds at a limit point, then the convergence rate is superlinear. The paper is organized as follows. In the next section, we introduce the notion of MCP2
function and show some useful properties of it. In Section 3, we present a smoothing function for an MCP function and investigate its properties. In Sections 4 and 5, we propose a smoothing Newton method and a smoothing quasi-Newton method, respectively, for solving MCP, and establish their global and superlinear convergence under appropriate conditions. Throughout the paper, we adopt the following convention: For any real number x, 1+x01 = x, x + 1 0 1 = x, 0 1 1 = 1 1 0 = 0 1 (01) = (01) 1 0 = 0, and 1=1 = 1. 2.
An MCP-Function and Its Properties
In this section, we rst introduce a natural extension of NCP-function. Recall that a function : R2 ! R is called an NCP-function if (a; b) = 0 holds if and only if a 0, b 0 and ab = 0. In other words, a function : R2 ! R is called an NCP-function if (a; b) = 0 is equivalent to minfa; bg = 0. Comparing (1.7) with (1.8), we see that the role of the mid function in MCP is very similar to that of the min function in NCP. Since x 0 l x 0 u for all x, we de ne an MCP-function as follows. De nition 1 Denote R 3 = (R [ f01g) 2 R 2 (R [ f+1g). A function : R 3 ! R is called an MCP-function if the following relation holds:
(a; b; c) = 0; a > c
mid fa; b; cg = 0; a > c: (2.1) Note that we allow a and c to be +1 and 01, respectively. In particular, when c = 01 the relation (2.1) is rewritten as (a; b; 01) = 0 () minfa; bg = 0: In other words, (a; b; 01) is an MCP-function if and only if the function ^(a; b) =4 (a; b; 01) is an NCP function. In this sense, the class of MCP-functions contains NCP-functions. We notice that MCP-function is di erent from the so-called BVIP-function in [25]. For any < , a function ( ; ) : R2 ! R is said to be a BVIP-function if ( ; ) (a; b) = 0 if and only if either a = and b 0, or a = and b 0, or a and b = 0. However, both MCP-function and BVIP-function are closely related to the MCP(F ). Indeed, it is easy to prove the following proposition. 4 Proposition 2.1 Let be an MCP-function and i = (l ;u ) be BVIP-functions for all i = 1; 2; : : : ; n. Then, the following statements are equivalent: ()
i
(i) x is a solution of MCP(F ); (ii) (xi 0 li ; Fi (x); xi 0 ui ) = 0 for each i = 1; 2; : : : ; n; (iii) i (xi ; Fi (x)) = 0 for each i = 1; 2; : : : ; n.
3
i
When a > c, the relation
midfa; b; cg = 0
is equivalent to a+b+c
= maxfa; b; cg + minfa; b; cg = maxfa; bg + minfb; cg = (a + b) +2 ja 0 bj + (b + c) 02 jb 0 cj 2 2 = (a + b) +2 (a 0 b) + (b + c) 02 (b 0 c) ; p
p
or equivalently,
a+c=
q
q
(a 0 b)2 0 (b 0 c)2 : Hence if we de ne the function : R 3 ! R by (a; b; c) = a + c 0 (a 0 b)2 + (b 0 c)2 ; (2.2) then is an MCP-function. Indeed it is not dicult to see that (a; b; c) = 2 midfa; b; cg; whenever a > c. The MCP-function de ned by (2.2) is not new but it will play an important role in this paper. We now reformulate the MCP(F ) by means of . In view of (1.7), let us de ne the function 8(x) = ( 1(x); 2 (x); : : : ; n (x))T , where for each i = 1; 2; : : : ; n i (x) = (xi 0 li ; Fi (x); xi 0 ui ) (2.3) = 2xi 0 (li + ui) 0 (xi 0 li 0 Fi(x))2 + (xi 0 ui 0 Fi(x))2 : Then the MCP(F ) is reformulated as the following nonsmooth equation: 8(x) = 0: (2.4) For the NCP(F ), the function i becomes (2.5) i (x) = xi + Fi (x) 0 (xi 0 Fi (x))2 : when li = 01 and ui = +1, the function i simply reduces to 2Fi. We now show some useful properties of . We rst consider the di erentiability of . It is obvious that and hence 8 are not di erentiable everywhere. Nevertheless, we can show that it has a property called semismoothness. The concept of semismoothness was introduced for scalar q
q
q
q
q
4
functions by Miin [21] and was extended to vector functions by Qi [24]. It has been widely used in the context of nonlinear complementarity problems and variational inequality problems [16, 23, 24]. A locally Lipschitz function 9 : Rn ! Rn is said to be semismooth at a point x if the limit lim 0 fV h0g V0 2 @ 9(x + h ) h !h
exists for any h 2 Rn, where @ 9(x) stands for the Clarke generalized Jacobian of 9 at x [7]. If 9 is semismooth everywhere, we say that 9 is a semismooth function. It is clear that a semismooth function is directionally di erentiable [27]. Let 90 (x; p) denote the directional derivative of 9 at x along direction p. Then the following lemma holds [27]. Lemma 2.1 Let 9 be semismooth at x. Then it holds that for any V 2 @ 9(x + h) and any h ! 0, 9(x + h) 0 9(x) 0 V h = o(khk) (2.6) and
V h 0 90 (x; p) = o(khk):
(2.7)
A locally Lipschitz function 9 is said to be strongly semismooth at x if V h 0 90(x; p) = O(khk2): (2.8) It is not dicult to show that if 9 is strongly semismooth at x, then we have for any V 2 @ 9(x) and any h ! 0, 9(x + h) 0 9(x) 0 V h = O(khk2 ): (2.9) It is obvious that di erentiable functions and strongly semismooth functions are semismooth functions. If 9 has Lipschitz continuous Jacobian, then it is a strongly semismooth function. The following proposition shows the semismoothness of the function 8 de ned by (2.3). Proposition 2.2 Let : R 3 ! R be de ned by (2.2) and 8(x) = ( 1 (x); 2 (x); : : : ; n (x))T be de ned by (2.3). Then (i) is strongly semismooth everywhere; (ii) if F is di erentiable, then 8 is semismooth; (iii) if F has Lipschitz continuous Jacobian, then 8 is strongly semismooth.
It is not dicult to show that the function g : x 7! jxj is strongly semismooth. From the de nition of , we have (a; b; c) = a + c 0 g(a 0 b) + g(b 0 c). By Theorem 19 in [12], it follows that is strongly semismooth. This concludes (i). Since 8 is (strongly) semismooth if and only if its elements i, i = 1; 2; : : : ; n are (strongly) semismooth, (ii) and (iii) follow again from Theorem 19 in [12]. 2 Proof
5
Denote (x) 1(x)
1(x) (x)
Then we have
= fi j xi 0 ui < Fi(x) < xi 0 lig; = fi j Fi (x) = xi 0 lig; 2 (x) = fi j Fi(x) = xi 0 uig; = fi j Fi (x) > xi 0 lig; 2(x) = fi j Fi(x) < xi 0 uig; = 1(x) [ 2 (x); (x) = 1 (x) [ 2 (x):
2Fi(x); if i 2 (x); i (x) = 2(xi 0 li ); if i 2 1(x) [ 1(x); 2(xi 0 ui); if i 2 2(x) [ 2(x): By using the chain rule for generalized derivatives of Lipschitz functions (see Theorem 2.3.9 (iii) in [7]), we have the following expression of @ i(x). Proposition 2.3 Let F : Rn ! Rn be di erentiable. Then we have f2rFi(x)g; if i 2 (x); (2.10) @ i (x) = f(1 + )ei + (1 0 )rFi (x) j 2 [01; 1]g; if i 2 1(x) [ 2 (x); f2eig; if i 2 1 (x) [ 2 (x); 8 > > < > > :
8 > > < > > :
where ei denotes the ith unit vector in Rn . It is clear that @8(x) @C 8(x) =4 @ 1(x) 2 1 11 2 @ n (x) @ 8(x) = @C 8(x) does not necessarily hold.
for any x 2 Rn , though the relation
The following proposition shows that 8 provides a global error bound for the MCP(F ) when F is a Lipschitz continuous uniform P function.
Proposition 2.4 Let F be Lipschitz continuous. If it is a uniform P function, then there are positive constants m M such that
mk8(x) 0 8(y)k kx 0 yk In particular, we have
M k8(x) 0 8(y)k; 8x; y 2 Rn:
kx 0 xk M k8(x)k; 8x 2 Rn ;
where x stands for the unique solution of the MCP(F).
(2.11) (2.12)
First we show that the left-hand inequality in (2.11), i.e., the Lipschitz continuity of 8. It is not dicult to show that for any x 2 Rn, 8(x) = 2fx 0 ProjX (x 0 F (x))g, where ProjX (x) denotes the projection of x onto the box X . The left-hand inequality of (2.11) then follows from the Lipschitz continuity of a projection operator.
Proof
6
Now we prove the right-hand inequality of (2.11). We rst show that there exists a constant c > 0 such that for every i = 1; 2; : : : ; n, and all x; y 2 Rn c(xi 0 yi )(Fi (x) 0 Fi (y)) k8(x) 0 8(y)kkx 0 yk: (2.13) First consider the case i 2 (x) = (x) \ (y) [ (x) \ (y) [ (x) \ (y) . When i 2 (x) \ (y) [ (y) , we have i(x) = 2Fi (x) and i(y) = 2Fi(y). It then follows that (Fi(x) 0 Fi(y))(xi 0 yi) = 12 ( i (x) 0 i(y))(xi 0 yi) 21 k8(x) 0 8(y)k kx 0 yk: When i 2 (x) \ 1 (y), we have i(x) = 2Fi(x) and i(y) = 2(yi 0 li). If xi yi, then we get (Fi(x) 0 Fi(y))(xi 0 yi ) Fi(x) 0 (yi 0 li) (xi 0 yi) = 21 ( i(x) 0 i(y))(xi 0 yi) 21 k8(x) 0 8(y)k kx 0 yk: If xi < yi, then Fi(x) < xi 0 li < yi 0 li < Fi(y). It follows that (Fi(x) 0 Fi(y))(xi 0 yi) = (Fi(x) 0 Fi(y)) (xi 0 li) 0 12 i(y) 21 (Fi(x) 0 Fi(y))( i(x) 0 i(y)) = 21 kF (x) 0 F (y)k k8(x) 0 8(y)k 21 Lk8(x) 0 8(y)kkx 0 yk; where L stands for the Lipschitz constant of F . Similarly, we can show that (2.13) holds for every i 2 (x) \ 2 (y). By similar arguments, it is not dicult to show that (2.13) holds for every i 2 (x) = (x) \ (y ) [ (x) \ (y ) [ (x) \ (y ) and i 2 (x) = (x) \ (y ) [ (x) \ (y ) [ (x) \ (y ) . Since F is a uniform P function, there is a constant c0 > 0 such that max (F (x) 0 Fi(y))(xi 0 yi) c0kx 0 yk2 ; 8x; y 2 Rn : 1in i This together with (2.13) implies k8(x) 0 8(y)kkx 0 yk cc0kx 0 yk2 ; 8x; y 2 Rn : Letting M = cc0, we get the right-hand inequality of (2.11). 2 Proposition 2.4, particularly, (2.12) shows that if F is a Lipschitz continuous uniform P function, then the level set fx j k8(x)k C g is bounded for every C > 0.
7
3.
Chen-Harker-Kanzow-Smale Smoothing Function for MCP
In this section, we consider a smooth function that approximates 8 and show its useful properties. For a scalar , let : R 3 ! R be de ned by (3.1) (a; b; c) = a + c 0 (a 0 b)2 + 2 + (b 0 c)2 + 2 : It is obvious that for any 6= 0, is smooth on R3, and 0(a; b; c) = (a; b; c) holds for any (a; b; c) 2 R3 . It is interesting to notice that though is obtained by smoothing the nonsmooth terms (a 0 b)2 and (b 0 c)2 in , the function 21 is just the Chen-Harker-Kanzow-Smale smoothing function [2, 17, 30], which is derived by letting 2 (s) = 2 (s + 4)3=2 in the Gabriel-More smoothing function [29] q
q
p
p
4 (; c; d; w) =
1 1
Z
midfc; d; w 0 sg(s)ds
with w = b, d = a and = 12 . Moreover, in the case where c = 01, we have q
(a 0 b)2 + 2: Notice that this function is essentially Chen-Harker-Kanzow-Smale smoothing function for NCP. In the following, we show that possesses similar properties to those of Chen-Harker-KanzowSmale smoothing function for NCP. First the following result shows that provides a uniform approximation to . Lemma 3.1 For any 0 > 0, we have j 0 (a; b; c) 0 (a; b; c)j 0 0 ; 8(a; b; c) 2 R 3 : (a; b; 01) = a + b 0
In particular, we have for every > 0
j (a; b; c) 0 (a; b; c)j : 8(a; b; c) 2 R 3:
From the de nition of and , we get j 0 (a; b; c) 0 (a; b; c)j = (b 0 c)2 + (0 )2 0 (b 0 c)2 + 2 0 (a 0 b)2 + (0 )2 + (a 0 b)2 + 2 2 02 2 2 2 02 2 2 = [((bb00cc))2++(()0 )2] 0+ [(b(b00c)c)2++ ]2 0 [((aa00bb))2++(()0 )2] 0+ [(a(a00b)b)2++ ]2
Proof
q
p
q
q
p
p
8
q
p
(0 )2 0 2 (0 )2 0 2 0 (b 0 c)2 + (0 )2 + (b 0 c)2 + 2 (a 0 b)2 + (0 )2 + (a 0 b)2 + 2 02 2 02 2 max (b 0 c)2 + ((0 ))2 +0 (b 0 c)2 + 2 ; (a 0 b)2 + ((0 ))2 0+ (a 0 b)2 + 2 (3.2) 02 2 ()0 +0 = 0 0 :
=
p
p
p
p
o
n
p
p
p
p
2
Lemma 3.1 particularly shows that ! uniformly as ! 0. It is then natural to construct a smoothing function for 8 based on . For each i = 1; 2; : : : ; n, de ne i : Rn ! R by i(x) = 2xi 0 (li + ui ) 0 (xi 0 li 0 Fi (x))2 + 2 + (xi 0 ui 0 Fi (x))2 + 2 : (3.3) Let 8 (x) = ( 1 (x); 2 (x); : : : ; n(x))T . It is obvious that 80(x) 8(x). Moreover, it follows from Lemma 3.1 that for any 0 > 0 k80 (x) 0 8(x)k pn(0 0 ); 8x 2 Rn ; (3.4) and (3.5) k8(x) 0 8(x)k pn; 8x 2 Rn: Note that, for every 6= 0, is di erentiable everywhere whenever F is di erentiable. Speci cally, by direct deduction we get for any 6= 0 and i = 1; 2; : : : ; n xi 0 li 0 Fi (x) r i(x) = 2 0 + xi 0 ui 0 Fi(x2) 2 ei 2 2 (xi 0 li 0 Fi(x)) + (xi 0 ui 0 Fi(x)) + (3.6) xi 0 li 0 Fi (x) xi 0 ui 0 Fi (x) 0 + r Fi (x): (xi 0 li 0 Fi(x))2 + 2 (xi 0 ui 0 Fi(x))2 + 2 q
q
q
q
q
q
Denote for each i = 1; 2; : : : ; n
xi 0 li 0 Fi (x) 0 q xi 0 ui 0 Fi(x2) 2 (xi 0 li 0 Fi(x))2 + 2 (xi 0 ui 0 Fi(x)) +
bi(x) = q
and Then we have or, in a matrix form,
(3.7)
ai(x) = 2 0 bi(x):
(3.8)
r i(x) = ai(x)ei + bi(x)rFi(x);
(3.9)
r8(x) = diag (ai(x)) + rF (x) diag (bi(x)):
(3.10)
9
Lemma 3.2 We have for every 6= 0 and any x 2 Rn
0 ai(x) < 2; 0 < bi (x) 2;
and
2
2
8i 62 1(x) [ 2 (x)
1 ai(x) 1 + 2(u 0 l )2 ; 1 0 2(u 0 l )2 bi(x) 1; i i i i
8i 2 1 (x) [ 2 (x):
(3.11) (3.12)
If either li or ui is nite, then all inequalities in (3.11) hold strictly.
Proof If li = 01 and ui = 1, then 1 (x) = 2 (x) = ;. Moreover, we have for each i 62 1(x) [ 2(x), bi(x) = 2 and ai (x) = 0, and hence (3.11) holds. Now, assume that either li or ui is nite. It is not dicult to see that for every xed 6= 0, the function t 7! pt2t+2 is increasing with t. It then follows that x 0 u 0 F (x) xi 0 li 0 Fi (x) q ; >q i i i (xi 0 li 0 Fi(x))2 + 2 (xi 0 ui 0 Fi(x))2 + 2 which implies that bi(x) > 0 and hence ai (x) < 2 for each i. On the other hand, it is obvious that for any 6= 0 xi 0 ui 0 Fi (x) xi 0 li 0 Fi (x) and (3.13) 1 1: q q 2 2 2 2 (xi 0 li 0 Fi(x)) + (xi 0 ui 0 Fi(x)) + Moreover, at least one of the inequalities in (3.13) must hold strictly because at least one of li and ui is nite. This yields bi(x) < 2 and hence ai(x) > 0. Thus we have (3.11). Note that, in
this case, the inequalities in (3.11) hold strictly. Next, we prove (3.12). If li or ui is in nite, then it follows from (3.7) that ai(x) = bi (x) = 1 for every i 2 (x). Suppose that both li and ui are nite. Then, we have for each i 2 (x) ui 0 li bi(x) = (ui 0 li)2 + 2 1: Moreover, we have ui 0 l i bi(x) = 1 0 1 + (u 0 l )2 + 2 p
p
= 10
i
i
2 (ui 0 li)2 + 2 + (ui 0 li) p(ui 0 li)2 + 2
p
1 0 2(u 10 l )2 2: i i
By the de nition of ai (x), it is not dicult to complete the proof. 2 The inequality (3.5) implies that 8 approximates 8 uniformly. In the following, we will see that r8 (x) serves as a good approximation to an element of @C 8(x). That is, r8 and r8 satisfy the Jacobian consistence property introduced by Chen, Qi and Sun [6]. 10
De nition 2 Let 9 = ( 1 ; 2 ; : : : ; n)T : Rn ! Rn be a Lipschitz continuous function. We call 9 : Rn ! Rn a smoothing approximation function of 9 if 9 is continuously di erentiable and there is a constant > 0 such that for any x 2 Rn and > 0
k9(x) 0 9(x)k :
(3.14)
lim dist(r9(x)T ; @C 9(x)) = 0; #0
(3.15)
Furthermore, if for any x 2 Rn
then we say 9 and 9 satisfy the Jacobian consistence property, where dist(r9 (x)T ; @C 9(x)) denotes the distance from the matrix r9 (x)T to the set of matrices @C 9(x) de ned by
@C 9(x) = @ 1 (x) 2 @ 2 (x) 2 1 11 2 @
n
(x):
From (3.5), it is obvious that the function 8 de ned by (3.3) is a smoothing function of 8 de ned by (2.3). We now show that 8 and 8 satisfy the Jacobian consistence property. To this end, we construct the matrix V (x) = (v1 (x); v2 (x); : : : ; vn (x)), where vectors vi (x) 2 Rn , i = 1; 2; : : : ; n, are de ned by 2rFi(x); i 2 (x); vi (x) = ai (x)ei + bi (x)rFi (x); i 2 1(x) [ 2 (x); (3.16) 2e i ; i 2 1 (x) [ 2 (x): Since ai (x) and bi (x) satisfy (3.8) and (3.11), it is clear from (2.10) and (3.16) that vi(x) 2 @ i(x) for each i = 1; 2; : : : ; n and any 6= 0, and hence V (x) 2 @C 8(x). By (3.8), we get ai(x)(ei 0 rFi (x)); i 2 (x); r i(x) 0 vi(x) = 0; (3.17) i 2 1(x) [ 2 (x); bi (x)(rFi (x) 0 ei ); i 2 1(x) [ 2(x): It is easy to see by the de nition of (3.7) and (3.8) of ai(x) and bi (x) that 8 > > < > > :
8 > > < > > :
lim b(x) = lim(2 0 ai(x)) = 0; 8i 2 1 (x) [ 2 (x); #0 i #0 and
lim a(x) = lim(2 0 bi(x)) = 0; 8i 2 (x): #0 i #0 We then deduce from (3.17) that for every i = 1; 2; : : : ; n lim (r i(x) 0 vi (x)) = 0: #0 11
(3.18)
Let V (x) = (v1(x); v2 (x); : : : ; vn (x)) =4 lim#0 V (x). Then we get from (3.12) and (3.16) 2rFi(x); i 2 (x); vi (x) = ei + rFi (x); i 2 1 (x) [ 2(x); 2e i ; i 2 1 (x) [ 2 (x): It is not dicult to see that V (x) 2 @C 8(x). Therefore, (3.18) implies lim dist(r8(x); @C 8(x)) lim kr8 (x) 0 V (x)k = 0: #0 #0 8 > > < > > :
The above discussion has shown the following proposition.
Proposition 3.1 Let F be continuously di erentiable. Then consistence property.
8
and
8
satisfy the Jacobian
In the next proposition, we give sucient conditions to ensure the nonsingularity of r8 (x). Proposition 3.2 Let F be continuously di erentiable. Then r8(x) is nonsingular for every 6= 0 if one of the following conditions holds. (i) rF (x) is a P matrix; (ii) rF (x) is a P0 matrix and, for each i 62 1(x) [ 2 (x), either li or ui is nite. Proof (i) From Lemma 3.2, ai (x) 0 and bi(x) > 0 hold for every i and any x. Therefore, in view of (3.10), the nonsingularity of r8 (x) is equivalent to the nonsingularity of the matrix diag ai (x)bi(x)01 + rF (x). If rF (x) is a P matrix, then so is diag ai (x)bi(x)01 + rF (x) and hence nonsingular. If rF (x) is a P0 matrix and for each i 62 1(x) [ 2 (x), either li or ui is nite, then by Lemma 3.2 again, the matrix diag ai(x)bi(x)01 is a diagonal matrix with positive diagonal elements, and hence the nonsingularity of the matrix diag ai(x)bi(x)01 + rF (x) follows from Lemma 5.1 in De Luca et al. [8]. 2 To conclude this section, we show that if F is a continuously di erentiable uniform P function, then the smooth equation 8(x) = 0 (3.19) has a unique solution x for each > 0, and x tends to the unique solution of MCP(F ) as ! 0.
Lemma 3.3 Let F be continuously di erentiable. If it is a uniform P function. then there is a constant M > 0 such that for any 0
kx 0 xk M pn;
where x is the unique solution of MCP(F ). In particular, we have
12
(3.20) lim#0 x = x.
We rst prove the existence of a unique solution of (3.19). By Proposition 2.4, the function k8k has bounded level sets. Since 8 satis es (3.5), it follows that the level set
= fx j k8(x)k C g is nonempty and bounded for an arbitrary constant C such that C > inf x k8(x)k. Moreover, it is clear that k8k2 attains its in mum at some point x 2 int . Since x satis es r8(x)8(x) = 0 and r8(x) is nonsingular by Proposition 3.2, we must have 8 (x ) = 0. This shows that (3.19) has a solution. The uniqueness of the solution of (3.19) follows from the nonsingularity of r8(x) for every x. Next, we show that x converges to x. Since x is the unique solution of MCP(F ), it is the unique zero point of 8. So, it follows from (2.12) in Proposition 2.4 that kx 0 xk M k8(x )k = M k8(x) 0 8(x )k M pn; where the last inequality follows from . 2 Proof
4.
A Smoothing Newton Method and Its Convergence
In this section, we propose a smoothing Newton method based on 8 for solving the MCP(F ). For simplicity, we abbreviate 8 , i , etc. as 8k , ki , etc., respectively. In the rest of the paper, we assume that F is continuously di erentiable. In the algorithm, the subproblem is the following linear equation r8k (xk )p + 8(xk ) = 0: (4.1) This type of subproblems has been used by many authors (see e.g., [6, 26, 29]). For line search, it is natural to use the squared residual function 41 k 2 (4.2) 'k (x) = 2 k8 (x)k : as the merit function. Generally, in Newton's method, the steplength k is determined by 'k (xk + k pk ) 'k (xk ) + k r'k (xk )T pk or 'k (xk + k pk ) 'k (xk ) + 2k '(xk ); where '(x) = '0(x), pk is the solution of (4.1), and > 0 is a suitable constant. Although it is possible to construct a smoothing Newton method with those line searches, we here adopt a di erent line search that is also e ective in the smoothing quasi-Newton method to be presented in the next section. Let fk g be a positive sequence satisfying k
k
1
X
k=1
k < 1;
13
(4.3)
where is a given positive constant. At iteration k, we determine a steplength k > 0 so that the following inequality holds for = k : k8k (xk + pk )k 0 k8k (xk )k 0kpk k2 + k ; (4.4) where > 0 is a constant. Clearly, at each iteration k, (4.4) holds for all > 0 suciently small since k is positive and independent of . Let 1i(x) = minf(xi 0 li 0 Fi(x))2 ; (xi 0 ui 0 Fi(x))2 g, i = 1; 2; : : : ; n, and 1(x) = minf1i(x) j i 2 (x) [ 1(x) [ 2(x)g: (4.5) The proposed smoothing Newton method is stated as follows.
Algorithm 1 (Smoothing Newton Method): Step 0. Choose constants 1 ; 2 2 (0; 1), 0 < < minf p1 ; p2 g, 1 ; 2 > 0. Select a 2 n n positive sequence fk g satisfying (4.3). Choose an initial point x1 2 Rn and a positive constant 1 2 k8(x1 )k. Let k := 1. Step 1. Stop, if 8(xk ) = 0. Otherwise, solve the linear equation (4.1) to get pk . Step 2. If k8k (xk + pk )k 2 k8k (xk )k 0 1kpk k2 ; (4.6)
then let k := 1 and go to Step 4. Step 3. Let k be the maximum number in the set f1; 1 ; 21 ; : : : ; g such that = i1 satis es the line search condition (4.4) with = 2. Step 4. Let xk+1 := xk + k pk . Step 5. Update k by the following rule: If (4.6) holds or k8(xk+1)k k , let
1 k+1 := minf k8(xk+1 )k; k ; 1(xk+1 )g; (4.7) 2 2 where 1(x) is de ned by (4.5). Otherwise, let k+1 := k . Step 6. Let k := k + 1. Go to Step 1. Remark (i) It is not dicult to see that if Algorithm 1 generates an in nite sequence fxk g, then 8k (xk ) 6= 0 for all k, and the positive sequence fk g is nonincreasing and satis es
k k8(xk )k;
It then follows from (3.5) that
(4.8)
8k: p
k k8(xk )k (k8k (xk )k + nk ):
Since pn < 12 by Step 0, we get
k k8k (xk )k;
14
(4.9)
where = =(1 0 pn ) 2 (0; p1n ). (ii) De ne index set K = K1 [ K2 , where K1 = fk j k8(xk+1)k k g; K2 = fk j k8k (xk + pk )k 2 k8k (xk )k 0 1 kpk k2 g:
(4.10)
Then, k 12 k01 if k 0 1 2 K , and k = k01 if k 0 1 62 K . Lemma 4.1 Let fxk g be generated by Algorithm 1. Then for every k , we have
k8k+1(xk+1 )k 2 k8k (xk )k 0 1kxk+1 0 xk k2 + pn(k 0 k+1)
(4.11)
if k is determined by Step 2, and
k8k+1(xk+1)k k8k (xk )k 0 2 kxk+1 0 xk k2 + pn(k 0 k+1 ) + k
(4.12)
if k is determined by Step 3. In particular, we have for every k
k8k+1(xk+1)k k8k (xk )k 0 kxk+1 0 xk k2 + pn(k 0 k+1 ) + k ;
where = minf1 ; 2 g. Consequently, we have 1 X kxk+1 0 xk k2 < 1: k=1
(4.13) (4.14)
Moreover, the sequence fk8k (xk )kg converges.
Proof
By (3.4), we clearly have for every k
k8k+1(xk+1)k k8k (xk+1)k + pn(k 0 k+1):
(4.15) If k is determined by Step 2, then (4.11) follows from (4.6) and (4.15). Since 2 2 (0; 1), (4.11) implies (4.13). If k is determined by Step 3, then p(4.12) follows from (4.15) and (4.4) with = 2 , and hence (4.13) holds. Let us denote k = n(k 0 k+1) + k > 0. Since k satis es 1 (4.3), we have k < 1. Therefore, (4.14) follows from (4.13) and, by Lemma 3.3 in [10], k=1 fk8k (xk )kg converges. 2 P
Lemma 4.2 Let fxk g be generated by Algorithm 1. Then the following four statements are equivalent. (i)
lim inf k8(xk )k = 0: k!1 15
(ii)
liminf k8k (xk )k = 0: k!1
(iii)
lim k8(xk )k = 0:
k!1
(iv)
lim k8k (xk )k = 0:
k!1
Since fk8k (xk )kg converges, by Lemma 4.1, (ii) and (iv) are equivalent. From (3.5) and (4.8), we have k8(xk )k k8k (xk )k + pnk k8k (xk )k + pnk8(xk )k: This implies (4.16) k8(xk )k 1 0 1pn k8k (xk )k: Similarly we have k8k (xk )k k8(xk )k + pnk (1 + pn)k8(xk )k: This together with (4.16) shows the equivalence between (i) and (ii) as well as the equivalence between (iii) and (iv). 2 Lemma 4.2 reveals that fxk g has an accumulation point that is a solution of MCP(F ), and every accumulation point of fxk g is a solution of MCP(F ).
Proof
Theorem 4.1 Let fxk g be generated by Algorithm 1. If the index set K de ned by (4.10) is in nite, then we have lim k8(xk )k = 0; (4.17) k!1
and every accumulation point of fxk g is a solution of MCP(F ).
The assumption that K is in nite means that either K2 is in nite, or K2 is nite but K1 is in nite. In the rst case, by Lemma 4.1, (4.11) holds in nitely often. Since 2 2 (0; 1), this implies limk!1 k8k (xk )k = 0. Then (4.17) follows from Lemma 4.2. Now consider the latter case. Without loss of generality, we assume that K1 = K = fk1 < k2 < 11 1g. By Step 5, we have k8(xk +1)k 01k = 01k 01+1 21 k8(xk 01 +1)k i01 11 1 21 k8(xk1+1)k: Proof
i
i
i
16
i
This shows that lim inf k!1 k8(xk )k = 0. Consequently, by Lemma 4.2 again, we get (4.17). Hence every accumulation point of fxk g is a zero point of 8, or equivalently, a solution of MCP(F ). 2 Theorem 4.1 shows that to prove global convergence of Algorithm 1, it suces to verify that the index set K de ned by (4.10) is in nite. To this end, we need the following assumption. Assumption A For each > 0, r8(x) is nonsingular for any x. Sucient conditions for Assumption A to hold are given in Proposition 3.2. The following theorem shows that under Assumption A, the index set K is in nite. Theorem 4.2 Let Assumption A hold and fxk g be generated by Algorithm 1. Then the index set K de ned by (4.10) is in nite and hence (4.17) holds. Consequently, every accumulation point of fxk g is a solution of MCP(F ).
We assume to the contrary that K is nite. Then by Step 5 of Algorithm 1, there is a positive integer k such that k = k holds for all k k, which in turn implies that 8k = 8k for all k k. Moreover, we have by (4.8), k8(xk )k 01 k > 0 for all k k and hence by Lemma 4.2, k8k (xk )k 0 for all k suciently large and some constant 0 > 0. We denote 8k by 8 . Since K2 is nite, k is determined by Step 3 for all k suciently large. So, we have for all k suciently large k8 (xk + k pk )k 0 k8 (xk )k 02kk pk k2 + k : (4.18) This particularly implies that kk pk k ! 0 as k ! 1. Let x be an accumulation point of fxk g and fxk gk2K be a subsequence converging to x. Since 8 is continuously di erentiable and r8 (x) is nonsingular, it follows that when k 2 K is suciently large, r8 (xk ) is uniformly nonsingular. Therefore, (4.1) implies that fpk gk2K is bounded. Without loss of generality, we assume that fpk gk2K converges to some vector p. Let = limsupk2K;k !1 k . Then 2 [0; 1]. If > 0, then p = 0. So it follows from (4.1) that 8(x) = 0. This is a contradiction. Now, suppose = 0, or equivalently, limk2K;k !1 k = 0. By Step 3 of Algorithm 1, when k 2 K is suciently large, 4 k =1 does not satisfy (4.18). So, we have 0k = k8 (xk + 0k pk )k 0 k8 (xk )k 02 k0k pk k2: Multiplying both sides by (0k )01 k8 (xk + 0k pk )k + k8 (xk )k and then taking the limit as k ! 1 with k 2 K yield 28 (x)T r8 (x)p 0: It then follows from (4.1) that 8 (x)T 8(x) 0: Proof
17
This together with (4.8) implies (1 0 n 2 )k8(x)k2 k8 (x)k2 + (1 0 n 2)k8(x)k2 k8 (x)k2 + k8(x)k2 0 n2 = k8 (x) 0 8(x)k2 0 n2 + 28 (x)8(x) 28 (x)T 8(x) 0; (4.19) where the third inequality follows from (3.5). Since < p1n , (4.19) is also a contradiction. Therefore, K must be in nite, and hence the assertion follows from Theorem 4.1. 2 De ne the level set (4.20)
= fx j k8(x)k 47 k8(x1 )k + g; where the positive constant is given in (4.3). The following theorem shows that if is bounded, then there is at least an accumulation point of fxk g that solves MCP(F ). Theorem 4.3 Let Assumption A hold and the level set be contained in a bounded convex set .
Then the sequence fxk g generated by Algorithm 1 has at least one accumulation point. Moreover, every accumulation point of fxk g is a solution of MCP(F ).
From Theorem 4.2, it suces to show that fxk g is bounded. By the boundedness of , it then suces to verify that fxk g . Indeed, from (4.13), we have for every k k8k+1(xk+1)k k8k (xk )k + pn(k 0 k+1 ) + k 11 1 k81 (x1)k + pn1 + k8(x1 )k + k81(x1 ) 0 8(x1 )k + pn1 + k8(x1 )k + 2pn1 + ; where the last inequality follows from (3.5). This implies
Proof
k8(xk+1)k k8k+1(xk+1)k + k8k+1(xk+1) 0 8(xk+1)k k8(x1 )k + 2pn1 + + pnk+1 k8(x1 )k + 3pn1 + k8(x1 )k + 23 pnk8(x1 )k + 47 k8(x1)k + ;
(4.21) where the last two inequalities follow from Step 0 of Algorithm 1. Since (4.21) holds for all k, we get fxk g as desired. 2 By Proposition 2.4, if F is a uniform P function, then the level set is bounded. So, we have the following corollary. Corollary 4.1 Let F be a continuously di erentiable uniform P function. Then fxk g generated by Algorithm 1 converges to the unique solution of MCP(F ).
18
V
We say that a semismooth function 9 : Rn ! Rn is BD-regular at a point x if every matrix 2 @ 9B (x) is nonsingular, where @B 9(x) is the B-di erential of 9 at x (see [24]). By using BD-regularity, we can prove a strong convergence property for Algorithm 1.
Theorem 4.4 Let Assumption A hold and fxk g be generated by Algorithm 1. Suppose that there is an accumulation point x3 of fxk g at which 8 is BD-regular. Then the whole sequence fxk g converges to x3 .
The BD-regularity implies that x3 is an isolated solution of MCP(F ) and hence an isolated accumulation point of fxk g ([24], Proposition 2.5). Since xk+1 0 xk ! 0 by Lemma 4.1, the whole sequence fxk g converges. 2 We now turn to proving superlinear/quadratic convergence of Algorithm 1. We rst prove a useful lemma. Lemma 4.3 Let V (x) be de ned by (3.16) and 1(x) be de ned by (4.5). If 0 < 1(x), then Proof
we have
kr8(x) 0 V (x)k pn(1 + krF (x)k):
(4.22)
It suces to show that for every i = 1; 2; : : : ; n kr i (x) 0 vi(x)k (1 + krF (x)k): (4.23) First, we have for any scalar t p 2 2 j 1 0 1 (4.24) 1 0 p1 + t2 = p1 +1 +t2 t j j1 0 1 + t2 j = 1 + pt 1 + t2 12 t2 : For i 2 1 (x), we have xi 0 li 0 Fi(x) < 0 and xi 0 ui 0 Fi(x) < 0 . Letting t = 0 xi 0 li 0 Fi(x) in (4.24), we get 2 xi 0 li 0 Fi (x) 1(x) 1 : + 1 2 2 2 2 2(xi 0 li 0 Fi(x)) 2(xi 0 li 0 Fi (x)) 2 (xi 0 li 0 Fi(x)) + Similarly, letting t = 0 xi 0 ui 0 Fi(x) in (4.24), we get 1 xi 0 ui 0 Fi (x) (xi 0 ui 0 Fi(x))2 + 2 + 1 2 : Therefore, we get for every i 2 1(x) jbi(x)j = (x x0i 0l l0i 0F F(xi ())x2) + 2 + 1 0 (x x0i 0u u0i 0F F(xi ())x2) + 2 0 1 i i i i i i xi 0 ui 0 Fi (x) xi 0 li 0 Fi (x) (x 0 l 0 F (x))2 + 2 + 1 + (x 0 u 0 F (x))2 + 2 + 1 i i i i i i Proof
p
p
p
p
p
p
:
19
p
This together with (3.17) implies that (4.23) holds for every i 2 1 (x). By a similar discussion, we can show that (4.23) holds for every other i. The proof is then complete. 2 Let K be de ned by (4.10). Then, by Step 5 of Algorithm 1, for every k with k 0 1 2 K , we have k 1(xk ). Since k k8(xk )k holds for all k, Theorem 4.2 implies limk!1 k = 0. So, we get the following corollary from Theorem 4.2 and Lemma 4.3. Corollary 4.2 Let the conditions of Theorem 4.2 hold. Then
lim kr8k (xk ) 0 Vk k = 0;
k!1 k012K
(4.25)
where Vk = V k (xk ) is de ned by (3.16).
The following theorem shows another good property of the smoothing function 8 . Theorem 4.5 Let Assumption A hold and fxk g be generated by Algorithm 1. Suppose that fxk g converges to x3 . Then we have
lim dist(r8k (xk ); @C 8(x3 )) = klim dist(Vk ; @C 8(x3 )) = 0; !1
k!1 k01 2 K
(4.26)
where Vk = V k (xk ) is de ned by (3.16).
Since (4.25) holds, we only need to verify the last equality. Let I1 [ I2 [ I3 be an arbitrary partition of the index set (x3 ) = 1 (x3 ) [ 2 (x3 ). That is, Ii \ Ij = ;, i 6= j, i; j = 1; 2; 3 and I1 [ I2 [ I3 = (x3 ). Since, by Proposition 2.3, f2ei ; 2rFi (x3 ); ei + rFi (x3 )g @ i (x3 ) for each 4 i 2 (x3 ), @C 8(x3 ) contains matrix V = (v1 ; v2 ; : : : ; vn) with 2rFi(x3); i 2 (x3 ) [ I1 ; (4.27) vi = 2 e i ; i 2 1(x3 ) [ 2 (x3) [ I2; ei + rFi (x3 ); i 2 I3 : We verify the last equality of (4.26) by showing that every accumulation point of fVk g is in @C 8(x3 ). It suces to show that for an arbitrary accumulation point V 3 = (vi3 ; v23 ; : : : ; vn3 ) of fVk g, there is a partition I1 [ I2 [ I3 of (x3) such that, for each i, vi3 is expressed in the form of (4.27). Let V 3 be an accumulation point of fVk g, and fVk gk2K be a subsequence converging to V 3 . For every i 2 (x3 ), we have i 2 (xk ) for all k 2 K suciently large. Therefore, from (3.16), we get vik = 2rFi(xk ) for every i 2 (x3 ) and all k suciently large, and hence vi3 = 2rFi (x3 ) for every i 2 (x3 ). Similarly, we have vi3 = 2ei for each i 2 (x3). Now, consider the indices i 2 (x3 ). By taking a further subsequence of fxk gk2K if necessary, we can partition the index (x3) as I1 , I2 , I3 such that I1 = fi 2 (x3 ) j i 2 (xk ) for all k 2 K g, Proof
8 > > < > > :
20
for all k 2 K g, I3 = fi 2 (x3 ) j i 2 (xk ) for all k 2 K g. Then, taking into account (3.12), we see from (3.16) that vi3 = 2rFi(x3 ) for each i 2 I1 , vi3 = 2ei(x3 ) for each i 2 I2 , and vi3 = ei + rFi(x3) for each i 2 I3 . Hence V 3 = (v13 ; v23 ; : : : ; vn3 ) is represented in the form of (4.27). 2 Now, we show superlinear/quadratic convergence of Algorithm 1. I2 = fi 2 (x3 ) j i 2 (xk )
Theorem 4.6 Let Assumption A hold and fxk g be generated by Algorithm 1. Suppose that there is an accumulation point x3 of fxk g such that all matrices in @C 8(x3 ) are nonsingular. Then fxk g converges to x3 superlinearly. If in addition, rF is Lipschitz continuous at x3, then the convergence rate is quadratic.
It is clear that the given conditions imply that 8 is BD-regular at x3 . Therefore, by Theorem 4.4, fxk g ! x3 . Let K be de ned by (4.10). By Theorem 4.5, there is a constant M1 > 0 such that kr8k (xk )01 k M1 for all k suciently large with k 0 1 2 K . So, it follows from (4.1) that for every k with k 0 1 2 K
Proof
kxk + pk 0 x3 k = k 0 r8k (xk )01 8(xk ) 0 8(x3 ) 0 r8k (xk )(xk 0 x3) k kr8k (xk )01 k k8(xk ) 0 8(x3 ) 0 Vk (xk 0 x3 )k + k(r8k (xk ) 0 Vk )(xk 0 x3 )k
n
X
M1 k i(xk ) 0 i(x3) 0 (vik )T (xk 0 x3 )k + kr8k (xk ) 0 Vk k kxk 0 x3 k i=1 = o(kxk 0 x3 k); (4.28)
where Vk = V (xk ) is given by (3.16), and the last equality follows from (4.25) and the fact that vik 2 @ i (xk ) together with the semismoothness of i . Then we have for all k suciently large with k 0 1 2 K k8k (xk + pk )k k8k (xk + pk ) 0 8(xk + pk )k + k8(xk + pk ) 0 8(x3 )k pnk + O(kxk + pk 0 x3 k) pnk8k (xk )k + o(kxk 0 x3 k) = 2 k8k (xk )k 0 (2 0 pn )k8k (xk )k + o(kxk 0 x3 k); (4.29) where the second inequality follows from (3.5) and the local Lipschitzian property of 8, and the third inequality follows from (4.9) and (4.28). On the other hand, since 8 is BD-regular at x3 , there is a constant m > 0 such that k8(x)k = k8(x) 0 8(x3 )k mkx 0 x3 k for all x near x3 . So, it follows that for all k suciently large with k
21
k012 K
k8k (xk )k
k8(xk )k 0 k8k (xk ) 0 8(xk )k k8(xk )k 0 pnk (1 0 pn )k8(xk ) 0 8(x3 )k (1 0 pn )mkxk 0 x3k = kxk 0 x3k;
(4.30) where = (1 0 pn )m > 0 and the third inequality follows from (4.8). Also it follows from (4.1) that for all k suciently large with k 0 1 2 K kpk k kr8k (xk )01k k8(xk )k M1 k8(xk ) 0 8(x3 )k M1 Lkxk 0 x3 k; (4.31) where L > 0 stands for the Lipschitz constant of 8 at x3. Therefore, it follows from (4.29), (4.30), and (4.31) that for all k suciently large with k 0 1 2 K k8k (xk + pk )k 0 2 k8k (xk )k + 1kpk k2 0(2 0 pn )k8k (xk )k + o(kxk 0 x3 k) 0: (4.32) This shows that (4.6) holds, and hence k 2 K . By induction, we claim that k 2 K for all k suciently large. Moreover, by Step 2 in Algorithm 1, (4.32) means that k = 1 is accepted. Consequently, (4.28) shows the superlinear convergence of fxk g. If rF is Lipschitz continuous at x3, then 8 is strongly semismooth at x3 . By means of (4.22) and (4.8), we have kr8k (xk ) 0 Vk k = O(kxk 0 x3 k2 ). Therefore, in a similar way to (4.28), we can deduce kxk + pk 0 x3 k = O(kxk 0 x3 k2 ) and hence, kxk+1 0 x3k = O(kxk 0 x3k2 ). That is, fxk g converges to x3 quadratically. 2 The proof of Theorem 4.6 shows that the index set K de ned by (4.10) contains all k suciently large. Therefore, we get the following corollary from Theorem 4.5. Corollary 4.3 Let the conditions of Theorem 4.4 hold. Then we have
lim dist(r8k (xk ); @C 8(x3 )) = 0: Since @C 8(x3 ) is closed, Corollary 4.3 shows that any accumulation point of fr8k (xk )g is an element of @C 8(x3 ). In particular, if 8 is di erentiable at x3 , then @C 8(x3 ) reduces to a singleton, and hence lim r8k (xk ) = r8(x3 ): k!1 k!1
5.
A Smoothing Quasi-Newton Method
In this section, based on the smoothing Newton method presented in Section 4, we propose a smoothing Broyden-like method for solving MCP(F ). The basic idea is to use a nonsingular matrix Bk instead of r8k (xk ) in Algorithm 1. The algorithm is stated as follows. 22
Algorithm 2 (Smoothing Broyden-like Method) Step 0. Choose constants 1 ; 2 2 (0; 1), 0 < < minf p1 ; p2 g, 1 ; 2 > 0. Choose an 2 n n n initial point x1 2 R , an n 2 n nonsingular matrix B1 , and positive constants 1 2 k8(x1 )k and 1 k8(x1 )k. Let k := 1. Step 1. Stop, if 8(xk ) = 0. Otherwise, solve the linear equation
to get pk . Step 2.
If
Bk p + 8(xk ) = 0
(5.1)
k8k (xk + pk )k 2 k8k (xk )k 0 1kpk k2 ;
(5.2)
then let k := 1 and go to Step 4. Step 3. Let k be the maximum number in the set f1; 1; 21 ; : : : ; g such that = i1 satis es the line search condition (4.4) with = 2 . Step 4. Let xk+1 := xk + k pk . Step 5. Update Bk by the Broyden-like update formula (y 0 Bk sk )sTk ; B = B + k k+1
k
k
ksk k2
where sk := xk+1 0 xk and yk := 8k (xk+1 ) 0 8k (xk ). The parameter k is chosen so that jk 0 1j for some constant 2 (0; 1) and Bk+1 is nonsingular. Step 6. Update k by the following rule: If (5.2) holds or k8(xk+1)k k , let
1 k+1 := minf k8(xk+1 )k; k g: 2 2 Otherwise, let k+1 := k . Step 7. Let k+1 := minf 21 k ; k8(xk+1 )kg and k := k + 1. Go to Step 1. Remark (i) Since k 12 k01 for all k, it is obvious that (4.3) holds. (ii) Similar to Algorithm 1, we have for every k k k8(xk )k; k k8k (xk )k and k k8(xk )k; (5.3) where = 1 0 pn 2 (0; p1n ) (see (4.16)). (iii) We still denote K = K1 [ K2 with K1 = fk j k8(xk+1 )k k g; K2 = fk jk8k (xk + pk )k 2k8k (xk )k 0 1 kpk k2g: (5.4) Then, k 21 k01 holds for every k with k 0 1 2 K , and k = k01 holds for every k with k 0 1 62 K . 23
(iv) It is not dicult to see that the sequence fxk g generated by Algorithm 2 is contained in the level set de ned by (4.20). Moreover, Lemmas 4.1, 4.2 and Theorem 4.1 remain true for Algorithm 2. This means that to show global convergence of Algorithm 2, it suces to verify that K de ned by (5.4) is in nite. In order to establish global convergence for Algorithm 2, we need the following assumption. Assumption B (i) The level set de ned by (4.20) is contained in a bounded convex set . (ii) The function F is continuously di erentiable and rF is Lipschitz continuous on . For given > 0 and > 0, de ne the set 5() = fx j (xi 0 li 0 Fi(x))2 + 2 ; (xi 0 ui 0 Fi(x))2 + 2 ; i = 1; 2; : : : ; ng \ : (5.5) The following lemma shows that under Assumption B, r8 is Lipschitz continuous on 5 (). Lemma 5.1 Let Assumption B hold. Then for any > 0 and > 0, there exists a positive constant L independent of , such that
kr8(x) 0 r8(y)k L kx 0 yk; 8x; y 2 5():
(5.6) We prove (5.6) by showing that there is a positive constant l such that for each i =
Proof 1; 2; : : : ; n
kr i (x) 0 r i (y)k l kx 0 yk; 8x; y 2 5():
We rst show that there is a constant l0 > 0 such that for each i = 1; 2; : : : ; n jai (x) 0 ai(y)j = jbi(x) 0 bi (y)j l0 kx 0 yk; 8x; y 2 5(): By an elementary deduction, we get for any scalars t1 ; t2 and > 0 t1 t2 jt1 0 t2 j + jt j 1 0 1 0 2 t21 + 2 t22 + 2 t21 + 2 t21 + 2 t22 + 2 j t2 + 2 0 t2 + 2j = jt1 20 t2 j2 + jt2 j 1 2 2 2 2 2 t1 + t1 + t2 + 21 2 jt1 0 t2 j + j t21 + 2 0 t22 + 2 j t1 + 2 2 = 21 2 jt1 0 t2 j + 2 jt12 0 t2 j 2 2 t1 + t1 + + t2 + 21 2 jt1 0 t2 j + (jt1j +jt jjt2+j)jjtt1j0 t2 j 1 2 t1 + = 2jt12 0 t22j : t1 + q
q
q
q
q
q
(5.8)
q
q
q
(5.7)
q
q
q
q
q
q
q
q
q
24
So, we get by the de nition (3.7) of bi jbi(x) 0 bi(y)j xi 0 li 0 Fi (x) xi 0 ui 0 Fi (x) 0 = 2 2 (xi 0 li 0 Fi(x)) + (xi 0 ui 0 Fi(x))2 + 2 0 (y y0i 0l l0i 0F F(yi())y2) + 2 0 (y y0i 0u u0i 0F F(yi())y2) + 2 i i i i i i xi 0 li 0 Fi (x) yi 0 li 0 Fi (y ) (x 0 l 0 F (x))2 + 2 0 (y 0 l 0 F (y))2 + 2 i i i i i i yi 0 ui 0 Fi (y ) xi 0 ui 0 Fi (x) + (x 0 u 0 F (x))2 + 2 0 (y 0 u 0 F (y))2 + 2 i i i i i i 2j(xi 0 li(0x F0i(lx))0 0F ((yxi))02 l+i 02Fi(y))j + 2j(xi 0 ui(x0 F0i(ux))00F ((yxi))02 u+i 02 Fi(y))j i i i i i i 4 jxi 0 yij + jFi (x) 0 Fi(y)j : Since F is continuously di erentiable, it is Lipschitz continuous on the bounded set 5 (). The last inequality then implies that (5.8) holds for each i with a suitable constant l0 independent of . Therefore, by (3.9), we get for each i kr i (x) 0 r i(y)k (ai(x) 0 ai (y))ei + bi(x)rFi(x) 0 bi (y)rFi(y) jai(x) 0 ai (y)j + k(bi(x) 0 bi(y))rFi(x)k + kbi(y)(rFi(x) 0 rFi(y))k: Since rF is Lipschitz continuous and jbi(x)j 2, the last inequality and (5.8) imply (5.7), which in turn implies (5.6) with some constant L > 0. 2 De ne ky 0 Bk sk k and A = 1 r8k (x + s )d : (5.9) k = k k k k ksk k 0 Then, we have yk = Ak sk and k(Ak 0 Bk )sk k = k(Ak 0 Bk )pk k : k = (5.10) ksk k kp k k So, it follows that k(Bk 0 r8k (xk ))pk k k(Ak 0 Bk )pk k + k(r8k (xk ) 0 Ak )pk k (k + kr8k (xk ) 0 Ak k)kpk k: (5.11) In a similar way to Lemma 4.1 in [20], it is not dicult to prove the following useful lemma. p
p
p
p
p
p
p
p
p
p
Z
Lemma 5.2 Let Assumption B hold. If the index set K de ned by (5.4) is nite, then we have
1 lim k!1 k
k
X
i=1
25
i2 = 0:
(5.12)
Now, we are ready to prove global convergence for Algorithm 2. Theorem 5.1 Let Assumptions A and B hold and fxk g be generated by Algorithm 2. Then the index set K must be in nite and hence
lim k8(xk )k = 0:
k!1
Moreover, every accumulation of fxk g is a solution of MCP(F ).
Proof Assume to the contrary that K is nite. Then there is an integer k such that k = k for all k k. Denote 8 = 8k . Notice that the niteness of K implies that K2 de ned by (5.4) is also nite, and hence there is an index k~ such that when k k~, the steplength k is determined by Step 3. That is, we have for every k maxfk; k~g
k8 (xk + k pk )k 0 k8 (xk )k 02kk pk k2 + k :
By (5.12), there is a subsequence fk gk2K converging to zero. Let x be an arbitrary accumulation point of the corresponding sequence fxk gk2K . Without loss of generality, we assume limk2K;k !1 xk = x. By (5.1) and (5.11), we have k8(xk )k k(Bk 0 r8 (xk ))pk k + kr8 (xk )pk k (k + kr8k (xk ) 0 Ak k + kr8 (xk )k)kpk k and kpk k =
kr8 (xk )01(8(xk ) + (Bk 0 r8 (xk ))pk k kr8 (xk )01k k8(xk )k + k(Bk 0 r8 (xk ))pk k k8 (xk )01k k8(xk )k + (k + kr8 (xk ) 0 Ak k) kpk k :
(5.13)
So, there exists a constant C > 0 such that for all k 2 K suciently large k8(xk )k C kpk k
and
kpk k C k8(xk )k:
(5.14)
In a similar way to the proof of Theorem 4.2, we can deduce a contradiction. 2 Following the same arguments as in the proof of Theorem 4.4, it is not dicult to prove the following theorem. Theorem 5.2 Let Assumptions A and B hold. Suppose that there is an accumulation point x3 of fxk g at which 8 is BD-regular. Then the whole sequence fxk g converges to x3 .
26
Now we turn to proving superlinear convergence of Algorithm 2. For this purpose, we make further assumptions. Assumption C (i) The sequence fxk g generated by Algorithm 2 converges to a solution x3 of MCP(F ) at which 8 is BD-regular. (ii) The strict complementarity holds at x3, i.e., 1(x3 ) = 2 (x3 ) = ;; in other words, (x3i 0 li)2 + Fi(x3)2 > 0 and (x3i 0 ui)2 + Fi (x3 )2 > 0; Let and
=
i = 1; 2; : : : ; n:
(5.15)
1 minfjx3 0 l 0 F (x3)j; jx3 0 u 0 F (x3 )jj i = 1; 2; : : : ; ng: i i i i i i 2
U 3 = fx j jxi 0 li 0 Fi (x)j ; jxi 0 ui 0 Fi (x)j ; i = 1; 2; : : : ; ng:
It is clear that, under Assumption C, > 0 and U 3 6= ;. It is also obvious that 1(x) = 2 (x) = ; for every x 2 U 3 , and hence, 8 is continuously di erentiable on U 3 . Speci cally, for every x 2 U 3 , we have r8(x) = (r 1 (x); r 2 (x); : : : ; r n (x)) with (5.16) r i(x) = 2rFi(x); if i 2 (x); 2e i ; if i 2 (x): Moreover, there exists a constant M2 > 0 such that k8(x) 0 8(x3 )k M2 kx 0 x3 k; 8x 2 U 3 : (5.17) Let 5() be de ned by (5.5). Then we have U 3 5 ( 2 ) for any . Therefore, in a similar way to Lemma 5.1, we can prove the following result. (
Lemma 5.3 Let Assumption B hold. Then there is a positive constant L0 independent of such that kr8(x) 0 r8(y)k L0 kx 0 yk; 8x; y 2 U 3 and
kr8 (x) 0 r8 (y)k L0 kx 0 yk; 8 > 0; 8x; y 2 U 3 :
Lemma 5.4 Let a solution x3 of MCP(F ) satisfy (5.15). Then there are constants C1 > 0 and C2 > 0 such that for every > 0
k8(x) 0 8(x)k C1 2; and
kr8(x) 0 r8(x)k C2 2 ;
27
8x 2 U 3 8x 2 U 3:
(5.18) (5.19)
We deduce from (3.2) that for every (a; b; c) 2 R 3 2 2 j (a; b; c) 0 (a; b; c)j max 2jb0 cj ; 2ja0 bj : Substituting xi 0 li for a, Fi(x) for b, and xi 0 ui for c in the last inequality yields 2 2 j i (x) 0 i(x)j max 2jx 0 u 0 F (x)j ; 2jx 0 l 0 F (x)j : i i i i i i 3 This implies that for all x 2 U 2 : j i(x) 0 i(x)j 2 pn Letting C1 = 2 , we get (5.18). Now we prove (5.19). From (3.9) and (5.16), we have for every > 0 and any x 2 U 3 i 2 (x); (5.20) r i(x) 0 r i(x) = ai (x)(ei 0 rFi(x)); bi (x)(0ei + rFi (x)); i 2 (x): For each i 2 1(x), we have by (3.7) jbi (x)j (x x0i 0l l0i 0F F(xi())x2) + 2 + 1 + (x x0i 0u u0i 0F F(xi())x2) + 2 + 1 : (5.21) i i i i i i Moreover, we have xi 0 li 0 Fi (x) (5.22) (xi 0 li 0 Fi(x))2 + 2 + 1 2 2 = (Fi(x) 0 xi +(xli0) 0l 0(Fxi(0x))li20+F2i(x)) + i i i 2 = (xi 0 li 0 Fi(x))2 + 2 (Fi(x) 0 xi + li) + (xi 0 li 0 Fi(x))2 + 2 (5.23) 21 2 2 : Similarly, we have 1 2 : xi 0 ui 0 Fi (x) (5.24) + 1 2 2 (xi 0 ui 0 Fi(x))2 + 2 Combining (5.23) and (5.24) with (5.21), we get jbi(x)j 12 2: (5.25) By a similar argument, we can show that (5.25) holds for each i 2 2(x), every > 0 and any x 2 U 3. Moreover, we have for each i 2 (x), every > 0 and any x 2 U 3 jai(x)j 12 2 : Then by using (5.20), we can deduce (5.19) with some constant C2 > 0. 2 Proof
o
n
n
o
(
p
p
p
p
p
p
p
p
28
Lemma 5.5 Let Assumptions B and C hold, fxk g be generated by Algorithm 2, and k be de ned by (5.9). Then there exist a constant > 0 and an integer k0 > 0 such that k = 1 whenever k and k k0 .
Since xk ! x3 , there is an index k10 such that xk 2 U 3 for all k k10 . Let Ak be de ned by (5.9). Then we have for all k k10 1 1 kr8k (xk + sk ) 0 r8(xk + sk )kd + kr8(xk + sk ) 0 r8(x3 )kd kAk 0 r8(x3 )k 0 0 Proof
Z
Z
Z 1 2 C2 k + L0 k (xk+1 0 x3 ) + (1 0 )(xk 0 x3)kd 0 C2 2 k8(xk ) 0 8(x3 )k2 + L0 (kxk+1 0 x3 k + kxk 0 x3k) C2 2 M22kxk 0 x3 k2 + L0(kxk+1 0 x3k + kxk 0 x3 k) (L0 + C2 2M22 kxk 0 x3k)kxk 0 x3 k + L0 kxk+1 0 x3k
= =4 k ; (5.26) where the second inequality follows from (5.19) and Lemma 5.3, and the third and the fourth inequalities follow from (5.3) and (5.17), respectively. It is obvious that k ! 0 as k ! 1. By using (5.1), we have r8(x3)pk = (r8(x3) 0 Ak )pk + (Ak 0 Bk )pk 0 F (xk ): This together with (5.10) yields kpk k kr8(x3 )k kr8(x3 ) 0 Ak kkpk k + k(Ak 0 Bk )pk k + kF (xk )k kr8(x3 )k(k + k )kpk k + kr8(x3)k kF (xk )k: Since k ! 0 as k ! 1, the last inequality shows that there are constants 0 > 0 and C3 > 0 such that when k 0 and k is suciently large kpk k C3k8(xk )k = k8(xk ) 0 8(x3 )k C3 M2 kxk 0 x3 k: (5.27) From (5.1) again, we have r8(x3 )(xk + pk 0 x3 ) = r8(x3 )(xk 0 x3 ) 0 8(xk ) 0 (Bk 0 r8(x3))pk : Let M3 = kr8(x3 )01k. It then follows that kxk + pk 0 x3 k kr8(x3 )01k 8(xk ) 0 r8(x3)(xk 0 x3 ) + (Bk 0 r8(x3))pk M3 k8(xk ) 0 8(x3 ) 0 r8(x3 )(xk 0 x3 )k + k(Bk 0 Ak )pk k + k(Ak 0 r8(x3 ))pk k k0 kxk 0 x3 k + M3 (k + k )kpk k (k0 + C3 M2 M3(k + k ))kxk 0 x3 k =4 (~k + C3 M2 M3k )kxk 0 x3 k; (5.28)
29
where k0 = M3k8(xk ) 0 8(x3 ) 0 r8(x3 )(xk 0 x3)k=kxk 0 x3k, ~k = k0 + C3 M2 M3 k ), the third inequality follows from (5.10) and (5.26), and the last inequality follows from (5.27). So, we deduce that when k 0 and k is suciently large k8k (xk + pk )k k8k (xk + pk ) 0 8(xk + pk )k + k8(xk + pk ) 0 8(x3 )k C1 2k + M2 kxk + pk 0 x3 k C1 2k8(xk ) 0 8(x3)k2 + M2 (~k + C3 M2M3k )kxk 0 x3 k (C1 2 M22 kxk 0 x3 k + M2(~k + C3 M2 M3 k ))kxk 0 x3 k =4 (^k + C4 k )kxk 0 x3 k; (5.29) where ^k = C1 2M22 kxk 0 x3 k + M2 ~k and C4 = C3 M22 M3, the second inequality follows from (5.18) and the third inequality follows from (5.3) and (5.28). On the other hand, by the given assumptions, it is not dicult to see that (4.30) still holds. Therefore, we get that when k 0 and k is suciently large k8k (xk + pk )k 0 2 k8k (xk )k + 1 kpk k2 (^k + C4 k )kxk 0 x3k 0 2 kxk 0 x3 k + 1 C32 M22 kxk 0 x3 k2 = 02 kxk 0 x3k + C4 k kxk 0 x3 k + o(kxk 0 x3 k); where the inequality follows from (5.29), (4.30) and (5.27). It is then not dicult to see that there exist a constant 0 such that when k and k is suciently large k8k (xk + pk )k 0 2 k8k (xk )k + 1kpk k2 0: This means k = 1. The proof is complete. 2 The following lemma shows that under Assumptions B and C, (5.12) holds even if K is in nite. Lemma 5.6 Let Assumptions B and C hold. Then
1 lim k!1 k
k
X
i=1
i2 = 0:
(5.30)
It is clear that when k is suciently large, xk + sk 2 U 3 for any 2 [0; 1]. It follows from Lemma 5.3 that there is a constant L0 > 0 such that for any 2 [0; 1] kr8k01 (xk + sk ) 0 r8k01 (xk01 + sk01 )k L0k(xk + sk ) 0 (xk01 + sk01 )k L0 ksk k + (1 0 )ksk01 k (5.31) L0 ksk k + ksk01k : Proof
30
Let Ak be de ned by (5.9). Then we have kAk 0 Ak01 k =
Z
1
Z
k(r8k (xk + sk ) 0 r8k01(xk + sk ))d k
0 1
0Z
r8 (xk + sk )d 0 k
Z
0
1
r8k01(xk01 + sk01)d
1
k(r8k01(xk + sk ) 0 r8k01(xk01 + sk01))d k pn(k01 0 k ) + L0 ksk k + ksk01k
+
0
=4 k ; where the last inequality follows from (3.4) and (5.31). It is easy to deduce that 1 1 X X 2 2 2 ksk k2 + ksk01 k2 : k 2n (k01 0 k ) + 4L0 k=1 k=1 k=1 1
X
Since 1k=1 ksk k2 < 1 (see Remark (iv) after Algorithm 2) and fk g is nonincreasing, it is clear that 1k=1 k2 < 1. This implies 1k=1 kAk 0 Ak01 k2 < 1. In a way similar to Lemma 4.1 in [20], we can then prove (5.30). 2 The following theorem shows superlinear convergence of Algorithm 2. P
P
P
Theorem 5.3 Let Assumptions B and C hold. Then fxk g converges superlinearly.
We rst show that there is a constant c > 0 such that for all k large enough k8(xk+1)k (!k + ck )k8(xk )k; (5.32) where f!k g is a positive sequence converging to zero. De ne the index set K~ = fk j k and k k0g, where and k0 are constants given in Lemma 5.5. Then we get for each k 2 K~ k8(xk+1)k k8(xk+1 ) 0 8k (xk+1)k + k8k (xk+1)k C12k + k8k (xk+1 )k C1 2 k8(xk ) 0 8(x3 )k2 + (^k + C4 k )kxk 0 x3 k (C1 2 M22 kxk 0 x3 k + ^k + C4 k )kxk 0 x3 k; (5.33) where the second inequality follows from (5.18), the third inequality follows from (5.3) and (5.29), and the fourth inequality follows from (5.17). By the nonsingularity of r8(x3 ), there exists a constant m > 0 such that k8(xk ) 0 8(x3)k m kxk 0 x3k for all k suciently large. Letting !k = m 01(C1 2 M22 kxk 0 x3 k + ^k ) and C5 = m 01C4 , it then follows from (5.33) that k8(xk+1)k (!k + C5 k )k8(xk )k: (5.34)
Proof
31
Clearly, !k ! 0 as k ! 1. By Steps 2 and 3, it is clear that k8k (xk+1 )k k8k (xk )k + k k8k (xk )k + k8(xk )k; where the last inequality follows from (5.3). Also we have k > for any k 62 K~ such that k > k0 . Therefore, we get for all k 62 K~ suciently large k8(xk+1)k k8(xk+1 ) 0 8k (xk+1 )k + k8k (xk+1 )k C1 2k + k8k (xk )k + k8(xk )k C1 2k + k8k (xk ) 0 8(xk )k + 2k8(xk )k 2C1 2k + 2k8(xk )k 2C1 2 k8(xk )k2 + xm2k8(xk )k = 2(C1 2k8(xk )k + 1)k8(xk )k (5.35) 2(C1 2k8(xk )k + k )k8(xk )k; where the second and the fourth inequalities follow from (5.18), and the fth inequality follows from (5.3). Letting !k = maxf!k ; 2C1 2 k8(xk )kg and c = maxfC5; 201g, we obtain (5.32) from (5.34) and (5.35). Next, we show the inequality 1 kxk 0 x3 k < 1: (5.36) X
k=1
Without loss of generality, we assume that (5.32) holds for all k. Multiplying those inequalities from k = 1 to t yields k8(xt+1 )k k8(x1 )k
t
Y
(!k + ck )
k=1 t
n Y
= k8(x1 )k
2
(2!k2 + 2c2 k2)
k=1 t X
k8(x1 )k 1t n
o1
k=1 t
n Y
k8(x1 )k
(!k + ck )2
= k8(x1 )k 2t n
k=1 t X k=1
o1
2
(2!k2 + 2c2k2 ) !k2 +
2c 2
t
X
t k=1
ot
2
ot
k2 2 :
Note that both terms in the last braces tend to zero as1 t ! 1, since !k converges to zero and (5.30) holds. Therefore we have limk!1 k8(xk+1 )k = 0 and hence 1k=1 k8(xk+1)k < k
32
P
1. However, k8(xk+1)k = k8(xk+1) 0 8(x3 )k m kxk+1 0 x3 k for all k suciently large.
Consequently, we get (5.36). Now, we show superlinear convergence of Algorithm 2. By virtue of (5.36), in a similar way to the proof of Lemma 5.4 in [20], we can verify that k ! 0 as k ! 1. Therefore, by Lemma 5.5, we have k 1 for all k suciently large. Consequently, (5.28) shows that fxk g converges to x3 superlinearly. 2 6.
Conclusion
In this paper, we proposed a smoothing Newton method and a smoothing quasi-Newton method for solving MCP. Under appropriate conditions, the proposed methods converge globally and superlinearly/quadratically. An advantage of these methods is that the dimension of the subproblems is the same as that of MCP. Moreover, the level set is bounded if the MCP involves a uniform P function. We showed an interesting property for the smoothing function. That is, every accumulation point of fr8k (xk )g is an element of @C 8(x3 ). In particular, if 8 is di erentiable at x3, which is particularly the case when the strict complementarity holds at x3 , then r8k (xk ) ! r8(x3). A drawback of the proposed quasi-Newton method is that it is not a descent method even if F is a strongly monotone function, though it converges globally. It is interesting to construct a descent quasi-Newton method for MCP with global and superlinear convergence. We leave it as a possible further research topic. References
[1] B. Chen, X. Chen and C. Kanzow, \A penalized Fischer-Burmeister NCP-function: Theoretical investigation and numerical results", Technical Report AMR 97/28, School of Mathematics, University of New South Wales, Sydney, September 1997. [2] B. Chen and P.T. Harker, \A non-interior-point continuation method for linear complementarity problems", SIAM J. Matrix Anal. Appl., vol. 14, pp. 1168-1190, 1993. [3] B. Chen and P.T. Harker, \Smooth approximations to nonlinear complementarity problems", SIAM J. Optim., vol. 7, pp. 403-420, 1997. [4] C. Chen and O.L. Mangasarian, \A class of smoothing functions for nonlinear and mixed complementarity problems", Comput. Optim. Appl., vol. 5, pp. 97-138, 1996. [5] X. Chen, Z. Nashed and L. Qi, \Convergence of Newton's method for singular smooth and nonsmooth equations using adaptive outer inverses", SIAM J. Optim., vol. 7, pp. 445-462, 1997. 33
[6] X. Chen, L. Qi and D. Sun, \Global and superlinear convergence of the smoothing Newton method and its application to general box constrained variational inequalities", Math. Comp., vol. 67, pp. 519-540, 1998. [7] F.H. Clarke, Optimization and Nonsmooth Analysis, John Wiley & Sons: New York, 1983. [8] T. De Luca, F. Facchinei and C. Kanzow, \A semismooth equation approach to the solution of nonlinear complementarity problems", Math. Programming, vol. 75, pp. 407-439, 1996. [9] T. De Luca, F. Facchinei and C. Kanzow, \A theoretical and numerical comparison of some semismooth algorithms for complementarity problems", Mathematical Programming Technical Report 97-15, Computer Sciences Department, University of Wisconsin - Madison, Madison, WI, December 1997. [10] J.E. Dennis, Jr. and J.J. More, \A characterization of superlinear convergence and its applications to quasi-Newton methods", Math. Comp., vol. 28, pp. 549-560, 1974. [11] F. Facchinei, A. Fischer and C. Kanzow, \A semismooth Newton method for variational inequalities: The case of box constraints", in Complementarity and Variational Problems: State of the Art, M.C. Ferris and J-S. Pang (Eds.), SIAM, Philadelphia, pp. 76-90, 1997. [12] A. Fischer, \Solution of monotone complementarity problems with locally Lipschitzian functions", Math. Programming, vol. 76, pp. 513-532, 1997. [13] M. Fukushima and L. Qi, (Eds.), Reformulation: Nonsmooth, Piecewise Smooth, Semismooth and Smoothing Methods, Kluwer Academic Publishers, Boston, 1998. [14] S.A. Gabriel and J.J. More, \Smoothing of mixed complementarity problems", in Complementarity and Variational Problems: State of the Art, M.C. Ferris and J.S. Pang (Eds.), SIAM, Philadelphia, pp. 105-116, 1997. [15] S.A. Gabriel and J.S. Pang, \An inexact NE/SQP method for solving the nonlinear complementarity problem", Comput. Optim. Appl., vol. 1, pp. 67-91, 1992. [16] H. Jiang, L. Qi, X. Chen and D. Sun, \Semismoothness and superlinear convergence in nonsmooth optimization and nonsmooth equations", in Nonlinear Optimization and Applications, G. Di Pillo and F. Giannessi (Eds.), Plenum Press: New York, pp. 197-212, 1996. [17] C. Kanzow, \Some noninterior continuation methods for linear complementarity problems", SIAM J. Matrix Anal. Appl., vol. 17, pp. 851-868, 1996. [18] C. Kanzow and H. Jiang, \A continuation method for (strongly) monotone variational inequalities", Math. Programming, vol. 81, pp. 103-125, 1998. 34
[19] C. Kanzow, N. Yamashita and M. Fukushima, \New NCP-functions and their properties", J. Optim. Theory Appl., vol. 94, pp. 115-135, 1997. [20] D.H. Li and M. Fukushima, \Broyden-like methods for variational inequality and nonlinear complementarity problems with global and superlinear convergence", Technical Report 98016, Department of Applied Mathematics and Physics, Kyoto University, Kyoto, September 1998. [21] R. Miin, \Semismooth and semiconvex functions in constrained optimization", SIAM J. Control Optim., vol. 15, pp. 957-972, 1977. [22] J.S. Pang, \A B-di erentiable equation based, globally, and locally quadratically convergent algorithm for nonlinear complementarity and variational inequality problems", Math. Programming, vol. 51, pp. 101-131, 1991. [23] J.S. Pang and L. Qi, \Nonsmooth equations: Motivation and algorithms", SIAM J. Optim., vol. 3, pp. 443-465, 1993. [24] L. Qi, \Convergence analysis of some algorithms for solving nonsmooth equations", Math. Oper. Res., vol. 18, pp. 227-244, 1993. [25] L. Qi, \Regular pseudo-smooth NCP and BVIP functions and globally and quadratically convergent generalized Newton methods for complementarity and variational inequality problems", Technical Report AMR 97/14, School of Mathematics, University of New South Wales, Sydney, July, 1997. [26] L. Qi and X. Chen, \A globally convergent successive approximation method for severely nonsmooth equations", SIAM J. Control Optim., vol. 33, pp. 402-418, 1995. [27] L. Qi and J. Sun, \A nonsmooth version of Newton's method", Math. Programming, vol. 58, pp. 353-367, 1993. [28] L. Qi and D. Sun, \Smoothing functions and a smoothing Newton method for complementarity problems and variational inequality problems", School of Mathematics, University of New South Wales, Sydney, October 1998. [29] L. Qi, D. Sun and G. Zhou, \A new look at smoothing Newton methods for nonlinear complementarity problems and box constrained variational inequalities", AMR 97/13, Applied Mathematics Report, University of New South Wales, Sydney, June 1997. [30] S. Smale, \Algorithms for solving the equations", in Proceedings of the International Congress of Mathematicians, Berkeley, California, pp. 172-195, 1986. 35
[31] D. Sun and R.S. Womersley, \A new unconstrained di erentiable merit function for box constrained variational inequality problems and a damped Gauss-Newton method", to appear in SIAM J. Optim.. [32] K. Yamada, N. Yamashita and M. Fukushima, \A new derivative-free descent method for the nonlinear complementarity problem", to appear in Nonlinear Optimization and Applications 2, G. Di Pillo and F. Giannessi (Eds.), Kluwer Academic Publishers: Boston. [33] N. Yamashita and M. Fukushima, \Modi ed Newton methods for solving a semismooth reformulation of monotone complementarity problems", Math. Programming, vol. 76, pp. 469-491, 1997. [34] N. Yamashita and M. Fukushima, \Equivalent unconstrained minimization and global error bounds for variational inequality problems", SIAM J. Control Optim., vol. 35, pp. 273-284, 1997.
36
Donghui Li 1 Department of Applied Mathematics Hunan University Changsha, China 410082 e-mail: [email protected] Masao Fukushima Department of Applied Mathematics and Physics Graduate School of Informatics Kyoto University Kyoto 606-8501, Japan e-mail: [email protected] February 3, 1999 Abstract The mixed complementarity problem can be reformulated as a nonsmooth equation by using the median operator. In this paper, we rst study some useful properties of this reformulation and then derive the Chen-Harker-Kanzow-Smale smoothing function for the mixed complementarity problem. On the basis of this smoothing function, we present a smoothing Newton method for solving the mixed complementarity problem. The smoothing Newton method converges globally if the problem involves a di erentiable P0 function. Under suitable conditions, the method exhibits a quadratic convergence property. We also present a smoothing Broyden-like method based on the same smoothing function. Under appropriate conditions, the method converges globally and superlinearly.
Key words:
Newton method
Mixed complementarity problem, smoothing function, Newton's method, quasi-
1 Present address (available until October, 1999): Department of Applied Mathematics and Physics, Graduate School of Informatics, Kyoto University, Kyoto 606-8501, Japan, e-mail: [email protected]
1.
Introduction
n
Let F = (F1 ; F2 ; : : : ; Fn ) be a mapping from Rn into itself and X = [li; ui ] be a box in Rn , i=1 where 01 li < ui 1, i = 1; 2; : : : ; n. The variational inequality problem of nding an x 2 X such that F (x)T (y 0 x) 0; 8y 2 X (1.1) is particularly called the mixed complementarity problem [4] and is simply denoted MCP(F ). The mixed complementarity problem is also called the box constrained variational inequality problem (BVIP) [1, 25, 29, 31]. MCP(F ) contains, as a special case, the nonlinear complementarity problem, NCP(F ), of nding an x 2 Rn such that x 0; F (x) 0; xT F (x) = 0; (1.2) which corresponds to the case where li = 0 and ui = +1, i = 1; 2; : : : ; n. Another extreme case of MCP(F ) is the problem with li = 01, i = 1; 2; : : : ; n and ui = +1, i = 1; 2; : : : ; n. In this case, MCP(F ) reduces to the nonlinear equation F (x) = 0: (1.3) Q
The study on iterative methods for solving NCP(F ) and MCP(F ) has been rapidly developed in the last decade. One of the most popular approaches is to reformulate NCP(F ) or MCP(F ) as an equivalent nonsmooth equation so that generalized Newton-type methods can be applied in a way similar to those for smooth equations. For example, consider the KKT system for the MCP(F ): F (x) 0 + = 0; (1.4) 0; x 0 l 0; T (x 0 l) = 0; 0; x 0 u 0; T (x 0 u) = 0: It is easy to see that the KKT system (1.4) can be rewritten as the equation F (x) 0 + (1.5) min(; x 0 l) = 0 min(; 0(x 0 u)) or equivalently min(F (x) + ; x 0 l) = 0: (1.6) min(; 0(x 0 u)) 8 > > < > > :
0
1
B B @
C C A
!
Another nonsmooth equation reformulation of MCP(F ) is the equation midfx 0 l; F (x); x 0 ug = 0: 1
(1.7)
Here and throughout the paper, the operator midfa; b; cg stands for the median of three scalars a; b; c 2 R [f61g. For vectors x; y; z 2 (R [f61g)n, the mid operation is done elementwise. We will focus on the reformulation (1.7) because it does not increase the dimension of the problem. For NCP(F ), (1.7) reduces to the following well known nonsmooth equation: minfx; F (x)g = 0: (1.8) There are a variety of nonsmooth Newton methods based on (1.5) { (1.8) and other reformulations for NCP and MCP. The interested readers may nd details in recent papers such as [8, 9, 11, 15, 22, 23, 27] and the references therein. In this paper, we pay our attention to another approach to NCP and MCP, called smoothing methods. A smoothing method tries to nd a solution of the nonsmooth equation (1.7) by solving a sequence of smooth equations that tend to the original equation (1.7). Smoothing methods for NCP and MCP have also received much attention in recent years [3, 4, 6, 14, 18, 28, 29]. In particular, Chen et al. [6] studied the Jacobian smoothing Newton method and established superlinear convergence of a smoothing Newton method. However, the merit function used in [6] may have an unbounded level set even if F is a uniform P function. Recently, Chen, Chen and Kanzow [1] proposed a new NCP-function by penalizing the Fisher-Burmeister function. The natural merit function of this new NCP-function has all nice features of KanzowYamashita-Fukushima merit function [19] for global convergence. In particular, it provides a bounded level set. By means of this merit function, a smoothing Newton method for solving NCP with global and superlinear convergence was proposed in [1]. Merit functions that possess similar properties have also been proposed recently in [32, 33]. Another interesting smoothing Newton method was proposed by Qi, Sun and Zhou [29] for solving MCP(F ) based on a 2ndimensional semismooth equation reformulation. An advantage of this method is that it only requires the function F to be de ned on the box X . Moreover, though the reformulation is 2n-dimensional, subproblems in the smoothing Newton method reduce to n-dimensional linear equations. However, it is not clear whether it is possible to develop a related quasi-Newton method in which subproblems still remain to be n-dimensional linear equations. In this paper, we present a new smoothing Newton method based on Chen-Harker-Kanzow-Smale smoothing function. The proposed smoothing Newton method retains global and superlinear convergence under the same assumptions as used by Chen et al. [6] and by Qi et al. [29], and the subproblems are n-dimensional linear equations. Moreover, we develop a smoothing quasi-Newton method where subproblems are also n-dimensional linear equations. We show that if F is a di erentiable P0 function with Lipschitz continuous Jacobian, then the smoothing quasi-Newton method converges globally. If in addition, the strict complementarity holds at a limit point, then the convergence rate is superlinear. The paper is organized as follows. In the next section, we introduce the notion of MCP2
function and show some useful properties of it. In Section 3, we present a smoothing function for an MCP function and investigate its properties. In Sections 4 and 5, we propose a smoothing Newton method and a smoothing quasi-Newton method, respectively, for solving MCP, and establish their global and superlinear convergence under appropriate conditions. Throughout the paper, we adopt the following convention: For any real number x, 1+x01 = x, x + 1 0 1 = x, 0 1 1 = 1 1 0 = 0 1 (01) = (01) 1 0 = 0, and 1=1 = 1. 2.
An MCP-Function and Its Properties
In this section, we rst introduce a natural extension of NCP-function. Recall that a function : R2 ! R is called an NCP-function if (a; b) = 0 holds if and only if a 0, b 0 and ab = 0. In other words, a function : R2 ! R is called an NCP-function if (a; b) = 0 is equivalent to minfa; bg = 0. Comparing (1.7) with (1.8), we see that the role of the mid function in MCP is very similar to that of the min function in NCP. Since x 0 l x 0 u for all x, we de ne an MCP-function as follows. De nition 1 Denote R 3 = (R [ f01g) 2 R 2 (R [ f+1g). A function : R 3 ! R is called an MCP-function if the following relation holds:
(a; b; c) = 0; a > c
mid fa; b; cg = 0; a > c: (2.1) Note that we allow a and c to be +1 and 01, respectively. In particular, when c = 01 the relation (2.1) is rewritten as (a; b; 01) = 0 () minfa; bg = 0: In other words, (a; b; 01) is an MCP-function if and only if the function ^(a; b) =4 (a; b; 01) is an NCP function. In this sense, the class of MCP-functions contains NCP-functions. We notice that MCP-function is di erent from the so-called BVIP-function in [25]. For any < , a function ( ; ) : R2 ! R is said to be a BVIP-function if ( ; ) (a; b) = 0 if and only if either a = and b 0, or a = and b 0, or a and b = 0. However, both MCP-function and BVIP-function are closely related to the MCP(F ). Indeed, it is easy to prove the following proposition. 4 Proposition 2.1 Let be an MCP-function and i = (l ;u ) be BVIP-functions for all i = 1; 2; : : : ; n. Then, the following statements are equivalent: ()
i
(i) x is a solution of MCP(F ); (ii) (xi 0 li ; Fi (x); xi 0 ui ) = 0 for each i = 1; 2; : : : ; n; (iii) i (xi ; Fi (x)) = 0 for each i = 1; 2; : : : ; n.
3
i
When a > c, the relation
midfa; b; cg = 0
is equivalent to a+b+c
= maxfa; b; cg + minfa; b; cg = maxfa; bg + minfb; cg = (a + b) +2 ja 0 bj + (b + c) 02 jb 0 cj 2 2 = (a + b) +2 (a 0 b) + (b + c) 02 (b 0 c) ; p
p
or equivalently,
a+c=
q
q
(a 0 b)2 0 (b 0 c)2 : Hence if we de ne the function : R 3 ! R by (a; b; c) = a + c 0 (a 0 b)2 + (b 0 c)2 ; (2.2) then is an MCP-function. Indeed it is not dicult to see that (a; b; c) = 2 midfa; b; cg; whenever a > c. The MCP-function de ned by (2.2) is not new but it will play an important role in this paper. We now reformulate the MCP(F ) by means of . In view of (1.7), let us de ne the function 8(x) = ( 1(x); 2 (x); : : : ; n (x))T , where for each i = 1; 2; : : : ; n i (x) = (xi 0 li ; Fi (x); xi 0 ui ) (2.3) = 2xi 0 (li + ui) 0 (xi 0 li 0 Fi(x))2 + (xi 0 ui 0 Fi(x))2 : Then the MCP(F ) is reformulated as the following nonsmooth equation: 8(x) = 0: (2.4) For the NCP(F ), the function i becomes (2.5) i (x) = xi + Fi (x) 0 (xi 0 Fi (x))2 : when li = 01 and ui = +1, the function i simply reduces to 2Fi. We now show some useful properties of . We rst consider the di erentiability of . It is obvious that and hence 8 are not di erentiable everywhere. Nevertheless, we can show that it has a property called semismoothness. The concept of semismoothness was introduced for scalar q
q
q
q
q
4
functions by Miin [21] and was extended to vector functions by Qi [24]. It has been widely used in the context of nonlinear complementarity problems and variational inequality problems [16, 23, 24]. A locally Lipschitz function 9 : Rn ! Rn is said to be semismooth at a point x if the limit lim 0 fV h0g V0 2 @ 9(x + h ) h !h
exists for any h 2 Rn, where @ 9(x) stands for the Clarke generalized Jacobian of 9 at x [7]. If 9 is semismooth everywhere, we say that 9 is a semismooth function. It is clear that a semismooth function is directionally di erentiable [27]. Let 90 (x; p) denote the directional derivative of 9 at x along direction p. Then the following lemma holds [27]. Lemma 2.1 Let 9 be semismooth at x. Then it holds that for any V 2 @ 9(x + h) and any h ! 0, 9(x + h) 0 9(x) 0 V h = o(khk) (2.6) and
V h 0 90 (x; p) = o(khk):
(2.7)
A locally Lipschitz function 9 is said to be strongly semismooth at x if V h 0 90(x; p) = O(khk2): (2.8) It is not dicult to show that if 9 is strongly semismooth at x, then we have for any V 2 @ 9(x) and any h ! 0, 9(x + h) 0 9(x) 0 V h = O(khk2 ): (2.9) It is obvious that di erentiable functions and strongly semismooth functions are semismooth functions. If 9 has Lipschitz continuous Jacobian, then it is a strongly semismooth function. The following proposition shows the semismoothness of the function 8 de ned by (2.3). Proposition 2.2 Let : R 3 ! R be de ned by (2.2) and 8(x) = ( 1 (x); 2 (x); : : : ; n (x))T be de ned by (2.3). Then (i) is strongly semismooth everywhere; (ii) if F is di erentiable, then 8 is semismooth; (iii) if F has Lipschitz continuous Jacobian, then 8 is strongly semismooth.
It is not dicult to show that the function g : x 7! jxj is strongly semismooth. From the de nition of , we have (a; b; c) = a + c 0 g(a 0 b) + g(b 0 c). By Theorem 19 in [12], it follows that is strongly semismooth. This concludes (i). Since 8 is (strongly) semismooth if and only if its elements i, i = 1; 2; : : : ; n are (strongly) semismooth, (ii) and (iii) follow again from Theorem 19 in [12]. 2 Proof
5
Denote (x) 1(x)
1(x) (x)
Then we have
= fi j xi 0 ui < Fi(x) < xi 0 lig; = fi j Fi (x) = xi 0 lig; 2 (x) = fi j Fi(x) = xi 0 uig; = fi j Fi (x) > xi 0 lig; 2(x) = fi j Fi(x) < xi 0 uig; = 1(x) [ 2 (x); (x) = 1 (x) [ 2 (x):
2Fi(x); if i 2 (x); i (x) = 2(xi 0 li ); if i 2 1(x) [ 1(x); 2(xi 0 ui); if i 2 2(x) [ 2(x): By using the chain rule for generalized derivatives of Lipschitz functions (see Theorem 2.3.9 (iii) in [7]), we have the following expression of @ i(x). Proposition 2.3 Let F : Rn ! Rn be di erentiable. Then we have f2rFi(x)g; if i 2 (x); (2.10) @ i (x) = f(1 + )ei + (1 0 )rFi (x) j 2 [01; 1]g; if i 2 1(x) [ 2 (x); f2eig; if i 2 1 (x) [ 2 (x); 8 > > < > > :
8 > > < > > :
where ei denotes the ith unit vector in Rn . It is clear that @8(x) @C 8(x) =4 @ 1(x) 2 1 11 2 @ n (x) @ 8(x) = @C 8(x) does not necessarily hold.
for any x 2 Rn , though the relation
The following proposition shows that 8 provides a global error bound for the MCP(F ) when F is a Lipschitz continuous uniform P function.
Proposition 2.4 Let F be Lipschitz continuous. If it is a uniform P function, then there are positive constants m M such that
mk8(x) 0 8(y)k kx 0 yk In particular, we have
M k8(x) 0 8(y)k; 8x; y 2 Rn:
kx 0 xk M k8(x)k; 8x 2 Rn ;
where x stands for the unique solution of the MCP(F).
(2.11) (2.12)
First we show that the left-hand inequality in (2.11), i.e., the Lipschitz continuity of 8. It is not dicult to show that for any x 2 Rn, 8(x) = 2fx 0 ProjX (x 0 F (x))g, where ProjX (x) denotes the projection of x onto the box X . The left-hand inequality of (2.11) then follows from the Lipschitz continuity of a projection operator.
Proof
6
Now we prove the right-hand inequality of (2.11). We rst show that there exists a constant c > 0 such that for every i = 1; 2; : : : ; n, and all x; y 2 Rn c(xi 0 yi )(Fi (x) 0 Fi (y)) k8(x) 0 8(y)kkx 0 yk: (2.13) First consider the case i 2 (x) = (x) \ (y) [ (x) \ (y) [ (x) \ (y) . When i 2 (x) \ (y) [ (y) , we have i(x) = 2Fi (x) and i(y) = 2Fi(y). It then follows that (Fi(x) 0 Fi(y))(xi 0 yi) = 12 ( i (x) 0 i(y))(xi 0 yi) 21 k8(x) 0 8(y)k kx 0 yk: When i 2 (x) \ 1 (y), we have i(x) = 2Fi(x) and i(y) = 2(yi 0 li). If xi yi, then we get (Fi(x) 0 Fi(y))(xi 0 yi ) Fi(x) 0 (yi 0 li) (xi 0 yi) = 21 ( i(x) 0 i(y))(xi 0 yi) 21 k8(x) 0 8(y)k kx 0 yk: If xi < yi, then Fi(x) < xi 0 li < yi 0 li < Fi(y). It follows that (Fi(x) 0 Fi(y))(xi 0 yi) = (Fi(x) 0 Fi(y)) (xi 0 li) 0 12 i(y) 21 (Fi(x) 0 Fi(y))( i(x) 0 i(y)) = 21 kF (x) 0 F (y)k k8(x) 0 8(y)k 21 Lk8(x) 0 8(y)kkx 0 yk; where L stands for the Lipschitz constant of F . Similarly, we can show that (2.13) holds for every i 2 (x) \ 2 (y). By similar arguments, it is not dicult to show that (2.13) holds for every i 2 (x) = (x) \ (y ) [ (x) \ (y ) [ (x) \ (y ) and i 2 (x) = (x) \ (y ) [ (x) \ (y ) [ (x) \ (y ) . Since F is a uniform P function, there is a constant c0 > 0 such that max (F (x) 0 Fi(y))(xi 0 yi) c0kx 0 yk2 ; 8x; y 2 Rn : 1in i This together with (2.13) implies k8(x) 0 8(y)kkx 0 yk cc0kx 0 yk2 ; 8x; y 2 Rn : Letting M = cc0, we get the right-hand inequality of (2.11). 2 Proposition 2.4, particularly, (2.12) shows that if F is a Lipschitz continuous uniform P function, then the level set fx j k8(x)k C g is bounded for every C > 0.
7
3.
Chen-Harker-Kanzow-Smale Smoothing Function for MCP
In this section, we consider a smooth function that approximates 8 and show its useful properties. For a scalar , let : R 3 ! R be de ned by (3.1) (a; b; c) = a + c 0 (a 0 b)2 + 2 + (b 0 c)2 + 2 : It is obvious that for any 6= 0, is smooth on R3, and 0(a; b; c) = (a; b; c) holds for any (a; b; c) 2 R3 . It is interesting to notice that though is obtained by smoothing the nonsmooth terms (a 0 b)2 and (b 0 c)2 in , the function 21 is just the Chen-Harker-Kanzow-Smale smoothing function [2, 17, 30], which is derived by letting 2 (s) = 2 (s + 4)3=2 in the Gabriel-More smoothing function [29] q
q
p
p
4 (; c; d; w) =
1 1
Z
midfc; d; w 0 sg(s)ds
with w = b, d = a and = 12 . Moreover, in the case where c = 01, we have q
(a 0 b)2 + 2: Notice that this function is essentially Chen-Harker-Kanzow-Smale smoothing function for NCP. In the following, we show that possesses similar properties to those of Chen-Harker-KanzowSmale smoothing function for NCP. First the following result shows that provides a uniform approximation to . Lemma 3.1 For any 0 > 0, we have j 0 (a; b; c) 0 (a; b; c)j 0 0 ; 8(a; b; c) 2 R 3 : (a; b; 01) = a + b 0
In particular, we have for every > 0
j (a; b; c) 0 (a; b; c)j : 8(a; b; c) 2 R 3:
From the de nition of and , we get j 0 (a; b; c) 0 (a; b; c)j = (b 0 c)2 + (0 )2 0 (b 0 c)2 + 2 0 (a 0 b)2 + (0 )2 + (a 0 b)2 + 2 2 02 2 2 2 02 2 2 = [((bb00cc))2++(()0 )2] 0+ [(b(b00c)c)2++ ]2 0 [((aa00bb))2++(()0 )2] 0+ [(a(a00b)b)2++ ]2
Proof
q
p
q
q
p
p
8
q
p
(0 )2 0 2 (0 )2 0 2 0 (b 0 c)2 + (0 )2 + (b 0 c)2 + 2 (a 0 b)2 + (0 )2 + (a 0 b)2 + 2 02 2 02 2 max (b 0 c)2 + ((0 ))2 +0 (b 0 c)2 + 2 ; (a 0 b)2 + ((0 ))2 0+ (a 0 b)2 + 2 (3.2) 02 2 ()0 +0 = 0 0 :
=
p
p
p
p
o
n
p
p
p
p
2
Lemma 3.1 particularly shows that ! uniformly as ! 0. It is then natural to construct a smoothing function for 8 based on . For each i = 1; 2; : : : ; n, de ne i : Rn ! R by i(x) = 2xi 0 (li + ui ) 0 (xi 0 li 0 Fi (x))2 + 2 + (xi 0 ui 0 Fi (x))2 + 2 : (3.3) Let 8 (x) = ( 1 (x); 2 (x); : : : ; n(x))T . It is obvious that 80(x) 8(x). Moreover, it follows from Lemma 3.1 that for any 0 > 0 k80 (x) 0 8(x)k pn(0 0 ); 8x 2 Rn ; (3.4) and (3.5) k8(x) 0 8(x)k pn; 8x 2 Rn: Note that, for every 6= 0, is di erentiable everywhere whenever F is di erentiable. Speci cally, by direct deduction we get for any 6= 0 and i = 1; 2; : : : ; n xi 0 li 0 Fi (x) r i(x) = 2 0 + xi 0 ui 0 Fi(x2) 2 ei 2 2 (xi 0 li 0 Fi(x)) + (xi 0 ui 0 Fi(x)) + (3.6) xi 0 li 0 Fi (x) xi 0 ui 0 Fi (x) 0 + r Fi (x): (xi 0 li 0 Fi(x))2 + 2 (xi 0 ui 0 Fi(x))2 + 2 q
q
q
q
q
q
Denote for each i = 1; 2; : : : ; n
xi 0 li 0 Fi (x) 0 q xi 0 ui 0 Fi(x2) 2 (xi 0 li 0 Fi(x))2 + 2 (xi 0 ui 0 Fi(x)) +
bi(x) = q
and Then we have or, in a matrix form,
(3.7)
ai(x) = 2 0 bi(x):
(3.8)
r i(x) = ai(x)ei + bi(x)rFi(x);
(3.9)
r8(x) = diag (ai(x)) + rF (x) diag (bi(x)):
(3.10)
9
Lemma 3.2 We have for every 6= 0 and any x 2 Rn
0 ai(x) < 2; 0 < bi (x) 2;
and
2
2
8i 62 1(x) [ 2 (x)
1 ai(x) 1 + 2(u 0 l )2 ; 1 0 2(u 0 l )2 bi(x) 1; i i i i
8i 2 1 (x) [ 2 (x):
(3.11) (3.12)
If either li or ui is nite, then all inequalities in (3.11) hold strictly.
Proof If li = 01 and ui = 1, then 1 (x) = 2 (x) = ;. Moreover, we have for each i 62 1(x) [ 2(x), bi(x) = 2 and ai (x) = 0, and hence (3.11) holds. Now, assume that either li or ui is nite. It is not dicult to see that for every xed 6= 0, the function t 7! pt2t+2 is increasing with t. It then follows that x 0 u 0 F (x) xi 0 li 0 Fi (x) q ; >q i i i (xi 0 li 0 Fi(x))2 + 2 (xi 0 ui 0 Fi(x))2 + 2 which implies that bi(x) > 0 and hence ai (x) < 2 for each i. On the other hand, it is obvious that for any 6= 0 xi 0 ui 0 Fi (x) xi 0 li 0 Fi (x) and (3.13) 1 1: q q 2 2 2 2 (xi 0 li 0 Fi(x)) + (xi 0 ui 0 Fi(x)) + Moreover, at least one of the inequalities in (3.13) must hold strictly because at least one of li and ui is nite. This yields bi(x) < 2 and hence ai(x) > 0. Thus we have (3.11). Note that, in
this case, the inequalities in (3.11) hold strictly. Next, we prove (3.12). If li or ui is in nite, then it follows from (3.7) that ai(x) = bi (x) = 1 for every i 2 (x). Suppose that both li and ui are nite. Then, we have for each i 2 (x) ui 0 li bi(x) = (ui 0 li)2 + 2 1: Moreover, we have ui 0 l i bi(x) = 1 0 1 + (u 0 l )2 + 2 p
p
= 10
i
i
2 (ui 0 li)2 + 2 + (ui 0 li) p(ui 0 li)2 + 2
p
1 0 2(u 10 l )2 2: i i
By the de nition of ai (x), it is not dicult to complete the proof. 2 The inequality (3.5) implies that 8 approximates 8 uniformly. In the following, we will see that r8 (x) serves as a good approximation to an element of @C 8(x). That is, r8 and r8 satisfy the Jacobian consistence property introduced by Chen, Qi and Sun [6]. 10
De nition 2 Let 9 = ( 1 ; 2 ; : : : ; n)T : Rn ! Rn be a Lipschitz continuous function. We call 9 : Rn ! Rn a smoothing approximation function of 9 if 9 is continuously di erentiable and there is a constant > 0 such that for any x 2 Rn and > 0
k9(x) 0 9(x)k :
(3.14)
lim dist(r9(x)T ; @C 9(x)) = 0; #0
(3.15)
Furthermore, if for any x 2 Rn
then we say 9 and 9 satisfy the Jacobian consistence property, where dist(r9 (x)T ; @C 9(x)) denotes the distance from the matrix r9 (x)T to the set of matrices @C 9(x) de ned by
@C 9(x) = @ 1 (x) 2 @ 2 (x) 2 1 11 2 @
n
(x):
From (3.5), it is obvious that the function 8 de ned by (3.3) is a smoothing function of 8 de ned by (2.3). We now show that 8 and 8 satisfy the Jacobian consistence property. To this end, we construct the matrix V (x) = (v1 (x); v2 (x); : : : ; vn (x)), where vectors vi (x) 2 Rn , i = 1; 2; : : : ; n, are de ned by 2rFi(x); i 2 (x); vi (x) = ai (x)ei + bi (x)rFi (x); i 2 1(x) [ 2 (x); (3.16) 2e i ; i 2 1 (x) [ 2 (x): Since ai (x) and bi (x) satisfy (3.8) and (3.11), it is clear from (2.10) and (3.16) that vi(x) 2 @ i(x) for each i = 1; 2; : : : ; n and any 6= 0, and hence V (x) 2 @C 8(x). By (3.8), we get ai(x)(ei 0 rFi (x)); i 2 (x); r i(x) 0 vi(x) = 0; (3.17) i 2 1(x) [ 2 (x); bi (x)(rFi (x) 0 ei ); i 2 1(x) [ 2(x): It is easy to see by the de nition of (3.7) and (3.8) of ai(x) and bi (x) that 8 > > < > > :
8 > > < > > :
lim b(x) = lim(2 0 ai(x)) = 0; 8i 2 1 (x) [ 2 (x); #0 i #0 and
lim a(x) = lim(2 0 bi(x)) = 0; 8i 2 (x): #0 i #0 We then deduce from (3.17) that for every i = 1; 2; : : : ; n lim (r i(x) 0 vi (x)) = 0: #0 11
(3.18)
Let V (x) = (v1(x); v2 (x); : : : ; vn (x)) =4 lim#0 V (x). Then we get from (3.12) and (3.16) 2rFi(x); i 2 (x); vi (x) = ei + rFi (x); i 2 1 (x) [ 2(x); 2e i ; i 2 1 (x) [ 2 (x): It is not dicult to see that V (x) 2 @C 8(x). Therefore, (3.18) implies lim dist(r8(x); @C 8(x)) lim kr8 (x) 0 V (x)k = 0: #0 #0 8 > > < > > :
The above discussion has shown the following proposition.
Proposition 3.1 Let F be continuously di erentiable. Then consistence property.
8
and
8
satisfy the Jacobian
In the next proposition, we give sucient conditions to ensure the nonsingularity of r8 (x). Proposition 3.2 Let F be continuously di erentiable. Then r8(x) is nonsingular for every 6= 0 if one of the following conditions holds. (i) rF (x) is a P matrix; (ii) rF (x) is a P0 matrix and, for each i 62 1(x) [ 2 (x), either li or ui is nite. Proof (i) From Lemma 3.2, ai (x) 0 and bi(x) > 0 hold for every i and any x. Therefore, in view of (3.10), the nonsingularity of r8 (x) is equivalent to the nonsingularity of the matrix diag ai (x)bi(x)01 + rF (x). If rF (x) is a P matrix, then so is diag ai (x)bi(x)01 + rF (x) and hence nonsingular. If rF (x) is a P0 matrix and for each i 62 1(x) [ 2 (x), either li or ui is nite, then by Lemma 3.2 again, the matrix diag ai(x)bi(x)01 is a diagonal matrix with positive diagonal elements, and hence the nonsingularity of the matrix diag ai(x)bi(x)01 + rF (x) follows from Lemma 5.1 in De Luca et al. [8]. 2 To conclude this section, we show that if F is a continuously di erentiable uniform P function, then the smooth equation 8(x) = 0 (3.19) has a unique solution x for each > 0, and x tends to the unique solution of MCP(F ) as ! 0.
Lemma 3.3 Let F be continuously di erentiable. If it is a uniform P function. then there is a constant M > 0 such that for any 0
kx 0 xk M pn;
where x is the unique solution of MCP(F ). In particular, we have
12
(3.20) lim#0 x = x.
We rst prove the existence of a unique solution of (3.19). By Proposition 2.4, the function k8k has bounded level sets. Since 8 satis es (3.5), it follows that the level set
= fx j k8(x)k C g is nonempty and bounded for an arbitrary constant C such that C > inf x k8(x)k. Moreover, it is clear that k8k2 attains its in mum at some point x 2 int . Since x satis es r8(x)8(x) = 0 and r8(x) is nonsingular by Proposition 3.2, we must have 8 (x ) = 0. This shows that (3.19) has a solution. The uniqueness of the solution of (3.19) follows from the nonsingularity of r8(x) for every x. Next, we show that x converges to x. Since x is the unique solution of MCP(F ), it is the unique zero point of 8. So, it follows from (2.12) in Proposition 2.4 that kx 0 xk M k8(x )k = M k8(x) 0 8(x )k M pn; where the last inequality follows from . 2 Proof
4.
A Smoothing Newton Method and Its Convergence
In this section, we propose a smoothing Newton method based on 8 for solving the MCP(F ). For simplicity, we abbreviate 8 , i , etc. as 8k , ki , etc., respectively. In the rest of the paper, we assume that F is continuously di erentiable. In the algorithm, the subproblem is the following linear equation r8k (xk )p + 8(xk ) = 0: (4.1) This type of subproblems has been used by many authors (see e.g., [6, 26, 29]). For line search, it is natural to use the squared residual function 41 k 2 (4.2) 'k (x) = 2 k8 (x)k : as the merit function. Generally, in Newton's method, the steplength k is determined by 'k (xk + k pk ) 'k (xk ) + k r'k (xk )T pk or 'k (xk + k pk ) 'k (xk ) + 2k '(xk ); where '(x) = '0(x), pk is the solution of (4.1), and > 0 is a suitable constant. Although it is possible to construct a smoothing Newton method with those line searches, we here adopt a di erent line search that is also e ective in the smoothing quasi-Newton method to be presented in the next section. Let fk g be a positive sequence satisfying k
k
1
X
k=1
k < 1;
13
(4.3)
where is a given positive constant. At iteration k, we determine a steplength k > 0 so that the following inequality holds for = k : k8k (xk + pk )k 0 k8k (xk )k 0kpk k2 + k ; (4.4) where > 0 is a constant. Clearly, at each iteration k, (4.4) holds for all > 0 suciently small since k is positive and independent of . Let 1i(x) = minf(xi 0 li 0 Fi(x))2 ; (xi 0 ui 0 Fi(x))2 g, i = 1; 2; : : : ; n, and 1(x) = minf1i(x) j i 2 (x) [ 1(x) [ 2(x)g: (4.5) The proposed smoothing Newton method is stated as follows.
Algorithm 1 (Smoothing Newton Method): Step 0. Choose constants 1 ; 2 2 (0; 1), 0 < < minf p1 ; p2 g, 1 ; 2 > 0. Select a 2 n n positive sequence fk g satisfying (4.3). Choose an initial point x1 2 Rn and a positive constant 1 2 k8(x1 )k. Let k := 1. Step 1. Stop, if 8(xk ) = 0. Otherwise, solve the linear equation (4.1) to get pk . Step 2. If k8k (xk + pk )k 2 k8k (xk )k 0 1kpk k2 ; (4.6)
then let k := 1 and go to Step 4. Step 3. Let k be the maximum number in the set f1; 1 ; 21 ; : : : ; g such that = i1 satis es the line search condition (4.4) with = 2. Step 4. Let xk+1 := xk + k pk . Step 5. Update k by the following rule: If (4.6) holds or k8(xk+1)k k , let
1 k+1 := minf k8(xk+1 )k; k ; 1(xk+1 )g; (4.7) 2 2 where 1(x) is de ned by (4.5). Otherwise, let k+1 := k . Step 6. Let k := k + 1. Go to Step 1. Remark (i) It is not dicult to see that if Algorithm 1 generates an in nite sequence fxk g, then 8k (xk ) 6= 0 for all k, and the positive sequence fk g is nonincreasing and satis es
k k8(xk )k;
It then follows from (3.5) that
(4.8)
8k: p
k k8(xk )k (k8k (xk )k + nk ):
Since pn < 12 by Step 0, we get
k k8k (xk )k;
14
(4.9)
where = =(1 0 pn ) 2 (0; p1n ). (ii) De ne index set K = K1 [ K2 , where K1 = fk j k8(xk+1)k k g; K2 = fk j k8k (xk + pk )k 2 k8k (xk )k 0 1 kpk k2 g:
(4.10)
Then, k 12 k01 if k 0 1 2 K , and k = k01 if k 0 1 62 K . Lemma 4.1 Let fxk g be generated by Algorithm 1. Then for every k , we have
k8k+1(xk+1 )k 2 k8k (xk )k 0 1kxk+1 0 xk k2 + pn(k 0 k+1)
(4.11)
if k is determined by Step 2, and
k8k+1(xk+1)k k8k (xk )k 0 2 kxk+1 0 xk k2 + pn(k 0 k+1 ) + k
(4.12)
if k is determined by Step 3. In particular, we have for every k
k8k+1(xk+1)k k8k (xk )k 0 kxk+1 0 xk k2 + pn(k 0 k+1 ) + k ;
where = minf1 ; 2 g. Consequently, we have 1 X kxk+1 0 xk k2 < 1: k=1
(4.13) (4.14)
Moreover, the sequence fk8k (xk )kg converges.
Proof
By (3.4), we clearly have for every k
k8k+1(xk+1)k k8k (xk+1)k + pn(k 0 k+1):
(4.15) If k is determined by Step 2, then (4.11) follows from (4.6) and (4.15). Since 2 2 (0; 1), (4.11) implies (4.13). If k is determined by Step 3, then p(4.12) follows from (4.15) and (4.4) with = 2 , and hence (4.13) holds. Let us denote k = n(k 0 k+1) + k > 0. Since k satis es 1 (4.3), we have k < 1. Therefore, (4.14) follows from (4.13) and, by Lemma 3.3 in [10], k=1 fk8k (xk )kg converges. 2 P
Lemma 4.2 Let fxk g be generated by Algorithm 1. Then the following four statements are equivalent. (i)
lim inf k8(xk )k = 0: k!1 15
(ii)
liminf k8k (xk )k = 0: k!1
(iii)
lim k8(xk )k = 0:
k!1
(iv)
lim k8k (xk )k = 0:
k!1
Since fk8k (xk )kg converges, by Lemma 4.1, (ii) and (iv) are equivalent. From (3.5) and (4.8), we have k8(xk )k k8k (xk )k + pnk k8k (xk )k + pnk8(xk )k: This implies (4.16) k8(xk )k 1 0 1pn k8k (xk )k: Similarly we have k8k (xk )k k8(xk )k + pnk (1 + pn)k8(xk )k: This together with (4.16) shows the equivalence between (i) and (ii) as well as the equivalence between (iii) and (iv). 2 Lemma 4.2 reveals that fxk g has an accumulation point that is a solution of MCP(F ), and every accumulation point of fxk g is a solution of MCP(F ).
Proof
Theorem 4.1 Let fxk g be generated by Algorithm 1. If the index set K de ned by (4.10) is in nite, then we have lim k8(xk )k = 0; (4.17) k!1
and every accumulation point of fxk g is a solution of MCP(F ).
The assumption that K is in nite means that either K2 is in nite, or K2 is nite but K1 is in nite. In the rst case, by Lemma 4.1, (4.11) holds in nitely often. Since 2 2 (0; 1), this implies limk!1 k8k (xk )k = 0. Then (4.17) follows from Lemma 4.2. Now consider the latter case. Without loss of generality, we assume that K1 = K = fk1 < k2 < 11 1g. By Step 5, we have k8(xk +1)k 01k = 01k 01+1 21 k8(xk 01 +1)k i01 11 1 21 k8(xk1+1)k: Proof
i
i
i
16
i
This shows that lim inf k!1 k8(xk )k = 0. Consequently, by Lemma 4.2 again, we get (4.17). Hence every accumulation point of fxk g is a zero point of 8, or equivalently, a solution of MCP(F ). 2 Theorem 4.1 shows that to prove global convergence of Algorithm 1, it suces to verify that the index set K de ned by (4.10) is in nite. To this end, we need the following assumption. Assumption A For each > 0, r8(x) is nonsingular for any x. Sucient conditions for Assumption A to hold are given in Proposition 3.2. The following theorem shows that under Assumption A, the index set K is in nite. Theorem 4.2 Let Assumption A hold and fxk g be generated by Algorithm 1. Then the index set K de ned by (4.10) is in nite and hence (4.17) holds. Consequently, every accumulation point of fxk g is a solution of MCP(F ).
We assume to the contrary that K is nite. Then by Step 5 of Algorithm 1, there is a positive integer k such that k = k holds for all k k, which in turn implies that 8k = 8k for all k k. Moreover, we have by (4.8), k8(xk )k 01 k > 0 for all k k and hence by Lemma 4.2, k8k (xk )k 0 for all k suciently large and some constant 0 > 0. We denote 8k by 8 . Since K2 is nite, k is determined by Step 3 for all k suciently large. So, we have for all k suciently large k8 (xk + k pk )k 0 k8 (xk )k 02kk pk k2 + k : (4.18) This particularly implies that kk pk k ! 0 as k ! 1. Let x be an accumulation point of fxk g and fxk gk2K be a subsequence converging to x. Since 8 is continuously di erentiable and r8 (x) is nonsingular, it follows that when k 2 K is suciently large, r8 (xk ) is uniformly nonsingular. Therefore, (4.1) implies that fpk gk2K is bounded. Without loss of generality, we assume that fpk gk2K converges to some vector p. Let = limsupk2K;k !1 k . Then 2 [0; 1]. If > 0, then p = 0. So it follows from (4.1) that 8(x) = 0. This is a contradiction. Now, suppose = 0, or equivalently, limk2K;k !1 k = 0. By Step 3 of Algorithm 1, when k 2 K is suciently large, 4 k =1 does not satisfy (4.18). So, we have 0k = k8 (xk + 0k pk )k 0 k8 (xk )k 02 k0k pk k2: Multiplying both sides by (0k )01 k8 (xk + 0k pk )k + k8 (xk )k and then taking the limit as k ! 1 with k 2 K yield 28 (x)T r8 (x)p 0: It then follows from (4.1) that 8 (x)T 8(x) 0: Proof
17
This together with (4.8) implies (1 0 n 2 )k8(x)k2 k8 (x)k2 + (1 0 n 2)k8(x)k2 k8 (x)k2 + k8(x)k2 0 n2 = k8 (x) 0 8(x)k2 0 n2 + 28 (x)8(x) 28 (x)T 8(x) 0; (4.19) where the third inequality follows from (3.5). Since < p1n , (4.19) is also a contradiction. Therefore, K must be in nite, and hence the assertion follows from Theorem 4.1. 2 De ne the level set (4.20)
= fx j k8(x)k 47 k8(x1 )k + g; where the positive constant is given in (4.3). The following theorem shows that if is bounded, then there is at least an accumulation point of fxk g that solves MCP(F ). Theorem 4.3 Let Assumption A hold and the level set be contained in a bounded convex set .
Then the sequence fxk g generated by Algorithm 1 has at least one accumulation point. Moreover, every accumulation point of fxk g is a solution of MCP(F ).
From Theorem 4.2, it suces to show that fxk g is bounded. By the boundedness of , it then suces to verify that fxk g . Indeed, from (4.13), we have for every k k8k+1(xk+1)k k8k (xk )k + pn(k 0 k+1 ) + k 11 1 k81 (x1)k + pn1 + k8(x1 )k + k81(x1 ) 0 8(x1 )k + pn1 + k8(x1 )k + 2pn1 + ; where the last inequality follows from (3.5). This implies
Proof
k8(xk+1)k k8k+1(xk+1)k + k8k+1(xk+1) 0 8(xk+1)k k8(x1 )k + 2pn1 + + pnk+1 k8(x1 )k + 3pn1 + k8(x1 )k + 23 pnk8(x1 )k + 47 k8(x1)k + ;
(4.21) where the last two inequalities follow from Step 0 of Algorithm 1. Since (4.21) holds for all k, we get fxk g as desired. 2 By Proposition 2.4, if F is a uniform P function, then the level set is bounded. So, we have the following corollary. Corollary 4.1 Let F be a continuously di erentiable uniform P function. Then fxk g generated by Algorithm 1 converges to the unique solution of MCP(F ).
18
V
We say that a semismooth function 9 : Rn ! Rn is BD-regular at a point x if every matrix 2 @ 9B (x) is nonsingular, where @B 9(x) is the B-di erential of 9 at x (see [24]). By using BD-regularity, we can prove a strong convergence property for Algorithm 1.
Theorem 4.4 Let Assumption A hold and fxk g be generated by Algorithm 1. Suppose that there is an accumulation point x3 of fxk g at which 8 is BD-regular. Then the whole sequence fxk g converges to x3 .
The BD-regularity implies that x3 is an isolated solution of MCP(F ) and hence an isolated accumulation point of fxk g ([24], Proposition 2.5). Since xk+1 0 xk ! 0 by Lemma 4.1, the whole sequence fxk g converges. 2 We now turn to proving superlinear/quadratic convergence of Algorithm 1. We rst prove a useful lemma. Lemma 4.3 Let V (x) be de ned by (3.16) and 1(x) be de ned by (4.5). If 0 < 1(x), then Proof
we have
kr8(x) 0 V (x)k pn(1 + krF (x)k):
(4.22)
It suces to show that for every i = 1; 2; : : : ; n kr i (x) 0 vi(x)k (1 + krF (x)k): (4.23) First, we have for any scalar t p 2 2 j 1 0 1 (4.24) 1 0 p1 + t2 = p1 +1 +t2 t j j1 0 1 + t2 j = 1 + pt 1 + t2 12 t2 : For i 2 1 (x), we have xi 0 li 0 Fi(x) < 0 and xi 0 ui 0 Fi(x) < 0 . Letting t = 0 xi 0 li 0 Fi(x) in (4.24), we get 2 xi 0 li 0 Fi (x) 1(x) 1 : + 1 2 2 2 2 2(xi 0 li 0 Fi(x)) 2(xi 0 li 0 Fi (x)) 2 (xi 0 li 0 Fi(x)) + Similarly, letting t = 0 xi 0 ui 0 Fi(x) in (4.24), we get 1 xi 0 ui 0 Fi (x) (xi 0 ui 0 Fi(x))2 + 2 + 1 2 : Therefore, we get for every i 2 1(x) jbi(x)j = (x x0i 0l l0i 0F F(xi ())x2) + 2 + 1 0 (x x0i 0u u0i 0F F(xi ())x2) + 2 0 1 i i i i i i xi 0 ui 0 Fi (x) xi 0 li 0 Fi (x) (x 0 l 0 F (x))2 + 2 + 1 + (x 0 u 0 F (x))2 + 2 + 1 i i i i i i Proof
p
p
p
p
p
p
:
19
p
This together with (3.17) implies that (4.23) holds for every i 2 1 (x). By a similar discussion, we can show that (4.23) holds for every other i. The proof is then complete. 2 Let K be de ned by (4.10). Then, by Step 5 of Algorithm 1, for every k with k 0 1 2 K , we have k 1(xk ). Since k k8(xk )k holds for all k, Theorem 4.2 implies limk!1 k = 0. So, we get the following corollary from Theorem 4.2 and Lemma 4.3. Corollary 4.2 Let the conditions of Theorem 4.2 hold. Then
lim kr8k (xk ) 0 Vk k = 0;
k!1 k012K
(4.25)
where Vk = V k (xk ) is de ned by (3.16).
The following theorem shows another good property of the smoothing function 8 . Theorem 4.5 Let Assumption A hold and fxk g be generated by Algorithm 1. Suppose that fxk g converges to x3 . Then we have
lim dist(r8k (xk ); @C 8(x3 )) = klim dist(Vk ; @C 8(x3 )) = 0; !1
k!1 k01 2 K
(4.26)
where Vk = V k (xk ) is de ned by (3.16).
Since (4.25) holds, we only need to verify the last equality. Let I1 [ I2 [ I3 be an arbitrary partition of the index set (x3 ) = 1 (x3 ) [ 2 (x3 ). That is, Ii \ Ij = ;, i 6= j, i; j = 1; 2; 3 and I1 [ I2 [ I3 = (x3 ). Since, by Proposition 2.3, f2ei ; 2rFi (x3 ); ei + rFi (x3 )g @ i (x3 ) for each 4 i 2 (x3 ), @C 8(x3 ) contains matrix V = (v1 ; v2 ; : : : ; vn) with 2rFi(x3); i 2 (x3 ) [ I1 ; (4.27) vi = 2 e i ; i 2 1(x3 ) [ 2 (x3) [ I2; ei + rFi (x3 ); i 2 I3 : We verify the last equality of (4.26) by showing that every accumulation point of fVk g is in @C 8(x3 ). It suces to show that for an arbitrary accumulation point V 3 = (vi3 ; v23 ; : : : ; vn3 ) of fVk g, there is a partition I1 [ I2 [ I3 of (x3) such that, for each i, vi3 is expressed in the form of (4.27). Let V 3 be an accumulation point of fVk g, and fVk gk2K be a subsequence converging to V 3 . For every i 2 (x3 ), we have i 2 (xk ) for all k 2 K suciently large. Therefore, from (3.16), we get vik = 2rFi(xk ) for every i 2 (x3 ) and all k suciently large, and hence vi3 = 2rFi (x3 ) for every i 2 (x3 ). Similarly, we have vi3 = 2ei for each i 2 (x3). Now, consider the indices i 2 (x3 ). By taking a further subsequence of fxk gk2K if necessary, we can partition the index (x3) as I1 , I2 , I3 such that I1 = fi 2 (x3 ) j i 2 (xk ) for all k 2 K g, Proof
8 > > < > > :
20
for all k 2 K g, I3 = fi 2 (x3 ) j i 2 (xk ) for all k 2 K g. Then, taking into account (3.12), we see from (3.16) that vi3 = 2rFi(x3 ) for each i 2 I1 , vi3 = 2ei(x3 ) for each i 2 I2 , and vi3 = ei + rFi(x3) for each i 2 I3 . Hence V 3 = (v13 ; v23 ; : : : ; vn3 ) is represented in the form of (4.27). 2 Now, we show superlinear/quadratic convergence of Algorithm 1. I2 = fi 2 (x3 ) j i 2 (xk )
Theorem 4.6 Let Assumption A hold and fxk g be generated by Algorithm 1. Suppose that there is an accumulation point x3 of fxk g such that all matrices in @C 8(x3 ) are nonsingular. Then fxk g converges to x3 superlinearly. If in addition, rF is Lipschitz continuous at x3, then the convergence rate is quadratic.
It is clear that the given conditions imply that 8 is BD-regular at x3 . Therefore, by Theorem 4.4, fxk g ! x3 . Let K be de ned by (4.10). By Theorem 4.5, there is a constant M1 > 0 such that kr8k (xk )01 k M1 for all k suciently large with k 0 1 2 K . So, it follows from (4.1) that for every k with k 0 1 2 K
Proof
kxk + pk 0 x3 k = k 0 r8k (xk )01 8(xk ) 0 8(x3 ) 0 r8k (xk )(xk 0 x3) k kr8k (xk )01 k k8(xk ) 0 8(x3 ) 0 Vk (xk 0 x3 )k + k(r8k (xk ) 0 Vk )(xk 0 x3 )k
n
X
M1 k i(xk ) 0 i(x3) 0 (vik )T (xk 0 x3 )k + kr8k (xk ) 0 Vk k kxk 0 x3 k i=1 = o(kxk 0 x3 k); (4.28)
where Vk = V (xk ) is given by (3.16), and the last equality follows from (4.25) and the fact that vik 2 @ i (xk ) together with the semismoothness of i . Then we have for all k suciently large with k 0 1 2 K k8k (xk + pk )k k8k (xk + pk ) 0 8(xk + pk )k + k8(xk + pk ) 0 8(x3 )k pnk + O(kxk + pk 0 x3 k) pnk8k (xk )k + o(kxk 0 x3 k) = 2 k8k (xk )k 0 (2 0 pn )k8k (xk )k + o(kxk 0 x3 k); (4.29) where the second inequality follows from (3.5) and the local Lipschitzian property of 8, and the third inequality follows from (4.9) and (4.28). On the other hand, since 8 is BD-regular at x3 , there is a constant m > 0 such that k8(x)k = k8(x) 0 8(x3 )k mkx 0 x3 k for all x near x3 . So, it follows that for all k suciently large with k
21
k012 K
k8k (xk )k
k8(xk )k 0 k8k (xk ) 0 8(xk )k k8(xk )k 0 pnk (1 0 pn )k8(xk ) 0 8(x3 )k (1 0 pn )mkxk 0 x3k = kxk 0 x3k;
(4.30) where = (1 0 pn )m > 0 and the third inequality follows from (4.8). Also it follows from (4.1) that for all k suciently large with k 0 1 2 K kpk k kr8k (xk )01k k8(xk )k M1 k8(xk ) 0 8(x3 )k M1 Lkxk 0 x3 k; (4.31) where L > 0 stands for the Lipschitz constant of 8 at x3. Therefore, it follows from (4.29), (4.30), and (4.31) that for all k suciently large with k 0 1 2 K k8k (xk + pk )k 0 2 k8k (xk )k + 1kpk k2 0(2 0 pn )k8k (xk )k + o(kxk 0 x3 k) 0: (4.32) This shows that (4.6) holds, and hence k 2 K . By induction, we claim that k 2 K for all k suciently large. Moreover, by Step 2 in Algorithm 1, (4.32) means that k = 1 is accepted. Consequently, (4.28) shows the superlinear convergence of fxk g. If rF is Lipschitz continuous at x3, then 8 is strongly semismooth at x3 . By means of (4.22) and (4.8), we have kr8k (xk ) 0 Vk k = O(kxk 0 x3 k2 ). Therefore, in a similar way to (4.28), we can deduce kxk + pk 0 x3 k = O(kxk 0 x3 k2 ) and hence, kxk+1 0 x3k = O(kxk 0 x3k2 ). That is, fxk g converges to x3 quadratically. 2 The proof of Theorem 4.6 shows that the index set K de ned by (4.10) contains all k suciently large. Therefore, we get the following corollary from Theorem 4.5. Corollary 4.3 Let the conditions of Theorem 4.4 hold. Then we have
lim dist(r8k (xk ); @C 8(x3 )) = 0: Since @C 8(x3 ) is closed, Corollary 4.3 shows that any accumulation point of fr8k (xk )g is an element of @C 8(x3 ). In particular, if 8 is di erentiable at x3 , then @C 8(x3 ) reduces to a singleton, and hence lim r8k (xk ) = r8(x3 ): k!1 k!1
5.
A Smoothing Quasi-Newton Method
In this section, based on the smoothing Newton method presented in Section 4, we propose a smoothing Broyden-like method for solving MCP(F ). The basic idea is to use a nonsingular matrix Bk instead of r8k (xk ) in Algorithm 1. The algorithm is stated as follows. 22
Algorithm 2 (Smoothing Broyden-like Method) Step 0. Choose constants 1 ; 2 2 (0; 1), 0 < < minf p1 ; p2 g, 1 ; 2 > 0. Choose an 2 n n n initial point x1 2 R , an n 2 n nonsingular matrix B1 , and positive constants 1 2 k8(x1 )k and 1 k8(x1 )k. Let k := 1. Step 1. Stop, if 8(xk ) = 0. Otherwise, solve the linear equation
to get pk . Step 2.
If
Bk p + 8(xk ) = 0
(5.1)
k8k (xk + pk )k 2 k8k (xk )k 0 1kpk k2 ;
(5.2)
then let k := 1 and go to Step 4. Step 3. Let k be the maximum number in the set f1; 1; 21 ; : : : ; g such that = i1 satis es the line search condition (4.4) with = 2 . Step 4. Let xk+1 := xk + k pk . Step 5. Update Bk by the Broyden-like update formula (y 0 Bk sk )sTk ; B = B + k k+1
k
k
ksk k2
where sk := xk+1 0 xk and yk := 8k (xk+1 ) 0 8k (xk ). The parameter k is chosen so that jk 0 1j for some constant 2 (0; 1) and Bk+1 is nonsingular. Step 6. Update k by the following rule: If (5.2) holds or k8(xk+1)k k , let
1 k+1 := minf k8(xk+1 )k; k g: 2 2 Otherwise, let k+1 := k . Step 7. Let k+1 := minf 21 k ; k8(xk+1 )kg and k := k + 1. Go to Step 1. Remark (i) Since k 12 k01 for all k, it is obvious that (4.3) holds. (ii) Similar to Algorithm 1, we have for every k k k8(xk )k; k k8k (xk )k and k k8(xk )k; (5.3) where = 1 0 pn 2 (0; p1n ) (see (4.16)). (iii) We still denote K = K1 [ K2 with K1 = fk j k8(xk+1 )k k g; K2 = fk jk8k (xk + pk )k 2k8k (xk )k 0 1 kpk k2g: (5.4) Then, k 21 k01 holds for every k with k 0 1 2 K , and k = k01 holds for every k with k 0 1 62 K . 23
(iv) It is not dicult to see that the sequence fxk g generated by Algorithm 2 is contained in the level set de ned by (4.20). Moreover, Lemmas 4.1, 4.2 and Theorem 4.1 remain true for Algorithm 2. This means that to show global convergence of Algorithm 2, it suces to verify that K de ned by (5.4) is in nite. In order to establish global convergence for Algorithm 2, we need the following assumption. Assumption B (i) The level set de ned by (4.20) is contained in a bounded convex set . (ii) The function F is continuously di erentiable and rF is Lipschitz continuous on . For given > 0 and > 0, de ne the set 5() = fx j (xi 0 li 0 Fi(x))2 + 2 ; (xi 0 ui 0 Fi(x))2 + 2 ; i = 1; 2; : : : ; ng \ : (5.5) The following lemma shows that under Assumption B, r8 is Lipschitz continuous on 5 (). Lemma 5.1 Let Assumption B hold. Then for any > 0 and > 0, there exists a positive constant L independent of , such that
kr8(x) 0 r8(y)k L kx 0 yk; 8x; y 2 5():
(5.6) We prove (5.6) by showing that there is a positive constant l such that for each i =
Proof 1; 2; : : : ; n
kr i (x) 0 r i (y)k l kx 0 yk; 8x; y 2 5():
We rst show that there is a constant l0 > 0 such that for each i = 1; 2; : : : ; n jai (x) 0 ai(y)j = jbi(x) 0 bi (y)j l0 kx 0 yk; 8x; y 2 5(): By an elementary deduction, we get for any scalars t1 ; t2 and > 0 t1 t2 jt1 0 t2 j + jt j 1 0 1 0 2 t21 + 2 t22 + 2 t21 + 2 t21 + 2 t22 + 2 j t2 + 2 0 t2 + 2j = jt1 20 t2 j2 + jt2 j 1 2 2 2 2 2 t1 + t1 + t2 + 21 2 jt1 0 t2 j + j t21 + 2 0 t22 + 2 j t1 + 2 2 = 21 2 jt1 0 t2 j + 2 jt12 0 t2 j 2 2 t1 + t1 + + t2 + 21 2 jt1 0 t2 j + (jt1j +jt jjt2+j)jjtt1j0 t2 j 1 2 t1 + = 2jt12 0 t22j : t1 + q
q
q
q
q
q
(5.8)
q
q
q
(5.7)
q
q
q
q
q
q
q
q
q
24
So, we get by the de nition (3.7) of bi jbi(x) 0 bi(y)j xi 0 li 0 Fi (x) xi 0 ui 0 Fi (x) 0 = 2 2 (xi 0 li 0 Fi(x)) + (xi 0 ui 0 Fi(x))2 + 2 0 (y y0i 0l l0i 0F F(yi())y2) + 2 0 (y y0i 0u u0i 0F F(yi())y2) + 2 i i i i i i xi 0 li 0 Fi (x) yi 0 li 0 Fi (y ) (x 0 l 0 F (x))2 + 2 0 (y 0 l 0 F (y))2 + 2 i i i i i i yi 0 ui 0 Fi (y ) xi 0 ui 0 Fi (x) + (x 0 u 0 F (x))2 + 2 0 (y 0 u 0 F (y))2 + 2 i i i i i i 2j(xi 0 li(0x F0i(lx))0 0F ((yxi))02 l+i 02Fi(y))j + 2j(xi 0 ui(x0 F0i(ux))00F ((yxi))02 u+i 02 Fi(y))j i i i i i i 4 jxi 0 yij + jFi (x) 0 Fi(y)j : Since F is continuously di erentiable, it is Lipschitz continuous on the bounded set 5 (). The last inequality then implies that (5.8) holds for each i with a suitable constant l0 independent of . Therefore, by (3.9), we get for each i kr i (x) 0 r i(y)k (ai(x) 0 ai (y))ei + bi(x)rFi(x) 0 bi (y)rFi(y) jai(x) 0 ai (y)j + k(bi(x) 0 bi(y))rFi(x)k + kbi(y)(rFi(x) 0 rFi(y))k: Since rF is Lipschitz continuous and jbi(x)j 2, the last inequality and (5.8) imply (5.7), which in turn implies (5.6) with some constant L > 0. 2 De ne ky 0 Bk sk k and A = 1 r8k (x + s )d : (5.9) k = k k k k ksk k 0 Then, we have yk = Ak sk and k(Ak 0 Bk )sk k = k(Ak 0 Bk )pk k : k = (5.10) ksk k kp k k So, it follows that k(Bk 0 r8k (xk ))pk k k(Ak 0 Bk )pk k + k(r8k (xk ) 0 Ak )pk k (k + kr8k (xk ) 0 Ak k)kpk k: (5.11) In a similar way to Lemma 4.1 in [20], it is not dicult to prove the following useful lemma. p
p
p
p
p
p
p
p
p
p
Z
Lemma 5.2 Let Assumption B hold. If the index set K de ned by (5.4) is nite, then we have
1 lim k!1 k
k
X
i=1
25
i2 = 0:
(5.12)
Now, we are ready to prove global convergence for Algorithm 2. Theorem 5.1 Let Assumptions A and B hold and fxk g be generated by Algorithm 2. Then the index set K must be in nite and hence
lim k8(xk )k = 0:
k!1
Moreover, every accumulation of fxk g is a solution of MCP(F ).
Proof Assume to the contrary that K is nite. Then there is an integer k such that k = k for all k k. Denote 8 = 8k . Notice that the niteness of K implies that K2 de ned by (5.4) is also nite, and hence there is an index k~ such that when k k~, the steplength k is determined by Step 3. That is, we have for every k maxfk; k~g
k8 (xk + k pk )k 0 k8 (xk )k 02kk pk k2 + k :
By (5.12), there is a subsequence fk gk2K converging to zero. Let x be an arbitrary accumulation point of the corresponding sequence fxk gk2K . Without loss of generality, we assume limk2K;k !1 xk = x. By (5.1) and (5.11), we have k8(xk )k k(Bk 0 r8 (xk ))pk k + kr8 (xk )pk k (k + kr8k (xk ) 0 Ak k + kr8 (xk )k)kpk k and kpk k =
kr8 (xk )01(8(xk ) + (Bk 0 r8 (xk ))pk k kr8 (xk )01k k8(xk )k + k(Bk 0 r8 (xk ))pk k k8 (xk )01k k8(xk )k + (k + kr8 (xk ) 0 Ak k) kpk k :
(5.13)
So, there exists a constant C > 0 such that for all k 2 K suciently large k8(xk )k C kpk k
and
kpk k C k8(xk )k:
(5.14)
In a similar way to the proof of Theorem 4.2, we can deduce a contradiction. 2 Following the same arguments as in the proof of Theorem 4.4, it is not dicult to prove the following theorem. Theorem 5.2 Let Assumptions A and B hold. Suppose that there is an accumulation point x3 of fxk g at which 8 is BD-regular. Then the whole sequence fxk g converges to x3 .
26
Now we turn to proving superlinear convergence of Algorithm 2. For this purpose, we make further assumptions. Assumption C (i) The sequence fxk g generated by Algorithm 2 converges to a solution x3 of MCP(F ) at which 8 is BD-regular. (ii) The strict complementarity holds at x3, i.e., 1(x3 ) = 2 (x3 ) = ;; in other words, (x3i 0 li)2 + Fi(x3)2 > 0 and (x3i 0 ui)2 + Fi (x3 )2 > 0; Let and
=
i = 1; 2; : : : ; n:
(5.15)
1 minfjx3 0 l 0 F (x3)j; jx3 0 u 0 F (x3 )jj i = 1; 2; : : : ; ng: i i i i i i 2
U 3 = fx j jxi 0 li 0 Fi (x)j ; jxi 0 ui 0 Fi (x)j ; i = 1; 2; : : : ; ng:
It is clear that, under Assumption C, > 0 and U 3 6= ;. It is also obvious that 1(x) = 2 (x) = ; for every x 2 U 3 , and hence, 8 is continuously di erentiable on U 3 . Speci cally, for every x 2 U 3 , we have r8(x) = (r 1 (x); r 2 (x); : : : ; r n (x)) with (5.16) r i(x) = 2rFi(x); if i 2 (x); 2e i ; if i 2 (x): Moreover, there exists a constant M2 > 0 such that k8(x) 0 8(x3 )k M2 kx 0 x3 k; 8x 2 U 3 : (5.17) Let 5() be de ned by (5.5). Then we have U 3 5 ( 2 ) for any . Therefore, in a similar way to Lemma 5.1, we can prove the following result. (
Lemma 5.3 Let Assumption B hold. Then there is a positive constant L0 independent of such that kr8(x) 0 r8(y)k L0 kx 0 yk; 8x; y 2 U 3 and
kr8 (x) 0 r8 (y)k L0 kx 0 yk; 8 > 0; 8x; y 2 U 3 :
Lemma 5.4 Let a solution x3 of MCP(F ) satisfy (5.15). Then there are constants C1 > 0 and C2 > 0 such that for every > 0
k8(x) 0 8(x)k C1 2; and
kr8(x) 0 r8(x)k C2 2 ;
27
8x 2 U 3 8x 2 U 3:
(5.18) (5.19)
We deduce from (3.2) that for every (a; b; c) 2 R 3 2 2 j (a; b; c) 0 (a; b; c)j max 2jb0 cj ; 2ja0 bj : Substituting xi 0 li for a, Fi(x) for b, and xi 0 ui for c in the last inequality yields 2 2 j i (x) 0 i(x)j max 2jx 0 u 0 F (x)j ; 2jx 0 l 0 F (x)j : i i i i i i 3 This implies that for all x 2 U 2 : j i(x) 0 i(x)j 2 pn Letting C1 = 2 , we get (5.18). Now we prove (5.19). From (3.9) and (5.16), we have for every > 0 and any x 2 U 3 i 2 (x); (5.20) r i(x) 0 r i(x) = ai (x)(ei 0 rFi(x)); bi (x)(0ei + rFi (x)); i 2 (x): For each i 2 1(x), we have by (3.7) jbi (x)j (x x0i 0l l0i 0F F(xi())x2) + 2 + 1 + (x x0i 0u u0i 0F F(xi())x2) + 2 + 1 : (5.21) i i i i i i Moreover, we have xi 0 li 0 Fi (x) (5.22) (xi 0 li 0 Fi(x))2 + 2 + 1 2 2 = (Fi(x) 0 xi +(xli0) 0l 0(Fxi(0x))li20+F2i(x)) + i i i 2 = (xi 0 li 0 Fi(x))2 + 2 (Fi(x) 0 xi + li) + (xi 0 li 0 Fi(x))2 + 2 (5.23) 21 2 2 : Similarly, we have 1 2 : xi 0 ui 0 Fi (x) (5.24) + 1 2 2 (xi 0 ui 0 Fi(x))2 + 2 Combining (5.23) and (5.24) with (5.21), we get jbi(x)j 12 2: (5.25) By a similar argument, we can show that (5.25) holds for each i 2 2(x), every > 0 and any x 2 U 3. Moreover, we have for each i 2 (x), every > 0 and any x 2 U 3 jai(x)j 12 2 : Then by using (5.20), we can deduce (5.19) with some constant C2 > 0. 2 Proof
o
n
n
o
(
p
p
p
p
p
p
p
p
28
Lemma 5.5 Let Assumptions B and C hold, fxk g be generated by Algorithm 2, and k be de ned by (5.9). Then there exist a constant > 0 and an integer k0 > 0 such that k = 1 whenever k and k k0 .
Since xk ! x3 , there is an index k10 such that xk 2 U 3 for all k k10 . Let Ak be de ned by (5.9). Then we have for all k k10 1 1 kr8k (xk + sk ) 0 r8(xk + sk )kd + kr8(xk + sk ) 0 r8(x3 )kd kAk 0 r8(x3 )k 0 0 Proof
Z
Z
Z 1 2 C2 k + L0 k (xk+1 0 x3 ) + (1 0 )(xk 0 x3)kd 0 C2 2 k8(xk ) 0 8(x3 )k2 + L0 (kxk+1 0 x3 k + kxk 0 x3k) C2 2 M22kxk 0 x3 k2 + L0(kxk+1 0 x3k + kxk 0 x3 k) (L0 + C2 2M22 kxk 0 x3k)kxk 0 x3 k + L0 kxk+1 0 x3k
= =4 k ; (5.26) where the second inequality follows from (5.19) and Lemma 5.3, and the third and the fourth inequalities follow from (5.3) and (5.17), respectively. It is obvious that k ! 0 as k ! 1. By using (5.1), we have r8(x3)pk = (r8(x3) 0 Ak )pk + (Ak 0 Bk )pk 0 F (xk ): This together with (5.10) yields kpk k kr8(x3 )k kr8(x3 ) 0 Ak kkpk k + k(Ak 0 Bk )pk k + kF (xk )k kr8(x3 )k(k + k )kpk k + kr8(x3)k kF (xk )k: Since k ! 0 as k ! 1, the last inequality shows that there are constants 0 > 0 and C3 > 0 such that when k 0 and k is suciently large kpk k C3k8(xk )k = k8(xk ) 0 8(x3 )k C3 M2 kxk 0 x3 k: (5.27) From (5.1) again, we have r8(x3 )(xk + pk 0 x3 ) = r8(x3 )(xk 0 x3 ) 0 8(xk ) 0 (Bk 0 r8(x3))pk : Let M3 = kr8(x3 )01k. It then follows that kxk + pk 0 x3 k kr8(x3 )01k 8(xk ) 0 r8(x3)(xk 0 x3 ) + (Bk 0 r8(x3))pk M3 k8(xk ) 0 8(x3 ) 0 r8(x3 )(xk 0 x3 )k + k(Bk 0 Ak )pk k + k(Ak 0 r8(x3 ))pk k k0 kxk 0 x3 k + M3 (k + k )kpk k (k0 + C3 M2 M3(k + k ))kxk 0 x3 k =4 (~k + C3 M2 M3k )kxk 0 x3 k; (5.28)
29
where k0 = M3k8(xk ) 0 8(x3 ) 0 r8(x3 )(xk 0 x3)k=kxk 0 x3k, ~k = k0 + C3 M2 M3 k ), the third inequality follows from (5.10) and (5.26), and the last inequality follows from (5.27). So, we deduce that when k 0 and k is suciently large k8k (xk + pk )k k8k (xk + pk ) 0 8(xk + pk )k + k8(xk + pk ) 0 8(x3 )k C1 2k + M2 kxk + pk 0 x3 k C1 2k8(xk ) 0 8(x3)k2 + M2 (~k + C3 M2M3k )kxk 0 x3 k (C1 2 M22 kxk 0 x3 k + M2(~k + C3 M2 M3 k ))kxk 0 x3 k =4 (^k + C4 k )kxk 0 x3 k; (5.29) where ^k = C1 2M22 kxk 0 x3 k + M2 ~k and C4 = C3 M22 M3, the second inequality follows from (5.18) and the third inequality follows from (5.3) and (5.28). On the other hand, by the given assumptions, it is not dicult to see that (4.30) still holds. Therefore, we get that when k 0 and k is suciently large k8k (xk + pk )k 0 2 k8k (xk )k + 1 kpk k2 (^k + C4 k )kxk 0 x3k 0 2 kxk 0 x3 k + 1 C32 M22 kxk 0 x3 k2 = 02 kxk 0 x3k + C4 k kxk 0 x3 k + o(kxk 0 x3 k); where the inequality follows from (5.29), (4.30) and (5.27). It is then not dicult to see that there exist a constant 0 such that when k and k is suciently large k8k (xk + pk )k 0 2 k8k (xk )k + 1kpk k2 0: This means k = 1. The proof is complete. 2 The following lemma shows that under Assumptions B and C, (5.12) holds even if K is in nite. Lemma 5.6 Let Assumptions B and C hold. Then
1 lim k!1 k
k
X
i=1
i2 = 0:
(5.30)
It is clear that when k is suciently large, xk + sk 2 U 3 for any 2 [0; 1]. It follows from Lemma 5.3 that there is a constant L0 > 0 such that for any 2 [0; 1] kr8k01 (xk + sk ) 0 r8k01 (xk01 + sk01 )k L0k(xk + sk ) 0 (xk01 + sk01 )k L0 ksk k + (1 0 )ksk01 k (5.31) L0 ksk k + ksk01k : Proof
30
Let Ak be de ned by (5.9). Then we have kAk 0 Ak01 k =
Z
1
Z
k(r8k (xk + sk ) 0 r8k01(xk + sk ))d k
0 1
0Z
r8 (xk + sk )d 0 k
Z
0
1
r8k01(xk01 + sk01)d
1
k(r8k01(xk + sk ) 0 r8k01(xk01 + sk01))d k pn(k01 0 k ) + L0 ksk k + ksk01k
+
0
=4 k ; where the last inequality follows from (3.4) and (5.31). It is easy to deduce that 1 1 X X 2 2 2 ksk k2 + ksk01 k2 : k 2n (k01 0 k ) + 4L0 k=1 k=1 k=1 1
X
Since 1k=1 ksk k2 < 1 (see Remark (iv) after Algorithm 2) and fk g is nonincreasing, it is clear that 1k=1 k2 < 1. This implies 1k=1 kAk 0 Ak01 k2 < 1. In a way similar to Lemma 4.1 in [20], we can then prove (5.30). 2 The following theorem shows superlinear convergence of Algorithm 2. P
P
P
Theorem 5.3 Let Assumptions B and C hold. Then fxk g converges superlinearly.
We rst show that there is a constant c > 0 such that for all k large enough k8(xk+1)k (!k + ck )k8(xk )k; (5.32) where f!k g is a positive sequence converging to zero. De ne the index set K~ = fk j k and k k0g, where and k0 are constants given in Lemma 5.5. Then we get for each k 2 K~ k8(xk+1)k k8(xk+1 ) 0 8k (xk+1)k + k8k (xk+1)k C12k + k8k (xk+1 )k C1 2 k8(xk ) 0 8(x3 )k2 + (^k + C4 k )kxk 0 x3 k (C1 2 M22 kxk 0 x3 k + ^k + C4 k )kxk 0 x3 k; (5.33) where the second inequality follows from (5.18), the third inequality follows from (5.3) and (5.29), and the fourth inequality follows from (5.17). By the nonsingularity of r8(x3 ), there exists a constant m > 0 such that k8(xk ) 0 8(x3)k m kxk 0 x3k for all k suciently large. Letting !k = m 01(C1 2 M22 kxk 0 x3 k + ^k ) and C5 = m 01C4 , it then follows from (5.33) that k8(xk+1)k (!k + C5 k )k8(xk )k: (5.34)
Proof
31
Clearly, !k ! 0 as k ! 1. By Steps 2 and 3, it is clear that k8k (xk+1 )k k8k (xk )k + k k8k (xk )k + k8(xk )k; where the last inequality follows from (5.3). Also we have k > for any k 62 K~ such that k > k0 . Therefore, we get for all k 62 K~ suciently large k8(xk+1)k k8(xk+1 ) 0 8k (xk+1 )k + k8k (xk+1 )k C1 2k + k8k (xk )k + k8(xk )k C1 2k + k8k (xk ) 0 8(xk )k + 2k8(xk )k 2C1 2k + 2k8(xk )k 2C1 2 k8(xk )k2 + xm2k8(xk )k = 2(C1 2k8(xk )k + 1)k8(xk )k (5.35) 2(C1 2k8(xk )k + k )k8(xk )k; where the second and the fourth inequalities follow from (5.18), and the fth inequality follows from (5.3). Letting !k = maxf!k ; 2C1 2 k8(xk )kg and c = maxfC5; 201g, we obtain (5.32) from (5.34) and (5.35). Next, we show the inequality 1 kxk 0 x3 k < 1: (5.36) X
k=1
Without loss of generality, we assume that (5.32) holds for all k. Multiplying those inequalities from k = 1 to t yields k8(xt+1 )k k8(x1 )k
t
Y
(!k + ck )
k=1 t
n Y
= k8(x1 )k
2
(2!k2 + 2c2 k2)
k=1 t X
k8(x1 )k 1t n
o1
k=1 t
n Y
k8(x1 )k
(!k + ck )2
= k8(x1 )k 2t n
k=1 t X k=1
o1
2
(2!k2 + 2c2k2 ) !k2 +
2c 2
t
X
t k=1
ot
2
ot
k2 2 :
Note that both terms in the last braces tend to zero as1 t ! 1, since !k converges to zero and (5.30) holds. Therefore we have limk!1 k8(xk+1 )k = 0 and hence 1k=1 k8(xk+1)k < k
32
P
1. However, k8(xk+1)k = k8(xk+1) 0 8(x3 )k m kxk+1 0 x3 k for all k suciently large.
Consequently, we get (5.36). Now, we show superlinear convergence of Algorithm 2. By virtue of (5.36), in a similar way to the proof of Lemma 5.4 in [20], we can verify that k ! 0 as k ! 1. Therefore, by Lemma 5.5, we have k 1 for all k suciently large. Consequently, (5.28) shows that fxk g converges to x3 superlinearly. 2 6.
Conclusion
In this paper, we proposed a smoothing Newton method and a smoothing quasi-Newton method for solving MCP. Under appropriate conditions, the proposed methods converge globally and superlinearly/quadratically. An advantage of these methods is that the dimension of the subproblems is the same as that of MCP. Moreover, the level set is bounded if the MCP involves a uniform P function. We showed an interesting property for the smoothing function. That is, every accumulation point of fr8k (xk )g is an element of @C 8(x3 ). In particular, if 8 is di erentiable at x3, which is particularly the case when the strict complementarity holds at x3 , then r8k (xk ) ! r8(x3). A drawback of the proposed quasi-Newton method is that it is not a descent method even if F is a strongly monotone function, though it converges globally. It is interesting to construct a descent quasi-Newton method for MCP with global and superlinear convergence. We leave it as a possible further research topic. References
[1] B. Chen, X. Chen and C. Kanzow, \A penalized Fischer-Burmeister NCP-function: Theoretical investigation and numerical results", Technical Report AMR 97/28, School of Mathematics, University of New South Wales, Sydney, September 1997. [2] B. Chen and P.T. Harker, \A non-interior-point continuation method for linear complementarity problems", SIAM J. Matrix Anal. Appl., vol. 14, pp. 1168-1190, 1993. [3] B. Chen and P.T. Harker, \Smooth approximations to nonlinear complementarity problems", SIAM J. Optim., vol. 7, pp. 403-420, 1997. [4] C. Chen and O.L. Mangasarian, \A class of smoothing functions for nonlinear and mixed complementarity problems", Comput. Optim. Appl., vol. 5, pp. 97-138, 1996. [5] X. Chen, Z. Nashed and L. Qi, \Convergence of Newton's method for singular smooth and nonsmooth equations using adaptive outer inverses", SIAM J. Optim., vol. 7, pp. 445-462, 1997. 33
[6] X. Chen, L. Qi and D. Sun, \Global and superlinear convergence of the smoothing Newton method and its application to general box constrained variational inequalities", Math. Comp., vol. 67, pp. 519-540, 1998. [7] F.H. Clarke, Optimization and Nonsmooth Analysis, John Wiley & Sons: New York, 1983. [8] T. De Luca, F. Facchinei and C. Kanzow, \A semismooth equation approach to the solution of nonlinear complementarity problems", Math. Programming, vol. 75, pp. 407-439, 1996. [9] T. De Luca, F. Facchinei and C. Kanzow, \A theoretical and numerical comparison of some semismooth algorithms for complementarity problems", Mathematical Programming Technical Report 97-15, Computer Sciences Department, University of Wisconsin - Madison, Madison, WI, December 1997. [10] J.E. Dennis, Jr. and J.J. More, \A characterization of superlinear convergence and its applications to quasi-Newton methods", Math. Comp., vol. 28, pp. 549-560, 1974. [11] F. Facchinei, A. Fischer and C. Kanzow, \A semismooth Newton method for variational inequalities: The case of box constraints", in Complementarity and Variational Problems: State of the Art, M.C. Ferris and J-S. Pang (Eds.), SIAM, Philadelphia, pp. 76-90, 1997. [12] A. Fischer, \Solution of monotone complementarity problems with locally Lipschitzian functions", Math. Programming, vol. 76, pp. 513-532, 1997. [13] M. Fukushima and L. Qi, (Eds.), Reformulation: Nonsmooth, Piecewise Smooth, Semismooth and Smoothing Methods, Kluwer Academic Publishers, Boston, 1998. [14] S.A. Gabriel and J.J. More, \Smoothing of mixed complementarity problems", in Complementarity and Variational Problems: State of the Art, M.C. Ferris and J.S. Pang (Eds.), SIAM, Philadelphia, pp. 105-116, 1997. [15] S.A. Gabriel and J.S. Pang, \An inexact NE/SQP method for solving the nonlinear complementarity problem", Comput. Optim. Appl., vol. 1, pp. 67-91, 1992. [16] H. Jiang, L. Qi, X. Chen and D. Sun, \Semismoothness and superlinear convergence in nonsmooth optimization and nonsmooth equations", in Nonlinear Optimization and Applications, G. Di Pillo and F. Giannessi (Eds.), Plenum Press: New York, pp. 197-212, 1996. [17] C. Kanzow, \Some noninterior continuation methods for linear complementarity problems", SIAM J. Matrix Anal. Appl., vol. 17, pp. 851-868, 1996. [18] C. Kanzow and H. Jiang, \A continuation method for (strongly) monotone variational inequalities", Math. Programming, vol. 81, pp. 103-125, 1998. 34
[19] C. Kanzow, N. Yamashita and M. Fukushima, \New NCP-functions and their properties", J. Optim. Theory Appl., vol. 94, pp. 115-135, 1997. [20] D.H. Li and M. Fukushima, \Broyden-like methods for variational inequality and nonlinear complementarity problems with global and superlinear convergence", Technical Report 98016, Department of Applied Mathematics and Physics, Kyoto University, Kyoto, September 1998. [21] R. Miin, \Semismooth and semiconvex functions in constrained optimization", SIAM J. Control Optim., vol. 15, pp. 957-972, 1977. [22] J.S. Pang, \A B-di erentiable equation based, globally, and locally quadratically convergent algorithm for nonlinear complementarity and variational inequality problems", Math. Programming, vol. 51, pp. 101-131, 1991. [23] J.S. Pang and L. Qi, \Nonsmooth equations: Motivation and algorithms", SIAM J. Optim., vol. 3, pp. 443-465, 1993. [24] L. Qi, \Convergence analysis of some algorithms for solving nonsmooth equations", Math. Oper. Res., vol. 18, pp. 227-244, 1993. [25] L. Qi, \Regular pseudo-smooth NCP and BVIP functions and globally and quadratically convergent generalized Newton methods for complementarity and variational inequality problems", Technical Report AMR 97/14, School of Mathematics, University of New South Wales, Sydney, July, 1997. [26] L. Qi and X. Chen, \A globally convergent successive approximation method for severely nonsmooth equations", SIAM J. Control Optim., vol. 33, pp. 402-418, 1995. [27] L. Qi and J. Sun, \A nonsmooth version of Newton's method", Math. Programming, vol. 58, pp. 353-367, 1993. [28] L. Qi and D. Sun, \Smoothing functions and a smoothing Newton method for complementarity problems and variational inequality problems", School of Mathematics, University of New South Wales, Sydney, October 1998. [29] L. Qi, D. Sun and G. Zhou, \A new look at smoothing Newton methods for nonlinear complementarity problems and box constrained variational inequalities", AMR 97/13, Applied Mathematics Report, University of New South Wales, Sydney, June 1997. [30] S. Smale, \Algorithms for solving the equations", in Proceedings of the International Congress of Mathematicians, Berkeley, California, pp. 172-195, 1986. 35
[31] D. Sun and R.S. Womersley, \A new unconstrained di erentiable merit function for box constrained variational inequality problems and a damped Gauss-Newton method", to appear in SIAM J. Optim.. [32] K. Yamada, N. Yamashita and M. Fukushima, \A new derivative-free descent method for the nonlinear complementarity problem", to appear in Nonlinear Optimization and Applications 2, G. Di Pillo and F. Giannessi (Eds.), Kluwer Academic Publishers: Boston. [33] N. Yamashita and M. Fukushima, \Modi ed Newton methods for solving a semismooth reformulation of monotone complementarity problems", Math. Programming, vol. 76, pp. 469-491, 1997. [34] N. Yamashita and M. Fukushima, \Equivalent unconstrained minimization and global error bounds for variational inequality problems", SIAM J. Control Optim., vol. 35, pp. 273-284, 1997.
36
