Globally Convergent Broyden-like Methods for ... - Semantic Scholar
Globally Convergent Broyden-like Methods for Semismooth Equations and Applications to VIP, NCP and MCP 1 Donghui Li Department of Applied Mathematics Hunan University Changsha, China 410082 e-mail: [email protected] Masao Fukushima Department of Applied Mathematics and Physics Graduate School of Informatics Kyoto University Kyoto 606-8501, Japan e-mail: [email protected] September 26, 1999
Abstract In this paper, we propose a general smoothing Broyden-like quasi-Newton
method for solving a class of nonsmooth equations. Under appropriate conditions, the proposed method converges to a solution of the equation globally and superlinearly. In particular, the proposed method provides the possibility of developing a quasiNewton method that enjoys superlinear convergence even if strict complementarity fails to hld. We pay particular attention to semismooth equations arising from nonlinear complementarity problems, mixed complementarity problems and variational inequality problems. We show that under certain conditions, the related methods based on the perturbed Fischer-Burmeister function, Chen-Harker-Kanzow-Smale smoothing function and Gabriel-More class of smoothing functions converge globally and superlinearly.
Key words: Nonsmooth equations, Broyden-like method, global convergence, superlinear convergence.
1
The work of the rst author was done while he was visiting Kyoto University. This paper is an extended version of the unpublished technical report [25] by the authors.
1 Introduction Let H be a function from Rn into itself that is not necessarily di erentiable in the sense of Frechet but semismooth in the sense of Miin [27] and Qi [29]. We are concerned with the solution of the semismooth equation H (x) = 0: (1.1) Semismooth equations have a rich practical background. Many problems such as nonlinear complementarity problems, mixed complementarity problems and variational inequality problems can be reformulated as semismooth equations. Newton's method for solving (1.1) has been extensively studied in recent years. In particular, Chen et al. [5] proposed a smoothing Newton method for solving a semismooth equation reformulation arising from the mixed complementarity problem that possesses global and superlinear convergence properties. We refer to [14] and [31] for reviews about Newton-type methods for nonsmooth and semismooth equations. Local convergence of quasi-Newton methods for nonsmooth and semismooth equations has also been studied by some authors [3, 4, 17, 21, 28, 33]. There are a few papers that deal with globally convergent quasi-Newton methods for semismooth equations arising from nonlinear complementarity problems and mixed complementarity problems [24, 26, 35]. In [26], by using a splitting technique of [30], a well de ned quasi-Newton method that possesses global and superlinear convergence properties was rst proposed for nonlinear complementarity problems. In [24], a smoothing quasi-Newton method was developed for a semismooth equation reformulation of the mixed complementarity problem. The method proposed in [24] is an improvement of the method proposed in [26]. In particular, when applied to nonlinear complementarity problems, the method proposed in [24] converges globally whenever the problem involves a P0 function with Lipschitz continuous derivatives. Moreover, if strict complementarity condition holds at the limit point, then the method converges superlinearly. The purpose of this paper is to develop a general smoothing quasi-Newton approach to the semismooth equation (1.1). Under appropriate conditions, the proposed method converges globally and superlinearly. In particular, the results obtained in this paper provides the possibility of developing a globally convergent quasi-Newton method that enjoys superlinear convergence even if strict complementarity fails to hold at the limit point. We pay particular attention to semismooth equations arising from nonlinear complementarity problems, mixed complementarity problems and variational inequality problems. The smoothing functions considered include the perturbed Fischer-Burmeister function, Chen-Harker-Kanzow-Smale smoothing function and Gabriel-More class of smoothing functions. The paper is organized as follows: In the next section, we propose a general quasi-Newton approach to the solution of semismooth equations. In Sections 3 and 4, we prove global and superlinear convergence of a general Broyden-like method, respectively. In Section 5, we specialize 1
the method to the nonlinear complementarity problem, the variational inequality problem and the mixed complementarity problem. Section 6 concludes the paper.
2 General Quasi-Newton Approach This section develops a general quasi-Newton approach to the solution of equation (1.1). Throughout, we assume that H is a semismooth function. The proposed method is a smoothing quasiNewton method. The feature of smoothing methods is to construct a smooth function H as an approximation of H satisfying H 0 (z ) = H (z ) for any z 2 Rn and
kH (z) ? H 0 (z)k ( ? 0); 8 > 0 0;
8z 2 Rn;
(2.1)
and then to generate a sequence fxk g where xk is a solution of the smooth equation
H (x) = 0 k
with k ! 0, so that the sequence fxk g may converge to a solution of (1.1). Smoothing Newton methods have been received much attention in the literature [1, 2, 5, 15, 19, 30, 31]. The quasiNewton method developed here is based on the smoothing Newton method in [5]. For simplicity, we abbreviate H as H k . In the proposed method, the subproblem is the linear equation k
Bk p + H 0(xk ) = 0;
(2.2)
where matrix Bk 2 Rnn is an approximation of rH k (xk ). If Bk = rH k (xk ), the linear equation (2.2) reduces to the subproblem of smoothing Newton methods [5, 30]. Unlike Newton's method, in a quasi-Newton method, derivative-free line search is indispensable because the computation of derivatives should be avoided. We now propose a derivative-free line search that is similar to those in [24]. Let fk g be a positive sequence satisfying 1 X k=1
k < 1;
(2.3)
where is a positive constant. At iteration k, given the current iterate zk and a direction pk , we determine a positive steplength k > 0 such that the following inequality holds for = k :
kH k (zk + pk )k ? kH k (zk )k ?1kH k (zk + pk ) ? H k (zk )k ? 2 k pk k2 + k ;
(2.4)
where 1 ; 2 > 0 are suitable constants. Note that (2.4) is satis ed for all suciently small > 0 since k is positive and independent of . Thus k can be obtained by a backtracking process. 2
We state a general smoothing quasi-Newton method for (1.1) as follows.
Algorithm 1 Step 0. Choose constants ; 2 (0; 1), 0 < < 1=, 1 2 (0; 11+? ), 2 > 0, 2 (0; 1). Select a positive sequence fk g satisfying (2.3). Choose an initial point z1 2 Rn and an initial nonsingular matrix B1 2 Rnn. Choose an 1 > 0 satisfying 1 2 kH 0 (z1 )k. Let k := 1. Step 1. If H 0 (zk ) = 0, stop. Otherwise, solve the linear equation (2.2) to get pk . Step 2. Let k be the maximum element in the set f1; ; 2 ; : : :g such that = i satis es
the line search condition (2.4). Step 3. Let zk+1 = zk + k pk . Step 4. Update Bk to get Bk+1. Step 5. Update k+1 by the following rule: If k < kH 0 (zk+1)k, set k+1 = k . Otherwise, choose k+1 such that 0 < k+1 minf 2 kH 0 (zk+1 )k; 12 k g: Step 6. Let k := k + 1. Go to Step 1. Remark Step 5 of Algorithm 1 ensures that fk g is monotonically nonincreasing and k
kH 0 (zk )k holds for every k. Moreover, if the algorithm does not terminate nitely, i.e., H 0(zk ) 6= 0 for all k, then k > 0 for all k. We also notice that fzk g satis es the inequality
kH k (zk+1)k ? kH k (zk )k ?1kH k (zk+1) ? H k (zk )k ? 2 ksk k2 + k ; where sk = zk+1 ? zk . By (2.1), it then follows that for every k kH k+1(zk+1 )k kH k (zk+1 )k + kH k+1(zk+1 ) ? H k (zk+1)k kH k (zk+1 )k + (k ? k+1) kH k (zk )k ? 1kH k (zk+1 ) ? H k (zk )k ? 2ksk k2 + k + (k ? k+1 ):
(2.5)
Moreover, (2.5) implies
kH k+1(zk+1 )k kH k (zk )k + k + (k ? k+1):
(2.6)
Since k satis es (2.3), we immediately get the following result.
Lemma 2.1 Let fzk g be generated by Algorithm 1. Then fkH k (zk )kg converges. Moreover, we
have and
1 X k=1
1 X k=1
kH k (zk+1 ) ? H k (zk )k < 1
ksk k2 =
1 X k=1
kzk+1 ? zk k2 < 1: 3
(2.7) (2.8)
Proof Since (2.3) implies
1 X
fk + (k ? k+1)g < 1;
k=1
it follows from (2.6) that fkH k (zk )kg converges. The inequality (2.7) then follows from (2.5) directly. Moreover, by (2.5), it is easy to see that
2ksk k2 kH k (zk )k ? kH k+1 (zk+1)k + k + (k ? k+1): 2
Summing the above inequalities yields (2.8). De ne the level set
kH 0 (z )k + 1 g;
= fz 2 Rn j kH 0 (z )k 11 + 1 ? 1 ?
(2.9)
where is a positive constant such that (2.3) holds. Notice that (2.1) particularly implies
kH (z) ? H (z)k ; 8z 2 Rn: (2.10) The following lemma shows that the sequence fzk g generated by Algorithm 1 is contained in the
level set .
Lemma 2.2 Let fzk g be generated by Algorithm 1. Then fzk g . Proof Summing (2.6), we get for every k kH k+1(zk+1)k kH 1 (z1 )k + + 1 kH 0 (z1 )k + + 21 (1 + )kH 0 (z1 )k + ;
(2.11)
where the second inequality follows from (2.10) and the last inequality follows from the choice of 1. By using (2.10) again, we have
kH 0 (zk )k kH k (zk )k + kH k (zk ) ? H 0 (zk )k kH k (zk )k + k kH k (zk )k + kH 0 (zk )k; where the last inequality holds since by Step 5, k kH 0 (zk )k for every k. So, we deduce from
(2.11) and the last inequality
1 : 0 k H ( z kH 0 (zk )k 1 ?1 kH k (zk )k 11 +? 1 )k + 1 ?
This means that fzk g as desired.
4
2
3 Global Convergence of Broyden-Like Methods In this section, we prove global convergence of Algorithm 1. Let us de ne the index set K f1; 2; : : :g by K = fk j k kH 0 (zk+1 )kg: (3.1) Then we have the following result. Theorem 3.1 Let fzk g be generated by Algorithm 1. If K is in nite, then lim inf kH 0 (zk )k = 0: k!1
(3.2)
Proof Let K = fk0 < k1 < k2 < g. Then by Step 5, for every k such that ki?1 < k ki , we
have k = k . So, it follows that i
kH 0 (zk +1)k ?1k = ?1 k ?1+1 12 kH 0 (zk ?1 +1)k i
i
i
i
( 21 )ikH 0 (zk0 +1)k ?1 ( 21 )i k0 ( 12 )i kH 0 (zk0 )k;
(3.3)
where the last inequality holds since by Step 5 of Algorithm 1, k kH 0 (zk )k holds for every k. This implies (3.2). 2 Theorem 3.1 reveals that to show that Algorithm 1 converges globally, it suces to verify that the index set K de ned by (3.1) is in nite. We focus our attention to Broyden-like methods. That is, in Step 4 of Algorithm 1, Bk+1 is updated by the Broyden-like formula
Bk+1 = Bk + k (yk ?ksBkks2k )sk ; T
k
(3.4)
where sk = zk+1 ? zk and yk = H k (zk+1 ) ? H k (zk ). The parameter k is chosen to satisfy jk ? 1j and Bk+1 is nonsingular [9]. We call the algorithm with (3.4) Broyden-like method. To prove global convergence of Broyden-like method, we need the following assumption. Assumption A (i) The level set de ned by (2.9) is contained in a bounded convex set D. (ii) For each > 0, function H is continuously di erentiable and rH is Lipschitz continuous on D. That is, there is a positive constant such that
krH (z) ? rH (z0 )k kz ? z0k; Let scalar k be de ned by and matrix Ak be de ned by
k = kyk ?ksBkk sk k k
Ak =
Z 0
1
rH k (zk + tsk )dt: 5
8z; z0 2 D:
(3.5) (3.6) (3.7)
Then by the mean value theorem, we have yk = Ak sk and hence = k(Ak ? Bk )sk k = k(Ak ? Bk )pk k : k
ksk k
kpk k
(3.8)
The next useful lemma follows from (2.8) and Lemma 2.6 in [23].
Lemma 3.1 Let Assumption A hold. If there is an index k such that H k (z) H k (z) for all k k, then k X 1 lim i2 = 0: k!1 k i=1
In particular, there is an in nite index set K 1 such that k ! 0 as k ! 1 with k 2 K 1 .
Now we prove global convergence of Broyden-like method. From Theorem 3.1, it suces to show that the index set K de ned by (3.1) is in nite.
Theorem 3.2 Let Assumption A hold. Assume that, for any accumulation point z of the sequence fzk g generated by Broyden-like method, rH k (z ) is nonsingular for all suciently large
k. Then the index set K de ned by (3.1) must be in nite. Moreover, we have lim kH 0 (zk )k = 0
k!1
(3.9)
and every accumulation point of fzk g is a solution of (1.1).
Proof We assume that K is nite to deduce a contradiction. By Step 5 of Algorithm 1, there is an index k such that k = for all k k with some constant > 0. Hence H k = H for all k k. Without loss of generality, we assume that k = and H k = H for all k with some function H : Rn ! Rn .
From Lemma 3.1, there is an in nite index set K 1 such that limk2K 1;k!1 k = 0. Since fzk gk2K 1 is bounded, there exists a subsequence fzk gk2K fzk gk2K 1 converging to some point z. Let Ak be de ned by (3.7). Then Ak ! rH (z ) as k ! 1 with k 2 K . Moreover, we get from (2.2) and (3.8)
kH 0 (zk )k = kBk pk k k(Bk ? Ak )pk k + kAk pk k k kpk k + kAk k kpk k: Since k ! 0 as k ! 1 with k 2 K 1 , there is a constant M1 > 0 such that for all k 2 K large enough kH 0 (zk )k M1 kpk k: (3.10) 6
Let = lim supk2K;k!1 k . If > 0, then we have limk2K;k!1 kpk k = 0 by (2.8). Hence (3.10) implies (3.9). Now consider the case = 0, that is, limk2K;k!1 k = 0. By the assumption that rH (z ) is nonsingular, it is clear that when k 2 K is suciently large, rH (zk ) and Ak are uniformly nonsingular. In particular, there exists a positive constant M2 > 0 such that krH (zk )?1 k M2 and kA?k 1 k M2 hold for all k 2 K large enough. This together with (2.2) yields
kpk k = kA?k 1 [(A k ? Bk )pk ? H 0(zk )]k kA?k 1 k k(Ak ? Bk )pk k + kH 0 (zk )k M2k kpk k + M2 kH 0(zk )k: Since k ! 0 as k ! 1 with k 2 K , the last inequality implies that there is a constant M3 > 0 such that for all k 2 K suciently large
kpk k M3kH 0 (zk )k:
(3.11)
In particular, fpk gk2K is bounded. Without loss of generality, we assume that pk ! p as k ! 1 with k 2 K . Again by (2.2) we have 0 = kBk pk + H 0 (zk )k kH 0(zk ) + Ak pk k ? k(Bk ? Ak )pk k = kH 0 (zk ) + Ak pk k ? k kpk k: Letting k 2 K go to in nity yields
rH (z)T p + H 0(z ) = 0:
(3.12)
On the other hand, the assumption that limk2K;k!1 k = 0 means that k < 1 for all k 2 K suciently large. By the line search rule, it means that when k 2 K is suciently large, 0k k = does not satisfy (2.4). In other words, we have
kH (zk + 0k pk )k ? kH (zk )k > ?1 kH (zk + 0k pk ) ? H (zk )k ? 2 k 0 pk k2 + k ?1 kH (zk + 0k pk ) ? H (zk )k ? 2 k 0 pk k2 : Multiplying the both sides by kH (zk + 0k pk )k + kH (zk )k yields
kH (zk + 0k pk )k2 ? kH (zk )k2 ?1 kH (zk + 0k pk )k + kH (zk )k kH (zk + 0k pk ) ? H (zk )k ? 2 k 0k pk k2 kH (zk + 0k pk )k + kH (zk )k : 7
Dividing the both sides by 0k and then taking the limit as k ! 1 with k 2 K yield 2H (z )T rH (z )T p ?21 kH (z )k krH (z )pk: This together with (3.12) implies
H (z )T H 0 (z) 1kH (z)k kH 0 (z )k:
(3.13)
Since kH 0 (zk )k for all k, we have kH 0 (z )k. It then follows from (2.10) that
kH (z ) ? H 0 (z)k = kH (z) ? H 0(z )k kH 0(z )k; and hence
(3.14)
kH (z )k kH 0 (z)k + kH (z) ? H 0(z )k (1 + )kH 0 (z)k:
Consequently, we get from the last inequality and (3.13)
1(1 + )kH 0 (z )k2 1kH (z)k kH 0 (z )k H (z)T H 0(z ) = kH 0 (z )k2 + (H (z ) ? H 0 (z ))T H 0 (z ) kH 0 (z)k2 ? kH (z ) ? H 0(z )k kH 0 (z )k (1 ? )kH 0 (z)k2 ; 0 z ) = 0, which where the last inequality follows from (3.14). However, since 1 < 11+? , we get H ( contradicts the fact that 0 < kH (z )k. The contradiction shows that K must be in nite. Therefore by Theorem 3.1, we have lim inf k!1 kH 0 (zk )k = 0. Now we show that (3.9) holds. Indeed, from (2.10), we get
kH k (zk )k kH 0 (zk )k + k (1 + )kH 0 (zk )k: This means that there is a subsequence of fH k (zk )g converging to zero. However, Lemma 2.1 shows that fkH k (zk )kg itself converges and hence it converges to zero. Again by (2.10) we get
kH 0 (zk )k kH k (zk )k + k kH k (zk )k + kH 0 (zk )k; which implies that
kH 0 (zk )k 1 ?1 kH k (zk )k:
This shows that (3.9) holds. Consequently, every accumulation point of fzk g is a solution of (1.1). 2 Remark (i) In Theorem 3.2, the assumption that rH k (z) is nonsingular for every accumulation point z can be replaced by the weaker assumption that there is an accumulation point z of the 8
subsequence fzk gk2K 1 such that rH k (z ) is nonsingular for all k large enough, where K 1 is the index set as speci ed in Lemma 3.1. (ii) Since by Step 5 in Broyden-like method, k kH (zk )k holds for every k and fk g is monotonically nonincreasing, Theorem 3.2 shows that lim k = 0:
k!1
(3.15)
4 Superlinear Convergence In this section, we prove superlinear convergence of Broyden-like method. To this end, we need further assumptions. Assumption B (i) The sequence fzk g generated by Broyden-like method converges to a solution z of (1.1) at which every element in @C H (z) is nonsingular, where @C H (z) = @H1(z) @H2 (z) @Hn(z), and @h(z) denotes the generalized Jacobian of function h : Rn ! R at z in the sense of Clarke [6]. (ii) Functions H k and H satisfy lim dist (rH k (zk ); @C H (z )) = 0:
k!1
(4.1)
(iii) There exist a neighbourhood U (z ) of z and a constant > 0 such that for all k suciently large krH k (z) ? rH k?1(z)k (k?1 ? k ); 8z 2 U (z ) (4.2) and krH k (zk + tsk ) ? rH k (zk?1 + tsk?1)k (ksk k + ksk?1k); 8t 2 (0; 1): (4.3) (iv) As k goes to in nity, we have
kH k (z ) ? H 0(z )k = o(kzk ? zk); kH k (zk ) ? H 0(zk )k = o(kzk ? zk):
(4.4)
Remark (i) Since @C H (z) is compact and @C H () is upper semicontinuous, it is not dicult to
see that conditions (i) and (ii) of Assumptions B imply that there is a neighbourhood U (z ) of z, a constant 0 > 0 and an index k such that inequality krH k (z )k 0 holds for all k k and any z 2 U (z ). Moreover, there are constants 0 < c C such that
ckz ? z0k kH k (z) ? H k (z0 )k C kz ? z0 k; 8k k; 8z; z0 2 U (z ): (ii) Inequality (4.3) holds particularly when H k satis es
krH k (z) ? rH k (z0)k kz ? z0 k; 8z; z0 2 U (z): 9
(4.5)
Lemma 4.1 Let Assumptions A and B hold and k be de ned by (3.6). Then we have lim k = 0:
k!1
(4.6)
Proof It follows from (2.7) in Lemma 2.1 and (4.5) that 1 X
k=1
ksk k =
1 X
k=1
kzk+1 ? zk k < 1:
(4.7)
Let Ak be de ned by (3.7). Then we have yk = Ak sk . Moreover, it follows from (3.4) that
T 2 kBk+1 ? Ak k2F =
Bk ? Ak + k (Ak ? Bk ) kssk skk2
F k 2 k ( = kBk ? Ak k2F + (k2 ? 2k ) Bk k?s Akk2)sk k k 2 k y ? B k = kBk ? Ak k2F + (k2 ? 2k ) ks kk2sk k k 2 2 2 = kBk ? Ak kF + (k ? 2k )k kBk ? Ak k2F ? (1 ? 2)k2 ; (4.8) where the last inequality holds since k is chosen to satisfy j1 ? k j in Step 4. Then we deduce
kAk ? Ak?1 k
Z
0
Z
1
krH k (zk + tsk ) ? rH k?1(zk?1 + tsk?1)kdt
1
krH k (zk + tsk ) ? rH k (zk?1 + tsk?1)kdt
0
Z
1
krH k (zk?1 + tsk?1) ? rH k?1(zk?1 + tsk?1)kdt ksk k + ksk?1k + (k?1 ? k ); +
0
where the last inequality follows from (4.2) and (4.3). This together with (4.7) implies 1 X
k=2
kAk ? Ak?1k < 1:
(4.9)
Therefore, we get from (4.8)
kBk+1 ? Ak kF kBk ? Ak kF kBk ? Ak?1kF + kAk ? Ak?1kF : This along with (4.9) implies that fkBk+1 ? Ak kF g converges. Thus we deduce from (4.8) again that (4.6) holds. 2 Lemma 4.2 Let z be a solution of (1.1). Then under Assumptions A and B, we have kzk + pk ? zk = 0: (4.10) lim k!1 kzk ? zk 10
Proof Let Ak be de ned by (3.7). It then follows from (4.3) that Ak ?rH k (zk ) ! 0. Moreover,
since (4.6) holds, using the arguments for showing (3.11) in the proof of Theorem 3.2, we can deduce that there is a positive constant M3 such that
kpk k M3 kH 0(zk )k LM3kzk ? zk;
(4.11)
where L > 0 is the Lipschitz constant of H 0 . We then get from (2.2)
kzk + pk ? zk = krH k (zk )?1 (rH k (zk )(zk ? z) + rH k (zk )pk ? Bk pk ? H 0 (zk ))k krH k (zk )?1 k k(rH k (zk ) ? Bk )pk k + krH k (zk )(zk ? z) + H 0 (z) ? H 0(zk )k 0 k(Ak ? Bk )pk k + k(rH k (zk ) ? Ak )pk k + krH k (zk )(zk ? z) + H 0 (z) ? H 0(zk )k 0 k kpk k + krH k (zk ) ? Ak k kpk k + krH k (zk )(zk ? z) + H k (z ) ? H k (zk )k + k[H 0 (z ) ? H 0 (zk )] ? [H k (z ) ? H k (zk )]k LM3 0 k + krH k (zk ) ? Ak k kzk ? zk + o(kzk ? zk) = o(kzk ? zk); where the third inequality follows from (3.6) and yk = Ak sk , the last inequality follows from (4.4) and (4.11), and the last equality follows from (4.6). 2 Now, we establish superlinear convergence of Broyden-like method. By Lemma 4.2, it suces to show that when k is suciently large, the unit steplength is always accepted.
Theorem 4.1 Under Assumptions A and B, the sequence fzk g generated by Broyden-like method converges superlinearly.
Proof By Lemma 4.2, it suces to show that when k is suciently large, = 1 satis es (2.4).
Indeed, it follows from (4.4), (4.5) and (4.10) that for all k large enough
kH k (zk + pk )k = kH k (zk + pk ) ? H 0(z )k kH k (zk + pk ) ? H k (z )k + kH k (z ) ? H 0 (z)k C kzk + pk ? zk + o(kzk ? zk) = o(kzk ? zk): On the other hand, we have
kH k (zk )k ? 1kH k (zk + pk ) ? H k (zk )k ? 2 kpk k2 = kH k (zk ) ? H 0 (z )k ? 1 kH k (zk + pk ) ? H k (zk )k + o(kzk ? zk) 11
(4.12)
kH k (zk ) ? H k (z )k ? kH k (z ) ? H (z )k ? kH k (zk + pk ) ? H k (z )k + kH k (z ) ? H k (zk )k + o(kzk ? zk) = (1 ? )kH k (zk ) ? H k (z )k + o(kzk ? zk) (1 ? )ckzk ? zk + o(kzk ? zk); 0
1
1
1
(4.13)
where the rst equality follows from (4.11), the last equality follows from (4.4), (4.5) and (4.10), and the last inequality follows from (4.5). Comparing (4.12) with (4.13), it is clear that when k is suciently large
kH k (zk + pk )k kH k (zk )k ? kH k (zk + pk ) ? H k (zk )k ? kpk k kH k (zk )k ? kH k (zk + pk ) ? H k (zk )k ? kpk k + k : 1
2
1
2
2
2
This means that when k is suciently large, = 1 is always accepted. 2 Theorem 4.1 reveals that to ensure superlinear convergence of Broyden-like method, the strict complementarity at z is not necessary.
5 Applications In this section, we apply the general smoothing quasi-Newton method proposed in Section 2 to nonlinear complementarity problems, mixed complementarity problems and variational inequality problems. We show that for some well known smoothing functions, the conditions in Assumptions A and B hold, and hence the related algorithms converge globally and superlinearly.
5.1 Nonlinear complementarity problems Let f : Rn ! Rn be continuously di erentiable. Consider the nonlinear complementarity problem of nding a z 2 Rn such that
z 0; f (z ) 0; zT f (z ) = 0:
(5.1)
There have been developed various reformulations of (5.1) and related smoothing functions. Fischer-Burmeister function as well as its perturbed version and Chen-Harker-Kanzow-Smale smoothing function are widely used [7, 10, 11, 12, 16, 18, 19, 34]. Fischer-Burmeister function FB : R2 ! R is de ned by p FB (a; b) = a2 + b2 ? (a + b): (5.2) It is easy to see that FB (a; b) = 0 holds if and only if a 0, b 0 and ab = 0. Fischer-Burmeister function is di erentiable at every point except the origin and is semismooth at the origin. By 12
introducing a perturbation parameter, we get a smoothing Fischer-Burmeister function
p
FB (a; b) = a2 + b2 + 2 ? (a + b);
(5.3)
where is a nonnegative parameter. It is clear that 0FB (a; b) FB (a; b). Let H (z ) = (H1 (z ); H2 (z ); : : : ; Hn (z ))T , where
Hi(z ) = FB (zi ; fi (z )); i = 1; 2; : : : ; n:
(5.4)
Then H is continuously di erentiable everywhere if f is continuously di erentiable and > 0. Moreover, (5.1) is equivalent to the semismooth equation H 0 (z ) = 0. Let zi ? 1; bi (z ) = q fi (z ) ? 1: ai(z ) = q zi2 + fi(z )2 + 2 zi2 + fi(z )2 + 2 Then it is obvious that for every > 0 and any x, we have
?2 < ai(z) < 0; ?2 < bi(z) < 0:
(5.5)
rH (z) = diag (ai(z)) + rf (z) diag (bi(z)):
(5.6)
Moreover, it is easy to deduce The following lemma gives some properties of FB and H .
Lemma 5.1 Let FB and H be de ned by (5.3) and (5.4), respectively. Suppose that f is
continuously di erentiable. Then the following statements hold. (i) For any > 0, functions FB and H are continuously di erentiable everywhere. (ii) For any > 0 > 0, we have 0 j FB (a; b) ? FB (a; b)j ? 0 ; 8(a; b) 2 R
and hence
2
kH (z) ? H 0 (z)k pn( ? 0 ); 8z 2 Rn:
(iii) For a given constant > 0, the inequality
j FB (a; b) ? FB (a; b)j 21 ? 0
1 2
holds for all (a; b) 2 R2 satisfying a2 + b2 2 . Consequently, the inequality kH (z) ? H 0 (z)k 12 pn?1 2 holds for all z 2 Rn satisfying zi2 + fi (z )2 2 , i = 1; 2; : : : ; n. (iv) If f is a P0 function, then rH (z ) is nonsingular for every > 0 and any z 2 Rn.
13
(5.7)
Proof The statements (i), (ii) and (iv) have been proved in [34]. We prove (iii). Let a + b . 2
Then, we get
2
p
p
j FB (a; b) ? FB (a; b)j =
2
a2 + b2 + 2 ? a2 + b2 2 = p 2 2 2 p 2 2 a +b + + a +b 1 2 ?1 2:
0
2
Now we make the following assumption. Assumption A0 (i) The level set de ned by (2.9) is contained in a bounded convex set D. (ii) Function f is continuously di erentiable and rf is Lipschitz continuous on D.
Lemma 5.2 Let Assumption A0 hold. Then for any > 0, there is a constant > 0 such that 1
krH (z) ? rH (z0 )k kz ? z0k;
8z; z0 2 D:
1
(5.8)
Proof It follows from (5.5) and (5.6) that krH (z) ? rH (z0 )k
=
diag (ai (z ) ? ai (z 0 )) + (rf (z ) ? rf (z 0 )) diag (bi (z )) + rf (z 0 ) diag (bi (z ) ? bi (z ))
k diag (ai(z) ? ai(z0 ))k + 2krf (z) ? rf (z0)k + krf (z0 )k k diag (bi(z) ? bi(z))k: It then suces to show that there is a constant 1 > 0 such that inequalities
jai(z) ? ai(z0 )j kz ? z0 k and jbi(z) ? bi(z0 )j kz ? z0 k 1
(5.9)
1
hold for any z; z 0 2 D. By the de nition of ai, we have
0 zi jai(z) ? ai(z0 )j = q ?q 0 z zi + fi (z ) + (zi ) + fi (z 0 ) + 0 0 1 ?q 0 1 0 q zi ? zi + jzi j q (zi ) + fi (z ) + zi + fi (z ) + zi + fi(z ) + q q ? zi + fi (z ) + ? (zi0 ) + fi(z 0 ) + ? jzi ? zi0 j + M 0 0 ? q jzi ? (zi ) + fiq(z ) ? fi (z ) j = ? jz ? z 0 j + M 1
1
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
zi2 + fi(z )2 + 2 + (zi0 )2 + fi (z 0 )2 + 2 ?3 (jzi2 ? (zi0 )2 j + jfi (z )2 ? fi (z 0 )2 j); ?1jzi ? zi0 j + 2?1 M 14
2
(5.10)
where M is a constant such that kz k M for all z 2 D. The last inequality implies the rst inequality of (5.9) with a suitable constant 1 > 0. The second inequality of (5.9) can be proved in a similar way. 2 Lemmas 5.1 and 5.2 imply the following global convergence theorem.
Theorem 5.1 Let Assumption A0 hold and H be de ned by (5.4). Suppose that f is a P function. Then the sequence fxk g generated by the related Broyden-like method satis es (3.9).
0
We say that z is a strictly complementary solution of (5.1) if it is a solution of (5.1) and satis es zi + fi(z ) > 0; i = 1; 2; : : : ; n: (5.11) We make another assumption as follows. Assumption B0 (i) The sequence fzk g generated by Broyden-like method converges to a solution z of (1.1) that is a strictly complementary solution of (5.1). (ii) rH 0 (z ) is nonsingular. Remark The strict complementarity (5.11) implies that H is di erentiable near z. Hence, rH 0(z ) exists. Moreover, @C H 0(z ) reduces to the singleton frH 0(z )g. Therefore, Assumption B0 implies condition (i) in Assumption B. The following lemma shows that Assumptions A0 and B0 imply Assumption B.
Lemma 5.3 Let FB and H be de ned by (5.3) and (5.4), respectively. If Assumptions A0 and
B0 hold, then Assumption B holds.
Proof The inequality (5.11) implies that there is a neighbourhood U (z) of z such that the 4 > 0 holds for any z 2 U (z ). It is then clear that H is inequality zi + fi (z ) (zi + fi (z )) = 2
2
1 2
2
2
2
0
di erentiable in U (z ) and hence equality (4.1) holds. Inequality (4.3) can be obtained in a way similar to Lemma 5.2. Inequality (4.4) follows from Lemma 5.1 (iii) and the fact k kH k (zk )k. It remains to show (4.2). By (5.6), it suces to show that there exists a constant > 0 such that the following inequalities hold for any > 0 and all z 2 U (z ): jai(z) ? ai 0 (z)j ( ? 0); jbi(z) ? bi 0 (z)j ( ? 0): (5.12) Indeed, by the de nition of ai , we have jai(z) ? ai 0 (z)j = jzij q 2 1 ?q 2 1 zi + fi(z )2 + 2 zi + fi (z )2 + (0)2 q q ?1jzi j zi2 + fi(z)2 + 2 ? zi2 + fi(z)2 + (0)2 2 ? q (0 )2 = ?1jzi j q zi2 + fi(z )2 + 2 + zi2 + fi (z )2 + (0)2 ?1jzi j( ? 0): 15
This shows the rst inequality of (5.12). The second inequality of (5.12) can be proved in a similar way. 2 In view of Lemmas 5.1, 5.2 and 5.3, and Theorem 4.1, we get the following superlinear convergence theorem.
Theorem 5.2 Let Assumptions A0 and B0 hold and H be de ned by (5.4). Then the sequence fzk g generated by the related Broyden-like method converges superlinearly. Chen-Harker-Kanzow-Smale smoothing function is de ned by
CHKS (a; b) =
p
a + b ? (a ? b)2 + 2 :
(5.13) 2 It is a smoothing function of 0CHKS (a; b) = minfa; bg. Functions CHKS and 0CHKS enjoy some important properties that FB and 0FB have. Let
H (z ) = ( CHKS (z1 ; f1(z )); : : : ; CHKS (zn ; fn(z )))T :
(5.14)
It is not dicult to prove that Lemmas 5.1, 5.2 and 5.3 remain true. It then follows that Theorems 5.1 and 5.2 hold even if FB is replaced by CHKS . We omit the detail.
5.2 KKT system of variational inequality problems Let F : Rn ! Rn be continuously di erentiable and X be a closed convex subset of Rn . The variational inequality problem VI(X; F ) is to nd an x 2 Rn such that
F (x)T (y ? x) 0;
8y 2 X:
(5.15)
In many cases, X is speci ed as
X = fx 2 Rn j gi (x) 0; i = 1; 2; : : : ; m1 ; hj (x) = 0; j = 1; 2; : : : ; m2 g; where gi : Rn ! R and hj : Rn ! R are twice continuously di erentiable functions. It has been shown that under a suitable constraint quali cation, a solution of (5.15) together with some vectors 2 Rm1 and 2 Rm2 is a solution of the following KKT system:
0 BB L(z) @ min(; g(x)) h(x)
1 CC A = 0;
(5.16)
where z = (x; ; ) 2 Rn+m1 +m2 ,
L(z ) = F (x) ? rg(x) ? rh(x); 16
(5.17)
and
min(; g(x)) = (min(1 ; g1 (x)); min(2 ; g2 (x)); : : : ; min(m1 ; gm1 (x)))T : By using Fischer-Burmeister function, (5.16) can be reformulated as
0 BB L(z) H (z ) = @ FB (; g(x)) h(x)
1 CC A = 0;
(5.18)
where FB (; g(x)) = ( FB (1 ; g1 (x)); : : : ; FB (m1 ; gm1 (x)))T and FB is de ned by (5.2). By using the perturbed Fischer-Burmeister function, we get a smoothing function to H
0 L(z ) B H (z ) = B @ FB (; g(x)) h(x)
1 CC = 0; A
(5.19)
where FB (; g(x)) = ( FB (1 ; g1 (x)); : : : ; FB (m1 ; gm1 (x)))T and FB is de ned by (5.3). Put i ? 1; bi (x; ) = q gi (x) ? 1: ai(x; ) = q 2i + gi (x)2 + 2 2i + gi(x)2 + 2 It is easy to show the following lemma by using the chain rule of derivatives for composite functions.
Lemma 5.4 Let F be continuously di erentiable and gi , i = 1; 2; : : : ; m , and hj , j = 1; 2; : : : ; m , 1
2
be twice continuously di erentiable. Then and H are continuously di erentiable for every 6= 0. Moreover, for every z = (x; ; ) 2 Rn+m1 +m2 and any 6= 0, we have the following expressions:
rx(x; ) = rg(x) diag (bi(x; )); and
r(x; ) = diag (ai(x; ));
1 0 ( x; )) ?r h ( x ) r L ( z ) ?r g ( x ) diag ( b i C B x T (x; )) rH (z) = B ?r g ( x ) ? diag ( a 0 C i A: @ T ?rh(x) 0 0
(5.20)
In the remainder of this section, we abbreviate ai (x; ) and bi (x; ) as ai and bi , respectively. It is clear that diag(ai ) and diag(bi) are nonsingular. Moreover, ai and bi are negative. This implies that for any (x; ) 2 Rn+m1 , the diagonal matrix diag ((ai )?1bi ) is positive de nite, and hence, the matrix rg(x)diag((ai )?1bi )rg(x)T is symmetric and positive semide nite for any (x; ) 2 Rn+m1 . The following lemma shows some conditions that ensure regularity of H . 17
Lemma 5.5 Let z = (x; ; ) 2 Rn
6= 0. If rhj (x); j = 1; 2; : : : ; m2 ; are linearly
m1 +m2 and
+
independent and the matrix
rxL(z) + rg(x)diag((ai)? bi)rg(x)T 1
is positive de nite on the subspace fp j rh(x)T p = 0g, then rH (z ) is nonsingular. In particular, if rx L(z ) is positive de nite, then rH (z ) is nonsingular.
Proof Let w = (p; q; r) 2 Rn
m1 +m2
+
0, i.e.,
be a solution of the system of linear equations rH (z )w =
10 1 0 bi ) ?rh(x) r L ( z ) ?r g ( x ) diag ( x CC BB p CC BB T ? diag (ai ) 0 A @ q A = 0: @ ?rg(x) T r ?rh(x) 0 0:
We will show that w = 0. Writing the above equations blockwise, we get 8 > < rxL(zT)p ? rg(x) diag (bi)q ? rh(x)r = 0; = 0; ?rg(x) p ? diag (ai )q > : ?rh(x)T p = 0; which implies
(5.21)
8 < rxL(z) + rg(x)diag((ai)? bi)rg(x)T p ? rh(x)r = 0; : rh(x)T p = 0: 1
Multiplying the rst equation by p and taking into account the second equation yield
pT rxL(z ) + rg(x)diag((ai )?1bi )rg(x)T p = 0:
(5.22)
However, since p lies in the subspace fp j rh(x)T p = 0g, it follows from the positive de niteness of rx L(z ) + rg(x)diag((ai )?1bi )rg(x)T on the subspace that p = 0. Then, the second equation in (5.21) implies that q = 0. Finally, since rhj (x), j = 1; 2; : : : ; m2 , are linearly independent, it follows from the rst equation of (5.21) that r = 0. Consequently, zero is the unique solution of (5.21), or equivalently, rH (z ) is nonsingular. The last part of the lemma is a direct consequence 2 of the rst part by the positive semide niteness of rg(x) diag ((ai )?1bi )rg(x)T . Similar to Assumption A0 , we make the following assumption. Assumption A00 (i) The level set de ned by (2.9) is contained in a bounded convex set D. (ii) The function F is continuously di erentiable on Dx , and gi , i = 1; 2; : : : ; m1 , and hj , j = 1; 2; : : : ; m2 , are twice continuously di erentiable on Dx, where Dx = fx 2 Rn j (x; ; ) 2 Dg. 18
(iii) rF , r2 g and r2 h are Lipschitz continuous on Dx , i.e., there is a constant > 0 such that 8 > < kr2F (x) ? rF2 (y)k kx ? yk; 8x; y 2 Dx kr g(x) ? r g(y)k kx ? yk; 8x; y 2 Dx > : kr2h(x) ? r2h(y)k kx ? yk; 8x; y 2 Dx: In a way similar to Lemma 5.2, we can prove that under Assumption A00 , rH is Lipschitz continuous on D. To be speci c, we have the following lemma. Lemma 5.6 Let Assumption A00 hold. Then for any > 0, there is a constant 2 > 0 such that
krH (z) ? rH (z0 )k kz ? z0k; 2
8z; z0 2 D:
(5.23)
Similar to Lemma 5.3, we can show that the following assumption implies Assumption B. Assumption B00 (i) The sequence fzk g generated by Algorithm 1 converges to a solution of (5.16), say z = (x; ; ), at which the strict complementarity condition holds, that is, i + gi(x) > 0; i = 1; 2; : : : ; m1 : (ii) rH 0 (z ) is nonsingular. Therefore, we have the following global and superlinear convergence theorem.
Theorem 5.3 Let Assumption A00 hold and the sequence fzk g be generated by Broyden-like method with H de ned by (5.19). Suppose that, for any accumulation point z of fzk g, matrices rH k (z ) are nonsingular for all suciently large k. Then we have lim kH (zk )k = 0 k!1 and every accumulation point of fzk g is a solution of (5.16). If in addition, Assumption B00 holds, then fzk g converges superlinearly. Remark (i) A condition that ensures nonsingularity of rH k (z ) has been given in Lemma 0
5.5.
(ii) A convergence theorem similar to Theorem 5.3 can be established even if FB is replaced by Chen-Harker-Kanzow-Smale smoothing function.
5.3 Mixed complementarity problems Let X = ni=1 [li ; ui ], where ?1 li < ui 1 for each i. Then problem (5.15) is called the mixed complementarity problem. The mixed complementarity problem is equivalent to nding a z 2 Rn such that 8 > if fi(z ) > 0; < zi = li; zi = ui ; if fi(z ) < 0; i = 1; 2; : : : ; n; > : li zi ui; if fi(z) = 0; 19
which can be further reformulated as mid fzi ? li ; zi ? ui ; fi (z )g = 0; i = 1; 2; : : : ; n;
(5.24)
where mid fa; b; cg denotes the median of a; b; c. Gabriel and More [15] introduced a class of smoothing functions for mixed complementarity problems, which is an extension of the class of smoothing functions introduced by Chen and Mangasarian [2]. We show that under suitable conditions, Gabriel-More smoothing function associated with the semismooth equation (5.24) satis es Assumptions A and B. Let : R ! R+ be a continuous density function with a bounded absolute mean, that is, 4 =
Z1
?1
jsj(s)ds < 1:
Then the Gabriel-More smoothing function is de ned by H (z ) = (H1 (z ); : : : ; Hn (z ))T , where
Hi (z ) =
Z1
?1
mid fzi ? li ; zi ? ui ; fi (z ) ? 2 sg(s)ds; i = 1; 2; : : : ; n:
(5.25)
Note that the function H given by (5.25) is slightly di erent from the original Gabriel-More smoothing function where 2 is replaced by . Gabriel-More smoothing function contains as a special case Chen-Harker-Kanzow-Smale smoothing function for (5.24), which corresponds to the choice (s) = (s2 +11)3=2 : For any > 0, H is continuously di erentiable, as long as f is continuously di erentiable. Moreover, the mixed complementarity problem is equivalent to the semismooth equation H 0 (z ) = 0. Function H possesses many good properties. Here we mention some properties that are particularly useful for the related smoothing quasi-Newton method. First, it has been proved in [5] that the density function satis es the inequality 2 (5.26) (s) < 1 for all s such that jsj is suciently large. The following lemma comes from [5] and [15].
s
Lemma 5.7 Let H be de ned by (5.25). Suppose that f is continuously di erentiable. Then
the following statements are true. (i) For any > 0 0 and all z 2 Rn , we have
jHi(z) ? Hi0 (z)j ( ? (0) ): 2
20
2
(5.27)
(ii) For every > 0, we have
rHi(z) = ei ?
Z
z ?fi (z)?li )=2
( i
z ?fi (z)?ui )=2
( i
(s)ds (ei ? rfi (z )); i = 1; 2; : : : ; n;
(5.28)
where ei is the i-th column of the n n identity matrix.
The following lemma gives a sucient condition for Lipschitz continuity of rH .
Lemma 5.8 Let H be de ned by (5.25). Suppose that f is continuously di erentiable and rf is Lipschitz continuous. Then for each > 0, there is a constant > 0 such that for any z; z 0 2 Rn krH (z) ? rH (z0 )k kz ? z0k: (5.29) Proof By using (5.28), it suces to verify that there is a constant > 0 such that for each 3
3
i = 1; 2; : : : ; n
3
Z z0 ?f z0 ?l =2 Z z ?f z ?l =2 z ?f z ?u =2 (s)ds ? z0 ?f z0 ?u =2 (s)ds kz ? z0k: ( i
( i
i(
i(
)
)
i(
( i
i)
( i
i)
i(
)
i)
)
2
i)
(5.30)
It is clear that is bounded. Let = sup (s). By direct deduction, we have
Z z0 ?f z0 ?l =2 Z z ?f z ?l =2 z ?f z ?u =2 (s)ds ? z0 ?f z0 ?u =2 (s)ds Z z ?f z ?u =2 Z z0 ?f z0 ?l =2 ( s ) ds ( s ) ds ? = z0 ?f z0 ?u =2 z ?f z ?l =2 Z z ?f z ?u =2 Z z0 ?f z0 ?l =2 ( s ) ds + ( s ) ds z0 ?f z0 ?u =2 z ?f z ?l =2 2? (jzi0 ? zij + jfi(z0 ) ? fi(z)j): ( i
( i
i(
i(
)
( i
i)
)
( i
i)
( i
( i
i(
i(
( i
( i
)
i(
i(
)
)
i(
)
i)
)
i)
( i
i)
i(
i(
( i
i)
)
i(
( i
i)
( i
i)
)
i)
)
i(
i(
i)
)
)
i)
i)
2
This together with Lipschitz continuity of f implies (5.30) with some constant 3 > 0 as desired.
2
Lemmas 5.7 and 5.8 show that Assumption A0 implies Assumption A. We turn to nding a sucient condition for Assumption B to hold. We say z is a strictly complementary solution of MCP if it is a solution of MCP and satis es
zi ? li + fi(z ) > 0; zi ? ui + fi(z ) < 0; i = 1; 2; : : : ; n: Let
= 12 1min fjz ? l + f (z )j; jzi ? ui + fi(z )jg: in i i i
If (5.31) holds, then > 0. Let U (z ) be a neighbourhood of z such that
U (z) fz j jzi ? li + fi(z )j ; jzi ? ui + fi(z )j g: 21
(5.31)
Then H 0 is continuously di erentiable on U (z ). Moreover, it has been shown in [5] that for any z 2 U (z ) ( (5.32) rHi0(z) = rfi(z); if zi ? fi(z) 2 (li; ui ); ei ; if zi ? fi(z ) 62 [li ; ui ]: The following lemma gives sucient conditions for Assumption B to hold.
Lemma 5.9 Let Assumption A0 and condition (i) in Assumption B hold. Suppose that z is a
strictly complementary solution of MCP. Then Assumption B holds.
Proof The semismoothness of H follows from the continuous di erentiability of f . In particular, rH (z ) exists and is given by (5.32) with z = z. If zi ? fi(z ) 2 (li; ui ), then we get from (5.28) 0
0
and (5.32)
rHik (zk ) ? rHi0(z )
Z
1?
=
(s)ds (ei ? rfi(zk )) + rfi(zk ) ? rfi(z )
z ?f (zk ))i ?ui ]=2k
Z 1
=
z ?f (zk ))i ?li ]=2k
[( k
[( k
(s)ds + 2
Z
z ?f (zk ))i ?li ]=k + rfi (zk ) ? rfi (z ): [( k
z ?f (zk ))i ?ui ]=2k
[( k
?1
(s)ds (ei ? rfi(zk ))
This implies rHik (zk ) ? rHi0 (z ) ! 0. If zi ? fi (z ) 62 [li ; ui ], we also get from (5.28) and (5.32)
rHik (zk ) ? rHi0(z ) = ?
Z
z ?f (zk ))i ?li ]=2k
[( k
(s)ds (ei ? rfi (zk )):
z ?f (zk ))i ?ui ]=2k
[( k
This also implies rHik (zk ) ? rHi0 (z ) ! 0. Therefore, we have shown (4.1). Since k ! 0, we have for all k suciently large and any z 2 U (z )
Z z ?f z ?l =2?1 Z z ?f z ?l =2 z ?f z ?u =2 (s)ds ? z ?f z ?u =2 (s)ds ?1 Z z ?f z ?l =2 Z z ?f z ?u =2?1 ( s ) ds + ( s ) ds z ?f z ?l =2 ?1 z ?f z ?u =2 Z z ?f z ?l =2 1 Z z ?f z ?u =2?1 1 ds + ds z ?f z ?u =2 s z ?f z ?l =2 ?1 s ? ? = jz ?k?f (z ) ?k l j + jz ?k?f (z ) ?k u j i i i i i i 2? (k? ? k ); ( i
i(
( i
i(
)
)
i)
i)
( i
( i
i(
( i
2
1
i(
)
i)
i)
)
i(
i)
)
i)
k
i)
k
( i
i(
( i
k
i(
( i
k
2
i)
)
k
( i
k
2
2
1
2
)
)
i(
i(
( i
k
i(
( i
( i
k
1
i(
i(
)
)
i)
)
)
i)
k
i)
i)
k
k
2
k
2
2
1
where the second inequality follows from (5.26). This together with (5.28) implies (4.2). Similarly, we have for all k suciently large and any z; z 0 2 U (z )
Z z0 ?f z0 ?l =2 Z z ?f z ?l =2 z ?f z ?u =2 (s)ds ? z0 ?f z0 ?u =2 (s)ds ( i
( i
i(
i(
)
)
i)
i)
k
k
( i
( i
i(
i(
)
)
i)
i)
k
k
22
=
Z z0 ?f z0 ?l =2 Z z ?f z ?u =2 z ?f z ?l =2 (s)ds + z0 ?f z0 ?u =2 (s)ds Z z0 ?f z0 ?l =2 1 Z z ?f z ?u =2 1 z ?f z ?l =2 s ds + z0 ?f z0 ?u =2 s ds k z ? f 1(z ) ? l ? z0 ? f (1z 0 ) ? l + k z0 ? f (1z 0 ) ? u ? z ? f (1z ) ? u i i i i i i i i i i i i 2k ? (jzi ? zi0 j + jfi (z ) ? fi (z 0 )j): i(
( i
( i
i(
)
i(
( i
( i
i(
)
)
i)
i)
)
( i
k
2
k
i(
i(
( i
k
i)
i)
( i
k
( i
i(
i(
)
i)
)
)
)
k
i)
i)
i)
k
k
2
k
2
2
2
2
This together with (5.28) yields (4.3). Equations (4.4) follow from (5.27) and the fact k kH 0 (zk )k immediately. The proof is then complete. 2 From Lemma 5.9, we see that the following assumption together with Assumption A0 ensures the conditions of Assumption B. Assumption B000 (i) The sequence fzk g generated by Algorithm 1 converges to a strictly complementary solution z of the mixed complementarity problem. (ii) rH 0 (z ) is nonsingular. We now state a global and superlinear convergence theorem for Broyden-like method for MCP.
Theorem 5.4 Let Assumption A0 hold and the sequence fzk g be generated by Broyden-like method with H de ned by (5.25). Suppose that, for any accumulation point z of fzk g, matrices rH k (z ) are nonsingular for all suciently large k. Then we have lim kH 0 (zk )k = 0
k!1
and every accumulation point of fzk g is a solution of (5.16). If in addition, Assumption B000 holds, then fzk g converges superlinearly.
6 Discussion In this paper, we have presented a general Broyden-like method for solving semismooth equations including semismooth reformulations of nonlinear complementarity problems, variational inequality problems and mixed complementarity problems. We have established global and superlinear convergence theorems under suitable conditions. For nonlinear complementarity problems, global convergence only requires that F is a P0 function with Lipschitz continuous Jacobian. On the other hand, for variational inequality problems, global convergence needs the assumption that rH k (z ) is nonsingular for all k suciently large, as in the global convergence theorems established in [26], [30] and [35]. Although this condition might appear somewhat arti cial, it is implied, as shown in Lemma 5.5, by a condition that is reminiscent of the second order sucient condition in 23
nonlinear programming. Conditions that guarantee nonsingularity of rH for some smoothing functions related to the mixed complementarity problem can be found in [32]. Theorem 4.1 is an important contribution of the paper. It shows that superlinear convergence of a smoothing Broyden-like method can be established without assuming the strict complementarity condition at the limit point. This provides the possibility of developing a globally convergent Broyden-like method that enjoys superlinear convergence even if strict complementarity fails to hold.
References [1] B. Chen and P.T. Harker, Smooth approximations to nonlinear complementarity problems, SIAM J. Optim., 7 (1997), 403-420. [2] C. Chen and O.L. Mangasarian, A class of smoothing functions for nonlinear and mixed complementarity problems, Comput. Optim. Appl., 5 (1996), 97-138. [3] X. Chen, Superlinear convergence of smoothing quasi-Newton methods for nonsmooth equations, J. Comp. Appl. Math., 80 (1997), 105-126. [4] X. Chen and L. Qi, A parameterized Newton method and a quasi-Newton method for nonsmooth equations, Comput. Optim. Appl., 3 (1994), 157{179. [5] X. Chen, L. Qi and D. Sun, Global and superlinear convergence of the smoothing Newton method and its application to general box constrained variational inequalities, Math. Comput., 67 (1998), 519-540. [6] F.H. Clarke, Optimization and Nonsmooth Analysis, Wiley New York, 1983. [7] T. De Luca, F. Facchinei and C. Kanzow, A semismooth equation approach to the solution of nonlinear complementarity problems, Math. Programming, 75 (1996), 407-440. [8] J.E. Dennis, Jr. and J.J. More, A characterization of superlinear convergence and its application to quasi-Newton methods, Math. Comput., 28 (1974), 549-560. [9] J.E. Dennis, Jr. and J.J. More, Quasi-Newton methods, Motivation and the theory, SIAM Rev., 19 (1977), 46{89. [10] F. Facchinei and J. Soares, A new merit function for nonlinear complementarity problems and related algorithm, SIAM J. Optim., 7 (1997), 225-247. [11] A. Fischer, A special Newton-type optimization method, Optimization, 24 (1992), 269-284. 24
[12] A. Fischer, A Newton-type method for positive semide nite linear complementarity problems, J. Optim. Theory Appl., 86 (1995), 585-608. [13] R. Fletcher, Practical Methods of Optimization, Second Edition, John Wiley & Sons, Chichester, 1987. [14] M. Fukushima and L. Qi, eds., Reformulation: Nonsmooth, Piecewise Smooth, Semismooth and Smoothing Methods, Kluwer Academic Publishers, Boston, 1998. [15] S.A. Gabriel and J.J. More, Smoothing of mixed complementarity problems, M.C. Ferris and J.S. Pang, eds., Complementarity and Variational Problems: State of the Art, SIAM Philadelpha, Pennsylvania, 1997, pp. 105-116. [16] C. Geiger and C. Kanzow, On the solution of monotone complementarity problems, Comput. Optim. Appl., 5 (1996), 155-173. [17] C. M. Ip and J. Kyparisis, Local convergence of quasi-Newton methods for B-di erentiable equations, Math. Programming, 56 (1992), 71-89. [18] H. Jiang and L. Qi, A new nonsmooth equations approach to nonlinear complementarity problems, SIAM J. Control Optim., 35 (1997), 178-193. [19] C. Kanzow and H. Jiang, A continuation method for (strongly) monotone variational inequalities, Math. Programming, 81 (1998), 103-125. [20] C. Kanzow and M. Fukushima, Equivalence of the generalized complementarity problem to di erentiable unconstrained minimization, J. Optim. Theory Appl., 90 (1996), 581-603. [21] M. Kojima and S. Shindo, Extension of Newton and quasi-Newton methods to systems of PC 1 equations, J. Oper. Res. Soc. Japan, 29 (1986), 352{374. [22] D. Li, Global convergence of nonsingular Broyden's method for solving unconstrained optimization, Chinese J. Numer. Math. Appl., 17 (1995), No.4 85-94. [23] D. Li and M. Fukushima, A derivative-free line seach and global convergence of Broyden-like method for nonlinear equations, Optim. Methods Softw., to appear. [24] D. Li and M. Fukushima, Smoothing Newton and quasi-Newton methods for mixed complementarity problems, Comput. Optim. Appl., to appear. [25] D. Li and M. Fukushima, Broyden-like methods for variational inequality and nonlinear complementarity problems with global and superlinear convergence, Technical Report 98016, Department of Applied Mathematics and Physics, Kyoto University, September 1998. 25
[26] D. Li and Y. Yang, A globally convergent quasi-Newton method for nonlinear complementarity problems, Math. Programming, submitted. [27] R. Miin, Semismooth and semiconvex functions in constrained optimization, SIAM J. Control Optim., 15 (1977), 957-972. [28] F.A. Potra, L. Qi and D. Sun, Secant methods for semismooth equations, Numerische Mathematik, 80 (1998), 305{324. [29] L. Qi, Convergence analysis of some algorithms for solving nonsmooth equations, Math. Oper. Res., 18 (1993), 227-244. [30] L. Qi and X. Chen, A globally convergent successive approximation method for severely nonsmooth equations, SIAM J. Control Optim., 33 (1995), 402-418. [31] L. Qi and J. Sun, Nonsmooth equations and smoothing Newton methods, Technical Report, School of Mathematics, University of New South Wales, Sydney, April 1998. [32] L. Qi, D. Sun and G. Zhou, A new look at smoothing Newton methods for nonlinear complementarity problems and box constrained variational inequalities, AMR 97/13, Applied Mathematics Report, University of New South Wales, Sydney, June 1997. [33] D. Sun and J. Han, Newton and quasi-Newton methods for a class of nonsmooth equations and related problems, SIAM J. Optim., 7 (1997), 463-480. [34] N. Yamashita and M. Fukushima, Modi ed Newton methods for solving a semismooth reformulation of monotone complementarity problems, Math. Programming, 76 (1997), 469-491. [35] S. Zhou, D. Li and J. Zeng, A successive approximation quasi-Newton process for nonlinear complementarity problems, D.Z. Du, L. Qi and R.S. Womersley, eds., Recent Advances in Nonsmooth Optimization, World Scienti c Publishers, Singapore, 1995, pp. 459-472.
26
Abstract In this paper, we propose a general smoothing Broyden-like quasi-Newton
method for solving a class of nonsmooth equations. Under appropriate conditions, the proposed method converges to a solution of the equation globally and superlinearly. In particular, the proposed method provides the possibility of developing a quasiNewton method that enjoys superlinear convergence even if strict complementarity fails to hld. We pay particular attention to semismooth equations arising from nonlinear complementarity problems, mixed complementarity problems and variational inequality problems. We show that under certain conditions, the related methods based on the perturbed Fischer-Burmeister function, Chen-Harker-Kanzow-Smale smoothing function and Gabriel-More class of smoothing functions converge globally and superlinearly.
Key words: Nonsmooth equations, Broyden-like method, global convergence, superlinear convergence.
1
The work of the rst author was done while he was visiting Kyoto University. This paper is an extended version of the unpublished technical report [25] by the authors.
1 Introduction Let H be a function from Rn into itself that is not necessarily di erentiable in the sense of Frechet but semismooth in the sense of Miin [27] and Qi [29]. We are concerned with the solution of the semismooth equation H (x) = 0: (1.1) Semismooth equations have a rich practical background. Many problems such as nonlinear complementarity problems, mixed complementarity problems and variational inequality problems can be reformulated as semismooth equations. Newton's method for solving (1.1) has been extensively studied in recent years. In particular, Chen et al. [5] proposed a smoothing Newton method for solving a semismooth equation reformulation arising from the mixed complementarity problem that possesses global and superlinear convergence properties. We refer to [14] and [31] for reviews about Newton-type methods for nonsmooth and semismooth equations. Local convergence of quasi-Newton methods for nonsmooth and semismooth equations has also been studied by some authors [3, 4, 17, 21, 28, 33]. There are a few papers that deal with globally convergent quasi-Newton methods for semismooth equations arising from nonlinear complementarity problems and mixed complementarity problems [24, 26, 35]. In [26], by using a splitting technique of [30], a well de ned quasi-Newton method that possesses global and superlinear convergence properties was rst proposed for nonlinear complementarity problems. In [24], a smoothing quasi-Newton method was developed for a semismooth equation reformulation of the mixed complementarity problem. The method proposed in [24] is an improvement of the method proposed in [26]. In particular, when applied to nonlinear complementarity problems, the method proposed in [24] converges globally whenever the problem involves a P0 function with Lipschitz continuous derivatives. Moreover, if strict complementarity condition holds at the limit point, then the method converges superlinearly. The purpose of this paper is to develop a general smoothing quasi-Newton approach to the semismooth equation (1.1). Under appropriate conditions, the proposed method converges globally and superlinearly. In particular, the results obtained in this paper provides the possibility of developing a globally convergent quasi-Newton method that enjoys superlinear convergence even if strict complementarity fails to hold at the limit point. We pay particular attention to semismooth equations arising from nonlinear complementarity problems, mixed complementarity problems and variational inequality problems. The smoothing functions considered include the perturbed Fischer-Burmeister function, Chen-Harker-Kanzow-Smale smoothing function and Gabriel-More class of smoothing functions. The paper is organized as follows: In the next section, we propose a general quasi-Newton approach to the solution of semismooth equations. In Sections 3 and 4, we prove global and superlinear convergence of a general Broyden-like method, respectively. In Section 5, we specialize 1
the method to the nonlinear complementarity problem, the variational inequality problem and the mixed complementarity problem. Section 6 concludes the paper.
2 General Quasi-Newton Approach This section develops a general quasi-Newton approach to the solution of equation (1.1). Throughout, we assume that H is a semismooth function. The proposed method is a smoothing quasiNewton method. The feature of smoothing methods is to construct a smooth function H as an approximation of H satisfying H 0 (z ) = H (z ) for any z 2 Rn and
kH (z) ? H 0 (z)k ( ? 0); 8 > 0 0;
8z 2 Rn;
(2.1)
and then to generate a sequence fxk g where xk is a solution of the smooth equation
H (x) = 0 k
with k ! 0, so that the sequence fxk g may converge to a solution of (1.1). Smoothing Newton methods have been received much attention in the literature [1, 2, 5, 15, 19, 30, 31]. The quasiNewton method developed here is based on the smoothing Newton method in [5]. For simplicity, we abbreviate H as H k . In the proposed method, the subproblem is the linear equation k
Bk p + H 0(xk ) = 0;
(2.2)
where matrix Bk 2 Rnn is an approximation of rH k (xk ). If Bk = rH k (xk ), the linear equation (2.2) reduces to the subproblem of smoothing Newton methods [5, 30]. Unlike Newton's method, in a quasi-Newton method, derivative-free line search is indispensable because the computation of derivatives should be avoided. We now propose a derivative-free line search that is similar to those in [24]. Let fk g be a positive sequence satisfying 1 X k=1
k < 1;
(2.3)
where is a positive constant. At iteration k, given the current iterate zk and a direction pk , we determine a positive steplength k > 0 such that the following inequality holds for = k :
kH k (zk + pk )k ? kH k (zk )k ?1kH k (zk + pk ) ? H k (zk )k ? 2 k pk k2 + k ;
(2.4)
where 1 ; 2 > 0 are suitable constants. Note that (2.4) is satis ed for all suciently small > 0 since k is positive and independent of . Thus k can be obtained by a backtracking process. 2
We state a general smoothing quasi-Newton method for (1.1) as follows.
Algorithm 1 Step 0. Choose constants ; 2 (0; 1), 0 < < 1=, 1 2 (0; 11+? ), 2 > 0, 2 (0; 1). Select a positive sequence fk g satisfying (2.3). Choose an initial point z1 2 Rn and an initial nonsingular matrix B1 2 Rnn. Choose an 1 > 0 satisfying 1 2 kH 0 (z1 )k. Let k := 1. Step 1. If H 0 (zk ) = 0, stop. Otherwise, solve the linear equation (2.2) to get pk . Step 2. Let k be the maximum element in the set f1; ; 2 ; : : :g such that = i satis es
the line search condition (2.4). Step 3. Let zk+1 = zk + k pk . Step 4. Update Bk to get Bk+1. Step 5. Update k+1 by the following rule: If k < kH 0 (zk+1)k, set k+1 = k . Otherwise, choose k+1 such that 0 < k+1 minf 2 kH 0 (zk+1 )k; 12 k g: Step 6. Let k := k + 1. Go to Step 1. Remark Step 5 of Algorithm 1 ensures that fk g is monotonically nonincreasing and k
kH 0 (zk )k holds for every k. Moreover, if the algorithm does not terminate nitely, i.e., H 0(zk ) 6= 0 for all k, then k > 0 for all k. We also notice that fzk g satis es the inequality
kH k (zk+1)k ? kH k (zk )k ?1kH k (zk+1) ? H k (zk )k ? 2 ksk k2 + k ; where sk = zk+1 ? zk . By (2.1), it then follows that for every k kH k+1(zk+1 )k kH k (zk+1 )k + kH k+1(zk+1 ) ? H k (zk+1)k kH k (zk+1 )k + (k ? k+1) kH k (zk )k ? 1kH k (zk+1 ) ? H k (zk )k ? 2ksk k2 + k + (k ? k+1 ):
(2.5)
Moreover, (2.5) implies
kH k+1(zk+1 )k kH k (zk )k + k + (k ? k+1):
(2.6)
Since k satis es (2.3), we immediately get the following result.
Lemma 2.1 Let fzk g be generated by Algorithm 1. Then fkH k (zk )kg converges. Moreover, we
have and
1 X k=1
1 X k=1
kH k (zk+1 ) ? H k (zk )k < 1
ksk k2 =
1 X k=1
kzk+1 ? zk k2 < 1: 3
(2.7) (2.8)
Proof Since (2.3) implies
1 X
fk + (k ? k+1)g < 1;
k=1
it follows from (2.6) that fkH k (zk )kg converges. The inequality (2.7) then follows from (2.5) directly. Moreover, by (2.5), it is easy to see that
2ksk k2 kH k (zk )k ? kH k+1 (zk+1)k + k + (k ? k+1): 2
Summing the above inequalities yields (2.8). De ne the level set
kH 0 (z )k + 1 g;
= fz 2 Rn j kH 0 (z )k 11 + 1 ? 1 ?
(2.9)
where is a positive constant such that (2.3) holds. Notice that (2.1) particularly implies
kH (z) ? H (z)k ; 8z 2 Rn: (2.10) The following lemma shows that the sequence fzk g generated by Algorithm 1 is contained in the
level set .
Lemma 2.2 Let fzk g be generated by Algorithm 1. Then fzk g . Proof Summing (2.6), we get for every k kH k+1(zk+1)k kH 1 (z1 )k + + 1 kH 0 (z1 )k + + 21 (1 + )kH 0 (z1 )k + ;
(2.11)
where the second inequality follows from (2.10) and the last inequality follows from the choice of 1. By using (2.10) again, we have
kH 0 (zk )k kH k (zk )k + kH k (zk ) ? H 0 (zk )k kH k (zk )k + k kH k (zk )k + kH 0 (zk )k; where the last inequality holds since by Step 5, k kH 0 (zk )k for every k. So, we deduce from
(2.11) and the last inequality
1 : 0 k H ( z kH 0 (zk )k 1 ?1 kH k (zk )k 11 +? 1 )k + 1 ?
This means that fzk g as desired.
4
2
3 Global Convergence of Broyden-Like Methods In this section, we prove global convergence of Algorithm 1. Let us de ne the index set K f1; 2; : : :g by K = fk j k kH 0 (zk+1 )kg: (3.1) Then we have the following result. Theorem 3.1 Let fzk g be generated by Algorithm 1. If K is in nite, then lim inf kH 0 (zk )k = 0: k!1
(3.2)
Proof Let K = fk0 < k1 < k2 < g. Then by Step 5, for every k such that ki?1 < k ki , we
have k = k . So, it follows that i
kH 0 (zk +1)k ?1k = ?1 k ?1+1 12 kH 0 (zk ?1 +1)k i
i
i
i
( 21 )ikH 0 (zk0 +1)k ?1 ( 21 )i k0 ( 12 )i kH 0 (zk0 )k;
(3.3)
where the last inequality holds since by Step 5 of Algorithm 1, k kH 0 (zk )k holds for every k. This implies (3.2). 2 Theorem 3.1 reveals that to show that Algorithm 1 converges globally, it suces to verify that the index set K de ned by (3.1) is in nite. We focus our attention to Broyden-like methods. That is, in Step 4 of Algorithm 1, Bk+1 is updated by the Broyden-like formula
Bk+1 = Bk + k (yk ?ksBkks2k )sk ; T
k
(3.4)
where sk = zk+1 ? zk and yk = H k (zk+1 ) ? H k (zk ). The parameter k is chosen to satisfy jk ? 1j and Bk+1 is nonsingular [9]. We call the algorithm with (3.4) Broyden-like method. To prove global convergence of Broyden-like method, we need the following assumption. Assumption A (i) The level set de ned by (2.9) is contained in a bounded convex set D. (ii) For each > 0, function H is continuously di erentiable and rH is Lipschitz continuous on D. That is, there is a positive constant such that
krH (z) ? rH (z0 )k kz ? z0k; Let scalar k be de ned by and matrix Ak be de ned by
k = kyk ?ksBkk sk k k
Ak =
Z 0
1
rH k (zk + tsk )dt: 5
8z; z0 2 D:
(3.5) (3.6) (3.7)
Then by the mean value theorem, we have yk = Ak sk and hence = k(Ak ? Bk )sk k = k(Ak ? Bk )pk k : k
ksk k
kpk k
(3.8)
The next useful lemma follows from (2.8) and Lemma 2.6 in [23].
Lemma 3.1 Let Assumption A hold. If there is an index k such that H k (z) H k (z) for all k k, then k X 1 lim i2 = 0: k!1 k i=1
In particular, there is an in nite index set K 1 such that k ! 0 as k ! 1 with k 2 K 1 .
Now we prove global convergence of Broyden-like method. From Theorem 3.1, it suces to show that the index set K de ned by (3.1) is in nite.
Theorem 3.2 Let Assumption A hold. Assume that, for any accumulation point z of the sequence fzk g generated by Broyden-like method, rH k (z ) is nonsingular for all suciently large
k. Then the index set K de ned by (3.1) must be in nite. Moreover, we have lim kH 0 (zk )k = 0
k!1
(3.9)
and every accumulation point of fzk g is a solution of (1.1).
Proof We assume that K is nite to deduce a contradiction. By Step 5 of Algorithm 1, there is an index k such that k = for all k k with some constant > 0. Hence H k = H for all k k. Without loss of generality, we assume that k = and H k = H for all k with some function H : Rn ! Rn .
From Lemma 3.1, there is an in nite index set K 1 such that limk2K 1;k!1 k = 0. Since fzk gk2K 1 is bounded, there exists a subsequence fzk gk2K fzk gk2K 1 converging to some point z. Let Ak be de ned by (3.7). Then Ak ! rH (z ) as k ! 1 with k 2 K . Moreover, we get from (2.2) and (3.8)
kH 0 (zk )k = kBk pk k k(Bk ? Ak )pk k + kAk pk k k kpk k + kAk k kpk k: Since k ! 0 as k ! 1 with k 2 K 1 , there is a constant M1 > 0 such that for all k 2 K large enough kH 0 (zk )k M1 kpk k: (3.10) 6
Let = lim supk2K;k!1 k . If > 0, then we have limk2K;k!1 kpk k = 0 by (2.8). Hence (3.10) implies (3.9). Now consider the case = 0, that is, limk2K;k!1 k = 0. By the assumption that rH (z ) is nonsingular, it is clear that when k 2 K is suciently large, rH (zk ) and Ak are uniformly nonsingular. In particular, there exists a positive constant M2 > 0 such that krH (zk )?1 k M2 and kA?k 1 k M2 hold for all k 2 K large enough. This together with (2.2) yields
kpk k = kA?k 1 [(A k ? Bk )pk ? H 0(zk )]k kA?k 1 k k(Ak ? Bk )pk k + kH 0 (zk )k M2k kpk k + M2 kH 0(zk )k: Since k ! 0 as k ! 1 with k 2 K , the last inequality implies that there is a constant M3 > 0 such that for all k 2 K suciently large
kpk k M3kH 0 (zk )k:
(3.11)
In particular, fpk gk2K is bounded. Without loss of generality, we assume that pk ! p as k ! 1 with k 2 K . Again by (2.2) we have 0 = kBk pk + H 0 (zk )k kH 0(zk ) + Ak pk k ? k(Bk ? Ak )pk k = kH 0 (zk ) + Ak pk k ? k kpk k: Letting k 2 K go to in nity yields
rH (z)T p + H 0(z ) = 0:
(3.12)
On the other hand, the assumption that limk2K;k!1 k = 0 means that k < 1 for all k 2 K suciently large. By the line search rule, it means that when k 2 K is suciently large, 0k k = does not satisfy (2.4). In other words, we have
kH (zk + 0k pk )k ? kH (zk )k > ?1 kH (zk + 0k pk ) ? H (zk )k ? 2 k 0 pk k2 + k ?1 kH (zk + 0k pk ) ? H (zk )k ? 2 k 0 pk k2 : Multiplying the both sides by kH (zk + 0k pk )k + kH (zk )k yields
kH (zk + 0k pk )k2 ? kH (zk )k2 ?1 kH (zk + 0k pk )k + kH (zk )k kH (zk + 0k pk ) ? H (zk )k ? 2 k 0k pk k2 kH (zk + 0k pk )k + kH (zk )k : 7
Dividing the both sides by 0k and then taking the limit as k ! 1 with k 2 K yield 2H (z )T rH (z )T p ?21 kH (z )k krH (z )pk: This together with (3.12) implies
H (z )T H 0 (z) 1kH (z)k kH 0 (z )k:
(3.13)
Since kH 0 (zk )k for all k, we have kH 0 (z )k. It then follows from (2.10) that
kH (z ) ? H 0 (z)k = kH (z) ? H 0(z )k kH 0(z )k; and hence
(3.14)
kH (z )k kH 0 (z)k + kH (z) ? H 0(z )k (1 + )kH 0 (z)k:
Consequently, we get from the last inequality and (3.13)
1(1 + )kH 0 (z )k2 1kH (z)k kH 0 (z )k H (z)T H 0(z ) = kH 0 (z )k2 + (H (z ) ? H 0 (z ))T H 0 (z ) kH 0 (z)k2 ? kH (z ) ? H 0(z )k kH 0 (z )k (1 ? )kH 0 (z)k2 ; 0 z ) = 0, which where the last inequality follows from (3.14). However, since 1 < 11+? , we get H ( contradicts the fact that 0 < kH (z )k. The contradiction shows that K must be in nite. Therefore by Theorem 3.1, we have lim inf k!1 kH 0 (zk )k = 0. Now we show that (3.9) holds. Indeed, from (2.10), we get
kH k (zk )k kH 0 (zk )k + k (1 + )kH 0 (zk )k: This means that there is a subsequence of fH k (zk )g converging to zero. However, Lemma 2.1 shows that fkH k (zk )kg itself converges and hence it converges to zero. Again by (2.10) we get
kH 0 (zk )k kH k (zk )k + k kH k (zk )k + kH 0 (zk )k; which implies that
kH 0 (zk )k 1 ?1 kH k (zk )k:
This shows that (3.9) holds. Consequently, every accumulation point of fzk g is a solution of (1.1). 2 Remark (i) In Theorem 3.2, the assumption that rH k (z) is nonsingular for every accumulation point z can be replaced by the weaker assumption that there is an accumulation point z of the 8
subsequence fzk gk2K 1 such that rH k (z ) is nonsingular for all k large enough, where K 1 is the index set as speci ed in Lemma 3.1. (ii) Since by Step 5 in Broyden-like method, k kH (zk )k holds for every k and fk g is monotonically nonincreasing, Theorem 3.2 shows that lim k = 0:
k!1
(3.15)
4 Superlinear Convergence In this section, we prove superlinear convergence of Broyden-like method. To this end, we need further assumptions. Assumption B (i) The sequence fzk g generated by Broyden-like method converges to a solution z of (1.1) at which every element in @C H (z) is nonsingular, where @C H (z) = @H1(z) @H2 (z) @Hn(z), and @h(z) denotes the generalized Jacobian of function h : Rn ! R at z in the sense of Clarke [6]. (ii) Functions H k and H satisfy lim dist (rH k (zk ); @C H (z )) = 0:
k!1
(4.1)
(iii) There exist a neighbourhood U (z ) of z and a constant > 0 such that for all k suciently large krH k (z) ? rH k?1(z)k (k?1 ? k ); 8z 2 U (z ) (4.2) and krH k (zk + tsk ) ? rH k (zk?1 + tsk?1)k (ksk k + ksk?1k); 8t 2 (0; 1): (4.3) (iv) As k goes to in nity, we have
kH k (z ) ? H 0(z )k = o(kzk ? zk); kH k (zk ) ? H 0(zk )k = o(kzk ? zk):
(4.4)
Remark (i) Since @C H (z) is compact and @C H () is upper semicontinuous, it is not dicult to
see that conditions (i) and (ii) of Assumptions B imply that there is a neighbourhood U (z ) of z, a constant 0 > 0 and an index k such that inequality krH k (z )k 0 holds for all k k and any z 2 U (z ). Moreover, there are constants 0 < c C such that
ckz ? z0k kH k (z) ? H k (z0 )k C kz ? z0 k; 8k k; 8z; z0 2 U (z ): (ii) Inequality (4.3) holds particularly when H k satis es
krH k (z) ? rH k (z0)k kz ? z0 k; 8z; z0 2 U (z): 9
(4.5)
Lemma 4.1 Let Assumptions A and B hold and k be de ned by (3.6). Then we have lim k = 0:
k!1
(4.6)
Proof It follows from (2.7) in Lemma 2.1 and (4.5) that 1 X
k=1
ksk k =
1 X
k=1
kzk+1 ? zk k < 1:
(4.7)
Let Ak be de ned by (3.7). Then we have yk = Ak sk . Moreover, it follows from (3.4) that
T 2 kBk+1 ? Ak k2F =
Bk ? Ak + k (Ak ? Bk ) kssk skk2
F k 2 k ( = kBk ? Ak k2F + (k2 ? 2k ) Bk k?s Akk2)sk k k 2 k y ? B k = kBk ? Ak k2F + (k2 ? 2k ) ks kk2sk k k 2 2 2 = kBk ? Ak kF + (k ? 2k )k kBk ? Ak k2F ? (1 ? 2)k2 ; (4.8) where the last inequality holds since k is chosen to satisfy j1 ? k j in Step 4. Then we deduce
kAk ? Ak?1 k
Z
0
Z
1
krH k (zk + tsk ) ? rH k?1(zk?1 + tsk?1)kdt
1
krH k (zk + tsk ) ? rH k (zk?1 + tsk?1)kdt
0
Z
1
krH k (zk?1 + tsk?1) ? rH k?1(zk?1 + tsk?1)kdt ksk k + ksk?1k + (k?1 ? k ); +
0
where the last inequality follows from (4.2) and (4.3). This together with (4.7) implies 1 X
k=2
kAk ? Ak?1k < 1:
(4.9)
Therefore, we get from (4.8)
kBk+1 ? Ak kF kBk ? Ak kF kBk ? Ak?1kF + kAk ? Ak?1kF : This along with (4.9) implies that fkBk+1 ? Ak kF g converges. Thus we deduce from (4.8) again that (4.6) holds. 2 Lemma 4.2 Let z be a solution of (1.1). Then under Assumptions A and B, we have kzk + pk ? zk = 0: (4.10) lim k!1 kzk ? zk 10
Proof Let Ak be de ned by (3.7). It then follows from (4.3) that Ak ?rH k (zk ) ! 0. Moreover,
since (4.6) holds, using the arguments for showing (3.11) in the proof of Theorem 3.2, we can deduce that there is a positive constant M3 such that
kpk k M3 kH 0(zk )k LM3kzk ? zk;
(4.11)
where L > 0 is the Lipschitz constant of H 0 . We then get from (2.2)
kzk + pk ? zk = krH k (zk )?1 (rH k (zk )(zk ? z) + rH k (zk )pk ? Bk pk ? H 0 (zk ))k krH k (zk )?1 k k(rH k (zk ) ? Bk )pk k + krH k (zk )(zk ? z) + H 0 (z) ? H 0(zk )k 0 k(Ak ? Bk )pk k + k(rH k (zk ) ? Ak )pk k + krH k (zk )(zk ? z) + H 0 (z) ? H 0(zk )k 0 k kpk k + krH k (zk ) ? Ak k kpk k + krH k (zk )(zk ? z) + H k (z ) ? H k (zk )k + k[H 0 (z ) ? H 0 (zk )] ? [H k (z ) ? H k (zk )]k LM3 0 k + krH k (zk ) ? Ak k kzk ? zk + o(kzk ? zk) = o(kzk ? zk); where the third inequality follows from (3.6) and yk = Ak sk , the last inequality follows from (4.4) and (4.11), and the last equality follows from (4.6). 2 Now, we establish superlinear convergence of Broyden-like method. By Lemma 4.2, it suces to show that when k is suciently large, the unit steplength is always accepted.
Theorem 4.1 Under Assumptions A and B, the sequence fzk g generated by Broyden-like method converges superlinearly.
Proof By Lemma 4.2, it suces to show that when k is suciently large, = 1 satis es (2.4).
Indeed, it follows from (4.4), (4.5) and (4.10) that for all k large enough
kH k (zk + pk )k = kH k (zk + pk ) ? H 0(z )k kH k (zk + pk ) ? H k (z )k + kH k (z ) ? H 0 (z)k C kzk + pk ? zk + o(kzk ? zk) = o(kzk ? zk): On the other hand, we have
kH k (zk )k ? 1kH k (zk + pk ) ? H k (zk )k ? 2 kpk k2 = kH k (zk ) ? H 0 (z )k ? 1 kH k (zk + pk ) ? H k (zk )k + o(kzk ? zk) 11
(4.12)
kH k (zk ) ? H k (z )k ? kH k (z ) ? H (z )k ? kH k (zk + pk ) ? H k (z )k + kH k (z ) ? H k (zk )k + o(kzk ? zk) = (1 ? )kH k (zk ) ? H k (z )k + o(kzk ? zk) (1 ? )ckzk ? zk + o(kzk ? zk); 0
1
1
1
(4.13)
where the rst equality follows from (4.11), the last equality follows from (4.4), (4.5) and (4.10), and the last inequality follows from (4.5). Comparing (4.12) with (4.13), it is clear that when k is suciently large
kH k (zk + pk )k kH k (zk )k ? kH k (zk + pk ) ? H k (zk )k ? kpk k kH k (zk )k ? kH k (zk + pk ) ? H k (zk )k ? kpk k + k : 1
2
1
2
2
2
This means that when k is suciently large, = 1 is always accepted. 2 Theorem 4.1 reveals that to ensure superlinear convergence of Broyden-like method, the strict complementarity at z is not necessary.
5 Applications In this section, we apply the general smoothing quasi-Newton method proposed in Section 2 to nonlinear complementarity problems, mixed complementarity problems and variational inequality problems. We show that for some well known smoothing functions, the conditions in Assumptions A and B hold, and hence the related algorithms converge globally and superlinearly.
5.1 Nonlinear complementarity problems Let f : Rn ! Rn be continuously di erentiable. Consider the nonlinear complementarity problem of nding a z 2 Rn such that
z 0; f (z ) 0; zT f (z ) = 0:
(5.1)
There have been developed various reformulations of (5.1) and related smoothing functions. Fischer-Burmeister function as well as its perturbed version and Chen-Harker-Kanzow-Smale smoothing function are widely used [7, 10, 11, 12, 16, 18, 19, 34]. Fischer-Burmeister function FB : R2 ! R is de ned by p FB (a; b) = a2 + b2 ? (a + b): (5.2) It is easy to see that FB (a; b) = 0 holds if and only if a 0, b 0 and ab = 0. Fischer-Burmeister function is di erentiable at every point except the origin and is semismooth at the origin. By 12
introducing a perturbation parameter, we get a smoothing Fischer-Burmeister function
p
FB (a; b) = a2 + b2 + 2 ? (a + b);
(5.3)
where is a nonnegative parameter. It is clear that 0FB (a; b) FB (a; b). Let H (z ) = (H1 (z ); H2 (z ); : : : ; Hn (z ))T , where
Hi(z ) = FB (zi ; fi (z )); i = 1; 2; : : : ; n:
(5.4)
Then H is continuously di erentiable everywhere if f is continuously di erentiable and > 0. Moreover, (5.1) is equivalent to the semismooth equation H 0 (z ) = 0. Let zi ? 1; bi (z ) = q fi (z ) ? 1: ai(z ) = q zi2 + fi(z )2 + 2 zi2 + fi(z )2 + 2 Then it is obvious that for every > 0 and any x, we have
?2 < ai(z) < 0; ?2 < bi(z) < 0:
(5.5)
rH (z) = diag (ai(z)) + rf (z) diag (bi(z)):
(5.6)
Moreover, it is easy to deduce The following lemma gives some properties of FB and H .
Lemma 5.1 Let FB and H be de ned by (5.3) and (5.4), respectively. Suppose that f is
continuously di erentiable. Then the following statements hold. (i) For any > 0, functions FB and H are continuously di erentiable everywhere. (ii) For any > 0 > 0, we have 0 j FB (a; b) ? FB (a; b)j ? 0 ; 8(a; b) 2 R
and hence
2
kH (z) ? H 0 (z)k pn( ? 0 ); 8z 2 Rn:
(iii) For a given constant > 0, the inequality
j FB (a; b) ? FB (a; b)j 21 ? 0
1 2
holds for all (a; b) 2 R2 satisfying a2 + b2 2 . Consequently, the inequality kH (z) ? H 0 (z)k 12 pn?1 2 holds for all z 2 Rn satisfying zi2 + fi (z )2 2 , i = 1; 2; : : : ; n. (iv) If f is a P0 function, then rH (z ) is nonsingular for every > 0 and any z 2 Rn.
13
(5.7)
Proof The statements (i), (ii) and (iv) have been proved in [34]. We prove (iii). Let a + b . 2
Then, we get
2
p
p
j FB (a; b) ? FB (a; b)j =
2
a2 + b2 + 2 ? a2 + b2 2 = p 2 2 2 p 2 2 a +b + + a +b 1 2 ?1 2:
0
2
Now we make the following assumption. Assumption A0 (i) The level set de ned by (2.9) is contained in a bounded convex set D. (ii) Function f is continuously di erentiable and rf is Lipschitz continuous on D.
Lemma 5.2 Let Assumption A0 hold. Then for any > 0, there is a constant > 0 such that 1
krH (z) ? rH (z0 )k kz ? z0k;
8z; z0 2 D:
1
(5.8)
Proof It follows from (5.5) and (5.6) that krH (z) ? rH (z0 )k
=
diag (ai (z ) ? ai (z 0 )) + (rf (z ) ? rf (z 0 )) diag (bi (z )) + rf (z 0 ) diag (bi (z ) ? bi (z ))
k diag (ai(z) ? ai(z0 ))k + 2krf (z) ? rf (z0)k + krf (z0 )k k diag (bi(z) ? bi(z))k: It then suces to show that there is a constant 1 > 0 such that inequalities
jai(z) ? ai(z0 )j kz ? z0 k and jbi(z) ? bi(z0 )j kz ? z0 k 1
(5.9)
1
hold for any z; z 0 2 D. By the de nition of ai, we have
0 zi jai(z) ? ai(z0 )j = q ?q 0 z zi + fi (z ) + (zi ) + fi (z 0 ) + 0 0 1 ?q 0 1 0 q zi ? zi + jzi j q (zi ) + fi (z ) + zi + fi (z ) + zi + fi(z ) + q q ? zi + fi (z ) + ? (zi0 ) + fi(z 0 ) + ? jzi ? zi0 j + M 0 0 ? q jzi ? (zi ) + fiq(z ) ? fi (z ) j = ? jz ? z 0 j + M 1
1
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
zi2 + fi(z )2 + 2 + (zi0 )2 + fi (z 0 )2 + 2 ?3 (jzi2 ? (zi0 )2 j + jfi (z )2 ? fi (z 0 )2 j); ?1jzi ? zi0 j + 2?1 M 14
2
(5.10)
where M is a constant such that kz k M for all z 2 D. The last inequality implies the rst inequality of (5.9) with a suitable constant 1 > 0. The second inequality of (5.9) can be proved in a similar way. 2 Lemmas 5.1 and 5.2 imply the following global convergence theorem.
Theorem 5.1 Let Assumption A0 hold and H be de ned by (5.4). Suppose that f is a P function. Then the sequence fxk g generated by the related Broyden-like method satis es (3.9).
0
We say that z is a strictly complementary solution of (5.1) if it is a solution of (5.1) and satis es zi + fi(z ) > 0; i = 1; 2; : : : ; n: (5.11) We make another assumption as follows. Assumption B0 (i) The sequence fzk g generated by Broyden-like method converges to a solution z of (1.1) that is a strictly complementary solution of (5.1). (ii) rH 0 (z ) is nonsingular. Remark The strict complementarity (5.11) implies that H is di erentiable near z. Hence, rH 0(z ) exists. Moreover, @C H 0(z ) reduces to the singleton frH 0(z )g. Therefore, Assumption B0 implies condition (i) in Assumption B. The following lemma shows that Assumptions A0 and B0 imply Assumption B.
Lemma 5.3 Let FB and H be de ned by (5.3) and (5.4), respectively. If Assumptions A0 and
B0 hold, then Assumption B holds.
Proof The inequality (5.11) implies that there is a neighbourhood U (z) of z such that the 4 > 0 holds for any z 2 U (z ). It is then clear that H is inequality zi + fi (z ) (zi + fi (z )) = 2
2
1 2
2
2
2
0
di erentiable in U (z ) and hence equality (4.1) holds. Inequality (4.3) can be obtained in a way similar to Lemma 5.2. Inequality (4.4) follows from Lemma 5.1 (iii) and the fact k kH k (zk )k. It remains to show (4.2). By (5.6), it suces to show that there exists a constant > 0 such that the following inequalities hold for any > 0 and all z 2 U (z ): jai(z) ? ai 0 (z)j ( ? 0); jbi(z) ? bi 0 (z)j ( ? 0): (5.12) Indeed, by the de nition of ai , we have jai(z) ? ai 0 (z)j = jzij q 2 1 ?q 2 1 zi + fi(z )2 + 2 zi + fi (z )2 + (0)2 q q ?1jzi j zi2 + fi(z)2 + 2 ? zi2 + fi(z)2 + (0)2 2 ? q (0 )2 = ?1jzi j q zi2 + fi(z )2 + 2 + zi2 + fi (z )2 + (0)2 ?1jzi j( ? 0): 15
This shows the rst inequality of (5.12). The second inequality of (5.12) can be proved in a similar way. 2 In view of Lemmas 5.1, 5.2 and 5.3, and Theorem 4.1, we get the following superlinear convergence theorem.
Theorem 5.2 Let Assumptions A0 and B0 hold and H be de ned by (5.4). Then the sequence fzk g generated by the related Broyden-like method converges superlinearly. Chen-Harker-Kanzow-Smale smoothing function is de ned by
CHKS (a; b) =
p
a + b ? (a ? b)2 + 2 :
(5.13) 2 It is a smoothing function of 0CHKS (a; b) = minfa; bg. Functions CHKS and 0CHKS enjoy some important properties that FB and 0FB have. Let
H (z ) = ( CHKS (z1 ; f1(z )); : : : ; CHKS (zn ; fn(z )))T :
(5.14)
It is not dicult to prove that Lemmas 5.1, 5.2 and 5.3 remain true. It then follows that Theorems 5.1 and 5.2 hold even if FB is replaced by CHKS . We omit the detail.
5.2 KKT system of variational inequality problems Let F : Rn ! Rn be continuously di erentiable and X be a closed convex subset of Rn . The variational inequality problem VI(X; F ) is to nd an x 2 Rn such that
F (x)T (y ? x) 0;
8y 2 X:
(5.15)
In many cases, X is speci ed as
X = fx 2 Rn j gi (x) 0; i = 1; 2; : : : ; m1 ; hj (x) = 0; j = 1; 2; : : : ; m2 g; where gi : Rn ! R and hj : Rn ! R are twice continuously di erentiable functions. It has been shown that under a suitable constraint quali cation, a solution of (5.15) together with some vectors 2 Rm1 and 2 Rm2 is a solution of the following KKT system:
0 BB L(z) @ min(; g(x)) h(x)
1 CC A = 0;
(5.16)
where z = (x; ; ) 2 Rn+m1 +m2 ,
L(z ) = F (x) ? rg(x) ? rh(x); 16
(5.17)
and
min(; g(x)) = (min(1 ; g1 (x)); min(2 ; g2 (x)); : : : ; min(m1 ; gm1 (x)))T : By using Fischer-Burmeister function, (5.16) can be reformulated as
0 BB L(z) H (z ) = @ FB (; g(x)) h(x)
1 CC A = 0;
(5.18)
where FB (; g(x)) = ( FB (1 ; g1 (x)); : : : ; FB (m1 ; gm1 (x)))T and FB is de ned by (5.2). By using the perturbed Fischer-Burmeister function, we get a smoothing function to H
0 L(z ) B H (z ) = B @ FB (; g(x)) h(x)
1 CC = 0; A
(5.19)
where FB (; g(x)) = ( FB (1 ; g1 (x)); : : : ; FB (m1 ; gm1 (x)))T and FB is de ned by (5.3). Put i ? 1; bi (x; ) = q gi (x) ? 1: ai(x; ) = q 2i + gi (x)2 + 2 2i + gi(x)2 + 2 It is easy to show the following lemma by using the chain rule of derivatives for composite functions.
Lemma 5.4 Let F be continuously di erentiable and gi , i = 1; 2; : : : ; m , and hj , j = 1; 2; : : : ; m , 1
2
be twice continuously di erentiable. Then and H are continuously di erentiable for every 6= 0. Moreover, for every z = (x; ; ) 2 Rn+m1 +m2 and any 6= 0, we have the following expressions:
rx(x; ) = rg(x) diag (bi(x; )); and
r(x; ) = diag (ai(x; ));
1 0 ( x; )) ?r h ( x ) r L ( z ) ?r g ( x ) diag ( b i C B x T (x; )) rH (z) = B ?r g ( x ) ? diag ( a 0 C i A: @ T ?rh(x) 0 0
(5.20)
In the remainder of this section, we abbreviate ai (x; ) and bi (x; ) as ai and bi , respectively. It is clear that diag(ai ) and diag(bi) are nonsingular. Moreover, ai and bi are negative. This implies that for any (x; ) 2 Rn+m1 , the diagonal matrix diag ((ai )?1bi ) is positive de nite, and hence, the matrix rg(x)diag((ai )?1bi )rg(x)T is symmetric and positive semide nite for any (x; ) 2 Rn+m1 . The following lemma shows some conditions that ensure regularity of H . 17
Lemma 5.5 Let z = (x; ; ) 2 Rn
6= 0. If rhj (x); j = 1; 2; : : : ; m2 ; are linearly
m1 +m2 and
+
independent and the matrix
rxL(z) + rg(x)diag((ai)? bi)rg(x)T 1
is positive de nite on the subspace fp j rh(x)T p = 0g, then rH (z ) is nonsingular. In particular, if rx L(z ) is positive de nite, then rH (z ) is nonsingular.
Proof Let w = (p; q; r) 2 Rn
m1 +m2
+
0, i.e.,
be a solution of the system of linear equations rH (z )w =
10 1 0 bi ) ?rh(x) r L ( z ) ?r g ( x ) diag ( x CC BB p CC BB T ? diag (ai ) 0 A @ q A = 0: @ ?rg(x) T r ?rh(x) 0 0:
We will show that w = 0. Writing the above equations blockwise, we get 8 > < rxL(zT)p ? rg(x) diag (bi)q ? rh(x)r = 0; = 0; ?rg(x) p ? diag (ai )q > : ?rh(x)T p = 0; which implies
(5.21)
8 < rxL(z) + rg(x)diag((ai)? bi)rg(x)T p ? rh(x)r = 0; : rh(x)T p = 0: 1
Multiplying the rst equation by p and taking into account the second equation yield
pT rxL(z ) + rg(x)diag((ai )?1bi )rg(x)T p = 0:
(5.22)
However, since p lies in the subspace fp j rh(x)T p = 0g, it follows from the positive de niteness of rx L(z ) + rg(x)diag((ai )?1bi )rg(x)T on the subspace that p = 0. Then, the second equation in (5.21) implies that q = 0. Finally, since rhj (x), j = 1; 2; : : : ; m2 , are linearly independent, it follows from the rst equation of (5.21) that r = 0. Consequently, zero is the unique solution of (5.21), or equivalently, rH (z ) is nonsingular. The last part of the lemma is a direct consequence 2 of the rst part by the positive semide niteness of rg(x) diag ((ai )?1bi )rg(x)T . Similar to Assumption A0 , we make the following assumption. Assumption A00 (i) The level set de ned by (2.9) is contained in a bounded convex set D. (ii) The function F is continuously di erentiable on Dx , and gi , i = 1; 2; : : : ; m1 , and hj , j = 1; 2; : : : ; m2 , are twice continuously di erentiable on Dx, where Dx = fx 2 Rn j (x; ; ) 2 Dg. 18
(iii) rF , r2 g and r2 h are Lipschitz continuous on Dx , i.e., there is a constant > 0 such that 8 > < kr2F (x) ? rF2 (y)k kx ? yk; 8x; y 2 Dx kr g(x) ? r g(y)k kx ? yk; 8x; y 2 Dx > : kr2h(x) ? r2h(y)k kx ? yk; 8x; y 2 Dx: In a way similar to Lemma 5.2, we can prove that under Assumption A00 , rH is Lipschitz continuous on D. To be speci c, we have the following lemma. Lemma 5.6 Let Assumption A00 hold. Then for any > 0, there is a constant 2 > 0 such that
krH (z) ? rH (z0 )k kz ? z0k; 2
8z; z0 2 D:
(5.23)
Similar to Lemma 5.3, we can show that the following assumption implies Assumption B. Assumption B00 (i) The sequence fzk g generated by Algorithm 1 converges to a solution of (5.16), say z = (x; ; ), at which the strict complementarity condition holds, that is, i + gi(x) > 0; i = 1; 2; : : : ; m1 : (ii) rH 0 (z ) is nonsingular. Therefore, we have the following global and superlinear convergence theorem.
Theorem 5.3 Let Assumption A00 hold and the sequence fzk g be generated by Broyden-like method with H de ned by (5.19). Suppose that, for any accumulation point z of fzk g, matrices rH k (z ) are nonsingular for all suciently large k. Then we have lim kH (zk )k = 0 k!1 and every accumulation point of fzk g is a solution of (5.16). If in addition, Assumption B00 holds, then fzk g converges superlinearly. Remark (i) A condition that ensures nonsingularity of rH k (z ) has been given in Lemma 0
5.5.
(ii) A convergence theorem similar to Theorem 5.3 can be established even if FB is replaced by Chen-Harker-Kanzow-Smale smoothing function.
5.3 Mixed complementarity problems Let X = ni=1 [li ; ui ], where ?1 li < ui 1 for each i. Then problem (5.15) is called the mixed complementarity problem. The mixed complementarity problem is equivalent to nding a z 2 Rn such that 8 > if fi(z ) > 0; < zi = li; zi = ui ; if fi(z ) < 0; i = 1; 2; : : : ; n; > : li zi ui; if fi(z) = 0; 19
which can be further reformulated as mid fzi ? li ; zi ? ui ; fi (z )g = 0; i = 1; 2; : : : ; n;
(5.24)
where mid fa; b; cg denotes the median of a; b; c. Gabriel and More [15] introduced a class of smoothing functions for mixed complementarity problems, which is an extension of the class of smoothing functions introduced by Chen and Mangasarian [2]. We show that under suitable conditions, Gabriel-More smoothing function associated with the semismooth equation (5.24) satis es Assumptions A and B. Let : R ! R+ be a continuous density function with a bounded absolute mean, that is, 4 =
Z1
?1
jsj(s)ds < 1:
Then the Gabriel-More smoothing function is de ned by H (z ) = (H1 (z ); : : : ; Hn (z ))T , where
Hi (z ) =
Z1
?1
mid fzi ? li ; zi ? ui ; fi (z ) ? 2 sg(s)ds; i = 1; 2; : : : ; n:
(5.25)
Note that the function H given by (5.25) is slightly di erent from the original Gabriel-More smoothing function where 2 is replaced by . Gabriel-More smoothing function contains as a special case Chen-Harker-Kanzow-Smale smoothing function for (5.24), which corresponds to the choice (s) = (s2 +11)3=2 : For any > 0, H is continuously di erentiable, as long as f is continuously di erentiable. Moreover, the mixed complementarity problem is equivalent to the semismooth equation H 0 (z ) = 0. Function H possesses many good properties. Here we mention some properties that are particularly useful for the related smoothing quasi-Newton method. First, it has been proved in [5] that the density function satis es the inequality 2 (5.26) (s) < 1 for all s such that jsj is suciently large. The following lemma comes from [5] and [15].
s
Lemma 5.7 Let H be de ned by (5.25). Suppose that f is continuously di erentiable. Then
the following statements are true. (i) For any > 0 0 and all z 2 Rn , we have
jHi(z) ? Hi0 (z)j ( ? (0) ): 2
20
2
(5.27)
(ii) For every > 0, we have
rHi(z) = ei ?
Z
z ?fi (z)?li )=2
( i
z ?fi (z)?ui )=2
( i
(s)ds (ei ? rfi (z )); i = 1; 2; : : : ; n;
(5.28)
where ei is the i-th column of the n n identity matrix.
The following lemma gives a sucient condition for Lipschitz continuity of rH .
Lemma 5.8 Let H be de ned by (5.25). Suppose that f is continuously di erentiable and rf is Lipschitz continuous. Then for each > 0, there is a constant > 0 such that for any z; z 0 2 Rn krH (z) ? rH (z0 )k kz ? z0k: (5.29) Proof By using (5.28), it suces to verify that there is a constant > 0 such that for each 3
3
i = 1; 2; : : : ; n
3
Z z0 ?f z0 ?l =2 Z z ?f z ?l =2 z ?f z ?u =2 (s)ds ? z0 ?f z0 ?u =2 (s)ds kz ? z0k: ( i
( i
i(
i(
)
)
i(
( i
i)
( i
i)
i(
)
i)
)
2
i)
(5.30)
It is clear that is bounded. Let = sup (s). By direct deduction, we have
Z z0 ?f z0 ?l =2 Z z ?f z ?l =2 z ?f z ?u =2 (s)ds ? z0 ?f z0 ?u =2 (s)ds Z z ?f z ?u =2 Z z0 ?f z0 ?l =2 ( s ) ds ( s ) ds ? = z0 ?f z0 ?u =2 z ?f z ?l =2 Z z ?f z ?u =2 Z z0 ?f z0 ?l =2 ( s ) ds + ( s ) ds z0 ?f z0 ?u =2 z ?f z ?l =2 2? (jzi0 ? zij + jfi(z0 ) ? fi(z)j): ( i
( i
i(
i(
)
( i
i)
)
( i
i)
( i
( i
i(
i(
( i
( i
)
i(
i(
)
)
i(
)
i)
)
i)
( i
i)
i(
i(
( i
i)
)
i(
( i
i)
( i
i)
)
i)
)
i(
i(
i)
)
)
i)
i)
2
This together with Lipschitz continuity of f implies (5.30) with some constant 3 > 0 as desired.
2
Lemmas 5.7 and 5.8 show that Assumption A0 implies Assumption A. We turn to nding a sucient condition for Assumption B to hold. We say z is a strictly complementary solution of MCP if it is a solution of MCP and satis es
zi ? li + fi(z ) > 0; zi ? ui + fi(z ) < 0; i = 1; 2; : : : ; n: Let
= 12 1min fjz ? l + f (z )j; jzi ? ui + fi(z )jg: in i i i
If (5.31) holds, then > 0. Let U (z ) be a neighbourhood of z such that
U (z) fz j jzi ? li + fi(z )j ; jzi ? ui + fi(z )j g: 21
(5.31)
Then H 0 is continuously di erentiable on U (z ). Moreover, it has been shown in [5] that for any z 2 U (z ) ( (5.32) rHi0(z) = rfi(z); if zi ? fi(z) 2 (li; ui ); ei ; if zi ? fi(z ) 62 [li ; ui ]: The following lemma gives sucient conditions for Assumption B to hold.
Lemma 5.9 Let Assumption A0 and condition (i) in Assumption B hold. Suppose that z is a
strictly complementary solution of MCP. Then Assumption B holds.
Proof The semismoothness of H follows from the continuous di erentiability of f . In particular, rH (z ) exists and is given by (5.32) with z = z. If zi ? fi(z ) 2 (li; ui ), then we get from (5.28) 0
0
and (5.32)
rHik (zk ) ? rHi0(z )
Z
1?
=
(s)ds (ei ? rfi(zk )) + rfi(zk ) ? rfi(z )
z ?f (zk ))i ?ui ]=2k
Z 1
=
z ?f (zk ))i ?li ]=2k
[( k
[( k
(s)ds + 2
Z
z ?f (zk ))i ?li ]=k + rfi (zk ) ? rfi (z ): [( k
z ?f (zk ))i ?ui ]=2k
[( k
?1
(s)ds (ei ? rfi(zk ))
This implies rHik (zk ) ? rHi0 (z ) ! 0. If zi ? fi (z ) 62 [li ; ui ], we also get from (5.28) and (5.32)
rHik (zk ) ? rHi0(z ) = ?
Z
z ?f (zk ))i ?li ]=2k
[( k
(s)ds (ei ? rfi (zk )):
z ?f (zk ))i ?ui ]=2k
[( k
This also implies rHik (zk ) ? rHi0 (z ) ! 0. Therefore, we have shown (4.1). Since k ! 0, we have for all k suciently large and any z 2 U (z )
Z z ?f z ?l =2?1 Z z ?f z ?l =2 z ?f z ?u =2 (s)ds ? z ?f z ?u =2 (s)ds ?1 Z z ?f z ?l =2 Z z ?f z ?u =2?1 ( s ) ds + ( s ) ds z ?f z ?l =2 ?1 z ?f z ?u =2 Z z ?f z ?l =2 1 Z z ?f z ?u =2?1 1 ds + ds z ?f z ?u =2 s z ?f z ?l =2 ?1 s ? ? = jz ?k?f (z ) ?k l j + jz ?k?f (z ) ?k u j i i i i i i 2? (k? ? k ); ( i
i(
( i
i(
)
)
i)
i)
( i
( i
i(
( i
2
1
i(
)
i)
i)
)
i(
i)
)
i)
k
i)
k
( i
i(
( i
k
i(
( i
k
2
i)
)
k
( i
k
2
2
1
2
)
)
i(
i(
( i
k
i(
( i
( i
k
1
i(
i(
)
)
i)
)
)
i)
k
i)
i)
k
k
2
k
2
2
1
where the second inequality follows from (5.26). This together with (5.28) implies (4.2). Similarly, we have for all k suciently large and any z; z 0 2 U (z )
Z z0 ?f z0 ?l =2 Z z ?f z ?l =2 z ?f z ?u =2 (s)ds ? z0 ?f z0 ?u =2 (s)ds ( i
( i
i(
i(
)
)
i)
i)
k
k
( i
( i
i(
i(
)
)
i)
i)
k
k
22
=
Z z0 ?f z0 ?l =2 Z z ?f z ?u =2 z ?f z ?l =2 (s)ds + z0 ?f z0 ?u =2 (s)ds Z z0 ?f z0 ?l =2 1 Z z ?f z ?u =2 1 z ?f z ?l =2 s ds + z0 ?f z0 ?u =2 s ds k z ? f 1(z ) ? l ? z0 ? f (1z 0 ) ? l + k z0 ? f (1z 0 ) ? u ? z ? f (1z ) ? u i i i i i i i i i i i i 2k ? (jzi ? zi0 j + jfi (z ) ? fi (z 0 )j): i(
( i
( i
i(
)
i(
( i
( i
i(
)
)
i)
i)
)
( i
k
2
k
i(
i(
( i
k
i)
i)
( i
k
( i
i(
i(
)
i)
)
)
)
k
i)
i)
i)
k
k
2
k
2
2
2
2
This together with (5.28) yields (4.3). Equations (4.4) follow from (5.27) and the fact k kH 0 (zk )k immediately. The proof is then complete. 2 From Lemma 5.9, we see that the following assumption together with Assumption A0 ensures the conditions of Assumption B. Assumption B000 (i) The sequence fzk g generated by Algorithm 1 converges to a strictly complementary solution z of the mixed complementarity problem. (ii) rH 0 (z ) is nonsingular. We now state a global and superlinear convergence theorem for Broyden-like method for MCP.
Theorem 5.4 Let Assumption A0 hold and the sequence fzk g be generated by Broyden-like method with H de ned by (5.25). Suppose that, for any accumulation point z of fzk g, matrices rH k (z ) are nonsingular for all suciently large k. Then we have lim kH 0 (zk )k = 0
k!1
and every accumulation point of fzk g is a solution of (5.16). If in addition, Assumption B000 holds, then fzk g converges superlinearly.
6 Discussion In this paper, we have presented a general Broyden-like method for solving semismooth equations including semismooth reformulations of nonlinear complementarity problems, variational inequality problems and mixed complementarity problems. We have established global and superlinear convergence theorems under suitable conditions. For nonlinear complementarity problems, global convergence only requires that F is a P0 function with Lipschitz continuous Jacobian. On the other hand, for variational inequality problems, global convergence needs the assumption that rH k (z ) is nonsingular for all k suciently large, as in the global convergence theorems established in [26], [30] and [35]. Although this condition might appear somewhat arti cial, it is implied, as shown in Lemma 5.5, by a condition that is reminiscent of the second order sucient condition in 23
nonlinear programming. Conditions that guarantee nonsingularity of rH for some smoothing functions related to the mixed complementarity problem can be found in [32]. Theorem 4.1 is an important contribution of the paper. It shows that superlinear convergence of a smoothing Broyden-like method can be established without assuming the strict complementarity condition at the limit point. This provides the possibility of developing a globally convergent Broyden-like method that enjoys superlinear convergence even if strict complementarity fails to hold.
References [1] B. Chen and P.T. Harker, Smooth approximations to nonlinear complementarity problems, SIAM J. Optim., 7 (1997), 403-420. [2] C. Chen and O.L. Mangasarian, A class of smoothing functions for nonlinear and mixed complementarity problems, Comput. Optim. Appl., 5 (1996), 97-138. [3] X. Chen, Superlinear convergence of smoothing quasi-Newton methods for nonsmooth equations, J. Comp. Appl. Math., 80 (1997), 105-126. [4] X. Chen and L. Qi, A parameterized Newton method and a quasi-Newton method for nonsmooth equations, Comput. Optim. Appl., 3 (1994), 157{179. [5] X. Chen, L. Qi and D. Sun, Global and superlinear convergence of the smoothing Newton method and its application to general box constrained variational inequalities, Math. Comput., 67 (1998), 519-540. [6] F.H. Clarke, Optimization and Nonsmooth Analysis, Wiley New York, 1983. [7] T. De Luca, F. Facchinei and C. Kanzow, A semismooth equation approach to the solution of nonlinear complementarity problems, Math. Programming, 75 (1996), 407-440. [8] J.E. Dennis, Jr. and J.J. More, A characterization of superlinear convergence and its application to quasi-Newton methods, Math. Comput., 28 (1974), 549-560. [9] J.E. Dennis, Jr. and J.J. More, Quasi-Newton methods, Motivation and the theory, SIAM Rev., 19 (1977), 46{89. [10] F. Facchinei and J. Soares, A new merit function for nonlinear complementarity problems and related algorithm, SIAM J. Optim., 7 (1997), 225-247. [11] A. Fischer, A special Newton-type optimization method, Optimization, 24 (1992), 269-284. 24
[12] A. Fischer, A Newton-type method for positive semide nite linear complementarity problems, J. Optim. Theory Appl., 86 (1995), 585-608. [13] R. Fletcher, Practical Methods of Optimization, Second Edition, John Wiley & Sons, Chichester, 1987. [14] M. Fukushima and L. Qi, eds., Reformulation: Nonsmooth, Piecewise Smooth, Semismooth and Smoothing Methods, Kluwer Academic Publishers, Boston, 1998. [15] S.A. Gabriel and J.J. More, Smoothing of mixed complementarity problems, M.C. Ferris and J.S. Pang, eds., Complementarity and Variational Problems: State of the Art, SIAM Philadelpha, Pennsylvania, 1997, pp. 105-116. [16] C. Geiger and C. Kanzow, On the solution of monotone complementarity problems, Comput. Optim. Appl., 5 (1996), 155-173. [17] C. M. Ip and J. Kyparisis, Local convergence of quasi-Newton methods for B-di erentiable equations, Math. Programming, 56 (1992), 71-89. [18] H. Jiang and L. Qi, A new nonsmooth equations approach to nonlinear complementarity problems, SIAM J. Control Optim., 35 (1997), 178-193. [19] C. Kanzow and H. Jiang, A continuation method for (strongly) monotone variational inequalities, Math. Programming, 81 (1998), 103-125. [20] C. Kanzow and M. Fukushima, Equivalence of the generalized complementarity problem to di erentiable unconstrained minimization, J. Optim. Theory Appl., 90 (1996), 581-603. [21] M. Kojima and S. Shindo, Extension of Newton and quasi-Newton methods to systems of PC 1 equations, J. Oper. Res. Soc. Japan, 29 (1986), 352{374. [22] D. Li, Global convergence of nonsingular Broyden's method for solving unconstrained optimization, Chinese J. Numer. Math. Appl., 17 (1995), No.4 85-94. [23] D. Li and M. Fukushima, A derivative-free line seach and global convergence of Broyden-like method for nonlinear equations, Optim. Methods Softw., to appear. [24] D. Li and M. Fukushima, Smoothing Newton and quasi-Newton methods for mixed complementarity problems, Comput. Optim. Appl., to appear. [25] D. Li and M. Fukushima, Broyden-like methods for variational inequality and nonlinear complementarity problems with global and superlinear convergence, Technical Report 98016, Department of Applied Mathematics and Physics, Kyoto University, September 1998. 25
[26] D. Li and Y. Yang, A globally convergent quasi-Newton method for nonlinear complementarity problems, Math. Programming, submitted. [27] R. Miin, Semismooth and semiconvex functions in constrained optimization, SIAM J. Control Optim., 15 (1977), 957-972. [28] F.A. Potra, L. Qi and D. Sun, Secant methods for semismooth equations, Numerische Mathematik, 80 (1998), 305{324. [29] L. Qi, Convergence analysis of some algorithms for solving nonsmooth equations, Math. Oper. Res., 18 (1993), 227-244. [30] L. Qi and X. Chen, A globally convergent successive approximation method for severely nonsmooth equations, SIAM J. Control Optim., 33 (1995), 402-418. [31] L. Qi and J. Sun, Nonsmooth equations and smoothing Newton methods, Technical Report, School of Mathematics, University of New South Wales, Sydney, April 1998. [32] L. Qi, D. Sun and G. Zhou, A new look at smoothing Newton methods for nonlinear complementarity problems and box constrained variational inequalities, AMR 97/13, Applied Mathematics Report, University of New South Wales, Sydney, June 1997. [33] D. Sun and J. Han, Newton and quasi-Newton methods for a class of nonsmooth equations and related problems, SIAM J. Optim., 7 (1997), 463-480. [34] N. Yamashita and M. Fukushima, Modi ed Newton methods for solving a semismooth reformulation of monotone complementarity problems, Math. Programming, 76 (1997), 469-491. [35] S. Zhou, D. Li and J. Zeng, A successive approximation quasi-Newton process for nonlinear complementarity problems, D.Z. Du, L. Qi and R.S. Womersley, eds., Recent Advances in Nonsmooth Optimization, World Scienti c Publishers, Singapore, 1995, pp. 459-472.
26
