On generalized semi-infinite optimization and ... - Semantic Scholar

European Journal of Operational Research 142 (2002) 444–462 www.elsevier.com/locate/dsw

Continuous Optimization

On generalized semi-infinite optimization and bilevel optimization Oliver Stein a, Georg Still b

b,*

a Department of Mathematics, RWTH Aachen, 52056 Aachen, Germany Faculty of Mathematical Sciences, University of Twente, P.O. Box 217, 7500 AE Enschede, Netherlands

Received 9 October 2000; accepted 7 September 2001

Abstract The paper studies the connections and differences between bilevel problems (BL) and generalized semi-infinite problems (GSIP). Under natural assumptions (GSIP) can be seen as a special case of a (BL). We consider the so-called reduction approach for (BL) and (GSIP) leading to optimality conditions and Newton-type methods for solving the problems. We show by a structural analysis that for (GSIP)-problems the regularity assumptions for the reduction approach can be expected to hold generically at a solution but for general (BL)-problems not. The genericity behavior of (BL) and (GSIP) is in particular studied for linear problems. Ó 2002 Elsevier Science B.V. All rights reserved. Keywords: Semi-infinite programming; Bilevel programming; Optimality conditions; Genericity behavior; Numerical methods

1. Introduction We consider generalized semi-infinite problems ðGSIPÞ:

min FGSIP ðxÞ x

s:t: x 2 MGSIP ¼ fx 2 Rn j Gðx; yÞ P 0 for all y 2 Y ðxÞg; where Y ðxÞ ¼ fy 2 Rm j gðx; yÞ P 0g:

For the special case, that the set Y ðxÞ ¼ Y does not depend on the variable x, this problem is a common semi-infinite problem (SIP). Bilevel problems are of the form ðBLÞ: min F ðx; yÞ s:t: Gðx; yÞ P 0 and y is a solution of x;y ð1Þ QðxÞ: min f ðx; yÞ s:t: y 2 Y ðxÞ y

with Y ðxÞ defined as in (GSIP). *

Corresponding author. Tel.: +31-53-489-34-04. E-mail address: [email protected] (G. Still).

0377-2217/02/$ - see front matter Ó 2002 Elsevier Science B.V. All rights reserved. PII: S 0 3 7 7 - 2 2 1 7 ( 0 1 ) 0 0 3 0 7 - 1

O. Stein, G. Still / European Journal of Operational Research 142 (2002) 444–462

445

Throughout the paper we assume FGSIP 2 CðRn ; RÞ; F 2 CðRn  Rm ; RÞ; G 2 CðRn  Rm ; Rp Þ, g 2 C ðR  Rm ; Rq Þ. We use the abbreviation J ¼ f1; . . . ; pg and I ¼ f1; . . . ; qg for the index sets of the constraints G and g. There is an extensive literature on bilevel optimization (see [16] and the references in this book). Semiinfinite programming (SIP) is an important field of research as well (cf. for example the survey article [7] with more than 300 references). Generalized semi-infinite problems are studied only recently (cf. e.g. [8,10,12,14,15,17–21,23]). Bilevel problems often arise as operations research problems in an economic context. They can be interpreted as a game between two players. Player 1 (upper level player) tries to minimize his object F depending on ðx; yÞ and player 2 (lower level player) who for given x chooses y as a solution of the lower level problem QðxÞ. Applications of (SIP) and (GSIP) mostly appear in technical sciences. For applications of (SIP) we refer to [7]. Applications of (GSIP) are e.g. the maneuverability problem in robotics (see [6]), the reverse Chebyshev approximation (see e.g. [9,20]) and time minimal control problems (see [12]). In this paper we will show that there is a strong connection between bilevel and generalized semi-infinite problems. Under certain assumptions (GSIP) can be seen as a special instance of a (BL). We will discuss the connections but also the differences between (GSIP) and (BL). The paper is organized as follows. In Section 2 we study the structure of the feasible sets of (BL) and (GSIP) and consider a natural condition under which (GSIP) becomes a special case of (BL). In Section 3 we apply the so-called local reduction approach. This technique leads to optimality conditions for (BL)- and (GSIP)-problems and gives the basis for (quasi-) Newton methods for solving the problems. We then ask whether the regularity assumptions used in the reduction approach are natural, i.e. assumptions which are generic. We analyze the difference between the structure of typical classes of (BL) problems and the class of (GSIP). It appears that for classes of bilevel problems the regularity assumptions are not generic. This shows that the regularity assumptions used in bilevel programming are often not valid at the solution of (BL). The structural discussion however leads to the conjecture that for (GSIP) the regularity assumptions for the ‘reduction’ can be expected to hold generically. In Section 4 we give a detailed analysis of linear problems and prove the genericity conjecture for linear generalized semi-infinite problems. Section 5 briefly describes the Kuhn–Tucker approach from bilevel programming for solving linear (GSIP) problems. n

2. Relations between GSIP- and BL-problems In this section we compare the structure of (GSIP) and (BL). We introduce some notations with x 2 Rn ; y 2 Rm : SðxÞ ¼ fy j y is a ðglobalÞ solution of QðxÞg S ¼ fðx; yÞ j y 2 SðxÞg MG ¼ fðx; yÞ j Gðx; yÞ P 0g X ¼ fx j Y ðxÞ 6¼ ;g ð dom Y Þ MBL ¼ fðx; yÞ j ðx; yÞ 2 MG ; y 2 SðxÞg

set of solutions ofQðxÞ; solution graph of Q; constrained set of upper level; domain of the mapping Y ; feasible set of ðBLÞ: m

on Rn , i.e. for any x 2 Rn We assume that the set-valued mapping Y : Rn ! 2R is uniformly compact S n there exists a ball Bq ðxÞ ¼ fx 2 R j kx  xk 6 qg; q > 0; such that closð x2Bq ðxÞ Y ðxÞÞ is compact. Then, under our assumptions, the mapping Y is closed and upper semi-continuous in the sense of Berge and the

446

O. Stein, G. Still / European Journal of Operational Research 142 (2002) 444–462

set X ð¼ dom Y Þ is a closed set. Moreover under this condition, the lower level problems QðxÞ; x 2 X , always have solutions. Now we try to transform GSIP into a problem of bilevel type. Let us define f ðx; yÞ :¼ min Gj ðx; yÞ;

ð2Þ

16j6p

where the minimum is taken over the components Gj of the vector-valued function G. Consider the parametric problem QðxÞ:

min f ðx; yÞ s:t: y 2 Y ðxÞ: y

Then, for x 2 Rn such that Y ðxÞ 6¼ ; we have f ðx; yÞ P 0 8y 2 Y ðxÞ, if and only if a solution y of QðxÞ (and thus all solutions) satisfies f ðx; yÞ P 0. Observe that f ðx; yÞ P 0 is equivalent with the inequality Gðx; yÞ P 0. So, by the relation between G and f, here for GSIP in contrast to the general BL-case, the feasibility condition Gðx; yÞ P 0 for the upper level holds for all solutions y of QðxÞ if it holds for one solution. Thus, for x with Y ðxÞ 6¼ ; the condition x 2 MGSIP is equivalent with x 2 prx ðMBLGSIP Þ, where MBLGSIP ¼ fðx; yÞ j Gðx; yÞ P 0 and y is a solution of QðxÞg: Here, prx denotes the orthogonal projection onto the space Rn (x-variable). Summarizing, the bilevel formulation of the generalized semi-infinite problem is given by BLGSIP

min FGSIP ðxÞ s:t: Gðx; yÞ P 0 and y is a solution of QðxÞ: min f ðx; yÞ s:t: y 2 Y ðxÞ;

ð3Þ

y

with the function f in (2). We have shown that if Y ðxÞ 6¼ ; holds for all x 2 Rn , then (GSIP) is equivalent with BLGSIP , i.e. the problem (GSIP) can be seen as a special instance of a (BL). Note however that for SðxÞ ¼ ;, x belongs to MGSIP , (no constraints for x) but not to prx ðMBLGSIP Þ. With the set ^ GSIP ¼ fx 2 dom Y j Gðx; yÞ P 0 for ðallÞ y 2 SðxÞg M

ð4Þ

c

^ GSIP [ ðdom Y Þ , where Ac denotes the complement of the set A in the correwe actually have MGSIP ¼ M sponding space. Thus, we have shown part (b) of the following lemma which provides different representations for the feasible sets of (GSIP) and (BL). Part (c) has been shown in [15], whereas part (a) follows directly from the definition. Lemma 1. The following holds. (a) MBL ¼ MG \ S. ^ GSIP [ ðdom Y Þc ¼ prx ðMBL Þ [ ðdom Y Þc . (b) MGSIP ¼ M GSIP (c) MGSIP ¼ ðprx ðMGc \ SÞÞc . In view of Lemma 1(a), since MG is closed, the set MBL is closed if S is closed. The set S is closed if the c mapping Y is (lower semi-) continuous on X ð¼ dom Y Þ. In view of Lemma 1(b), since ðdom Y Þ is open, the set MGSIP need not be closed, even when prx MBLGSIP is closed. For further details on the feasible set of (GSIP) we refer to [15]. Let be given ðx; yÞ; y 2 Y ðxÞ. We say that at y the Linear Independency Constraint Qualification (LICQ) is satisfied for the lower level problem QðxÞ if Dy gi ðx; yÞ;

i 2 Iðx; yÞ :¼ fi 2 I j gi ðx; yÞ ¼ 0g are linearly independent:

At y 2 Y ðxÞ, the weaker Mangasarian Fromovitz Constraint Qualification (MFCQ) is said to hold for QðxÞ if there exists a vector n such that

O. Stein, G. Still / European Journal of Operational Research 142 (2002) 444–462

Dy gi ðx; yÞn > 0

447

for all i 2 Iðx; yÞ:

The next lemma lists some standard sufficient conditions for the continuity of Y. Lemma 2. Under our assumptions on the set-valued mapping Y we have: (a) If the function gðx; yÞ ¼ Ax þ By  b is affine linear then Y is continuous on X. (b) Let U  X be open. Let for any x 2 U the function gðx; yÞ be convex in y and let for any x 2 U the Slater condition holds: There exists y ¼ yðxÞ 2 Rm such that gðx; yÞ > 0. Then Y is continuous in U. (c) Let U  X be open. Let for any x 2 U the condition (MFCQ) be fulfilled at all y 2 Y ðxÞ. Then Y is continuous in U. Lemma 2(a) shows that if g is affine linear then the feasible set MBL is closed (see also Theorem 1(a)). We give an example which shows that this need not be more true if g is not affine linear. Example 1. Consider the bilevel problem min F ðx; yÞ :¼ y  x

s:t: x 6 1 and y is a solution of QðxÞ : min f ðx; yÞ ¼ x  y s:t:  xy 6 0; 0 6 y 6 2:

Then,  Y ðxÞ ¼

f0g if x < 0; ½0; 2 if x P 0;

 and

SðxÞ ¼

f0g f2g

if x < 0; if x P 0:

These mappings are not continuous at the point x ¼ 0. We find MBL ¼ fðx; 0Þ j x < 0g [ fðx; 2Þ j 0 6 x 6 1g. Obviously, MBL is not closed and a global solution of (BL) does not exist. A local minimizer is ðx; yÞ ¼ ð1; 2Þ. A similar counterexample is given for (GSIP). Example 2. Consider the generalized semi-infinite problem min F ðx; yÞ :¼ x s:t: x 6 1 and Gðx; yÞ :¼ x  y P 0 for all y 2 Y ðxÞ where Y ðxÞ ¼ fy 2 R j  xy 6 0; 0 6 y 6 2g: Then, for the bilevel problem BLGSIP we find with the sets Y ðxÞ, SðxÞ in Example 1, MBLGSIP ¼ fðx; 0Þ j x < 0g and MGSIP ¼ prx MBLGSIP ¼ ð1; 0Þ: Again, MGSIP is not closed and a solution of (GSIP) does not exist. In view of these negative examples it seems natural to assume that Y is continuous on MBL . For (GSIP) we have to sharpen this condition slightly. Let be given a point x 62 dom Y , i.e. Y ðxÞ ¼ ;. Then, since ðdom Y Þc is open, around x the problem (GSIP) can be regarded as an unconstrained problem, i.e. around x the problem does not have the structure of a real ‘infinitely constrained problem’. Thus, to exclude this degenerate situation, in the sequel, we will assume that MGSIP  int X ;

where X ¼ dom Y :

ð5Þ

This condition can always be satisfied by adding to the original constraints G P 0 appropriate extra conditions (such as jxi j 6 q). Remember that by the discussion above, assumption (5) implies that the problem (GSIP) is a (BL) problem with the special structure that lower level object function coincides with the upper level constraint. In addition to (5) we assume that Y ðxÞ is continuous on MGSIP . A natural sufficient condition for the continuity of Y is the condition (see Lemma 2(b)): For some open set U we have MGSIP  U and

448

O. Stein, G. Still / European Journal of Operational Research 142 (2002) 444–462

for all x 2 U ;

ðMFCQÞ is satisfied for all y 2 Y ðxÞ:

ð6Þ

This condition also implies (5). In the following we are going to compare the structure of a general (BL) with the structure of a (BLGSIP ) satisfying (6).

3. Local reduction approach A possible theoretical and practical approach for solving (BL) and (GSIP) is the so-called local reduction. For (SIP) and (GSIP) this is a standard approach introduced in [5] (see also [7,20]). It is also used in bilevel programming (see e.g. [2]). The idea is to transform the problem locally into a common finite optimization problem. Such a transformation is possible if certain regularity assumptions hold. Under these assumptions we obtain a system of optimality conditions for a minimizer of (BL) and (GSIP) and the solution can be computed by applying a (quasi-) Newton method for solving this system of equations. In the present section we derive the optimality conditions and discuss the question as to whether the regularity assumptions are natural conditions which can be expected to hold at the solution in the generic case. 3.1. Local reduction for general (BL) Let be given x 2 Rn ; y 2 SðxÞ; ðx; yÞ 2 MG , i.e. ðx; yÞ is feasible for (BL). Let the following assumption hold: A1BL

There exist a neighborhood U ðxÞ of x and a C 1 -function y : U ðxÞ ! Rm such that yðxÞ ¼ y and for any x 2 U ðxÞ the vector yðxÞ is the (unique, global) solution of QðxÞ.

The following is a standard assumption in parametric optimization and sufficient for A1BL . (This assumption is often used in nonlinear bilevel programming see e.g. [2, Assumption A2].) A2BL

All problem functions of (BL) are C 2 -functions and at the unique solution y of QðxÞ we have (1) (LICQ) is satisfied and the Kuhn–Tucker condition with multipliers ci > 0 (strict complementary slackness) X Dy Ly ðx; y; cÞ :¼ Dy f ðx; yÞ  ci Dy gi ðx; yÞ ¼ 0: i2Iðx;yÞ

(2) The standard second-order sufficiency optimality condition: The Hessian D2y Ly ðx; y; cÞ is positive definite on the tangent space fn 2 Rq jDy gi ðx; yÞn ¼ 0; i 2 Iðx; yÞg. Obviously, under A1BL , on U ðxÞ, the problem (BL) is equivalent with the so-called locally reduced problem BLx :

min F^ðxÞ :¼ F ðx; yðxÞÞ x

^ðxÞ :¼ Gðx; yðxÞÞ P 0: s:t: G

BLx is a finite optimization problem and standard optimality conditions applied to this problem lead to optimality conditions for (BL) as follows. ^j ðxÞ ¼ Dx Gj ðx; yÞ þ Dy Gj ðx; yÞg; j 2 J ðxÞ :¼ fj 2 f1; . . . ; pg j Gj ðx; yÞ ¼ Suppose, the active gradients DG 0g are linearly independent, where we set g ¼ DyðxÞ. Then, a necessary optimality condition for ðx; yÞ to solve (BL) is: There exist multipliers lj P 0 such that

O. Stein, G. Still / European Journal of Operational Research 142 (2002) 444–462

DF^ðxÞ 

X

^j ðxÞ ¼ Dx F ðx; yÞ  lj DG

j2J ðxÞ

X

lj Dx Gj ðx; yÞ þ

j2J ðxÞ

Dy F ðx; yÞ 

X

449

! lj Dy Gj ðx; yÞ g ¼ 0:

ð7Þ

j2J ðxÞ

Consider now the Kuhn–Tucker conditions for a solution y ¼ yðxÞ of QðxÞ Dy Ly ðx; y; cÞ ¼ 0; gi ðx; yÞ ¼ 0;

H ðx; y; cÞ :¼

ð8Þ

i 2 Iðx; yÞ:

Under assumption A2BL , by applying the implicit function theorem to H ¼ 0, it follows that there exist a neighborhood U ðxÞ of x and C 1 -functions yðxÞ; cðxÞ such that H ðx; yðxÞ; cðxÞÞ ¼ 0; x 2 U ðxÞ and yðxÞ is the solution function in A1BL with corresponding multiplier cðxÞ. By differentiating the relation H ðx; yðxÞ; cðxÞÞ ¼ 0 w.r.t. x we find Dx H ðx; y; cÞ þ Dy H ðx; y; cÞg þ Dc H ðx; y; cÞh ¼ 0; where h ¼ DcðxÞ. Altogether, in view of (7) and (8), we obtain the following system of equations for a solution ðx; yÞ of (BL) and the corresponding multipliers and their derivatives ðl; c; g; hÞ: X X Dx F ðx; yÞ  lj Dx Gj ðx; yÞ þ ðDy F ðx; yÞ  lj Dy Gj ðx; yÞÞg ¼ 0; j2J ðxÞ

Gj ðx; yÞ ¼ 0;

j2J ðxÞ

j 2 J ðxÞ;

ð9Þ

y

Dy L ðx; y; cÞ ¼ 0; gi ðx; yÞ ¼ 0; i 2 Iðx; yÞ; Dx H ðx; y; cÞ þ Dy H ðx; y; cÞg þ Dc H ðx; y; cÞh ¼ 0:

This is a system of n þ m þ jJ ðxÞj þ jIðx; yÞj þ nm þ njIðx; yÞj equations for the same number of unknowns x; y; l; c; g; h. To compute a solution ðx; yÞ of (BL), we could apply a (quasi-) Newton procedure for solving system (9). We are now interested in knowing whether assumption A2BL – essential for the local reduction – is a natural condition. We first give an illustrative example. Example 3. Consider the bilevel problem P1 :

max F ðx; yÞ :¼ x þ y

s:t: Gðx; yÞ :¼ x þ 2y  8 6 0 and y solves QðxÞ: max f ðx; yÞ :¼ x þ y s:t: 0 6 y 6 4:

Here, SðxÞ ¼ fyðxÞ ¼ 4g and the feasible set is MBL ¼ fðx; 4Þ j x 6 0g. The optimal solution yðxÞ ¼ 4 of QðxÞ is feasible w.r.t. G P 0 only for x 6 0. Hence, the solution of (BL) is ðx; yÞ ¼ ð0; 4Þ with value F ðx; yÞ ¼ f ðx; yÞ ¼ 4. It is easily checked that at the solution yðxÞ ¼ 4 of QðxÞ the conditions of A2BL (1) are fulfilled (the second-order condition (2) is superfluous since QðxÞ is a linear program). Now let us consider the problem P2 obtained from P1 by only moving the condition G P 0 to the lower level P2 :

max F ðx; yÞ ¼ x þ y

s:t: y solves QðxÞ: max f ðx; yÞ ¼ x þ y s:t: 0 6 y 6 4; x þ 2y  8 6 0:

x; y^Þ ¼ ð8; 0Þ is Then for P2 also the points fðx; yÞ j y ¼ 4  ðx=2Þ; x 2 ½0; 8g become feasible and the point ð^ optimal (with a better value of F ð^ x; y^Þ ¼ f ð^ x; y^Þ ¼ 8). However now, for P2 the solution y^ ¼ 0 of Qð^ xÞ does not fulfill the assumptions A2BL (1). The point y^ ¼ 0 is a degenerate vertex solution of the linear program Qð^ xÞ. At y^ ¼ 0, even the (MFCQ) is not valid for Qð^ xÞ.

450

O. Stein, G. Still / European Journal of Operational Research 142 (2002) 444–462

Remark. By definition of (LICQ), the assumption A2BL ð1Þ can only hold if there are not more than m lower level constraints active at a solution ð^ x; y^Þ of (BL). We will see in Section 4 for the linear case (cf. Theorem 2) that the number of active lower level constraints is directly related to the number of active upper level constraints. In fact, as in the preceding examples, it will appear that (LICQ) is not valid in the lower level if there are not enough ( P n) upper level constraints active at ð^ x; y^Þ. In this case the ‘reduction approach’ is not possible. Note that a typical bilevel model in operations research need not have any upper level constraints. In fact viewing a (BL) in (1) as a game between an upper level player 1 and a lower level player 2, the player 2 could accept the upper level constraints G P 0 in his lower level problem QðxÞ. In the example above, the strategy to pass the upper level constraints to the lower level problem even leads to a better object value for both players. The next lemma shows that for the upper level player such a policy is always an advantage (for the lower level player it may be advantageous but also unfavorable depending on whether his object is ‘similar’ f denote the bilevel problem obtained from (BL) by passing or ‘adverse’ to the upper level object). Let ( BL BL) e BL denote the corresponding feasible set. the constraints G P 0 to the lower level constraints g P 0 and let M e BL and for the solutions ðx; yÞ of (BL), Lemma 3. Let be given a bilevel problem (1). Then we have MBL  M f respectively, it follows F ð~x; y~Þ 6 F ðx; yÞ. ð~x; y~Þ of ( BL BL), Proof. Let ðx; yÞ be feasible for (BL), i.e. y is a solution of QðxÞ and Gðx; yÞ P 0. Then y is also feasible for f the lower level problem of ( BL BL), e ðxÞ: Q

min f ðx; yÞ y

s:t: Gðx; yÞ P 0; gðx; yÞ P 0:

e ðxÞ is contained in the feasible set Y ðxÞ of QðxÞ, y must also be a solution of Since the feasible set Ye ðxÞ of Q e f Q ðxÞ, i.e. ðx; yÞ is feasible for ( BL BL).  The situation for (GSIP) and its bilevel formulation BLGSIP (see (3)) is quite different. Let x be feasible for (GSIP). We define the set of active points   Y0 ðxÞ ¼ y 2 Y ðxÞ j min Gj ðx; yÞ ¼ 0 : 16j6p

Note that x is feasible if and only if Gðx; yÞ P 0 for all y 2 Y ðxÞ. Thus for x 2 MGSIP every point y 2 Y0 ðxÞ is a global solution of QðxÞ. Suppose now that x is a solution of (GSIP) and that Y0 ðxÞ ¼ ;. Then by continuity assumptions in Section 1, near x the problem (GSIP) is equivalent with the unconstrained problem minx FGSIP ðxÞ. We exclude such a situation by assuming that in addition to (5) the following holds: Y0 ðxÞ 6¼ ;

for any local solution x of ðGSIPÞ:

Consequently, the bilevel problems BLGSIP related to generalized semi-infinite problems intrinsicly have at least one upper level constraint active in the solution. Moreover, typically in (GSIP) the degree of freedom in the variable x ‘forces’ the solution x of BLGSIP to a location such that as many active points y l 2 Y0 ðxÞ occur, (i.e. solutions y l of QðxÞ) as the degree of freedom in the minimization model allows. This behavior is illustrated with the following geometrical interpretation of (GSIP) (cf. also [20]). Given the region MG ¼ fðx; yÞ j Gðx; yÞ P 0g in Rn  Rm . Then we have to find x such that the set fxg  Y ðxÞ is contained in MG and such that some functional f ðxÞ is maximized. Often the function f ðxÞ can be viewed as the volume of the set Y ðxÞ. Fig. 1 illustrates the situation of such a problem at the solution x of (GSIP). The different points y 1 ; y 2 ; y 3 where the set Y ðxÞ touches the set fy j Gðx; yÞ ¼ 0g are the solutions of the lower level problem

O. Stein, G. Still / European Journal of Operational Research 142 (2002) 444–462

451

Fig. 1. Illustration of a (GSIP) at the solution x.

QðxÞ, i.e. Y0 ðxÞ ¼ fy 1 ; y 2 ; y 3 g. So typically in semi-infinite optimization we have to admit different solutions of QðxÞ. 3.2. Local reduction for (GSIP) In semi-infinite optimization the local reduction is a standard technique. As motivated above, because of the special structure of (GSIP), in contrast to the general (BL) case, at a solution x of (GSIP) typically different solutions of QðxÞ must be considered, i.e. Y0 ðxÞ need not to be a singleton. Let be given x 2 MGSIP and let Y0 ðxÞ consist of finitely many points, Y0 ðxÞ ¼ fy 1 ; . . . ; y r g, r P 1. We make the following assumption: A1GSIP

There exist a neighborhood U ðxÞ of x and r C 1 -functions y l : U ðxÞ ! Rm , such that y l ðxÞ ¼ y l ; and for any x 2 U ðxÞ the values y l ðxÞ; l ¼ 1; . . . ; r, include all (global) solution of QðxÞ.

As in the (BL) case we give a natural sufficiency condition for A1GSIP . A2GSIP

All problem functions of (GSIP) are C 2 -functions. Let for y 2 Y0 ðxÞ be defined Jy :¼ fj 2 J j Gj ðx; yÞ ¼ 0g and for j 2 Jy Qj ðxÞ:

min Gj ðx; yÞ s:t: y 2 Y ðxÞ:

For all y 2 Y0 ðxÞ and allPj 2 Jy we have with the Lagrange function Lyj ðx; y; cÞ ¼ Gj ðx; yÞ  i2Iðx;yÞ ci gi ðx; yÞ: (1) (LICQ) is satisfied at y for Qj ðxÞ together with the Kuhn–Tucker condition Dy Lyj ðx; y; cÞ ¼ 0 with multipliers cji > 0 (strict complementary slackness). (2) The standard second-order sufficiency optimality condition at y for Qj ðxÞ: The Hessian D2y Lyj ðx; y; cÞ is positive definite on fn 2 Rq j Dy gi ðx; yÞn ¼ 0; i 2 Iðx; yÞg.

452

O. Stein, G. Still / European Journal of Operational Research 142 (2002) 444–462

Under A2GSIP the set Y0 ðxÞ must be finite, Y0 ðxÞ ¼ fy l ; l ¼ 1; . . . ; rg, and locally in a neighborhood U ðxÞ of x, the problem (GSIP) is equivalent with the locally reduced problem GSIPx :

min FGSIP ðxÞ

^l ðxÞ :¼ Gj ðx; y l ðxÞÞ P 0; j 2 J l ; l ¼ 1; . . . ; r: s:t: G j y

Again GSIPx is a finite optimization problem and optimality conditions of finite optimization applied to this problem lead to optimality conditions for (GSIP). We only give the conditions for the case p ¼ 1 (see also [8,20]; the modification to the case p > 1 is straightforward). Similar to (7) with Y0 ðxÞ ¼ fy l ; l ¼ 1; . . . ; rg we obtain for p ¼ 1 the optimality condition (putting l y ¼ y l ðxÞ) DFGSIP ðxÞ 

r X

 ll Dx Gðx; y l Þ þ Dy Gðx; y l ÞDy l ¼ 0

ð10Þ

l¼1

with multipliers ll P 0. For (GSIP) however the equations simplify. Consider the Kuhn–Tucker condition for QðxÞ at the solutions y l : X cli ðxÞDy gi ðx; y l Þ Dy Gðx; y l Þ ¼ i2Iðx;y l Þ

and gi ðx; y l Þ ¼ 0; i 2 Iðx; y l Þ. By differentiating the relation gi ðx; y l ðxÞÞ ¼ 0 we find Dx gi ðx; y l Þ ¼ Dy gi ðx; y l ÞDy l and X X cli Dy gi ðx; y l ÞDy l ¼  cli Dx gi ðx; y l Þ: Dy Gðx; y l ÞDy l ¼ i2Iðx;y l Þ

i2Iðx;y l Þ

Substituting in (10) we obtain the following system of optimality conditions: 0 1 r X X DFGSIP ðxÞ  ll @Dx Gðx; y l Þ  cli Dx gi ðx; y l ÞA ¼ 0; i2Iðx;y l Þ

l¼1

Gðx; y l Þ ¼ 0;

l ¼ 1; . . . ; r; ð11Þ

and for l ¼ 1; . . . ; r; X j Dy Gðx; y l Þ  ci Dy gi ðx; y l Þ ¼ 0; i2Iðx;y l Þ

gi ðx; y l Þ ¼ 0;

i 2 Iðx; y l Þ:

This system consists of K :¼ n þ r þ

r X

m þ jIðx; y l Þj



l¼1 l

equations for the K unknowns x 2 Rn ; ll 2 R; y l 2 Rm ; cl 2 RjIðx;y Þj ; l ¼ 1; . . . ; r. In [22] it has been shown that under the assumption A2GSIP at a solution x of (GSIP) and the additional assumptions that the gradients 0 1 X j @Dx Gðx; y l Þ  ci Dx gi ðx; y l ÞA; l ¼ 1; . . . ; r; i2Iðx;y l Þ

O. Stein, G. Still / European Journal of Operational Research 142 (2002) 444–462

453

are linearly independent, the Jacobian of (11) is regular at the solution ðx; y 1 ; c1 ; . . . ; y r ; cr Þ. Hence, to solve (GSIP) numerically, we can apply a (quasi-) Newton method to (11). See e.g. [7] for solving common (SIP) problems by Newton methods. Note that in contrast to Eq. (9) for the (BL)-problems the optimality condition (11) for BLGSIP does not contain the derivatives Dy l ; Dcl as unknowns. The reason is that for the problem BLGSIP the upper level constraints Gj coincide with the lower level objects. So only the information of the value function of QðxÞ is really needed in the upper level and not the full information about the solution yðxÞ. We are now going to discuss the question whether the assumption A2BL or A2GSIP for the local reduction can be expected to hold generically at a solution. By a generic subset S of a problem set P we roughly mean a subset which is open and dense in P (in some appropriate topology). For the problem P2 of Example 3 the assumption (LICQ) in A2BL (even (MFCQ)) is not valid. This negative behavior is stable w.r.t. smooth nonlinear (small) perturbations. Hence we can state. For the general class of (BL) problems the assumption A2BL is not generic at a local solution ðx; yÞ. For typical classes of bilevel problems, in particular problems without upper level constraints, (LICQ) or even (MFCQ) will not be satisfied at the solution y of QðxÞ. For such problems we cannot expect a ‘nice’ system of optimality conditions for ðx; yÞ which can be solved with smooth methods. Consequently in this situation the ‘reduction approach’ can only be used with caution. As indicated above, the special class of bilevel problems BLGSIP related to (GSIP) may have a better genericity behavior. For the sub-class of common semi-infinite problems it has been shown in [11] that A2GSIP is generically fulfilled at each local solution. A similar genericity analysis for (GSIP) has not yet been done. In [17] some particular results are obtained. It has been proven for example that generically for (GSIP) the number jY0 ðxÞj of lower level local minima at a solution x is bounded by n, jY0 ðxÞj 6 n. We will show for the linear case in the following section that generically jY0 ðxÞj ¼ n holds. We formulate the following: Conjecture. In the class BLGSIP (appropriately defined) the assumption A2GSIP holds generically at a solution x of a (GSIP) problem. In particular generically, all local minima y l ; l ¼ 1; . . . ; r, of QðxÞ are nondegenerate minima. In the following section this conjecture is proven for the special case of linear problems (see Theorem 3). We also will present a detailed analysis of the negative results for the general class of linear bilevel problems. Summarizing, roughly speaking, generically for classes of general (BL) problems from operational research, at a solution ðx; yÞ, the minimizer y of QðxÞ will be a unique minimizer but (LICQ) (or even (MFCQ)) will not be satisfied at y. In contrast, for (GSIP) generically we expect at a solution different minimizer y l of QðxÞ but each solution will be non-degenerate, such that a smooth approach for solving (GSIP) is possible. To analyze the difference between (BL) and (GSIP) in the following section, we have to modify the bilevel formulation of (GSIP) in (3). In formulation (3), when the upper level contains different (smooth) constraints, i.e. if p > 1, then the object function f ðx; yÞ ¼ min1 6 j 6 p Gj ðx; yÞ is not a C 1 -function (only Lipschitz-continuous). To transform (GSIP) into a smooth (BL) we consider the following generalization of the bilevel problem (3): ðBLÞ:

min F ðx; yÞ x;y

s:t: Gðx; y1 ; . . . ; yr Þ P 0 and for l ¼ 1; . . . ; r; yl solves Ql ðxÞ: min fl ðx; yl Þ s:t: gl ðx; yl Þ P 0; yl

ð12Þ

454

O. Stein, G. Still / European Journal of Operational Research 142 (2002) 444–462

where x 2 Rn , y ¼ ðy1 ; . . . ; yr Þ; yl 2 Rml ; l ¼ 1; . . . ; r, and the functions are defined accordingly. This problem can be seen as a game between an upper level player and r different lower level players with r different lower level problems Ql . The problem BLGSIP (and (GSIP)) can be written as ðBLGSIP Þ:

min FGSIP ðxÞ

s:t: Gl ðx; yl Þ P 0; l ¼ 1; . . . ; p; where yl solves Ql ðxÞ: min Gl ðx; yl Þ s:t: gðx; yl Þ P 0:

ð13Þ

yl

This represents a bilevel problem (12) with the special conditions fl ¼ Gl , ml ¼ m, r ¼ p and gl ¼ g not depending on l. 4. Linear problems In this section we are concerned with linear (GSIP) and (BL), i.e. all problem functions are affine linear. We describe the structure of the feasible sets and analyze which kind of regularity can be expected at a solution ðx; yÞ of (BL) or at a solution x of (GSIP). We will show that a general (BL) and the special case of a bilevel problem BLGSIP arising from (GSIP) may have different generic behaviors. We consider the following linear bilevel problem (cf. (12)) with y ¼ ðy1 ; . . . ; yr Þ 2 Rm0 , yl 2 Rml ; l ¼ 1; . . . ; r; m0 ¼ m1 þ    þ mr : ðLBLÞ:

min cT0 x þ d0T y

s:t: A0 x þ B0 y  b0 P 0 and for l ¼ 1; . . . ; r; yl is a solution of

ð14Þ

Ql ðxÞ: min cTl x þ dlT yl s:t: Al x þ Bl yl  bl P 0: yl

n

m0

Here, c0 2 R ; d0 2 R ; A0 is a p  n-matrix, B0 is a p  m0 -matrix, Al are ql  n-matrices, Bl are ql  ml matrices, etc. A linear (GSIP) is of the form ðLGSIPÞ:

min cT0 x s:t: A0 x þ B0 y  b0 P 0 for all y in YðxÞ where Y ðxÞ ¼ fy 2 Rm j Ax þ By  b P 0g:

Let in the sequel a0l ; b0l denote the rows of A0 ; B0 , respectively, and let b0l be the components of b0 . Then (LGSIP) can be written in the form of an (LBL): ðBLLGSIP Þ:

min cT0 x

T

T

s:t: Gl ðx; yÞ :¼ ða0l Þ x þ ðb0l Þ yl  b0l P 0; l ¼ 1; . . . ; p; and for l ¼ 1; . . . ; p; yl is a solution of T

T

Ql ðxÞ: min ða0l Þ x þ ðb0l Þ yl s:t: Ax þ Byl  b P 0: yl

with Al ¼ A; Bl ¼ B; bl ¼ b not depending on l and p ¼ r. We have to complete our notation. With y ¼ ðy1 ; . . . ; yr Þ we define for l ¼ 1; . . . ; r: Yl ðxÞ ¼ fyl 2 Rml ; ðy 2 Rm0 Þ j Al x þ Bl yl  bl P 0g Yl ¼ fðx; yl Þ; ððx; yÞÞ j yl 2 Yl ðxÞg Sl ðxÞ ¼ fyl 2 Rml ; ðy 2 Rm0 Þ j yl solves Ql ðxÞg MG ¼ fðx; yÞ j Gðx; yÞ :¼ A0 x þ B0 y  b0 P 0g X ¼ \l dom Yl ð¼ \l fx j Yl ðxÞ 6¼ ;gÞ Msem ¼ MG \ ð\l Yl Þ S ¼ fðx; yÞ j yl 2 Sl ðxÞ; l ¼ 1; . . . ; rg MBL ¼ fðx; yÞ j ðx; yÞ 2 Msem \ Sg

feasible set of Ql ðxÞ; the graph of Yl ðxÞ; set of solutions of Ql ðxÞ; upper level constraints; the semi-feasible set; the solution graph; feasible set of ðLBLÞ:

O. Stein, G. Still / European Journal of Operational Research 142 (2002) 444–462

455

We will regard the sets Yl ðxÞ; Sl ðxÞ as sets in Rml or as sets in Rm0 depending on the context. We introduce the following assumptions: AL1 AL2

The sets Sl ðxÞ are compact subsets of Rml , l ¼ 1; . . . ; r, for all x 2 Rn . This assumption in particular implies that Ql ðxÞ always has a vertex solution (if Sl ðxÞ 6¼ ;). The polyhedron Msem  Rn  Rm0 is bounded (thus compact).

Recall that for Y ðxÞ the Slater condition is said to hold if there exists y~ ¼ y~ðxÞ 2 Rm such that Ax þ B~ y  b > 0 (see Lemma 1(b)). In view of our regularity assumption (6) for (LGSIP) we consider the following assumption: AL3

For (LGSIP) let the Slater condition be satisfied for all x 2 MGSIP . (Then in particular, (LGSIP) is equivalent with BLLGSIP .)

The following theorem contains the main results on the structure of the feasible set and the solution of an (LBL). Theorem 1. For (LBL) the following holds. (a) The feasible set MBL ¼ Msem \ S consists of a union of finitely many faces f k of the polyhedron Msem MBL ¼

K [

f k:

k¼1

In particular, MBL is a closed set in Rn  Rm0 . (b) If no upper level constraints are present, then the set MBL is pathwise connected. If the assumptions AL1 and AL2 hold then we have (c) The solution of (LBL) occurs at a vertex of some face f k0 ; k0 2 f1; . . . ; Kg, and thus at a vertex of Msem . (d) The value functions vl ðxÞ :¼ minyl 2Yl ðxÞ cTl x þ dlT yl of Ql ðxÞ, l ¼ 1; . . . ; r, are convex and Lipschitzcontinuous on dom Yl .

Proof. (For a proof of (a) and (c) for the case r ¼ 1 we refer e.g. to [16].) For completeness we give a proof for the general case r P 1. (a) The set Msem is a polyhedron in Rn  Rm0 . Let f 0 be a d-dimensional face of Msem , with d P 1 (i.e. f 0 is not a vertex). Let ðx0 ; y 0 Þ be a point in the relative interior of f 0 and let ðx0 ; y 0 Þ belong to MBL . We now show that the whole face f 0 belongs to MBL . Then, since every point of Msem which is not a vertex is contained in the relative interior of some face of Msem , the proof is completed. Let ðx1 ; y 1 Þ 2 f 0 be arbitrary. Since ðx0 ; y 0 Þ is a point in the relative interior of f 0 there exists a point 2 2 ðx ; y Þ in f 0 such that with some k; 0 < k < 1 ðx0 ; y 0 Þ ¼ kðx1 ; y 1 Þ þ ð1  kÞðx2 ; y 2 Þ: T Given any point ðx1 ; yÞ in l Yl , i.e. y l 2 Yl ðx1 Þ, we consider ðx0 ; y  Þ :¼ kðx1 ; yÞ þ ð1  kÞðx2 ; y 2 Þ: It follows yl 2 Yl ðx0 Þ. Since ðx0 ; y 0 Þ 2 MBL , i.e. yl0 are solutions of Ql ðx0 Þ, in view of (15) we have dlT yl0 ¼ kdlT yl1 þ ð1  kÞdlT yl2 6 dlT yl ¼ kdlT y l þ ð1  kÞdlT yl2 and dlT yl1 6 dlT y l . This implies that yl1 are solutions of Ql ðx1 Þ and ðx1 ; y 1 Þ 2 MBL .

ð15Þ

456

O. Stein, G. Still / European Journal of Operational Research 142 (2002) 444–462

(b) See [22] for r ¼ 1. The generalization to r > 1 is not difficult. (c) In view of (a) the feasible set MBL consists of the union of faces f 1 ; . . . ; f K of the compact set Msem (cf. AL2 ). A solution of (LBL) exists and must be contained in some of these faces say f k0 . Thus the problem (LBL) can be replaced by the problem min cT0 x þ d0T y x;y

s:t: ðx; yÞ 2 f k0 :

Since f k0 is a bounded polyhedron the minimum is attained at a vertex ðx; yÞ of f k0 . Since f k0 is a face of Msem the point ðx; yÞ is also a vertex of Msem . (d) See [16].



Note that the set MLBL , as the union of faces of the polyhedron Msem , is typically a non-convex set. The same holds for prx MBLLGSIP which may have re-entrant corners (see e.g. [17]). From Theorem 1 in view of Lemma 1(b) and using AL3 we directly obtain the following corollary for ðBLLGSIP Þ. Recall that FGSIP does not depend on y and that MLGSIP is a subset of Rn . Corollary 1. Let be given (LGSIP) satisfying AL3 . Then the feasible set MLGSIP ¼ prx MBLLGSIP is the subset of the polyhedron prx Msem given by a union of polyhedra prx f k ; k ¼ 1; . . . ; K. In particular, MLGSIP is closed. Proof. We have only to note that the projections of the polyhedra Msem , f k are again polyhedra.



We are now going to describe the structural difference between a general (LBL) and a problem (BLLGSIP ). Theorem 2. Let ðx; yÞ be a vertex solution of (LBL) in (14), i.e. ðx; yÞ is a vertex of Msem . Suppose, for the number p of upper level constraints we have p < n. Then at least one of the solutions y l of Ql ðxÞ, say y l0 , does not fulfill the condition (LICQ) for Ql0 ðxÞ (or even not (MFCQ)), i.e. y l0 is a degenerate vertex solution of the linear problem Ql0 ðxÞ. Proof. If (LICQ) is satisfied at y l 2 Rml for Ql ðxÞ, then at most ml of the inequalities Al x þ Bl y l  bl P 0 can be active (l ¼ 1; . . . ; r). Together with maximally p activePconstraints in the upper level, the number of active constraints for ðx; yÞ is less than or equal to p þ rl¼1 ml ¼ p þ m0 < n þ m0 . Consequently ðx; yÞ cannot be a vertex of the polyhedron Msem in Rn  Rm0 .  In view of Theorem 2, when the number p of constraints G P 0 in the upper level is too small, the regularity assumption A2BL (1) in Section 2 cannot hold. (Note that this situation is stable under small smooth nonlinear perturbations.) In the extreme case, a general (BL) may have no constraints in the upper level (i.e. p ¼ 0.) By definition, as we have discussed in Section 2, a BLGSIP always has at least one upper level constraint. This difference makes the generalized semi-infinite problems behave better. We give an illustrative example. Consider the (BL) without constraints in the upper level max x þ y

s:t: y is a solution of QðxÞ: max 2y s:t: 0 6 y 6 12x; 2y þ x  1 6 0: y

Here, the feasible set MLBL is given by the union f 1 [ f 2 of faces f 1 ¼ fðx; yÞ j y ¼ 12x; 0 6 x 6 12g and f 2 ¼ fðx; yÞ j 2y þ x  1 ¼ 0; 12 6 x 6 1g of Msem . The solution is attained at the vertex ðx; yÞ ¼ ð1; 0Þ. At the solution y of the one-dimensional problem QðxÞ two lower level constraints y ¼ 0; 2y þ x  1 ¼ 0 are

O. Stein, G. Still / European Journal of Operational Research 142 (2002) 444–462

457

active. Thus, y is a degenerate vertex of Y ðxÞ. Since Y ðxÞ ¼ fyg consists only of one point, the Slater condition (MFCQ) is not valid. Consider a similar (LGSIP) problem. max x xP0

s:t: 2y þ x  1 6 1 for all y 2 Y ðxÞ :¼ fyj0 6 y 6 12xg

with the bilevel formulation max x xP0

s:t: 2y þ x  1 6 0 and y is a solution of QðxÞ: max 2y s:t: 0 6 y 6 12x: y

(The condition x P 0 is added to yield the assumption AL3 .) Here, the feasible set MBLLGSIP consists of the face f 1 ¼ fðx; yÞ j y ¼ 12x; 0 6 x 6 12g of Msem . The solution of BLLGSIP is attained at the vertex ðx; yÞ ¼ ð12; 12Þ. In contrast to the solution of the (LBL) above, here, at the solution y of QðxÞ only one lower level constraint y ¼ 12x is active and y is a non-degenerate vertex of QðxÞ. The solution x ¼ 12 of (GSIP) is a vertex of the feasible set MLGSIP ¼ prx f 1 ¼ ½0; 12. We now show that the regularity properties of this example hold generically in (LGSIP). We have to introduce some definitions and facts from genericity theory. Firstly we define the problem set for (LBL) and BLLGSIP . Let us fix the vector s ¼ ðn; r; p; m1 ; q1 ; . . . ; mr ; qr Þ. A problem (LBL) in (14) can be seen as an element from Ps ¼ fP ¼ ðAl ; Bl ; bl ; cl ; dl ; l ¼ 0; . . . ; rÞg; where the dimensions of Al ; Bl ; etc. are defined by s. The set Ps can be identified with RK , where r X K :¼ ðn þ 1Þp þ n þ ðp þ 2Þml þ ðn þ ml þ 1Þql : l¼1

For BLLGSIP in view of Al ¼ A; Bl ¼ B; ml ¼ m; ql ¼ q and r ¼ p we define sGSIP ¼ ðn; p; m; qÞ and the corresponding set of BLLGSIP problems PsGSIP ¼ fPGSIP ¼ ðA0 ; B0 ; A; B; b; b0 ; c0 Þg  RKGSIP with KGSIP :¼ ðn þ m þ 1Þðp þ qÞ þ n. In the sequel, by a generic subset V of RK we mean a set which is open and has a complement c V ¼ RK n V of measure zero (notation lðVc Þ ¼ 0). Note that lðVc Þ ¼ 0 implies that the set V is dense in RK . For definitions and details on the genericity and stratification theory we refer to [3]. The whole genericity analysis can be based on the following general result (see [3] for a proof). Lemma 4. Let h: RK ! R be a polynomial function, h 6 0. Then the solution set h1 ð0Þ ¼ fw 2 RK j hðwÞ ¼ 0g is a closed set of measure zero. Equivalently the complement V ¼ RK n h1 ð0Þ is a generic set in RK . This lemma will be used in a way indicated in the following lemma. Lemma 5. Let Vl denote the set of real ðl  lÞ-matrices, Vl ¼ fA ¼ ðaij Þi;j¼1;...;l jaij 2 Rg  Rll . Then, the set Vl0 ¼ fA 2 Vl j det A ¼ 0g is a closed set of measure zero in Rll . Equivalently the set Vlr ¼ Vl n Vl0 of regular matrices is generic in Rll . P Proof. In view of the Laplace expansion det A ¼ p2Pl signpa1pð1Þ    alpðlÞ the mapping h: Rll ! R, hðAÞ ¼ det A, is a polynomial. Since hðIÞ ¼ 1 we have h 6 0 (I denotes the unit matrix). The result now follows from Lemma 4. 

458

O. Stein, G. Still / European Journal of Operational Research 142 (2002) 444–462

In the proofs later on we tacitly make use of the following simple facts: q If V is a generic subset in RT , then Rs  V is generic in Rs  Rq . Let V1 ; . . . ; Vr be generic subsets of Rq . r Then the intersection V ¼ i¼1 Vi is generic in Rq .

We give the first genericity result. Lemma 6. The problem set Ps (or PsGSIP ) contains a generic subset V such that for any problem P in V the following holds: (a) All vertices of the semi-feasible set Msem of P are non-degenerate. All local solutions of P are locally unique and occur at vertices of Msem . All local solutions have different object values. In particular, the problem P has a unique global (vertex-) solution. (b) For any x 2 X and l, if Ql ðxÞ has a solution, then this solution yl ðxÞ is unique and occurs at a vertex of Yl ðxÞ.

Proof. (a) For r ¼ 1, the result is proven in [22, Theorem 3(a),(c)]. The generalization to the case r > 1 is not difficult (we have only to take care of the fact that now the problem matrices have block-structure). (b) Choose x 2 X arbitrarily and l 2 f1; . . . ; rg. Consider the lower level problem Ql ðxÞ:

min dl yl yl

s:t: Bl yl 6 bl  Al x:

Suppose yl is a solution of Ql ðxÞ. Then there exist Il , Il  f1; . . . ; ql g, jIl j 6 ml (by Caratheodory’s Theorem), 0 < ul 2 RjIl j such that uTl ðBl ÞIl ¼ dlT ;

ðBl Þj yl ¼ ðbl Þj  ðAl Þj x; j 2 Il :

ð16Þ

Here ðBl ÞIl denotes the sub-matrix of Bl only containing the rows with indices in Il . Generically, jIl j P ml , i.e. we can assume jIl j ¼ ml . In fact, if jIl j < ml then in view of uTl ðBl ÞIl ¼ dlT the ðjIl j þ 1Þ  ðjIl j þ 1Þmatrix (assume for brevity Il ¼ f1; . . . ; jIl jg and we denote the elements of Bl by ðBl Þij ) 2 3 B^ :¼ 4ðBl Þ i¼1;...;jI jþ1 b^5 l ij j¼1;...;jI lj

with

b^ :¼ ððdl Þ1 ; . . . ; ðdl ÞjIl jþ1 ÞT

would satisfy detðB^Þ ¼ 0 which can generically be avoided. Since generically (with jIl j ¼ ml ) the matrix ðBl ÞIl , is regular, a solution yl of Ql ðxÞ is generically a vertex of the polyhedron Yl ðxÞ. Since ul > 0 this solution is unique.  For the analysis of (LGSIP) we define the set PsrGSIP ¼ fP 2 PsGSIP jthe assumption AL3 holdg: It is not difficult to show that the problem set PsrGSIP is open in RKGSIP . The following theorem describes the difference between general (BL) and (GSIP) problems (see also Theorem 2). It shows that for BLLGSIP , in the generic case, n upper level constraints must be active at a solution x of (LGSIP) and that the regularity assumption A2LGSIP in Section 3 holds. Theorem 3. The problem set PsrGSIP contains a generic subset V such that for any BLLGSIP problem P in V the following holds.

O. Stein, G. Still / European Journal of Operational Research 142 (2002) 444–462

459

If P has a local solution ðx; yÞ (vertex solution see Lemma 6 ) then, precisely n upper-level constraints are active, i.e. there exist n indices in the index set J ¼ f1; . . . ; pg, say l ¼ 1; . . . ; n such that with the solutions y l of Ql ðxÞ we have ða0l ÞT x þ ðb0l ÞT y l  b0l ¼ 0;

l ¼ 1; . . . ; n:

These solutions y l are non-degenerate-vertices (i.e. (LICQ) holds). Moreover the local solution x of (LGSIP) is attained at a vertex of prx Msem . In particular, if p < n holds, then generically BLLGSIP and the corresponding (LGSIP) does not have a solution, i.e. the problem is unbounded. Proof. First we show that generically at least n constraints must be active at the local solution ðx; yÞ of BLLGSIP or the local solution x of (LGSIP). Suppose that k < n points are active at x. This means that there are k < n vertex solutions y l of Ql ðxÞ, say l ¼ 1; . . . ; k, active, i.e. we have ða0l ÞT x þ ðb0l ÞT y l  b0l ¼ 0;

l ¼ 1; . . . ; k;

T ða0l Þ x

l ¼ k þ 1; . . . ; p:

þ

T ðb0l Þ y l



b0l

> 0;

ð17Þ

We show then that generically AL3 is violated (see the definition of PrsGSIP ). With the value function vl ðxÞ of Ql ðxÞ, the local solution x of (LGSIP) must be a local minimizer of the problem min cT0 x s:t: vl ðxÞ P 0; l ¼ 1; . . . ; k:

ð18Þ

Consider the optimality conditions (Kuhn–Tucker and complementary conditions) for the solutions y l of Ql ðxÞ: BT kl  b0l ¼ 0;

ð19Þ

kTl ðAx þ Byl  bÞ ¼ 0:

Generically the solutions y l of Ql ðxÞ are unique (cf. Lemma 6(b)). Let Dl denote the set of Lagrange multipliers kl satisfying (19). Suppose AL3 is satisfied (Slater condition). Then by a well-known theorem (see e.g. [13]) the value functions vl are directionally differentiable and with the Lagrange function T Ll ðx; y; kÞ :¼ ða0l x þ b0l y  b0l Þ  kTl ðAx þ By  bÞ the directional derivative Dvl ðx; dÞ :¼ limt#0 ½vl ðx þ tdÞ  vl ðxÞ=t is given by T

Dvl ðx; dÞ ¼ max Dx Ll ðx; yl ; kl Þd ¼ max ða0l  AT kl Þ d: kl 2Dl

kl 2Dl

1

Choosing one kl 2 Dl arbitrarily (e.g. the multiplier kl ¼ ðBTIl Þ b0l ; see (16) in the proof of Lemma 6(b)), then obviously T

ða0l  AT kl Þ d 6 Dvl ðx; dÞ: Generically we can assume that the vectors c0 ; ða0l  AT kl Þ; l ¼ 1; . . . k; (k < n) are linearly independent. Thus, there is a solution d of cT0 d ¼ 1

T

ða0l  AT kl Þ d ¼ 1; l ¼ 1; . . . k:

This implies that for xt ¼ x þ td; t > 0 small, we have cT0 xt < cT0 x and in view of vl ðxÞ ¼ 0 T

vl ðxt Þ ¼ vl ðxÞ þ tDvl ðx; dÞ þ oðtÞ P tða0l  AT kl Þ d þ oðtÞ > 0; l ¼ 1; . . . k; contradicting the fact that x is a local solution of (18).

460

O. Stein, G. Still / European Journal of Operational Research 142 (2002) 444–462

We now show that k ¼ n must be valid. In view of Lemma 6(a) generically the local solution z :¼ ðx; y 1 ; . . . ; y k Þ is a non-degenerate vertex of the semi-feasible set Msem  Rnþkm of the problem ðLBLk Þ:

min cT0 x s:t: Gl ðx; yÞ :¼ a0l x þ b0l y  b0l P 0; l ¼ 1; . . . ; k; and for l ¼ 1; . . . ; k; yl is a solution of Ql ðxÞ: min ða0l ÞT x þ ðb0l ÞT yl s:t: Ax þ Byl  b P 0: yl

Thus n þ km constraints must be active in z. So for the number Na of active constraints we must have Na ¼ n þ km ¼ k þ

k X

jIl j:

l¼1

Using k P n and jIl j P m (yl are vertices of Ql ðxÞ) we find k 6 n; jIl j 6 m, i.e. k¼n

and

jIl j ¼ m; l ¼ 1; . . . ; k:

In view of Lemma 5 generically the ðm  mÞ-matrices BIl are regular. Thus (LICQ) is fulfilled generically. We now show that generically the solution x is attained at a vertex of prx Msem . The Kuhn–Tucker condition for the solutions y l of Ql ðxÞ reads BTIl kl ¼ b0l ;

kl P 0:

Generically we must have kl > 0 (see the proof of Lemma 6(b)). By standard sensitivity analysis it follows that locally near x the solutions yl ðxÞ of Ql ðxÞ; l ¼ 1; . . . ; n, (k ¼ n) with yl ðxÞ ¼ y l are given by AIl x þ BIl yl ðxÞ  bIl ¼ 0 or

yl ðxÞ ¼ B1 Il ðbIl  AIl xÞ:

By substituting this solution into (17) we see that a point x near x is feasible if and only if     T T 0 0 T 1 ða0l Þ  ðb0l Þ B1 A Þ B b x  b  ðb P 0; l ¼ 1; . . . ; n: I I l l Il l Il l T

ð20Þ

T

Generically the vectors ða0l Þ  ðb0l Þ B1 Il bIl ; l ¼ 1; . . . n; must be linearly independent. Thus the n inequalities in (20) define the vertex x of the polyhedron prx Msem . 

5. Algorithm for linear GSIP In the preceding sections we have seen that (GSIP) can be regarded as a special instance of a bilevel problem. Because of the special structure of (GSIP) not all approaches for semi-infinite programming are appropriate for general (BL) problems (for example the reduction approach). However any method for bilevel problems can be used to solve the bilevel formulation of (GSIP) problems. We refer to [16] for a survey of methods for solving (BL) (for the case r ¼ 1). Here we only consider the linear case and briefly outline the generalization to r > 1 (r lower level players) of an algorithm due to Bard and Moore (cf. [1]) which is based on a so-called Kuhn–Tucker approach. With this method, also (LGSIP) can be solved. Consider the necessary and sufficient Kuhn–Tucker optimality conditions for a solution yl of the linear program Ql ðxÞ: Introducing slack variables vl 2 Rql with the Lagrange multiplier vectors kl 2 Rql these conditions are:

O. Stein, G. Still / European Journal of Operational Research 142 (2002) 444–462

461

Al x þ Bl yl  bl  vl ¼ 0; kTl Bl  dl ¼ 0; kl P 0; kTl vl

¼0

vl P 0; ðcomplementarity conditionsÞ:

It follows that ðx; yÞ is a solution of (LBL) (cf. (14)) if and only if with slack vectors vl and multipliers kl the point ðx; yÞ solves the optimization problem min cT0 x þ d0T y x;y

s:t: A0 x þ B0 y  b0 P 0 and for l ¼ 1; . . . ; r; Al x þ Bl yl  bl  vl ¼ 0; ð21Þ

kTl Bl  dl ¼ 0; kl P 0; vl P 0; kTl vl ¼ 0:

Apart from the complementarity conditions kTl vl ¼ 0 this problem is linear. A branch and bound method to solve (21) is as follows. We define q :¼ q1 þ    þ qr , the vectors K :¼ ðk1 ; . . . ; kr Þ, V :¼ ðv1 ; . . . ; vr Þ in Rq and the index set K :¼ f1; . . . ; qg. In view of K; V P 0, the complementarity condition KT V in (21) is equivalent with Ki Vi ¼ 0 for all i 2 K. For given index sets K þ ; K   K; K þ \ K  ¼ ; we define the sets KðK þ Þ ¼ fK P 0 j Ki ¼ 0; i 2 K þ g;

V ðK  Þ ¼ fV P 0 j Vi ¼ 0; i 2 K  g:

For any pair K; V with K 2 KðK þ Þ, V 2 V ðK  Þ let LBLðK þ ; K  Þ denote the problem obtained by replacing in the (LBL) problem (21) the complementarity condition KV ¼ 0 by the conditions K 2 KðK þ Þ, V 2 V ðK  Þ. The problems LBLðK þ ; K  Þ are linear programs and for the right choice of K þ ; K  the solution of LBLðK þ ; K  Þ coincides with the solution of (LBL). The idea of the Bard/Moore algorithm is to examine in a branch and bound search all possible choices of K þ ; K  (see [1] for further details). The algorithm starts with K þ ¼ K  ¼ ;. Obviously, the value of LBLð;; ;Þ (a relaxation of (LBL)) gives a lower bound for the value of (LBL). Branch and bound algorithm. Start: Put k ¼ 0; K0þ ¼ ;; K0 ¼ ;, val ¼ 1. Step k ! k þ 1: Given Kkþ ; Kk , try to calculate a solution xk ; y k ; Kk ; V k of LBLðKkþ ; Kk Þ with value valk . 1. If LBLðKkþ ; Kk Þ is infeasible or if valk P val goto 3. If Ki Vi ¼ 0 for all i 2 K put val ¼ valk , goto 3. þ  2. (Branching w.r.t. K) Select an index ik 2 K n Kkþ such that Kik Vik > 0, put Kkþ1 ¼ Kkþ [ fik g, Kkþ1 ¼ Kk , goto 4. 3. Perform backtracking (see [1]) for details), goto 4. 4. k þ 1 ! k. With this method problems of size up to n ¼ m ¼ 100 (for r ¼ 1) can be solved (cf. [1,4] for numerical experiments). References [1] J.F. Bard, J.T. Moore, A branch and bound algorithm for the bilevel programming problem, SIAM Journal of Science and Statistical Computation 11 (2) (1990) 281–292. [2] E.J. Falk, J. Liu, On bilevel programming, Part I: General nonlinear cases, Mathematical Programming 70 (1995) 47–72.

462

O. Stein, G. Still / European Journal of Operational Research 142 (2002) 444–462

[3] C.G. Gibson, K. Wirthm€ uller, A.A. Du Plessis, E.J.N. Looijenga, in: Topological Stability of Smooth Mappings, Lecture Notes in Mathematics, vol. 552, Springer, Berlin, 1976. [4] R. Hamming, On the bilevel programming problem, Thesis in the Faculty of Mathematics, University of Twente, 1998. [5] R. Hettich, H.Th. Jongen, Semi-infinite programming: Conditions of optimality and applications, in: J. Stoer (Ed.), Optimization Techniques, vol. 2, Springer, Berlin, 1978, pp. 1–11. [6] R. Hettich, G. Still, Semi-infinite programming models in Robotics, in: Guddat et al. (Eds.), Parametric Optimization and Related Topics II, Akademie Verlag, Berlin, 1991. [7] R. Hettich, K. Kortanek, Semi-infinite programming: Theory methods and applications, SIAM Review 35 (3) (1993) 380–429. [8] R. Hettich, G. Still, Second order optimality conditions for generalized semi-infinite programming problems, Optimization 34 (1995) 195–211. [9] A. Hoffmann, R. Reinhardt, On reverse Chebyshev approximation problems, Technical University of Illmenau, Preprint No. M08/94, 1994. [10] H.Th. Jongen, J.-J. R€ uckmann, O. Stein, Generalized semi-infinite optimization: A first order optimality condition and examples, Mathematical Programming 83 (1998) 145–158. [11] H.Th. Jongen, G. Zwier, On the local structure of the feasible set in semi-infinite optimization, in: D Brosowski (Ed.), International Series of Numerical Mathematics, vol. 72, Birkh€auser, Basel, 1984, pp. 185–202. [12] A. Kaplan, R. Tichatschke, On a class of terminal variational problems, in: Guddat et al. (Eds.), Parametric Optimization and Related Topics IV, Peter Lang, Frankfurt a.M, 1997. [13] R.T. Rockafellar, Directional differentiability of the optimal value function in nonlinear programming problem, Mathematical Programming Study 21 (1994) 213–226. [14] J.-J. R€ uckmann, A. Shapiro, First-order optimality conditions in generalized semi-infinite programming, Journal of Optimization Theory and Applications 101 (1999) 677–691. [15] J.-J. R€ uckmann, O. Stein, On linear and linearized generalized semi-infinite problems, Annals of Operations Research 101 (2001) 191–208. [16] K. Shimizu, Y. Ishizuka, J. Bard, Nondifferentiable and Two-level Mathematical Programming, Kluwer Academic Publishers, Boston, 1997. [17] O. Stein, On level sets of marginal functions, Optimization 48 (2000) 43–67. [18] O. Stein, First order optimality conditions for degenerate index sets in generalized semi-infinite programming, Preprint No. 91, Lehrstuhl C f€ ur Mathematik, RWTH Aachen, 2000. [19] O. Stein, G. Still, On optimality conditions for generalized semi-infinite programming problems, Journal of Optimization Theory and Applications 104 (2000) 443–458. [20] G. Still, Generalized semi-infinite programming: Theory and methods, European Journal of Operational Research 119 (1999) 301–313. [21] G. Still, Generalized semi-infinite programming: Numerical aspects, Optimization 49 (2001) 223–242. [22] G. Still, Linear bilevel problems: Genericity results and an efficient method for computing local minima, Mathematical Methods of Operations Research, to appear. [23] G.-W. Weber, Generalized semi-infinite optimization and related topics, Habilitation Thesis, Darmstadt University of Technology, 1999.