Optimality conditions in global optimization and their applications
Math. Program., Ser. B DOI 10.1007/s10107-007-0142-4 FULL LENGTH PAPER
Optimality conditions in global optimization and their applications A. M. Rubinov · Z. Y. Wu
Received: 15 September 2005 / Accepted: 15 January 2006 © Springer-Verlag 2007
Abstract In this paper we derive necessary and sufficient conditions for some problems of global minimization. Our approach is based on methods of abstract convexity: we use a representation of an upper semicontinuous function as the lower envelope of a family of convex functions. We discuss applications of conditions obtained to the examination of some tractable sufficient conditions for the global minimum and to the theory of inequalities. Keywords Global optimization · Necessary and sufficient conditions · Abstract convexity · Inequalities Mathematics Subject Classification (2000) 90C30 · 90C46 · 41A65 1 Introduction The theory of local optimization is based on a local approximation of functions and sets that can be accomplished by methods of calculus and its modern generalizations. However, local approximation alone cannot help to examine global optimization problems, so different tools should be used instead of calculus or together with calculus. One of these tools is abstract convexity (see, for example, [9,12,14]) which deals with
The work was supported by a grant from the Australian Research Council. A. M. Rubinov · Z. Y. Wu (B) School of Information Technology and Mathematical Sciences, University of Ballarat, Ballarat 3353, VIC, Australia e-mail: [email protected] A. M. Rubinov e-mail: [email protected]
123
A. M. Rubinov, Z. Y. Wu
functions that can be represented as the upper envelope or the lower envelope of a subset of a set of sufficiently simple functions. We use methods of abstract convexity in this paper. We consider the global minimization of a function f over a convex set assuming that f can be represented as the infimum of a family ( f t )t∈T of convex functions and derive necessary and sufficient conditions for the global minimum. Conditions obtained are expressed in terms of εt -subdifferentials of functions f t with a certain εt ≥ 0. It is known (see for example [12]) that for each upper semicontinuous finite function f defined on a subset Ω of a Hilbert space X and bounded from above on Ω by a quadratic function of the form h(x) = ax2 +[b, x]+c, there exists a family ( f t )t∈T of convex quadratic functions such that f (x) = inf t∈T f t (x). For applications we need to know an explicit description of such a family. We give the required description for a differentiable function f with the Lipschitz continuous gradient mapping. Necessary and sufficient conditions or only sufficient conditions in global optimization are known for some classes of problems, in particular for the minimization of DC functions (including concave functions) over a convex set and also for the minimization of quadratic functions subject to quadratic constraints and/or box constraints (see, for example [1,2,4,5,7,8,10,11,15–18] and [Jeyakumar V., Rubinov A.M., Wu Z.Y. in Non-convex quadratic minimization problems with quadratic constraints: global optimality conditions, submitted paper; Jeyakumar V., Rubinov A.M., Wu Z.Y. in Sufficient global optimality conditions for non-convex quadratic minimization problems with box constraints submitted paper and Wu Z.Y., Jeyakumar V., Rubinov, A.M. in Sufficient conditions for globally optimality of bivalent nonconvex quadratic programs, submitted paper]). We consider a different kind of problems in this paper. In the simplest case of the unconstrained minimization of a function f : X → IR such that ∇ f (x) − ∇ f (y) ≤ ax − y for all x, y ∈ X we obtain the following 1 ∇ f (t)2 ≥ f (x) ¯ result: if a point x¯ is a global minimizer of f over X then f (t) − 4a for all t ∈ X . In other words ( f (t)− f (x) ¯ ≥ 0 ∀ t ∈ X ) ⇒ ( f (t)− f (x) ¯ ≥
1 ∇ f (t)2 ∀ t ∈ X ). 4a
(1.1)
Conditions obtained are not tractable. Nevertheless these conditions have some interesting applications and we examine two of them. First, we apply the conditions obtained to examination of the tractable sufficient condition for a global minimum that was used in papers [Jeyakumar V., Rubinov A.M., Wu Z.Y. in Non-convex quadratic minimization problems with quadratic constraints: global optimality conditions, submitted paper; Jeyakumar V., Rubinov A.M., Wu Z.Y. in Sufficient global optimality conditions for non-convex quadratic minimization problems with box constraints, submitted paper and Wu Z.Y., Jeyakumar V., Rubinov A.M. in Sufficient conditions for globally optimality of bivalent nonconvex quadratic programs,. submitted paper]. This condition is presented in terms of the abstract convex subdifferential ∂ L f (x) , where L is a certain set of elementary functions. By definition ∂ L f (x) = l ∈ L : f (y) ≥ f (x) − l(x) + l(y) for all y ∈ X .
123
Optimality conditions in global optimization and their applications
If subdifferential calculus rules are valid for ∂ L then the sufficient condition under consideration is also necessary one. However, calculus rules do not hold for many sets L of nonlinear elementary functions, so it is interesting to estimate how the sufficient condition under consideration is far from a necessary condition. The developed approach allows as to examine this problem. Second, we apply the conditions obtained to the theory of inequalities. Many inequalities (for examples, inequalities involving means) can be represented in the form f (x) − f (x) ¯ ≥ 0 where f is a certain function. We say that the inequality f (x) − f (x) ¯ ≥ u(x) with u(x) ≥ 0 is sharper that the inequality f (x) − f (x) ¯ ≥0 if there exists x with u(x) > 0. It was observed in [13] that certain conditions for global minimum can be used for sharpening some special inequalities. We demonstrate that conditions similar to (1.1) lead to nontrivial sharpening some well-known inequalities. In particular, we sharpen the inequality between the arithmetic mean and the geometric mean. The outline of the paper is as follows. Section 2 collects definitions and preliminary results from abstract convexity. Section 3 contains a result about abstract concavity with respect to a set of quadratic functions. Approximate subdifferentiability is discussed in Sect. 4. Section 5 provides necessary and sufficient conditions for the global minimum over a convex set. A sufficient condition for global minimum is examined in Sect. 6. The inequality (1.1) and its generalization are discussed in Sect. 7. Application to the theory of inequalities is presented in Sect. 8.
2 Preliminaries 2.1 Notation We use the following notation: ¯ = IR ∪ {+∞} ∪ {−∞}. IR is the real line, IR+∞ = IR ∪ {+∞}, IR−∞ = IR ∪ {−∞}, IR √ X is a Hilbert space with the inner product [·, ·] and the norm x = [x, x]. B(y, r ) = {x ∈ X : x − y ≤ r }. cl Ω is the closure of a set Ω ⊂ H . IRn is an-dimensional Euclidean space. IRn+ = x = (x1 , . . . , xn )T ∈ IRn : xi ≥ 0, i = 1, . . . , n IRn++ = x = (x1 , . . . , xn )T ∈ IRn : xi > 0, i = 1, . . . , n . ¯ Then dom f := x ∈ Ω : −∞ < f (x) < +∞ . Let Ω be a set and f : Ω → IR. ¯ and g : Ω → IR ¯ then f ≤ g means that f (x) ≤ g(x) for all x ∈ Ω. If f : Ω → IR If f : X → IR+∞ is a convex function and x ∈ dom f , then ∂ f (x) and ∂ε f (x) are the subdifferential and ε-subdifferential of f at x, respectively, in the sense of convex analysis. If Ω ⊂ X is a convex set, x ∈ cl Ω, and ε ≥ 0, then Nε,Ω (x) = {u ∈ X : [u, y] − [u, x] ≤ ε for all y ∈ Ω}. In particular NΩ (x) ≡ N0,Ω (x) is the normal cone of Ω at x. We assume that the infimum over the empty set is equal to +∞ and the supremum over the empty set is equal to −∞.
123
A. M. Rubinov, Z. Y. Wu
2.2 Abstract convexity and abstract concavity In this subsection we present basic definitions from abstract convexity that will be used throughout the paper. (See [9,12,14] for detailed presentation of different aspects of abstract convexity.) Let Ω be a set and H be a set of functions h : Ω → IR−∞ . A function f : Ω → IR+∞ is called abstract convex with respect to H or H -convex if there exists a set U ⊂ H such that f (x) = suph∈U h(x) for all x ∈ Ω. The set H is called a set of elementary functions in such a setting. Let H be a set of functions h : Ω → IR+∞ . A function f : Ω → IR−∞ is called abstract concave with respect to H or H -concave if there exists a set V ⊂ H such that f (x) = inf h∈V h(x) for all x ∈ Ω. We again call H a set of elementary functions. If Ω is a convex closed subset of a Hilbert space X and H consists of convex functions h : Ω → IR+∞ , we use the term inf-convex functions for H -concave functions. Let Ω ⊂ Ω ⊂ X , f : Ω → IR+∞ and x0 ∈ dom f . Let L be a set of functions l : Ω → IR−∞ . An element l ∈ L is called an L-subgradient of f at the point x0 if x0 ∈ dom l and f (x) ≥ f (x0 ) + l(x) − l(x0 ) for each x ∈ Ω. The set ∂ L f (x0 ) of all L-subgradients of f at x0 is referred to as L−subdifferential of f at x0 . An element l ∈ L is called an (ε, L)-subgradient of f at a point x0 ∈ dom f if x0 ∈ dom l and f (x) ≥ f (x0 ) + l(x) − l(x0 ) − ε, for each x ∈ Ω. The set ∂ε,L f (x) of all (ε, L)-subgradients of f at x0 is referred to as (ε, L)subdifferential of f at x0 . If L is the set of linear functions defined on X , f : X → IR+∞ is a lower semicontinuous convex function and x ∈ dom f then ∂ L f (x) = ∂ f (x), ∂ε,L f (x) = ∂ε f (x), where ∂ f (x) and ∂ε f (x) are the subdifferential and ε−subdifferential in the sense of convex analysis, respectively. For properties of L-subdifferentials and (ε, L)-subdifferentials, see, for example, [12]. For a set Ω ⊂ X the indicator function δΩ is defined as δΩ (x) =
0 +∞
if x ∈ Ω if x ∈ / Ω.
Let L be a set of elementary functions l : X → IR−∞ such that dom l ⊃ Ω for all l ∈ L and let x ∈ Ω. The normal set of Ω at x with respect to L is given by N L ,Ω (x) := l ∈ L : l(y) − l(x) ≤ 0 for all y ∈ Ω . (If L is a cone in a vector space, that is (l ∈ L , λ > 0) ⇒ λl ∈ L, then N L ,Ω is also a cone so we can use the term “normal cone” in such a case.) The ε-normal set
123
Optimality conditions in global optimization and their applications
of Ω at x with respect to L is given by Nε,L ,Ω (x) := l ∈ L : l(y) − l(x) ≤ ε for all y ∈ Ω . It is easy to see that N L ,Ω (x) = ∂ L δΩ (x),
Nε,L ,Ω (x) = ∂ε,L δΩ (x), x ∈ Ω.
2.3 Approximate subdifferential of the sum The following result will be intensively used in the paper (we formulate it only in the Hilbert space setting): Theorem 1 Let mf 0 , f 1 , . . . , f m be proper convex functions defined on a Hilbert space X and f = i=0 f i . Let ε > 0. Assume that there exists x˜ ∈ dom f 0 at which functions f 1 , . . . , f m are finite and continuous. Then ∂ε f (x0 ) =
m
∂εi f i (x0 ) : εi ≥ 0, i = 0, . . . , m,
i=0
for each x0 ∈
m
εi = ε .
i=0
m
i=0 dom f i .
This result can be found in [3] (see also [19]). It was assumed in [3] that m = 1. The validity of the result for an arbitrary m can be easily obtained by induction.
3 Abstract concavity with respect to a set of quadratic functions Let H be the set of all quadratic functions h of the form h(x) = ax2 + [l, x] + c, x ∈ X,
(3.1)
where a > 0, l ∈ X and c ∈ IR. We say that a function f : Ω → IR−∞ is majorized by H if there exists h ∈ H such that h ≥ f . The following result holds (see [12], Example 6.2). Theorem 2 Let Ω ⊂ X and H be the set of quadratic functions defined by (3.1). Then a function f : Ω → IR−∞ is H -concave if and only if f is majorized by H and upper semicontinuous. Since H consists of convex functions it follows that a majorized by H upper semicontinuous function is inf-convex. It follows from Theorem 2 that for each majorized by H upper semicontinuous function f : Ω → IR there exists a family ( f t )t∈T of quadratic functions f t ∈ H such that f (x) = inf t∈T f t (x) (x ∈ Ω). For applications we need to have an explicit description of such a family. It is also important to describe functions f and families
123
A. M. Rubinov, Z. Y. Wu
( f t )t∈T such that f (x) = mint∈T f t (x). We now describe such a family for functions f with the Lipschitz continuous gradient mapping x → ∇ f (x). It follows from the Proposition 1 below that such a function is majorized by the set H of quadratics. Proposition 1 Let Ω ⊂ X be a convex set and let f be a differentiable function defined on an open set containing Ω. Assume that the mapping x → ∇ f (x) is Lipschitz continuous on Ω: K :=
sup
x,y∈Ω,x= y
∇ f (x) − ∇ f (y) < +∞. x − y
(3.2)
Let a ≥ K . For each t ∈ Ω consider the function
f t (x) = f (t) + ∇ f (t), x − t + ax − t2 , (x ∈ X ).
(3.3)
Then f (x) = mint∈Ω f t (x), x ∈ Ω. Proof Applying the mean value theorem we have for each x, y ∈ Ω:
f (x) − f (y) = ∇ f (y + θ (x − y)), x − y , θ ∈ [0, 1], therefore
f (x) − f (y) − ∇ f (y), x − y = ∇ f (y + θ (x − y)) − ∇ f (y), x − y ≤ ∇ f (y + θ (x − y)) − ∇ f (y)x − y ≤ K θ x − y2 ≤ K x − y2 ≤ ax − y2 . This means that
f (x) ≤ f (y) + ∇ f (y), x − y + ax − y2 := g y (x), x ∈ Ω. Since f (x) = gx (x) it follows that f (x) = min y∈Ω g y (x) for all x ∈ Ω.
4 Approximate L k -subdifferentiability We start with the following assertion: Proposition 2 Let L be a set of continuous concave functions l : X → IR. Let Ω ⊂ X be a nonempty convex set and let f : Ω → IR. Assume that f (x) = inf t∈T f t (x) (x ∈ Ω), where f t : X → IR is a continuous convex function (t ∈ T ). Let y ∈ Ω, η ≥ 0 and εt = f t (y) − f (y), (t ∈ T ). Then l ∈ ∂η,L f (y) if and only if for each t ∈ T there exists εi,t ≥ 0 (i = 1, 2, 3) such that ε1,t + ε2,t + ε3,t = εt + η and 0 ∈ ∂ε1,t f t (y) + ∂ε2 ,t (−l)(y) + Nε3 ,t,Ω (y).
123
(4.1)
Optimality conditions in global optimization and their applications
Proof We have: ∂η,L f (y) = l ∈ = l∈ = l∈ = l∈
L : f (x) − l(x) ≥ f (y) − l(y) − η ∀x ∈ Ω L : inf f t (x) − l(x) ≥ f (y) − l(y) − η ∀ x ∈ Ω
(4.2)
t∈T
L : f t (x) − l(x) ≥ f (y) − l(y) − η ∀ t ∈ T, ∀ x ∈ Ω L : f t (x) + (−l)(x) + δΩ (x) ≥ f t (y) + (−l(y)) + δΩ (y) −εt − η ∀ t ∈ T, ∀ x ∈ X (4.3)
Thus
∂η,L f (y) = l ∈ L : 0 ∈ ∂εt +η ( f t + (−l) + δΩ )(y) .
(4.4)
Convexity and nonemptiness of Ω implies that δΩ is a proper convex function. Since the functions f t and −l are continuous and convex we can use Theorem 1. Then we obtain: ∂εt +η ( f t + (−l)+δΩ )(y) =
{∂ε1,t f t (y)+∂ε2,t (−l)(y)+∂ε3,t δΩ (y) : ε1,t , ε2,t , ε3,t ≥ 0, ε1,t+ε2,t+ε3,t =εt +η},
so 0 ∈ ∂εt +η ( f t + (−l) + δΩ )(y) if and only if there exist ε1,t , ε2,t , ε3,t ≥ 0, ε1,t + ε2,t + ε3,t = εt + η such that 0 ∈ ∂ε1,t f t (y) + ∂ε2,t (−l)(y) + ∂ε3,t δΩ (y). Since ∂ε3,t δΩ = Nε3,t ,Ω , we conclude that (4.5) is equivalent to (4.1).
(4.5)
Remark 1 Assume that assumptions of Proposition 2 hold. Let f˜t (x) = f t +δΩ . Then l ∈ ∂η,L f (y) if and only if there exists ε1,t ≥ 0, ε2,t ≥ 0 such that ε1,t + ε2,t = εt + η and 0 ∈ ∂ε1 ,t f˜t (y) + ∂ε2 ,t (−l)(y). The proof of this assertion is similar to the proof of Proposition 2 and we omit it. We now consider a more general case when L is a set of concave functions l : X → IR−∞ . Proposition 3 Let Ω ⊂ X be a non empty convex set, f : Ω → IR and L be a set of concave functions l : X → IR−∞ such that dom l ⊃ Ω. Assume that f (x) = inf t∈T f t (x) (x ∈ Ω), where f t : X → IR is a continuous convex function (t ∈ T ). Let y ∈ Ω, η ≥ 0 and εt = f t (y) − f (y), (t ∈ T ). Then l ∈ ∂η,L f (y) if and only if y ∈ dom l and for each t ∈ T there exists ε1,t ≥ 0 and ε2,t ≥ 0 such that ε1,t + ε2,t = εt + η and 0 ∈ ∂ε1,t f t (y) + ∂ε2 ,t l (y). where l = −l + δΩ .
123
A. M. Rubinov, Z. Y. Wu
Proof Since (4.4) can be represented in the form 0 ∈ ∂εt +η ( f t + l )(y), the desired result follows from Theorem 1. Let k ≥ 1. For each y ∈ X and a < 0 consider the function
Let
k (x) = ax − yk , x ∈ X. ϕ y,a
(4.6)
k L k = ϕ y,a : y ∈ X, a < 0
(4.7)
be a set of elementary functions. We only consider cases k = 1 and k = 2. If k = 2 then 2 ϕ y,a (x) := ax − y2 = ax2 + [m, x] + c, where m = −2ay, c = ay2 .
This presentation sometimes is more convenient. In this section we will study the approximate L k -subdifferentiability of inf-convex functions f : Ω → IR for k = 1, 2, where Ω ⊂ X . It is known, (see [12]) that for each Lipschitz function f the subdifferential ∂ L 1 f (x) is not empty; for f ∈ C 2 (Ω) which is minorized by a quadratic function, the subdifferential ∂ L 2 f (x) is not empty. Proposition 2 can be simplified if either L = L 1 or L = L 2 . Let L = L 1 then l ∈ −L if and only if l(x) = ax − z with a > 0 and z ∈ X . Proposition 4 Let a > 0 and y, z ∈ X . Let l(x) = ax − z (x ∈ X ) and let cε (m) = ay − z − [m, y − z] − ε,
(4.8)
where ε ≥ 0. Then ∂ε l(y) = m ∈ B(0, a) : cε (m) ≤ 0 .
Proof Let p(x) = x. Then ∂ p(0) = B(0, 1). We have ∂ε l(y) = a∂ε/a p(y−z). Since p is a positively homogeneous function it follows (see, for example, [12], Proposition 7.19) that ∂ε/a p(y − z) = {m ∈ B(0, 1) : [m , y − z] ≥ y − z − ε/a}, so ∂ε l(y) = m ∈ B(0, a) : [m, y − z] ≥ ay − z − ε = m ∈ B(0, a) : cε (m) ≤ 0 .
Corollary 1 Assume that assumptions of Proposition 2 hold and L = L 1 . Let l(x) = −ax − z, where a > 0 and z ∈ X . Let y ∈ Ω. Then l ∈ ∂η,L 1 f (y) if and only if for each t ∈ T there exists ε1,t and ε2,t with the following properties: (i) ε1,t ≥ 0, ε2,t ≥ 0, ε1,t + ε2,t = εt + η; (ii) there exists m ∈ ∂ε1,t f˜t (y) such that m ≤ a and ay − z − [m, y − z] ≤ ε2,t , where f˜t = f t + δΩ .
123
Optimality conditions in global optimization and their applications
It follows from Proposition 4 and Remark 1. Remark 2 If y = z then cε (m) ≤ 0 for all m ∈ X and ε ≥ 0 so ∂ε l(z) = B(0, a). In this case ∂ε l(z) does not depend on ε. We also need the following statement: Proposition 5 Let a > √ 0 and y, z ∈ X . Let l(x) = ax − z2 , (x ∈ X ). Then ∂ε l(y) = B(2a(y − z), 2 aε) for each ε ≥ 0. The result fairly easily follows from [6, Chap. 11]. The following argument can be also used: Let g(x) = ax − z2 − ay − z2 − [m, x − y] + ε, x ∈ X. Then m ∈ ∂ε l(y) if and only if g(x) ≥ 0 for all x ∈ X . Calculating minimum of the quadratic function g we obtain the desired result. 5 Necessary and sufficient condition for the global minimum over a convex set Let f : X → IR+∞ and Ω ⊂ X . Definition 1 Let η ≥ 0. We say that x¯ is an η-global minimizer of f over Ω if f (x) − f (x) ¯ ≥ −ηx − x, ¯ x ∈ Ω.
Let f Ω = f +δΩ . Definition 1 can be represented in the following form: A point x¯ ∈ Ω ¯ where lη,x¯ (x) = −ηx − x. ¯ is an η-global minimizer of f over Ω if lη,x¯ ∈ ∂ L 1 f Ω (x) It is clear that x¯ is a global minimizer if and only if it is an η-global minimizer for all η > 0. We now present necessary and sufficient conditions for an approximate global minimizer of an inf-convex function over a convex set in terms of approximate subdifferentials of corresponding convex functions. Theorem 3 (1) Let Ω ⊂ X be a convex nonempty set and ( f t )t∈T be a family of convex continuous functions defined on X . Let f : Ω → IR be a function such ¯ − f (x). ¯ Then that f (x) = inf t∈T f t (x). Let x¯ ∈ Ω and εt = f t (x) (2) x¯ is an η-global minimizer with η > 0 over Ω if and only if for each t ∈ T there exists ε1,t ≥ 0, ε2,t ≥ 0 such that ε1,t + ε2,t = εt and 0 ∈ ∂ε1,t f t (x) ¯ + Nε2,t ,Ω (x) ¯ + B(0, η). (3)
(5.1)
x¯ is a global minimizer over Ω if and only if for each t ∈ T there exists ε1,t ≥ 0, ε2,t ≥ 0 such that ε1,t + ε2,t = εt and ∂ε1,t f t (x) ¯ ∩ (−Nε2,t ,Ω (x)) ¯ = ∅.
(5.2)
123
A. M. Rubinov, Z. Y. Wu
Proof (1) Let η > 0. A point x¯ is an η-global minimizer if and only if the function ¯ belongs to ∂ L 1 f (x). ¯ In view of Proposition 2 this means lη (x) = −ηx − x that for all t ∈ Ω there exists ε1,t ≥ 0, ε2,t ≥ 0 and ε3,t ≥ 0 such that ε1,t + ε2,t + ε3,t = εt and 0 ∈ ∂ε1,t f t (x) ¯ + Nε2,t ,Ω (x) ¯ + ∂ε3 ,t (−lη )(x)). ¯
(5.3)
In view of Corollary 2 we get ∂ε (−lη )(x) ¯ = B(0, η) for all ε > 0, so (5.3) can be rewritten as 0 ∈ ∂ε1,t f t (x) ¯ + Nε2 ,t,Ω (x) ¯ + B(0, η),
(2)
where ε1,t + ε2,t ≤ εt . Since ε - subdifferential contains ε -subdifferential for ε ≥ ε , we can choose ε1,t and ε2,t for which ε1,t + ε2,t = εt . Let x¯ be a global minimizer. Then (5.1) holds for each η > 0; in other words, for each η > 0 there exist numbers ε1,t,η ≥ 0 and ε2,t,η ≥ 0, such that ε1,t,η + ¯ bη ∈ Nε2,t,η ,Ω (x) ¯ and ε2,t,η = εt and there exist vectors aη ∈ ∂ε1,t,η f t (x), cη ∈ B(0, η) such that aη + bη + cη = 0. Then cη → 0 as η → 0+. Without loss of generality we can assume that there exist limη→0+ ε1,t,η := ε1,t ≥ 0 and limη→0+ ε2,t,η := ε2,t ≥ 0. Obviously ε1,t + ε2,t = εt . Since ε-subdifferential is bounded we can assume without loss of generality that there exists a weak limit limη→0+ aη := a. Then there exists a weak limit limη→0+ bη := b. We ¯ b ∈ Nε2,t ,Ω (x) ¯ and a + b = 0 so (5.2) holds. have a ∈ ∂ε1,t f t (x),
Corollary 2 Assume that assumptions of Theorem 3 hold and Ω = X . Then (1)
x¯ is an η-global minimizer with η > 0 over X if and only if for each t ∈ T ∂εt f t (x) ¯ ∩ B(0, η) = ∅.
(2)
(5.4)
x¯ is a global minimizer over X if and only if for each t ∈ T , 0 ∈ ∂εt f t (x). ¯
(5.5)
Indeed, (5.4) and (5.5) follow from (5.1) and (5.2), respectively, and the fact that Nε,X (x) ¯ = {0} for all ε. We now consider a special case where ( f t )t∈T is the family of quadratic functions from Proposition 1. Theorem 4 Let Ω ⊂ X be a convex nonempty set and f be a continuously differentiable function defined on an open set containing Ω. Assume that the mapping x → ∇ f (x) is Lipschitz continuous on Ω with the Lipschitz constant K . Let f t be the function defined on X by (3.3) and f˜t = f t + δΩ . Let η > 0 and a ≥ K . Then
123
Optimality conditions in global optimization and their applications
(i) x¯ ∈ Ω is an η-global minimizer of f over Ω if and only if ¯ ≤ η, t ∈ Ω, min{y : y ∈ ∂εt f˜t (x)}
(5.6)
where ¯ f (x) ¯ = [∇ f (t), x¯ −t]+ax¯ −t2 + f (t)− f (x), ¯ t ∈ Ω. εt = f t (x)−
(5.7)
(ii) If Ω = X then x¯ is an η- global minimizer if and only if 2a(x¯ −t)+∇ f (t) ≤ 2 [a∇ f (t), x¯ −t]+a 2 x¯ − t2 + a[ f (t) − f (x)]+η, ¯ t ∈ X. (5.8) Proof
(i) For t ∈ Ω consider the function (3.3): f t (x) = f (t) + [∇ f (t), x − t] + ax − t2 , x ∈ X.
It follows from Proposition 1 that f (x) = mint∈Ω f t (x). It is easy to check that f t (x) = avt − x2 − avt − t2 + f (t), with vt = t −
1 ∇ f (t). 2a
In view of Theorem 3 a point x¯ is an η-global minimizer over Ω if and only if for each t ∈ T there exist ε1,t and ε2,t such that ε1,t ≥ 0, ε2,t ≥ 0, ε1,t + ε2,t = εt
(5.9)
¯ + B(0, η) + Nε2 ,t,Ω (x). ¯ 0 ∈ ∂ε1,t f t (x)
(5.10)
and (5.1) holds: Since f˜t = f t + δΩ , we have ¯ = ∂εt f˜t,ε (x)
∂ε1,t f t (x) ¯ + Nε2 ,t,Ω (x), ¯
ε1,t ≥0,ε2,t ≥0, ε1,t +ε2,t =εt
so the intersection B(0, η) ∩ ∂εt f˜t (x) ¯ is not empty if and only if there exist ε1,t and ε2,t such that (5.9) and (5.10) holds. This implies that (5.10) is equivalent to (5.6). (ii) Let Ω = X and let lt (x) = avt − x2 , (x ∈ X ), (t ∈ Ω). ¯ = ∂ε lt (x) ¯ for all ε ≥ 0. Since Ω = X it follows that f˜t = f t , Clearly ∂ε f t (x) therefore due to (5.6), x¯ is an η-global minimizer if and only if ¯ ≤ η. min y : y ∈ ∂εt lt (x)
123
A. M. Rubinov, Z. Y. Wu
√ Applying Proposition 5 we conclude that ∂εt lt (x) ¯ = B(2a(x¯ − vt ), 2 aεt ) so we need to calculate √ min y : y ∈ B(2a(x¯ − vt ), 2 aεt ) . It is easy to check that this minimum is equal to max(0, qt ) with qt := 2a(x¯ − √ vt ) − 2 aεt , so the inequality (5.6) holds if and only if qt ≤ η. Since 1 ∇ f (t) = 2a(x¯ − t) + ∇ f (t) 2a(x¯ − vt ) = 2a x¯ − t + 2a and
aεt = a∇ f (t), x¯ − t + a 2 x¯ − t2 + a f (t) − f (x) ¯ we have qt = 2a(x¯ − t) + ∇ f (t)
¯ . −2 a∇ f (t), x¯ − t + a 2 x¯ − t2 + a f (t) − f (x) Thus the result follows.
6 Sufficient conditions for the constrained global minimum Necessary and sufficient conditions presented in the previous section are not tractable. However, results obtained in this section can be useful for examination of some tractable conditions. In this section we investigate a sufficient condition for the global minimum that is based on abstract convexity. Consider the following optimization problem (P): minimize f (x) subject to x ∈ Ω, (6.1) where f : X → IR and Ω ⊂ X . Proposition 6 (Sufficient condition for a global minimizer) Let x¯ ∈ Ω. Let L be a ¯ = ∅. If there exists a set of elementary functions l : X → R−∞ such that ∂ L f (x) function l ∈ L such that l ∈ ∂ L f (x) ¯ and l(x) ≥ l(x) ¯ f or all x ∈ Ω
(6.2)
then x¯ is a global minimizer of (P). Proof We have: f (x) − f (x) ¯ ≥ l(x) − l(x) ¯ ≥ 0, x ∈ Ω.
123
Optimality conditions in global optimization and their applications
We can express the result of Proposition 6 in terms of the L-normal set N−L ,Ω (x). ¯ ¯ so the sufficient Indeed, l(x) ≥ l(x) ¯ for all x ∈ Ω means that −l ∈ N−L ,Ω (x), condition under consideration can be represented in the following form: Proposition 7 Let x¯ ∈ Ω. If ∂ L f (x)∩(−N ¯ ¯ = ∅ then x¯ is a global minimizer −L ,Ω )( x) of (P). Remark 3 Assume that subdifferential calculus rules hold for the set L and functions f and δ S , in particular ∂ L ( f + δΩ )(x) ⊂ ∂ L f (x) + ∂ L δΩ (x).
(6.3)
Then (6.2) is also a necessary condition. Indeed, the definition of a global minimizer can be presented in terms of the L-subdifferentials as follows: 0 ∈ ∂ L ( f + δΩ )(x). ¯
(6.4)
If (6.3) holds and x¯ is a global minimizer then 0 ∈ ∂ L f (x) ¯ + ∂ L δΩ (x). ¯ This implies ¯ ∩ (−∂ L δΩ )(x) ¯ = ∅. Since l ∈ −∂ L δΩ (x) ¯ means that l(x) ≥ l(x) ¯ for all ∂ L f (x) x ∈ Ω, we get (6.2). If L is an additive set (l1 , l2 ∈ L ⇒ l1 + l2 ∈ L) then the inclusion opposite to (6.3) holds so (6.3) is equivalent to ∂ L ( f + δΩ )(x) = ∂ L f (x) + ∂ L δΩ (x).
(6.5)
Unfortunately (6.5) does not hold for many sets L of elementary functions (see detailed discussion in Jeyakumar V., Rubinov A.M., Wu Z.Y. in Generalized Fenchel’s conjugation formulas and duality for abstract convex functions, submitted paper). There is an example in (Jeyakumar V., Rubinov A.M., Wu Z.Y. in Sufficient global optimality conditions for non-convex quadratic minimization problems with box constraints. Submitted paper) which shows that (6.2) is not a necessary condition for a set L of quadratic functions. Condition (6.2) is tractable in some important cases (see [Jeyakumar V., Rubinov A.M., Wu Z.Y. in Non-convex quadratic minimization problems with quadratic constraints: global optimality conditions, submitted paper; Jeyakumar V., Rubinov A.M., Wu Z.Y. in Sufficient global optimality conditions for non-convex quadratic minimization problems with box constraints, submitted paper and Wu Z.Y., Jeyakumar V., Rubinov A.M. in Sufficient conditions for globally optimality of bivalent nonconvex quadratic programs, submitted paper] for details) and there is a hope that the number of these cases can be extended. It is interesting to compare (6.2) with a necessary condition in general situation, where (6.3) is not valid. We will compare (6.2) with the necessary and sufficient condition (5.2) given in Theorem 3. Assume that Ω is a nonempty convex set; f : X → IR is a function such that f (x) = inf t∈T f t (x) for x ∈ X , where f t : X → IR is a continuous convex function (t ∈ T ); L is a set of concave functions l : X → IR−∞ such that dom l ⊃ Ω for all l ∈ L.
123
A. M. Rubinov, Z. Y. Wu
First we present condition (6.2) it terms that were used in Theorem 3. Let x¯ ∈ Ω. ¯ and Assume that there exists a function l such that (6.2) holds, that is l ∈ ∂ L f (x) ¯ if and only if for l(x) − l(x) ¯ ≥ 0 for all x ∈ Ω. In view of Proposition 3, l ∈ ∂ L f (x) ¯ − f (x) ¯ and each t ∈ T there exist ε1,t ≥ 0 and ε2,t ≥ 0 such that ε1,t + ε2,t = f t (x) ∂ε1,t f t (x) ¯ ∩ (−∂ε2 ,t (−l)(x)) ¯ = ∅.
(6.6)
Condition (5.2) can be rewritten in the following form: x¯ is a global minimizer if and only if for each t ∈ T there exist ε1,t ≥ 0 and ε2,t ≥ 0 such that ε1,t + ε2,t = ¯ − f (x) ¯ and f t (x) ¯ ∩ (−Nε2 ,t,Ω (x)) ¯ = ∅. (6.7) ∂ε1,t f t (x) It follows from the aforesaid that there is only one distinction between sufficient condition (6.2) and necessary and sufficient condition (5.2): the subdifferential ¯ in (6.2) is replaced with the normal cone Nε2 ,t,Ω (x) ¯ in (6.7). Note that −∂ε2 ,t (−l)(x) ¯ and we do not need to (6.2) is expressed in terms of the L-subdifferentials ∂ L f (x) have a set of elementary function L in (5.2). We can provide a more precise comparison between (6.2) and (5.2) for a special set of elementary function L. This class depends on a point x. ¯ Recall that a function l : X → IR−∞ is called superlinear if l(x + y) ≥ l(x) + l(y) for all x, y ∈ X and l(λx) = λl(x) for x ∈ X and λ > 0. An upper semicontinuous superlinear function l has the following representation
where
¯ l(x) = inf{[u, x] : u ∈ ∂l(0)}, x ∈ X,
(6.8)
¯ ∂l(0) := {u ∈ X : [u, x] ≥ l(x)} for all x ∈ X.
(6.9)
If l : X → IR−∞ is a superlinear function then the function l = −l is sublinear and ¯ its subdifferential at zero ∂l (0) coincides with −∂l(0). Let x¯ be a global minimizer of the problem (P) defined by (6.1). For the investigation of this global minimizer we will use a set L x¯ which consists of the functions of the form m(x) = l(x − x), ¯ where l is a superlinear function. For the sake of a simplicity we assume that x¯ = 0; then we consider superlinear functions themselves. Proposition 8 Consider the problem (P). Assume that Ω is a nonempty convex set with 0 ∈ Ω and f : X → IR is a function such that f (x) = inf t∈T f t (x), x ∈ X , where f t : X → IR is a continuous convex function (t ∈ T ). Let L be the set of all superlinear functions l : X → IR−∞ with dom l ⊃ Ω. Then ∂ L f (0) ∩ (−N−L ,Ω (0)) = ∅
(6.10)
if and only if ∂εt f t (0) ∩ (−NΩ (0)) = ∅, where εt = f t (0) − f (0).
123
∀t ∈ T ,
(6.11)
Optimality conditions in global optimization and their applications
Proof Assume that (6.10) holds, so there exists l ∈ L such that l ∈ ∂ L f (0) and −l ∈ N−L ,Ω (0). This means that l(x) ≤ f (x) − f (0) ∀ x ∈ X and
l(x) ≥ 0 ∀ x ∈ Ω.
(6.12)
It follows from (6.8) and the second inequality in (6.12) that ∂(−l)(0) ⊂ NΩ (0).
(6.13)
The first inequality in (6.12) is equivalent to l(x) ≤ f t (x) − f t (0) + εt for each x ∈ X and t ∈ T , that is 0 ∈ ∂εt ( f + (−l))(0) for every t ∈ T . Applying Theorem 1 we obtain that ∂εt f t + (−l) (0) =
∂ε1 f t (0) + ∂ε2 (−l)(0) .
(6.14)
ε1 ,ε2 ≥0, ε1 +ε2 =εt
It is well-known (see for example [12]) and easy to check that ε-subdifferential of a sublinear function at zero with ε > 0 coincides with its subdifferential at zero; therefore (6.14) can be represented in the form: ∂εt f t + (−l) (0) = ∂ε f t (0) + ∂(−l)(0) .
(6.15)
0≤ε≤εt
Since ∂εt f t (0) ⊃ ∂ε f t (0) for 0 ≤ ε ≤ εt , (6.15) implies ∂εt f t + (−l) (0) = ∂εt f t (0) + ∂(−l)(0).
(6.16)
It follows from (6.16) and (6.13) that (6.11) holds. Conversely, let (6.11) hold, i.e., for any t ∈ T , there exists u t ∈ X such that u t ∈ ∂εt f t (0) and −u t ∈ NΩ (0). Let l(x) = inf t∈T [u t , x]. Then l : X → IR−∞ is ¯ a superlinear function and ∂l(0) = cl co{u t }t∈T . Since NΩ (0) is a closed and convex ¯ set, this implies that ∂l(0) ⊂ −NΩ (0). The inclusion u t ∈ −NΩ (0) also implies that l(x) = inf t∈T [u t , x] ≥ 0, x ∈ Ω so dom l ⊃ Ω. This means that l ∈ L. We have for x ∈ X and t ∈ T : l(x) ≤ [u t , x] ≤ f t (x) − f t (0) + εt = f t (x) − f (0) and so l(x) ≤ f (x) − f (0) for all x ∈ X . Hence l ∈ ∂ L f (0). Since −u t ∈ NΩ (0) it follows that [u t , x] ≥ 0 for each x ∈ Ω and t ∈ T . Hence l(x) ≥ 0 for each x ∈ Ω, whence −l ∈ N−L ,Ω (0). Proposition 8 allows us to compare necessary and sufficient condition (5.2) and sufficient condition (6.2) in the case under consideration. Let x¯ = 0 be a solution of the problem (P). Let L be the set of all superlinear functions l : X → IR−∞ with dom l ⊃ Ω. Then (5.2) can be represented in the following form: for any t ∈ T , there exist ε1,t ≥ 0 and ε2,t ≥ 0 such that ε1,t + ε2,t = εt and ∂ε1,t f t (0) ∩ − Nε2,t ,Ω (0) = ∅.
(6.17)
123
A. M. Rubinov, Z. Y. Wu
Due to Proposition 8 sufficient condition (6.2) can be represented in the form ∂εt f t (0) ∩ − NΩ (0) = ∅,
∀t ∈ T.
(6.18)
Thus (6.2) is a special case of (5.2) corresponding to ε1,t = εt , ε2,t = 0. 7 Some properties of the global minimum In this section we will examine some properties of the global minimum. We start with functions f : X → IR having the Lipschitz continuous gradient mapping. Theorem 5 Let f ∈ C 1 (X ) and let the mapping x → ∇ f (x) be Lipschitz continuous: K :=
sup
x,y∈X,x= y
∇ f (x) − ∇ f (y) < +∞. x − y
Let a ≥ K . Let a point x¯ ∈ X be a global minimizer of f . Then 1 ∇ f (t)2 ≤ f (t) − f (x), ¯ t ∈ X. 4a
(7.1)
Proof A point x¯ is a global minimizer on X if and only if (5.8) holds for all η > 0, that is
2a(x¯ − t)+∇ f (t) ≤ 2 a∇ f (t), x¯ − t]+a 2 x¯ − t2 +a[ f (t) − f (x) ¯ , t ∈ X. (7.2) This inequality can be represented in the form
∇ f (t), ∇ f (t) − 4a f (t) − f (x) ¯ ≤ 0, t ∈ X
which is equivalent to (7.1).
We now give a version of Theorem 5 for functions defined on a set Ω ⊂ X with the nonempty interior. Theorem 6 Consider the space X and assume that X is equipped with not only the norm · , but also with a norm · ◦ , which is equivalent to · . Let Ω ⊂ X be a set with int Ω = ∅ and let f be a continuously differentiable function defined on an open set containing Ω. Assume that the mapping x → ∇ f (x) is Lipschitz on Ω: K :=
sup
x= y,x,y∈Ω
∇ f (x) − ∇ f (y) < +∞. x − y
Let x¯ ∈ int Ω be a global minimizer of f over Ω. Consider the ball B◦ (x, ¯ r) = {x : x − x ¯ ◦ ≤ r } ⊂ int Ω and let M := max ∇ f (t)◦ : t ∈ B◦ (x, ¯ r) .
123
(7.3)
Optimality conditions in global optimization and their applications
Let q > 0 be a number such that B◦ (x, ¯ r + q) ⊂ Ω and let
M a ≥ max K , 2q Then
.
(7.4)
1 ¯ t ∈ B◦ (x, ¯ r ). ∇ f (t)2 ≤ f (t) − f (x), 4a
(7.5)
M . Then r1 ≤ r + q, so B◦ (x, ¯ r1 ) ⊂ Ω. Applying Proposition 1 Proof Let r1 := r + 2a ¯ r1 ), where f t (x) = we conclude that f (x) = mint∈B◦ (x,r ¯ 1 ) f t (x) for any x ∈ B◦ ( x, f (t) + [∇ f (t), x − t] + ax − t2 . Thus, f (x) ≤ mint∈B◦ (x,r ¯ ) f t (x) for any x ∈ ¯ r1 ). Let us calculate vt ∈ argmin x∈X f t (x). Since vt is a solution of the equation B◦ (x, ∇ f t (x) = 0, we have
vt = t −
∇ f (t) . 2a
So 1 1 ∇ f (t)2 min f t (x) = f t (vt ) = f (t) + ∇ f (t), − ∇ f (t) + x∈X 2a 4a 1 ∇ f (t)2 . = f (t) − 4a For any t ∈ B◦ (x, ¯ r ) it holds: ∇ f (t) ¯ ◦ ≤ t − x ¯ ◦+ vt − x 2a ≤ r + q = r1 . ◦ So, vt ∈ B◦ (x, ¯ r1 ) and f (t) −
1 ∇ f (t)2 = f t (vt ) = min f t (x) = min f t (x). x∈Ω x∈B◦ (x,r ¯ 1) 4a
Then f (x) ¯ = ≤
min
f (x) =
min
min
x∈B◦ (x,r ¯ )
x∈B◦ (x,r ¯ 1 ) t∈B◦ (x,r ¯ )
Thus (7.5) is valid.
min
x∈B◦ (x,r ¯ 1)
f t (x) =
f (x) min
t∈B◦ (x,r ¯ )
f (t) −
1 ∇ f (t)2 . 4a
We now present a version of Theorem 6 for functions f : X → IR that have the boundedly Lipschitz continuous gradient mapping x → ∇ f (x). This means that for
123
A. M. Rubinov, Z. Y. Wu
all r > 0 it holds: K r :=
∇ f (x) − ∇ f (y) < +∞. x − y
sup
x= y,x,y≤r
(7.6)
Theorem 7 Let f ∈ C 1 (X ). Let the mapping x → ∇ f (x) be boundedly Lipschitz continuous. For any r > 0, let Mr :=
sup ∇ f (t) and r1 := r +
t∈B(0,r )
Mr , 2K r
where K r is defined by (7.6). If x¯ is a global minimizer of f then for all r ≥ x ¯ it holds: 1 ∇ f (t)2 ≤ f (t) − f (x), ¯ t ∈ B(0, r ), (7.7) ar1 where ar1 ≥ K r1 is an arbitrary number. The proof is similar to that of Theorem 6 and we present only its scheme. Proof Let r > 0. It follows from Proposition 1 that f (x) = mint∈B(0,r1 ) f t (x) for any x ∈ B(0, r1 ), where f t (x) = f (t) + [∇ f (t), x − t] + ar1 x − t2 and ar1 ≥ K r1 . Thus, f (x) ≤ inf t∈B(0,r ) f t (x) for any x ∈ B(0, r1 ). The same argument as in the proof of Theorem 6 shows that the global minimizer x¯t of f t (x) over X is located in B(0, r1 ). Therefore f t (x¯t ) = min f t (x) = x∈X
min
x∈B(0,r1 )
f t (x) = f (t) −
1 ∇ f (t)2 . 4ar1
If x¯ is a global minimizer of f (x) on X and r ≥ x, ¯ then f (x) ¯ = ≤
min
x∈B(0,r )
f (x) =
min ( f (t) −
t∈B(0,r1 )
min
x∈B(0,r1 )
f (x)
1 ∇ f (t)2 , forany ar ≥ K r1 . 4ar
Thus, (7.5) holds. 8 Global optimization and inequalities
In this section we demonstrate that a theory of global optimization has some interesting application to the theory of inequalities. Indeed, many inequalities can be presented in the form f (x) ≥ f (x), ¯ x ∈Ω
123
Optimality conditions in global optimization and their applications
where Ω ⊂ X , so the examination of these inequalities can be reduced to examination of a global optimization problem minimize f (x) subject to x ∈ Ω. Assume that we have two inequalities: f (x) − f (x) ¯ ≥0
(8.1)
f (x) − f (x) ¯ ≥ u(x) where u(x) ≥ 0.
(8.2)
and We say that (8.2) is sharper that (8.1) if there is a vector x such that u(x) > 0. Results obtained in the previous section can be used for construction of inequalities that are sharper than the given one. For the sake of definiteness we consider the situation described in Theorem 6. We assume that conditions of this theorem hold and we will use the same notation. Consider the function Q a ( f ) := g : Ω → IR, where g(x) = f (x) −
1 ∇ f (x)2 , x ∈ Ω. 4a
(8.3)
Here a is the number defined by (7.4). The function g enjoys the following properties: 1. 2.
g(x) ≤ f (x) for all x ∈ Ω; g(x) = f (x) if and only if x is a stationary point of f ; it follows directly from the definition of g. Let x¯ ∈ int Ω be a global minimizer of f over Ω, so x¯ is a global minimizer of f ¯ r + q). Then x¯ is also a global minimizer of g over B◦ (x, ¯ r + q) and over B◦ (x, min x∈B f (x) = min x∈B g(x). Indeed, since x¯ is a global minimizer of f then (7.5) holds; the inclusion x¯ ∈ int Ω implies ∇ f (x) ¯ = 0. Therefore (7.5) can be presented in the following form: f (x) ¯ −
1 1 ∇ f (x) ¯ 2 ≤ f (x) − ∇ f (x)2 , (x ∈ B◦ (x, ¯ r + q)). 4a 4a
¯ r + q)). Hence x¯ is a global This can be rewritten as g(x) ¯ ≤ g(x), (x ∈ B◦ (x, ¯ r + q). Since ∇ f (x) ¯ = 0, it follows that g(x) ¯ = f (x). ¯ minimizer of g over B◦ (x, Then g(x) = min f (x). (8.4) min x∈B◦ (x,r ¯ +q)
x∈B◦ (x,r ¯ +q)
If f is a twice-continuously differentiable function then the function g is differentiable and 1 2 1 2 ∇ f (x)∇ f (x) = I d − ∇ f (x) ∇ f (x). ∇g(x) = ∇ f (x) − (8.5) 2a 2a It follows from (8.5) that a point y is a stationary point of g if either y is a stationary point of f or ∇ f (y) is an eigenvector of ∇ 2 f (y) corresponding to the eigenvalue 2a.
123
A. M. Rubinov, Z. Y. Wu
The inequality g(x) − g(x) ¯ ≥ 0 can be rewritten in the form f (x) − f (x) ¯ ≥
1 ∇ f (x)2 , 4a
which is sharper than the inequality f (x)− f (x) ¯ ≥ 0. If f is a convex function defined on a convex set Ω then g(x) > f (x) for all x ∈ int Ω which are different from a global minimizer of f . Remark 4 Let Ω = X and f ∈ C ∞ (X ) be a boundedly Lipschitz function. Let a > 0 be an arbitrary number and Q a : C ∞ (X ) → C ∞ (X ) be an operator defined by Q a ( f )(x) = f (x) −
1 ∇ f (x)2 . 4a
Then Q a ( f )(x) ≤ f (x) for all x ∈ X and Q a ( f )(x) = f (x) if x is a stationary point. Let 1(x) = 1 for all x ∈ X . Then Q a (c1) = c1 for all c ∈ IR. Let a k be a sequence of positive numbers. Then f ≥ Q a1 f ≥ Q a2 Q a1 f ≥ · · · ≥ Q ak . . . Q a2 Q a1 f and for a stationary point x of f we have f (x) = Q a 1 f (x) = Q a 2 Q a 1 f (x) = · · · = Q a k . . . Q a 2 Q a 1 f (x). Using the described construction we will present a sharper version of the wellknown inequality between the arithmetic mean and the geometric mean which asserts that 1 1 (x1 + · · · + xn ) > (x1 . . . xn ) n , x ∈ IRn+ , x = λ1 with λ ≥ 0. n
(8.6)
Here 1 = (1, . . . , 1)T ∈ IRn+ . Recall that IRn+ is the cone of n-vectors with nonnegative coordinates and IRn++ is the cone of n-vectors with positive coordinates. Theorem 8 Let λ > r be positive numbers. Let
aλ,r
⎞ ⎛ n−1 n λ+r 1 − 1⎟ ⎜ n − 1 2 (λ + d) λ−r ⎟. = min max ⎜ , ⎝ r d > 0 consider the ball Vλ,d := B∞ (λ1, d) = x ∈ IRn : λ1 − x∞ ≤ d = {x ∈ IRn : λ − d ≤ xi ≤ λ + d, . . . , i = 1, . . . , n}.
(8.11) (8.12)
1
Since d < λ it follows that Vλ,d ⊂ ∇ρi (x) for x ∈ Vλ,d . We have
IRn++ .
πn (x) n Let ρi (x) = . We need to estimate xi
! ! ! ∂ρi ! 1 1 1 1 λ+d ! != (x) !∂x ! n x x (x1 . . . xn ) n ≤ n (λ − d)2 i = j. j i j ! ! ! ∂ρi ! n−1 1 1 n−1 λ+d ! ! ! ∂ x (x)! = n x 2 (x1 . . . xn ) n ≤ n (λ − d)2 , i i so
n − 1 (λ + d)2 (n − 1)2 (λ + d)2 ∇ρi (x) ≤ + 2 4 n (λ − d) n2 (λ − d)4 1 n − 1 2 (λ + d) = , (x ∈ Vλ,d ). n (λ − d)2
21
(8.13)
123
A. M. Rubinov, Z. Y. Wu
Let x, y ∈ Vλ,d . Applying the mean value theorem we conclude that there exist numbers θi ∈ [0, 1], i = 1, . . . , n such that 1 ∇ f (x) − ∇ f (y) = n
n
1 2
(ρi (x) − ρi (y))2
i=1
1 n 2 1 = [∇ρi (x + θi (y − x), (x − y)]2 n i=1
1 n 2 1 2 2 ≤ ∇ρi (x + θi (y − x) x − y . n i=1
Since x, y ∈ Vλ,d it follows that x + θi (y − x) ∈ Vd for all i. Applying (8.13) we conclude that ∇ f (x) − ∇ f (y) ≤ a1 (λ, d)x − y, x, y ∈ Vλ,d where
a1 (λ, d) =
n−1 n
1 2
(λ + d) . (λ − d)2
(8.14)
Hence mapping x → ∇ f (x) is Lipschitz continuous on Vλ,d with the Lipschitz constant K ≤ a1 (λ, d). We will apply Theorem 6 to the set Ω = Vλ,d where r < d < λ and the global minimizer x¯ = λ1 of the function f . Assume that the norm ·◦ that was used in Theorem 6 coincides with ·∞ . Let q = d−r . Let us estimate M = max{∇ f (x)∞ : x ∈ Vλ,r }. Due to symmetry it is enough to estimate only the first coordinate 1 − gradient ∇ f (x). It is easy to see that ! 1 ! πn (x) n ! M ≤ max !1 − x∈Vλ,r ! x1
of the
! ! 1 ! n−1 ! ! x2 λ+r n xn n !! ! ! ... − 1. ! = max !1 − !≤ ! x∈Vλ,r ! x1 x1 ! λ−r
Let a2 (λ, d, r ) =
λ+r λ−r
n−1 n
−1
2(d − r )
and a(λ, d, r ) = max(a1 (λ, d), a2 (λ, d, r )) ⎛ ⎞ n−1 n λ+r 1 − 1⎟ ⎜ n − 1 2 (λ + d) λ−r ⎟. = max ⎜ , ⎝ 2 n (λ − d) 2(d − r ) ⎠
123
1
πn (x) n x1
Optimality conditions in global optimization and their applications
Note that limd→λ−0 a(λ, d, r ) = limd→r +0 a(λ, d, r ) = +∞ so the function d → a(λ, d, r ) attains its minimum on the segment (r, λ). Let aλ,r = minr
Optimality conditions in global optimization and their applications A. M. Rubinov · Z. Y. Wu
Received: 15 September 2005 / Accepted: 15 January 2006 © Springer-Verlag 2007
Abstract In this paper we derive necessary and sufficient conditions for some problems of global minimization. Our approach is based on methods of abstract convexity: we use a representation of an upper semicontinuous function as the lower envelope of a family of convex functions. We discuss applications of conditions obtained to the examination of some tractable sufficient conditions for the global minimum and to the theory of inequalities. Keywords Global optimization · Necessary and sufficient conditions · Abstract convexity · Inequalities Mathematics Subject Classification (2000) 90C30 · 90C46 · 41A65 1 Introduction The theory of local optimization is based on a local approximation of functions and sets that can be accomplished by methods of calculus and its modern generalizations. However, local approximation alone cannot help to examine global optimization problems, so different tools should be used instead of calculus or together with calculus. One of these tools is abstract convexity (see, for example, [9,12,14]) which deals with
The work was supported by a grant from the Australian Research Council. A. M. Rubinov · Z. Y. Wu (B) School of Information Technology and Mathematical Sciences, University of Ballarat, Ballarat 3353, VIC, Australia e-mail: [email protected] A. M. Rubinov e-mail: [email protected]
123
A. M. Rubinov, Z. Y. Wu
functions that can be represented as the upper envelope or the lower envelope of a subset of a set of sufficiently simple functions. We use methods of abstract convexity in this paper. We consider the global minimization of a function f over a convex set assuming that f can be represented as the infimum of a family ( f t )t∈T of convex functions and derive necessary and sufficient conditions for the global minimum. Conditions obtained are expressed in terms of εt -subdifferentials of functions f t with a certain εt ≥ 0. It is known (see for example [12]) that for each upper semicontinuous finite function f defined on a subset Ω of a Hilbert space X and bounded from above on Ω by a quadratic function of the form h(x) = ax2 +[b, x]+c, there exists a family ( f t )t∈T of convex quadratic functions such that f (x) = inf t∈T f t (x). For applications we need to know an explicit description of such a family. We give the required description for a differentiable function f with the Lipschitz continuous gradient mapping. Necessary and sufficient conditions or only sufficient conditions in global optimization are known for some classes of problems, in particular for the minimization of DC functions (including concave functions) over a convex set and also for the minimization of quadratic functions subject to quadratic constraints and/or box constraints (see, for example [1,2,4,5,7,8,10,11,15–18] and [Jeyakumar V., Rubinov A.M., Wu Z.Y. in Non-convex quadratic minimization problems with quadratic constraints: global optimality conditions, submitted paper; Jeyakumar V., Rubinov A.M., Wu Z.Y. in Sufficient global optimality conditions for non-convex quadratic minimization problems with box constraints submitted paper and Wu Z.Y., Jeyakumar V., Rubinov, A.M. in Sufficient conditions for globally optimality of bivalent nonconvex quadratic programs, submitted paper]). We consider a different kind of problems in this paper. In the simplest case of the unconstrained minimization of a function f : X → IR such that ∇ f (x) − ∇ f (y) ≤ ax − y for all x, y ∈ X we obtain the following 1 ∇ f (t)2 ≥ f (x) ¯ result: if a point x¯ is a global minimizer of f over X then f (t) − 4a for all t ∈ X . In other words ( f (t)− f (x) ¯ ≥ 0 ∀ t ∈ X ) ⇒ ( f (t)− f (x) ¯ ≥
1 ∇ f (t)2 ∀ t ∈ X ). 4a
(1.1)
Conditions obtained are not tractable. Nevertheless these conditions have some interesting applications and we examine two of them. First, we apply the conditions obtained to examination of the tractable sufficient condition for a global minimum that was used in papers [Jeyakumar V., Rubinov A.M., Wu Z.Y. in Non-convex quadratic minimization problems with quadratic constraints: global optimality conditions, submitted paper; Jeyakumar V., Rubinov A.M., Wu Z.Y. in Sufficient global optimality conditions for non-convex quadratic minimization problems with box constraints, submitted paper and Wu Z.Y., Jeyakumar V., Rubinov A.M. in Sufficient conditions for globally optimality of bivalent nonconvex quadratic programs,. submitted paper]. This condition is presented in terms of the abstract convex subdifferential ∂ L f (x) , where L is a certain set of elementary functions. By definition ∂ L f (x) = l ∈ L : f (y) ≥ f (x) − l(x) + l(y) for all y ∈ X .
123
Optimality conditions in global optimization and their applications
If subdifferential calculus rules are valid for ∂ L then the sufficient condition under consideration is also necessary one. However, calculus rules do not hold for many sets L of nonlinear elementary functions, so it is interesting to estimate how the sufficient condition under consideration is far from a necessary condition. The developed approach allows as to examine this problem. Second, we apply the conditions obtained to the theory of inequalities. Many inequalities (for examples, inequalities involving means) can be represented in the form f (x) − f (x) ¯ ≥ 0 where f is a certain function. We say that the inequality f (x) − f (x) ¯ ≥ u(x) with u(x) ≥ 0 is sharper that the inequality f (x) − f (x) ¯ ≥0 if there exists x with u(x) > 0. It was observed in [13] that certain conditions for global minimum can be used for sharpening some special inequalities. We demonstrate that conditions similar to (1.1) lead to nontrivial sharpening some well-known inequalities. In particular, we sharpen the inequality between the arithmetic mean and the geometric mean. The outline of the paper is as follows. Section 2 collects definitions and preliminary results from abstract convexity. Section 3 contains a result about abstract concavity with respect to a set of quadratic functions. Approximate subdifferentiability is discussed in Sect. 4. Section 5 provides necessary and sufficient conditions for the global minimum over a convex set. A sufficient condition for global minimum is examined in Sect. 6. The inequality (1.1) and its generalization are discussed in Sect. 7. Application to the theory of inequalities is presented in Sect. 8.
2 Preliminaries 2.1 Notation We use the following notation: ¯ = IR ∪ {+∞} ∪ {−∞}. IR is the real line, IR+∞ = IR ∪ {+∞}, IR−∞ = IR ∪ {−∞}, IR √ X is a Hilbert space with the inner product [·, ·] and the norm x = [x, x]. B(y, r ) = {x ∈ X : x − y ≤ r }. cl Ω is the closure of a set Ω ⊂ H . IRn is an-dimensional Euclidean space. IRn+ = x = (x1 , . . . , xn )T ∈ IRn : xi ≥ 0, i = 1, . . . , n IRn++ = x = (x1 , . . . , xn )T ∈ IRn : xi > 0, i = 1, . . . , n . ¯ Then dom f := x ∈ Ω : −∞ < f (x) < +∞ . Let Ω be a set and f : Ω → IR. ¯ and g : Ω → IR ¯ then f ≤ g means that f (x) ≤ g(x) for all x ∈ Ω. If f : Ω → IR If f : X → IR+∞ is a convex function and x ∈ dom f , then ∂ f (x) and ∂ε f (x) are the subdifferential and ε-subdifferential of f at x, respectively, in the sense of convex analysis. If Ω ⊂ X is a convex set, x ∈ cl Ω, and ε ≥ 0, then Nε,Ω (x) = {u ∈ X : [u, y] − [u, x] ≤ ε for all y ∈ Ω}. In particular NΩ (x) ≡ N0,Ω (x) is the normal cone of Ω at x. We assume that the infimum over the empty set is equal to +∞ and the supremum over the empty set is equal to −∞.
123
A. M. Rubinov, Z. Y. Wu
2.2 Abstract convexity and abstract concavity In this subsection we present basic definitions from abstract convexity that will be used throughout the paper. (See [9,12,14] for detailed presentation of different aspects of abstract convexity.) Let Ω be a set and H be a set of functions h : Ω → IR−∞ . A function f : Ω → IR+∞ is called abstract convex with respect to H or H -convex if there exists a set U ⊂ H such that f (x) = suph∈U h(x) for all x ∈ Ω. The set H is called a set of elementary functions in such a setting. Let H be a set of functions h : Ω → IR+∞ . A function f : Ω → IR−∞ is called abstract concave with respect to H or H -concave if there exists a set V ⊂ H such that f (x) = inf h∈V h(x) for all x ∈ Ω. We again call H a set of elementary functions. If Ω is a convex closed subset of a Hilbert space X and H consists of convex functions h : Ω → IR+∞ , we use the term inf-convex functions for H -concave functions. Let Ω ⊂ Ω ⊂ X , f : Ω → IR+∞ and x0 ∈ dom f . Let L be a set of functions l : Ω → IR−∞ . An element l ∈ L is called an L-subgradient of f at the point x0 if x0 ∈ dom l and f (x) ≥ f (x0 ) + l(x) − l(x0 ) for each x ∈ Ω. The set ∂ L f (x0 ) of all L-subgradients of f at x0 is referred to as L−subdifferential of f at x0 . An element l ∈ L is called an (ε, L)-subgradient of f at a point x0 ∈ dom f if x0 ∈ dom l and f (x) ≥ f (x0 ) + l(x) − l(x0 ) − ε, for each x ∈ Ω. The set ∂ε,L f (x) of all (ε, L)-subgradients of f at x0 is referred to as (ε, L)subdifferential of f at x0 . If L is the set of linear functions defined on X , f : X → IR+∞ is a lower semicontinuous convex function and x ∈ dom f then ∂ L f (x) = ∂ f (x), ∂ε,L f (x) = ∂ε f (x), where ∂ f (x) and ∂ε f (x) are the subdifferential and ε−subdifferential in the sense of convex analysis, respectively. For properties of L-subdifferentials and (ε, L)-subdifferentials, see, for example, [12]. For a set Ω ⊂ X the indicator function δΩ is defined as δΩ (x) =
0 +∞
if x ∈ Ω if x ∈ / Ω.
Let L be a set of elementary functions l : X → IR−∞ such that dom l ⊃ Ω for all l ∈ L and let x ∈ Ω. The normal set of Ω at x with respect to L is given by N L ,Ω (x) := l ∈ L : l(y) − l(x) ≤ 0 for all y ∈ Ω . (If L is a cone in a vector space, that is (l ∈ L , λ > 0) ⇒ λl ∈ L, then N L ,Ω is also a cone so we can use the term “normal cone” in such a case.) The ε-normal set
123
Optimality conditions in global optimization and their applications
of Ω at x with respect to L is given by Nε,L ,Ω (x) := l ∈ L : l(y) − l(x) ≤ ε for all y ∈ Ω . It is easy to see that N L ,Ω (x) = ∂ L δΩ (x),
Nε,L ,Ω (x) = ∂ε,L δΩ (x), x ∈ Ω.
2.3 Approximate subdifferential of the sum The following result will be intensively used in the paper (we formulate it only in the Hilbert space setting): Theorem 1 Let mf 0 , f 1 , . . . , f m be proper convex functions defined on a Hilbert space X and f = i=0 f i . Let ε > 0. Assume that there exists x˜ ∈ dom f 0 at which functions f 1 , . . . , f m are finite and continuous. Then ∂ε f (x0 ) =
m
∂εi f i (x0 ) : εi ≥ 0, i = 0, . . . , m,
i=0
for each x0 ∈
m
εi = ε .
i=0
m
i=0 dom f i .
This result can be found in [3] (see also [19]). It was assumed in [3] that m = 1. The validity of the result for an arbitrary m can be easily obtained by induction.
3 Abstract concavity with respect to a set of quadratic functions Let H be the set of all quadratic functions h of the form h(x) = ax2 + [l, x] + c, x ∈ X,
(3.1)
where a > 0, l ∈ X and c ∈ IR. We say that a function f : Ω → IR−∞ is majorized by H if there exists h ∈ H such that h ≥ f . The following result holds (see [12], Example 6.2). Theorem 2 Let Ω ⊂ X and H be the set of quadratic functions defined by (3.1). Then a function f : Ω → IR−∞ is H -concave if and only if f is majorized by H and upper semicontinuous. Since H consists of convex functions it follows that a majorized by H upper semicontinuous function is inf-convex. It follows from Theorem 2 that for each majorized by H upper semicontinuous function f : Ω → IR there exists a family ( f t )t∈T of quadratic functions f t ∈ H such that f (x) = inf t∈T f t (x) (x ∈ Ω). For applications we need to have an explicit description of such a family. It is also important to describe functions f and families
123
A. M. Rubinov, Z. Y. Wu
( f t )t∈T such that f (x) = mint∈T f t (x). We now describe such a family for functions f with the Lipschitz continuous gradient mapping x → ∇ f (x). It follows from the Proposition 1 below that such a function is majorized by the set H of quadratics. Proposition 1 Let Ω ⊂ X be a convex set and let f be a differentiable function defined on an open set containing Ω. Assume that the mapping x → ∇ f (x) is Lipschitz continuous on Ω: K :=
sup
x,y∈Ω,x= y
∇ f (x) − ∇ f (y) < +∞. x − y
(3.2)
Let a ≥ K . For each t ∈ Ω consider the function
f t (x) = f (t) + ∇ f (t), x − t + ax − t2 , (x ∈ X ).
(3.3)
Then f (x) = mint∈Ω f t (x), x ∈ Ω. Proof Applying the mean value theorem we have for each x, y ∈ Ω:
f (x) − f (y) = ∇ f (y + θ (x − y)), x − y , θ ∈ [0, 1], therefore
f (x) − f (y) − ∇ f (y), x − y = ∇ f (y + θ (x − y)) − ∇ f (y), x − y ≤ ∇ f (y + θ (x − y)) − ∇ f (y)x − y ≤ K θ x − y2 ≤ K x − y2 ≤ ax − y2 . This means that
f (x) ≤ f (y) + ∇ f (y), x − y + ax − y2 := g y (x), x ∈ Ω. Since f (x) = gx (x) it follows that f (x) = min y∈Ω g y (x) for all x ∈ Ω.
4 Approximate L k -subdifferentiability We start with the following assertion: Proposition 2 Let L be a set of continuous concave functions l : X → IR. Let Ω ⊂ X be a nonempty convex set and let f : Ω → IR. Assume that f (x) = inf t∈T f t (x) (x ∈ Ω), where f t : X → IR is a continuous convex function (t ∈ T ). Let y ∈ Ω, η ≥ 0 and εt = f t (y) − f (y), (t ∈ T ). Then l ∈ ∂η,L f (y) if and only if for each t ∈ T there exists εi,t ≥ 0 (i = 1, 2, 3) such that ε1,t + ε2,t + ε3,t = εt + η and 0 ∈ ∂ε1,t f t (y) + ∂ε2 ,t (−l)(y) + Nε3 ,t,Ω (y).
123
(4.1)
Optimality conditions in global optimization and their applications
Proof We have: ∂η,L f (y) = l ∈ = l∈ = l∈ = l∈
L : f (x) − l(x) ≥ f (y) − l(y) − η ∀x ∈ Ω L : inf f t (x) − l(x) ≥ f (y) − l(y) − η ∀ x ∈ Ω
(4.2)
t∈T
L : f t (x) − l(x) ≥ f (y) − l(y) − η ∀ t ∈ T, ∀ x ∈ Ω L : f t (x) + (−l)(x) + δΩ (x) ≥ f t (y) + (−l(y)) + δΩ (y) −εt − η ∀ t ∈ T, ∀ x ∈ X (4.3)
Thus
∂η,L f (y) = l ∈ L : 0 ∈ ∂εt +η ( f t + (−l) + δΩ )(y) .
(4.4)
Convexity and nonemptiness of Ω implies that δΩ is a proper convex function. Since the functions f t and −l are continuous and convex we can use Theorem 1. Then we obtain: ∂εt +η ( f t + (−l)+δΩ )(y) =
{∂ε1,t f t (y)+∂ε2,t (−l)(y)+∂ε3,t δΩ (y) : ε1,t , ε2,t , ε3,t ≥ 0, ε1,t+ε2,t+ε3,t =εt +η},
so 0 ∈ ∂εt +η ( f t + (−l) + δΩ )(y) if and only if there exist ε1,t , ε2,t , ε3,t ≥ 0, ε1,t + ε2,t + ε3,t = εt + η such that 0 ∈ ∂ε1,t f t (y) + ∂ε2,t (−l)(y) + ∂ε3,t δΩ (y). Since ∂ε3,t δΩ = Nε3,t ,Ω , we conclude that (4.5) is equivalent to (4.1).
(4.5)
Remark 1 Assume that assumptions of Proposition 2 hold. Let f˜t (x) = f t +δΩ . Then l ∈ ∂η,L f (y) if and only if there exists ε1,t ≥ 0, ε2,t ≥ 0 such that ε1,t + ε2,t = εt + η and 0 ∈ ∂ε1 ,t f˜t (y) + ∂ε2 ,t (−l)(y). The proof of this assertion is similar to the proof of Proposition 2 and we omit it. We now consider a more general case when L is a set of concave functions l : X → IR−∞ . Proposition 3 Let Ω ⊂ X be a non empty convex set, f : Ω → IR and L be a set of concave functions l : X → IR−∞ such that dom l ⊃ Ω. Assume that f (x) = inf t∈T f t (x) (x ∈ Ω), where f t : X → IR is a continuous convex function (t ∈ T ). Let y ∈ Ω, η ≥ 0 and εt = f t (y) − f (y), (t ∈ T ). Then l ∈ ∂η,L f (y) if and only if y ∈ dom l and for each t ∈ T there exists ε1,t ≥ 0 and ε2,t ≥ 0 such that ε1,t + ε2,t = εt + η and 0 ∈ ∂ε1,t f t (y) + ∂ε2 ,t l (y). where l = −l + δΩ .
123
A. M. Rubinov, Z. Y. Wu
Proof Since (4.4) can be represented in the form 0 ∈ ∂εt +η ( f t + l )(y), the desired result follows from Theorem 1. Let k ≥ 1. For each y ∈ X and a < 0 consider the function
Let
k (x) = ax − yk , x ∈ X. ϕ y,a
(4.6)
k L k = ϕ y,a : y ∈ X, a < 0
(4.7)
be a set of elementary functions. We only consider cases k = 1 and k = 2. If k = 2 then 2 ϕ y,a (x) := ax − y2 = ax2 + [m, x] + c, where m = −2ay, c = ay2 .
This presentation sometimes is more convenient. In this section we will study the approximate L k -subdifferentiability of inf-convex functions f : Ω → IR for k = 1, 2, where Ω ⊂ X . It is known, (see [12]) that for each Lipschitz function f the subdifferential ∂ L 1 f (x) is not empty; for f ∈ C 2 (Ω) which is minorized by a quadratic function, the subdifferential ∂ L 2 f (x) is not empty. Proposition 2 can be simplified if either L = L 1 or L = L 2 . Let L = L 1 then l ∈ −L if and only if l(x) = ax − z with a > 0 and z ∈ X . Proposition 4 Let a > 0 and y, z ∈ X . Let l(x) = ax − z (x ∈ X ) and let cε (m) = ay − z − [m, y − z] − ε,
(4.8)
where ε ≥ 0. Then ∂ε l(y) = m ∈ B(0, a) : cε (m) ≤ 0 .
Proof Let p(x) = x. Then ∂ p(0) = B(0, 1). We have ∂ε l(y) = a∂ε/a p(y−z). Since p is a positively homogeneous function it follows (see, for example, [12], Proposition 7.19) that ∂ε/a p(y − z) = {m ∈ B(0, 1) : [m , y − z] ≥ y − z − ε/a}, so ∂ε l(y) = m ∈ B(0, a) : [m, y − z] ≥ ay − z − ε = m ∈ B(0, a) : cε (m) ≤ 0 .
Corollary 1 Assume that assumptions of Proposition 2 hold and L = L 1 . Let l(x) = −ax − z, where a > 0 and z ∈ X . Let y ∈ Ω. Then l ∈ ∂η,L 1 f (y) if and only if for each t ∈ T there exists ε1,t and ε2,t with the following properties: (i) ε1,t ≥ 0, ε2,t ≥ 0, ε1,t + ε2,t = εt + η; (ii) there exists m ∈ ∂ε1,t f˜t (y) such that m ≤ a and ay − z − [m, y − z] ≤ ε2,t , where f˜t = f t + δΩ .
123
Optimality conditions in global optimization and their applications
It follows from Proposition 4 and Remark 1. Remark 2 If y = z then cε (m) ≤ 0 for all m ∈ X and ε ≥ 0 so ∂ε l(z) = B(0, a). In this case ∂ε l(z) does not depend on ε. We also need the following statement: Proposition 5 Let a > √ 0 and y, z ∈ X . Let l(x) = ax − z2 , (x ∈ X ). Then ∂ε l(y) = B(2a(y − z), 2 aε) for each ε ≥ 0. The result fairly easily follows from [6, Chap. 11]. The following argument can be also used: Let g(x) = ax − z2 − ay − z2 − [m, x − y] + ε, x ∈ X. Then m ∈ ∂ε l(y) if and only if g(x) ≥ 0 for all x ∈ X . Calculating minimum of the quadratic function g we obtain the desired result. 5 Necessary and sufficient condition for the global minimum over a convex set Let f : X → IR+∞ and Ω ⊂ X . Definition 1 Let η ≥ 0. We say that x¯ is an η-global minimizer of f over Ω if f (x) − f (x) ¯ ≥ −ηx − x, ¯ x ∈ Ω.
Let f Ω = f +δΩ . Definition 1 can be represented in the following form: A point x¯ ∈ Ω ¯ where lη,x¯ (x) = −ηx − x. ¯ is an η-global minimizer of f over Ω if lη,x¯ ∈ ∂ L 1 f Ω (x) It is clear that x¯ is a global minimizer if and only if it is an η-global minimizer for all η > 0. We now present necessary and sufficient conditions for an approximate global minimizer of an inf-convex function over a convex set in terms of approximate subdifferentials of corresponding convex functions. Theorem 3 (1) Let Ω ⊂ X be a convex nonempty set and ( f t )t∈T be a family of convex continuous functions defined on X . Let f : Ω → IR be a function such ¯ − f (x). ¯ Then that f (x) = inf t∈T f t (x). Let x¯ ∈ Ω and εt = f t (x) (2) x¯ is an η-global minimizer with η > 0 over Ω if and only if for each t ∈ T there exists ε1,t ≥ 0, ε2,t ≥ 0 such that ε1,t + ε2,t = εt and 0 ∈ ∂ε1,t f t (x) ¯ + Nε2,t ,Ω (x) ¯ + B(0, η). (3)
(5.1)
x¯ is a global minimizer over Ω if and only if for each t ∈ T there exists ε1,t ≥ 0, ε2,t ≥ 0 such that ε1,t + ε2,t = εt and ∂ε1,t f t (x) ¯ ∩ (−Nε2,t ,Ω (x)) ¯ = ∅.
(5.2)
123
A. M. Rubinov, Z. Y. Wu
Proof (1) Let η > 0. A point x¯ is an η-global minimizer if and only if the function ¯ belongs to ∂ L 1 f (x). ¯ In view of Proposition 2 this means lη (x) = −ηx − x that for all t ∈ Ω there exists ε1,t ≥ 0, ε2,t ≥ 0 and ε3,t ≥ 0 such that ε1,t + ε2,t + ε3,t = εt and 0 ∈ ∂ε1,t f t (x) ¯ + Nε2,t ,Ω (x) ¯ + ∂ε3 ,t (−lη )(x)). ¯
(5.3)
In view of Corollary 2 we get ∂ε (−lη )(x) ¯ = B(0, η) for all ε > 0, so (5.3) can be rewritten as 0 ∈ ∂ε1,t f t (x) ¯ + Nε2 ,t,Ω (x) ¯ + B(0, η),
(2)
where ε1,t + ε2,t ≤ εt . Since ε - subdifferential contains ε -subdifferential for ε ≥ ε , we can choose ε1,t and ε2,t for which ε1,t + ε2,t = εt . Let x¯ be a global minimizer. Then (5.1) holds for each η > 0; in other words, for each η > 0 there exist numbers ε1,t,η ≥ 0 and ε2,t,η ≥ 0, such that ε1,t,η + ¯ bη ∈ Nε2,t,η ,Ω (x) ¯ and ε2,t,η = εt and there exist vectors aη ∈ ∂ε1,t,η f t (x), cη ∈ B(0, η) such that aη + bη + cη = 0. Then cη → 0 as η → 0+. Without loss of generality we can assume that there exist limη→0+ ε1,t,η := ε1,t ≥ 0 and limη→0+ ε2,t,η := ε2,t ≥ 0. Obviously ε1,t + ε2,t = εt . Since ε-subdifferential is bounded we can assume without loss of generality that there exists a weak limit limη→0+ aη := a. Then there exists a weak limit limη→0+ bη := b. We ¯ b ∈ Nε2,t ,Ω (x) ¯ and a + b = 0 so (5.2) holds. have a ∈ ∂ε1,t f t (x),
Corollary 2 Assume that assumptions of Theorem 3 hold and Ω = X . Then (1)
x¯ is an η-global minimizer with η > 0 over X if and only if for each t ∈ T ∂εt f t (x) ¯ ∩ B(0, η) = ∅.
(2)
(5.4)
x¯ is a global minimizer over X if and only if for each t ∈ T , 0 ∈ ∂εt f t (x). ¯
(5.5)
Indeed, (5.4) and (5.5) follow from (5.1) and (5.2), respectively, and the fact that Nε,X (x) ¯ = {0} for all ε. We now consider a special case where ( f t )t∈T is the family of quadratic functions from Proposition 1. Theorem 4 Let Ω ⊂ X be a convex nonempty set and f be a continuously differentiable function defined on an open set containing Ω. Assume that the mapping x → ∇ f (x) is Lipschitz continuous on Ω with the Lipschitz constant K . Let f t be the function defined on X by (3.3) and f˜t = f t + δΩ . Let η > 0 and a ≥ K . Then
123
Optimality conditions in global optimization and their applications
(i) x¯ ∈ Ω is an η-global minimizer of f over Ω if and only if ¯ ≤ η, t ∈ Ω, min{y : y ∈ ∂εt f˜t (x)}
(5.6)
where ¯ f (x) ¯ = [∇ f (t), x¯ −t]+ax¯ −t2 + f (t)− f (x), ¯ t ∈ Ω. εt = f t (x)−
(5.7)
(ii) If Ω = X then x¯ is an η- global minimizer if and only if 2a(x¯ −t)+∇ f (t) ≤ 2 [a∇ f (t), x¯ −t]+a 2 x¯ − t2 + a[ f (t) − f (x)]+η, ¯ t ∈ X. (5.8) Proof
(i) For t ∈ Ω consider the function (3.3): f t (x) = f (t) + [∇ f (t), x − t] + ax − t2 , x ∈ X.
It follows from Proposition 1 that f (x) = mint∈Ω f t (x). It is easy to check that f t (x) = avt − x2 − avt − t2 + f (t), with vt = t −
1 ∇ f (t). 2a
In view of Theorem 3 a point x¯ is an η-global minimizer over Ω if and only if for each t ∈ T there exist ε1,t and ε2,t such that ε1,t ≥ 0, ε2,t ≥ 0, ε1,t + ε2,t = εt
(5.9)
¯ + B(0, η) + Nε2 ,t,Ω (x). ¯ 0 ∈ ∂ε1,t f t (x)
(5.10)
and (5.1) holds: Since f˜t = f t + δΩ , we have ¯ = ∂εt f˜t,ε (x)
∂ε1,t f t (x) ¯ + Nε2 ,t,Ω (x), ¯
ε1,t ≥0,ε2,t ≥0, ε1,t +ε2,t =εt
so the intersection B(0, η) ∩ ∂εt f˜t (x) ¯ is not empty if and only if there exist ε1,t and ε2,t such that (5.9) and (5.10) holds. This implies that (5.10) is equivalent to (5.6). (ii) Let Ω = X and let lt (x) = avt − x2 , (x ∈ X ), (t ∈ Ω). ¯ = ∂ε lt (x) ¯ for all ε ≥ 0. Since Ω = X it follows that f˜t = f t , Clearly ∂ε f t (x) therefore due to (5.6), x¯ is an η-global minimizer if and only if ¯ ≤ η. min y : y ∈ ∂εt lt (x)
123
A. M. Rubinov, Z. Y. Wu
√ Applying Proposition 5 we conclude that ∂εt lt (x) ¯ = B(2a(x¯ − vt ), 2 aεt ) so we need to calculate √ min y : y ∈ B(2a(x¯ − vt ), 2 aεt ) . It is easy to check that this minimum is equal to max(0, qt ) with qt := 2a(x¯ − √ vt ) − 2 aεt , so the inequality (5.6) holds if and only if qt ≤ η. Since 1 ∇ f (t) = 2a(x¯ − t) + ∇ f (t) 2a(x¯ − vt ) = 2a x¯ − t + 2a and
aεt = a∇ f (t), x¯ − t + a 2 x¯ − t2 + a f (t) − f (x) ¯ we have qt = 2a(x¯ − t) + ∇ f (t)
¯ . −2 a∇ f (t), x¯ − t + a 2 x¯ − t2 + a f (t) − f (x) Thus the result follows.
6 Sufficient conditions for the constrained global minimum Necessary and sufficient conditions presented in the previous section are not tractable. However, results obtained in this section can be useful for examination of some tractable conditions. In this section we investigate a sufficient condition for the global minimum that is based on abstract convexity. Consider the following optimization problem (P): minimize f (x) subject to x ∈ Ω, (6.1) where f : X → IR and Ω ⊂ X . Proposition 6 (Sufficient condition for a global minimizer) Let x¯ ∈ Ω. Let L be a ¯ = ∅. If there exists a set of elementary functions l : X → R−∞ such that ∂ L f (x) function l ∈ L such that l ∈ ∂ L f (x) ¯ and l(x) ≥ l(x) ¯ f or all x ∈ Ω
(6.2)
then x¯ is a global minimizer of (P). Proof We have: f (x) − f (x) ¯ ≥ l(x) − l(x) ¯ ≥ 0, x ∈ Ω.
123
Optimality conditions in global optimization and their applications
We can express the result of Proposition 6 in terms of the L-normal set N−L ,Ω (x). ¯ ¯ so the sufficient Indeed, l(x) ≥ l(x) ¯ for all x ∈ Ω means that −l ∈ N−L ,Ω (x), condition under consideration can be represented in the following form: Proposition 7 Let x¯ ∈ Ω. If ∂ L f (x)∩(−N ¯ ¯ = ∅ then x¯ is a global minimizer −L ,Ω )( x) of (P). Remark 3 Assume that subdifferential calculus rules hold for the set L and functions f and δ S , in particular ∂ L ( f + δΩ )(x) ⊂ ∂ L f (x) + ∂ L δΩ (x).
(6.3)
Then (6.2) is also a necessary condition. Indeed, the definition of a global minimizer can be presented in terms of the L-subdifferentials as follows: 0 ∈ ∂ L ( f + δΩ )(x). ¯
(6.4)
If (6.3) holds and x¯ is a global minimizer then 0 ∈ ∂ L f (x) ¯ + ∂ L δΩ (x). ¯ This implies ¯ ∩ (−∂ L δΩ )(x) ¯ = ∅. Since l ∈ −∂ L δΩ (x) ¯ means that l(x) ≥ l(x) ¯ for all ∂ L f (x) x ∈ Ω, we get (6.2). If L is an additive set (l1 , l2 ∈ L ⇒ l1 + l2 ∈ L) then the inclusion opposite to (6.3) holds so (6.3) is equivalent to ∂ L ( f + δΩ )(x) = ∂ L f (x) + ∂ L δΩ (x).
(6.5)
Unfortunately (6.5) does not hold for many sets L of elementary functions (see detailed discussion in Jeyakumar V., Rubinov A.M., Wu Z.Y. in Generalized Fenchel’s conjugation formulas and duality for abstract convex functions, submitted paper). There is an example in (Jeyakumar V., Rubinov A.M., Wu Z.Y. in Sufficient global optimality conditions for non-convex quadratic minimization problems with box constraints. Submitted paper) which shows that (6.2) is not a necessary condition for a set L of quadratic functions. Condition (6.2) is tractable in some important cases (see [Jeyakumar V., Rubinov A.M., Wu Z.Y. in Non-convex quadratic minimization problems with quadratic constraints: global optimality conditions, submitted paper; Jeyakumar V., Rubinov A.M., Wu Z.Y. in Sufficient global optimality conditions for non-convex quadratic minimization problems with box constraints, submitted paper and Wu Z.Y., Jeyakumar V., Rubinov A.M. in Sufficient conditions for globally optimality of bivalent nonconvex quadratic programs, submitted paper] for details) and there is a hope that the number of these cases can be extended. It is interesting to compare (6.2) with a necessary condition in general situation, where (6.3) is not valid. We will compare (6.2) with the necessary and sufficient condition (5.2) given in Theorem 3. Assume that Ω is a nonempty convex set; f : X → IR is a function such that f (x) = inf t∈T f t (x) for x ∈ X , where f t : X → IR is a continuous convex function (t ∈ T ); L is a set of concave functions l : X → IR−∞ such that dom l ⊃ Ω for all l ∈ L.
123
A. M. Rubinov, Z. Y. Wu
First we present condition (6.2) it terms that were used in Theorem 3. Let x¯ ∈ Ω. ¯ and Assume that there exists a function l such that (6.2) holds, that is l ∈ ∂ L f (x) ¯ if and only if for l(x) − l(x) ¯ ≥ 0 for all x ∈ Ω. In view of Proposition 3, l ∈ ∂ L f (x) ¯ − f (x) ¯ and each t ∈ T there exist ε1,t ≥ 0 and ε2,t ≥ 0 such that ε1,t + ε2,t = f t (x) ∂ε1,t f t (x) ¯ ∩ (−∂ε2 ,t (−l)(x)) ¯ = ∅.
(6.6)
Condition (5.2) can be rewritten in the following form: x¯ is a global minimizer if and only if for each t ∈ T there exist ε1,t ≥ 0 and ε2,t ≥ 0 such that ε1,t + ε2,t = ¯ − f (x) ¯ and f t (x) ¯ ∩ (−Nε2 ,t,Ω (x)) ¯ = ∅. (6.7) ∂ε1,t f t (x) It follows from the aforesaid that there is only one distinction between sufficient condition (6.2) and necessary and sufficient condition (5.2): the subdifferential ¯ in (6.2) is replaced with the normal cone Nε2 ,t,Ω (x) ¯ in (6.7). Note that −∂ε2 ,t (−l)(x) ¯ and we do not need to (6.2) is expressed in terms of the L-subdifferentials ∂ L f (x) have a set of elementary function L in (5.2). We can provide a more precise comparison between (6.2) and (5.2) for a special set of elementary function L. This class depends on a point x. ¯ Recall that a function l : X → IR−∞ is called superlinear if l(x + y) ≥ l(x) + l(y) for all x, y ∈ X and l(λx) = λl(x) for x ∈ X and λ > 0. An upper semicontinuous superlinear function l has the following representation
where
¯ l(x) = inf{[u, x] : u ∈ ∂l(0)}, x ∈ X,
(6.8)
¯ ∂l(0) := {u ∈ X : [u, x] ≥ l(x)} for all x ∈ X.
(6.9)
If l : X → IR−∞ is a superlinear function then the function l = −l is sublinear and ¯ its subdifferential at zero ∂l (0) coincides with −∂l(0). Let x¯ be a global minimizer of the problem (P) defined by (6.1). For the investigation of this global minimizer we will use a set L x¯ which consists of the functions of the form m(x) = l(x − x), ¯ where l is a superlinear function. For the sake of a simplicity we assume that x¯ = 0; then we consider superlinear functions themselves. Proposition 8 Consider the problem (P). Assume that Ω is a nonempty convex set with 0 ∈ Ω and f : X → IR is a function such that f (x) = inf t∈T f t (x), x ∈ X , where f t : X → IR is a continuous convex function (t ∈ T ). Let L be the set of all superlinear functions l : X → IR−∞ with dom l ⊃ Ω. Then ∂ L f (0) ∩ (−N−L ,Ω (0)) = ∅
(6.10)
if and only if ∂εt f t (0) ∩ (−NΩ (0)) = ∅, where εt = f t (0) − f (0).
123
∀t ∈ T ,
(6.11)
Optimality conditions in global optimization and their applications
Proof Assume that (6.10) holds, so there exists l ∈ L such that l ∈ ∂ L f (0) and −l ∈ N−L ,Ω (0). This means that l(x) ≤ f (x) − f (0) ∀ x ∈ X and
l(x) ≥ 0 ∀ x ∈ Ω.
(6.12)
It follows from (6.8) and the second inequality in (6.12) that ∂(−l)(0) ⊂ NΩ (0).
(6.13)
The first inequality in (6.12) is equivalent to l(x) ≤ f t (x) − f t (0) + εt for each x ∈ X and t ∈ T , that is 0 ∈ ∂εt ( f + (−l))(0) for every t ∈ T . Applying Theorem 1 we obtain that ∂εt f t + (−l) (0) =
∂ε1 f t (0) + ∂ε2 (−l)(0) .
(6.14)
ε1 ,ε2 ≥0, ε1 +ε2 =εt
It is well-known (see for example [12]) and easy to check that ε-subdifferential of a sublinear function at zero with ε > 0 coincides with its subdifferential at zero; therefore (6.14) can be represented in the form: ∂εt f t + (−l) (0) = ∂ε f t (0) + ∂(−l)(0) .
(6.15)
0≤ε≤εt
Since ∂εt f t (0) ⊃ ∂ε f t (0) for 0 ≤ ε ≤ εt , (6.15) implies ∂εt f t + (−l) (0) = ∂εt f t (0) + ∂(−l)(0).
(6.16)
It follows from (6.16) and (6.13) that (6.11) holds. Conversely, let (6.11) hold, i.e., for any t ∈ T , there exists u t ∈ X such that u t ∈ ∂εt f t (0) and −u t ∈ NΩ (0). Let l(x) = inf t∈T [u t , x]. Then l : X → IR−∞ is ¯ a superlinear function and ∂l(0) = cl co{u t }t∈T . Since NΩ (0) is a closed and convex ¯ set, this implies that ∂l(0) ⊂ −NΩ (0). The inclusion u t ∈ −NΩ (0) also implies that l(x) = inf t∈T [u t , x] ≥ 0, x ∈ Ω so dom l ⊃ Ω. This means that l ∈ L. We have for x ∈ X and t ∈ T : l(x) ≤ [u t , x] ≤ f t (x) − f t (0) + εt = f t (x) − f (0) and so l(x) ≤ f (x) − f (0) for all x ∈ X . Hence l ∈ ∂ L f (0). Since −u t ∈ NΩ (0) it follows that [u t , x] ≥ 0 for each x ∈ Ω and t ∈ T . Hence l(x) ≥ 0 for each x ∈ Ω, whence −l ∈ N−L ,Ω (0). Proposition 8 allows us to compare necessary and sufficient condition (5.2) and sufficient condition (6.2) in the case under consideration. Let x¯ = 0 be a solution of the problem (P). Let L be the set of all superlinear functions l : X → IR−∞ with dom l ⊃ Ω. Then (5.2) can be represented in the following form: for any t ∈ T , there exist ε1,t ≥ 0 and ε2,t ≥ 0 such that ε1,t + ε2,t = εt and ∂ε1,t f t (0) ∩ − Nε2,t ,Ω (0) = ∅.
(6.17)
123
A. M. Rubinov, Z. Y. Wu
Due to Proposition 8 sufficient condition (6.2) can be represented in the form ∂εt f t (0) ∩ − NΩ (0) = ∅,
∀t ∈ T.
(6.18)
Thus (6.2) is a special case of (5.2) corresponding to ε1,t = εt , ε2,t = 0. 7 Some properties of the global minimum In this section we will examine some properties of the global minimum. We start with functions f : X → IR having the Lipschitz continuous gradient mapping. Theorem 5 Let f ∈ C 1 (X ) and let the mapping x → ∇ f (x) be Lipschitz continuous: K :=
sup
x,y∈X,x= y
∇ f (x) − ∇ f (y) < +∞. x − y
Let a ≥ K . Let a point x¯ ∈ X be a global minimizer of f . Then 1 ∇ f (t)2 ≤ f (t) − f (x), ¯ t ∈ X. 4a
(7.1)
Proof A point x¯ is a global minimizer on X if and only if (5.8) holds for all η > 0, that is
2a(x¯ − t)+∇ f (t) ≤ 2 a∇ f (t), x¯ − t]+a 2 x¯ − t2 +a[ f (t) − f (x) ¯ , t ∈ X. (7.2) This inequality can be represented in the form
∇ f (t), ∇ f (t) − 4a f (t) − f (x) ¯ ≤ 0, t ∈ X
which is equivalent to (7.1).
We now give a version of Theorem 5 for functions defined on a set Ω ⊂ X with the nonempty interior. Theorem 6 Consider the space X and assume that X is equipped with not only the norm · , but also with a norm · ◦ , which is equivalent to · . Let Ω ⊂ X be a set with int Ω = ∅ and let f be a continuously differentiable function defined on an open set containing Ω. Assume that the mapping x → ∇ f (x) is Lipschitz on Ω: K :=
sup
x= y,x,y∈Ω
∇ f (x) − ∇ f (y) < +∞. x − y
Let x¯ ∈ int Ω be a global minimizer of f over Ω. Consider the ball B◦ (x, ¯ r) = {x : x − x ¯ ◦ ≤ r } ⊂ int Ω and let M := max ∇ f (t)◦ : t ∈ B◦ (x, ¯ r) .
123
(7.3)
Optimality conditions in global optimization and their applications
Let q > 0 be a number such that B◦ (x, ¯ r + q) ⊂ Ω and let
M a ≥ max K , 2q Then
.
(7.4)
1 ¯ t ∈ B◦ (x, ¯ r ). ∇ f (t)2 ≤ f (t) − f (x), 4a
(7.5)
M . Then r1 ≤ r + q, so B◦ (x, ¯ r1 ) ⊂ Ω. Applying Proposition 1 Proof Let r1 := r + 2a ¯ r1 ), where f t (x) = we conclude that f (x) = mint∈B◦ (x,r ¯ 1 ) f t (x) for any x ∈ B◦ ( x, f (t) + [∇ f (t), x − t] + ax − t2 . Thus, f (x) ≤ mint∈B◦ (x,r ¯ ) f t (x) for any x ∈ ¯ r1 ). Let us calculate vt ∈ argmin x∈X f t (x). Since vt is a solution of the equation B◦ (x, ∇ f t (x) = 0, we have
vt = t −
∇ f (t) . 2a
So 1 1 ∇ f (t)2 min f t (x) = f t (vt ) = f (t) + ∇ f (t), − ∇ f (t) + x∈X 2a 4a 1 ∇ f (t)2 . = f (t) − 4a For any t ∈ B◦ (x, ¯ r ) it holds: ∇ f (t) ¯ ◦ ≤ t − x ¯ ◦+ vt − x 2a ≤ r + q = r1 . ◦ So, vt ∈ B◦ (x, ¯ r1 ) and f (t) −
1 ∇ f (t)2 = f t (vt ) = min f t (x) = min f t (x). x∈Ω x∈B◦ (x,r ¯ 1) 4a
Then f (x) ¯ = ≤
min
f (x) =
min
min
x∈B◦ (x,r ¯ )
x∈B◦ (x,r ¯ 1 ) t∈B◦ (x,r ¯ )
Thus (7.5) is valid.
min
x∈B◦ (x,r ¯ 1)
f t (x) =
f (x) min
t∈B◦ (x,r ¯ )
f (t) −
1 ∇ f (t)2 . 4a
We now present a version of Theorem 6 for functions f : X → IR that have the boundedly Lipschitz continuous gradient mapping x → ∇ f (x). This means that for
123
A. M. Rubinov, Z. Y. Wu
all r > 0 it holds: K r :=
∇ f (x) − ∇ f (y) < +∞. x − y
sup
x= y,x,y≤r
(7.6)
Theorem 7 Let f ∈ C 1 (X ). Let the mapping x → ∇ f (x) be boundedly Lipschitz continuous. For any r > 0, let Mr :=
sup ∇ f (t) and r1 := r +
t∈B(0,r )
Mr , 2K r
where K r is defined by (7.6). If x¯ is a global minimizer of f then for all r ≥ x ¯ it holds: 1 ∇ f (t)2 ≤ f (t) − f (x), ¯ t ∈ B(0, r ), (7.7) ar1 where ar1 ≥ K r1 is an arbitrary number. The proof is similar to that of Theorem 6 and we present only its scheme. Proof Let r > 0. It follows from Proposition 1 that f (x) = mint∈B(0,r1 ) f t (x) for any x ∈ B(0, r1 ), where f t (x) = f (t) + [∇ f (t), x − t] + ar1 x − t2 and ar1 ≥ K r1 . Thus, f (x) ≤ inf t∈B(0,r ) f t (x) for any x ∈ B(0, r1 ). The same argument as in the proof of Theorem 6 shows that the global minimizer x¯t of f t (x) over X is located in B(0, r1 ). Therefore f t (x¯t ) = min f t (x) = x∈X
min
x∈B(0,r1 )
f t (x) = f (t) −
1 ∇ f (t)2 . 4ar1
If x¯ is a global minimizer of f (x) on X and r ≥ x, ¯ then f (x) ¯ = ≤
min
x∈B(0,r )
f (x) =
min ( f (t) −
t∈B(0,r1 )
min
x∈B(0,r1 )
f (x)
1 ∇ f (t)2 , forany ar ≥ K r1 . 4ar
Thus, (7.5) holds. 8 Global optimization and inequalities
In this section we demonstrate that a theory of global optimization has some interesting application to the theory of inequalities. Indeed, many inequalities can be presented in the form f (x) ≥ f (x), ¯ x ∈Ω
123
Optimality conditions in global optimization and their applications
where Ω ⊂ X , so the examination of these inequalities can be reduced to examination of a global optimization problem minimize f (x) subject to x ∈ Ω. Assume that we have two inequalities: f (x) − f (x) ¯ ≥0
(8.1)
f (x) − f (x) ¯ ≥ u(x) where u(x) ≥ 0.
(8.2)
and We say that (8.2) is sharper that (8.1) if there is a vector x such that u(x) > 0. Results obtained in the previous section can be used for construction of inequalities that are sharper than the given one. For the sake of definiteness we consider the situation described in Theorem 6. We assume that conditions of this theorem hold and we will use the same notation. Consider the function Q a ( f ) := g : Ω → IR, where g(x) = f (x) −
1 ∇ f (x)2 , x ∈ Ω. 4a
(8.3)
Here a is the number defined by (7.4). The function g enjoys the following properties: 1. 2.
g(x) ≤ f (x) for all x ∈ Ω; g(x) = f (x) if and only if x is a stationary point of f ; it follows directly from the definition of g. Let x¯ ∈ int Ω be a global minimizer of f over Ω, so x¯ is a global minimizer of f ¯ r + q). Then x¯ is also a global minimizer of g over B◦ (x, ¯ r + q) and over B◦ (x, min x∈B f (x) = min x∈B g(x). Indeed, since x¯ is a global minimizer of f then (7.5) holds; the inclusion x¯ ∈ int Ω implies ∇ f (x) ¯ = 0. Therefore (7.5) can be presented in the following form: f (x) ¯ −
1 1 ∇ f (x) ¯ 2 ≤ f (x) − ∇ f (x)2 , (x ∈ B◦ (x, ¯ r + q)). 4a 4a
¯ r + q)). Hence x¯ is a global This can be rewritten as g(x) ¯ ≤ g(x), (x ∈ B◦ (x, ¯ r + q). Since ∇ f (x) ¯ = 0, it follows that g(x) ¯ = f (x). ¯ minimizer of g over B◦ (x, Then g(x) = min f (x). (8.4) min x∈B◦ (x,r ¯ +q)
x∈B◦ (x,r ¯ +q)
If f is a twice-continuously differentiable function then the function g is differentiable and 1 2 1 2 ∇ f (x)∇ f (x) = I d − ∇ f (x) ∇ f (x). ∇g(x) = ∇ f (x) − (8.5) 2a 2a It follows from (8.5) that a point y is a stationary point of g if either y is a stationary point of f or ∇ f (y) is an eigenvector of ∇ 2 f (y) corresponding to the eigenvalue 2a.
123
A. M. Rubinov, Z. Y. Wu
The inequality g(x) − g(x) ¯ ≥ 0 can be rewritten in the form f (x) − f (x) ¯ ≥
1 ∇ f (x)2 , 4a
which is sharper than the inequality f (x)− f (x) ¯ ≥ 0. If f is a convex function defined on a convex set Ω then g(x) > f (x) for all x ∈ int Ω which are different from a global minimizer of f . Remark 4 Let Ω = X and f ∈ C ∞ (X ) be a boundedly Lipschitz function. Let a > 0 be an arbitrary number and Q a : C ∞ (X ) → C ∞ (X ) be an operator defined by Q a ( f )(x) = f (x) −
1 ∇ f (x)2 . 4a
Then Q a ( f )(x) ≤ f (x) for all x ∈ X and Q a ( f )(x) = f (x) if x is a stationary point. Let 1(x) = 1 for all x ∈ X . Then Q a (c1) = c1 for all c ∈ IR. Let a k be a sequence of positive numbers. Then f ≥ Q a1 f ≥ Q a2 Q a1 f ≥ · · · ≥ Q ak . . . Q a2 Q a1 f and for a stationary point x of f we have f (x) = Q a 1 f (x) = Q a 2 Q a 1 f (x) = · · · = Q a k . . . Q a 2 Q a 1 f (x). Using the described construction we will present a sharper version of the wellknown inequality between the arithmetic mean and the geometric mean which asserts that 1 1 (x1 + · · · + xn ) > (x1 . . . xn ) n , x ∈ IRn+ , x = λ1 with λ ≥ 0. n
(8.6)
Here 1 = (1, . . . , 1)T ∈ IRn+ . Recall that IRn+ is the cone of n-vectors with nonnegative coordinates and IRn++ is the cone of n-vectors with positive coordinates. Theorem 8 Let λ > r be positive numbers. Let
aλ,r
⎞ ⎛ n−1 n λ+r 1 − 1⎟ ⎜ n − 1 2 (λ + d) λ−r ⎟. = min max ⎜ , ⎝ r d > 0 consider the ball Vλ,d := B∞ (λ1, d) = x ∈ IRn : λ1 − x∞ ≤ d = {x ∈ IRn : λ − d ≤ xi ≤ λ + d, . . . , i = 1, . . . , n}.
(8.11) (8.12)
1
Since d < λ it follows that Vλ,d ⊂ ∇ρi (x) for x ∈ Vλ,d . We have
IRn++ .
πn (x) n Let ρi (x) = . We need to estimate xi
! ! ! ∂ρi ! 1 1 1 1 λ+d ! != (x) !∂x ! n x x (x1 . . . xn ) n ≤ n (λ − d)2 i = j. j i j ! ! ! ∂ρi ! n−1 1 1 n−1 λ+d ! ! ! ∂ x (x)! = n x 2 (x1 . . . xn ) n ≤ n (λ − d)2 , i i so
n − 1 (λ + d)2 (n − 1)2 (λ + d)2 ∇ρi (x) ≤ + 2 4 n (λ − d) n2 (λ − d)4 1 n − 1 2 (λ + d) = , (x ∈ Vλ,d ). n (λ − d)2
21
(8.13)
123
A. M. Rubinov, Z. Y. Wu
Let x, y ∈ Vλ,d . Applying the mean value theorem we conclude that there exist numbers θi ∈ [0, 1], i = 1, . . . , n such that 1 ∇ f (x) − ∇ f (y) = n
n
1 2
(ρi (x) − ρi (y))2
i=1
1 n 2 1 = [∇ρi (x + θi (y − x), (x − y)]2 n i=1
1 n 2 1 2 2 ≤ ∇ρi (x + θi (y − x) x − y . n i=1
Since x, y ∈ Vλ,d it follows that x + θi (y − x) ∈ Vd for all i. Applying (8.13) we conclude that ∇ f (x) − ∇ f (y) ≤ a1 (λ, d)x − y, x, y ∈ Vλ,d where
a1 (λ, d) =
n−1 n
1 2
(λ + d) . (λ − d)2
(8.14)
Hence mapping x → ∇ f (x) is Lipschitz continuous on Vλ,d with the Lipschitz constant K ≤ a1 (λ, d). We will apply Theorem 6 to the set Ω = Vλ,d where r < d < λ and the global minimizer x¯ = λ1 of the function f . Assume that the norm ·◦ that was used in Theorem 6 coincides with ·∞ . Let q = d−r . Let us estimate M = max{∇ f (x)∞ : x ∈ Vλ,r }. Due to symmetry it is enough to estimate only the first coordinate 1 − gradient ∇ f (x). It is easy to see that ! 1 ! πn (x) n ! M ≤ max !1 − x∈Vλ,r ! x1
of the
! ! 1 ! n−1 ! ! x2 λ+r n xn n !! ! ! ... − 1. ! = max !1 − !≤ ! x∈Vλ,r ! x1 x1 ! λ−r
Let a2 (λ, d, r ) =
λ+r λ−r
n−1 n
−1
2(d − r )
and a(λ, d, r ) = max(a1 (λ, d), a2 (λ, d, r )) ⎛ ⎞ n−1 n λ+r 1 − 1⎟ ⎜ n − 1 2 (λ + d) λ−r ⎟. = max ⎜ , ⎝ 2 n (λ − d) 2(d − r ) ⎠
123
1
πn (x) n x1
Optimality conditions in global optimization and their applications
Note that limd→λ−0 a(λ, d, r ) = limd→r +0 a(λ, d, r ) = +∞ so the function d → a(λ, d, r ) attains its minimum on the segment (r, λ). Let aλ,r = minr
