Inner Approximations of Completely Positive ... - Semantic Scholar
Inner Approximations of Completely Positive Reformulations of Mixed Binary Quadratic Optimization Problems: A Unified Analysis∗ E. Alper Yıldırım† August 24, 2015 Abstract Every quadratic optimization problem with a mix of continuous and binary variables can be equivalently reformulated as a completely positive optimization problem, i.e., a linear optimization problem over the convex but computationally intractable cone of completely positive matrices. In this paper, we focus on general inner approximations of the cone of completely positive matrices on instances of completely positive optimization problems that arise from the reformulation of mixed binary quadratic optimization problems. We provide a characterization of the feasibility of such an inner approximation as well as the optimal value of a feasible inner approximation. For polyhedral inner approximations, our characterization implies that computing the corresponding approximate solution reduces to an optimization problem over a finite set. Our characterization yields, as a byproduct, an upper bound on the gap between the optimal value of an inner approximation and that of the original instance. We discuss the implications of the error bound for standard and box-constrained quadratic optimization problems. ∗
¨ ITAK ˙ This work was supported in part by TUB (The Scientific and Technological Research Council of
Turkey) Grant 112M870. † Department of Industrial
Engineering,
Ko¸c
University,
34450
Sarıyer,
Istanbul,
Turkey
¨ ˙ (Turkish Academy of ([email protected]). The author was supported in part by TUBA-GEB IP Sciences Young Scientists Award Program).
1
Key words: Completely positive cone, mixed binary quadratic optimization problems, inner approximations, polyhedral approximations. AMS Subject Classifications: 90C20, 90C25.
1
Introduction
The cone of completely positive matrices is given by C n := conv{uuT : u ∈ Rn+ }, where conv{·} denotes the convex hull and Rn+ denotes the nonnegative orthant in Rn . A completely positive optimization problem is a linear optimization problem over an affine subset of the cone of completely positive matrices: (CoP) Here, hU, V i :=
min {hC, Xi : hAi , Xi = fi ,
Pn Pn i=1
j=1
i = 1, . . . , p,
X ∈ C n} .
Uij Vij for any U ∈ S n and V ∈ S n , where S n denotes the set of
real symmetric n × n matrices; C ∈ S n , Ai ∈ S n , i = 1, . . . , p, and f ∈ Rp constitute the parameters; and X ∈ S n denotes the decision variable. Burer [6] showed that every quadratic optimization problem with a mix of binary and continuous variables, henceforth referred to as a mixed binary quadratic optimization problem, can be equivalently reformulated as a completely positive optimization problem. On the one hand, this result implies that a rather large class of nonconvex optimization problems admits a compact reformulation as a convex optimization problem. On the other hand, it follows from this result that a completely positive optimization problem is, in general, NP-hard. Indeed, deciding if a given matrix belongs to C n is a computationally intractable problem [10, 16]. Despite the fact that Burer’s reformulation result does not seem to be helpful from a computational complexity point of view, it provides a new theoretical and computational perspective on approximately solving mixed binary quadratic optimization problems. For 2
instance, the computationally intractable cone of completely positive matrices can be approximated from the inside or from the outside by a tractable convex cone. Therefore, for a given mixed binary quadratic optimization problem, one can obtain upper and lower bounds on its optimal value by utilizing approximations of C n in the corresponding completely positive reformulation. Indeed, various tractable inner and outer approximations of C n have been proposed in the literature. For further details, the reader is referred to [2, 7, 11]. In this paper, we focus on inner approximations of the cone of completely positive matrices in the context of completely positive optimization problems arising from the reformulation of mixed binary quadratic optimization problems. Given a particular inner approximation of C n and an instance of such a completely positive optimization problem, we provide a characterization of the feasibility of the corresponding approximation as well as the optimal value of a feasible inner approximation. For the special case of polyhedral inner approximations, our characterization implies that computing the corresponding approximate solution reduces to an optimization problem over a finite set. Finally, as a byproduct, our characterization yields an upper bound on the approximation error, i.e., the difference between the optimal value of the inner approximation and that of the original mixed binary quadratic optimization problem. We discuss the implications of the error bound for standard and box-constrained quadratic optimization problems. This paper is organized as follows. We define our notation in Section 1.1. We review Burer’s reformulation of mixed binary quadratic optimization problems and present a useful decomposition result for this formulation in Section 2. Section 3 is devoted to the characterization of the feasibility, optimal value, and the error bound for general inner approximations. We focus on inner approximations generated by a set of nonnegative rank one matrices in Section 4. For this particular family of inner approximations, we show that our results can be stated in terms of the original mixed binary quadratic optimization problem, in contrast with our corresponding results for general inner approximations which are stated in terms of the completely positive formulation in Section 3. We also consider mixed binary quadratic optimization problems with a bounded feasible region and discuss the implications of our re-
3
sults on standard and box-constrained quadratic optimization problems. Section 5 concludes the paper.
1.1
Notation
We use Rn , Rn+ , and S n to denote the n-dimensional Euclidean space, the nonnegative orthant, and the space of n × n real symmetric matrices, respectively. We reserve calligraphic letters for subsets of S n . The set of completely positive matrices is denoted by C n . The inner P P product on S n is the trace inner product given by hU, V i := ni=1 nj=1 Uij Vij for any U ∈ S n and V ∈ S n . The norm associated with the trace inner product is the Frobenius norm denoted by kU kF := hU, U i1/2 . The `p -norm on Rn is denoted by k · kp , where p ∈ {1, 2, ∞}. The set of nonnegative integers is denoted by N. Similarly, we use Nn to denote the set of ndimensional vectors all of whose components are given by nonnegative integers. For u ∈ Rn , we denote its jth component by uj , j = 1, . . . , n. For u ∈ Rn+1 , we start the indexing from zero. Similarly, Uij denotes the (i, j) entry of a matrix U ∈ S n , i = 1, . . . , n; j = 1, . . . , n, and we start both indices from zero for U ∈ S n+1 . We adopt Matlab-like notation. For a vector u, we use u1:k ∈ Rk to denote the subvector given by the components of u indexed by 1, . . . , k. We use similar notation for matrices, i.e., UI,J denotes the submatrix of U whose rows and columns are given by the rows and columns of U indexed by the sets I and J, respectively. For two vectors u ∈ Rn1 and v ∈ Rn2 , we use [u; v] ∈ Rn1 +n2 to denote the vector obtained by the vertical concatenation of u and v. The unit vectors in Rn are denoted by ej , j = 1, . . . , n, with the convention that indexing starts from zero for Rn+1 . We reserve e to denote the vector of all ones whose dimension will always be clear from the context. We use 0 to denote the real number zero, the vector of all zeroes as well as the matrix of all zeroes in the appropriate dimension, which will always be clear from the context.
4
2
Burer’s Reformulation and a Decomposition Result
A mixed binary quadratic optimization problem is given by (BQP)
min xT Qx + 2cT x
s.t. aTi x = bi ,
i = 1, . . . , m,
x ≥ 0, xj ∈ {0, 1},
j ∈ B,
where Q ∈ S n , c ∈ Rn , ai ∈ Rn , i = 1, . . . , m, bi ∈ R, i = 1, . . . , m, and B ⊆ {1, . . . , n} constitute the data of the problem and x ∈ Rn denotes the decision variable. We denote the feasible region of (BQP) by F , i.e., F := {x ∈ Rn : aTi x = bi ,
i = 1, . . . , m,
x ≥ 0,
xj ∈ {0, 1},
j ∈ B},
(1)
and we assume that F 6= ∅. The set obtained from F by ignoring the binary constraints is denoted by L: L := {x ∈ Rn : aTi x = bi ,
i = 1, . . . , m,
x ≥ 0}.
(2)
i = 1, . . . , m,
d ≥ 0}.
(3)
The recession cone of L is given by L∞ := {d ∈ Rn : aTi d = 0,
Burer [6] makes the following assumption, referred to as the key assumption: x ∈ L ⇒ 0 ≤ xj ≤ 1,
j ∈ B.
(4)
Note that the key assumption can always be enforced by adding the redundant constraints xj ≤ 1 for each j ∈ B, if necessary, and then introducing slack variables to convert the formulation into the form of (BQP). The key assumption implies that d ∈ L∞ ⇒ dj = 0,
5
j ∈ B.
(5)
Under the key assumption (4), Burer [6] shows that (BQP) is equivalent to the following completely positive optimization problem: (CP) min hQ, Xi + 2 cT x aTi x = bi ,
i = 1, . . . , m,
aTi Xai = b2i ,
i = 1, . . . , m,
s.t.
j ∈ B,
Xjj = xj ,
1 xT x X
∈ C n+1 .
Note that (CP) can be easily represented in the form of (CoP) by using the identity aTi Xai = hai aTi , Xi for i = 1, . . . , m. Similarly, let us denote the feasible region of (CP) by aTi x = bi , 1 xT ∈ C n+1 : aT Xai = b2 , F= i i x X X =x , jj
j
F: i = 1, . . . , m, i = 1, . . . , m, . j∈B
The recession cone of F is given by T 0 0T a Dai = 0, i = 1, . . . , m, ∈ C n+1 : i L∞ = . 0 D Djj = 0, j∈B
(6)
(7)
By (1) and (6), x∈F ⇒
1 x
1 x
T ∈ F.
(8)
Similarly, by (3), (5), and (7), d ∈ L∞ ⇒
0 d
0 d
T ∈ L∞ .
The equivalence between (BQP) and (CP) relies on the following main result.
6
(9)
Proposition 2.1 (Burer [6]) Given an instance of (BQP), let Y ∈ C n+1 be a feasible solution of (CP) given by Y =
X h∈H
ρh rh
T
ρh
,
rh
where H is a finite set, ρh ∈ R+ and rh ∈ Rn+ for each h ∈ H. Let H+ := {h ∈ H : ρh > 0},
H0 := {h ∈ H : ρh = 0}.
Then, (i) (1/ρh )rh ∈ F for each h ∈ H+ , and (ii) rh ∈ L∞ for each h ∈ H0 . Our next proposition presents a useful decomposition result for a given feasible solution of (CP). Proposition 2.2 Let Y ∈ C n+1 be a feasible solution of (CP) and suppose that Y =
X
Vp+
p∈P
X
W z,
z∈Z
where P and Z are finite sets, V p ∈ C n+1 and V00p > 0 for each p ∈ P , W z ∈ C n+1 and z = 0 for each z ∈ Z. Then, we have (i) P 6= ∅, (ii) (1/V00p )V p ∈ F for each p ∈ P , and W00
(iii) W z ∈ L∞ for each z ∈ Z. Proof. Since Y ∈ F, it follows that Y00 = 1, which implies that P 6= ∅ establishing (i). Suppose that V p , p ∈ P , and W z , z ∈ Z, admit the following decompositions: Vp =
X h∈H0p
Wz =
X h∈H0z
0 v hp 0 w
hz
T
0 v hp 0 w
hz
+
X p h∈H+
σhp shp
T ,
7
z ∈ Z,
σhp shp
T ,
p ∈ P,
where H0p and H+p are finite sets for each p ∈ P ; H0z is a finite set for each z ∈ Z; v hp ∈ Rn+ for each p ∈ P and h ∈ H0p ; shp ∈ Rn+ and σhp > 0 for each p ∈ P and h ∈ H+p ; and whz ∈ Rn+ for each z ∈ Z and h ∈ H0z . Note that H+p 6= ∅ since V00p > 0 for each p ∈ P . By Proposition 2.1, we obtain v hp ∈ L∞ ,
h ∈ H0p ,
p ∈ P,
where L∞ is given by (3), and
1 σhp
and whz ∈ L∞ ,
shp ∈ F,
p ∈ P,
z ∈ Z,
h ∈ H0z ,
h ∈ H+p ,
(10)
(11)
where F is defined as in (1). Since L∞ is a convex cone, it follows from (9) and (10) that W z ∈ L∞ for each z ∈ Z, which establishes (iii), and V p,0 ∈ L∞ for each p ∈ P , where T X 0 0 , p ∈ P. V p,0 := hp hp v v h∈H0p Similarly, let X
V p,+ :=
p h∈H+
σhp s
hp
σhp s
hp
T ,
p ∈ P,
so that V p = V p,+ + V p,0 for each p ∈ P . Let us now define ˆp
V :=
1 V00p
p
V =
1 V00p,+
(V p,+ + V p,0 ),
p ∈ P.
We need to show that Vˆ p ∈ F for each p ∈ P . Let us fix p ∈ P . By (8) and (11), we have T 1 1 ∈ F, h ∈ H+p . (12) 1 1 hp hp s s σhp σhp Note that Vˆ p =
!
1 P
p h∈H+
2 σhp
X 2 σhp p h∈H+
1
1 σhp
8
shp
1
1 σhp
T shp
+ V p,0 .
By (12), the first term above is given by a convex combination of the elements of F and the second term, which is a positive multiple of V p,0 , belongs to L∞ . It follows that Vˆ p ∈ F as desired, thereby establishing (ii).
3
General Inner Approximations
In this section, we consider replacing the difficult conic constraint in (CP) by an arbitrary inner approximation. Let K be a (possibly infinite) nonempty index set and let M k ∈ C n+1 be a nonzero matrix for each k ∈ K. Let M := {M k : k ∈ K} ⊆ C n+1 . We consider the following cone: I(M) := cone(M),
(13)
where cone(·) denotes the conic hull, i.e., the smallest convex cone (with respect to inclusion) that contains M ∪ {0}. Note that, by construction, we have I(M) ⊆ C n+1 . We remark that any inner approximation of C n+1 can be represented by (13) for an appropriate choice of M. In fact, there exist inner approximation hierarchies for the cone of completely positive matrices, i.e., there are sequences of tractable nested cones that provide increasingly better inner approximations with the property that such sequences are, in some sense, exact in the limit. The reader is referred to [17] and [15] for inner approximation hierarchies consisting of polyhedral and spectrahedral cones (i.e., cones given by the intersection of a linear subspace and the set of positive semidefinite matrices), respectively. For instance, each inner approximation in either of these hierarchies can be represented in the form of (13) for an appropriate choice of M. By Carath´eodory’s theorem for conic hulls, we have ¯ ⊆ K, Y ∈ I(M) ⇔ ∃ K
¯ ≤ |K|
(n + 1)(n + 2) , 2
Y =
X
λk M k ,
λk ≥ 0,
¯ k ∈ K,
¯ k∈K
(14) 9
i.e., any element of I(M) can be represented by a nonnegative combination of a finite number of matrices M k . We will refer to (14) as a finite conic representation of Y ∈ I(M). We consider replacing the conic constraint X ∈ C n+1 in (CP) by X ∈ I(M). The next proposition provides a characterization of the feasibility of the corresponding inner approximation of (CP). Furthermore, for any feasible solution of the resulting inner approximation, we present a useful representation result that will subsequently be used to characterize the corresponding optimal solution. Proposition 3.1 Given an instance of (CP), a nonempty index set K, and M := {M k : k ∈ K} ⊆ C n+1 , suppose that the conic constraint X ∈ C n+1 in (CP) is replaced by X ∈ I(M), where I(M) is defined as in (13). Let k K0 = k ∈ K : M00 =0 ,
k K+ = k ∈ K : M00 >0 .
(15)
(i) We have F ∩I(M) 6= ∅ (i.e., the inner approximation of (CP) has a nonempty feasible region) if and only if K++ 6= ∅, where k k K++ := k ∈ K+ : F ∩ cone(M k ) 6= ∅ = k ∈ K+ : 1/M00 M ∈F .
(16)
(ii) Suppose that F ∩I(M) 6= ∅ and let Y ∈ F ∩I(M). For any finite conic representation ¯ ∩ K++ such that λk > 0. Furthermore, of Y given by (14), there exists k ∈ K ¯ ∩ K+ , ∀k∈K
λk > 0 ⇒ k ∈ K++ ,
¯ ∩ K0 , ∀k∈K
λk > 0 ⇒ M k ∈ L∞ .
k Proof. Let us first consider part (i). If K++ 6= ∅, then (1/M00 )M k ∈ F ∩ I(M) for any
k ∈ K++ , which implies that F ∩ I(M) 6= ∅. Note that the reverse implication of part (i) directly follows from part (ii). Therefore, we will proceed with the proof of part (ii). Suppose that F ∩ I(M) 6= ∅ and let Y ∈ F ∩ I(M). For any finite conic representation of Y given by (14), let us decompose Y = Y + + Y 0 , where Y + :=
X
λk M k ,
¯ K∩K +
Y 0 :=
X ¯ K∩K 0
10
λk M k .
¯ By Proposition 2.2, Y + 6= 0. Therefore, Note that M k ∈ C n+1 and λk ≥ 0 for each k ∈ K. ¯ ∩ K+ such that λk > 0, i.e., K ¯ ∩ K+ 6= ∅. Furthermore, for each there exists k ∈ K k k ¯ ∩ K+ such that λk > 0, we have (1/λk M00 )M k ∈ F , i.e., k ∈ K++ . )λk M k = (1/M00 k∈K
Therefore, K++ 6= ∅, which establishes the reverse implication of (i). Finally, λk M k ∈ L∞ ¯ ∩ K0 , i.e., for each k ∈ K ¯ ∩ K0 such that λk > 0, we have M k ∈ L∞ , which for each k ∈ K concludes the proof of part (ii).
By Proposition 3.1, the feasibility of an inner approximation of (CP) is simply determined by whether the cone generated by any element of M intersects the feasible region F of (CP). The next example illustrates that such a characterization does not necessarily hold for the feasible region of an arbitrary completely positive optimization problem. Example 3.1 Consider the following set: + * 2 0 , X = 1, F := X ∈ C 2 : 0 −1
*
0 1 1 0
+
,X
=2 .
One can easily verify that 1 1 1 0 + µ :µ≥0 . F= 1 1 0 2 Let
1 0 1 1 ⊂ C 2. M= , 0 1 1 2
We have
2 1 , F ∩ I(M) = 1 3
whereas the cone generated by either matrix in M does not intersect F. Our next result gives a useful characterization of the upper bound obtained by replacing the conic constraint X ∈ C n+1 in (CP) by X ∈ I(M). 11
Proposition 3.2 Given an instance of (CP), a nonempty index set K, and M := {M k : k ∈ K} ⊆ C n+1 , suppose that the conic constraint X ∈ C n+1 in (CP) is replaced by X ∈ I(M), where I(M) is defined as in (13). Let K0 and K+ be defined as in (15) and K++ as in (16) and let ν(M) denote the optimal value of the corresponding inner approximation of (CP). (i) If K++ = ∅, then the inner approximation of (CP) is infeasible and ν(M) := +∞. k (ii) If K++ 6= ∅ and there exists k ∈ K0 such that M k ∈ L∞ and hQ, M1:n,1:n i < 0, then
the inner approximation of (CP) is unbounded below and ν(M) = −∞. k (iii) If K++ 6= ∅ and inf k∈K0 :M k ∈L∞ hQ, M1:n,1:n i ≥ 0, then
k k k ν(M) = inf (1/M00 ) hQ, M1:n,1:n i + 2cT M1:n,1 . k∈K++
Proof. Note that (i) follows directly from part (i) of Proposition 3.1. Considering (ii), let k∗ ∈ K++ and suppose that there exists k ∈ K0 such that M k ∈ L∞ k∗ k and hQ, M1:n,1:n i < 0. Then, Y µ := (1/M00 )M k∗ + µM k ∈ F ∩ I(M) for each µ ≥ 0 by
Proposition 3.1. Clearly, µ µ k∗ k∗ k∗ k hQ, Y1:n,1:n i + 2cT Y1:n,1 = (1/M00 ) hQ, M1:n,1:n i + 2cT M1:n,1 + µhQ, M1:n,1:n i → −∞ as µ → +∞, which establishes (ii). k Finally, suppose that K++ 6= ∅ and inf k∈K0 :M k ∈L∞ hQ, M1:n,1:n i ≥ 0. By Proposition 3.1, k )M k ∈ F for each k ∈ K++ . Therefore, (1/M00
k k k . ν(M) ≤ η := inf (1/M00 ) hQ, M1:n,1:n i + 2cT M1:n,1 k∈K++
(17)
We will use a contradiction argument to establish the reverse inequality. Suppose, for a contradiction, that ν(M) < η. Then, there exists a sequence {Y r : r ∈ N} ⊆ F ∩ I(M) r r such that hQ, Y1:n,1:n i + 2cT Y1:n,1 → ν(M) as r → ∞. Therefore, there exists r∗ ∈ N such r∗ r∗ that hQ, Y1:n,1:n i + 2cT Y1:n,1 < η. Consider any finite representation of Y r∗ given by (14):
Y r∗ =
X
λk M k +
¯ k∈K∩K +
X ¯ k∈K∩K 0
12
λk M k .
¯ ∩ K+ such that λk > 0 and for each k ∈ K ¯ ∩ K+ , By Proposition 3.1, there exists k ∈ K ¯ ∩ K0 , λk > 0 implies M k ∈ L∞ . λk > 0 implies k ∈ K++ . Furthermore, for each k ∈ K Therefore, r∗ r∗ , i + 2cT Y1:n,1 η > hQ, Y1:n,1:n X k i+ = λk hQ, M1:n,1:n ¯ k∈K∩K 0
X
k k , i + 2cT M1:n,1 λk hQ, M1:n,1:n
¯ k∈K∩K ++
1 k k , i + 2cT M1:n,1 hQ, M1:n,1:n k M00 ¯ k∈K∩K++ 1 k k ≥ min , i + 2cT M1:n,1 hQ, M1:n,1:n k ¯ M00 k∈K∩K ++ :λk >0 ≥ η, ≥
X
k λk M00
k i ≥ 0 in the third line, the where we used the assumption that inf k∈K0 :M k ∈L∞ hQ, M1:n,1:n
identity Y00r∗ =
X
k λk M00 =1
¯ k∈K∩K ++
in the fourth line, and the definition of η in (17) in the last line. Therefore, (iii) follows from the contradiction that arises from the chain of inequalities above.
3.1
Error Bound
In this section, we consider the gap between the upper bound that arises from an inner approximation of an instance of (CP) and its optimal value. The next proposition presents the main result. Proposition 3.3 Given an instance of (CP), a nonempty index set K, and M := {M k : k ∈ K} ⊆ C n+1 , suppose that the conic constraint X ∈ C n+1 in (CP) is replaced by X ∈ I(M), where I(M) is defined as in (13). Let K0 and K+ be defined as in (15) and K++ as in (16) and let ν(M) and ν ∗ denote the optimal values of the corresponding inner approximation and the original instance of (CP), respectively. Suppose that ν ∗ is finite. Then, ν(M) − ν ∗ ≤ kQk2F + 2kck22 13
1/2
δ1 (F, M),
(18)
where
k )M k F . δ1 (F, M) := sup inf Y − (1/M00 Y ∈F k∈K++
(19)
Proof. If K++ = ∅, then δ1 (F, M) := +∞ by Proposition 3.2 and (18) is trivially satisfied. Suppose now that K++ 6= ∅. Since ν ∗ is finite, there exists x∗ ∈ Rn such that ν ∗ = (x∗ )T Qx∗ + 2cT x∗ by the Frank-Wolfe theorem [12] (note that, in the presence of binary variables, ν ∗ is given by the minimum of the optimal values of a finite number of quadratic optimization problems in smaller dimensions and the optimal solution is attained for at least one of those quadratic optimization problems since ν ∗ is finite). Therefore, by Burer’s equivalence result [6] and by (8), Y∗ =
1 x
∗
1 x
∗
T ∈F
k∗ is an optimal solution of (CP). By (19), there exists k∗ ∈ K++ such that kY ∗ −(1/M00 )M k∗ kF ≤
δ1 (F, M). By Proposition 3.2, ν(M) − ν ∗ =
k k k inf (1/M00 ) hQ, M1:n,1:n i + 2cT M1:n,1 − ν ∗, k∈K++ k∗ k∗ k∗ ≤ 1/M00 hQ, M1:n,1:n i + 2cT M1:n,1 − ν ∗, * + 0 cT 1 , = M k∗ − Y ∗ , k∗ M00 c Q 1/2 ≤ kQk2F + 2kck22 δ1 (F, M).
3.2
Polyhedral Inner Approximations
We close this section by specializing our results to the case of polyhedral inner approximations. Since every polyhedral cone is generated by a finite number of matrices, such a cone admits a representation given by (13) in which K is a finite set. In this case, most of our earlier discussions simplify considerably. For instance, by Proposition 3.1, checking the feasibility of the corresponding inner approximation can be performed in finite time by simply 14
checking whether the cone generated by each matrix intersects the feasible region of (CP). Similarly, it follows from Proposition 3.2 that computing the corresponding upper bound reduces to solving an optimization problem over a finite set. Indeed, since K0 and K+ given by (15) are both finite sets, one can compute in a finite number of steps whether the inner approximation of (CP) is unbounded below or has a finite optimal value. Finally, unless F is bounded, we clearly have δ1 (F, M) = +∞ for any finite set K. Therefore, Proposition 3.3 does not yield a nontrivial upper bound in this case. However, we remark that, for any fixed finite set K, it is easy to construct an instance of (CP) for which ν ∗ is finite while ν(M) − ν ∗ is as large as possible. It follows that a general finite error bound cannot be established for polyhedral inner approximations unless additional assumptions are made about F. We will address this issue in the next section.
4
Inner Approximations Generated by Nonnegative Rank One Matrices
In this section, we focus on inner approximations generated by nonnegative rank one matrices. Note that C n+1 = cone{vv T : v ∈ ∆n+1 }, where ∆n+1 = {x ∈ Rn+1 : eT x = 1} denotes the unit simplex in Rn+1 . Since the extreme + rays of C n are given by rank one matrices [1], we can define inner approximations generated by the rank one matrices ppT , where p ∈ S and S is a nonempty subset of ∆n+1 , so that the extreme rays of the inner approximation coincide with those of C n+1 . Given a nonempty subset S ⊆ ∆n+1 , let us therefore define MS := {ppT : p ∈ S} ⊆ C n+1 .
(20)
The following result is a direct consequence of Proposition 3.2. Corollary 4.1 Given an instance of (BQP) and its completely positive reformulation (CP), suppose that the conic constraint X ∈ C n+1 in (CP) is replaced by X ∈ I(MS ), where 15
S ⊆ ∆n+1 is nonempty and MS is defined as in (20). Let S0 := {p ∈ S : p0 = 0},
S+ := {p ∈ S : p0 > 0},
S++ := {p ∈ S+ : (1/p0 )p1:n ∈ F }, (21)
where F defined as in (1) denotes the feasible region of (BQP). Let ν(MS ) denote the optimal value of the corresponding inner approximation of (CP). (i) If S++ = ∅, then the inner approximation of (CP) is infeasible and ν(MS ) = +∞. (ii) If S++ 6= ∅ and there exists p ∈ S0 such that p1:n ∈ L∞ , where L∞ is given by (3), and pT1:n Qp1:n < 0, then the inner approximation of (CP) is unbounded below and ν(MS ) = −∞. (iii) If S++ 6= ∅ and inf p∈S0 :p1:n ∈L∞ pT1:n Qp1:n ≥ 0, then ) ( 2 1 1 pT1:n Qp1:n + 2 cT p1:n . ν(MS ) = inf p∈S++ p0 p0 p Proof. Let us define M p := ppT for each p ∈ S. Note that M00 > 0 if and only if p ∈ S+ . p For p ∈ S+ , we have (1/M00 )M p ∈ F if and only if aTi (1/p0 )p1:n = bi for each i = 1, . . . , m, p p p p = (1/p20 )eTj p pT ej = (1/p20 )p2j = (1/M00 )M0j = (1/p0 )pj for each j ∈ B, and (1/M00 )Mjj
which holds if and only if (1/p0 )p1:n ∈ F since p ∈ Rn+1 + . For p ∈ S++ , p p p (1/M00 ) hQ, M1:n,1:n i + 2cT M1:n,1 = (1/p0 )2 pT1:n Qp1:n + 2(1/p0 )cT p1:n . Similarly, for p ∈ P0 , we have M p ∈ L∞ if and only if p20 = 0, aTi p1:n pT1:n ai = (aTi p1:n )2 = 0 p for each i = 1, . . . , m, and Mjj = eTj p pT ej = p2j = 0 for each j ∈ B, which holds if and only
if p ∈ S0 and p1:n ∈ L∞ . Furthermore, for p ∈ P0 , p hQ, M1:n,1:n i = pT1:n Qp1:n .
The assertions now follow from the observations above and Proposition 3.2. An interesting and useful observation regarding Corollary 4.1 is that any inner approximation of (CP) given by a cone generated by rank one matrices is equivalent to approximating 16
the original problem (BQP) by evaluating the objective function only on a subset of the feasible solutions. Furthermore, if S ⊆ ∆n+1 is a nonempty finite set, then I(MS ) is a polyhedral inner approximation of C n+1 and the resulting inner approximation of (CP) reduces to a finite discretization of the feasible region of (BQP). On the other hand, if S = ∆n+1 , then I(MS ) = C n+1 , i.e., the inner approximation of C n+1 is exact. In this case, x ∈ F if and only if (1/p0 )p1:n = x, where p := (1/(eT x + 1))[1; x] ∈ ∆n+1 . We therefore recover Burer’s reformulation. Recall that Proposition 3.3 presents an upper bound on the difference between the upper bound arising from an inner approximation of an instance of (CP) and its optimal value. Clearly, under the hypotheses of Proposition 3.3, the same error bound holds for inner approximations generated by nonnegative rank one matrices. By Corollary 4.1, such inner approximations are equivalent to evaluating the objective function only on a subset of the feasible solutions of (BQP). Therefore, an interesting question is whether a similar error bound can be presented based on the maximum distance between any feasible solution of (BQP) and the subset of feasible solutions of (BQP) corresponding to the particular inner approximation as opposed to the corresponding maximum distance in the space of the feasible solutions of (CP) as in Proposition 3.3. Note that the objective function of (CP) is linear in the space of (n + 1) × (n + 1) symmetric matrices and this observation allows us to bound the difference in the objective function as a function of the distance between two feasible solutions of (CP). On the other hand, the objective function of (BQP) is quadratic on Rn . Therefore, we cannot find a global Lipschitz constant, unless we make further additional assumptions on (BQP). Henceforth, we will assume that the feasible region F of (BQP) is a bounded set, which allows us to use the following Lipschitz-type error bound. Lemma 4.1 Given an instance of (BQP) with a nonempty and bounded feasible region F defined as in (1), we have |f (y) − f (x)| ≤ 2 max kQz + ck1 ky − xk∞ , z∈L
x ∈ F,
y ∈ F,
where f (x) := xT Qx + 2cT x and L given by (2) denotes the linear portion of F . 17
(22)
Proof. Note that F is bounded if and only if L is bounded due to (4). Let x ∈ F and y ∈ F be such that x 6= y. By the mean value theorem, there exists µ ∈ (0, 1) such that f (y) − f (x) = ∇f ((1 − µ)x + µy)T (y − x). Therefore, |f (y) − f (x)| ≤ k∇f ((1 − µ)x + µy)k1 ky − xk∞ . Since ∇f ((1 − µ)x + µy) = 2Q((1 − µ)x + µy)) + c and L is a convex set, the assertion follows.
Next, using the simple characterization of the inner approximation given by Corollary 4.1, we present an error bound on the gap between the corresponding upper bound and the optimal value of (CP) under the assumption that the feasible region F of (BQP) is nonempty and bounded. Proposition 4.1 Given an instance of (BQP) with a nonempty and bounded feasible region F , consider its completely positive reformulation (CP). Suppose that the conic constraint X ∈ C n+1 in (CP) is replaced by X ∈ I(MS ), where S ⊆ ∆n+1 is nonempty and MS is defined as in (20). Let S++ be defined as in (21) and let ν(MS ) and ν ∗ denote the optimal values of the corresponding inner approximation and the original instance of (CP), respectively. Then, ν(MS ) − ν ≤ 2 max kQz + ck1 δ2 (F, S), ∗
z∈L
where L denotes the linear portion of F given by (2) and δ2 (F, S) := sup inf kx − (1/p0 )pk∞ . x∈F p∈S++
Proof. The proof follows from Lemma 4.1 and a similar argument as in the proof of Proposition 3.3.
18
4.1
Burer’s Reduced Formulation
Given an instance of (BQP), suppose that m
∃ y ∈ R such that g :=
m X
yi ai ∈
Rn+ ,
i=1
m X
yi bi = 1.
(23)
i=1
By (1) and (23), we obtain x ∈ F ⇒ g T x = 1. Under the assumption (23), Burer [6] shows that (BQP) admits the following simpler completely positive formulation: (CPr) min hQ, Xi + 2 cT Xg s.t.
aTi Xg = bi ,
i = 1, . . . , m,
aTi Xai = b2i ,
i = 1, . . . , m,
(Xg)j = Xjj ,
j ∈ B,
g T Xg = 1, X ∈ C n. Suppose now that (BQP) has a nonempty and bounded feasible region F . Therefore, there exists a positive β ∈ R+ such that x ∈ F implies eT x ≤ β. Adding this redundant (m + 1)st constraint to (BQP) and the corresponding slack variable to convert the inequality constraint to an equality constraint, it follows that the assumption (23) is satisfied by simply defining yi = 0 for i = 1, . . . , m and ym+1 = 1/β, where am+1 := e ∈ Rn+1 . Therefore, by adding additional variables if necessary, every instance of (BQP) with a nonempty and bounded feasible region F can be formulated as an instance of (CPr). Note that we obtain g = (1/β)e, which implies that g ∈ Rn+ can be chosen to be strictly positive under this assumption (after possibly redefining n to account for the slack variable). In a similar fashion, the conic constraint X ∈ C n in (CPr) can be replaced by an inner approximation I(MS ), where S ⊆ ∆n is nonempty. We first present a characterization of the feasibility of the corresponding inner approximation and the resulting upper bound.
19
Proposition 4.2 Given an instance of (BQP) with a nonempty and bounded feasible region F given by (1), consider its completely positive reformulation (CPr). Suppose that the conic constraint X ∈ C n in (CPr) is replaced by X ∈ I(MS ), where S ⊆ ∆n is nonempty and MS is defined as in (20). Let ν(MS ) denote the optimal value of the corresponding inner approximation. (i) We have F 0 ∩I(MS ) 6= ∅ (i.e., the inner approximation has a nonempty feasible region) if and only if S 0 6= ∅, where F 0 denotes the feasible region of (CPr) and S 0 := p ∈ S : (1/g T p)p ∈ F ,
(24)
where g ∈ Rn+ is given by (23). (ii) ( ν(MS ) = inf0 p∈S
2
1 gT p
pT Qp + 2
1 gT p
) cT p .
Proof. Under the assumption that F is nonempty and bounded, note that g ∈ Rn+ given by (23) can be chosen to be strictly positive by our discussion preceding the statement. Since S ⊆ ∆n , it follows that g T p > 0 for each p ∈ S. Let us consider (i). If S 0 6= ∅, it is easy to verify that (1/(g T p)2 )ppT ∈ F 0 ∩ I(MS ). Conversely, suppose that X ∈ F 0 ∩ I(MS ). By Carath´eodory’s theorem for conic hulls, ¯ ≤ n(n + 1)/2 and there exists S¯ ⊂ S such that |S| X=
X
λp ppT .
p∈S¯
Since X ∈ F 0 , we obtain X
ˆ p aT pˆ = bi , λ i
i = 1, . . . , m,
(25)
ˆ p aT pˆ 2 = b2 , λ i i
i = 1, . . . , m,
(26)
p∈S¯
X p∈S¯
X
ˆ p pˆj = λ
p∈S¯
X
X
ˆ p pˆ2 , λ j
j ∈ B,
(27)
p∈S¯
ˆ p = 1, λ
p∈S¯
20
(28)
where ˆ p := λp (g T p)2 , λ
pˆ :=
1 gT p
p,
¯ p ∈ S.
(29)
ˆ p ≥ 0 and pˆ ∈ Rn for each p ∈ S. ¯ Let aT pˆ = bi + αip for each p ∈ S¯ and Note that λ + i P ˆ p αip = 0 for each i = 1, . . . , m. Together i = 1, . . . , m. By (25) and (28), we obtain p∈S¯ λ with (26), X
X ˆ p α 2 = b2 , ˆ p b2 + 2αip bi + α2 = b2 + λ λ i i ip ip i
i = 1, . . . , m,
p∈S¯
p∈S¯
ˆ p > 0. which implies that αip = 0 for each i = 1, . . . , m and for each p ∈ S¯ such that λ ˆ p > 0, where L given by (2) denotes the linear Therefore, pˆ ∈ L for each p ∈ S¯ such that λ portion of F . By (4), we obtain that 0 ≤ pˆj ≤ 1, which implies that pˆj − pˆ2j ≥ 0 for each P ˆ p (ˆ pj − pˆ2j ) = 0, which holds if and only if j ∈ B. Therefore, it follows from (27) that p∈S¯ λ ˆ p > 0. We pˆj − pˆ2j = 0, or equivalently, pˆj ∈ {0, 1} for each j ∈ B and each p ∈ S¯ such that λ ˆ p > 0, or equivalently, S 0 6= ∅ by (29) therefore obtain that pˆ ∈ F for each p ∈ S¯ such that λ ˆ p > 0 by (28). This completes the proof since there should be at least one p ∈ S¯ such that λ of (i). The proof of part (ii) is very similar to the proof of part (iii) of Proposition 3.2 and is therefore omitted. The next result presents an error bound on the difference between the corresponding upper bound and the optimal value of (CPr). Proposition 4.3 Given an instance of (BQP) with a nonempty and bounded feasible region F given by (1), consider its completely positive reformulation (CPr). Suppose that the conic constraint X ∈ C n in (CPr) is replaced by X ∈ I(MS ), where S ⊆ ∆n is nonempty and MS is defined as in (20). Let ν(MS ) and ν ∗ denote the optimal values of the corresponding inner approximation and the original instance of (CPr), respectively. Then, ∗ ν(MS ) − ν ≤ 2 max kQz + ck1 δ3 (F, S), z∈L
21
where L given by (2) denotes the linear portion of F and
1
,
p δ3 (F, S) := sup inf0 x − g T p ∞ x∈F p∈S where g ∈ Rn+ given by (23) is strictly positive and S 0 is defined as in (24). Proof. The proof follows from Lemma 4.1, Proposition 4.2, and a similar argument as in the proof of Proposition 3.3.
4.2
Implications of Error Bounds
In this section, we discuss the implications of the error bound of Proposition 4.3 on standard quadratic optimization problems and box-constrained quadratic optimization problems, which constitute two important classes of quadratic optimization problems. 4.2.1
Standard Quadratic Optimization
A standard quadratic optimization problem is given by (StQP) ν ∗ = min xT Qx,
(30)
x∈∆n
where Q ∈ S n . Clearly, the feasible region is bounded since it is given by the unit simplex. Since there is a single equality constraint, it follows that (23) is satisfied by defining y1 = 1 and we obtain g = e ∈ Rn+ . Therefore, (StQP) admits the following completely positive formulation in the form of (CPr) (see also [4] for an alternative derivation): (CP1) ν ∗ = min {hQ, Xi : hE, Xi = 1,
X ∈ C n} ,
where E = eeT ∈ S n is the matrix of all ones. Let us consider the following subset of ∆n (see, e.g., [3, 5, 17]): Snr := {p ∈ ∆n : rp ∈ Nn },
22
r = 1, 2, . . . .
(31)
Consider replacing the conic constraint X ∈ C n in (CP1) by X ∈ I(MSnr ), r = 1, 2, . . .. Note that F = ∆n in this case. Furthermore, for each p ∈ Snr , (1/g T p)p = p ∈ F , which implies that (Snr )0 = Snr , where (Snr )0 is defined similarly as in (24). By Proposition 4.2, ( ) 2 T 1 T ν(MSnr ) = min 0 p Qp . p Qp = min r r) p∈Sn gT p p∈(Sn We refer the reader to [3, 17] for an alternative derivation of the same result. Furthermore, by [5], δ3 (F, Snr )
= sup infr kx − pk∞ x∈∆n p∈Sn
1 = r
1 1− , n
r = 1, 2, . . . .
Together with Proposition 4.3, we obtain 2 1 ∗ r ν(MSnr ) − ν ≤ 2 max kQzk1 δ(F, Sn ) = 1− max kQ1:n,i k1 , r = 1, 2, . . . . z∈F r n i=1,...,n Since |Snr | = n+r−1 , which is polynomial for each fixed value of r, we obtain a polynomialr time approximation scheme for standard quadratic optimization. We refer the reader to [3, 5, 8, 9, 17] for similar approximation results. 4.2.2
Box-Constrained Quadratic Optimization
A box-constrained quadratic optimization problem is given by (BoxQP) ν ∗ = min{xT Qx + 2cT x : 0 ≤ x ≤ e}, where Q ∈ S n and c ∈ Rn . By adding slack variables, (BoxQP) can be formulated in the form of (BQP), where F = {[x; s] ∈ R2n : xi + si = 1, i = 1, . . . , n}. Clearly, F is bounded and (23) is satisfied by defining yi = 1/n for i = 1, . . . , n, in which case g = (1/n)e ∈ R2n + . Therefore, (BoxQP) admits the following completely positive formulation in the form of (CPr): (CP2) ν ∗ = minhQ, X1:n,1:n i + (1/n)cT X1:n,1:2n e s.t.
(Xe)i + (Xe)n+i = n,
i = 1, . . . , n,
Xii + 2Xi,n+i + Xn+i,n+i = 1,
i = 1, . . . , n,
X ∈ C 2n . 23
r Let us consider polyhedral inner approximations corresponding to S2n given by (31), i.e., r ), r = 1, 2, . . .. Let us fix r. By Proposition 4.2, we replace X ∈ C 2n in (CP2) by X ∈ I(MS2n
the corresponding inner approximation has a nonempty feasible region if and only if there r exists p ∈ S2n such that (1/g T p)p = np ∈ F , or equivalently, n(pi + pn+i ) = 1 for each r i = 1, . . . , n. By (31), p ∈ S2n if and only if pj = qj /r, where qj ∈ {0, 1, . . . , r} for each P2n j = 1, . . . , 2n and j=1 qj = r. Therefore, the inner approximation is feasible if and only if
(n/r)(qi + qn+i ) = 1, or equivalently, qi + qn+i = r/n for each i = 1, . . . , n. Since qj ∈ N for each j = 1, . . . , 2n, this system has a solution if and only if r/n ∈ N, or equivalently, r = κn, where κ ∈ N. This discussion shows that polyhedral inner approximations corresponding to r are feasible if and only if r = κn, where κ ∈ N. S2n
Let us next consider the upper bound arising from the polyhedral inner approximation r for r = κn, where κ ∈ N. Let us define corresponding to S2n κn 0 κn (S2n ) := {p ∈ S2n : np ∈ F } . κn 0 ) if and only if there exists [x; s] ∈ F such that [x; s] = np, or equivalently, Note that p ∈ (S2n κn (1/n)[x; s] ∈ S2n . By the preceding discussion, this holds if and only if there exists q ∈ N2n
such that (1/n)xi = qi /(κn) and (1/n)si = qn+i /(κn), or equivalently, xi = qi /κ and si = κn 0 ) if and only if there exists [x; s] ∈ F such qn+i /κ for each i = 1, . . . , n. Therefore, p ∈ (S2n
that κ[x; s] ∈ N2n . Combining this observation with Proposition 4.2, we obtain min 0 n2 pT1:n Qp1:n + 2ncT p1:n , κn p∈(S2n ) = min 2n xT Qx + 2cT x .
κn ) ν(MS2n =
[x;s]∈F :κ[x;s]∈N
Finally, we consider the approximation error for r = κn, where κ ∈ N. We need to κn κn 0 characterize δ3 (F, S2n ). By the preceding discussion, p ∈ (S2n ) if and only if there exists
[x; s] ∈ F such that κ[x; s] ∈ N2n , or equivalently, xi = qi /κ and si = (κ − qi )/κ, where qi ∈ {0, 1, . . . , κ} for i = 1, . . . , n. Therefore, for any feasible solution [ˆ x; sˆ] ∈ R2n of (BoxQP), κn 0 there exists p ∈ (S2n ) such that k[ˆ x; sˆ] − npk∞ ≤ 1/(2κ). In fact, it is straightforward to
verify that we can have equality by using a feasible solution with xˆ1 = 1/(2κ). Therefore, it 24
κn follows that δ3 (F, S2n ) = 1/(2κ). By Proposition 4.3, we obtain ∗ κn κn ) − ν ν(MS2n ≤ 2 max kQx + ck1 δ3 (F, S2n ), 0≤x≤e 1 max kQxk1 + kck1 , ≤ 0≤x≤e κ
! n
X 1
Q1:n,i xi + kck1 , = max 0≤x≤e
κ i=1 1 ! n X 1 ≤ max kQ1:n,i k1 |xi | + kck1 , 0≤x≤e κ i=1 ! X n X n 1 ≤ |Qij | + kck1 , κ = 1, 2, . . . . κ i=1 j=1
In contrast with standard quadratic optimization, this error bound does not translate into κn 0 ) | = (κ + 1)n , which is not polynomial a polynomial-time approximation scheme since | (S2n
for fixed κ. In fact, this is an expected result since, for instance, the max-cut problem can be formulated as an instance of (BoxQP) (see, e.g., [13]) and hardness of approximation results are known for the max-cut problem [14].
5
Concluding Remarks
In this paper, we focused on inner approximations of the cone of completely positive matrices on instances of completely positive optimization problems that arise from the reformulation of mixed binary quadratic optimization problems. We provided characterizations of the feasibility of such inner approximations and the optimal value of the corresponding approximation. We presented an upper bound on the gap between the optimal value of an inner approximation and that of the original instance. We considered general inner approximations as well as those generated by a set of nonnegative rank one matrices. We also discussed the special case of polyhedral approximations. For inner approximations generated by nonnegative rank one matrices, our characterization reveals that the corresponding inner approximation reduces to sampling a subset of points from the feasible region of the original mixed binary quadratic optimization prob25
lem. An interesting research direction is the investigation of tractable inner approximation schemes that would provide a reasonably good approximation guarantee for a given class of mixed binary quadratic optimization problems.
References [1] A. Berman and N. Shaked-Monderer. Completely Positive Matrices. World Scientific, 2003. [2] I. M. Bomze. Copositive optimization - recent developments and applications. European Journal of Operational Research, 216(3):509–520, 2012. [3] I. M. Bomze and E. De Klerk. Solving standard quadratic optimization problems via linear, semidefinite and copositive programming. Journal of Global Optimization, 24(2):163–185, 2002. [4] I. M. Bomze, M. D¨ ur, E. de Klerk, C. Roos, A. J. Quist, and T. Terlaky. On copositive programming and standard quadratic optimization problems. Journal of Global Optimization, 18(4):301–320, 2000. [5] I. M. Bomze, S. Gollowitzer, and E. A. Yıldırım. Rounding on the standard simplex: Regular grids for global optimization. Journal of Global Optimization, 59(2-3):243–258, 2014. [6] S. Burer. On the copositive representation of binary and continuous nonconvex quadratic programs. Math. Program., 120(2):479–495, 2009. [7] S. Burer. Copositive programming. In Miguel F. Anjos and Jean B. Lasserre, editors, Handbook on Semidefinite, Conic and Polynomial Optimization, volume 166 of International Series in Operations Research & Management Science, pages 201–218. Springer, New York, 2012.
26
[8] E. de Klerk, M. Laurent, and Z. Sun. An error analysis for polynomial optimization over the simplex based on the multivariate hypergeometric distribution. Technical report, Tilburg University, Tilburg, the Netherlands, 2014. [9] E. de Klerk, M. Laurent, and Z. Sun. An alternative proof of a PTAS for fixed-degree polynomial optimization over the simplex. Math. Program., 151(2):433–457, 2015. [10] P. J. C. Dickinson and L. Gijben. On the computational complexity of membership problems for the completely positive cone and its dual. Comp. Opt. and Appl., 57(2):403–415, 2014. [11] M. D¨ ur. Copositive programming–a survey. In Moritz Diehl, Francois Glineur, Elias Jarlebring, and Wim Michiels, editors, Recent Advances in Optimization and its Applications in Engineering, pages 3–20. Springer Berlin Heidelberg, 2010. [12] M. Frank and P. Wolfe. An algorithm for quadratic programming. Naval Research Logistics Quarterly, 3(1-2):95–110, 1956. [13] M. X. Goemans and D. P. Williamson.
Improved approximation algorithms for
maximum cut and satisfiability problems using semidefinite programming. J. ACM, 42(6):1115–1145, 1995. [14] J. H˚ astad. Some optimal inapproximability results. J. ACM, 48(4):798–859, 2001. [15] J. B. Lasserre. New approximations for the cone of copositive matrices and its dual. Mathematical Programming, 144(1-2):265–276, 2014. [16] K. G. Murty and S .N. Kabadi. Some NP-complete problems in quadratic and nonlinear programming. Math. Program., 39(2):117–129, 1987. [17] E. A. Yıldırım. On the accuracy of uniform polyhedral approximations of the copositive cone. Optimization Methods and Software, 27(1):155–173, 2012.
27
¨ ITAK ˙ This work was supported in part by TUB (The Scientific and Technological Research Council of
Turkey) Grant 112M870. † Department of Industrial
Engineering,
Ko¸c
University,
34450
Sarıyer,
Istanbul,
Turkey
¨ ˙ (Turkish Academy of ([email protected]). The author was supported in part by TUBA-GEB IP Sciences Young Scientists Award Program).
1
Key words: Completely positive cone, mixed binary quadratic optimization problems, inner approximations, polyhedral approximations. AMS Subject Classifications: 90C20, 90C25.
1
Introduction
The cone of completely positive matrices is given by C n := conv{uuT : u ∈ Rn+ }, where conv{·} denotes the convex hull and Rn+ denotes the nonnegative orthant in Rn . A completely positive optimization problem is a linear optimization problem over an affine subset of the cone of completely positive matrices: (CoP) Here, hU, V i :=
min {hC, Xi : hAi , Xi = fi ,
Pn Pn i=1
j=1
i = 1, . . . , p,
X ∈ C n} .
Uij Vij for any U ∈ S n and V ∈ S n , where S n denotes the set of
real symmetric n × n matrices; C ∈ S n , Ai ∈ S n , i = 1, . . . , p, and f ∈ Rp constitute the parameters; and X ∈ S n denotes the decision variable. Burer [6] showed that every quadratic optimization problem with a mix of binary and continuous variables, henceforth referred to as a mixed binary quadratic optimization problem, can be equivalently reformulated as a completely positive optimization problem. On the one hand, this result implies that a rather large class of nonconvex optimization problems admits a compact reformulation as a convex optimization problem. On the other hand, it follows from this result that a completely positive optimization problem is, in general, NP-hard. Indeed, deciding if a given matrix belongs to C n is a computationally intractable problem [10, 16]. Despite the fact that Burer’s reformulation result does not seem to be helpful from a computational complexity point of view, it provides a new theoretical and computational perspective on approximately solving mixed binary quadratic optimization problems. For 2
instance, the computationally intractable cone of completely positive matrices can be approximated from the inside or from the outside by a tractable convex cone. Therefore, for a given mixed binary quadratic optimization problem, one can obtain upper and lower bounds on its optimal value by utilizing approximations of C n in the corresponding completely positive reformulation. Indeed, various tractable inner and outer approximations of C n have been proposed in the literature. For further details, the reader is referred to [2, 7, 11]. In this paper, we focus on inner approximations of the cone of completely positive matrices in the context of completely positive optimization problems arising from the reformulation of mixed binary quadratic optimization problems. Given a particular inner approximation of C n and an instance of such a completely positive optimization problem, we provide a characterization of the feasibility of the corresponding approximation as well as the optimal value of a feasible inner approximation. For the special case of polyhedral inner approximations, our characterization implies that computing the corresponding approximate solution reduces to an optimization problem over a finite set. Finally, as a byproduct, our characterization yields an upper bound on the approximation error, i.e., the difference between the optimal value of the inner approximation and that of the original mixed binary quadratic optimization problem. We discuss the implications of the error bound for standard and box-constrained quadratic optimization problems. This paper is organized as follows. We define our notation in Section 1.1. We review Burer’s reformulation of mixed binary quadratic optimization problems and present a useful decomposition result for this formulation in Section 2. Section 3 is devoted to the characterization of the feasibility, optimal value, and the error bound for general inner approximations. We focus on inner approximations generated by a set of nonnegative rank one matrices in Section 4. For this particular family of inner approximations, we show that our results can be stated in terms of the original mixed binary quadratic optimization problem, in contrast with our corresponding results for general inner approximations which are stated in terms of the completely positive formulation in Section 3. We also consider mixed binary quadratic optimization problems with a bounded feasible region and discuss the implications of our re-
3
sults on standard and box-constrained quadratic optimization problems. Section 5 concludes the paper.
1.1
Notation
We use Rn , Rn+ , and S n to denote the n-dimensional Euclidean space, the nonnegative orthant, and the space of n × n real symmetric matrices, respectively. We reserve calligraphic letters for subsets of S n . The set of completely positive matrices is denoted by C n . The inner P P product on S n is the trace inner product given by hU, V i := ni=1 nj=1 Uij Vij for any U ∈ S n and V ∈ S n . The norm associated with the trace inner product is the Frobenius norm denoted by kU kF := hU, U i1/2 . The `p -norm on Rn is denoted by k · kp , where p ∈ {1, 2, ∞}. The set of nonnegative integers is denoted by N. Similarly, we use Nn to denote the set of ndimensional vectors all of whose components are given by nonnegative integers. For u ∈ Rn , we denote its jth component by uj , j = 1, . . . , n. For u ∈ Rn+1 , we start the indexing from zero. Similarly, Uij denotes the (i, j) entry of a matrix U ∈ S n , i = 1, . . . , n; j = 1, . . . , n, and we start both indices from zero for U ∈ S n+1 . We adopt Matlab-like notation. For a vector u, we use u1:k ∈ Rk to denote the subvector given by the components of u indexed by 1, . . . , k. We use similar notation for matrices, i.e., UI,J denotes the submatrix of U whose rows and columns are given by the rows and columns of U indexed by the sets I and J, respectively. For two vectors u ∈ Rn1 and v ∈ Rn2 , we use [u; v] ∈ Rn1 +n2 to denote the vector obtained by the vertical concatenation of u and v. The unit vectors in Rn are denoted by ej , j = 1, . . . , n, with the convention that indexing starts from zero for Rn+1 . We reserve e to denote the vector of all ones whose dimension will always be clear from the context. We use 0 to denote the real number zero, the vector of all zeroes as well as the matrix of all zeroes in the appropriate dimension, which will always be clear from the context.
4
2
Burer’s Reformulation and a Decomposition Result
A mixed binary quadratic optimization problem is given by (BQP)
min xT Qx + 2cT x
s.t. aTi x = bi ,
i = 1, . . . , m,
x ≥ 0, xj ∈ {0, 1},
j ∈ B,
where Q ∈ S n , c ∈ Rn , ai ∈ Rn , i = 1, . . . , m, bi ∈ R, i = 1, . . . , m, and B ⊆ {1, . . . , n} constitute the data of the problem and x ∈ Rn denotes the decision variable. We denote the feasible region of (BQP) by F , i.e., F := {x ∈ Rn : aTi x = bi ,
i = 1, . . . , m,
x ≥ 0,
xj ∈ {0, 1},
j ∈ B},
(1)
and we assume that F 6= ∅. The set obtained from F by ignoring the binary constraints is denoted by L: L := {x ∈ Rn : aTi x = bi ,
i = 1, . . . , m,
x ≥ 0}.
(2)
i = 1, . . . , m,
d ≥ 0}.
(3)
The recession cone of L is given by L∞ := {d ∈ Rn : aTi d = 0,
Burer [6] makes the following assumption, referred to as the key assumption: x ∈ L ⇒ 0 ≤ xj ≤ 1,
j ∈ B.
(4)
Note that the key assumption can always be enforced by adding the redundant constraints xj ≤ 1 for each j ∈ B, if necessary, and then introducing slack variables to convert the formulation into the form of (BQP). The key assumption implies that d ∈ L∞ ⇒ dj = 0,
5
j ∈ B.
(5)
Under the key assumption (4), Burer [6] shows that (BQP) is equivalent to the following completely positive optimization problem: (CP) min hQ, Xi + 2 cT x aTi x = bi ,
i = 1, . . . , m,
aTi Xai = b2i ,
i = 1, . . . , m,
s.t.
j ∈ B,
Xjj = xj ,
1 xT x X
∈ C n+1 .
Note that (CP) can be easily represented in the form of (CoP) by using the identity aTi Xai = hai aTi , Xi for i = 1, . . . , m. Similarly, let us denote the feasible region of (CP) by aTi x = bi , 1 xT ∈ C n+1 : aT Xai = b2 , F= i i x X X =x , jj
j
F: i = 1, . . . , m, i = 1, . . . , m, . j∈B
The recession cone of F is given by T 0 0T a Dai = 0, i = 1, . . . , m, ∈ C n+1 : i L∞ = . 0 D Djj = 0, j∈B
(6)
(7)
By (1) and (6), x∈F ⇒
1 x
1 x
T ∈ F.
(8)
Similarly, by (3), (5), and (7), d ∈ L∞ ⇒
0 d
0 d
T ∈ L∞ .
The equivalence between (BQP) and (CP) relies on the following main result.
6
(9)
Proposition 2.1 (Burer [6]) Given an instance of (BQP), let Y ∈ C n+1 be a feasible solution of (CP) given by Y =
X h∈H
ρh rh
T
ρh
,
rh
where H is a finite set, ρh ∈ R+ and rh ∈ Rn+ for each h ∈ H. Let H+ := {h ∈ H : ρh > 0},
H0 := {h ∈ H : ρh = 0}.
Then, (i) (1/ρh )rh ∈ F for each h ∈ H+ , and (ii) rh ∈ L∞ for each h ∈ H0 . Our next proposition presents a useful decomposition result for a given feasible solution of (CP). Proposition 2.2 Let Y ∈ C n+1 be a feasible solution of (CP) and suppose that Y =
X
Vp+
p∈P
X
W z,
z∈Z
where P and Z are finite sets, V p ∈ C n+1 and V00p > 0 for each p ∈ P , W z ∈ C n+1 and z = 0 for each z ∈ Z. Then, we have (i) P 6= ∅, (ii) (1/V00p )V p ∈ F for each p ∈ P , and W00
(iii) W z ∈ L∞ for each z ∈ Z. Proof. Since Y ∈ F, it follows that Y00 = 1, which implies that P 6= ∅ establishing (i). Suppose that V p , p ∈ P , and W z , z ∈ Z, admit the following decompositions: Vp =
X h∈H0p
Wz =
X h∈H0z
0 v hp 0 w
hz
T
0 v hp 0 w
hz
+
X p h∈H+
σhp shp
T ,
7
z ∈ Z,
σhp shp
T ,
p ∈ P,
where H0p and H+p are finite sets for each p ∈ P ; H0z is a finite set for each z ∈ Z; v hp ∈ Rn+ for each p ∈ P and h ∈ H0p ; shp ∈ Rn+ and σhp > 0 for each p ∈ P and h ∈ H+p ; and whz ∈ Rn+ for each z ∈ Z and h ∈ H0z . Note that H+p 6= ∅ since V00p > 0 for each p ∈ P . By Proposition 2.1, we obtain v hp ∈ L∞ ,
h ∈ H0p ,
p ∈ P,
where L∞ is given by (3), and
1 σhp
and whz ∈ L∞ ,
shp ∈ F,
p ∈ P,
z ∈ Z,
h ∈ H0z ,
h ∈ H+p ,
(10)
(11)
where F is defined as in (1). Since L∞ is a convex cone, it follows from (9) and (10) that W z ∈ L∞ for each z ∈ Z, which establishes (iii), and V p,0 ∈ L∞ for each p ∈ P , where T X 0 0 , p ∈ P. V p,0 := hp hp v v h∈H0p Similarly, let X
V p,+ :=
p h∈H+
σhp s
hp
σhp s
hp
T ,
p ∈ P,
so that V p = V p,+ + V p,0 for each p ∈ P . Let us now define ˆp
V :=
1 V00p
p
V =
1 V00p,+
(V p,+ + V p,0 ),
p ∈ P.
We need to show that Vˆ p ∈ F for each p ∈ P . Let us fix p ∈ P . By (8) and (11), we have T 1 1 ∈ F, h ∈ H+p . (12) 1 1 hp hp s s σhp σhp Note that Vˆ p =
!
1 P
p h∈H+
2 σhp
X 2 σhp p h∈H+
1
1 σhp
8
shp
1
1 σhp
T shp
+ V p,0 .
By (12), the first term above is given by a convex combination of the elements of F and the second term, which is a positive multiple of V p,0 , belongs to L∞ . It follows that Vˆ p ∈ F as desired, thereby establishing (ii).
3
General Inner Approximations
In this section, we consider replacing the difficult conic constraint in (CP) by an arbitrary inner approximation. Let K be a (possibly infinite) nonempty index set and let M k ∈ C n+1 be a nonzero matrix for each k ∈ K. Let M := {M k : k ∈ K} ⊆ C n+1 . We consider the following cone: I(M) := cone(M),
(13)
where cone(·) denotes the conic hull, i.e., the smallest convex cone (with respect to inclusion) that contains M ∪ {0}. Note that, by construction, we have I(M) ⊆ C n+1 . We remark that any inner approximation of C n+1 can be represented by (13) for an appropriate choice of M. In fact, there exist inner approximation hierarchies for the cone of completely positive matrices, i.e., there are sequences of tractable nested cones that provide increasingly better inner approximations with the property that such sequences are, in some sense, exact in the limit. The reader is referred to [17] and [15] for inner approximation hierarchies consisting of polyhedral and spectrahedral cones (i.e., cones given by the intersection of a linear subspace and the set of positive semidefinite matrices), respectively. For instance, each inner approximation in either of these hierarchies can be represented in the form of (13) for an appropriate choice of M. By Carath´eodory’s theorem for conic hulls, we have ¯ ⊆ K, Y ∈ I(M) ⇔ ∃ K
¯ ≤ |K|
(n + 1)(n + 2) , 2
Y =
X
λk M k ,
λk ≥ 0,
¯ k ∈ K,
¯ k∈K
(14) 9
i.e., any element of I(M) can be represented by a nonnegative combination of a finite number of matrices M k . We will refer to (14) as a finite conic representation of Y ∈ I(M). We consider replacing the conic constraint X ∈ C n+1 in (CP) by X ∈ I(M). The next proposition provides a characterization of the feasibility of the corresponding inner approximation of (CP). Furthermore, for any feasible solution of the resulting inner approximation, we present a useful representation result that will subsequently be used to characterize the corresponding optimal solution. Proposition 3.1 Given an instance of (CP), a nonempty index set K, and M := {M k : k ∈ K} ⊆ C n+1 , suppose that the conic constraint X ∈ C n+1 in (CP) is replaced by X ∈ I(M), where I(M) is defined as in (13). Let k K0 = k ∈ K : M00 =0 ,
k K+ = k ∈ K : M00 >0 .
(15)
(i) We have F ∩I(M) 6= ∅ (i.e., the inner approximation of (CP) has a nonempty feasible region) if and only if K++ 6= ∅, where k k K++ := k ∈ K+ : F ∩ cone(M k ) 6= ∅ = k ∈ K+ : 1/M00 M ∈F .
(16)
(ii) Suppose that F ∩I(M) 6= ∅ and let Y ∈ F ∩I(M). For any finite conic representation ¯ ∩ K++ such that λk > 0. Furthermore, of Y given by (14), there exists k ∈ K ¯ ∩ K+ , ∀k∈K
λk > 0 ⇒ k ∈ K++ ,
¯ ∩ K0 , ∀k∈K
λk > 0 ⇒ M k ∈ L∞ .
k Proof. Let us first consider part (i). If K++ 6= ∅, then (1/M00 )M k ∈ F ∩ I(M) for any
k ∈ K++ , which implies that F ∩ I(M) 6= ∅. Note that the reverse implication of part (i) directly follows from part (ii). Therefore, we will proceed with the proof of part (ii). Suppose that F ∩ I(M) 6= ∅ and let Y ∈ F ∩ I(M). For any finite conic representation of Y given by (14), let us decompose Y = Y + + Y 0 , where Y + :=
X
λk M k ,
¯ K∩K +
Y 0 :=
X ¯ K∩K 0
10
λk M k .
¯ By Proposition 2.2, Y + 6= 0. Therefore, Note that M k ∈ C n+1 and λk ≥ 0 for each k ∈ K. ¯ ∩ K+ such that λk > 0, i.e., K ¯ ∩ K+ 6= ∅. Furthermore, for each there exists k ∈ K k k ¯ ∩ K+ such that λk > 0, we have (1/λk M00 )M k ∈ F , i.e., k ∈ K++ . )λk M k = (1/M00 k∈K
Therefore, K++ 6= ∅, which establishes the reverse implication of (i). Finally, λk M k ∈ L∞ ¯ ∩ K0 , i.e., for each k ∈ K ¯ ∩ K0 such that λk > 0, we have M k ∈ L∞ , which for each k ∈ K concludes the proof of part (ii).
By Proposition 3.1, the feasibility of an inner approximation of (CP) is simply determined by whether the cone generated by any element of M intersects the feasible region F of (CP). The next example illustrates that such a characterization does not necessarily hold for the feasible region of an arbitrary completely positive optimization problem. Example 3.1 Consider the following set: + * 2 0 , X = 1, F := X ∈ C 2 : 0 −1
*
0 1 1 0
+
,X
=2 .
One can easily verify that 1 1 1 0 + µ :µ≥0 . F= 1 1 0 2 Let
1 0 1 1 ⊂ C 2. M= , 0 1 1 2
We have
2 1 , F ∩ I(M) = 1 3
whereas the cone generated by either matrix in M does not intersect F. Our next result gives a useful characterization of the upper bound obtained by replacing the conic constraint X ∈ C n+1 in (CP) by X ∈ I(M). 11
Proposition 3.2 Given an instance of (CP), a nonempty index set K, and M := {M k : k ∈ K} ⊆ C n+1 , suppose that the conic constraint X ∈ C n+1 in (CP) is replaced by X ∈ I(M), where I(M) is defined as in (13). Let K0 and K+ be defined as in (15) and K++ as in (16) and let ν(M) denote the optimal value of the corresponding inner approximation of (CP). (i) If K++ = ∅, then the inner approximation of (CP) is infeasible and ν(M) := +∞. k (ii) If K++ 6= ∅ and there exists k ∈ K0 such that M k ∈ L∞ and hQ, M1:n,1:n i < 0, then
the inner approximation of (CP) is unbounded below and ν(M) = −∞. k (iii) If K++ 6= ∅ and inf k∈K0 :M k ∈L∞ hQ, M1:n,1:n i ≥ 0, then
k k k ν(M) = inf (1/M00 ) hQ, M1:n,1:n i + 2cT M1:n,1 . k∈K++
Proof. Note that (i) follows directly from part (i) of Proposition 3.1. Considering (ii), let k∗ ∈ K++ and suppose that there exists k ∈ K0 such that M k ∈ L∞ k∗ k and hQ, M1:n,1:n i < 0. Then, Y µ := (1/M00 )M k∗ + µM k ∈ F ∩ I(M) for each µ ≥ 0 by
Proposition 3.1. Clearly, µ µ k∗ k∗ k∗ k hQ, Y1:n,1:n i + 2cT Y1:n,1 = (1/M00 ) hQ, M1:n,1:n i + 2cT M1:n,1 + µhQ, M1:n,1:n i → −∞ as µ → +∞, which establishes (ii). k Finally, suppose that K++ 6= ∅ and inf k∈K0 :M k ∈L∞ hQ, M1:n,1:n i ≥ 0. By Proposition 3.1, k )M k ∈ F for each k ∈ K++ . Therefore, (1/M00
k k k . ν(M) ≤ η := inf (1/M00 ) hQ, M1:n,1:n i + 2cT M1:n,1 k∈K++
(17)
We will use a contradiction argument to establish the reverse inequality. Suppose, for a contradiction, that ν(M) < η. Then, there exists a sequence {Y r : r ∈ N} ⊆ F ∩ I(M) r r such that hQ, Y1:n,1:n i + 2cT Y1:n,1 → ν(M) as r → ∞. Therefore, there exists r∗ ∈ N such r∗ r∗ that hQ, Y1:n,1:n i + 2cT Y1:n,1 < η. Consider any finite representation of Y r∗ given by (14):
Y r∗ =
X
λk M k +
¯ k∈K∩K +
X ¯ k∈K∩K 0
12
λk M k .
¯ ∩ K+ such that λk > 0 and for each k ∈ K ¯ ∩ K+ , By Proposition 3.1, there exists k ∈ K ¯ ∩ K0 , λk > 0 implies M k ∈ L∞ . λk > 0 implies k ∈ K++ . Furthermore, for each k ∈ K Therefore, r∗ r∗ , i + 2cT Y1:n,1 η > hQ, Y1:n,1:n X k i+ = λk hQ, M1:n,1:n ¯ k∈K∩K 0
X
k k , i + 2cT M1:n,1 λk hQ, M1:n,1:n
¯ k∈K∩K ++
1 k k , i + 2cT M1:n,1 hQ, M1:n,1:n k M00 ¯ k∈K∩K++ 1 k k ≥ min , i + 2cT M1:n,1 hQ, M1:n,1:n k ¯ M00 k∈K∩K ++ :λk >0 ≥ η, ≥
X
k λk M00
k i ≥ 0 in the third line, the where we used the assumption that inf k∈K0 :M k ∈L∞ hQ, M1:n,1:n
identity Y00r∗ =
X
k λk M00 =1
¯ k∈K∩K ++
in the fourth line, and the definition of η in (17) in the last line. Therefore, (iii) follows from the contradiction that arises from the chain of inequalities above.
3.1
Error Bound
In this section, we consider the gap between the upper bound that arises from an inner approximation of an instance of (CP) and its optimal value. The next proposition presents the main result. Proposition 3.3 Given an instance of (CP), a nonempty index set K, and M := {M k : k ∈ K} ⊆ C n+1 , suppose that the conic constraint X ∈ C n+1 in (CP) is replaced by X ∈ I(M), where I(M) is defined as in (13). Let K0 and K+ be defined as in (15) and K++ as in (16) and let ν(M) and ν ∗ denote the optimal values of the corresponding inner approximation and the original instance of (CP), respectively. Suppose that ν ∗ is finite. Then, ν(M) − ν ∗ ≤ kQk2F + 2kck22 13
1/2
δ1 (F, M),
(18)
where
k )M k F . δ1 (F, M) := sup inf Y − (1/M00 Y ∈F k∈K++
(19)
Proof. If K++ = ∅, then δ1 (F, M) := +∞ by Proposition 3.2 and (18) is trivially satisfied. Suppose now that K++ 6= ∅. Since ν ∗ is finite, there exists x∗ ∈ Rn such that ν ∗ = (x∗ )T Qx∗ + 2cT x∗ by the Frank-Wolfe theorem [12] (note that, in the presence of binary variables, ν ∗ is given by the minimum of the optimal values of a finite number of quadratic optimization problems in smaller dimensions and the optimal solution is attained for at least one of those quadratic optimization problems since ν ∗ is finite). Therefore, by Burer’s equivalence result [6] and by (8), Y∗ =
1 x
∗
1 x
∗
T ∈F
k∗ is an optimal solution of (CP). By (19), there exists k∗ ∈ K++ such that kY ∗ −(1/M00 )M k∗ kF ≤
δ1 (F, M). By Proposition 3.2, ν(M) − ν ∗ =
k k k inf (1/M00 ) hQ, M1:n,1:n i + 2cT M1:n,1 − ν ∗, k∈K++ k∗ k∗ k∗ ≤ 1/M00 hQ, M1:n,1:n i + 2cT M1:n,1 − ν ∗, * + 0 cT 1 , = M k∗ − Y ∗ , k∗ M00 c Q 1/2 ≤ kQk2F + 2kck22 δ1 (F, M).
3.2
Polyhedral Inner Approximations
We close this section by specializing our results to the case of polyhedral inner approximations. Since every polyhedral cone is generated by a finite number of matrices, such a cone admits a representation given by (13) in which K is a finite set. In this case, most of our earlier discussions simplify considerably. For instance, by Proposition 3.1, checking the feasibility of the corresponding inner approximation can be performed in finite time by simply 14
checking whether the cone generated by each matrix intersects the feasible region of (CP). Similarly, it follows from Proposition 3.2 that computing the corresponding upper bound reduces to solving an optimization problem over a finite set. Indeed, since K0 and K+ given by (15) are both finite sets, one can compute in a finite number of steps whether the inner approximation of (CP) is unbounded below or has a finite optimal value. Finally, unless F is bounded, we clearly have δ1 (F, M) = +∞ for any finite set K. Therefore, Proposition 3.3 does not yield a nontrivial upper bound in this case. However, we remark that, for any fixed finite set K, it is easy to construct an instance of (CP) for which ν ∗ is finite while ν(M) − ν ∗ is as large as possible. It follows that a general finite error bound cannot be established for polyhedral inner approximations unless additional assumptions are made about F. We will address this issue in the next section.
4
Inner Approximations Generated by Nonnegative Rank One Matrices
In this section, we focus on inner approximations generated by nonnegative rank one matrices. Note that C n+1 = cone{vv T : v ∈ ∆n+1 }, where ∆n+1 = {x ∈ Rn+1 : eT x = 1} denotes the unit simplex in Rn+1 . Since the extreme + rays of C n are given by rank one matrices [1], we can define inner approximations generated by the rank one matrices ppT , where p ∈ S and S is a nonempty subset of ∆n+1 , so that the extreme rays of the inner approximation coincide with those of C n+1 . Given a nonempty subset S ⊆ ∆n+1 , let us therefore define MS := {ppT : p ∈ S} ⊆ C n+1 .
(20)
The following result is a direct consequence of Proposition 3.2. Corollary 4.1 Given an instance of (BQP) and its completely positive reformulation (CP), suppose that the conic constraint X ∈ C n+1 in (CP) is replaced by X ∈ I(MS ), where 15
S ⊆ ∆n+1 is nonempty and MS is defined as in (20). Let S0 := {p ∈ S : p0 = 0},
S+ := {p ∈ S : p0 > 0},
S++ := {p ∈ S+ : (1/p0 )p1:n ∈ F }, (21)
where F defined as in (1) denotes the feasible region of (BQP). Let ν(MS ) denote the optimal value of the corresponding inner approximation of (CP). (i) If S++ = ∅, then the inner approximation of (CP) is infeasible and ν(MS ) = +∞. (ii) If S++ 6= ∅ and there exists p ∈ S0 such that p1:n ∈ L∞ , where L∞ is given by (3), and pT1:n Qp1:n < 0, then the inner approximation of (CP) is unbounded below and ν(MS ) = −∞. (iii) If S++ 6= ∅ and inf p∈S0 :p1:n ∈L∞ pT1:n Qp1:n ≥ 0, then ) ( 2 1 1 pT1:n Qp1:n + 2 cT p1:n . ν(MS ) = inf p∈S++ p0 p0 p Proof. Let us define M p := ppT for each p ∈ S. Note that M00 > 0 if and only if p ∈ S+ . p For p ∈ S+ , we have (1/M00 )M p ∈ F if and only if aTi (1/p0 )p1:n = bi for each i = 1, . . . , m, p p p p = (1/p20 )eTj p pT ej = (1/p20 )p2j = (1/M00 )M0j = (1/p0 )pj for each j ∈ B, and (1/M00 )Mjj
which holds if and only if (1/p0 )p1:n ∈ F since p ∈ Rn+1 + . For p ∈ S++ , p p p (1/M00 ) hQ, M1:n,1:n i + 2cT M1:n,1 = (1/p0 )2 pT1:n Qp1:n + 2(1/p0 )cT p1:n . Similarly, for p ∈ P0 , we have M p ∈ L∞ if and only if p20 = 0, aTi p1:n pT1:n ai = (aTi p1:n )2 = 0 p for each i = 1, . . . , m, and Mjj = eTj p pT ej = p2j = 0 for each j ∈ B, which holds if and only
if p ∈ S0 and p1:n ∈ L∞ . Furthermore, for p ∈ P0 , p hQ, M1:n,1:n i = pT1:n Qp1:n .
The assertions now follow from the observations above and Proposition 3.2. An interesting and useful observation regarding Corollary 4.1 is that any inner approximation of (CP) given by a cone generated by rank one matrices is equivalent to approximating 16
the original problem (BQP) by evaluating the objective function only on a subset of the feasible solutions. Furthermore, if S ⊆ ∆n+1 is a nonempty finite set, then I(MS ) is a polyhedral inner approximation of C n+1 and the resulting inner approximation of (CP) reduces to a finite discretization of the feasible region of (BQP). On the other hand, if S = ∆n+1 , then I(MS ) = C n+1 , i.e., the inner approximation of C n+1 is exact. In this case, x ∈ F if and only if (1/p0 )p1:n = x, where p := (1/(eT x + 1))[1; x] ∈ ∆n+1 . We therefore recover Burer’s reformulation. Recall that Proposition 3.3 presents an upper bound on the difference between the upper bound arising from an inner approximation of an instance of (CP) and its optimal value. Clearly, under the hypotheses of Proposition 3.3, the same error bound holds for inner approximations generated by nonnegative rank one matrices. By Corollary 4.1, such inner approximations are equivalent to evaluating the objective function only on a subset of the feasible solutions of (BQP). Therefore, an interesting question is whether a similar error bound can be presented based on the maximum distance between any feasible solution of (BQP) and the subset of feasible solutions of (BQP) corresponding to the particular inner approximation as opposed to the corresponding maximum distance in the space of the feasible solutions of (CP) as in Proposition 3.3. Note that the objective function of (CP) is linear in the space of (n + 1) × (n + 1) symmetric matrices and this observation allows us to bound the difference in the objective function as a function of the distance between two feasible solutions of (CP). On the other hand, the objective function of (BQP) is quadratic on Rn . Therefore, we cannot find a global Lipschitz constant, unless we make further additional assumptions on (BQP). Henceforth, we will assume that the feasible region F of (BQP) is a bounded set, which allows us to use the following Lipschitz-type error bound. Lemma 4.1 Given an instance of (BQP) with a nonempty and bounded feasible region F defined as in (1), we have |f (y) − f (x)| ≤ 2 max kQz + ck1 ky − xk∞ , z∈L
x ∈ F,
y ∈ F,
where f (x) := xT Qx + 2cT x and L given by (2) denotes the linear portion of F . 17
(22)
Proof. Note that F is bounded if and only if L is bounded due to (4). Let x ∈ F and y ∈ F be such that x 6= y. By the mean value theorem, there exists µ ∈ (0, 1) such that f (y) − f (x) = ∇f ((1 − µ)x + µy)T (y − x). Therefore, |f (y) − f (x)| ≤ k∇f ((1 − µ)x + µy)k1 ky − xk∞ . Since ∇f ((1 − µ)x + µy) = 2Q((1 − µ)x + µy)) + c and L is a convex set, the assertion follows.
Next, using the simple characterization of the inner approximation given by Corollary 4.1, we present an error bound on the gap between the corresponding upper bound and the optimal value of (CP) under the assumption that the feasible region F of (BQP) is nonempty and bounded. Proposition 4.1 Given an instance of (BQP) with a nonempty and bounded feasible region F , consider its completely positive reformulation (CP). Suppose that the conic constraint X ∈ C n+1 in (CP) is replaced by X ∈ I(MS ), where S ⊆ ∆n+1 is nonempty and MS is defined as in (20). Let S++ be defined as in (21) and let ν(MS ) and ν ∗ denote the optimal values of the corresponding inner approximation and the original instance of (CP), respectively. Then, ν(MS ) − ν ≤ 2 max kQz + ck1 δ2 (F, S), ∗
z∈L
where L denotes the linear portion of F given by (2) and δ2 (F, S) := sup inf kx − (1/p0 )pk∞ . x∈F p∈S++
Proof. The proof follows from Lemma 4.1 and a similar argument as in the proof of Proposition 3.3.
18
4.1
Burer’s Reduced Formulation
Given an instance of (BQP), suppose that m
∃ y ∈ R such that g :=
m X
yi ai ∈
Rn+ ,
i=1
m X
yi bi = 1.
(23)
i=1
By (1) and (23), we obtain x ∈ F ⇒ g T x = 1. Under the assumption (23), Burer [6] shows that (BQP) admits the following simpler completely positive formulation: (CPr) min hQ, Xi + 2 cT Xg s.t.
aTi Xg = bi ,
i = 1, . . . , m,
aTi Xai = b2i ,
i = 1, . . . , m,
(Xg)j = Xjj ,
j ∈ B,
g T Xg = 1, X ∈ C n. Suppose now that (BQP) has a nonempty and bounded feasible region F . Therefore, there exists a positive β ∈ R+ such that x ∈ F implies eT x ≤ β. Adding this redundant (m + 1)st constraint to (BQP) and the corresponding slack variable to convert the inequality constraint to an equality constraint, it follows that the assumption (23) is satisfied by simply defining yi = 0 for i = 1, . . . , m and ym+1 = 1/β, where am+1 := e ∈ Rn+1 . Therefore, by adding additional variables if necessary, every instance of (BQP) with a nonempty and bounded feasible region F can be formulated as an instance of (CPr). Note that we obtain g = (1/β)e, which implies that g ∈ Rn+ can be chosen to be strictly positive under this assumption (after possibly redefining n to account for the slack variable). In a similar fashion, the conic constraint X ∈ C n in (CPr) can be replaced by an inner approximation I(MS ), where S ⊆ ∆n is nonempty. We first present a characterization of the feasibility of the corresponding inner approximation and the resulting upper bound.
19
Proposition 4.2 Given an instance of (BQP) with a nonempty and bounded feasible region F given by (1), consider its completely positive reformulation (CPr). Suppose that the conic constraint X ∈ C n in (CPr) is replaced by X ∈ I(MS ), where S ⊆ ∆n is nonempty and MS is defined as in (20). Let ν(MS ) denote the optimal value of the corresponding inner approximation. (i) We have F 0 ∩I(MS ) 6= ∅ (i.e., the inner approximation has a nonempty feasible region) if and only if S 0 6= ∅, where F 0 denotes the feasible region of (CPr) and S 0 := p ∈ S : (1/g T p)p ∈ F ,
(24)
where g ∈ Rn+ is given by (23). (ii) ( ν(MS ) = inf0 p∈S
2
1 gT p
pT Qp + 2
1 gT p
) cT p .
Proof. Under the assumption that F is nonempty and bounded, note that g ∈ Rn+ given by (23) can be chosen to be strictly positive by our discussion preceding the statement. Since S ⊆ ∆n , it follows that g T p > 0 for each p ∈ S. Let us consider (i). If S 0 6= ∅, it is easy to verify that (1/(g T p)2 )ppT ∈ F 0 ∩ I(MS ). Conversely, suppose that X ∈ F 0 ∩ I(MS ). By Carath´eodory’s theorem for conic hulls, ¯ ≤ n(n + 1)/2 and there exists S¯ ⊂ S such that |S| X=
X
λp ppT .
p∈S¯
Since X ∈ F 0 , we obtain X
ˆ p aT pˆ = bi , λ i
i = 1, . . . , m,
(25)
ˆ p aT pˆ 2 = b2 , λ i i
i = 1, . . . , m,
(26)
p∈S¯
X p∈S¯
X
ˆ p pˆj = λ
p∈S¯
X
X
ˆ p pˆ2 , λ j
j ∈ B,
(27)
p∈S¯
ˆ p = 1, λ
p∈S¯
20
(28)
where ˆ p := λp (g T p)2 , λ
pˆ :=
1 gT p
p,
¯ p ∈ S.
(29)
ˆ p ≥ 0 and pˆ ∈ Rn for each p ∈ S. ¯ Let aT pˆ = bi + αip for each p ∈ S¯ and Note that λ + i P ˆ p αip = 0 for each i = 1, . . . , m. Together i = 1, . . . , m. By (25) and (28), we obtain p∈S¯ λ with (26), X
X ˆ p α 2 = b2 , ˆ p b2 + 2αip bi + α2 = b2 + λ λ i i ip ip i
i = 1, . . . , m,
p∈S¯
p∈S¯
ˆ p > 0. which implies that αip = 0 for each i = 1, . . . , m and for each p ∈ S¯ such that λ ˆ p > 0, where L given by (2) denotes the linear Therefore, pˆ ∈ L for each p ∈ S¯ such that λ portion of F . By (4), we obtain that 0 ≤ pˆj ≤ 1, which implies that pˆj − pˆ2j ≥ 0 for each P ˆ p (ˆ pj − pˆ2j ) = 0, which holds if and only if j ∈ B. Therefore, it follows from (27) that p∈S¯ λ ˆ p > 0. We pˆj − pˆ2j = 0, or equivalently, pˆj ∈ {0, 1} for each j ∈ B and each p ∈ S¯ such that λ ˆ p > 0, or equivalently, S 0 6= ∅ by (29) therefore obtain that pˆ ∈ F for each p ∈ S¯ such that λ ˆ p > 0 by (28). This completes the proof since there should be at least one p ∈ S¯ such that λ of (i). The proof of part (ii) is very similar to the proof of part (iii) of Proposition 3.2 and is therefore omitted. The next result presents an error bound on the difference between the corresponding upper bound and the optimal value of (CPr). Proposition 4.3 Given an instance of (BQP) with a nonempty and bounded feasible region F given by (1), consider its completely positive reformulation (CPr). Suppose that the conic constraint X ∈ C n in (CPr) is replaced by X ∈ I(MS ), where S ⊆ ∆n is nonempty and MS is defined as in (20). Let ν(MS ) and ν ∗ denote the optimal values of the corresponding inner approximation and the original instance of (CPr), respectively. Then, ∗ ν(MS ) − ν ≤ 2 max kQz + ck1 δ3 (F, S), z∈L
21
where L given by (2) denotes the linear portion of F and
1
,
p δ3 (F, S) := sup inf0 x − g T p ∞ x∈F p∈S where g ∈ Rn+ given by (23) is strictly positive and S 0 is defined as in (24). Proof. The proof follows from Lemma 4.1, Proposition 4.2, and a similar argument as in the proof of Proposition 3.3.
4.2
Implications of Error Bounds
In this section, we discuss the implications of the error bound of Proposition 4.3 on standard quadratic optimization problems and box-constrained quadratic optimization problems, which constitute two important classes of quadratic optimization problems. 4.2.1
Standard Quadratic Optimization
A standard quadratic optimization problem is given by (StQP) ν ∗ = min xT Qx,
(30)
x∈∆n
where Q ∈ S n . Clearly, the feasible region is bounded since it is given by the unit simplex. Since there is a single equality constraint, it follows that (23) is satisfied by defining y1 = 1 and we obtain g = e ∈ Rn+ . Therefore, (StQP) admits the following completely positive formulation in the form of (CPr) (see also [4] for an alternative derivation): (CP1) ν ∗ = min {hQ, Xi : hE, Xi = 1,
X ∈ C n} ,
where E = eeT ∈ S n is the matrix of all ones. Let us consider the following subset of ∆n (see, e.g., [3, 5, 17]): Snr := {p ∈ ∆n : rp ∈ Nn },
22
r = 1, 2, . . . .
(31)
Consider replacing the conic constraint X ∈ C n in (CP1) by X ∈ I(MSnr ), r = 1, 2, . . .. Note that F = ∆n in this case. Furthermore, for each p ∈ Snr , (1/g T p)p = p ∈ F , which implies that (Snr )0 = Snr , where (Snr )0 is defined similarly as in (24). By Proposition 4.2, ( ) 2 T 1 T ν(MSnr ) = min 0 p Qp . p Qp = min r r) p∈Sn gT p p∈(Sn We refer the reader to [3, 17] for an alternative derivation of the same result. Furthermore, by [5], δ3 (F, Snr )
= sup infr kx − pk∞ x∈∆n p∈Sn
1 = r
1 1− , n
r = 1, 2, . . . .
Together with Proposition 4.3, we obtain 2 1 ∗ r ν(MSnr ) − ν ≤ 2 max kQzk1 δ(F, Sn ) = 1− max kQ1:n,i k1 , r = 1, 2, . . . . z∈F r n i=1,...,n Since |Snr | = n+r−1 , which is polynomial for each fixed value of r, we obtain a polynomialr time approximation scheme for standard quadratic optimization. We refer the reader to [3, 5, 8, 9, 17] for similar approximation results. 4.2.2
Box-Constrained Quadratic Optimization
A box-constrained quadratic optimization problem is given by (BoxQP) ν ∗ = min{xT Qx + 2cT x : 0 ≤ x ≤ e}, where Q ∈ S n and c ∈ Rn . By adding slack variables, (BoxQP) can be formulated in the form of (BQP), where F = {[x; s] ∈ R2n : xi + si = 1, i = 1, . . . , n}. Clearly, F is bounded and (23) is satisfied by defining yi = 1/n for i = 1, . . . , n, in which case g = (1/n)e ∈ R2n + . Therefore, (BoxQP) admits the following completely positive formulation in the form of (CPr): (CP2) ν ∗ = minhQ, X1:n,1:n i + (1/n)cT X1:n,1:2n e s.t.
(Xe)i + (Xe)n+i = n,
i = 1, . . . , n,
Xii + 2Xi,n+i + Xn+i,n+i = 1,
i = 1, . . . , n,
X ∈ C 2n . 23
r Let us consider polyhedral inner approximations corresponding to S2n given by (31), i.e., r ), r = 1, 2, . . .. Let us fix r. By Proposition 4.2, we replace X ∈ C 2n in (CP2) by X ∈ I(MS2n
the corresponding inner approximation has a nonempty feasible region if and only if there r exists p ∈ S2n such that (1/g T p)p = np ∈ F , or equivalently, n(pi + pn+i ) = 1 for each r i = 1, . . . , n. By (31), p ∈ S2n if and only if pj = qj /r, where qj ∈ {0, 1, . . . , r} for each P2n j = 1, . . . , 2n and j=1 qj = r. Therefore, the inner approximation is feasible if and only if
(n/r)(qi + qn+i ) = 1, or equivalently, qi + qn+i = r/n for each i = 1, . . . , n. Since qj ∈ N for each j = 1, . . . , 2n, this system has a solution if and only if r/n ∈ N, or equivalently, r = κn, where κ ∈ N. This discussion shows that polyhedral inner approximations corresponding to r are feasible if and only if r = κn, where κ ∈ N. S2n
Let us next consider the upper bound arising from the polyhedral inner approximation r for r = κn, where κ ∈ N. Let us define corresponding to S2n κn 0 κn (S2n ) := {p ∈ S2n : np ∈ F } . κn 0 ) if and only if there exists [x; s] ∈ F such that [x; s] = np, or equivalently, Note that p ∈ (S2n κn (1/n)[x; s] ∈ S2n . By the preceding discussion, this holds if and only if there exists q ∈ N2n
such that (1/n)xi = qi /(κn) and (1/n)si = qn+i /(κn), or equivalently, xi = qi /κ and si = κn 0 ) if and only if there exists [x; s] ∈ F such qn+i /κ for each i = 1, . . . , n. Therefore, p ∈ (S2n
that κ[x; s] ∈ N2n . Combining this observation with Proposition 4.2, we obtain min 0 n2 pT1:n Qp1:n + 2ncT p1:n , κn p∈(S2n ) = min 2n xT Qx + 2cT x .
κn ) ν(MS2n =
[x;s]∈F :κ[x;s]∈N
Finally, we consider the approximation error for r = κn, where κ ∈ N. We need to κn κn 0 characterize δ3 (F, S2n ). By the preceding discussion, p ∈ (S2n ) if and only if there exists
[x; s] ∈ F such that κ[x; s] ∈ N2n , or equivalently, xi = qi /κ and si = (κ − qi )/κ, where qi ∈ {0, 1, . . . , κ} for i = 1, . . . , n. Therefore, for any feasible solution [ˆ x; sˆ] ∈ R2n of (BoxQP), κn 0 there exists p ∈ (S2n ) such that k[ˆ x; sˆ] − npk∞ ≤ 1/(2κ). In fact, it is straightforward to
verify that we can have equality by using a feasible solution with xˆ1 = 1/(2κ). Therefore, it 24
κn follows that δ3 (F, S2n ) = 1/(2κ). By Proposition 4.3, we obtain ∗ κn κn ) − ν ν(MS2n ≤ 2 max kQx + ck1 δ3 (F, S2n ), 0≤x≤e 1 max kQxk1 + kck1 , ≤ 0≤x≤e κ
! n
X 1
Q1:n,i xi + kck1 , = max 0≤x≤e
κ i=1 1 ! n X 1 ≤ max kQ1:n,i k1 |xi | + kck1 , 0≤x≤e κ i=1 ! X n X n 1 ≤ |Qij | + kck1 , κ = 1, 2, . . . . κ i=1 j=1
In contrast with standard quadratic optimization, this error bound does not translate into κn 0 ) | = (κ + 1)n , which is not polynomial a polynomial-time approximation scheme since | (S2n
for fixed κ. In fact, this is an expected result since, for instance, the max-cut problem can be formulated as an instance of (BoxQP) (see, e.g., [13]) and hardness of approximation results are known for the max-cut problem [14].
5
Concluding Remarks
In this paper, we focused on inner approximations of the cone of completely positive matrices on instances of completely positive optimization problems that arise from the reformulation of mixed binary quadratic optimization problems. We provided characterizations of the feasibility of such inner approximations and the optimal value of the corresponding approximation. We presented an upper bound on the gap between the optimal value of an inner approximation and that of the original instance. We considered general inner approximations as well as those generated by a set of nonnegative rank one matrices. We also discussed the special case of polyhedral approximations. For inner approximations generated by nonnegative rank one matrices, our characterization reveals that the corresponding inner approximation reduces to sampling a subset of points from the feasible region of the original mixed binary quadratic optimization prob25
lem. An interesting research direction is the investigation of tractable inner approximation schemes that would provide a reasonably good approximation guarantee for a given class of mixed binary quadratic optimization problems.
References [1] A. Berman and N. Shaked-Monderer. Completely Positive Matrices. World Scientific, 2003. [2] I. M. Bomze. Copositive optimization - recent developments and applications. European Journal of Operational Research, 216(3):509–520, 2012. [3] I. M. Bomze and E. De Klerk. Solving standard quadratic optimization problems via linear, semidefinite and copositive programming. Journal of Global Optimization, 24(2):163–185, 2002. [4] I. M. Bomze, M. D¨ ur, E. de Klerk, C. Roos, A. J. Quist, and T. Terlaky. On copositive programming and standard quadratic optimization problems. Journal of Global Optimization, 18(4):301–320, 2000. [5] I. M. Bomze, S. Gollowitzer, and E. A. Yıldırım. Rounding on the standard simplex: Regular grids for global optimization. Journal of Global Optimization, 59(2-3):243–258, 2014. [6] S. Burer. On the copositive representation of binary and continuous nonconvex quadratic programs. Math. Program., 120(2):479–495, 2009. [7] S. Burer. Copositive programming. In Miguel F. Anjos and Jean B. Lasserre, editors, Handbook on Semidefinite, Conic and Polynomial Optimization, volume 166 of International Series in Operations Research & Management Science, pages 201–218. Springer, New York, 2012.
26
[8] E. de Klerk, M. Laurent, and Z. Sun. An error analysis for polynomial optimization over the simplex based on the multivariate hypergeometric distribution. Technical report, Tilburg University, Tilburg, the Netherlands, 2014. [9] E. de Klerk, M. Laurent, and Z. Sun. An alternative proof of a PTAS for fixed-degree polynomial optimization over the simplex. Math. Program., 151(2):433–457, 2015. [10] P. J. C. Dickinson and L. Gijben. On the computational complexity of membership problems for the completely positive cone and its dual. Comp. Opt. and Appl., 57(2):403–415, 2014. [11] M. D¨ ur. Copositive programming–a survey. In Moritz Diehl, Francois Glineur, Elias Jarlebring, and Wim Michiels, editors, Recent Advances in Optimization and its Applications in Engineering, pages 3–20. Springer Berlin Heidelberg, 2010. [12] M. Frank and P. Wolfe. An algorithm for quadratic programming. Naval Research Logistics Quarterly, 3(1-2):95–110, 1956. [13] M. X. Goemans and D. P. Williamson.
Improved approximation algorithms for
maximum cut and satisfiability problems using semidefinite programming. J. ACM, 42(6):1115–1145, 1995. [14] J. H˚ astad. Some optimal inapproximability results. J. ACM, 48(4):798–859, 2001. [15] J. B. Lasserre. New approximations for the cone of copositive matrices and its dual. Mathematical Programming, 144(1-2):265–276, 2014. [16] K. G. Murty and S .N. Kabadi. Some NP-complete problems in quadratic and nonlinear programming. Math. Program., 39(2):117–129, 1987. [17] E. A. Yıldırım. On the accuracy of uniform polyhedral approximations of the copositive cone. Optimization Methods and Software, 27(1):155–173, 2012.
27
