COMPUTATIONAL COMPLEXITY OF TENSOR ... - Semantic Scholar
COMPUTATIONAL COMPLEXITY OF TENSOR NUCLEAR NORM SHMUEL FRIEDLAND AND LEK-HENG LIM Abstract. The main result of this paper is that the weak membership problem in the unit ball of a given norm is NP-hard if and only if the weak membership problem in the unit ball of the dual norm is NP-hard. Equivalently, the approximation of a given norm is polynomial time if and only if the approximation of the dual norm is polynomial time. Using the NP-hardness of the approximation of spectral norm of tensors we prove that the approximation of nuclear norm of tensors is NP-hard. In addition, we show that bipartite separability of a density matrix is equivalent its corresponding 4-tensor having unit nuclear norm, relating these results to quantum information theory.
1. Introduction The nuclear norm of a 2-tensor (or, in coordinate form, a matrix) has recently found widespread use as a convex surrogate for relaxing various intractable non-convex problems into tractable convex problems. The motivation for this article is to investigate the computational complexity of the nuclear norm for higher order tensors. We investigate three specific problems: (i) the weak membership problem for the nuclear norm unit ball of 3-tensors over R, (ii) the weak membership problem for the nuclear norm unit ball of 4-tensors over R and C, (iii) the approximation of nuclear norm of 4-tensors over R and C. We will show that all these problems are NP-hard. In this article we use the term d-tensor to mean a tensor of order d. Our investigation leads to a more general result that applies to all norms, namely, the weak membership problem for a norm ball and that for its dual norm ball have equivalent complexity. Another interesting side result that we obtained in the course of our study is that a density matrix is bipartite separable iff its associated 4-tensor has unit nuclear norm. Taken together with Gurvits’s result [8] on the NP-hardness of bipartite separability, we deduce that the membership problem for the nuclear norm unit ball of 4-tensors over C is NP-hard, which is of course also implied by (ii) above. In short, the nuclear norm is NP-hard to compute for real 3-tensors and complex 4-tensors. As usual, our proofs depend on reductions to existing problems known to be NP-hard. The NPhardness of the nuclear norm of real 3-tensor relies on that of the spectral norm of real 3-tensors [9]. The NP-hardness of the nuclear norm of complex 4-tensors relies on a characterization of the clique number of a graph as the spectral norm of a 4-tensor that is bi-hermitian, positive semidefinite, and nonnegative-valued, together with the fact that clique number is NP-hard. 1.1. Outline. In Section 2 we define the nuclear and spectral norms for tensors of arbitrary orders over C and R. In Section 3 we discuss bipartite separability in quantum mechanics and show that a bipartite state is separable if and only if its nuclear norm is one. Using Gurvits’s result [8] that bipartite membership is NP-hard we deduce that deciding membership in the nuclear norm unit ball is NP-hard. In Section 4 we relate Motzkin–Strauss’s characterization of the clique number of a graph as the spectral norm of a 4-tensor defined by the graph. It then follows that -approximation of the nuclear norm for 4-tensors over C and R, or equivalently, of biparitite density matrices, is NP-hard. In Section 5 we show that weak membership in the unit ball of a given norm can be decided in polynomial time if and only if weak membership in the unit ball of its dual norm can be decided in polynomial time. In Section 6 we show that weak membership in the unit ball of a given 1
2
S. FRIEDLAND AND L.-H. LIM
norm can be decided in polynomial if and only if -approximation of the given norm is polynomial time. In Section 7 we apply our results to show that deciding membership in the tensor nuclear norm unit ball is NP-hard. 2. Tensor nuclear and spectral norms Let F = R or C. Let Fn1 ×···×nd := Fn1 ⊗ · · · ⊗ Fnd be the space of d-tensors of dimensions n1 , . . . , nd ∈ N. If desired, these may be viewed as d-dimensional hypermatrices A = (ai1 ···id ) with entries ai1 ···id ∈ F. The hermitian inner product of two d-tensors A, B ∈ Cn1 ×···×nd is given by Xn1 ,...,nd ai1 ···id bi1 ···id . hA, Bi = i1 ,...,id =1
This induces the Hilbert–Schmidt norm, denoted by 1 Xn1 ,...,nd p 2 |ai1 ···id |2 . kAk = hA, Ai = i1 ,...,id =1
We adopt the convention that an unlabeled k · k will always denote the Hilbert–Schmidt norm. Note that when d = 1, this is just the regular hermitian or l2 -norm of a vector in Cn and when d = 2, this is the Frobenius norm of a matrix in Cm×n . The norms of greatest interest to us in this article are the spectral norm and nuclear norm of a d-tensor A ∈ Cn1 ×···×nd . These are denoted and defined respectively by Re(hA, x1 ⊗ · · · ⊗ xd i) |hA, x1 ⊗ · · · ⊗ xd i| = max , max x1 ,...,xd 6=0 kx1 k · · · kxd k kx1 k · · · kxd k nXr o Xr kAk∗ := min kx1,p k · · · kxd,p k : A = x1,p ⊗ · · · ⊗ xd,p , r ∈ N .
kAkσ :=
x1 ,...,xd 6=0
p=1
p=1
(1) (2)
It is easy to see that k · kσ and k · k∗ are dual norms [12, Lemma 21] and that kx1 ⊗ · · · ⊗ xd kσ = kx1 ⊗ · · · ⊗ xd k∗ = kx1 k · · · kxd k. We would like to point out that (2) is the definition of tensor nuclear norm as originally defined by Grothendieck [6] and Schatten [16]. An alternate definition of ‘tensor nuclear norm’ as the average of nuclear norms of matrices obtained from flattenings of a tensor has gained recent popularity. While this alternate definition may be useful for various purposes, it is nevertheless not the definition commonly accepted in mathematics [2, 15, 13, 17] (see also [4, 12]). The nuclear norm defined in (2) is precisely the dual norm of the spectral norm in (1) and is naturally related to the notion of tensor rank (cf. [11]). Moreover, we will prove in Section 3 that nuclear norm as defined in (2) has physical meaning — equivalent to bipartite separability of quantum states in an appropriate sense. As such, a tensor nuclear norm in this article will always be the one in (2). Tensor rank is known to depend on the choice of base field [11]. We do not know if it might be the same for nuclear and spectral norms. As such, we need a more careful definition. Let B(Fn ) := {x ∈ Fn : kxk ≤ 1},
S(Fn ) := {x ∈ Fn : kxk = 1},
be the Euclidean unit ball and sphere in Fn (recall that k · k denotes the Euclidean norm on Fn ). Then for A ∈ Fn1 ×···×nd , we define n o kAkσ,F := max |hA, x1 ⊗ · · · ⊗ xd i| : xi ∈ B(Fni ) , (3) nXr Yd o Xr kAk∗,F := min kxi,p k : A = x1,p ⊗ · · · ⊗ xd,p , xi,p ∈ Fni , r ∈ N . (4) p=1
i=1
p=1
The indices i and p are always assumed to run over i = 1, . . . , d and p = 1, . . . , r respectively. Clearly, for any A ∈ Rn1 ×···×nd , kAkσ,R ≤ kAkσ,C ,
kAk∗,R ≥ kAkσ,C .
(5)
COMPUTATIONAL COMPLEXITY OF TENSOR NUCLEAR NORM
3
It is well known that for d = 2 we have equality signs in the above inequalities. We suspect that this is not the case for d ≥ 3. Let x = (x1 , . . . , xn )T ∈ Cn . Denote by |x| := (|x1 |, . . . , |xn |)T . Then x is called a nonnegative vector, denoted as x ≥ 0, if x = |x|. We will also use this notation for tensors in Cn1 ×···×nd . Lemma 2.1. Let A ∈ Cn1 ×···×nd . Then kAkσ,C ≤ k|A|kσ,C ,
k|A|kσ,C = k|A|kσ,R .
Proof. The triangle inequality yields |hA, x1 ⊗ · · · ⊗ xd i| ≤ h|A|, x1 ⊗ · · · ⊗ xd i. Recall that the Euclidean norm on Cn is an absolute norm, i.e., kxk = k|x|k. The definitions of k · kσ,C and k · kσ,R and the above inequality yields the result. The equality in the above lemma fails for nuclear norm even for matrices. Indeed, let √ √ 1/ √2 1/√2 . Q= −1/ 2 1/ 2 √ Then kQk∗ = 2 > k|Q|k∗ = 2. A recent example of R. Tomioka shows that the equality in Lemma 2.1 fails for 4 × 4 hermitian matrices. Henceforth, by spectral and nuclear norm we will always mean over C, unless stated otherwise. We will denote k · kσ := k · kσ,C ,
k · k∗ := k · k∗,C .
In what follows we show that kAkσ,F and kAk∗,F are NP-hard to compute for d ≥ 3 by using appropriate results. If we show that kAkσ,R is NP-hard to compute for A ≥ 0, then Lemma 2.1 yields that kAkσ,C is also NP-hard to compute. 3. Bipartite separability and nuclear norm We show that every density matrix corresponds uniquely and naturally to a 4-tensor. Furthermore the density matrix is bipartite separable if and only if its corresponding 4-tensor has unit nuclear norm. It then follows from the well-known NP-hardness of bipartite separability [8] that tensor nuclear norm is also NP-hard. Assume throughout the following that mj+d = mj
for all j = 1, . . . , d.
(6)
Then we may write Cm1 ×...md ×md+1 ×···×m2d = Cm1 ×···×md ⊗ Cm1 ×···×md = (Cm1 ×···×md )⊗2 . For each A = (ai1 ···i2d ) ∈ (Cm1 ×···×md )⊗2 , we define the trace as Xm1 ,...,md tr(A) := ai1 ···id i1 ···id . i1 =···=id =1
(7)
It is straightforward to see that tr(x1 ⊗ · · · ⊗ x2d ) =
Yd j=1
xT j+d xj
(8)
for every xj , xj+d ∈ Cmj , j = 1, . . . , d. Let Hm×m ⊂ Cm×m be the real vector subspace of hermitian matrices. Let Hm×m be the cone + m×m of nonnegative definite hermitian matrices and Hρ be the convex set of density matrices, i.e., nonnegative definite hermitian matrices of trace one. Clearly, Hm×m ⊂ Hm×m ⊂ Hm×m . ρ + Since (Cm1 ×···×md )⊗2 ∼ = Cm1 ···md ×m1 ···md , we may identify tensors of order 2d satisfying (6) with m1 · · · md × m1 · · · md matrices. We will adopt this identification in subsequent discussions and will regard 2d-tensors satisfying (6) interchangeably with m1 · · · md × m1 · · · md matrices. In which
4
S. FRIEDLAND AND L.-H. LIM
case, Hermitian matrices Hm1 ···md ×m1 ···md ⊂ Cm1 ···md ×m1 ···md may be identified with a subset of 2d-tensors that we denote by H(m1 , . . . , md ) ⊂ (Cm1 ×···×md )⊗2 . We will call H(m1 , . . . , md ) the space of d-hermitian tensors. The density matrices and nonnegative definite hermitian matrices may then be regarded as d-hermitian tensors. We denote these by Hρ (m1 , . . . , md ) ⊂ H+ (m1 , . . . , md ) ⊂ H(m1 , . . . , md ). Note that a d-hermitian tensor is a tensor of order 2d. Alternatively, one may define them directly by stating that A = (ai1 ···i2d ) ∈ (Cm1 ×···×md )⊗2 is a d-hermitian tensor if and only if ai1 ···id id+1 ···i2d = aid+1 ···i2d i1 ···id
for all ij , id+j = 1, . . . , mj , j = 1, . . . , d.
We call the convex set Hρ (m1 , . . . , md ) the set of d-partite density matrices. An element of m ×m the form A1 ⊗ · · · ⊗ Ad ∈ Hρ (m1 , . . . , md ) is called a product density matrix if Aj ∈ Hρ j j for j = 1, . . . , d. Let Hsep (m1 , . . . , md ) be the convex set spanned by product density matrices. Since Hm×m is a convex combination of rank-one hermitian matrices xx∗ , x∗ x = 1, called pure states, it ρ follows that Hsep (m1 , . . . , md ) is a convex combination of product pure states (x1 x∗1 ) ⊗ · · · ⊗ (xd x∗d ),
x∗j xj = 1,
j = 1, . . . , d.
A density matrix A ∈ Hρ (m1 , . . . , md ) is called d-partite separable if A ∈ Hsep (m1 , . . . , md ). Lemma 3.1. Let A ∈ (Cm1 ×···×md )⊗2 . Then |tr(A)| ≤ kAk∗ ,
(9)
and equality holds if and only if A = zB for some z ∈ C and B ∈ Hsep (m1 , . . . , md ). Furthermore, a density matrix A ∈ Hρ (m1 , . . . , md ) is d-partite separable if and only if it has unit nuclear norm. P Q P Proof. Let A = ri=1 x1,i ⊗ · · · ⊗ x2d,i , where kAk∗ = ri=1 2d j=1 kxj,i k > 0. In view of (8), tr(A) = Pr Qd T T i=1 j=1 (xj+d,i xj,i ). The Cauchy–Schwarz inequality yields that |xj+d,i xj,i | ≤ kxj,i kkxj+d,i k. Equality holds if and only if xj+d,i = zj,i x ¯j,i for some zj,i ∈ C. Thus X Y Xr Yd r 2d T ≤ | tr A| ≤ x ) (x kxj,i k = kAk∗ . j,i j+d,i i=1 j=1 i=1 j=1 This establishes (9). Suppose that equality holds in (9). Then x1,i ⊗ · · · ⊗ x2d,i is of the form zj,i (x1,i x∗1,i ) ⊗ · · · ⊗ (xd,i x∗d,i ). Observe that Yd tr zj,i (x1,i x∗1,i ) ⊗ · · · ⊗ (xd,i x∗d,i ) = zj,i kxj,i k2 . j=1
Without loss of generality we may assume that kxj,i k = 1 for j = 1, . . . , d. Since equality holds in the triangle inequality it follows that all zj,i must have the same arguments. Hence A = zB where Xr B= ti (x1,i x∗1,i ) ⊗ · · · ⊗ (xd,i x∗d,i ), (10) i=1 P where x∗j,i xj,i = 1 for j = 1, . . . , d, and ri=1 ti = 1, ti ≥ 0, for i = 1, . . . , r. Conversely, suppose B is d-partite separable. Hence B is of the above form. Therefore Xr Yd kBk∗ ≤ ti kxj,i k2 = 1. i=1
j=1
Clearly, tr(B) = 1. In view of (9), it follows that kBk∗ = 1. Hence a decomposition (10) of B is minimal with respect to the nuclear norm. We will use the following hardness result from [8] (see also [5]). Theorem 3.2 (Gurvits). Deciding whether a given density matrix is bipartite separable is an NPhard problem.
COMPUTATIONAL COMPLEXITY OF TENSOR NUCLEAR NORM
5
From this and Lemma 3.1, we immediately deduce our first hardness result for tensor nuclear norm. Corollary 3.3. Deciding whether a given 4-tensor is in the nuclear norm unit ball is an NP-hard problem. In the following, we will show that a stronger result, namely, not only membership but even weak membership in the nuclear norm unit ball is also NP-hard to decide over R and C. 4. Clique number and nuclear norm Let G = (V, E) be a graph with V := {1, . . . , n} and the set of undirected edges E = {(ik , jk ) : ik = 1, . . . , jk − 1, k = 1, . . . , m}. Let κ(G) be the clique number of G, i.e., the size of the maximal clique in G. Denote by AG the adjacency matrix of G. Let ∆n be the simplex of probability vectors on Rn . Motzkin and Strauss showed that κ(G) − 1 = maxn xT AG x. (11) x∈∆ κ(G) Equality is attained when x is uniformly distributed on the largest clique. We now transform (11) to one involving 4-tensors. Let x = y ◦2 , i.e., x = (y12 , . . . , yn2 )T . Then X (y ◦2 )T AG y ◦2 = yi2 yj2 , y T y = 1. (12) (i,j)∈E(G)
We will see how a term of the form
ys2 yt2
may be regarded as a 4-tensor.
m×n×m×n be a 4-tensor. We call it bi-symmetric if Definition 4.1. Let A = (aijpq )m,n,m,n i,j,p,q=1 ∈ C
aijpq = apqij
for all i, p = 1, . . . , m, j, q = 1, . . . , n,
(13)
aijpq = a ¯pqij
for all i, p = 1, . . . , m, j, q = 1, . . . , n.
(14)
and bi-hermitian if
A bi-hermitian tensor A = (aijpq ) is said to be positive semidefinite if Xm,n,m,n aijpq xij x ¯pq ≥ 0 i,j,p,q=1
for all X = [xij ] ∈ Cm×n . This is a special case of the terms introduced in Section 3 for d = 2, except that we use the more verbal ‘bi-hermitian’ for ‘2-hermitian’, and likewise for our use of the term ‘bipartite’ for m×n×m×n as a 2-partite later. As in Section 3, we may regard a 4-tensor A = (aijpq )m,n,m,n i,j,p,q=1 ∈ C matrix C(A) = [c(i,j),(p,q) ] ∈ Cmn×mn , where c(i,j),(p,q) = aijpq . Evidently, A is bi-symmetric (resp. bi-hermitian) if and only if C(A) is symmetric (resp. hermitian). (s,t) For integers 1 ≤ s < t ≤ n, we consider Ast = (aijpq )ni,j,p,q=1 ∈ Cn×n×n×n defined by 1/2 i = s, j = t, p = s, q = t, 1/2 i = t, j = s, p = t, q = s, (s,t) aijpq = 1/2 i = s, j = t, p = t, q = s, (15) 1/2 i = t, j = s, p = s, q = t, 0 otherwise. Note that the quartic form hAst , y ⊗ y ⊗ y ⊗ yi = 2ys2 yt2 . Lemma 4.2. The tensor Ast is bi-hermitian, positive semidefinite, and has all entries nonnegative.
6
S. FRIEDLAND AND L.-H. LIM
Proof. It is easy to see that Ast is bi-hermitian and nonnegative. It is positive semidefinite because Xn,n,n,n (s,t) 1 a xij x ¯pq = (xst + xts )(¯ xst + x ¯ts ) ≥ 0 i,j,s,t=1 ijpq 2 for all X = [xij ] ∈ Cn×n . C(Ast ) is evidently a nonnegative definite, rank-one matrix with trace one. Hence C(Ast ) represents a density matrix on a bipartite state. For any graph G, we will let 1 X Ast ∈ Cn×n×n×n , (16) AG := (s,t)∈E(G) m and C(AG ) ∈ Cn entries.
2 ×n2
be its corresponding matrix. Note that AG and C(AG ) have real-valued
Theorem 4.3. Let G be a simple undirected graph on n vertices with m edges. Denote AG as above. Then |hAG , x ⊗ y ⊗ u ⊗ vi| kAG kσ := max n 06=x,y,u,v∈C kxkkykkukkvk hAG , x ⊗ y ⊗ u ⊗ vi = max n (17) 06=x,y,u,v∈R+ kxkkykkukkvk = maxn
06=y∈R+
hAG , y ⊗ y ⊗ y ⊗ yi . kyk4
(18)
Since all entries of AG are nonnegative we immediately deduce the equality (17). If the AG is a symmetric 4-tensor as opposed to merely bisymmetric, then we may apply a classical result of Banach [1, 3] to deduce that the maximum is attained at x = y = u = v and deduce (18). Unfortunately for us, AG is not symmetric and we need to prove Theorem 4.3 from scratch. We start with the following lemma which may be of independent interest. Lemma 4.4. Let H = [h(i,j),(pq) ] ∈ Cmn×mn be a hermitian nonnegative definite matrix. Define A = (aijpq ) ∈ Cm×n×m×n be the equality aijpq = h(i,j),(p,q) . Then kAkσ =
max
06=x∈Cm , 06=y∈Cn
hA, x ⊗ y ⊗ x ¯ ⊗ y¯i . kxk2 kyk2
(19)
Proof. Recall that we may always write H = R2 for some hermitian R ∈ Cmn×mn . Consider the sesquilinear form w ¯ T Hz = (Rw)T (Rz). Use Cauchy–Schwarz inequality to see that √ √ |¯ z T Hw| ≤ z¯T Hz w ¯ T Hw ≤ max(¯ z T Hz, w ¯ T Hw). Now let z := x ⊗ y, w := u ¯ ⊗ v¯ and deduce the lemma.
Proof of Theorem 4.3. We first apply Lemma 4.4 to (17). Hence we can assume that u = x ≥ 0, y = v ≥ 0. Hence 2hAst , x ⊗ y ⊗ x ⊗ yi = (xs yt + xt ys )2 . Use Cauchy–Schwarz inequality to see that (x2 + ys2 ) (x2t + yt2 ) (xs yt + xt ys )2 ≤ 4 s × . 2 2 p p So introducing two new variables as = (x2s + ys2 )/2 and at = (x2t + yt2 )/2, we reduced our problem to the problem of degree 4 in one vector variable a = (a1 , . . . , an ), where kak = 1. This is exactly the characterization (18). It was shown in [9] that a computation and approximation of spectral norm of d-tensor, for d ≥ 3, in NP-hard over the reals. We now give a version of the NP-hardness of a computation and approximation of tensor spectral norm of bi-hermitian and real bi-symmetric 4-tensors over complexes and reals:
COMPUTATIONAL COMPLEXITY OF TENSOR NUCLEAR NORM
7
Theorem 4.5. It is NP-hard to approximate tensor spectral norm of bi-hermitian 4-tensors over C and bi-symmetric real 4-tensor over R corresponding to bipartite density matrices. Proof. Let AG be defined by (16). Then AG is a nonnegative bi-symmetric tensor. Furthermore, the matrix C(AG ) is positive semi-definite. Since m is the number of edges in E(G) it follows that the trace of C(AG ) is 1. Hence C(AG ) represents a real bipartite density matrix. Motzkin–Strauss theorem yields that κ(G) − 1 kAG kσ = kAG kσ,R = . (20) mκ(G) Since the computation of clique number of a graph is NP-hard, the above identity implies that the computation of the spectral norm corresponding to bipartite density matrices is NP-hard over C and R. Since the clique number of a graph is an integer, it follows that it is NP-hard to approximate the spectral norms of the corresponding 4-tensors over complex or real numbers. 5. Weak membership in a norm unit ball Let F = R or C and ν : Fn → [0, ∞) be a norm. Denote by Bν := {x ∈ Fn : ν(x) ≤ 1} the closed unit ball with respect to ν. Let k · k be the Euclidean norm on Fn and B(a, r) := {x ∈ Fn : kx−ak ≤ r}. Since all norms in Fn are equivalent, it follows that there exist constants Kν ≥ kν > 0 such that kν kxk ≤ ν(x) ≤ Kν kxk, for all x ∈ Fn . (21) In what follows we assume that kν , Kν are rational. We denote by hkν i, hKν i the number of bits corresponding to kν , Kν . Q Identify Fn1 ×···×nd with Fn , where n = di=1 ni . Assume that A ∈ Fn1 ×···×nd . Then kAk is the Hilbert–Schmidt norm, which is the Euclidean norm in Fn . Lemma 5.1. Let k · kσ,F be the spectral norm in Fn1 ×···×nd . Then √
1 n1 n2 . . . nd
kAk ≤ kAkσ,F ≤ kAk
for all A ∈ Fn1 ×···×nd .
(22)
Proof. Clearly, kAkσ,F ≤ kAk. Assume that A = (ai1 ···id ). Let kAkmax = max{|ai1 ···id | : ij = √ 1, . . . , nj , j = 1, . . . , d}. Clearly, kAk ≤ n1 n2 . . . nd kAkmax . Choose in characterization (3) each xj ∈ Fnj to be a standard unit vector. It then follows that kAkmax ≤ kAkσ,F . Hence the left-hand side of (22) holds. Recall that the dual norm of ν, denoted by ν∗ , is given by ν∗ (x) = max{Re(y ∗ x) : ν(y) ≤ 1}. Hence 1 1 kxk ≤ ν∗ (x) ≤ kxk. Kν kν In particular kAk ≤ kAk∗,F ≤
√
n1 n2 . . . nd kAk
for all A ∈ Fn1 ×···×nd .
(23)
(24)
In what follows, for simplicity of the exposition we assume√that F = R. It will be convenient to identify Cn with R2n ≡ Rn × Rn . So z ∈ Cn is viewed as x + −1y, where (x, y) ∈ Rn ×√Rn . Hence a norm ν : Cn → [0, ∞) induces a norm ν˜ : R2n → [0, ∞), where ν˜((xT , y T )T ) := ν(x + −1y). By abusing the notation we will identify ν with with ν˜. Note that the Euclidean norm on Cn gives rise to the Euclidean norm on R2n . Hence, from the complexity point of view it is enough to consider norms over the real valued spaces.
8
S. FRIEDLAND AND L.-H. LIM
In what follows we use the definitions and results from [7]. Observe first that Bν is a well-bounded 0-centered set. More precisely B(0, K1ν ) ⊂ Bν ⊂ B(0, k1ν ). Hence hBν i := hni + hkν i + hKν i is the encoding length bits of Bν . Recall that for > 0, [ S(Bν , ) = B(x, ), S(Bν , −) = {x ∈ Bν : B(x, ) ⊂ Bν }. x∈Bν
The membership problem (mem) for Bν is to determine if a given y ∈ Rn is in Bν . A weak membership problem (wmem) in Bν is: given y ∈ Qn and a rational number δ > 0 assert that y ∈ S(Bν , δ) or y 6∈ S(Bν , −δ). The weak validity problem (wval) problem for Bν is as follows. Given a vector c ∈ Qn and rational number γ, > 0 either assert that cT x ≤ γ + for all x ∈ S(Bν , −), or assert cT x ≥ γ − for some x ∈ S(Bν , ). The fundamental Yudin–Nemirovski theorem [7] implies that if there exists a deterministic algorithm solving wmem problem for Bν , y, δ in Poly(hBν i + hδi) then there exists a deterministic algorithm solving wval problem for Bν , c, γ, δ in Poly(hBν i + hci + hγi + hδi). Theorem 5.2. The wmem in the unit ball of a norm ν is polynomial if and only if the wmem for the unit balls in the dual norm ν∗ is polynomial. To prove this theorem we give a number of estimations. Some of them are already in [8]. For a compact set K ⊂ Rn and c ∈ Rn let M (K, c) := maxx∈K cT x. Recall that 1 1 . ν(x) = M (Bν∗ , x), Kν∗ = , kν∗ = kν Kν Lemma 5.3. Let ν be a norm on Rn and δ > 0. Then (1 + kν δ)Bν ⊂ S(Bν , δ) ⊂ (1 + Kν δ)Bν ,
(25)
(1 − Kν δ)Bν ⊂ S(Bν , −δ) ⊂ (1 − kν δ)Bν for Kν δ < 1, δ δ δ 1− < 1, ν(x) ≥ M (S(Bν∗ , −δ), x) ≥ 1 − ν(x) for Kν kν kν δ δ ν(x) ≤ M (S(Bν∗ , δ), x) ≤ 1 + ν(x). 1+ Kν kν
(26) (27) (28)
Proof. First observe the containments kν Bν ⊂ Bk·k ⊂ Kν Bν . So if we replace in definition of S(Bν , δ) the set B(x, δ) = {y ∈ Rn : ky − xk ≤ δ}, where x ∈ Bν , by {y ∈ Rn : ν(y − x) ≤ Kν δ} we increase the set S(Bν , δ). This proves the right-hand side of (25). On the other hand, if we replace B(x, δ) by {y ∈ Rn : ν(y − x) ≤ kν δ} we decrease S(Bν , δ). This proves the right-hand side of (25). S To prove (26) we argue as follows. Let T = x:ν(x)=1 {y ∈ Rn : ky − xk < δ}. Then S(Bν , −δ) = Bν \ T . Let [ [ T1 = {y ∈ Rn : ν(y − x) < Kν δ}, T2 = {y ∈ Rn : ν(y − x) < kν δ}. x:ν(x)=1
x:ν(x)=1
So T1 ⊃ T ⇒ S(Bν , −δ) ⊃ Bν \ T1 , T2 ⊂ T ⇒ S(Bν , −δ) ⊂ Bν \ T2 . This establishes (26). To show the last two inequalities use the first two inequalities and (23).
The above results show that the wmem in Bν is polynomially equivalent to the problem of finding any (rational) δ > 0 approximation of ν(x), where x ∈ Qn . (See next section for more details.) Lemma 5.4. Assume that kν ≥ 2. Then wval in Bν∗ implies wmem in Bν .
COMPUTATIONAL COMPLEXITY OF TENSOR NUCLEAR NORM
9
Proof. Let x ∈ Qn and a rational number δ ∈ (0, 12 ). Choose γ = 1. Suppose that xT y ≤ 1 + δ for 1+δ all y ∈ S(Bν∗ , −δ). Hence M (S(Bν∗ , −δ), x) ≤ 1 + δ. Use (27) to deduce that ν(x) ≤ 1−δ/k . Since ν kν ≥ 2 it follows that
1+δ 1−δ/kν
≤ 1 + kν δ. Use (25) to deduce that x ∈ S(Bν , δ).
xT y
Suppose that > 1 − δ for some y ∈ S(Bν∗ , δ). Hence S(Bν , δ), δ) > 1 − δ. Use (28) to 1−δ 1−δ deduce that ν(x) > 1+δ/k . As straightforward calculation shows that 1+δ/k ≥ 1 − kν δ. Use (26) ν ν to deduce that x 6∈ S(Bν , −δ). The assumption that kν ≥ 2 is not restrictive. Let r ≥ 2/kν . Then a new norm νr (x) = rν(x) satisfies the assumption of the above lemma. Note that x ∈ Bν if and only if 1r x ∈ Bνr . The proof of Theorem 5.2 follows from the above lemmas. 6. Weak membership and norm approximation In this section we show that for a given norm ν : Rn → [0, ∞) satisfying (21), where kν , Kν are rational, the weak membership in Bν with respect to rational δ is polynomially equivalent to δ approximation of the norm ν. We say that we can approximate a norm ν, satisfying (21) polynomially if the following conditions are satisfied. Let kxk = 1 and δ ∈ (0, κν ) be rational. Then we can compute in polynomial time in n + hδi + hKν i + hkν i an approximation ω(x) such that ω(x) − δ < ν(x) < ω(x) + δ, for kxk = 1.
(29)
Theorem 6.1. Assume that the norm ν : Rn → [0, ∞) is satisfying (21), where kν , Kν are rational. Then the following are equivalent: (i) ν is polynomially approximable. (ii) The weak membership in Bν is polynomial. Proof. Suppose that (i) holds. Let x ∈ Rn and a rational δ > 0 is given. If kxk ≤ K1ν then ν(x) ≤ 1. Hence x ∈ S(Bν , δ). Suppose that kxk ≥ k1ν . So ν(x) ≥ 1. Hence x 6∈ S(Bν , −δ). So we can assume that kxk ∈ ( K1ν , k1ν ). Let y := x/kxk. (Observe that ν(y) ∈ [kν , Kν ].) Let = approximation of ν(y). Assume first that kν δ kxkω(y) ≤ 1 + kν δ − =1+ . kν 2 We then claim that x ∈ S(Bν , δ). Indeed ν(x) < kxk(ω(y) + ) < kxkω(y) + (25) yields that x ∈ S(Bν , δ). Assume now that kxkω(y) > 1 +
kν δ 2 .
kν2 δ 2 .
Let ω(y) be an
1 ≤ 1 + kν δ. kν
Then
ν(x) > kxk(ω(y) − ) ≥ kxkω(y) −
kν δ kν δ >1+ − = 1. kν 2 2
So x 6∈ S(Bν , −δ). Assume now that (ii) holds. Let x, kxk = 1 and a rational δ is given. So ν(x) ∈ [kν , Kν ]. Let ν ν −kν a = kν +K and = 2KK . 2 ν (Kν +kν ) 1 We now consider y = a x. Suppose first that y ∈ S(Bν , ). Then it follows that ν(x) ≤ 34 Kν + 14 kν . (Use (25).) Suppose that now that y 6∈ S(Bν , −). (26) yields that 3 1 ν(x) ≥ a(1 − Kν ) = Kν + kν . 4 4
10
S. FRIEDLAND AND L.-H. LIM
So in the beginning of our process we know that ν(x) is in the interval of length Kν − kν . After one iteration we know that ν(x) is in the interval of length 34 (Kν − kν ). Let m be the smallest integer that ( 34 )m (Kν − kν ) < 2δ. m is polynomial in hKν i + hkν i + hδi. Repeating our process m times we deduce that we obtain in interval of length ( 43 )m (Kν − kν ) in the interval [kν , Kν ] were ν(x) is located. Letting ω(x) to be equal to the middle of this interval gives the δ approximation of ν(x). 7. Weak membership in tensor nuclear norm unit ball is NP-hard We show that the NP-hardness of the weak membership problem for the nuclear norm unit ball of 4-tensor over C and R. In the following, we write Q[i] := {a + bi : a, b ∈ Q} for the Gaussian rationals. Theorem 7.1. Given A ∈ Q[i]n×n×n×n or A ∈ Qn×n×n×n and 0 < δ ∈ Q, deciding whether A ∈ S(Bk · k∗ , δ) or A 6∈ S(Bk · k∗ , −δ) is an NP-hard problem. Proof. By Theorems 6.1 and 4.5, we deduce that wmem for the tensor spectral norm unit ball is NP-hard over C and R. Since the tensor spectral and nuclear norms are dual to each other over C and R, by Theorem 5.2, the wmem for the tensor nuclear norm unit ball is also NP-hard over C and R. References ¨ [1] S. Banach, “Uber homogene Polynome in (L ),” Studia Math., 7 (1938), pp. 36–44. [2] A. Defant and K. Floret, Tensor Norms and Operator Ideals, North-Holland, Amsterdam, 1993. [3] S. Friedland, “Best rank-one approximation of real symmetric tensors can be chosen symmetric,” Front. Math. China, 8 (2013), pp. 19–40. [4] S. Friedland, “Variation of tensor powers and spectra,” Linear and Multilinear Algebra, 12 (1982/83), no. 2, pp. 81–98. [5] S. Gharibian, “Strong NP-hardness of the Quantum separability problem,” Quantum Inf. Comput., 10 (2010), no. 3–4, pp. 343–360. [6] A. Grothendieck, “Produits tensoriels topologiques et espaces nucl´eaires,” Mem. Amer. Math. Soc., 1955 (1955), no. 16, 140 pp. [7] M. Gr¨ otzschel, L. Lov´ asz, and A. Schrijver, Geometric Algorithms and Combinatorial Optimization, Springer, Berlin, 1988. [8] L. Gurvits, “Classical deterministic complexity of Edmonds problem and quantum entanglement,” Proc. ACM Symp. Theory Comput. (STOC), 35, pp. 10–19, New York, NY, ACM Press, 2003. [9] C. J. Hillar and L.-H. Lim, ”Most tensor problems are NP-hard,” J. ACM, 60 (2013), no. 6, Art. 45, 39 pp. [10] P. Horodecki, “Separability criterion and inseparable mixed states with positive partial transposition,” Physics Letters A, 232 (1997), pp. 333–339. [11] L.-H. Lim, “Tensors and hypermatrices,” Handbook of Linear Algebra, 2nd Ed., CRC Press, Boca Raton, FL, 2013. [12] L.-H. Lim and P. Comon, “Blind multilinear identification,” IEEE Trans. Inform. Theory, 60 (2014), no. 2, pp. 1260–1280. [13] A. Pappas, Y. Sarantopoulos, and A. Tonge, “Norm attaining polynomials,” Bull. Lond. Math. Soc., 39 (2007), no. 2, pp. 255–264. [14] T. S. Motzkin and E. G. Strauss, “Maxima for graphs and a new proof of T´ uran,” Canadian J. Math., 17 (1965), pp. 533–540. [15] R. A. Ryan, Introduction to Tensor Products of Banach Spaces, Springer-Verlag, London, 2002. [16] R. Schatten, A Theory of Cross-Spaces, Princeton University Press, Princeton, NJ, 1950. [17] Y. C. Wong, Schwartz Spaces, Nuclear Spaces and Tensor Products, Lecture Notes in Mathematics, 726, Springer, Berlin, 1979. 2
Department of Mathematics, Statistics and Computer Science, University of Illinois, Chicago E-mail address: [email protected] Computational and Applied Mathematics Initiative, Department of Statistics, University of Chicago E-mail address: [email protected]
1. Introduction The nuclear norm of a 2-tensor (or, in coordinate form, a matrix) has recently found widespread use as a convex surrogate for relaxing various intractable non-convex problems into tractable convex problems. The motivation for this article is to investigate the computational complexity of the nuclear norm for higher order tensors. We investigate three specific problems: (i) the weak membership problem for the nuclear norm unit ball of 3-tensors over R, (ii) the weak membership problem for the nuclear norm unit ball of 4-tensors over R and C, (iii) the approximation of nuclear norm of 4-tensors over R and C. We will show that all these problems are NP-hard. In this article we use the term d-tensor to mean a tensor of order d. Our investigation leads to a more general result that applies to all norms, namely, the weak membership problem for a norm ball and that for its dual norm ball have equivalent complexity. Another interesting side result that we obtained in the course of our study is that a density matrix is bipartite separable iff its associated 4-tensor has unit nuclear norm. Taken together with Gurvits’s result [8] on the NP-hardness of bipartite separability, we deduce that the membership problem for the nuclear norm unit ball of 4-tensors over C is NP-hard, which is of course also implied by (ii) above. In short, the nuclear norm is NP-hard to compute for real 3-tensors and complex 4-tensors. As usual, our proofs depend on reductions to existing problems known to be NP-hard. The NPhardness of the nuclear norm of real 3-tensor relies on that of the spectral norm of real 3-tensors [9]. The NP-hardness of the nuclear norm of complex 4-tensors relies on a characterization of the clique number of a graph as the spectral norm of a 4-tensor that is bi-hermitian, positive semidefinite, and nonnegative-valued, together with the fact that clique number is NP-hard. 1.1. Outline. In Section 2 we define the nuclear and spectral norms for tensors of arbitrary orders over C and R. In Section 3 we discuss bipartite separability in quantum mechanics and show that a bipartite state is separable if and only if its nuclear norm is one. Using Gurvits’s result [8] that bipartite membership is NP-hard we deduce that deciding membership in the nuclear norm unit ball is NP-hard. In Section 4 we relate Motzkin–Strauss’s characterization of the clique number of a graph as the spectral norm of a 4-tensor defined by the graph. It then follows that -approximation of the nuclear norm for 4-tensors over C and R, or equivalently, of biparitite density matrices, is NP-hard. In Section 5 we show that weak membership in the unit ball of a given norm can be decided in polynomial time if and only if weak membership in the unit ball of its dual norm can be decided in polynomial time. In Section 6 we show that weak membership in the unit ball of a given 1
2
S. FRIEDLAND AND L.-H. LIM
norm can be decided in polynomial if and only if -approximation of the given norm is polynomial time. In Section 7 we apply our results to show that deciding membership in the tensor nuclear norm unit ball is NP-hard. 2. Tensor nuclear and spectral norms Let F = R or C. Let Fn1 ×···×nd := Fn1 ⊗ · · · ⊗ Fnd be the space of d-tensors of dimensions n1 , . . . , nd ∈ N. If desired, these may be viewed as d-dimensional hypermatrices A = (ai1 ···id ) with entries ai1 ···id ∈ F. The hermitian inner product of two d-tensors A, B ∈ Cn1 ×···×nd is given by Xn1 ,...,nd ai1 ···id bi1 ···id . hA, Bi = i1 ,...,id =1
This induces the Hilbert–Schmidt norm, denoted by 1 Xn1 ,...,nd p 2 |ai1 ···id |2 . kAk = hA, Ai = i1 ,...,id =1
We adopt the convention that an unlabeled k · k will always denote the Hilbert–Schmidt norm. Note that when d = 1, this is just the regular hermitian or l2 -norm of a vector in Cn and when d = 2, this is the Frobenius norm of a matrix in Cm×n . The norms of greatest interest to us in this article are the spectral norm and nuclear norm of a d-tensor A ∈ Cn1 ×···×nd . These are denoted and defined respectively by Re(hA, x1 ⊗ · · · ⊗ xd i) |hA, x1 ⊗ · · · ⊗ xd i| = max , max x1 ,...,xd 6=0 kx1 k · · · kxd k kx1 k · · · kxd k nXr o Xr kAk∗ := min kx1,p k · · · kxd,p k : A = x1,p ⊗ · · · ⊗ xd,p , r ∈ N .
kAkσ :=
x1 ,...,xd 6=0
p=1
p=1
(1) (2)
It is easy to see that k · kσ and k · k∗ are dual norms [12, Lemma 21] and that kx1 ⊗ · · · ⊗ xd kσ = kx1 ⊗ · · · ⊗ xd k∗ = kx1 k · · · kxd k. We would like to point out that (2) is the definition of tensor nuclear norm as originally defined by Grothendieck [6] and Schatten [16]. An alternate definition of ‘tensor nuclear norm’ as the average of nuclear norms of matrices obtained from flattenings of a tensor has gained recent popularity. While this alternate definition may be useful for various purposes, it is nevertheless not the definition commonly accepted in mathematics [2, 15, 13, 17] (see also [4, 12]). The nuclear norm defined in (2) is precisely the dual norm of the spectral norm in (1) and is naturally related to the notion of tensor rank (cf. [11]). Moreover, we will prove in Section 3 that nuclear norm as defined in (2) has physical meaning — equivalent to bipartite separability of quantum states in an appropriate sense. As such, a tensor nuclear norm in this article will always be the one in (2). Tensor rank is known to depend on the choice of base field [11]. We do not know if it might be the same for nuclear and spectral norms. As such, we need a more careful definition. Let B(Fn ) := {x ∈ Fn : kxk ≤ 1},
S(Fn ) := {x ∈ Fn : kxk = 1},
be the Euclidean unit ball and sphere in Fn (recall that k · k denotes the Euclidean norm on Fn ). Then for A ∈ Fn1 ×···×nd , we define n o kAkσ,F := max |hA, x1 ⊗ · · · ⊗ xd i| : xi ∈ B(Fni ) , (3) nXr Yd o Xr kAk∗,F := min kxi,p k : A = x1,p ⊗ · · · ⊗ xd,p , xi,p ∈ Fni , r ∈ N . (4) p=1
i=1
p=1
The indices i and p are always assumed to run over i = 1, . . . , d and p = 1, . . . , r respectively. Clearly, for any A ∈ Rn1 ×···×nd , kAkσ,R ≤ kAkσ,C ,
kAk∗,R ≥ kAkσ,C .
(5)
COMPUTATIONAL COMPLEXITY OF TENSOR NUCLEAR NORM
3
It is well known that for d = 2 we have equality signs in the above inequalities. We suspect that this is not the case for d ≥ 3. Let x = (x1 , . . . , xn )T ∈ Cn . Denote by |x| := (|x1 |, . . . , |xn |)T . Then x is called a nonnegative vector, denoted as x ≥ 0, if x = |x|. We will also use this notation for tensors in Cn1 ×···×nd . Lemma 2.1. Let A ∈ Cn1 ×···×nd . Then kAkσ,C ≤ k|A|kσ,C ,
k|A|kσ,C = k|A|kσ,R .
Proof. The triangle inequality yields |hA, x1 ⊗ · · · ⊗ xd i| ≤ h|A|, x1 ⊗ · · · ⊗ xd i. Recall that the Euclidean norm on Cn is an absolute norm, i.e., kxk = k|x|k. The definitions of k · kσ,C and k · kσ,R and the above inequality yields the result. The equality in the above lemma fails for nuclear norm even for matrices. Indeed, let √ √ 1/ √2 1/√2 . Q= −1/ 2 1/ 2 √ Then kQk∗ = 2 > k|Q|k∗ = 2. A recent example of R. Tomioka shows that the equality in Lemma 2.1 fails for 4 × 4 hermitian matrices. Henceforth, by spectral and nuclear norm we will always mean over C, unless stated otherwise. We will denote k · kσ := k · kσ,C ,
k · k∗ := k · k∗,C .
In what follows we show that kAkσ,F and kAk∗,F are NP-hard to compute for d ≥ 3 by using appropriate results. If we show that kAkσ,R is NP-hard to compute for A ≥ 0, then Lemma 2.1 yields that kAkσ,C is also NP-hard to compute. 3. Bipartite separability and nuclear norm We show that every density matrix corresponds uniquely and naturally to a 4-tensor. Furthermore the density matrix is bipartite separable if and only if its corresponding 4-tensor has unit nuclear norm. It then follows from the well-known NP-hardness of bipartite separability [8] that tensor nuclear norm is also NP-hard. Assume throughout the following that mj+d = mj
for all j = 1, . . . , d.
(6)
Then we may write Cm1 ×...md ×md+1 ×···×m2d = Cm1 ×···×md ⊗ Cm1 ×···×md = (Cm1 ×···×md )⊗2 . For each A = (ai1 ···i2d ) ∈ (Cm1 ×···×md )⊗2 , we define the trace as Xm1 ,...,md tr(A) := ai1 ···id i1 ···id . i1 =···=id =1
(7)
It is straightforward to see that tr(x1 ⊗ · · · ⊗ x2d ) =
Yd j=1
xT j+d xj
(8)
for every xj , xj+d ∈ Cmj , j = 1, . . . , d. Let Hm×m ⊂ Cm×m be the real vector subspace of hermitian matrices. Let Hm×m be the cone + m×m of nonnegative definite hermitian matrices and Hρ be the convex set of density matrices, i.e., nonnegative definite hermitian matrices of trace one. Clearly, Hm×m ⊂ Hm×m ⊂ Hm×m . ρ + Since (Cm1 ×···×md )⊗2 ∼ = Cm1 ···md ×m1 ···md , we may identify tensors of order 2d satisfying (6) with m1 · · · md × m1 · · · md matrices. We will adopt this identification in subsequent discussions and will regard 2d-tensors satisfying (6) interchangeably with m1 · · · md × m1 · · · md matrices. In which
4
S. FRIEDLAND AND L.-H. LIM
case, Hermitian matrices Hm1 ···md ×m1 ···md ⊂ Cm1 ···md ×m1 ···md may be identified with a subset of 2d-tensors that we denote by H(m1 , . . . , md ) ⊂ (Cm1 ×···×md )⊗2 . We will call H(m1 , . . . , md ) the space of d-hermitian tensors. The density matrices and nonnegative definite hermitian matrices may then be regarded as d-hermitian tensors. We denote these by Hρ (m1 , . . . , md ) ⊂ H+ (m1 , . . . , md ) ⊂ H(m1 , . . . , md ). Note that a d-hermitian tensor is a tensor of order 2d. Alternatively, one may define them directly by stating that A = (ai1 ···i2d ) ∈ (Cm1 ×···×md )⊗2 is a d-hermitian tensor if and only if ai1 ···id id+1 ···i2d = aid+1 ···i2d i1 ···id
for all ij , id+j = 1, . . . , mj , j = 1, . . . , d.
We call the convex set Hρ (m1 , . . . , md ) the set of d-partite density matrices. An element of m ×m the form A1 ⊗ · · · ⊗ Ad ∈ Hρ (m1 , . . . , md ) is called a product density matrix if Aj ∈ Hρ j j for j = 1, . . . , d. Let Hsep (m1 , . . . , md ) be the convex set spanned by product density matrices. Since Hm×m is a convex combination of rank-one hermitian matrices xx∗ , x∗ x = 1, called pure states, it ρ follows that Hsep (m1 , . . . , md ) is a convex combination of product pure states (x1 x∗1 ) ⊗ · · · ⊗ (xd x∗d ),
x∗j xj = 1,
j = 1, . . . , d.
A density matrix A ∈ Hρ (m1 , . . . , md ) is called d-partite separable if A ∈ Hsep (m1 , . . . , md ). Lemma 3.1. Let A ∈ (Cm1 ×···×md )⊗2 . Then |tr(A)| ≤ kAk∗ ,
(9)
and equality holds if and only if A = zB for some z ∈ C and B ∈ Hsep (m1 , . . . , md ). Furthermore, a density matrix A ∈ Hρ (m1 , . . . , md ) is d-partite separable if and only if it has unit nuclear norm. P Q P Proof. Let A = ri=1 x1,i ⊗ · · · ⊗ x2d,i , where kAk∗ = ri=1 2d j=1 kxj,i k > 0. In view of (8), tr(A) = Pr Qd T T i=1 j=1 (xj+d,i xj,i ). The Cauchy–Schwarz inequality yields that |xj+d,i xj,i | ≤ kxj,i kkxj+d,i k. Equality holds if and only if xj+d,i = zj,i x ¯j,i for some zj,i ∈ C. Thus X Y Xr Yd r 2d T ≤ | tr A| ≤ x ) (x kxj,i k = kAk∗ . j,i j+d,i i=1 j=1 i=1 j=1 This establishes (9). Suppose that equality holds in (9). Then x1,i ⊗ · · · ⊗ x2d,i is of the form zj,i (x1,i x∗1,i ) ⊗ · · · ⊗ (xd,i x∗d,i ). Observe that Yd tr zj,i (x1,i x∗1,i ) ⊗ · · · ⊗ (xd,i x∗d,i ) = zj,i kxj,i k2 . j=1
Without loss of generality we may assume that kxj,i k = 1 for j = 1, . . . , d. Since equality holds in the triangle inequality it follows that all zj,i must have the same arguments. Hence A = zB where Xr B= ti (x1,i x∗1,i ) ⊗ · · · ⊗ (xd,i x∗d,i ), (10) i=1 P where x∗j,i xj,i = 1 for j = 1, . . . , d, and ri=1 ti = 1, ti ≥ 0, for i = 1, . . . , r. Conversely, suppose B is d-partite separable. Hence B is of the above form. Therefore Xr Yd kBk∗ ≤ ti kxj,i k2 = 1. i=1
j=1
Clearly, tr(B) = 1. In view of (9), it follows that kBk∗ = 1. Hence a decomposition (10) of B is minimal with respect to the nuclear norm. We will use the following hardness result from [8] (see also [5]). Theorem 3.2 (Gurvits). Deciding whether a given density matrix is bipartite separable is an NPhard problem.
COMPUTATIONAL COMPLEXITY OF TENSOR NUCLEAR NORM
5
From this and Lemma 3.1, we immediately deduce our first hardness result for tensor nuclear norm. Corollary 3.3. Deciding whether a given 4-tensor is in the nuclear norm unit ball is an NP-hard problem. In the following, we will show that a stronger result, namely, not only membership but even weak membership in the nuclear norm unit ball is also NP-hard to decide over R and C. 4. Clique number and nuclear norm Let G = (V, E) be a graph with V := {1, . . . , n} and the set of undirected edges E = {(ik , jk ) : ik = 1, . . . , jk − 1, k = 1, . . . , m}. Let κ(G) be the clique number of G, i.e., the size of the maximal clique in G. Denote by AG the adjacency matrix of G. Let ∆n be the simplex of probability vectors on Rn . Motzkin and Strauss showed that κ(G) − 1 = maxn xT AG x. (11) x∈∆ κ(G) Equality is attained when x is uniformly distributed on the largest clique. We now transform (11) to one involving 4-tensors. Let x = y ◦2 , i.e., x = (y12 , . . . , yn2 )T . Then X (y ◦2 )T AG y ◦2 = yi2 yj2 , y T y = 1. (12) (i,j)∈E(G)
We will see how a term of the form
ys2 yt2
may be regarded as a 4-tensor.
m×n×m×n be a 4-tensor. We call it bi-symmetric if Definition 4.1. Let A = (aijpq )m,n,m,n i,j,p,q=1 ∈ C
aijpq = apqij
for all i, p = 1, . . . , m, j, q = 1, . . . , n,
(13)
aijpq = a ¯pqij
for all i, p = 1, . . . , m, j, q = 1, . . . , n.
(14)
and bi-hermitian if
A bi-hermitian tensor A = (aijpq ) is said to be positive semidefinite if Xm,n,m,n aijpq xij x ¯pq ≥ 0 i,j,p,q=1
for all X = [xij ] ∈ Cm×n . This is a special case of the terms introduced in Section 3 for d = 2, except that we use the more verbal ‘bi-hermitian’ for ‘2-hermitian’, and likewise for our use of the term ‘bipartite’ for m×n×m×n as a 2-partite later. As in Section 3, we may regard a 4-tensor A = (aijpq )m,n,m,n i,j,p,q=1 ∈ C matrix C(A) = [c(i,j),(p,q) ] ∈ Cmn×mn , where c(i,j),(p,q) = aijpq . Evidently, A is bi-symmetric (resp. bi-hermitian) if and only if C(A) is symmetric (resp. hermitian). (s,t) For integers 1 ≤ s < t ≤ n, we consider Ast = (aijpq )ni,j,p,q=1 ∈ Cn×n×n×n defined by 1/2 i = s, j = t, p = s, q = t, 1/2 i = t, j = s, p = t, q = s, (s,t) aijpq = 1/2 i = s, j = t, p = t, q = s, (15) 1/2 i = t, j = s, p = s, q = t, 0 otherwise. Note that the quartic form hAst , y ⊗ y ⊗ y ⊗ yi = 2ys2 yt2 . Lemma 4.2. The tensor Ast is bi-hermitian, positive semidefinite, and has all entries nonnegative.
6
S. FRIEDLAND AND L.-H. LIM
Proof. It is easy to see that Ast is bi-hermitian and nonnegative. It is positive semidefinite because Xn,n,n,n (s,t) 1 a xij x ¯pq = (xst + xts )(¯ xst + x ¯ts ) ≥ 0 i,j,s,t=1 ijpq 2 for all X = [xij ] ∈ Cn×n . C(Ast ) is evidently a nonnegative definite, rank-one matrix with trace one. Hence C(Ast ) represents a density matrix on a bipartite state. For any graph G, we will let 1 X Ast ∈ Cn×n×n×n , (16) AG := (s,t)∈E(G) m and C(AG ) ∈ Cn entries.
2 ×n2
be its corresponding matrix. Note that AG and C(AG ) have real-valued
Theorem 4.3. Let G be a simple undirected graph on n vertices with m edges. Denote AG as above. Then |hAG , x ⊗ y ⊗ u ⊗ vi| kAG kσ := max n 06=x,y,u,v∈C kxkkykkukkvk hAG , x ⊗ y ⊗ u ⊗ vi = max n (17) 06=x,y,u,v∈R+ kxkkykkukkvk = maxn
06=y∈R+
hAG , y ⊗ y ⊗ y ⊗ yi . kyk4
(18)
Since all entries of AG are nonnegative we immediately deduce the equality (17). If the AG is a symmetric 4-tensor as opposed to merely bisymmetric, then we may apply a classical result of Banach [1, 3] to deduce that the maximum is attained at x = y = u = v and deduce (18). Unfortunately for us, AG is not symmetric and we need to prove Theorem 4.3 from scratch. We start with the following lemma which may be of independent interest. Lemma 4.4. Let H = [h(i,j),(pq) ] ∈ Cmn×mn be a hermitian nonnegative definite matrix. Define A = (aijpq ) ∈ Cm×n×m×n be the equality aijpq = h(i,j),(p,q) . Then kAkσ =
max
06=x∈Cm , 06=y∈Cn
hA, x ⊗ y ⊗ x ¯ ⊗ y¯i . kxk2 kyk2
(19)
Proof. Recall that we may always write H = R2 for some hermitian R ∈ Cmn×mn . Consider the sesquilinear form w ¯ T Hz = (Rw)T (Rz). Use Cauchy–Schwarz inequality to see that √ √ |¯ z T Hw| ≤ z¯T Hz w ¯ T Hw ≤ max(¯ z T Hz, w ¯ T Hw). Now let z := x ⊗ y, w := u ¯ ⊗ v¯ and deduce the lemma.
Proof of Theorem 4.3. We first apply Lemma 4.4 to (17). Hence we can assume that u = x ≥ 0, y = v ≥ 0. Hence 2hAst , x ⊗ y ⊗ x ⊗ yi = (xs yt + xt ys )2 . Use Cauchy–Schwarz inequality to see that (x2 + ys2 ) (x2t + yt2 ) (xs yt + xt ys )2 ≤ 4 s × . 2 2 p p So introducing two new variables as = (x2s + ys2 )/2 and at = (x2t + yt2 )/2, we reduced our problem to the problem of degree 4 in one vector variable a = (a1 , . . . , an ), where kak = 1. This is exactly the characterization (18). It was shown in [9] that a computation and approximation of spectral norm of d-tensor, for d ≥ 3, in NP-hard over the reals. We now give a version of the NP-hardness of a computation and approximation of tensor spectral norm of bi-hermitian and real bi-symmetric 4-tensors over complexes and reals:
COMPUTATIONAL COMPLEXITY OF TENSOR NUCLEAR NORM
7
Theorem 4.5. It is NP-hard to approximate tensor spectral norm of bi-hermitian 4-tensors over C and bi-symmetric real 4-tensor over R corresponding to bipartite density matrices. Proof. Let AG be defined by (16). Then AG is a nonnegative bi-symmetric tensor. Furthermore, the matrix C(AG ) is positive semi-definite. Since m is the number of edges in E(G) it follows that the trace of C(AG ) is 1. Hence C(AG ) represents a real bipartite density matrix. Motzkin–Strauss theorem yields that κ(G) − 1 kAG kσ = kAG kσ,R = . (20) mκ(G) Since the computation of clique number of a graph is NP-hard, the above identity implies that the computation of the spectral norm corresponding to bipartite density matrices is NP-hard over C and R. Since the clique number of a graph is an integer, it follows that it is NP-hard to approximate the spectral norms of the corresponding 4-tensors over complex or real numbers. 5. Weak membership in a norm unit ball Let F = R or C and ν : Fn → [0, ∞) be a norm. Denote by Bν := {x ∈ Fn : ν(x) ≤ 1} the closed unit ball with respect to ν. Let k · k be the Euclidean norm on Fn and B(a, r) := {x ∈ Fn : kx−ak ≤ r}. Since all norms in Fn are equivalent, it follows that there exist constants Kν ≥ kν > 0 such that kν kxk ≤ ν(x) ≤ Kν kxk, for all x ∈ Fn . (21) In what follows we assume that kν , Kν are rational. We denote by hkν i, hKν i the number of bits corresponding to kν , Kν . Q Identify Fn1 ×···×nd with Fn , where n = di=1 ni . Assume that A ∈ Fn1 ×···×nd . Then kAk is the Hilbert–Schmidt norm, which is the Euclidean norm in Fn . Lemma 5.1. Let k · kσ,F be the spectral norm in Fn1 ×···×nd . Then √
1 n1 n2 . . . nd
kAk ≤ kAkσ,F ≤ kAk
for all A ∈ Fn1 ×···×nd .
(22)
Proof. Clearly, kAkσ,F ≤ kAk. Assume that A = (ai1 ···id ). Let kAkmax = max{|ai1 ···id | : ij = √ 1, . . . , nj , j = 1, . . . , d}. Clearly, kAk ≤ n1 n2 . . . nd kAkmax . Choose in characterization (3) each xj ∈ Fnj to be a standard unit vector. It then follows that kAkmax ≤ kAkσ,F . Hence the left-hand side of (22) holds. Recall that the dual norm of ν, denoted by ν∗ , is given by ν∗ (x) = max{Re(y ∗ x) : ν(y) ≤ 1}. Hence 1 1 kxk ≤ ν∗ (x) ≤ kxk. Kν kν In particular kAk ≤ kAk∗,F ≤
√
n1 n2 . . . nd kAk
for all A ∈ Fn1 ×···×nd .
(23)
(24)
In what follows, for simplicity of the exposition we assume√that F = R. It will be convenient to identify Cn with R2n ≡ Rn × Rn . So z ∈ Cn is viewed as x + −1y, where (x, y) ∈ Rn ×√Rn . Hence a norm ν : Cn → [0, ∞) induces a norm ν˜ : R2n → [0, ∞), where ν˜((xT , y T )T ) := ν(x + −1y). By abusing the notation we will identify ν with with ν˜. Note that the Euclidean norm on Cn gives rise to the Euclidean norm on R2n . Hence, from the complexity point of view it is enough to consider norms over the real valued spaces.
8
S. FRIEDLAND AND L.-H. LIM
In what follows we use the definitions and results from [7]. Observe first that Bν is a well-bounded 0-centered set. More precisely B(0, K1ν ) ⊂ Bν ⊂ B(0, k1ν ). Hence hBν i := hni + hkν i + hKν i is the encoding length bits of Bν . Recall that for > 0, [ S(Bν , ) = B(x, ), S(Bν , −) = {x ∈ Bν : B(x, ) ⊂ Bν }. x∈Bν
The membership problem (mem) for Bν is to determine if a given y ∈ Rn is in Bν . A weak membership problem (wmem) in Bν is: given y ∈ Qn and a rational number δ > 0 assert that y ∈ S(Bν , δ) or y 6∈ S(Bν , −δ). The weak validity problem (wval) problem for Bν is as follows. Given a vector c ∈ Qn and rational number γ, > 0 either assert that cT x ≤ γ + for all x ∈ S(Bν , −), or assert cT x ≥ γ − for some x ∈ S(Bν , ). The fundamental Yudin–Nemirovski theorem [7] implies that if there exists a deterministic algorithm solving wmem problem for Bν , y, δ in Poly(hBν i + hδi) then there exists a deterministic algorithm solving wval problem for Bν , c, γ, δ in Poly(hBν i + hci + hγi + hδi). Theorem 5.2. The wmem in the unit ball of a norm ν is polynomial if and only if the wmem for the unit balls in the dual norm ν∗ is polynomial. To prove this theorem we give a number of estimations. Some of them are already in [8]. For a compact set K ⊂ Rn and c ∈ Rn let M (K, c) := maxx∈K cT x. Recall that 1 1 . ν(x) = M (Bν∗ , x), Kν∗ = , kν∗ = kν Kν Lemma 5.3. Let ν be a norm on Rn and δ > 0. Then (1 + kν δ)Bν ⊂ S(Bν , δ) ⊂ (1 + Kν δ)Bν ,
(25)
(1 − Kν δ)Bν ⊂ S(Bν , −δ) ⊂ (1 − kν δ)Bν for Kν δ < 1, δ δ δ 1− < 1, ν(x) ≥ M (S(Bν∗ , −δ), x) ≥ 1 − ν(x) for Kν kν kν δ δ ν(x) ≤ M (S(Bν∗ , δ), x) ≤ 1 + ν(x). 1+ Kν kν
(26) (27) (28)
Proof. First observe the containments kν Bν ⊂ Bk·k ⊂ Kν Bν . So if we replace in definition of S(Bν , δ) the set B(x, δ) = {y ∈ Rn : ky − xk ≤ δ}, where x ∈ Bν , by {y ∈ Rn : ν(y − x) ≤ Kν δ} we increase the set S(Bν , δ). This proves the right-hand side of (25). On the other hand, if we replace B(x, δ) by {y ∈ Rn : ν(y − x) ≤ kν δ} we decrease S(Bν , δ). This proves the right-hand side of (25). S To prove (26) we argue as follows. Let T = x:ν(x)=1 {y ∈ Rn : ky − xk < δ}. Then S(Bν , −δ) = Bν \ T . Let [ [ T1 = {y ∈ Rn : ν(y − x) < Kν δ}, T2 = {y ∈ Rn : ν(y − x) < kν δ}. x:ν(x)=1
x:ν(x)=1
So T1 ⊃ T ⇒ S(Bν , −δ) ⊃ Bν \ T1 , T2 ⊂ T ⇒ S(Bν , −δ) ⊂ Bν \ T2 . This establishes (26). To show the last two inequalities use the first two inequalities and (23).
The above results show that the wmem in Bν is polynomially equivalent to the problem of finding any (rational) δ > 0 approximation of ν(x), where x ∈ Qn . (See next section for more details.) Lemma 5.4. Assume that kν ≥ 2. Then wval in Bν∗ implies wmem in Bν .
COMPUTATIONAL COMPLEXITY OF TENSOR NUCLEAR NORM
9
Proof. Let x ∈ Qn and a rational number δ ∈ (0, 12 ). Choose γ = 1. Suppose that xT y ≤ 1 + δ for 1+δ all y ∈ S(Bν∗ , −δ). Hence M (S(Bν∗ , −δ), x) ≤ 1 + δ. Use (27) to deduce that ν(x) ≤ 1−δ/k . Since ν kν ≥ 2 it follows that
1+δ 1−δ/kν
≤ 1 + kν δ. Use (25) to deduce that x ∈ S(Bν , δ).
xT y
Suppose that > 1 − δ for some y ∈ S(Bν∗ , δ). Hence S(Bν , δ), δ) > 1 − δ. Use (28) to 1−δ 1−δ deduce that ν(x) > 1+δ/k . As straightforward calculation shows that 1+δ/k ≥ 1 − kν δ. Use (26) ν ν to deduce that x 6∈ S(Bν , −δ). The assumption that kν ≥ 2 is not restrictive. Let r ≥ 2/kν . Then a new norm νr (x) = rν(x) satisfies the assumption of the above lemma. Note that x ∈ Bν if and only if 1r x ∈ Bνr . The proof of Theorem 5.2 follows from the above lemmas. 6. Weak membership and norm approximation In this section we show that for a given norm ν : Rn → [0, ∞) satisfying (21), where kν , Kν are rational, the weak membership in Bν with respect to rational δ is polynomially equivalent to δ approximation of the norm ν. We say that we can approximate a norm ν, satisfying (21) polynomially if the following conditions are satisfied. Let kxk = 1 and δ ∈ (0, κν ) be rational. Then we can compute in polynomial time in n + hδi + hKν i + hkν i an approximation ω(x) such that ω(x) − δ < ν(x) < ω(x) + δ, for kxk = 1.
(29)
Theorem 6.1. Assume that the norm ν : Rn → [0, ∞) is satisfying (21), where kν , Kν are rational. Then the following are equivalent: (i) ν is polynomially approximable. (ii) The weak membership in Bν is polynomial. Proof. Suppose that (i) holds. Let x ∈ Rn and a rational δ > 0 is given. If kxk ≤ K1ν then ν(x) ≤ 1. Hence x ∈ S(Bν , δ). Suppose that kxk ≥ k1ν . So ν(x) ≥ 1. Hence x 6∈ S(Bν , −δ). So we can assume that kxk ∈ ( K1ν , k1ν ). Let y := x/kxk. (Observe that ν(y) ∈ [kν , Kν ].) Let = approximation of ν(y). Assume first that kν δ kxkω(y) ≤ 1 + kν δ − =1+ . kν 2 We then claim that x ∈ S(Bν , δ). Indeed ν(x) < kxk(ω(y) + ) < kxkω(y) + (25) yields that x ∈ S(Bν , δ). Assume now that kxkω(y) > 1 +
kν δ 2 .
kν2 δ 2 .
Let ω(y) be an
1 ≤ 1 + kν δ. kν
Then
ν(x) > kxk(ω(y) − ) ≥ kxkω(y) −
kν δ kν δ >1+ − = 1. kν 2 2
So x 6∈ S(Bν , −δ). Assume now that (ii) holds. Let x, kxk = 1 and a rational δ is given. So ν(x) ∈ [kν , Kν ]. Let ν ν −kν a = kν +K and = 2KK . 2 ν (Kν +kν ) 1 We now consider y = a x. Suppose first that y ∈ S(Bν , ). Then it follows that ν(x) ≤ 34 Kν + 14 kν . (Use (25).) Suppose that now that y 6∈ S(Bν , −). (26) yields that 3 1 ν(x) ≥ a(1 − Kν ) = Kν + kν . 4 4
10
S. FRIEDLAND AND L.-H. LIM
So in the beginning of our process we know that ν(x) is in the interval of length Kν − kν . After one iteration we know that ν(x) is in the interval of length 34 (Kν − kν ). Let m be the smallest integer that ( 34 )m (Kν − kν ) < 2δ. m is polynomial in hKν i + hkν i + hδi. Repeating our process m times we deduce that we obtain in interval of length ( 43 )m (Kν − kν ) in the interval [kν , Kν ] were ν(x) is located. Letting ω(x) to be equal to the middle of this interval gives the δ approximation of ν(x). 7. Weak membership in tensor nuclear norm unit ball is NP-hard We show that the NP-hardness of the weak membership problem for the nuclear norm unit ball of 4-tensor over C and R. In the following, we write Q[i] := {a + bi : a, b ∈ Q} for the Gaussian rationals. Theorem 7.1. Given A ∈ Q[i]n×n×n×n or A ∈ Qn×n×n×n and 0 < δ ∈ Q, deciding whether A ∈ S(Bk · k∗ , δ) or A 6∈ S(Bk · k∗ , −δ) is an NP-hard problem. Proof. By Theorems 6.1 and 4.5, we deduce that wmem for the tensor spectral norm unit ball is NP-hard over C and R. Since the tensor spectral and nuclear norms are dual to each other over C and R, by Theorem 5.2, the wmem for the tensor nuclear norm unit ball is also NP-hard over C and R. References ¨ [1] S. Banach, “Uber homogene Polynome in (L ),” Studia Math., 7 (1938), pp. 36–44. [2] A. Defant and K. Floret, Tensor Norms and Operator Ideals, North-Holland, Amsterdam, 1993. [3] S. Friedland, “Best rank-one approximation of real symmetric tensors can be chosen symmetric,” Front. Math. China, 8 (2013), pp. 19–40. [4] S. Friedland, “Variation of tensor powers and spectra,” Linear and Multilinear Algebra, 12 (1982/83), no. 2, pp. 81–98. [5] S. Gharibian, “Strong NP-hardness of the Quantum separability problem,” Quantum Inf. Comput., 10 (2010), no. 3–4, pp. 343–360. [6] A. Grothendieck, “Produits tensoriels topologiques et espaces nucl´eaires,” Mem. Amer. Math. Soc., 1955 (1955), no. 16, 140 pp. [7] M. Gr¨ otzschel, L. Lov´ asz, and A. Schrijver, Geometric Algorithms and Combinatorial Optimization, Springer, Berlin, 1988. [8] L. Gurvits, “Classical deterministic complexity of Edmonds problem and quantum entanglement,” Proc. ACM Symp. Theory Comput. (STOC), 35, pp. 10–19, New York, NY, ACM Press, 2003. [9] C. J. Hillar and L.-H. Lim, ”Most tensor problems are NP-hard,” J. ACM, 60 (2013), no. 6, Art. 45, 39 pp. [10] P. Horodecki, “Separability criterion and inseparable mixed states with positive partial transposition,” Physics Letters A, 232 (1997), pp. 333–339. [11] L.-H. Lim, “Tensors and hypermatrices,” Handbook of Linear Algebra, 2nd Ed., CRC Press, Boca Raton, FL, 2013. [12] L.-H. Lim and P. Comon, “Blind multilinear identification,” IEEE Trans. Inform. Theory, 60 (2014), no. 2, pp. 1260–1280. [13] A. Pappas, Y. Sarantopoulos, and A. Tonge, “Norm attaining polynomials,” Bull. Lond. Math. Soc., 39 (2007), no. 2, pp. 255–264. [14] T. S. Motzkin and E. G. Strauss, “Maxima for graphs and a new proof of T´ uran,” Canadian J. Math., 17 (1965), pp. 533–540. [15] R. A. Ryan, Introduction to Tensor Products of Banach Spaces, Springer-Verlag, London, 2002. [16] R. Schatten, A Theory of Cross-Spaces, Princeton University Press, Princeton, NJ, 1950. [17] Y. C. Wong, Schwartz Spaces, Nuclear Spaces and Tensor Products, Lecture Notes in Mathematics, 726, Springer, Berlin, 1979. 2
Department of Mathematics, Statistics and Computer Science, University of Illinois, Chicago E-mail address: [email protected] Computational and Applied Mathematics Initiative, Department of Statistics, University of Chicago E-mail address: [email protected]
