Weighted NP Optimization Problems: Logical De ... - Semantic Scholar
Weighted NP Optimization Problems: Logical De nability and Approximation Properties Marius Zimand
Abstract.
Extending a well-known property of NP optimization problems in which the value of the optimum is guaranteed to be polynomially bounded in the length of the input, it is observed that, by attaching weights to tuples over the domain of the input, all NP optimization problems admit a logical characterization. It is shown that any NP optimization problem can be stated as a problem in which the constraint conditions can be expressed by a 2 rst-order formula. The paper analyzes the weighted analogue of all syntactically de ned classes of optimization problems that are known to have good approximation properties in the nonweighted case. Dramatic changes occur when negative weights are allowed. KeywordsNP optimization problems, logical de nability, approximation properties, multiprover interactive systems. AMS Classi cation 68Q15, 68Q25.
1 Introduction Recent years have seen considerable progress in the understanding of the approximation properties of NP optimization problems. Two approaches have been the main cause for the new advancements: (a) the development of a robust model for optimization problems based on the logical de nability of this type of problems, and (b) the characterization of NP in terms of multiprover interactive proof (MIP) or probabilistically checking proof (PCP) systems which allowed new reductions that preserve approximation properties between NP complete problems. The rst approach was initiated by Papadimitriou and Yannakakis [PY91] who introduced the classes MAX SNP and MAX NP (later called MAX 0 and MAX 1 by Kolaitis and Thakur [KT94] in an attempt to uniformize notation). A maximization problem is in MAX SNP (MAX NP) if: (1) the input I is viewed as a nite structure (2) the set of feasible solutions of an input I is given by the set of nite structures S having the same domain as I and satisfying a 0 (1 ) rst-order formula having a tuple of free variables, and (3) the objective function maps each feasible solution S to the cardinality of tuples over the domain of I satisfying when substituting the tuple of free variables and School of Computer & Applied Sciences, Georgia Southwestern State University, Americus, GA 31709, email: [email protected]. Supported in part by grant NSF-CCR-8957604, NSF-INT9116781/JSPS-ENG-207 and NSF-CCR-9322513 and by the Romanian Department of Education and Science grant 4975-92. A preliminary version appeared in \Proceedings 10th IEEE Structure in Complexity Theory, 1995", pp. 12-28. Work done in part while at the University of Rochester.
1
WEIGHTED NP OPTIMIZATION PROBLEMS
2
when the relation symbols in are interpreted as the relations of I and S . MAX SNP and MAX NP contain many natural problems, including MAX 3SAT, MAX CUT and MAX SAT, and have the nice property that all the problems in these classes are approximable by polynomial-time algorithms with constant approximation ratio. Moreover, Khanna, Motwani, Sudan and Vazirani [KMSV94] have shown that the class of NP optimization problems admitting polynomial-time constant ratio approximating algorithms coincides with the class of problems that are reducible to MAX SNP via a certain type of reduction that preserves approximation properties. It is proved in [ALM+ 92], via a reduction from a PCP, that no complete problem in MAX SNP can be approximated with arbitrarily small constant approximation ratio (i.e., have PTAS ), unless NP = P. Many important minimization problems, like VERTEX COVER, DOMINATING SET, STEINER TREE, and TRAVELING SALESMAN with edge weights 1 and 2, SHORTEST COMMON SUPERSTRING, are MAX SNP hard via reductions that preserve approximation properties (see Johnson [Joh92] for references and an excellent survey) and, thus, have the same approximation properties. Hence, the approximation capabilities of a large number of natural important optimization problems have been fully understood by the joint contribution of the two approaches (a) and (b). However, MAX SNP and MAX NP are far from being the whole story. Panconesi and Ranjan [PR93] showed that MAX CLIQUE is not in MAX NP. This problem belongs to the class MAX 1 in which the feasible solutions are characterized by 1 formulae. By imposing a syntactical restriction on , Panconesi and Ranjan identi ed, inside MAX 1 , the class RMAX(2) containing MAX CLIQUE and having the property that all problems in the class are self-improvable, i.e., the existence of approximation polynomial-time algorithms with a constant ratio of approximation implies the existence of PTAS. It has been shown, again by a reduction from a certain type of PCP, that there is no constant ratio polynomial approximation algorithm for MAX CLIQUE, unless NP = P. Therefore, no complete problem in RMAX(2) has PTAS, unless NP = P. In a later development, Kolaitis and Thakur undertook in [KT94] a comprehensive investigation of the logical characterizations of NP optimization problems. Considering arbitrary number of alternations of blocks of quanti ers, they introduced the classes MAX n , MAX n, MIN n , MIN n , n 0. They showed that all NP optimization problems in which the value of the optimum is polynomially bounded in the length of the input are in MAX 2 , in the case of maximization problems, and in MIN 1 , in the case of minimization problems. Moreover, they proved that all polynomially bounded NP maximization and minimization problems can be placed in one of the levels of the following proper hierarchies: MAX 0 $ MAX 1 $ MAX 1 = MAX 2 $ MAX 2 , for maximization problems, and MIN 0 = MIN 1 $ MIN 1 = MIN 2 = MIN 2 , for minimization problems. Kolaitis and Thakur have also introduced and investigated in [KT94] and [KT95] the classes MAX Fn , MAX Fn , MIN Fn , MIN Fn , n 1, which are closely related to the above classes and o er a more natural logical description for some NP polynomially bounded optimization problems. This time the formula characterizing the set of feasible solutions is closed (i.e., it has no free variables) and the objective function is de ned to be the number of tuples satisfying a speci ed relation from the structure S over which we maximize or minimize. They identi ed two syntactically de ned
WEIGHTED NP OPTIMIZATION PROBLEMS
3
classes, MIN F+ 1 and MIN F+ 2 (1), containing many natural important problems and having good approximation properties. Thus, MIN F+ 1 contains VERTEX COVER and many minimization node and edge deletion problems. All the problems in this class have polynomial-time algorithms with constant approximation ratio. MIN F+ 2 (1) contains SET COVER, DOMINATING SET, HITTING SET and other natural problems. All the problems in this class have polynomial-time approximation algorithms with the approximation ratio bounded by log n, where n is the length of the input. A further step was taken by Behrendt, Compton and Gradel [BCG92] who considered more powerful logics obtained by adding the least xpoint operator to the rst-order syntax. It is observed that MAX FP 1 , which is just MAX 1 extended with the least xpoint operator, continues to be approximable within a constant factor. In [BCG92] and [GM93], limit laws are identi ed for many of the above classes. These laws, similar to the 0-1 laws for various logics (see the survey paper [Com88]) constitute useful necessary criteria for the expressibility of a problem in the restricted syntax which de ne the respective classes. In spite of the new techniques that have emerged in recent years, with one important exception, there has been no thorough investigation of arbitrary NP optimization problems, i.e., problems in which the optimum value is no longer polynomially bounded in the length of the input. Since these problems usually arise when numerical weights are added to the input data, we call them weighted NP optimization problems. There is no need to argue about the large percentage of problems of this type that appear in real applications. The exception paper alluded above is again the work of Papadimitriou and Yannakakis [PY91]. They have considered the variants of MAX SNP and MAX NP in which positive weights are attached to the tuples over the domain of the input structure that can be substituted for the free variables ~x and showed that these more general classes continue to have polynomialtime algorithms with constant ratio. In this work, we undertake a more comprehensive examination of weighted NP optimization problems. We consider the classes weight-MAX Fn , weight-MAX Fn , weightMIN Fn , weight-MIN Fn , n 1, which are just the weighted variants of the MAX Fn , MAX Fn , MIN Fn and MIN Fn classes from [KT94] and [KT95]. For example, a problem is in weight-MAX F2 if: (1) the set of feasible solutions for an input structure I = (DI ; RI ) is given by the nite structures S having the same domain as I and the relation S1 ; S2 ; : : : ; Sp and satisfying a formula of the form 8~x 9~y (~x; ~y; RI ; S1 ; : : : ; Sp) with closed and quanti er-free, (2) each m-tuple (x1 ; : : : ; xm ) over the domain of I has a realvalued weight, where m is the arity of S1 , and (3) the objective function is to maximize over all structures S as above the weight of tuples in S1 . If the weights are positive, then the problem is in weight(+)-MAX F2 . We notice that all NP optimization problems, not just the polynomially bounded ones, admit a logical characterization. More precisely, all NP maximization (minimization) problems are in weight-MAX F2 (weight-MIN F2 ). It is easy to note that all problems in weight-MAX F1 and weight(+)-MAX F1 are solvable in polynomial time, and weight-MAX F2 and weight(+)-MAX F2 can be reduced in a way that preserves approximation properties to weight-MAX F1 and, respectively to
WEIGHTED NP OPTIMIZATION PROBLEMS weight(+)-MAX F Σ 1
4
weight(+)-MAX F Π1
weight(+)-MAX F Π2
weight(+)-MAX F Σ 2
weight(+)- MAX NP
weight(+)-MIN F Σ 1
weight(+)- MIN F Π2 weight(+)-MIN F Σ 2
weight(+)-MIN F Π1
weight(+)- MIN NP
weight-MAX F Σ 1
weight-MAX F Π2 weight-MAX F Σ 2
weight-MAX F Π1
weight- MAX NP weight-MIN F Π2
weight - MIN F Σ 1 weight-MIN F Σ 2 weight-MIN F Π1
weight- MIN NP
Figure 1: The relations between the syntactically de ned NP optimization classes. ( denotes incomparability.) weight(+)-MAX F1 . Similar properties hold for the analogue minimization classes and, consequently, these classes are less interesting from the point of view of approximation properties. From the syntactical point of view, the weight(+) classes satisfy the diagram in Figure 1, which is identical with the one satis ed by their non-weighted analogues. For the classes with arbitrary weights we only know the trivial relations represented in Figure 1. Making these relations more precise is left open. For all classes C earlier identi ed to have polynomial time approximation algorithms with guaranteed low approximation ratio, we analyze the weight-C and weight(+)-C variants. We consider C 2 fMAX SNP; MAX NP; MAX SNP(); MAX F1 ; MAX F+ 2 (1)g, (MAX SNP() is a subclass of MAX SNP in which the structure S over which the maximum is searched is required to be a permutation of the domain of the input structure). In all cases, weight(+)-C continues to have the same approximation properties as C and weight-C fails to do likewise unless very unlikely hypotheses hold. Table 1 summarizes the approximation properties of the classes analyzed in this paper. The notation is standard and most of it is explicitly introduced in the text. We note here only that is the set of nite binary strings and =n is the set of binary strings of length n; if x 2 , then jxj is the length of string x; if S is a set, then jS j is the cardinality of S ; and, nally, if y is a real number, then jyj is the modulus of y. P~ denotes DTIME[2logO n ] and log n is the integer part of log2 n. (1)
2 Logical De nability The elements which de ne an optimization problem A are:
WEIGHTED NP OPTIMIZATION PROBLEMS
5
Syntax MAX SNP
Positive weights Arbitrary weights const. approximable not approx. with ratio < n1=4 unless P = NP MAX NP const. approximable not approx. with ratio < n1=4 unless P = NP MAX SNP() const. approximable not approx. with ratio 2log n , for some > 0, unless NP DTIME[2logOq n ] MIN F+ 1 const. approximable not approx. with ratio 2n , any q, unless P = NP MIN F+ 2 (1) log. approximable not approx. with ratio 2nq , any q, unless P = NP (1)
Table 1: Approximation properties of the classes with good approximation properties in the nonweighted case (1) a set IA of input instances; we assume that this set can be recognized in polynomial time, (2) for each I 2 IA, a set FA (I ) of feasible solutions associated to each input instance; we assume that each element in FA (I ) has size polynomially bounded in the size of I , and (3) an objective function fA which maps to real numbers each pair (I; J ) with I 2 IA and J 2 FA (I ); we assume that this function is computable in polynomial time. There is also a default value for the cases when the set of feasible solutions is empty. If the objective function takes only nonnegative values then A is called a positive optimization problem. Given an instance I 2 IA, the goal is to nd optJ 2FA (I ) fA(I; J ) or output the default value in case FA (I ) is empty, where opt is max or min depending on what kind of an optimization we have. It is convenient to denote optJ 2FA (I ) fA(I; J ) by optA (I ). A max (min) optimization problem A is an NP optimization problem if the following associated decision problem B is in NP: Instance: An input instance I 2 IA and k 2 Z. Question: Does there exists a feasible solution J 2 FA (I ) such that fA(I; J ) k (fA (I; J ) k, in the case of a min problem)?
Kolaitis and Thakur [KT94] have shown that each NP optimization problem in which the optimum is polynomially bounded in the size of an encoding of the input instance can be syntactically described as an optimization problem in which the goal is to nd the max (or min) cardinality of a relation which together with some other relations satis es a given rst-order formula. Thus the objective function is the cardinality of tuples satisfying a
WEIGHTED NP OPTIMIZATION PROBLEMS
6
relation of some speci ed arity and the set of feasible solutions is given by the structures satisfying a rst-order formula. In order to make the above statements more precise we need some standard de nitions from descriptive computational complexity that we introduce in a simpli ed manner which mixes together syntactical and semantical notions. For a rigorous treatment see [Fag74], [Gra84], [Imm89], [Lyn82]. De nition 2.1 A nite type is a nite sequence of nonnegative integers. Given a nite type T = (n1 ; n2 ; : : : ; nk ), a nite T -structure is a (k + 1)-tuple F = (X; f1 ; f2 ; : : : ; fk ), where X is a non-empty nite set called the domain of the structure F and, for all i, fi is a relation over X of arity ni. Input instances of problems can naturally be viewed as nite structures of some nite type. For example, graphs are nite structures of the form (V; E ), where V is the domain (the set of nodes) and E is a relation of arity 2 (the set of edges). Boolean formulae in CNF are nite structures of the form (fx1 ; x2 ; : : : ; xn ; c1 ; : : : ; cm g; P; N ), where fx1 ; x2 ; : : : ; xn ; c1 ; : : : ; cm g is the set of variables and clauses and P and N are relations of arity 2 such that P (xi ; cj )(N (xi ; cj )) has the signi cance that variable xi appears positively (negatively) in clause cj . The following well-known theorem of Fagin [Fag74] characterizes the class of NP decision problems in logical terms. Theorem 2.2 [Fag74] Let T be a nite type. A set L of nite T -structures is in NP if and only if there exists a nite type S and a quanti er-free rst-order closed formula such that for all input structures I : I 2 L $ 9S 8~x 9~y (~x; ~y; I; S ); where S is a nite structure of type S having the same domain as I and ~x and ~y are tuples of variables ranging over I 's domain. The reader should be aware that the notation in the above statement of Theorem 2.2 is highly abusive. While the \I" in the left-hand side denotes a whole nite structure, the \I" and both \S"'s in the right-hand side are a shorthand notation for the relations of the corresponding structures denoted by the same symbols. We have implicitly assumed (by notation) that is compatible with I and S ; i.e., for each relation symbol occurring in there is a correspondent relation in I or S and there is agreement on arities. Also, in 8~x and 9~y, the quanti er is applied to all the components in the tuple ~x and, respectively, ~y. These conventions will be used throughout the rest of the paper. As an example, if I is a nite structure describing a CNF boolean formula as above, then I 2 SAT $ 9T 8c 9x [(P (x; c) ^ T (x)) _ (N (x; c) ^ :T (x))]: As mentioned above, Kolaitis and Thakur [KT94] have shown a similar property for polynomially bounded NP optimization problems. Theorem 2.3 [KT94] Let A be a polynomially bounded NP optimization problem. There exists a nite type S and a closed rst-order formula such that for each input structure I optA(I ) = optS fjS1 j : (~x; I; S )g;
WEIGHTED NP OPTIMIZATION PROBLEMS
7
where S is a nite structure of type S with the same domain as I and relations S1 ; S2 ; : : : ; Sk , ~x is a tuple of variables ranging over I 's domain and opt is max or min. Moreover, formula has the form 8~x 9~y (~x; ~y; I; S ) with quanti er-free. Lautemann has observed in [Lau92] that this result can be improved to state that all the values in the range of the objective function, not just the optimal one, can be obtained through logical descriptions. Related to the above example, consider the NP optimization problem MAX SAT. Input structures I are boolean formulae in CNF and the goal is to nd the maximum number of clauses that can be simultaneously satis ed by some truth assignment. This problem can be expressed as
max Max Sat (I ) = maxC;T fjC j : 8c 9x[C (c) ! [(P (x; c) ^ T (x)) _ (N (x; c) ^ :T (x))]]g: We generalize the above result to all NP optimization problems. The price for removing the polynomial bounding restriction is the introduction of weights for all n1 -tuples over the I 's domain, where n1 is the arity of S1. Note that the domain of I , being nite, can be identi ed with a set of the form f1; 2; : : : ; ng. De nition 2.4 (1) Let k 2 N. A k-weight assignment is a sequence of recursive functions fwi gi2N , where each wi : f1; 2; : : : ; igk ! R. For each k-tuplet ~x and all i and j , if wi (~x) and wj (~x) are de ned, then wi(~x) = wj (~x). (2) A k-positive weight assignment is a sequence of recursive functions fwi gi2N , where each wi : f1; 2; : : : ; igk ! R+ . For each k-tuplet ~x and all i and j , if wi (~x) and wj (~x) are de ned, then wi (~x) = wj (~x). (3) If S is a relation of arity k over f1; 2; : : : ; ig and w is a k-weight assignment, the P weight of S is w(S ) = ~x2S wi (~x). Theorem 2.5 Let A be a (positive) NP optimization problem. There exists a nite type S = (n1; n2 ; : : : ; nk ), a closed rst-order formula , and an n1-weight (positive) assignment w such that for each input instance I whose set of feasible solutions is not empty,
optA(I ) = optS fw(S1 ) : (~x; I; S )g;
(1)
where S is a nite structure of type S with the same domain as I and relations S1 ; S2 ; : : : ; Sk , ~x is a tuple of variables ranging over I 's domain and opt is max or min. Moreover, formula has the form 8~x 9~y (~x; ~y; I; S ) with quanti er-free.
Proof : Let A be an NP optimization problem. For simplicity we assume that the objective function fA is integer-valued. Let I be an input structure with domain f1; 2; : : : ; ng.
The structure I is encoded by a string whose length is bounded by a polynomial in n. Since the objective function fA is polynomial time computable, there exists a constant d such that joptA I j 2nd ? 1, for all I , where n, as explained, is the size of I . We de ne inductively the following (d + 1)-weight assignment w. Initially we order in the lexicographical order the (d + 1)-tuples over f1; 2g and we assign to them, in this order, the
WEIGHTED NP OPTIMIZATION PROBLEMS
8
weights ?22d ?1 ; : : : ; ?d20 ; 20 ; 21 ; : : : ; 22d ?1 ; 20 ; : : :d ; 20 . At the end of stage n, we have assigned the values ?2n ?1 ; : : : ; ?20 ; 20 ; 21 ; : : : ; 2n ?1 to all (d + 1)-tuples over f1; : : : ; ng. At stage n +1, we order lexicographically all (d +1)-tuples over f1; : : : ; n +1g that contain n +1 and assign to them the values ?2(n+1)d ?1 ; : : : ; ?2nd ; 2nd ; : : : ; 2(n+1)d ?1 ; 20 ; : : : ; 20 . There are (n + 1)d+1 ? nd+1 such tuples and 2((n + 1)d ? nd ) values of the form 2k (k 6= 0) to assign and thus, such an assignment is dpossible. The fact that we need is that for each integer d n m in the interval [?(2 ? 1); 2n ? 1] there exists a set of (d + 1)-tuples over f1; : : : ; ng whose w weights sums to m. Let us suppose that A is a maximization problem (the case of a minimization problem is similar). We consider the following decision problem B . Instance: A nite input structure I 2 IA with domain f1; 2; : : : ; ng, a relation U over f1; 2; : : : ; ng of arity (d + 1), the (d + 1)-weight assignment w de ned above. Question: Is there a feasible solution J 2 FA (I ) such that fA (I; J ) w(U ) ?
This is the decision problem associated to the NP optimization problem A and therefore is in NP. Consequently, by Fagin's Theorem 2.2, (I; U ) is a YES instance to B if and only if there exists a quanti er-free rst-order formula and a nite structure R such that 8~x 9~y (~x; ~y; I; U; R), where ~x; ~y and R satisfy the conditions in Theorem 2.2. Now it is easy to see that if is the formula 8~x 9~y (~x; ~y; I; U; R), then
optA(I ) = maxU;R fw(U ) : (~x; ~y; I; U; R)g: Indeed, let m = optA (I ). By a previous observation, there exists a relation U of arity (d +1) such that w(U ) = m . It follows that (I; U ) is a YES instance to the decision problem B and consequently maxU;R fw(U ) : (~x; ~y; I; U; R)g m . Conversely, let U; R be relations such that (~x; ~y; I; U; R) holds and w(U ) is maximum. Then (I; U ) is a YES instance to problem B . Therefore there exists a feasible solution J 2 FA (I ) such that fA (J ) w(U ) and, consequently, m maxU;R fw(U ) : (~x; ~y ; I; U; R)g. The proof for an arbitrary objective function (i.e., not necessarily integer-valued) is similar but more tedious. The key observation is that in polynomial time fA can compute only rational values with a number of digits that is polynomial in n and all these possible values can be covered by weights assigned to tuples of constant arity. The proof for positive NP optimization problems is absolutely similar. 2 It should be remarked that, except for the arity, the weight assignment built in the proof of the above theorem is independent of the problem and thus, we have proved a stronger fact. More precisely, there exists a canonical family of weight assignments fw1 ; : : : ; wk ; : : :g, where, for each k, wk is a k-assignment, such that for any NP optimization problem the weight assignment w in the expression (1) belongs to the family. Thus, the situation is completely similar to the polynomially bounded case from Theorem 2.3 (where the canonical weight assignment assigns value 1 to each tuplet). We can now classify all NP optimization problems with respect to the quanti er structure of the formulae describing them.
WEIGHTED NP OPTIMIZATION PROBLEMS
9
De nition 2.6 For n 1, a n(n) formula is a rst-order closed formula in prenex normal form that has n alternations of quanti ers starting with a block of existential (universal) quanti ers. A 0 or 0 formula is a rst-order quanti er-free closed formula.
De nition 2.7 (1) For each n 1, weight-MAX Fn is the set of optimization problems that can be expressed as
8 > < maxS fw(S1 ) : (~x; I; S )g if there is S such that (~x; I; S ); maxA (I ) = > : default; otherwise,
where S; S1 ; w are as in Theorem 2.5, default is a real constant, and is a n formula compatible with I and S . (2) For each n 1, weight(+)-MAX Fn is the subclass of weight-MAX Fn in which w is a positive weight assignment. (3) The classes weight-MAX Fn , weight(+)-MAX Fn , weight-MIN Fn , weight(+)MIN Fn , weight-MIN Fn , weight(+)-MIN Fn are de ned in the similar obvious way.
It follows from Theorem 2.5 that all NP maximization (minimization) problems (we denote these two classes by weight-MAX NP and weight-MIN NP) are in weight-MAX F2 (weight-MIN F2 ). Also, all positive NP maximization (minimization) problems (denoted weight(+)-MAX NP and weight(+)-MIN NP) are in weight(+)-MAX F2 (weight(+)MIN F2 ). From an algorithmical point of view, we observe rst (by considering all polynomially many possible substitutions for the existentially quanti ed variables) that all problems in C F1 , where C 2 fweight(+)-MAX; weight(+)-MIN; weight-MAX; weight-MINg can be solved in polynomial time and the problems in C F2 can be reduced to problems in C F1 with C as before. From the syntactical characterization point of view, we can prove the following relations. Theorem 2.8 (1) weight(+)-MAX F1 $ weight(+)-MAX F1 = weight(+)MAX F2 $ weight(+)-MAX F2 = weight(+)-MAX NP. (2) weight(+)-MIN F1 (weight(+)-MIN F1 ) $ weight(+)-MIN F2 $ weight(+)-MIN F2 = weight(+)-MIN NP, and weight(+)-MIN F1 and weight(+)-MIN F1 are incomparable. (3) weight-MAX F1 (weight-MAX F1 ) weight-MAX F2 weight-MAX F2 = weight-MAX NP, and weight-MAX F1 and weight-MAX F1 are not equal. (4) weight-MIN F1 (weight-MIN F1 ) weight-MIN F2 weight-MIN F2 = weightMIN NP, and weight-MIN F1 and weight-MIN F1 are not equal. Proof : Most of the proof follows closely the arguments in the similar proofs in [KT95]. For convenience, we provide the appropriate pointers.
WEIGHTED NP OPTIMIZATION PROBLEMS
10
By Theorem 2.5, C F2 = C NP, where C 2 fweight(+)-MAX, weight(+)-MIN, weightMAX, weight-MINg and thus, just from the syntactical characterization, (C F1 ; C F1 ) C F2 C F2 . As in [KT95, Theorem 6.2], MIN VERTEX COVER is in weight(+)-MIN F1 and in weight-MIN F1 but not in weight(+)-MAX F1 and not in weight-MIN F1 . By the arguments in [PR93], it can be shown that MAX CLIQUE is in weight(+)-MAX F1 and in weight-MAX F1 but not in weight(+)-MAX F1 and not in weight-MAX F1 . As in [KT95, Remark 2.1 and Theorem 2.1], weight(+)-MAX F1 = weight(+)-MAX 1 and weight(+)-MAX Fi = weight(+)-MAX i , i = 1; 2 (where weight(+)-MAX i and weight(+)-MAX i are the analogues of MAX i and MAX i but with positive weights on tuples). As in [KT94, Theorem 2] weight(+)-MAX 2 = weight(+)-MAX 1 . Thus, weight(+)-MAX F2 = weight(+)-MAX 2 = weight(+)-MAX 1 = weight(+)-MAX F1 . As in [KT95, Proposition 2.1], weight(+)-MIN F1 = weight(+)-MIN 1 and there exists a problem in weight(+)-MIN F1 which is not in weight(+)-MIN 1 and, therefore, is not in weight(+)-MIN F1 . We have shown that weight(+)-MIN F1 and weight(+)-MIN F1 are incomparable and, thus, both classes are properly included in weight(+)-MIN F2 . As in [KT95, Theorem 6.2], MIN CYCLE (given a graph G, return the size of the shortest cycle in G) is in weight(+)-MIN F2 but not in weight(+)-MIN F2 . The only relation left unproven is weight(+)-MAX F2 $ weight(+)-MAX F2 . Consider the following problem P : given a graph G = (V; E ), nd the length of the longest cycle in G or output 0 if no cycle exists. By Theorem 2.5, P is in weight(+)-MAX F2 . Since weight(+)-MAX F2 = weight(+)-MAX F1 , it is enough to show that P is not in weight(+)-MAX F1 . So, suppose that maxP (G) = maxS ;:::;Sq fw(S1 ) : 8~x (~x; G; S1 ; : : : ; Sq )g: Let the arity of S1 be k and consider the graph G consisting of a cycle a1 ; a2 ; : : : ; an , i.e., (ai ; ai+1 ) 2 E and (an ; a1 ) 2 E and these are the only tuples in E . Take n > k. Let S = (S1 ; : : : ; Sq) be a relation that is optimal for G with respect to the above formula. Since w(S1 ) = n, there must be a k-tuple (ai ; : : : ; aik ) such that w(ai ; : : : ; aik ) > 0. Let H be the subgraph of G obtained by restricting G to the vertices ai ; : : : ; aik . Since 1 formu ) holds true, where S = las are closed under taking substructures, 8~x (~x; H; S1;H ; : : : ; Sq;H H ) is the restriction of S to ai ; : : : ; ai . Therefore, maxP (H ) w(S ) > 0. (S1;H ; : : : ; Sq;H q k But H is not a cycle, and thus, maxP (H ) = 0. 2 1
1
1
1
1
3 Approximation Properties As explained in the Introduction, the logical de nability of optimization problems has in many important cases a direct impact on their approximation properties. The syntactical form of formulae describing the constraint conditions of a problem can imply the existence of polynomial time algorithms achieving a speci ed ratio of approximation. Or, on the other
WEIGHTED NP OPTIMIZATION PROBLEMS
11
hand, if a problem is hard under L- or A- reductions (these are some restricted variants of reductions that preserve approximation properties; they were introduced by Papadimitriou and Yannakakis in [PY91] and Panconesi and Ranjan in [PR93]), then the existence of good approximations algorithms is highly improbable. We investigate to what extent the known properties of polynomially bounded optimization problems remain valid when we pass to the positively weighted and arbitrarily weighted corresponding problems. De nition 3.1 (1) Let A be an optimization problem. An approximation algorithm B for A is a function that maps input instances I 2 IA to feasible solutions in FA (I ). As a technical convenience, we require that fA(I; B (I )) and opt A(I ) have strictly positive value for all I . The approximation algorithm B has approximation ratio rB : N ! [1; +1) if for all input instances I : 8 fA(I;B(I )) if A is a minimization problem, > < optAI rB (jI j) > : optAI if A is a maximization problem. fA (I;B(I ))
(2) An optimization problem A is constant (log)-approximable if there exists a polynomial time approximation algorithm B for A such that rB (jI j) = O(1) (rB (jI j) = O(log(jI j))) for all input structures I of A. (3) An optimization problem A is superpolylog approximable if there exists a DTIME[2logO n ] approximation algorithm B for A and a constant > 0 such that rB (jI j) = 2log jI j for all input structures I of A. For the sake of simplicity, the above de nition assumes that the approximation algorithm B outputs the result of the objective function applied to a feasible solution. However, all the following results hold in the more liberal setting in which B is allowed to compute just an approximation (from below or above) of the optimum. (1)
De nition 3.2 The optimization problem L-reduces to the optimization problem 0 if
there are two polynomial-time algorithms f and g and constants ; > 0 such that (1) given any instance I of , f produces an instance I 0 of 0 such that opt0 I 0 opt I , (2) given a feasible solution of I 0 of cost c0 , g produces a feasible solution of I of cost c such that jc ? optI j jc0 ? opt0 I 0 j. We next take into review all syntactically de ned classes of NP optimization problems that have been identi ed in earlier works to have good approximation properties.
MAX SNP and MAX NP. A polynomially bounded maximization problem A is in MAX NP if maxA (I ) = maxS jf~x : 9~y (~x; ~y; I; S )gj, where is a rst-order quanti erfree formula. If maxA(I ) = maxS jf~x : (~x; I; S )gj, then A is in MAX SNP. It is known
from [PY91] that if A is in MAX NP or in MAX SNP, then A is constant approximable.
WEIGHTED NP OPTIMIZATION PROBLEMS
12
The weighted variants of these classes are de ned by considering m-weight assignments for the ~x tuples in the formulae above (m is the arity of ~x) and proceeding as in the de nition of the classes MAX Fi or MAX Fi . Papadimitriou and Yannakakis have shown in [PY91] that all problems in weight(+)-MAX NP and weight(+)-MAX SNP are constant approximable. This property does not extend to weight-MAX SNP (and to weight-MAX NP), unless NP P~ . In order to prove this we consider the following well-known problem: weight-MAX 2SAT: an instance of this problem is a CNF formula with two literals per clause and with real-valued weights assigned to each clause. The goal is to maximize the total weight of the clauses that can be simultaneously satis ed by a truth assignment.
The problem is in weight-MAX SNP by the following logical description. An input structure consists of a pair (V ar; C0 ; C1 ; C2 ; C3 ), where the domain V ar is the set of variables and Ci are relations of arity 4 such that x1 ; x2 ; x1 ; x2 2 C0 means that there exists a clause c = x1 _x2 , x1 ; x2 ; x2 ; x1 2 C1 means that there exists a clause c = x1 _x2 , x1 ; x2 ; x1 ; x1 2 C2 means that there exists a clause c = x1 _ x2 , and x1 ; x2 ; x2 ; x2 2 C3 means that there exists a clause c = x1 _ x2 (we assume that all clauses have distinct variables). Assigning weights to any 4-tuple of variables in the obvious way, we can write maxMAX 2SAT (C ) = maxT w(f(t; u; v; w) : (C0 (t; u; v; w) ! (:T (t) _ :T (u))) ^ (C1 (t; u; v; w) ! (T (t) _ :T (u))) ^ (C2 (t; u; v; w) ! (:T (t) _ T (u))) ^ (C3 (t; u; v; w) ! (T (t) _ T (u)))g). The following lemma, whose proof is deferred until the end of this section, holds. Lemma 3.3 If NP 6= P, then weight-MAX 2SAT is not approximable within ratio nc for any c < 1=4, and, if NP 6= coRP, then weight-MAX 2SAT is not approximable within ratio nc for any c < 1=3. Consequently we obtain the following theorem. Theorem 3.4 If NP 6= P, there are problems in weight-MAX SNP that are not approximable with approximation ratio nc for any c < 1=4. If NP 6= coRP, there are problems in weight-MAX SNP that are not approximable with approximation ratio nc for any c < 1=3. We do not know if weight-MAX 2SAT is complete for weight-MAX SNP under L- (or A-) reductions. Nevertheless, this problem could be a good starting point for a chain of Lreductions showing that other natural problems do not possess the superpolylog approximation property under the same hypothesis. For example, the reduction from [PY91] shows that weight-MAX NOT-ALL-EQUAL-2 SAT falls into this category. Papadimitriou and Yannakakis have considered in [PY91] a variant of MAX SNP, called MAX SNP(), in which the structure over which we maximize is required to be a total linear ordering (i.e., a permutation) of the domain of the input structure. This class contains natural problems like MAX SUBDAG: given a directed graph G = (V; E ), nd an acyclic subgraph G0 = (V; E 0 ) with E 0 as large as possible. The weighted version of this problem is obtained, of course, by assigning weights to edges. It was shown in [PY91] that MAX SUBDAG is L-complete for MAX SNP() and it is straightforward to extend this
WEIGHTED NP OPTIMIZATION PROBLEMS
13
result to weight(+)-MAX SUBDAG. MAX SUBDAG and weight(+)-MAX SUBDAG can be approximated in polynomial time with ratio 2: take any permutation of the vertices and its reverse and choose the one that yields an orientation whose weight is larger. It follows that all problems in MAX SNP() and weight(+)-MAX SNP() are constant approximable. Clearly, weight-MAX SUBDAG continues to be constant approximable, because we can just disregard the edges with negative weights. We consider another important natural problem (see [GM84, p. 465 .] ) in weight-MAX SNP(), which is closely related to MAX SUBDAG. weight-PRIORITY ORDERING: given a set X and real-valued weights w to all pairs of P distinct elements in X , nd the maximum over all permutations of f(x;y):(x)
Abstract.
Extending a well-known property of NP optimization problems in which the value of the optimum is guaranteed to be polynomially bounded in the length of the input, it is observed that, by attaching weights to tuples over the domain of the input, all NP optimization problems admit a logical characterization. It is shown that any NP optimization problem can be stated as a problem in which the constraint conditions can be expressed by a 2 rst-order formula. The paper analyzes the weighted analogue of all syntactically de ned classes of optimization problems that are known to have good approximation properties in the nonweighted case. Dramatic changes occur when negative weights are allowed. KeywordsNP optimization problems, logical de nability, approximation properties, multiprover interactive systems. AMS Classi cation 68Q15, 68Q25.
1 Introduction Recent years have seen considerable progress in the understanding of the approximation properties of NP optimization problems. Two approaches have been the main cause for the new advancements: (a) the development of a robust model for optimization problems based on the logical de nability of this type of problems, and (b) the characterization of NP in terms of multiprover interactive proof (MIP) or probabilistically checking proof (PCP) systems which allowed new reductions that preserve approximation properties between NP complete problems. The rst approach was initiated by Papadimitriou and Yannakakis [PY91] who introduced the classes MAX SNP and MAX NP (later called MAX 0 and MAX 1 by Kolaitis and Thakur [KT94] in an attempt to uniformize notation). A maximization problem is in MAX SNP (MAX NP) if: (1) the input I is viewed as a nite structure (2) the set of feasible solutions of an input I is given by the set of nite structures S having the same domain as I and satisfying a 0 (1 ) rst-order formula having a tuple of free variables, and (3) the objective function maps each feasible solution S to the cardinality of tuples over the domain of I satisfying when substituting the tuple of free variables and School of Computer & Applied Sciences, Georgia Southwestern State University, Americus, GA 31709, email: [email protected]. Supported in part by grant NSF-CCR-8957604, NSF-INT9116781/JSPS-ENG-207 and NSF-CCR-9322513 and by the Romanian Department of Education and Science grant 4975-92. A preliminary version appeared in \Proceedings 10th IEEE Structure in Complexity Theory, 1995", pp. 12-28. Work done in part while at the University of Rochester.
1
WEIGHTED NP OPTIMIZATION PROBLEMS
2
when the relation symbols in are interpreted as the relations of I and S . MAX SNP and MAX NP contain many natural problems, including MAX 3SAT, MAX CUT and MAX SAT, and have the nice property that all the problems in these classes are approximable by polynomial-time algorithms with constant approximation ratio. Moreover, Khanna, Motwani, Sudan and Vazirani [KMSV94] have shown that the class of NP optimization problems admitting polynomial-time constant ratio approximating algorithms coincides with the class of problems that are reducible to MAX SNP via a certain type of reduction that preserves approximation properties. It is proved in [ALM+ 92], via a reduction from a PCP, that no complete problem in MAX SNP can be approximated with arbitrarily small constant approximation ratio (i.e., have PTAS ), unless NP = P. Many important minimization problems, like VERTEX COVER, DOMINATING SET, STEINER TREE, and TRAVELING SALESMAN with edge weights 1 and 2, SHORTEST COMMON SUPERSTRING, are MAX SNP hard via reductions that preserve approximation properties (see Johnson [Joh92] for references and an excellent survey) and, thus, have the same approximation properties. Hence, the approximation capabilities of a large number of natural important optimization problems have been fully understood by the joint contribution of the two approaches (a) and (b). However, MAX SNP and MAX NP are far from being the whole story. Panconesi and Ranjan [PR93] showed that MAX CLIQUE is not in MAX NP. This problem belongs to the class MAX 1 in which the feasible solutions are characterized by 1 formulae. By imposing a syntactical restriction on , Panconesi and Ranjan identi ed, inside MAX 1 , the class RMAX(2) containing MAX CLIQUE and having the property that all problems in the class are self-improvable, i.e., the existence of approximation polynomial-time algorithms with a constant ratio of approximation implies the existence of PTAS. It has been shown, again by a reduction from a certain type of PCP, that there is no constant ratio polynomial approximation algorithm for MAX CLIQUE, unless NP = P. Therefore, no complete problem in RMAX(2) has PTAS, unless NP = P. In a later development, Kolaitis and Thakur undertook in [KT94] a comprehensive investigation of the logical characterizations of NP optimization problems. Considering arbitrary number of alternations of blocks of quanti ers, they introduced the classes MAX n , MAX n, MIN n , MIN n , n 0. They showed that all NP optimization problems in which the value of the optimum is polynomially bounded in the length of the input are in MAX 2 , in the case of maximization problems, and in MIN 1 , in the case of minimization problems. Moreover, they proved that all polynomially bounded NP maximization and minimization problems can be placed in one of the levels of the following proper hierarchies: MAX 0 $ MAX 1 $ MAX 1 = MAX 2 $ MAX 2 , for maximization problems, and MIN 0 = MIN 1 $ MIN 1 = MIN 2 = MIN 2 , for minimization problems. Kolaitis and Thakur have also introduced and investigated in [KT94] and [KT95] the classes MAX Fn , MAX Fn , MIN Fn , MIN Fn , n 1, which are closely related to the above classes and o er a more natural logical description for some NP polynomially bounded optimization problems. This time the formula characterizing the set of feasible solutions is closed (i.e., it has no free variables) and the objective function is de ned to be the number of tuples satisfying a speci ed relation from the structure S over which we maximize or minimize. They identi ed two syntactically de ned
WEIGHTED NP OPTIMIZATION PROBLEMS
3
classes, MIN F+ 1 and MIN F+ 2 (1), containing many natural important problems and having good approximation properties. Thus, MIN F+ 1 contains VERTEX COVER and many minimization node and edge deletion problems. All the problems in this class have polynomial-time algorithms with constant approximation ratio. MIN F+ 2 (1) contains SET COVER, DOMINATING SET, HITTING SET and other natural problems. All the problems in this class have polynomial-time approximation algorithms with the approximation ratio bounded by log n, where n is the length of the input. A further step was taken by Behrendt, Compton and Gradel [BCG92] who considered more powerful logics obtained by adding the least xpoint operator to the rst-order syntax. It is observed that MAX FP 1 , which is just MAX 1 extended with the least xpoint operator, continues to be approximable within a constant factor. In [BCG92] and [GM93], limit laws are identi ed for many of the above classes. These laws, similar to the 0-1 laws for various logics (see the survey paper [Com88]) constitute useful necessary criteria for the expressibility of a problem in the restricted syntax which de ne the respective classes. In spite of the new techniques that have emerged in recent years, with one important exception, there has been no thorough investigation of arbitrary NP optimization problems, i.e., problems in which the optimum value is no longer polynomially bounded in the length of the input. Since these problems usually arise when numerical weights are added to the input data, we call them weighted NP optimization problems. There is no need to argue about the large percentage of problems of this type that appear in real applications. The exception paper alluded above is again the work of Papadimitriou and Yannakakis [PY91]. They have considered the variants of MAX SNP and MAX NP in which positive weights are attached to the tuples over the domain of the input structure that can be substituted for the free variables ~x and showed that these more general classes continue to have polynomialtime algorithms with constant ratio. In this work, we undertake a more comprehensive examination of weighted NP optimization problems. We consider the classes weight-MAX Fn , weight-MAX Fn , weightMIN Fn , weight-MIN Fn , n 1, which are just the weighted variants of the MAX Fn , MAX Fn , MIN Fn and MIN Fn classes from [KT94] and [KT95]. For example, a problem is in weight-MAX F2 if: (1) the set of feasible solutions for an input structure I = (DI ; RI ) is given by the nite structures S having the same domain as I and the relation S1 ; S2 ; : : : ; Sp and satisfying a formula of the form 8~x 9~y (~x; ~y; RI ; S1 ; : : : ; Sp) with closed and quanti er-free, (2) each m-tuple (x1 ; : : : ; xm ) over the domain of I has a realvalued weight, where m is the arity of S1 , and (3) the objective function is to maximize over all structures S as above the weight of tuples in S1 . If the weights are positive, then the problem is in weight(+)-MAX F2 . We notice that all NP optimization problems, not just the polynomially bounded ones, admit a logical characterization. More precisely, all NP maximization (minimization) problems are in weight-MAX F2 (weight-MIN F2 ). It is easy to note that all problems in weight-MAX F1 and weight(+)-MAX F1 are solvable in polynomial time, and weight-MAX F2 and weight(+)-MAX F2 can be reduced in a way that preserves approximation properties to weight-MAX F1 and, respectively to
WEIGHTED NP OPTIMIZATION PROBLEMS weight(+)-MAX F Σ 1
4
weight(+)-MAX F Π1
weight(+)-MAX F Π2
weight(+)-MAX F Σ 2
weight(+)- MAX NP
weight(+)-MIN F Σ 1
weight(+)- MIN F Π2 weight(+)-MIN F Σ 2
weight(+)-MIN F Π1
weight(+)- MIN NP
weight-MAX F Σ 1
weight-MAX F Π2 weight-MAX F Σ 2
weight-MAX F Π1
weight- MAX NP weight-MIN F Π2
weight - MIN F Σ 1 weight-MIN F Σ 2 weight-MIN F Π1
weight- MIN NP
Figure 1: The relations between the syntactically de ned NP optimization classes. ( denotes incomparability.) weight(+)-MAX F1 . Similar properties hold for the analogue minimization classes and, consequently, these classes are less interesting from the point of view of approximation properties. From the syntactical point of view, the weight(+) classes satisfy the diagram in Figure 1, which is identical with the one satis ed by their non-weighted analogues. For the classes with arbitrary weights we only know the trivial relations represented in Figure 1. Making these relations more precise is left open. For all classes C earlier identi ed to have polynomial time approximation algorithms with guaranteed low approximation ratio, we analyze the weight-C and weight(+)-C variants. We consider C 2 fMAX SNP; MAX NP; MAX SNP(); MAX F1 ; MAX F+ 2 (1)g, (MAX SNP() is a subclass of MAX SNP in which the structure S over which the maximum is searched is required to be a permutation of the domain of the input structure). In all cases, weight(+)-C continues to have the same approximation properties as C and weight-C fails to do likewise unless very unlikely hypotheses hold. Table 1 summarizes the approximation properties of the classes analyzed in this paper. The notation is standard and most of it is explicitly introduced in the text. We note here only that is the set of nite binary strings and =n is the set of binary strings of length n; if x 2 , then jxj is the length of string x; if S is a set, then jS j is the cardinality of S ; and, nally, if y is a real number, then jyj is the modulus of y. P~ denotes DTIME[2logO n ] and log n is the integer part of log2 n. (1)
2 Logical De nability The elements which de ne an optimization problem A are:
WEIGHTED NP OPTIMIZATION PROBLEMS
5
Syntax MAX SNP
Positive weights Arbitrary weights const. approximable not approx. with ratio < n1=4 unless P = NP MAX NP const. approximable not approx. with ratio < n1=4 unless P = NP MAX SNP() const. approximable not approx. with ratio 2log n , for some > 0, unless NP DTIME[2logOq n ] MIN F+ 1 const. approximable not approx. with ratio 2n , any q, unless P = NP MIN F+ 2 (1) log. approximable not approx. with ratio 2nq , any q, unless P = NP (1)
Table 1: Approximation properties of the classes with good approximation properties in the nonweighted case (1) a set IA of input instances; we assume that this set can be recognized in polynomial time, (2) for each I 2 IA, a set FA (I ) of feasible solutions associated to each input instance; we assume that each element in FA (I ) has size polynomially bounded in the size of I , and (3) an objective function fA which maps to real numbers each pair (I; J ) with I 2 IA and J 2 FA (I ); we assume that this function is computable in polynomial time. There is also a default value for the cases when the set of feasible solutions is empty. If the objective function takes only nonnegative values then A is called a positive optimization problem. Given an instance I 2 IA, the goal is to nd optJ 2FA (I ) fA(I; J ) or output the default value in case FA (I ) is empty, where opt is max or min depending on what kind of an optimization we have. It is convenient to denote optJ 2FA (I ) fA(I; J ) by optA (I ). A max (min) optimization problem A is an NP optimization problem if the following associated decision problem B is in NP: Instance: An input instance I 2 IA and k 2 Z. Question: Does there exists a feasible solution J 2 FA (I ) such that fA(I; J ) k (fA (I; J ) k, in the case of a min problem)?
Kolaitis and Thakur [KT94] have shown that each NP optimization problem in which the optimum is polynomially bounded in the size of an encoding of the input instance can be syntactically described as an optimization problem in which the goal is to nd the max (or min) cardinality of a relation which together with some other relations satis es a given rst-order formula. Thus the objective function is the cardinality of tuples satisfying a
WEIGHTED NP OPTIMIZATION PROBLEMS
6
relation of some speci ed arity and the set of feasible solutions is given by the structures satisfying a rst-order formula. In order to make the above statements more precise we need some standard de nitions from descriptive computational complexity that we introduce in a simpli ed manner which mixes together syntactical and semantical notions. For a rigorous treatment see [Fag74], [Gra84], [Imm89], [Lyn82]. De nition 2.1 A nite type is a nite sequence of nonnegative integers. Given a nite type T = (n1 ; n2 ; : : : ; nk ), a nite T -structure is a (k + 1)-tuple F = (X; f1 ; f2 ; : : : ; fk ), where X is a non-empty nite set called the domain of the structure F and, for all i, fi is a relation over X of arity ni. Input instances of problems can naturally be viewed as nite structures of some nite type. For example, graphs are nite structures of the form (V; E ), where V is the domain (the set of nodes) and E is a relation of arity 2 (the set of edges). Boolean formulae in CNF are nite structures of the form (fx1 ; x2 ; : : : ; xn ; c1 ; : : : ; cm g; P; N ), where fx1 ; x2 ; : : : ; xn ; c1 ; : : : ; cm g is the set of variables and clauses and P and N are relations of arity 2 such that P (xi ; cj )(N (xi ; cj )) has the signi cance that variable xi appears positively (negatively) in clause cj . The following well-known theorem of Fagin [Fag74] characterizes the class of NP decision problems in logical terms. Theorem 2.2 [Fag74] Let T be a nite type. A set L of nite T -structures is in NP if and only if there exists a nite type S and a quanti er-free rst-order closed formula such that for all input structures I : I 2 L $ 9S 8~x 9~y (~x; ~y; I; S ); where S is a nite structure of type S having the same domain as I and ~x and ~y are tuples of variables ranging over I 's domain. The reader should be aware that the notation in the above statement of Theorem 2.2 is highly abusive. While the \I" in the left-hand side denotes a whole nite structure, the \I" and both \S"'s in the right-hand side are a shorthand notation for the relations of the corresponding structures denoted by the same symbols. We have implicitly assumed (by notation) that is compatible with I and S ; i.e., for each relation symbol occurring in there is a correspondent relation in I or S and there is agreement on arities. Also, in 8~x and 9~y, the quanti er is applied to all the components in the tuple ~x and, respectively, ~y. These conventions will be used throughout the rest of the paper. As an example, if I is a nite structure describing a CNF boolean formula as above, then I 2 SAT $ 9T 8c 9x [(P (x; c) ^ T (x)) _ (N (x; c) ^ :T (x))]: As mentioned above, Kolaitis and Thakur [KT94] have shown a similar property for polynomially bounded NP optimization problems. Theorem 2.3 [KT94] Let A be a polynomially bounded NP optimization problem. There exists a nite type S and a closed rst-order formula such that for each input structure I optA(I ) = optS fjS1 j : (~x; I; S )g;
WEIGHTED NP OPTIMIZATION PROBLEMS
7
where S is a nite structure of type S with the same domain as I and relations S1 ; S2 ; : : : ; Sk , ~x is a tuple of variables ranging over I 's domain and opt is max or min. Moreover, formula has the form 8~x 9~y (~x; ~y; I; S ) with quanti er-free. Lautemann has observed in [Lau92] that this result can be improved to state that all the values in the range of the objective function, not just the optimal one, can be obtained through logical descriptions. Related to the above example, consider the NP optimization problem MAX SAT. Input structures I are boolean formulae in CNF and the goal is to nd the maximum number of clauses that can be simultaneously satis ed by some truth assignment. This problem can be expressed as
max Max Sat (I ) = maxC;T fjC j : 8c 9x[C (c) ! [(P (x; c) ^ T (x)) _ (N (x; c) ^ :T (x))]]g: We generalize the above result to all NP optimization problems. The price for removing the polynomial bounding restriction is the introduction of weights for all n1 -tuples over the I 's domain, where n1 is the arity of S1. Note that the domain of I , being nite, can be identi ed with a set of the form f1; 2; : : : ; ng. De nition 2.4 (1) Let k 2 N. A k-weight assignment is a sequence of recursive functions fwi gi2N , where each wi : f1; 2; : : : ; igk ! R. For each k-tuplet ~x and all i and j , if wi (~x) and wj (~x) are de ned, then wi(~x) = wj (~x). (2) A k-positive weight assignment is a sequence of recursive functions fwi gi2N , where each wi : f1; 2; : : : ; igk ! R+ . For each k-tuplet ~x and all i and j , if wi (~x) and wj (~x) are de ned, then wi (~x) = wj (~x). (3) If S is a relation of arity k over f1; 2; : : : ; ig and w is a k-weight assignment, the P weight of S is w(S ) = ~x2S wi (~x). Theorem 2.5 Let A be a (positive) NP optimization problem. There exists a nite type S = (n1; n2 ; : : : ; nk ), a closed rst-order formula , and an n1-weight (positive) assignment w such that for each input instance I whose set of feasible solutions is not empty,
optA(I ) = optS fw(S1 ) : (~x; I; S )g;
(1)
where S is a nite structure of type S with the same domain as I and relations S1 ; S2 ; : : : ; Sk , ~x is a tuple of variables ranging over I 's domain and opt is max or min. Moreover, formula has the form 8~x 9~y (~x; ~y; I; S ) with quanti er-free.
Proof : Let A be an NP optimization problem. For simplicity we assume that the objective function fA is integer-valued. Let I be an input structure with domain f1; 2; : : : ; ng.
The structure I is encoded by a string whose length is bounded by a polynomial in n. Since the objective function fA is polynomial time computable, there exists a constant d such that joptA I j 2nd ? 1, for all I , where n, as explained, is the size of I . We de ne inductively the following (d + 1)-weight assignment w. Initially we order in the lexicographical order the (d + 1)-tuples over f1; 2g and we assign to them, in this order, the
WEIGHTED NP OPTIMIZATION PROBLEMS
8
weights ?22d ?1 ; : : : ; ?d20 ; 20 ; 21 ; : : : ; 22d ?1 ; 20 ; : : :d ; 20 . At the end of stage n, we have assigned the values ?2n ?1 ; : : : ; ?20 ; 20 ; 21 ; : : : ; 2n ?1 to all (d + 1)-tuples over f1; : : : ; ng. At stage n +1, we order lexicographically all (d +1)-tuples over f1; : : : ; n +1g that contain n +1 and assign to them the values ?2(n+1)d ?1 ; : : : ; ?2nd ; 2nd ; : : : ; 2(n+1)d ?1 ; 20 ; : : : ; 20 . There are (n + 1)d+1 ? nd+1 such tuples and 2((n + 1)d ? nd ) values of the form 2k (k 6= 0) to assign and thus, such an assignment is dpossible. The fact that we need is that for each integer d n m in the interval [?(2 ? 1); 2n ? 1] there exists a set of (d + 1)-tuples over f1; : : : ; ng whose w weights sums to m. Let us suppose that A is a maximization problem (the case of a minimization problem is similar). We consider the following decision problem B . Instance: A nite input structure I 2 IA with domain f1; 2; : : : ; ng, a relation U over f1; 2; : : : ; ng of arity (d + 1), the (d + 1)-weight assignment w de ned above. Question: Is there a feasible solution J 2 FA (I ) such that fA (I; J ) w(U ) ?
This is the decision problem associated to the NP optimization problem A and therefore is in NP. Consequently, by Fagin's Theorem 2.2, (I; U ) is a YES instance to B if and only if there exists a quanti er-free rst-order formula and a nite structure R such that 8~x 9~y (~x; ~y; I; U; R), where ~x; ~y and R satisfy the conditions in Theorem 2.2. Now it is easy to see that if is the formula 8~x 9~y (~x; ~y; I; U; R), then
optA(I ) = maxU;R fw(U ) : (~x; ~y; I; U; R)g: Indeed, let m = optA (I ). By a previous observation, there exists a relation U of arity (d +1) such that w(U ) = m . It follows that (I; U ) is a YES instance to the decision problem B and consequently maxU;R fw(U ) : (~x; ~y; I; U; R)g m . Conversely, let U; R be relations such that (~x; ~y; I; U; R) holds and w(U ) is maximum. Then (I; U ) is a YES instance to problem B . Therefore there exists a feasible solution J 2 FA (I ) such that fA (J ) w(U ) and, consequently, m maxU;R fw(U ) : (~x; ~y ; I; U; R)g. The proof for an arbitrary objective function (i.e., not necessarily integer-valued) is similar but more tedious. The key observation is that in polynomial time fA can compute only rational values with a number of digits that is polynomial in n and all these possible values can be covered by weights assigned to tuples of constant arity. The proof for positive NP optimization problems is absolutely similar. 2 It should be remarked that, except for the arity, the weight assignment built in the proof of the above theorem is independent of the problem and thus, we have proved a stronger fact. More precisely, there exists a canonical family of weight assignments fw1 ; : : : ; wk ; : : :g, where, for each k, wk is a k-assignment, such that for any NP optimization problem the weight assignment w in the expression (1) belongs to the family. Thus, the situation is completely similar to the polynomially bounded case from Theorem 2.3 (where the canonical weight assignment assigns value 1 to each tuplet). We can now classify all NP optimization problems with respect to the quanti er structure of the formulae describing them.
WEIGHTED NP OPTIMIZATION PROBLEMS
9
De nition 2.6 For n 1, a n(n) formula is a rst-order closed formula in prenex normal form that has n alternations of quanti ers starting with a block of existential (universal) quanti ers. A 0 or 0 formula is a rst-order quanti er-free closed formula.
De nition 2.7 (1) For each n 1, weight-MAX Fn is the set of optimization problems that can be expressed as
8 > < maxS fw(S1 ) : (~x; I; S )g if there is S such that (~x; I; S ); maxA (I ) = > : default; otherwise,
where S; S1 ; w are as in Theorem 2.5, default is a real constant, and is a n formula compatible with I and S . (2) For each n 1, weight(+)-MAX Fn is the subclass of weight-MAX Fn in which w is a positive weight assignment. (3) The classes weight-MAX Fn , weight(+)-MAX Fn , weight-MIN Fn , weight(+)MIN Fn , weight-MIN Fn , weight(+)-MIN Fn are de ned in the similar obvious way.
It follows from Theorem 2.5 that all NP maximization (minimization) problems (we denote these two classes by weight-MAX NP and weight-MIN NP) are in weight-MAX F2 (weight-MIN F2 ). Also, all positive NP maximization (minimization) problems (denoted weight(+)-MAX NP and weight(+)-MIN NP) are in weight(+)-MAX F2 (weight(+)MIN F2 ). From an algorithmical point of view, we observe rst (by considering all polynomially many possible substitutions for the existentially quanti ed variables) that all problems in C F1 , where C 2 fweight(+)-MAX; weight(+)-MIN; weight-MAX; weight-MINg can be solved in polynomial time and the problems in C F2 can be reduced to problems in C F1 with C as before. From the syntactical characterization point of view, we can prove the following relations. Theorem 2.8 (1) weight(+)-MAX F1 $ weight(+)-MAX F1 = weight(+)MAX F2 $ weight(+)-MAX F2 = weight(+)-MAX NP. (2) weight(+)-MIN F1 (weight(+)-MIN F1 ) $ weight(+)-MIN F2 $ weight(+)-MIN F2 = weight(+)-MIN NP, and weight(+)-MIN F1 and weight(+)-MIN F1 are incomparable. (3) weight-MAX F1 (weight-MAX F1 ) weight-MAX F2 weight-MAX F2 = weight-MAX NP, and weight-MAX F1 and weight-MAX F1 are not equal. (4) weight-MIN F1 (weight-MIN F1 ) weight-MIN F2 weight-MIN F2 = weightMIN NP, and weight-MIN F1 and weight-MIN F1 are not equal. Proof : Most of the proof follows closely the arguments in the similar proofs in [KT95]. For convenience, we provide the appropriate pointers.
WEIGHTED NP OPTIMIZATION PROBLEMS
10
By Theorem 2.5, C F2 = C NP, where C 2 fweight(+)-MAX, weight(+)-MIN, weightMAX, weight-MINg and thus, just from the syntactical characterization, (C F1 ; C F1 ) C F2 C F2 . As in [KT95, Theorem 6.2], MIN VERTEX COVER is in weight(+)-MIN F1 and in weight-MIN F1 but not in weight(+)-MAX F1 and not in weight-MIN F1 . By the arguments in [PR93], it can be shown that MAX CLIQUE is in weight(+)-MAX F1 and in weight-MAX F1 but not in weight(+)-MAX F1 and not in weight-MAX F1 . As in [KT95, Remark 2.1 and Theorem 2.1], weight(+)-MAX F1 = weight(+)-MAX 1 and weight(+)-MAX Fi = weight(+)-MAX i , i = 1; 2 (where weight(+)-MAX i and weight(+)-MAX i are the analogues of MAX i and MAX i but with positive weights on tuples). As in [KT94, Theorem 2] weight(+)-MAX 2 = weight(+)-MAX 1 . Thus, weight(+)-MAX F2 = weight(+)-MAX 2 = weight(+)-MAX 1 = weight(+)-MAX F1 . As in [KT95, Proposition 2.1], weight(+)-MIN F1 = weight(+)-MIN 1 and there exists a problem in weight(+)-MIN F1 which is not in weight(+)-MIN 1 and, therefore, is not in weight(+)-MIN F1 . We have shown that weight(+)-MIN F1 and weight(+)-MIN F1 are incomparable and, thus, both classes are properly included in weight(+)-MIN F2 . As in [KT95, Theorem 6.2], MIN CYCLE (given a graph G, return the size of the shortest cycle in G) is in weight(+)-MIN F2 but not in weight(+)-MIN F2 . The only relation left unproven is weight(+)-MAX F2 $ weight(+)-MAX F2 . Consider the following problem P : given a graph G = (V; E ), nd the length of the longest cycle in G or output 0 if no cycle exists. By Theorem 2.5, P is in weight(+)-MAX F2 . Since weight(+)-MAX F2 = weight(+)-MAX F1 , it is enough to show that P is not in weight(+)-MAX F1 . So, suppose that maxP (G) = maxS ;:::;Sq fw(S1 ) : 8~x (~x; G; S1 ; : : : ; Sq )g: Let the arity of S1 be k and consider the graph G consisting of a cycle a1 ; a2 ; : : : ; an , i.e., (ai ; ai+1 ) 2 E and (an ; a1 ) 2 E and these are the only tuples in E . Take n > k. Let S = (S1 ; : : : ; Sq) be a relation that is optimal for G with respect to the above formula. Since w(S1 ) = n, there must be a k-tuple (ai ; : : : ; aik ) such that w(ai ; : : : ; aik ) > 0. Let H be the subgraph of G obtained by restricting G to the vertices ai ; : : : ; aik . Since 1 formu ) holds true, where S = las are closed under taking substructures, 8~x (~x; H; S1;H ; : : : ; Sq;H H ) is the restriction of S to ai ; : : : ; ai . Therefore, maxP (H ) w(S ) > 0. (S1;H ; : : : ; Sq;H q k But H is not a cycle, and thus, maxP (H ) = 0. 2 1
1
1
1
1
3 Approximation Properties As explained in the Introduction, the logical de nability of optimization problems has in many important cases a direct impact on their approximation properties. The syntactical form of formulae describing the constraint conditions of a problem can imply the existence of polynomial time algorithms achieving a speci ed ratio of approximation. Or, on the other
WEIGHTED NP OPTIMIZATION PROBLEMS
11
hand, if a problem is hard under L- or A- reductions (these are some restricted variants of reductions that preserve approximation properties; they were introduced by Papadimitriou and Yannakakis in [PY91] and Panconesi and Ranjan in [PR93]), then the existence of good approximations algorithms is highly improbable. We investigate to what extent the known properties of polynomially bounded optimization problems remain valid when we pass to the positively weighted and arbitrarily weighted corresponding problems. De nition 3.1 (1) Let A be an optimization problem. An approximation algorithm B for A is a function that maps input instances I 2 IA to feasible solutions in FA (I ). As a technical convenience, we require that fA(I; B (I )) and opt A(I ) have strictly positive value for all I . The approximation algorithm B has approximation ratio rB : N ! [1; +1) if for all input instances I : 8 fA(I;B(I )) if A is a minimization problem, > < optAI rB (jI j) > : optAI if A is a maximization problem. fA (I;B(I ))
(2) An optimization problem A is constant (log)-approximable if there exists a polynomial time approximation algorithm B for A such that rB (jI j) = O(1) (rB (jI j) = O(log(jI j))) for all input structures I of A. (3) An optimization problem A is superpolylog approximable if there exists a DTIME[2logO n ] approximation algorithm B for A and a constant > 0 such that rB (jI j) = 2log jI j for all input structures I of A. For the sake of simplicity, the above de nition assumes that the approximation algorithm B outputs the result of the objective function applied to a feasible solution. However, all the following results hold in the more liberal setting in which B is allowed to compute just an approximation (from below or above) of the optimum. (1)
De nition 3.2 The optimization problem L-reduces to the optimization problem 0 if
there are two polynomial-time algorithms f and g and constants ; > 0 such that (1) given any instance I of , f produces an instance I 0 of 0 such that opt0 I 0 opt I , (2) given a feasible solution of I 0 of cost c0 , g produces a feasible solution of I of cost c such that jc ? optI j jc0 ? opt0 I 0 j. We next take into review all syntactically de ned classes of NP optimization problems that have been identi ed in earlier works to have good approximation properties.
MAX SNP and MAX NP. A polynomially bounded maximization problem A is in MAX NP if maxA (I ) = maxS jf~x : 9~y (~x; ~y; I; S )gj, where is a rst-order quanti erfree formula. If maxA(I ) = maxS jf~x : (~x; I; S )gj, then A is in MAX SNP. It is known
from [PY91] that if A is in MAX NP or in MAX SNP, then A is constant approximable.
WEIGHTED NP OPTIMIZATION PROBLEMS
12
The weighted variants of these classes are de ned by considering m-weight assignments for the ~x tuples in the formulae above (m is the arity of ~x) and proceeding as in the de nition of the classes MAX Fi or MAX Fi . Papadimitriou and Yannakakis have shown in [PY91] that all problems in weight(+)-MAX NP and weight(+)-MAX SNP are constant approximable. This property does not extend to weight-MAX SNP (and to weight-MAX NP), unless NP P~ . In order to prove this we consider the following well-known problem: weight-MAX 2SAT: an instance of this problem is a CNF formula with two literals per clause and with real-valued weights assigned to each clause. The goal is to maximize the total weight of the clauses that can be simultaneously satis ed by a truth assignment.
The problem is in weight-MAX SNP by the following logical description. An input structure consists of a pair (V ar; C0 ; C1 ; C2 ; C3 ), where the domain V ar is the set of variables and Ci are relations of arity 4 such that x1 ; x2 ; x1 ; x2 2 C0 means that there exists a clause c = x1 _x2 , x1 ; x2 ; x2 ; x1 2 C1 means that there exists a clause c = x1 _x2 , x1 ; x2 ; x1 ; x1 2 C2 means that there exists a clause c = x1 _ x2 , and x1 ; x2 ; x2 ; x2 2 C3 means that there exists a clause c = x1 _ x2 (we assume that all clauses have distinct variables). Assigning weights to any 4-tuple of variables in the obvious way, we can write maxMAX 2SAT (C ) = maxT w(f(t; u; v; w) : (C0 (t; u; v; w) ! (:T (t) _ :T (u))) ^ (C1 (t; u; v; w) ! (T (t) _ :T (u))) ^ (C2 (t; u; v; w) ! (:T (t) _ T (u))) ^ (C3 (t; u; v; w) ! (T (t) _ T (u)))g). The following lemma, whose proof is deferred until the end of this section, holds. Lemma 3.3 If NP 6= P, then weight-MAX 2SAT is not approximable within ratio nc for any c < 1=4, and, if NP 6= coRP, then weight-MAX 2SAT is not approximable within ratio nc for any c < 1=3. Consequently we obtain the following theorem. Theorem 3.4 If NP 6= P, there are problems in weight-MAX SNP that are not approximable with approximation ratio nc for any c < 1=4. If NP 6= coRP, there are problems in weight-MAX SNP that are not approximable with approximation ratio nc for any c < 1=3. We do not know if weight-MAX 2SAT is complete for weight-MAX SNP under L- (or A-) reductions. Nevertheless, this problem could be a good starting point for a chain of Lreductions showing that other natural problems do not possess the superpolylog approximation property under the same hypothesis. For example, the reduction from [PY91] shows that weight-MAX NOT-ALL-EQUAL-2 SAT falls into this category. Papadimitriou and Yannakakis have considered in [PY91] a variant of MAX SNP, called MAX SNP(), in which the structure over which we maximize is required to be a total linear ordering (i.e., a permutation) of the domain of the input structure. This class contains natural problems like MAX SUBDAG: given a directed graph G = (V; E ), nd an acyclic subgraph G0 = (V; E 0 ) with E 0 as large as possible. The weighted version of this problem is obtained, of course, by assigning weights to edges. It was shown in [PY91] that MAX SUBDAG is L-complete for MAX SNP() and it is straightforward to extend this
WEIGHTED NP OPTIMIZATION PROBLEMS
13
result to weight(+)-MAX SUBDAG. MAX SUBDAG and weight(+)-MAX SUBDAG can be approximated in polynomial time with ratio 2: take any permutation of the vertices and its reverse and choose the one that yields an orientation whose weight is larger. It follows that all problems in MAX SNP() and weight(+)-MAX SNP() are constant approximable. Clearly, weight-MAX SUBDAG continues to be constant approximable, because we can just disregard the edges with negative weights. We consider another important natural problem (see [GM84, p. 465 .] ) in weight-MAX SNP(), which is closely related to MAX SUBDAG. weight-PRIORITY ORDERING: given a set X and real-valued weights w to all pairs of P distinct elements in X , nd the maximum over all permutations of f(x;y):(x)
