Size-Depth Tradeo s for Algebraic Formulae - Semantic Scholar
Size-Depth Tradeo s for Algebraic Formulae Nader H. Bshouty
Richard Cleve
Wayne Eberly
Department of Computer Science University of Calgary Calgary, Alberta, Canada T2N 1N4
Abstract We prove some tradeo s between the size and depth of algebraic formulae. In particular, we show that, for any xed > 0, any algebraic formula of size S can be converted into an equivalent formula of depth O(log S ) and size O(S 1+ ). This result is an improvement over previously-known results where, to obtain the same depth bound, the formula-size is (S ), with 2.
1 Introduction A classical result, due to Brent (1974), implies that for any algebraic formula there is an algebraic circuit of \small" depth and \similar" size that computes the same function. More precisely, if the formula has size S then the circuit has depth O(log S ) and size O(S ). This result holds for formulae over any eld. We believe that a natural question to consider is whether for any algebraic formula there is an equivalent formula of small depth and similar size. Since any circuit of depth O(log S ) can be transformed into a formula of the same depth and with size polynomial in S , it follows immediately from Brent's result that there is also a formula of depth O(log S ) and size S O(1) that computes the same function as the original formula of size S . Applying this to the speci c circuits that result from Brent's construction yields formulae with size as large as (S ), with 2. Simple changes in Brent's construction may improve the exponent, but straightforward modi cations do not appear to result in exponents arbitrarily close to one. A widely-investigated problem that is related to Brent's result, as well as our work, is the \formula evaluation problem," where the goal is to construct a \universal formula evaluator" algorithm. Such an algorithm takes as input a description of a formula, with all of its inputs speci ed, and produces as output the value of the formula. Parallel algorithms for this problem have been proposed by Cook and Gupta (1985); Miller and Reif (1985); Buss (1987); Buss, Cook, Gupta, and Ramachandran (1989); and Kosaraju and Delcher (1990). These yield NC algorithms for the problem that also produce, for any given formula of size S , a circuit of depth O(log S ). When these circuits are expressed as formulae, the sizes are (S ), for various 2. In the case of division-free formulae, the exponents are smaller, but nevertheless bounded above one. As an example, in Section 5, we exhibit division-free formulae for which Miller and Reif's method produces formulae with such a polynomial size blowup. In this paper, we show that, over any eld F , for any xed > 0, for any formula of size S with operations from f+; ?; ; g [ F , there are equivalent formulae with: Depth O(log S ) and size O(S 1+ ). Depth O(log1+ S ) and size S 1+O( S ) . 1 log log
A preliminary version of this paper was presented at the 32nd Annual ACM Symposium on Foundations of Computer Science, San Juan, Puerto Rico (1991). This research was supported in part by NSERC of Canada.
1
Depth O(S ) and size O(S ) In the latter result, the method we use will add new variables to the formula when the eld size is less than S . Also, for Boolean formulae with operations from f^; _; :g, we obtain similar conditions as above. The techniques that we use include a multi-level extension of Brent's tree-decomposition method, as well as other restructuring methods. The organization of the remainder of this paper is as follows. Section 2 contains basic de nitions and notation. Section 3 contains the main result (expressed in Theorem 6, and interpreted in Corollary 7). Section 4 concerns additional results that apply for special classes of formulae, such as division-free, Boolean, and \simple" formulae. Section 5 describes some speci c formulae that appear to exhibit increases in size when their depth is reduced, and some known lower bounds on the size-depth tradeo due to Commentz-Walter (1979) and Commentz-Walter and Sattler (1980).
2 De nitions and Notation Algebraic Formulae: For a eld F , a formula over (F ; +; ?; ; ) of depth d is de ned as follows. A depth 0 formula is either c, for some c 2 F (a constant) or xu , for some u 2 f1; 2; : : :g (an input). For d > 0, a depth d formula is (F G), where 2 f+; ?; ; g, F and G are formulae of depth dF and dG respectively and d = max(dF ; dG ) + 1. The size of a formula F , denoted as jF j, is, informally, the number of occurrences of inputs and constants in the formula. More formally, a depth 0 formula has size one, and j(F G)j = jF j + jGj ( 2 f+; ?; ; g). A formula over (F ; +; ?; ; ) corresponds to a rational function in F (x1 ; :::; xn ) (for some n) in a natural way, provided that it does not involve a division by a formula equivalent to zero. For formulae F and G, F G denotes that they correspond to the same rational function. Hence, denotes equivalence in the function semantics sense. Division-Free Formulae: A division-free formula is one that has no divisions. Clearly, division-free formulae correspond to polynomials.
Simple Formulae: A simple formula is one that is division-free, and for which at least one argument of each multiplication operation is either an input or a constant. Thus, a depth 0 simple formula is either a constant or an input, and for depth d > 0 a depth d simple formula is (F G) where F and G are simple formulae, 2 f+; ?; g, and if = then either F or G has depth 0. Extended Formulae: In order to denote decompositions of a formula, we de ne an extended formula, which is allowed to take auxiliary inputs, which are input symbols that are not from fx1 ; x2 ; : : :g. For clarity, in extended
formulae, we write all auxiliary inputs as \arguments" to the formula. For example, the extended formula F (y) has auxiliary input symbol y. If G is a formula, then F (G) denotes the formula F (y) modi ed by substituting G for the symbol y. The size of an extended formula is de ned recursively as above, except that auxiliary inputs are not counted (that is, an auxiliary input has size 0). Also, we use special terminology to denote the number of occurrences of auxiliary variables in extended formulae. For A fy1 ; :::; ymg, jG(y1 ; :::; ym )jA denotes precisely the total number of occurrences of inputs from A in G(y1 ; :::; ym ). In particular, an extended formula G(y1 ; :::; ym) is read-once with respect to an auxiliary input yi if and only if jG(y1 ; :::; ym )jfyi g = 1. Note that, for formulae F1 ; :::; Fm ,
jG(F1 ; :::; Fm )j = jG(y1 ; :::; ym )j +
m X i=1
jFi j jG(y1 ; :::; ym)jfyi g :
Binary Strings: As usual, for k 0, f0; 1gk denotes all binary strings of length k. Furthermore, " denotes the empty string, and f0; 1gk denotes all binary strings of length less than or equal to k.
2
3 Main Result Our main result is Theorem 6 (Corollary 7 presents some consequences of this result). Brent's result is partially based on the following lemma, which concerns ways of partitioning trees into pieces of various sizes.
Lemma 1 (Brent, 1974): For any formula F and any m such that 1 < m jF j there exists an extended formula G(y) that is read-once with respect to y, formulae U and V , and an operation such that: F = G(U V ). jG(y)j jF j ? m and jU j; jV j < m. For a formula F , Brent applies Lemma 1, with m = d 21 jF je, thereby \decomposing" F into three pieces G(y), U , and V , each of size at most d 21 jF je. Then, using a recursive technique, he: translates G(y) into a circuit of size O(jG(y)j) and depth O(log jG(y)j) that computes A, B , C , and D such that y) + B ; G(y) ((CA y) + D translates U into an equivalent circuit U^ of size O(jU j) and depth O(log jU j), and similarly translates V into V^ . Finally, Brent expresses F as the required circuit by the identity ^ ^ F (A (U^ V^ )) + B : (C (U V )) + D Lemma 2 (below) is a multi-level version of the decomposition that Brent uses. Informally, it states that every formula F can be partitioned into 4k ? 3 or fewer pieces, each of size at most d k1 jF je.
Lemma 2: For any formula F , and any positive integer k such that 2 k < jF j, there exist nite sets of indices Interior; Border f0; 1g, extended formulae G (y) and operations for all 2 Interior, and formulae G for all 2 Border, such that these form a well de ned decomposition of F : (a) " 2 Interior; (b) Interior \ Border = ;; (c) For all 2 Interior, both 0 and 1 are in Interior [ Border; (d) For all 2 Border, neither 0 nor 1 is in Interior [ Border; (e) For all 2 Interior, jG (y)j b k1 jF jc and, for all 2 Border, jG j d k1 jF je; (f) If formula U is de ned recursively for all 2 Interior [ Border by the rule G (U 0 U 1) if 2 Interior U = G if 2 Border
then F = U" . In addition, these form a small decomposition of F : (g) Interior f0; 1gk?2 and jInteriorj 2k ? 2; (h) Border f0; 1gk?1 and jBorderj 2k ? 1.
3
Proof: Let F be an arbitrary formula and let k be an integer such that 2 k < jF j. We shall rst give a construction for the sets Interior and Border, the extended formulae G (y) and operations for 2 Interior, and formulae G for 2 Border, and demonstrate that these give a well de ned decomposition of F . We shall then argue that this decomposition is small. To begin, initialize Interior and Border to be empty, and set U" = F , so that jU" j = jF j = jF j ? j"j d k1 jF je. Since k 2, jU" j > d k1 jF je. To continue, let 2 f0; 1g such that U has been de ned with jU j jF j ? j j d k1 jF je, and such that has not been added (yet) to either Interior or Border | ending the construction if no such string exists. If jU j d k1 jF je then add to Border and set G to be U . Otherwise, add to Interior and apply Lemma 1 (Brent) to U with m = jU j ? b k1 jF jc, to de ne an extended formula G (y) that is read-once with respect to y, an operation , and formulae U 0, U 1 such that U = G (U 0 U 1); jG (y)j jU j ? m = b k1 jF jc; jU 0j; jU 1j < m. Note that it follows that jU 0j jF j ? j 0j d k1 jF je and jU 1 j jF j ? j 1j d k1 jF je. It is clear that properties (a) { (f) are established if this construction terminates. To see that it does, consider
2 f0; 1g such that U is de ned during the construction. Clearly, 0 < jU j jF j ? j j d k1 jF je, so j j < k. Therefore the construction does eventually halt, de ning sets Border f0; 1gk?1 and Interior f0; 1gk?2, and extended formulae, operations and formulae such that properties (a) { (f) hold. For 2 Interior de ne Interior Interior as Interior = Interior \ f 2 f0; 1g : is a pre x of g: Then it is easily established by structural induction that jInterior j 2 jU j k ? 2
jF j
for all 2 Interior: If 2 Interior and both of 0; 1 are in Border, then jInterior j = 1 2 jU j k ? 2;
jF j
since jU j > jFk j . If 2 Interior and exactly one (say, 0) of 0 and 1 is in Interior, then
jInterior j = jInterior 0j + 1 2 jUjF 0j j k ? 1 2 jjUF jj k ? 2;
since jU j ? jU 0 j jFk j . Finally, if all of , 0, and 1 are in Interior, then, since jU j jU 0 j + jU 1 j,
jInterior j = jInterior 0j +jInterior
1j + 1 2 jUjF 0j j k + 2 jUjF 1j j k ? 3 2 j U j k 2 j U j k
0
1 +1 ?3 jF j + jF j as desired. 2 jjUF jj k ? 2;
m l Therefore, since F = U" and Interior = Interior" , jInteriorj 2jjFF jj k ? 2 = 2k ? 2, which is sucient to establish property (g). Property (h) also follows because elements of the sets Interior and Border correspond respectively to the internal nodes and leaves of a binary tree, so jBorderj = jInteriorj + 1. 2
4
In order to control the formula size, we use a di erent approach than Brent for restructuring extended formulae of the form G(y). Lemma 3 (below) achieves the restructuring; however, it introduces new auxiliary inputs. In Lemmas 4 and 5, we show how to eliminate the auxiliary inputs (roughly, by substituting constants for them|or small polynomials, if the eld is too small|with special care to avoid introducing divisions by a zero formula).
Lemma 3: For any extended formula G(y) that is read-once with respect to y there exists an extended formula H (y; g1; g2 ; g3 ; z1 ; z2; z3 ) such that: G(y) H (y; G(z1 ); G(z2 ); G(z3 ); z1 ; z2 ; z3 ). H (y; g1; g2 ; g3 ; z1; z2 ; z3 ) is read-once with respect to y. jH (y; g1; g2 ; g3 ; z1; z2 ; z3 )j = 0. jH (y; g1; g2 ; g3 ; z1; z2 ; z3 )jfg ;g ;g g 44. jH (y; g1; g2 ; g3 ; z1; z2 ; z3 )jfz ;z ;z g 42. depth(H (y; g1 ; g2 ; g3; z1 ; z2 ; z3 )) 9. 1
2
3
1
2
3
Proof: Since G(y) is read-once with respect to y, there exist formulae P , Q, and R such that either G(y) (P y) + Q y+R
or
G(y) (P y) + Q:
The existence of P , Q, and R can be shown by considering the functions computed along the path in G(y) from y to its root: each such function is the quotient of two ane linear functions of y. Although this establishes the existence of P , Q, and R, this does not lead to an ecient way to construct these formulae: in the general case (with divisions), the resulting formula size may be exponential in jG(y)j. Instead, we use the method below. First, we consider the case where y) + Q : G(y) (P y+R
Substituting three distinct new auxiliary variables z1 , z2 , and z3 for y in this equation for G(y) results in the system of equations 2 3 2 3 2 3 z1 1 ?G(z1 ) P G(z1 ) z1 4 z2 1 ?G(z2 ) 5 4 Q 5 4 G(z2 ) z2 5 : z3 1 ?G(z3 ) R G(z3 ) z3 Since z1 , z2 , and z3 are new auxiliary variables and G(y) is not of the form (P y) + Q, z1 1 ?G(z1 ) z2 1 ?G(z2 ) 6 0; z3 1 ?G(z3 )
which implies that there are unique rational functions for P , Q, and R that satisfy the linear system. In particular, ( (
z3 G(z3 )) ? (z2 G(z2 )) G(z2 ) ? G(z3 ) z2 G(z 2 )) ? (z1 G(z1 )) G(z1 ) ? G(z2 ) : P z3 ? z2 G(z2 ) ? G(z3 ) z2 ? z1 G(z1 ) ? G(z2 )
5
By examining the expressions for the above determinants, we deduce that1 ^ ~ P P (G~ (~z );~z ) ; D(G(~z );~z ) where P^ (~g;~z) and D(~g;~z) are as follows. – x
x
–
–
x z1
g2
x g1
z2
– g3
–
x
g2
z2
g1
x g2
z3
g2
g3
Formula P^ (~g;~z) – x
x
– z1
– z2
–
g2
g3
–
z2
z3
g1
g2
Formula D(~g;~z) Clearly, depth(P^ (~g;~z )) = 4, depth(D(~g;~z )) = 3, jP^ (~g;~z)j = 0, jP^ (~g;~z)jf~gg = 8, jP^ (~g;~z)jf~zg = 4, jD(~g;~z)j = 0,
jD(~g;~z)jf~gg = 4, and jD(~g;~z)jf~zg = 4. Similarly,
^ ~ Q Q(G~ (~z );~z ) D(G(~z );~z ) and where Q^ (~g;~z) and R^ (~g;~z) are as shown below. 1
Here ( ( ) ) denotes ( ( 1 ) ( 2 ) ( 3 ) ~ ~ G z
;~ z
G z
;G z
;G z
^ ~ R R(G~ (~z );~z ) ; D(G(~z );~z )
; z 1 ; z2 ; z3
) and (
~ g; ~ z
6
) denotes (
g 1 ; g2 ; g3 ; z1 ; z2 ; z3
).
– x
x
– x z2
z1
–
z3
x g1
–
x
g2
g2
x
x g3
z3
g2
z2
–
z1
x g3
g1
g2
Formula Q^ (~g;~z) – x
x
–
–
z2
z3
–
x z1
z1
x g1
z2
– z2
x
g2
z2
x g2
z3
g3
Formula R^ (~g;~z) It is easily checked that depth(Q^ (~g;~z )) = depth(R^ (~g;~z )) = 4, jQ^ (~g;~z)j = 0, jQ^ (~g;~z)jf~gg = 8, jQ^ (~g;~z)jf~zg = 6, ^ jR(~g;~z)j = 0, jR^ (~g;~z)jf~gg = 4, and jR^ (~g;~z)jf~zg = 8. Clearly, the desired H (y;~g;~z ) can be expressed in terms of P^ (~g;~z ), Q^ (~g;~z ), R^ (~g;~z ), and D(~g;~z ); however, prior to completing the construction, we restructure the expression ((P y) + Q) (y + R) so that it is read-once with respect to y. This accomplished by performing a \polynomial division" of y + R into (P y) + Q, resulting in the identity ((P y) + Q) (y + R) = H~ (y; P; Q; R; D), for the formula H~ (y; p; q; r; d) given below. ÷ +
d
÷
p
–
+
x q
x d
p
x r
y
r d
Formula H~ (y; p; q; r; d) Set H (y;~g;~z) = H~ (y; P^ (~g;~z); Q^ (~g;~z); R^ (~g;~z); D(~g;~z)); then it is easily checked that depth(H (y;~g;~z)) = 9, 7
jH (y;~g;~z)j = 0, jH (y;~g;~z)jfyg = 1, jH (y;~g;~z)jf~gg = 44, and jH (y;~g;~z)jf~zg = 42, as claimed. Furthermore, it follows by the construction of these formulae that
G(y) H (y; G~ (~z );~z ) which completes the proof for the case where G(y) ((P y) + Q) (y + R). The simpler case where G(y) (P y) + Q is handled similarly. 2 The formulae introduced in Lemma 3 introduce new variables, z1 , z2, and z3 . The next lemmas establish that these can be replaced by elements of the ground eld F , provided F is suciently large.
Lemma 4:
(i) Suppose G(z ) = GL (z ) GR(z ) for 2 f+; ?; ; g and that no proper subformula of G(z ) is identically zero. Let dL (respectively, dR ) be an upper bound on the degree in z of the numerator and denominator of the rational function GL (z ) (respectively, GR (z )). If 2 f+; ?g then there are at most dL + dR elements 2 F such that G( ) is identically zero but none of GL ( ), GR ( ), or any of their subformulae are identically zero. If 2 f; g then there are no elements 2 F such that G( ) is identically zero but none of GL ( ), GR ( ), or any of their proper subformulae are identically zero. (ii) Suppose G(z ) is read-once with respect to z and that no subformula of G(z ) is identically zero. Then there are at most 1 + depth(G(z )) jG(z )j elements 2 F such that a subformula of G( ) is identically zero.
Proof: The rst claim is easily veri ed by expressing the numerator of G(z) as a function of the numerators and denominators of GL (z ) and GR (z ). If 2 f+; ?g then this numerator is a nonzero polynomial with degree at most dL + dR in z . If 2 f; g then the numerator is a product of numerators of subformulae of GL(z ) and GR (z ).
The second claim can be proved using induction on the depth of G(z ) and the fact that, if G(z ) is read-once with respect to z , then every subformula of G(z ) that includes z is the quotient of two ane linear functions of z , while every other subformula of G(z ) has degree zero in z . 2
Lemma 5: Suppose the extended formula G(y) is read-once with respect to y, jFj jG(y)j + 9, and let S be a subset of F with at least jG(y)j + 9 distinct elements. Then there exists an extended formula H^ (y; a1 ; a2 ; a3 ) and formulae A1 , A2 , and A3 such that: G(y) H^ (y; A1 ; A2 ; A3 ). jH^ (y; a1 ; a2 ; a3 )j 42. jH^ (y; a1 ; a2 ; a3 )jfa ;a ;a g 44. H^ (y; a1 ; a2 ; a3 ) is read-once with respect to y. depth(H^ (y; a1 ; a2 ; a3 )) 9. 1
2
3
jA1 j; jA2 j; jA3 j jG(y)j + 1. The only constants occurring as a subformula of H^ (y; a1 ; a2 ; a3 ), A1 , A2 , or A3 either occur as a subformula of G(y) or belong to S .
Proof: As argued in the proof of lemma 3, since G(y) is read-once with respect to y, there exist formulae P , Q, and R such that either G(y) (P y) + Q y+R
8
or
G(y) (P y) + Q: Suppose the second case holds, and let H (y; g1; g2 ; g3 ; z1 ; z2; z3 ) be the formula that exists by applying Lemma 3 to G(y). If, as in the proof of Lemma 3,
? z2 g2 ? g3 D(g1 ; g2; g3 ; z1 ; z2 ; z3) = zz3 ? 2 z1 g1 ? g2 then, for any 1 ; 2 ; 3 2 F , the formula H (y; G(1 ); G(2 ); G(3 ); 1 ; 2 ; 3 ) is well-de ned and equivalent to G(y) provided that D(G(1 ); G(2 ); G(3 ); 1 ; 2 ; 3 ) is well-de ned and not equivalent to the zero function (again, see
the proof of Lemma 3 for details). We next show that there exist 1 ; 2 ; 3 2 F with the above properties. Lemma 4 (ii) implies that the set F includes at most jG(y)j elements such that either G( ) or one of its subformulae is identically zero. Applying this and the fact that G(y) is read-once with respect to y (and, hence, the quotient of two ane linear functions) to Lemma 4 (i) implies that there are at most jG(y)j+8 elements 1 2 F such that D(G(1 ); G(z2 ); G(z3 ); 1 ; z2 ; z3 ) or one of its subformulae is identically zero. Since jSj > jG(y)j +8, such a 1 can be found in S . The same argument can be applied twice more to prove the existence of elements 2 ; 3 of S such that D(G(1 ); G(2 ); G(z3 ); 1 ; 2 ; z3 ) and D(G(1 ); G(2 ); G(3 ); 1 ; 2 ; 3 ) are both well-de ned and nonzero, as desired. Now, set H^ (y; a1 ; a2 ; a3 ) = H (y; a1 ; a2 ; a3 ; 1 ; 2 ; 3 ); A1 = G(1 ); A2 = G(2 ); A3 = G(3 ): Clearly, H^ (y; A1 ; A2 ; A3 ) H (y; G(1 ); G(2 ); G(3 ); 1 ; 2 ; 3 ) G(y), the size bounds stated above for A1 , A2 , and A3 follow immediately from the de nitions of these formulae, and the size and depth bounds for H^ (a1 ; a2 ; a3 ) and H^ (A1 ; A2 ; A3 ) follow directly from those given for H (y; g1 ; g2 ; g3; y1 ; y2 ; y3 ) in the statement of Lemma 3. The simpler case where G(y) (P y) + Q is handled similarly. 2
Theorem 6: For any formula F of size S jFj ? 9, any subset S of F with size at least S + 9, and any integer k 2, there exists a formula G that is equivalent to F and has depth bounded by 9 k ) log S + 9 k + 3 ( log k
and size bounded by
1+ k 64 S ; such that the only constants appearing as a subformula of G either appear as a subformula of F or belong to S . For any formula F of size S > jFj ? 9, and any integer k 2, there exists a formula G that is equivalent to F and has depth bounded by 9 k + 2) log S + 9 k + 11 ( log k and size bounded by 1+ k 2 log S ( log jFj + 9): 64 S 6 log
6 log
Proof: Let F be an arbitrary formula of size S jFj ? 9, let S be a subset of F with at least S + 9 elements, and let k be an arbitrary integer such that k 2. Since the formula F has the depth and size stated in the lemma if k > S9 , we will assume k S9 . Using Lemma 2 and Lemma 5, we shall show that F can be restructured in a particular way and then iterate this restructuring process on a series of subformulae. Let Interior f0; 1gk?2, Border f0; 1gk?1, G (y) and (for all 2 Interior), and G (for all 2 Border) be the result of applying Lemma 2 to F and k. Let H^ (y; a 1 ; a 2 ; a 3 ) and A 1 , A 2 , A 3 ( 2 Interior) be the result of applying Lemma 5 to G (y) ( 2 Interior), respectively. Intuitively, the next step is to \reassemble" the 9
formula F substituting H^ (y; a 1 ; a 2 ; a 3 ) in place of each G (y). Prior to doing this, we simplify our subscript notation as follows. Let = (f1; 2; 3g Interior) [ Border; and, for each 2 , let (i; ) 2 f1; 2; 3g Interior w = ag i ifif = = 2 Border
and
if = (i; ) 2 f1; 2; 3g Interior i W = A G if = 2 Border. Now, de ne E (w : 2) (for all 2 Interior [ Border) recursively as
^ (E 0 (w : 2) E 1 (w : 2); a 1 ; a 2 ; a 3 ) if 2 Interior E (w : 2) = H E (w : 2) = g if 2 Border. It follows from the above and the properties of H^ (y; a 1 ; a 2 ; a 3 ) ( 2 Interior) that: F E" (W : 2). depth(E" (w : 2)) 9 k. jE" (w : 2)j 42 jInteriorj. For all 2 Interior, jE" (w : 2)jfw ; ;w ; ;w ; g 44 (1
and
)
(2
)
(3
)
jW(1; ) j; jW(2; ) j; jW(3; ) j jG (y)j + 1 b k1 jF jc + 1: For all 2 Border, jE" (w : 2)jfw g = 1 and jW j = jG j d k1 jF je. X X jG (y)j + jG j = jF j. 2Interior
2Border
As well, jInteriorj 2k. Therefore,
X
X
jE" (W : 2)j 42 jInteriorj + 44 (jG (y)j + 1) + jG j 44jF j + 172k 64jF j, 2 Interior 2 Border since k jF9 j . depth(E" (w : 2)) 9 k. For all 2 , jW j b k1 jF jc + 1 k1 S + 1. Now, by iterating this entire restructuring process on all the formulae W ( 2 ) i times, we obtain extended formulae E"i (w~ : ~2i ) and formulae W~ (for all ~ 2 i ) such that: F E"i (W~ : ~2i). jE"i (W~ : ~2i )j 64iS . depth(E"i (w~ : ~2i)) 9 k i. For all ~ 2 i , jW~ j ( k1 )i S + 2.
10
S i Therefore, after i = d log log k e iterations, jW~ j 3 (for all ~ 2 ) so
l m S +3 depth(E"i (W~ : ~2i)) 9 k log log k 9 k ( log k ) log S + 9 k + 3
and
jE"i (W~ : ~2i)j 64d
log log
Se k
S 64 S 1+
6 log
k
as required. Suppose now that F is nite and that F is a formula with size S > jFj ? 9. Let E = F (x1 ), and consider F as a formula of size S over the in nite eld E ; the only \constants" arising as subformulae of F are x1 and elements of the small eld F . Let S E include all elements of F [x1 ] whose degree in x1 is at most logjFj(S + 9); clearly, jSj S + 9, and (by the claim for formulae over large elds) there exists a formula G^ equivalent to F that has depth bounded by 9 k ) log S + 9 k + 3 ( log k and size bounded by 1+ k ; 64 S such that the only constants appearing as a subformula of G^ either appear as a subformula of F (hence are x1 or belong to F ) or belong to S . Now, each element of S is equivalent to a formula (with constants in F and variable x1 ) with size at most 2 logjFj(S + 9) + 1 2 logjFj S + 9, and depth at most 2 logjFj(S + 9) 2 logjFj S + 8. Therefore, the formula G^ can be used to obtain an equivalent formula G with constants in F and variables x1 ; x2 ; : : : such that jGj jG^ j(2 logjFj S + 9) and depth(G) depth(G^ ) + 2 logjFj S + 8, as is required to prove the claim for the case that F is small. 2 6 log
Corollary 7: Over any eld F , for any xed > 0, for any formula of size S with operations from f+; ?; ; g[F , there are equivalent formulae with: Depth O(log S ) and size O(S 1+ ). Depth O(log1+ S ) and size S 1+O(
1 log log
S)
S Depth O(S ) and size OO((SS )log S ) ifif jFj jFj < S. log jFj
Proof: Apply Theorem 4 setting k = 2 (in the rst case), k = log S (in the second case), and k = S (in the 6
2
third case). 2
4 Additional Results Division-Free Formulae: It should be noted that several parts of the proofs in Section 3 are signi cantly simpler in the case of division-free formulae. In particular, the process of introducing new variables to a formula and then eliminating these variables so as to avoid division by a zero formula (in Lemmas 3, 4, and 5) is unnecessary. Lemmas 3, 4, and 5 may be replaced by the following. Lemma 8: For a division-free extended formula G(y) that is read-once with respect to y, there exists an extended formula H (y; a1; a2 ), and formulae A1 and A2 such that: G(y) H (y; A1 ; A2 ) jH (y; a1; a2 )j = 0 11
jH (y; a1; a2 )jfa ;a g = 3 1
2
H (y; a1; a2 ) is read-once with respect to y depth(H (y; a1 ; a2 )) = 3 jA1 j; jA2 j jG(y)j + 1.
Proof: Clearly, since G(y) is read-once with respect to y, there exist formulae P and Q such that G(y) (P y) + Q: By substituting y = 0 and y = 1 in the above equivalence, we obtain Q G(0) and P G(1) ? G(0). Thus, by setting H (y; a1 ; a1 ) = ((a2 ? a1 ) y) + a1 ; and A1 = G(0) and A2 = G(1), the required properties are satis ed. 2
Following this, all references to Lemma 5 in Theorem 6 may be replaced by references to Lemma 8. Although some constants are smaller for the division-free case, the ultimate tradeo s (expressed in Corollary 7) are the same, except that, in the third case of Corollary 7, the formula size can be linear, regardless of the size of the eld.
Boolean Formulae: It is straightforward to adapt our techniques to Boolean formulae over the basis f^; _; :g, since any such formula of size S is equivalent to a Boolean formula of size O(S ) over the basis f^; ; 1g: Each _ in a formula can be replaced by a ^ and three :'s, by De Morgan's law, F _ G :(:F ^ :G); then, each : can be replaced by a and a 1: :F 1 F: Now, since Boolean formulae over the basis f^; ; 1g can be regarded as division-free formula over the eld GF (2), the above comments for division-free formula apply.
^ Simple Formulae: Kosaraju p (1986) showed that any simple formula F is equivalent to a division free formula F ^ of depth at most log jF j + 2 log jF j + d, for some constant d; clearly such a formula F can have size at most p cjF j1+ jF j 2 O(jF j1+" ) for some constant c and for arbitrary " > 0. We show how Brent's construction can 2 log
be modi ed to improve the bound on formula size implied by Kosaraju (1986) for the restructuring of simple formulae.
Theorem 9: For any simple formula F there exists an equivalent division-free formula G (not generally simple) with depth at most 3 log jF j such that jGj jF j + 21 jF j log jF j.
Proof: We prove the result by induction on the size of F . The result is trivial if jF j 2, since it is sucient to set G = F . Suppose jF j > 2 and set m = d 12 jF je. By Lemma 1 there exists an extended formula G(y) that is read-once with respect to y, formulae U and V , and an operation such that F = G(U V ), jG(y)j jF j ? m = b 21 jF jc, and jU j; jV j m ? 1 b 21 jF jc. Since F is simple, G(y), U and V are simple as well. Furthermore, there exist formulae A and B such that jAj; jB j jG(y)j, A and B are both simple, and such that G(y) (A y) + B ; Brent's construction ensures that jU j + jV j + jB j jF j. Since G(y) is simple the only operation used in A will be ; consequently, since at least one argument of every gate in A has depth 0, A is equivalent to a balanced formula A^ such that jAj = jA^j and such that the depth of A^ is at most blog jAjc (log jF j) ? 1. Now F F1 = (A^ (U V )) + B: 12
By the inductive hypothesis U is equivalent to a formula U^ such the depth of U^ is at most 3 log jU j and U^ jU j + 21 jU j log jU j. Similarly, V is equivalent to a formula V^ whose depth is at most 3 log jV j and whose size is at most jV j + 12 jV j log jV j, and B is equivalent to a formula B^ with depth at most 3 log jB j and size at most jB j + 12 jB j log jB j. Set ^ G = (A^ (U^ V^ )) + B: Then G F , and the depth of G is the maximum of 1+depth(B^ ), 2+depth(A^), 3+depth(U^ ), and 3+depth(V^ ). Since the depths of U^ , V^ , and B^ are at most 3 logb(jF j=2)c and the depth of A^ is at most (log jF j)?1 3 log jF j?2, the depth of G is at most 3 log jF j. Also, jGj jA^j + jU^ j + jV^ j + jB^ j b 12 jF jc + (jU j + jV j + jB j) + 21 (jU j + jV j + jB j) logb 12 jF jc b 21 jF jc + jF j + 21 jF j((log jF j) ? 1) jF j + 12 jF j log jF j; as desired. 2
5 Speci c Formulae and Known Lower Bounds As mentioned in Section 1, parallel algorithms for the formula evaluation problem can be modi ed to transform formulae into small-depth circuits, which can in turn be transformed into formulae of the same depth. We conclude with an example illustrating that polynomial size blow-up can arise from this approach, even if one is restricted to division-free formulae. In particular, we shall exhibit a formula of size n such that when the formula evaluation algorithm of Miller and Reif (1985) is applied to it, the resulting formula is of size (n1+ ), for a xed > 0. For each n, de ne the formula Fn (x1 ; x2 ; :::; x2n+1 ) as
Fn (x1 ; x2 ; :::; x2n+1 ) =( (((x1 x2 ) + x3 ) x4 ) + x2n ) + x2n+1 : Clearly, depth(Fn (x1 ; :::; x2n+1 )) = 2 n and jFn (x1 ; :::; x2n+1 )j = 2 n + 1. Commentz-Walter (1979) shows that, over the Boolean semi-ring (f0; 1g; ^; _) (where negations are disallowed), there are formulae equivalent to Fn (x1 ; :::; x2n+1 ) with depth O(log n), but that all such formulae have size
(n log n). Furthermore, Commentz-Walter and Sattler (1980) show that, even if negations can be introduced, any log log n ). (This nonmonotonic formula of depth O(log n) that computes Fn (x1 ; :::; x2n+1 ) must have size ( lognlog log log n lower bound does not apply if operations can be introduced.) Now consider the formula G(nk) , where G(1) n (x1 ; : : : ; x2n+1 ) = Fn (x1 ; : : : ; x2n+1 ); for Fn (x1 ; x2 ; : : : ; x2n+1 ) as above, and G(nk) (x1 ; : : : ; x(2n+1)k ) = F (Gn(k?1) (x1(k?1) ); : : : ; Gn(k?1) (x2(kn?+11) )) for k > 1, where xi(k?1) = (x(i?1)(2n+1)k? +1 ; : : : ; xi(2n+1)k? ), i = 1; : : : ; 2n + 1: For n = 3 the method of Miller and Reif balances G(1) 3 (x1 ; : : : ; x7 ) to (x1 x2 + x3 )x6 + x4 + x6 x5 + x7 which induces one more occurrence of the variable x6 . When we try to balance G(3k) using Miller and Reif's method we induce two copies of each G(3i) (x(6i) ) in each level of the formula G(3k) . This implies that if the balanced formula is of size S (N ), for N = 7k , then S (N ) = 8S (N=7) which implies that Miller and Reif's method gives a formula of size n = n1:0686 . 1
1
log 8 log 7
13
Acknowledgements We thank Allan Borodin for pointing out the work of Commentz-Walter (1979) and Commentz-Walter and Sattler (1980).
14
References Brent, R.P. (1974), \The Parallel Evaluation of General Arithmetic Expressions," J. ACM, Vol. 21, No. 2, pp. 201{206. Bshouty, N.H., R. Cleve, and W. Eberly (1991), \Size-Depth Tradeo s for Algebraic Formulae," Proc. 32th Ann. IEEE Symp. on Foundations of Computer Sci., pp. 334{341. Buss, S.R. (1987), \The Boolean Formula Value Problem is in ALOGTIME," Proc. 19th Ann. ACM Symp. on Theory of Comput., pp. 123{131. Buss, S.R., S. Cook, A. Gupta, and V. Ramachandran (1989), \An Optimal Parallel Algorithm for Formula Evaluation," Manuscript (submitted for publication). Commentz-Walter, B. (1979), \Size-Depth Tradeo in Monotone Boolean Formulae," Acta Informatica Vol. 12, pp. 227{243. Commentz-Walter, B., J. Sattler (1980), \Size-Depth Tradeo in Non-monotone Boolean Formulae," Acta Informatica Vol. 14, pp. 257{269. Gupta, A. (1985), \A Fast Parallel Algorithm for Recognition of Parenthesis Languages," Master's thesis, University of Toronto. Kosaraju, S.A. (1986), \Parallel Evaluation of Division-Free Arithmetic Expressions," Proc. 18th Ann. ACM Symp. on Theory of Comput., pp. 231{239. Kosaraju, S.A., and A.L. Delcher (1990), \A Tree Partitioning Technique With Applications to Expression Evaluation and Term Matching," Proc. 31th Ann. IEEE Symp. on Foundations of Computer Sci., pp. 163{172. Miller, G.L., and J. Reif (1985), \Parallel Tree Contraction and its Application," Proc. 26th Ann. IEEE Symp. on Foundations of Computer Sci., pp. 478{489. Muller, D.E., and F.P. Preparata (1976), \Restructuring of Arithmetic Expressions for Parallel Evaluation," J. ACM, Vol. 23, No. 3, pp. 534{543.
15
Richard Cleve
Wayne Eberly
Department of Computer Science University of Calgary Calgary, Alberta, Canada T2N 1N4
Abstract We prove some tradeo s between the size and depth of algebraic formulae. In particular, we show that, for any xed > 0, any algebraic formula of size S can be converted into an equivalent formula of depth O(log S ) and size O(S 1+ ). This result is an improvement over previously-known results where, to obtain the same depth bound, the formula-size is (S ), with 2.
1 Introduction A classical result, due to Brent (1974), implies that for any algebraic formula there is an algebraic circuit of \small" depth and \similar" size that computes the same function. More precisely, if the formula has size S then the circuit has depth O(log S ) and size O(S ). This result holds for formulae over any eld. We believe that a natural question to consider is whether for any algebraic formula there is an equivalent formula of small depth and similar size. Since any circuit of depth O(log S ) can be transformed into a formula of the same depth and with size polynomial in S , it follows immediately from Brent's result that there is also a formula of depth O(log S ) and size S O(1) that computes the same function as the original formula of size S . Applying this to the speci c circuits that result from Brent's construction yields formulae with size as large as (S ), with 2. Simple changes in Brent's construction may improve the exponent, but straightforward modi cations do not appear to result in exponents arbitrarily close to one. A widely-investigated problem that is related to Brent's result, as well as our work, is the \formula evaluation problem," where the goal is to construct a \universal formula evaluator" algorithm. Such an algorithm takes as input a description of a formula, with all of its inputs speci ed, and produces as output the value of the formula. Parallel algorithms for this problem have been proposed by Cook and Gupta (1985); Miller and Reif (1985); Buss (1987); Buss, Cook, Gupta, and Ramachandran (1989); and Kosaraju and Delcher (1990). These yield NC algorithms for the problem that also produce, for any given formula of size S , a circuit of depth O(log S ). When these circuits are expressed as formulae, the sizes are (S ), for various 2. In the case of division-free formulae, the exponents are smaller, but nevertheless bounded above one. As an example, in Section 5, we exhibit division-free formulae for which Miller and Reif's method produces formulae with such a polynomial size blowup. In this paper, we show that, over any eld F , for any xed > 0, for any formula of size S with operations from f+; ?; ; g [ F , there are equivalent formulae with: Depth O(log S ) and size O(S 1+ ). Depth O(log1+ S ) and size S 1+O( S ) . 1 log log
A preliminary version of this paper was presented at the 32nd Annual ACM Symposium on Foundations of Computer Science, San Juan, Puerto Rico (1991). This research was supported in part by NSERC of Canada.
1
Depth O(S ) and size O(S ) In the latter result, the method we use will add new variables to the formula when the eld size is less than S . Also, for Boolean formulae with operations from f^; _; :g, we obtain similar conditions as above. The techniques that we use include a multi-level extension of Brent's tree-decomposition method, as well as other restructuring methods. The organization of the remainder of this paper is as follows. Section 2 contains basic de nitions and notation. Section 3 contains the main result (expressed in Theorem 6, and interpreted in Corollary 7). Section 4 concerns additional results that apply for special classes of formulae, such as division-free, Boolean, and \simple" formulae. Section 5 describes some speci c formulae that appear to exhibit increases in size when their depth is reduced, and some known lower bounds on the size-depth tradeo due to Commentz-Walter (1979) and Commentz-Walter and Sattler (1980).
2 De nitions and Notation Algebraic Formulae: For a eld F , a formula over (F ; +; ?; ; ) of depth d is de ned as follows. A depth 0 formula is either c, for some c 2 F (a constant) or xu , for some u 2 f1; 2; : : :g (an input). For d > 0, a depth d formula is (F G), where 2 f+; ?; ; g, F and G are formulae of depth dF and dG respectively and d = max(dF ; dG ) + 1. The size of a formula F , denoted as jF j, is, informally, the number of occurrences of inputs and constants in the formula. More formally, a depth 0 formula has size one, and j(F G)j = jF j + jGj ( 2 f+; ?; ; g). A formula over (F ; +; ?; ; ) corresponds to a rational function in F (x1 ; :::; xn ) (for some n) in a natural way, provided that it does not involve a division by a formula equivalent to zero. For formulae F and G, F G denotes that they correspond to the same rational function. Hence, denotes equivalence in the function semantics sense. Division-Free Formulae: A division-free formula is one that has no divisions. Clearly, division-free formulae correspond to polynomials.
Simple Formulae: A simple formula is one that is division-free, and for which at least one argument of each multiplication operation is either an input or a constant. Thus, a depth 0 simple formula is either a constant or an input, and for depth d > 0 a depth d simple formula is (F G) where F and G are simple formulae, 2 f+; ?; g, and if = then either F or G has depth 0. Extended Formulae: In order to denote decompositions of a formula, we de ne an extended formula, which is allowed to take auxiliary inputs, which are input symbols that are not from fx1 ; x2 ; : : :g. For clarity, in extended
formulae, we write all auxiliary inputs as \arguments" to the formula. For example, the extended formula F (y) has auxiliary input symbol y. If G is a formula, then F (G) denotes the formula F (y) modi ed by substituting G for the symbol y. The size of an extended formula is de ned recursively as above, except that auxiliary inputs are not counted (that is, an auxiliary input has size 0). Also, we use special terminology to denote the number of occurrences of auxiliary variables in extended formulae. For A fy1 ; :::; ymg, jG(y1 ; :::; ym )jA denotes precisely the total number of occurrences of inputs from A in G(y1 ; :::; ym ). In particular, an extended formula G(y1 ; :::; ym) is read-once with respect to an auxiliary input yi if and only if jG(y1 ; :::; ym )jfyi g = 1. Note that, for formulae F1 ; :::; Fm ,
jG(F1 ; :::; Fm )j = jG(y1 ; :::; ym )j +
m X i=1
jFi j jG(y1 ; :::; ym)jfyi g :
Binary Strings: As usual, for k 0, f0; 1gk denotes all binary strings of length k. Furthermore, " denotes the empty string, and f0; 1gk denotes all binary strings of length less than or equal to k.
2
3 Main Result Our main result is Theorem 6 (Corollary 7 presents some consequences of this result). Brent's result is partially based on the following lemma, which concerns ways of partitioning trees into pieces of various sizes.
Lemma 1 (Brent, 1974): For any formula F and any m such that 1 < m jF j there exists an extended formula G(y) that is read-once with respect to y, formulae U and V , and an operation such that: F = G(U V ). jG(y)j jF j ? m and jU j; jV j < m. For a formula F , Brent applies Lemma 1, with m = d 21 jF je, thereby \decomposing" F into three pieces G(y), U , and V , each of size at most d 21 jF je. Then, using a recursive technique, he: translates G(y) into a circuit of size O(jG(y)j) and depth O(log jG(y)j) that computes A, B , C , and D such that y) + B ; G(y) ((CA y) + D translates U into an equivalent circuit U^ of size O(jU j) and depth O(log jU j), and similarly translates V into V^ . Finally, Brent expresses F as the required circuit by the identity ^ ^ F (A (U^ V^ )) + B : (C (U V )) + D Lemma 2 (below) is a multi-level version of the decomposition that Brent uses. Informally, it states that every formula F can be partitioned into 4k ? 3 or fewer pieces, each of size at most d k1 jF je.
Lemma 2: For any formula F , and any positive integer k such that 2 k < jF j, there exist nite sets of indices Interior; Border f0; 1g, extended formulae G (y) and operations for all 2 Interior, and formulae G for all 2 Border, such that these form a well de ned decomposition of F : (a) " 2 Interior; (b) Interior \ Border = ;; (c) For all 2 Interior, both 0 and 1 are in Interior [ Border; (d) For all 2 Border, neither 0 nor 1 is in Interior [ Border; (e) For all 2 Interior, jG (y)j b k1 jF jc and, for all 2 Border, jG j d k1 jF je; (f) If formula U is de ned recursively for all 2 Interior [ Border by the rule G (U 0 U 1) if 2 Interior U = G if 2 Border
then F = U" . In addition, these form a small decomposition of F : (g) Interior f0; 1gk?2 and jInteriorj 2k ? 2; (h) Border f0; 1gk?1 and jBorderj 2k ? 1.
3
Proof: Let F be an arbitrary formula and let k be an integer such that 2 k < jF j. We shall rst give a construction for the sets Interior and Border, the extended formulae G (y) and operations for 2 Interior, and formulae G for 2 Border, and demonstrate that these give a well de ned decomposition of F . We shall then argue that this decomposition is small. To begin, initialize Interior and Border to be empty, and set U" = F , so that jU" j = jF j = jF j ? j"j d k1 jF je. Since k 2, jU" j > d k1 jF je. To continue, let 2 f0; 1g such that U has been de ned with jU j jF j ? j j d k1 jF je, and such that has not been added (yet) to either Interior or Border | ending the construction if no such string exists. If jU j d k1 jF je then add to Border and set G to be U . Otherwise, add to Interior and apply Lemma 1 (Brent) to U with m = jU j ? b k1 jF jc, to de ne an extended formula G (y) that is read-once with respect to y, an operation , and formulae U 0, U 1 such that U = G (U 0 U 1); jG (y)j jU j ? m = b k1 jF jc; jU 0j; jU 1j < m. Note that it follows that jU 0j jF j ? j 0j d k1 jF je and jU 1 j jF j ? j 1j d k1 jF je. It is clear that properties (a) { (f) are established if this construction terminates. To see that it does, consider
2 f0; 1g such that U is de ned during the construction. Clearly, 0 < jU j jF j ? j j d k1 jF je, so j j < k. Therefore the construction does eventually halt, de ning sets Border f0; 1gk?1 and Interior f0; 1gk?2, and extended formulae, operations and formulae such that properties (a) { (f) hold. For 2 Interior de ne Interior Interior as Interior = Interior \ f 2 f0; 1g : is a pre x of g: Then it is easily established by structural induction that jInterior j 2 jU j k ? 2
jF j
for all 2 Interior: If 2 Interior and both of 0; 1 are in Border, then jInterior j = 1 2 jU j k ? 2;
jF j
since jU j > jFk j . If 2 Interior and exactly one (say, 0) of 0 and 1 is in Interior, then
jInterior j = jInterior 0j + 1 2 jUjF 0j j k ? 1 2 jjUF jj k ? 2;
since jU j ? jU 0 j jFk j . Finally, if all of , 0, and 1 are in Interior, then, since jU j jU 0 j + jU 1 j,
jInterior j = jInterior 0j +jInterior
1j + 1 2 jUjF 0j j k + 2 jUjF 1j j k ? 3 2 j U j k 2 j U j k
0
1 +1 ?3 jF j + jF j as desired. 2 jjUF jj k ? 2;
m l Therefore, since F = U" and Interior = Interior" , jInteriorj 2jjFF jj k ? 2 = 2k ? 2, which is sucient to establish property (g). Property (h) also follows because elements of the sets Interior and Border correspond respectively to the internal nodes and leaves of a binary tree, so jBorderj = jInteriorj + 1. 2
4
In order to control the formula size, we use a di erent approach than Brent for restructuring extended formulae of the form G(y). Lemma 3 (below) achieves the restructuring; however, it introduces new auxiliary inputs. In Lemmas 4 and 5, we show how to eliminate the auxiliary inputs (roughly, by substituting constants for them|or small polynomials, if the eld is too small|with special care to avoid introducing divisions by a zero formula).
Lemma 3: For any extended formula G(y) that is read-once with respect to y there exists an extended formula H (y; g1; g2 ; g3 ; z1 ; z2; z3 ) such that: G(y) H (y; G(z1 ); G(z2 ); G(z3 ); z1 ; z2 ; z3 ). H (y; g1; g2 ; g3 ; z1; z2 ; z3 ) is read-once with respect to y. jH (y; g1; g2 ; g3 ; z1; z2 ; z3 )j = 0. jH (y; g1; g2 ; g3 ; z1; z2 ; z3 )jfg ;g ;g g 44. jH (y; g1; g2 ; g3 ; z1; z2 ; z3 )jfz ;z ;z g 42. depth(H (y; g1 ; g2 ; g3; z1 ; z2 ; z3 )) 9. 1
2
3
1
2
3
Proof: Since G(y) is read-once with respect to y, there exist formulae P , Q, and R such that either G(y) (P y) + Q y+R
or
G(y) (P y) + Q:
The existence of P , Q, and R can be shown by considering the functions computed along the path in G(y) from y to its root: each such function is the quotient of two ane linear functions of y. Although this establishes the existence of P , Q, and R, this does not lead to an ecient way to construct these formulae: in the general case (with divisions), the resulting formula size may be exponential in jG(y)j. Instead, we use the method below. First, we consider the case where y) + Q : G(y) (P y+R
Substituting three distinct new auxiliary variables z1 , z2 , and z3 for y in this equation for G(y) results in the system of equations 2 3 2 3 2 3 z1 1 ?G(z1 ) P G(z1 ) z1 4 z2 1 ?G(z2 ) 5 4 Q 5 4 G(z2 ) z2 5 : z3 1 ?G(z3 ) R G(z3 ) z3 Since z1 , z2 , and z3 are new auxiliary variables and G(y) is not of the form (P y) + Q, z1 1 ?G(z1 ) z2 1 ?G(z2 ) 6 0; z3 1 ?G(z3 )
which implies that there are unique rational functions for P , Q, and R that satisfy the linear system. In particular, ( (
z3 G(z3 )) ? (z2 G(z2 )) G(z2 ) ? G(z3 ) z2 G(z 2 )) ? (z1 G(z1 )) G(z1 ) ? G(z2 ) : P z3 ? z2 G(z2 ) ? G(z3 ) z2 ? z1 G(z1 ) ? G(z2 )
5
By examining the expressions for the above determinants, we deduce that1 ^ ~ P P (G~ (~z );~z ) ; D(G(~z );~z ) where P^ (~g;~z) and D(~g;~z) are as follows. – x
x
–
–
x z1
g2
x g1
z2
– g3
–
x
g2
z2
g1
x g2
z3
g2
g3
Formula P^ (~g;~z) – x
x
– z1
– z2
–
g2
g3
–
z2
z3
g1
g2
Formula D(~g;~z) Clearly, depth(P^ (~g;~z )) = 4, depth(D(~g;~z )) = 3, jP^ (~g;~z)j = 0, jP^ (~g;~z)jf~gg = 8, jP^ (~g;~z)jf~zg = 4, jD(~g;~z)j = 0,
jD(~g;~z)jf~gg = 4, and jD(~g;~z)jf~zg = 4. Similarly,
^ ~ Q Q(G~ (~z );~z ) D(G(~z );~z ) and where Q^ (~g;~z) and R^ (~g;~z) are as shown below. 1
Here ( ( ) ) denotes ( ( 1 ) ( 2 ) ( 3 ) ~ ~ G z
;~ z
G z
;G z
;G z
^ ~ R R(G~ (~z );~z ) ; D(G(~z );~z )
; z 1 ; z2 ; z3
) and (
~ g; ~ z
6
) denotes (
g 1 ; g2 ; g3 ; z1 ; z2 ; z3
).
– x
x
– x z2
z1
–
z3
x g1
–
x
g2
g2
x
x g3
z3
g2
z2
–
z1
x g3
g1
g2
Formula Q^ (~g;~z) – x
x
–
–
z2
z3
–
x z1
z1
x g1
z2
– z2
x
g2
z2
x g2
z3
g3
Formula R^ (~g;~z) It is easily checked that depth(Q^ (~g;~z )) = depth(R^ (~g;~z )) = 4, jQ^ (~g;~z)j = 0, jQ^ (~g;~z)jf~gg = 8, jQ^ (~g;~z)jf~zg = 6, ^ jR(~g;~z)j = 0, jR^ (~g;~z)jf~gg = 4, and jR^ (~g;~z)jf~zg = 8. Clearly, the desired H (y;~g;~z ) can be expressed in terms of P^ (~g;~z ), Q^ (~g;~z ), R^ (~g;~z ), and D(~g;~z ); however, prior to completing the construction, we restructure the expression ((P y) + Q) (y + R) so that it is read-once with respect to y. This accomplished by performing a \polynomial division" of y + R into (P y) + Q, resulting in the identity ((P y) + Q) (y + R) = H~ (y; P; Q; R; D), for the formula H~ (y; p; q; r; d) given below. ÷ +
d
÷
p
–
+
x q
x d
p
x r
y
r d
Formula H~ (y; p; q; r; d) Set H (y;~g;~z) = H~ (y; P^ (~g;~z); Q^ (~g;~z); R^ (~g;~z); D(~g;~z)); then it is easily checked that depth(H (y;~g;~z)) = 9, 7
jH (y;~g;~z)j = 0, jH (y;~g;~z)jfyg = 1, jH (y;~g;~z)jf~gg = 44, and jH (y;~g;~z)jf~zg = 42, as claimed. Furthermore, it follows by the construction of these formulae that
G(y) H (y; G~ (~z );~z ) which completes the proof for the case where G(y) ((P y) + Q) (y + R). The simpler case where G(y) (P y) + Q is handled similarly. 2 The formulae introduced in Lemma 3 introduce new variables, z1 , z2, and z3 . The next lemmas establish that these can be replaced by elements of the ground eld F , provided F is suciently large.
Lemma 4:
(i) Suppose G(z ) = GL (z ) GR(z ) for 2 f+; ?; ; g and that no proper subformula of G(z ) is identically zero. Let dL (respectively, dR ) be an upper bound on the degree in z of the numerator and denominator of the rational function GL (z ) (respectively, GR (z )). If 2 f+; ?g then there are at most dL + dR elements 2 F such that G( ) is identically zero but none of GL ( ), GR ( ), or any of their subformulae are identically zero. If 2 f; g then there are no elements 2 F such that G( ) is identically zero but none of GL ( ), GR ( ), or any of their proper subformulae are identically zero. (ii) Suppose G(z ) is read-once with respect to z and that no subformula of G(z ) is identically zero. Then there are at most 1 + depth(G(z )) jG(z )j elements 2 F such that a subformula of G( ) is identically zero.
Proof: The rst claim is easily veri ed by expressing the numerator of G(z) as a function of the numerators and denominators of GL (z ) and GR (z ). If 2 f+; ?g then this numerator is a nonzero polynomial with degree at most dL + dR in z . If 2 f; g then the numerator is a product of numerators of subformulae of GL(z ) and GR (z ).
The second claim can be proved using induction on the depth of G(z ) and the fact that, if G(z ) is read-once with respect to z , then every subformula of G(z ) that includes z is the quotient of two ane linear functions of z , while every other subformula of G(z ) has degree zero in z . 2
Lemma 5: Suppose the extended formula G(y) is read-once with respect to y, jFj jG(y)j + 9, and let S be a subset of F with at least jG(y)j + 9 distinct elements. Then there exists an extended formula H^ (y; a1 ; a2 ; a3 ) and formulae A1 , A2 , and A3 such that: G(y) H^ (y; A1 ; A2 ; A3 ). jH^ (y; a1 ; a2 ; a3 )j 42. jH^ (y; a1 ; a2 ; a3 )jfa ;a ;a g 44. H^ (y; a1 ; a2 ; a3 ) is read-once with respect to y. depth(H^ (y; a1 ; a2 ; a3 )) 9. 1
2
3
jA1 j; jA2 j; jA3 j jG(y)j + 1. The only constants occurring as a subformula of H^ (y; a1 ; a2 ; a3 ), A1 , A2 , or A3 either occur as a subformula of G(y) or belong to S .
Proof: As argued in the proof of lemma 3, since G(y) is read-once with respect to y, there exist formulae P , Q, and R such that either G(y) (P y) + Q y+R
8
or
G(y) (P y) + Q: Suppose the second case holds, and let H (y; g1; g2 ; g3 ; z1 ; z2; z3 ) be the formula that exists by applying Lemma 3 to G(y). If, as in the proof of Lemma 3,
? z2 g2 ? g3 D(g1 ; g2; g3 ; z1 ; z2 ; z3) = zz3 ? 2 z1 g1 ? g2 then, for any 1 ; 2 ; 3 2 F , the formula H (y; G(1 ); G(2 ); G(3 ); 1 ; 2 ; 3 ) is well-de ned and equivalent to G(y) provided that D(G(1 ); G(2 ); G(3 ); 1 ; 2 ; 3 ) is well-de ned and not equivalent to the zero function (again, see
the proof of Lemma 3 for details). We next show that there exist 1 ; 2 ; 3 2 F with the above properties. Lemma 4 (ii) implies that the set F includes at most jG(y)j elements such that either G( ) or one of its subformulae is identically zero. Applying this and the fact that G(y) is read-once with respect to y (and, hence, the quotient of two ane linear functions) to Lemma 4 (i) implies that there are at most jG(y)j+8 elements 1 2 F such that D(G(1 ); G(z2 ); G(z3 ); 1 ; z2 ; z3 ) or one of its subformulae is identically zero. Since jSj > jG(y)j +8, such a 1 can be found in S . The same argument can be applied twice more to prove the existence of elements 2 ; 3 of S such that D(G(1 ); G(2 ); G(z3 ); 1 ; 2 ; z3 ) and D(G(1 ); G(2 ); G(3 ); 1 ; 2 ; 3 ) are both well-de ned and nonzero, as desired. Now, set H^ (y; a1 ; a2 ; a3 ) = H (y; a1 ; a2 ; a3 ; 1 ; 2 ; 3 ); A1 = G(1 ); A2 = G(2 ); A3 = G(3 ): Clearly, H^ (y; A1 ; A2 ; A3 ) H (y; G(1 ); G(2 ); G(3 ); 1 ; 2 ; 3 ) G(y), the size bounds stated above for A1 , A2 , and A3 follow immediately from the de nitions of these formulae, and the size and depth bounds for H^ (a1 ; a2 ; a3 ) and H^ (A1 ; A2 ; A3 ) follow directly from those given for H (y; g1 ; g2 ; g3; y1 ; y2 ; y3 ) in the statement of Lemma 3. The simpler case where G(y) (P y) + Q is handled similarly. 2
Theorem 6: For any formula F of size S jFj ? 9, any subset S of F with size at least S + 9, and any integer k 2, there exists a formula G that is equivalent to F and has depth bounded by 9 k ) log S + 9 k + 3 ( log k
and size bounded by
1+ k 64 S ; such that the only constants appearing as a subformula of G either appear as a subformula of F or belong to S . For any formula F of size S > jFj ? 9, and any integer k 2, there exists a formula G that is equivalent to F and has depth bounded by 9 k + 2) log S + 9 k + 11 ( log k and size bounded by 1+ k 2 log S ( log jFj + 9): 64 S 6 log
6 log
Proof: Let F be an arbitrary formula of size S jFj ? 9, let S be a subset of F with at least S + 9 elements, and let k be an arbitrary integer such that k 2. Since the formula F has the depth and size stated in the lemma if k > S9 , we will assume k S9 . Using Lemma 2 and Lemma 5, we shall show that F can be restructured in a particular way and then iterate this restructuring process on a series of subformulae. Let Interior f0; 1gk?2, Border f0; 1gk?1, G (y) and (for all 2 Interior), and G (for all 2 Border) be the result of applying Lemma 2 to F and k. Let H^ (y; a 1 ; a 2 ; a 3 ) and A 1 , A 2 , A 3 ( 2 Interior) be the result of applying Lemma 5 to G (y) ( 2 Interior), respectively. Intuitively, the next step is to \reassemble" the 9
formula F substituting H^ (y; a 1 ; a 2 ; a 3 ) in place of each G (y). Prior to doing this, we simplify our subscript notation as follows. Let = (f1; 2; 3g Interior) [ Border; and, for each 2 , let (i; ) 2 f1; 2; 3g Interior w = ag i ifif = = 2 Border
and
if = (i; ) 2 f1; 2; 3g Interior i W = A G if = 2 Border. Now, de ne E (w : 2) (for all 2 Interior [ Border) recursively as
^ (E 0 (w : 2) E 1 (w : 2); a 1 ; a 2 ; a 3 ) if 2 Interior E (w : 2) = H E (w : 2) = g if 2 Border. It follows from the above and the properties of H^ (y; a 1 ; a 2 ; a 3 ) ( 2 Interior) that: F E" (W : 2). depth(E" (w : 2)) 9 k. jE" (w : 2)j 42 jInteriorj. For all 2 Interior, jE" (w : 2)jfw ; ;w ; ;w ; g 44 (1
and
)
(2
)
(3
)
jW(1; ) j; jW(2; ) j; jW(3; ) j jG (y)j + 1 b k1 jF jc + 1: For all 2 Border, jE" (w : 2)jfw g = 1 and jW j = jG j d k1 jF je. X X jG (y)j + jG j = jF j. 2Interior
2Border
As well, jInteriorj 2k. Therefore,
X
X
jE" (W : 2)j 42 jInteriorj + 44 (jG (y)j + 1) + jG j 44jF j + 172k 64jF j, 2 Interior 2 Border since k jF9 j . depth(E" (w : 2)) 9 k. For all 2 , jW j b k1 jF jc + 1 k1 S + 1. Now, by iterating this entire restructuring process on all the formulae W ( 2 ) i times, we obtain extended formulae E"i (w~ : ~2i ) and formulae W~ (for all ~ 2 i ) such that: F E"i (W~ : ~2i). jE"i (W~ : ~2i )j 64iS . depth(E"i (w~ : ~2i)) 9 k i. For all ~ 2 i , jW~ j ( k1 )i S + 2.
10
S i Therefore, after i = d log log k e iterations, jW~ j 3 (for all ~ 2 ) so
l m S +3 depth(E"i (W~ : ~2i)) 9 k log log k 9 k ( log k ) log S + 9 k + 3
and
jE"i (W~ : ~2i)j 64d
log log
Se k
S 64 S 1+
6 log
k
as required. Suppose now that F is nite and that F is a formula with size S > jFj ? 9. Let E = F (x1 ), and consider F as a formula of size S over the in nite eld E ; the only \constants" arising as subformulae of F are x1 and elements of the small eld F . Let S E include all elements of F [x1 ] whose degree in x1 is at most logjFj(S + 9); clearly, jSj S + 9, and (by the claim for formulae over large elds) there exists a formula G^ equivalent to F that has depth bounded by 9 k ) log S + 9 k + 3 ( log k and size bounded by 1+ k ; 64 S such that the only constants appearing as a subformula of G^ either appear as a subformula of F (hence are x1 or belong to F ) or belong to S . Now, each element of S is equivalent to a formula (with constants in F and variable x1 ) with size at most 2 logjFj(S + 9) + 1 2 logjFj S + 9, and depth at most 2 logjFj(S + 9) 2 logjFj S + 8. Therefore, the formula G^ can be used to obtain an equivalent formula G with constants in F and variables x1 ; x2 ; : : : such that jGj jG^ j(2 logjFj S + 9) and depth(G) depth(G^ ) + 2 logjFj S + 8, as is required to prove the claim for the case that F is small. 2 6 log
Corollary 7: Over any eld F , for any xed > 0, for any formula of size S with operations from f+; ?; ; g[F , there are equivalent formulae with: Depth O(log S ) and size O(S 1+ ). Depth O(log1+ S ) and size S 1+O(
1 log log
S)
S Depth O(S ) and size OO((SS )log S ) ifif jFj jFj < S. log jFj
Proof: Apply Theorem 4 setting k = 2 (in the rst case), k = log S (in the second case), and k = S (in the 6
2
third case). 2
4 Additional Results Division-Free Formulae: It should be noted that several parts of the proofs in Section 3 are signi cantly simpler in the case of division-free formulae. In particular, the process of introducing new variables to a formula and then eliminating these variables so as to avoid division by a zero formula (in Lemmas 3, 4, and 5) is unnecessary. Lemmas 3, 4, and 5 may be replaced by the following. Lemma 8: For a division-free extended formula G(y) that is read-once with respect to y, there exists an extended formula H (y; a1; a2 ), and formulae A1 and A2 such that: G(y) H (y; A1 ; A2 ) jH (y; a1; a2 )j = 0 11
jH (y; a1; a2 )jfa ;a g = 3 1
2
H (y; a1; a2 ) is read-once with respect to y depth(H (y; a1 ; a2 )) = 3 jA1 j; jA2 j jG(y)j + 1.
Proof: Clearly, since G(y) is read-once with respect to y, there exist formulae P and Q such that G(y) (P y) + Q: By substituting y = 0 and y = 1 in the above equivalence, we obtain Q G(0) and P G(1) ? G(0). Thus, by setting H (y; a1 ; a1 ) = ((a2 ? a1 ) y) + a1 ; and A1 = G(0) and A2 = G(1), the required properties are satis ed. 2
Following this, all references to Lemma 5 in Theorem 6 may be replaced by references to Lemma 8. Although some constants are smaller for the division-free case, the ultimate tradeo s (expressed in Corollary 7) are the same, except that, in the third case of Corollary 7, the formula size can be linear, regardless of the size of the eld.
Boolean Formulae: It is straightforward to adapt our techniques to Boolean formulae over the basis f^; _; :g, since any such formula of size S is equivalent to a Boolean formula of size O(S ) over the basis f^; ; 1g: Each _ in a formula can be replaced by a ^ and three :'s, by De Morgan's law, F _ G :(:F ^ :G); then, each : can be replaced by a and a 1: :F 1 F: Now, since Boolean formulae over the basis f^; ; 1g can be regarded as division-free formula over the eld GF (2), the above comments for division-free formula apply.
^ Simple Formulae: Kosaraju p (1986) showed that any simple formula F is equivalent to a division free formula F ^ of depth at most log jF j + 2 log jF j + d, for some constant d; clearly such a formula F can have size at most p cjF j1+ jF j 2 O(jF j1+" ) for some constant c and for arbitrary " > 0. We show how Brent's construction can 2 log
be modi ed to improve the bound on formula size implied by Kosaraju (1986) for the restructuring of simple formulae.
Theorem 9: For any simple formula F there exists an equivalent division-free formula G (not generally simple) with depth at most 3 log jF j such that jGj jF j + 21 jF j log jF j.
Proof: We prove the result by induction on the size of F . The result is trivial if jF j 2, since it is sucient to set G = F . Suppose jF j > 2 and set m = d 12 jF je. By Lemma 1 there exists an extended formula G(y) that is read-once with respect to y, formulae U and V , and an operation such that F = G(U V ), jG(y)j jF j ? m = b 21 jF jc, and jU j; jV j m ? 1 b 21 jF jc. Since F is simple, G(y), U and V are simple as well. Furthermore, there exist formulae A and B such that jAj; jB j jG(y)j, A and B are both simple, and such that G(y) (A y) + B ; Brent's construction ensures that jU j + jV j + jB j jF j. Since G(y) is simple the only operation used in A will be ; consequently, since at least one argument of every gate in A has depth 0, A is equivalent to a balanced formula A^ such that jAj = jA^j and such that the depth of A^ is at most blog jAjc (log jF j) ? 1. Now F F1 = (A^ (U V )) + B: 12
By the inductive hypothesis U is equivalent to a formula U^ such the depth of U^ is at most 3 log jU j and U^ jU j + 21 jU j log jU j. Similarly, V is equivalent to a formula V^ whose depth is at most 3 log jV j and whose size is at most jV j + 12 jV j log jV j, and B is equivalent to a formula B^ with depth at most 3 log jB j and size at most jB j + 12 jB j log jB j. Set ^ G = (A^ (U^ V^ )) + B: Then G F , and the depth of G is the maximum of 1+depth(B^ ), 2+depth(A^), 3+depth(U^ ), and 3+depth(V^ ). Since the depths of U^ , V^ , and B^ are at most 3 logb(jF j=2)c and the depth of A^ is at most (log jF j)?1 3 log jF j?2, the depth of G is at most 3 log jF j. Also, jGj jA^j + jU^ j + jV^ j + jB^ j b 12 jF jc + (jU j + jV j + jB j) + 21 (jU j + jV j + jB j) logb 12 jF jc b 21 jF jc + jF j + 21 jF j((log jF j) ? 1) jF j + 12 jF j log jF j; as desired. 2
5 Speci c Formulae and Known Lower Bounds As mentioned in Section 1, parallel algorithms for the formula evaluation problem can be modi ed to transform formulae into small-depth circuits, which can in turn be transformed into formulae of the same depth. We conclude with an example illustrating that polynomial size blow-up can arise from this approach, even if one is restricted to division-free formulae. In particular, we shall exhibit a formula of size n such that when the formula evaluation algorithm of Miller and Reif (1985) is applied to it, the resulting formula is of size (n1+ ), for a xed > 0. For each n, de ne the formula Fn (x1 ; x2 ; :::; x2n+1 ) as
Fn (x1 ; x2 ; :::; x2n+1 ) =( (((x1 x2 ) + x3 ) x4 ) + x2n ) + x2n+1 : Clearly, depth(Fn (x1 ; :::; x2n+1 )) = 2 n and jFn (x1 ; :::; x2n+1 )j = 2 n + 1. Commentz-Walter (1979) shows that, over the Boolean semi-ring (f0; 1g; ^; _) (where negations are disallowed), there are formulae equivalent to Fn (x1 ; :::; x2n+1 ) with depth O(log n), but that all such formulae have size
(n log n). Furthermore, Commentz-Walter and Sattler (1980) show that, even if negations can be introduced, any log log n ). (This nonmonotonic formula of depth O(log n) that computes Fn (x1 ; :::; x2n+1 ) must have size ( lognlog log log n lower bound does not apply if operations can be introduced.) Now consider the formula G(nk) , where G(1) n (x1 ; : : : ; x2n+1 ) = Fn (x1 ; : : : ; x2n+1 ); for Fn (x1 ; x2 ; : : : ; x2n+1 ) as above, and G(nk) (x1 ; : : : ; x(2n+1)k ) = F (Gn(k?1) (x1(k?1) ); : : : ; Gn(k?1) (x2(kn?+11) )) for k > 1, where xi(k?1) = (x(i?1)(2n+1)k? +1 ; : : : ; xi(2n+1)k? ), i = 1; : : : ; 2n + 1: For n = 3 the method of Miller and Reif balances G(1) 3 (x1 ; : : : ; x7 ) to (x1 x2 + x3 )x6 + x4 + x6 x5 + x7 which induces one more occurrence of the variable x6 . When we try to balance G(3k) using Miller and Reif's method we induce two copies of each G(3i) (x(6i) ) in each level of the formula G(3k) . This implies that if the balanced formula is of size S (N ), for N = 7k , then S (N ) = 8S (N=7) which implies that Miller and Reif's method gives a formula of size n = n1:0686 . 1
1
log 8 log 7
13
Acknowledgements We thank Allan Borodin for pointing out the work of Commentz-Walter (1979) and Commentz-Walter and Sattler (1980).
14
References Brent, R.P. (1974), \The Parallel Evaluation of General Arithmetic Expressions," J. ACM, Vol. 21, No. 2, pp. 201{206. Bshouty, N.H., R. Cleve, and W. Eberly (1991), \Size-Depth Tradeo s for Algebraic Formulae," Proc. 32th Ann. IEEE Symp. on Foundations of Computer Sci., pp. 334{341. Buss, S.R. (1987), \The Boolean Formula Value Problem is in ALOGTIME," Proc. 19th Ann. ACM Symp. on Theory of Comput., pp. 123{131. Buss, S.R., S. Cook, A. Gupta, and V. Ramachandran (1989), \An Optimal Parallel Algorithm for Formula Evaluation," Manuscript (submitted for publication). Commentz-Walter, B. (1979), \Size-Depth Tradeo in Monotone Boolean Formulae," Acta Informatica Vol. 12, pp. 227{243. Commentz-Walter, B., J. Sattler (1980), \Size-Depth Tradeo in Non-monotone Boolean Formulae," Acta Informatica Vol. 14, pp. 257{269. Gupta, A. (1985), \A Fast Parallel Algorithm for Recognition of Parenthesis Languages," Master's thesis, University of Toronto. Kosaraju, S.A. (1986), \Parallel Evaluation of Division-Free Arithmetic Expressions," Proc. 18th Ann. ACM Symp. on Theory of Comput., pp. 231{239. Kosaraju, S.A., and A.L. Delcher (1990), \A Tree Partitioning Technique With Applications to Expression Evaluation and Term Matching," Proc. 31th Ann. IEEE Symp. on Foundations of Computer Sci., pp. 163{172. Miller, G.L., and J. Reif (1985), \Parallel Tree Contraction and its Application," Proc. 26th Ann. IEEE Symp. on Foundations of Computer Sci., pp. 478{489. Muller, D.E., and F.P. Preparata (1976), \Restructuring of Arithmetic Expressions for Parallel Evaluation," J. ACM, Vol. 23, No. 3, pp. 534{543.
15
