Average Case Complexity of Unbounded Fanin Circuits
Average Case Complexity of Unbounded Fanin Circuits Andreas Jakoby and Rudiger Reischuk Institut fur Theoretische Informatik Med. Universitat zu Lubeck October 1998
Abstract
Hastad has shown that functions like PARITY cannot be computed by unbounded fanin circuits of small depth and polynomial size. We generalize this result in two directions. First, we obtain the same tight lower bound for the average case. This is done by estimating the average delay { the natural generalization of circuit depth to an average case measure { of unbounded fanin circuits of polynomial size, resp. their error probability given an upper bound on the maximal delay. These bounds are obtained by extending the probabilistic restriction method to an average case setting. Secondly, we completely classify the set of parallel pre x functions { for which PARITY is just one example { with respect to their average delay in unbounded fanin circuits of a given size. It is shown that only two cases can occur: a parallel pre x functions either has the same complexity as PARITY, that is the average delay has to be of order (log n= log log s) for circuit of size s , or it can be computed with constant average delay and almost linear size { there is nothing in between. This classi cation is achieved by analyzing the algebraic structure of the semigroups that correspond to parallel pre x functions. It extends methods developed by the rst author in his Ph.D. Thesis.
1 Introduction To analyse the average-case complexity of Boolean functions we have introduced an average-case time complexity measure for Boolean circuits based on the notion of delay in [JRS94]. For the classical model with OR, AND and NOT gates of bounded fanin and fanout we have shown that n -ary functions like the OR, the addition or threshold functions with a xed threshold can be computed much faster on the average compared to the worst-case for which the trivial log n lower bound holds. Other functions like PARITY do not allow a signi cant speedup of the average time complexity, regardless of the size of the circuits. In [JRSW94] we have generalized our investigations to computing arbitrary pre x functions. Tight bounds for the average complexity have been obtained by analysing the algebraic structure of the semigroups that correspond to the pre x operator. These results have been extended in [Jak98a, Jak98b] to parallel pre x computations in other computational models like linear arrays and networks of automata. A di erent line of research has considered the complexity of Boolean functions in a circuit model where gates of arbitrary large fanin can be used. It is well known that in this model every Boolean function can be computed in constant depth if one allows circuits of exponential size. Functions like the OR or the addition again can be computed much more ecient, that is in constant depth and polynomial size [ChFL83a]. But PARITY remains dicult { the best speedup for polynomial size one can obtain is depth (log n= loglog n) [Ya85]. Hastad has proven optimal lower size bounds with respect to the circuit depth by analysing the probabilistic restriction method very carefully. In this paper we combine both questions and asks whether PARITY can be computed more eciently by unbounded fanin circuits on the average. At the same time we try to classify Boolean functions according to their average case complexity in the unbounded fanin model. Our work starts by investigating algebraic 1
Average Case Complexity of Unbounded Fanin Circuits
2
properties of the corresponding semigroups in a similar spirit as in [JRSW94, Jak98a] now tailored to unbounded fanin computations. For the worst-case complexity Chandra, Fortune and Lipton have investigated the algebraic structure of pre x operators and shown that constant depth and polynomial size cannot be achieved simultaneously if the semigroup contains a subgroup [ChFL83a]. These results cannot simply be translated to the average case because they do not hold under this setting. Consider the following simple example of an extension of the PARITY operator , which still contains the 2-element group as a subgroup. Let us call this function parity with reset, : f0; 1; rg ! f0; 1g , de ned by
(w) :=
(
if w = or w[jwj] = r
0
( (w[1; jwj ? 1]); w[jwj]) else.
An input value r resets the value of the current pre x to 0. Similarly, the classi cation of Bilardi, Preparata for constant depth and linear size pre x circuits based on the notion of monoidel cycles [BiPr90] does not extend to the average case. The second major step in our analysis of the problem is a nontrivial extension of the probabilistic restriction method to arbitrary nite domains. This way, we can deduce bounds on the error probabilities of averagecase delay bounded circuits with unbounded fanin gates from which the lower bounds for pre x functions of certain types will follow. The results are as follows. For parity and other pre x functions with a group like structure \embedded" in the semigroup (in a sense to be made precise below) the same lower bounds are obtained for the average complexity as in the worst case. For all other functions we establish constant average time complexity by polynomial size circuits. Thus, we obtain a strict separation with a large complexity gap from constant to (log n= loglog n). The paper is organized as follows. After introducing the necessary formal models and notation we state the results in a precise form. Then we outline the main techniques and arguments to obtain the lower and upper bounds. Due to space limitations proofs can only be sketched.
2 De nitions Let ; I ; O be nite alphabets, and let denote the empty word. In the following, I and O will typically be subsets of . For a string w := w[1]w[2] : : : 2 and for 1 i j jwj de ne w[i; j ] := w[i] : : : w[j ]. For easier notion de ne llog n := log log n . 2.1 Circuit Delay The depth of a circuit is a static and thus worst-case measure for the (parallel) time complexity of computing functions in the circuit model. In [JRS94] we have given a de nition of delay for the circuit model and used this notion to de ne an average-case measure for the time complexity. De nition 1 Let C be a circuit over he standard basis fOR; AND; NOTg , where the fanout of OR- and ANDgates may be bounded or unbounded. For input gates and constant gates v of C let timev (x) := 0 . For an internal nonconstant gate v with k direct predecessors v1 ; : : : ; vk we de ne
time (x) := 1 + minft j the values resvi (x) with timevi (x) t uniquely determine resv (x)g : v
For the circuit C itself with output gates y1 ; : : : ; ym the global time function is given by time (x) := maxi timeyi (x) : C
Average Case Complexity of Unbounded Fanin Circuits
3
In order to compute functions over arbitrary nite domains I in the Boolean circuit model, symbols x 2 I will be coded in a appropriate way, for example as binary vectors. A circuit C is said to compute a function f : nI ! mO if C transforms the corresponding code vectors in the appropriate way. For a function f : I ! O let UbCir(f ; s) denote the set of all circuits with unbounded fanin and fanout over the standard basis and size at most s that compute f restricted to nI , denoted by fn . A family of circuits of size s for a function f : I ! O is a sequence C = C1 ; C2 ; : : : of circuits with Cn computing fn We will analyse the average case complexity with respect to the family of uniform uni ?n probability distributions (uni n )n=1;2;::: , where n assigns probability jI j to every string over I of length n . De nition 2 For a circuit Cn for inputs from nI de ne the expected time, resp. the worst-case time as X uni timeCn (w) : n (w) timeCn (w) and time(C ) := wmax etime(C ) := 2n n
n
n
w2I n
I
If T is a function over the natural numbers and C a family of circuits we write
etime(C ) T
:()
etime(Cn ) T (n) 8n :
2.2 Pre x Functions De nition 3 A groupoid h; i consists of a set (in the following also called alphabet) and a binary operator : ! . The operator does not have to be associative. We extend the operator to strings of arbitrary length ` 1 . For ` > 1 and w 2 ` de ne
(w)
:= (: : : ((w[1] w[2]) w[3]) : : : w[`]) ;
while (w) := w for jwj = 1 . Often the rst symbol of a word over will have special importance. Thus for q 2 [ fg and w 2 ` , when considering the word q w we will also use the notation (w) := (qw) . Extending this to sets Q and X we de ne (Q; X ) := f q (w) j q 2 Q; w 2 X g : De nition 4 For a groupoid h; i and a subset I we de ne the n -ary parallel pre x functions PPh i I : nI ! n as follows. An input w = w[1] : : : w[n] is transformed into the vector q
n
;
;
( (w[1]); (w[1; 2]); : : : ; (w[1; n])) : If the domain I equals we simply write PPnh; i . A function f : I ! O is called a pre x function if there exists a groupoid h; i such that f restricted to nI is identical to PPnh; i;I . Note that this implicitly requires that jf (w)j = jwj for all arguments w of f and contains both I and O . The set of all pre x functions will be denoted by Fpre x . A groupoid h; i is called minimal for a pre x function f if for any pair q1 ; q2 2 there exists a string w 2 I such that q1 (w) 6= q2 (w) . In the following we will always assume that corresponding groupoids are minimal, which are unique up to isomorphisms. Let us denote such a minimal groupoid by h[ ]; [ ]i . If for a minimal groupoid h[f ]; [f ] i the corresponding pre x function f is de nite given by the context we will skip the indices. Groupoids can be represented by directed graphs which somehow resemble the transition diagrams of nite automata (also called Cayley machines [BiPr89]). The symbols in correspond to the states of the machine. f
f
Average Case Complexity of Unbounded Fanin Circuits
4
De nition 5 For a groupoid h; i , an input alphabet I de ne a transition graph Gh i I := (; E ) by 8q1; q2 2 : (q1; q2 ) 2 E :() 9q3 2 I : q1 q3 = q2 : ;
;
Furthermore, for a pre x function f let Gf denote the transition graph of a minimal groupoid for f . We call a groupoid h; i , resp. a pre x function f closed if Gh; i;I is strongly connected. The set of closed . all closed pre x functions will be denoted by Fpre x The di erence between this de nition of a transition graph and the concept of a Cayley graph is the absence of a distinguished starting state.
2.3 Con uences and Diuences Based on the groupoid h; i of a pre x function f and its transition graph Gf we will now de ne some basic properties, which will allow us to analyse the average-case complexity of f in the unbounded fanin circuit model. For t 2 IN, the set of nodes of Gf that can be reached in exactly t steps starting at node q 2 is denoted by R (t) := (q; tI ) Furthermore, for t 1 let R (t) := (; tI ). It is easy to see that these reachability sets share the following property: f;q
8 t1; t2 2 IN :
f
Rf;q (t1 ) = Rf;q (t2 )
()
Rf;q (t1 + i) = Rf;q (t2 + i) 8i 2 IN :
Since there are at most 2jj di erent possibilities for Rf;q (t) this property implies that there exist numbers ; 2 f1; 2; : : : ; 2jj g with Rf ( ) = Rf ( + i ) for all i 2 IN. Let us call the smallest such the start of periodicity, (f ) , and the smallest such the period, (f ) of f . In [Jak98a] it is shown that the start of the period has to occur earlier than the trivial counting argument above implies. Proposition 1 For every pre x function f 2 Fpre x the start of periodicity can be bounded by (f ) closed it holds in addition: If we augment the parallel pre x function by a symbol jj2 + jj . For f 2 Fpre x q 2 at the beginning, that is consider the function PPh; q i;I , then the periodicity e (f ) is independent of q and can be bounded by . The periodic or nonperiodic behavior of the transition graph turns out to be an important classi cation tool. De nition 6 For q1; q2 2 , a string w 2 I with q1 (w) = q2 (w) is called a con uence of these nodes. w 2 I is called a con uence for a subset 0 if j (0 ; w)j = 1 . f will be called con uent if Rf ( (f )) has a con uence. Let (f ) denote the minimal length of such con uences. Let Fconf denote the set of all con uent functions in Fpre x . A pre x function f is called strictly con uent if there exists some ` 2 IN such that each string w 2 `I is a con uence for Rf ( (f )) . De ne Fstr-conf as the set of all strictly con uent functions. For a subset 0 , a nonempty string w 2 I is a diuence if 0 contains two distinct nodes q1 ; q2 such that (fq1 ; q2 g; w) = fq1 ; q2 g . f is diuent if Rf ( (f )) has a diuence. A pre x function f is called strictly diuent if
9q1; q2 2 Rf ( (f )) 8w 2 I 9u 2 I : (fq1; q2 g; wu) = fq1; q2 g : De ne Fdi resp. Fstr-di as the set of all (strictly) diuent functions in Fpre x .
It can be shown that in the de nition of strictly con uent the value of ` can be bounded by jj2 =2. In [Jak98a, Jak98b] it proven that Fstr-conf , Fconf \ Fdi , and Fstr-di form a partition of Fpre x . Furthermore, we have sown in [JRSW94, Jak98a, Jak98b] that the con uence property of a pre x function f is strongly related to the average time complexity in bounded degree network and bounded fanin and
Average Case Complexity of Unbounded Fanin Circuits
5
bounded fanout circuit model. If we investigate circuit with bounded fanin and unbounded fanout we have to extend the notion of a con uence to the strongly connected components of the transition graph. A subset 0 is called a component of f if 0 is a maximal strongly connected component in Gf . For q 2 let Q[q] denotes the component that contains q . A component 0 is called closed if any node that is reachable from an element of 0 belongs to 0 . Let Q be the set of all elements of belonging to a closed component of f and Q := fQ[q]jq 2 Qf g . Using this notion we have subdivided Fpre x into three classes when considering the expected delay of circuits with bounded fanin and unbounded fanout analogously to the bounded fanout case [Jak98b]. f
f
3 Main Result For the model of circuits with bounded fanin it has been shown in [Jak98b]: Theorem A For any pre x function f 2 Fpre x the expected time for computing f in the bounded fanin and bounded fanout circuit model is either constant i f is strictly con uent, or of order llog jwj i f is con uent and diuent, or of logarithmic order i f is strictly diuent. Theorem B For any pre x function f 2 Fpre x the expected time for computing f in the bounded fanin and unbounded fanout circuit model is either constant i any closed component of f is strictly con uent, or of order llog jwj i every closed component f is con uent and diuent, or of logarithmic order i at least one closed component of f is strictly diuent. If we allow circuits with unbounded fanin and fanout the classi cations collapse such that only two cases are possible. As the main result of this work we will give an asymptotically tight classi cation of pre x functions according to the average time complexity of ub-circuits, i.e. Theorem 1 Let f 2 Fpre x and s n1+" for arbitrary " > 0 . Then it holds for every Cn;s 2 UbCir(fn; s) etime(Cn;s) =
(
(1); if every closed component of f is con uent; (log n= llog s); if there exists a strictly diuent closed component of f:
For pre x functions like PARITY this implies: Corollary 1 etime(PARITYn; POL) 2 (log n= llog n) . P The same also holds for every pre x sum mod k function with k 2 , which are de ned by ( `i=1 w[i]) 0 mod k . The proof of the lower bound for a strictly diuent pre x function f follows by an estimation of the number of inputs that leads to an error of an o(log = llog)-depth and polynomial size bounded ub-circuit computing f . For this we will extend the probabilistic restriction from the binary setting to arbitrarily large input alphabets. Before going into details we will make a simpli cation of the investigated set of functions. It is shown in [Jak98b] that for every pre x function f holds: if we choose independently and uniformly a string w = w[1] w[2] : : : w[`] over I , where each w[i] has equal probability for the symbols in I then the probability that (w) is not an element of a closed component decreases exponentially with ` . Hence, by an ub-circuit of constant depth and linear size considering only a pre x of w of logarithmic length we can determine the component Q[ (w)] with high probability, i.e. with a probability polynomial in ` . This leads to the following strategy for computing f (w): We partition w into a pre x of logarithmic length w1 and a sux w2 . In a rst step we will use a constant depth and linear size ub-circuit C 1 to determine f (w1 ). If (w1 ) is not an element of a closed component of f we use an standard circuit C 2 of logarithmic depth and linear size to compute f (w). Elsewhere, we can use an average ecient ub-circuit
Average Case Complexity of Unbounded Fanin Circuits
6
C 3 to compute the pre x function f 0(w2 ) over the groupoid h;
(w1 )i . Note that the expected delay of the resulting circuit C is given by C 2 ) + etime(C 3 ) 2 O(etime(C 3) + 1) etime(C` ) depth(C 1 ) + depth( `" for some " > 0. To simplify our analysis we will therefore restrict ourself in the following to closed pre x closed . functions Fpre x
4 Upper Bounds
For ` 2 IN de ne a `f -gate with inputs v0 ; : : : ; v` and outputs u0 ; : : : ; u` by resu0 (w) := ( ((resv1 (w) resv2 (w)) resv3 (w)) ) resv` (w) and for all i 2 f1; : : : ; `g resui (w) := ( ((resv0 (w) resv1 (w)) resv2 (w)) ) resvi (w) To achieve a minimal delay, we construct a circuit of c n=` gates of type `f as described in Figure 1. y[10; 12] y[1; 3] y[4; 6] y[7; 9] y[13; 14] w1 w2 w3 w4 w5 w6 w7 w8 w9 w10 w11 w12 w13 w14 e
v1 v2 v3 v0 u u1 u0 u3 2 v1 v2 v3 v0 u u1 u0 u3 2
v1 v2 v3 v0 u u1 u0 u3 2
v1 v2 v3 v0 u u1 u0 u3 2
v1 v2 v3 v0 u u1 u0 u3 2
v1 v2 v3 v0 u u1 u0 u3 2
v1 v2 v3 v0 u u1 u0 u3 2
v1 v2 v3 v0 u u1 u0 u3 2
Figure 1: An expected delay optimal 3f -circuit. Using the standard basis fOR; AND; NOTg of unbounded fanin, any `f -gate can be replaced by a subcircuit of depth 3 and size O( ` jI j`+1 ). Hence, the size of the resulting circuit is given by O( n jI j`+1 ). This implies: closed then there exists a family of conTheorem 2 For every " > 0 and s (n1+") holds if f 2 Fpre x structible ub-circuit C = C1 ; C2 ; : : : of size s for f with average delay ( closed ; O(1) if f 2 Fconf etime(Cn ) closed : O(log n= llog s) if f 2 Fstr-di The di erences in the average delay in these two cases is due to the di erences of the expected length of an interval of the input that do not contain a con uence. This value is of order logarithmic in the input length if f is con uent, and of linear order elsewhere.
5 Group Properties of Strictly Diuent Functions To investigate the average time complexity of pre x functions in the unbounded fanin circuit model in detail we have to discuss group-like properties of closed strict diuences. Let f be a closed pre x function with
Average Case Complexity of Unbounded Fanin Circuits
7
domain I and h; i be the corresponding groupoid. If (; ) is a group then obviously f is strictly diuent. In the following we will focus on some weaker properties that characterise strictly diuent functions. De nition 7 For k 2 IN we consider the k -dimensional space k . Its elements will be denoted by q[k] or hq1 ; : : : ; qk i , where qi 2 . q[k] will be called injektive if its components qi are pairwise di erent. For w 2 I de ne a transition function on k by
(hq1; : : : ; qk i; x) := h q1 (w); : : : ; qk (w)i : This induces a k -dimensional graph G := (k ; E k ) with k f
E k := f(q[k] ; (q[k]; x) j q[k] 2 k ; x 2 I g : Let u; w be strings in I . u will be called an inverse of (q[k]; w) if ( (q[k]; w); u) = q[k] . u is a loop string of q[k] if it is an inverse of (q[k]; ) , that is (q[k]; u) = q[k] . For a shift string u of a vector hq1; q2 ; : : : ; qk i we require that
(hq1; q2; : : : ; qk i; u) = hq2 ; : : : ; qk ; q1i : We call the groupoid h; i for f ` -robust if for some k 2 there exists a injective node q[k] = hq1; : : : ; qk i in Gkf such that 1.) q1 ; : : : ; qk 2 q (`I ) for some q 2 ,
2.) q[k] has a loop string and a shift string both of length ` , and 3.) for any w 2 I there exists a string u of length at most ` that is an inverse of (q[k]; w) , and furthermore, jw uj mod ` = 0 . ` -robustness of a groupoid is closely related to the property of having a subgroup. Since there are ` -robust groupoids without a neutral element there are ` -robust groupoids which are no groups. But, it is easy to see that each group is a 1-robust groupoid. To see that there are groupoids with a subgroup that are not ` -robust consider the con uent pre x function parity with reset de ned in the introduction. Note that the parity function without reset can trivially be embedded into this extension. Furthermore, hf0; 1; rg; i has a con uence r . But, hf0; 1; rg; i is not ` -robust for any ` 2 IN, i.e. there are no con uent ` -robust groupoids. On the other hand, each 1-robust groupoid contains a subgroup. Theorem 3 Every strictly diuent closed pre x function is ` -robust for some ` 2 IN . Proof: In the following we will successively construct a sequence hq1 ; : : : ; qk i of states ful lling the property ` -robustness for some ` 2 IN, that means hq1 ; : : : ; qk i 2 V k is an injective node of a closed component of Gkf with k 2. Furthermore, hq1 ; : : : ; qk i is chosen such that there exists a shift and a loop string of the same length. closed each subfunction h; q i with q 2 has In [Jak98a] it has been shown that for a function f 2 Fpre x the same period length e (f ) jj . Choose ` jj2 + 2 jj as the least multiple of e (f ) such that there exists a state q with q 2 Rf;q1 (`). Further, let D Rf;q (`) be the set of states which are pairwise strictly diuent to q . Then it holds: closed and all q1 2 with q1 2 Rf;q (`) there exists q2 2 Dq Lemma 1 For every function f 2 Fstr-di 1 1 j ` and two strings wloop ; wshift 2 I for some j 2 IN such that q1 = q1 (wloop ); q2 = q2 (wloop ) and q2 = q1 (wshift ) . Further, q2 can be chosen such that for any w 2 I there exists a inverse string of (hq1 ; q2 i; q) . q
Average Case Complexity of Unbounded Fanin Circuits
8
Proof: For w1 2 `I with q1 = q1 (w1 ) let Dw1 Dq1 be the set of states q2 such that w1c maps q2 to itself for some c 1. From the de nition of strict diuence and the set Dq1 it follows that Dw1 6= ; . Furthermore, we choose for q 2 Dw1 the value cq minimal such that q = q (w1cq ). Let clcm be the least common multiple of all these values cq and jj . For a string w 2 I with q1 = q1 (w) choose u 2 I such that q1 = q1 (wu) and jwuj mod ` = 0. Since each element q2 2 Dw1 is strictly diuent to q1 it holds that q2 (wuw1jj ) 2 Dw1 . So we can construct a directed graph Gw1 = (Dw1 ; Ew1 ) where
(q; q0 ) 2 Ew1
9i 2 IN 9w 2 iI` : q0 = q (w) and q1 = q1 (w) :
()
Let q be a node of a closed component of Gw1 and D 1 be the set of states of these component. Then for any strings w with jwj mod ` = 0 and q1 = q1 (w) it holds that q (ww1jj ) 2 Dw1 ;q : Note that a node hq1 ; q2 i of G2f is an element of a closed component of G2f if for all w 2 I there exists inverse string for hq1 ; q2 i and w . Hence, it follows that there exists an inverse string u for q and ww1jj where juj mod ` = 0 and q1 = q1 (u). The second part of the claim follows directly. Let q0 2 Dw1 ;q Dw1 Dq1 Rf;q1 (`) and w2 2 `I with q0 = q1 (w2 ). Furthermore, de ne wloop := w1clcm , wshift := w2 w1clcm?1 and q2 := q1 (wshift ). Hence, the rst part of the claim follows directly from the de nition of Dw1 ;q1 . Using q1 ; q2 as well as the strings wloop = w1clcm and wshift constructed in Lemma 1 we can successively [1] ; : : : ; w[m] generate a sequence hq1 ; q2 ; q3 ; : : : ; qm i of elements of as well as two sequences of strings wloop loop [ m ] [1] and wshift ; : : : ; wshift such that for all j 2 [1; m] the following properties holds: [j ] ). 1) For all i 2 [1; j ] it holds qi = qi (wloop [j ] ). 2) For all i 2 [1; j ? 1] it holds qi+1 = qi (wshift 3) There exists an inverse string u for hq1 ; q2 ; q3 ; : : : ; qm i 2 Vfm and each string w with jwuj mod ` = 0. [m] ). 4) There exists r 2 [1; m] with qr = qm (wshift This can be done analogously to the construction given in the proof of Lemma 1. Figure 2 illustrates the rst three conditions of the sequence hq1 ; q2 ; q3 ; q4 ; q5 ; q6 i with r = 3. w ;q
[2]
wloop
[1] wshift [1] wloop
q1
[3]
wloop
[2]
wshift q2
[4]
[3]
q3
= q7 [6]
q6
wloop
wshift
[6]
wshift
[5]
wshift
wloop q4 [4] wshift q5 [5] wloop
Figure 2: Sequence hq1 ; q2 ; q3 ; q4 ; q5 ; q6 i of a closed component of Gkf . This implies Theorem 3. closed choose `(f ) jj2 + 2 jj minimal such that f is ` -robust, i.e. there exists For a function f 2 Fstr-di [m] ; w[m] 2 `(f ) as constructed in the last a sequence hq1 ; : : : ; qm i as well as a loop and a shift string wloop I shift Theorem. Note that if the underlying groupoid of f is a semigroup we get `(f ) := jj2 + 2 jj . closed and an input w 2 de ne f (w) := f (w)[1 `(f )]f (w)[2 `(f )] : : : For a pre x function f 2 Fstr-di `(f ) I as the subfunction of f computing only the values of output positions that are multiples of `(f ). Note
Average Case Complexity of Unbounded Fanin Circuits
9
that f`(f ) is equivalent to a pre x function over the input alphabet 0I := I`(f ) . Furthermore, f`(f ) is 1 robust. On the other hand a lower bound of the expected delay of a ub-circuits computing f`(f ) gives also a lower bound for the expected delay of f . Therefore, we will restrict ourself to the following kind of pre x functions: closed where the transition graph De nition 8 Let Frobust be the set of all 1 -robust pre x functions f 2 Fstr-di of f is a complete graph (i.e. for all pairs q; q0 2 there exists a symbol x 2 I such that q = q0 (x) ). If we restrict ourself to pre x functions f with are de ned over an associative operator it can easily be shown that f`(f ) 2 Frobust .
6 Probabilistic Restriction for Arbitrary Alphabets { Error Bounds for Depth-Bounded Circuits In the following we will calculate an upper bound on the number of inputs for which a small depth ub-circuit gives correct result for a given pre x function f 2 Fsi-str-di . Since the standard probabilistic restriction method works only on circuits over a binary input alphabet it cannot be used directly to estimate the desired quantity. More precisely, using the calculations of Hastad (see [Hast86b]), for depth d ub-circuit of polynomial size with d 2 o(log = llog) the number of inputs with a wrong result can only be bounded by 1 1=d ) n ?
(( n= log n ) jj jj2 ? 2 jjn?2 =2 : If jj is not anpower of 2 this approximation implies a lower bound for the error probability of at most jj 1 2jj2 jj+1 . Note that this bound tends towards zero for growing n . Hence, this estimation cannot be used to give a meaningful bound for the expected delay of an ub-circuit in general. Therefore, we will introduce a generalisation of the probabilistic restriction method: De nition 9 For 0 p 1 , the probabilistic restriction over the alphabet , [p; ] sets the input variables Bi independently to the following values: Pr[[p; I ](Bi ) = ?] = p and 8x 2 I : Pr[[p; I ](Bi ) = x] = 1 ? p : I 0 n In the following we will restrict ourself to functions f : I ! f0; 1g which decide for a given pre x function f , an input w 2 nI , and an symbol x 2 whether f (w)[n] = x , that means whether the last value of f (w) is x or not. For a circuit Cn with n inputs and a string w 2 (I [f?g)n de ne C d as the simpli ed circuit (resp. the function computd by the circuit) for which all inputs vi with w[i] 6= ? are determined by w[i]. To consider arbitrary input alphabets I we will use special input gates [B 2 Z ] for an input variable B and a subset Z I which are evaluated to 1 i B 2 Z . A conjunction resp. disjunction of the variables B1 ; : : : ; Bn is a term of type ^ _ K := [Bi (j ) 2 Zi;j ]; resp. D := [Bi (j ) 2 Zi;j ] I
I
w
i
j 2f1;:::;jKijg
i
j 2f1;:::;jDijg
with Z I . A conjunction Ki is called a conjunctive min-term if 8w 2 nI : Ki (w) = 1 ! f (w) = 1 and 8 k 2 [1; jKi j] 9 w 2 nI : f (w) = 0 ^ Ki;k (w) = 1 ; V where Ki;k := j 2f1;:::;jKijg and j 6=k [Bi (j ) 2 Zi;j ] : A disjunctive min-term is de ned analogously. Further, min(C ) denotes the maximum number of variables of a conjunctive min-term of C . Analogously to [Hast86b] it can be shown: i;j
Average Case Complexity of Unbounded Fanin Circuits
10
Lemma 2 Let C = Vki=1 Di where Di denotes a disjunction of at most t variables. Further let F : I ! f0; 1g be an arbitrary function and [p; I ] a probabilistic restriction. Then it holds Pr[min(C d[p;I ] ) s j F d[p;I ] 1] s ; where is the unique positive solution of !t !t 2p j j j j p 1 1 I I 1 + 1 + (j j ? 1) p = 1 + 1 + (j j ? 1) p + 1 : I I
Investigating F d[p;I ] 0 instead of F d[p;I ] 1 the same technique can be used to transform a disjunction of conjunctions into a conjunction of disjunctions. The value of can be estimated as following inequality: Lemma 3 The value of can be bounded by:
t (jI j ln 2 ? 1) jI j p < t jI j2 p (1 + (jI j ? 1) p) ln 2 (1 + (jI j ? 1) p) ln 2 : For a gate v of C de ne accept(v) as the set of inputs w 2 nI such that resv (w) = 1. Furthermore, let accept(C ) := accept(v) where v denotes the output gate of C . If we allow arbitrary conjunctions for C d (not only conjunctive min-terms) we can improve Lemma 2 to: Corollary 2 Let C = Vki=1 Di where Di denotes a disjunction of at most t variables. Further, let be a probabilistic restriction of Rp;I . Then the probability that C d can be represented by a disjunction of conjunctions Kj with maximal degree s is at least 1 ? s . Furthermore, this conjunction can be chosen such that the sets accept(Ki ) are pairwise disjoint. Using a probabilistic restriction Rp;I and the robustness of strictly diuent pre x functions we can show an upper bound on the maximum rank of correctness of a d -depth and s -size bounded ub-circuit C computing a pre x function f on inputs of length n . We will denote this value by E (f; d; s; n) , i.e. there exists at most E (f; d; s; n) jI jn inputs w 2 nI with C (w) = f (w). last it holds Theorem 4 For f 2 Frobust
E (f; d; s; n)
jI j2 ? 1 jI j2
+
where
8 > > > >
jI j? (n=(logjI j s) ) if logjI j s > pd n; > > > : j j? ((n= logjI j s)1=d ) if log s < p d n: I jI j
7 Lower Bounds closed and all bounds d , s , and n the value of d (1 ? E (f; d; s; n)) gives Note that for any function f 2 Fpre x a lower bound for the expected delay of an ub-circuit computing f . Hence it follows from Theorem 4: closed then it holds for any s Lemma 4 Let f 2 Fstr-di log n etime(UbCir(fn ; s)) llog s :
Proof: We will use a technique presented in [JaRS95]. Any family of circuits C1 ; C2 ; : : : of size s computing f within an expected delay of t can be transformed into a depth t bounded family of circuits C1[t]; C2[t] ; : : : of size at most quadratic in s computing f with a decreasing error rate. This implies 1 ?E (f; t; s; n) 2 o(1).
Average Case Complexity of Unbounded Fanin Circuits
11
Considering the value of this means that we have to choose a bound t for the expected delay such that tends to 1=jI j2 for growing n , resp. > 1=jI j3 for almost all n 2 IN. After someparithmetic transformations we get the following cases: If t = 2 then 3 (n= logjI j s). If logjI j s > t n it holds p that 3 (n= logtj?I1j s) and if logjI j s < t n then 3 ((n= logjI j s)1=t ). For the rst case it follows that s is exponential in n . From the precondition of the second case, i.e. p t logjI j s > n , it follows that t 2 llog s?logllogn jI j + (1). For the third case we get
p
3 2 ((n= logjI j s)1=t ) (n(t?1)=t2 ) =) 9 2 ( t n) =) t 2 log n + (1) : The third case implies that s has to be a constant { a contradiction. So, only the rst and the second case can occurs and the claim follows directly. Note that Theorem 1 follows by combining the lemma above with the show upper bounds.
References [BiPr89] [BiPr90] [ChFL83a] [ChFL83b] [FuSS84] [Hast86a] [Hast86b] [Jak98a] [Jak98b] [JRS94] [JaRS95] [JRSW94] [LaFi80] [Ya85]
G. Bilardi, F. Preparata, Size-Time Complexity of Boolean Networks for Pre x Computations, J. of the ACM 36, 1989, 362-382. G. Bilardi, F. Preparata, Characterization of Associative Operations with Pre x Circuits of Constant Depth and Linear Size, SIAM J. Comput 19, 1990, 246-255. A. Chandra, S. Fortune, R. Lipton, Unbounded Fan-in Circuits and Associative Functions, Proc. 15. SToC'83, 1983, 52-60. A. Chandra, S. Fortune, R. Lipton, Lower Bounds for Constant Depth Circuits for Pre x Problems, Proc. 10. ICALP'83, 1983, 109-116. M. Furst, J. B. Saxe, M. Sipser, Parity, Circuits, and the Polynomial-Time Hierarchy, Proc. 22. FoCS'81, 1981, 260-270, and Math. Systems Theory 17, 1984, 13-27. J. Hastad, Almost Optimal Lower Bounds for Small Depth Circuits, Proc. 18. SToC'86, 1986, 6-20. J. Hastad, Computational Limitations for Small-Depth Circuits, ACM Doctoral Dissertation Award 1986, MIT Press, 1986. A. Jakoby, Die Komplexitat von Pra xfunktionen bezuglich ihres mittleren Zeitverhaltens, Dissertation, Med. Universitat zu Lubeck, 1998. A. Jakoby, The Average Time Complexity to Compute Pre x Functions in Processor Networks, Technical Report, Med. Universitat zu Lubeck, 1998. A. Jakoby, R. Reischuk, C. Schindelhauer, Circuit Complexity: from the Worst Case to the Average Case, Proc. 26. ACM Symposium on the Theory of Computing, 1994, 58-67. A. Jakoby, R. Reischuk, C. Schindelhauer, Malign Distributions for Average Case Circuit Complexity, Proc. 12. STACS'95, 1995, 628-639. A. Jakoby, R. Reischuk, C. Schindelhauer, S. Weis, The Average Case Complexity of the Parallel Pre x Problem, Proc. 21. ICALP'94, 1994, 593-604. R. Ladner, M. Fischer, Parallel Pre x Computation, J. of the ACM 27, 1980, 831-838. A. Yao, Separating the Polynomial Time Hierarchy by Oracles, Proc. 26. FoCS, 1985, 1-10.
Abstract
Hastad has shown that functions like PARITY cannot be computed by unbounded fanin circuits of small depth and polynomial size. We generalize this result in two directions. First, we obtain the same tight lower bound for the average case. This is done by estimating the average delay { the natural generalization of circuit depth to an average case measure { of unbounded fanin circuits of polynomial size, resp. their error probability given an upper bound on the maximal delay. These bounds are obtained by extending the probabilistic restriction method to an average case setting. Secondly, we completely classify the set of parallel pre x functions { for which PARITY is just one example { with respect to their average delay in unbounded fanin circuits of a given size. It is shown that only two cases can occur: a parallel pre x functions either has the same complexity as PARITY, that is the average delay has to be of order (log n= log log s) for circuit of size s , or it can be computed with constant average delay and almost linear size { there is nothing in between. This classi cation is achieved by analyzing the algebraic structure of the semigroups that correspond to parallel pre x functions. It extends methods developed by the rst author in his Ph.D. Thesis.
1 Introduction To analyse the average-case complexity of Boolean functions we have introduced an average-case time complexity measure for Boolean circuits based on the notion of delay in [JRS94]. For the classical model with OR, AND and NOT gates of bounded fanin and fanout we have shown that n -ary functions like the OR, the addition or threshold functions with a xed threshold can be computed much faster on the average compared to the worst-case for which the trivial log n lower bound holds. Other functions like PARITY do not allow a signi cant speedup of the average time complexity, regardless of the size of the circuits. In [JRSW94] we have generalized our investigations to computing arbitrary pre x functions. Tight bounds for the average complexity have been obtained by analysing the algebraic structure of the semigroups that correspond to the pre x operator. These results have been extended in [Jak98a, Jak98b] to parallel pre x computations in other computational models like linear arrays and networks of automata. A di erent line of research has considered the complexity of Boolean functions in a circuit model where gates of arbitrary large fanin can be used. It is well known that in this model every Boolean function can be computed in constant depth if one allows circuits of exponential size. Functions like the OR or the addition again can be computed much more ecient, that is in constant depth and polynomial size [ChFL83a]. But PARITY remains dicult { the best speedup for polynomial size one can obtain is depth (log n= loglog n) [Ya85]. Hastad has proven optimal lower size bounds with respect to the circuit depth by analysing the probabilistic restriction method very carefully. In this paper we combine both questions and asks whether PARITY can be computed more eciently by unbounded fanin circuits on the average. At the same time we try to classify Boolean functions according to their average case complexity in the unbounded fanin model. Our work starts by investigating algebraic 1
Average Case Complexity of Unbounded Fanin Circuits
2
properties of the corresponding semigroups in a similar spirit as in [JRSW94, Jak98a] now tailored to unbounded fanin computations. For the worst-case complexity Chandra, Fortune and Lipton have investigated the algebraic structure of pre x operators and shown that constant depth and polynomial size cannot be achieved simultaneously if the semigroup contains a subgroup [ChFL83a]. These results cannot simply be translated to the average case because they do not hold under this setting. Consider the following simple example of an extension of the PARITY operator , which still contains the 2-element group as a subgroup. Let us call this function parity with reset, : f0; 1; rg ! f0; 1g , de ned by
(w) :=
(
if w = or w[jwj] = r
0
( (w[1; jwj ? 1]); w[jwj]) else.
An input value r resets the value of the current pre x to 0. Similarly, the classi cation of Bilardi, Preparata for constant depth and linear size pre x circuits based on the notion of monoidel cycles [BiPr90] does not extend to the average case. The second major step in our analysis of the problem is a nontrivial extension of the probabilistic restriction method to arbitrary nite domains. This way, we can deduce bounds on the error probabilities of averagecase delay bounded circuits with unbounded fanin gates from which the lower bounds for pre x functions of certain types will follow. The results are as follows. For parity and other pre x functions with a group like structure \embedded" in the semigroup (in a sense to be made precise below) the same lower bounds are obtained for the average complexity as in the worst case. For all other functions we establish constant average time complexity by polynomial size circuits. Thus, we obtain a strict separation with a large complexity gap from constant to (log n= loglog n). The paper is organized as follows. After introducing the necessary formal models and notation we state the results in a precise form. Then we outline the main techniques and arguments to obtain the lower and upper bounds. Due to space limitations proofs can only be sketched.
2 De nitions Let ; I ; O be nite alphabets, and let denote the empty word. In the following, I and O will typically be subsets of . For a string w := w[1]w[2] : : : 2 and for 1 i j jwj de ne w[i; j ] := w[i] : : : w[j ]. For easier notion de ne llog n := log log n . 2.1 Circuit Delay The depth of a circuit is a static and thus worst-case measure for the (parallel) time complexity of computing functions in the circuit model. In [JRS94] we have given a de nition of delay for the circuit model and used this notion to de ne an average-case measure for the time complexity. De nition 1 Let C be a circuit over he standard basis fOR; AND; NOTg , where the fanout of OR- and ANDgates may be bounded or unbounded. For input gates and constant gates v of C let timev (x) := 0 . For an internal nonconstant gate v with k direct predecessors v1 ; : : : ; vk we de ne
time (x) := 1 + minft j the values resvi (x) with timevi (x) t uniquely determine resv (x)g : v
For the circuit C itself with output gates y1 ; : : : ; ym the global time function is given by time (x) := maxi timeyi (x) : C
Average Case Complexity of Unbounded Fanin Circuits
3
In order to compute functions over arbitrary nite domains I in the Boolean circuit model, symbols x 2 I will be coded in a appropriate way, for example as binary vectors. A circuit C is said to compute a function f : nI ! mO if C transforms the corresponding code vectors in the appropriate way. For a function f : I ! O let UbCir(f ; s) denote the set of all circuits with unbounded fanin and fanout over the standard basis and size at most s that compute f restricted to nI , denoted by fn . A family of circuits of size s for a function f : I ! O is a sequence C = C1 ; C2 ; : : : of circuits with Cn computing fn We will analyse the average case complexity with respect to the family of uniform uni ?n probability distributions (uni n )n=1;2;::: , where n assigns probability jI j to every string over I of length n . De nition 2 For a circuit Cn for inputs from nI de ne the expected time, resp. the worst-case time as X uni timeCn (w) : n (w) timeCn (w) and time(C ) := wmax etime(C ) := 2n n
n
n
w2I n
I
If T is a function over the natural numbers and C a family of circuits we write
etime(C ) T
:()
etime(Cn ) T (n) 8n :
2.2 Pre x Functions De nition 3 A groupoid h; i consists of a set (in the following also called alphabet) and a binary operator : ! . The operator does not have to be associative. We extend the operator to strings of arbitrary length ` 1 . For ` > 1 and w 2 ` de ne
(w)
:= (: : : ((w[1] w[2]) w[3]) : : : w[`]) ;
while (w) := w for jwj = 1 . Often the rst symbol of a word over will have special importance. Thus for q 2 [ fg and w 2 ` , when considering the word q w we will also use the notation (w) := (qw) . Extending this to sets Q and X we de ne (Q; X ) := f q (w) j q 2 Q; w 2 X g : De nition 4 For a groupoid h; i and a subset I we de ne the n -ary parallel pre x functions PPh i I : nI ! n as follows. An input w = w[1] : : : w[n] is transformed into the vector q
n
;
;
( (w[1]); (w[1; 2]); : : : ; (w[1; n])) : If the domain I equals we simply write PPnh; i . A function f : I ! O is called a pre x function if there exists a groupoid h; i such that f restricted to nI is identical to PPnh; i;I . Note that this implicitly requires that jf (w)j = jwj for all arguments w of f and contains both I and O . The set of all pre x functions will be denoted by Fpre x . A groupoid h; i is called minimal for a pre x function f if for any pair q1 ; q2 2 there exists a string w 2 I such that q1 (w) 6= q2 (w) . In the following we will always assume that corresponding groupoids are minimal, which are unique up to isomorphisms. Let us denote such a minimal groupoid by h[ ]; [ ]i . If for a minimal groupoid h[f ]; [f ] i the corresponding pre x function f is de nite given by the context we will skip the indices. Groupoids can be represented by directed graphs which somehow resemble the transition diagrams of nite automata (also called Cayley machines [BiPr89]). The symbols in correspond to the states of the machine. f
f
Average Case Complexity of Unbounded Fanin Circuits
4
De nition 5 For a groupoid h; i , an input alphabet I de ne a transition graph Gh i I := (; E ) by 8q1; q2 2 : (q1; q2 ) 2 E :() 9q3 2 I : q1 q3 = q2 : ;
;
Furthermore, for a pre x function f let Gf denote the transition graph of a minimal groupoid for f . We call a groupoid h; i , resp. a pre x function f closed if Gh; i;I is strongly connected. The set of closed . all closed pre x functions will be denoted by Fpre x The di erence between this de nition of a transition graph and the concept of a Cayley graph is the absence of a distinguished starting state.
2.3 Con uences and Diuences Based on the groupoid h; i of a pre x function f and its transition graph Gf we will now de ne some basic properties, which will allow us to analyse the average-case complexity of f in the unbounded fanin circuit model. For t 2 IN, the set of nodes of Gf that can be reached in exactly t steps starting at node q 2 is denoted by R (t) := (q; tI ) Furthermore, for t 1 let R (t) := (; tI ). It is easy to see that these reachability sets share the following property: f;q
8 t1; t2 2 IN :
f
Rf;q (t1 ) = Rf;q (t2 )
()
Rf;q (t1 + i) = Rf;q (t2 + i) 8i 2 IN :
Since there are at most 2jj di erent possibilities for Rf;q (t) this property implies that there exist numbers ; 2 f1; 2; : : : ; 2jj g with Rf ( ) = Rf ( + i ) for all i 2 IN. Let us call the smallest such the start of periodicity, (f ) , and the smallest such the period, (f ) of f . In [Jak98a] it is shown that the start of the period has to occur earlier than the trivial counting argument above implies. Proposition 1 For every pre x function f 2 Fpre x the start of periodicity can be bounded by (f ) closed it holds in addition: If we augment the parallel pre x function by a symbol jj2 + jj . For f 2 Fpre x q 2 at the beginning, that is consider the function PPh; q i;I , then the periodicity e (f ) is independent of q and can be bounded by . The periodic or nonperiodic behavior of the transition graph turns out to be an important classi cation tool. De nition 6 For q1; q2 2 , a string w 2 I with q1 (w) = q2 (w) is called a con uence of these nodes. w 2 I is called a con uence for a subset 0 if j (0 ; w)j = 1 . f will be called con uent if Rf ( (f )) has a con uence. Let (f ) denote the minimal length of such con uences. Let Fconf denote the set of all con uent functions in Fpre x . A pre x function f is called strictly con uent if there exists some ` 2 IN such that each string w 2 `I is a con uence for Rf ( (f )) . De ne Fstr-conf as the set of all strictly con uent functions. For a subset 0 , a nonempty string w 2 I is a diuence if 0 contains two distinct nodes q1 ; q2 such that (fq1 ; q2 g; w) = fq1 ; q2 g . f is diuent if Rf ( (f )) has a diuence. A pre x function f is called strictly diuent if
9q1; q2 2 Rf ( (f )) 8w 2 I 9u 2 I : (fq1; q2 g; wu) = fq1; q2 g : De ne Fdi resp. Fstr-di as the set of all (strictly) diuent functions in Fpre x .
It can be shown that in the de nition of strictly con uent the value of ` can be bounded by jj2 =2. In [Jak98a, Jak98b] it proven that Fstr-conf , Fconf \ Fdi , and Fstr-di form a partition of Fpre x . Furthermore, we have sown in [JRSW94, Jak98a, Jak98b] that the con uence property of a pre x function f is strongly related to the average time complexity in bounded degree network and bounded fanin and
Average Case Complexity of Unbounded Fanin Circuits
5
bounded fanout circuit model. If we investigate circuit with bounded fanin and unbounded fanout we have to extend the notion of a con uence to the strongly connected components of the transition graph. A subset 0 is called a component of f if 0 is a maximal strongly connected component in Gf . For q 2 let Q[q] denotes the component that contains q . A component 0 is called closed if any node that is reachable from an element of 0 belongs to 0 . Let Q be the set of all elements of belonging to a closed component of f and Q := fQ[q]jq 2 Qf g . Using this notion we have subdivided Fpre x into three classes when considering the expected delay of circuits with bounded fanin and unbounded fanout analogously to the bounded fanout case [Jak98b]. f
f
3 Main Result For the model of circuits with bounded fanin it has been shown in [Jak98b]: Theorem A For any pre x function f 2 Fpre x the expected time for computing f in the bounded fanin and bounded fanout circuit model is either constant i f is strictly con uent, or of order llog jwj i f is con uent and diuent, or of logarithmic order i f is strictly diuent. Theorem B For any pre x function f 2 Fpre x the expected time for computing f in the bounded fanin and unbounded fanout circuit model is either constant i any closed component of f is strictly con uent, or of order llog jwj i every closed component f is con uent and diuent, or of logarithmic order i at least one closed component of f is strictly diuent. If we allow circuits with unbounded fanin and fanout the classi cations collapse such that only two cases are possible. As the main result of this work we will give an asymptotically tight classi cation of pre x functions according to the average time complexity of ub-circuits, i.e. Theorem 1 Let f 2 Fpre x and s n1+" for arbitrary " > 0 . Then it holds for every Cn;s 2 UbCir(fn; s) etime(Cn;s) =
(
(1); if every closed component of f is con uent; (log n= llog s); if there exists a strictly diuent closed component of f:
For pre x functions like PARITY this implies: Corollary 1 etime(PARITYn; POL) 2 (log n= llog n) . P The same also holds for every pre x sum mod k function with k 2 , which are de ned by ( `i=1 w[i]) 0 mod k . The proof of the lower bound for a strictly diuent pre x function f follows by an estimation of the number of inputs that leads to an error of an o(log = llog)-depth and polynomial size bounded ub-circuit computing f . For this we will extend the probabilistic restriction from the binary setting to arbitrarily large input alphabets. Before going into details we will make a simpli cation of the investigated set of functions. It is shown in [Jak98b] that for every pre x function f holds: if we choose independently and uniformly a string w = w[1] w[2] : : : w[`] over I , where each w[i] has equal probability for the symbols in I then the probability that (w) is not an element of a closed component decreases exponentially with ` . Hence, by an ub-circuit of constant depth and linear size considering only a pre x of w of logarithmic length we can determine the component Q[ (w)] with high probability, i.e. with a probability polynomial in ` . This leads to the following strategy for computing f (w): We partition w into a pre x of logarithmic length w1 and a sux w2 . In a rst step we will use a constant depth and linear size ub-circuit C 1 to determine f (w1 ). If (w1 ) is not an element of a closed component of f we use an standard circuit C 2 of logarithmic depth and linear size to compute f (w). Elsewhere, we can use an average ecient ub-circuit
Average Case Complexity of Unbounded Fanin Circuits
6
C 3 to compute the pre x function f 0(w2 ) over the groupoid h;
(w1 )i . Note that the expected delay of the resulting circuit C is given by C 2 ) + etime(C 3 ) 2 O(etime(C 3) + 1) etime(C` ) depth(C 1 ) + depth( `" for some " > 0. To simplify our analysis we will therefore restrict ourself in the following to closed pre x closed . functions Fpre x
4 Upper Bounds
For ` 2 IN de ne a `f -gate with inputs v0 ; : : : ; v` and outputs u0 ; : : : ; u` by resu0 (w) := ( ((resv1 (w) resv2 (w)) resv3 (w)) ) resv` (w) and for all i 2 f1; : : : ; `g resui (w) := ( ((resv0 (w) resv1 (w)) resv2 (w)) ) resvi (w) To achieve a minimal delay, we construct a circuit of c n=` gates of type `f as described in Figure 1. y[10; 12] y[1; 3] y[4; 6] y[7; 9] y[13; 14] w1 w2 w3 w4 w5 w6 w7 w8 w9 w10 w11 w12 w13 w14 e
v1 v2 v3 v0 u u1 u0 u3 2 v1 v2 v3 v0 u u1 u0 u3 2
v1 v2 v3 v0 u u1 u0 u3 2
v1 v2 v3 v0 u u1 u0 u3 2
v1 v2 v3 v0 u u1 u0 u3 2
v1 v2 v3 v0 u u1 u0 u3 2
v1 v2 v3 v0 u u1 u0 u3 2
v1 v2 v3 v0 u u1 u0 u3 2
Figure 1: An expected delay optimal 3f -circuit. Using the standard basis fOR; AND; NOTg of unbounded fanin, any `f -gate can be replaced by a subcircuit of depth 3 and size O( ` jI j`+1 ). Hence, the size of the resulting circuit is given by O( n jI j`+1 ). This implies: closed then there exists a family of conTheorem 2 For every " > 0 and s (n1+") holds if f 2 Fpre x structible ub-circuit C = C1 ; C2 ; : : : of size s for f with average delay ( closed ; O(1) if f 2 Fconf etime(Cn ) closed : O(log n= llog s) if f 2 Fstr-di The di erences in the average delay in these two cases is due to the di erences of the expected length of an interval of the input that do not contain a con uence. This value is of order logarithmic in the input length if f is con uent, and of linear order elsewhere.
5 Group Properties of Strictly Diuent Functions To investigate the average time complexity of pre x functions in the unbounded fanin circuit model in detail we have to discuss group-like properties of closed strict diuences. Let f be a closed pre x function with
Average Case Complexity of Unbounded Fanin Circuits
7
domain I and h; i be the corresponding groupoid. If (; ) is a group then obviously f is strictly diuent. In the following we will focus on some weaker properties that characterise strictly diuent functions. De nition 7 For k 2 IN we consider the k -dimensional space k . Its elements will be denoted by q[k] or hq1 ; : : : ; qk i , where qi 2 . q[k] will be called injektive if its components qi are pairwise di erent. For w 2 I de ne a transition function on k by
(hq1; : : : ; qk i; x) := h q1 (w); : : : ; qk (w)i : This induces a k -dimensional graph G := (k ; E k ) with k f
E k := f(q[k] ; (q[k]; x) j q[k] 2 k ; x 2 I g : Let u; w be strings in I . u will be called an inverse of (q[k]; w) if ( (q[k]; w); u) = q[k] . u is a loop string of q[k] if it is an inverse of (q[k]; ) , that is (q[k]; u) = q[k] . For a shift string u of a vector hq1; q2 ; : : : ; qk i we require that
(hq1; q2; : : : ; qk i; u) = hq2 ; : : : ; qk ; q1i : We call the groupoid h; i for f ` -robust if for some k 2 there exists a injective node q[k] = hq1; : : : ; qk i in Gkf such that 1.) q1 ; : : : ; qk 2 q (`I ) for some q 2 ,
2.) q[k] has a loop string and a shift string both of length ` , and 3.) for any w 2 I there exists a string u of length at most ` that is an inverse of (q[k]; w) , and furthermore, jw uj mod ` = 0 . ` -robustness of a groupoid is closely related to the property of having a subgroup. Since there are ` -robust groupoids without a neutral element there are ` -robust groupoids which are no groups. But, it is easy to see that each group is a 1-robust groupoid. To see that there are groupoids with a subgroup that are not ` -robust consider the con uent pre x function parity with reset de ned in the introduction. Note that the parity function without reset can trivially be embedded into this extension. Furthermore, hf0; 1; rg; i has a con uence r . But, hf0; 1; rg; i is not ` -robust for any ` 2 IN, i.e. there are no con uent ` -robust groupoids. On the other hand, each 1-robust groupoid contains a subgroup. Theorem 3 Every strictly diuent closed pre x function is ` -robust for some ` 2 IN . Proof: In the following we will successively construct a sequence hq1 ; : : : ; qk i of states ful lling the property ` -robustness for some ` 2 IN, that means hq1 ; : : : ; qk i 2 V k is an injective node of a closed component of Gkf with k 2. Furthermore, hq1 ; : : : ; qk i is chosen such that there exists a shift and a loop string of the same length. closed each subfunction h; q i with q 2 has In [Jak98a] it has been shown that for a function f 2 Fpre x the same period length e (f ) jj . Choose ` jj2 + 2 jj as the least multiple of e (f ) such that there exists a state q with q 2 Rf;q1 (`). Further, let D Rf;q (`) be the set of states which are pairwise strictly diuent to q . Then it holds: closed and all q1 2 with q1 2 Rf;q (`) there exists q2 2 Dq Lemma 1 For every function f 2 Fstr-di 1 1 j ` and two strings wloop ; wshift 2 I for some j 2 IN such that q1 = q1 (wloop ); q2 = q2 (wloop ) and q2 = q1 (wshift ) . Further, q2 can be chosen such that for any w 2 I there exists a inverse string of (hq1 ; q2 i; q) . q
Average Case Complexity of Unbounded Fanin Circuits
8
Proof: For w1 2 `I with q1 = q1 (w1 ) let Dw1 Dq1 be the set of states q2 such that w1c maps q2 to itself for some c 1. From the de nition of strict diuence and the set Dq1 it follows that Dw1 6= ; . Furthermore, we choose for q 2 Dw1 the value cq minimal such that q = q (w1cq ). Let clcm be the least common multiple of all these values cq and jj . For a string w 2 I with q1 = q1 (w) choose u 2 I such that q1 = q1 (wu) and jwuj mod ` = 0. Since each element q2 2 Dw1 is strictly diuent to q1 it holds that q2 (wuw1jj ) 2 Dw1 . So we can construct a directed graph Gw1 = (Dw1 ; Ew1 ) where
(q; q0 ) 2 Ew1
9i 2 IN 9w 2 iI` : q0 = q (w) and q1 = q1 (w) :
()
Let q be a node of a closed component of Gw1 and D 1 be the set of states of these component. Then for any strings w with jwj mod ` = 0 and q1 = q1 (w) it holds that q (ww1jj ) 2 Dw1 ;q : Note that a node hq1 ; q2 i of G2f is an element of a closed component of G2f if for all w 2 I there exists inverse string for hq1 ; q2 i and w . Hence, it follows that there exists an inverse string u for q and ww1jj where juj mod ` = 0 and q1 = q1 (u). The second part of the claim follows directly. Let q0 2 Dw1 ;q Dw1 Dq1 Rf;q1 (`) and w2 2 `I with q0 = q1 (w2 ). Furthermore, de ne wloop := w1clcm , wshift := w2 w1clcm?1 and q2 := q1 (wshift ). Hence, the rst part of the claim follows directly from the de nition of Dw1 ;q1 . Using q1 ; q2 as well as the strings wloop = w1clcm and wshift constructed in Lemma 1 we can successively [1] ; : : : ; w[m] generate a sequence hq1 ; q2 ; q3 ; : : : ; qm i of elements of as well as two sequences of strings wloop loop [ m ] [1] and wshift ; : : : ; wshift such that for all j 2 [1; m] the following properties holds: [j ] ). 1) For all i 2 [1; j ] it holds qi = qi (wloop [j ] ). 2) For all i 2 [1; j ? 1] it holds qi+1 = qi (wshift 3) There exists an inverse string u for hq1 ; q2 ; q3 ; : : : ; qm i 2 Vfm and each string w with jwuj mod ` = 0. [m] ). 4) There exists r 2 [1; m] with qr = qm (wshift This can be done analogously to the construction given in the proof of Lemma 1. Figure 2 illustrates the rst three conditions of the sequence hq1 ; q2 ; q3 ; q4 ; q5 ; q6 i with r = 3. w ;q
[2]
wloop
[1] wshift [1] wloop
q1
[3]
wloop
[2]
wshift q2
[4]
[3]
q3
= q7 [6]
q6
wloop
wshift
[6]
wshift
[5]
wshift
wloop q4 [4] wshift q5 [5] wloop
Figure 2: Sequence hq1 ; q2 ; q3 ; q4 ; q5 ; q6 i of a closed component of Gkf . This implies Theorem 3. closed choose `(f ) jj2 + 2 jj minimal such that f is ` -robust, i.e. there exists For a function f 2 Fstr-di [m] ; w[m] 2 `(f ) as constructed in the last a sequence hq1 ; : : : ; qm i as well as a loop and a shift string wloop I shift Theorem. Note that if the underlying groupoid of f is a semigroup we get `(f ) := jj2 + 2 jj . closed and an input w 2 de ne f (w) := f (w)[1 `(f )]f (w)[2 `(f )] : : : For a pre x function f 2 Fstr-di `(f ) I as the subfunction of f computing only the values of output positions that are multiples of `(f ). Note
Average Case Complexity of Unbounded Fanin Circuits
9
that f`(f ) is equivalent to a pre x function over the input alphabet 0I := I`(f ) . Furthermore, f`(f ) is 1 robust. On the other hand a lower bound of the expected delay of a ub-circuits computing f`(f ) gives also a lower bound for the expected delay of f . Therefore, we will restrict ourself to the following kind of pre x functions: closed where the transition graph De nition 8 Let Frobust be the set of all 1 -robust pre x functions f 2 Fstr-di of f is a complete graph (i.e. for all pairs q; q0 2 there exists a symbol x 2 I such that q = q0 (x) ). If we restrict ourself to pre x functions f with are de ned over an associative operator it can easily be shown that f`(f ) 2 Frobust .
6 Probabilistic Restriction for Arbitrary Alphabets { Error Bounds for Depth-Bounded Circuits In the following we will calculate an upper bound on the number of inputs for which a small depth ub-circuit gives correct result for a given pre x function f 2 Fsi-str-di . Since the standard probabilistic restriction method works only on circuits over a binary input alphabet it cannot be used directly to estimate the desired quantity. More precisely, using the calculations of Hastad (see [Hast86b]), for depth d ub-circuit of polynomial size with d 2 o(log = llog) the number of inputs with a wrong result can only be bounded by 1 1=d ) n ?
(( n= log n ) jj jj2 ? 2 jjn?2 =2 : If jj is not anpower of 2 this approximation implies a lower bound for the error probability of at most jj 1 2jj2 jj+1 . Note that this bound tends towards zero for growing n . Hence, this estimation cannot be used to give a meaningful bound for the expected delay of an ub-circuit in general. Therefore, we will introduce a generalisation of the probabilistic restriction method: De nition 9 For 0 p 1 , the probabilistic restriction over the alphabet , [p; ] sets the input variables Bi independently to the following values: Pr[[p; I ](Bi ) = ?] = p and 8x 2 I : Pr[[p; I ](Bi ) = x] = 1 ? p : I 0 n In the following we will restrict ourself to functions f : I ! f0; 1g which decide for a given pre x function f , an input w 2 nI , and an symbol x 2 whether f (w)[n] = x , that means whether the last value of f (w) is x or not. For a circuit Cn with n inputs and a string w 2 (I [f?g)n de ne C d as the simpli ed circuit (resp. the function computd by the circuit) for which all inputs vi with w[i] 6= ? are determined by w[i]. To consider arbitrary input alphabets I we will use special input gates [B 2 Z ] for an input variable B and a subset Z I which are evaluated to 1 i B 2 Z . A conjunction resp. disjunction of the variables B1 ; : : : ; Bn is a term of type ^ _ K := [Bi (j ) 2 Zi;j ]; resp. D := [Bi (j ) 2 Zi;j ] I
I
w
i
j 2f1;:::;jKijg
i
j 2f1;:::;jDijg
with Z I . A conjunction Ki is called a conjunctive min-term if 8w 2 nI : Ki (w) = 1 ! f (w) = 1 and 8 k 2 [1; jKi j] 9 w 2 nI : f (w) = 0 ^ Ki;k (w) = 1 ; V where Ki;k := j 2f1;:::;jKijg and j 6=k [Bi (j ) 2 Zi;j ] : A disjunctive min-term is de ned analogously. Further, min(C ) denotes the maximum number of variables of a conjunctive min-term of C . Analogously to [Hast86b] it can be shown: i;j
Average Case Complexity of Unbounded Fanin Circuits
10
Lemma 2 Let C = Vki=1 Di where Di denotes a disjunction of at most t variables. Further let F : I ! f0; 1g be an arbitrary function and [p; I ] a probabilistic restriction. Then it holds Pr[min(C d[p;I ] ) s j F d[p;I ] 1] s ; where is the unique positive solution of !t !t 2p j j j j p 1 1 I I 1 + 1 + (j j ? 1) p = 1 + 1 + (j j ? 1) p + 1 : I I
Investigating F d[p;I ] 0 instead of F d[p;I ] 1 the same technique can be used to transform a disjunction of conjunctions into a conjunction of disjunctions. The value of can be estimated as following inequality: Lemma 3 The value of can be bounded by:
t (jI j ln 2 ? 1) jI j p < t jI j2 p (1 + (jI j ? 1) p) ln 2 (1 + (jI j ? 1) p) ln 2 : For a gate v of C de ne accept(v) as the set of inputs w 2 nI such that resv (w) = 1. Furthermore, let accept(C ) := accept(v) where v denotes the output gate of C . If we allow arbitrary conjunctions for C d (not only conjunctive min-terms) we can improve Lemma 2 to: Corollary 2 Let C = Vki=1 Di where Di denotes a disjunction of at most t variables. Further, let be a probabilistic restriction of Rp;I . Then the probability that C d can be represented by a disjunction of conjunctions Kj with maximal degree s is at least 1 ? s . Furthermore, this conjunction can be chosen such that the sets accept(Ki ) are pairwise disjoint. Using a probabilistic restriction Rp;I and the robustness of strictly diuent pre x functions we can show an upper bound on the maximum rank of correctness of a d -depth and s -size bounded ub-circuit C computing a pre x function f on inputs of length n . We will denote this value by E (f; d; s; n) , i.e. there exists at most E (f; d; s; n) jI jn inputs w 2 nI with C (w) = f (w). last it holds Theorem 4 For f 2 Frobust
E (f; d; s; n)
jI j2 ? 1 jI j2
+
where
8 > > > >
jI j? (n=(logjI j s) ) if logjI j s > pd n; > > > : j j? ((n= logjI j s)1=d ) if log s < p d n: I jI j
7 Lower Bounds closed and all bounds d , s , and n the value of d (1 ? E (f; d; s; n)) gives Note that for any function f 2 Fpre x a lower bound for the expected delay of an ub-circuit computing f . Hence it follows from Theorem 4: closed then it holds for any s Lemma 4 Let f 2 Fstr-di log n etime(UbCir(fn ; s)) llog s :
Proof: We will use a technique presented in [JaRS95]. Any family of circuits C1 ; C2 ; : : : of size s computing f within an expected delay of t can be transformed into a depth t bounded family of circuits C1[t]; C2[t] ; : : : of size at most quadratic in s computing f with a decreasing error rate. This implies 1 ?E (f; t; s; n) 2 o(1).
Average Case Complexity of Unbounded Fanin Circuits
11
Considering the value of this means that we have to choose a bound t for the expected delay such that tends to 1=jI j2 for growing n , resp. > 1=jI j3 for almost all n 2 IN. After someparithmetic transformations we get the following cases: If t = 2 then 3 (n= logjI j s). If logjI j s > t n it holds p that 3 (n= logtj?I1j s) and if logjI j s < t n then 3 ((n= logjI j s)1=t ). For the rst case it follows that s is exponential in n . From the precondition of the second case, i.e. p t logjI j s > n , it follows that t 2 llog s?logllogn jI j + (1). For the third case we get
p
3 2 ((n= logjI j s)1=t ) (n(t?1)=t2 ) =) 9 2 ( t n) =) t 2 log n + (1) : The third case implies that s has to be a constant { a contradiction. So, only the rst and the second case can occurs and the claim follows directly. Note that Theorem 1 follows by combining the lemma above with the show upper bounds.
References [BiPr89] [BiPr90] [ChFL83a] [ChFL83b] [FuSS84] [Hast86a] [Hast86b] [Jak98a] [Jak98b] [JRS94] [JaRS95] [JRSW94] [LaFi80] [Ya85]
G. Bilardi, F. Preparata, Size-Time Complexity of Boolean Networks for Pre x Computations, J. of the ACM 36, 1989, 362-382. G. Bilardi, F. Preparata, Characterization of Associative Operations with Pre x Circuits of Constant Depth and Linear Size, SIAM J. Comput 19, 1990, 246-255. A. Chandra, S. Fortune, R. Lipton, Unbounded Fan-in Circuits and Associative Functions, Proc. 15. SToC'83, 1983, 52-60. A. Chandra, S. Fortune, R. Lipton, Lower Bounds for Constant Depth Circuits for Pre x Problems, Proc. 10. ICALP'83, 1983, 109-116. M. Furst, J. B. Saxe, M. Sipser, Parity, Circuits, and the Polynomial-Time Hierarchy, Proc. 22. FoCS'81, 1981, 260-270, and Math. Systems Theory 17, 1984, 13-27. J. Hastad, Almost Optimal Lower Bounds for Small Depth Circuits, Proc. 18. SToC'86, 1986, 6-20. J. Hastad, Computational Limitations for Small-Depth Circuits, ACM Doctoral Dissertation Award 1986, MIT Press, 1986. A. Jakoby, Die Komplexitat von Pra xfunktionen bezuglich ihres mittleren Zeitverhaltens, Dissertation, Med. Universitat zu Lubeck, 1998. A. Jakoby, The Average Time Complexity to Compute Pre x Functions in Processor Networks, Technical Report, Med. Universitat zu Lubeck, 1998. A. Jakoby, R. Reischuk, C. Schindelhauer, Circuit Complexity: from the Worst Case to the Average Case, Proc. 26. ACM Symposium on the Theory of Computing, 1994, 58-67. A. Jakoby, R. Reischuk, C. Schindelhauer, Malign Distributions for Average Case Circuit Complexity, Proc. 12. STACS'95, 1995, 628-639. A. Jakoby, R. Reischuk, C. Schindelhauer, S. Weis, The Average Case Complexity of the Parallel Pre x Problem, Proc. 21. ICALP'94, 1994, 593-604. R. Ladner, M. Fischer, Parallel Pre x Computation, J. of the ACM 27, 1980, 831-838. A. Yao, Separating the Polynomial Time Hierarchy by Oracles, Proc. 26. FoCS, 1985, 1-10.
