Improved Pseudorandom Generators for Combinatorial ... - CiteSeerX
Improved Pseudorandom Generators for Combinatorial Rectangles Chi-Jen Lu Computer Science Department University of Massachusetts Amherst, MA 01003 U.S.A. Email: [email protected]
Abstract
We construct a pseudorandom generator which uses O(log m + log d + log3=2 1=) bits and approximates the volume of any combinatorial rectangle in f1; : : : ; mgd to within error. This improves on the previous construction by Armoni, Saks, Wigderson, and Zhou [3] using O(log m + log d + log2 1=) bits. For a subclass of rectangles with at most t log 1= nontrivial dimensions and each dimension being an interval, we also give a pseudorandom generator using O(log log d + log 1= log1=2 logt1= ) bits, which again improves the previous upper bound O(log log d + log 1= log logt1= ) by Chari, Rohatgi, and Srinivasan [4].
1 Introduction Pseudorandom generators for combinatorial rectangles have been actively studied recently, because they are closely related to some fundamental problems in theoretical computer science, such as derandomizing RL, DNF approximate counting, and approximating the distributions of independent multivalued random variables. Let V be a nite set with the uniform distribution. The volume of a set A V is de ned as
vol(A) = Px2V [x 2 A]: Let A be a family of subsets from V . We want to sample from a much smaller space, instead of from V , and still be able to approximate the volume of any subset A 2 A. We call a function g : f0; 1gl ! V an -generator using l bits for A, if for all A 2 A,
jPy2f0;1gl [g(y) 2 A] ? vol(A)j : For positive integers m and d, a combinatorial rectangle of type (m; d) is a subset of [m]d = f1; : : : ; mgd of the form R1 Rd, where Ri [m] for all i 2 [d]. Let Q R(m; d) denote the family of all such rectangles. The volume of a rectangle R 2 R(m; d) is now i2[d] jRmi j . Our goal is to nd explicit -generators with small l for R(m; d) and some particular subclasses of it. As observed by Even, Goldreich, Luby, Nisan, and Velickovic [5], this is a special case of constructing pseudorandom generators for RL. Nisan's generator for RL [10] is currently the best, using O(log2 n) bits. Because it has many important applications and no improvement has been made for several years, one might hope that solving this special case could shed some light on the general problem. 1
It's easy to show that a random function mapping from O(log m +log d +log 1=) bits to [m]d is very likely to be an -generator for R(m; d). However, the ecient construction of an explicit one still remains open. Even et al. [5] gave two -generators. One uses O((log m +log d +log 1=) log 1=) bits based on k-wise independence, and the other uses O((log m +log d +log 1=) log d) bits based on Nisan's generator for RL. Armoni at al. [3] observed that the generator of Impagliazzo, Nisan, and Wigderson [7] for communication networks also gives an -generator for R(m; d) using O(log m + (log d + log 1=) log d) bits, which is good when d is small. They then reduced the original problem to the case when d is small (a formal de nition of reductions will be given in the next section), and used the INW-generator to get an -generator for R(m; d) using O(log m + log d + log2 1=) bits. When m, d, and 1= are polynomially related, say all n(1) , all these previous generators still use (log2 n) bits, which is the current barrier for its generalized problem | constructing generators for RL. We break this barrier for the rst time, and give an -generator for R(m; d) using O(log m + log d + log3=2 1=) bits. Our construction is based on that of Armoni at al. [3], and uses two more reductions to further reduce the dimension before applying the INW-generator. The overall construction can be seen as a composition of several generators for rectangles. Independently, Radhakrishnan and Ta-shma [13] have a slightly weaker result using a very similar idea. We also observe that further improvements can be made if one can do better for a special case. Let R(m; d; k) be the set of rectangles from R(m; d) with at most k nontrivial dimensions (those not equal to [m]). We show that if an explicit -generator using O(k +log m +log d +log 1=) bits for R(m; d; k) exists, we can construct an explicit -generator using O(log m +log d +log 1= log log 1=) bits for R(m; d). Unfortunately we still don't know how to construct such a generator for R(m; d; k). Another interesting special case is for rectangles where each dimension is an interval. Let B(m; d; k) be the set of rectangles from R(m; d; k) with each dimension being an interval. Even et al. [5] observed that the problem of approximating the distribution of independent multivalued random variables can be reduced to this case. They gave a generator using O(k +log d +log 1=) bits. This is good when k = O(log 1=). For the case k log 1=, Chari et al. [4] gave a generator using O(log log d+log 1= log logk1= ) bits. Here, we improve this again to O(log log d+log 1= log1=2 logk1= ). We will not emphasize the eciency of our generators, but one can easily check that all the generators can be computed in simultaneous (md=)O(1) time and O(log m + log d + log 1=) space. It's worth mentioning that the hitting version of our problem has already been settled. Linial, Luby, Saks, and Zuckerman [8] gave an explicit generator using O(log m + log log d + log 1=) bits that can hit any rectangle in R(m; d) of volume at least . In fact the work of Armoni at al. [3] followed this result closely, and so does ours.
2 Preliminaries 2.1 Notations
For a set V , we let 2V denote the family of subsets of V . For a rectangle R 2 R(m; d) and a set of indices I [d], we let RI denote the subrectangle of R restricted to those dimensions in I . Similarly, for a vector v 2 [m]d and I [d], we de ne vI to be the subvector of v restricted to those dimensions in I . Let Hk (n1 ; n2 ) denote the standard family of k-wise independent hash functions from [n1 ] to [n2 ]. It can be identi ed with [jHk (n1 ; n2 )j] in the sense that there is a one-to-one mapping from [jHk (n1 ; n2 )j] onto Hk (n1 ; n2 ) that can be eciently computed. Whenever we can identify a class F of functions with [jFj], we can use numbers in [jFj] to represent functions in F . There is a natural correspondence between functions from A to B and vectors in B jAj. So 2
Hk (n1 ; n2) can be seen as jHk (n1; n2)j vectors in [n2]n . For a function f : A ! [m]d, an element x 2 A, and an index y 2 [d], we will use f (x)(y) to denote the yth dimension in the vector f (x) 2 [m]d . 1
When we sample from a nite set, the default distribution is the uniform distribution over that set. All the logarithms throughout this paper will have base 2.
2.2 Reductions
We adopt the notion of reduction introduced by Armoni at al. [3]. It enables us to reduce a harder problem to an easier one, and then focus our attention on solving the easier problem. A class F of functions from a set V2 to a set V1 de nes a reduction from V1 to V2 . Let A1 2V and A2 2V . F is said to be (A1 ; A2; )-good, if for each R 2 A1 the following hold: 1. 8f 2 F ; f ?1 (R) 2 A2 , and 2. jEf 2F [vol(f ?1 (R))] ? vol(R)j . Suppose now that F is (A1 ; A2 ; 1 ) good and g : f0; 1gs ! V2 is an 2 -generator for A2 . Armoni at al. [3] showed that the function g0 : f0; 1gs F ! V1 , de ned as g0 (y; f ) = (f g)(y), is an (1 + 2 )-generator for A1 . The reduction cost of F is log jFj, which is the number of extra bits needed for the new generator. The following lemma follows immediately. 1
2
Lemma 2.1 For each i, 0 i l, let Vi be a set and Ai 2Vi . Suppose that Fi is (Ai?1; Ai; i?1 )good for 1 i l, and g : f0; 1gs ! Vl is an l -generator for Al . Then P the function g0 : f0; 1gs F1 Fl ! V0 de ned as g0 (x; f1 ; : : : ; fl ) = (f1 fl g)(x), is a ( li=0 i)-genertor for A0 . So to construct a generator for A0 , it suces to nd a series of reductions from A0 to Al , and then nd a generator for Al . Notice that an (A1 ; A2 ; )-good reduction F actually corresponds to a special kind of -generator for A1 . Let h : V2 F ! V1 be de ned as h(y; f ) = f (y). Then the second condition guarantees that for all R 2 A1 , jP(y;f )2V F [h(y; f ) 2 R] ? vol(R)j = jEf 2F [Py2V [f (y) 2 R]] ? vol(R)j = jEf 2F [vol(f ?1 (R))] ? vol(R)j : 2
2
The rst condition guarantees that one part of h's input, V2 , can come from the output of a generator for A2 , and makes the composition of generators possible. So one way of nding a reduction is to use some generator that might use many bits but can be composed with other generators.
3 The Pseudorandom Generator for R(m; d)
3.1 The Overview of the Construction
The INW-generator uses O(log m + (log d + log 1=) log d) bits, which is good when d is small. The idea of Armoni at al. [3] is to reduce the dimension of rectangles rst to d0 = (1=)O(1) before applying the INW-generator. 3
In addition to that, we also reduce m to m0 = (1=)O(1) . The INW-generator for R(m0 ; d0 ) needs O(log m0 + (log d0 + log 1=) log d0 ) = O(log 1= + log2 1=) bits. Observe plog 1=)that we do not lose by O ( 0 0 00 letting m increase By allowing m to grow to m = (1=) , we are able to reduce plog 1a=little. 3 = 2 ) O ( 0 00 d to d = 2 . The INW-generator now uses O(log 1=) bits for R(m00 ; d00 ). The total reduction cost is O(log m + log d + log3=2 1=), and we have the desired generator for R(m; d). More precisely, we will use the following three reductions. F1 is called the rst dimension reduction family, and is used to reduce d to d1 = (1=)O(1) . It is (R(m; d); R(m1 ; d1 ); =4)-good, where m1 = (md)O(1) . The reduction cost is O(log d). F2 is called the range reduction family, and is used to reduce m1 to m2 = (1=)O(1) . It is (R(m1 ; d1 ); R(m2 ; d2 ); =4)-good, where d2 = d1 . The reduction cost is O(log m + log d + log 1=). F3 is called the second dimension reduction family, and is used to reduce d2 to d3 = 2(3 log )=(k?1) , where k is a parameter to be chosen to optimize our construction. It is (R(m2 ; d2 ); R(m3 ; d3 ); =4)-good, where m3 = jHk (d2 ; m2 )j = (1=)O(k) . The reduction cost is log jHk (d2 ; d3 )j = O(k log 1=). Together with the =4-generator for R(d3 ; m3 ) from the INW-generator, we have an -generator for R(m; d). The number of bits used depends on k, and choosing k = log1=2 1= results in the minimum O(log m + log d + log3=2 1=). 12
3.2 The First Dimension Reduction Function Family
Let F1 be the reduction family used by Armoni at al. [3], which is the composition of three reduction families. F1 is (R(m; d); R(m1 ; d1 ); =4)-good, where m1 = (dm)O(1) , d1 = (1=)O(1) , and each is a power of 2. jF1 j = dO(1) . Let V1 = [m1 ]d . 1
3.3 The Range Reduction Function Family
We will use a generator of Nisan and Zuckerman [12], based on an extractor of Goldreich and Wigderson [6]. The idea of using extractors for range reduction was inspired by that of Radhakrishnan and Ta-Shma [13]. We choose a more appropriate extractor and get a better reduction. De nition 3.1 A function E : f0; 1gs f0; 1gt ! f0; 1gl is an (s; r; t; l; )-extractor if for x chosen from a distribution over f0; 1gs with min-entropy1 r, and y chosen from the uniform distribution over f0; 1gt , the distribution E (x; y) has distance2 at most to the uniform distribution over f0; 1gl . Extractors are used to extract randomness from weakly random sources, and have many other applications. For more details, please refer to an excellent survey by Nisan [11]. We use an extractor due to Goldreich and Wigderson [6]. Lemma 3.1 There are constants c1 and c2 such that for any s, , and with s > , s ? > l, and > 2?(s?l?c )=c , an explicit (s; s ? ; O( + log 1 ); l; )-extractor exists. Choose = =(4d1 ) = O(1) , = dlog 1=e, t = O( ), and l = log m1 . Choose s = l + c = O(log(md) + log 1=) = O(log(md=)) for some constant c, such that 2?(s?l?c )=c = 2?((c?c )=c ) log 1= < 2log = . We have the following extractor for this setting. 1
2
1
1
1 2
2
The min-entropy of a distribution D on a set S is mina2S log D(1a) . The distance of two distributions D1 and D2 over a set S is de ned as maxAS jD1 (A) ? D2 (A)j
4
2
Corollary 3.1 There exists an explicit (s; s ? ; t; l; )-extractor A. The building block of Nisan and Zuckerman's generator for space bounded Turing machines [12], when using the extractor A, has the form G : f0; 1gs f0; 1gtd ! [m1 ]d , where 1
1
G(x; y1; : : : ; yd ) = (A(x; y1 ); : : : ; A(x; yd )): 1
1
For R = R1 Rd 2 R(m1 ; d1 ), we rst show that 1
jPx;y ;:::;yd [(A(x; y1 ); : : : ; A(x; yd )) 2 R] ? vol(R)j d1 = 4 : 1
1
1
For i 2 [d1 ], let Di (x; y1 ; : : : ; yi ) denote the event that (A(x; y1 ); : : : ; A(x; yi )) is in R1 Ri . Also let pi = Px;y ;:::;yi [Di(x; y1 ; : : : ; yi )], qi = Qij=1 jmRj j , and ri+1 = Px;y ;:::;yi [A(x; yi+1 ) 2 Ri+1 jDi (x; y1 ; : : : ; yi )]. The following is a simpli ed version of a lemma due to Nisan and Zuckerman [12], based on our choice of parameters. 1
1
1
+1
Lemma 3.2 For any i 2 [d1 ], jpi ? qij i. Proof: Use induction on i. It's true for i = 1 from the de nition of extractors. Assuming that jpi ? qij i, we'll show that jpi+1 ? qi+1j (i + 1). jp ? q j = jp r ? p jRi+1j + p jRi+1 j ? q jRi+1 j j i+1
i+1
i i+1
i
m1
i
i
m1 pijri+1 ? jRmi+1j j + jpi ? qij:
m1
1
There are two cases. pi :
jpi+1 ? qi+1j + i = (i + 1):
pi > :
The distribution of x conditioned on Di (x; y1 ; : : : ; yi ) has min-entropy at least s ? log 1= s ? . From Corollary 3.1, jPx;y ;:::;yi [A(x; yi+1 ) 2 Ri+1 jDi (x; y1 ; : : : ; yi )] ? jRmi j j , and jpi+1 ? qi+1 j + i = (i + 1): Now let m2 = 2t = (1=)O(1) = (d1 =)O(1) = (1=)O(1) , d2 = d1 , and V2 = [m2 ]d . Consider the reduction F2 = ffx j x 2 f0; 1gs g, where fx : V2 ! V1 is de ned as follows +1
1
1
2
fx (y1 ; : : : ; yd ) = G(x; y1 ; : : : ; yd ): 2
1
Then fx?1 (R) = R10 Rd0 2 R(m2 ; d2 ), where Ri0 = fyi j A(x; yi ) 2 Ri g. Also, 2
jEx[vol(fx?1 (R))] ? vol(R)j = jPx;y ;:::;yd [G(x; y1 ; : : : ; yd ) 2 R] ? vol(R)j =4: 1
So we have the following lemma. Lemma 3.3 F2 is (R(m1 ; d1 ); R(m2 ; d2 ); =4)-good. 5
1
1
3.4 The Second Dimension Reduction Function Family
Let R = R1 Rd 2 R(m2 ; d2 ). We want to partition the d2 dimensions of R into d3 parts using some function h : [d2 ] ! [d3 ] in the natural Q way. For q 2 [d3], those dimensions of R that are mapped to q form a subrectangle Rh? (q) = i2h? (q) Ri . Based on the idea of Even et al. [5], its volume can be approximated by sampling from the k-wise independent space G = Hk (d2 ; m2 ). We use d3 copies of G, one for each subrectangle. The corresponding rectangle R(h) = R10 Rd0 , where Rq0 = fp 2 G : ph? (q) 2 Rh? (q) g, should have a volume close to that of R. The error depends on the choice of k and h. We will show that for k = O(log1=2 1=) and h chosen uniformly from H = H(d2 ; d3 ), the expected error is at most =4. More formally, let d3 = 2(3 log )=(k?1) , m3 = jGj, V3 = [m3 ]d , and F3 = ffh : h 2 H g, where fh : V3 ! V2 is de ned as follows 2
1
1
3
1
1
12
3
fh(p1 ; : : : ; pd ) = (ph(1) (1); : : : ; ph(d ) (d2 )): 3
2
Then for R 2 R(m2 ; d2 ) and fh 2 F3 , fh?1 (R) = R(h) 2 R(m3 ; d3 ). We also need the following notation for the proofs below. For R = R1 Rd 2 R(m2 ; d2 ), let R~ denote the rectangle R1 Rd 2 R(m2 ; d2 ), where Ri = [m2 ] n Ri . For i; j 2 [d2 ], and I [d2 ], denote 2
2
Ri j ; i = jm 2 Y (I ) = vol(R~ I ) = i; i2I
(I ) = Pp2G [pI 2 R~ I ]; and X (I ): j (I ) = J I;jJ j=j
The approximation error of each subrectangle can be bounded in the following way. Proposition 3.1 8I [d2 ]; jPp2G [pI 2 RI ] ? vol(RI )j k (I ) Proof: Because G is a k-wise independent space, for J I with jJ j k, (J ) = (J ). From the principle of inclusion and exclusion, we have the following.
Y
vol(RI ) = (1 ? i ) = i2I
Pp2G [pI 2 RI ] =
jI j X (?1)jJ j(J ) = Xk (?1)j (I ) + X (?1)j j (I ): j j =0
J I
j =k+1
jI j X X (?1)jJ j (J ) = Xk (?1)j (I ) + X (?1)j j
J I
j =0
j =k+1
J I;jJ j=j
(J ):
P ?1(?1)j j (I ) Now the proposition follows because both vol(RI ) and Pp2G [pI 2 RI ] fall between kj =0 P and kj=0 (?1)j j (I ). The approximation error of any partition can be bounded by the following. Lemma 3.4 8h : [d2 ] ! [d3 ]; jvol(R(h) ) ? vol(R)j Pq2[d ] k (h?1 (q)) 3
6
Proof:
Y vol(R ? ): h (q) q2[d ] Y P [p ? 2 R ?
vol(R) =
1
3
vol(R(h) ) =
q2[d3 ]
h 1 (q) ]:
p2G h 1 (q)
lemma follows from the previous proposition and the known fact that j Pli=1This jxi ? yij when 0 xi; yi 1 for all i 2 [l].
Qli=1 xi ? Qli=1 yij
Finally, we can bound the expected approximation error. Lemma 3.5 For R 2 R(m2 ; d2 ), jEh2H [vol(R(h) )] ? vol(R)j 4 .
Proof:
jEh2H [vol(R(h) )] ? vol(R)j Eh2H [jvol (R(h) ) ? vol(R)j] X Eh2H [ k (h?1 (q))]
X Eq2[d ][ X (I )] h2H q2[d ] X XIh?P(q);jI[j=8ik 2 I h(i) = q](I ) h2H q2[d ] I [d ];jI j=k X X (1=d )k (I ) 3
=
1
3
=
3
= =
P
2
q2[d3 ] I [d2 ];jI j=k (1=d3 )k?1 k ([d2 ]):
3
Let = i2[d ] i . There are two cases depending on the value of . log 12 : k ([d2 ]) gets it maximum value when i = d for all i 2 [d2 ]. So k ([d2 ]) ( edk )k ( d )k ( e logk )k , which is again maximized when k = log 12 . So for d3 = 2(3 log )=(k?1) , we have 2
2
2
12
12
2
jEh2H [vol(R(h) )] ? vol(R)j 2?3 log elog 12
= 2?(3?log e) log = ( 12 )3?log e
12 12
12 : > log 12 :
In this case, both Eh2H [vol (R(h) )] and vol(R) are small, so their di erence is small. First, P vol(R) = Qi2[d ] (1 ? i ) 2? i2 d i < 12 . we show that Eh2H [vol(R(h) )] 3 . Let d0 be the smallest integer such that log 12 ? 1 < Pi2[Next, 12 0 d ?d0 . From the previous case Eh2H [vol(R0(h) )] vol(R0 ) + d0 ] i log . Let R = R[d0 ] [m2 ] 2
[ 2]
2
7
12 . So
Eh2H [vol(R(h) )] Eh2H [vol(R0(h) )] vol(R0) + 12 P 0 i ? 2 i2 d + 12 2? log +1 + 12 = : [
]
12
4 0 Eh2H [vol(R(h) )]; vol(R) 4 . So jEh2H [vol(R(h) )] ? vol(R)j 4 . So we have the following lemma.
Lemma 3.6 F3 is (R(m2 ; d2 ); R(m3 ; d3 ); 4 )-good. Theorem 3.1 There is an explicit -generator for R(m; d), using O(log m +log d +log3=2 1=) bits. Proof: jF1 j = dO(1) , jF2 j = 2s = (md=)O(1) , and jF3 j = jHk (d2 ; d3 )j dk2 = (1=)O(k) . The INW-generator gives us an 4 -generator for R(m3 ; d3 ) using O(log m3 + (log d3 + log 1=) log d3 ) = O(k log 1= + 1=k log2 1=) bits. From Lemma 2.1, we have an -generator for R(m; d) using
O(log m + log d + k log 1= + 1=k log2 1=) bits. When k = O(log1=2 1=), the number of bits used gets its minimum value O(log m + log d + log3=2 1=).
4 A Potential Improvement
For F3 , we can replace the k-wise independent space H by an approximate k-wise independent space H 0 , over [d3 ]d , such that for any I [d2 ] with jI j k and for any y 2 [d3 ]d , jP 0 [x = y ] ? ( 1 )jI jj O( ): 2
2
x2H
I
I
d3
d3
A simple generalization from the constructions of Alon, Goldreich, Hastad, and Peralta [1], or Naor and Naor [9] gives us jH 0 j = ( k log d )O(1) = ( 1 )O(1) . H 0 can also be identi ed eciently with jH 0 j. One can easily verify that only an additional O() error is introduced in Lemma 3.5, and now the reduction cost for F3 is O(log 1=). From now on we will use H 0 instead of H in F3 . Recall from the previous section that m3 = jHk (d2 ; m2 )j = (1=)O(k) and d3 = 2(3 log 1=)=(k?1) . Larger k implies smaller d3 but larger m3 . The optimum is attained at k = (log1=2 1=). If we can replace Hk (d2 ; m2 ) by a smaller space, we might be able to choose a larger k and get a smaller d3 . Remeber that d3 copies of Hk (d2 ; m2 ) are used to approximate the volumes of the d3 subrectangles of R partitioned by a function h : [d2 ] ! [d3 ]. The approximation is guaranteed by the fact that for R 2 R(m2 ; d2 ) and for J [d2 ] with jJ j k, j(J ) ? (J )j = jvol(R~J ) ? Pp2G[pJ 2 R~J ]j = 0: 2
We want to use a smaller space by allowing a small error 0 instead of 0 above. The approximate k-wise independent space does not help here, because it needs (k log m2 + log 1=0 ) bits to achieve an error 0 here, no better than G. However, observe that what we need here is to approximate the 8
volume of a rectangle with at most k nontrivial dimensions. This turns out to be a special case of our original problem | constructing a pseudorandom generator for R(m; d; k). Suppose that g : f0; 1gs ! [m2 ]d is an 0 -generator for R(m; d; k). In F3 , we replace Hk (d2 ; m2 ) by the space generated by g. Let m3 = 2s . For h 2 H 0 , let fh : [m3 ]d ! [m2 ]d be de ned as follows fh(x1 ; : : : ; xd ) = (g(xh(1) )(1); : : : ; g(xh(d ))(d2 )): For R = R1 Rd 2 R(m2 ; d2 ), fh?1 (R) = R(h) = R10 Rd0 2 R(m3 ; d3 ), where Rq0 = fx 2 [m3 ] : g(x)h? (q) 2 Rh? (q) g. Then for J h?1 (q) with jJ j k, jPx2[m ][g(x)J 2 R~ J ] ? vol(R~J )j 0. So, for any h : [d2 ] ! [d3 ], X (( X 0) + (h?1(q))) jvol(R) ? vol(R(h) )j k 2
3
3
2
2
2
3
1
1
3
q2[d3 ] J h?1 (q);jJ jk dk2+20 + k (h?1 (q)); q2[d3 ] and jEh2H 0 [vol(R(h) )] ? vol(R)j dk2 +2 0 + 4 = =2, for 0 = 4dk+2 . 2
X
d3 .
So if we have a better 0 -generator for R(m2 ; d2 ; k), we can choose a larger k and thus a smaller
Theorem 4.1 If there exists an explicit -generator for R(m; d; k) using O(k +log d +log m +log 1 ) bits, then there exists an explicit -generator for R(m; d) using O(log d +log m +log 1 log log 1 ) bits. k2
d m )O(1) = ( d )O(1) . We want Proof: Using the 0 -generator for R(m2 ; d2 ; k) in F3 gives m3 = ( 2k=d k 2
2
to repeatedly reduce the dimensions of rectangles. Notice that each time the dimension is reduced, we can choose a larger next k. For 3 i l = log log 12 ? log log log 12 , let ki = 2 i ; di = 2O((log )=ki ) = 2O((log )=2i ) ; i = ki +2 = O(1) ; and 4di?1 ki m = ( 2 mi?1 di?1 )O(1) = ( 1 )O(i) 2
1
i
1
i
For 3 i l, let Fi be the dimension reduction discussed before, using the assumed i -generator for R(mi?1 ; di?1 ; ki ). One can check that each Fi is (R(mi?1 ; di?1 ); R(mi ; di ); )-good. Using the INW-generator as an -generator for R(ml ; dl ), we have an O( log log 1 )-generator for R(m; d). The total number of bits used is O(log m + log d + log 1 + log 1 log log 1 + log ml + (log dl + log 1 ) log dl ) = O(log m + log d + log 1 log log 1 ):
Replacing log log 1 by , we have an -generator for R(m; d) using O(log m +log d +log 1 log log 1 ) bits. We don't know yet how to construct such an explicit -generator for R(m; d; k) using O(log k + log d + log m + log 1=) bits. Using an idea of Auer, Long, and Srinivasan [2], we can derive one using O(log k + log m + log3=2 1=) bits, which improves their upper bound, but does not serve our purpose here. 9
5 The Pseudorandom Generator for B(m; d; t)
Recall that B(m; d; t) denotes the class of rectangles from R(m; d) with at most t nontrivial dimensions and each dimension being an interval. For B(m; d; t), Even et al. [5] have an -generator using O(t + log log d + log 1=) bits. Unfortunately, we cannot apply the iterative procedure in the previous section to B(m; d; d) because after applying the dimension reduction once, each dimension is no longer an interval. For t log 1=, Chari et al. [4] had an -generator for B(m; d; t) using O(log log d+log 1= log logt1= ) bits, a signi cant improvement in the dependence on t. This is improved again by the following theorem.
Theorem 5.1 For t log 1=, there is an explicit -generator for B(m; d; t), using O(log log d + log 1= log1=2 logt1= ) bits.
Proof: Here we use only one reduction, the modi ed dimension reduction discussed in the previous section. Let k t be a parameter to be chosen later. For 0 = ( kt )O(k) , let g : f0; 1gs ! [m]d be the 0 -generator of Even et al. for B(m; d; k). Let m0 = 2s and d0 = 2(3 log )=(k?1) . Given R = R1 Rd 2 B(m; d; t), assume w.l.o.g. that the rst t dimensions are nontrivial. For h 2 H 0 let R(h) = fh?1 (R) = R10 Rd0 , where Rq0 = fx 2 [m0 ] : g(x)h? (q) 2 Rh? (q) g. For J 6 [t], jvol(R~J ) ? Px2[m0][g(x)J 2 R~J ]j = 0; because Rj = ; for j 62 [t]. For J [t] with jJ j k, jvol(R~J ) ? Px2[m0 ][g(x)J 2 R~J ]j 2jJ j0; as each R~ J is the union of at most 2jJ j rectangles from B(m; jJ j; jJ j). Then 12
1
jEh2H 0 [vol(R(h) )] ? vol(R)j
Xk X
2j 0 + Eh2H 0 [
j =0 J [t];jJ j=j ( t )O(k) 0 + O()
X (h?1(q))]
q2[d0 ]
k O():
1
k
Combined with the INW-generator, we get an -generator using O(log k log d + k log kt +log 1 logk1= ) = 1= O(log k +log log d +log 1= + k log kt + log k1= ) bits. Choosing k = log log = t results in O (log log d + = log 1= log1=2 logt1= ). 2
1 2
log 1
6 Acknowledgements We would like to thank David Barrington for correcting some mistakes and making useful suggestions. We would like to thank Shiyu Zhou for telling us the result in [13] and for some helpful comments. We would also like to thank Amnon Ta-Shma, Jaikumar Radhakrishnan, and Avi Wigderson for reading this paper.
10
References [1] N. Alon, O. Goldreich, J. Hastad, and R. Peralta, Simple constructions of almost k-wise independent random variables, Random Structures and Algorithms, 3(3), pages 289-303, 1992. [2] P. Auer, P. Long, and A. Srinivasan, Approximating hyper-rectangles: learning and pseudorandom sets, In Proceedings of the 29th Annual ACM Symposium on Theory of Computing, 1997. [3] R. Armoni, M. Saks, A. Wigderson, and S. Zhou, Discrepancy sets and pseudorandom generators for combinatorial rectangles, In Proceedings of the 37th Annual IEEE Symposium on Foundations of Computer Science, pages 412-421, 1996. [4] S. Chari, P. Rohatgi, and A. Srinivasan, Improved algorithms via approximations of probability distributions, In Proceedings of the 26th Annual ACM Symposium on Theory of Computing, pages 584-592, 1994. [5] G. Even, O. Goldreich, M. Luby, N. Nisan, and B. Velickovic, Approximations of general independent distributions. In Proceedings of the 24th Annual ACM Symposium on Theory of Computing, pages 10-16, 1992. [6] O. Goldreich and A. Wigderson, Tiny families of functions with random properties: a qualitysize trade-o for hashing, In Proceedings of the 26th Annual ACM Symposium on Theory of Computing, pages 574-583, 1994. [7] R. Impagliazzo, N. Nisan, and A. Wigderson, Pseudorandomness for network algorithms, In Proceedings of the 26th Annual ACM Symposium on Theory of Computing, pages 356-364, 1994. [8] N. Linial, M. Luby, M. Saks, and D. Zuckerman, Ecient construction of a small hitting set for combinatorial rectangles in high dimension, Combinatorica, 17, pages 215-234, 1997. [9] J. Naor and M. Naor, Small-bias probability spaces: ecient constructions and applications, SIAM Journal on Computing, 22(4), pages 838-856, 1990. [10] N. Nisan, Pseudorandom generators for space-bounded computation, Combinatorica, 12, pages 449-461, 1992. [11] N. Nisan, Extracting randomness: how and why - a survey, In Proceedings of the 11th Annual IEEE Conference on Computational Complexity, pages 44-58, 1996. [12] N. Nisan and D. Zuckerman, Randomness is linear in space, Journal of Computer and System Sciences, 52(1), pages 43-52, 1996. [13] J. Radhakrishnan and A. Ta-shma, Private communication.
11
Abstract
We construct a pseudorandom generator which uses O(log m + log d + log3=2 1=) bits and approximates the volume of any combinatorial rectangle in f1; : : : ; mgd to within error. This improves on the previous construction by Armoni, Saks, Wigderson, and Zhou [3] using O(log m + log d + log2 1=) bits. For a subclass of rectangles with at most t log 1= nontrivial dimensions and each dimension being an interval, we also give a pseudorandom generator using O(log log d + log 1= log1=2 logt1= ) bits, which again improves the previous upper bound O(log log d + log 1= log logt1= ) by Chari, Rohatgi, and Srinivasan [4].
1 Introduction Pseudorandom generators for combinatorial rectangles have been actively studied recently, because they are closely related to some fundamental problems in theoretical computer science, such as derandomizing RL, DNF approximate counting, and approximating the distributions of independent multivalued random variables. Let V be a nite set with the uniform distribution. The volume of a set A V is de ned as
vol(A) = Px2V [x 2 A]: Let A be a family of subsets from V . We want to sample from a much smaller space, instead of from V , and still be able to approximate the volume of any subset A 2 A. We call a function g : f0; 1gl ! V an -generator using l bits for A, if for all A 2 A,
jPy2f0;1gl [g(y) 2 A] ? vol(A)j : For positive integers m and d, a combinatorial rectangle of type (m; d) is a subset of [m]d = f1; : : : ; mgd of the form R1 Rd, where Ri [m] for all i 2 [d]. Let Q R(m; d) denote the family of all such rectangles. The volume of a rectangle R 2 R(m; d) is now i2[d] jRmi j . Our goal is to nd explicit -generators with small l for R(m; d) and some particular subclasses of it. As observed by Even, Goldreich, Luby, Nisan, and Velickovic [5], this is a special case of constructing pseudorandom generators for RL. Nisan's generator for RL [10] is currently the best, using O(log2 n) bits. Because it has many important applications and no improvement has been made for several years, one might hope that solving this special case could shed some light on the general problem. 1
It's easy to show that a random function mapping from O(log m +log d +log 1=) bits to [m]d is very likely to be an -generator for R(m; d). However, the ecient construction of an explicit one still remains open. Even et al. [5] gave two -generators. One uses O((log m +log d +log 1=) log 1=) bits based on k-wise independence, and the other uses O((log m +log d +log 1=) log d) bits based on Nisan's generator for RL. Armoni at al. [3] observed that the generator of Impagliazzo, Nisan, and Wigderson [7] for communication networks also gives an -generator for R(m; d) using O(log m + (log d + log 1=) log d) bits, which is good when d is small. They then reduced the original problem to the case when d is small (a formal de nition of reductions will be given in the next section), and used the INW-generator to get an -generator for R(m; d) using O(log m + log d + log2 1=) bits. When m, d, and 1= are polynomially related, say all n(1) , all these previous generators still use (log2 n) bits, which is the current barrier for its generalized problem | constructing generators for RL. We break this barrier for the rst time, and give an -generator for R(m; d) using O(log m + log d + log3=2 1=) bits. Our construction is based on that of Armoni at al. [3], and uses two more reductions to further reduce the dimension before applying the INW-generator. The overall construction can be seen as a composition of several generators for rectangles. Independently, Radhakrishnan and Ta-shma [13] have a slightly weaker result using a very similar idea. We also observe that further improvements can be made if one can do better for a special case. Let R(m; d; k) be the set of rectangles from R(m; d) with at most k nontrivial dimensions (those not equal to [m]). We show that if an explicit -generator using O(k +log m +log d +log 1=) bits for R(m; d; k) exists, we can construct an explicit -generator using O(log m +log d +log 1= log log 1=) bits for R(m; d). Unfortunately we still don't know how to construct such a generator for R(m; d; k). Another interesting special case is for rectangles where each dimension is an interval. Let B(m; d; k) be the set of rectangles from R(m; d; k) with each dimension being an interval. Even et al. [5] observed that the problem of approximating the distribution of independent multivalued random variables can be reduced to this case. They gave a generator using O(k +log d +log 1=) bits. This is good when k = O(log 1=). For the case k log 1=, Chari et al. [4] gave a generator using O(log log d+log 1= log logk1= ) bits. Here, we improve this again to O(log log d+log 1= log1=2 logk1= ). We will not emphasize the eciency of our generators, but one can easily check that all the generators can be computed in simultaneous (md=)O(1) time and O(log m + log d + log 1=) space. It's worth mentioning that the hitting version of our problem has already been settled. Linial, Luby, Saks, and Zuckerman [8] gave an explicit generator using O(log m + log log d + log 1=) bits that can hit any rectangle in R(m; d) of volume at least . In fact the work of Armoni at al. [3] followed this result closely, and so does ours.
2 Preliminaries 2.1 Notations
For a set V , we let 2V denote the family of subsets of V . For a rectangle R 2 R(m; d) and a set of indices I [d], we let RI denote the subrectangle of R restricted to those dimensions in I . Similarly, for a vector v 2 [m]d and I [d], we de ne vI to be the subvector of v restricted to those dimensions in I . Let Hk (n1 ; n2 ) denote the standard family of k-wise independent hash functions from [n1 ] to [n2 ]. It can be identi ed with [jHk (n1 ; n2 )j] in the sense that there is a one-to-one mapping from [jHk (n1 ; n2 )j] onto Hk (n1 ; n2 ) that can be eciently computed. Whenever we can identify a class F of functions with [jFj], we can use numbers in [jFj] to represent functions in F . There is a natural correspondence between functions from A to B and vectors in B jAj. So 2
Hk (n1 ; n2) can be seen as jHk (n1; n2)j vectors in [n2]n . For a function f : A ! [m]d, an element x 2 A, and an index y 2 [d], we will use f (x)(y) to denote the yth dimension in the vector f (x) 2 [m]d . 1
When we sample from a nite set, the default distribution is the uniform distribution over that set. All the logarithms throughout this paper will have base 2.
2.2 Reductions
We adopt the notion of reduction introduced by Armoni at al. [3]. It enables us to reduce a harder problem to an easier one, and then focus our attention on solving the easier problem. A class F of functions from a set V2 to a set V1 de nes a reduction from V1 to V2 . Let A1 2V and A2 2V . F is said to be (A1 ; A2; )-good, if for each R 2 A1 the following hold: 1. 8f 2 F ; f ?1 (R) 2 A2 , and 2. jEf 2F [vol(f ?1 (R))] ? vol(R)j . Suppose now that F is (A1 ; A2 ; 1 ) good and g : f0; 1gs ! V2 is an 2 -generator for A2 . Armoni at al. [3] showed that the function g0 : f0; 1gs F ! V1 , de ned as g0 (y; f ) = (f g)(y), is an (1 + 2 )-generator for A1 . The reduction cost of F is log jFj, which is the number of extra bits needed for the new generator. The following lemma follows immediately. 1
2
Lemma 2.1 For each i, 0 i l, let Vi be a set and Ai 2Vi . Suppose that Fi is (Ai?1; Ai; i?1 )good for 1 i l, and g : f0; 1gs ! Vl is an l -generator for Al . Then P the function g0 : f0; 1gs F1 Fl ! V0 de ned as g0 (x; f1 ; : : : ; fl ) = (f1 fl g)(x), is a ( li=0 i)-genertor for A0 . So to construct a generator for A0 , it suces to nd a series of reductions from A0 to Al , and then nd a generator for Al . Notice that an (A1 ; A2 ; )-good reduction F actually corresponds to a special kind of -generator for A1 . Let h : V2 F ! V1 be de ned as h(y; f ) = f (y). Then the second condition guarantees that for all R 2 A1 , jP(y;f )2V F [h(y; f ) 2 R] ? vol(R)j = jEf 2F [Py2V [f (y) 2 R]] ? vol(R)j = jEf 2F [vol(f ?1 (R))] ? vol(R)j : 2
2
The rst condition guarantees that one part of h's input, V2 , can come from the output of a generator for A2 , and makes the composition of generators possible. So one way of nding a reduction is to use some generator that might use many bits but can be composed with other generators.
3 The Pseudorandom Generator for R(m; d)
3.1 The Overview of the Construction
The INW-generator uses O(log m + (log d + log 1=) log d) bits, which is good when d is small. The idea of Armoni at al. [3] is to reduce the dimension of rectangles rst to d0 = (1=)O(1) before applying the INW-generator. 3
In addition to that, we also reduce m to m0 = (1=)O(1) . The INW-generator for R(m0 ; d0 ) needs O(log m0 + (log d0 + log 1=) log d0 ) = O(log 1= + log2 1=) bits. Observe plog 1=)that we do not lose by O ( 0 0 00 letting m increase By allowing m to grow to m = (1=) , we are able to reduce plog 1a=little. 3 = 2 ) O ( 0 00 d to d = 2 . The INW-generator now uses O(log 1=) bits for R(m00 ; d00 ). The total reduction cost is O(log m + log d + log3=2 1=), and we have the desired generator for R(m; d). More precisely, we will use the following three reductions. F1 is called the rst dimension reduction family, and is used to reduce d to d1 = (1=)O(1) . It is (R(m; d); R(m1 ; d1 ); =4)-good, where m1 = (md)O(1) . The reduction cost is O(log d). F2 is called the range reduction family, and is used to reduce m1 to m2 = (1=)O(1) . It is (R(m1 ; d1 ); R(m2 ; d2 ); =4)-good, where d2 = d1 . The reduction cost is O(log m + log d + log 1=). F3 is called the second dimension reduction family, and is used to reduce d2 to d3 = 2(3 log )=(k?1) , where k is a parameter to be chosen to optimize our construction. It is (R(m2 ; d2 ); R(m3 ; d3 ); =4)-good, where m3 = jHk (d2 ; m2 )j = (1=)O(k) . The reduction cost is log jHk (d2 ; d3 )j = O(k log 1=). Together with the =4-generator for R(d3 ; m3 ) from the INW-generator, we have an -generator for R(m; d). The number of bits used depends on k, and choosing k = log1=2 1= results in the minimum O(log m + log d + log3=2 1=). 12
3.2 The First Dimension Reduction Function Family
Let F1 be the reduction family used by Armoni at al. [3], which is the composition of three reduction families. F1 is (R(m; d); R(m1 ; d1 ); =4)-good, where m1 = (dm)O(1) , d1 = (1=)O(1) , and each is a power of 2. jF1 j = dO(1) . Let V1 = [m1 ]d . 1
3.3 The Range Reduction Function Family
We will use a generator of Nisan and Zuckerman [12], based on an extractor of Goldreich and Wigderson [6]. The idea of using extractors for range reduction was inspired by that of Radhakrishnan and Ta-Shma [13]. We choose a more appropriate extractor and get a better reduction. De nition 3.1 A function E : f0; 1gs f0; 1gt ! f0; 1gl is an (s; r; t; l; )-extractor if for x chosen from a distribution over f0; 1gs with min-entropy1 r, and y chosen from the uniform distribution over f0; 1gt , the distribution E (x; y) has distance2 at most to the uniform distribution over f0; 1gl . Extractors are used to extract randomness from weakly random sources, and have many other applications. For more details, please refer to an excellent survey by Nisan [11]. We use an extractor due to Goldreich and Wigderson [6]. Lemma 3.1 There are constants c1 and c2 such that for any s, , and with s > , s ? > l, and > 2?(s?l?c )=c , an explicit (s; s ? ; O( + log 1 ); l; )-extractor exists. Choose = =(4d1 ) = O(1) , = dlog 1=e, t = O( ), and l = log m1 . Choose s = l + c = O(log(md) + log 1=) = O(log(md=)) for some constant c, such that 2?(s?l?c )=c = 2?((c?c )=c ) log 1= < 2log = . We have the following extractor for this setting. 1
2
1
1
1 2
2
The min-entropy of a distribution D on a set S is mina2S log D(1a) . The distance of two distributions D1 and D2 over a set S is de ned as maxAS jD1 (A) ? D2 (A)j
4
2
Corollary 3.1 There exists an explicit (s; s ? ; t; l; )-extractor A. The building block of Nisan and Zuckerman's generator for space bounded Turing machines [12], when using the extractor A, has the form G : f0; 1gs f0; 1gtd ! [m1 ]d , where 1
1
G(x; y1; : : : ; yd ) = (A(x; y1 ); : : : ; A(x; yd )): 1
1
For R = R1 Rd 2 R(m1 ; d1 ), we rst show that 1
jPx;y ;:::;yd [(A(x; y1 ); : : : ; A(x; yd )) 2 R] ? vol(R)j d1 = 4 : 1
1
1
For i 2 [d1 ], let Di (x; y1 ; : : : ; yi ) denote the event that (A(x; y1 ); : : : ; A(x; yi )) is in R1 Ri . Also let pi = Px;y ;:::;yi [Di(x; y1 ; : : : ; yi )], qi = Qij=1 jmRj j , and ri+1 = Px;y ;:::;yi [A(x; yi+1 ) 2 Ri+1 jDi (x; y1 ; : : : ; yi )]. The following is a simpli ed version of a lemma due to Nisan and Zuckerman [12], based on our choice of parameters. 1
1
1
+1
Lemma 3.2 For any i 2 [d1 ], jpi ? qij i. Proof: Use induction on i. It's true for i = 1 from the de nition of extractors. Assuming that jpi ? qij i, we'll show that jpi+1 ? qi+1j (i + 1). jp ? q j = jp r ? p jRi+1j + p jRi+1 j ? q jRi+1 j j i+1
i+1
i i+1
i
m1
i
i
m1 pijri+1 ? jRmi+1j j + jpi ? qij:
m1
1
There are two cases. pi :
jpi+1 ? qi+1j + i = (i + 1):
pi > :
The distribution of x conditioned on Di (x; y1 ; : : : ; yi ) has min-entropy at least s ? log 1= s ? . From Corollary 3.1, jPx;y ;:::;yi [A(x; yi+1 ) 2 Ri+1 jDi (x; y1 ; : : : ; yi )] ? jRmi j j , and jpi+1 ? qi+1 j + i = (i + 1): Now let m2 = 2t = (1=)O(1) = (d1 =)O(1) = (1=)O(1) , d2 = d1 , and V2 = [m2 ]d . Consider the reduction F2 = ffx j x 2 f0; 1gs g, where fx : V2 ! V1 is de ned as follows +1
1
1
2
fx (y1 ; : : : ; yd ) = G(x; y1 ; : : : ; yd ): 2
1
Then fx?1 (R) = R10 Rd0 2 R(m2 ; d2 ), where Ri0 = fyi j A(x; yi ) 2 Ri g. Also, 2
jEx[vol(fx?1 (R))] ? vol(R)j = jPx;y ;:::;yd [G(x; y1 ; : : : ; yd ) 2 R] ? vol(R)j =4: 1
So we have the following lemma. Lemma 3.3 F2 is (R(m1 ; d1 ); R(m2 ; d2 ); =4)-good. 5
1
1
3.4 The Second Dimension Reduction Function Family
Let R = R1 Rd 2 R(m2 ; d2 ). We want to partition the d2 dimensions of R into d3 parts using some function h : [d2 ] ! [d3 ] in the natural Q way. For q 2 [d3], those dimensions of R that are mapped to q form a subrectangle Rh? (q) = i2h? (q) Ri . Based on the idea of Even et al. [5], its volume can be approximated by sampling from the k-wise independent space G = Hk (d2 ; m2 ). We use d3 copies of G, one for each subrectangle. The corresponding rectangle R(h) = R10 Rd0 , where Rq0 = fp 2 G : ph? (q) 2 Rh? (q) g, should have a volume close to that of R. The error depends on the choice of k and h. We will show that for k = O(log1=2 1=) and h chosen uniformly from H = H(d2 ; d3 ), the expected error is at most =4. More formally, let d3 = 2(3 log )=(k?1) , m3 = jGj, V3 = [m3 ]d , and F3 = ffh : h 2 H g, where fh : V3 ! V2 is de ned as follows 2
1
1
3
1
1
12
3
fh(p1 ; : : : ; pd ) = (ph(1) (1); : : : ; ph(d ) (d2 )): 3
2
Then for R 2 R(m2 ; d2 ) and fh 2 F3 , fh?1 (R) = R(h) 2 R(m3 ; d3 ). We also need the following notation for the proofs below. For R = R1 Rd 2 R(m2 ; d2 ), let R~ denote the rectangle R1 Rd 2 R(m2 ; d2 ), where Ri = [m2 ] n Ri . For i; j 2 [d2 ], and I [d2 ], denote 2
2
Ri j ; i = jm 2 Y (I ) = vol(R~ I ) = i; i2I
(I ) = Pp2G [pI 2 R~ I ]; and X (I ): j (I ) = J I;jJ j=j
The approximation error of each subrectangle can be bounded in the following way. Proposition 3.1 8I [d2 ]; jPp2G [pI 2 RI ] ? vol(RI )j k (I ) Proof: Because G is a k-wise independent space, for J I with jJ j k, (J ) = (J ). From the principle of inclusion and exclusion, we have the following.
Y
vol(RI ) = (1 ? i ) = i2I
Pp2G [pI 2 RI ] =
jI j X (?1)jJ j(J ) = Xk (?1)j (I ) + X (?1)j j (I ): j j =0
J I
j =k+1
jI j X X (?1)jJ j (J ) = Xk (?1)j (I ) + X (?1)j j
J I
j =0
j =k+1
J I;jJ j=j
(J ):
P ?1(?1)j j (I ) Now the proposition follows because both vol(RI ) and Pp2G [pI 2 RI ] fall between kj =0 P and kj=0 (?1)j j (I ). The approximation error of any partition can be bounded by the following. Lemma 3.4 8h : [d2 ] ! [d3 ]; jvol(R(h) ) ? vol(R)j Pq2[d ] k (h?1 (q)) 3
6
Proof:
Y vol(R ? ): h (q) q2[d ] Y P [p ? 2 R ?
vol(R) =
1
3
vol(R(h) ) =
q2[d3 ]
h 1 (q) ]:
p2G h 1 (q)
lemma follows from the previous proposition and the known fact that j Pli=1This jxi ? yij when 0 xi; yi 1 for all i 2 [l].
Qli=1 xi ? Qli=1 yij
Finally, we can bound the expected approximation error. Lemma 3.5 For R 2 R(m2 ; d2 ), jEh2H [vol(R(h) )] ? vol(R)j 4 .
Proof:
jEh2H [vol(R(h) )] ? vol(R)j Eh2H [jvol (R(h) ) ? vol(R)j] X Eh2H [ k (h?1 (q))]
X Eq2[d ][ X (I )] h2H q2[d ] X XIh?P(q);jI[j=8ik 2 I h(i) = q](I ) h2H q2[d ] I [d ];jI j=k X X (1=d )k (I ) 3
=
1
3
=
3
= =
P
2
q2[d3 ] I [d2 ];jI j=k (1=d3 )k?1 k ([d2 ]):
3
Let = i2[d ] i . There are two cases depending on the value of . log 12 : k ([d2 ]) gets it maximum value when i = d for all i 2 [d2 ]. So k ([d2 ]) ( edk )k ( d )k ( e logk )k , which is again maximized when k = log 12 . So for d3 = 2(3 log )=(k?1) , we have 2
2
2
12
12
2
jEh2H [vol(R(h) )] ? vol(R)j 2?3 log elog 12
= 2?(3?log e) log = ( 12 )3?log e
12 12
12 : > log 12 :
In this case, both Eh2H [vol (R(h) )] and vol(R) are small, so their di erence is small. First, P vol(R) = Qi2[d ] (1 ? i ) 2? i2 d i < 12 . we show that Eh2H [vol(R(h) )] 3 . Let d0 be the smallest integer such that log 12 ? 1 < Pi2[Next, 12 0 d ?d0 . From the previous case Eh2H [vol(R0(h) )] vol(R0 ) + d0 ] i log . Let R = R[d0 ] [m2 ] 2
[ 2]
2
7
12 . So
Eh2H [vol(R(h) )] Eh2H [vol(R0(h) )] vol(R0) + 12 P 0 i ? 2 i2 d + 12 2? log +1 + 12 = : [
]
12
4 0 Eh2H [vol(R(h) )]; vol(R) 4 . So jEh2H [vol(R(h) )] ? vol(R)j 4 . So we have the following lemma.
Lemma 3.6 F3 is (R(m2 ; d2 ); R(m3 ; d3 ); 4 )-good. Theorem 3.1 There is an explicit -generator for R(m; d), using O(log m +log d +log3=2 1=) bits. Proof: jF1 j = dO(1) , jF2 j = 2s = (md=)O(1) , and jF3 j = jHk (d2 ; d3 )j dk2 = (1=)O(k) . The INW-generator gives us an 4 -generator for R(m3 ; d3 ) using O(log m3 + (log d3 + log 1=) log d3 ) = O(k log 1= + 1=k log2 1=) bits. From Lemma 2.1, we have an -generator for R(m; d) using
O(log m + log d + k log 1= + 1=k log2 1=) bits. When k = O(log1=2 1=), the number of bits used gets its minimum value O(log m + log d + log3=2 1=).
4 A Potential Improvement
For F3 , we can replace the k-wise independent space H by an approximate k-wise independent space H 0 , over [d3 ]d , such that for any I [d2 ] with jI j k and for any y 2 [d3 ]d , jP 0 [x = y ] ? ( 1 )jI jj O( ): 2
2
x2H
I
I
d3
d3
A simple generalization from the constructions of Alon, Goldreich, Hastad, and Peralta [1], or Naor and Naor [9] gives us jH 0 j = ( k log d )O(1) = ( 1 )O(1) . H 0 can also be identi ed eciently with jH 0 j. One can easily verify that only an additional O() error is introduced in Lemma 3.5, and now the reduction cost for F3 is O(log 1=). From now on we will use H 0 instead of H in F3 . Recall from the previous section that m3 = jHk (d2 ; m2 )j = (1=)O(k) and d3 = 2(3 log 1=)=(k?1) . Larger k implies smaller d3 but larger m3 . The optimum is attained at k = (log1=2 1=). If we can replace Hk (d2 ; m2 ) by a smaller space, we might be able to choose a larger k and get a smaller d3 . Remeber that d3 copies of Hk (d2 ; m2 ) are used to approximate the volumes of the d3 subrectangles of R partitioned by a function h : [d2 ] ! [d3 ]. The approximation is guaranteed by the fact that for R 2 R(m2 ; d2 ) and for J [d2 ] with jJ j k, j(J ) ? (J )j = jvol(R~J ) ? Pp2G[pJ 2 R~J ]j = 0: 2
We want to use a smaller space by allowing a small error 0 instead of 0 above. The approximate k-wise independent space does not help here, because it needs (k log m2 + log 1=0 ) bits to achieve an error 0 here, no better than G. However, observe that what we need here is to approximate the 8
volume of a rectangle with at most k nontrivial dimensions. This turns out to be a special case of our original problem | constructing a pseudorandom generator for R(m; d; k). Suppose that g : f0; 1gs ! [m2 ]d is an 0 -generator for R(m; d; k). In F3 , we replace Hk (d2 ; m2 ) by the space generated by g. Let m3 = 2s . For h 2 H 0 , let fh : [m3 ]d ! [m2 ]d be de ned as follows fh(x1 ; : : : ; xd ) = (g(xh(1) )(1); : : : ; g(xh(d ))(d2 )): For R = R1 Rd 2 R(m2 ; d2 ), fh?1 (R) = R(h) = R10 Rd0 2 R(m3 ; d3 ), where Rq0 = fx 2 [m3 ] : g(x)h? (q) 2 Rh? (q) g. Then for J h?1 (q) with jJ j k, jPx2[m ][g(x)J 2 R~ J ] ? vol(R~J )j 0. So, for any h : [d2 ] ! [d3 ], X (( X 0) + (h?1(q))) jvol(R) ? vol(R(h) )j k 2
3
3
2
2
2
3
1
1
3
q2[d3 ] J h?1 (q);jJ jk dk2+20 + k (h?1 (q)); q2[d3 ] and jEh2H 0 [vol(R(h) )] ? vol(R)j dk2 +2 0 + 4 = =2, for 0 = 4dk+2 . 2
X
d3 .
So if we have a better 0 -generator for R(m2 ; d2 ; k), we can choose a larger k and thus a smaller
Theorem 4.1 If there exists an explicit -generator for R(m; d; k) using O(k +log d +log m +log 1 ) bits, then there exists an explicit -generator for R(m; d) using O(log d +log m +log 1 log log 1 ) bits. k2
d m )O(1) = ( d )O(1) . We want Proof: Using the 0 -generator for R(m2 ; d2 ; k) in F3 gives m3 = ( 2k=d k 2
2
to repeatedly reduce the dimensions of rectangles. Notice that each time the dimension is reduced, we can choose a larger next k. For 3 i l = log log 12 ? log log log 12 , let ki = 2 i ; di = 2O((log )=ki ) = 2O((log )=2i ) ; i = ki +2 = O(1) ; and 4di?1 ki m = ( 2 mi?1 di?1 )O(1) = ( 1 )O(i) 2
1
i
1
i
For 3 i l, let Fi be the dimension reduction discussed before, using the assumed i -generator for R(mi?1 ; di?1 ; ki ). One can check that each Fi is (R(mi?1 ; di?1 ); R(mi ; di ); )-good. Using the INW-generator as an -generator for R(ml ; dl ), we have an O( log log 1 )-generator for R(m; d). The total number of bits used is O(log m + log d + log 1 + log 1 log log 1 + log ml + (log dl + log 1 ) log dl ) = O(log m + log d + log 1 log log 1 ):
Replacing log log 1 by , we have an -generator for R(m; d) using O(log m +log d +log 1 log log 1 ) bits. We don't know yet how to construct such an explicit -generator for R(m; d; k) using O(log k + log d + log m + log 1=) bits. Using an idea of Auer, Long, and Srinivasan [2], we can derive one using O(log k + log m + log3=2 1=) bits, which improves their upper bound, but does not serve our purpose here. 9
5 The Pseudorandom Generator for B(m; d; t)
Recall that B(m; d; t) denotes the class of rectangles from R(m; d) with at most t nontrivial dimensions and each dimension being an interval. For B(m; d; t), Even et al. [5] have an -generator using O(t + log log d + log 1=) bits. Unfortunately, we cannot apply the iterative procedure in the previous section to B(m; d; d) because after applying the dimension reduction once, each dimension is no longer an interval. For t log 1=, Chari et al. [4] had an -generator for B(m; d; t) using O(log log d+log 1= log logt1= ) bits, a signi cant improvement in the dependence on t. This is improved again by the following theorem.
Theorem 5.1 For t log 1=, there is an explicit -generator for B(m; d; t), using O(log log d + log 1= log1=2 logt1= ) bits.
Proof: Here we use only one reduction, the modi ed dimension reduction discussed in the previous section. Let k t be a parameter to be chosen later. For 0 = ( kt )O(k) , let g : f0; 1gs ! [m]d be the 0 -generator of Even et al. for B(m; d; k). Let m0 = 2s and d0 = 2(3 log )=(k?1) . Given R = R1 Rd 2 B(m; d; t), assume w.l.o.g. that the rst t dimensions are nontrivial. For h 2 H 0 let R(h) = fh?1 (R) = R10 Rd0 , where Rq0 = fx 2 [m0 ] : g(x)h? (q) 2 Rh? (q) g. For J 6 [t], jvol(R~J ) ? Px2[m0][g(x)J 2 R~J ]j = 0; because Rj = ; for j 62 [t]. For J [t] with jJ j k, jvol(R~J ) ? Px2[m0 ][g(x)J 2 R~J ]j 2jJ j0; as each R~ J is the union of at most 2jJ j rectangles from B(m; jJ j; jJ j). Then 12
1
jEh2H 0 [vol(R(h) )] ? vol(R)j
Xk X
2j 0 + Eh2H 0 [
j =0 J [t];jJ j=j ( t )O(k) 0 + O()
X (h?1(q))]
q2[d0 ]
k O():
1
k
Combined with the INW-generator, we get an -generator using O(log k log d + k log kt +log 1 logk1= ) = 1= O(log k +log log d +log 1= + k log kt + log k1= ) bits. Choosing k = log log = t results in O (log log d + = log 1= log1=2 logt1= ). 2
1 2
log 1
6 Acknowledgements We would like to thank David Barrington for correcting some mistakes and making useful suggestions. We would like to thank Shiyu Zhou for telling us the result in [13] and for some helpful comments. We would also like to thank Amnon Ta-Shma, Jaikumar Radhakrishnan, and Avi Wigderson for reading this paper.
10
References [1] N. Alon, O. Goldreich, J. Hastad, and R. Peralta, Simple constructions of almost k-wise independent random variables, Random Structures and Algorithms, 3(3), pages 289-303, 1992. [2] P. Auer, P. Long, and A. Srinivasan, Approximating hyper-rectangles: learning and pseudorandom sets, In Proceedings of the 29th Annual ACM Symposium on Theory of Computing, 1997. [3] R. Armoni, M. Saks, A. Wigderson, and S. Zhou, Discrepancy sets and pseudorandom generators for combinatorial rectangles, In Proceedings of the 37th Annual IEEE Symposium on Foundations of Computer Science, pages 412-421, 1996. [4] S. Chari, P. Rohatgi, and A. Srinivasan, Improved algorithms via approximations of probability distributions, In Proceedings of the 26th Annual ACM Symposium on Theory of Computing, pages 584-592, 1994. [5] G. Even, O. Goldreich, M. Luby, N. Nisan, and B. Velickovic, Approximations of general independent distributions. In Proceedings of the 24th Annual ACM Symposium on Theory of Computing, pages 10-16, 1992. [6] O. Goldreich and A. Wigderson, Tiny families of functions with random properties: a qualitysize trade-o for hashing, In Proceedings of the 26th Annual ACM Symposium on Theory of Computing, pages 574-583, 1994. [7] R. Impagliazzo, N. Nisan, and A. Wigderson, Pseudorandomness for network algorithms, In Proceedings of the 26th Annual ACM Symposium on Theory of Computing, pages 356-364, 1994. [8] N. Linial, M. Luby, M. Saks, and D. Zuckerman, Ecient construction of a small hitting set for combinatorial rectangles in high dimension, Combinatorica, 17, pages 215-234, 1997. [9] J. Naor and M. Naor, Small-bias probability spaces: ecient constructions and applications, SIAM Journal on Computing, 22(4), pages 838-856, 1990. [10] N. Nisan, Pseudorandom generators for space-bounded computation, Combinatorica, 12, pages 449-461, 1992. [11] N. Nisan, Extracting randomness: how and why - a survey, In Proceedings of the 11th Annual IEEE Conference on Computational Complexity, pages 44-58, 1996. [12] N. Nisan and D. Zuckerman, Randomness is linear in space, Journal of Computer and System Sciences, 52(1), pages 43-52, 1996. [13] J. Radhakrishnan and A. Ta-shma, Private communication.
11
