On Construction of k-wise Independent Random Variables
On Construction of k-wise Independent Random Variables Howard Karlo
Yishay Mansour
y
Abstract A 0-1 probability space is a probability space ( ; 2 ; P ), where the sample space f0; 1gn for some n. A probability space is k-wise independent if, when Yi is de ned to be the ith coordinate of the random n-vector, then any subset of k of the Yi 's is (mutually) independent, and it is said to be a probability space for p1 ; p2 ; :::;pn if P [Yi = 1] = pi . We study constructions of k-wise independent 0-1 probability spaces in which the pi 's are arbitrary. It? was ?known ?that for any ?p1; p2; :::;pn , a k-wise independent probability space of size m(n; k) = nk + k?n 1 + k?n 2 + + n0 always exists. We prove that for some p1 ; p2 ; :::;pn 2 [0; 1], m(n; k) is a lower bound on the size of any k-wise independent 0-1 probability space. For each xed k, we prove that every k-wise independent 0-1 probability space when each pi = k=n has size (nk ). For a very large degree of independence|k = b nc, for > 1=2|and all pi = 1=2, we prove a lower bound on the size of 2n (1 ? 21 ). We also give explicit constructions of k-wise independent 0-1 probability spaces.
1 Introduction A 0-1 probability space H is a probability space ( ; 2 ; P) whose sample space, , is a set of 0-1 strings of length n (for some n). The size of a probability space H = ( ; 2 ; P) is j j. H is k-wise independent if, when Yi is de ned to be the ith coordinate of the random n-vector, then any subset of k of the Yi 's is (mutually) independent, and it is said to be a probability space for p1; p2; :::; pn if P[Yi = 1] = pi . A great deal of research in theoretical computer science recently has dealt with the problem of constructing small probability spaces satisfying certain independence constraints, most commonly kwise independence [Jo e74, Lub86, ABI86, Lub88]. Aside from its interest as a fundamental issue, such small probability spaces can often be used for derandomizing randomized algorithms [ABI86, Lub86, KM93, Sch92, BR89, MNN89]: if one has a randomized algorithm which needs to be run on a k-wise independent 0-1 probability space, then it can be derandomized by running it deterministically on all the points in such a space (in parallel, if possible). This deterministic algorithm will be ecient if the size of the sample space is small, so that not too much time (or processors, in the parallel case) will be needed, and if the algorithm runs quickly on each point of the sample space.
College of Computing, Georgia Institute of Technology, Atlanta, GA 30332-0280. This author was supported in part by NSF grant CCR 9107349. y Computer Science Dept., Tel-Aviv University. This research was supported in part by the Israel Science Foundation administered by the Israel Academy of Science and Humanities and by a grant of the Israeli Ministry of Science and Technology.
1
Almost all previous work has dealt with the construction of uniform k-wise independent 0-1 probability spaces. These are spaces in which all points ! 2 have the same probability. For derandomization, nonuniform spaces are as good as uniform ones. A sample space whose j j points have many di erent probabilities will serve just as well. Suppose, for example, that a probabilistic construction needs a pairwise independent 0-1 probability space for p1; p2; :::; pn. The conclusion of the existence proof is that some structure exists with positive probability. Then if we have a \small" pairwise independent 0-1 probability space for p1 ; p2; :::; pn, then we can try each of the points of , and we will be guaranteed to nd one for which the structure exists. We start with lower bounds. De ne m(n; k) = nk + k ?n 1 + k ?n 2 + + n0 : For constant k, m(n; k) is (nk ). The best general lower bound is from [ABI86, CGHFRS85]; cf. [AS92]. Theorem 1. [ABI86, CGHFRS85] If H = ( ; 2 ; P) is a k-wise independent 0-1 probability space for p1 ; p2; :::; pn and all pi 2 (0; 1), then j j m(n; bk=2c). In fact, the theorem of [ABI86, CGHFRS85] is actually stronger, since it applies to random variables with arbitrary real range. We show that this lower bound is not always tight. We prove a lower bound of m(n; k) for certain speci ed probabilities: Theorem 2. Let n be a positive integer and let 1 k n. Then there exist p1; p2; :::; pn 2 (0; 1) such that the size of every k-wise independent 0-1 probability space for p1 ; p2; :::; pn is at least m(n; k). Even when all the probabilities are identical, we can prove a lower bound on the size that is (nk ), provided that k is xed: Theorem 3. Fix k 2. The size of every k-wise independent 0-1 probability space when every pi = k=n is (nk ). In fact, we show that the result holds even if each pi is suciently close to k=n. Note that neither Theorem 2 nor Theorem 3 can be improved by allowing the probabilities to be arbitrary, for there is a k-wise independent 0-1 probability space where all pi = 1=2 of size at most 2(2n)bk=2c [ABI86]. How close to optimal are these theorems? It is implicit in the work of D. Koller and N. Megiddo [KM93], who seem to have initiated the systematic study of nonuniform spaces in theoretical computer science, that there is always a space of size at most m(n; k): Theorem 4. [KM93] Let n 1, let 1 d n, and let p1; p2; :::; pn 2 [0; 1]. Then there is a k-wise independent 0-1 probability space for p1 ; :::; pn of size at most m(n; k). This implies that Theorem 2 is exactly tight and the bound of Theorem 3 is tight to within a constant factor (which depends on k but not n). We also study k-wise independent 0-1 probability spaces when k is very large, say, k = b nc for 2 (1=2; 1]. In this case the best lower bound on size was m(n; bk=2c) (from Theorem 1). This quantity is bounded above by n , where < 2 is a function only of . Based on the techniques of [Fri92] we improve this bound as well. 2
Theorem 5. Let 1=2 < 1. Then the size of any b nc-wise independent 0-1 probability space when all pi = 1=2 is at least 2n(1 ? 21 ). Since it is trivial to build an n-wise independent 0-1 probability space on = f0; 1gn for any sequence p1 ; p2; :::; pn, this theorem is also tight to within a constant factor. Furthermore, when 2 (0; 1=2), the
upper bound of Theorem 4 implies that there is a k-wise independent 0-1 probability space of size at most n for some < 2, so one cannot give an (2n) lower bound for any xed < 1=2. We now turn to construction of small k-wise independent 0-1 probability spaces. We rst give a general construction for arbitrary pi. Namely, given p1; p2; :::; pn 2 [0; 1], which are possibly irrational, we give an explicit construction of a k-wise independent 0-1 probability space for p1; :::; pn. The size of the sample space is at most (8n)2k . Note that for a xed k the size of the sample space is bounded by a polynomial in the lower bound. (In fact, the Koller-Megiddo technique shows how to generate in polynomial time a k-wise independent 0-1 probability space of size at most m(n; k), for any p1; :::; pn, but it is not a \construction." One has to execute n times an algorithm that takes a feasible solution to a linear program and produces a basic feasible solution. On the other hand, their method is more general in that it is not restricted to k-wise independence.) An interesting and well-studied special case of k-wise independent 0-1 random variables is the case in which k = 2 and every pi = p. For this case we give a construction with a sample space of size O(n2), which is the best one could hope for if p is to be arbitrary, by Theorem 3. For some speci c ranges we show that linear-size constructions are possible. The paper is organized as follows. In Section 2 we give the lower bound in the case that the probabilities are di erent. Section 3 derives a lower bound in the case that the probabilities are the same. The lower bound for a large degree of independence appears in Section 4. The general construction with arbitrary probabilities is in Section 5. The construction for pairwise independent random variables is in Section 6.
2 Lower Bound for Di erent Probabilities Throughout this section, let 0 < < 1=m(n; k) and let pi = 2i?1 : We derive a tight lower bound of m(n; k) on the size of any k-wise independent 0-1 probability space H for p1; p2; :::; pn. First we de ne a lexicographic ordering between sets.
De nition. Let S = fs1 ; : : :; sl g and T = ft1 ; : : :; tjg be distinct subsets of f1; 2; :::;ng, where s1 > s2 > > sl and t1 > t2 > > tj . Then S T if either (1) S is a proper superset of T, or (2) there exists an m such that si = ti for i = 1; 2; :::; m and sm+1 > tm+1 . (Notice that either S T or T S if S 6= T.) Q For a set S f1; 2; :::; ng and for any y 2 f0; 1gn, with ith coordinate yi , let IS (y) = i2S yi . Since the distribution is k-wise independent, for any speci c set S of Q size at most k the probability of the Q event [IS (y) = 1] is exactly i2S pi, so let us use qS to denote i2S pi. The following lemma shows that the q's of di erent sets are well separated. 3
Lemma 6. Let S = fs1; : : :; sl g and T = ft1 ; : : :; tj g be sets of size at most k such that S T. Then qT =qS = (pt1 ptj )=(ps1 psl ) 1= > m(n; k):
Proof. If T is a proper subset of S, then the claim follows from the fact that for any i, 1=pi 1= > m(n; k), so assume T is not a proper subset of S. Without loss of generality t1 > t2 > >Q tj , s1 > s2 > Q > sl , si = ti for i Qm, sm+1 >Qtm+1 . By de nition, qT = Q pt1 pt2 pQ tj = ( im pti )( i>m pti ); and qS = ps1 ps2 psl = ( im psi )( i>m psi ): Thus qT =qS = i>m pQti = i>m pQ si : Let r = tm+1 . Since r sm+1 tm+1 + 1 = r + 1, the numerator of the quotient is at least i=1 pi = ri=1 2i?1 = 2r ?1: The denominator is at most psm+1 = 2sm+1 ?1 2r : Thus qT =qS (2r ?1)?2r = 1= > m(n; k): Lemma 7. Let H be a k-wise independent 0-1 probability space for the p1 ; p2; :::; pn de ned above. For any subset T of f1; 2; :::;ng of size at most k, there is a vector xT assigned positive probability by H such that IT (xT ) = 1, and IS (xT ) = 0 for all S such that S T and jS j k. Proof. Consider the following event: [IT (x) = 1 and IS (x) = 0 for all S such that S T, jS j k]. The probability of this event is bounded from below by P[IT (x) = 1] ?
X
S T;
P[IS (x) = 1]:
jS jk
Since the size of T is at most k and the size of each S is at most k,P the probability of the event can be bounded from below via k-wise independence. The result is qT ? S T; qS : By Lemma 6, for any jS jk P S T, qT qS = . Since m(n; k) < 1= , we have qT ? S T; qS > 0: Thus the probability of the event jS jk
[IT (x) = 1 and IS (x) = 0 for all S such that jS j k and S T] is positive. It follows that there is some vector xT assigned positive probability by H that has the required properties. Now Theorem 2 follows from Theorem 8.
Theorem 8. Let H = ( ; 2 ; P) be any k-wise independent 0-1 probability space for the p1; :::; pn de ned above. Then j j m(n; k): Proof. By Lemma 7, for any set T of size at most k, there is a vector xT 2 f0; 1gn having positive probability in H such that IT (xT ) = 1 and also IS (xT ) = 0 for any S of size at most k such that S T. We need to show for any S and T, both of size at most k and satisfying S = 6 T, that xS = 6 xT . Without loss of generality assume that S T. Hence, IS (xT ) = 0. But by the de nition of xS , IS (xS ) = 1. Therefore xS = 6 xT . Remark: It is easy to see that it is not necessary that each probability be exactly pi; there is some
slack in the proof. This means that for each n and i there is a subinterval of [0; 1] of positive length so that if p0i is in the given interval for all i, then the size of any k-wise independent 0-1 probability space for p01; p02; :::; p0n is at least m(n; k).
4
3 Lower Bound for Identical Probabilities Fix k 2. Let p1; p2; :::; pn be a sequence of probabilities, 2 k n. Choose reals M; M 0 0 such that M is a lower bound on the product of the smallest k ? 1 probabilities and M 0 is an upper bound on the product of the k (not k ? 1) largest probabilities. In this section we study k-wise independent 0-1 probability spaces for p1 ; p2; :::; pn. We prove an (nk ) lower bound on the size for p1 ; p2; :::; pn, provided that
?
1. M > 1:8= k?n 1 , and 2. M > M 0 0:9n=k. We will see later that these conditions are satis ed if (M; M 0 ) = ((k=n)k?1; (k=n)k ), i.e., if pi = k=n for all i. But notice that since both conditions are strict inequalities, if they are satis ed by (M; M 0 ) as above, then they are also satis ed by (M ? ; M 0 + ), where > 0 depends on n and k. Thus these conditions hold if each pi is in a small neighborhood around k=n. First, we need the following technical lemma. It will help us prove that there are many strings in
with few ones apiece. Lemma 9. Let ; > 0. Suppose S1; S2; :::; Sm f1; 2; :::;N g and xi > 0 is associated with Si such that
P
1. For j = 1; 2; :::; N, i:Si 3j xi , and P 2. mi=1 xi . Then for any integer l 2, there are at least [l ? N ]=[l(l ? 1) ] pairwise disjoint Si 's, each of size less than l. Proof. Let Tj = fi : Si 3 jg. From the assumptions about the xi's, for j = 1; 2; :::; N; Pi2Tj xi , P P and therefore Nj=1 i2Tj xi N : Order the sets so that jS1 j jS2 j P jSm jP . Choose the m x m jS jx minimal p so that j S j l, choosing p = m + 1 if all j S j < l. Then l p i i=p i i i=p i Pm x N =l: Since the sum Pm jS jx N . Therefore of all x 's is bounded below by , we i i i i i=1 P ?1 x + Pm x Pi=p?p 1 x + N =l: Therefore Pp?1 x ? N =l: have pi=1 i i=p i i=1 i i=1 i We may assume that ? N =l > 0, since otherwise the lemmaP is trivially true. Let I = ;. Let S = f1; 2; :::; p ? 1g. All sets Si for i 2 S have size less than l. Since i2S xi ? N =l > 0, there is a set St of size less than l with t 2 S . Add t to I, and eliminate from S all i such that Si \ St 6= ; (including t itself). The sum of the x's of the sets remaining in S is at least ( ? N =l) ? (l ? 1) ; since the sum of xi over all Si with Si intersecting St is at most jStj. If ( ? N =l) ? (l ? 1) > 0, we repeat the process. This process can be repeated at least [ ? N =l]=[(l ? 1) ] times. At the end, we have a set of at least [ l ? N ]=[l(l ? 1) ] pairwise disjoint sets, each of size less than l.
De nition. Identify B f1; 2; :::;ng with its characteristic n-vector. We use P[B] to denote the probability assigned by H to the characteristic vector of B. 5
Given p1; p2; :::; pn and M; M 0 as above, and a k-wise independent 0-1 probability space H, say a (k ? 1)-subset S of f1; 2; :::;ng is valid if according to H = ( ; 2 ; P), P[S] < 2M=3: We rst show that at least one sixth of the sets of size k ? 1 are valid.
Lemma 10. For every k-wise independent 0-1 probability space H as above, with the given restrictions on M and M 0, at least one sixth of the sets of size k ? 1 are valid. Proof. If S is not valid, then the characteristic vector of S must be assigned probability at least 2M=3. Clearly, the sum of the probabilities assigned by H to characteristic vectors of (k ? 1)-sets at ? n is, we most 1. Therefore the? number of invalid (k ? 1)-sets is at most 3=(2M). Because M > 1:8= k?1 have 3=(2M) < (5=6) k?n 1 .
Theorem 11. Every k-wise independent 0-1 probability space H for p1; p2; :::; pn as above has size at ? ? 7 k n n 1 least 6 k?1 45k2 = k?1 : If n 2k, then this is at least c m(n; k)=(k3 27k ) for a universal constant c > 0. Proof. Suppose f1; 2; :::; k ? 1g is valid. Let U1 ; U2; :::; Um be the proper supersets of f1; 2; :::; k ? 1g such that P[Ui] > 0. Let xi = P[Ui] and Si = Ui ? f1; 2; :::;k ? 1g. Because H is a k-wise independent 0-1 probability space for p1 ; p2; :::; pn, the chance of having ones simultaneously in P positions 1; 2; :::;k ? 1 P and j (for j k) is at most M 0 . But this probability is exactly i:Si 3j xi , so i:Si 3j xi M 0. We
have
1. Si fk; k + 1; :::; ng.
P
2. For j = k; k + 1; :::; n, i:Si 3j xi M 0, and P 3. mi=1 xi > M=3 (because f1; 2; :::;k ? 1g is valid). Apply Lemma 9 with N = n ? (k ? 1), = M 0 , = M=3. By the lemma, for all l 2, there are at least 1 1 lM ? (n ? k + 1)M 0 0 3 3 M=M ? (n ? k + 1)=l = l(l ? 1)M 0 l?1
pairwise disjoint Si 's, each of size at most l ? 1. Now set l = 6k + 1. Because M=M 0 0:9n=k, we have at least 1 1 M ? n n 6k 3 M 0 6k 45k2 pairwise disjoint Si 's, each of size at most 6k ? 1. Thus there are at least 45nk2 strings having positive probability in H with ones in positions 1; 2; :::;k ? 1 and at most (k ? 1)+(l ? 1) = 7k ? 1 ones in total. ? Now this argument obviously goes through ?for each of the 16 k?n 1 or more valid sets. This means 1 n n that, counting multiplicities, there? are at least 6 k?1 45k2 strings with at most 7k ? 1 ones each. Since ? each such string contains at most 7kk??11 < k7?k1 subsets of size k ? 1, each such string is generated fewer ? 7 k than k?1? times by?di erent valid sets. Thus the number of di erent strings of positive probability 7 k n n 1 exceeds 6 k?1 45k2 = k?1 : ) Proving that this last quantity is at least c mk3(n;k 27k for a universal ? ? constant ? c > 0 ?if k n=2 is not n dicult, if one uses facts such as (k + 1) k m(n; k) if k n=2, k?n 1 n nk , and k7?k1 27k . 6
Now the reader can verify that the two conditions above on M and M 0 are satis ed if M 0 = (k=n)k ? a k ? 1 and M = (k=n) , using inequalities such as b (a=b)b if a b 1 and [k=(k ? 1)]k?1 2 for all k 2. Theorem 3 follows.
4 Lower Bounds for Large Independence In this section we show how to derive a lower bound for the case in which the degree of independence is very large. The main tool in proving our result is the following theorem, in which Cd denotes the set of all d-dimensional subcubes of the n-dimensional cube C = f1; ?1gn. Theorem 12. Let n 1. Let g be a 2n-dimensional probability vector viewed as a function on the vertices of the n-dimensional cube C . P Let 1 d n. Let N = jfv 2 C : g(v) > 0gj. Suppose that there is an such that for all C 2 Cd , = v2C g(v): Then N 2n?1(n + 2 ? 2d)=(n + 1 ? d): The proof of the above theorem uses the Fourier analysis techniques of [Fri92]. Before giving the proof, we derive an interesting consequence.
Corollary 13. Let H = ( ; 2 ; P) be a k-wise independent 0-1 probability space where all pi = 1=2. ?n n n Then j j 2n( 2k2+2 k+2 ) = 2 (1 ? 2k+2 ): Proof. For all (n ? k)-dimensional subcubes C, Pv2H g(v) = 2?k : So let d = n ? k, = 2?k . Then N 2n?1(n + 2 ? 2(n ? k))=(n + 1 ? (n ? k)) = 2n (2k ? n + 2)=(2 + 2k): Corollary 14. Let 0 < 1 and b nc k n, and suppose that H = ( ; 2 ; P) is a k-wise independent 0-1 probability space where all pi = 1=2. Then j j (1 ? 1=(2 )) 2n : Proof. Because k b nc, 1? 2(kn+1) 1?1=(2 ). By Corollary 13, we have j j 2n(1?n=(2k + 2)) 2n(1 ? 1=(2 )). Corollary 14 implies Theorem 5. Now we do the proof of Theorem 12. We start with the following simple technical lemma, which follows from the convexity of the function f(x) = x2.
Lemma 15. Let g be a probability vector. Suppose that N = jfi : g(i) > 0gj. Then jgj2 = Pv g2(v) 1=N.
We use A to denote the 2n 2n adjacency matrix of C , i.e., the (u; v) entry A(u; v) of A is 1 if u and v di er in exactly one position, and 0 otherwise. For Q each T f1; 2; :::;ng, let T be a vector, viewed as a function onPC , whose vth coordinate T (v) = i2T vi , where v 2 f+1; ?1gn. If f; h : C ! IR, let < f; h >= 2?n v2C f(v)h(v):
Lemma 16. Let 0 r n. Then for each T f1; 2; :::;ng of size r, T is an eigenvector of A, corresponding to eigenvalue n ? 2r. Proof. Let T f1; 2; :::;ng with jT j = r. Choose any u 2 C. The uth entry of A T is
X v2C
A(u; v) T (v) =
7
X
v2N (u)
T (v);
where N(u) denotes those v 2 C di ering from u in exactly one position. Of the n v's in N(u), n ? r v's di er from u in a bit position not in T. These v's satisfy T (v) = T (u). The remaining r v's di er from u in one position in T and satisfy T (v) = ? T (u). It follows that
P
= (n ? r) T (u) ? r T (u) = (n ? 2r) T (u): Since u was arbitrary, we have A T = (n ? 2r) T : v2N (u) T (v)
It is easy to see that if T 6= U, then < T ; U >= 0, and that < T ; T >= 1 for all T. This means that the T 's are orthogonal and hence f T : T f1; 2; :::;ngg isPlinearly independent. Thus any 2n -dimensional vector g can be written as a linear combination g = T T T for some reals T . Proof of Theorem 12. As the theorem is vacuous if d = n, we assume without loss of generality that d < n. Fixing g, letPus set g^(U) =< g; U > for any U. It is easy to see, from the orthogonality of the T 's, that g(v) = T g^(T) T (v). P We are given that v2C g(v) = for all C 2 Cd . Choose any U f1; 2; :::;ng such that 1 jU j n ? d. g^(U) = < g;P U > ? n = 2 v g(v) U (v): Let us use \vjU = s" to mean that the restriction of v to the bit positions in U is s. 2ng^(U) = = =
P g(v) (v) U P Pv Ps2f1;?1gjjUUjj(QvjU:vjjUs=)(s g(v) P U (v)g(v)): i=1 i
s2f1;?1g
v:vjU =s
Now n ? jU j d and fv : vjU = sg is a cube of dimension n ? jU j. This (n ? jU j)-dimensional cube can be partitioned into 2n?jU j=2d d-dimensional subcubes. As the sum of g(v) over any d-dimensional subcube is , n?jU j X g(v) = 2 2d : v:vjU =s
Thus 2n g^(U) = Now
1 0 jU j 1 0 jU j n?jU j X Y X X @Y si : si A @ g(v)A = 2 2d
s2f1;?1gjU j i=1
v:vjU =s
jU j X Y s2f1;?1gjU j i=1
s2f1;?1gjU j i=1
si = [1 + (?1)]jU j = 0;
since jU j 1. Therefore g^(U) = 0 for any U such that 1 jU j n ? d. P Let us de ne h(v) = (Ag)v = u2N (v) g(u). Now h(v) = (Ag)v = A
"X T
g^(T) T
#!
v
=
X T
g^(T)(A T )v =
X T
(n ? 2jT j)^g(T) T (v);
where the third equality follows from the linearity of A and the last equality follows from Lemma 16. Above we showed that for any T with 1 jT j n ? d, g^(T) = 0. However, g^(;) = 1=2n, since g is a probability vector. 8
We would like to derive an upper bound on jgj2. To do this we consider < h; g >. Clearly, < h; g > 0. On the other hand, < h; g >= 2?n
X v
g(v)h(v) = 2?n
= 2?n
XX
T U n The last summation equals 2 < T ; U
XXX v U T
[(n ? 2jT j)^g(T) T (v)] [^g (U) U (v)]
g^(U)(n ? 2jT j)^g(T)
X v
T (v) U (v):
>, which is 0 if U 6= T and 2n if U = T. Therefore X X (n ? 2jT j)^g2(T): < h; g >= (n ? 2jT j)^g2(T) = 4nn + T T :jT jn?d+1
We can bound < h; g > as follows.
X 2 0 < h; g > 4nn + (2d ? 2 ? n) g^ (T): T :jT jn?d+1 P
By Parseval's identity, jjgjj2 = T g^2 (T), where jjgjj2 =< g; g >. Hence X 2 g^ (T) : < g; g >= 2?njgj2 = 41n + T :jT jn?d+1 Using this identity we have 2 0 4nn + (2d ? 2 ? n) j2gnj ? 41n = 2n +42n? 2d + 2d ?22n ? n jgj2: Therefore 1 2n n + 2 ? 2d : jgj2 2 n + 1 ? d By Lemma 15, jgj2 1=N. It follows that n n + 2 ? 2d 1 2 N jgj2 2 n + 1 ? d :
5 General Construction In this section we give a general construction for k-wise independent 0-1 random variables. The construction receives as input the desired p1 ; p2; :::; pn. We rst need a simple lemma, which is an easy consequence of the Moebius inversion formula. Lemma 17. Let H = ( ; 2 ; P) be a 0-1 probability space. De ne Yi to be the ith bit of an n-vector chosen according to H. Let 1 k n. Suppose that thereQ are reals p1; p2; :::; pn such that for all subsets S of f1; 2; :::;ng of size at most k, P[\i2S [Yi = 1]] = i2S pi : Then H is a k-wise independent 0-1 probability space for p1; p2; :::; pn. We omit the simple proof of the following corollary. Corollary 18. Suppose H is a k-wise independent 0-1 probability space for p1; p2; :::; pn. Suppose that p0i = 1 ? pi if i 2 S, p0i = pi otherwise. A k-wise independent 0-1 probability space for p01; p02; :::; p0n can be obtained from H by interchanging zeroes and ones in every position i 2 S in each point of the probability space. 9
Our general construction for p1; p2; :::; pn consists of three parts: (1) a construction for probabilities ti very close to 1, (2) a construction for s1 ; s2 ; :::; sn such that each si either equals pi or is slightly larger, and (3) a method of combining the two probability spaces so as to get p1; :::; pn exactly. We must give a construction for very large probabilities. Since it is easier to understand what's going on when the probabilities are very small, we will assume ti 1=(2n) for all i, and then invoke Corollary 18 to get the space for ti 1 ? 1=(2n).
Lemma 19. Let = 1=(2n). Suppose that 0 t1; t2; :::; tn . There is a k-wise independent 0-1 probability space H for t1 ; t2; :::; tn whose points are the elements of f0; 1gn with at most k ones. Its size is at most m(n; k). Proof. We assign a probability to each 0-1 vector as follows. Identify B f1; 2; :::; ng with its characteristic n-vector. We use P[B] to denote the probability that H assigns to the characteristic vector of B. Set P[B] = 0 if jB j > k. We will create probabilities P[B] for each B f1; 2; :::; ng, jB j k, such that for all A f1; 2; :::;ng such that jAj k, PB:BA P[B] = qA = Qi2A ti : In H, wherePYi denotes the random variable which is the ith bit of the random n-vector, we have P[\i2A[Yi = 1]] = B:BA P[B] = qA: By Lemma 17, this is enough. We assign probabilities P[B] by downward induction on jB j. As each P[B] is de ned, we prove inductively that 0 P[B] qB . If jB j = k, set P[B] = qB . (Notice that 0 P[B] qB .) Now suppose that jB j = l, l < k, and P[C] has been de ned for all C of size exceeding l. Set P[B] = qB ?
X
C :C B
P[C] = qB ?
6?
X
C :C B;jC jk
P[C]:
6?
P It is obvious with this de nition that qB = C :C B P[C]: Now if 0 P[C] qC for all C of size exceeding l, then clearly P[B] qB . Also,
P
P[B] qB ? C :C B;jC jk qC P 6??l ?n?li qB ? qBP ki=1 i i qB [1 ? P1 i=1 (n) ] 1 1 i = qB [1 ? i=1 ( 2 ) ] = 0: That m(n; k) is the number of 0-1 vectors with at most k ones is obvious. Now we give a construction and lemma due to Jo e [Jo e74]. Let r n be a prime. Suppose that s1 ; s2 ; :::; sn 2 [0; 1] satisfy si = ji =r for all i, for some integers ji . De ne a probability space, with (at most) nk points, as follows. Choose a0; a1; :::; ak?1 uniformly and independently at random from ZZr , and let Xi = a0 + a1i + a2i2 + + ak?1ik?1 mod r, for 1 i n. For each point < a0 ; a1; :::; ak?1 > in the sample space, let Yi = 1 if Xi 2 f0; 1; :::; ji ? 1g and let Yi = 0 otherwise, 1 i n. Associate with < a0 ; a1; :::; ak?1 > the vector < Y1 ; Y2; :::; Yn > and give each of the rk r-vectors probability 1=rk (and then combine identical r-vectors, if necessary).
Lemma 20. The random variables X1 ; X2; :::; Xn are k-wise independent and each Xi is uniform on ZZr . It follows that the 0-1 probability space de ned by the vectors < Y1 ; Y2; :::; Yn > is a k-wise independent 0-1 probability space for s1 ; s2 ; :::; sn. 10
Lemma 20 follows easily from the fact that Van der Monde matrices are invertible over ZZr . We now present an almost trivial technique that takes a k-wise independent 0-1 probability space A for s1 ; s2; :::; sn, and a k-wise independent 0-1 probability space A0 for t1 ; t2; :::; tn, and produces a k-wise independent 0-1 probability space H for s1 t1 ; s2t2 ; :::; sntn . To generate an element of H, simply independently pick x from A and x0 from A0 , and output the bit-wise AND x ^ x0 2 f0; 1gn. Clearly the size of the implied space H is at most jAjjA0j.
Lemma 21. The above construction of H from A and A0 guarantees that H is a k-wise independent 0-1 probability space for s1 t1 ; s2t2 ; :::; sntn .
Proof. From Lemma Q 17, it is sucient to show that for any subset S f1; 2; :::; ng of size at most k, P[\i2S [Yi = 1]] = i2S (si ti ), where Yi is the ith bit of the random n-vector x ^ x0 . But the ith bit of x ^ x0 is 1 if and onlyQif the ithQbit of x and the ith bit of x0 are both one. By independence, this occurs with probability ( i2S si )( i2S ti ): The probability space for the given probabilities p1 ; p2; :::; pn is obtained routinely as follows. 1. Assume without loss of generality that every pi 1=2. 2. Build a probability space in which the expectation si of the ith variable satis es si 2 [pi; pi + 1=(4n)]. This is done via Lemma 20 with r being a prime in [4n; 8n]. 3. Build a space for t1; t2 ; :::; tn, where ti = pi=si and hence ti 2 [1 ? 1=(2n); 1]. We do this via Lemma 19. 4. Use Lemma 21 to form a space for p1 ; :::; pn where pi = ti si . We conclude with Theorem 22. The new space is a k-wise independent 0-1 probability space for the given probabilities p1 ; p2; :::; pn. Its size is at most (8n)k m(n; k) (8n)2k .
6 Construction for Pairwise Independence In this section we exhibit a space of size at most 4n2 for pairwise independent random variables in the case that all the probabilities are equal. From the lower bound of Theorem 11 it follows that this construction is optimal in size, to within a constant factor.
De nition. Let H be a 0-1 probability space. H is an (a; b; n)-space if the probability of a 1 in position i is a for all i, and the probability of simultaneous ones in positions i and j is b, for all i < j. For a pairwise independent 0-1 probability space, we would want b = a2 . However, here we will need b < a2 . The following lemma shows that given such an (a; b; n)-space, we can build a pairwise independent 0-1 probability space if all pi = p, and if p is in a certain interval:
11
Lemma 23. Let H = ( ; 2 ; P) be an (a; b; n)-space with b < a2. Then there is a construction of a pairwise independent 0-1 probability space H if all pi = p, on a sample space of size at most j j + 2, provided that p 2 [b=a; (a ? b)=(1 ? a)]. Proof. The new probability space will be de ned in the following way. Let x; y 0 such that x + y 1. With probability x, choose a point in (according to the probabilities in P). With probability y, choose the all-ones row. With probability 1 ? x ? y, choose the all-zeroes row. It is clear that the implied probability space has size at most j j + 2. In order that the new space be pairwise independent for all pi = p, it suces to ensure that the probability of a 1 in position i is p and the probability of simultaneous ones in positions i and j is p2 (and that the sum of the probabilities of the points is 1). In other words, it is sucient to satisfy the following system: xa+y 1 = p x b + y 1 = p2 and satisfy x + y 1, x; y 0. The unique solution to the system is x = (p ? p2 )=(a ? b); y = p(ap ? b)=(a ? b): We have x + y 1, x; y 0 if and only if p 2 [b=a; (a ? b)=(1 ? a)]: Note that since b < a2 , b=a < (a ? b)=(1 ? a). Based on the above lemma we can exhibit a quadratic size space for an arbitrary probability p. Theorem 24. For any n > 1, for any 0 p 1, there is a construction of a pairwise independent 0-1 probability space of size at most 4n2 if every pi = p. Proof. First we claim that if q n is a prime, then for any l, 0 l q, there is an (l=q; (l2 ? l)=(q2 ? q); n)-space. The points of the sample space are the q2 ? q pairs (c; d), c; d 2 ZZq , d 6= 0. Let Xi (c; d) = c + di mod q and Yi (c; d) = 1 if Xi (c; d) 2 f0; 1; 2; :::;l ? 1g and Yi (c; d) = 0 otherwise, 1 i n. Build a space with at most q2 ? q vectors by associating with point (c; d) the n-vector (Y1 ; Y2; :::; Yn), and then combining identical n-vectors. That this is an (l=q; (l2 ? l)=(q2 ? q); n)-space can easily be veri ed by noting that the q omitted pairs (c; 0) de ne l rows that are all 1 and q ? l rows that are all 0, and if the q omitted rows were added, we'd have a pairwise independent space in which each random variable is 1 with probability l=q. Now for the construction of the pairwise independent probability space. Choose a prime q such that n q 2n. Choose an l, 1 l q ? 1, such that p 2 [(l ? 1)=(q ? 1); l=(q ? 1)]. The construction of this proof gives us an (l=q; (l2 ? l)=(q2 ? q); n)-space of size at most q2 ? q. Lemma 23 now gives us a pairwise independent sample space, of size at most (q2 ? q) + 2 q2 (2n)2 = 4n2, with pi = p for all i, provided that p 2 [b=a; (a ? b)=(1 ? a)]. A simple calculation with a = l=q, b = (l2 ? l)=(q2 ? q) shows that [b=a; (a ? b)=(1 ? a)] = [(l ? 1)=(q ? 1); l=(q ? 1)], which contains p. For certain ranges of p, better constructions are possible. Theorem 25 uses Lemma 23 to get a linear construction for p 1=(n ? 1), while Corollary 28 exhibits a linear construction for values in the neighborhood of 1=2. Theorem 25. For p 1=(n ? 1) there is a pairwise independent 0-1 probability space of size at most n + 2 with pi = p for all i. 12
Proof. We exhibit a (1=n; 0; n)-space; this is enough for all p 2 [b=a; (a ? b)=(1 ? a)] = [0; (1=n)=(1 ? 1=n)] = [0; 1=(n ? 1)]. This is simply the identity matrix in which each row gets probability 1=n. By Lemma 23 there is a construction of size at most n + 2. We can also get linear size constructions around 1=2. Lemma 26. Let A be a normalized mm Hadamard matrix (i.e., an mm (+1; ?1) matrix satisfying AT A = mI, whose rst row and rst column are all +1's). Let A0 be obtained from A by omitting the rst row and rst column and then replacing all ?1's by 0's. The probability space obtained by giving each of the m ? 1 rows of A0 probability 1=(m ? 1) is a ((1=2)(m ? 2)=(m ? 1); (1=4)(m ? 4)=(m ? 1); m ? 1)space. The proof is simple and is omitted.
Theorem 27. Let n > 1 and let A be a normalized m m Hadamard matrix with m n + 1. There is a pairwise independent 0-1 probability space (whose points are elements of f0; 1gn) of size at most m + 1 with pi = p for all i if p 2 [1=2 ? 1=(m ? 2); 1=2 + 1=(m ? 2)]. Proof. With a = (1=2)(m ? 2)=(m ? 1), b = (1=4)(m ? 4)=(m ? 1) (from Lemma 26), we have b=a = 1=2 ? 1=(m ? 2) and (a ? b)=(1 ? a) = 1=2. This gives a construction for p 2 [1=2 ? 1=(m ? 2); 1=2]. Its size is at most (m ? 1) + 2 = m + 1. For p 2 [1=2; 1=2+ 1=(m ? 2)], we rst build a space for 1 ? p and then exchange zeroes and ones.
Corollary 28. For any n > 1, for any p 2 [1=2 ? 1=(2n ? 2); 1=2 + 1=(2n ? 2)], there is a pairwise
independent 0-1 probability space of size at most 2n + 1 with all pi = p.
Proof. For any t, there is a 2t 2t Hadamard matrix. Given n, take m to be the least power of two exceeding n; m 2n. By Theorem 27, there is a construction of size at most m + 1 2n + 1 for any p 2 [1=2 ? 1=(m ? 2); 1=2 + 1=(m ? 2)] [1=2 ? 1=(2n ? 2); 1=2 + 1=(2n ? 2)].
7 Acknowledgments For their help in the early stages of our work, we are very grateful to D. Koller and N. Megiddo, whose paper [KM93], in opening up the area of nonuniform spaces in theoretical computer science, inspired this one. We thank Shi-Chun Tsai for pointing out a typo.
References [ABI86]
N. Alon, L. Babai, and A. Itai, \A Fast and Simple Randomized Parallel Algorithm for the Maximal Independent Set Problem," Journal of Algorithms 7 (1986), pages 567{583.
[AS92]
N. Alon and J. Spencer, The Probabilistic Method, Wiley, 1992.
[BR89]
B. Berger and J. Rompel, \Simulating (logc n)-wise Independence in NC," Proc. 30th IEEE Symposium on Foundations of Computer Science, 1989, pages 2{7.
13
[CGHFRS85] B. Chor, O. Goldreich, J. Hastad, J. Friedman, S. Rudich, and R. Smolensky, \The Bit Extraction Problem or t-Resilient Functions," Proc. 26th IEEE Symposium on Foundations of Computer Science, 1985, pages 396{407. [Fri92]
J. Friedman, \On the Bit Extraction Problem," Proc. 33rd IEEE Symposium on Foundations of Computer Science, 1992, pages 314{319.
[Jo e74]
A. Jo e, \On a Set of Almost Deterministic k-Independent Random Variables," Annals of Probability 2 (1974), pages 161{162.
[KM93]
D. Koller and N. Megiddo, Constructing Small Sample Spaces Satisfying Given Constraints," Proc. 25th ACM Symposium on Theory of Computing, 1993, pages 268{277.
[Lub86]
M. Luby, \A Simple Parallel Algorithm for the Maximal Independent Set Problem," SIAM Journal on Computing 15 (1986), pages 1036{1053.
[Lub88]
M. Luby, \Removing Randomness in Parallel Computation Without a Processor Penalty," Proc. 29th IEEE Symposium on Foundations of Computer Science, 1988, pages 162-173.
[MNN89]
R. Motwani, J. Naor, and J. Naor, \The Probabilistic Method Yields Deterministic Parallel Algorithms," Proc. 30th IEEE Symposium on Foundations of Computer Science, 1989, pages 8{13.
[Sch92]
L. Schulman, \Sample Spaces Uniform on Neighborhoods," Proc. 24th ACM Symposium on Theory of Computing, 1992, pages 17{25.
14
Yishay Mansour
y
Abstract A 0-1 probability space is a probability space ( ; 2 ; P ), where the sample space f0; 1gn for some n. A probability space is k-wise independent if, when Yi is de ned to be the ith coordinate of the random n-vector, then any subset of k of the Yi 's is (mutually) independent, and it is said to be a probability space for p1 ; p2 ; :::;pn if P [Yi = 1] = pi . We study constructions of k-wise independent 0-1 probability spaces in which the pi 's are arbitrary. It? was ?known ?that for any ?p1; p2; :::;pn , a k-wise independent probability space of size m(n; k) = nk + k?n 1 + k?n 2 + + n0 always exists. We prove that for some p1 ; p2 ; :::;pn 2 [0; 1], m(n; k) is a lower bound on the size of any k-wise independent 0-1 probability space. For each xed k, we prove that every k-wise independent 0-1 probability space when each pi = k=n has size (nk ). For a very large degree of independence|k = b nc, for > 1=2|and all pi = 1=2, we prove a lower bound on the size of 2n (1 ? 21 ). We also give explicit constructions of k-wise independent 0-1 probability spaces.
1 Introduction A 0-1 probability space H is a probability space ( ; 2 ; P) whose sample space, , is a set of 0-1 strings of length n (for some n). The size of a probability space H = ( ; 2 ; P) is j j. H is k-wise independent if, when Yi is de ned to be the ith coordinate of the random n-vector, then any subset of k of the Yi 's is (mutually) independent, and it is said to be a probability space for p1; p2; :::; pn if P[Yi = 1] = pi . A great deal of research in theoretical computer science recently has dealt with the problem of constructing small probability spaces satisfying certain independence constraints, most commonly kwise independence [Jo e74, Lub86, ABI86, Lub88]. Aside from its interest as a fundamental issue, such small probability spaces can often be used for derandomizing randomized algorithms [ABI86, Lub86, KM93, Sch92, BR89, MNN89]: if one has a randomized algorithm which needs to be run on a k-wise independent 0-1 probability space, then it can be derandomized by running it deterministically on all the points in such a space (in parallel, if possible). This deterministic algorithm will be ecient if the size of the sample space is small, so that not too much time (or processors, in the parallel case) will be needed, and if the algorithm runs quickly on each point of the sample space.
College of Computing, Georgia Institute of Technology, Atlanta, GA 30332-0280. This author was supported in part by NSF grant CCR 9107349. y Computer Science Dept., Tel-Aviv University. This research was supported in part by the Israel Science Foundation administered by the Israel Academy of Science and Humanities and by a grant of the Israeli Ministry of Science and Technology.
1
Almost all previous work has dealt with the construction of uniform k-wise independent 0-1 probability spaces. These are spaces in which all points ! 2 have the same probability. For derandomization, nonuniform spaces are as good as uniform ones. A sample space whose j j points have many di erent probabilities will serve just as well. Suppose, for example, that a probabilistic construction needs a pairwise independent 0-1 probability space for p1; p2; :::; pn. The conclusion of the existence proof is that some structure exists with positive probability. Then if we have a \small" pairwise independent 0-1 probability space for p1 ; p2; :::; pn, then we can try each of the points of , and we will be guaranteed to nd one for which the structure exists. We start with lower bounds. De ne m(n; k) = nk + k ?n 1 + k ?n 2 + + n0 : For constant k, m(n; k) is (nk ). The best general lower bound is from [ABI86, CGHFRS85]; cf. [AS92]. Theorem 1. [ABI86, CGHFRS85] If H = ( ; 2 ; P) is a k-wise independent 0-1 probability space for p1 ; p2; :::; pn and all pi 2 (0; 1), then j j m(n; bk=2c). In fact, the theorem of [ABI86, CGHFRS85] is actually stronger, since it applies to random variables with arbitrary real range. We show that this lower bound is not always tight. We prove a lower bound of m(n; k) for certain speci ed probabilities: Theorem 2. Let n be a positive integer and let 1 k n. Then there exist p1; p2; :::; pn 2 (0; 1) such that the size of every k-wise independent 0-1 probability space for p1 ; p2; :::; pn is at least m(n; k). Even when all the probabilities are identical, we can prove a lower bound on the size that is (nk ), provided that k is xed: Theorem 3. Fix k 2. The size of every k-wise independent 0-1 probability space when every pi = k=n is (nk ). In fact, we show that the result holds even if each pi is suciently close to k=n. Note that neither Theorem 2 nor Theorem 3 can be improved by allowing the probabilities to be arbitrary, for there is a k-wise independent 0-1 probability space where all pi = 1=2 of size at most 2(2n)bk=2c [ABI86]. How close to optimal are these theorems? It is implicit in the work of D. Koller and N. Megiddo [KM93], who seem to have initiated the systematic study of nonuniform spaces in theoretical computer science, that there is always a space of size at most m(n; k): Theorem 4. [KM93] Let n 1, let 1 d n, and let p1; p2; :::; pn 2 [0; 1]. Then there is a k-wise independent 0-1 probability space for p1 ; :::; pn of size at most m(n; k). This implies that Theorem 2 is exactly tight and the bound of Theorem 3 is tight to within a constant factor (which depends on k but not n). We also study k-wise independent 0-1 probability spaces when k is very large, say, k = b nc for 2 (1=2; 1]. In this case the best lower bound on size was m(n; bk=2c) (from Theorem 1). This quantity is bounded above by n , where < 2 is a function only of . Based on the techniques of [Fri92] we improve this bound as well. 2
Theorem 5. Let 1=2 < 1. Then the size of any b nc-wise independent 0-1 probability space when all pi = 1=2 is at least 2n(1 ? 21 ). Since it is trivial to build an n-wise independent 0-1 probability space on = f0; 1gn for any sequence p1 ; p2; :::; pn, this theorem is also tight to within a constant factor. Furthermore, when 2 (0; 1=2), the
upper bound of Theorem 4 implies that there is a k-wise independent 0-1 probability space of size at most n for some < 2, so one cannot give an (2n) lower bound for any xed < 1=2. We now turn to construction of small k-wise independent 0-1 probability spaces. We rst give a general construction for arbitrary pi. Namely, given p1; p2; :::; pn 2 [0; 1], which are possibly irrational, we give an explicit construction of a k-wise independent 0-1 probability space for p1; :::; pn. The size of the sample space is at most (8n)2k . Note that for a xed k the size of the sample space is bounded by a polynomial in the lower bound. (In fact, the Koller-Megiddo technique shows how to generate in polynomial time a k-wise independent 0-1 probability space of size at most m(n; k), for any p1; :::; pn, but it is not a \construction." One has to execute n times an algorithm that takes a feasible solution to a linear program and produces a basic feasible solution. On the other hand, their method is more general in that it is not restricted to k-wise independence.) An interesting and well-studied special case of k-wise independent 0-1 random variables is the case in which k = 2 and every pi = p. For this case we give a construction with a sample space of size O(n2), which is the best one could hope for if p is to be arbitrary, by Theorem 3. For some speci c ranges we show that linear-size constructions are possible. The paper is organized as follows. In Section 2 we give the lower bound in the case that the probabilities are di erent. Section 3 derives a lower bound in the case that the probabilities are the same. The lower bound for a large degree of independence appears in Section 4. The general construction with arbitrary probabilities is in Section 5. The construction for pairwise independent random variables is in Section 6.
2 Lower Bound for Di erent Probabilities Throughout this section, let 0 < < 1=m(n; k) and let pi = 2i?1 : We derive a tight lower bound of m(n; k) on the size of any k-wise independent 0-1 probability space H for p1; p2; :::; pn. First we de ne a lexicographic ordering between sets.
De nition. Let S = fs1 ; : : :; sl g and T = ft1 ; : : :; tjg be distinct subsets of f1; 2; :::;ng, where s1 > s2 > > sl and t1 > t2 > > tj . Then S T if either (1) S is a proper superset of T, or (2) there exists an m such that si = ti for i = 1; 2; :::; m and sm+1 > tm+1 . (Notice that either S T or T S if S 6= T.) Q For a set S f1; 2; :::; ng and for any y 2 f0; 1gn, with ith coordinate yi , let IS (y) = i2S yi . Since the distribution is k-wise independent, for any speci c set S of Q size at most k the probability of the Q event [IS (y) = 1] is exactly i2S pi, so let us use qS to denote i2S pi. The following lemma shows that the q's of di erent sets are well separated. 3
Lemma 6. Let S = fs1; : : :; sl g and T = ft1 ; : : :; tj g be sets of size at most k such that S T. Then qT =qS = (pt1 ptj )=(ps1 psl ) 1= > m(n; k):
Proof. If T is a proper subset of S, then the claim follows from the fact that for any i, 1=pi 1= > m(n; k), so assume T is not a proper subset of S. Without loss of generality t1 > t2 > >Q tj , s1 > s2 > Q > sl , si = ti for i Qm, sm+1 >Qtm+1 . By de nition, qT = Q pt1 pt2 pQ tj = ( im pti )( i>m pti ); and qS = ps1 ps2 psl = ( im psi )( i>m psi ): Thus qT =qS = i>m pQti = i>m pQ si : Let r = tm+1 . Since r sm+1 tm+1 + 1 = r + 1, the numerator of the quotient is at least i=1 pi = ri=1 2i?1 = 2r ?1: The denominator is at most psm+1 = 2sm+1 ?1 2r : Thus qT =qS (2r ?1)?2r = 1= > m(n; k): Lemma 7. Let H be a k-wise independent 0-1 probability space for the p1 ; p2; :::; pn de ned above. For any subset T of f1; 2; :::;ng of size at most k, there is a vector xT assigned positive probability by H such that IT (xT ) = 1, and IS (xT ) = 0 for all S such that S T and jS j k. Proof. Consider the following event: [IT (x) = 1 and IS (x) = 0 for all S such that S T, jS j k]. The probability of this event is bounded from below by P[IT (x) = 1] ?
X
S T;
P[IS (x) = 1]:
jS jk
Since the size of T is at most k and the size of each S is at most k,P the probability of the event can be bounded from below via k-wise independence. The result is qT ? S T; qS : By Lemma 6, for any jS jk P S T, qT qS = . Since m(n; k) < 1= , we have qT ? S T; qS > 0: Thus the probability of the event jS jk
[IT (x) = 1 and IS (x) = 0 for all S such that jS j k and S T] is positive. It follows that there is some vector xT assigned positive probability by H that has the required properties. Now Theorem 2 follows from Theorem 8.
Theorem 8. Let H = ( ; 2 ; P) be any k-wise independent 0-1 probability space for the p1; :::; pn de ned above. Then j j m(n; k): Proof. By Lemma 7, for any set T of size at most k, there is a vector xT 2 f0; 1gn having positive probability in H such that IT (xT ) = 1 and also IS (xT ) = 0 for any S of size at most k such that S T. We need to show for any S and T, both of size at most k and satisfying S = 6 T, that xS = 6 xT . Without loss of generality assume that S T. Hence, IS (xT ) = 0. But by the de nition of xS , IS (xS ) = 1. Therefore xS = 6 xT . Remark: It is easy to see that it is not necessary that each probability be exactly pi; there is some
slack in the proof. This means that for each n and i there is a subinterval of [0; 1] of positive length so that if p0i is in the given interval for all i, then the size of any k-wise independent 0-1 probability space for p01; p02; :::; p0n is at least m(n; k).
4
3 Lower Bound for Identical Probabilities Fix k 2. Let p1; p2; :::; pn be a sequence of probabilities, 2 k n. Choose reals M; M 0 0 such that M is a lower bound on the product of the smallest k ? 1 probabilities and M 0 is an upper bound on the product of the k (not k ? 1) largest probabilities. In this section we study k-wise independent 0-1 probability spaces for p1 ; p2; :::; pn. We prove an (nk ) lower bound on the size for p1 ; p2; :::; pn, provided that
?
1. M > 1:8= k?n 1 , and 2. M > M 0 0:9n=k. We will see later that these conditions are satis ed if (M; M 0 ) = ((k=n)k?1; (k=n)k ), i.e., if pi = k=n for all i. But notice that since both conditions are strict inequalities, if they are satis ed by (M; M 0 ) as above, then they are also satis ed by (M ? ; M 0 + ), where > 0 depends on n and k. Thus these conditions hold if each pi is in a small neighborhood around k=n. First, we need the following technical lemma. It will help us prove that there are many strings in
with few ones apiece. Lemma 9. Let ; > 0. Suppose S1; S2; :::; Sm f1; 2; :::;N g and xi > 0 is associated with Si such that
P
1. For j = 1; 2; :::; N, i:Si 3j xi , and P 2. mi=1 xi . Then for any integer l 2, there are at least [l ? N ]=[l(l ? 1) ] pairwise disjoint Si 's, each of size less than l. Proof. Let Tj = fi : Si 3 jg. From the assumptions about the xi's, for j = 1; 2; :::; N; Pi2Tj xi , P P and therefore Nj=1 i2Tj xi N : Order the sets so that jS1 j jS2 j P jSm jP . Choose the m x m jS jx minimal p so that j S j l, choosing p = m + 1 if all j S j < l. Then l p i i=p i i i=p i Pm x N =l: Since the sum Pm jS jx N . Therefore of all x 's is bounded below by , we i i i i i=1 P ?1 x + Pm x Pi=p?p 1 x + N =l: Therefore Pp?1 x ? N =l: have pi=1 i i=p i i=1 i i=1 i We may assume that ? N =l > 0, since otherwise the lemmaP is trivially true. Let I = ;. Let S = f1; 2; :::; p ? 1g. All sets Si for i 2 S have size less than l. Since i2S xi ? N =l > 0, there is a set St of size less than l with t 2 S . Add t to I, and eliminate from S all i such that Si \ St 6= ; (including t itself). The sum of the x's of the sets remaining in S is at least ( ? N =l) ? (l ? 1) ; since the sum of xi over all Si with Si intersecting St is at most jStj. If ( ? N =l) ? (l ? 1) > 0, we repeat the process. This process can be repeated at least [ ? N =l]=[(l ? 1) ] times. At the end, we have a set of at least [ l ? N ]=[l(l ? 1) ] pairwise disjoint sets, each of size less than l.
De nition. Identify B f1; 2; :::;ng with its characteristic n-vector. We use P[B] to denote the probability assigned by H to the characteristic vector of B. 5
Given p1; p2; :::; pn and M; M 0 as above, and a k-wise independent 0-1 probability space H, say a (k ? 1)-subset S of f1; 2; :::;ng is valid if according to H = ( ; 2 ; P), P[S] < 2M=3: We rst show that at least one sixth of the sets of size k ? 1 are valid.
Lemma 10. For every k-wise independent 0-1 probability space H as above, with the given restrictions on M and M 0, at least one sixth of the sets of size k ? 1 are valid. Proof. If S is not valid, then the characteristic vector of S must be assigned probability at least 2M=3. Clearly, the sum of the probabilities assigned by H to characteristic vectors of (k ? 1)-sets at ? n is, we most 1. Therefore the? number of invalid (k ? 1)-sets is at most 3=(2M). Because M > 1:8= k?1 have 3=(2M) < (5=6) k?n 1 .
Theorem 11. Every k-wise independent 0-1 probability space H for p1; p2; :::; pn as above has size at ? ? 7 k n n 1 least 6 k?1 45k2 = k?1 : If n 2k, then this is at least c m(n; k)=(k3 27k ) for a universal constant c > 0. Proof. Suppose f1; 2; :::; k ? 1g is valid. Let U1 ; U2; :::; Um be the proper supersets of f1; 2; :::; k ? 1g such that P[Ui] > 0. Let xi = P[Ui] and Si = Ui ? f1; 2; :::;k ? 1g. Because H is a k-wise independent 0-1 probability space for p1 ; p2; :::; pn, the chance of having ones simultaneously in P positions 1; 2; :::;k ? 1 P and j (for j k) is at most M 0 . But this probability is exactly i:Si 3j xi , so i:Si 3j xi M 0. We
have
1. Si fk; k + 1; :::; ng.
P
2. For j = k; k + 1; :::; n, i:Si 3j xi M 0, and P 3. mi=1 xi > M=3 (because f1; 2; :::;k ? 1g is valid). Apply Lemma 9 with N = n ? (k ? 1), = M 0 , = M=3. By the lemma, for all l 2, there are at least 1 1 lM ? (n ? k + 1)M 0 0 3 3 M=M ? (n ? k + 1)=l = l(l ? 1)M 0 l?1
pairwise disjoint Si 's, each of size at most l ? 1. Now set l = 6k + 1. Because M=M 0 0:9n=k, we have at least 1 1 M ? n n 6k 3 M 0 6k 45k2 pairwise disjoint Si 's, each of size at most 6k ? 1. Thus there are at least 45nk2 strings having positive probability in H with ones in positions 1; 2; :::;k ? 1 and at most (k ? 1)+(l ? 1) = 7k ? 1 ones in total. ? Now this argument obviously goes through ?for each of the 16 k?n 1 or more valid sets. This means 1 n n that, counting multiplicities, there? are at least 6 k?1 45k2 strings with at most 7k ? 1 ones each. Since ? each such string contains at most 7kk??11 < k7?k1 subsets of size k ? 1, each such string is generated fewer ? 7 k than k?1? times by?di erent valid sets. Thus the number of di erent strings of positive probability 7 k n n 1 exceeds 6 k?1 45k2 = k?1 : ) Proving that this last quantity is at least c mk3(n;k 27k for a universal ? ? constant ? c > 0 ?if k n=2 is not n dicult, if one uses facts such as (k + 1) k m(n; k) if k n=2, k?n 1 n nk , and k7?k1 27k . 6
Now the reader can verify that the two conditions above on M and M 0 are satis ed if M 0 = (k=n)k ? a k ? 1 and M = (k=n) , using inequalities such as b (a=b)b if a b 1 and [k=(k ? 1)]k?1 2 for all k 2. Theorem 3 follows.
4 Lower Bounds for Large Independence In this section we show how to derive a lower bound for the case in which the degree of independence is very large. The main tool in proving our result is the following theorem, in which Cd denotes the set of all d-dimensional subcubes of the n-dimensional cube C = f1; ?1gn. Theorem 12. Let n 1. Let g be a 2n-dimensional probability vector viewed as a function on the vertices of the n-dimensional cube C . P Let 1 d n. Let N = jfv 2 C : g(v) > 0gj. Suppose that there is an such that for all C 2 Cd , = v2C g(v): Then N 2n?1(n + 2 ? 2d)=(n + 1 ? d): The proof of the above theorem uses the Fourier analysis techniques of [Fri92]. Before giving the proof, we derive an interesting consequence.
Corollary 13. Let H = ( ; 2 ; P) be a k-wise independent 0-1 probability space where all pi = 1=2. ?n n n Then j j 2n( 2k2+2 k+2 ) = 2 (1 ? 2k+2 ): Proof. For all (n ? k)-dimensional subcubes C, Pv2H g(v) = 2?k : So let d = n ? k, = 2?k . Then N 2n?1(n + 2 ? 2(n ? k))=(n + 1 ? (n ? k)) = 2n (2k ? n + 2)=(2 + 2k): Corollary 14. Let 0 < 1 and b nc k n, and suppose that H = ( ; 2 ; P) is a k-wise independent 0-1 probability space where all pi = 1=2. Then j j (1 ? 1=(2 )) 2n : Proof. Because k b nc, 1? 2(kn+1) 1?1=(2 ). By Corollary 13, we have j j 2n(1?n=(2k + 2)) 2n(1 ? 1=(2 )). Corollary 14 implies Theorem 5. Now we do the proof of Theorem 12. We start with the following simple technical lemma, which follows from the convexity of the function f(x) = x2.
Lemma 15. Let g be a probability vector. Suppose that N = jfi : g(i) > 0gj. Then jgj2 = Pv g2(v) 1=N.
We use A to denote the 2n 2n adjacency matrix of C , i.e., the (u; v) entry A(u; v) of A is 1 if u and v di er in exactly one position, and 0 otherwise. For Q each T f1; 2; :::;ng, let T be a vector, viewed as a function onPC , whose vth coordinate T (v) = i2T vi , where v 2 f+1; ?1gn. If f; h : C ! IR, let < f; h >= 2?n v2C f(v)h(v):
Lemma 16. Let 0 r n. Then for each T f1; 2; :::;ng of size r, T is an eigenvector of A, corresponding to eigenvalue n ? 2r. Proof. Let T f1; 2; :::;ng with jT j = r. Choose any u 2 C. The uth entry of A T is
X v2C
A(u; v) T (v) =
7
X
v2N (u)
T (v);
where N(u) denotes those v 2 C di ering from u in exactly one position. Of the n v's in N(u), n ? r v's di er from u in a bit position not in T. These v's satisfy T (v) = T (u). The remaining r v's di er from u in one position in T and satisfy T (v) = ? T (u). It follows that
P
= (n ? r) T (u) ? r T (u) = (n ? 2r) T (u): Since u was arbitrary, we have A T = (n ? 2r) T : v2N (u) T (v)
It is easy to see that if T 6= U, then < T ; U >= 0, and that < T ; T >= 1 for all T. This means that the T 's are orthogonal and hence f T : T f1; 2; :::;ngg isPlinearly independent. Thus any 2n -dimensional vector g can be written as a linear combination g = T T T for some reals T . Proof of Theorem 12. As the theorem is vacuous if d = n, we assume without loss of generality that d < n. Fixing g, letPus set g^(U) =< g; U > for any U. It is easy to see, from the orthogonality of the T 's, that g(v) = T g^(T) T (v). P We are given that v2C g(v) = for all C 2 Cd . Choose any U f1; 2; :::;ng such that 1 jU j n ? d. g^(U) = < g;P U > ? n = 2 v g(v) U (v): Let us use \vjU = s" to mean that the restriction of v to the bit positions in U is s. 2ng^(U) = = =
P g(v) (v) U P Pv Ps2f1;?1gjjUUjj(QvjU:vjjUs=)(s g(v) P U (v)g(v)): i=1 i
s2f1;?1g
v:vjU =s
Now n ? jU j d and fv : vjU = sg is a cube of dimension n ? jU j. This (n ? jU j)-dimensional cube can be partitioned into 2n?jU j=2d d-dimensional subcubes. As the sum of g(v) over any d-dimensional subcube is , n?jU j X g(v) = 2 2d : v:vjU =s
Thus 2n g^(U) = Now
1 0 jU j 1 0 jU j n?jU j X Y X X @Y si : si A @ g(v)A = 2 2d
s2f1;?1gjU j i=1
v:vjU =s
jU j X Y s2f1;?1gjU j i=1
s2f1;?1gjU j i=1
si = [1 + (?1)]jU j = 0;
since jU j 1. Therefore g^(U) = 0 for any U such that 1 jU j n ? d. P Let us de ne h(v) = (Ag)v = u2N (v) g(u). Now h(v) = (Ag)v = A
"X T
g^(T) T
#!
v
=
X T
g^(T)(A T )v =
X T
(n ? 2jT j)^g(T) T (v);
where the third equality follows from the linearity of A and the last equality follows from Lemma 16. Above we showed that for any T with 1 jT j n ? d, g^(T) = 0. However, g^(;) = 1=2n, since g is a probability vector. 8
We would like to derive an upper bound on jgj2. To do this we consider < h; g >. Clearly, < h; g > 0. On the other hand, < h; g >= 2?n
X v
g(v)h(v) = 2?n
= 2?n
XX
T U n The last summation equals 2 < T ; U
XXX v U T
[(n ? 2jT j)^g(T) T (v)] [^g (U) U (v)]
g^(U)(n ? 2jT j)^g(T)
X v
T (v) U (v):
>, which is 0 if U 6= T and 2n if U = T. Therefore X X (n ? 2jT j)^g2(T): < h; g >= (n ? 2jT j)^g2(T) = 4nn + T T :jT jn?d+1
We can bound < h; g > as follows.
X 2 0 < h; g > 4nn + (2d ? 2 ? n) g^ (T): T :jT jn?d+1 P
By Parseval's identity, jjgjj2 = T g^2 (T), where jjgjj2 =< g; g >. Hence X 2 g^ (T) : < g; g >= 2?njgj2 = 41n + T :jT jn?d+1 Using this identity we have 2 0 4nn + (2d ? 2 ? n) j2gnj ? 41n = 2n +42n? 2d + 2d ?22n ? n jgj2: Therefore 1 2n n + 2 ? 2d : jgj2 2 n + 1 ? d By Lemma 15, jgj2 1=N. It follows that n n + 2 ? 2d 1 2 N jgj2 2 n + 1 ? d :
5 General Construction In this section we give a general construction for k-wise independent 0-1 random variables. The construction receives as input the desired p1 ; p2; :::; pn. We rst need a simple lemma, which is an easy consequence of the Moebius inversion formula. Lemma 17. Let H = ( ; 2 ; P) be a 0-1 probability space. De ne Yi to be the ith bit of an n-vector chosen according to H. Let 1 k n. Suppose that thereQ are reals p1; p2; :::; pn such that for all subsets S of f1; 2; :::;ng of size at most k, P[\i2S [Yi = 1]] = i2S pi : Then H is a k-wise independent 0-1 probability space for p1; p2; :::; pn. We omit the simple proof of the following corollary. Corollary 18. Suppose H is a k-wise independent 0-1 probability space for p1; p2; :::; pn. Suppose that p0i = 1 ? pi if i 2 S, p0i = pi otherwise. A k-wise independent 0-1 probability space for p01; p02; :::; p0n can be obtained from H by interchanging zeroes and ones in every position i 2 S in each point of the probability space. 9
Our general construction for p1; p2; :::; pn consists of three parts: (1) a construction for probabilities ti very close to 1, (2) a construction for s1 ; s2 ; :::; sn such that each si either equals pi or is slightly larger, and (3) a method of combining the two probability spaces so as to get p1; :::; pn exactly. We must give a construction for very large probabilities. Since it is easier to understand what's going on when the probabilities are very small, we will assume ti 1=(2n) for all i, and then invoke Corollary 18 to get the space for ti 1 ? 1=(2n).
Lemma 19. Let = 1=(2n). Suppose that 0 t1; t2; :::; tn . There is a k-wise independent 0-1 probability space H for t1 ; t2; :::; tn whose points are the elements of f0; 1gn with at most k ones. Its size is at most m(n; k). Proof. We assign a probability to each 0-1 vector as follows. Identify B f1; 2; :::; ng with its characteristic n-vector. We use P[B] to denote the probability that H assigns to the characteristic vector of B. Set P[B] = 0 if jB j > k. We will create probabilities P[B] for each B f1; 2; :::; ng, jB j k, such that for all A f1; 2; :::;ng such that jAj k, PB:BA P[B] = qA = Qi2A ti : In H, wherePYi denotes the random variable which is the ith bit of the random n-vector, we have P[\i2A[Yi = 1]] = B:BA P[B] = qA: By Lemma 17, this is enough. We assign probabilities P[B] by downward induction on jB j. As each P[B] is de ned, we prove inductively that 0 P[B] qB . If jB j = k, set P[B] = qB . (Notice that 0 P[B] qB .) Now suppose that jB j = l, l < k, and P[C] has been de ned for all C of size exceeding l. Set P[B] = qB ?
X
C :C B
P[C] = qB ?
6?
X
C :C B;jC jk
P[C]:
6?
P It is obvious with this de nition that qB = C :C B P[C]: Now if 0 P[C] qC for all C of size exceeding l, then clearly P[B] qB . Also,
P
P[B] qB ? C :C B;jC jk qC P 6??l ?n?li qB ? qBP ki=1 i i qB [1 ? P1 i=1 (n) ] 1 1 i = qB [1 ? i=1 ( 2 ) ] = 0: That m(n; k) is the number of 0-1 vectors with at most k ones is obvious. Now we give a construction and lemma due to Jo e [Jo e74]. Let r n be a prime. Suppose that s1 ; s2 ; :::; sn 2 [0; 1] satisfy si = ji =r for all i, for some integers ji . De ne a probability space, with (at most) nk points, as follows. Choose a0; a1; :::; ak?1 uniformly and independently at random from ZZr , and let Xi = a0 + a1i + a2i2 + + ak?1ik?1 mod r, for 1 i n. For each point < a0 ; a1; :::; ak?1 > in the sample space, let Yi = 1 if Xi 2 f0; 1; :::; ji ? 1g and let Yi = 0 otherwise, 1 i n. Associate with < a0 ; a1; :::; ak?1 > the vector < Y1 ; Y2; :::; Yn > and give each of the rk r-vectors probability 1=rk (and then combine identical r-vectors, if necessary).
Lemma 20. The random variables X1 ; X2; :::; Xn are k-wise independent and each Xi is uniform on ZZr . It follows that the 0-1 probability space de ned by the vectors < Y1 ; Y2; :::; Yn > is a k-wise independent 0-1 probability space for s1 ; s2 ; :::; sn. 10
Lemma 20 follows easily from the fact that Van der Monde matrices are invertible over ZZr . We now present an almost trivial technique that takes a k-wise independent 0-1 probability space A for s1 ; s2; :::; sn, and a k-wise independent 0-1 probability space A0 for t1 ; t2; :::; tn, and produces a k-wise independent 0-1 probability space H for s1 t1 ; s2t2 ; :::; sntn . To generate an element of H, simply independently pick x from A and x0 from A0 , and output the bit-wise AND x ^ x0 2 f0; 1gn. Clearly the size of the implied space H is at most jAjjA0j.
Lemma 21. The above construction of H from A and A0 guarantees that H is a k-wise independent 0-1 probability space for s1 t1 ; s2t2 ; :::; sntn .
Proof. From Lemma Q 17, it is sucient to show that for any subset S f1; 2; :::; ng of size at most k, P[\i2S [Yi = 1]] = i2S (si ti ), where Yi is the ith bit of the random n-vector x ^ x0 . But the ith bit of x ^ x0 is 1 if and onlyQif the ithQbit of x and the ith bit of x0 are both one. By independence, this occurs with probability ( i2S si )( i2S ti ): The probability space for the given probabilities p1 ; p2; :::; pn is obtained routinely as follows. 1. Assume without loss of generality that every pi 1=2. 2. Build a probability space in which the expectation si of the ith variable satis es si 2 [pi; pi + 1=(4n)]. This is done via Lemma 20 with r being a prime in [4n; 8n]. 3. Build a space for t1; t2 ; :::; tn, where ti = pi=si and hence ti 2 [1 ? 1=(2n); 1]. We do this via Lemma 19. 4. Use Lemma 21 to form a space for p1 ; :::; pn where pi = ti si . We conclude with Theorem 22. The new space is a k-wise independent 0-1 probability space for the given probabilities p1 ; p2; :::; pn. Its size is at most (8n)k m(n; k) (8n)2k .
6 Construction for Pairwise Independence In this section we exhibit a space of size at most 4n2 for pairwise independent random variables in the case that all the probabilities are equal. From the lower bound of Theorem 11 it follows that this construction is optimal in size, to within a constant factor.
De nition. Let H be a 0-1 probability space. H is an (a; b; n)-space if the probability of a 1 in position i is a for all i, and the probability of simultaneous ones in positions i and j is b, for all i < j. For a pairwise independent 0-1 probability space, we would want b = a2 . However, here we will need b < a2 . The following lemma shows that given such an (a; b; n)-space, we can build a pairwise independent 0-1 probability space if all pi = p, and if p is in a certain interval:
11
Lemma 23. Let H = ( ; 2 ; P) be an (a; b; n)-space with b < a2. Then there is a construction of a pairwise independent 0-1 probability space H if all pi = p, on a sample space of size at most j j + 2, provided that p 2 [b=a; (a ? b)=(1 ? a)]. Proof. The new probability space will be de ned in the following way. Let x; y 0 such that x + y 1. With probability x, choose a point in (according to the probabilities in P). With probability y, choose the all-ones row. With probability 1 ? x ? y, choose the all-zeroes row. It is clear that the implied probability space has size at most j j + 2. In order that the new space be pairwise independent for all pi = p, it suces to ensure that the probability of a 1 in position i is p and the probability of simultaneous ones in positions i and j is p2 (and that the sum of the probabilities of the points is 1). In other words, it is sucient to satisfy the following system: xa+y 1 = p x b + y 1 = p2 and satisfy x + y 1, x; y 0. The unique solution to the system is x = (p ? p2 )=(a ? b); y = p(ap ? b)=(a ? b): We have x + y 1, x; y 0 if and only if p 2 [b=a; (a ? b)=(1 ? a)]: Note that since b < a2 , b=a < (a ? b)=(1 ? a). Based on the above lemma we can exhibit a quadratic size space for an arbitrary probability p. Theorem 24. For any n > 1, for any 0 p 1, there is a construction of a pairwise independent 0-1 probability space of size at most 4n2 if every pi = p. Proof. First we claim that if q n is a prime, then for any l, 0 l q, there is an (l=q; (l2 ? l)=(q2 ? q); n)-space. The points of the sample space are the q2 ? q pairs (c; d), c; d 2 ZZq , d 6= 0. Let Xi (c; d) = c + di mod q and Yi (c; d) = 1 if Xi (c; d) 2 f0; 1; 2; :::;l ? 1g and Yi (c; d) = 0 otherwise, 1 i n. Build a space with at most q2 ? q vectors by associating with point (c; d) the n-vector (Y1 ; Y2; :::; Yn), and then combining identical n-vectors. That this is an (l=q; (l2 ? l)=(q2 ? q); n)-space can easily be veri ed by noting that the q omitted pairs (c; 0) de ne l rows that are all 1 and q ? l rows that are all 0, and if the q omitted rows were added, we'd have a pairwise independent space in which each random variable is 1 with probability l=q. Now for the construction of the pairwise independent probability space. Choose a prime q such that n q 2n. Choose an l, 1 l q ? 1, such that p 2 [(l ? 1)=(q ? 1); l=(q ? 1)]. The construction of this proof gives us an (l=q; (l2 ? l)=(q2 ? q); n)-space of size at most q2 ? q. Lemma 23 now gives us a pairwise independent sample space, of size at most (q2 ? q) + 2 q2 (2n)2 = 4n2, with pi = p for all i, provided that p 2 [b=a; (a ? b)=(1 ? a)]. A simple calculation with a = l=q, b = (l2 ? l)=(q2 ? q) shows that [b=a; (a ? b)=(1 ? a)] = [(l ? 1)=(q ? 1); l=(q ? 1)], which contains p. For certain ranges of p, better constructions are possible. Theorem 25 uses Lemma 23 to get a linear construction for p 1=(n ? 1), while Corollary 28 exhibits a linear construction for values in the neighborhood of 1=2. Theorem 25. For p 1=(n ? 1) there is a pairwise independent 0-1 probability space of size at most n + 2 with pi = p for all i. 12
Proof. We exhibit a (1=n; 0; n)-space; this is enough for all p 2 [b=a; (a ? b)=(1 ? a)] = [0; (1=n)=(1 ? 1=n)] = [0; 1=(n ? 1)]. This is simply the identity matrix in which each row gets probability 1=n. By Lemma 23 there is a construction of size at most n + 2. We can also get linear size constructions around 1=2. Lemma 26. Let A be a normalized mm Hadamard matrix (i.e., an mm (+1; ?1) matrix satisfying AT A = mI, whose rst row and rst column are all +1's). Let A0 be obtained from A by omitting the rst row and rst column and then replacing all ?1's by 0's. The probability space obtained by giving each of the m ? 1 rows of A0 probability 1=(m ? 1) is a ((1=2)(m ? 2)=(m ? 1); (1=4)(m ? 4)=(m ? 1); m ? 1)space. The proof is simple and is omitted.
Theorem 27. Let n > 1 and let A be a normalized m m Hadamard matrix with m n + 1. There is a pairwise independent 0-1 probability space (whose points are elements of f0; 1gn) of size at most m + 1 with pi = p for all i if p 2 [1=2 ? 1=(m ? 2); 1=2 + 1=(m ? 2)]. Proof. With a = (1=2)(m ? 2)=(m ? 1), b = (1=4)(m ? 4)=(m ? 1) (from Lemma 26), we have b=a = 1=2 ? 1=(m ? 2) and (a ? b)=(1 ? a) = 1=2. This gives a construction for p 2 [1=2 ? 1=(m ? 2); 1=2]. Its size is at most (m ? 1) + 2 = m + 1. For p 2 [1=2; 1=2+ 1=(m ? 2)], we rst build a space for 1 ? p and then exchange zeroes and ones.
Corollary 28. For any n > 1, for any p 2 [1=2 ? 1=(2n ? 2); 1=2 + 1=(2n ? 2)], there is a pairwise
independent 0-1 probability space of size at most 2n + 1 with all pi = p.
Proof. For any t, there is a 2t 2t Hadamard matrix. Given n, take m to be the least power of two exceeding n; m 2n. By Theorem 27, there is a construction of size at most m + 1 2n + 1 for any p 2 [1=2 ? 1=(m ? 2); 1=2 + 1=(m ? 2)] [1=2 ? 1=(2n ? 2); 1=2 + 1=(2n ? 2)].
7 Acknowledgments For their help in the early stages of our work, we are very grateful to D. Koller and N. Megiddo, whose paper [KM93], in opening up the area of nonuniform spaces in theoretical computer science, inspired this one. We thank Shi-Chun Tsai for pointing out a typo.
References [ABI86]
N. Alon, L. Babai, and A. Itai, \A Fast and Simple Randomized Parallel Algorithm for the Maximal Independent Set Problem," Journal of Algorithms 7 (1986), pages 567{583.
[AS92]
N. Alon and J. Spencer, The Probabilistic Method, Wiley, 1992.
[BR89]
B. Berger and J. Rompel, \Simulating (logc n)-wise Independence in NC," Proc. 30th IEEE Symposium on Foundations of Computer Science, 1989, pages 2{7.
13
[CGHFRS85] B. Chor, O. Goldreich, J. Hastad, J. Friedman, S. Rudich, and R. Smolensky, \The Bit Extraction Problem or t-Resilient Functions," Proc. 26th IEEE Symposium on Foundations of Computer Science, 1985, pages 396{407. [Fri92]
J. Friedman, \On the Bit Extraction Problem," Proc. 33rd IEEE Symposium on Foundations of Computer Science, 1992, pages 314{319.
[Jo e74]
A. Jo e, \On a Set of Almost Deterministic k-Independent Random Variables," Annals of Probability 2 (1974), pages 161{162.
[KM93]
D. Koller and N. Megiddo, Constructing Small Sample Spaces Satisfying Given Constraints," Proc. 25th ACM Symposium on Theory of Computing, 1993, pages 268{277.
[Lub86]
M. Luby, \A Simple Parallel Algorithm for the Maximal Independent Set Problem," SIAM Journal on Computing 15 (1986), pages 1036{1053.
[Lub88]
M. Luby, \Removing Randomness in Parallel Computation Without a Processor Penalty," Proc. 29th IEEE Symposium on Foundations of Computer Science, 1988, pages 162-173.
[MNN89]
R. Motwani, J. Naor, and J. Naor, \The Probabilistic Method Yields Deterministic Parallel Algorithms," Proc. 30th IEEE Symposium on Foundations of Computer Science, 1989, pages 8{13.
[Sch92]
L. Schulman, \Sample Spaces Uniform on Neighborhoods," Proc. 24th ACM Symposium on Theory of Computing, 1992, pages 17{25.
14
