On the Thinnest Coverings of Spheres and Ellipsoids with Balls in on ...
On the Thinnest Coverings of Spheres and Ellipsoids with Balls in on the Thinnest Coverings of Spheres and Ellipsoids I. Dumer , M.S. Pinsker, and V.V. Prelov
Abstract. In this paper, we present some new results on the thinnest coverings that can be obtained in Hamming or Euclidean spaces if spheres and ellipsoids are covered with balls of some radius ε. In particular, we tighten the bounds currently known for the ε-entropy of Hamming spheres of an arbitrary radius r. New bounds for the ε-entropy of Hamming balls are also derived. If both parameters ε and r are linear in dimension n, then the upper bounds exceed the lower ones by an additive term of order log n. We also present the uniform bounds valid for all values of ε and r. In the second part of the paper, new sufficient conditions are obtained, which allow one to verify the validity of the asymptotic formula for the size of an ellipsoid in a Hamming space. Finally, we survey recent results concerning coverings of ellipsoids in Hamming and Euclidean spaces.
1
Introduction
Let En be the Hamming space of binary vectors x = (x1 , . . . , xn ) of length n. Given an integer ε, let B n (y, ε) be the ball of radius ε centered at the point y ∈ En , i.e., B n (y, ε) := {x ∈ En | d(x, y) ≤ ε} , where d(x, y) is the Hamming distance between x and y, that is the number of coordinate positions in which xi = yi . We say that a subset Mε (A) ⊆ En forms an ε-covering of a set A ⊆ En if A belongs to the union of the balls B n (y, ε) centered at points y ∈ Mε (A), i.e., A⊆ B n (y, ε). y∈Mε (A)
The ε-entropy [1] Hε (A) of a set A is the logarithm of the size of its minimal ε-covering: Hε (A) := log min |Mε (A)|,
Supported in part by NSF grant CCR-0097125. Supported in part by the Russian Foundation for Basic Research (project no. 03– 01–00592) and INTAS (project 00-738). Supported in part by ZIF (project “General Theory of Information Transfer and Combinatorics”), the Russian Foundation for Basic Research (project no. 03–01– 00592) and INTAS (project 00-738).
R. Ahlswede et al. (Eds.): Information Transfer and Combinatorics, LNCS 4123, pp. 883–910, 2006. c Springer-Verlag Berlin Heidelberg 2006
884
I. Dumer, M.S. Pinsker, and V.V. Prelov
where the minimum is taken over all ε-coverings Mε (A) and log denotes logarithm base 2. Given a vector v = (v1 , . . . , vn ), vi ∈ [0, ∞), i = 1, . . . , n, the ellipsoid Evn is defined by the equality n n n Ev := x ∈ E | vi xi ≤ 1 , (1) i=1
where Note that the inequality n all operations are performed over real numbers. n 2 i=1 vi xi ≤ 1 is equivalent to the inequality i=1 vi xi ≤ 1. This fact explains the notion of ellipsoid for Evn by analogy with that in the Euclidean space. Below we study the asymptotic behavior (as n → ∞) of the ε-entropy of an arbitrary ellipsoid Evn and, in particular, the ε-entropy of a ball B n (r) := B n (0, r) and a sphere S n (r) := {x ∈ En | d(x, 0) = r} , where r is an integer. The following notation will be used throughout the paper. Consider the binary entropy function h(t) := −t log t − (1 − t) log(1 − t),
0 ≤ t ≤ 1.
Then, given any integer n and real-valued vectors P = P (n) = (p1 , . . . , pn ) and Q = Q(n) = (q1 , . . . , qn ) such that 0 ≤ pi ≤ 1, 0 ≤ qi ≤ 1, i = 1, . . . , n, define the function h(P, Q) :=
n
h(pi , qi ) = h(P ) − h(Q),
i=1
where h(pi , qi ) := h(pi ) − h(qi ),
h(P ) :=
n i=1
h(pi ),
h(Q) :=
n
h(qi ).
i=1
Here and below, we sometimes omit an argument n or a superscript n in our notation of vectors, ellipsoids, balls, and spheres. In this paper, we also consider an arbitrary ellipsoid Ea = Ean defined in the n-dimensional Euclidean space Rn as n x2i n n Ea := x = (x1 , . . . , xn ) ∈ R | ≤ 1, (2) a2 i=1 i where a = (a1 , . . . , an ) is a real-valued vector with n positive coordinates. We will use the same notation B n (y, ε) and S n (r) for balls and spheres in the Euclidean space Rn as the one used in the Hamming space Rn . The definitions of B n (y, ε), S n (r) and the ε-entropy Hε (A) of a bounded set A ⊂ Rn in the
On the Thinnest Coverings of Spheres and Ellipsoids
885
Euclidean space are absolutely similar to those in the Hamming space. The only difference arises in the definition of the function d(x, y): now d(x, y) is the Euclidean distance between real-valued vectors x and y. Also, in the Euclidean space, the radii r, ε, and many other parameters can take arbitrary positive values instead of integers used in the Hamming space. In both spaces we are interested in the asymptotic ε-entropy of any ellipsoid Evn or Ean Note that ellipsoids in the Hamming spaces arise in various problems related to combinatorics, decoding, and data compression. For example, ellipsoids in the form of (1) can be considered as Boolean threshold functions. Also, ellipsoids emerge in maximum likelihood decoding for binary memoryless channels. On the other hand, the problem of coverings of spheres and ellipsoids in Euclidean spaces often arises in vector quantizers. In the next section, we better the bounds recently obtained for the ε-entropy of spheres in the Hamming space. We also extend these results for balls and derive new uniform bounds. In Section 3, we shortly survey a few results already known for the size and the ε-entropy of ellipsoids in the Hamming space. There we also derive some new sufficient conditions for the validity of the asymptotic formula for the size of an ellipsoid. Section 4 is devoted to recent results obtained for the ε-entropy of ellipsoids in Euclidean spaces.
2
Covering of Spheres and Balls in Hamming Spaces
Below we assume in this section that n, r, and ε are some positive integers such that the ratios ρ := r/n, σ := ε/n satisfy condition 0 < σ < ρ ≤ 1/2. Firstly, note that the whole space En can be considered as a special case of a ball B n (r) when r = n. It is shown (see, e.g., [2, 3]) that the normalized ε-entropy H ε (En ) := Hε (En )/n satisfies equality log n n H ε (E ) = 1 − h(σ) + O , n → ∞, n for any fixed σ. Non-uniform lower and upper bounds on the ε-entropy of spheres in the Hamming space are obtained in [4]. It is proven there that for any fixed ρ < 1/2 and σ there exist some constants c(ρ, σ) and C(ρ, σ), which are independent of n but can depend on ρ and σ such that the normalized ε-entropy H ε (S(r)) satisfies the following inequalities: h(ρ) − h(σ) +
c(ρ, σ) 3 log n C(ρ, σ) ≤ H ε (S(r)) ≤ h(ρ) − h(σ) + + . n 2n n
The following theorem improves this bound and also gives the uniform (with respect to parameters ρ and σ) lower and upper bounds on the ε-entropy of
886
I. Dumer, M.S. Pinsker, and V.V. Prelov
spheres in the Hamming space. Such bounds are of independent interest, and can be used, for example, in deriving the upper bounds for the ε-entropy of ellipsoids. Theorem 1 1. For all n, r ≤ n/2, and ε < r, the normalized ε-entropy of an n-dimensional sphere S(r) satisfies the following lower and upper bounds: h(ρ) − h(σ) −
c 3 log n log(2n) ≤ H ε (S(r)) ≤ h(ρ) − h(σ) + + , 2n 2n n
(3)
where c is an absolute constant. Moreover, the lower bound in (3) can be improved if σ or both σ and ρ are constants independent of n. Namely, 2. If σ is a constant, then there exists a constant c1 (σ) such that H ε (S(r)) ≥ h(ρ) − h(σ) +
c1 (σ) . n
(4)
3. If both ρ and σ are constants, then there exists a constant c2 (ρ, σ) such that H ε (S(r)) ≥ h(ρ) − h(σ) +
log n c2 (ρ, σ) + . 2n n
(5)
Proof. This theorem will be proven using Proposition 1 given in the sequel. However, the lower bounds in (3) and (4) are almost trivial. Indeed, we first use the following well-known inequalities (see, e.g., (10.16) and (10.20) in [5]): (8nρ(1 − ρ))−1/2 2nh(ρ) ≤ |S(r)| ≤ (2πnρ(1 − ρ))−1/2 2nh(ρ)
(6)
|B(ε)| ≤ 2nh(σ) .
(7)
and Applying (6) and (7) we obtain the lower (packing) bound |S(r)| 1 log ≥ h(ρ) − h(σ)− n |B(ε)| log(2n) log 8nρ(1 − ρ) ≥ h(ρ) − h(σ) − , 2n 2n H ε (S(r)) ≥
which gives the left inequality in (3). Similarly, note that for σ < 1/2, we have |B(ε)| ≤ |S(ε)| i=0 ε
σ 1−σ
i ≤
1−σ , 1 − 2σ
which follows from the inequality |S(ε − 1)|/|S(ε)| ≤ σ/(1 − σ).
(8)
On the Thinnest Coverings of Spheres and Ellipsoids
887
Therefore, using (6) and (8), we obtain inequality (4): |S(r)| 1 |S(r)| 1 |B(ε)| 1 log = log − log n |B(ε)| n |S(ε)| n |S(ε)| log(2n) log(2πnσ(1 − σ)) 1 1−σ ≥ h(ρ) − h(σ) − + − log . 2n 2n n 1 − 2σ
H ε (S(r)) ≥
To proceed with the other bounds of Theorem 1, we shall extensively use the following notation. Given any integer w ∈ [r − ε, r + ε], we consider a point x ∈ S(w) and the subset Ar (x, ε) := B(x, ε) ∩ S(r). Given any y ∈ S(r), we also consider the set Aw (y, ε) := B(y, ε) ∩ S(w). Now we can prove that H ε (S(r)) can be tightly related to the function Ω(r, ε) := max |Ar (x, ε)|, w
x ∈ S(w).
(9)
Namely, we shall prove that the ε-entropy of the sphere S(r) satisfies the bounds 1 |S(r)| |S(r)| log(n ln 2) log 2h(ρ) 1 log ≤ H ε (S(r)) ≤ log + + . n Ω(r, ε) n Ω(r, ε) n n
(10)
1. The lower bound in (10) is based on a standard packing argument. Here we use the fact that each ball B(x, ε) can cover at most Ω(r, ε) points on S(r). 2. The upper bound is obtained using random coverings. Given a fixed integer N, we choose the radius w, for which equality (9) holds. Then we perform N trials choosing the centers x of an ε-covering independently and uniformly on S(w). Then any point y ∈ S(r) is covered in one trial with the same probability γ=
|Aw (y, ε)| . |S(w)|
(11)
Now we employ a straightforward argument (used by Bassalygo in 1965; see also Lemma 6 in [4]) that states that for any x ∈ S(w) and any y ∈ S(r), the two sets Ar (x, ε) and Aw (y, ε) cover the same fraction of spheres S(r) and S(w), respectively. Therefore, |Ar (x, ε)| . (12) γ= |S(r)| Now note that any y ∈ S(r) is not covered in N trials with probability PN = (1 − γ)N . Choosing 1 + ln |S(r)| , (13) N= γ
888
I. Dumer, M.S. Pinsker, and V.V. Prelov
we obtain PN ≤ (e|S(r)|)−1 . Therefore the whole sphere S(r) is covered in N trials with a probability P N (S(r)) ≥ 1 − PN · |S(r)| ≥ 1 − e−1 . In this case, there exists an ε-covering of size N. Using (12), we obtain the upper bound H ε (S(r)) ≤
1 |S(r)| 1 log N = log + log(1 + ln |S(r)|). n n Ω(r, ε) n
(14)
Now the upper bound in (10) follows from (6). Thus, we see that to prove Theorem 1 we need to derive rather tight lower and upper bounds on the quantity Ω(r, ε). These bounds - derived in the sequel in Proposition 1 - conclude the proof of Theorem 1. In fact, slightly more precise calculations show that the universal constant c belongs to the interval (0, 2). Proposition 1 1. For all n, r ≤ n/2, and ε < r, the following lower and upper bounds hold: nh(σ) − log 8n ≤ log Ω(r, ε) ≤ nh(σ).
(15)
Moreover, 2. If σ is a constant, then there exists a constant c(σ) such that log Ω(r, ε) ≤ nh(σ) − (log n)/2 + c(σ). 3. If ρ and σ are constants, then there exists a constant c(ρ, σ) such that log Ω(r, ε) ≤ nh(σ) − log n + c(ρ, σ). Proposition 1 will be proven in the Appendix. In the following theorem, we present the corresponding non-uniform and uniform lower and upper bounds for the ε-entropy of balls in the Hamming space. Theorem 2 1. For all n, r ≤ n/2, and ε < r, the normalized ε-entropy of an n-dimensional ball B(r) satisfies the following lower and upper bounds: h(ρ) − h(σ) −
2 log n C log(2n) ≤ H ε (B(r)) ≤ h(ρ) − h(σ) + + , 2n n n
(16)
where C is an absolute constant. Moreover, 2. If σ is a constant, then there exists a constant C1 (σ) such that H ε (B(r)) ≥ h(ρ) − h(σ) +
C1 (σ) . n
(17)
3. If both ρ < 1/2 and σ are constants, then there exists a constant C2 (ρ, σ) such that h(ρ) − h(σ) +
3 log n C log n C2 (ρ, σ) + ≤ H ε (B(r)) ≤ h(ρ) − h(σ) + + . (18) 2n n 2n n
On the Thinnest Coverings of Spheres and Ellipsoids
889
Proof. Note first that the lower bounds in (16) - (18) immediately follow from the corresponding lower bounds in (3) - (5) since S(r) ⊂ B(r) for all r.1 Before starting to prove the upper bounds in (16)–(18), note that for all r and ε we have (19) Hε (B(r)) ≤ Hε (S(r)) + log n r since B(r) = k=0 S(k) and Hε (S(k)) ≤ Hε (S(r)) for all k = 0, . . . , r. Therefore, to derive some upper bounds for Hε (B(r)) we can use (19) together with the upper bounds in (3) and (4). By doing this, we obtain the upper bounds which are worse than those in (16)–(18). The latter means that the trivial inequality (19) does not allow us to prove the upper bounds in (16)–(18). In order to prove these upper bounds, let us use the following modification of the random choice method described in the proof of Theorem 1. Namely, let an integer N be fixed. Then we choose N points x(1) , . . . , x(N ) of an εcovering for the ball B(r) randomly according to the following rule. In the ith trial, i = 1, . . . , N , independently of all other trials, we choose an integer u, ε + 1 ≤ u ≤ r, at random with probability p(u) =
|S(u)| , |B(r)| − |B(ε)|
ε + 1 ≤ u ≤ r.
Given u, the i-th point x(i) of the ε-covering is chosen randomly on the sphere S(u ), where
u−ε u := 1 − 2σ Similarly to the proof of Theorem 1 (see also formula (89) in the Appendix), here we also use the fact that our choice of u maximizes the covering area Au (x, ε) generated by the ball B(x, ε) with any center x ∈ S(u ), so that |Au (x, ε)| = Ω(u, ε). Now we estimate the probability γy that any fixed point y ∈ S(u) is covered by a ball B(x(i) , ε). First, note that similarly to equalities (11) and (12), for any x ∈ S(u ) and y ∈ S(u) we have the equality u A (y, ε) |Au (x, ε)| = . |S(u )| |S(u)| Then we use inequality (15) for Ω(u, ε) and see that any point y ∈ S(u) is covered by a ball B(x(i) , ε) with probability γy ≥
|S(u)| |B(r)|−|B(ε)|
≥ γr := 1
·
Ω(u,ε) |S(u)|
2nh(σ)−log 8n |B(r)|−|B(ε)| .
(20)
Note that the size of the covering is always bounded from below by |B(r)|/|B(ε)|. This fact allows one to remove the term −(log n)/2n from the lower bound (16). We omit the corresponding calculations.
890
I. Dumer, M.S. Pinsker, and V.V. Prelov
It is important that for any fixed point y ∈ B(r), the probability γy of its covering in one trial is bounded from below in (20) by a quantity γr that does not depend on y. The rest of the proof of the theorem almost coincides with that of Theorem 1 and therefore we omit it here. The main difference arises from the fact that now we use a bigger set B(r) in (20) instead of S(r) employed in (12). Note also that now we take N=
1 (1 + ln |B(r)|), γr
instead of (13), but this change does not affect the asymptotics of H ε (B(r)). Finally, we apply the inequalities |B(r)| ≤
1−ρ |S(r)| 1 − 2ρ
(cf. (8)) in the proof of (18) and |B(r)| ≤ 2nh(ρ) in the proof of (16) and (17). This completes the proof of Theorem 2.
The main conclusion we derive from Theorems 1 and 2 is that for both spheres and balls with fixed relative radii ρ and ε, the corresponding upper and lower bounds differ only by an additive term of order log n. Note that this term is related to our random-covering algorithm used for upper bounds. Thus, any further tightening of the above bounds is only possible if there exist constructive coverings that surpass their randomly chosen counterparts.
3 3.1
Covering of Ellipsoids in Hamming Spaces The Size of Ellipsoids
Consider an ellipsoid Ev = Evn defined in (1). The first interesting question is the problem of finding the number of different ellipsoids in En . It was proven 2 long ago [6] that this number is upper bounded by 2n . The problem had been addressed in many publications for over 140 years until Zuev proved [7] that the 2 number of different ellipsoids is lower bounded by 2n (1−10/ ln n) thus solving the 2 problem up to the exponential order of 2n . Another important problem is to derive the size of any ellipsoid Ev . This quantity |Ev | is necessary, for example, to write out the packing (Hamming) bound for the ε-entropy of Ev . The main term of the asymptotics of log |Ev | was found by Pinsker [8]. To state his result, let us consider the vector P ∗ = P ∗ (n) = (p∗1 , . . . , p∗n ) with components
−1 p∗i = p∗i (n) := 1 + 2λvi ,
i = 1, . . . , n,
(21)
On the Thinnest Coverings of Spheres and Ellipsoids
where parameter λ = λ(n) is defined by the equalities
n n λvi −1 = 1 if 12 i=1 vi > 1, i=1 vi 1 + 2 n λ=0 if 12 i=1 vi ≤ 1. Let
891
(22)
Hn := h(P ∗ (n)).
(23)
Then the following statement holds. Theorem 3. [8].
If lim
n→∞
Hn = ∞, log n
(24)
then Hn (1 + o(1)) ≤ log |Ev | ≤ Hn ,
n → ∞,
(25)
and, in particular, log |Ev | = Hn (1 + o(1)),
n → ∞.
(26)
The proof of the upper bound in (25) given in [8]) (see Lemma 1 there) is rather simple and short and therefore for reader’s convenience we reproduce it here. Indeed, given the uniform distribution Pr(x) = 2−n , x ∈ En , we have n vi Xi ≤ 1 + n, (27) log |Ev | = log Pr(Ev ) + n = log Pr i=1
where Xi , i = 1, . . . , n, are independent binary random variables taking values n vi /2 > 1, then 0, 1 with probabilities Pr(0) = Pr(1) = 1/2. It is clear that if Pr
n
vi Xi ≤ 1
= Pr 2
−λ
n i=1
i=1
vi Xj
≥2
−λ
≤ 2 E2 λ
−λ
n
vi Xi
i=1
i=1
= 2λ
n
E 2−λvi Xi = 2λ
i=1
n 1 + 2−λvi i=1
2
.
Therefore, formulas (27) and (28) imply log |Ev | ≤ λ +
n
log 1 + 2−λvi
i=1
=
n
−1 λvi 1 + 2λvi + log 1 + 2−λvi
i=1
=
n i=1
h(p∗i ) = h(P ∗ ) = Hn .
(28)
892
I. Dumer, M.S. Pinsker, and V.V. Prelov
For the case
n i=1
vi /2 ≤ 1, we have λ = 0 and p∗i = 1/2, i = 1, . . . , n, and
therefore the right-hand side inequality in (25) is trivially fulfilled. The proof of the lower bound in (25) is more involved. The idea is to construct a sequence of sets An,u ⊆ Ev , n = 1, 2, . . . that are the direct products of balls of a special radius u in the corresponding subspaces and then to evaluate the asymptotic behavior of the size of An,u . For details of the proof we refer to [8]. Without loss of generality, below we will assume that the coefficients vi , i = 1, . . . , n, of an ellipsoid Ev form a non-increasing sequence, i.e., v1 ≥ v2 ≥ . . . ≥ vn .
(29)
This can always be attained by renumbering the elements of the sequence {vi }. Further, we will consider two different cases: (i) the whole sequence {vi }∞ i=1 is given, i.e., vi does not depend on n for any fixed i = 1, 2, . . .; (ii) ”scheme of series”: each vi = vi (n) can depend on n but for any given n the elements vi (n), i = 1, . . . , n, satisfy condition (29). The following statement, for the case (i), gives a simple necessary and sufficient condition on coefficients {vi } under which the main condition (24) of Theorem 3 is fulfilled. Proposition 2. Given a non-increasing sequence {vi }∞ i=1 , condition (24) holds if and only if lim vn = 0. (30) n→∞
Corollary 1. Given a non-increasing sequence {vi }∞ i=1 , Theorem 3 can be reformulated as follows: if lim vn = 0, then inequalities (25) and equality (26) n→∞ are valid. Proof. It can easily be seen that Hn = max h(P ),
(31)
P
where Hn is defined in (23) and the maximum in (31) is taken over all vectors P = (p1 , . . . , pn ) such that 0 ≤ p1 , . . . , pn ≤ 1/2,
n
vi pi ≤ 1.
i=1
We will use equality (31) below. 1. Assume first that lim vn = 0, i.e., there exists a positive constant v such n→∞ that vn ≥ v > 0 for all n. Then we have h(P ), Hn = max h(P ) ≤ max P
P
(32)
On the Thinnest Coverings of Spheres and Ellipsoids
893
where the second maximum in (32) is taken over all vectors P = (p1 , . . . , pn ) such that n 0 ≤ p1 , . . . , pn ≤ 1/2, vpi ≤ 1. i=1
It is clear that h(P ) = nh max P
1 nv
=
1 (log n)(1 + o(1)), v
n → ∞.
(33)
Relations (32) and (33) show that condition (24) is not satisfied if lim vn = 0. n→∞
2. Assume now that lim vn = 0. Let k = k(n) = αn where α, 0 < α < 1, is n→∞
a fixed constant.2 Then we clearly have n kh(1/(kvn−k+1 )) Hn ≥ max h( pi (k)) = k P (k) i=n−k+1
if if
kvn−k+1 ≥ 2, kvn−k+1 ≤ 2,
(34)
where the maximum in (34) is taken over all k-dimensional vectors P (k) = ( pn−k+1 (k), . . . , pn (k)) such that n
0 ≤ pn−k+1 (k), . . . , pn (k) ≤ 1/2,
vn−k+1 pi (k) ≤ 1.
i=n−k+1
Let a rather large positive constant A > 2 be fixed. Consider first a subset N1 of integers n for which kvn−k+1 ≤ A, n ∈ N1 . For such n ∈ N1 , inequality (34) shows that Hn ≥ kh(1/A) = αnh(1/A),
(35)
and (24) holds. On the other hand, for n ∈ N2 for which kvn−k+1 > A,
n ∈ N2 ,
it follows from (34) that Hn ≥
log(kvn−k+1 ) . vn−k+1
(36)
Therefore, if vn−k+1 ≤
1 , log k
n ∈ N2 ,
then we have Hn ≥ (log A · log k), 2
n ∈ N2 .
(37)
More precisely, we should write k = αn instead of k = αn but this difference does not affect further asymptotic relations.
894
I. Dumer, M.S. Pinsker, and V.V. Prelov
At the same time if
1 , n ∈ N2 , kc where c, 0 < c < 1, is a constant, then we obtain from (36) that vn−k+1 ≥
Hn ≥
log k − log(1/vn−k+1 ) (1 − c) log k ≥ , vn−k+1 vn−k+1
Hence, inequalities (37) and (38) imply that 1−c Hn ≥ min log A, (log k), vn−k+1
n ∈ N2 .
n ∈ N2 .
(38)
(39)
Finally, taking A → ∞ rather slowly, we conclude from (35) and (39) that lim
n→∞
Hn = ∞. log n
Proposition 2 is proved.
Consider now the general case - “scheme of series” - where each vi = vi (n) may depend on n but for any given n the sequence is non-increasing. For such a situation the following statement holds. Proposition 3. Assume that for any given n the sequence vi = vi (n), i = 1, . . . , n, does not increase. Then (a) If lim vn = 0, then condition (24) is not fulfilled. n→∞
(b) If there exists a sequence k = k(n) such that vn−k+1 → 0 as n → ∞, log k lim inf log n > 0, n→∞
then condition (24) does hold. Proof. The proof of part (a) does not differ from that of Proposition 2. To prove the second statement, note that in the proof of the direct part of Proposition 2 we have only used the facts that lim (log k/ log n) > 0 for subsequence k = k(n) and n→∞
that vn−k+1 → 0 as n → ∞ (which in turn follows from condition (30) used in Proposition 2). Now the two latter conditions are introduced in the formulation of Proposition 3. Therefore, taking into account this observation, we can claim that Proposition 3 holds. Remark 2. It easily follows from inequality (34) (which also holds for the scheme of series) that if there exists a sequence k = k(n) such that n i=n−k+1
vi ≤ 2 and
k → ∞ as n → ∞ log n
On the Thinnest Coverings of Spheres and Ellipsoids
or n
n
kh 1/
i=n−k+1
vi > 2 and
log n
i=n−k+1
895
vi → ∞ as n → ∞,
then condition (24) is fulfilled. In particular, this holds if n
vi ≤ 2
i=1
n kh 1/ vi
or n
vi > 2 and
i=1
i=1
log n
→ ∞ as n → ∞.
Remark 3. It should be mentioned that using some results known from the theory of limit theorems and, in particular, the theory of large deviations for the sums of independent but not-identically distributed random variables, the asymptotics of log |Ev | can be expressed in a different form. Moreover, in certain special cases, the next terms of the asymptotics of log |Ev | can also be found [9]. 3.2
The ε-Entropy of Ellipsoids
Below in this subsection, we will always assume, as was already mention earlier, that the coefficients vi = vi (n), i = 1, . . . , n, do not increase for any given n and, moreover, that ε = ε(n) is an integer such that 1 ≤ ε < n/2. Otherwise (i.e., if ε ≥ n/2) the equality Hε (Evn ) = O(log n),
n → ∞,
holds for arbitrary ellipsoids Evn since Evn ⊆ En and it is obvious that Hε (En ) = O(log n) (cf. also (2)). Lower bound Using relation (26) for the size of ellipsoid Ev and inequality (7) for the size of the ball B(ε), we can easily write out the packing (Hamming) bound for the ε-entropy Hε (Evn ): Hε (Evn ) ≥ log+
|Ev | ≥ [Hn − nh(ε/n)]+ (1 + o(1)), |B(ε)|
n → ∞,
(40)
if condition (24) holds. Here and throughout the rest of the paper we use notation u+ := max{u, 0}. It is clear that, in general, this lower bound is not asymptotically tight, and we will discuss this fact later. Let k = h(P ∗ (k)), k = 1, . . . , n, H (41)
896
I. Dumer, M.S. Pinsker, and V.V. Prelov
where P ∗ (k) = ( p∗n−k+1 , . . . , p∗n ) is defined similar to P ∗ (n) (cf. (21) and (22)): −1 p∗i = p∗i (k) := 1 + 2λvi ,
i = n − k + 1, . . . , n,
= λ(k) and parameter λ is defined by the equalities ⎧ −1 n i ⎨ n λv = 1 if 12 i=n−k+1 vi > 1, i=n−k+1 vi 1 + 2 n ⎩λ =0 if 12 i=n−k+1 vi ≤ 1.
(42)
(43)
In particular, comparing relations (21)–(23) and (41)–(43), we observe that λ(n) = λ(n),
P∗ (n)P ∗ (n),
n = Hn . H
Now, define the quantity Rn (ε) :=
max
{k: 2ε
Abstract. In this paper, we present some new results on the thinnest coverings that can be obtained in Hamming or Euclidean spaces if spheres and ellipsoids are covered with balls of some radius ε. In particular, we tighten the bounds currently known for the ε-entropy of Hamming spheres of an arbitrary radius r. New bounds for the ε-entropy of Hamming balls are also derived. If both parameters ε and r are linear in dimension n, then the upper bounds exceed the lower ones by an additive term of order log n. We also present the uniform bounds valid for all values of ε and r. In the second part of the paper, new sufficient conditions are obtained, which allow one to verify the validity of the asymptotic formula for the size of an ellipsoid in a Hamming space. Finally, we survey recent results concerning coverings of ellipsoids in Hamming and Euclidean spaces.
1
Introduction
Let En be the Hamming space of binary vectors x = (x1 , . . . , xn ) of length n. Given an integer ε, let B n (y, ε) be the ball of radius ε centered at the point y ∈ En , i.e., B n (y, ε) := {x ∈ En | d(x, y) ≤ ε} , where d(x, y) is the Hamming distance between x and y, that is the number of coordinate positions in which xi = yi . We say that a subset Mε (A) ⊆ En forms an ε-covering of a set A ⊆ En if A belongs to the union of the balls B n (y, ε) centered at points y ∈ Mε (A), i.e., A⊆ B n (y, ε). y∈Mε (A)
The ε-entropy [1] Hε (A) of a set A is the logarithm of the size of its minimal ε-covering: Hε (A) := log min |Mε (A)|,
Supported in part by NSF grant CCR-0097125. Supported in part by the Russian Foundation for Basic Research (project no. 03– 01–00592) and INTAS (project 00-738). Supported in part by ZIF (project “General Theory of Information Transfer and Combinatorics”), the Russian Foundation for Basic Research (project no. 03–01– 00592) and INTAS (project 00-738).
R. Ahlswede et al. (Eds.): Information Transfer and Combinatorics, LNCS 4123, pp. 883–910, 2006. c Springer-Verlag Berlin Heidelberg 2006
884
I. Dumer, M.S. Pinsker, and V.V. Prelov
where the minimum is taken over all ε-coverings Mε (A) and log denotes logarithm base 2. Given a vector v = (v1 , . . . , vn ), vi ∈ [0, ∞), i = 1, . . . , n, the ellipsoid Evn is defined by the equality n n n Ev := x ∈ E | vi xi ≤ 1 , (1) i=1
where Note that the inequality n all operations are performed over real numbers. n 2 i=1 vi xi ≤ 1 is equivalent to the inequality i=1 vi xi ≤ 1. This fact explains the notion of ellipsoid for Evn by analogy with that in the Euclidean space. Below we study the asymptotic behavior (as n → ∞) of the ε-entropy of an arbitrary ellipsoid Evn and, in particular, the ε-entropy of a ball B n (r) := B n (0, r) and a sphere S n (r) := {x ∈ En | d(x, 0) = r} , where r is an integer. The following notation will be used throughout the paper. Consider the binary entropy function h(t) := −t log t − (1 − t) log(1 − t),
0 ≤ t ≤ 1.
Then, given any integer n and real-valued vectors P = P (n) = (p1 , . . . , pn ) and Q = Q(n) = (q1 , . . . , qn ) such that 0 ≤ pi ≤ 1, 0 ≤ qi ≤ 1, i = 1, . . . , n, define the function h(P, Q) :=
n
h(pi , qi ) = h(P ) − h(Q),
i=1
where h(pi , qi ) := h(pi ) − h(qi ),
h(P ) :=
n i=1
h(pi ),
h(Q) :=
n
h(qi ).
i=1
Here and below, we sometimes omit an argument n or a superscript n in our notation of vectors, ellipsoids, balls, and spheres. In this paper, we also consider an arbitrary ellipsoid Ea = Ean defined in the n-dimensional Euclidean space Rn as n x2i n n Ea := x = (x1 , . . . , xn ) ∈ R | ≤ 1, (2) a2 i=1 i where a = (a1 , . . . , an ) is a real-valued vector with n positive coordinates. We will use the same notation B n (y, ε) and S n (r) for balls and spheres in the Euclidean space Rn as the one used in the Hamming space Rn . The definitions of B n (y, ε), S n (r) and the ε-entropy Hε (A) of a bounded set A ⊂ Rn in the
On the Thinnest Coverings of Spheres and Ellipsoids
885
Euclidean space are absolutely similar to those in the Hamming space. The only difference arises in the definition of the function d(x, y): now d(x, y) is the Euclidean distance between real-valued vectors x and y. Also, in the Euclidean space, the radii r, ε, and many other parameters can take arbitrary positive values instead of integers used in the Hamming space. In both spaces we are interested in the asymptotic ε-entropy of any ellipsoid Evn or Ean Note that ellipsoids in the Hamming spaces arise in various problems related to combinatorics, decoding, and data compression. For example, ellipsoids in the form of (1) can be considered as Boolean threshold functions. Also, ellipsoids emerge in maximum likelihood decoding for binary memoryless channels. On the other hand, the problem of coverings of spheres and ellipsoids in Euclidean spaces often arises in vector quantizers. In the next section, we better the bounds recently obtained for the ε-entropy of spheres in the Hamming space. We also extend these results for balls and derive new uniform bounds. In Section 3, we shortly survey a few results already known for the size and the ε-entropy of ellipsoids in the Hamming space. There we also derive some new sufficient conditions for the validity of the asymptotic formula for the size of an ellipsoid. Section 4 is devoted to recent results obtained for the ε-entropy of ellipsoids in Euclidean spaces.
2
Covering of Spheres and Balls in Hamming Spaces
Below we assume in this section that n, r, and ε are some positive integers such that the ratios ρ := r/n, σ := ε/n satisfy condition 0 < σ < ρ ≤ 1/2. Firstly, note that the whole space En can be considered as a special case of a ball B n (r) when r = n. It is shown (see, e.g., [2, 3]) that the normalized ε-entropy H ε (En ) := Hε (En )/n satisfies equality log n n H ε (E ) = 1 − h(σ) + O , n → ∞, n for any fixed σ. Non-uniform lower and upper bounds on the ε-entropy of spheres in the Hamming space are obtained in [4]. It is proven there that for any fixed ρ < 1/2 and σ there exist some constants c(ρ, σ) and C(ρ, σ), which are independent of n but can depend on ρ and σ such that the normalized ε-entropy H ε (S(r)) satisfies the following inequalities: h(ρ) − h(σ) +
c(ρ, σ) 3 log n C(ρ, σ) ≤ H ε (S(r)) ≤ h(ρ) − h(σ) + + . n 2n n
The following theorem improves this bound and also gives the uniform (with respect to parameters ρ and σ) lower and upper bounds on the ε-entropy of
886
I. Dumer, M.S. Pinsker, and V.V. Prelov
spheres in the Hamming space. Such bounds are of independent interest, and can be used, for example, in deriving the upper bounds for the ε-entropy of ellipsoids. Theorem 1 1. For all n, r ≤ n/2, and ε < r, the normalized ε-entropy of an n-dimensional sphere S(r) satisfies the following lower and upper bounds: h(ρ) − h(σ) −
c 3 log n log(2n) ≤ H ε (S(r)) ≤ h(ρ) − h(σ) + + , 2n 2n n
(3)
where c is an absolute constant. Moreover, the lower bound in (3) can be improved if σ or both σ and ρ are constants independent of n. Namely, 2. If σ is a constant, then there exists a constant c1 (σ) such that H ε (S(r)) ≥ h(ρ) − h(σ) +
c1 (σ) . n
(4)
3. If both ρ and σ are constants, then there exists a constant c2 (ρ, σ) such that H ε (S(r)) ≥ h(ρ) − h(σ) +
log n c2 (ρ, σ) + . 2n n
(5)
Proof. This theorem will be proven using Proposition 1 given in the sequel. However, the lower bounds in (3) and (4) are almost trivial. Indeed, we first use the following well-known inequalities (see, e.g., (10.16) and (10.20) in [5]): (8nρ(1 − ρ))−1/2 2nh(ρ) ≤ |S(r)| ≤ (2πnρ(1 − ρ))−1/2 2nh(ρ)
(6)
|B(ε)| ≤ 2nh(σ) .
(7)
and Applying (6) and (7) we obtain the lower (packing) bound |S(r)| 1 log ≥ h(ρ) − h(σ)− n |B(ε)| log(2n) log 8nρ(1 − ρ) ≥ h(ρ) − h(σ) − , 2n 2n H ε (S(r)) ≥
which gives the left inequality in (3). Similarly, note that for σ < 1/2, we have |B(ε)| ≤ |S(ε)| i=0 ε
σ 1−σ
i ≤
1−σ , 1 − 2σ
which follows from the inequality |S(ε − 1)|/|S(ε)| ≤ σ/(1 − σ).
(8)
On the Thinnest Coverings of Spheres and Ellipsoids
887
Therefore, using (6) and (8), we obtain inequality (4): |S(r)| 1 |S(r)| 1 |B(ε)| 1 log = log − log n |B(ε)| n |S(ε)| n |S(ε)| log(2n) log(2πnσ(1 − σ)) 1 1−σ ≥ h(ρ) − h(σ) − + − log . 2n 2n n 1 − 2σ
H ε (S(r)) ≥
To proceed with the other bounds of Theorem 1, we shall extensively use the following notation. Given any integer w ∈ [r − ε, r + ε], we consider a point x ∈ S(w) and the subset Ar (x, ε) := B(x, ε) ∩ S(r). Given any y ∈ S(r), we also consider the set Aw (y, ε) := B(y, ε) ∩ S(w). Now we can prove that H ε (S(r)) can be tightly related to the function Ω(r, ε) := max |Ar (x, ε)|, w
x ∈ S(w).
(9)
Namely, we shall prove that the ε-entropy of the sphere S(r) satisfies the bounds 1 |S(r)| |S(r)| log(n ln 2) log 2h(ρ) 1 log ≤ H ε (S(r)) ≤ log + + . n Ω(r, ε) n Ω(r, ε) n n
(10)
1. The lower bound in (10) is based on a standard packing argument. Here we use the fact that each ball B(x, ε) can cover at most Ω(r, ε) points on S(r). 2. The upper bound is obtained using random coverings. Given a fixed integer N, we choose the radius w, for which equality (9) holds. Then we perform N trials choosing the centers x of an ε-covering independently and uniformly on S(w). Then any point y ∈ S(r) is covered in one trial with the same probability γ=
|Aw (y, ε)| . |S(w)|
(11)
Now we employ a straightforward argument (used by Bassalygo in 1965; see also Lemma 6 in [4]) that states that for any x ∈ S(w) and any y ∈ S(r), the two sets Ar (x, ε) and Aw (y, ε) cover the same fraction of spheres S(r) and S(w), respectively. Therefore, |Ar (x, ε)| . (12) γ= |S(r)| Now note that any y ∈ S(r) is not covered in N trials with probability PN = (1 − γ)N . Choosing 1 + ln |S(r)| , (13) N= γ
888
I. Dumer, M.S. Pinsker, and V.V. Prelov
we obtain PN ≤ (e|S(r)|)−1 . Therefore the whole sphere S(r) is covered in N trials with a probability P N (S(r)) ≥ 1 − PN · |S(r)| ≥ 1 − e−1 . In this case, there exists an ε-covering of size N. Using (12), we obtain the upper bound H ε (S(r)) ≤
1 |S(r)| 1 log N = log + log(1 + ln |S(r)|). n n Ω(r, ε) n
(14)
Now the upper bound in (10) follows from (6). Thus, we see that to prove Theorem 1 we need to derive rather tight lower and upper bounds on the quantity Ω(r, ε). These bounds - derived in the sequel in Proposition 1 - conclude the proof of Theorem 1. In fact, slightly more precise calculations show that the universal constant c belongs to the interval (0, 2). Proposition 1 1. For all n, r ≤ n/2, and ε < r, the following lower and upper bounds hold: nh(σ) − log 8n ≤ log Ω(r, ε) ≤ nh(σ).
(15)
Moreover, 2. If σ is a constant, then there exists a constant c(σ) such that log Ω(r, ε) ≤ nh(σ) − (log n)/2 + c(σ). 3. If ρ and σ are constants, then there exists a constant c(ρ, σ) such that log Ω(r, ε) ≤ nh(σ) − log n + c(ρ, σ). Proposition 1 will be proven in the Appendix. In the following theorem, we present the corresponding non-uniform and uniform lower and upper bounds for the ε-entropy of balls in the Hamming space. Theorem 2 1. For all n, r ≤ n/2, and ε < r, the normalized ε-entropy of an n-dimensional ball B(r) satisfies the following lower and upper bounds: h(ρ) − h(σ) −
2 log n C log(2n) ≤ H ε (B(r)) ≤ h(ρ) − h(σ) + + , 2n n n
(16)
where C is an absolute constant. Moreover, 2. If σ is a constant, then there exists a constant C1 (σ) such that H ε (B(r)) ≥ h(ρ) − h(σ) +
C1 (σ) . n
(17)
3. If both ρ < 1/2 and σ are constants, then there exists a constant C2 (ρ, σ) such that h(ρ) − h(σ) +
3 log n C log n C2 (ρ, σ) + ≤ H ε (B(r)) ≤ h(ρ) − h(σ) + + . (18) 2n n 2n n
On the Thinnest Coverings of Spheres and Ellipsoids
889
Proof. Note first that the lower bounds in (16) - (18) immediately follow from the corresponding lower bounds in (3) - (5) since S(r) ⊂ B(r) for all r.1 Before starting to prove the upper bounds in (16)–(18), note that for all r and ε we have (19) Hε (B(r)) ≤ Hε (S(r)) + log n r since B(r) = k=0 S(k) and Hε (S(k)) ≤ Hε (S(r)) for all k = 0, . . . , r. Therefore, to derive some upper bounds for Hε (B(r)) we can use (19) together with the upper bounds in (3) and (4). By doing this, we obtain the upper bounds which are worse than those in (16)–(18). The latter means that the trivial inequality (19) does not allow us to prove the upper bounds in (16)–(18). In order to prove these upper bounds, let us use the following modification of the random choice method described in the proof of Theorem 1. Namely, let an integer N be fixed. Then we choose N points x(1) , . . . , x(N ) of an εcovering for the ball B(r) randomly according to the following rule. In the ith trial, i = 1, . . . , N , independently of all other trials, we choose an integer u, ε + 1 ≤ u ≤ r, at random with probability p(u) =
|S(u)| , |B(r)| − |B(ε)|
ε + 1 ≤ u ≤ r.
Given u, the i-th point x(i) of the ε-covering is chosen randomly on the sphere S(u ), where
u−ε u := 1 − 2σ Similarly to the proof of Theorem 1 (see also formula (89) in the Appendix), here we also use the fact that our choice of u maximizes the covering area Au (x, ε) generated by the ball B(x, ε) with any center x ∈ S(u ), so that |Au (x, ε)| = Ω(u, ε). Now we estimate the probability γy that any fixed point y ∈ S(u) is covered by a ball B(x(i) , ε). First, note that similarly to equalities (11) and (12), for any x ∈ S(u ) and y ∈ S(u) we have the equality u A (y, ε) |Au (x, ε)| = . |S(u )| |S(u)| Then we use inequality (15) for Ω(u, ε) and see that any point y ∈ S(u) is covered by a ball B(x(i) , ε) with probability γy ≥
|S(u)| |B(r)|−|B(ε)|
≥ γr := 1
·
Ω(u,ε) |S(u)|
2nh(σ)−log 8n |B(r)|−|B(ε)| .
(20)
Note that the size of the covering is always bounded from below by |B(r)|/|B(ε)|. This fact allows one to remove the term −(log n)/2n from the lower bound (16). We omit the corresponding calculations.
890
I. Dumer, M.S. Pinsker, and V.V. Prelov
It is important that for any fixed point y ∈ B(r), the probability γy of its covering in one trial is bounded from below in (20) by a quantity γr that does not depend on y. The rest of the proof of the theorem almost coincides with that of Theorem 1 and therefore we omit it here. The main difference arises from the fact that now we use a bigger set B(r) in (20) instead of S(r) employed in (12). Note also that now we take N=
1 (1 + ln |B(r)|), γr
instead of (13), but this change does not affect the asymptotics of H ε (B(r)). Finally, we apply the inequalities |B(r)| ≤
1−ρ |S(r)| 1 − 2ρ
(cf. (8)) in the proof of (18) and |B(r)| ≤ 2nh(ρ) in the proof of (16) and (17). This completes the proof of Theorem 2.
The main conclusion we derive from Theorems 1 and 2 is that for both spheres and balls with fixed relative radii ρ and ε, the corresponding upper and lower bounds differ only by an additive term of order log n. Note that this term is related to our random-covering algorithm used for upper bounds. Thus, any further tightening of the above bounds is only possible if there exist constructive coverings that surpass their randomly chosen counterparts.
3 3.1
Covering of Ellipsoids in Hamming Spaces The Size of Ellipsoids
Consider an ellipsoid Ev = Evn defined in (1). The first interesting question is the problem of finding the number of different ellipsoids in En . It was proven 2 long ago [6] that this number is upper bounded by 2n . The problem had been addressed in many publications for over 140 years until Zuev proved [7] that the 2 number of different ellipsoids is lower bounded by 2n (1−10/ ln n) thus solving the 2 problem up to the exponential order of 2n . Another important problem is to derive the size of any ellipsoid Ev . This quantity |Ev | is necessary, for example, to write out the packing (Hamming) bound for the ε-entropy of Ev . The main term of the asymptotics of log |Ev | was found by Pinsker [8]. To state his result, let us consider the vector P ∗ = P ∗ (n) = (p∗1 , . . . , p∗n ) with components
−1 p∗i = p∗i (n) := 1 + 2λvi ,
i = 1, . . . , n,
(21)
On the Thinnest Coverings of Spheres and Ellipsoids
where parameter λ = λ(n) is defined by the equalities
n n λvi −1 = 1 if 12 i=1 vi > 1, i=1 vi 1 + 2 n λ=0 if 12 i=1 vi ≤ 1. Let
891
(22)
Hn := h(P ∗ (n)).
(23)
Then the following statement holds. Theorem 3. [8].
If lim
n→∞
Hn = ∞, log n
(24)
then Hn (1 + o(1)) ≤ log |Ev | ≤ Hn ,
n → ∞,
(25)
and, in particular, log |Ev | = Hn (1 + o(1)),
n → ∞.
(26)
The proof of the upper bound in (25) given in [8]) (see Lemma 1 there) is rather simple and short and therefore for reader’s convenience we reproduce it here. Indeed, given the uniform distribution Pr(x) = 2−n , x ∈ En , we have n vi Xi ≤ 1 + n, (27) log |Ev | = log Pr(Ev ) + n = log Pr i=1
where Xi , i = 1, . . . , n, are independent binary random variables taking values n vi /2 > 1, then 0, 1 with probabilities Pr(0) = Pr(1) = 1/2. It is clear that if Pr
n
vi Xi ≤ 1
= Pr 2
−λ
n i=1
i=1
vi Xj
≥2
−λ
≤ 2 E2 λ
−λ
n
vi Xi
i=1
i=1
= 2λ
n
E 2−λvi Xi = 2λ
i=1
n 1 + 2−λvi i=1
2
.
Therefore, formulas (27) and (28) imply log |Ev | ≤ λ +
n
log 1 + 2−λvi
i=1
=
n
−1 λvi 1 + 2λvi + log 1 + 2−λvi
i=1
=
n i=1
h(p∗i ) = h(P ∗ ) = Hn .
(28)
892
I. Dumer, M.S. Pinsker, and V.V. Prelov
For the case
n i=1
vi /2 ≤ 1, we have λ = 0 and p∗i = 1/2, i = 1, . . . , n, and
therefore the right-hand side inequality in (25) is trivially fulfilled. The proof of the lower bound in (25) is more involved. The idea is to construct a sequence of sets An,u ⊆ Ev , n = 1, 2, . . . that are the direct products of balls of a special radius u in the corresponding subspaces and then to evaluate the asymptotic behavior of the size of An,u . For details of the proof we refer to [8]. Without loss of generality, below we will assume that the coefficients vi , i = 1, . . . , n, of an ellipsoid Ev form a non-increasing sequence, i.e., v1 ≥ v2 ≥ . . . ≥ vn .
(29)
This can always be attained by renumbering the elements of the sequence {vi }. Further, we will consider two different cases: (i) the whole sequence {vi }∞ i=1 is given, i.e., vi does not depend on n for any fixed i = 1, 2, . . .; (ii) ”scheme of series”: each vi = vi (n) can depend on n but for any given n the elements vi (n), i = 1, . . . , n, satisfy condition (29). The following statement, for the case (i), gives a simple necessary and sufficient condition on coefficients {vi } under which the main condition (24) of Theorem 3 is fulfilled. Proposition 2. Given a non-increasing sequence {vi }∞ i=1 , condition (24) holds if and only if lim vn = 0. (30) n→∞
Corollary 1. Given a non-increasing sequence {vi }∞ i=1 , Theorem 3 can be reformulated as follows: if lim vn = 0, then inequalities (25) and equality (26) n→∞ are valid. Proof. It can easily be seen that Hn = max h(P ),
(31)
P
where Hn is defined in (23) and the maximum in (31) is taken over all vectors P = (p1 , . . . , pn ) such that 0 ≤ p1 , . . . , pn ≤ 1/2,
n
vi pi ≤ 1.
i=1
We will use equality (31) below. 1. Assume first that lim vn = 0, i.e., there exists a positive constant v such n→∞ that vn ≥ v > 0 for all n. Then we have h(P ), Hn = max h(P ) ≤ max P
P
(32)
On the Thinnest Coverings of Spheres and Ellipsoids
893
where the second maximum in (32) is taken over all vectors P = (p1 , . . . , pn ) such that n 0 ≤ p1 , . . . , pn ≤ 1/2, vpi ≤ 1. i=1
It is clear that h(P ) = nh max P
1 nv
=
1 (log n)(1 + o(1)), v
n → ∞.
(33)
Relations (32) and (33) show that condition (24) is not satisfied if lim vn = 0. n→∞
2. Assume now that lim vn = 0. Let k = k(n) = αn where α, 0 < α < 1, is n→∞
a fixed constant.2 Then we clearly have n kh(1/(kvn−k+1 )) Hn ≥ max h( pi (k)) = k P (k) i=n−k+1
if if
kvn−k+1 ≥ 2, kvn−k+1 ≤ 2,
(34)
where the maximum in (34) is taken over all k-dimensional vectors P (k) = ( pn−k+1 (k), . . . , pn (k)) such that n
0 ≤ pn−k+1 (k), . . . , pn (k) ≤ 1/2,
vn−k+1 pi (k) ≤ 1.
i=n−k+1
Let a rather large positive constant A > 2 be fixed. Consider first a subset N1 of integers n for which kvn−k+1 ≤ A, n ∈ N1 . For such n ∈ N1 , inequality (34) shows that Hn ≥ kh(1/A) = αnh(1/A),
(35)
and (24) holds. On the other hand, for n ∈ N2 for which kvn−k+1 > A,
n ∈ N2 ,
it follows from (34) that Hn ≥
log(kvn−k+1 ) . vn−k+1
(36)
Therefore, if vn−k+1 ≤
1 , log k
n ∈ N2 ,
then we have Hn ≥ (log A · log k), 2
n ∈ N2 .
(37)
More precisely, we should write k = αn instead of k = αn but this difference does not affect further asymptotic relations.
894
I. Dumer, M.S. Pinsker, and V.V. Prelov
At the same time if
1 , n ∈ N2 , kc where c, 0 < c < 1, is a constant, then we obtain from (36) that vn−k+1 ≥
Hn ≥
log k − log(1/vn−k+1 ) (1 − c) log k ≥ , vn−k+1 vn−k+1
Hence, inequalities (37) and (38) imply that 1−c Hn ≥ min log A, (log k), vn−k+1
n ∈ N2 .
n ∈ N2 .
(38)
(39)
Finally, taking A → ∞ rather slowly, we conclude from (35) and (39) that lim
n→∞
Hn = ∞. log n
Proposition 2 is proved.
Consider now the general case - “scheme of series” - where each vi = vi (n) may depend on n but for any given n the sequence is non-increasing. For such a situation the following statement holds. Proposition 3. Assume that for any given n the sequence vi = vi (n), i = 1, . . . , n, does not increase. Then (a) If lim vn = 0, then condition (24) is not fulfilled. n→∞
(b) If there exists a sequence k = k(n) such that vn−k+1 → 0 as n → ∞, log k lim inf log n > 0, n→∞
then condition (24) does hold. Proof. The proof of part (a) does not differ from that of Proposition 2. To prove the second statement, note that in the proof of the direct part of Proposition 2 we have only used the facts that lim (log k/ log n) > 0 for subsequence k = k(n) and n→∞
that vn−k+1 → 0 as n → ∞ (which in turn follows from condition (30) used in Proposition 2). Now the two latter conditions are introduced in the formulation of Proposition 3. Therefore, taking into account this observation, we can claim that Proposition 3 holds. Remark 2. It easily follows from inequality (34) (which also holds for the scheme of series) that if there exists a sequence k = k(n) such that n i=n−k+1
vi ≤ 2 and
k → ∞ as n → ∞ log n
On the Thinnest Coverings of Spheres and Ellipsoids
or n
n
kh 1/
i=n−k+1
vi > 2 and
log n
i=n−k+1
895
vi → ∞ as n → ∞,
then condition (24) is fulfilled. In particular, this holds if n
vi ≤ 2
i=1
n kh 1/ vi
or n
vi > 2 and
i=1
i=1
log n
→ ∞ as n → ∞.
Remark 3. It should be mentioned that using some results known from the theory of limit theorems and, in particular, the theory of large deviations for the sums of independent but not-identically distributed random variables, the asymptotics of log |Ev | can be expressed in a different form. Moreover, in certain special cases, the next terms of the asymptotics of log |Ev | can also be found [9]. 3.2
The ε-Entropy of Ellipsoids
Below in this subsection, we will always assume, as was already mention earlier, that the coefficients vi = vi (n), i = 1, . . . , n, do not increase for any given n and, moreover, that ε = ε(n) is an integer such that 1 ≤ ε < n/2. Otherwise (i.e., if ε ≥ n/2) the equality Hε (Evn ) = O(log n),
n → ∞,
holds for arbitrary ellipsoids Evn since Evn ⊆ En and it is obvious that Hε (En ) = O(log n) (cf. also (2)). Lower bound Using relation (26) for the size of ellipsoid Ev and inequality (7) for the size of the ball B(ε), we can easily write out the packing (Hamming) bound for the ε-entropy Hε (Evn ): Hε (Evn ) ≥ log+
|Ev | ≥ [Hn − nh(ε/n)]+ (1 + o(1)), |B(ε)|
n → ∞,
(40)
if condition (24) holds. Here and throughout the rest of the paper we use notation u+ := max{u, 0}. It is clear that, in general, this lower bound is not asymptotically tight, and we will discuss this fact later. Let k = h(P ∗ (k)), k = 1, . . . , n, H (41)
896
I. Dumer, M.S. Pinsker, and V.V. Prelov
where P ∗ (k) = ( p∗n−k+1 , . . . , p∗n ) is defined similar to P ∗ (n) (cf. (21) and (22)): −1 p∗i = p∗i (k) := 1 + 2λvi ,
i = n − k + 1, . . . , n,
= λ(k) and parameter λ is defined by the equalities ⎧ −1 n i ⎨ n λv = 1 if 12 i=n−k+1 vi > 1, i=n−k+1 vi 1 + 2 n ⎩λ =0 if 12 i=n−k+1 vi ≤ 1.
(42)
(43)
In particular, comparing relations (21)–(23) and (41)–(43), we observe that λ(n) = λ(n),
P∗ (n)P ∗ (n),
n = Hn . H
Now, define the quantity Rn (ε) :=
max
{k: 2ε
