Error Decay of (almost) Consistent Signal Estimations from Quantized ...
Error Decay of (almost) Consistent Signal Estimations from Quantized Random Gaussian Projections Laurent Jacques∗ June 3, 2014
arXiv:1406.0022v1 [cs.IT] 30 May 2014
Abstract This paper provides new error bounds on consistent reconstruction methods for signals observed from quantized random sensing. Those signal estimation techniques guarantee a perfect matching between the available quantized data and a reobservation of the estimated signal under the same sensing model. Focusing on dithered uniform scalar quantization of resolution δ > 0, we prove first that, given a random Gaussian frame of RN with M vectors, the worst case `2 -error of consistent signal reconstruction decays with high probability as N O( M log √MN ) uniformly for all signals of the unit ball BN ⊂ RN . Up to a log factor, this matches a known lower bound in Ω(N/M ). Equivalently, with a minimal number of frame √ N N coefficients behaving like M = O( 0 log( 0 )), any vectors in BN with M identical quantized projections are at most 0 apart with high probability. Second, in the context of Quantized Compressed Sensing with M random Gaussian measurements and under the same scalar quantization scheme, consistent reconstructions of K-sparse signals of RN have a worst case K MN error that decreases with high probability as O( M log √ 3 ) uniformly for all such signals. K Finally, we show that the strict consistency condition can be slightly relaxed, e.g., allowing for a bounded level of error in the quantization process, while still guaranteeing a proximity between the original and the estimated signal. In particular, if this quantization error is of order O(1) with respect to M , similar worst case error decays are reached for reconstruction methods adjusted to such an approximate consistency.
1
Introduction
Since the advent of the digital signal processing era and of analog to digital converters an intense field of research has been concerned by the following non-linear sensing model q = Q[Ax] ∈ J ,
(1)
where A ∈ RM ×N is a matrix representing a linear transformation (or sensing) of a signal x taken in some bounded subset K of RN and Q stands for a quantization of Ax mapping AK ⊂ RM to a finite set of vectors J ⊂ RM , e.g., using a given number of bits [7, 16].
Since the bounded space K contains a priori an infinite number of signals, the model (1) is of course lossy and x cannot be recovered exactly from q. Quantifying this loss of information in function of both the signal reconstruction method and of the key elements A, N , M , K and Q has therefore been the source of numerous research at the frontier of information theory, high dimensional geometry, signal processing and statistics. ∗
LJ is with the ICTEAM institute, ELEN Department, Universit´e catholique de Louvain (UCL), Belgium.
1
The general model (1) is for instance the one adopted in Quantized Compressed Sensing (QCS) [7, 11, 19] when the signal x is assumed sparse or compressible in an orthonormal basis Ψ of RN and when the sensing matrix is generated randomly, e.g., from random Gaussian ensembles [10]. When M > N , Eq. (1) is also a model for frame coefficient quantization (FCQ) of signals in RN , i.e., when the coefficients Ax = (aT1 x, · · · , aTM x)T of x in a overcomplete frame of RN are quantized in q = Q(Ax) ∈ J , A = (a1 , · · · , aM )T representing then the matrix whose row set {aj ∈ RN : 1 6 j 6 M } collects the frame vectors [3, 14, 15].
In this work, we restrict the analysis of (1) to a scalar, regular and uniform quantizer, i.e., when Q is a scalar operation applied componentwise on Ax ∈ RM , and when the 1-D quantization cells defined by the set function Q−1 [c] = {λ : Q[λ] = c} ⊂ R are convex and have all the same size. We refer the reader for instance to [7] for a review of scalar and Σ∆quantization [17] in the QCS literature, to [4] for a theoretical analysis of non-regular scalar quantizers, or to [27] for an example of vector quantization by frame permutation. In particular, we adopt a uniform midrise quantizer Qδ (λ) := δ(b λδ c + 21 ) ∈ δ(Z + 21 ) =: Zδ
(2)
of resolution δ > 0 applied componentwise on vectors. Moreover, we focus on the interplay of Qδ with both a random Gaussian matrix A = Φ ∼ N (0, 1)M ×N and a dithering ξ ∼ U M ([0, δ]), where DM ×N (η) and DM (η) denote a M × N random matrix or a M -length random vector, respectively, whose entries are identically and independently distributed as the probability distribution D(η) of parameters η = (η1 , · · · , ηP ), e.g., the standard normal distribution N (0, 1) or the uniform distribution U([0, δ]). Such a dithering is often used for improving the statistical properties of the quantizer by randomizing the unquantized input location inside the quantization cell [16]. As will become clear later (see Sec. 6), this uniform dithering allows us also to bridge our analysis with a geometrical probability context inspired by Buffon’s needle problem [8, 21]. Consequently, given a signal x in a bounded set K ⊂ RN , the quantized sensing scenario studied in this paper reads (3) q = Qδ [Φx + ξ] ∈ ZM δ . This is either a sensing model for QCS with a random Gaussian sensing Φ, or a quantization scheme for FCQ when the overcomplete frame is made of M vectors in RN (with M > N ) that are randomly and independently drawn from N (0, 1N ×N ). This guarantees that they are also linearly independent with probability 1, i.e., we obtain a random Gaussian frame (RGF) of RN . Our main objective in order to quantify the information loss in (3) while trying to estimate x is to characterize the worst case error Eδ (Φ, ξ, K) := max kx − x∗ k x∈K
of any consistent reconstruction method whose output x∗ is determined by the following formal program: find any x∗ ∈ RN such that Qδ (Φx∗ + ξ) = Qδ (Φx + ξ) and x∗ ∈ K.
(4)
In the case of FCQ of signals in a RGF (i.e., M > N ), we set K = BN , while in the context of QCS we take K = ΣK (Ψ) ∩ BN where ΣK (Ψ) = {v = Ψα ∈ RN : kαk0 6 K}, with kαk0 := #{j ∈ [N ] : αj 6= 0}, is the space of K-sparse signals in the orthonormal basis Ψ ∈ RN ×N . For the sake of simplicity, we work with the canonical basis Ψ = 1N ×N . However, all our results can be applied to Ψ 6= 1N ×N from the rotational invariance of the random Gaussian matrix Φ ∼ N M ×N (0, 1) in RN [11]. 2
We acknowledge the fact that for K = ΣK ∩ BN the program (4) is possibly NP hard, e.g., if x∗ is found by minimizing the `0 “norm” under the consistency constraint [26]. However, similarly to the procedure developed in [23], we are anyway interested in studying its reconstruction error, remembering that similar ideal reconstructions in CS and in QCS have often driven the determination of feasible programs [11, 13, 19, 20, 23, 28, 32]. Notice that the error Eδ is also associated to the biggest size, with respect to all x ∈ K, of all consistency cells Cx := {x∗ ∈ K : Qδ (Φx∗ + ξ) = Qδ (Φx + ξ)}, i.e., Eδ (Φ, ξ, K) = max max kx − x∗ k, ∗ x∈K x ∈Cx
which shows that the characterization of Eδ is actually a high dimensional geometric problem, whose general formulation can be connected to the problem of finding a finite covering of K of minimal size [7]. Our first contribution is an upper bound on this worst case error for consistent signal reconstruction in the context of Random Gaussian Frame Coefficient Quantization (RG-FCQ). Theorem 1. Let us fix 0 > 0, 0 < η < 1, δ > 0 and M > N such that M>
4δ + 20 0
√ 1 N log( 290 N ) + log 2η .
Let us randomly draw a RGF Φ ∼ N M ×N (0, 1) and a dithering ξ ∼ U M ([0, δ]). Then, with a probability higher than 1 − η, for all x ∈ K = BN sensed by (3), any solution x∗ to (4) is such that kx − x∗ k 6 0 , or, equivalently, Eδ (Φ, ξ, BN ) 6 0 . As a corollary, the asymptotic decay of Eδ in function of M , N , δ and of the probability η can be established. Corollary 1. Given M > 0, 0 < η < 1, δ > 0 with δ = O(1), there exists a constant C > 0 such that N P Eδ (Φ, ξ, BN ) 6 C M log √MN + log η2 > 1 − η, where the probability is computed with respected to both the RGF Φ ∼ N M ×N (0, 1) and the random dithering ξ ∼ U M ([0, δ]). Loosely speaking, this corollary says that if δ = O(1), with a probability higher than 1 − η, N Eδ (Φ, ξ, BN ) = O( M log √MN + log η2 ).
(5)
As shown in Sec. 4, it is then straightforward to generalize Theorem 1 to the set of sparse signals. Theorem 2. Let us fix 0 > 0, 0 < η < 1, δ > 0 and M such that 1 M > 4δ +0 20 2K log( √56N ) + log( 2η ) . K 0
Let us randomly draw a Gaussian sensing matrix Φ ∼ N M ×N (0, 1) and a dithering ξ ∼ U M ([0, δ]). Then, with a probability higher than 1 − η, for all x ∈ K = ΣK ∩ B N sensed by (3), any solution x∗ to (4) is such that kx − x∗ k 6 0 , or, equivalently, Eδ (Φ, ξ, ΣK ∩ BN ) 6 0 .
3
In such a case, as for Corollary 1 in the context of RGF, Sec. 4 also shows that, for any M > 0, K MN Eδ (Φ, ξ, ΣK ∩ BN ) = O( M log( √ ) + log η2 ), 3 K
with a probability higher than 1 − η.
As explained in Sec. 2, this matches existing error bounds for 1-bit compressed sensing in the case√ of Gaussian random projections [23]. It also improves previousp known bounds decaying as O(1/ M ) for linear reconstruction methods in FCQ [14] and as O( K/M ) for QCS, while a known lower bound in Ω(K/M ) exists [7]. Our result behaves also similarly to the bound on the mean worst case error (mean established with respect to Φ and ξ in the context of our notations) of consistent reconstruction methods obtained in [30] in the case of random frames over SN −1 . As a last contribution, we show that slight deviations to strict consistency is also possible while keeping similar proximity relations between almost consistent vectors. This occurs for instance if the sensing model (3) is moderately corrupted, i.e., if there exists a noise n ∈ RM such that, for any x ∈ K, kQδ ((Φx + n) + ξ) − Qδ (Φx + ξ)k1 6 r δ. In such a case, we can relax the formal reconstruction (4) and ask to reconstruct x∗ from find any x∗ ∈ RN such that kQδ (Φx∗ + ξ) − Qδ (Φx + ξ)k1 6 r δ and x∗ ∈ K.
(6)
This leads to study a new worst case reconstruction error associated to the definition of relaxed consistency cells Cxr := {x∗ ∈ K : kQδ (Φx∗ + ξ) − Qδ (Φx + ξ)k1 6 r δ}, i.e., kx − x∗ k. Eδr (Φ, ξ, K) := max max ∗ r x∈K x ∈Cx
Sec. 5 proves that, with a probability higher than 1 − η with respect to the random draw of Φ ∼ N M ×N (0, 1) and ξ ∼ U M ([0, δ]), the worst case error on almost consistent vectors behaves like ) 1 Eδr (Φ, ξ, BN ) = O NM+r (log( M max(N,M ) + log( )) N 2η for RG-FCQ if M > N , while for QCS and K-sparse signals, Eδr (Φ, ξ, ΣK ∩ BN ) = O
M max(N,M ) K+r ) M (log( K
1 + log( 2η )) .
These two results show that, in an almost consistent regime where r = O(1) with respect to M , e.g., if the dynamics of the noise components nj decreases with j, the impact of the noise n on the reconstruction of x is controlled and does not change the asymptotic decay of the worst case reconstruction error. However, in the case of stronger noise where r = O(M ), the worst case reconstruction error can suffer from a systematic bias in O(r/M ). This bias is due to our ignorance of the locations of the corrupted quantized projections in the almost consistent sensing above. The rest of the paper is structured as follows. In Sec. 2, our main results are discussed under the light of related prior works, i.e., with respect to known upper and lower bounds on reconstruction errors for RG-FCQ and in the QCS setting. Sec. 3 provides the proof of our main results above in the context of RG-FCQ, while Sec. 4 focuses on QCS by extending the analysis to the set of K-sparse signals of RN . Sec. 5 studies the proximity of almost consistent vectors. Our last section, Sec. 6, provides finally the proof of a key Lemma sustaining all our developments. This one is connected to a geometric equivalence between quantization of random projections in RN and a variant of Buffon’s needle problem [18, 21]. 4
Conventions: We find useful to present here the rest of the notations used throughout this paper. Domain dimensions are denoted by capital roman letters, e.g., M, N, . . . Vectors and matrices are associated to bold symbols while lowercase light letters are associated to scalar values, e.g., Φ ∈ RM ×N or u ∈ RM . The identity matrix in RD reads 1D×D . The ith component of a vector (or of a vector function) u reads either ui or (u)i , while the vector ui may refer to the ith element of a set of vectors. The set of indices in RD is [D] = {1, · · · , D} and for any S ⊂ [D] of cardinality S = #S, uS ∈ R#S denotes the restriction of u to S. For materializing this last operation, we also introduce the linear restriction operator RS such that RS u = uS , i.e., RS = ((1M ×M )S )T , where B S denotes the matrix obtained by restricting the of Pcolumns p D×D p B ∈R to those indexed in S. For any p > 1, the `p -norm of u is kukp = i |ui | with k·k = k·k2 . The (N − 1)-sphere in RN is SN −1 = {x ∈ RN : kxk = 1} while the unit ball is N denoted BN = {x ∈ RN : kxk 6 1}. More generally, we note BN s (q) = {x ∈ R : kx − qk 6 s}. For asymptotic relations, we use the common Landau family of notations, i.e., the symbols O, Ω and Θ [24]. The positive thresholding function is defined by (λ)+ := 12 (λ + |λ|) for any λ ∈ R.
2
Background and Discussion
In the context of uniform scalar quantization of resolution δ > 0, many works have addressed the model (1) by observing that the distortion induced by quantization compared to a linear model Ax is the one of an additive measurement noise n = Q[Ax] − Ax with ni ∈ [−δ/2, δ/2], i.e., q = Ax + n. (7) When the resolution is small compared to the dynamics of Ax, i.e., under the high resolution assumption [7, 16, 19], or if a random dithering is added to the quantization [16], each component of the noise can be assumed uniformly distributed within [−δ/2, δ/2]. This allows one to assume 2 2 2 n independent √ of Ax and to bound the power of this noise, i.e., Eknk = M δ /12 and knk 6 1 2 12 δ (M + ζ M ) with high probability for ζ = O(1) (see, e.g., [19]). When a noise n of bounded power knk 6 ε corrupts the compressive observation of a sparse signal x as in (7), a worst case reconstruction error in √ kx − x∗ k = O(ε/ M ), can be reached by various reconstruction methods (e.g., Basis Pursuit DeNoise [10, 12] or Iterative Hard Thresholding [2]) as soon as the rescaled sensing matrix √1M A respects the restricted isometry property (RIP) [10]. When the compressive observations of a K-sparse signal undergo uniform scalar quantization, it is√then expected that, with high probability, kx − x∗ k = O(δ) by setting ε2 = 1 2 12 δ (M + ζ M ). The constancy of this error with respect to M is also known as the classical error limit of the pulse code modulation scheme (PCM) in CS [17]. However, most of these reconstruction techniques enforce a `2 -norm fidelity with q, e.g., by imposing kAx∗ − qk 6 ε, and the reconstructed signal is not guaranteed to be consistent with the observations, i.e., Q[Ax∗ ] 6= q. The knowledge of the sensing model is thus not fully exploited for reconstructing x from q. In the context of frame coefficient quantization (FCQ), it is also known that linear reconstruction methods resynthesizing the signal from its frame coefficients corrupted by an additive noise of variance σ 2 have a root mean square error (RMSE) lower bounded by [15, 30] √ 1 (Ekx − x∗ k2 ) 2 > N σ/ M , 5
where the frame is assumed made of unit norm elements and the expectation is taken with respect to noise. A better approach for improving the reconstruction error decay in QCS or in FCQ consists in explicitly enforcing consistency between the estimated signal and the quantized data, as formally described by the formal program (4). Such a procedure was initially introduced in [14] in the context of quantized overcomplete expansion of signals. It was also shown there that, given a random model on the generation of the sensed signal, the RMSE of any reconstruction method is lower bounded by Ω(N/M ). Interestingly, the same lower bound can also be obtained on the worst case error reconstruction without requiring a random generation model on the source [7]. While conjectured for general (M/N )-redundant frames, the combination of a tight frame formed by an oversampled Discrete Fourier Transform (DFT) with a consistent signal reconstruction reaches this lower bound, i.e., in this case the RMSE is upper bounded by O(N/M ) [14]. Consistent reconstruction methods have been applied to QCS in the high resolution regime (i.e., for δ 1 when K ⊂ BN ) [7, 13, 20, 20], for uniform (or bounded) noise for FCQ [30], in the extreme 1-bit CS setting where quantization reduces to the application of a sign operator [5, 23, 28, 29], or even for non-regular quantization scheme in CS [4, 6]. Recently, Powell and Whitehouse in [30] have analyzed a model equivalent to (3) by adopting a geometric standpoint. In particular, adapting their work to our notations, they have studied the sensing model q = Ax + n, where A = (aT1 , · · · , aTM )T ∈ RM ×N is a frame whose elements aj are drawn from a suitable distribution on SN −1 and the uniform noise n ∼ U M ([−δ, δ]) stands for, e.g., a dithered uniform scalar quantization of Ax. They observe that the consistent reconstruction polytope QM := {u ∈ RN : kAu − qk∞ 6 δ} can be seen a translation of an error polytope PN , i.e., for any consistent reconstruction x∗ ∈ QN (x∗ − x) ∈ PM := {u ∈ RN : kAu − nk∞ 6 δ}. Therefore, for a given A, analyzing the worst case error of any consistent reconstruction amounts to estimating the width of PN , i.e., WM = max{kuk : u ∈ PN } = max{ku − xk : u ∈ QN }. Authors in [30] estimate the expected worst case square error E|WM |2 with respect to the distribution of the random vectors {aj : 1 6 j 6 M } on SN −1 . Relating this estimation to coverage processes on the unit sphere [9], they show that, under general assumption on the distribution of these unit frame vectors, 1
(E|WM |2 ) 2 6
Cδ M,
with C > 0 depending on this distribution. In particular, for M frame vectors uniformly drawn at random over SN −1 , C = O(N 3/2 ) so that 1
3/2
(E|WM |2 ) 2 = O( N M δ ).
(8)
Despite a slightly different context where the results above focus on an expected worst case analysis, the behavior of these bounds is highly similar to the one we get in Corollary 1 for 6
consistent reconstruction of signals in the case of RG-FCQ: we observe that, for one random draw 1 N of this (M/N )-redundant RGF and of the quantization dithering, Eδ = O( M (log √MN + log 2η )) with probability higher than 1 − η. At first sight the dependence in N 3/2 of (8) may seem less optimal than the dependence in N of (5). However, the first bound is adjusted to random frame vectors drawn uniformly at random over SN −1 √ [30], i.e., they have all a unit norm while the RGF vectors have an expected length equal to N . Keeping in mind the difficulty to compare a bound on the expectation of a random event with a probabilistic bound on this event itself, we can notice, however, that √ rescaling the result of [30] to uniform random frames over the dilated sphere N SN −1 , or √ 1 conversely rescaling δ into δ/ N in (8), provides an error decay in (E|WM |2 ) 2 = O( NMδ ).
We can notice that the decay in (log M )/M of our bound (5) with respect to M suffers from an extra log factor compared to the decay of (E|WM |2 )1/2 . As will become clear later, this factor actually comes from a union bound argument for upper bounding the probability of failure of our error bounds over all elements of a covering set of K (see Sec. 3).
To conclude this section, as pointed out by the known lower bounds described above, let us mention that regular scalar quantization provides a rather limited decay of the reconstruction error, both for FCQ and QCS contexts. Recent developments in vector quantization for FCQ [27], in the use of feedback quantization and of Σ∆ scheme for FCQ [25] and QCS [7, 17], and finally non-regular quantization schemes where Q is periodic over its range [4, 6], provide all faster reconstruction error bounds decaying polynomially or even exponentially in M . The implicit objective of this paper is therefore to improve our understanding of one of the simplest quantization scheme, basically a dithered round off operation when combined with random Gaussian projections.
3
Quantization of Random Gaussian Frames
This section is dedicated to the proofs of Theorem 1 and of its Corollary 1. Following an argument developed in [4] for non-regular scalar quantization, proving that Eδ (Φ, ξ, BN ) = max max kx − yk x∈BN y∈Cx
6
0
(9)
holds with a probability higher than 1−η on the random draw of a RGF Φ = (ϕ1 , · · · , ϕM )T ∼ N M ×N (0, 1) and of a dithering ξ ∼ U M ([0, δ]), amounts to showing that P[∀x, y ∈ BN , Qδ [Φx + ξ] = Qδ [Φy + ξ] ⇒ kx − yk 6 0 > 1 − η, where P is computed with respect to the random quantities Φ and ξ. Taking the contraposition, we can alternatively demonstrate that, Pfail := P ∃ x, y ∈ BN , kx − yk > 0 s.t. Qδ [Φx + ξ] = Qδ [Φy + ξ] 6 η. For upper bounding Pfail , we take a s-covering of the unit ball BN , i.e., a finite point set Ls ¯ ∈ Ls at most s far apart from v, i.e., kv−¯ such that for any v ∈ BN , there exists a point v v k 6 s. N The cardinality Ls = #Ls of this covering set is known to be bounded as Ls 6 (3/s) [1]. Therefore, if x, y ∈ BN are such that kx − yk > 0 , taking their respective closest points ¯, y ¯ ∈ Ls , we have k¯ ¯ k > 0 − 2s. Consequently, it is easy to show that x x−y ¯ ∈ Ls : k¯ ¯ k > 0 − 2s), p, q p−q Pfail 6 P (∃¯ ∃u ∈ Bs (¯ p), ∃v ∈ Bs (¯ p) : Qδ [Φu + ξ] = Qδ [Φv + ξ] . 7
¯=x ¯, Indeed, if the event whose probability is measured by Pfail is verified for x and y, taking p ¯=y ¯ , u = x and v = y shows that the event associated to the probability of the RHS above q occurs. If one can find an upper bound P0 on ¯ k > 0 − 2s 6 P0 P ∃u ∈ Bs (¯ p), ∃v ∈ Bs (¯ p), Qδ [Φu + ξ] = Qδ [Φv + ξ] k¯ p−q ¯ and q ¯ , since the number of pair of points in Ls is bounded by that is independent of p independently of any conditions on them, an union bound provides
LS 2
< 12 L2s
Pfail 6 12 L2s P0 . The following key Lemma allows one to estimate P0 . ˜, q ˜ be two points in RN . There exists a radius s0 > Lemma 1. Let p Φ∼
N N ×M (0, 1)
and ξ ∼
U M ×1 ([0, δ]),
the probability
√1 k˜ ˜k p−q 8 N
such that, for
Ps0 (α, M ) := P ∃u ∈ Bs0 (˜ p), ∃v ∈ Bs0 (˜ q ), Qδ [Φu + ξ] = Qδ [Φv + ξ] satisfies 1−
Ps0 (α, M ) 6
M 3α , 8 + 4α
(10)
˜ k/δ. with α = k˜ p−q As explained in its proof (see Sec. 6), this Lemma is determined by an equivalence with Buffon’s Needle problem in N dimensions [18], where the needle is actually replaced by a ˜ and q ˜. “dumbbell” shape whose two balls are associated to the two neighborhoods of p The quantity Pλ (α, M ) defined in Lemma 1 increases with λ > 0. Therefore, for finding an estimation of P0 which is associated to the covering radius s, we must guarantee that s 6 ¯ k > 0 − 2s and 2s0 > √1 k¯ ¯ k. This is achieved by imposing s0 , knowing that k¯ p−q p−q 4 N √1 ( 4 N 0
− 2s) = 2s, i.e.,
2s =
This provides also 0 − 2s =
√ 4 N √ 4 N +1 0
√0 . 4 N +1
> 45 0 if N > 2.
Consequently, using (10) and observing that 1 − 3α/(4 + 8α) decays with α, ¯ k > 0 − 2s P ∃u ∈ Bs (¯ p), ∃v ∈ Bs (¯ p), Qδ [Φu + ξ] = Qδ [Φv + ξ] k¯ p−q ¯ k, M | k¯ ¯ k > 0 − 2s = Ps 1δ k¯ p−q p−q ¯ k, M | k¯ ¯ k > 0 − 2s p−q p−q 6 Ps0 1δ k¯ M M 34 0 6 1 − 8δ +5 404 < 1 − 8δ 2 6 exp(− 4δ M+020 ), + 40 5 0
where the conditioning of Ps simply records the extra information without altering its definition. We can then set P0 = exp(− 4δ M+020 ) so that finally Pfail = P Qδ [Φx + ξ] = Qδ [Φy + ξ] kx − yk > 0 6 12 ( 3s )N exp(− 4δ M+020 ) √
N +6 ) − 4δ M+020 ) 0 √ log( 290 N ) − 4δ M+020 ).
=
1 2
exp(N log( 24
6
1 2
exp(N
8
Therefore, if we want Pfail 6 η for some 0 < η < 1, it suffices to impose M>
4δ + 20 0
√ 1 N log( 290 N ) + log 2η ,
which determines the condition invoked in Theorem 1. Knowing that we have necessarily 0 6 2 since x, y ∈ BN , a stronger condition for (9) to occur with the same lower bound on its probability reads M>
4(δ + 1) 0
√ 1 N log( 290 N ) + log 2η .
(11)
Alternatively, saturating this condition, we have 0 =
4(δ + 1) M
√
1 N log( 290 N ) + log 2η
6
4(δ + 1) M
√ ) + log 1 . N log( 25M 2η N
where we used the fact that, from (11), M √ N
>
4 0
√
√
N log( 290 N ) >
4 0
√ 29 ) log( √ N> 2
2 29 5 0
√
N,
since 0 6 2 and assuming N > 2. In other words, assuming δ = O(1), there exists a constant C > 0 such that, 1 1 N log( √MN ) + M log 2η > 1 − η, P Eδ (Φ, ξ, BN ) 6 C M which proves Corollary 1.
4
Extension to K-Sparse Vectors of RN
We study now how the minimal number of measurements evolves in the statement of Theorem 1 when both the original signal and the consistent reconstruction are additionally assumed to be K-sparse in BN ⊂ RN , i.e., they belong to K = ΣK ∩ BN with ΣK := {w ∈ RN : kwk0 6 K}.
Notice first that, given a fixed support T0 ⊂ [N ] with #T0 = 2K, thanks to the developments of Sec. 3, P (∃x, x∗ ∈ BN : kx − x∗ k > 0 , supp x ∪ supp x∗ ⊂ T0 ) : Qδ [Φx + ξ] = Qδ [Φx∗ + ξ] 6
1 2
√
exp(2K log( 29 02K ) −
M 0 4δ + 20 ),
since the subspace of vectors supported in T0 is equivalent to R2K . N eN 2K Since there are no more than 2K 6 ( 2K ) choices of 2K-length supports in [N ], another union bound provides P (∃x, x∗ ∈ BN ∩ ΣK : kx − x∗ k > 0 ) : Qδ [Φx + ξ] = Qδ [Φx∗ + ξ] 6 P (∃T ⊂ [N ] : #T = 2K), (∃x, x∗ ∈ BN : kx − x∗ k > 0 , supp x ∪ supp x∗ ⊂ T ) : Qδ [Φx + ξ] = Qδ [Φx∗ + ξ] √ N 6 12 2K exp(2K log( 29 02K ) − 4δ M+020 ) 6 12 exp(2K log( √29eN ) − 4δ M+020 ). 2K 0
Again, willing to have this last probability smaller than η ∈ (0, 1) leads to imposing 1 M > 4δ +0 20 2K log( √29eN ) + log( ) , 2η 2K 0
9
√ which, by noting that 29e/ 2 < 56, provides the key condition of Theorem 2. Since 0 6 2, a stronger condition reads M>
4(δ + 1) 0
1 ) + log( 2η 2K log( √56N ) , K 0
which gives the crude estimation MN √ K3
>
8N √ 0 K
)> log( √56N K 0
8N √ 0 K
√
log( 560 N ) >
1 √56N 2 K0 ,
using K 6 N and N > 2. Therefore, saturating the condition on M above, 1 √ N ) + log( 1 ) , 0 = 4(δ M+ 1) 2K log( √56N ) + log( 2η ) 6 4(δ M+ 1) 2K log( 2M 3 2η K K
0
which shows that, if δ = O(1), there exists a constant C > 0 for which 1 K MN 1 )+ M P Eδ (Φ, ξ, ΣK ∩ BN ) 6 C M log( √ log( 2η ) > 1 − η. 3 K
This determines the bound stated at the end of the Introduction.
5
Proximity of Almost Consistent Signals
The strict consistency between the quantized projections of two vectors of K ⊂ RN can be slightly relaxed while still keeping their maximal distance bounded. To show this, we follow a similar procedure developed in [22] in the case of 1-bit quantized random projections. We may first observe that if kQδ (Φx + ξ) − Qδ (Φy + ξ)k1 6 r δ for some r ∈ N, at most r measurements differ between Qδ (Φx + ξ) and Qδ (Φy + ξ). There exists thus a subset T of [M ] with size at least M − r such that RT Qδ (Φx + ξ) = RT Qδ (Φy + ξ), with the corresponding restriction operator RT defined in the Introduction.
Therefore, for K ⊂ RN and writing [M ]r the set of all subsets of [M ] of size M − r, a union bound provides Pr := P ∃ x, y ∈ K : kx − yk > 0 s.t. kQδ (Φx + ξ) − Qδ (Φy + ξ)k1 6 r δ 6 P ∃T ⊂ [M ]r , ∃ x, y ∈ K : kx − yk > 0 s.t. RT Qδ (Φx + ξ) = RT Qδ (Φy + ξ) [ 6 P ∃ x, y ∈ K : kx − yk > 0 s.t. Qδ (RTc Φx + ξ T ) = Qδ (RT Φy + ξ T ) . T ⊂[M ]r
We have now to split the analysis between the RG-FCQ and the QCS cases. RG-FCQ case:
If K = BN with M > N , using Sec. 3 and and Pr 6
1 2
6
1 2
M M −r
M M −r
=
M r
6 (eM/r)r ,
√
(M −r)0 2δ + 40 ) √ (M −r)0 log( 290 N ) − 4δ + 20 ).
exp(N log( 290 N ) −
exp(r log( eM r ) + N
Willing to have this last probability smaller than η ∈ (0, 1), we find that, as soon as √ 29 N 1 M > r + 4δ +0 20 r log( eM r ) + N log( 0 ) + log( 2η ) , and given Φ ∼ N M ×N (0, 1) and ξ ∼ U M ([0, δ]), the event ∀x, y ∈ K,
kQδ (Φx + ξ) − Qδ (Φy + ξ)k1 6 r δ 10
⇒
kx − yk 6 0 ,
holds with probability higher than 1 − η.
Aiming a slightly stronger condition on M by a series of crude upper bounds, we observe that √ 1 29 N ) + N log( r + 4δ +0 20 r log( eM r 0 ) + log( 2η ) √ 1 6 4δ +0 20 32 r log(eM ) + 32 N log( 290 N ) + log( 2η ) , √ 1) 3 3 1 2eM 29 N 6 4(δ + 2 r log( 0 ) + 2 N log( 0 ) + log( 2η ) , 0 √ (δ + 1) 3 3 1 2eM N 29 N N ) + ) + log( = 4N r log( N log( ) , 0 0 0 2 2 2η (N +r)0 (N +r)0 (N +r)0 (δ + 1) 3 3 1 2eM 29N 6 4N 2 r log( 0 ) + 2 N log( 0 ) + log( 2η ) , (N +r)0 0
0
6
4N (δ + 1) 3 (r (N +r)00 2
=
6(δ + 1) N 00
0
+ N ) log( 29 max(N,M/5) ) + 0
4(δ + 1) 00
0
log( 29 max(N,M/5) ) + 0 0
4(δ + 1) 00
1 log( 2η ),
1 log( 2η ),
using the variable change 0 = NN+r 00 , 0 6 2 and 2e/29 < 1/5. Notice that in the case where r = 0, remembering that the term r log(eM/r) above comes from a bound on log M r , we can assume r log(eM/r) = 0, and since r ∈ N, we can write r log(eM/r) 6 r log(eM ).
Therefore, reexpressing everything in function of 00 and forgetting the prime symbol, we find that, as soon as M>
6(δ + 1) N 0
log( 29 max(N,M/5) ) + 0
4(δ + 1) 0
1 log( 2η ),
and given a random draw of Φ ∼ N M ×N (0, 1) and ξ ∼ U M ([0, δ]), the event ∀x, y ∈ K,
kQδ (Φx + ξ) − Qδ (Φy + ξ)k1 6 r δ
⇒
kx − yk 6
N +r N 0 ,
(12)
holds with probability higher than 1 − η.
Saturating the condition on M above and since N > 2 > 0 , we find M>
6(δ + 1) N 0
log( 29 max(N,M/5) )> 0
6 0 N
log(29) >
2 29N 3 0 ,
and 0 =
0 such that ) 1 P Eδr (Φ, ξ, BN ) 6 C NM+r log( M max(N,M ) + log( 2η ) > 1 − η. N QCS case:
If K = ΣK ∩ BN , using the proof of Sec. 4, we have similarly (M −r)0 Pr 6 12 MM−r exp(2K log( √29eN ) − 4δ + 20 ) 2K 0
6
1 2
exp(r log( eM r )
+ 2K log( √29eN ) − 2K 0
(M −r)0 4δ + 20 ).
Again, willing to have this last probability smaller than η ∈ (0, 1), we find that, as soon as 1 √56N M > r + 4δ +0 20 r log( eM r ) + 2K log( K ) + log( 2η ) , 0
11
and given Φ ∼ N M ×N (0, 1) and ξ ∼ U M ([0, δ]), the event ∀x, y ∈ K,
kQδ (Φx + ξ) − Qδ (Φy + ξ)k1 6 r δ
⇒
kx − yk 6 0 ,
holds with probability higher than 1 − η.
Targeting a slightly stronger condition on M by a series of crude upper bounds, we observe that 4δ + 20 1 √56N r(1 + 4δ +0 20 ) log( eM 0 r ) + 2K log( K0 ) + log( 2η ) , 1 ) + log( ) , 6 4δ +0 20 2r log(eM ) + 2K log( √56N 2η K0 1) 1 √56N 6 4(δ + 2r log( 2eM 0 ) + 2K log( K0 ) + log( 2η ) , 0 √ + 1) 2eM K 56N K 1 = 4K(δ 2r log( (K+r) 0 ) + 2K log( (K+r)0 ) + log( 2η ) , (K+r)00 0 0 4K(δ + 1) 2eM 56N 1 ) , 6 (K+r)0 2r log( 0 ) + 2K log( 0 ) + log( 2η 0
6 =
0
4K(δ + 1) 2(r (K+r)00 8(δ + 1) K 00
0
+ K) log( 56 max(N,M/10) )+ 0 0
log( 56 max(N,M/10) )+ 0 0
4(δ + 1) 00
4(δ + 1) 00
1 log( 2η ),
1 log( 2η ),
0 using the variable change 0 = K+r K 0 , 0 6 2 and 2e/56 < 1/10, and with the same remark on the vanishing value of r log(eM/r) when r = 0.
Therefore, reexpressing everything in function of 00 and forgetting the prime symbol, we find that, as soon as M>
8(δ + 1) K 0
log( 56 max(N,M/10) )+ 0
4(δ + 1) 0
1 log( 2η ),
and given a random draw of Φ ∼ N M ×N (0, 1) and ξ ∼ U M ([0, δ]), the event ∀x, y ∈ K,
kQδ (Φx + ξ) − Qδ (Φy + ξ)k1 6 r δ
⇒
kx − yk 6
K+r K 0 ,
(13)
holds with probability higher than 1 − η.
Saturating the condition on M above, since for this M M>
8(δ + 1) K 0
log( 56 max(N,M/10) )> 0
8 0 K
log(56) >
1 56 2 0 K,
we find 0 =
0 such that ) 1 P Eδr (Φ, ξ, ΣK ∩ BN ) 6 C K+r log( M max(N,M ) + log( 2η )) > 1 − η. M K
6
Proof of Lemma 1
˜, q ˜ ∈ RN , there exists a Let us recall the context. We want to show that, given two points p 1 0 N ×M M ×1 ˜ k such that, for Φ ∼ N radius s > 8√N k˜ p−q (0, 1) and ξ ∼ U ([0, δ]), the probability Ps0 (α, M ) := P ∃u ∈ Bs0 (˜ p), ∃v ∈ Bs0 (˜ q ), Qδ [Φu + ξ] = Qδ [Φv + ξ] 12
satisfies Ps0 (α, M ) 6
1−
M 3α , 8 + 4α
˜ k/δ. with α = k˜ p−q
Notice first that we can focus on upper bounding the probability associated to a single projection by the random vector ϕ ∼ N N ×1 (0, 1) quantized with Qδ with a scalar dithering ξ ∼ U([0, δ]), the result for M dithered quantized projections simply following by raising the single measurement bound to the power M , i.e., Ps0 (α, M ) 6 (Ps0 (α, 1))M . ˆ where ϕ ˆ ∈ SN −1 is uniformly distributed at random over SN −1 and the We write ϕ = φ ϕ, length φ = kϕk ∼ χ(N ) follows a χ distribution with N degrees of freedom. We are going first to estimate the following conditional probability: Ps0 (α, 1|φ) := P ∃u ∈ Bs0 (˜ p), ∃v ∈ Bs0 (˜ q ), Qδ [ϕT u + ξ] = Qδ [ϕT v + ξ] kϕk = φ ˆ T u + ξ] = Qδ [φϕ ˆ T v + ξ] kϕk = φ = P ∃u ∈ Bs0 (˜ p), ∃v ∈ Bs0 (˜ q ), Qδ [φϕ ˆ T u + ξ] = Qδ [ϕ ˆ T v + ξ] kϕk = φ , (14) = P ∃u ∈ Br (p), ∃v ∈ Br (q), Qδ [ϕ ˆ = ϕ. Notice that 2r/kp − qk = with the variable change r = φs0 , p = φ˜ p and q = φ˜ q and φϕ ˜ k. Let us focus on this last probability keeping in mind the relationships between 2s0 /k˜ p−q these parameters for estimating later a result which is not conditioned to the knowledge of φ.
We follow the procedure described in [21]. In this work, from a generalization of the Buffon’s needle problem [8, 18] in N dimensions, it is shown that when r = 0, i.e., u = p and v = q, computing Ps0 (α, 1|φ) above is equivalent to estimating the probability that a segment (or needle) of length L = kp − qk uniformly “thrown” at random in RN , both spatially and in orientation, does not intersect a fixed set of parallel (N − 1)-dimensional hyperplanes spaced by a distance δ. ˆ ∈ SN −1 and ξ ∈ [0, δ], the function f (v) := Qδ (ϕ ˆ T v + ξ) is piecewise More precisely, given ϕ N constant in R and the frontiers where its value changes correspond to a set of parallel (N − 1)dimensional hyperplanes in RN . These hyperplanes are equi-spaced with a separating distance ˆ Consequently, the quantity X := 1δ Qδ (ϕ ˆTp + δ and they are all normal to the direction ϕ. ˆ T q + ξ) ∈ Z counts the number of such hyperplanes intersecting the segment pq. In ξ) − Qδ (ϕ this scenario, this segment is thus fixed and the hyperplanes are randomly oriented and shifted ˆ and ξ, respectively. by ϕ However, we can reverse the point of view and rather consider those hyperplanes as fixed and normal, e.g., to the first canonical axis e1 of RN . This is allowed by considering the affine mapping Aϕ,ξ : RN → RN implicitly defined by any combination of a rotation and of ˆ ˆ T v + ξ for all v ∈ RN . In words, thanks to a translation in RN such that eT1 Aϕ,ξ ˆ (v) = ϕ N ˆ and shifting the result by ξ is Aϕ,ξ ˆ , projecting a point v ∈ R onto the random orientation ϕ equivalent to projecting the random point Aϕ,ξ ˆ (v) onto e1 .
0 Therefore, denoting p0 = Aϕ,ξ ˆ (q), it is easy to see that the L-length ˆ (p) and q = Aϕ,ξ 0 0 segment p q , i.e., our needle, is then oriented uniformly at random over SN −1 while the distance of its centrum 12 (p0 + q 0 ) to the closest hyperplane follows a uniform random variable over the interval [0, δ/2]. Moreover, we have X = 1δ Qδ (eT1 p0 ) − Qδ (eT1 q 0 ) ,
so that X actuallySmeasures the number of intersections the segment p0 q 0 makes with the set of hyperplanes Gδ = k∈Z {x : eT1 x = k}. In [21], the distribution of the discrete bounded random variable X is actually fully determined and denoted Buffon(L/δ, N ).
13
G
r w ✓
r
L
Figure 1: A Buffon “dumbbell” problem in 2-D.
For r > 0, Eq.(14) shows that we must now consider the two neighboring `2 -balls of p and q in Ps0 (α, 1|φ) and estimate the probability that at least two points of these balls share the same ˆ Following the same argument as above, this new problem dithered quantized projection onto ϕ. is now equivalent to a new Buffon experiment if the previous needle is ended with two balls. In other words, we create a dumbbell shape formed by a segment of length L on the extremities of which two balls of radius r are centered (see Fig. 1). It is then easy to see that Ps0 (α, 1|φ) is equivalent to the probability that there is no hyperplane of Gδ intersecting only the part of the segment outside of the two balls when the dumbbell is thrown randomly in RN as for previous Buffon’s needle. Otherwise, having such an intersection would mean that no pair of points (taken in distinct balls) lie in the same subvolume delimited by two consecutive hyperplanes, i.e., they do not have the same quantized projection, and conversely. Let us parametrize this dumbbell by its distance w ∼ U([0, δ/2]) (estimated from the middle of the segment) to the closest hyperplane Gδ and by its orientation drawn uniformly at random in SN −1 . By symmetry, only the angle θ ∈ [0, π] made by the dumbbell with the normal vector e1 to Gδ is important in this parametrization [21]. Moreover, from Fig. 1, the absence of intersection amounts to imposing w > 12 L| cos θ| − r. The probability Ps0 (α, 1|φ) is thus obtained by Z π Z δ/2 N −2 Ps0 (α, 1|φ) = κN (sin θ) dθ I(w > L2 | cos θ| − r) 2δ dw, 0
=
4κN δ
0
Z
π/2
(sin θ)
N −2
Z dθ
0
δ/2
I(w > 0
L 2
cos θ − r) dw,
where κN (sin θ)N −2 dθ is the area (normalized to the one of SN −1 ) of the thin spherical segment ˆ ) ∈ [θ, θ + dθ]}, where κN := Sdθ (θ) := {ˆ v ∈ SN −1 : arccos(eT1 v B(k, l) = Γ(k)Γ(l)/Γ(k + l) is the Beta function.
√
Γ( N ) 2 π Γ( N 2−1 )
= B( 12 , N 2−1 )−1 and
It is important to remark that, from [21, 31], √ √2 π
1
(N + 1)− 2 6
2κN N −1
6
√ √2 π
1
(N − 1)− 2 ,
so that, for N > 2, √1 2π
1
(N + 1) 2 − 1 < κN 6
√1 2π
14
1
(N − 1) 2
q N ⇒ κN = Θ( 2π ).
(15)
Let us define two angles 0 6 θ0 6 θ1 6 π/2 such that cos θ0 = min( δ+2r L , 1) and cos θ1 = 2r 0 L , assuming 2r 6 L (otherwise, Ps = 1). The angular integration domain can be split in three intervals: [0, θ0 ], [θ0 , θ1 ] and [θ1 , π/2]. Over the first interval, the integral is always zero since, either we have a zero measure interval (θ0 = 0) or I(w > L2 cos θ − r) = 0 since L L 2 cos θ > 2 cos θ0 = δ/2 + r and 0 6 w 6 δ/2. Moreover, over the last interval [θ1 , π/2], L I(w > 2 cos θ − r) = 1 Therefore, writing a = L/δ, Z θ1 4κN (sin θ)N −2 ( 2δ − Ps0 (α, 1|φ) = δ θ0
=1+ =1−
4κN δ 4κN δ
Z
θ0 θ1
Z
4κN δ
=1 −
+
θ1
(sin θ)N −2 ( 2δ
L 2
Z
−
L 2
cos θ + r) dθ − 2κN
θ0
Z
θ1
4κN δ
= 1 − 2κN a
Z
0
4κN δ
θ0
Z 0
Z
θ1
(sin θ)N −2 dθ
0
(sin θ)N −2 2δ dθ
(sin θ)N −2 ( L2 cos θ − r) dθ θ0
0
Z
(sin θ)N −2 dθ
θ1
(sin θ)N −2 ( L2 cos θ − r) dθ −
0
π/2
cos θ + r) dθ + 2κN
(sin θ)N −2 ( L2 cos θ − (r + 2δ )) dθ
1
(1 − v 2 )
N −3 2
(v −
2r L )+
− (v −
2r+δ L )+
dv,
applying a variable change v = cos θ on the last line. Let us study this R vlast integral and the function f (v) = (v − verify that F (v) := 0 f (v 0 )dv 0 is convex and reads 0, 2 2r+δ 2 2 2F (v) = (v − 2r ) − (v − ) = (v − 2r + L + L L) , δ 4r+δ L (2v − L ),
2r L )+
− (v −
2r+δ L )+ .
We can
if v 6 2r L, 2r if L < v 6 2r+δ L , 2r+δ if v > L .
Moreover, by integrating by part, Z 1 Z 1 N −5 2 N 2−3 (1 − v ) f (v) dv = (N − 3) v (1 − v 2 ) 2 F (v) dv, 0
0
N −5
The positive measure µ(v) = (N −3) v (1−v 2 ) 2 has unit mass over [0, 1] so that, by convexity of F and using the Jensen inequality, Z 1 R1 F (v) µ(v) dv > F 0 v µ(v) dv . 0
However, since (N − 3)
R1 0
(1 −
N −5 v2) 2
vq
dv =
N −3 2
N −3 B( q+1 2 , 2 )
we find R1 0
v µ(v) dv = (N − 3)
R1
2 0 (1 − v )
N −5 2
=
√
v 2 dv =
q+1 N −1 Γ( 2 ) 2 N +q−2 , Γ 2
π Γ( N 2−1 ) 2 Γ( N ) 2
and Ps0 (α, 1|φ) 6 1 − 2κN a F ( 2κ1N ). 15
Γ
=
1 2κN ,
(16)
(17)
From the definition of F above, if 2κ1N 6 2r L , F = 0 and we cannot show anything. Let us λ thus set 2r = 2κN L, where λ ∈ (0, 1) will be determined later. Notice that, since s0 = φ r and ˜ k = φ L, we implicitly impose 2r/L = 2s0 /k˜ ˜ k = 2κλN . k˜ p−q p−q Then,
( 2F ( 2κ1N ) = 2κN 1−λ ,
so that, writing φ0 =
1 (1 − λ)2 , 4κ2N 1 1 1 a ( κN (1 − λ) − a ),
if a 6 if a >
2κN 1−λ , 2κN 1−λ ,
we have
Ps0 (α, 1|φ) 6
( 1−
λ+
a 2φ0 (1 − λ), φ0 2a (1 − λ),
if a 6 φ0 , if a > φ0 .
Let us recall that Ps0 (α, 1|φ) is defined conditionally to φ = kϕk with φ ∼ χ(N ). Moreover, ˜ k/δ = αφ with α = k˜ ˜ k/δ. Denoting the pdf of χ(N ) by γN (φ) = a = kp − qk/δ = φk˜ p−q p−q R +∞ φ2 1− N N N −1 2 cN φ exp(− 2 ) and cN = 2 /Γ( 2 ), we can develop Ps0 (α, 1) = 0 Ps0 (α, 1|φ) γN (φ) dφ as follows Z φ0 /α Z +∞ αφ φ0 (1 − 2φ0 (1 − λ)) γN (φ) dφ + Ps0 (α, 1) 6 (λ + 2αφ (1 − λ)) γN (φ) dφ 0
φ0 /α
= λ + (1 − λ) = λ + (1 − λ)
Z
φ0 /α
(1 −
0
Z
+∞
0
αφ 2φ0 ) γN (φ) dφ
+ (1 − λ)
Z
+∞
φ0 /α
φ0 2αφ γN (φ) dφ
ϕ( αφ φ0 ) γN (φ)dφ,
with ϕ(t) = 1 − 12 t if 0 6 t < 1 and ϕ(t) =
1 2t
if t > 1.
We can notice that tϕ(t), which is equal to 12 t(2 − t) over [0, 1] and to concave function. Therefore, by Jensen inequality, Z +∞ Z +∞ αφ cN ϕ( φ0 ) γN (φ)dφ = cN −1 φ ϕ( αφ φ0 ) γN −1 (φ)dφ 0
1 2
for t > 1, is a
0
6
cN
cN −1 (EγN −1 φ) ϕ(
αEγN −1 φ ) φ0
√ √ √ We have also cN /cN −1 = Γ( N 2−1 )/( 2Γ( N2 )) and EγN −1 φ = 2Γ( N2 )/Γ( N 2−1 ) = 2πκN , so that cNcN−1 (EγN −1 φ) = 1 and αEγN −1 φ φ0
Consequently, since ϕ(t) 6
1−λ = α 2κ EγN −1 φ = N
6 λ + (1 − λ)
q
2+
2 π
2
π 2 (1
√ π2
2+
√π (1−λ)2 α 2 =1− √ π
q
π 2 (1
− λ)α.
2 2+t ,
Ps0 (α, 1) 6 λ + (1 − λ)ϕ
taking (1 − λ) =
q
2
(1−λ)α
> 3/4. 16
− λ)α
(1−λ)α q
= 1−
2 α π
2+α
< 1−
3α 8+4α
Moreover, from the bounds on κN given in (15), this shows also that q q 1 2s0 2r λ 2 π √1 = = > (1 − L 2κN π) 2 (N −1)1/2 > 4 N , k˜ p−˜ qk as stated at the beginning of Lemma 1.
Acknowledgements Laurent Jacques is a Research Associate funded by the Belgian F.R.S.-FNRS.
References [1] R. Baraniuk, M. Davenport, R. DeVore, and M. Wakin. A simple proof of the restricted isometry property for random matrices. Constructive Approximation, 28(3):253–263, 2008. [2] T. Blumensath and M. Davies. Iterative hard thresholding for compressive sensing. Appl. Comput. Harmon. Anal., 27(3):265–274, 2009. [3] P. T. Boufounos. Quantization and Erasures in Frame Representations. D.Sc. Thesis, MIT EECS, Cambridge, MA, January 2006. [4] P. T. Boufounos. Universal rate-efficient scalar quantization. IEEE Trans. Info. Theory, 58(3):1861–1872, March 2012. [5] P. T. Boufounos and R. G. Baraniuk. 1-bit compressive sensing. In Proc. Conf. Inform. Science and Systems (CISS), Princeton, NJ, March 19-21 2008. [6] P. T. Boufounos and S. Rane. Efficient coding of signal distances using universal quantized embeddings. In Proc. Data Compression Conference (DCC), Snowbird, UT, March 20-22 2013. [7] P. T. Boufounos, L. Jacques, F. Krahmer, and R. Saab. Quantization and compressive sensing. In book ”Compressed Sensing and its Applications”, Springer, 2014. [8] G. C. Buffon. Essai d’arithm´etique morale, volume 4 of Suppl´ement a ` lhistoire naturelle, 1777. http://www.buffon.cnrs.fr. [9] P. B¨ urgisser, F. Cucker, M. Lotz, et al. Coverage processes on spheres and condition numbers for linear programming. The Annals of Probability, 38(2):570–604, 2010. [10] E. Cand`es, J. Romberg, and T. Tao. Stable signal recovery from incomplete and inaccurate measurements. Comm. Pure Appl. Math, 59(8):1207–1223, 2006. [11] E. J. Candes and T. Tao. Near-optimal signal recovery from random projections: Universal encoding strategies? Information Theory, IEEE Transactions on, 52(12):5406–5425, 2006. [12] S. S. Chen, D. L. Donoho, and M. A. Saunders. Atomic Decomposition by Basis Pursuit. SIAM Journal on Scientific Computing, 20(1):33–61, 1998. [13] W. Dai, H. V. Pham, and O. Milenkovic. Distortion-Rate Functions for Quantized Compressive Sensing. Technical Report arXiv:0901.0749, 2009. [14] V. K Goyal, M. Vetterli, and N. T. Thao. Quantized overcomplete expansions in RN : Analysis, synthesis, and algorithms. IEEE Trans. Info. Theory, 44(1):16–31, 1998. [15] V. K. Goyal, J. Kovaˇcevi´c, and J. A. Kelner. Quantized frame expansions with erasures. Applied and Computational Harmonic Analysis, 10(3):203–233, 2001. [16] R.M. Gray and D.L. Neuhoff. Quantization. IEEE Trans. Info. Theory, 44(6):2325–2383, Oct 1998.
17
¨ Yılmaz. Sobolev duals for random frames and σδ [17] C. S. G¨ unt¨ urk, M. Lammers, A. M. Powell, R. Saab, and O. quantization of compressed sensing measurements. Foundations of Computational Mathematics, 13(1):1–36, 2013. [18] J. D. Hey, T. M. Neugebauer, and C. M. Pasca. Georges-Louis Leclerc de Buffons Essays on Moral Arithmetic. In The Selten School of Behavioral Economics, pages 245–282. Springer, 2010. [19] L. Jacques, D. K. Hammond, and M. J. Fadili. Dequantizing Compressed Sensing: When Oversampling and Non-Gaussian Constraints Combine. IEEE Trans. Inf. Theory, 57(1):559–571, January 2011. [20] L. Jacques, D. K. Hammond, and M. J. Fadili. Stabilizing Nonuniformly Quantized Compressed Sensing With Scalar Companders. IEEE Trans. Inf. Theory, 59(12):7969 – 7984, January 2013. [21] L. Jacques. A Quantized Johnson Lindenstrauss Lemma: The Finding of Buffon’s Needle. arXiv preprint arXiv:1309.1507, 2013. [22] L. Jacques, K. Degraux, and C. De Vleeschouwer. Quantized iterative hard thresholding: Bridging 1-bit and high-resolution quantized compressed sensing. In 10th international conference on Sampling Theory and Applications (SampTA 2013), arXiv:1305.1786, pages 105–108, Bremen, Germany, 2013. [23] L. Jacques, J. N. Laska, P. T. Boufounos, and R. G. Baraniuk. Robust 1-bit compressive sensing via binary stable embeddings of sparse vectors. IEEE Transactions on Information Theory, 59(4):2082–2102, 2013. [24] D. E. Knuth. Big omicron and big omega and big theta. ACM Sigact News, 8(2):18–24, 1976. [25] F. Krahmer, R. Saab, and R. Ward. Root-exponential accuracy for coarse quantization of finite frame expansions. IEEE Trans. Inform. Theory, 58(2):1069–1079, 2012. [26] B. K. Natarajan. Sparse approximate solutions to linear systems. SIAM journal on computing, 24(2):227– 234, 1995. [27] H. Q. Nguyen, V.K. Goyal, and L.R. Varshney. Frame Permutation Quantization. Applied and Computational Harmonic Analysis (ACHA), Nov. 2010. [28] Y. Plan and R. Vershynin. arXiv:1111.4452, 2011.
Dimension reduction by random hyperplane tessellations.
arXiv preprint
[29] Y. Plan and R. Vershynin. One-bit compressed sensing by linear programming. Communications on Pure and Applied Mathematics, 66(8):1275–1297, 2013. [30] A. M. Powell and J. T. Whitehouse. Error bounds for consistent reconstruction: Random polytopes and coverage processes. arXiv preprint arXiv:1405.7094, 2014. [31] F. Qi, Q.-M. Luo, et al. Bounds for the ratio of two gamma functionsfrom wendels and related inequalities to logarithmically completely monotonic functions. Banach J. Math. Anal, 6(2):132–158, 2012. [32] J. A. Tropp. Just relax: Convex programming methods for identifying sparse signals in noise. Information Theory, IEEE Transactions on, 52(3):1030–1051, 2006.
18
arXiv:1406.0022v1 [cs.IT] 30 May 2014
Abstract This paper provides new error bounds on consistent reconstruction methods for signals observed from quantized random sensing. Those signal estimation techniques guarantee a perfect matching between the available quantized data and a reobservation of the estimated signal under the same sensing model. Focusing on dithered uniform scalar quantization of resolution δ > 0, we prove first that, given a random Gaussian frame of RN with M vectors, the worst case `2 -error of consistent signal reconstruction decays with high probability as N O( M log √MN ) uniformly for all signals of the unit ball BN ⊂ RN . Up to a log factor, this matches a known lower bound in Ω(N/M ). Equivalently, with a minimal number of frame √ N N coefficients behaving like M = O( 0 log( 0 )), any vectors in BN with M identical quantized projections are at most 0 apart with high probability. Second, in the context of Quantized Compressed Sensing with M random Gaussian measurements and under the same scalar quantization scheme, consistent reconstructions of K-sparse signals of RN have a worst case K MN error that decreases with high probability as O( M log √ 3 ) uniformly for all such signals. K Finally, we show that the strict consistency condition can be slightly relaxed, e.g., allowing for a bounded level of error in the quantization process, while still guaranteeing a proximity between the original and the estimated signal. In particular, if this quantization error is of order O(1) with respect to M , similar worst case error decays are reached for reconstruction methods adjusted to such an approximate consistency.
1
Introduction
Since the advent of the digital signal processing era and of analog to digital converters an intense field of research has been concerned by the following non-linear sensing model q = Q[Ax] ∈ J ,
(1)
where A ∈ RM ×N is a matrix representing a linear transformation (or sensing) of a signal x taken in some bounded subset K of RN and Q stands for a quantization of Ax mapping AK ⊂ RM to a finite set of vectors J ⊂ RM , e.g., using a given number of bits [7, 16].
Since the bounded space K contains a priori an infinite number of signals, the model (1) is of course lossy and x cannot be recovered exactly from q. Quantifying this loss of information in function of both the signal reconstruction method and of the key elements A, N , M , K and Q has therefore been the source of numerous research at the frontier of information theory, high dimensional geometry, signal processing and statistics. ∗
LJ is with the ICTEAM institute, ELEN Department, Universit´e catholique de Louvain (UCL), Belgium.
1
The general model (1) is for instance the one adopted in Quantized Compressed Sensing (QCS) [7, 11, 19] when the signal x is assumed sparse or compressible in an orthonormal basis Ψ of RN and when the sensing matrix is generated randomly, e.g., from random Gaussian ensembles [10]. When M > N , Eq. (1) is also a model for frame coefficient quantization (FCQ) of signals in RN , i.e., when the coefficients Ax = (aT1 x, · · · , aTM x)T of x in a overcomplete frame of RN are quantized in q = Q(Ax) ∈ J , A = (a1 , · · · , aM )T representing then the matrix whose row set {aj ∈ RN : 1 6 j 6 M } collects the frame vectors [3, 14, 15].
In this work, we restrict the analysis of (1) to a scalar, regular and uniform quantizer, i.e., when Q is a scalar operation applied componentwise on Ax ∈ RM , and when the 1-D quantization cells defined by the set function Q−1 [c] = {λ : Q[λ] = c} ⊂ R are convex and have all the same size. We refer the reader for instance to [7] for a review of scalar and Σ∆quantization [17] in the QCS literature, to [4] for a theoretical analysis of non-regular scalar quantizers, or to [27] for an example of vector quantization by frame permutation. In particular, we adopt a uniform midrise quantizer Qδ (λ) := δ(b λδ c + 21 ) ∈ δ(Z + 21 ) =: Zδ
(2)
of resolution δ > 0 applied componentwise on vectors. Moreover, we focus on the interplay of Qδ with both a random Gaussian matrix A = Φ ∼ N (0, 1)M ×N and a dithering ξ ∼ U M ([0, δ]), where DM ×N (η) and DM (η) denote a M × N random matrix or a M -length random vector, respectively, whose entries are identically and independently distributed as the probability distribution D(η) of parameters η = (η1 , · · · , ηP ), e.g., the standard normal distribution N (0, 1) or the uniform distribution U([0, δ]). Such a dithering is often used for improving the statistical properties of the quantizer by randomizing the unquantized input location inside the quantization cell [16]. As will become clear later (see Sec. 6), this uniform dithering allows us also to bridge our analysis with a geometrical probability context inspired by Buffon’s needle problem [8, 21]. Consequently, given a signal x in a bounded set K ⊂ RN , the quantized sensing scenario studied in this paper reads (3) q = Qδ [Φx + ξ] ∈ ZM δ . This is either a sensing model for QCS with a random Gaussian sensing Φ, or a quantization scheme for FCQ when the overcomplete frame is made of M vectors in RN (with M > N ) that are randomly and independently drawn from N (0, 1N ×N ). This guarantees that they are also linearly independent with probability 1, i.e., we obtain a random Gaussian frame (RGF) of RN . Our main objective in order to quantify the information loss in (3) while trying to estimate x is to characterize the worst case error Eδ (Φ, ξ, K) := max kx − x∗ k x∈K
of any consistent reconstruction method whose output x∗ is determined by the following formal program: find any x∗ ∈ RN such that Qδ (Φx∗ + ξ) = Qδ (Φx + ξ) and x∗ ∈ K.
(4)
In the case of FCQ of signals in a RGF (i.e., M > N ), we set K = BN , while in the context of QCS we take K = ΣK (Ψ) ∩ BN where ΣK (Ψ) = {v = Ψα ∈ RN : kαk0 6 K}, with kαk0 := #{j ∈ [N ] : αj 6= 0}, is the space of K-sparse signals in the orthonormal basis Ψ ∈ RN ×N . For the sake of simplicity, we work with the canonical basis Ψ = 1N ×N . However, all our results can be applied to Ψ 6= 1N ×N from the rotational invariance of the random Gaussian matrix Φ ∼ N M ×N (0, 1) in RN [11]. 2
We acknowledge the fact that for K = ΣK ∩ BN the program (4) is possibly NP hard, e.g., if x∗ is found by minimizing the `0 “norm” under the consistency constraint [26]. However, similarly to the procedure developed in [23], we are anyway interested in studying its reconstruction error, remembering that similar ideal reconstructions in CS and in QCS have often driven the determination of feasible programs [11, 13, 19, 20, 23, 28, 32]. Notice that the error Eδ is also associated to the biggest size, with respect to all x ∈ K, of all consistency cells Cx := {x∗ ∈ K : Qδ (Φx∗ + ξ) = Qδ (Φx + ξ)}, i.e., Eδ (Φ, ξ, K) = max max kx − x∗ k, ∗ x∈K x ∈Cx
which shows that the characterization of Eδ is actually a high dimensional geometric problem, whose general formulation can be connected to the problem of finding a finite covering of K of minimal size [7]. Our first contribution is an upper bound on this worst case error for consistent signal reconstruction in the context of Random Gaussian Frame Coefficient Quantization (RG-FCQ). Theorem 1. Let us fix 0 > 0, 0 < η < 1, δ > 0 and M > N such that M>
4δ + 20 0
√ 1 N log( 290 N ) + log 2η .
Let us randomly draw a RGF Φ ∼ N M ×N (0, 1) and a dithering ξ ∼ U M ([0, δ]). Then, with a probability higher than 1 − η, for all x ∈ K = BN sensed by (3), any solution x∗ to (4) is such that kx − x∗ k 6 0 , or, equivalently, Eδ (Φ, ξ, BN ) 6 0 . As a corollary, the asymptotic decay of Eδ in function of M , N , δ and of the probability η can be established. Corollary 1. Given M > 0, 0 < η < 1, δ > 0 with δ = O(1), there exists a constant C > 0 such that N P Eδ (Φ, ξ, BN ) 6 C M log √MN + log η2 > 1 − η, where the probability is computed with respected to both the RGF Φ ∼ N M ×N (0, 1) and the random dithering ξ ∼ U M ([0, δ]). Loosely speaking, this corollary says that if δ = O(1), with a probability higher than 1 − η, N Eδ (Φ, ξ, BN ) = O( M log √MN + log η2 ).
(5)
As shown in Sec. 4, it is then straightforward to generalize Theorem 1 to the set of sparse signals. Theorem 2. Let us fix 0 > 0, 0 < η < 1, δ > 0 and M such that 1 M > 4δ +0 20 2K log( √56N ) + log( 2η ) . K 0
Let us randomly draw a Gaussian sensing matrix Φ ∼ N M ×N (0, 1) and a dithering ξ ∼ U M ([0, δ]). Then, with a probability higher than 1 − η, for all x ∈ K = ΣK ∩ B N sensed by (3), any solution x∗ to (4) is such that kx − x∗ k 6 0 , or, equivalently, Eδ (Φ, ξ, ΣK ∩ BN ) 6 0 .
3
In such a case, as for Corollary 1 in the context of RGF, Sec. 4 also shows that, for any M > 0, K MN Eδ (Φ, ξ, ΣK ∩ BN ) = O( M log( √ ) + log η2 ), 3 K
with a probability higher than 1 − η.
As explained in Sec. 2, this matches existing error bounds for 1-bit compressed sensing in the case√ of Gaussian random projections [23]. It also improves previousp known bounds decaying as O(1/ M ) for linear reconstruction methods in FCQ [14] and as O( K/M ) for QCS, while a known lower bound in Ω(K/M ) exists [7]. Our result behaves also similarly to the bound on the mean worst case error (mean established with respect to Φ and ξ in the context of our notations) of consistent reconstruction methods obtained in [30] in the case of random frames over SN −1 . As a last contribution, we show that slight deviations to strict consistency is also possible while keeping similar proximity relations between almost consistent vectors. This occurs for instance if the sensing model (3) is moderately corrupted, i.e., if there exists a noise n ∈ RM such that, for any x ∈ K, kQδ ((Φx + n) + ξ) − Qδ (Φx + ξ)k1 6 r δ. In such a case, we can relax the formal reconstruction (4) and ask to reconstruct x∗ from find any x∗ ∈ RN such that kQδ (Φx∗ + ξ) − Qδ (Φx + ξ)k1 6 r δ and x∗ ∈ K.
(6)
This leads to study a new worst case reconstruction error associated to the definition of relaxed consistency cells Cxr := {x∗ ∈ K : kQδ (Φx∗ + ξ) − Qδ (Φx + ξ)k1 6 r δ}, i.e., kx − x∗ k. Eδr (Φ, ξ, K) := max max ∗ r x∈K x ∈Cx
Sec. 5 proves that, with a probability higher than 1 − η with respect to the random draw of Φ ∼ N M ×N (0, 1) and ξ ∼ U M ([0, δ]), the worst case error on almost consistent vectors behaves like ) 1 Eδr (Φ, ξ, BN ) = O NM+r (log( M max(N,M ) + log( )) N 2η for RG-FCQ if M > N , while for QCS and K-sparse signals, Eδr (Φ, ξ, ΣK ∩ BN ) = O
M max(N,M ) K+r ) M (log( K
1 + log( 2η )) .
These two results show that, in an almost consistent regime where r = O(1) with respect to M , e.g., if the dynamics of the noise components nj decreases with j, the impact of the noise n on the reconstruction of x is controlled and does not change the asymptotic decay of the worst case reconstruction error. However, in the case of stronger noise where r = O(M ), the worst case reconstruction error can suffer from a systematic bias in O(r/M ). This bias is due to our ignorance of the locations of the corrupted quantized projections in the almost consistent sensing above. The rest of the paper is structured as follows. In Sec. 2, our main results are discussed under the light of related prior works, i.e., with respect to known upper and lower bounds on reconstruction errors for RG-FCQ and in the QCS setting. Sec. 3 provides the proof of our main results above in the context of RG-FCQ, while Sec. 4 focuses on QCS by extending the analysis to the set of K-sparse signals of RN . Sec. 5 studies the proximity of almost consistent vectors. Our last section, Sec. 6, provides finally the proof of a key Lemma sustaining all our developments. This one is connected to a geometric equivalence between quantization of random projections in RN and a variant of Buffon’s needle problem [18, 21]. 4
Conventions: We find useful to present here the rest of the notations used throughout this paper. Domain dimensions are denoted by capital roman letters, e.g., M, N, . . . Vectors and matrices are associated to bold symbols while lowercase light letters are associated to scalar values, e.g., Φ ∈ RM ×N or u ∈ RM . The identity matrix in RD reads 1D×D . The ith component of a vector (or of a vector function) u reads either ui or (u)i , while the vector ui may refer to the ith element of a set of vectors. The set of indices in RD is [D] = {1, · · · , D} and for any S ⊂ [D] of cardinality S = #S, uS ∈ R#S denotes the restriction of u to S. For materializing this last operation, we also introduce the linear restriction operator RS such that RS u = uS , i.e., RS = ((1M ×M )S )T , where B S denotes the matrix obtained by restricting the of Pcolumns p D×D p B ∈R to those indexed in S. For any p > 1, the `p -norm of u is kukp = i |ui | with k·k = k·k2 . The (N − 1)-sphere in RN is SN −1 = {x ∈ RN : kxk = 1} while the unit ball is N denoted BN = {x ∈ RN : kxk 6 1}. More generally, we note BN s (q) = {x ∈ R : kx − qk 6 s}. For asymptotic relations, we use the common Landau family of notations, i.e., the symbols O, Ω and Θ [24]. The positive thresholding function is defined by (λ)+ := 12 (λ + |λ|) for any λ ∈ R.
2
Background and Discussion
In the context of uniform scalar quantization of resolution δ > 0, many works have addressed the model (1) by observing that the distortion induced by quantization compared to a linear model Ax is the one of an additive measurement noise n = Q[Ax] − Ax with ni ∈ [−δ/2, δ/2], i.e., q = Ax + n. (7) When the resolution is small compared to the dynamics of Ax, i.e., under the high resolution assumption [7, 16, 19], or if a random dithering is added to the quantization [16], each component of the noise can be assumed uniformly distributed within [−δ/2, δ/2]. This allows one to assume 2 2 2 n independent √ of Ax and to bound the power of this noise, i.e., Eknk = M δ /12 and knk 6 1 2 12 δ (M + ζ M ) with high probability for ζ = O(1) (see, e.g., [19]). When a noise n of bounded power knk 6 ε corrupts the compressive observation of a sparse signal x as in (7), a worst case reconstruction error in √ kx − x∗ k = O(ε/ M ), can be reached by various reconstruction methods (e.g., Basis Pursuit DeNoise [10, 12] or Iterative Hard Thresholding [2]) as soon as the rescaled sensing matrix √1M A respects the restricted isometry property (RIP) [10]. When the compressive observations of a K-sparse signal undergo uniform scalar quantization, it is√then expected that, with high probability, kx − x∗ k = O(δ) by setting ε2 = 1 2 12 δ (M + ζ M ). The constancy of this error with respect to M is also known as the classical error limit of the pulse code modulation scheme (PCM) in CS [17]. However, most of these reconstruction techniques enforce a `2 -norm fidelity with q, e.g., by imposing kAx∗ − qk 6 ε, and the reconstructed signal is not guaranteed to be consistent with the observations, i.e., Q[Ax∗ ] 6= q. The knowledge of the sensing model is thus not fully exploited for reconstructing x from q. In the context of frame coefficient quantization (FCQ), it is also known that linear reconstruction methods resynthesizing the signal from its frame coefficients corrupted by an additive noise of variance σ 2 have a root mean square error (RMSE) lower bounded by [15, 30] √ 1 (Ekx − x∗ k2 ) 2 > N σ/ M , 5
where the frame is assumed made of unit norm elements and the expectation is taken with respect to noise. A better approach for improving the reconstruction error decay in QCS or in FCQ consists in explicitly enforcing consistency between the estimated signal and the quantized data, as formally described by the formal program (4). Such a procedure was initially introduced in [14] in the context of quantized overcomplete expansion of signals. It was also shown there that, given a random model on the generation of the sensed signal, the RMSE of any reconstruction method is lower bounded by Ω(N/M ). Interestingly, the same lower bound can also be obtained on the worst case error reconstruction without requiring a random generation model on the source [7]. While conjectured for general (M/N )-redundant frames, the combination of a tight frame formed by an oversampled Discrete Fourier Transform (DFT) with a consistent signal reconstruction reaches this lower bound, i.e., in this case the RMSE is upper bounded by O(N/M ) [14]. Consistent reconstruction methods have been applied to QCS in the high resolution regime (i.e., for δ 1 when K ⊂ BN ) [7, 13, 20, 20], for uniform (or bounded) noise for FCQ [30], in the extreme 1-bit CS setting where quantization reduces to the application of a sign operator [5, 23, 28, 29], or even for non-regular quantization scheme in CS [4, 6]. Recently, Powell and Whitehouse in [30] have analyzed a model equivalent to (3) by adopting a geometric standpoint. In particular, adapting their work to our notations, they have studied the sensing model q = Ax + n, where A = (aT1 , · · · , aTM )T ∈ RM ×N is a frame whose elements aj are drawn from a suitable distribution on SN −1 and the uniform noise n ∼ U M ([−δ, δ]) stands for, e.g., a dithered uniform scalar quantization of Ax. They observe that the consistent reconstruction polytope QM := {u ∈ RN : kAu − qk∞ 6 δ} can be seen a translation of an error polytope PN , i.e., for any consistent reconstruction x∗ ∈ QN (x∗ − x) ∈ PM := {u ∈ RN : kAu − nk∞ 6 δ}. Therefore, for a given A, analyzing the worst case error of any consistent reconstruction amounts to estimating the width of PN , i.e., WM = max{kuk : u ∈ PN } = max{ku − xk : u ∈ QN }. Authors in [30] estimate the expected worst case square error E|WM |2 with respect to the distribution of the random vectors {aj : 1 6 j 6 M } on SN −1 . Relating this estimation to coverage processes on the unit sphere [9], they show that, under general assumption on the distribution of these unit frame vectors, 1
(E|WM |2 ) 2 6
Cδ M,
with C > 0 depending on this distribution. In particular, for M frame vectors uniformly drawn at random over SN −1 , C = O(N 3/2 ) so that 1
3/2
(E|WM |2 ) 2 = O( N M δ ).
(8)
Despite a slightly different context where the results above focus on an expected worst case analysis, the behavior of these bounds is highly similar to the one we get in Corollary 1 for 6
consistent reconstruction of signals in the case of RG-FCQ: we observe that, for one random draw 1 N of this (M/N )-redundant RGF and of the quantization dithering, Eδ = O( M (log √MN + log 2η )) with probability higher than 1 − η. At first sight the dependence in N 3/2 of (8) may seem less optimal than the dependence in N of (5). However, the first bound is adjusted to random frame vectors drawn uniformly at random over SN −1 √ [30], i.e., they have all a unit norm while the RGF vectors have an expected length equal to N . Keeping in mind the difficulty to compare a bound on the expectation of a random event with a probabilistic bound on this event itself, we can notice, however, that √ rescaling the result of [30] to uniform random frames over the dilated sphere N SN −1 , or √ 1 conversely rescaling δ into δ/ N in (8), provides an error decay in (E|WM |2 ) 2 = O( NMδ ).
We can notice that the decay in (log M )/M of our bound (5) with respect to M suffers from an extra log factor compared to the decay of (E|WM |2 )1/2 . As will become clear later, this factor actually comes from a union bound argument for upper bounding the probability of failure of our error bounds over all elements of a covering set of K (see Sec. 3).
To conclude this section, as pointed out by the known lower bounds described above, let us mention that regular scalar quantization provides a rather limited decay of the reconstruction error, both for FCQ and QCS contexts. Recent developments in vector quantization for FCQ [27], in the use of feedback quantization and of Σ∆ scheme for FCQ [25] and QCS [7, 17], and finally non-regular quantization schemes where Q is periodic over its range [4, 6], provide all faster reconstruction error bounds decaying polynomially or even exponentially in M . The implicit objective of this paper is therefore to improve our understanding of one of the simplest quantization scheme, basically a dithered round off operation when combined with random Gaussian projections.
3
Quantization of Random Gaussian Frames
This section is dedicated to the proofs of Theorem 1 and of its Corollary 1. Following an argument developed in [4] for non-regular scalar quantization, proving that Eδ (Φ, ξ, BN ) = max max kx − yk x∈BN y∈Cx
6
0
(9)
holds with a probability higher than 1−η on the random draw of a RGF Φ = (ϕ1 , · · · , ϕM )T ∼ N M ×N (0, 1) and of a dithering ξ ∼ U M ([0, δ]), amounts to showing that P[∀x, y ∈ BN , Qδ [Φx + ξ] = Qδ [Φy + ξ] ⇒ kx − yk 6 0 > 1 − η, where P is computed with respect to the random quantities Φ and ξ. Taking the contraposition, we can alternatively demonstrate that, Pfail := P ∃ x, y ∈ BN , kx − yk > 0 s.t. Qδ [Φx + ξ] = Qδ [Φy + ξ] 6 η. For upper bounding Pfail , we take a s-covering of the unit ball BN , i.e., a finite point set Ls ¯ ∈ Ls at most s far apart from v, i.e., kv−¯ such that for any v ∈ BN , there exists a point v v k 6 s. N The cardinality Ls = #Ls of this covering set is known to be bounded as Ls 6 (3/s) [1]. Therefore, if x, y ∈ BN are such that kx − yk > 0 , taking their respective closest points ¯, y ¯ ∈ Ls , we have k¯ ¯ k > 0 − 2s. Consequently, it is easy to show that x x−y ¯ ∈ Ls : k¯ ¯ k > 0 − 2s), p, q p−q Pfail 6 P (∃¯ ∃u ∈ Bs (¯ p), ∃v ∈ Bs (¯ p) : Qδ [Φu + ξ] = Qδ [Φv + ξ] . 7
¯=x ¯, Indeed, if the event whose probability is measured by Pfail is verified for x and y, taking p ¯=y ¯ , u = x and v = y shows that the event associated to the probability of the RHS above q occurs. If one can find an upper bound P0 on ¯ k > 0 − 2s 6 P0 P ∃u ∈ Bs (¯ p), ∃v ∈ Bs (¯ p), Qδ [Φu + ξ] = Qδ [Φv + ξ] k¯ p−q ¯ and q ¯ , since the number of pair of points in Ls is bounded by that is independent of p independently of any conditions on them, an union bound provides
LS 2
< 12 L2s
Pfail 6 12 L2s P0 . The following key Lemma allows one to estimate P0 . ˜, q ˜ be two points in RN . There exists a radius s0 > Lemma 1. Let p Φ∼
N N ×M (0, 1)
and ξ ∼
U M ×1 ([0, δ]),
the probability
√1 k˜ ˜k p−q 8 N
such that, for
Ps0 (α, M ) := P ∃u ∈ Bs0 (˜ p), ∃v ∈ Bs0 (˜ q ), Qδ [Φu + ξ] = Qδ [Φv + ξ] satisfies 1−
Ps0 (α, M ) 6
M 3α , 8 + 4α
(10)
˜ k/δ. with α = k˜ p−q As explained in its proof (see Sec. 6), this Lemma is determined by an equivalence with Buffon’s Needle problem in N dimensions [18], where the needle is actually replaced by a ˜ and q ˜. “dumbbell” shape whose two balls are associated to the two neighborhoods of p The quantity Pλ (α, M ) defined in Lemma 1 increases with λ > 0. Therefore, for finding an estimation of P0 which is associated to the covering radius s, we must guarantee that s 6 ¯ k > 0 − 2s and 2s0 > √1 k¯ ¯ k. This is achieved by imposing s0 , knowing that k¯ p−q p−q 4 N √1 ( 4 N 0
− 2s) = 2s, i.e.,
2s =
This provides also 0 − 2s =
√ 4 N √ 4 N +1 0
√0 . 4 N +1
> 45 0 if N > 2.
Consequently, using (10) and observing that 1 − 3α/(4 + 8α) decays with α, ¯ k > 0 − 2s P ∃u ∈ Bs (¯ p), ∃v ∈ Bs (¯ p), Qδ [Φu + ξ] = Qδ [Φv + ξ] k¯ p−q ¯ k, M | k¯ ¯ k > 0 − 2s = Ps 1δ k¯ p−q p−q ¯ k, M | k¯ ¯ k > 0 − 2s p−q p−q 6 Ps0 1δ k¯ M M 34 0 6 1 − 8δ +5 404 < 1 − 8δ 2 6 exp(− 4δ M+020 ), + 40 5 0
where the conditioning of Ps simply records the extra information without altering its definition. We can then set P0 = exp(− 4δ M+020 ) so that finally Pfail = P Qδ [Φx + ξ] = Qδ [Φy + ξ] kx − yk > 0 6 12 ( 3s )N exp(− 4δ M+020 ) √
N +6 ) − 4δ M+020 ) 0 √ log( 290 N ) − 4δ M+020 ).
=
1 2
exp(N log( 24
6
1 2
exp(N
8
Therefore, if we want Pfail 6 η for some 0 < η < 1, it suffices to impose M>
4δ + 20 0
√ 1 N log( 290 N ) + log 2η ,
which determines the condition invoked in Theorem 1. Knowing that we have necessarily 0 6 2 since x, y ∈ BN , a stronger condition for (9) to occur with the same lower bound on its probability reads M>
4(δ + 1) 0
√ 1 N log( 290 N ) + log 2η .
(11)
Alternatively, saturating this condition, we have 0 =
4(δ + 1) M
√
1 N log( 290 N ) + log 2η
6
4(δ + 1) M
√ ) + log 1 . N log( 25M 2η N
where we used the fact that, from (11), M √ N
>
4 0
√
√
N log( 290 N ) >
4 0
√ 29 ) log( √ N> 2
2 29 5 0
√
N,
since 0 6 2 and assuming N > 2. In other words, assuming δ = O(1), there exists a constant C > 0 such that, 1 1 N log( √MN ) + M log 2η > 1 − η, P Eδ (Φ, ξ, BN ) 6 C M which proves Corollary 1.
4
Extension to K-Sparse Vectors of RN
We study now how the minimal number of measurements evolves in the statement of Theorem 1 when both the original signal and the consistent reconstruction are additionally assumed to be K-sparse in BN ⊂ RN , i.e., they belong to K = ΣK ∩ BN with ΣK := {w ∈ RN : kwk0 6 K}.
Notice first that, given a fixed support T0 ⊂ [N ] with #T0 = 2K, thanks to the developments of Sec. 3, P (∃x, x∗ ∈ BN : kx − x∗ k > 0 , supp x ∪ supp x∗ ⊂ T0 ) : Qδ [Φx + ξ] = Qδ [Φx∗ + ξ] 6
1 2
√
exp(2K log( 29 02K ) −
M 0 4δ + 20 ),
since the subspace of vectors supported in T0 is equivalent to R2K . N eN 2K Since there are no more than 2K 6 ( 2K ) choices of 2K-length supports in [N ], another union bound provides P (∃x, x∗ ∈ BN ∩ ΣK : kx − x∗ k > 0 ) : Qδ [Φx + ξ] = Qδ [Φx∗ + ξ] 6 P (∃T ⊂ [N ] : #T = 2K), (∃x, x∗ ∈ BN : kx − x∗ k > 0 , supp x ∪ supp x∗ ⊂ T ) : Qδ [Φx + ξ] = Qδ [Φx∗ + ξ] √ N 6 12 2K exp(2K log( 29 02K ) − 4δ M+020 ) 6 12 exp(2K log( √29eN ) − 4δ M+020 ). 2K 0
Again, willing to have this last probability smaller than η ∈ (0, 1) leads to imposing 1 M > 4δ +0 20 2K log( √29eN ) + log( ) , 2η 2K 0
9
√ which, by noting that 29e/ 2 < 56, provides the key condition of Theorem 2. Since 0 6 2, a stronger condition reads M>
4(δ + 1) 0
1 ) + log( 2η 2K log( √56N ) , K 0
which gives the crude estimation MN √ K3
>
8N √ 0 K
)> log( √56N K 0
8N √ 0 K
√
log( 560 N ) >
1 √56N 2 K0 ,
using K 6 N and N > 2. Therefore, saturating the condition on M above, 1 √ N ) + log( 1 ) , 0 = 4(δ M+ 1) 2K log( √56N ) + log( 2η ) 6 4(δ M+ 1) 2K log( 2M 3 2η K K
0
which shows that, if δ = O(1), there exists a constant C > 0 for which 1 K MN 1 )+ M P Eδ (Φ, ξ, ΣK ∩ BN ) 6 C M log( √ log( 2η ) > 1 − η. 3 K
This determines the bound stated at the end of the Introduction.
5
Proximity of Almost Consistent Signals
The strict consistency between the quantized projections of two vectors of K ⊂ RN can be slightly relaxed while still keeping their maximal distance bounded. To show this, we follow a similar procedure developed in [22] in the case of 1-bit quantized random projections. We may first observe that if kQδ (Φx + ξ) − Qδ (Φy + ξ)k1 6 r δ for some r ∈ N, at most r measurements differ between Qδ (Φx + ξ) and Qδ (Φy + ξ). There exists thus a subset T of [M ] with size at least M − r such that RT Qδ (Φx + ξ) = RT Qδ (Φy + ξ), with the corresponding restriction operator RT defined in the Introduction.
Therefore, for K ⊂ RN and writing [M ]r the set of all subsets of [M ] of size M − r, a union bound provides Pr := P ∃ x, y ∈ K : kx − yk > 0 s.t. kQδ (Φx + ξ) − Qδ (Φy + ξ)k1 6 r δ 6 P ∃T ⊂ [M ]r , ∃ x, y ∈ K : kx − yk > 0 s.t. RT Qδ (Φx + ξ) = RT Qδ (Φy + ξ) [ 6 P ∃ x, y ∈ K : kx − yk > 0 s.t. Qδ (RTc Φx + ξ T ) = Qδ (RT Φy + ξ T ) . T ⊂[M ]r
We have now to split the analysis between the RG-FCQ and the QCS cases. RG-FCQ case:
If K = BN with M > N , using Sec. 3 and and Pr 6
1 2
6
1 2
M M −r
M M −r
=
M r
6 (eM/r)r ,
√
(M −r)0 2δ + 40 ) √ (M −r)0 log( 290 N ) − 4δ + 20 ).
exp(N log( 290 N ) −
exp(r log( eM r ) + N
Willing to have this last probability smaller than η ∈ (0, 1), we find that, as soon as √ 29 N 1 M > r + 4δ +0 20 r log( eM r ) + N log( 0 ) + log( 2η ) , and given Φ ∼ N M ×N (0, 1) and ξ ∼ U M ([0, δ]), the event ∀x, y ∈ K,
kQδ (Φx + ξ) − Qδ (Φy + ξ)k1 6 r δ 10
⇒
kx − yk 6 0 ,
holds with probability higher than 1 − η.
Aiming a slightly stronger condition on M by a series of crude upper bounds, we observe that √ 1 29 N ) + N log( r + 4δ +0 20 r log( eM r 0 ) + log( 2η ) √ 1 6 4δ +0 20 32 r log(eM ) + 32 N log( 290 N ) + log( 2η ) , √ 1) 3 3 1 2eM 29 N 6 4(δ + 2 r log( 0 ) + 2 N log( 0 ) + log( 2η ) , 0 √ (δ + 1) 3 3 1 2eM N 29 N N ) + ) + log( = 4N r log( N log( ) , 0 0 0 2 2 2η (N +r)0 (N +r)0 (N +r)0 (δ + 1) 3 3 1 2eM 29N 6 4N 2 r log( 0 ) + 2 N log( 0 ) + log( 2η ) , (N +r)0 0
0
6
4N (δ + 1) 3 (r (N +r)00 2
=
6(δ + 1) N 00
0
+ N ) log( 29 max(N,M/5) ) + 0
4(δ + 1) 00
0
log( 29 max(N,M/5) ) + 0 0
4(δ + 1) 00
1 log( 2η ),
1 log( 2η ),
using the variable change 0 = NN+r 00 , 0 6 2 and 2e/29 < 1/5. Notice that in the case where r = 0, remembering that the term r log(eM/r) above comes from a bound on log M r , we can assume r log(eM/r) = 0, and since r ∈ N, we can write r log(eM/r) 6 r log(eM ).
Therefore, reexpressing everything in function of 00 and forgetting the prime symbol, we find that, as soon as M>
6(δ + 1) N 0
log( 29 max(N,M/5) ) + 0
4(δ + 1) 0
1 log( 2η ),
and given a random draw of Φ ∼ N M ×N (0, 1) and ξ ∼ U M ([0, δ]), the event ∀x, y ∈ K,
kQδ (Φx + ξ) − Qδ (Φy + ξ)k1 6 r δ
⇒
kx − yk 6
N +r N 0 ,
(12)
holds with probability higher than 1 − η.
Saturating the condition on M above and since N > 2 > 0 , we find M>
6(δ + 1) N 0
log( 29 max(N,M/5) )> 0
6 0 N
log(29) >
2 29N 3 0 ,
and 0 =
0 such that ) 1 P Eδr (Φ, ξ, BN ) 6 C NM+r log( M max(N,M ) + log( 2η ) > 1 − η. N QCS case:
If K = ΣK ∩ BN , using the proof of Sec. 4, we have similarly (M −r)0 Pr 6 12 MM−r exp(2K log( √29eN ) − 4δ + 20 ) 2K 0
6
1 2
exp(r log( eM r )
+ 2K log( √29eN ) − 2K 0
(M −r)0 4δ + 20 ).
Again, willing to have this last probability smaller than η ∈ (0, 1), we find that, as soon as 1 √56N M > r + 4δ +0 20 r log( eM r ) + 2K log( K ) + log( 2η ) , 0
11
and given Φ ∼ N M ×N (0, 1) and ξ ∼ U M ([0, δ]), the event ∀x, y ∈ K,
kQδ (Φx + ξ) − Qδ (Φy + ξ)k1 6 r δ
⇒
kx − yk 6 0 ,
holds with probability higher than 1 − η.
Targeting a slightly stronger condition on M by a series of crude upper bounds, we observe that 4δ + 20 1 √56N r(1 + 4δ +0 20 ) log( eM 0 r ) + 2K log( K0 ) + log( 2η ) , 1 ) + log( ) , 6 4δ +0 20 2r log(eM ) + 2K log( √56N 2η K0 1) 1 √56N 6 4(δ + 2r log( 2eM 0 ) + 2K log( K0 ) + log( 2η ) , 0 √ + 1) 2eM K 56N K 1 = 4K(δ 2r log( (K+r) 0 ) + 2K log( (K+r)0 ) + log( 2η ) , (K+r)00 0 0 4K(δ + 1) 2eM 56N 1 ) , 6 (K+r)0 2r log( 0 ) + 2K log( 0 ) + log( 2η 0
6 =
0
4K(δ + 1) 2(r (K+r)00 8(δ + 1) K 00
0
+ K) log( 56 max(N,M/10) )+ 0 0
log( 56 max(N,M/10) )+ 0 0
4(δ + 1) 00
4(δ + 1) 00
1 log( 2η ),
1 log( 2η ),
0 using the variable change 0 = K+r K 0 , 0 6 2 and 2e/56 < 1/10, and with the same remark on the vanishing value of r log(eM/r) when r = 0.
Therefore, reexpressing everything in function of 00 and forgetting the prime symbol, we find that, as soon as M>
8(δ + 1) K 0
log( 56 max(N,M/10) )+ 0
4(δ + 1) 0
1 log( 2η ),
and given a random draw of Φ ∼ N M ×N (0, 1) and ξ ∼ U M ([0, δ]), the event ∀x, y ∈ K,
kQδ (Φx + ξ) − Qδ (Φy + ξ)k1 6 r δ
⇒
kx − yk 6
K+r K 0 ,
(13)
holds with probability higher than 1 − η.
Saturating the condition on M above, since for this M M>
8(δ + 1) K 0
log( 56 max(N,M/10) )> 0
8 0 K
log(56) >
1 56 2 0 K,
we find 0 =
0 such that ) 1 P Eδr (Φ, ξ, ΣK ∩ BN ) 6 C K+r log( M max(N,M ) + log( 2η )) > 1 − η. M K
6
Proof of Lemma 1
˜, q ˜ ∈ RN , there exists a Let us recall the context. We want to show that, given two points p 1 0 N ×M M ×1 ˜ k such that, for Φ ∼ N radius s > 8√N k˜ p−q (0, 1) and ξ ∼ U ([0, δ]), the probability Ps0 (α, M ) := P ∃u ∈ Bs0 (˜ p), ∃v ∈ Bs0 (˜ q ), Qδ [Φu + ξ] = Qδ [Φv + ξ] 12
satisfies Ps0 (α, M ) 6
1−
M 3α , 8 + 4α
˜ k/δ. with α = k˜ p−q
Notice first that we can focus on upper bounding the probability associated to a single projection by the random vector ϕ ∼ N N ×1 (0, 1) quantized with Qδ with a scalar dithering ξ ∼ U([0, δ]), the result for M dithered quantized projections simply following by raising the single measurement bound to the power M , i.e., Ps0 (α, M ) 6 (Ps0 (α, 1))M . ˆ where ϕ ˆ ∈ SN −1 is uniformly distributed at random over SN −1 and the We write ϕ = φ ϕ, length φ = kϕk ∼ χ(N ) follows a χ distribution with N degrees of freedom. We are going first to estimate the following conditional probability: Ps0 (α, 1|φ) := P ∃u ∈ Bs0 (˜ p), ∃v ∈ Bs0 (˜ q ), Qδ [ϕT u + ξ] = Qδ [ϕT v + ξ] kϕk = φ ˆ T u + ξ] = Qδ [φϕ ˆ T v + ξ] kϕk = φ = P ∃u ∈ Bs0 (˜ p), ∃v ∈ Bs0 (˜ q ), Qδ [φϕ ˆ T u + ξ] = Qδ [ϕ ˆ T v + ξ] kϕk = φ , (14) = P ∃u ∈ Br (p), ∃v ∈ Br (q), Qδ [ϕ ˆ = ϕ. Notice that 2r/kp − qk = with the variable change r = φs0 , p = φ˜ p and q = φ˜ q and φϕ ˜ k. Let us focus on this last probability keeping in mind the relationships between 2s0 /k˜ p−q these parameters for estimating later a result which is not conditioned to the knowledge of φ.
We follow the procedure described in [21]. In this work, from a generalization of the Buffon’s needle problem [8, 18] in N dimensions, it is shown that when r = 0, i.e., u = p and v = q, computing Ps0 (α, 1|φ) above is equivalent to estimating the probability that a segment (or needle) of length L = kp − qk uniformly “thrown” at random in RN , both spatially and in orientation, does not intersect a fixed set of parallel (N − 1)-dimensional hyperplanes spaced by a distance δ. ˆ ∈ SN −1 and ξ ∈ [0, δ], the function f (v) := Qδ (ϕ ˆ T v + ξ) is piecewise More precisely, given ϕ N constant in R and the frontiers where its value changes correspond to a set of parallel (N − 1)dimensional hyperplanes in RN . These hyperplanes are equi-spaced with a separating distance ˆ Consequently, the quantity X := 1δ Qδ (ϕ ˆTp + δ and they are all normal to the direction ϕ. ˆ T q + ξ) ∈ Z counts the number of such hyperplanes intersecting the segment pq. In ξ) − Qδ (ϕ this scenario, this segment is thus fixed and the hyperplanes are randomly oriented and shifted ˆ and ξ, respectively. by ϕ However, we can reverse the point of view and rather consider those hyperplanes as fixed and normal, e.g., to the first canonical axis e1 of RN . This is allowed by considering the affine mapping Aϕ,ξ : RN → RN implicitly defined by any combination of a rotation and of ˆ ˆ T v + ξ for all v ∈ RN . In words, thanks to a translation in RN such that eT1 Aϕ,ξ ˆ (v) = ϕ N ˆ and shifting the result by ξ is Aϕ,ξ ˆ , projecting a point v ∈ R onto the random orientation ϕ equivalent to projecting the random point Aϕ,ξ ˆ (v) onto e1 .
0 Therefore, denoting p0 = Aϕ,ξ ˆ (q), it is easy to see that the L-length ˆ (p) and q = Aϕ,ξ 0 0 segment p q , i.e., our needle, is then oriented uniformly at random over SN −1 while the distance of its centrum 12 (p0 + q 0 ) to the closest hyperplane follows a uniform random variable over the interval [0, δ/2]. Moreover, we have X = 1δ Qδ (eT1 p0 ) − Qδ (eT1 q 0 ) ,
so that X actuallySmeasures the number of intersections the segment p0 q 0 makes with the set of hyperplanes Gδ = k∈Z {x : eT1 x = k}. In [21], the distribution of the discrete bounded random variable X is actually fully determined and denoted Buffon(L/δ, N ).
13
G
r w ✓
r
L
Figure 1: A Buffon “dumbbell” problem in 2-D.
For r > 0, Eq.(14) shows that we must now consider the two neighboring `2 -balls of p and q in Ps0 (α, 1|φ) and estimate the probability that at least two points of these balls share the same ˆ Following the same argument as above, this new problem dithered quantized projection onto ϕ. is now equivalent to a new Buffon experiment if the previous needle is ended with two balls. In other words, we create a dumbbell shape formed by a segment of length L on the extremities of which two balls of radius r are centered (see Fig. 1). It is then easy to see that Ps0 (α, 1|φ) is equivalent to the probability that there is no hyperplane of Gδ intersecting only the part of the segment outside of the two balls when the dumbbell is thrown randomly in RN as for previous Buffon’s needle. Otherwise, having such an intersection would mean that no pair of points (taken in distinct balls) lie in the same subvolume delimited by two consecutive hyperplanes, i.e., they do not have the same quantized projection, and conversely. Let us parametrize this dumbbell by its distance w ∼ U([0, δ/2]) (estimated from the middle of the segment) to the closest hyperplane Gδ and by its orientation drawn uniformly at random in SN −1 . By symmetry, only the angle θ ∈ [0, π] made by the dumbbell with the normal vector e1 to Gδ is important in this parametrization [21]. Moreover, from Fig. 1, the absence of intersection amounts to imposing w > 12 L| cos θ| − r. The probability Ps0 (α, 1|φ) is thus obtained by Z π Z δ/2 N −2 Ps0 (α, 1|φ) = κN (sin θ) dθ I(w > L2 | cos θ| − r) 2δ dw, 0
=
4κN δ
0
Z
π/2
(sin θ)
N −2
Z dθ
0
δ/2
I(w > 0
L 2
cos θ − r) dw,
where κN (sin θ)N −2 dθ is the area (normalized to the one of SN −1 ) of the thin spherical segment ˆ ) ∈ [θ, θ + dθ]}, where κN := Sdθ (θ) := {ˆ v ∈ SN −1 : arccos(eT1 v B(k, l) = Γ(k)Γ(l)/Γ(k + l) is the Beta function.
√
Γ( N ) 2 π Γ( N 2−1 )
= B( 12 , N 2−1 )−1 and
It is important to remark that, from [21, 31], √ √2 π
1
(N + 1)− 2 6
2κN N −1
6
√ √2 π
1
(N − 1)− 2 ,
so that, for N > 2, √1 2π
1
(N + 1) 2 − 1 < κN 6
√1 2π
14
1
(N − 1) 2
q N ⇒ κN = Θ( 2π ).
(15)
Let us define two angles 0 6 θ0 6 θ1 6 π/2 such that cos θ0 = min( δ+2r L , 1) and cos θ1 = 2r 0 L , assuming 2r 6 L (otherwise, Ps = 1). The angular integration domain can be split in three intervals: [0, θ0 ], [θ0 , θ1 ] and [θ1 , π/2]. Over the first interval, the integral is always zero since, either we have a zero measure interval (θ0 = 0) or I(w > L2 cos θ − r) = 0 since L L 2 cos θ > 2 cos θ0 = δ/2 + r and 0 6 w 6 δ/2. Moreover, over the last interval [θ1 , π/2], L I(w > 2 cos θ − r) = 1 Therefore, writing a = L/δ, Z θ1 4κN (sin θ)N −2 ( 2δ − Ps0 (α, 1|φ) = δ θ0
=1+ =1−
4κN δ 4κN δ
Z
θ0 θ1
Z
4κN δ
=1 −
+
θ1
(sin θ)N −2 ( 2δ
L 2
Z
−
L 2
cos θ + r) dθ − 2κN
θ0
Z
θ1
4κN δ
= 1 − 2κN a
Z
0
4κN δ
θ0
Z 0
Z
θ1
(sin θ)N −2 dθ
0
(sin θ)N −2 2δ dθ
(sin θ)N −2 ( L2 cos θ − r) dθ θ0
0
Z
(sin θ)N −2 dθ
θ1
(sin θ)N −2 ( L2 cos θ − r) dθ −
0
π/2
cos θ + r) dθ + 2κN
(sin θ)N −2 ( L2 cos θ − (r + 2δ )) dθ
1
(1 − v 2 )
N −3 2
(v −
2r L )+
− (v −
2r+δ L )+
dv,
applying a variable change v = cos θ on the last line. Let us study this R vlast integral and the function f (v) = (v − verify that F (v) := 0 f (v 0 )dv 0 is convex and reads 0, 2 2r+δ 2 2 2F (v) = (v − 2r ) − (v − ) = (v − 2r + L + L L) , δ 4r+δ L (2v − L ),
2r L )+
− (v −
2r+δ L )+ .
We can
if v 6 2r L, 2r if L < v 6 2r+δ L , 2r+δ if v > L .
Moreover, by integrating by part, Z 1 Z 1 N −5 2 N 2−3 (1 − v ) f (v) dv = (N − 3) v (1 − v 2 ) 2 F (v) dv, 0
0
N −5
The positive measure µ(v) = (N −3) v (1−v 2 ) 2 has unit mass over [0, 1] so that, by convexity of F and using the Jensen inequality, Z 1 R1 F (v) µ(v) dv > F 0 v µ(v) dv . 0
However, since (N − 3)
R1 0
(1 −
N −5 v2) 2
vq
dv =
N −3 2
N −3 B( q+1 2 , 2 )
we find R1 0
v µ(v) dv = (N − 3)
R1
2 0 (1 − v )
N −5 2
=
√
v 2 dv =
q+1 N −1 Γ( 2 ) 2 N +q−2 , Γ 2
π Γ( N 2−1 ) 2 Γ( N ) 2
and Ps0 (α, 1|φ) 6 1 − 2κN a F ( 2κ1N ). 15
Γ
=
1 2κN ,
(16)
(17)
From the definition of F above, if 2κ1N 6 2r L , F = 0 and we cannot show anything. Let us λ thus set 2r = 2κN L, where λ ∈ (0, 1) will be determined later. Notice that, since s0 = φ r and ˜ k = φ L, we implicitly impose 2r/L = 2s0 /k˜ ˜ k = 2κλN . k˜ p−q p−q Then,
( 2F ( 2κ1N ) = 2κN 1−λ ,
so that, writing φ0 =
1 (1 − λ)2 , 4κ2N 1 1 1 a ( κN (1 − λ) − a ),
if a 6 if a >
2κN 1−λ , 2κN 1−λ ,
we have
Ps0 (α, 1|φ) 6
( 1−
λ+
a 2φ0 (1 − λ), φ0 2a (1 − λ),
if a 6 φ0 , if a > φ0 .
Let us recall that Ps0 (α, 1|φ) is defined conditionally to φ = kϕk with φ ∼ χ(N ). Moreover, ˜ k/δ = αφ with α = k˜ ˜ k/δ. Denoting the pdf of χ(N ) by γN (φ) = a = kp − qk/δ = φk˜ p−q p−q R +∞ φ2 1− N N N −1 2 cN φ exp(− 2 ) and cN = 2 /Γ( 2 ), we can develop Ps0 (α, 1) = 0 Ps0 (α, 1|φ) γN (φ) dφ as follows Z φ0 /α Z +∞ αφ φ0 (1 − 2φ0 (1 − λ)) γN (φ) dφ + Ps0 (α, 1) 6 (λ + 2αφ (1 − λ)) γN (φ) dφ 0
φ0 /α
= λ + (1 − λ) = λ + (1 − λ)
Z
φ0 /α
(1 −
0
Z
+∞
0
αφ 2φ0 ) γN (φ) dφ
+ (1 − λ)
Z
+∞
φ0 /α
φ0 2αφ γN (φ) dφ
ϕ( αφ φ0 ) γN (φ)dφ,
with ϕ(t) = 1 − 12 t if 0 6 t < 1 and ϕ(t) =
1 2t
if t > 1.
We can notice that tϕ(t), which is equal to 12 t(2 − t) over [0, 1] and to concave function. Therefore, by Jensen inequality, Z +∞ Z +∞ αφ cN ϕ( φ0 ) γN (φ)dφ = cN −1 φ ϕ( αφ φ0 ) γN −1 (φ)dφ 0
1 2
for t > 1, is a
0
6
cN
cN −1 (EγN −1 φ) ϕ(
αEγN −1 φ ) φ0
√ √ √ We have also cN /cN −1 = Γ( N 2−1 )/( 2Γ( N2 )) and EγN −1 φ = 2Γ( N2 )/Γ( N 2−1 ) = 2πκN , so that cNcN−1 (EγN −1 φ) = 1 and αEγN −1 φ φ0
Consequently, since ϕ(t) 6
1−λ = α 2κ EγN −1 φ = N
6 λ + (1 − λ)
q
2+
2 π
2
π 2 (1
√ π2
2+
√π (1−λ)2 α 2 =1− √ π
q
π 2 (1
− λ)α.
2 2+t ,
Ps0 (α, 1) 6 λ + (1 − λ)ϕ
taking (1 − λ) =
q
2
(1−λ)α
> 3/4. 16
− λ)α
(1−λ)α q
= 1−
2 α π
2+α
< 1−
3α 8+4α
Moreover, from the bounds on κN given in (15), this shows also that q q 1 2s0 2r λ 2 π √1 = = > (1 − L 2κN π) 2 (N −1)1/2 > 4 N , k˜ p−˜ qk as stated at the beginning of Lemma 1.
Acknowledgements Laurent Jacques is a Research Associate funded by the Belgian F.R.S.-FNRS.
References [1] R. Baraniuk, M. Davenport, R. DeVore, and M. Wakin. A simple proof of the restricted isometry property for random matrices. Constructive Approximation, 28(3):253–263, 2008. [2] T. Blumensath and M. Davies. Iterative hard thresholding for compressive sensing. Appl. Comput. Harmon. Anal., 27(3):265–274, 2009. [3] P. T. Boufounos. Quantization and Erasures in Frame Representations. D.Sc. Thesis, MIT EECS, Cambridge, MA, January 2006. [4] P. T. Boufounos. Universal rate-efficient scalar quantization. IEEE Trans. Info. Theory, 58(3):1861–1872, March 2012. [5] P. T. Boufounos and R. G. Baraniuk. 1-bit compressive sensing. In Proc. Conf. Inform. Science and Systems (CISS), Princeton, NJ, March 19-21 2008. [6] P. T. Boufounos and S. Rane. Efficient coding of signal distances using universal quantized embeddings. In Proc. Data Compression Conference (DCC), Snowbird, UT, March 20-22 2013. [7] P. T. Boufounos, L. Jacques, F. Krahmer, and R. Saab. Quantization and compressive sensing. In book ”Compressed Sensing and its Applications”, Springer, 2014. [8] G. C. Buffon. Essai d’arithm´etique morale, volume 4 of Suppl´ement a ` lhistoire naturelle, 1777. http://www.buffon.cnrs.fr. [9] P. B¨ urgisser, F. Cucker, M. Lotz, et al. Coverage processes on spheres and condition numbers for linear programming. The Annals of Probability, 38(2):570–604, 2010. [10] E. Cand`es, J. Romberg, and T. Tao. Stable signal recovery from incomplete and inaccurate measurements. Comm. Pure Appl. Math, 59(8):1207–1223, 2006. [11] E. J. Candes and T. Tao. Near-optimal signal recovery from random projections: Universal encoding strategies? Information Theory, IEEE Transactions on, 52(12):5406–5425, 2006. [12] S. S. Chen, D. L. Donoho, and M. A. Saunders. Atomic Decomposition by Basis Pursuit. SIAM Journal on Scientific Computing, 20(1):33–61, 1998. [13] W. Dai, H. V. Pham, and O. Milenkovic. Distortion-Rate Functions for Quantized Compressive Sensing. Technical Report arXiv:0901.0749, 2009. [14] V. K Goyal, M. Vetterli, and N. T. Thao. Quantized overcomplete expansions in RN : Analysis, synthesis, and algorithms. IEEE Trans. Info. Theory, 44(1):16–31, 1998. [15] V. K. Goyal, J. Kovaˇcevi´c, and J. A. Kelner. Quantized frame expansions with erasures. Applied and Computational Harmonic Analysis, 10(3):203–233, 2001. [16] R.M. Gray and D.L. Neuhoff. Quantization. IEEE Trans. Info. Theory, 44(6):2325–2383, Oct 1998.
17
¨ Yılmaz. Sobolev duals for random frames and σδ [17] C. S. G¨ unt¨ urk, M. Lammers, A. M. Powell, R. Saab, and O. quantization of compressed sensing measurements. Foundations of Computational Mathematics, 13(1):1–36, 2013. [18] J. D. Hey, T. M. Neugebauer, and C. M. Pasca. Georges-Louis Leclerc de Buffons Essays on Moral Arithmetic. In The Selten School of Behavioral Economics, pages 245–282. Springer, 2010. [19] L. Jacques, D. K. Hammond, and M. J. Fadili. Dequantizing Compressed Sensing: When Oversampling and Non-Gaussian Constraints Combine. IEEE Trans. Inf. Theory, 57(1):559–571, January 2011. [20] L. Jacques, D. K. Hammond, and M. J. Fadili. Stabilizing Nonuniformly Quantized Compressed Sensing With Scalar Companders. IEEE Trans. Inf. Theory, 59(12):7969 – 7984, January 2013. [21] L. Jacques. A Quantized Johnson Lindenstrauss Lemma: The Finding of Buffon’s Needle. arXiv preprint arXiv:1309.1507, 2013. [22] L. Jacques, K. Degraux, and C. De Vleeschouwer. Quantized iterative hard thresholding: Bridging 1-bit and high-resolution quantized compressed sensing. In 10th international conference on Sampling Theory and Applications (SampTA 2013), arXiv:1305.1786, pages 105–108, Bremen, Germany, 2013. [23] L. Jacques, J. N. Laska, P. T. Boufounos, and R. G. Baraniuk. Robust 1-bit compressive sensing via binary stable embeddings of sparse vectors. IEEE Transactions on Information Theory, 59(4):2082–2102, 2013. [24] D. E. Knuth. Big omicron and big omega and big theta. ACM Sigact News, 8(2):18–24, 1976. [25] F. Krahmer, R. Saab, and R. Ward. Root-exponential accuracy for coarse quantization of finite frame expansions. IEEE Trans. Inform. Theory, 58(2):1069–1079, 2012. [26] B. K. Natarajan. Sparse approximate solutions to linear systems. SIAM journal on computing, 24(2):227– 234, 1995. [27] H. Q. Nguyen, V.K. Goyal, and L.R. Varshney. Frame Permutation Quantization. Applied and Computational Harmonic Analysis (ACHA), Nov. 2010. [28] Y. Plan and R. Vershynin. arXiv:1111.4452, 2011.
Dimension reduction by random hyperplane tessellations.
arXiv preprint
[29] Y. Plan and R. Vershynin. One-bit compressed sensing by linear programming. Communications on Pure and Applied Mathematics, 66(8):1275–1297, 2013. [30] A. M. Powell and J. T. Whitehouse. Error bounds for consistent reconstruction: Random polytopes and coverage processes. arXiv preprint arXiv:1405.7094, 2014. [31] F. Qi, Q.-M. Luo, et al. Bounds for the ratio of two gamma functionsfrom wendels and related inequalities to logarithmically completely monotonic functions. Banach J. Math. Anal, 6(2):132–158, 2012. [32] J. A. Tropp. Just relax: Convex programming methods for identifying sparse signals in noise. Information Theory, IEEE Transactions on, 52(3):1030–1051, 2006.
18
