colored random projections for compressed sensing - Semantic Scholar
COLORED RANDOM PROJECTIONS FOR COMPRESSED SENSING
† † ‡ Gonzalo R. Arce , Zhongmin Wang and Jose L. Paredes †
Department of Electrical and Computer Engineering University of Delaware, Newark, DE, USA, 19716 ‡ Electrical Engineering Department, University of Los Andes, M´ erida, Venezuela 5101
email: [email protected], [email protected], [email protected]
problem [4]. Matching Pursuit (MP) and Orthogonal Matching Pur-
ABSTRACT This papers discusses the reconstruction of sparse signal from a few of incoherent projections based on the theory of Compressed Sensing (CS) when some a priori information about the frequency structure of the signal is available. A new method to design the random projections is introduced that achieves a notably reduced number of measurements while at the same time increases the probability of successful signal reconstruction.
The essential improvement is
achieved by using colored random projections instead of i.i.d. random projections for the measurements. The proposed approach uses color dithered random matrix as the random measurement matrix since it can achieve desired frequency structure that matches the frequency content of the underlying signal and also has simple spatial form that makes the hardware implementation very convenient. Computationally efcient algorithms to design such color dithered measurement matrix have been developed. Simulation results show the effectiveness of the proposed method.
suit (OMP) have also be proposed in the literature [3, 5]. the sparse signal.
CS relies on the only assumption that the underlying signals are sparse on some basis
Index Terms Compressed Sensing, sparse signal, colored noise,
Ψ and that the measurements are i.i.d.
random
projections [2]. In practice, many signals not only are sparse, but also have specic structures in basis other than
Ψ.
For example, a
signal sparse in the wavelet domain may have most of its energy concentrated on the low frequency band. Can we design more efcient random projections by exploiting the characteristics of the signal so that the signal can be reconstructed with higher probability using less measurements? This paper addresses this problem in detail. Our research is motivated by the fact that if we have enough knowledge about the spectrum characteristics of the signal, we can recover the signal with much less measurements [6]. Actually, if the signal is these
color dithered random projection.
These
methods are based on greedy algorithms that iteratively reconstruct
T
T
sparse in the frequency domain and if the locations of
frequency components are known, then we can recover the
signal with only
T
random projections or samples [6]. On the other
hand, it has been shown that passing the signal through a lter with random coefcients has the similar effect of making multiple projections of the signal onto random vectors in CS [7]. If we project the
1. INTRODUCTION
signal to random vectors that have band-pass characteristics (colored
Compressed sensing, initially proposed in [1, 2, 3], provides a new way to acquire and represent sparse signals that requires less sam-
random vectors) multiple times, it would be equivalent to get measurements by ltering the signal through a colored random lter and
the dominate frequency components of the signal can be enhanced pling resource and computational capability than traditional approaches. N in the measurements. Thus, it is more likely that less projections Given a T sparse signal x ∈ R on some basis Ψ = [ψ1 , ψ2 , . . . , ψN ], are needed to capture the salient information of the underlying sigthat is, x can be approximated by a linear combination of T vectors PT nal. Colored random projections could help reduce the search space from Ψ, ie, x ≈ i=1 θni ψni with T ¿ N , then the theory of to nd the sparse components of the signal in the Basis Pursuit alCompressed Sensing shows that x can be recovered from M rangorithm. Furthermore, an exact knowledge of the signal frequency dom projections with high probability when M = CT log N ¿ N , spectrum is not required for colored random projections. where C ≥ 1 is the oversampling factor. The projections are given Random projection vector that has band-pass characteristics and by y = Φx, where Φ is a M × N random measurement matrix with its rows incoherent with the columns of
Ψ.
Commonly used ran-
also can be easily implemented via hardware is preferred. This moti-
dom measurement matrices for CS are random Gaussian matrices √
vates the application of color dithered random vectors for CS. Multi-
and partial Fourier matrices. In [2], it is shown that a matrix satisfy-
digital halftoning methods [8].
(Φij
∈ {N (0, 1/N )),
Rademacher matrices (Φij
∈ {±1/ N })
level color dithered random vectors can also be easily generated by
ing the incoherent condition is so ubiquitous that nearly all matrices are CS matrices.
l1 norm minimization problem: min kθk1 subject to ΦΨθ = y, if M ≥ CT log N . V = ΦΨ is called the holographic basis. Minimizing the l1 norm
2. COLORED RANDOM PROJECTIONS
Signal reconstruction is achieved by solving a
2.1. Motivating experiments
1024-point band-pass input signal x(n) as shown in Fig. 1
yields solutions that are zero except at a small number of isolated val-
Suppose a
ues and can be solved by efcient linear programming algorithms.
is to be reconstructed via multiple random projections. Its frequency
One method called Basis Pursuit is direct l1 minimization method
components are also shown in Fig. 1 where the pass-band of the
using, for instance, interior-point method to solve the optimization
signal lies between normalized frequency
0.07
and
0.2.
Note that
the signal has a sparsity
1024-point
T = 10 in the frequency domain.
A set of
random vectors are generated by ltering a normalized
24-tap band-pass FIR lter h(n) which has a pass-band 0.05 < ω < 0.2. The output of the lter is colored white Gaussian noise with a
random noise that is not evenly distributed in the frequency domain and the noise values are correlated.
concentrated on its pass-band, it spreads over the whole spectrum. This property can be enhanced if the random vector is dithered with limited levels, as will be discussed shortly. Therefore, it is easy to incoherent with any xed basis
10
Ψ.
By matching the frequency components of the colored random
5
Value
First note that although the energy of the colored random vector is
construct universal colored random measurement matrix so that it is
15
0
vector and the underlying signal, the major frequency components
−5
of the signal are enhanced in the measurements and fewer measurements would sufce to contain all the salient information of the sig-
−10 −15
100
200
300
400
500 600 Time
700
800
900
1000
nal of interest. Furthermore, the energy of the vectors can be evaluated by integrating over their spectrum. If the peak amplitude of the
3000
colored random vectors in the pass-band is the same as the amplitude of the i.i.d. random vectors in the frequency domain, then it is
2000
Value
2.2. Frequency matching expedites signal recovery
easy to see that colored random projections also lead to less energy consumption in the real applications.
1000
From the signal recovery point of view, colored random projec-
0 −0.5
0 Normalized frquency
0.5
tion reduces the search space in the optimization by Basis Pursuit
Φx = y , the opkθk1 by searching the space expanded by the rows of measurement matrix Φ. If we study the prob-
algorithm. More precisely, given the measurement timization process tries to minimize Fig. 1. Test signal and its spectrum representation.
lem in the frequency domain, then it is clear that the solution only exists in the space expanded approximately by the major frequency components of the colored random vectors and the cost function deColored random vector
Frequency structure of colored random vector 80
1.5 1
Value
Value
0.5 0 −0.5 −1 −1.5
3
projections with i.i.d. random projections, we know that i.i.d. ran-
20
dom projections, although more universal and robust, is not efcient in that it proves a much larger searching space.
400 600 800 1000 0 0.1 0.2 0.3 0.4 Time Normalized frquency i.i.d. random vector Frequency structure of the i.i.d. random vector 80
3. COLORED RANDOM MEASUREMENT MATRIX
60 Value
Value
covery the signal with high probability. Comparing colored random
40
200
1 0 −1 −2 −3
That is the reason why much less measurements are sufcient to re-
60
0
2
creases rapidly in the direction where these major components exist.
200
400 600 Time
40
The colored random projection is effective in the CS framework and
20
is simple to generate.
0
800 1000
However, their spectrum magnitude on the
stop-band may be close to 0
0.1 0.2 0.3 0.4 Normalized frquency
0
leading to singularity problem in the
signal reconstruction. This problem can be alleviated if we quantize the colored random vectors. The quantized colored random vectors can be generated by feeding normalized Gaussian noise to a band-
Fig. 2. Colored random vectors and their spectrum representations.
pass lter and quantizing the lter output to a nite number of levels. Figure 3(a) shows the frequency representation of a
2-level
color
dithered random vector which is obtained by feeding normalized Figure 2 shows one realization of colored random vector, i.i.d.
Gaussian noise to a band-pass lter with pass-band
±1.
0.05 < ω < 0.2
random vector and their corresponding frequency structure. The en-
and dithering the output to
ergy of the colored random vector is also concentrated between nor-
structure is still preserved after quantization. Furthermore, its ro-
malized frequency
0.05 < ω < 0.2.
It can be shown that the signal
can be exactly recovered with probability suit with only
M = 50
residual error is less than
p > 90% using Basis Pur-
colored random projections. The norm of 10−4 . On the other hand, simulation result
in [3] shows that with only
50
white Gaussian random projections,
It is clear that the desired frequency
bustness is improved. Note that the frequency content for
ω > 0.2 is
no longer close to zero. Figure 3(b) shows the ensemble average of different spectrum band of different color dithered projection vectors with 100 realizations for each band. A more appealing method to generate color dithered random
p > 90%, which is only achievable when the number of measurements is over 80.
vectors is to use halftoning green noise error diffusion structure [8]
To make our observations more convincible, next we choose an-
that green noise can generate aperiodic, uncorrelated structure with
the signal can not be recovered with probability
other band-pass FIR lter
h(n)
with pass-band
0.2 < ω < 0.3
which appears to be more exible and easy tuning. It is well known desired frequency concentration.
Figure 4 shows the structure of
which is outside the pass-band of the underlying signal. Using the
green noise error diffusion for color dithered random process.
colored random vectors generated from this lter, as expected, the
this structure,
signal can not be recovered with probability number of random projections is greater than
p ≥ 90% unless the 600. This illustrative
example shows clearly that frequency matching is very important for efcient recovery of the signal, especially band-pass signal.
In
h > 0 is called the hysteresis constant which denes
the degree of output dots clustering. Different degree of clustering corresponds to different frequency concentration. The spatial lter P a(n), with N n=1 a(n) = 1, is designed in such a way that the output dots are more likely to occur in clusters. b(n) is a normal error
0.9
quency structure of the
4
0.8
on the mid-frequency with a central frequency around
Ensemble average magnitude
3.5 3 Value
50th row.
4.5
2.5 2 1.5 1
0
0
0.1
0.2 0.3 Normalized frquency
0.4
random vector generation reduces to the design of a band-pass l-
0.6
ter with the same pass-band of the underlying signal or setting the 0.5
hysteresis constant
0.4
the signal is not known in advance. Similar to the matching lter
Red noise Yellow noise Green noise Blue noise 0
0.1
0.2 0.3 Normalized frquency
(a) Frequency representation of color (b) Ensemble dithered random vector.
h for the desired central frequency.
A more difcult situation arises when the spectrum content of
0.3
0.1
0.5
0.2.
When the frequency content of the underlying signal is known,
0.7
0.2
0.5
Note that the energy concentrates
average
0.4
design, the method we propose here is to use the correlation be0.5
tween the normalized colored random vectors and underlying signal to evaluate the frequency matching between them.
of
If the green noise error diffusion structure is used, we can vary
different
color dithered vectors in Fourier domain.
the hysteresis constant malized testing vectors
h at coarse scale and generate L ¿ M norti ∈ RN , i ∈ {1, 2, . . . , L} that covers all
the possible frequency range of interest. Next we project the underFig. 3. Frequency spectrum of color dithered vectors.
lying signal
s
on those testing vectors. The measurement gives a
rough description of the frequency content of the signal of interest. h
The absolute value of these projections are evaluated and the value
a(n)
of + +
x(n)
corresponding to the maximal
ments, we can tune the hysterolysis constant
h around hmax
to gen-
erate more color dithered random vectors with similar frequency
-+
+
hmax = maxi∈{1,2,...,L} |hti , si|
correlation is located. To dene the random basis for CS measure-
y(n)
structure for measurements. In practice, we can use e(n)
N ×N
red-to-
blue color dithered matrix for signal acquisition and recovery. Since
b(n)
the number of measurements
M ≤ N,
only a small portion of the
rows that have consecutive row indexes need to be used. Although there are central frequency shifting within these rows, the require-
Fig. 4. Error Diffusion for color dithered random process.
ments of frequency matching can be approximately satised. The colored random projections generated are suitable for the diffusion lter that tries to spread out the errors as homogeneously
CS when Basis Pursuit algorithm is used. It is not suitable for MP al-
as possible. The input to the green noise generator is a matrix with
gorithm if used directly. The reason is that MP prefers uncorrelation
uniform density
0 < p < 1 and each row of the output matrix is a de-
between the columns of the holographic basis
V.
The holographic
sired color dithered random vector. Note that the hysteresis constant
basis from color dithered random projections has more correlation
h
between its columns than that from i.i.d. random projections.
can be a function of row index
ri
leading to a tunable structure
Given signal statistics, one approach to apply computationally
in which the central frequency can vary for each row in the output
efcient MP algorithm in colored random projections based CS is
matrix.
to perform unitary transform approximations [9] of the red-to-blue h=0
color dithered matrix. Given the autocorrelation matrix
40
input signal and the trix
A,
the desired unitary matrix is given by
30
Q1
and
Q2
Value
of the
B = Q1 Q2 ,
where
are the unitary matrices derived from the singular value
C = ARxx as C = Q1 ΛQ2 .
25
decomposition (SVD) of
20
property of the unitary transform approximation is that it preserves
15
the structure of the original matrix
5
though 0
An attractive
A on some basis [9] which makes A and Rxx , B also = (A − B)x. Al-
frequency matching projections feasible. Given H minimizes the error energy J = E[ε ε] for ε
10
0
0.1
0.2 0.3 Normalized frquency
0.4
0.5
(a) Color dithered matrix with fre- (b) Frequency spectrum of a green quency shifting.
Rxx
dimensional red-to-blue color dithered ma-
35
50th
h=4
N
B
is not a halftone matrix, it makes the application of MP
algorithm in colored random projections possible. Note that the correlation matrix
Rxx
can be estimated adaptively.
random vector.
4. SIMULATION RESULTS Fig. 5. Color dithered measurement vector generated by error diffusion.
In this section, several simulations are presented that illustrate the effectiveness of CS based on the color dithered projections. In simu-
Figure 5(a) shows a
256 × 256
color dithered matrix created
by the green noise error diffusion with frequency shifting from high
128-point, band-pass sparse signal 0.21 < ω < 0.26. The signal is sparse in the frequency domain with sparsity T = 10. The approach
lation 1, the signal of interest is a s ∈ RN with a frequency band
to low along the rows as h increases from 0 to 4. Note that for h = 0, high frequency components are present. As h increases, the
to estimate the frequency content of the signal is used to locate the
central frequency shifts to the lower end. The entries are dithered
used for random projections. The reconstruction signal is computed
to
1(white)
or
−1(black).
central row index
hmax of the red-to-blue color dithered matrix to be
Such a matrix is called red-to-blue color
via Basis Pursuit [4]. Figure 6(a) shows the relationship between the
dithered matrix. As an illustrative example, Fig. 5(b) shows the fre-
number of measurements and the the possibility of successful signal
10 less mea-
1 0.9
surements are achieved by color dithered random projections for the
0.8
same probability of successful reconstruction compared with i.i.d.
0.7
random projections. Note that when the signal length
0.6
more reduction in the number of measurements is expected.
Probability of successful reconstruction.
0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 20
30 40 50 Number of Measurements.
60
0.4
1
0.3
0.9
0.2 0.1
Color dithered projeciton Bernoulli random projection
0 10
Frequency matching projeciton Bernoulli random projection
0 10
70
20
30 40 Number of Measurements.
50
60
(a) Probability of successful recon- (b) Probability of successful reconstruction using Basis Pursuit.
N
increases,
0.5
struction using Matching Pursuit.
Probability of successful reconstruction.
Probability of successful reconstruction.
The simulation results is shown in Fig. 8. On average,
1 0.9
Fig. 6. Signal reconstruction via colored random projections.
0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 45
Frequency matching projeciton Bernoulli random projection 50
55
60 65 70 75 Number of Measurements.
80
85
90
reconstruction. For comparative purpose, the probability of successful reconstruction as a function of the number of i.i.d.
Bernoulli
random projections is also shown in Fig. 6(a). At each data point for
1000 trials are performed and the reconstruction proba1000 trials that results in success. A trial is successful when the error ε = s − s ˆ between the original signal s and the reconstructed signal sˆ has its norm kεk2 ≤ 0.01ksk2 . As
Fig. 8. Probability of successful reconstruction of signal 'Blocks'.
simulation,
5. CONCLUSION
bility is the fraction of the
can be seen from Fig. 6(a), color dithered random projections reduce notably the number of necessary measurements. In the second simulation, the signal autocorrelation matrix
Rxx
is known and MP algorithm is used for signal reconstruction. The P4 input signal is modelled by the statistic model s(n) = j=0 αj sin(2πn(j +1+βj )/128), where random variable αj ∼ U nif orm
(−0.5, 0.5)
and
βj ∼ Bernuli{0, 1}.
The signal is band-limited
It has been shown that when there is a frequency matching between the colored random projection vector and the signal, the sparse signal can be recovered with less measurements compared with i.i.d. random projections. This concepts can be extended to the cases where there is strong correlation between the random projection vector and the signal to be measured. In the future, we will extend our current work to 2-D signals and use the similar idea to reconstruct compressible 2-D signal via colored random projections.
0 < ω < 0.1. Uni128 × 128 red-to-blue ma-
and has its main frequency components within tary matrix transform is performed on an
6. REFERENCES
trix and the resultant unitary matrix is used for color random projections. Figure 6(b) shows the simulation results. It is clear that
[1] Emmanuel Candt´ es, Justin Romberg, and Terence Tao, Robust
Matching Pursuit can be as efcient as Basis Pursuit in CS via col-
uncertainty principles: Exact signal reconstruction from highly
ored random projections.
incomplete frequency information, IEEE Trans on Information Theory, vol. 52, no. 2, pp. 489509, Feb 2006.
Frequency domain representation 6
200
[2] David Donoho, Compressed sensing, IEEE Trans on Infor-
150
5
100
mation Theory, vol. 52, no. 4, pp. 12891306, April 2006.
4 50 3 Value
0
20
40
60
80
100
120
2
mation Via Orthogonal Matching Pursuit, preprint, 2005.
Haar wavelet domain representation. 20
1
[4] Emmanuel Cand´ es and Justin Romberg, L1-magic: Recovery of
10 0 0 −1 −2
sparse signals via convex programming, October 2005.
−10
20
40
60 Time
80
100
(a) Signal 'Blocks'.
−20
120
[3] J. Tropp and A. Gilbert, Signal Recovery From Partial Infor-
20
40
60
80
100
120
[5] M. Duarte, M. Davenport, M. Wakin, and R. Baraniuk, Sparse signal detection from incoherent projections, in Proc. of IEEE
(b)
Representation
of
signal
'Blocks' in Fourier domain and Haar wavelet domain.
ICASSP, 2006. [6] Shlomo Levy and Peter K. Fullagar, Reconstruction of a sparse spike train from a postion of its spectrum and application to
Fig. 7. Signal 'Blocks' and its transform representation. In the third simulation, the objective is to show that color dithered frequency matching is effective even if the signal is not sparse in the frequency domain, as long as it still has energy concentration on some frequency band. The input is the
128-point standard piecewise
constant signal 'Blocks', as shown in Fig. 7(a). Its Fourier domain representation and Haar wavelet domain representation are shown in Fig. 7(b). Note that a 6-level Haar wavelet transformation yields a sparse representation in the wavelet domain and most of its energy is in the low frequency. Color dithered random projections are used for measurements and Basis Pursuit is used for signal reconstruction.
high-resolution deconvolution, Geophysics, vol. 46, pp. 1235 1243, 1981. [7] J. Tropp, M. Wakin, M. Duarte, D. Baron, and R. Baraniuk, Random lters for compressive sampling and reconstruction, in Proc. of IEEE ICASSP, Toulouse, France, May 2006. [8] Daniel L. Lau and Gonzalo R. Arce, Modern Digital Halftoning, Mrcel Dekker, Inc, 2001. [9] Ricardo L. de Queiroz, maitons,
On Unitary Transform Approxi-
IEEE Signal Processing Letters, vol. 5, no. 2, pp.
4647, Feb 1998.
† † ‡ Gonzalo R. Arce , Zhongmin Wang and Jose L. Paredes †
Department of Electrical and Computer Engineering University of Delaware, Newark, DE, USA, 19716 ‡ Electrical Engineering Department, University of Los Andes, M´ erida, Venezuela 5101
email: [email protected], [email protected], [email protected]
problem [4]. Matching Pursuit (MP) and Orthogonal Matching Pur-
ABSTRACT This papers discusses the reconstruction of sparse signal from a few of incoherent projections based on the theory of Compressed Sensing (CS) when some a priori information about the frequency structure of the signal is available. A new method to design the random projections is introduced that achieves a notably reduced number of measurements while at the same time increases the probability of successful signal reconstruction.
The essential improvement is
achieved by using colored random projections instead of i.i.d. random projections for the measurements. The proposed approach uses color dithered random matrix as the random measurement matrix since it can achieve desired frequency structure that matches the frequency content of the underlying signal and also has simple spatial form that makes the hardware implementation very convenient. Computationally efcient algorithms to design such color dithered measurement matrix have been developed. Simulation results show the effectiveness of the proposed method.
suit (OMP) have also be proposed in the literature [3, 5]. the sparse signal.
CS relies on the only assumption that the underlying signals are sparse on some basis
Index Terms Compressed Sensing, sparse signal, colored noise,
Ψ and that the measurements are i.i.d.
random
projections [2]. In practice, many signals not only are sparse, but also have specic structures in basis other than
Ψ.
For example, a
signal sparse in the wavelet domain may have most of its energy concentrated on the low frequency band. Can we design more efcient random projections by exploiting the characteristics of the signal so that the signal can be reconstructed with higher probability using less measurements? This paper addresses this problem in detail. Our research is motivated by the fact that if we have enough knowledge about the spectrum characteristics of the signal, we can recover the signal with much less measurements [6]. Actually, if the signal is these
color dithered random projection.
These
methods are based on greedy algorithms that iteratively reconstruct
T
T
sparse in the frequency domain and if the locations of
frequency components are known, then we can recover the
signal with only
T
random projections or samples [6]. On the other
hand, it has been shown that passing the signal through a lter with random coefcients has the similar effect of making multiple projections of the signal onto random vectors in CS [7]. If we project the
1. INTRODUCTION
signal to random vectors that have band-pass characteristics (colored
Compressed sensing, initially proposed in [1, 2, 3], provides a new way to acquire and represent sparse signals that requires less sam-
random vectors) multiple times, it would be equivalent to get measurements by ltering the signal through a colored random lter and
the dominate frequency components of the signal can be enhanced pling resource and computational capability than traditional approaches. N in the measurements. Thus, it is more likely that less projections Given a T sparse signal x ∈ R on some basis Ψ = [ψ1 , ψ2 , . . . , ψN ], are needed to capture the salient information of the underlying sigthat is, x can be approximated by a linear combination of T vectors PT nal. Colored random projections could help reduce the search space from Ψ, ie, x ≈ i=1 θni ψni with T ¿ N , then the theory of to nd the sparse components of the signal in the Basis Pursuit alCompressed Sensing shows that x can be recovered from M rangorithm. Furthermore, an exact knowledge of the signal frequency dom projections with high probability when M = CT log N ¿ N , spectrum is not required for colored random projections. where C ≥ 1 is the oversampling factor. The projections are given Random projection vector that has band-pass characteristics and by y = Φx, where Φ is a M × N random measurement matrix with its rows incoherent with the columns of
Ψ.
Commonly used ran-
also can be easily implemented via hardware is preferred. This moti-
dom measurement matrices for CS are random Gaussian matrices √
vates the application of color dithered random vectors for CS. Multi-
and partial Fourier matrices. In [2], it is shown that a matrix satisfy-
digital halftoning methods [8].
(Φij
∈ {N (0, 1/N )),
Rademacher matrices (Φij
∈ {±1/ N })
level color dithered random vectors can also be easily generated by
ing the incoherent condition is so ubiquitous that nearly all matrices are CS matrices.
l1 norm minimization problem: min kθk1 subject to ΦΨθ = y, if M ≥ CT log N . V = ΦΨ is called the holographic basis. Minimizing the l1 norm
2. COLORED RANDOM PROJECTIONS
Signal reconstruction is achieved by solving a
2.1. Motivating experiments
1024-point band-pass input signal x(n) as shown in Fig. 1
yields solutions that are zero except at a small number of isolated val-
Suppose a
ues and can be solved by efcient linear programming algorithms.
is to be reconstructed via multiple random projections. Its frequency
One method called Basis Pursuit is direct l1 minimization method
components are also shown in Fig. 1 where the pass-band of the
using, for instance, interior-point method to solve the optimization
signal lies between normalized frequency
0.07
and
0.2.
Note that
the signal has a sparsity
1024-point
T = 10 in the frequency domain.
A set of
random vectors are generated by ltering a normalized
24-tap band-pass FIR lter h(n) which has a pass-band 0.05 < ω < 0.2. The output of the lter is colored white Gaussian noise with a
random noise that is not evenly distributed in the frequency domain and the noise values are correlated.
concentrated on its pass-band, it spreads over the whole spectrum. This property can be enhanced if the random vector is dithered with limited levels, as will be discussed shortly. Therefore, it is easy to incoherent with any xed basis
10
Ψ.
By matching the frequency components of the colored random
5
Value
First note that although the energy of the colored random vector is
construct universal colored random measurement matrix so that it is
15
0
vector and the underlying signal, the major frequency components
−5
of the signal are enhanced in the measurements and fewer measurements would sufce to contain all the salient information of the sig-
−10 −15
100
200
300
400
500 600 Time
700
800
900
1000
nal of interest. Furthermore, the energy of the vectors can be evaluated by integrating over their spectrum. If the peak amplitude of the
3000
colored random vectors in the pass-band is the same as the amplitude of the i.i.d. random vectors in the frequency domain, then it is
2000
Value
2.2. Frequency matching expedites signal recovery
easy to see that colored random projections also lead to less energy consumption in the real applications.
1000
From the signal recovery point of view, colored random projec-
0 −0.5
0 Normalized frquency
0.5
tion reduces the search space in the optimization by Basis Pursuit
Φx = y , the opkθk1 by searching the space expanded by the rows of measurement matrix Φ. If we study the prob-
algorithm. More precisely, given the measurement timization process tries to minimize Fig. 1. Test signal and its spectrum representation.
lem in the frequency domain, then it is clear that the solution only exists in the space expanded approximately by the major frequency components of the colored random vectors and the cost function deColored random vector
Frequency structure of colored random vector 80
1.5 1
Value
Value
0.5 0 −0.5 −1 −1.5
3
projections with i.i.d. random projections, we know that i.i.d. ran-
20
dom projections, although more universal and robust, is not efcient in that it proves a much larger searching space.
400 600 800 1000 0 0.1 0.2 0.3 0.4 Time Normalized frquency i.i.d. random vector Frequency structure of the i.i.d. random vector 80
3. COLORED RANDOM MEASUREMENT MATRIX
60 Value
Value
covery the signal with high probability. Comparing colored random
40
200
1 0 −1 −2 −3
That is the reason why much less measurements are sufcient to re-
60
0
2
creases rapidly in the direction where these major components exist.
200
400 600 Time
40
The colored random projection is effective in the CS framework and
20
is simple to generate.
0
800 1000
However, their spectrum magnitude on the
stop-band may be close to 0
0.1 0.2 0.3 0.4 Normalized frquency
0
leading to singularity problem in the
signal reconstruction. This problem can be alleviated if we quantize the colored random vectors. The quantized colored random vectors can be generated by feeding normalized Gaussian noise to a band-
Fig. 2. Colored random vectors and their spectrum representations.
pass lter and quantizing the lter output to a nite number of levels. Figure 3(a) shows the frequency representation of a
2-level
color
dithered random vector which is obtained by feeding normalized Figure 2 shows one realization of colored random vector, i.i.d.
Gaussian noise to a band-pass lter with pass-band
±1.
0.05 < ω < 0.2
random vector and their corresponding frequency structure. The en-
and dithering the output to
ergy of the colored random vector is also concentrated between nor-
structure is still preserved after quantization. Furthermore, its ro-
malized frequency
0.05 < ω < 0.2.
It can be shown that the signal
can be exactly recovered with probability suit with only
M = 50
residual error is less than
p > 90% using Basis Pur-
colored random projections. The norm of 10−4 . On the other hand, simulation result
in [3] shows that with only
50
white Gaussian random projections,
It is clear that the desired frequency
bustness is improved. Note that the frequency content for
ω > 0.2 is
no longer close to zero. Figure 3(b) shows the ensemble average of different spectrum band of different color dithered projection vectors with 100 realizations for each band. A more appealing method to generate color dithered random
p > 90%, which is only achievable when the number of measurements is over 80.
vectors is to use halftoning green noise error diffusion structure [8]
To make our observations more convincible, next we choose an-
that green noise can generate aperiodic, uncorrelated structure with
the signal can not be recovered with probability
other band-pass FIR lter
h(n)
with pass-band
0.2 < ω < 0.3
which appears to be more exible and easy tuning. It is well known desired frequency concentration.
Figure 4 shows the structure of
which is outside the pass-band of the underlying signal. Using the
green noise error diffusion for color dithered random process.
colored random vectors generated from this lter, as expected, the
this structure,
signal can not be recovered with probability number of random projections is greater than
p ≥ 90% unless the 600. This illustrative
example shows clearly that frequency matching is very important for efcient recovery of the signal, especially band-pass signal.
In
h > 0 is called the hysteresis constant which denes
the degree of output dots clustering. Different degree of clustering corresponds to different frequency concentration. The spatial lter P a(n), with N n=1 a(n) = 1, is designed in such a way that the output dots are more likely to occur in clusters. b(n) is a normal error
0.9
quency structure of the
4
0.8
on the mid-frequency with a central frequency around
Ensemble average magnitude
3.5 3 Value
50th row.
4.5
2.5 2 1.5 1
0
0
0.1
0.2 0.3 Normalized frquency
0.4
random vector generation reduces to the design of a band-pass l-
0.6
ter with the same pass-band of the underlying signal or setting the 0.5
hysteresis constant
0.4
the signal is not known in advance. Similar to the matching lter
Red noise Yellow noise Green noise Blue noise 0
0.1
0.2 0.3 Normalized frquency
(a) Frequency representation of color (b) Ensemble dithered random vector.
h for the desired central frequency.
A more difcult situation arises when the spectrum content of
0.3
0.1
0.5
0.2.
When the frequency content of the underlying signal is known,
0.7
0.2
0.5
Note that the energy concentrates
average
0.4
design, the method we propose here is to use the correlation be0.5
tween the normalized colored random vectors and underlying signal to evaluate the frequency matching between them.
of
If the green noise error diffusion structure is used, we can vary
different
color dithered vectors in Fourier domain.
the hysteresis constant malized testing vectors
h at coarse scale and generate L ¿ M norti ∈ RN , i ∈ {1, 2, . . . , L} that covers all
the possible frequency range of interest. Next we project the underFig. 3. Frequency spectrum of color dithered vectors.
lying signal
s
on those testing vectors. The measurement gives a
rough description of the frequency content of the signal of interest. h
The absolute value of these projections are evaluated and the value
a(n)
of + +
x(n)
corresponding to the maximal
ments, we can tune the hysterolysis constant
h around hmax
to gen-
erate more color dithered random vectors with similar frequency
-+
+
hmax = maxi∈{1,2,...,L} |hti , si|
correlation is located. To dene the random basis for CS measure-
y(n)
structure for measurements. In practice, we can use e(n)
N ×N
red-to-
blue color dithered matrix for signal acquisition and recovery. Since
b(n)
the number of measurements
M ≤ N,
only a small portion of the
rows that have consecutive row indexes need to be used. Although there are central frequency shifting within these rows, the require-
Fig. 4. Error Diffusion for color dithered random process.
ments of frequency matching can be approximately satised. The colored random projections generated are suitable for the diffusion lter that tries to spread out the errors as homogeneously
CS when Basis Pursuit algorithm is used. It is not suitable for MP al-
as possible. The input to the green noise generator is a matrix with
gorithm if used directly. The reason is that MP prefers uncorrelation
uniform density
0 < p < 1 and each row of the output matrix is a de-
between the columns of the holographic basis
V.
The holographic
sired color dithered random vector. Note that the hysteresis constant
basis from color dithered random projections has more correlation
h
between its columns than that from i.i.d. random projections.
can be a function of row index
ri
leading to a tunable structure
Given signal statistics, one approach to apply computationally
in which the central frequency can vary for each row in the output
efcient MP algorithm in colored random projections based CS is
matrix.
to perform unitary transform approximations [9] of the red-to-blue h=0
color dithered matrix. Given the autocorrelation matrix
40
input signal and the trix
A,
the desired unitary matrix is given by
30
Q1
and
Q2
Value
of the
B = Q1 Q2 ,
where
are the unitary matrices derived from the singular value
C = ARxx as C = Q1 ΛQ2 .
25
decomposition (SVD) of
20
property of the unitary transform approximation is that it preserves
15
the structure of the original matrix
5
though 0
An attractive
A on some basis [9] which makes A and Rxx , B also = (A − B)x. Al-
frequency matching projections feasible. Given H minimizes the error energy J = E[ε ε] for ε
10
0
0.1
0.2 0.3 Normalized frquency
0.4
0.5
(a) Color dithered matrix with fre- (b) Frequency spectrum of a green quency shifting.
Rxx
dimensional red-to-blue color dithered ma-
35
50th
h=4
N
B
is not a halftone matrix, it makes the application of MP
algorithm in colored random projections possible. Note that the correlation matrix
Rxx
can be estimated adaptively.
random vector.
4. SIMULATION RESULTS Fig. 5. Color dithered measurement vector generated by error diffusion.
In this section, several simulations are presented that illustrate the effectiveness of CS based on the color dithered projections. In simu-
Figure 5(a) shows a
256 × 256
color dithered matrix created
by the green noise error diffusion with frequency shifting from high
128-point, band-pass sparse signal 0.21 < ω < 0.26. The signal is sparse in the frequency domain with sparsity T = 10. The approach
lation 1, the signal of interest is a s ∈ RN with a frequency band
to low along the rows as h increases from 0 to 4. Note that for h = 0, high frequency components are present. As h increases, the
to estimate the frequency content of the signal is used to locate the
central frequency shifts to the lower end. The entries are dithered
used for random projections. The reconstruction signal is computed
to
1(white)
or
−1(black).
central row index
hmax of the red-to-blue color dithered matrix to be
Such a matrix is called red-to-blue color
via Basis Pursuit [4]. Figure 6(a) shows the relationship between the
dithered matrix. As an illustrative example, Fig. 5(b) shows the fre-
number of measurements and the the possibility of successful signal
10 less mea-
1 0.9
surements are achieved by color dithered random projections for the
0.8
same probability of successful reconstruction compared with i.i.d.
0.7
random projections. Note that when the signal length
0.6
more reduction in the number of measurements is expected.
Probability of successful reconstruction.
0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 20
30 40 50 Number of Measurements.
60
0.4
1
0.3
0.9
0.2 0.1
Color dithered projeciton Bernoulli random projection
0 10
Frequency matching projeciton Bernoulli random projection
0 10
70
20
30 40 Number of Measurements.
50
60
(a) Probability of successful recon- (b) Probability of successful reconstruction using Basis Pursuit.
N
increases,
0.5
struction using Matching Pursuit.
Probability of successful reconstruction.
Probability of successful reconstruction.
The simulation results is shown in Fig. 8. On average,
1 0.9
Fig. 6. Signal reconstruction via colored random projections.
0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 45
Frequency matching projeciton Bernoulli random projection 50
55
60 65 70 75 Number of Measurements.
80
85
90
reconstruction. For comparative purpose, the probability of successful reconstruction as a function of the number of i.i.d.
Bernoulli
random projections is also shown in Fig. 6(a). At each data point for
1000 trials are performed and the reconstruction proba1000 trials that results in success. A trial is successful when the error ε = s − s ˆ between the original signal s and the reconstructed signal sˆ has its norm kεk2 ≤ 0.01ksk2 . As
Fig. 8. Probability of successful reconstruction of signal 'Blocks'.
simulation,
5. CONCLUSION
bility is the fraction of the
can be seen from Fig. 6(a), color dithered random projections reduce notably the number of necessary measurements. In the second simulation, the signal autocorrelation matrix
Rxx
is known and MP algorithm is used for signal reconstruction. The P4 input signal is modelled by the statistic model s(n) = j=0 αj sin(2πn(j +1+βj )/128), where random variable αj ∼ U nif orm
(−0.5, 0.5)
and
βj ∼ Bernuli{0, 1}.
The signal is band-limited
It has been shown that when there is a frequency matching between the colored random projection vector and the signal, the sparse signal can be recovered with less measurements compared with i.i.d. random projections. This concepts can be extended to the cases where there is strong correlation between the random projection vector and the signal to be measured. In the future, we will extend our current work to 2-D signals and use the similar idea to reconstruct compressible 2-D signal via colored random projections.
0 < ω < 0.1. Uni128 × 128 red-to-blue ma-
and has its main frequency components within tary matrix transform is performed on an
6. REFERENCES
trix and the resultant unitary matrix is used for color random projections. Figure 6(b) shows the simulation results. It is clear that
[1] Emmanuel Candt´ es, Justin Romberg, and Terence Tao, Robust
Matching Pursuit can be as efcient as Basis Pursuit in CS via col-
uncertainty principles: Exact signal reconstruction from highly
ored random projections.
incomplete frequency information, IEEE Trans on Information Theory, vol. 52, no. 2, pp. 489509, Feb 2006.
Frequency domain representation 6
200
[2] David Donoho, Compressed sensing, IEEE Trans on Infor-
150
5
100
mation Theory, vol. 52, no. 4, pp. 12891306, April 2006.
4 50 3 Value
0
20
40
60
80
100
120
2
mation Via Orthogonal Matching Pursuit, preprint, 2005.
Haar wavelet domain representation. 20
1
[4] Emmanuel Cand´ es and Justin Romberg, L1-magic: Recovery of
10 0 0 −1 −2
sparse signals via convex programming, October 2005.
−10
20
40
60 Time
80
100
(a) Signal 'Blocks'.
−20
120
[3] J. Tropp and A. Gilbert, Signal Recovery From Partial Infor-
20
40
60
80
100
120
[5] M. Duarte, M. Davenport, M. Wakin, and R. Baraniuk, Sparse signal detection from incoherent projections, in Proc. of IEEE
(b)
Representation
of
signal
'Blocks' in Fourier domain and Haar wavelet domain.
ICASSP, 2006. [6] Shlomo Levy and Peter K. Fullagar, Reconstruction of a sparse spike train from a postion of its spectrum and application to
Fig. 7. Signal 'Blocks' and its transform representation. In the third simulation, the objective is to show that color dithered frequency matching is effective even if the signal is not sparse in the frequency domain, as long as it still has energy concentration on some frequency band. The input is the
128-point standard piecewise
constant signal 'Blocks', as shown in Fig. 7(a). Its Fourier domain representation and Haar wavelet domain representation are shown in Fig. 7(b). Note that a 6-level Haar wavelet transformation yields a sparse representation in the wavelet domain and most of its energy is in the low frequency. Color dithered random projections are used for measurements and Basis Pursuit is used for signal reconstruction.
high-resolution deconvolution, Geophysics, vol. 46, pp. 1235 1243, 1981. [7] J. Tropp, M. Wakin, M. Duarte, D. Baron, and R. Baraniuk, Random lters for compressive sampling and reconstruction, in Proc. of IEEE ICASSP, Toulouse, France, May 2006. [8] Daniel L. Lau and Gonzalo R. Arce, Modern Digital Halftoning, Mrcel Dekker, Inc, 2001. [9] Ricardo L. de Queiroz, maitons,
On Unitary Transform Approxi-
IEEE Signal Processing Letters, vol. 5, no. 2, pp.
4647, Feb 1998.
