Sign-Aware Distributed Approximate Message Passing
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
Sign-Aware Distributed Approximate Message Passing
Puxiao Han and Ruixin Niu Department of Electrical and Computer Engineering Virginia Commonwealth University, Richmond, VA
August 10, 2016
1 / 15
Outline
Introduction
1
Introduction
2
DiAMP
3
GC Algorithm in DiAMP
DiAMP
GC Algorithm in DiAMP
2 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
Motivation of Distributed Compressed Sensing (CS)
CS is facing challenges: Due to the curse of dimensionality, it can be highly demanding to perform CS on a single processor.
Distributed CS (DCS) has the potential to overcome these problems: Each distributed sensor has much less data to process compared to the centralized case. Distributed data querying algorithms can be applied to reduce the communication cost in data transmission.
3 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
CCS (left) vs. DCS (right) in a Sensor Network intermediate data
w t2
on
Sensor 2
Sensor 1
Exchange
Ite rat ive com pu (y 1 , A 1 tatio n ,x t)
(y 2 , A2 )
w1t
x*
i tat pu com ) ive 2 , x t rat 2 , A Ite (y
(y 1 , A1 )
Center
y1 A1 y 2 ,A 2 y A CS Recovery
xt+1
Sensor 2
A 2s
1
e
1 s 0+
=A
1
e
y 2=
1
1 s 0+
y
=A
y2 =A2 s0 + e2
1
y
0 +e 2
Sensor 1
Sparse signal s0 Sparse signal s0 4 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
Centralized AMP Algorithm Given y and A, AMP repeatedly updates an estimate xt of s0 : xt+1 = η(xt + AT zt ; τ σt ) zt+1 = y − Axt+1 +
kxt+1 k0 zt M
σt2 is the element-wise MSE of xt + AT zt − s0 , and is replaced by its estimator σ ˆt2 =
kzt k22 M
if the distribution of s0 is unknown.
τ : a positive parameter that needs to be tuned. η(x; β) is the soft thresholding function: x − β, η(x; β) = x + β, 0
x > β; x < −β; |x| ≤ β 5 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
The Distributed AMP (DiAMP) Algorithm Partition the sensing matrix A by rows: A1 . A = .. AP
Local Computation (LC) zpt = yp −Ap xt +
wtp =
kxt k0 p z , ∀p = 1, · · · , P M t−1
xt + (Ap )T zpt
p=1
(Ap )T zp
o.w.
t
Partition the measurement: y1 A1 e1 .. .. . . = . s0 + .. P P y A eP Global Computation (GC) v u P uX p σ ˆt = t kzt k22 /M p=1
ft =
P X
wtp
p=1
xt+1 = η (ft ; τ σ ˆt )
6 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
New DiAMP Framework Gaussianity in DiAMP [1] Define ωp = Mp /M, where Mp is # of rows in Ap , if we change wtp into wtp = ωp xt + (Ap )T zpt , numerical results show that wtp − ωp s0 is N (0, ωp σt2 ).
A new estimator of σt2 Partition Ap equally by rows into Ap,1 and Ap,2 , wtp,1 − (ωp /2)s0 = (ωp /2)(xt − s0 ) + (Ap,1 )T zp,1 ∼ N (0, ωp σt2 /2), t wtp,2 − (ωp /2)s0 = (ωp /2)(xt − s0 ) + (Ap,2 )T zp,2 ∼ N (0, ωp σt2 /2), t wtp,1 − wtp,2 ∼ N (0, ωp σt2 ). We know wtp,1 and wtp,2 , so we can obtain σ ˆt2 . [1] Han and Niu, State Evolution for DiAMP, near completion and to be submitted to IEEE T-IT, 2016. 7 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
Global Computation of AMP (GCAMP) Only the global computation step requires data exchange The naive approach: sending wtp from all sensors p ≥ 2 to Sensor 1 This approach requires N(P − 1) messages. Sending all the data is unnecessary: we only need to find all the components in ft with magnitudes greater than βt = τ σ ˆt .
The essence of GCAMP Use a small amount of data to obtain an upper bound U(n) on |ft (n)|, ∀n = 1, · · · , N. For all the n’s that U(n) > βt , sending all the corresponding data from other sensors to Sensor 1 to compute ft (n).
Question: How to obtain U(n)? 8 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
Computation of U(n) Set a local threshold T > 0 for each Sensor p ≥ 2 Sensor p sends a data point (n, wtp (n)) to Sensor 1 only if |wtp (n)| > T . Finding: for all n ∈ {1, · · · , N} P P X X 1 p 1 U(n) = wt (n) + wt (n)I (|wt (n)| > T ) + T I (|wtp (n)| ≤ T ) p=2 p=2 is an upper bound on |ft (n)| according to triangular inequality.
How do we choose T ? Consider the case where all the data are positive (negative) but we do not know it in advance. If T ≥
βt , P−1
then U(n) > βt for all n ∈ {1, · · · , N}, i.e., we still need to
send all the data. βt Choose θ ∈ (0, 1), and set T = θ P−1 . 9 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
Remarks
GCAMP saves communication cost by a sifting process. The sifting is done by computing U(n) and comparing it with βt . A tighter U(n) means a better performance.
What constrains us in obtaining a tighter U(n)? The unawareness of signs of data, and the constraint θ < 1.
Solution: Add a sign-aware step before sending any data. By doing this, we actually use number of communication bits, instead of number of communication messages to evaluate communication cost.
10 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
Sign-Aware GC (SAGC) We obtain a range L R (p, n), R U (p, n) on
w tp ( n )
wtp (n).
w tp ( n ) > T
The new upper bound is P X U(n) = max RU (p, n), p=1
P X − RL (p, n) . p=1
N
w tp ( n ) ≥ 0
Y
sign-awareness N
Y R
L
U
R
( p, n ) = w ( n ) ( p, n ) = w tp ( n ) p t
R
L
U
R
( p, n ) = 0 ( p, n ) = T
R L ( p, n ) = −T RU ( p, n ) = 0
L U R ( p, n ) , R ( p, n )
11 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
Numerical Results
N = 10, 000, κ ∈ [0.2, 0.4], ρ ∈ [0.1, 0.2], M = Nκ, K = Mρ. s0 is with random support, and its non-zero entries follow i.i.d. N (0, 1). e ∼ N (0, σ 2 IM ) such that SNR = 10 log10 (ρ/σ 2 ) ∈ [10, 20] dB. θ = 0.8 for GCAMP and θ = 1.1 for SAGC. We evaluate the efficiency of GC algorithms in terms of µM =
# of Bits of GC algorithms # of Bits if sending all data
We compare the µM obtained from GCAMP and SAGC.
12 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
Performance of GCAMP and SAGC
ρ = 0.1,SNR = 20dB
ρ = 0.2,SNR = 20dB
κ
0.20
0.30
0.40
0.20
0.30
0.40
GCAMP (%)
42.02
44.11
45.60
53.29
55.57
57.70
SAGC (%)
32.25
34.99
36.95
42.34
45.43
48.23
ρ = 0.1,SNR = 10dB
ρ = 0.2,SNR = 10dB
κ
0.20
0.30
0.40
0.20
0.30
0.40
GCAMP (%)
43.77
45.77
47.57
53.23
55.13
57.03
SAGC (%)
34.23
36.97
39.27
42.59
45.34
47.92
13 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
Accuracy of Proposed σ ˆt2 2 σt,S : the value predicted by theory. 2 σt,E : empirical value of σt2 . 1
|σ ! t2 − σ t2, S | /σ t2, S
|σ ! t2 − σ t2, E | /σ t2, E
0.9 0.8
E mp i r i c al C D F
0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Ab sol t e val u e of r e l at i v e e r r or of σ ! t2
0.9
1
14 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
Conclusion
A new estimator is proposed for σt2 in DiAMP. A new GC algorithm SAGC is proposed and outperforms GCAMP. The sign-awareness step yields a tighter upper bound U(n). Knowing the signs in advance makes the choice of θ more flexible.
15 / 15
Sensor Selection for Target Tracking in Sensor Networks Based on a Proximal Algorithm Xiaojun Yangǂ and Ruixin NiuƗ ǂ School of Information Engineering, Chang’an University Ɨ Department of ECE, Virginia Commonwealth University
August 10, 2016 InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
Motivation In a wireless sensor network, it is desirable to select a small subset of sensors due to limited communication bandwidth and sensor battery power. Adaptive sensor selection: to attain an optimal tradeoff between estimation accuracy and sensing & communication costs. Sensor selection is an integer programming problem with combinatorial complexity sparsity-promoting techniques with polynomial complexities.
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
1
Related work A convex optimization procedure was developed to solve the sensor selection problem in [1]. In [2], Kalman filter gain matrix was obtained as the solution to an optimization problem where a sparsity-promoting penalty function is added to the cost function. These methods are limited to linear estimators in which the optimized variable is the Kalman filter gain or linear estimator gain. Sparsity-promoting sensor selection problem was solved in [3] for parameter estimation problem. [1] S. Joshi and S. Boyd, “Sensor selection via convex optimization,” IEEE Transactions on Signal Processing, vol. 57, pp. 451–462, Feb. 2009. [2] E. Masazade, M. Fardad, and P. Varshney, “Sparsity-promoting extended kalman filtering for target tracking in wireless sensor networks,” IEEE Signal Processing Letters, vol. 19, pp. 845–848, Dec. 2012. [3] S. Chepuri and G. Leus, “Sparsity-promoting sensor selection for nonlinear measurement models,” IEEE Transactions on Signal Processing, vol. 63, pp. 684–698, Feb. 2015. InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
2
Problem Formulation Track a moving target in a WSN, with the following models
x t 1 f t ( x t ) v t zi ,t hi ,t ( x t ) nt , i 1, 2, ..., N
Sensor selection based on feedback from a nonlinear filter InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
3
Problem Formulation
Sensor selection strategy at time t is determined by a selection vector w t [ w1,t , w2,t , ..., wN ,t ]T
where the elements are binary, i.e., wi,t = 1 if the i-th sensor is selected; wi,t = 0, otherwise.
Conditional PCRLB (Zuo, Niu, and Varsheny, IEEE T-SP’11) provides a lower bound on the mean squared error (MSE) of the target state estimate, conditioned on the measurements up to the current time.
E [xˆ t 1 x t 1 ][xˆ t 1 x t 1 ]T z1:t J t 11 ( x t 1 z1:t ) Conditional Fisher information matrix (FIM) at time step t: N
J t ( w t ) wi ,t J iZ,t J tX i 1
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
4
Problem Formulation Sensor selection is formulated as an optimization problem by promoting the sparsity of selection vector.
w*t arg min ( w t ) w t wt
0
s.t . w t {0, 1}N ψ(wt): performance measure; ||wt||0: sparsity-promoting penalty function; γ characterizes the relative emphasis of accurate estimation versus selecting a small number of sensors.
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
5
l1 -Norm Minimization Algorithm Replacing non-convex Boolean constraints wi,t ∈ {0, 1} with convex box constraints wi,t ∈ [0, 1] and ℓ0 norm with ℓ1 norm, we have the following convex problem minimize ( w t ) w t wt
1
subjectto wi ,t 1,i 1, 2,
,N
This can be efficiently solved in polynomial time using interior-point methods (e.g. Newton’s method using an approximate logarithmic barrier function). InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
6
Accelerated Proximal Gradient Method If number of sensors is large, Newton’s method, which is a second order approach, is computationally inefficient proximal algorithm, a first-order method Rewrite the optimization problem as
min ( w t ) ( w t ) wt
where ϕ(wt) = ψ(wt)+γ ||wt||1, and φ(wt) is defined by an indicator function
0 ( w t )=
if 0 w1,t , ..., wN ,t 1 otherwise
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
7
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
8
APGM Algorithm In step 3, the proximal operator Proxλφ(y) of the scaled function λφ is given by 1 2 Prox ( y ) min ( x ) x y 2 x 2 For indicator function φ, the proximal operator of φ, i.e., Proxλφ, reduces to Euclidean projection onto convex box C ={0 ≤ w1,t, …, wN,t ≤1}
Prox ( v ) C v arg min x v xC
2
Projection above takes a simple form 0 vi 0 ( v ) C i vi 0 vi 1 1 v 1 i Windsor Locks, CT, August 2016 InfoSymbiotics/DDDAS Conference,
9
APGM Algorithm In Step 4, the function ˆ is defined as ˆ ( z, y ) ( y ) ( y )T ( z y ) (1 / 2 ) z y
2 2
Function ˆ can be interpreted as a quadratic approximation of ϕ(y) . Steps 3-5 constitute a linear search for step size λ(l) called backtracking.
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
10
Simulation Results 10 13
11
12
true target trajectory
8
l1 minimization APGM
6
sensor node
14
y-axis coordinates
4
2
0
15
7
16
6
-2
-4 -6
-8 1 -10 -10
3
2 -8
-6
-4
-2
0
4 2
4
5 6
8
10
x-axis coordinates
Fig.1 Sensor placement, target trajectory, and its estimate InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
11
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
12
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
13
Conclusion and Future Work •
• •
• •
We considered general scenario with nonlinear state and measurement models. The proposed sensor selection approach is not restricted to any specific estimator. We employ the conditional PCRLB as the cost function to obtain a trade-off between the estimation performance and the number of selected sensors. Based on a proximal algorithm, we developed a fast algorithm to handle large-scale sensor networks. We are developing alternating direction method of multipliers (ADMM) and distributed ADMM algorithms for large-scale sensor networks. InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
14
Overview of Other Ongoing Research Activities •
Joint Subspace Tracking and Manifold Learning for Fusing Multi-Modality Sensor Data
f:M1 M2
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
15
Other Research Activities •
Distributed Sparse Signal Recovery and Low-Rank Matrix Completion (with Han, Jowkar, Baron, Eldar, and Huie)
Message Approximate Passing (AMP) Message Passing (AMP)
[Donoho et al. 2009]
𝐳𝐳
[Donoho et al. 2009]
𝑀𝑀×𝑁𝑁 𝑡𝑡 𝐀 𝐀∈ ℝ ö s 𝒩 𝐰 𝜎𝑡𝑡2𝕀𝕀 𝐲𝐲 ( 𝐟𝐟) 𝐱𝐱 𝐰𝐱 𝐱æçè 0,∼ 𝐳 I𝐳÷ 𝒩0, 𝜎 P ø
𝑡𝑡T x ft = t + A p z tp s0 / P wtp ~ P
2
p
decouple
=
=
Node p
+
t
N
ft = å p=1𝑡𝑡 ft p
𝐟𝐟 P
𝐱s𝐱 0
wt ~
Uplink: send ft p
decouple
+
Downlink: broadcast
=
𝐰𝐰(𝑡𝑡0,s∼I )𝒩𝒩0, 𝜎𝜎𝑡𝑡2𝕀𝕀 2
t
N
+
x t+1 = h t (ft )
Fusion Center
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016 correction term correction
Iterate:
16 term
Other Research Activities • Fast Image Inpainting Based on Sparse Reconstruction
From left to right: original, damaged, and reconstructed images (with Almshaal) InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
Other Research Activities •
Secure Statistical Signal Processing in the Presence of False Information Attacks (with Lu) Analysis of impact of attacks on state estimation Attack detection/identification Game theoretic attack mitigation schemes
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
18
Other Research Activities •
Joint Group Testing of Time-Varying Faulty Sensors and State Estimation (with Ren)
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
19
Introduction
DiAMP
GC Algorithm in DiAMP
Sign-Aware Distributed Approximate Message Passing
Puxiao Han and Ruixin Niu Department of Electrical and Computer Engineering Virginia Commonwealth University, Richmond, VA
August 10, 2016
1 / 15
Outline
Introduction
1
Introduction
2
DiAMP
3
GC Algorithm in DiAMP
DiAMP
GC Algorithm in DiAMP
2 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
Motivation of Distributed Compressed Sensing (CS)
CS is facing challenges: Due to the curse of dimensionality, it can be highly demanding to perform CS on a single processor.
Distributed CS (DCS) has the potential to overcome these problems: Each distributed sensor has much less data to process compared to the centralized case. Distributed data querying algorithms can be applied to reduce the communication cost in data transmission.
3 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
CCS (left) vs. DCS (right) in a Sensor Network intermediate data
w t2
on
Sensor 2
Sensor 1
Exchange
Ite rat ive com pu (y 1 , A 1 tatio n ,x t)
(y 2 , A2 )
w1t
x*
i tat pu com ) ive 2 , x t rat 2 , A Ite (y
(y 1 , A1 )
Center
y1 A1 y 2 ,A 2 y A CS Recovery
xt+1
Sensor 2
A 2s
1
e
1 s 0+
=A
1
e
y 2=
1
1 s 0+
y
=A
y2 =A2 s0 + e2
1
y
0 +e 2
Sensor 1
Sparse signal s0 Sparse signal s0 4 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
Centralized AMP Algorithm Given y and A, AMP repeatedly updates an estimate xt of s0 : xt+1 = η(xt + AT zt ; τ σt ) zt+1 = y − Axt+1 +
kxt+1 k0 zt M
σt2 is the element-wise MSE of xt + AT zt − s0 , and is replaced by its estimator σ ˆt2 =
kzt k22 M
if the distribution of s0 is unknown.
τ : a positive parameter that needs to be tuned. η(x; β) is the soft thresholding function: x − β, η(x; β) = x + β, 0
x > β; x < −β; |x| ≤ β 5 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
The Distributed AMP (DiAMP) Algorithm Partition the sensing matrix A by rows: A1 . A = .. AP
Local Computation (LC) zpt = yp −Ap xt +
wtp =
kxt k0 p z , ∀p = 1, · · · , P M t−1
xt + (Ap )T zpt
p=1
(Ap )T zp
o.w.
t
Partition the measurement: y1 A1 e1 .. .. . . = . s0 + .. P P y A eP Global Computation (GC) v u P uX p σ ˆt = t kzt k22 /M p=1
ft =
P X
wtp
p=1
xt+1 = η (ft ; τ σ ˆt )
6 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
New DiAMP Framework Gaussianity in DiAMP [1] Define ωp = Mp /M, where Mp is # of rows in Ap , if we change wtp into wtp = ωp xt + (Ap )T zpt , numerical results show that wtp − ωp s0 is N (0, ωp σt2 ).
A new estimator of σt2 Partition Ap equally by rows into Ap,1 and Ap,2 , wtp,1 − (ωp /2)s0 = (ωp /2)(xt − s0 ) + (Ap,1 )T zp,1 ∼ N (0, ωp σt2 /2), t wtp,2 − (ωp /2)s0 = (ωp /2)(xt − s0 ) + (Ap,2 )T zp,2 ∼ N (0, ωp σt2 /2), t wtp,1 − wtp,2 ∼ N (0, ωp σt2 ). We know wtp,1 and wtp,2 , so we can obtain σ ˆt2 . [1] Han and Niu, State Evolution for DiAMP, near completion and to be submitted to IEEE T-IT, 2016. 7 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
Global Computation of AMP (GCAMP) Only the global computation step requires data exchange The naive approach: sending wtp from all sensors p ≥ 2 to Sensor 1 This approach requires N(P − 1) messages. Sending all the data is unnecessary: we only need to find all the components in ft with magnitudes greater than βt = τ σ ˆt .
The essence of GCAMP Use a small amount of data to obtain an upper bound U(n) on |ft (n)|, ∀n = 1, · · · , N. For all the n’s that U(n) > βt , sending all the corresponding data from other sensors to Sensor 1 to compute ft (n).
Question: How to obtain U(n)? 8 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
Computation of U(n) Set a local threshold T > 0 for each Sensor p ≥ 2 Sensor p sends a data point (n, wtp (n)) to Sensor 1 only if |wtp (n)| > T . Finding: for all n ∈ {1, · · · , N} P P X X 1 p 1 U(n) = wt (n) + wt (n)I (|wt (n)| > T ) + T I (|wtp (n)| ≤ T ) p=2 p=2 is an upper bound on |ft (n)| according to triangular inequality.
How do we choose T ? Consider the case where all the data are positive (negative) but we do not know it in advance. If T ≥
βt , P−1
then U(n) > βt for all n ∈ {1, · · · , N}, i.e., we still need to
send all the data. βt Choose θ ∈ (0, 1), and set T = θ P−1 . 9 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
Remarks
GCAMP saves communication cost by a sifting process. The sifting is done by computing U(n) and comparing it with βt . A tighter U(n) means a better performance.
What constrains us in obtaining a tighter U(n)? The unawareness of signs of data, and the constraint θ < 1.
Solution: Add a sign-aware step before sending any data. By doing this, we actually use number of communication bits, instead of number of communication messages to evaluate communication cost.
10 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
Sign-Aware GC (SAGC) We obtain a range L R (p, n), R U (p, n) on
w tp ( n )
wtp (n).
w tp ( n ) > T
The new upper bound is P X U(n) = max RU (p, n), p=1
P X − RL (p, n) . p=1
N
w tp ( n ) ≥ 0
Y
sign-awareness N
Y R
L
U
R
( p, n ) = w ( n ) ( p, n ) = w tp ( n ) p t
R
L
U
R
( p, n ) = 0 ( p, n ) = T
R L ( p, n ) = −T RU ( p, n ) = 0
L U R ( p, n ) , R ( p, n )
11 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
Numerical Results
N = 10, 000, κ ∈ [0.2, 0.4], ρ ∈ [0.1, 0.2], M = Nκ, K = Mρ. s0 is with random support, and its non-zero entries follow i.i.d. N (0, 1). e ∼ N (0, σ 2 IM ) such that SNR = 10 log10 (ρ/σ 2 ) ∈ [10, 20] dB. θ = 0.8 for GCAMP and θ = 1.1 for SAGC. We evaluate the efficiency of GC algorithms in terms of µM =
# of Bits of GC algorithms # of Bits if sending all data
We compare the µM obtained from GCAMP and SAGC.
12 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
Performance of GCAMP and SAGC
ρ = 0.1,SNR = 20dB
ρ = 0.2,SNR = 20dB
κ
0.20
0.30
0.40
0.20
0.30
0.40
GCAMP (%)
42.02
44.11
45.60
53.29
55.57
57.70
SAGC (%)
32.25
34.99
36.95
42.34
45.43
48.23
ρ = 0.1,SNR = 10dB
ρ = 0.2,SNR = 10dB
κ
0.20
0.30
0.40
0.20
0.30
0.40
GCAMP (%)
43.77
45.77
47.57
53.23
55.13
57.03
SAGC (%)
34.23
36.97
39.27
42.59
45.34
47.92
13 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
Accuracy of Proposed σ ˆt2 2 σt,S : the value predicted by theory. 2 σt,E : empirical value of σt2 . 1
|σ ! t2 − σ t2, S | /σ t2, S
|σ ! t2 − σ t2, E | /σ t2, E
0.9 0.8
E mp i r i c al C D F
0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Ab sol t e val u e of r e l at i v e e r r or of σ ! t2
0.9
1
14 / 15
Outline
Introduction
DiAMP
GC Algorithm in DiAMP
Conclusion
A new estimator is proposed for σt2 in DiAMP. A new GC algorithm SAGC is proposed and outperforms GCAMP. The sign-awareness step yields a tighter upper bound U(n). Knowing the signs in advance makes the choice of θ more flexible.
15 / 15
Sensor Selection for Target Tracking in Sensor Networks Based on a Proximal Algorithm Xiaojun Yangǂ and Ruixin NiuƗ ǂ School of Information Engineering, Chang’an University Ɨ Department of ECE, Virginia Commonwealth University
August 10, 2016 InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
Motivation In a wireless sensor network, it is desirable to select a small subset of sensors due to limited communication bandwidth and sensor battery power. Adaptive sensor selection: to attain an optimal tradeoff between estimation accuracy and sensing & communication costs. Sensor selection is an integer programming problem with combinatorial complexity sparsity-promoting techniques with polynomial complexities.
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
1
Related work A convex optimization procedure was developed to solve the sensor selection problem in [1]. In [2], Kalman filter gain matrix was obtained as the solution to an optimization problem where a sparsity-promoting penalty function is added to the cost function. These methods are limited to linear estimators in which the optimized variable is the Kalman filter gain or linear estimator gain. Sparsity-promoting sensor selection problem was solved in [3] for parameter estimation problem. [1] S. Joshi and S. Boyd, “Sensor selection via convex optimization,” IEEE Transactions on Signal Processing, vol. 57, pp. 451–462, Feb. 2009. [2] E. Masazade, M. Fardad, and P. Varshney, “Sparsity-promoting extended kalman filtering for target tracking in wireless sensor networks,” IEEE Signal Processing Letters, vol. 19, pp. 845–848, Dec. 2012. [3] S. Chepuri and G. Leus, “Sparsity-promoting sensor selection for nonlinear measurement models,” IEEE Transactions on Signal Processing, vol. 63, pp. 684–698, Feb. 2015. InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
2
Problem Formulation Track a moving target in a WSN, with the following models
x t 1 f t ( x t ) v t zi ,t hi ,t ( x t ) nt , i 1, 2, ..., N
Sensor selection based on feedback from a nonlinear filter InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
3
Problem Formulation
Sensor selection strategy at time t is determined by a selection vector w t [ w1,t , w2,t , ..., wN ,t ]T
where the elements are binary, i.e., wi,t = 1 if the i-th sensor is selected; wi,t = 0, otherwise.
Conditional PCRLB (Zuo, Niu, and Varsheny, IEEE T-SP’11) provides a lower bound on the mean squared error (MSE) of the target state estimate, conditioned on the measurements up to the current time.
E [xˆ t 1 x t 1 ][xˆ t 1 x t 1 ]T z1:t J t 11 ( x t 1 z1:t ) Conditional Fisher information matrix (FIM) at time step t: N
J t ( w t ) wi ,t J iZ,t J tX i 1
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
4
Problem Formulation Sensor selection is formulated as an optimization problem by promoting the sparsity of selection vector.
w*t arg min ( w t ) w t wt
0
s.t . w t {0, 1}N ψ(wt): performance measure; ||wt||0: sparsity-promoting penalty function; γ characterizes the relative emphasis of accurate estimation versus selecting a small number of sensors.
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
5
l1 -Norm Minimization Algorithm Replacing non-convex Boolean constraints wi,t ∈ {0, 1} with convex box constraints wi,t ∈ [0, 1] and ℓ0 norm with ℓ1 norm, we have the following convex problem minimize ( w t ) w t wt
1
subjectto wi ,t 1,i 1, 2,
,N
This can be efficiently solved in polynomial time using interior-point methods (e.g. Newton’s method using an approximate logarithmic barrier function). InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
6
Accelerated Proximal Gradient Method If number of sensors is large, Newton’s method, which is a second order approach, is computationally inefficient proximal algorithm, a first-order method Rewrite the optimization problem as
min ( w t ) ( w t ) wt
where ϕ(wt) = ψ(wt)+γ ||wt||1, and φ(wt) is defined by an indicator function
0 ( w t )=
if 0 w1,t , ..., wN ,t 1 otherwise
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
7
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
8
APGM Algorithm In step 3, the proximal operator Proxλφ(y) of the scaled function λφ is given by 1 2 Prox ( y ) min ( x ) x y 2 x 2 For indicator function φ, the proximal operator of φ, i.e., Proxλφ, reduces to Euclidean projection onto convex box C ={0 ≤ w1,t, …, wN,t ≤1}
Prox ( v ) C v arg min x v xC
2
Projection above takes a simple form 0 vi 0 ( v ) C i vi 0 vi 1 1 v 1 i Windsor Locks, CT, August 2016 InfoSymbiotics/DDDAS Conference,
9
APGM Algorithm In Step 4, the function ˆ is defined as ˆ ( z, y ) ( y ) ( y )T ( z y ) (1 / 2 ) z y
2 2
Function ˆ can be interpreted as a quadratic approximation of ϕ(y) . Steps 3-5 constitute a linear search for step size λ(l) called backtracking.
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
10
Simulation Results 10 13
11
12
true target trajectory
8
l1 minimization APGM
6
sensor node
14
y-axis coordinates
4
2
0
15
7
16
6
-2
-4 -6
-8 1 -10 -10
3
2 -8
-6
-4
-2
0
4 2
4
5 6
8
10
x-axis coordinates
Fig.1 Sensor placement, target trajectory, and its estimate InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
11
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
12
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
13
Conclusion and Future Work •
• •
• •
We considered general scenario with nonlinear state and measurement models. The proposed sensor selection approach is not restricted to any specific estimator. We employ the conditional PCRLB as the cost function to obtain a trade-off between the estimation performance and the number of selected sensors. Based on a proximal algorithm, we developed a fast algorithm to handle large-scale sensor networks. We are developing alternating direction method of multipliers (ADMM) and distributed ADMM algorithms for large-scale sensor networks. InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
14
Overview of Other Ongoing Research Activities •
Joint Subspace Tracking and Manifold Learning for Fusing Multi-Modality Sensor Data
f:M1 M2
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
15
Other Research Activities •
Distributed Sparse Signal Recovery and Low-Rank Matrix Completion (with Han, Jowkar, Baron, Eldar, and Huie)
Message Approximate Passing (AMP) Message Passing (AMP)
[Donoho et al. 2009]
𝐳𝐳
[Donoho et al. 2009]
𝑀𝑀×𝑁𝑁 𝑡𝑡 𝐀 𝐀∈ ℝ ö s 𝒩 𝐰 𝜎𝑡𝑡2𝕀𝕀 𝐲𝐲 ( 𝐟𝐟) 𝐱𝐱 𝐰𝐱 𝐱æçè 0,∼ 𝐳 I𝐳÷ 𝒩0, 𝜎 P ø
𝑡𝑡T x ft = t + A p z tp s0 / P wtp ~ P
2
p
decouple
=
=
Node p
+
t
N
ft = å p=1𝑡𝑡 ft p
𝐟𝐟 P
𝐱s𝐱 0
wt ~
Uplink: send ft p
decouple
+
Downlink: broadcast
=
𝐰𝐰(𝑡𝑡0,s∼I )𝒩𝒩0, 𝜎𝜎𝑡𝑡2𝕀𝕀 2
t
N
+
x t+1 = h t (ft )
Fusion Center
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016 correction term correction
Iterate:
16 term
Other Research Activities • Fast Image Inpainting Based on Sparse Reconstruction
From left to right: original, damaged, and reconstructed images (with Almshaal) InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
Other Research Activities •
Secure Statistical Signal Processing in the Presence of False Information Attacks (with Lu) Analysis of impact of attacks on state estimation Attack detection/identification Game theoretic attack mitigation schemes
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
18
Other Research Activities •
Joint Group Testing of Time-Varying Faulty Sensors and State Estimation (with Ren)
InfoSymbiotics/DDDAS Conference, Windsor Locks, CT, August 2016
19
