PARTIAL UPDATE EDS ALGORITHMS FOR ADAPTIVE FILTERING
PARTIAL UPDATE EDS ALGORITHMS FOR ADAPTIVE FILTERING Bei Xie and Tamal Bose Wireless @ Virginia Tech Bradley Department of Electrical and Computer Engineering Virginia Polytechnic Institute and State University Blacksburg, VA 24061, USA ABSTRACT
2. PARTIAL UPDATE EDS
In practice, computational complexity is an important consideration of an adaptive signal processing system. A well-known approach to controlling computational complexity is applying partial update (PU) adaptive filters. In this paper, a partial update Euclidean Direction Search (EDS) algorithm is employed. The theoretical analyses of mean and mean-square performance are presented. The simulation results of different PU EDS are shown. Index Terms— EDS, partial updates
The basic adaptive filter system model can be written as: d(n) = xT (n)w∗ + v(n),
where d(n) is the desired signal, x(n) = [x(n), x(n − 1), ..., x(n − N + 1)]T is the input data vector of the unknown system, w∗ (n) = ∗ T [w1∗ , w2∗ , ..., wN ] is the impulse response vector of the unknown system, and v(n) is zero-mean white noise, which is independent of any other signals. Let w be the coefficient vector of an adaptive filter. The estimated signal y(n) is defined as
1. INTRODUCTION
y(n) = xT (n)w(n),
Adaptive filters play an important role in the fields related to digital signal processing, such as system identification, noise cancellation, and channel equalization. In the real world, the computational complexity of an adaptive filter is an important consideration for applications which need long filters. Usually algorithms with high computational complexity, such as Recursive Least Square (RLS) and EDS [1], give better convergence performance. Therefore, a tradeoff must be made between computational complexity and performance. One option is to use partial update techniques [2] to reduce the computational complexity. The partial update adaptive filter only updates part of the coefficient vector instead of updating the entire vector. The theoretical results on the full-update case may not apply to the partial update case. Therefore, performance analysis of the partial update adaptive filter is very meaningful. The EDS algorithm, developed by Xu and Bose [1], is similar to the RLS. It has fast a convergence rate and high computational complexity O(N 2 ). A fast EDS algorithm was developed to achieve O(N ) computational complexity [1]. It combines the benefits of fast convergence of the RLS and the computational efficiency of the Least Mean Square (LMS) algorithm. Since the EDS algorithm is relatively new, there has been little attempt to analyze the performance of the PU EDS. In [3] and [4], the deficient-length EDS, which is a special case of PU EDS, was studied. In this paper, partial update techniques are applied to the EDS algorithm. The theoretical mean and mean-square expressions of the PU EDS are derived. The performance of the PU EDS is analyzed. In Section 2, different PU EDS algorithms are developed. Theoretical analyses of PU EDS are given in Section 3. In Section 4, the simulation results for correlated inputs are shown. This work was supported in part by NASA grant #NNG06GE95G, NSF Grant #0809036, and the Institute for Critical Technology and Applied Science.
978-1-4244-4296-6/10/$25.00 ©2010 IEEE
(1)
3750
(2)
and the output signal error is defined as e(n) = d(n) − xT (n)w(n).
(3)
The partial update method aims to reduce the computational cost of the adaptive filters. Instead of updating all the N × 1 coefficients, it usually only updates M × 1 coefficients, where M < N . Basic partial update methods include periodic PU, sequential PU, the deficient-length method, stochastic PU, the MMax update method, etc. These methods will be applied to the EDS algorithm. The partial update EDS has the uniform update equation: w(n + 1) = w(n) −
ˆ −ˆ r(n)) ggT (Q(n)w(n) , ˆ gT Q(n)g
(4)
where ˆ Q(n)
=
n
λn−i IM x(i)xT (i)ITM
i=1
= ˆ r(n)
= =
n i=1 n i=1 n
λn−i x ˆ(i)ˆ xT (i),
(5)
λn−i d(i)IM x(i) λn−i x ˆ(i)d(i),
(6)
i=1
⎡ ⎢ ⎢ IM (n) = ⎢ ⎢ ⎣
i1 (n)
0
0 .. . 0
i2 (n) .. . ...
... .. . .. . 0
0 .. . 0 iN (n)
⎤ ⎥ ⎥ ⎥, ⎥ ⎦
(7)
ICASSP 2010
N
ik (n) = M,
ik (n) ∈ {0, 1},
(8)
k=1
where x ˆ = IM x and the N × 1 vector g is the search direction at iteration n, which is taken to be “Euclidean directions.” It is defined as gi = [0, · · · , 0, 1, 0, · · · , 0]T , where the 1 appears in the i-th position. For PU EDS, the entire weight vector w(n) is partially updated by cycling through the Euclidean directions gk at each iteration n, and k is determined by ik . This means if ik = 1, then the kth element of g will be set to be 1. If ik = 0, then the kth element of w(n) will not be updated. The total number of multiplications needed for this uniform PU EDS becomes M (2N + M + 3). The original EDS needs N (3N + 3) [1] multiplications. For different partial update algorithms, only IM will be different. 2.1. Periodic Partial Update EDS The periodic partial update method [2] only updates the coefficients at every S th iteration and copies the coefficients at the other iterations. The periodic PU EDS doesn’t partially update the coefficient vector, but updates the entire vector at every Sth iteration. The update equation for EDS can be written as: w((n + 1)S) = w(nS) − N
gT (Q(nS)w(nS) − r(nS))g , gT Q(nS)g
(9)
N . The total number of where S = M , which is the ceiling of M multiplications needed for every S number of samples is N (3N +3). Since the periodic PU EDS still updates the whole vector, the steadystate performance will be the same as the original EDS. However, the convergence rate of periodic PU EDS will be S times slower than the original EDS.
2.2. Deficient-length EDS
2.5. MMax EDS The update equations (4) and (7) can still be applied to the MMax EDS algorithm. The condition of ik (n) [2] becomes 1 if |xk (n)| ∈ max1≤l≤N {|xl (n)|, M } , (14) ik (n) = 0 otherwise which means the elements of the weight vector w are updated according to the first M max elements of the input x. 3. PERFORMANCE ANALYSIS OF PARTIAL UPDATE EDS To determine the steady-state behaviors of partial update EDS, two assumptions are needed. Assumption I: The coefficient error z(n) is independent of the input signal x(n) in steady state. Assumption II: Noise v(n) is independent of the input signal x(n) and has zero mean. The desired signal can be rewritten as ¯T (n)w∗ + v(n), d(n) = x ˆT (n)w∗ + x
z(n) = w(n) − w∗ , and subtracting w from both sides of (4), we have z(n + 1) = z(n) −
ˆ ggT (Q(n)w(n) −ˆ r(n)) . ˆ gT Q(n)g
(17)
ˆ ˆ Now consider the term Q(n)w(n) −ˆ r(n). Since Q(n) = T ˆ r(n) = λˆ r(n − 1) + d(n)ˆ x(n), and λQ(n − 1) + x ˆ(n)ˆ x (n), ˆ from (15), (16), we get ˆ Q(n)w(n) −ˆ r(n) = (
n
λn−i x ˆ(i)ˆ xT (i))z(n)
i=1
−
(
n
λn−i x ˆ(i)¯ xT (i))w∗ −
i=1
2.3. Sequential Partial Update EDS
=
n
λn−i v(i)ˆ x(i)
i=1
ˆ Q(n)z(n) − B(n)w∗ − C(n),
(18)
where (11)
B(n) =
n
λn−i x ˆ(i)¯ xT (i),
i=1 n
where K1 = {1, 2, . . . , M }, K2 = {M + 1, M + 2, . . . , 2M }, ... , KS = {(S − 1)M + 1, (S − 1)M + 2, . . . , N }.
C(n) =
λn−i v(i)ˆ x(i).
(19) (20)
i=1
2.4. Stochastic Partial Update EDS The stochastic partial update algorithm chooses coefficients subsets randomly. The ik (n) becomes 1 if k ∈ Km(n) , (12) ik (n) = 0 otherwise where m(n) is a random process with probability mass function [2]: P r{m(n) = i} = pi , i = 1, . . . , S,
(16)
∗
The deficient-length method only updates the first 1 . . . M elements of the coefficient vector w at each iteration. Then ik (n) becomes 1 if k = 1, . . . , M . (10) ik (n) = 0 otherwise
The sequential partial update method [2] designs ik (n) as: 1 if k ∈ Kn mod S+1 , ik (n) = 0 otherwise
(15)
IN−M x. where x ¯ =x−x ˆ = (I − IM )x = ¯ Introducing the coefficient error vector
S
pi = 1.
(13)
i=1
Usually a uniformly distributed random process will be applied. Therefore, for each iteration, M of N weight elements will be chosen and updated with probabilty pi = 1/S.
3751
Now we can rewrite the PU EDS coefficient error recursion in (17) as z(n + 1) = z(n) −
ˆ ggT (Q(n)z(n) − B(n)w∗ − C(n)) . (21) ˆ gT Q(n)g
Using the direct-averaging method [6], (21) can be replaced by z(n + 1)
=
z(n) − E{
+
E{
ˆ ggT Q(n) }z(n) T ˆ g Q(n)g
ggT C(n) ggT B(n) }w∗ + . T ˆ ˆ g Q(n)g gT Q(n)g
Note w∗ is independent of the input signal.
(22)
3.1. Mean Performance
v(n) x(n)
The mean behavior of the EDS coefficient error vector can be determined by taking the expectation of both sides of (21) E{z(n + 1)} = (I − J(n)) E{z(n)} + S(n), T
(23)
ε(n)
d(n) _
Unknown system
Adaptive filter
T
ˆ
B(n) Q(n) } and S(n) = E{ gggT Q(n)g }w∗ . We where J(n) = E{ gg ˆ ˆ gT Q(n)g T
C(n) } = 0. have used E{ gggT Q(n)g ˆ
Fig. 1. System identification model.
The necessary and sufficient condition for E{z} to converge is ρ (I − J(n)) < 1,
100
(24)
60
where ρ is the spectral radius and this condition is equivalent to
40
(25)
The full-update EDS algorithm was proven to converge to the optimal filter w∗ in [5]. Since the maximal eigenvalue of J(n) must be T
Q(n) smaller than or equal to the maximal eigenvalue of E{ gg } for gT Q(n)g stationary signals, the PU EDS can also converge, but to the optimal filter plus an error weight. The convergence rate of the PU EDS deˆ is pends on the term J(n). As g is only the direction and gT Q(n)g ˆ only a scalar to normalize the Q(n), the convergence can be further ˆ = E{ˆ related to the partial input correlation matrix R xx ˆT }. The steady-state mean weight error of the PU EDS is
ˆ −1 h, E{z(∞)} = R
MSE (dB)
0 < λmax (J(n)) < 2.
MMax EDS Deficient−length EDS Periodic PU EDS Sequential PU EDS Stochastic PU EDS EDS
80
20 0 −20 −40 −60 −80 −100 0
50
100
150 200 250 Iteration number
300
350
400
Fig. 2. Comparison of MSE of partial update EDS with correlated input.
(26)
ˆ = E{ˆ xx ¯T }w∗ . From (23) and (26), where R xx ˆT } and h = E{ˆ ˆ has a large efwe can see that the partial input correlation matrix R fect on the convergence rate and the steady-state mean weight error. ˆ = M R. For the sequential PU EDS and the stochastic PU EDS R N Obviously, the error will increase and the convergence rate will deˆ crease when M decreases. For the MMax EDS, R is close to R, therefore the convergence rate should be close to that of the EDS. For the deficient-length EDS, the convergence rate should be slightly faster than the EDS because it uses a shorter filter. For white inputs, E{ˆ xx ¯T } = 0. Therefore, h = 0. The steadystate mean weight is equal to the optimal weight.
For the white noise input case, R = σx2 I, where σx2 is the variance of input signal. The MSE becomes (n) = min + σx2 tr(K(n)).
(29)
Since S(n) = 0 in (28), K(n) can be simplified to
+
K(n + 1) = K(n) − K(n)JT (n) − J(n)K(n) T ˜ ggT Q(n)gg J(n)K(n)JT (n) + min E{ }, (30) T 2 ˆ (g Q(n)g)
2(n−i) ˜ where Q(n) = ( n x ˆ(i)ˆ xT (i)). The MSE of white ini=1 λ puts is not affected by the mean behavior.
3.2. Mean-Square Performance The MSE equation of the PU EDS algorithm is the same as the fullupdate EDS, which is (n) = min + tr(RE{z(n)zT (n)}),
(27)
2
where min = E{v (n)} is the variance of the input noise, and R = E{x(n)xT (n)} is the autocorrelation matrix of the input data. The only difference for PU EDS is the weight-error autocorrelation matrix K(n) = E{z(n)zT (n)}. By using Assumptions I, II, and (22), K(n) is determined as follows:
T
T
+
J(n)K(n)J (n) + E{z(n)}S (n) + S(n)E{z (n)}
−
J(n)E{z(n)}ST (n) − S(n)E{zT (n)}JT (n)
+
S(n)ST (n) + E{
ggT C(n) CT (n)ggT }. ˆ ˆ gT Q(n)g gT Q(n)g
4.1. System Identification The system identification model is shown in Fig. 1 and is taken from [3]. The same unknown system was used as in [7]. It was a 16-order FIR filter (N=16), with the impulse response w∗ = [0.01, 0.02, −0.04, −0.08, 0.15, −0.3, 0.45, 0.6, 0.6, 0.45, −0.3, 0.15, −0.08, −0.04, 0.02, 0.01]T . In our simulations, the length of the partial update filter was M=12. The variance of the input noise v(n) was min = 0.0001. The initial weights were w = 0. The parameter λ of EDS was equal to 0.9995. The results were obtained by averaging 400 independent runs. The input of the system [3] has the following form
K(n + 1) = K(n) − K(n)JT (n) − J(n)K(n) T
4. SIMULATIONS
(28)
x(n) = 0.8x(n − 1) + η(n),
(31)
where η(n) was zero-mean white Gaussian noise with unit variance.
The MSE convergence is affected by the mean convergence.
3752
0
0.06
10
0.04
10
EDS Deficient−length EDS MMax EDS Stochastic PU EDS Sequential PU EDS
−1
−2
10
SER
EZ8(n)
0.02
0
−0.04
−0.06 0
−4
10
Deficient−length EDS simulation Deficient−length EDS theory Sequential PU EDS simulation Sequential PU EDS theory Stochastic PU EDS simulation Stochastic PU EDS theory MMax EDS simulation MMax EDS theory
−0.02
50
100
150 200 250 iteration number
300
350
−3
10
−5
10
−6
10
400
5
6
7
8
9
10 11 SNR(dB)
12
13
14
15
Fig. 5. Comparison of SER among PU EDS algorithms. Fig. 3. Comparison of theoretical and simulation curves of the mean behavior of the 8th coefficient error of the PU EDS. We assume the full length of the equalizer is 30 and let the PU length be 15. The input sequence is 4-QAM. The results are obtained by averaging 50 independent runs. Fig. 5 illustrates the symbol-error-rate (SER) in log-scale among PU EDS algorithms. The SER performance of different PU EDS algorithms is still related to the MSE performance shown in Fig. 2. When SNR equals 15 dB, the SERs of the deficient-length EDS and the MMax EDS become zeros, which are the same as the full-update EDS. However, the sequential and the stochastic EDS algorithms perform much worse than the full-update EDS for high SNR.
100 Deficient−length EDS simulation Deficient−length EDS theory Sequential PU EDS simulation Sequential PU EDS theory Stochastic PU EDS simulation Stochastic PU EDS theory MMax EDS simulation MMax EDS theory
80 60
MSE(dB)
40 20 0 −20
5. CONCLUSION
−40
In this paper, different PU EDS algorithms are developed. The expressions for mean and mean-square errors of the PU EDS are derived for both correlated and white inputs. Simulation results agree with the derived theoretical expressions in steady state.
−60 0
50
100
150 200 250 iteration number
300
350
400
Fig. 4. Comparison of theoretical and simulation curves of the meansquare behavior of the PU EDS for correlated input. Fig. 2 shows the mean-square behaviors of the partial update EDS with the correlated input signal. From the figure, the steadystate MSE of the periodic PU EDS is the same as that of EDS. The convergence rates of the periodic, sequential, and stochastic PU EDS algorithms are about S times slower than the EDS algorithm. The sequential PU EDS and the stochastic PU EDS have high MSE. The steady-state MSE of the deficient-length EDS and MMax EDS are almost 40dB worse than that of the EDS. Fig. 3 and Fig. 4 show the theoretical and simulation curves of the mean and mean-square behaviors of PU EDS for correlated input. The theoretical results are obtained by using (23), (27), and (28). In steady state, the simulation results agree with the theoretical results. Since Assumption I is only valid for steady state, the results do not match well in transient state. The results for white input are omitted due to space limitations. 4.2. Channel Equalization Equalization performance is also examined among PU EDS algorithms. We use a simple short FIR channel [8] C(z) = 0.5 + 1.2z −1 + 1.5z −2 − z −3 .
(32)
3753
6. REFERENCES [1] T. Bose, Digital Signal and Image Processing. New York: Wiley, 2004. [2] K. Do˘ganc¸ay, Partial-Update Adaptive Signal Processing. Academic Press, 2008. [3] Z. K. Zhang, T. Bose, L. Xiao, and R. Thamvichai, “Performance analysis of the deficient length EDS adaptive algorithms,” in Proc. APCCAS, Dec. 2006, pp. 222-226. [4] B. Xie, T. Bose, and Z. K. Zhang, “Performance analysis of deficient-length RLS and EDS algorithms,” in Proc. IEEE DSP/SPE Workshop, Jan. 2009, pp. 1-6. [5] G. F. Xu, “Fast algorithms for digital filtering: Theory and applications,” Ph.D. dissertation, University of Colorado, 1999. [6] S. Haykin, Adaptive Filter Theory. Prentice-Hall, 1996.
3th ed. New Jersey:
[7] K. Mayyas, “Performance analysis of the deficient length LMS adaptive algorithm,” IEEE Trans. Signal Processing, vol. 53, no. 8, pp. 2727-2734, Aug. 2005. [8] A. H. Sayed, Fundamentals of Adaptive Filtering. New York: Wiley, 2003.
2. PARTIAL UPDATE EDS
In practice, computational complexity is an important consideration of an adaptive signal processing system. A well-known approach to controlling computational complexity is applying partial update (PU) adaptive filters. In this paper, a partial update Euclidean Direction Search (EDS) algorithm is employed. The theoretical analyses of mean and mean-square performance are presented. The simulation results of different PU EDS are shown. Index Terms— EDS, partial updates
The basic adaptive filter system model can be written as: d(n) = xT (n)w∗ + v(n),
where d(n) is the desired signal, x(n) = [x(n), x(n − 1), ..., x(n − N + 1)]T is the input data vector of the unknown system, w∗ (n) = ∗ T [w1∗ , w2∗ , ..., wN ] is the impulse response vector of the unknown system, and v(n) is zero-mean white noise, which is independent of any other signals. Let w be the coefficient vector of an adaptive filter. The estimated signal y(n) is defined as
1. INTRODUCTION
y(n) = xT (n)w(n),
Adaptive filters play an important role in the fields related to digital signal processing, such as system identification, noise cancellation, and channel equalization. In the real world, the computational complexity of an adaptive filter is an important consideration for applications which need long filters. Usually algorithms with high computational complexity, such as Recursive Least Square (RLS) and EDS [1], give better convergence performance. Therefore, a tradeoff must be made between computational complexity and performance. One option is to use partial update techniques [2] to reduce the computational complexity. The partial update adaptive filter only updates part of the coefficient vector instead of updating the entire vector. The theoretical results on the full-update case may not apply to the partial update case. Therefore, performance analysis of the partial update adaptive filter is very meaningful. The EDS algorithm, developed by Xu and Bose [1], is similar to the RLS. It has fast a convergence rate and high computational complexity O(N 2 ). A fast EDS algorithm was developed to achieve O(N ) computational complexity [1]. It combines the benefits of fast convergence of the RLS and the computational efficiency of the Least Mean Square (LMS) algorithm. Since the EDS algorithm is relatively new, there has been little attempt to analyze the performance of the PU EDS. In [3] and [4], the deficient-length EDS, which is a special case of PU EDS, was studied. In this paper, partial update techniques are applied to the EDS algorithm. The theoretical mean and mean-square expressions of the PU EDS are derived. The performance of the PU EDS is analyzed. In Section 2, different PU EDS algorithms are developed. Theoretical analyses of PU EDS are given in Section 3. In Section 4, the simulation results for correlated inputs are shown. This work was supported in part by NASA grant #NNG06GE95G, NSF Grant #0809036, and the Institute for Critical Technology and Applied Science.
978-1-4244-4296-6/10/$25.00 ©2010 IEEE
(1)
3750
(2)
and the output signal error is defined as e(n) = d(n) − xT (n)w(n).
(3)
The partial update method aims to reduce the computational cost of the adaptive filters. Instead of updating all the N × 1 coefficients, it usually only updates M × 1 coefficients, where M < N . Basic partial update methods include periodic PU, sequential PU, the deficient-length method, stochastic PU, the MMax update method, etc. These methods will be applied to the EDS algorithm. The partial update EDS has the uniform update equation: w(n + 1) = w(n) −
ˆ −ˆ r(n)) ggT (Q(n)w(n) , ˆ gT Q(n)g
(4)
where ˆ Q(n)
=
n
λn−i IM x(i)xT (i)ITM
i=1
= ˆ r(n)
= =
n i=1 n i=1 n
λn−i x ˆ(i)ˆ xT (i),
(5)
λn−i d(i)IM x(i) λn−i x ˆ(i)d(i),
(6)
i=1
⎡ ⎢ ⎢ IM (n) = ⎢ ⎢ ⎣
i1 (n)
0
0 .. . 0
i2 (n) .. . ...
... .. . .. . 0
0 .. . 0 iN (n)
⎤ ⎥ ⎥ ⎥, ⎥ ⎦
(7)
ICASSP 2010
N
ik (n) = M,
ik (n) ∈ {0, 1},
(8)
k=1
where x ˆ = IM x and the N × 1 vector g is the search direction at iteration n, which is taken to be “Euclidean directions.” It is defined as gi = [0, · · · , 0, 1, 0, · · · , 0]T , where the 1 appears in the i-th position. For PU EDS, the entire weight vector w(n) is partially updated by cycling through the Euclidean directions gk at each iteration n, and k is determined by ik . This means if ik = 1, then the kth element of g will be set to be 1. If ik = 0, then the kth element of w(n) will not be updated. The total number of multiplications needed for this uniform PU EDS becomes M (2N + M + 3). The original EDS needs N (3N + 3) [1] multiplications. For different partial update algorithms, only IM will be different. 2.1. Periodic Partial Update EDS The periodic partial update method [2] only updates the coefficients at every S th iteration and copies the coefficients at the other iterations. The periodic PU EDS doesn’t partially update the coefficient vector, but updates the entire vector at every Sth iteration. The update equation for EDS can be written as: w((n + 1)S) = w(nS) − N
gT (Q(nS)w(nS) − r(nS))g , gT Q(nS)g
(9)
N . The total number of where S = M , which is the ceiling of M multiplications needed for every S number of samples is N (3N +3). Since the periodic PU EDS still updates the whole vector, the steadystate performance will be the same as the original EDS. However, the convergence rate of periodic PU EDS will be S times slower than the original EDS.
2.2. Deficient-length EDS
2.5. MMax EDS The update equations (4) and (7) can still be applied to the MMax EDS algorithm. The condition of ik (n) [2] becomes 1 if |xk (n)| ∈ max1≤l≤N {|xl (n)|, M } , (14) ik (n) = 0 otherwise which means the elements of the weight vector w are updated according to the first M max elements of the input x. 3. PERFORMANCE ANALYSIS OF PARTIAL UPDATE EDS To determine the steady-state behaviors of partial update EDS, two assumptions are needed. Assumption I: The coefficient error z(n) is independent of the input signal x(n) in steady state. Assumption II: Noise v(n) is independent of the input signal x(n) and has zero mean. The desired signal can be rewritten as ¯T (n)w∗ + v(n), d(n) = x ˆT (n)w∗ + x
z(n) = w(n) − w∗ , and subtracting w from both sides of (4), we have z(n + 1) = z(n) −
ˆ ggT (Q(n)w(n) −ˆ r(n)) . ˆ gT Q(n)g
(17)
ˆ ˆ Now consider the term Q(n)w(n) −ˆ r(n). Since Q(n) = T ˆ r(n) = λˆ r(n − 1) + d(n)ˆ x(n), and λQ(n − 1) + x ˆ(n)ˆ x (n), ˆ from (15), (16), we get ˆ Q(n)w(n) −ˆ r(n) = (
n
λn−i x ˆ(i)ˆ xT (i))z(n)
i=1
−
(
n
λn−i x ˆ(i)¯ xT (i))w∗ −
i=1
2.3. Sequential Partial Update EDS
=
n
λn−i v(i)ˆ x(i)
i=1
ˆ Q(n)z(n) − B(n)w∗ − C(n),
(18)
where (11)
B(n) =
n
λn−i x ˆ(i)¯ xT (i),
i=1 n
where K1 = {1, 2, . . . , M }, K2 = {M + 1, M + 2, . . . , 2M }, ... , KS = {(S − 1)M + 1, (S − 1)M + 2, . . . , N }.
C(n) =
λn−i v(i)ˆ x(i).
(19) (20)
i=1
2.4. Stochastic Partial Update EDS The stochastic partial update algorithm chooses coefficients subsets randomly. The ik (n) becomes 1 if k ∈ Km(n) , (12) ik (n) = 0 otherwise where m(n) is a random process with probability mass function [2]: P r{m(n) = i} = pi , i = 1, . . . , S,
(16)
∗
The deficient-length method only updates the first 1 . . . M elements of the coefficient vector w at each iteration. Then ik (n) becomes 1 if k = 1, . . . , M . (10) ik (n) = 0 otherwise
The sequential partial update method [2] designs ik (n) as: 1 if k ∈ Kn mod S+1 , ik (n) = 0 otherwise
(15)
IN−M x. where x ¯ =x−x ˆ = (I − IM )x = ¯ Introducing the coefficient error vector
S
pi = 1.
(13)
i=1
Usually a uniformly distributed random process will be applied. Therefore, for each iteration, M of N weight elements will be chosen and updated with probabilty pi = 1/S.
3751
Now we can rewrite the PU EDS coefficient error recursion in (17) as z(n + 1) = z(n) −
ˆ ggT (Q(n)z(n) − B(n)w∗ − C(n)) . (21) ˆ gT Q(n)g
Using the direct-averaging method [6], (21) can be replaced by z(n + 1)
=
z(n) − E{
+
E{
ˆ ggT Q(n) }z(n) T ˆ g Q(n)g
ggT C(n) ggT B(n) }w∗ + . T ˆ ˆ g Q(n)g gT Q(n)g
Note w∗ is independent of the input signal.
(22)
3.1. Mean Performance
v(n) x(n)
The mean behavior of the EDS coefficient error vector can be determined by taking the expectation of both sides of (21) E{z(n + 1)} = (I − J(n)) E{z(n)} + S(n), T
(23)
ε(n)
d(n) _
Unknown system
Adaptive filter
T
ˆ
B(n) Q(n) } and S(n) = E{ gggT Q(n)g }w∗ . We where J(n) = E{ gg ˆ ˆ gT Q(n)g T
C(n) } = 0. have used E{ gggT Q(n)g ˆ
Fig. 1. System identification model.
The necessary and sufficient condition for E{z} to converge is ρ (I − J(n)) < 1,
100
(24)
60
where ρ is the spectral radius and this condition is equivalent to
40
(25)
The full-update EDS algorithm was proven to converge to the optimal filter w∗ in [5]. Since the maximal eigenvalue of J(n) must be T
Q(n) smaller than or equal to the maximal eigenvalue of E{ gg } for gT Q(n)g stationary signals, the PU EDS can also converge, but to the optimal filter plus an error weight. The convergence rate of the PU EDS deˆ is pends on the term J(n). As g is only the direction and gT Q(n)g ˆ only a scalar to normalize the Q(n), the convergence can be further ˆ = E{ˆ related to the partial input correlation matrix R xx ˆT }. The steady-state mean weight error of the PU EDS is
ˆ −1 h, E{z(∞)} = R
MSE (dB)
0 < λmax (J(n)) < 2.
MMax EDS Deficient−length EDS Periodic PU EDS Sequential PU EDS Stochastic PU EDS EDS
80
20 0 −20 −40 −60 −80 −100 0
50
100
150 200 250 Iteration number
300
350
400
Fig. 2. Comparison of MSE of partial update EDS with correlated input.
(26)
ˆ = E{ˆ xx ¯T }w∗ . From (23) and (26), where R xx ˆT } and h = E{ˆ ˆ has a large efwe can see that the partial input correlation matrix R fect on the convergence rate and the steady-state mean weight error. ˆ = M R. For the sequential PU EDS and the stochastic PU EDS R N Obviously, the error will increase and the convergence rate will deˆ crease when M decreases. For the MMax EDS, R is close to R, therefore the convergence rate should be close to that of the EDS. For the deficient-length EDS, the convergence rate should be slightly faster than the EDS because it uses a shorter filter. For white inputs, E{ˆ xx ¯T } = 0. Therefore, h = 0. The steadystate mean weight is equal to the optimal weight.
For the white noise input case, R = σx2 I, where σx2 is the variance of input signal. The MSE becomes (n) = min + σx2 tr(K(n)).
(29)
Since S(n) = 0 in (28), K(n) can be simplified to
+
K(n + 1) = K(n) − K(n)JT (n) − J(n)K(n) T ˜ ggT Q(n)gg J(n)K(n)JT (n) + min E{ }, (30) T 2 ˆ (g Q(n)g)
2(n−i) ˜ where Q(n) = ( n x ˆ(i)ˆ xT (i)). The MSE of white ini=1 λ puts is not affected by the mean behavior.
3.2. Mean-Square Performance The MSE equation of the PU EDS algorithm is the same as the fullupdate EDS, which is (n) = min + tr(RE{z(n)zT (n)}),
(27)
2
where min = E{v (n)} is the variance of the input noise, and R = E{x(n)xT (n)} is the autocorrelation matrix of the input data. The only difference for PU EDS is the weight-error autocorrelation matrix K(n) = E{z(n)zT (n)}. By using Assumptions I, II, and (22), K(n) is determined as follows:
T
T
+
J(n)K(n)J (n) + E{z(n)}S (n) + S(n)E{z (n)}
−
J(n)E{z(n)}ST (n) − S(n)E{zT (n)}JT (n)
+
S(n)ST (n) + E{
ggT C(n) CT (n)ggT }. ˆ ˆ gT Q(n)g gT Q(n)g
4.1. System Identification The system identification model is shown in Fig. 1 and is taken from [3]. The same unknown system was used as in [7]. It was a 16-order FIR filter (N=16), with the impulse response w∗ = [0.01, 0.02, −0.04, −0.08, 0.15, −0.3, 0.45, 0.6, 0.6, 0.45, −0.3, 0.15, −0.08, −0.04, 0.02, 0.01]T . In our simulations, the length of the partial update filter was M=12. The variance of the input noise v(n) was min = 0.0001. The initial weights were w = 0. The parameter λ of EDS was equal to 0.9995. The results were obtained by averaging 400 independent runs. The input of the system [3] has the following form
K(n + 1) = K(n) − K(n)JT (n) − J(n)K(n) T
4. SIMULATIONS
(28)
x(n) = 0.8x(n − 1) + η(n),
(31)
where η(n) was zero-mean white Gaussian noise with unit variance.
The MSE convergence is affected by the mean convergence.
3752
0
0.06
10
0.04
10
EDS Deficient−length EDS MMax EDS Stochastic PU EDS Sequential PU EDS
−1
−2
10
SER
EZ8(n)
0.02
0
−0.04
−0.06 0
−4
10
Deficient−length EDS simulation Deficient−length EDS theory Sequential PU EDS simulation Sequential PU EDS theory Stochastic PU EDS simulation Stochastic PU EDS theory MMax EDS simulation MMax EDS theory
−0.02
50
100
150 200 250 iteration number
300
350
−3
10
−5
10
−6
10
400
5
6
7
8
9
10 11 SNR(dB)
12
13
14
15
Fig. 5. Comparison of SER among PU EDS algorithms. Fig. 3. Comparison of theoretical and simulation curves of the mean behavior of the 8th coefficient error of the PU EDS. We assume the full length of the equalizer is 30 and let the PU length be 15. The input sequence is 4-QAM. The results are obtained by averaging 50 independent runs. Fig. 5 illustrates the symbol-error-rate (SER) in log-scale among PU EDS algorithms. The SER performance of different PU EDS algorithms is still related to the MSE performance shown in Fig. 2. When SNR equals 15 dB, the SERs of the deficient-length EDS and the MMax EDS become zeros, which are the same as the full-update EDS. However, the sequential and the stochastic EDS algorithms perform much worse than the full-update EDS for high SNR.
100 Deficient−length EDS simulation Deficient−length EDS theory Sequential PU EDS simulation Sequential PU EDS theory Stochastic PU EDS simulation Stochastic PU EDS theory MMax EDS simulation MMax EDS theory
80 60
MSE(dB)
40 20 0 −20
5. CONCLUSION
−40
In this paper, different PU EDS algorithms are developed. The expressions for mean and mean-square errors of the PU EDS are derived for both correlated and white inputs. Simulation results agree with the derived theoretical expressions in steady state.
−60 0
50
100
150 200 250 iteration number
300
350
400
Fig. 4. Comparison of theoretical and simulation curves of the meansquare behavior of the PU EDS for correlated input. Fig. 2 shows the mean-square behaviors of the partial update EDS with the correlated input signal. From the figure, the steadystate MSE of the periodic PU EDS is the same as that of EDS. The convergence rates of the periodic, sequential, and stochastic PU EDS algorithms are about S times slower than the EDS algorithm. The sequential PU EDS and the stochastic PU EDS have high MSE. The steady-state MSE of the deficient-length EDS and MMax EDS are almost 40dB worse than that of the EDS. Fig. 3 and Fig. 4 show the theoretical and simulation curves of the mean and mean-square behaviors of PU EDS for correlated input. The theoretical results are obtained by using (23), (27), and (28). In steady state, the simulation results agree with the theoretical results. Since Assumption I is only valid for steady state, the results do not match well in transient state. The results for white input are omitted due to space limitations. 4.2. Channel Equalization Equalization performance is also examined among PU EDS algorithms. We use a simple short FIR channel [8] C(z) = 0.5 + 1.2z −1 + 1.5z −2 − z −3 .
(32)
3753
6. REFERENCES [1] T. Bose, Digital Signal and Image Processing. New York: Wiley, 2004. [2] K. Do˘ganc¸ay, Partial-Update Adaptive Signal Processing. Academic Press, 2008. [3] Z. K. Zhang, T. Bose, L. Xiao, and R. Thamvichai, “Performance analysis of the deficient length EDS adaptive algorithms,” in Proc. APCCAS, Dec. 2006, pp. 222-226. [4] B. Xie, T. Bose, and Z. K. Zhang, “Performance analysis of deficient-length RLS and EDS algorithms,” in Proc. IEEE DSP/SPE Workshop, Jan. 2009, pp. 1-6. [5] G. F. Xu, “Fast algorithms for digital filtering: Theory and applications,” Ph.D. dissertation, University of Colorado, 1999. [6] S. Haykin, Adaptive Filter Theory. Prentice-Hall, 1996.
3th ed. New Jersey:
[7] K. Mayyas, “Performance analysis of the deficient length LMS adaptive algorithm,” IEEE Trans. Signal Processing, vol. 53, no. 8, pp. 2727-2734, Aug. 2005. [8] A. H. Sayed, Fundamentals of Adaptive Filtering. New York: Wiley, 2003.
