A modified frequency-domain block LMS algorithm with guaranteed ...
Signal Processing 104 (2014) 27–32
Contents lists available at ScienceDirect
Signal Processing journal homepage: www.elsevier.com/locate/sigpro
Fast communication
A modified frequency-domain block LMS algorithm with guaranteed optimal steady-state performance Jing Lu n, Xiaojun Qiu, Haishan Zou Key Laboratory of Modern Acoustics, Institute of Acoustics, Nanjing University, Nanjing 210093, China
a r t i c l e i n f o
abstract
Article history: Received 29 November 2013 Received in revised form 13 March 2014 Accepted 21 March 2014 Available online 28 March 2014
The bin-normalized frequency-domain block LMS (FBLMS) algorithm has low computational burden and potential fast convergence; however, it suffers from a biased steadystate solution when the reference signal lags behind the desired signal or the adaptive filter is of insufficient length. This paper proposes a unified framework for the FBLMS algorithm, which can be used to comprehensively analyze its steady-state behavior. Furthermore, a modified FBLMS algorithm with guaranteed optimal steady-state performance is proposed based on the framework. Simulations are carried out to demonstrate the benefit of the proposed algorithm. & 2014 Elsevier B.V. All rights reserved.
Key words: Adaptive filters Frequency-domain implementation Steady-state behavior
1. Introduction Adaptive filtering has been widely used in many situations such as telecommunication systems, acoustic echo cancellation, active noise control, and array processing, where the least-mean-square (LMS) algorithm is commonly used due to its simplicity and robustness [1,2]. Unfortunately, it suffers from slow convergence for reference signals with large eigenvalue disparity, and moreover, its computational burden is too heavy in many application scenarios because the filter length has to be set very large [1,2]. To overcome the problem of slow convergence, transform-domain LMS (TDLMS) algorithms [1–4] have been suggested, which preprocess the reference signal by using orthogonal transforms such as the discrete Fourier transform (DFT), discrete cosine transform (DCT), discrete sine transform (DST), and discrete Hartely transform (DHT), and then set power-normalized step sizes. The improvement of the convergence rate has been proven by many
n
Corresponding author. Tel.: þ86 25 83593571. E-mail address: [email protected] (J. Lu).
http://dx.doi.org/10.1016/j.sigpro.2014.03.029 0165-1684/& 2014 Elsevier B.V. All rights reserved.
researchers [3,4]. However, the computational burden of the TDLMS algorithms is substantially heavier than that of the LMS algorithm because the orthogonal transforms are often performed for each new input sample. Although partial updating and sliding transform techniques [5,6] can be used to mitigate the problem, the computational burden is still a challenge for implementation of the TDLMS algorithms in real-time systems. Apart from the application of the TDLMS algorithm, the DFT in particular can also be used to realize the frequencydomain block least-mean-square (FBLMS) algorithm [7], which is a computational efficient implementation of the block LMS (BLMS) algorithm. The computational burden of the FBLMS algorithm is significantly less than that of the LMS algorithm because the fast Fourier transform (FFT) is used to calculate both the block filtering output and the update terms in the frequency domain. Furthermore, when the step size of the adaptive filter is normalized by the reference signal power in each frequency bin, the convergence speed of the FBLMS algorithm can be significantly increased for reference signals with large power spectral disparity [7,8]. Therefore the bin-normalized FBLMS algorithm is widely used in many applications that require
28
J. Lu et al. / Signal Processing 104 (2014) 27–32
large filter length and fast convergence, e.g., acoustic echo cancellation, active noise control, channel estimation, and equalization [9–11]. Nevertheless, it has been pointed out that the bin-normalized FBLMS algorithm suffers from an increase in the steady-state mean-square error in noncausal circumstances [12] or with insufficient filter length [13], which is very common in many applications of adaptive filters. For example, the reference signal can lag behind the desired signal in an adaptive equalizer or in an adaptive feedback active noise control system. On the other hand, for an acoustic echo cancellation systems used in a room with long reverberation time, it is often the case that the adaptive filter is of insufficient length. A frequency-domain Newton's algorithm has been derived in [12] to improve the steady-state behavior in non-causal circumstances. However, it requires a spectral factorization of the estimated power spectral density of the reference signal, which forms an obstacle to its implementation. In this paper, a unified framework of the FBLMS algorithm without any assumptions on the signal and system model is proposed, which can be used to comprehensively analyze the steady-state behavior of the algorithm. Based on this framework, a modification is proposed on the existing algorithm that guarantees optimal steady-state behavior. Throughout this paper, lowercase letters are used for scalar quantities, bold lowercase for vectors, and bold uppercase for matrices. Subscript f denotes frequency-domain representation of each signal and k is reserved for the block index. 2. Analysis of FBLMS steady-state behavior based on a unified framework Let x(k) ¼[x(kN N), x(kN N þ1), …, x(kN þN 1)]T be the reference signal vector, where the superscript T represents the transpose operation, w(k)¼[w0(k), w1(k), …, wN 1(k)]T be the N-tap filter, and d(k)¼ [d(kN N), d(kNNþ1), …, d(kNþN1)]T be the desired signal vector. Then the error vector in the frequency domain can be described as ef ðkÞ ¼ FG0;N F 1 df ðkÞ Xf ðkÞwf ðkÞ ð1Þ where F represents the 2N 2N discrete Fourier transform (DFT) matrix, df(k)¼F[01 N, dT(k)]T, Xf(k)¼diag[xf(k)]¼diag [Fx(k)], wf(k)¼F[wT(k), 01 N]T, and " # 0NN 0NN G0;N ¼ : ð2Þ 0NN INN There are two kinds of FBLMS algorithms: constrained and unconstrained [7]. The unconstrained FBLMS algorithm is more computationally efficient by removing the constrained operations; however, the aliasing caused by circular convolution leads to poorer convergence behavior [8]. Therefore this paper focuses only on the constrained algorithm. The constrained filter update equation in the frequency domain is given by [7] wf ðk þ 1Þ ¼ wf ðkÞ þFGN;0 F 1 μΜf XH f ðkÞef ðkÞ
ð3Þ
where the superscript H represents the conjugate transpose operation, μ is a constant step size, Mf ¼diag[ξ] is a
diagonal matrix with ξ representing a vector containing the normalizing factors for each frequency bin, and " # INN 0NN GN;0 ¼ : ð4Þ 0NN 0NN Note that the non-causal part of the filter coefficients is not affected by the updating process, due to the constraint operation in (3). Therefore, multiplying both sides of (3) by F 1 yields " # " # " # 0N1 wðkÞ wðk þ1Þ ¼ þ μGN;0 ΜXðkÞ ; ð5Þ eðkÞ 0N1 0N1 where e(k)¼[e(kN), e(kN þ1), …, e(kN þN 1)]T, " # X1 X2 XðkÞ ¼ F 1 XH f ðkÞF ¼ X2 X1 is a circulant matrix whose first row is x(k), and " # Μ1 Μ2 Μ ¼ F 1 Μf F ¼ Μ2 Μ1
ð6Þ
ð7Þ
is also a circulant matrix whose first column is F 1 ξ (the inverse Fourier transform of the normalizing vector). With simple derivation, (5) becomes wðk þ1Þ ¼ wðkÞ þ μ½Μ1 X2 þΜ2 X1 eðkÞ
ð8Þ
where eðkÞ ¼ dðkÞ XT2 wðkÞ:
ð9Þ
Taking expectation on both sides of (8) and using the independence assumption with respect to the reference signal and filter coefficients [1,2] yields i h E wðk þ1Þ ¼ INN μΜ1 R μΜ2 R^ E wðkÞ þ μΜ1 r þ μΜ2 r^ ; ð10Þ with
h i R ¼ E X2 XT2 ¼ NR x h i R^ ¼ E X1 XT2 r ¼ E X2 dðkÞ ¼ Nrdx ; r^ ¼ E X1 dðkÞ ;
ð11Þ
where Rx represents the autocorrelation matrix of the reference signal and rdx represents the correlation vector between the reference signal and the desired signal, and both of these are needed for the Wiener solution. The steady-state solution of (10) is i 1 h ð12Þ E w1 ðkÞ ¼ Μ1 R þ Μ2 R^ Μ1 r þ Μ2 r^ : Eq. (12) is a unified description without any assumption on the signal and system models, based on which, the steady-state behavior of the FBLMS algorithm can be investigated. If a constant normalizing factor is used in the frequency domain, i.e., Mf ¼ξI2N 2N, then M2 ¼0N N according to (7), so that E w1 ðkÞ ¼ R 1 r ¼ R x 1 rdx ; ð13Þ which is exactly the causal Wiener filter [1,2].
J. Lu et al. / Signal Processing 104 (2014) 27–32
To accelerate the convergence, the normalizing factors can be set as the reciprocal of the reference signal power spectrum as 1 1 1 ; ; …; ; ð14Þ Mf ¼ diag P0 P1 P 2N 1 where Pi represents the power of the ith frequency bin. For the bin-normalized FBLMS algorithm, the optimal solution can still be achieved under the following two conditions. (a) Sufficient filter length When the adaptive filter is of sufficient length, the desired signal can be described as h iT T ð15Þ dðkÞ ¼ XT2 h ; 01ðN LÞ þuðkÞ; where u(k) is an independent additive noise sequence and h¼[h0, h1, …, hL 1]T is the L-tap (L rN) impulse response of the unknown system. It can be easily found that h iT T ð16Þ r ¼ R h ; 01ðN LÞ and h iT T r^ ¼ R^ h ; 01ðN LÞ :
ð17Þ
Substituting (16) and (17) into (6), the steady-state solution simplifies to E[w1(k)]¼[hT, 01 (N L)]T. This means that there is a perfect match between the adaptive filter and the unknown system. (b) White noise reference signal The normalizing matrix simplifies to Mf ¼I2N 2N/P for a white noise reference signal, where P stands for the constant power spectrum level of the signal. This results in convergence to the Wiener filter since the normalizing factor is uniformly constant. For a colored reference signal, if the reference signal lags behind the desired signal or the control filter is of insufficient length, (12) cannot be simplified to the Wiener solution, which indicates a deterioration of the steady-state behavior of the bin-normalized FBLMS algorithm. These two phenomena have been addressed in [12] (for linear prediction) and [13] (for system identification), respectively. The contribution of this paper is that a unified framework as described above is proposed for analyzing the steady-state behavior of the FBLMS algorithm without making any assumption on the signal and system model, so it can be used to inspect the behavior in all circumstances. Based on the proposed framework, some modifications on the existing algorithm can also be made to improve the performance, as shown below.
then taking expectation on both sides of (18) yields E wðk þ 1Þ ¼ ½INN μΜ1 RE wðkÞ þ μΜ1 r:
29
ð19Þ
The steady-state solution of (19) is E[w1(k)]¼R 1r, and obviously, this means unconditional convergence to the Wiener solution. The updating equation of the FBLMS algorithm needs to be changed to obtain (18). After careful inspection of (3) and (5), it can be found that a simple change of the position of the normalizing factor matrix will achieve this goal. The modified updating equation is wf ðk þ 1Þ ¼ wf ðkÞ þ μΜf FGN;0 F 1 XH f ðkÞef ðkÞ:
ð20Þ
Applying inverse Fourier transformation on both sides of (20) leads to " # " # " # 0N1 wðkÞ wðk þ1Þ ; ð21Þ ¼ þ μΜGN;0 XðkÞ eðkÞ wnc ðkÞ wnc ðk þ 1Þ where wnc(k) represents the non-causal part of the adaptive filter, which does not influence the filtered output. From (21), the updating of the causal part of the filter can then be described exactly in the form of (18). Comparing (3) and (20), it can be found that the computational load of the proposed algorithm for updating the filter coefficients is the same as that of the ordinary FBLMS algorithm. However, some extra constraint operations are needed to eliminate the influence of the noncausal part wnc(k) on the output calculation, as depicted in (1). Block diagrams of both the conventional FBLMS algorithm and the modified FBLMS algorithm are combined in Fig. 1. It can be found that the modified algorithm is easy to implement and the increase of the whole computational load is moderate with only one more FFT/IFFT pair in each block.
4. Simulation To demonstrate the effectiveness of the proposed method, two simulations are conducted with the same setups as those described in [12] (non-causal case) and [13] (deficient filter-length case), respectively. For convenience, the bin-normalized FBLMS algorithm is abbreviated as NFBLMS, and the proposed modified bin-normalized method as MFBLMS. To compare the convergence of these algorithms, coefficient-deviation G is defined as the norm of the difference between the steady-state filter coefficients and the optimal Wiener solution, i.e., qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi G ¼ ðw1 wo ÞT ðw1 wo Þ; ð22Þ where w1 represents the filter coefficients after convergence and wo represents the Wiener solution. 4.1. Non-causal case
3. Modification of the FBLMS algorithm From (8), it can be found that the term M2X1 forms an obstacle to convergence of the FBLMS algorithm. Actually, if the updating of the causal part of the filter can be described as wðk þ1Þ ¼ wðkÞ þ μΜ1 X2 eðkÞ;
ð18Þ
In this example, the reference signal is generated by passing Gaussian white noise with unit variance through a lowpass filter with transfer function H(z)¼[(1–0.5z 1)/ (1–0.6z 1)]16. The desired signal is one sample ahead of the reference signal, resulting in a typical linear prediction problem. A white noise signal, uncorrelated with the reference signal, is added to the desired signal so that the maximum
30
J. Lu et al. / Signal Processing 104 (2014) 27–32
Fig. 1. Block diagram of the conventional and modified FBLMS algorithms. The solid red line and the dotted blue line indicate the signal flow of the conventional and modified FBLMS algorithms, respectively, and the solid black line indicates the signal flow of both algorithms.
1.6
0
NFBLMS
1.2
−5
Amplitude
MSE (dB)
Wiener MFBLMS NFBLMS
1.4
−2.8 dB
−10
−15
1 0.8 0.6 0.4 0.2
MFBLMS −18.7 dB
0 −0.2
−20 0
1
2
3
Samples
4
5
x 104
Fig. 2. Convergence of NFBLMS and MFBLMS algorithms under non-causal conditions.
−0.4 0
2
4
6
8
Filter coefficient index Fig. 3. Steady-state solution of the first 9 filter coefficients in the non-causal case.
attenuation of the mean-square error (MSE) obtained by the Wiener solution is about 18.7 dB. The simulation results are averaged over 100 independent trials. The adaptive filter length N is 128 and a 256-point FFT is used. The step sizes for the NFBLMS and MFBLMS algorithms are 0.001 and 0.00005, respectively, which are close to the upper limits to guarantee both the fastest convergence and stable steadystate behavior. Fig. 2 presents the convergence curves of these two algorithms. It is clearly shown that the steady-state performance of the NFBLMS algorithm deteriorates seriously, with only 2.8 dB attenuation of the MSE. The convergence
speed of the MFBLMS algorithm is initially very fast with 18 dB attenuation obtained within 1500 samples. After that, the slow mode dominates the convergence process and the optimal steady-state performance is achieved at about 20000 samples. The steady-state filter coefficients are depicted in Fig. 3. Note that only the first 9 coefficients are depicted since the rest of the coefficients are all very close to 0. The deviation of the NFBLMS algorithm is obvious, with a coefficient-deviation of 1.0458, while the MFBLMS algorithm converges to the Wiener solution, with a coefficient-deviation of 0.0722.
J. Lu et al. / Signal Processing 104 (2014) 27–32
4.2. Deficient filter-length case In this example, the reference signal is generated by passing Gaussian white noise with unit variance through a 4-tap FIR filter with coefficients [0.1 0.2 0.4 0.7]. The desired signal is generated by passing the reference signal through a 16-tap FIR filter with coefficients [0.01 0.02 0.04 0.08 0.15 0.3 0.45 0.6 0.6 0.45 0.3 0.15 0.08 0.04 0.02 0.01]. A 10-tap adaptive filter is used, resulting in a typical deficient-filter-length scenario. The step size of both algorithms is 0.0001. The simulation results are also averaged over 100 independent trials. From the convergence curve shown in Fig. 4, it can be found that the two algorithms have roughly the same convergence speed for the system modeling problem. However, the MFBLMS algorithm benefits from a lower MSE. As depicted in Fig. 5, the steady-state solution of the NFBLMS algorithm deviates from the Wiener solution, especially at the last several points, while the MFBLMS algorithm converges precisely to the Wiener solution. The
2
A unified framework of the FBLMS algorithm has been proposed in this paper, which can be used to analyze its steady-state behavior in all circumstances. Based on this framework, a modified FBLMS algorithm is proposed to overcome the problem of non-optimal steady-state performance of the commonly used bin-normalized FBLMS algorithm under non-causal and deficient-filter-length conditions. The proposed algorithm guarantees the optimal steady-state behavior at the cost of a moderate increase in computational load. The benefit of the proposed algorithm is clearly demonstrated by the simulations.
MSE (dB)
−2 −4 −6
NFBLMS
−8
MFBLMS 0
2
4
6
8 x 104
Samples
Fig. 4. Convergence of NFBLMS and MFBLMS algorithms under deficientfilter-length conditions.
1
Wiener NFBLMS MFBLMS
0.8
0.4 0.2 0 −0.2
−0.4
0
2
Acknowledgements This work was supported by the National Science Foundation of China Nos. 11374156 and 11204130. The authors would like to extend sincere gratitude to the anonymous reviewer whose comments and suggestions greatly improve the quality of this paper. References
0.6
Amplitude
coefficient-deviation of the NFBLMS and MFBLMS algorithms are 0.2618 and 0.0063, respectively. It should be pointed out that both algorithms perform poorly with less than 10 dB MSE reduction because the filter-length is significantly shorter than that of the unknown model. With an increase of the filter length, both algorithms perform better, but the steady-state filter coefficients of the NFBLMS algorithm still deviates more than that of the MFBLMS algorithm, as long as the filter is of deficient length. For example, with a filter length of 13, the MSE reduction level of the NFBLMS algorithm is 25.6 dB with a coefficient-deviation of 0.0366, while the MSE reduction level of the MFBLMS algorithm is 27.1 dB with a coefficient-deviation of 0.0006. These simulations clearly demonstrate the superior steady-state behavior of the proposed MFBLMS algorithm in both non-causal and deficient filter-length circumstances. Although the convergence speed might be slower than that of the NFBLMS algorithm in some situations, the guaranteed optimal steady-state behavior still makes this algorithm a good option for many applications. 5. Conclusions
0
−10
31
4
6
8
Filter coefficient index Fig. 5. Steady-state solution of the filter coefficients in the deficientfilter-length case.
[1] S. Haykin, Adaptive Filter Theory, fourth ed. Prentice Hall, New Jersey, 2001. [2] B. Farhang-Boroujeny, Adaptive Filters: Theory and Applications, Wiley, Chichester, U.K., 1998. [3] M. Kamenetsky, Accelerating the Convergence of the LMS Adaptive Algorithm, Ph.D. Dissertation, Stanford University, Stanford, CA, 2005. [4] S. Zhao, Z. Man, S. Khoo, H. Wu, Stability and convergence analysis of transform-domain LMS adaptive filters with second-order autoregressive process, IEEE Trans. Signal Process. 57 (1) (2009) 119–130. [5] K. Dogancay, Complexity consideration for transform-domain adaptive filters, Signal Process. 83 (2003) 1177–1192. [6] B. Farhang-Boroujeny, Y. Lee, C.C. Ko, Sliding transforms for efficient implementation of transform domain adaptive filters, Signal Process. 52 (1996) 83–96. [7] J.J. Shynk, Frequency-domain and multirate adaptive filtering, IEEE Signal Process. Mag. (1992) 14–37. (Jan).
32
J. Lu et al. / Signal Processing 104 (2014) 27–32
[8] B. Farhang-Boroujeny, K.S. Chan, Analysis of the frequency-domain block LMS algorithm, IEEE Trans. Signal Process. 48 (8) (2000) 2332–2342. [9] H. Buchner, J. Benesty, W. Kellermann, Generalized multichannel frequency-domain adaptive filtering: efficient realization and application to hands-free speech communication, Signal Process. 85 (2005) 549–570. [10] M. Wu, G. Chen, X. Qiu, An improved active noise control algorithm without secondary path identification based on the frequencydomain subband architecture, IEEE Trans. Audio Speech Lang. Process. 16 (8) (2008) 1409–1419.
[11] Y. Huang, J. Benesty, A class of frequency-domain adaptive approaches to blind multichannel identification, IEEE Trans. Signal Process. 51 (1) (2003) 11–24. [12] S.J. Elliott, B. Rafaely, Frequency-domain adaptation of causal digital filters, IEEE Trans. Signal Process. 48 (5) (2000) 1354–1364. [13] M. Wu, J. Yang, Y. Xu, X. Qiu, Steady-state solution of the deficient length constrained FBLMS algorithm, IEEE Trans. Signal Process. 60 (12) (2012) 6681–6687.
Contents lists available at ScienceDirect
Signal Processing journal homepage: www.elsevier.com/locate/sigpro
Fast communication
A modified frequency-domain block LMS algorithm with guaranteed optimal steady-state performance Jing Lu n, Xiaojun Qiu, Haishan Zou Key Laboratory of Modern Acoustics, Institute of Acoustics, Nanjing University, Nanjing 210093, China
a r t i c l e i n f o
abstract
Article history: Received 29 November 2013 Received in revised form 13 March 2014 Accepted 21 March 2014 Available online 28 March 2014
The bin-normalized frequency-domain block LMS (FBLMS) algorithm has low computational burden and potential fast convergence; however, it suffers from a biased steadystate solution when the reference signal lags behind the desired signal or the adaptive filter is of insufficient length. This paper proposes a unified framework for the FBLMS algorithm, which can be used to comprehensively analyze its steady-state behavior. Furthermore, a modified FBLMS algorithm with guaranteed optimal steady-state performance is proposed based on the framework. Simulations are carried out to demonstrate the benefit of the proposed algorithm. & 2014 Elsevier B.V. All rights reserved.
Key words: Adaptive filters Frequency-domain implementation Steady-state behavior
1. Introduction Adaptive filtering has been widely used in many situations such as telecommunication systems, acoustic echo cancellation, active noise control, and array processing, where the least-mean-square (LMS) algorithm is commonly used due to its simplicity and robustness [1,2]. Unfortunately, it suffers from slow convergence for reference signals with large eigenvalue disparity, and moreover, its computational burden is too heavy in many application scenarios because the filter length has to be set very large [1,2]. To overcome the problem of slow convergence, transform-domain LMS (TDLMS) algorithms [1–4] have been suggested, which preprocess the reference signal by using orthogonal transforms such as the discrete Fourier transform (DFT), discrete cosine transform (DCT), discrete sine transform (DST), and discrete Hartely transform (DHT), and then set power-normalized step sizes. The improvement of the convergence rate has been proven by many
n
Corresponding author. Tel.: þ86 25 83593571. E-mail address: [email protected] (J. Lu).
http://dx.doi.org/10.1016/j.sigpro.2014.03.029 0165-1684/& 2014 Elsevier B.V. All rights reserved.
researchers [3,4]. However, the computational burden of the TDLMS algorithms is substantially heavier than that of the LMS algorithm because the orthogonal transforms are often performed for each new input sample. Although partial updating and sliding transform techniques [5,6] can be used to mitigate the problem, the computational burden is still a challenge for implementation of the TDLMS algorithms in real-time systems. Apart from the application of the TDLMS algorithm, the DFT in particular can also be used to realize the frequencydomain block least-mean-square (FBLMS) algorithm [7], which is a computational efficient implementation of the block LMS (BLMS) algorithm. The computational burden of the FBLMS algorithm is significantly less than that of the LMS algorithm because the fast Fourier transform (FFT) is used to calculate both the block filtering output and the update terms in the frequency domain. Furthermore, when the step size of the adaptive filter is normalized by the reference signal power in each frequency bin, the convergence speed of the FBLMS algorithm can be significantly increased for reference signals with large power spectral disparity [7,8]. Therefore the bin-normalized FBLMS algorithm is widely used in many applications that require
28
J. Lu et al. / Signal Processing 104 (2014) 27–32
large filter length and fast convergence, e.g., acoustic echo cancellation, active noise control, channel estimation, and equalization [9–11]. Nevertheless, it has been pointed out that the bin-normalized FBLMS algorithm suffers from an increase in the steady-state mean-square error in noncausal circumstances [12] or with insufficient filter length [13], which is very common in many applications of adaptive filters. For example, the reference signal can lag behind the desired signal in an adaptive equalizer or in an adaptive feedback active noise control system. On the other hand, for an acoustic echo cancellation systems used in a room with long reverberation time, it is often the case that the adaptive filter is of insufficient length. A frequency-domain Newton's algorithm has been derived in [12] to improve the steady-state behavior in non-causal circumstances. However, it requires a spectral factorization of the estimated power spectral density of the reference signal, which forms an obstacle to its implementation. In this paper, a unified framework of the FBLMS algorithm without any assumptions on the signal and system model is proposed, which can be used to comprehensively analyze the steady-state behavior of the algorithm. Based on this framework, a modification is proposed on the existing algorithm that guarantees optimal steady-state behavior. Throughout this paper, lowercase letters are used for scalar quantities, bold lowercase for vectors, and bold uppercase for matrices. Subscript f denotes frequency-domain representation of each signal and k is reserved for the block index. 2. Analysis of FBLMS steady-state behavior based on a unified framework Let x(k) ¼[x(kN N), x(kN N þ1), …, x(kN þN 1)]T be the reference signal vector, where the superscript T represents the transpose operation, w(k)¼[w0(k), w1(k), …, wN 1(k)]T be the N-tap filter, and d(k)¼ [d(kN N), d(kNNþ1), …, d(kNþN1)]T be the desired signal vector. Then the error vector in the frequency domain can be described as ef ðkÞ ¼ FG0;N F 1 df ðkÞ Xf ðkÞwf ðkÞ ð1Þ where F represents the 2N 2N discrete Fourier transform (DFT) matrix, df(k)¼F[01 N, dT(k)]T, Xf(k)¼diag[xf(k)]¼diag [Fx(k)], wf(k)¼F[wT(k), 01 N]T, and " # 0NN 0NN G0;N ¼ : ð2Þ 0NN INN There are two kinds of FBLMS algorithms: constrained and unconstrained [7]. The unconstrained FBLMS algorithm is more computationally efficient by removing the constrained operations; however, the aliasing caused by circular convolution leads to poorer convergence behavior [8]. Therefore this paper focuses only on the constrained algorithm. The constrained filter update equation in the frequency domain is given by [7] wf ðk þ 1Þ ¼ wf ðkÞ þFGN;0 F 1 μΜf XH f ðkÞef ðkÞ
ð3Þ
where the superscript H represents the conjugate transpose operation, μ is a constant step size, Mf ¼diag[ξ] is a
diagonal matrix with ξ representing a vector containing the normalizing factors for each frequency bin, and " # INN 0NN GN;0 ¼ : ð4Þ 0NN 0NN Note that the non-causal part of the filter coefficients is not affected by the updating process, due to the constraint operation in (3). Therefore, multiplying both sides of (3) by F 1 yields " # " # " # 0N1 wðkÞ wðk þ1Þ ¼ þ μGN;0 ΜXðkÞ ; ð5Þ eðkÞ 0N1 0N1 where e(k)¼[e(kN), e(kN þ1), …, e(kN þN 1)]T, " # X1 X2 XðkÞ ¼ F 1 XH f ðkÞF ¼ X2 X1 is a circulant matrix whose first row is x(k), and " # Μ1 Μ2 Μ ¼ F 1 Μf F ¼ Μ2 Μ1
ð6Þ
ð7Þ
is also a circulant matrix whose first column is F 1 ξ (the inverse Fourier transform of the normalizing vector). With simple derivation, (5) becomes wðk þ1Þ ¼ wðkÞ þ μ½Μ1 X2 þΜ2 X1 eðkÞ
ð8Þ
where eðkÞ ¼ dðkÞ XT2 wðkÞ:
ð9Þ
Taking expectation on both sides of (8) and using the independence assumption with respect to the reference signal and filter coefficients [1,2] yields i h E wðk þ1Þ ¼ INN μΜ1 R μΜ2 R^ E wðkÞ þ μΜ1 r þ μΜ2 r^ ; ð10Þ with
h i R ¼ E X2 XT2 ¼ NR x h i R^ ¼ E X1 XT2 r ¼ E X2 dðkÞ ¼ Nrdx ; r^ ¼ E X1 dðkÞ ;
ð11Þ
where Rx represents the autocorrelation matrix of the reference signal and rdx represents the correlation vector between the reference signal and the desired signal, and both of these are needed for the Wiener solution. The steady-state solution of (10) is i 1 h ð12Þ E w1 ðkÞ ¼ Μ1 R þ Μ2 R^ Μ1 r þ Μ2 r^ : Eq. (12) is a unified description without any assumption on the signal and system models, based on which, the steady-state behavior of the FBLMS algorithm can be investigated. If a constant normalizing factor is used in the frequency domain, i.e., Mf ¼ξI2N 2N, then M2 ¼0N N according to (7), so that E w1 ðkÞ ¼ R 1 r ¼ R x 1 rdx ; ð13Þ which is exactly the causal Wiener filter [1,2].
J. Lu et al. / Signal Processing 104 (2014) 27–32
To accelerate the convergence, the normalizing factors can be set as the reciprocal of the reference signal power spectrum as 1 1 1 ; ; …; ; ð14Þ Mf ¼ diag P0 P1 P 2N 1 where Pi represents the power of the ith frequency bin. For the bin-normalized FBLMS algorithm, the optimal solution can still be achieved under the following two conditions. (a) Sufficient filter length When the adaptive filter is of sufficient length, the desired signal can be described as h iT T ð15Þ dðkÞ ¼ XT2 h ; 01ðN LÞ þuðkÞ; where u(k) is an independent additive noise sequence and h¼[h0, h1, …, hL 1]T is the L-tap (L rN) impulse response of the unknown system. It can be easily found that h iT T ð16Þ r ¼ R h ; 01ðN LÞ and h iT T r^ ¼ R^ h ; 01ðN LÞ :
ð17Þ
Substituting (16) and (17) into (6), the steady-state solution simplifies to E[w1(k)]¼[hT, 01 (N L)]T. This means that there is a perfect match between the adaptive filter and the unknown system. (b) White noise reference signal The normalizing matrix simplifies to Mf ¼I2N 2N/P for a white noise reference signal, where P stands for the constant power spectrum level of the signal. This results in convergence to the Wiener filter since the normalizing factor is uniformly constant. For a colored reference signal, if the reference signal lags behind the desired signal or the control filter is of insufficient length, (12) cannot be simplified to the Wiener solution, which indicates a deterioration of the steady-state behavior of the bin-normalized FBLMS algorithm. These two phenomena have been addressed in [12] (for linear prediction) and [13] (for system identification), respectively. The contribution of this paper is that a unified framework as described above is proposed for analyzing the steady-state behavior of the FBLMS algorithm without making any assumption on the signal and system model, so it can be used to inspect the behavior in all circumstances. Based on the proposed framework, some modifications on the existing algorithm can also be made to improve the performance, as shown below.
then taking expectation on both sides of (18) yields E wðk þ 1Þ ¼ ½INN μΜ1 RE wðkÞ þ μΜ1 r:
29
ð19Þ
The steady-state solution of (19) is E[w1(k)]¼R 1r, and obviously, this means unconditional convergence to the Wiener solution. The updating equation of the FBLMS algorithm needs to be changed to obtain (18). After careful inspection of (3) and (5), it can be found that a simple change of the position of the normalizing factor matrix will achieve this goal. The modified updating equation is wf ðk þ 1Þ ¼ wf ðkÞ þ μΜf FGN;0 F 1 XH f ðkÞef ðkÞ:
ð20Þ
Applying inverse Fourier transformation on both sides of (20) leads to " # " # " # 0N1 wðkÞ wðk þ1Þ ; ð21Þ ¼ þ μΜGN;0 XðkÞ eðkÞ wnc ðkÞ wnc ðk þ 1Þ where wnc(k) represents the non-causal part of the adaptive filter, which does not influence the filtered output. From (21), the updating of the causal part of the filter can then be described exactly in the form of (18). Comparing (3) and (20), it can be found that the computational load of the proposed algorithm for updating the filter coefficients is the same as that of the ordinary FBLMS algorithm. However, some extra constraint operations are needed to eliminate the influence of the noncausal part wnc(k) on the output calculation, as depicted in (1). Block diagrams of both the conventional FBLMS algorithm and the modified FBLMS algorithm are combined in Fig. 1. It can be found that the modified algorithm is easy to implement and the increase of the whole computational load is moderate with only one more FFT/IFFT pair in each block.
4. Simulation To demonstrate the effectiveness of the proposed method, two simulations are conducted with the same setups as those described in [12] (non-causal case) and [13] (deficient filter-length case), respectively. For convenience, the bin-normalized FBLMS algorithm is abbreviated as NFBLMS, and the proposed modified bin-normalized method as MFBLMS. To compare the convergence of these algorithms, coefficient-deviation G is defined as the norm of the difference between the steady-state filter coefficients and the optimal Wiener solution, i.e., qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi G ¼ ðw1 wo ÞT ðw1 wo Þ; ð22Þ where w1 represents the filter coefficients after convergence and wo represents the Wiener solution. 4.1. Non-causal case
3. Modification of the FBLMS algorithm From (8), it can be found that the term M2X1 forms an obstacle to convergence of the FBLMS algorithm. Actually, if the updating of the causal part of the filter can be described as wðk þ1Þ ¼ wðkÞ þ μΜ1 X2 eðkÞ;
ð18Þ
In this example, the reference signal is generated by passing Gaussian white noise with unit variance through a lowpass filter with transfer function H(z)¼[(1–0.5z 1)/ (1–0.6z 1)]16. The desired signal is one sample ahead of the reference signal, resulting in a typical linear prediction problem. A white noise signal, uncorrelated with the reference signal, is added to the desired signal so that the maximum
30
J. Lu et al. / Signal Processing 104 (2014) 27–32
Fig. 1. Block diagram of the conventional and modified FBLMS algorithms. The solid red line and the dotted blue line indicate the signal flow of the conventional and modified FBLMS algorithms, respectively, and the solid black line indicates the signal flow of both algorithms.
1.6
0
NFBLMS
1.2
−5
Amplitude
MSE (dB)
Wiener MFBLMS NFBLMS
1.4
−2.8 dB
−10
−15
1 0.8 0.6 0.4 0.2
MFBLMS −18.7 dB
0 −0.2
−20 0
1
2
3
Samples
4
5
x 104
Fig. 2. Convergence of NFBLMS and MFBLMS algorithms under non-causal conditions.
−0.4 0
2
4
6
8
Filter coefficient index Fig. 3. Steady-state solution of the first 9 filter coefficients in the non-causal case.
attenuation of the mean-square error (MSE) obtained by the Wiener solution is about 18.7 dB. The simulation results are averaged over 100 independent trials. The adaptive filter length N is 128 and a 256-point FFT is used. The step sizes for the NFBLMS and MFBLMS algorithms are 0.001 and 0.00005, respectively, which are close to the upper limits to guarantee both the fastest convergence and stable steadystate behavior. Fig. 2 presents the convergence curves of these two algorithms. It is clearly shown that the steady-state performance of the NFBLMS algorithm deteriorates seriously, with only 2.8 dB attenuation of the MSE. The convergence
speed of the MFBLMS algorithm is initially very fast with 18 dB attenuation obtained within 1500 samples. After that, the slow mode dominates the convergence process and the optimal steady-state performance is achieved at about 20000 samples. The steady-state filter coefficients are depicted in Fig. 3. Note that only the first 9 coefficients are depicted since the rest of the coefficients are all very close to 0. The deviation of the NFBLMS algorithm is obvious, with a coefficient-deviation of 1.0458, while the MFBLMS algorithm converges to the Wiener solution, with a coefficient-deviation of 0.0722.
J. Lu et al. / Signal Processing 104 (2014) 27–32
4.2. Deficient filter-length case In this example, the reference signal is generated by passing Gaussian white noise with unit variance through a 4-tap FIR filter with coefficients [0.1 0.2 0.4 0.7]. The desired signal is generated by passing the reference signal through a 16-tap FIR filter with coefficients [0.01 0.02 0.04 0.08 0.15 0.3 0.45 0.6 0.6 0.45 0.3 0.15 0.08 0.04 0.02 0.01]. A 10-tap adaptive filter is used, resulting in a typical deficient-filter-length scenario. The step size of both algorithms is 0.0001. The simulation results are also averaged over 100 independent trials. From the convergence curve shown in Fig. 4, it can be found that the two algorithms have roughly the same convergence speed for the system modeling problem. However, the MFBLMS algorithm benefits from a lower MSE. As depicted in Fig. 5, the steady-state solution of the NFBLMS algorithm deviates from the Wiener solution, especially at the last several points, while the MFBLMS algorithm converges precisely to the Wiener solution. The
2
A unified framework of the FBLMS algorithm has been proposed in this paper, which can be used to analyze its steady-state behavior in all circumstances. Based on this framework, a modified FBLMS algorithm is proposed to overcome the problem of non-optimal steady-state performance of the commonly used bin-normalized FBLMS algorithm under non-causal and deficient-filter-length conditions. The proposed algorithm guarantees the optimal steady-state behavior at the cost of a moderate increase in computational load. The benefit of the proposed algorithm is clearly demonstrated by the simulations.
MSE (dB)
−2 −4 −6
NFBLMS
−8
MFBLMS 0
2
4
6
8 x 104
Samples
Fig. 4. Convergence of NFBLMS and MFBLMS algorithms under deficientfilter-length conditions.
1
Wiener NFBLMS MFBLMS
0.8
0.4 0.2 0 −0.2
−0.4
0
2
Acknowledgements This work was supported by the National Science Foundation of China Nos. 11374156 and 11204130. The authors would like to extend sincere gratitude to the anonymous reviewer whose comments and suggestions greatly improve the quality of this paper. References
0.6
Amplitude
coefficient-deviation of the NFBLMS and MFBLMS algorithms are 0.2618 and 0.0063, respectively. It should be pointed out that both algorithms perform poorly with less than 10 dB MSE reduction because the filter-length is significantly shorter than that of the unknown model. With an increase of the filter length, both algorithms perform better, but the steady-state filter coefficients of the NFBLMS algorithm still deviates more than that of the MFBLMS algorithm, as long as the filter is of deficient length. For example, with a filter length of 13, the MSE reduction level of the NFBLMS algorithm is 25.6 dB with a coefficient-deviation of 0.0366, while the MSE reduction level of the MFBLMS algorithm is 27.1 dB with a coefficient-deviation of 0.0006. These simulations clearly demonstrate the superior steady-state behavior of the proposed MFBLMS algorithm in both non-causal and deficient filter-length circumstances. Although the convergence speed might be slower than that of the NFBLMS algorithm in some situations, the guaranteed optimal steady-state behavior still makes this algorithm a good option for many applications. 5. Conclusions
0
−10
31
4
6
8
Filter coefficient index Fig. 5. Steady-state solution of the filter coefficients in the deficientfilter-length case.
[1] S. Haykin, Adaptive Filter Theory, fourth ed. Prentice Hall, New Jersey, 2001. [2] B. Farhang-Boroujeny, Adaptive Filters: Theory and Applications, Wiley, Chichester, U.K., 1998. [3] M. Kamenetsky, Accelerating the Convergence of the LMS Adaptive Algorithm, Ph.D. Dissertation, Stanford University, Stanford, CA, 2005. [4] S. Zhao, Z. Man, S. Khoo, H. Wu, Stability and convergence analysis of transform-domain LMS adaptive filters with second-order autoregressive process, IEEE Trans. Signal Process. 57 (1) (2009) 119–130. [5] K. Dogancay, Complexity consideration for transform-domain adaptive filters, Signal Process. 83 (2003) 1177–1192. [6] B. Farhang-Boroujeny, Y. Lee, C.C. Ko, Sliding transforms for efficient implementation of transform domain adaptive filters, Signal Process. 52 (1996) 83–96. [7] J.J. Shynk, Frequency-domain and multirate adaptive filtering, IEEE Signal Process. Mag. (1992) 14–37. (Jan).
32
J. Lu et al. / Signal Processing 104 (2014) 27–32
[8] B. Farhang-Boroujeny, K.S. Chan, Analysis of the frequency-domain block LMS algorithm, IEEE Trans. Signal Process. 48 (8) (2000) 2332–2342. [9] H. Buchner, J. Benesty, W. Kellermann, Generalized multichannel frequency-domain adaptive filtering: efficient realization and application to hands-free speech communication, Signal Process. 85 (2005) 549–570. [10] M. Wu, G. Chen, X. Qiu, An improved active noise control algorithm without secondary path identification based on the frequencydomain subband architecture, IEEE Trans. Audio Speech Lang. Process. 16 (8) (2008) 1409–1419.
[11] Y. Huang, J. Benesty, A class of frequency-domain adaptive approaches to blind multichannel identification, IEEE Trans. Signal Process. 51 (1) (2003) 11–24. [12] S.J. Elliott, B. Rafaely, Frequency-domain adaptation of causal digital filters, IEEE Trans. Signal Process. 48 (5) (2000) 1354–1364. [13] M. Wu, J. Yang, Y. Xu, X. Qiu, Steady-state solution of the deficient length constrained FBLMS algorithm, IEEE Trans. Signal Process. 60 (12) (2012) 6681–6687.
