Performance of Belief Propagation for Decoding LDPC codes in the ...
Performance of Belief Propagation for Decoding LDPC codes in the Presence of Channel Estimation Error1 Hamid Saeedi and Amir H. Banihashemi Dept. of Systems and Computer Engineering, Carleton University 1125 Colonel By Dr., K1S 5B6, Ottawa, Canada {hsaeedi, ahashemi}@sce.carleton.ca
Abstract— In this paper we investigate the performance of belief propagation (BP) algorithm for decoding low density parity check (LDPC) codes over the additive white Gaussian noise (AWGN) channel when there is an incorrect estimate of the channel signal to noise ratio (SNR) (referred to as SNR mismatch) at the decoder. We propose a computationally efficient method based on the Gaussian approximation of density evolution to compute the threshold values for regular and irregular codes in the presence of SNR mismatch and show that these values are consistent with simulation results for the codes with finite block lengths. At the extremes for over- and under-estimation of SNR, the performance of BP tends to that of min-sum (MS) algorithm and the channel bit error rate, respectively. Our results for regular codes indicate that the sensitivity to mismatch increases by increasing the variable node degree and by decreasing the check node degree. The effect of variable node degree however appears to be more profound such that at a given rate, the codes with the smallest variable and check degrees are more robust against SNR mismatch. For irregular codes, by comparing the thresholds of a few ensembles, we demonstrate that the ensemble which performs better in the absence of mismatch can perform worse in the presence of it. Index Terms -- LDPC codes, SNR mismatch, SNR estimation error, Belief propagation, Min-sum, SNR threshold, , Density evolution, Gaussian approximation 1
A Preliminary version of this work has been accepted for presentation in part at IEEE VTC2005-Spring, Stockholm, Sweden, May 30 - June 1, 2005 and in part at CWIT 2005, Montreal, Quebec, Canada, June 5 – 8, 2005.
I.
INTRODUCTION AND PROBLEM FORMULATION
Different tools have been proposed to analyze asymptotic performance of LDPC codes. The most accurate method called density evolution (DE) [1], [2] is based on iterative calculation of densities of the messages passed between check nodes and variable nodes. For soft messages, however, this method has a high complexity. Other methods have been proposed in [3]-[5] based on Gaussian approximation of densities to reduce the complexity at the expense of less accuracy. The method suggested in [3] is based on tracking the mean of the Gaussian estimated version of the messages while the approximation proposed in [4], [5] is based on keeping track of the mutual information between the transmitted bit and the output messages of variable nodes and check nodes through extrinsic information transfer (EXIT) charts. In belief propagation (BP) algorithm for LDPC codes over the binary input additive white Gaussian noise (BIAWGN) channel, the knowledge of the channel noise power is necessary to achieve the ultimate performance of the algorithm. Consider a BIAWGN channel with input X that can take values ±1, and output Y=X + N, where N is a Gaussian random variable with mean zero and variance σ2 (N (0, σ2)). In the log-likelihood ratio (LLR) domain, the input to the BP algorithm for each bit should be M0=2Y/σ2. Starting from such initial messages, which is based on a perfect knowledge of the noise power σ2, BP algorithm performs optimally in the sense that it provides the a-posteriori probabilities (APP) for bits assuming a tree-like (cycle-free) graph. Suppose that the all-one codeword is transmitted. The input LLRs for all the bits are then i.i.d. with distribution N (2/σ2, 4/σ2). This distribution is symmetric (consistent) [1]-[3] with the ratio of variance to mean (VMR) equal to two. As the symmetry property is preserved throughout the iterations of BP algorithm, the method of Gaussian approximation (GA) proposed in [3] can be used to only track the mean of the Gaussian densities throughout the iterative process. This
reduces the dimension of the problem from infinity (for full DE) to one, and significantly simplifies the analysis.
) Now consider the case where the noise power is estimated erroneously as σ 2 ≠ σ2. We refer to
) this as “signal to noise ratio (SNR) mismatch,” and define the SNR mismatch ratio η by σ2 / σ 2 . The value of η in dB is called SNR offset. In the presence of SNR mismatch, the input LLRs to BP
) ) algorithm have distribution N (2/ σ 2 , 4σ2/ σ 4 ). This distribution is not symmetric and has VMR = ) 2 σ2/ σ 2 . In Fig. 1, for the first nine iterations, we have shown the values of VMR at the output of variable nodes (input of check nodes) for an ensemble of (4,6)-regular LDPC codes at SNR = 3 dB and for η = ±2 dB. (Note that SNR = Eb/N0 = 1/(2Rσ2), where Eb, N0 and R are the average energy per information bit, one-sided power spectral density of AWGN and the code rate, respectively.) This figure shows that in the case of mismatch, not only VMR is not equal to 2, but also it changes throughout the iterative process. In this paper to investigate the asymptotic performance of BP in the presence of mismatch, we employ a GA method in which both the mean and the variance of the distributions are tracked. The performance of BP with imperfect channel estimation has been addressed in [6]-[8] for turbo codes and in [9], [10] for LDPC codes. In [9], simulation results for a few codes with finite block lengths are presented, while in [10], DE has also been used to analyze the performance of BP with mismatch in the context of magnetic recording channels for a regular LDPC code ensemble. For the AWGN channel, the results of [9] and [10] indicate that the algorithm is more sensitive to under-estimation of SNR than it is to its over-estimation. In this work, we perform asymptotic analysis of BP with mismatch over the AWGN channel for both regular and irregular codes and demonstrate that our asymptotic results are consistent with finite-length simulations. Compared to the DE approach of [10], our proposed method, which is based on GA of DE, is much simpler and thus can be easily applied to irregular codes. We demonstrate that in the limit when the mismatch offset tends to infinity, the performance of BP tends to that of min-sum (MS)
algorithm [11] and the channel bit error rate (BER) for over- and under-estimation of SNR, respectively. This explains why the algorithm is in general more sensitive to under-estimation than it is to over-estimation of SNR. Using the proposed method, we also investigate the sensitivity of BP to SNR mismatch as a function of variable and check node degree distributions. Our results show that for regular codes the sensitivity increases by increasing the variable degree (dv) and by decreasing the check degree (dc) with the effect of the former more prominent than that of the latter. This means that at a given rate the codes with smaller dv and dc are more robust against SNR mismatch. An implication of this is that, at least for the tested cases, the regular codes of a given rate which perform better in the absence of SNR mismatch are also more robust in the presence of it. For irregular codes however this is not necessarily the case as shown by an example. It is well-known that BP is superior to MS in performance. Our result that MS is in fact the limit of BP as the SNR overestimation tends to infinity makes it possible to apply the results of this work to relate the degradation in performance of MS with respect to BP to the degree distributions of the code. This, for example, implies that among regular LDPC codes of the same rate, codes with smaller dv and dc experience smaller gap between the performances of BP and MS. The rest of the paper is organized as follows: In the next section, we explain the proposed GA method, define the SNR threshold value of an ensemble of codes for a given mismatch offset, and also define mismatch thresholds for a given channel SNR. We end the section by discussing the asymptotic performance of BP as mismatch offset tends to ± ∞. In section III, simulation results are presented and discussed. Section VI is devoted to some concluding remarks.
II. GAUSSIAN APPROXIMATION, THRESHOLDS AND ASYMPTOTIC PERFORMANCE OF BP IN THE PRESENCE OF MISMATCH A. The proposed method based on Gaussian approximation For regular LDPC codes, the method of [3] approximates the density of messages throughout the iterative process by symmetric Gaussian distributions and tracks the mean of these distributions using the following two equations at variable and check nodes, respectively: mout = (dv-1) min + m0 ,
(1)
mout = g-1( g(min) dc-1) ,
(2)
where min and mout are the means of the input and output distributions, respectively, m0 is the mean of the initial LLRs inputted to the BP algorithm, and function g(x) is the expected value of tanh(N (x,2x)/2). For the case of mismatch, as the symmetry condition is violated and as the ratio of variance to mean changes with iterations (see Fig. 1), we need to track both the mean and the variance of the random variables. We however maintain the Gaussian assumption. For variable nodes, we still use (1) to track the means and employ the following equation for variances: σ2out = (dv-1) σ2in + σ20 .
(3)
For check nodes, we define the following 2-input 2-output mapping: G(m, σ)=( g1(m,σ), g2(m,σ) ) =( E[tanh(N(m,σ2)/2], E[tanh2(N(m,σ2)/2)] ) ,
(4)
where E[.] denotes the expected value, and track the means and variances using (mout , σout )= G-1( g1d c −1 (min , σin ), g 2d c −1 (min , σin )) .
(5)
We compute the mappings G and G-1 numerically and use look-up tables to track the means and variances. In Figures 2(a) and (b), we have shown the functions g1(m,σ) and g2(m,σ), respectively.
To generalize this analysis to irregular codes similar steps as those described in [3] are taken. Dv
Dc
i=2
i =2
Assume an irregular code ensemble with degree distributions λ ( x) = ∑ λi x i −1 and ρ ( x) = ∑ ρ i x i −1 for variable nodes and check nodes respectively, where the coefficient of xi represents the percentage of edges connected to nodes of degree i+1, and Dv and Dc are the maximum variable and check node degrees, respectively. The distribution of messages passed between variable nodes and check nodes in this case are approximated by Gaussian mixtures. Suppose that, at a given iteration ℓ, the Gaussian mixture at the output of check nodes and input of variable nodes has mean and variance min and σ2in, respectively. For variable nodes of degree i, we estimate the output by a Gaussian and compute its mean and variance using
mout,i = (i-1) min + m0 , σ2out,i = (i-1) σ2in + σ20
,
respectively. The density of the messages from variable nodes to check nodes is then approximated by the Gaussian mixture ∑i λi N (mout,i , σ2out,i). We now define h1 and h2 as follows: Dv
(h1 , h2 ) = ∑ λi G(mout,i , σ2out,i) . i=2
The output of a check node of degree j is then approximated by a Gaussian with the following mean and variance (mout,j , σ2out,j ) = G -1 (h1j −1 , h2j −1 ) . The distribution of messages from check nodes to variable nodes is then approximated by the Gaussian mixture ∑j ρj N (mout,j , σ2out,j). The mean and the variance of this distribution are then obtained by the following equations to be used as inputs to variable nodes for iteration ℓ+1. Dc
mout = ∑ ρ j mout , j , j =2
σ
2 out
Dc
(
)
2 2 2 = ∑ ρ j σ out , j + mout , j − mout . j =2
To examine the accuracy of the proposed method, we have compared its threshold results to those generated by density evolution for a few cases. Our results indicate that with steps of 0.01 for mean and variance, the difference between the exact and approximate thresholds is at most 0.1dB and 0.2dB for regular and irregular codes, respectively2. We note that this inaccuracy is mainly due to the Gaussian approximation [3], but marginal improvement can be obtained by reducing the step size. B. Thresholds Threshold values are used to measure the asymptotic performance of code ensembles under iterative decoding algorithms (see, e.g., [1]-[3]). For a given ensemble and an SNR offset η, we define the SNR threshold α(η) as the infimum of all channel SNRs such that the bit error rate converges to zero as the iteration number tends to infinity. It is well-known that SNR thresholds in the presence of mismatch are always higher than that of no-mismatch case, i.e., α(η) > α(0), ∀ η ≠ 0 dB [10]. We measure the sensitivity of an ensemble of codes to mismatch by the
difference ∆α(η) = α(η) - α(0) (dB). To quantify the amount of tolerable mismatch for a given code ensemble at a given SNR
s > α(0), we also define mismatch thresholds β ( o ) ( s ) and β (u ) ( s ) for over- and underestimation of SNR, respectively. For over- (under)estimation, this is defined as supremum (infimum) of SNR offset such that the bit error rate converges to zero as the iteration number tends to infinity. We use the difference ∆β(s) = β ( o ) ( s ) - β (u ) ( s ) as the measure of tolerance. C. Asymptotic performance of BP at offset limits 1) SNR underestimation
) Assume that the mismatch offset tends to minus infinity. In this case the estimate σ 2 of the ) channel noise variance is very large. This makes the initial LLR values of m0 = 2y/ σ 2 very small. Consider the first iteration of BP, where the initial messages are passed from variable 2
In line with our results, it can be seen in [3] as well that the Gaussian approximation is generally less accurate for irregular codes.
nodes to check nodes. Without loss of generality, consider a check node of degree three with two non-negative extrinsic inputs L1 and L2. The output is then equal to (see, e.g., [11]) L3 = min( L1 , L2 ) + ln(1 + e − ( L1 + L2 ) ) − ln(1 + e
− L1 − L2
).
(6)
Using the Taylor series, for small values of x, we have ln(1 + e − x ) ≈ 1 − x / 2 . Applying this to the
second and third terms of (6), we obtain L3 ≈ 0 . This implies that the messages passed by check nodes to variable nodes are negligible compared to the initial LLR values. This situation will continue throughout the iterations and thus the bit error rate tends to that of the channel, which is ∞
equal to Q(1/σ), where Q( x) = 1 / 2π ∫ e − x x
2
/2
dx .
2) SNR overestimation
In the case of SNR overestimation, the initial LLR values, and thus all the messages passed from variable nodes to check nodes throughout the iterations, are very large. In this case, as the magnitude of the difference between the second and the third terms of (6) is upper bounded by ln(2), we have L3 ≈ min( L1 , L2 ) . This is the check operation of min-sum (see, e.g., [11], [12]). As the variable operations of BP and MS are also the same, this implies that the performance of BP tends to that of min-sum as mismatch offset tends to plus infinity.
III. SIMULATION RESULTS We first consider the ensemble of (3,6)-regular LDPC codes and investigate its asymptotic performance under BP in the presence of mismatch by computing the mismatch thresholds
β ( o ) ( s) and β (u ) ( s) for s = 1.5, 2.0 and 2.5 dB, using our proposed GA method. These are shown in Fig. 3. Note that β (u ) (2) = β (u ) (2.5) = +∞. This is in agreement with the fact that both SNR values are larger than the MS threshold for this ensemble which is about 1.7 dB [12]. In Fig. 3, we have also plotted BER vs. SNR offset for a randomly constructed (3,6)-regular code of length 105. As can be seen, the waterfall region of the BER curves start at close vicinity of the computed
threshold values. (For all the BER simulations in this paper, the maximum number of iterations is 100 and at least 50 codeword errors have been observed.) Similar results for a rate one half irregular LDPC code ensemble and a corresponding random code are presented in Fig. 4. The degree distributions for variable and check nodes are
λ ( x) = 0.24 x + 0.21x 2 + 0.04 x 3 + 0.12 x 4 + 0.39 x14 ,
(7)
ρ ( x) = x 7 , respectively. This ensemble, which is check-regular, is a slight modification of the first ensemble in Table II of [2]. Again the simulation results of the randomly constructed code of length 20,000 are consistent with the asymptotic results derived by the proposed GA method. Both Figures 3 and 4 show that the tolerance of BP algorithm to SNR mismatch increases with increasing SNR, i.e., ∆β (s ) is an increasing function of s, and that the algorithm is more sensitive to SNR underestimation than it is to SNR overestimation, i.e., β ( o ) ( s ) > β (u ) ( s ) , ∀s . To have a better picture of the sensitivity of BP to SNR mismatch and to investigate its variation with the degrees of variable and check nodes for regular LDPC codes, in Fig. 5, we have plotted the normalized SNR thresholds ∆α(η) for a number of (dv,dc) ensembles and for the range of SNR offset η between -6 and +6dB. As can be seen, the sensitivity can change significantly by the variation of dv and dc, particularly for SNR overestimation. For example, while 4dB of SNR overestimation degrades the SNR threshold by only about 0.1dB for (3,27) codes, it degrades the threshold by more than 1.1dB for (5,6) codes. In general, the sensitivity increases by increasing the variable node degree and by decreasing the check node degree. It also appears that the effect of variable node degree is more prominent than that of the check node degree such that for a given rate R=1-dv/dc, the codes with smaller dv and dc are less sensitive to mismatch. For example among rate ½ ensembles, the (3,6) codes are the least sensitive, followed by (4,8) and (5,10) ensembles, respectively. Fig. 5 also shows that while in general SNR underestimation is more deteriorating than SNR overestimation, for some ensembles and over a
certain range of SNR mismatch, overestimation is almost as harmful as underestimation is (see, e.g., (5,6) and (6,8) ensembles for η ∈ [−2,2] ). Curves of Fig. 5 asymptotically tend to the
performance gap between BP and MS as η → +∞ . This implies that this gap decreases by decreasing dv and increasing dc with the effect of dv being more profound. As an example, for rate ½ regular LDPC codes, the performance gap between BP and MS is about 0.6dB for (3,6) ensemble [12]. This increases to about 0.88 dB and 1.06 dB for (4,8) and (5,10) ensembles, respectively [12]. In Fig. 6, we have plotted the SNR threshold vs. SNR offset for the irregular ensemble described by (7) and two other rate ½ irregular ensembles described by the following degree distributions [2, Table 1]:
λ1 ( x ) = 0.38354 x + 0.04237 x 2 + 0.57409 x 3 ,
ρ1 ( x) = 0.24123 x 4 + 0.75877 x 5 , and [2, Table 2]:
λ 2 ( x ) = 0.19606 x + 0.24039 x 2 + 0.00228 x 5 + 0.05516 x 6 + 0.16602 x 7 + 0.04088 x 8 + 0.01064 x 9 + 0.00221 x 27 + 0.28636 x 29
ρ 2 ( x ) = 0.00749 x 7 + 0.99101x 8 + 0.00150 x 9 Fig. 6 shows that while the ensemble with Dv = 30 outperforms the other two ensembles in the absence of mismatch, the situation changes in the presence of mismatch. The ensemble with Dv=15 takes over outside the range of η ∈ [-0.7dB,1.5dB], while beyond the SNR mismatch of
about +4.9dB the ensemble with Dv = 4 is superior.
IV. CONCLUSIONS In this paper, we have investigated the performance of belief propagation (BP) for the decoding of regular and irregular LDPC codes in the presence of channel estimation error. For
the asymptotic analysis, we propose a method based on the Gaussian approximation of density evolution which tracks both the mean and the variance of message distributions. The method is simple and thus can be easily applied to irregular codes. Threshold values are computed based on the proposed method and shown to match the finite-length simulations. It is shown that at the limit of infinite SNR mismatch, the performance of BP tends to that of min-sum and the channel BER, for over- and under-estimation, respectively. This provides a proof that asymptotically BP is more sensitive to under-estimation of SNR than it is to over-estimation of it. It is however observed that for some ensembles of LDPC codes and over finite ranges of SNR mismatch, overestimation can be almost as harmful as under-estimation is. We demonstrate that the sensitivity of BP to SNR mismatch increases with increasing the variable degree (dv) and decreasing the check degree (dc) for regular LDPC codes. The effect of variable degree appears to be more profound such that among the codes with the same rate, those with smaller dv and dc are less sensitive to mismatch. For irregular LDPC codes, we show that highly irregular codes, which perform very well in the absence of mismatch, face significant degradation in their performance in the presence of mismatch such that they may perform worse than codes with less irregularity.
ACKNOWLEDGMENT The authors would like to thank Mr. H. Xiao for constructing the codes used in Figures 3 and 4.
REFERENCES [1] T. J. Richardson and R. L. Urbanke, “The capacity of low-density parity-check codes under message-passing decoding,” IEEE Trans. Inform. Theory, vol. 47, pp. 599-618, Feb. 2001.
[2] T. J. Richardson, A. Shokrollahi, and R. L. Urbanke, “Design of capacity approaching irregular low density parity check codes,” IEEE Trans. Inform. Theory, vol. 47, no. 2, pp. 619–637, Feb. 2001. [3] S. Y. Chung, T.J. Richardson, and R. L. Urbanke, “Analysis of sum-product decoding of LDPC codes using a Gaussian approximation”, IEEE Trans. Inform. Theory, vol. 47, no. 2, pp. 657-670, Feb. 2001. [4] S. ten Brink, G. Kramer, and A. Ashikhmin, “Design of low-density parity check codes for modulation and detection,” IEEE Trans. Comm., vol. 52, no. 4, pp. 670-678, April 2004. [5] Alexei Ashikhmin, Gerhard Kramer, and Stephan ten Brink, “Extrinsic Information Transfer Functions: Model and Erasure Channel Properties,” IEEE Trans. Inform., vol. 50, no. 11, pp. 2657-2673, Nov. 2004. [6]
M. Jordan and R. Nichols, “The effects of channel characteristics on turbo code
performance,” in Proc. MILCOM Conf. McLean, VA, pp. 17-21, Oct. 1996. [7] A. Worm, P. Hoeher and N. When, “Turbo-decoding without SNR estimation,” IEEE Comm. Letters., vol. 4, no. 6, pp. 193-195, June 2000.
[8] T. A. Summers and S. G. Wilson, “SNR mismatch and online estimation in turbo decoding,” IEEE Trans. Com., vol. 46, no. 4,pp. 421-423, Apr. 1998.
[9] D. MacKay and C. Hesketh, “Performance of Low Density Parity Check Codes as a Function of Actual and Assumed Noise Levels,” Electronic Notes in Theoretical Comp. Science, vol. 74, Elsevier Science B.V., 2003. [10] W. Tan and J.R. Cruz, “Signal-to-noise ratio mismatch for low-density parity-check coded magnetic recording channels,” IEEE Trans. Magnetics, vol. 40, no. 2, pp. 498-506, March 2004. [11] A. Anastasopoulos, ``A comparison between the sum-product and the min-sum iterative detection algorithms based on density evolution,” Proc. IEEE Globecom, San Antonio, TX, USA, pp. 1021-1025, Nov. 2001.
[12] J. Chen and M. Fossorier,”Density evolution for two improved BP-Based decoding algorithms of LDPC codes,” IEEE Communications Letters, vol. 6, no. 5, pp. 208 - 210 , May 2002 .
4 offset = −2 offset = 0 offset = +2
Varaiance to Mean Ratio (VMR)
3.5
3
2.5
2
1.5
1
0.5
1
2
3
4
5 Iteration number
6
7
8
9
Fig. 1: Variance to mean ratio (VMR) at the output of varaible nodes as a function of iteration number for a (4,6)-regular ensemble with SNR offsets η = -2,0 and 2 dB.
1 1 0.8
0.5
0.4
0.2
0 20
0 20
40
15 40
15
20
30
10
5
20 5 Mean
30
10
10 0
10 0
Mean
0 Variance
0 Varaince
(a)
(b)
Fig. 2: Mapping G ; (a) g1(m,σ), (b) g2(m,σ).
0
10
Eb/No=1.5dB Eb/No=2.0dB Eb/No=2.5dB −1
10
−2
10
BER
g1
g2
0.6
−3
10
−4
10
−3.7dB −2.9dB
−6
−4
−1.8dB
−2
Mismatch Thresholds
0 Eb/No offset(dB)
2
3.5dB
4
6
Fig. 3: BER of a randomly constructed (3,6)-regular LDPC code of length 105 at different SNR values in the presence of SNR mismatch. Corresponding mismatch thresholds are also given.
0
10
Eb/No=0.8dB Eb/No=1.3dB Eb/No=1.8dB −1
10
−2
BER
10
−3
10
Mismatch Thresholds −4
10
−3.3dB
−6
−4
−2.4dB
−2
−1.3dB
2.3dB
0 Eb/No offset (dB)
2
6.0dB
4
6
Fig. 4: BER of a randomly constructed rate-1/2 irregular LDPC code of length 20,000 with degree distributions given in (7) at different SNR values in the presence of SNR mismatch. Corresponding mismatch thresholds are also given. 3.5
3
Normalized Threshold (dB)
2.5
2
(5 6)
1.5
(6 8) (5 10)
1
(4 6)
(4 8) 0.5
(3 6) (3 8) (3 27)
0 −6
−4
−2
0 Eb/No offset (dB)
2
4
6
Fig. 5: Normalized threshold values ∆α(η) for a number of (dv,dc) ensembles of regular LDPC codes and for the range of SNR offset η between -6 and +6dB.
4 Dv=4 Dv=15 Dv=30
Threshold (dB)
3
2
1
0 −6
−4
−2
0 offset(dB)
2
4
6
Fig. 6: SNR threshold values for three rate ½ ensembles of irrregular LDPC codes for the range of SNR offset η between -6 and +6dB.
Abstract— In this paper we investigate the performance of belief propagation (BP) algorithm for decoding low density parity check (LDPC) codes over the additive white Gaussian noise (AWGN) channel when there is an incorrect estimate of the channel signal to noise ratio (SNR) (referred to as SNR mismatch) at the decoder. We propose a computationally efficient method based on the Gaussian approximation of density evolution to compute the threshold values for regular and irregular codes in the presence of SNR mismatch and show that these values are consistent with simulation results for the codes with finite block lengths. At the extremes for over- and under-estimation of SNR, the performance of BP tends to that of min-sum (MS) algorithm and the channel bit error rate, respectively. Our results for regular codes indicate that the sensitivity to mismatch increases by increasing the variable node degree and by decreasing the check node degree. The effect of variable node degree however appears to be more profound such that at a given rate, the codes with the smallest variable and check degrees are more robust against SNR mismatch. For irregular codes, by comparing the thresholds of a few ensembles, we demonstrate that the ensemble which performs better in the absence of mismatch can perform worse in the presence of it. Index Terms -- LDPC codes, SNR mismatch, SNR estimation error, Belief propagation, Min-sum, SNR threshold, , Density evolution, Gaussian approximation 1
A Preliminary version of this work has been accepted for presentation in part at IEEE VTC2005-Spring, Stockholm, Sweden, May 30 - June 1, 2005 and in part at CWIT 2005, Montreal, Quebec, Canada, June 5 – 8, 2005.
I.
INTRODUCTION AND PROBLEM FORMULATION
Different tools have been proposed to analyze asymptotic performance of LDPC codes. The most accurate method called density evolution (DE) [1], [2] is based on iterative calculation of densities of the messages passed between check nodes and variable nodes. For soft messages, however, this method has a high complexity. Other methods have been proposed in [3]-[5] based on Gaussian approximation of densities to reduce the complexity at the expense of less accuracy. The method suggested in [3] is based on tracking the mean of the Gaussian estimated version of the messages while the approximation proposed in [4], [5] is based on keeping track of the mutual information between the transmitted bit and the output messages of variable nodes and check nodes through extrinsic information transfer (EXIT) charts. In belief propagation (BP) algorithm for LDPC codes over the binary input additive white Gaussian noise (BIAWGN) channel, the knowledge of the channel noise power is necessary to achieve the ultimate performance of the algorithm. Consider a BIAWGN channel with input X that can take values ±1, and output Y=X + N, where N is a Gaussian random variable with mean zero and variance σ2 (N (0, σ2)). In the log-likelihood ratio (LLR) domain, the input to the BP algorithm for each bit should be M0=2Y/σ2. Starting from such initial messages, which is based on a perfect knowledge of the noise power σ2, BP algorithm performs optimally in the sense that it provides the a-posteriori probabilities (APP) for bits assuming a tree-like (cycle-free) graph. Suppose that the all-one codeword is transmitted. The input LLRs for all the bits are then i.i.d. with distribution N (2/σ2, 4/σ2). This distribution is symmetric (consistent) [1]-[3] with the ratio of variance to mean (VMR) equal to two. As the symmetry property is preserved throughout the iterations of BP algorithm, the method of Gaussian approximation (GA) proposed in [3] can be used to only track the mean of the Gaussian densities throughout the iterative process. This
reduces the dimension of the problem from infinity (for full DE) to one, and significantly simplifies the analysis.
) Now consider the case where the noise power is estimated erroneously as σ 2 ≠ σ2. We refer to
) this as “signal to noise ratio (SNR) mismatch,” and define the SNR mismatch ratio η by σ2 / σ 2 . The value of η in dB is called SNR offset. In the presence of SNR mismatch, the input LLRs to BP
) ) algorithm have distribution N (2/ σ 2 , 4σ2/ σ 4 ). This distribution is not symmetric and has VMR = ) 2 σ2/ σ 2 . In Fig. 1, for the first nine iterations, we have shown the values of VMR at the output of variable nodes (input of check nodes) for an ensemble of (4,6)-regular LDPC codes at SNR = 3 dB and for η = ±2 dB. (Note that SNR = Eb/N0 = 1/(2Rσ2), where Eb, N0 and R are the average energy per information bit, one-sided power spectral density of AWGN and the code rate, respectively.) This figure shows that in the case of mismatch, not only VMR is not equal to 2, but also it changes throughout the iterative process. In this paper to investigate the asymptotic performance of BP in the presence of mismatch, we employ a GA method in which both the mean and the variance of the distributions are tracked. The performance of BP with imperfect channel estimation has been addressed in [6]-[8] for turbo codes and in [9], [10] for LDPC codes. In [9], simulation results for a few codes with finite block lengths are presented, while in [10], DE has also been used to analyze the performance of BP with mismatch in the context of magnetic recording channels for a regular LDPC code ensemble. For the AWGN channel, the results of [9] and [10] indicate that the algorithm is more sensitive to under-estimation of SNR than it is to its over-estimation. In this work, we perform asymptotic analysis of BP with mismatch over the AWGN channel for both regular and irregular codes and demonstrate that our asymptotic results are consistent with finite-length simulations. Compared to the DE approach of [10], our proposed method, which is based on GA of DE, is much simpler and thus can be easily applied to irregular codes. We demonstrate that in the limit when the mismatch offset tends to infinity, the performance of BP tends to that of min-sum (MS)
algorithm [11] and the channel bit error rate (BER) for over- and under-estimation of SNR, respectively. This explains why the algorithm is in general more sensitive to under-estimation than it is to over-estimation of SNR. Using the proposed method, we also investigate the sensitivity of BP to SNR mismatch as a function of variable and check node degree distributions. Our results show that for regular codes the sensitivity increases by increasing the variable degree (dv) and by decreasing the check degree (dc) with the effect of the former more prominent than that of the latter. This means that at a given rate the codes with smaller dv and dc are more robust against SNR mismatch. An implication of this is that, at least for the tested cases, the regular codes of a given rate which perform better in the absence of SNR mismatch are also more robust in the presence of it. For irregular codes however this is not necessarily the case as shown by an example. It is well-known that BP is superior to MS in performance. Our result that MS is in fact the limit of BP as the SNR overestimation tends to infinity makes it possible to apply the results of this work to relate the degradation in performance of MS with respect to BP to the degree distributions of the code. This, for example, implies that among regular LDPC codes of the same rate, codes with smaller dv and dc experience smaller gap between the performances of BP and MS. The rest of the paper is organized as follows: In the next section, we explain the proposed GA method, define the SNR threshold value of an ensemble of codes for a given mismatch offset, and also define mismatch thresholds for a given channel SNR. We end the section by discussing the asymptotic performance of BP as mismatch offset tends to ± ∞. In section III, simulation results are presented and discussed. Section VI is devoted to some concluding remarks.
II. GAUSSIAN APPROXIMATION, THRESHOLDS AND ASYMPTOTIC PERFORMANCE OF BP IN THE PRESENCE OF MISMATCH A. The proposed method based on Gaussian approximation For regular LDPC codes, the method of [3] approximates the density of messages throughout the iterative process by symmetric Gaussian distributions and tracks the mean of these distributions using the following two equations at variable and check nodes, respectively: mout = (dv-1) min + m0 ,
(1)
mout = g-1( g(min) dc-1) ,
(2)
where min and mout are the means of the input and output distributions, respectively, m0 is the mean of the initial LLRs inputted to the BP algorithm, and function g(x) is the expected value of tanh(N (x,2x)/2). For the case of mismatch, as the symmetry condition is violated and as the ratio of variance to mean changes with iterations (see Fig. 1), we need to track both the mean and the variance of the random variables. We however maintain the Gaussian assumption. For variable nodes, we still use (1) to track the means and employ the following equation for variances: σ2out = (dv-1) σ2in + σ20 .
(3)
For check nodes, we define the following 2-input 2-output mapping: G(m, σ)=( g1(m,σ), g2(m,σ) ) =( E[tanh(N(m,σ2)/2], E[tanh2(N(m,σ2)/2)] ) ,
(4)
where E[.] denotes the expected value, and track the means and variances using (mout , σout )= G-1( g1d c −1 (min , σin ), g 2d c −1 (min , σin )) .
(5)
We compute the mappings G and G-1 numerically and use look-up tables to track the means and variances. In Figures 2(a) and (b), we have shown the functions g1(m,σ) and g2(m,σ), respectively.
To generalize this analysis to irregular codes similar steps as those described in [3] are taken. Dv
Dc
i=2
i =2
Assume an irregular code ensemble with degree distributions λ ( x) = ∑ λi x i −1 and ρ ( x) = ∑ ρ i x i −1 for variable nodes and check nodes respectively, where the coefficient of xi represents the percentage of edges connected to nodes of degree i+1, and Dv and Dc are the maximum variable and check node degrees, respectively. The distribution of messages passed between variable nodes and check nodes in this case are approximated by Gaussian mixtures. Suppose that, at a given iteration ℓ, the Gaussian mixture at the output of check nodes and input of variable nodes has mean and variance min and σ2in, respectively. For variable nodes of degree i, we estimate the output by a Gaussian and compute its mean and variance using
mout,i = (i-1) min + m0 , σ2out,i = (i-1) σ2in + σ20
,
respectively. The density of the messages from variable nodes to check nodes is then approximated by the Gaussian mixture ∑i λi N (mout,i , σ2out,i). We now define h1 and h2 as follows: Dv
(h1 , h2 ) = ∑ λi G(mout,i , σ2out,i) . i=2
The output of a check node of degree j is then approximated by a Gaussian with the following mean and variance (mout,j , σ2out,j ) = G -1 (h1j −1 , h2j −1 ) . The distribution of messages from check nodes to variable nodes is then approximated by the Gaussian mixture ∑j ρj N (mout,j , σ2out,j). The mean and the variance of this distribution are then obtained by the following equations to be used as inputs to variable nodes for iteration ℓ+1. Dc
mout = ∑ ρ j mout , j , j =2
σ
2 out
Dc
(
)
2 2 2 = ∑ ρ j σ out , j + mout , j − mout . j =2
To examine the accuracy of the proposed method, we have compared its threshold results to those generated by density evolution for a few cases. Our results indicate that with steps of 0.01 for mean and variance, the difference between the exact and approximate thresholds is at most 0.1dB and 0.2dB for regular and irregular codes, respectively2. We note that this inaccuracy is mainly due to the Gaussian approximation [3], but marginal improvement can be obtained by reducing the step size. B. Thresholds Threshold values are used to measure the asymptotic performance of code ensembles under iterative decoding algorithms (see, e.g., [1]-[3]). For a given ensemble and an SNR offset η, we define the SNR threshold α(η) as the infimum of all channel SNRs such that the bit error rate converges to zero as the iteration number tends to infinity. It is well-known that SNR thresholds in the presence of mismatch are always higher than that of no-mismatch case, i.e., α(η) > α(0), ∀ η ≠ 0 dB [10]. We measure the sensitivity of an ensemble of codes to mismatch by the
difference ∆α(η) = α(η) - α(0) (dB). To quantify the amount of tolerable mismatch for a given code ensemble at a given SNR
s > α(0), we also define mismatch thresholds β ( o ) ( s ) and β (u ) ( s ) for over- and underestimation of SNR, respectively. For over- (under)estimation, this is defined as supremum (infimum) of SNR offset such that the bit error rate converges to zero as the iteration number tends to infinity. We use the difference ∆β(s) = β ( o ) ( s ) - β (u ) ( s ) as the measure of tolerance. C. Asymptotic performance of BP at offset limits 1) SNR underestimation
) Assume that the mismatch offset tends to minus infinity. In this case the estimate σ 2 of the ) channel noise variance is very large. This makes the initial LLR values of m0 = 2y/ σ 2 very small. Consider the first iteration of BP, where the initial messages are passed from variable 2
In line with our results, it can be seen in [3] as well that the Gaussian approximation is generally less accurate for irregular codes.
nodes to check nodes. Without loss of generality, consider a check node of degree three with two non-negative extrinsic inputs L1 and L2. The output is then equal to (see, e.g., [11]) L3 = min( L1 , L2 ) + ln(1 + e − ( L1 + L2 ) ) − ln(1 + e
− L1 − L2
).
(6)
Using the Taylor series, for small values of x, we have ln(1 + e − x ) ≈ 1 − x / 2 . Applying this to the
second and third terms of (6), we obtain L3 ≈ 0 . This implies that the messages passed by check nodes to variable nodes are negligible compared to the initial LLR values. This situation will continue throughout the iterations and thus the bit error rate tends to that of the channel, which is ∞
equal to Q(1/σ), where Q( x) = 1 / 2π ∫ e − x x
2
/2
dx .
2) SNR overestimation
In the case of SNR overestimation, the initial LLR values, and thus all the messages passed from variable nodes to check nodes throughout the iterations, are very large. In this case, as the magnitude of the difference between the second and the third terms of (6) is upper bounded by ln(2), we have L3 ≈ min( L1 , L2 ) . This is the check operation of min-sum (see, e.g., [11], [12]). As the variable operations of BP and MS are also the same, this implies that the performance of BP tends to that of min-sum as mismatch offset tends to plus infinity.
III. SIMULATION RESULTS We first consider the ensemble of (3,6)-regular LDPC codes and investigate its asymptotic performance under BP in the presence of mismatch by computing the mismatch thresholds
β ( o ) ( s) and β (u ) ( s) for s = 1.5, 2.0 and 2.5 dB, using our proposed GA method. These are shown in Fig. 3. Note that β (u ) (2) = β (u ) (2.5) = +∞. This is in agreement with the fact that both SNR values are larger than the MS threshold for this ensemble which is about 1.7 dB [12]. In Fig. 3, we have also plotted BER vs. SNR offset for a randomly constructed (3,6)-regular code of length 105. As can be seen, the waterfall region of the BER curves start at close vicinity of the computed
threshold values. (For all the BER simulations in this paper, the maximum number of iterations is 100 and at least 50 codeword errors have been observed.) Similar results for a rate one half irregular LDPC code ensemble and a corresponding random code are presented in Fig. 4. The degree distributions for variable and check nodes are
λ ( x) = 0.24 x + 0.21x 2 + 0.04 x 3 + 0.12 x 4 + 0.39 x14 ,
(7)
ρ ( x) = x 7 , respectively. This ensemble, which is check-regular, is a slight modification of the first ensemble in Table II of [2]. Again the simulation results of the randomly constructed code of length 20,000 are consistent with the asymptotic results derived by the proposed GA method. Both Figures 3 and 4 show that the tolerance of BP algorithm to SNR mismatch increases with increasing SNR, i.e., ∆β (s ) is an increasing function of s, and that the algorithm is more sensitive to SNR underestimation than it is to SNR overestimation, i.e., β ( o ) ( s ) > β (u ) ( s ) , ∀s . To have a better picture of the sensitivity of BP to SNR mismatch and to investigate its variation with the degrees of variable and check nodes for regular LDPC codes, in Fig. 5, we have plotted the normalized SNR thresholds ∆α(η) for a number of (dv,dc) ensembles and for the range of SNR offset η between -6 and +6dB. As can be seen, the sensitivity can change significantly by the variation of dv and dc, particularly for SNR overestimation. For example, while 4dB of SNR overestimation degrades the SNR threshold by only about 0.1dB for (3,27) codes, it degrades the threshold by more than 1.1dB for (5,6) codes. In general, the sensitivity increases by increasing the variable node degree and by decreasing the check node degree. It also appears that the effect of variable node degree is more prominent than that of the check node degree such that for a given rate R=1-dv/dc, the codes with smaller dv and dc are less sensitive to mismatch. For example among rate ½ ensembles, the (3,6) codes are the least sensitive, followed by (4,8) and (5,10) ensembles, respectively. Fig. 5 also shows that while in general SNR underestimation is more deteriorating than SNR overestimation, for some ensembles and over a
certain range of SNR mismatch, overestimation is almost as harmful as underestimation is (see, e.g., (5,6) and (6,8) ensembles for η ∈ [−2,2] ). Curves of Fig. 5 asymptotically tend to the
performance gap between BP and MS as η → +∞ . This implies that this gap decreases by decreasing dv and increasing dc with the effect of dv being more profound. As an example, for rate ½ regular LDPC codes, the performance gap between BP and MS is about 0.6dB for (3,6) ensemble [12]. This increases to about 0.88 dB and 1.06 dB for (4,8) and (5,10) ensembles, respectively [12]. In Fig. 6, we have plotted the SNR threshold vs. SNR offset for the irregular ensemble described by (7) and two other rate ½ irregular ensembles described by the following degree distributions [2, Table 1]:
λ1 ( x ) = 0.38354 x + 0.04237 x 2 + 0.57409 x 3 ,
ρ1 ( x) = 0.24123 x 4 + 0.75877 x 5 , and [2, Table 2]:
λ 2 ( x ) = 0.19606 x + 0.24039 x 2 + 0.00228 x 5 + 0.05516 x 6 + 0.16602 x 7 + 0.04088 x 8 + 0.01064 x 9 + 0.00221 x 27 + 0.28636 x 29
ρ 2 ( x ) = 0.00749 x 7 + 0.99101x 8 + 0.00150 x 9 Fig. 6 shows that while the ensemble with Dv = 30 outperforms the other two ensembles in the absence of mismatch, the situation changes in the presence of mismatch. The ensemble with Dv=15 takes over outside the range of η ∈ [-0.7dB,1.5dB], while beyond the SNR mismatch of
about +4.9dB the ensemble with Dv = 4 is superior.
IV. CONCLUSIONS In this paper, we have investigated the performance of belief propagation (BP) for the decoding of regular and irregular LDPC codes in the presence of channel estimation error. For
the asymptotic analysis, we propose a method based on the Gaussian approximation of density evolution which tracks both the mean and the variance of message distributions. The method is simple and thus can be easily applied to irregular codes. Threshold values are computed based on the proposed method and shown to match the finite-length simulations. It is shown that at the limit of infinite SNR mismatch, the performance of BP tends to that of min-sum and the channel BER, for over- and under-estimation, respectively. This provides a proof that asymptotically BP is more sensitive to under-estimation of SNR than it is to over-estimation of it. It is however observed that for some ensembles of LDPC codes and over finite ranges of SNR mismatch, overestimation can be almost as harmful as under-estimation is. We demonstrate that the sensitivity of BP to SNR mismatch increases with increasing the variable degree (dv) and decreasing the check degree (dc) for regular LDPC codes. The effect of variable degree appears to be more profound such that among the codes with the same rate, those with smaller dv and dc are less sensitive to mismatch. For irregular LDPC codes, we show that highly irregular codes, which perform very well in the absence of mismatch, face significant degradation in their performance in the presence of mismatch such that they may perform worse than codes with less irregularity.
ACKNOWLEDGMENT The authors would like to thank Mr. H. Xiao for constructing the codes used in Figures 3 and 4.
REFERENCES [1] T. J. Richardson and R. L. Urbanke, “The capacity of low-density parity-check codes under message-passing decoding,” IEEE Trans. Inform. Theory, vol. 47, pp. 599-618, Feb. 2001.
[2] T. J. Richardson, A. Shokrollahi, and R. L. Urbanke, “Design of capacity approaching irregular low density parity check codes,” IEEE Trans. Inform. Theory, vol. 47, no. 2, pp. 619–637, Feb. 2001. [3] S. Y. Chung, T.J. Richardson, and R. L. Urbanke, “Analysis of sum-product decoding of LDPC codes using a Gaussian approximation”, IEEE Trans. Inform. Theory, vol. 47, no. 2, pp. 657-670, Feb. 2001. [4] S. ten Brink, G. Kramer, and A. Ashikhmin, “Design of low-density parity check codes for modulation and detection,” IEEE Trans. Comm., vol. 52, no. 4, pp. 670-678, April 2004. [5] Alexei Ashikhmin, Gerhard Kramer, and Stephan ten Brink, “Extrinsic Information Transfer Functions: Model and Erasure Channel Properties,” IEEE Trans. Inform., vol. 50, no. 11, pp. 2657-2673, Nov. 2004. [6]
M. Jordan and R. Nichols, “The effects of channel characteristics on turbo code
performance,” in Proc. MILCOM Conf. McLean, VA, pp. 17-21, Oct. 1996. [7] A. Worm, P. Hoeher and N. When, “Turbo-decoding without SNR estimation,” IEEE Comm. Letters., vol. 4, no. 6, pp. 193-195, June 2000.
[8] T. A. Summers and S. G. Wilson, “SNR mismatch and online estimation in turbo decoding,” IEEE Trans. Com., vol. 46, no. 4,pp. 421-423, Apr. 1998.
[9] D. MacKay and C. Hesketh, “Performance of Low Density Parity Check Codes as a Function of Actual and Assumed Noise Levels,” Electronic Notes in Theoretical Comp. Science, vol. 74, Elsevier Science B.V., 2003. [10] W. Tan and J.R. Cruz, “Signal-to-noise ratio mismatch for low-density parity-check coded magnetic recording channels,” IEEE Trans. Magnetics, vol. 40, no. 2, pp. 498-506, March 2004. [11] A. Anastasopoulos, ``A comparison between the sum-product and the min-sum iterative detection algorithms based on density evolution,” Proc. IEEE Globecom, San Antonio, TX, USA, pp. 1021-1025, Nov. 2001.
[12] J. Chen and M. Fossorier,”Density evolution for two improved BP-Based decoding algorithms of LDPC codes,” IEEE Communications Letters, vol. 6, no. 5, pp. 208 - 210 , May 2002 .
4 offset = −2 offset = 0 offset = +2
Varaiance to Mean Ratio (VMR)
3.5
3
2.5
2
1.5
1
0.5
1
2
3
4
5 Iteration number
6
7
8
9
Fig. 1: Variance to mean ratio (VMR) at the output of varaible nodes as a function of iteration number for a (4,6)-regular ensemble with SNR offsets η = -2,0 and 2 dB.
1 1 0.8
0.5
0.4
0.2
0 20
0 20
40
15 40
15
20
30
10
5
20 5 Mean
30
10
10 0
10 0
Mean
0 Variance
0 Varaince
(a)
(b)
Fig. 2: Mapping G ; (a) g1(m,σ), (b) g2(m,σ).
0
10
Eb/No=1.5dB Eb/No=2.0dB Eb/No=2.5dB −1
10
−2
10
BER
g1
g2
0.6
−3
10
−4
10
−3.7dB −2.9dB
−6
−4
−1.8dB
−2
Mismatch Thresholds
0 Eb/No offset(dB)
2
3.5dB
4
6
Fig. 3: BER of a randomly constructed (3,6)-regular LDPC code of length 105 at different SNR values in the presence of SNR mismatch. Corresponding mismatch thresholds are also given.
0
10
Eb/No=0.8dB Eb/No=1.3dB Eb/No=1.8dB −1
10
−2
BER
10
−3
10
Mismatch Thresholds −4
10
−3.3dB
−6
−4
−2.4dB
−2
−1.3dB
2.3dB
0 Eb/No offset (dB)
2
6.0dB
4
6
Fig. 4: BER of a randomly constructed rate-1/2 irregular LDPC code of length 20,000 with degree distributions given in (7) at different SNR values in the presence of SNR mismatch. Corresponding mismatch thresholds are also given. 3.5
3
Normalized Threshold (dB)
2.5
2
(5 6)
1.5
(6 8) (5 10)
1
(4 6)
(4 8) 0.5
(3 6) (3 8) (3 27)
0 −6
−4
−2
0 Eb/No offset (dB)
2
4
6
Fig. 5: Normalized threshold values ∆α(η) for a number of (dv,dc) ensembles of regular LDPC codes and for the range of SNR offset η between -6 and +6dB.
4 Dv=4 Dv=15 Dv=30
Threshold (dB)
3
2
1
0 −6
−4
−2
0 offset(dB)
2
4
6
Fig. 6: SNR threshold values for three rate ½ ensembles of irrregular LDPC codes for the range of SNR offset η between -6 and +6dB.
