Optimization of Scaling Soft Information in Iterative ... - Semantic Scholar
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 53, NO. 6, JUNE 2005
957
Optimization of Scaling Soft Information in Iterative Decoding Via Density Evolution Methods Jun Heo, Member, IEEE, and Keith M. Chugg, Member, IEEE
Abstract—Density evolution has recently been used to analyze iterative decoding and explain many characteristics of iterative decoding including convergence of performance and preferred structures for the constituent codes. The scaling of extrinsic information (messages) has been heuristically used to enhance the performance in the iterative decoding literature, particularly based on the min-sum message passing algorithm. In this paper, it is demonstrated that density evolution can be used to obtain the optimal scaling factor and also estimate the maximum achievable scaling gain. For low density parity check (LDPC) codes and serially concatenated convolutional codes (SCCC) with two-state constituent codes, the analytic density evolution technique is used, while the signal-to-noise ratio (SNR) evolution technique and the EXIT chart technique is used for SCCC with more than 2 state constituent codes. Simulation results show that the scaling gain predicted by density evolution or SNR evolution matches well with the scaling gain observed by simulation. Index Terms—Iterative decoding, density evolution, low-density parity-check (LDPC) codes, serially concatenated convolutional codes (SCCC).
I. INTRODUCTION
R
ECENTLY developed density evolution techniques have been used to determine the analytic capacity of LDPC codes. In [1], the density evolution technique was recursively used to track the density of extrinsic information between the variable nodes and check nodes of an LDPC code. A simplified version of the density evolution technique was introduced with a Gaussian approximation in [2]. While these two approaches [1], [2] were based on the sum-product message passing algorithm, in [3]–[5] a density evolution technique based on the min-sum message passing was introduced. Similar to [1], the probability density of extrinsic information was tracked during iterative decoding in [3]–[5]. During the review of this letter, we found an independent work [5] which derived the min-sum density evolution without a Gaussian approximation. Because the LDPC decoding algorithm can be represented by a simple bipartite graph, most of the analytical work has been focused on LDPC codes. Meanwhile, similar attempts have been made for Turbo codes based on tracking the evolution of signal-to-noise ratio (SNR) Paper approved by H. El-Gamal, the Editor for Space-Time Coding and Spread Spectrum of the IEEE Communications Society. Manuscript received October 1, 2002; revised March 1, 2004. This work was supported in part by Grant R08-2003-000-10165-0 from the Basic Research Program of the Korea Science and Engineering Foundation and by the National Science Foundation under CCR-0082987. J. Heo was with the Communication Sciences Institute, University of Southern California, Los Angeles, CA 90089-2565 USA, and is now with the NITRI (Next Generation Innovative Technology Research Institute), Konkuk University, Seoul 143-701, Korea (e-mail: [email protected]). K. M. Chugg is with the Communication Sciences Institute, Electrical Engineering Department, University of Southern California, Los Angeles, CA 90089-2565 USA (e-mail: [email protected]). Digital Object Identifier 10.1109/TCOMM.2005.849782
values or mutual information of the extrinsic information. In [6]–[8] the mutual information transfer characteristic was used to show the convergence properties of turbo codes. In [9] the SNR evolution technique was used to achieve the asymptotic capacity (i.e., threshold) of Turbo codes as the interleaver size and the number of iterations go to infinity. The obtained thresholds agree well with the region where the bit error curves start to fall down. In [10], the SNR evolution technique was used to explain many mysteries of turbo codes and serially concatenated convolutional codes (SCCCs) (e.g., the role of systematic bits and the importance of recursive constituent codes). The SNR evolution techniques were classified into two categories. The actual SNR curves were obtained during a standard iterative decoding procedure by collecting the extrinsic information, while the Gaussian SNR curves were obtained by generating Gaussian random variables and feeding these into each constituent decoder. In the literature [11]–[15], damping of soft information in order to allow the suboptimal algorithm to better refine the beliefs has been used to improve performance. As a simple and effective method of damping soft information, scaling1 of extrinsic information was introduced in the iterative decoding literature [16] where the scaling technique (i.e., multiplying a constant value) was applied to the extrinsic information between the concatenated systems. Another approach is to view the scaling as a method to reduce the overestimation of the extrinsic information in the min-sum algorithm [17] where the scaling was applied to every marginalization (i.e., minimization) of the soft information. It was claimed that the performance of the min-sum algorithm could approach that of the sum-product algorithm by reducing the overestimation error. In [13]–[15], scaling was applied to LDPC codes and the scaling factor was calculated as a ratio of expectation value of log likelihood ratio (LLR). Sum-product message-passing can be run in the negative log domain using a slight modification of min-sum processing. Specifically, the minimum function can be augmented by a look-up table that accounts for the difference (e.g., see [12]). However, the look-up table method may require many more operations compared to that of scaling. Consider, for example, an 8-state parallel-concatenated code with an input block bits. For each constituent decoder activation, there size of will be (forward recursion), (backward recursion), (completion) table look-ups per activation, respecand multiplications in tively. This is to be compared against scaled-min-sum processing. Moreover, building a multiplier for scaling factor can be much simpler when the scaling factor is known at design time. Besides the number of operations, the scaled-min-sum computation can be desirable for a very 1As an alternative method, filtering of extrinsic information was introduced in [11], [12].
0090-6778/$20.00 © 2005 IEEE
958
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 53, NO. 6, JUNE 2005
fast decoder and a very flexible decoder which supports a wide range of code rates and different modulation formats. For such a flexible decoder, many look-up tables may be required to optimize the performance. For the scaled-min-sum algorithm, however, it is not clear when scaling results in a significant performance gain and the optimal scaling factor is determined by trial and error. In addition, it was recently shown that scaling greatly enhances performance when a relatively strong convolutional code is serially concatenated with a differential encoder under the min-sum decoding algorithm [18]. In this letter, we propose the min-sum density evolution techniques with a Gaussian approximation, which makes the calculation much simpler by tracking only the mean and variance instead of the probability density function (pdf). Using this proposed technique, density evolution is used to analyze the scaling gain for LDPC codes. The threshold values are calculated for various scaling factors, and the scaling factor showing the highest threshold in noise level, equivalently the lowest threshold in signal-to-noise ratio, is determined as the optimal scaling factor. At the same time, the maximum achievable scaling gain is estimated by the difference of threshold values. Simulation results for various scaling factors validates the analytical results by density evolution. Due to the similarity in a graphical representation, the SCCC with two-state constituent codes can also be analyzed similarly and the optimal scaling factor can be obtained as well. When the constituent codes have more than 2 states, the SNR evolution technique and the extrinsic mutual information transfer (EXIT) chart technique are used instead of density evolution. The rest of this letter is organized as follows. In Section II, density evolution based on the min-sum algorithm with a Gaussian approximation is developed for the LDPC codes and it is applied for the analysis of scaling. For SCCCs with two-state constituent codes, min-sum density evolution technique and the analysis of scaling are followed in Section III. In Section IV, the SNR evolution and the EXIT chart are used for scaling in more general SCCCs (more than two-state constituent codes). Finally, conclusions are provided in Section V.
where is the message output from a variable node and is represents the the message output from a check node, and received message from the channel. This min-sum message passing algorithm at a check node was shown in [3], [19], and [20]. According to this expression, the output message from a check node can be represented by the sign of the product of the incoming messages and the minimum absolute value among the incoming messages, where the message from the node receiving the output message is excluded. On the min-sum density evolution with a Gaussian assumpand variance of the tion, our goal is to track the mean message during iterative decoding. With the independent and identically distributed (iid) assumption for the exchanged mescan be obtained by (1) as: sages, and (3) (4) is more involved. When The calculation of and , the probability density function (pdf) of the output message from a check node based on (2) was derived in [3] as
(5) (6) and are the pdf and the cumulative density where is function (cdf), respectively. For the case of obtained by recursive update [i.e., (5)–(6)] with additional pdf up to as (7) a shorthand notation of (5)–(6), which is the pdf of where the output message for the case of . Once we have at th iteration, the mean and variance at th iteration are tracked as2
II. ANALYSIS OF SCALED MIN-SUM DECODING OF LDPC CODES
(8)
In this section, density evolution is used to explain the gain obtained by scaling in min-sum decoding of LDPC codes. The scaling is applied for the extrinsic information which is obtained based on the min-sum algorithm. Therefore, to analyze the scaling, a min-sum density evolution technique is required. In [3], min-sum density evolution was introduced without a Gaussian approximation. Combining this with Gaussian approximation, we present a simpler min-sum density evolution LDPC codes the min-sum technique. For regular message passing algorithm at variable and check nodes is represented by (1) (2)
where the scaling factor multiplies the message before it goes of message is cominto the check nodes, hence the pdf puted based on , instead of . Using the Gaussian assumption on , the pdf and cdf are obtained from . Then, the pdf is obtained by (5)–(6). The mean and variance are numerically calculated from . Finally, at th iteration is obtained by (3)–(4). The iterative mean and variance tracking is initialized by the received message from channel as: . This recursive calculation is executed for a sufficiently large number of iterations (e.g., 1000) to determine whether the message converges to correct codewords at a certain channel noise level. The convergence is determined 2In [3] and [4], the pdfs are tracked as: Gaussian assumption.
without
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 53, NO. 6, JUNE 2005
959
Fig. 2. (a) Block diagram of SCCC1 with two-state constituent codes . (b) Associated serially concatenated bipartite graph with scaling factor .
Fig. 1.
Simulation results for regular (3, 6) LDPC code for block size with various scaling factor . The corresponding thresholds ( dB) are shown in the parentheses.
in
when the probability of error, equivalently the tail part of the pdf goes to zero. The threshold is defined as the maximum channel noise level for which the message converges. We consider a rate 1/2, regular (3,6) LDPC code and compute the threshold of scaled min-sum decoding with different scale factors . The standard decoding without scaling corresponds to . The thresholds and corresponding performance (by simulation) is shown in Fig. 1 for the codeword block size of 10 000 bits. It is noted that the threshold strongly depends on the scaling factor and the scaling factor showing the highest threshold (equivalently, the lowest in dB) is optimal. , the threshold is 0.65 dB lower With the optimal than that (1.83 dB) of standard min-sum decoding. It was also seen that the simulation results follow the calculated thresholds. Moreover, the order of thresholds is maintained in the order of threshold always simulation performances (i.e., a lower results in better performance).
outer to the inner decoder. This is based on the experimental rule that the scaling should be applied to soft information transferred from the stronger code to the weaker code. The messages from check node to information and parity and respectively. variable nodes are represented by Similarly, the messages from information and parity variable respectively. nodes to check node are represented by and Based on the min-sum message passing algorithm, the messages at the inner code are represented by (9) (10) (11) (12) (13) (14) represents a received message from channel and where represents input extrinsic message from the outer constituent code. The messages at the outer code are also similarly represented. The messages exchanged between the inner and outer decoder are
III. ANALYSIS FOR SCCC CODES WITH TWO-STATE CONSTITUENT CODES In this section, density evolution is used to explain the gain obtained by scaling of the soft information in a SCCC system. This approach shows that scaling allows a higher noise level for convergence of soft information. We consider , two-state convolutional SCCC1 consisting of a rate code and a differential encoder which has a generator matrix . The block diagram of encoder SCCC1 is shown in Fig. 2(a). In this simple constituent and outputs have the relationship: code, the input . Using this relationship, we can draw a Tanner graph of this constituent code with variable nodes and check nodes as that of LDPC code. This makes it possible to adopt the analytic methods which have been used for LDPC codes. We can draw a serially concatenated bipartite graph with scaling coefficient , which is shown in Fig. 2(b). The scaling coefficient is only applied to the soft information from the
where represents the output extrinsic message from the inner represent the output extrinsic informacode, and , and from tion from the outer code. The input extrinsic message outer code is equally likely selected from two available extrinsic through a uniform interleaver as messages (15) Because of the independent and identically distributed assumption [1], [21] for the messages, we omit any time index in the message representation. The evolved mean and variance at the variable nodes are calculated by corresponding equation for the messages and iid assumption, whereas the evolved mean and
960
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 53, NO. 6, JUNE 2005
variance at check nodes are numerically evaluated by the relationship between input and output pdf and Gaussian assumption. ), once we Similar to the evolution of LDPC codes (with at th iteration by taking expectation have on both sides of (9)–(14), the mean and variance of and are numerically evaluated as
(16) Given the mean and variance of and , the extrinsic message fed into the outer code is
Similar to the inner code, the mean and variance of and at the check nodes of the outer code are numerically evaluated. The extrinsic messages from the outer code are
The next recursion for th iteration is started using the obtained values . This recursive calculation is initialized for the first iteration as
where the symmetry condition: [22] has been used for the received message from channel, because a symmetric channel is considered. This recursive calculation is executed for a sufficient number of iterations (e.g., 1000) to see whether the message converges or not given a certain channel noise level. Fig. 3 shows the threshold values for various scaling coefficients . The standard min-sum decoding without scaling cor. Note that with the optimal scaling coeffiresponds to , the threshold is noticeably (0.25 dB) lower than cient that without scaling. This suggests that the performance with dB) than without scaling in this optimal scaling is better ( particular system. The performance curves are also shown in Fig. 3 and the computed thresholds are plotted on it to show how the analytic threshold is matched to simulation results.
Fig. 3. (a) Comparison between simulation results and convergence thresholds , for SCCC1 with and without scaling ( 10 iterations), (b) The thresholds for SCCC2 on analytic computation with and without scaling.
IV. ANALYSIS OF SCALING FOR SCCCS WITH CONSTITUENT CODES HAVING MORE THAN TWO STATES In this section, the scaling of soft information is analyzed on a system having more than two-state outer convolutional code (in this paper, 4-state nonsystematic nonrecursive convolutional code is considered) with an inner differential encoder (SCCC2). Because it is difficult to get analytic expressions for this 4-state convolutional code, the simulation based SNR evolution and the EXIT chart using mutual information transfer characteristic are used to explain the scaling gain. The SNR evolution is widely used in the literature [9], [10], particulary, when it is difficult to apply the previous analytic methods as in the case of Turbo code and SCCC with complex constituent codes. The SNR evolution is obtained based on the collected messages at each iterative decoding step. From the collected messages, the mean for the exchanged soft information between two constituent codes is using the symcomputed. The SNR is approximated as metry condition for the exchanged message [22]. Instead of the collected LLRs, a generated Gaussian random soft information can be used as well. The EXIT chart technique was introduced in [6] which showed the mutual information transfer characteristic. An simplified construction of the EXIT chart was presented in [8]. For the EXIT chart, the mutual information is computed for both the input Gaussian soft information and the collected output extrinsic information. The considered system (SCCC2) is obtained by replacing the outer code of Section III with a 4-state convolutional code, . The inner differential encoder is unchanged. For this system, the thresholds are obtained by actual SNR ), the threshold is evolution. Without scaling (i.e., dB whereas with scaling , the threshold dB. The SNR evolution curves of the conis sidered system are shown in Fig. 4(a) at dB, which is the threshold without scaling. It is noted that the SNR have an open evolution curves with the scaling factor iteration tunnel, while they touch each other without scaling. In other words, the threshold with scaling is lower than that without scaling, which agrees to the results on previous SCCC1 system. The EXIT chart was presented in Fig. 4(b) for the same considered system. Similarly in the previous SNR evolution, an
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 53, NO. 6, JUNE 2005
961
REFERENCES
Fig. 4. (a) Actual SNR evolution with and without scaling for SCCC2 at the dB) without scaling. (b) EXIT chart with and without threshold ( dB) without scaling. scaling for SCCC2 at the threshold ( (c) Comparison between simulation results and convergence thresholds for SCCC2 with and without scaling ( , 10 iterations).
open iteration tunnel was observed with the same scaling factor , while they touched each other without scaling. The simulation results and the thresholds are shown in Fig. 4(c). With a stronger outer code (4-state CC), the performance gain by scaling is more significant than that with a weaker outer code (two-state CC) (see Fig. 3). As the previous system, the simulation results follow the calculated thresholds. As the interleaver size and number of iteration go to infinity, the simulation gain will approach to the computed threshold gain. V. CONCLUSION In this paper, a simpler min-sum density evolution technique was presented with the Gaussian approximation. Based on this method, the scaling of soft information on iterative decoding was analyzed on LDPC code and SCCC with two-state constituent codes. For more complex constituent codes, the SNR evolution and the EXIT chart techniques were used to explain the scaling gain. The optimal scaling gain and the expected maximum scaling gain were obtained by density evolution and it was demonstrated that these results predict simulation results well. ACKNOWLEDGMENT The authors would like to thank J. Melzer, Ph.D. candidate at the University of Southern California, for helpful discussions and generation of the EXIT charts.
[1] T. J. Richardson and R. L. Urbanke, “The capacity of low-density paritycheck codes under message-passing decoding,” IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 599–618, Feb. 2001. [2] S. Y. Chung, T. J. Richardson, and R. L. Urbanke, “Analysis of sumproduct decoding of low-density parity-check codes using a gaussian approximation,” IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 657–670, Feb. 2001. [3] A. Anastasopoulos, “A comparison between the sum-product and the min-sum iterative detection algorithms based on density evolution,” in Proc. IEEE GLOBECOM, 2001, pp. 1021–1025. [4] X. Wei and A. Akansu, “Density evolution for low-density parity-check codes under max-log-map decoding,” Electron. Lett., vol. 37, pp. 1125–1126, Aug. 2001. [5] J. Chen and M. P. C. Fossorier, “Density evolution for two improved bp-based decoding algorithm of LDPC codes,” IEEE Commun. Lett., vol. 6, no. 5, pp. 208–210, May 2002. [6] S. Ten Brink, “Convergence of iterative decoding,” Electron. Lett., vol. 35, pp. 806–808, May 1999. [7] , “Convergence behavior of iteratively decoded parallel concatenated codes,” IEEE Trans. Commun., vol. 49, no. 10, pp. 1727–1737, Oct. 2001. [8] M. Tuchler and J. Hagenauer, “Exit charts of irregular codes,” presented at the Conf. Information Sciences and Systems, Princeton, NJ, Mar. 2002. [9] H. E. Gamal and J. A. R. Hammons, “Analyzing the turbo decoder using the Gaussian approximation,” IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 671–686, Feb. 2001. [10] D. Divsalar, S. Dolinar, and F. Pollara, “Iterative turbo decoder analysis based on density evolution,” Jet Propulsion Lab., TMO Progress Reps., Feb. 2001. [11] X. Chen, K. M. Chugg, and M. A. Neifeld, “Near-optimal parallel distributed data detection for page-oriented optical memories,” IEEE J. Sel. Topics Quantum Electron., vol. 4, no. 9, pp. 866–879, Sep. 1998. [12] K. Chugg, A. Anastasopoulos, and X. Chen, Iterative Detection-Adaptivity, Complexity Reduction, and Applications. Boston, MA: Kluwer, 2001. [13] J. Chen and M. P. C. Fossorier, “Near optimum universal belief propagation based decoding of LDPC codes and extension to turbo decoding,” presented at the ISIT2001, Washington, DC, Jun. 2001. , “Decoding low-density parity check codes with normalized APP[14] based algorithm,” presented at the IEEE GLOBECOM, San Antonio, TX, Nov. 2001. [15] , “Near optimum universal belief propagation based decoding of low-density parity check codes,” IEEE Trans. Commun., vol. 50, no. 3, pp. 406–414, Mar. 2002. [16] K. Gracie, S. Crozier, and A. Hunt, “Performance of a low-complexity turbo decoder with a simple early stopping criterion implemented on a sharc processor,” presented at the Int. Mobile Satellite Conf., Ottawa, ON, Canada, Jun. 1999. [17] G. Colavolpe, G. Ferrari, and R. Raheli, “Extrinsic information in turbo decoding: A unified view,” in Proc. IEEE Globecom, 1999, pp. 505–509. [18] E. Ettelaie, “Slowing the convergence using filtering,” Univ. Southern California, Los Angeles, Class Project Report, EE666, Spring 2001. [19] J. Hagenauer, E. Offer, and L. Papke, “Iterative decoding of binary block and convolutional codes,” IEEE Trans. Inf. Theory, vol. 42, no. 2, pp. 429–445, Mar. 1996. [20] L. Ping, S. Chan, and K. L. Yeung, “Iterative decoding of multidimensional concatenated single parity check codes,” in Proc. IEEE Int. Conf. Communications, Jun. 1998, pp. 131–135. [21] S. Y. Chung, G. D. Forney, T. J. Richardson, and R. L. Urbanke, “On the design of low-density parity-check codes within 0.0045 db of the shannon limit,” IEEE Commun. Lett., vol. 5, pp. 58–60, Feb. 2001. [22] T. J. Richardson, M. A. Shokrollahi, and R. L. Urbanke, “Design of capacity-approaching irregular low-density parity-check codes,” IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 619–637, Feb. 2001.
957
Optimization of Scaling Soft Information in Iterative Decoding Via Density Evolution Methods Jun Heo, Member, IEEE, and Keith M. Chugg, Member, IEEE
Abstract—Density evolution has recently been used to analyze iterative decoding and explain many characteristics of iterative decoding including convergence of performance and preferred structures for the constituent codes. The scaling of extrinsic information (messages) has been heuristically used to enhance the performance in the iterative decoding literature, particularly based on the min-sum message passing algorithm. In this paper, it is demonstrated that density evolution can be used to obtain the optimal scaling factor and also estimate the maximum achievable scaling gain. For low density parity check (LDPC) codes and serially concatenated convolutional codes (SCCC) with two-state constituent codes, the analytic density evolution technique is used, while the signal-to-noise ratio (SNR) evolution technique and the EXIT chart technique is used for SCCC with more than 2 state constituent codes. Simulation results show that the scaling gain predicted by density evolution or SNR evolution matches well with the scaling gain observed by simulation. Index Terms—Iterative decoding, density evolution, low-density parity-check (LDPC) codes, serially concatenated convolutional codes (SCCC).
I. INTRODUCTION
R
ECENTLY developed density evolution techniques have been used to determine the analytic capacity of LDPC codes. In [1], the density evolution technique was recursively used to track the density of extrinsic information between the variable nodes and check nodes of an LDPC code. A simplified version of the density evolution technique was introduced with a Gaussian approximation in [2]. While these two approaches [1], [2] were based on the sum-product message passing algorithm, in [3]–[5] a density evolution technique based on the min-sum message passing was introduced. Similar to [1], the probability density of extrinsic information was tracked during iterative decoding in [3]–[5]. During the review of this letter, we found an independent work [5] which derived the min-sum density evolution without a Gaussian approximation. Because the LDPC decoding algorithm can be represented by a simple bipartite graph, most of the analytical work has been focused on LDPC codes. Meanwhile, similar attempts have been made for Turbo codes based on tracking the evolution of signal-to-noise ratio (SNR) Paper approved by H. El-Gamal, the Editor for Space-Time Coding and Spread Spectrum of the IEEE Communications Society. Manuscript received October 1, 2002; revised March 1, 2004. This work was supported in part by Grant R08-2003-000-10165-0 from the Basic Research Program of the Korea Science and Engineering Foundation and by the National Science Foundation under CCR-0082987. J. Heo was with the Communication Sciences Institute, University of Southern California, Los Angeles, CA 90089-2565 USA, and is now with the NITRI (Next Generation Innovative Technology Research Institute), Konkuk University, Seoul 143-701, Korea (e-mail: [email protected]). K. M. Chugg is with the Communication Sciences Institute, Electrical Engineering Department, University of Southern California, Los Angeles, CA 90089-2565 USA (e-mail: [email protected]). Digital Object Identifier 10.1109/TCOMM.2005.849782
values or mutual information of the extrinsic information. In [6]–[8] the mutual information transfer characteristic was used to show the convergence properties of turbo codes. In [9] the SNR evolution technique was used to achieve the asymptotic capacity (i.e., threshold) of Turbo codes as the interleaver size and the number of iterations go to infinity. The obtained thresholds agree well with the region where the bit error curves start to fall down. In [10], the SNR evolution technique was used to explain many mysteries of turbo codes and serially concatenated convolutional codes (SCCCs) (e.g., the role of systematic bits and the importance of recursive constituent codes). The SNR evolution techniques were classified into two categories. The actual SNR curves were obtained during a standard iterative decoding procedure by collecting the extrinsic information, while the Gaussian SNR curves were obtained by generating Gaussian random variables and feeding these into each constituent decoder. In the literature [11]–[15], damping of soft information in order to allow the suboptimal algorithm to better refine the beliefs has been used to improve performance. As a simple and effective method of damping soft information, scaling1 of extrinsic information was introduced in the iterative decoding literature [16] where the scaling technique (i.e., multiplying a constant value) was applied to the extrinsic information between the concatenated systems. Another approach is to view the scaling as a method to reduce the overestimation of the extrinsic information in the min-sum algorithm [17] where the scaling was applied to every marginalization (i.e., minimization) of the soft information. It was claimed that the performance of the min-sum algorithm could approach that of the sum-product algorithm by reducing the overestimation error. In [13]–[15], scaling was applied to LDPC codes and the scaling factor was calculated as a ratio of expectation value of log likelihood ratio (LLR). Sum-product message-passing can be run in the negative log domain using a slight modification of min-sum processing. Specifically, the minimum function can be augmented by a look-up table that accounts for the difference (e.g., see [12]). However, the look-up table method may require many more operations compared to that of scaling. Consider, for example, an 8-state parallel-concatenated code with an input block bits. For each constituent decoder activation, there size of will be (forward recursion), (backward recursion), (completion) table look-ups per activation, respecand multiplications in tively. This is to be compared against scaled-min-sum processing. Moreover, building a multiplier for scaling factor can be much simpler when the scaling factor is known at design time. Besides the number of operations, the scaled-min-sum computation can be desirable for a very 1As an alternative method, filtering of extrinsic information was introduced in [11], [12].
0090-6778/$20.00 © 2005 IEEE
958
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 53, NO. 6, JUNE 2005
fast decoder and a very flexible decoder which supports a wide range of code rates and different modulation formats. For such a flexible decoder, many look-up tables may be required to optimize the performance. For the scaled-min-sum algorithm, however, it is not clear when scaling results in a significant performance gain and the optimal scaling factor is determined by trial and error. In addition, it was recently shown that scaling greatly enhances performance when a relatively strong convolutional code is serially concatenated with a differential encoder under the min-sum decoding algorithm [18]. In this letter, we propose the min-sum density evolution techniques with a Gaussian approximation, which makes the calculation much simpler by tracking only the mean and variance instead of the probability density function (pdf). Using this proposed technique, density evolution is used to analyze the scaling gain for LDPC codes. The threshold values are calculated for various scaling factors, and the scaling factor showing the highest threshold in noise level, equivalently the lowest threshold in signal-to-noise ratio, is determined as the optimal scaling factor. At the same time, the maximum achievable scaling gain is estimated by the difference of threshold values. Simulation results for various scaling factors validates the analytical results by density evolution. Due to the similarity in a graphical representation, the SCCC with two-state constituent codes can also be analyzed similarly and the optimal scaling factor can be obtained as well. When the constituent codes have more than 2 states, the SNR evolution technique and the extrinsic mutual information transfer (EXIT) chart technique are used instead of density evolution. The rest of this letter is organized as follows. In Section II, density evolution based on the min-sum algorithm with a Gaussian approximation is developed for the LDPC codes and it is applied for the analysis of scaling. For SCCCs with two-state constituent codes, min-sum density evolution technique and the analysis of scaling are followed in Section III. In Section IV, the SNR evolution and the EXIT chart are used for scaling in more general SCCCs (more than two-state constituent codes). Finally, conclusions are provided in Section V.
where is the message output from a variable node and is represents the the message output from a check node, and received message from the channel. This min-sum message passing algorithm at a check node was shown in [3], [19], and [20]. According to this expression, the output message from a check node can be represented by the sign of the product of the incoming messages and the minimum absolute value among the incoming messages, where the message from the node receiving the output message is excluded. On the min-sum density evolution with a Gaussian assumpand variance of the tion, our goal is to track the mean message during iterative decoding. With the independent and identically distributed (iid) assumption for the exchanged mescan be obtained by (1) as: sages, and (3) (4) is more involved. When The calculation of and , the probability density function (pdf) of the output message from a check node based on (2) was derived in [3] as
(5) (6) and are the pdf and the cumulative density where is function (cdf), respectively. For the case of obtained by recursive update [i.e., (5)–(6)] with additional pdf up to as (7) a shorthand notation of (5)–(6), which is the pdf of where the output message for the case of . Once we have at th iteration, the mean and variance at th iteration are tracked as2
II. ANALYSIS OF SCALED MIN-SUM DECODING OF LDPC CODES
(8)
In this section, density evolution is used to explain the gain obtained by scaling in min-sum decoding of LDPC codes. The scaling is applied for the extrinsic information which is obtained based on the min-sum algorithm. Therefore, to analyze the scaling, a min-sum density evolution technique is required. In [3], min-sum density evolution was introduced without a Gaussian approximation. Combining this with Gaussian approximation, we present a simpler min-sum density evolution LDPC codes the min-sum technique. For regular message passing algorithm at variable and check nodes is represented by (1) (2)
where the scaling factor multiplies the message before it goes of message is cominto the check nodes, hence the pdf puted based on , instead of . Using the Gaussian assumption on , the pdf and cdf are obtained from . Then, the pdf is obtained by (5)–(6). The mean and variance are numerically calculated from . Finally, at th iteration is obtained by (3)–(4). The iterative mean and variance tracking is initialized by the received message from channel as: . This recursive calculation is executed for a sufficiently large number of iterations (e.g., 1000) to determine whether the message converges to correct codewords at a certain channel noise level. The convergence is determined 2In [3] and [4], the pdfs are tracked as: Gaussian assumption.
without
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 53, NO. 6, JUNE 2005
959
Fig. 2. (a) Block diagram of SCCC1 with two-state constituent codes . (b) Associated serially concatenated bipartite graph with scaling factor .
Fig. 1.
Simulation results for regular (3, 6) LDPC code for block size with various scaling factor . The corresponding thresholds ( dB) are shown in the parentheses.
in
when the probability of error, equivalently the tail part of the pdf goes to zero. The threshold is defined as the maximum channel noise level for which the message converges. We consider a rate 1/2, regular (3,6) LDPC code and compute the threshold of scaled min-sum decoding with different scale factors . The standard decoding without scaling corresponds to . The thresholds and corresponding performance (by simulation) is shown in Fig. 1 for the codeword block size of 10 000 bits. It is noted that the threshold strongly depends on the scaling factor and the scaling factor showing the highest threshold (equivalently, the lowest in dB) is optimal. , the threshold is 0.65 dB lower With the optimal than that (1.83 dB) of standard min-sum decoding. It was also seen that the simulation results follow the calculated thresholds. Moreover, the order of thresholds is maintained in the order of threshold always simulation performances (i.e., a lower results in better performance).
outer to the inner decoder. This is based on the experimental rule that the scaling should be applied to soft information transferred from the stronger code to the weaker code. The messages from check node to information and parity and respectively. variable nodes are represented by Similarly, the messages from information and parity variable respectively. nodes to check node are represented by and Based on the min-sum message passing algorithm, the messages at the inner code are represented by (9) (10) (11) (12) (13) (14) represents a received message from channel and where represents input extrinsic message from the outer constituent code. The messages at the outer code are also similarly represented. The messages exchanged between the inner and outer decoder are
III. ANALYSIS FOR SCCC CODES WITH TWO-STATE CONSTITUENT CODES In this section, density evolution is used to explain the gain obtained by scaling of the soft information in a SCCC system. This approach shows that scaling allows a higher noise level for convergence of soft information. We consider , two-state convolutional SCCC1 consisting of a rate code and a differential encoder which has a generator matrix . The block diagram of encoder SCCC1 is shown in Fig. 2(a). In this simple constituent and outputs have the relationship: code, the input . Using this relationship, we can draw a Tanner graph of this constituent code with variable nodes and check nodes as that of LDPC code. This makes it possible to adopt the analytic methods which have been used for LDPC codes. We can draw a serially concatenated bipartite graph with scaling coefficient , which is shown in Fig. 2(b). The scaling coefficient is only applied to the soft information from the
where represents the output extrinsic message from the inner represent the output extrinsic informacode, and , and from tion from the outer code. The input extrinsic message outer code is equally likely selected from two available extrinsic through a uniform interleaver as messages (15) Because of the independent and identically distributed assumption [1], [21] for the messages, we omit any time index in the message representation. The evolved mean and variance at the variable nodes are calculated by corresponding equation for the messages and iid assumption, whereas the evolved mean and
960
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 53, NO. 6, JUNE 2005
variance at check nodes are numerically evaluated by the relationship between input and output pdf and Gaussian assumption. ), once we Similar to the evolution of LDPC codes (with at th iteration by taking expectation have on both sides of (9)–(14), the mean and variance of and are numerically evaluated as
(16) Given the mean and variance of and , the extrinsic message fed into the outer code is
Similar to the inner code, the mean and variance of and at the check nodes of the outer code are numerically evaluated. The extrinsic messages from the outer code are
The next recursion for th iteration is started using the obtained values . This recursive calculation is initialized for the first iteration as
where the symmetry condition: [22] has been used for the received message from channel, because a symmetric channel is considered. This recursive calculation is executed for a sufficient number of iterations (e.g., 1000) to see whether the message converges or not given a certain channel noise level. Fig. 3 shows the threshold values for various scaling coefficients . The standard min-sum decoding without scaling cor. Note that with the optimal scaling coeffiresponds to , the threshold is noticeably (0.25 dB) lower than cient that without scaling. This suggests that the performance with dB) than without scaling in this optimal scaling is better ( particular system. The performance curves are also shown in Fig. 3 and the computed thresholds are plotted on it to show how the analytic threshold is matched to simulation results.
Fig. 3. (a) Comparison between simulation results and convergence thresholds , for SCCC1 with and without scaling ( 10 iterations), (b) The thresholds for SCCC2 on analytic computation with and without scaling.
IV. ANALYSIS OF SCALING FOR SCCCS WITH CONSTITUENT CODES HAVING MORE THAN TWO STATES In this section, the scaling of soft information is analyzed on a system having more than two-state outer convolutional code (in this paper, 4-state nonsystematic nonrecursive convolutional code is considered) with an inner differential encoder (SCCC2). Because it is difficult to get analytic expressions for this 4-state convolutional code, the simulation based SNR evolution and the EXIT chart using mutual information transfer characteristic are used to explain the scaling gain. The SNR evolution is widely used in the literature [9], [10], particulary, when it is difficult to apply the previous analytic methods as in the case of Turbo code and SCCC with complex constituent codes. The SNR evolution is obtained based on the collected messages at each iterative decoding step. From the collected messages, the mean for the exchanged soft information between two constituent codes is using the symcomputed. The SNR is approximated as metry condition for the exchanged message [22]. Instead of the collected LLRs, a generated Gaussian random soft information can be used as well. The EXIT chart technique was introduced in [6] which showed the mutual information transfer characteristic. An simplified construction of the EXIT chart was presented in [8]. For the EXIT chart, the mutual information is computed for both the input Gaussian soft information and the collected output extrinsic information. The considered system (SCCC2) is obtained by replacing the outer code of Section III with a 4-state convolutional code, . The inner differential encoder is unchanged. For this system, the thresholds are obtained by actual SNR ), the threshold is evolution. Without scaling (i.e., dB whereas with scaling , the threshold dB. The SNR evolution curves of the conis sidered system are shown in Fig. 4(a) at dB, which is the threshold without scaling. It is noted that the SNR have an open evolution curves with the scaling factor iteration tunnel, while they touch each other without scaling. In other words, the threshold with scaling is lower than that without scaling, which agrees to the results on previous SCCC1 system. The EXIT chart was presented in Fig. 4(b) for the same considered system. Similarly in the previous SNR evolution, an
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 53, NO. 6, JUNE 2005
961
REFERENCES
Fig. 4. (a) Actual SNR evolution with and without scaling for SCCC2 at the dB) without scaling. (b) EXIT chart with and without threshold ( dB) without scaling. scaling for SCCC2 at the threshold ( (c) Comparison between simulation results and convergence thresholds for SCCC2 with and without scaling ( , 10 iterations).
open iteration tunnel was observed with the same scaling factor , while they touched each other without scaling. The simulation results and the thresholds are shown in Fig. 4(c). With a stronger outer code (4-state CC), the performance gain by scaling is more significant than that with a weaker outer code (two-state CC) (see Fig. 3). As the previous system, the simulation results follow the calculated thresholds. As the interleaver size and number of iteration go to infinity, the simulation gain will approach to the computed threshold gain. V. CONCLUSION In this paper, a simpler min-sum density evolution technique was presented with the Gaussian approximation. Based on this method, the scaling of soft information on iterative decoding was analyzed on LDPC code and SCCC with two-state constituent codes. For more complex constituent codes, the SNR evolution and the EXIT chart techniques were used to explain the scaling gain. The optimal scaling gain and the expected maximum scaling gain were obtained by density evolution and it was demonstrated that these results predict simulation results well. ACKNOWLEDGMENT The authors would like to thank J. Melzer, Ph.D. candidate at the University of Southern California, for helpful discussions and generation of the EXIT charts.
[1] T. J. Richardson and R. L. Urbanke, “The capacity of low-density paritycheck codes under message-passing decoding,” IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 599–618, Feb. 2001. [2] S. Y. Chung, T. J. Richardson, and R. L. Urbanke, “Analysis of sumproduct decoding of low-density parity-check codes using a gaussian approximation,” IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 657–670, Feb. 2001. [3] A. Anastasopoulos, “A comparison between the sum-product and the min-sum iterative detection algorithms based on density evolution,” in Proc. IEEE GLOBECOM, 2001, pp. 1021–1025. [4] X. Wei and A. Akansu, “Density evolution for low-density parity-check codes under max-log-map decoding,” Electron. Lett., vol. 37, pp. 1125–1126, Aug. 2001. [5] J. Chen and M. P. C. Fossorier, “Density evolution for two improved bp-based decoding algorithm of LDPC codes,” IEEE Commun. Lett., vol. 6, no. 5, pp. 208–210, May 2002. [6] S. Ten Brink, “Convergence of iterative decoding,” Electron. Lett., vol. 35, pp. 806–808, May 1999. [7] , “Convergence behavior of iteratively decoded parallel concatenated codes,” IEEE Trans. Commun., vol. 49, no. 10, pp. 1727–1737, Oct. 2001. [8] M. Tuchler and J. Hagenauer, “Exit charts of irregular codes,” presented at the Conf. Information Sciences and Systems, Princeton, NJ, Mar. 2002. [9] H. E. Gamal and J. A. R. Hammons, “Analyzing the turbo decoder using the Gaussian approximation,” IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 671–686, Feb. 2001. [10] D. Divsalar, S. Dolinar, and F. Pollara, “Iterative turbo decoder analysis based on density evolution,” Jet Propulsion Lab., TMO Progress Reps., Feb. 2001. [11] X. Chen, K. M. Chugg, and M. A. Neifeld, “Near-optimal parallel distributed data detection for page-oriented optical memories,” IEEE J. Sel. Topics Quantum Electron., vol. 4, no. 9, pp. 866–879, Sep. 1998. [12] K. Chugg, A. Anastasopoulos, and X. Chen, Iterative Detection-Adaptivity, Complexity Reduction, and Applications. Boston, MA: Kluwer, 2001. [13] J. Chen and M. P. C. Fossorier, “Near optimum universal belief propagation based decoding of LDPC codes and extension to turbo decoding,” presented at the ISIT2001, Washington, DC, Jun. 2001. , “Decoding low-density parity check codes with normalized APP[14] based algorithm,” presented at the IEEE GLOBECOM, San Antonio, TX, Nov. 2001. [15] , “Near optimum universal belief propagation based decoding of low-density parity check codes,” IEEE Trans. Commun., vol. 50, no. 3, pp. 406–414, Mar. 2002. [16] K. Gracie, S. Crozier, and A. Hunt, “Performance of a low-complexity turbo decoder with a simple early stopping criterion implemented on a sharc processor,” presented at the Int. Mobile Satellite Conf., Ottawa, ON, Canada, Jun. 1999. [17] G. Colavolpe, G. Ferrari, and R. Raheli, “Extrinsic information in turbo decoding: A unified view,” in Proc. IEEE Globecom, 1999, pp. 505–509. [18] E. Ettelaie, “Slowing the convergence using filtering,” Univ. Southern California, Los Angeles, Class Project Report, EE666, Spring 2001. [19] J. Hagenauer, E. Offer, and L. Papke, “Iterative decoding of binary block and convolutional codes,” IEEE Trans. Inf. Theory, vol. 42, no. 2, pp. 429–445, Mar. 1996. [20] L. Ping, S. Chan, and K. L. Yeung, “Iterative decoding of multidimensional concatenated single parity check codes,” in Proc. IEEE Int. Conf. Communications, Jun. 1998, pp. 131–135. [21] S. Y. Chung, G. D. Forney, T. J. Richardson, and R. L. Urbanke, “On the design of low-density parity-check codes within 0.0045 db of the shannon limit,” IEEE Commun. Lett., vol. 5, pp. 58–60, Feb. 2001. [22] T. J. Richardson, M. A. Shokrollahi, and R. L. Urbanke, “Design of capacity-approaching irregular low-density parity-check codes,” IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 619–637, Feb. 2001.
