Partial Iterative Decoding for Binary Turbo Codes via ... - IEEE Xplore
3298
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 57, NO. 11, NOVEMBER 2009
Partial Iterative Decoding for Binary Turbo Codes via Cross-Entropy Based Bit Selection Jinhong Wu, Student Member, IEEE, Zhengdao Wang, Senior Member, IEEE, and Branimir R. Vojcic, Senior Member, IEEE
Abstract—Near-capacity performance of turbo codes is generally achieved with a large number of decoding iterations. Various iteration stopping rules introduced in the literature often induce performance loss. This paper proposes a novel partial decoding iteration scheme using a bit-level convergence test. We first establish decoding optimality of windowed partial iteration for non-converged bits given that convergence has been achieved on window boundaries. We next present two criteria for testing bit convergence based on cross-entropy, and propose a windowed partial iteration algorithm. The overall complexity and memory requirements of the new algorithm are evaluated and compared with known algorithms. Simulations reveal that the proposed scheme suffers essentially no performance loss compared to full iterations, while reducing the decoding complexity. We also briefly discuss possible extensions of the proposed scheme to general iterative receivers. Index Terms—Turbo decoding, partial iteration, stopping rule, cross entropy, parallel concatenated convolutional codes.
I. I NTRODUCTION
S
INCE the discovery of turbo codes[1], iterative decoding and receiver design have been widely adopted as an effective way to approach the information theoretic limits. Such capacity-approaching performance, however, is in general achieved with a large number of iterations. In the literature, two families of iteration stopping rules were developed to reduce the average number of iterations. Frame-based stopping rules examine the convergence status of a whole frame after each iteration. Depending on whether a criterion is met, the decoding process either stops completely, or continues with another complete iteration. In [2], [3], a cross-entropy criterion is proposed to measure frame convergence. Based on the cross-entropy concept, two simplified rules were introduced in [4]. In [5], frame error probability is estimated from decoder soft decisions and the decoding is stopped if the estimated probability is low enough. A hard decision based rule [3] stops iterations when the hard decisions do not change in several consecutive iterations. Among these frame-based stopping rules, cross-entropy based rules have Paper approved by T.-K. Truong, the Editor for Coding Theory and Techniques of the IEEE Communications Society. Manuscript received April 10, 2008; revised December 11, 2008 and April 21, 2009. J. Wu and B. Vojcic are with the Department of Electrical and Computer Engineering, George Washington University, Washington, DC, USA (e-mail: {johnnywu, vojcic}@gwu.edu). Z. Wang is with the Department of Electrical and Computer Engineering, Iowa State University, Ames, Iowa, USA (e-mail: [email protected]). This work was presented in part at the 2007 IEEE Global Telecommunications Conference (GLOBECOM). Digital Object Identifier 10.1109/TCOMM.2009.11.080182
been shown to yield the best performance. The problem with these frame-based methods is that frame convergence, being an average measure over all bits, may not reflect the convergence status of every bit and a few erroneous bits may still remain after the iteration is stopped. The second family is bit-level based. The goal is to identify possible erroneous bits and then design a partial iterative scheme on such “unreliable” bits. In [6], after decoding by each component decoder (half an iteration), bit reliability is determined by a threshold value on the magnitude of the loglikelihood ratio (LLR). The unreliable bits are kept in a single new, shortened trellis for decoding at the other decoder. In this approach, a higher threshold value keeps more bits in the next round of decoding, which results in better performance, but higher complexity, and vice versa. In [7], in the context of decoding with the soft output Viterbi algorithm (SOVA), bits are classified by the probability of a hard decision change in the next iteration. Bits with higher probabilities of changes are singled out and a window is applied for each selected bit. In the next iteration, decoding is only performed within the windows. This method is shown to attain a performance achievable with four complete SOVA iterations. Both methods, however, in general do not achieve the performance of a sufficient number of complete iterations. Aiming to achieve the performance of complete iterations, this paper proposes a bit-level partial iteration algorithm. When describing the algorithm, we will use the parallel concatenated convolutional code (PCCC) with BCJR decoding [8] as an example. But the discussion applies to other iterative algorithms as well. We use bit-level cross-entropy estimation to gauge convergence. We do not use LLRs or hard decisions because a bit with LLR above a certain threshold or with a fixed hard decision may still provide changing extrinsic information useful for other bits. From the belief propagation point of view [9], the evolving extrinsic information from individual bits is essential for the iterative gains. The cross-entropy value more accurately indicates whether the soft output of a bit has approached its final value. We devise a windowing scheme for performing partial iterations. First we define a short window (the primary window) on each non-converged bit. Then we apply a secondary window for each bit in the primary window on the other component decoder. Such a primary-secondary windowing scheme mimics full iterations, but only for non-converged bits. As a result, it essentially maintains the performance of complete iterations
c 2009 IEEE 0090-6778/09$25.00 ⃝
WU et al.: PARTIAL ITERATIVE DECODING FOR BINARY TURBO CODES VIA CROSS-ENTROPY BASED BIT SELECTION
as we will show using numerical simulations. Our primary window selection is similar to the one proposed for SOVA in [7]. The difference is in the windows boundary setup. In [7], the windowed trellises are terminated by making hard decisions on boundary variables. Whereas in this paper, the soft states from the previous iteration are kept, and they play an essential role in the subsequent partial iterations. The rest of the paper is organized as follows. In Sec. II, we briefly review the maximum a posteriori (MAP) decoding algorithm, and show the optimality of the partial MAP scheme for selected bits when boundary bits have converged. Sec. III discusses the suitability of cross-entropy measurements for bit convergence and introduces our proposed algorithm. In Sec. IV, we evaluate the computational complexity and memory requirements of the new scheme, and briefly discuss the implementation design. The effectiveness of the proposed scheme is shown in several simulations in Sec. V. Finally, Sec. VI presents the conclusions.
II. MAP DETECTION WITH REDUCED BLOCK LENGTH FOR SINGLE DECODER
√ 𝑦𝑗 (𝑛) = 𝑎𝑗 (𝑛) 𝐸𝑏 𝑟𝑐𝑗 (𝑛) + 𝑣𝑗 (𝑛),
(1)
where 𝑣𝑗 (𝑛) is zero mean additive white Gaussian noise (AWGN) with variance 𝑁20 , and 𝑎𝑗 (𝑛) is the fading attenuation for the transmitted bit. 𝐸𝑏 represents the energy per information bit and 𝑟 is the code rate. Following the well-known BCJR-MAP algorithm [1], [8], at each component decoder output, the a posteriori LLR for each bit is [10]: 𝐿(𝑛) = 𝐿(𝑥(𝑛) ∣ y) ∑ = ln
(𝑆 ′ ,𝑆)⇒𝑥(𝑛)=1
∑
(𝑆 ′ ,𝑆)⇒𝑥(𝑛)=0
𝛼(𝑛 − 1, 𝑆 ′ )𝛾(𝑛, 𝑆 ′ , 𝑆)𝛽(𝑛, 𝑆) 𝛼(𝑛 − 1, 𝑆 ′ )𝛾(𝑛, 𝑆 ′ , 𝑆)𝛽(𝑛, 𝑆)
𝛽(𝑁, 𝑆) values1 , 𝛼(𝑛, 𝑆) and 𝛽(𝑛, 𝑆) are calculated by [8]: ∑ 𝛼(𝑛, 𝑆) = 𝛾(𝑛, 𝑆 ′ , 𝑆)𝛼(𝑛 − 1, 𝑆 ′ ) (3) ′
′
𝛽(𝑛 − 1, 𝑆 ) =
𝑆 ∑
𝛽(𝑛, 𝑆)𝛾(𝑛, 𝑆 ′ , 𝑆)
. (2)
In the above, y is the vector representation of the received sequence; 𝛼(𝑛, 𝑆) and 𝛽(𝑛, 𝑆) are the forward and backward probabilistic metrics; 𝑆 or 𝑆 ′ refers to the encoder state: (𝑆 ′ , 𝑆) ⇒ 𝑥(𝑛) = 1 denotes the set of encoder state transitions from 𝑆 ′ to 𝑆 caused by the input 𝑥(𝑛) = 1, and similarly for (𝑆 ′ , 𝑆) ⇒ 𝑥(𝑛) = 0. With properly initialized 𝛼(0, 𝑆) and
(4)
𝑆
𝛾(𝑛, 𝑆 ′ , 𝑆) = ⎞ ⎛ 1 ∑ 1 𝐸𝑏 𝑟 √ exp ⎝− 2 [𝑦𝑗 (𝑛) − 𝑎𝑗 (𝑛)𝑐𝑗 (𝑛)]2 ⎠ 𝑃 (𝑥(𝑛)), 2 2𝜎 2𝜋𝜎 𝑗=0 (5) where 𝑐𝑗 (𝑛), 𝑗 = 0, 1, are the coded bits that can cause the channel state transition from 𝑆 ′ to 𝑆. The a priori probability 𝑃 (𝑥(𝑛)) and a priori LLR are related by 𝑒0.5(1+𝑥(𝑛))𝐿𝑎 (𝑛) . (6) 1 + 𝑒𝐿𝑎 (𝑛) Furthermore, the extrinsic LLR of each bit after decoding is [10] 𝑃 (𝑥(𝑛)) =
𝐿𝑒 (𝑛) = 𝐿(𝑛) − 𝐿𝑎 (𝑛) − 4𝑎0 (𝑛)
Consider MAP decoding for PCCC with binary antipodal modulation. We consider recursive systematic convolutional encoders whereby a data frame consisting of 𝑁 information bits 𝑥(𝑛) ∈ {0, 1} produces coded bits 𝑐𝑗 (𝑛) at the output of the a component encoder, 𝑗 = 0, 1, 𝑛 = 1, 2, ⋅ ⋅ ⋅ , 𝑁 . It is assumed that the channel is non-dispersive or has been rendered non-dispersive using schemes such as orthogonal frequency-division multiplexing (OFDM). The received signal for the corresponding component decoder after demodulation and de-multiplexing is
3299
𝐸𝑏 𝑟 𝑦0 (𝑛). 𝑁0
(7)
The extrinsic LLR after interleaving/de-interleaving becomes the a priori LLR for the other decoder. From (3)–(5), it is clear that after all 𝐿𝑎 (𝑛) and hence 𝑃 (𝑥(𝑛)) have converged, there will be no more changes in further iterations. Furthermore, {𝛼(𝑛 − 1, 𝑆), 𝑛 = 1, 2, ⋅ ⋅ ⋅ , 𝑁 } and {𝛽(𝑛, 𝑆), 𝑛 = 1, 2, ⋅ ⋅ ⋅ , 𝑁 } form two forward and backward Markov chains. When some a priori LLRs and the corresponding 𝛼, 𝛽 values have converged, the MAP computations of (3)–(5) among converged bits will not change in future iterations. This observation is illustrated in Fig. 1, where the shaded bits indicate that their LLRs have converged. Therefore, the computations for consecutive converged bits become redundant. For non-converged bits, a MAP with reduced block length (bounded by two converged bits) is sufficient. For this reason, computations can be saved for future iterations without compromising the performance. III. C ROSS - ENTROPY BASED BIT SELECTION AND WINDOWED PARTIAL ITERATION
The above observation indicates that for one component decoder, if good estimates of the convergence status of the coded bits can be obtained, we can continue with the next iteration performing MAP decoding only over small windows, each of which contains a non-converged bit with converged bits on its sides; MAP computations for consecutive converged bits can be omitted. In practice, if a significant number of bits reach convergence according to certain criteria, then we can group several likely converged bits on each side of the window. As soft information for these converged boundary bits fluctuate little in subsequent iterations, such an approach may be very close to the full MAP iteration in performance. 1 The initial values are generally chosen as 𝛼(0, 0) = 1, 𝛼(0, 𝑆) = 0 for all 𝑆 ∕= 0, and for terminated frames: 𝛽(𝑁, 0) = 1, 𝛽(𝑁, 𝑆) = 0 for all 𝑆 ∕= 0, while for unterminated frames: 𝛽(𝑁, 𝑆) = 1 for all 𝑆.
3300
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 57, NO. 11, NOVEMBER 2009
The cross-entropy criterion has been applied to determine frame convergence [2], [3]. In the following, we use this criterion to detect bit-level convergence and design a windowing scheme that performs decoding for the non-converged bits.
Denote the joint a posteriori probability distributions of the information bits x = [𝑥(1), 𝑥(2), ⋅ ⋅ ⋅ , 𝑥(𝑁 )] at the two decoders as 𝑄1 (x) and 𝑄2 (x), respectively. Assuming that the a priori LLRs of the information bits are statistically independent after each iteration, the cross-entropy between the two joint distributions is defined as [3], [11] { { } 𝑁∑ } −1 𝑄2 (x) 𝑄2 (𝑥(𝑛)) 𝐸𝑄2 log 𝑇𝑏𝑙𝑜𝑐𝑘 = 𝐸𝑄2 log = , 𝑄1 (x) 𝑄1 (𝑥(𝑛)) 𝑛=0 (8) where 𝐸𝑄2 {} denotes the expectation with respect to probability distribution 𝑄2 (𝑥(𝑛)). For binary codes, after the 𝑖th iteration, using the LLRs for the first decoder, the crossentropy for bit 𝑥(𝑛) can be approximated as [2], [3] ∣Δ𝐿𝑖𝑎 (𝑛)∣2 , exp (∣𝐿𝑖 (𝑛)∣)
(9)
𝑖 where Δ𝐿𝑖𝑎 (𝑛) = 𝐿𝑖+1 𝑎 (𝑛) − 𝐿𝑎 (𝑛). The block cross-entropy 𝑁 ∑ 𝑖 ≈ 𝑇 𝑖 (𝑛), and the frame cross-entropy stopping is 𝑇𝑏𝑙𝑜𝑐𝑘
rule uses
𝑛=1
𝑖 1 𝑇𝑏𝑙𝑜𝑐𝑘 < 𝑇𝑏𝑙𝑜𝑐𝑘 ⋅ 10−3
(10)
as the stopping criterion [2], [3]. It has been shown that (10) gives satisfactory performance in general. Iterative MAP decoding is essentially a cross-entropy minimization process [12]. Strict decoding convergence is reached when cross-entropy is zero, i.e., the two decoders provide the same distributions on all bits. In practice, (10) indicates that the distributions by the two decoders are very close. Note that (10) essentially compares the averaged convergence status of all bits and does not reflect the convergence status of individual bits. To perform windowed iterations, we next propose two bit convergence criteria. The first criterion is Criterion-I:
𝑇 𝑖 (𝑛)
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 57, NO. 11, NOVEMBER 2009
Partial Iterative Decoding for Binary Turbo Codes via Cross-Entropy Based Bit Selection Jinhong Wu, Student Member, IEEE, Zhengdao Wang, Senior Member, IEEE, and Branimir R. Vojcic, Senior Member, IEEE
Abstract—Near-capacity performance of turbo codes is generally achieved with a large number of decoding iterations. Various iteration stopping rules introduced in the literature often induce performance loss. This paper proposes a novel partial decoding iteration scheme using a bit-level convergence test. We first establish decoding optimality of windowed partial iteration for non-converged bits given that convergence has been achieved on window boundaries. We next present two criteria for testing bit convergence based on cross-entropy, and propose a windowed partial iteration algorithm. The overall complexity and memory requirements of the new algorithm are evaluated and compared with known algorithms. Simulations reveal that the proposed scheme suffers essentially no performance loss compared to full iterations, while reducing the decoding complexity. We also briefly discuss possible extensions of the proposed scheme to general iterative receivers. Index Terms—Turbo decoding, partial iteration, stopping rule, cross entropy, parallel concatenated convolutional codes.
I. I NTRODUCTION
S
INCE the discovery of turbo codes[1], iterative decoding and receiver design have been widely adopted as an effective way to approach the information theoretic limits. Such capacity-approaching performance, however, is in general achieved with a large number of iterations. In the literature, two families of iteration stopping rules were developed to reduce the average number of iterations. Frame-based stopping rules examine the convergence status of a whole frame after each iteration. Depending on whether a criterion is met, the decoding process either stops completely, or continues with another complete iteration. In [2], [3], a cross-entropy criterion is proposed to measure frame convergence. Based on the cross-entropy concept, two simplified rules were introduced in [4]. In [5], frame error probability is estimated from decoder soft decisions and the decoding is stopped if the estimated probability is low enough. A hard decision based rule [3] stops iterations when the hard decisions do not change in several consecutive iterations. Among these frame-based stopping rules, cross-entropy based rules have Paper approved by T.-K. Truong, the Editor for Coding Theory and Techniques of the IEEE Communications Society. Manuscript received April 10, 2008; revised December 11, 2008 and April 21, 2009. J. Wu and B. Vojcic are with the Department of Electrical and Computer Engineering, George Washington University, Washington, DC, USA (e-mail: {johnnywu, vojcic}@gwu.edu). Z. Wang is with the Department of Electrical and Computer Engineering, Iowa State University, Ames, Iowa, USA (e-mail: [email protected]). This work was presented in part at the 2007 IEEE Global Telecommunications Conference (GLOBECOM). Digital Object Identifier 10.1109/TCOMM.2009.11.080182
been shown to yield the best performance. The problem with these frame-based methods is that frame convergence, being an average measure over all bits, may not reflect the convergence status of every bit and a few erroneous bits may still remain after the iteration is stopped. The second family is bit-level based. The goal is to identify possible erroneous bits and then design a partial iterative scheme on such “unreliable” bits. In [6], after decoding by each component decoder (half an iteration), bit reliability is determined by a threshold value on the magnitude of the loglikelihood ratio (LLR). The unreliable bits are kept in a single new, shortened trellis for decoding at the other decoder. In this approach, a higher threshold value keeps more bits in the next round of decoding, which results in better performance, but higher complexity, and vice versa. In [7], in the context of decoding with the soft output Viterbi algorithm (SOVA), bits are classified by the probability of a hard decision change in the next iteration. Bits with higher probabilities of changes are singled out and a window is applied for each selected bit. In the next iteration, decoding is only performed within the windows. This method is shown to attain a performance achievable with four complete SOVA iterations. Both methods, however, in general do not achieve the performance of a sufficient number of complete iterations. Aiming to achieve the performance of complete iterations, this paper proposes a bit-level partial iteration algorithm. When describing the algorithm, we will use the parallel concatenated convolutional code (PCCC) with BCJR decoding [8] as an example. But the discussion applies to other iterative algorithms as well. We use bit-level cross-entropy estimation to gauge convergence. We do not use LLRs or hard decisions because a bit with LLR above a certain threshold or with a fixed hard decision may still provide changing extrinsic information useful for other bits. From the belief propagation point of view [9], the evolving extrinsic information from individual bits is essential for the iterative gains. The cross-entropy value more accurately indicates whether the soft output of a bit has approached its final value. We devise a windowing scheme for performing partial iterations. First we define a short window (the primary window) on each non-converged bit. Then we apply a secondary window for each bit in the primary window on the other component decoder. Such a primary-secondary windowing scheme mimics full iterations, but only for non-converged bits. As a result, it essentially maintains the performance of complete iterations
c 2009 IEEE 0090-6778/09$25.00 ⃝
WU et al.: PARTIAL ITERATIVE DECODING FOR BINARY TURBO CODES VIA CROSS-ENTROPY BASED BIT SELECTION
as we will show using numerical simulations. Our primary window selection is similar to the one proposed for SOVA in [7]. The difference is in the windows boundary setup. In [7], the windowed trellises are terminated by making hard decisions on boundary variables. Whereas in this paper, the soft states from the previous iteration are kept, and they play an essential role in the subsequent partial iterations. The rest of the paper is organized as follows. In Sec. II, we briefly review the maximum a posteriori (MAP) decoding algorithm, and show the optimality of the partial MAP scheme for selected bits when boundary bits have converged. Sec. III discusses the suitability of cross-entropy measurements for bit convergence and introduces our proposed algorithm. In Sec. IV, we evaluate the computational complexity and memory requirements of the new scheme, and briefly discuss the implementation design. The effectiveness of the proposed scheme is shown in several simulations in Sec. V. Finally, Sec. VI presents the conclusions.
II. MAP DETECTION WITH REDUCED BLOCK LENGTH FOR SINGLE DECODER
√ 𝑦𝑗 (𝑛) = 𝑎𝑗 (𝑛) 𝐸𝑏 𝑟𝑐𝑗 (𝑛) + 𝑣𝑗 (𝑛),
(1)
where 𝑣𝑗 (𝑛) is zero mean additive white Gaussian noise (AWGN) with variance 𝑁20 , and 𝑎𝑗 (𝑛) is the fading attenuation for the transmitted bit. 𝐸𝑏 represents the energy per information bit and 𝑟 is the code rate. Following the well-known BCJR-MAP algorithm [1], [8], at each component decoder output, the a posteriori LLR for each bit is [10]: 𝐿(𝑛) = 𝐿(𝑥(𝑛) ∣ y) ∑ = ln
(𝑆 ′ ,𝑆)⇒𝑥(𝑛)=1
∑
(𝑆 ′ ,𝑆)⇒𝑥(𝑛)=0
𝛼(𝑛 − 1, 𝑆 ′ )𝛾(𝑛, 𝑆 ′ , 𝑆)𝛽(𝑛, 𝑆) 𝛼(𝑛 − 1, 𝑆 ′ )𝛾(𝑛, 𝑆 ′ , 𝑆)𝛽(𝑛, 𝑆)
𝛽(𝑁, 𝑆) values1 , 𝛼(𝑛, 𝑆) and 𝛽(𝑛, 𝑆) are calculated by [8]: ∑ 𝛼(𝑛, 𝑆) = 𝛾(𝑛, 𝑆 ′ , 𝑆)𝛼(𝑛 − 1, 𝑆 ′ ) (3) ′
′
𝛽(𝑛 − 1, 𝑆 ) =
𝑆 ∑
𝛽(𝑛, 𝑆)𝛾(𝑛, 𝑆 ′ , 𝑆)
. (2)
In the above, y is the vector representation of the received sequence; 𝛼(𝑛, 𝑆) and 𝛽(𝑛, 𝑆) are the forward and backward probabilistic metrics; 𝑆 or 𝑆 ′ refers to the encoder state: (𝑆 ′ , 𝑆) ⇒ 𝑥(𝑛) = 1 denotes the set of encoder state transitions from 𝑆 ′ to 𝑆 caused by the input 𝑥(𝑛) = 1, and similarly for (𝑆 ′ , 𝑆) ⇒ 𝑥(𝑛) = 0. With properly initialized 𝛼(0, 𝑆) and
(4)
𝑆
𝛾(𝑛, 𝑆 ′ , 𝑆) = ⎞ ⎛ 1 ∑ 1 𝐸𝑏 𝑟 √ exp ⎝− 2 [𝑦𝑗 (𝑛) − 𝑎𝑗 (𝑛)𝑐𝑗 (𝑛)]2 ⎠ 𝑃 (𝑥(𝑛)), 2 2𝜎 2𝜋𝜎 𝑗=0 (5) where 𝑐𝑗 (𝑛), 𝑗 = 0, 1, are the coded bits that can cause the channel state transition from 𝑆 ′ to 𝑆. The a priori probability 𝑃 (𝑥(𝑛)) and a priori LLR are related by 𝑒0.5(1+𝑥(𝑛))𝐿𝑎 (𝑛) . (6) 1 + 𝑒𝐿𝑎 (𝑛) Furthermore, the extrinsic LLR of each bit after decoding is [10] 𝑃 (𝑥(𝑛)) =
𝐿𝑒 (𝑛) = 𝐿(𝑛) − 𝐿𝑎 (𝑛) − 4𝑎0 (𝑛)
Consider MAP decoding for PCCC with binary antipodal modulation. We consider recursive systematic convolutional encoders whereby a data frame consisting of 𝑁 information bits 𝑥(𝑛) ∈ {0, 1} produces coded bits 𝑐𝑗 (𝑛) at the output of the a component encoder, 𝑗 = 0, 1, 𝑛 = 1, 2, ⋅ ⋅ ⋅ , 𝑁 . It is assumed that the channel is non-dispersive or has been rendered non-dispersive using schemes such as orthogonal frequency-division multiplexing (OFDM). The received signal for the corresponding component decoder after demodulation and de-multiplexing is
3299
𝐸𝑏 𝑟 𝑦0 (𝑛). 𝑁0
(7)
The extrinsic LLR after interleaving/de-interleaving becomes the a priori LLR for the other decoder. From (3)–(5), it is clear that after all 𝐿𝑎 (𝑛) and hence 𝑃 (𝑥(𝑛)) have converged, there will be no more changes in further iterations. Furthermore, {𝛼(𝑛 − 1, 𝑆), 𝑛 = 1, 2, ⋅ ⋅ ⋅ , 𝑁 } and {𝛽(𝑛, 𝑆), 𝑛 = 1, 2, ⋅ ⋅ ⋅ , 𝑁 } form two forward and backward Markov chains. When some a priori LLRs and the corresponding 𝛼, 𝛽 values have converged, the MAP computations of (3)–(5) among converged bits will not change in future iterations. This observation is illustrated in Fig. 1, where the shaded bits indicate that their LLRs have converged. Therefore, the computations for consecutive converged bits become redundant. For non-converged bits, a MAP with reduced block length (bounded by two converged bits) is sufficient. For this reason, computations can be saved for future iterations without compromising the performance. III. C ROSS - ENTROPY BASED BIT SELECTION AND WINDOWED PARTIAL ITERATION
The above observation indicates that for one component decoder, if good estimates of the convergence status of the coded bits can be obtained, we can continue with the next iteration performing MAP decoding only over small windows, each of which contains a non-converged bit with converged bits on its sides; MAP computations for consecutive converged bits can be omitted. In practice, if a significant number of bits reach convergence according to certain criteria, then we can group several likely converged bits on each side of the window. As soft information for these converged boundary bits fluctuate little in subsequent iterations, such an approach may be very close to the full MAP iteration in performance. 1 The initial values are generally chosen as 𝛼(0, 0) = 1, 𝛼(0, 𝑆) = 0 for all 𝑆 ∕= 0, and for terminated frames: 𝛽(𝑁, 0) = 1, 𝛽(𝑁, 𝑆) = 0 for all 𝑆 ∕= 0, while for unterminated frames: 𝛽(𝑁, 𝑆) = 1 for all 𝑆.
3300
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 57, NO. 11, NOVEMBER 2009
The cross-entropy criterion has been applied to determine frame convergence [2], [3]. In the following, we use this criterion to detect bit-level convergence and design a windowing scheme that performs decoding for the non-converged bits.
Denote the joint a posteriori probability distributions of the information bits x = [𝑥(1), 𝑥(2), ⋅ ⋅ ⋅ , 𝑥(𝑁 )] at the two decoders as 𝑄1 (x) and 𝑄2 (x), respectively. Assuming that the a priori LLRs of the information bits are statistically independent after each iteration, the cross-entropy between the two joint distributions is defined as [3], [11] { { } 𝑁∑ } −1 𝑄2 (x) 𝑄2 (𝑥(𝑛)) 𝐸𝑄2 log 𝑇𝑏𝑙𝑜𝑐𝑘 = 𝐸𝑄2 log = , 𝑄1 (x) 𝑄1 (𝑥(𝑛)) 𝑛=0 (8) where 𝐸𝑄2 {} denotes the expectation with respect to probability distribution 𝑄2 (𝑥(𝑛)). For binary codes, after the 𝑖th iteration, using the LLRs for the first decoder, the crossentropy for bit 𝑥(𝑛) can be approximated as [2], [3] ∣Δ𝐿𝑖𝑎 (𝑛)∣2 , exp (∣𝐿𝑖 (𝑛)∣)
(9)
𝑖 where Δ𝐿𝑖𝑎 (𝑛) = 𝐿𝑖+1 𝑎 (𝑛) − 𝐿𝑎 (𝑛). The block cross-entropy 𝑁 ∑ 𝑖 ≈ 𝑇 𝑖 (𝑛), and the frame cross-entropy stopping is 𝑇𝑏𝑙𝑜𝑐𝑘
rule uses
𝑛=1
𝑖 1 𝑇𝑏𝑙𝑜𝑐𝑘 < 𝑇𝑏𝑙𝑜𝑐𝑘 ⋅ 10−3
(10)
as the stopping criterion [2], [3]. It has been shown that (10) gives satisfactory performance in general. Iterative MAP decoding is essentially a cross-entropy minimization process [12]. Strict decoding convergence is reached when cross-entropy is zero, i.e., the two decoders provide the same distributions on all bits. In practice, (10) indicates that the distributions by the two decoders are very close. Note that (10) essentially compares the averaged convergence status of all bits and does not reflect the convergence status of individual bits. To perform windowed iterations, we next propose two bit convergence criteria. The first criterion is Criterion-I:
𝑇 𝑖 (𝑛)
