Structured IRA Codes: Performance Analysis and ... - CiteSeerX
1
Structured IRA Codes: Performance Analysis and Construction Yifei Zhang, Student Member, IEEE, and William E. Ryan, Senior Member, IEEE
Abstract In this paper, we present design techniques for structured irregular repeat-accumulate (S-IRA) codes with low error-rate floors. These S-IRA codes need not be quasi-cyclic, permitting flexibility in code dimension, length, and rate. We present a simple ensemble estimate of the level of the error-rate floor of finite-length IRA codes on the additive white Gaussian noise (AWGN) channel. This performance estimate provides guidance on the choice of IRA code column weights which yield low floors. We also present two design algorithms for S-IRA codes accompanied by software- and hardware-based performance results which demonstrate their low floors. Lastly, we present two design algorithms for multi-rate S-IRA code families implementable by a single encoder/decoder.
I. I NTRODUCTION Low-density parity-check (LDPC) codes were introduced by Gallager circa 1960 [1] and were re-discovered in 1996 by MacKay and Neal [2]. Since the work of MacKay and Neal, several LDPC code subclasses have been proposed, perhaps most notably, the accumulator-based LDPC codes. This subclass includes the repeat-accumulate (RA) codes [3], [4], the irregular repeat-accumulate (IRA) codes [5], the extended IRA (eIRA) codes [6], the accumulate-repeat-accumulate (ARA) codes [7], and the accumulate-repeat-accumulate-accumulate (ARAA) codes [8]. Accumulator-based codes offer the advantages of linearcomplexity encoding and low error-rate floors while sacrificing little in the decoding threshold [3], [5], [6], [7], [8]. We will not consider ARA or ARAA codes in this paper. An RA code is a serial concatenation of a repetition code and an accumulator through an interleaver: a length-k information block is repeated q times, scrambled by a qk × qk interleaver Π, and then encoded by a rate-1 accumulator having transfer function 1/(1 ⊕ D). When the information bits are transmitted along with the accumulator output, the result is a rate-1/(q + 1) systematic RA code; otherwise, it is a rate-1/q nonsystematic RA code. We emphasize systematic codes in this paper. The Tanner graph for the systematic RA code is given in Fig. 1(a). Note, as indicated in the figure for an RA code, each information bit This work was funded in part by NASA grants NAG5-13254 and NNG04GI06G and a gift from the INSIC EHDR program.
2
node connects to exactly q check nodes (db,i = q) and each check node connects to exactly one information bit node (dc,i = 1). Further, the number of parity bits m is given by m = qk. The systematic IRA codes generalize the systematic RA codes in that the repetition rate may differ across the k information bits. Further, according to Figure 1 in [5] and its associated discussion, m > k and the check nodes all connect to exactly a ≥ 1 information bit nodes. In the context of Fig. 1(a), this means that db,i varies with i and dc,i = a for some fixed a ≥ 1. However, as pointed out by the reviewers and H. Jin [9], the constraints m > k and dc,i = a are relaxed later in [5]. Thus, what have been called eIRA codes in [6] – db,i and dc,i are both variable and m need not be greater than k – can simply be called IRA codes and we shall do so hereafter.
Fig. 1: Structures of RA and IRA codes: (a) Tanner graph. (b) Systematic encoder.
The parity-check matrix for systematic RA and IRA codes has the form
H = [H1 H2 ],
(1)
3
where H2 is an m × m “dual-diagonal” square matrix, 1 1 H2 =
1 ..
. 1
1 1
1
.
(2)
For RA codes, H1 is a regular matrix having column weight q and row weight 1. For IRA codes, H1 has column weights {db,i } and row weights {dc,i }. The encoder of Fig. 1(b) will be useful to our ensemble analysis and is obtained by writing H1 as ΠT BT , where Π is a permutation matrix. In this paper, we extend Tanner’s quasi-cyclic RA code designs [4] to quasi-cyclic and near quasi-cyclic IRA code designs, which we simply call structured IRA (S-IRA) codes. We first present a simple ensemble estimate of the level of the errorrate floor of finite-length IRA codes on the AWGN channel. This performance estimate provides a guide to the selection of the design parameters {db,i } for IRA codes with low floors. We then present our definition of S-IRA codes, a construction algorithm, and simulation results. Lastly, we demonstrate the versatility of the S-IRA code class in that a multi-rate code family has been designed which may be implemented with a single encoder/decoder. These results also demonstrate that these codes are flexible in that designs with very specific information word lengths and code rates are possible, which is not the case for many other code classes, such as the ones in [4]. Related work may be found in [3], [4], [5], [6], [10], [11] and [12]. As indicated above, our contribution is the design of structured IRA codes with low floors for both high-rate and low-rate codes, in contrast to the works in [4] and [10] which focus on structured RA and IRA codes with low rates. Our analysis of the IRA code ensemble exploits the method of [3] which was used to prove the near-capacity capabilities of accumulator-based “turbo-like” codes, specifically, RA codes, on the AWGN channel. Our result provides insight into the design of IRA codes with low floors, in contrast to the work in [5] which deals with the design of good-threshold IRA codes using the Gaussian approximation. Furthermore, we focus on the structured IRA codes design as opposed to design of random and semi-random RA/IRA codes in [12]. The rest of this paper is organized as follows. Section II derives an estimate on the floor performance of a subset of IRA codes and presents some numerical examples for the binary-input AWGN channel. Sections III and IV introduce the S-IRA codes and two design algorithms. Section V presents two S-IRA code family design algorithms and example simulation results.
4
II. IRA C ODE E NSEMBLE P ERFORMANCE E STIMATE A. The Union Bound and Estimate Error floor estimation via the union bound is particularly valuable for signal-to-noise ratios (SNRs) that are beyond the reach of simulations. We have found it difficult to analyze truly irregular IRA codes and so we consider a subset of the full IRA ensemble. The IRA code ensemble we will analyze can be defined by parameter triple < k, m, wc >, where k is the number of information bit nodes, m is the number of check nodes, and wc is the column weight of H1 (db,i = wc ). We leave the row weight distribution {dc,i } of H1 unspecified. To simplify the analysis, we assume the wc 1’s in each column of H1 are uniformly distributed and the 1’s in any two rows have at most one position in common, i.e., there are no 4-cycles in the graph of H1 . Note that the ensemble we consider has finite length. The union bound on the bit error rate (BER) is given by
Pb
0
, Z=e
(3)
−Rc Eb /N0
where X
Dd =
w:w+h=d
w Aw,h . k
(4)
Here, Aw,h is the average input-output weight enumerator (IOWE) over the IRA code ensemble, where w and h are the Hamming weights of information block and parity block, respectively. To facilitate the analysis, we use the encoder structure in Fig. 1 (b) composed of the matrix B = HT1 Π−1 , the uniform interleaver Π, and the accumulator. Then, by analogy to serial turbo codes, the average IOWE of our IRA code ensemble can be written as
Aw,h =
X
(acc)
(1) Al,h Aw,l · ¡m¢ ,
(5)
l
l (1)
(acc)
where Aw,l is the average IOWE of the outer code corresponding to B and Al,h
is the IOWE of the inner accumulator code.
Per Fig. 1 (b), l and h are the accumulator input and output weights, respectively. The IOWE of the accumulator is given by [3]
µ (acc) Al,h
=
m−h bl/2c
¶µ
¶ h−1 . dl/2e − 1
(6)
The average IOWE of the outer code is a function of the input-output weight transfer probability (IOWTP) [13], Pw,l , as follows: (1)
Aw,l =
µ ¶ k Pw,l . w
(7)
5
The computation of the IOWTP Pw,l for all possible w and l is a difficult problem. Fortunately, from the analysis of turbolike codes, we know that the bound behavior at high SNRs is dominated by low-weight encoder inputs (i.e., small w). Hence, we consider the computation of an approximation of the union bound which exploits this fact. Suppose the input block has weight w = 1, the output block of the HT1 encoder, p1 , is precisely a row of HT1 , which means P1,l is 1 for l = wc and 0 for l 6= wc . For weight-2 inputs (w = 2), the weight of p1 will either be 2wc or 2(wc − 1) since no 4-cycles exist in HT1 . The IOWTP can be computed using
Nw,l Pw,l = P , Nw,l0 l
(8)
0
where Nw,l is the number of ways that p1 has weight l given weight-w inputs. In the case of w = 2, the two terms can be calculated by [11]
µ N2,2·wc =
and
m wc
¶µ
¶ m − wc /2 wc
µ N2,2·(wc −1) =
¶ µ ¶ m m − wc wc /2. wc wc − 1
The results in [11] show that P1,l and P2,l dominate the code ensemble performance in the error floor region. Thus, we omit all terms corresponding to w ≥ 3 in the following numerical examples.
B. Numerical Results We applied our performance estimate to the following IRA code < k, m, wc > ensembles, grouped so that codes of like parameters may be compared: 1) < 100, 100, 3 > and < 100, 100, 5 > 2) < 1000, 1000, 3 > and < 1600, 400, 3 > 3) < 100, 100, 3 > and < 1000, 1000, 3 > 4) < 3141, 730, 3 >, < 3141, 730, 4 >, and < 3141, 730, 5 >. For each ensemble in the first three groups, two codes are randomly selected (designed). In the last group, only one code is designed for each ensemble. The results of the above four comparison groups are shown in Fig. 2. The simulated bit error rates have solid curves and the error floor estimates have dashed curves. We observe the floor estimates are consistent with the simulation results for the randomly selected codes. Moreover, we notice that increasing wc (to at least 5) tends to lower the floor.
6
IRA ensembles
IRA ensembles vs
BER
−5
10
BER
−5
10
−10
10
−10
2
4 Eb/N0 [dB]
10
6
2
IRA ensembles vs
10
w =4
BER
wc=3
c
BER
10
6
IRA ensembles
−5
−5
4 Eb/N0 [dB]
wc=5
−10
10 −10
10
2
4 E /N [dB] b
6
2
3
0
4 E /N [dB] b
5
6
0
Fig. 2: Comparison of the performance of randomly selected IRA codes with ensemble estimates. III. S TRUCTURED IRA C ODES To specify the class of S-IRA codes, we first define the submatrix, P, of one version of the parity-check matrix as discussed below. The matrix P is an L × J array of Q × Q (Q can be any integer) circulant permutation submatrices, which can be represented by P=
b0,0
b0,1
...
b0,J−1
b1,0
b1,1
...
b1,J−1
.. . bL−1,0
bL−1,1
... bL−1,J−1
,
(9)
L×J
where bl,j ∈ { ∞, 0, 1, ..., Q − 1} is the exponent of the circulant permutation matrix π formed by cyclically shifting (mod Q) each row of a Q × Q identity matrix IQ to the right once. Thus, (9) is a short-hand notation in which each entry bl,j should be replaced by π bl,j . We define π ∞ to be the Q × Q zero matrix OQ which corresponds to the “masked out” entries of P [14]. These entries are recorded in a L × J binary masking matrix [14] which is chosen in accordance with a targeted degree distribution. Because of the structure of P, rows and columns occur in groups of size Q. We will refer to row group l,
7
0 ≤ l ≤ L, and column group j, 0 ≤ j ≤ J. Thus, the masking matrix determines the edge connections between row groups and column groups. It is easy to show that using matrices with circulant submatrices such as P in place of H1 will produce a poor LDPC code in the sense of minimum distance (consider weight-2 encoder inputs with adjacent ones). In view of this, to create the parity-check matrix for our S-IRA codes, instead of replacing H1 with P, we replace BT (see Fig. 1) with P so that H1 = ΠT P and
H = [ΠT P H2 ].
(10)
Our goal of explicitly adding the interleaver Π is to achieve a larger minimum codeword weight — at least for codewords corresponding to weight-2 inputs — via proper choice of the interleaver. We would simultaneously like to impart an organized structure in H, which constrains Π to be deterministic rather than (pseudo-)random. Thus, we choose Π so that row qL + l in row group q of H1 is row lQ + q in row group l of P, where 0 ≤ l < L and 0 ≤ q < Q. Observe that this choice of interleaver has the effect of spreading the 1’s in the word p1 for weight-2 inputs. The idea of spreading 1’s appears also in [4]. Note that an alternative parity-check matrix for a given S-IRA code is a row-permuted version of the one above:
H0 = Π−T H = [P ΠH2 ],
(11)
where we have used the fact that Π−T = Π. There is yet a third parity-check matrix form, first given by Tanner in the context of quasi-cyclic RA codes [4]:
H00 = [P
ΠH2 Π].
(12)
Observe that H00 describes essentially the same code as H0 , except for a re-ordering of the parity bits. We remark that, since H2 is not circulant, there is a minor difference between the form given by Tanner and that of H00 : a missing 1 in the upper-right corner of H00 . Further, S-IRA codes need not be quasi-cyclic for two reasons. First, we do not require the number of columns of H2 to be a multiple of Q. Second, we allow deletions of columns and rows of P so that m and k are not necessarily multiples of Q.
8
IV. C ODE D ESIGNS We discuss two S-IRA code design algorithms in this section and present selected performance results for each. In the first algorithm, we require the following three conditions. Condition 1: To avoid short cycles which have two nodes in H1 , we require that the bi,j ’s in one column of P be distinct. Condition 2: There will be no length-4 cycles within the systematic (H1 ) part of the code’s graph provided [15]
(bl1 ,j1 − bl1 ,j2 )
mod Q 6= (bl2 ,j1 − bl2 ,j2 ) mod Q,
(13)
where l1 , l2 ∈ [0, L), l1 6= l2 and j1 , j2 ∈ [0, J), j1 6= j2 . There will be no length-4 cycles across the H1 and H2 parts of H provided
bl,j 6= bl+1,j and (bL−1,j − b0,j ) mod Q 6= 1,
(14)
where l ∈ [0, L − 1), j ∈ [0, J). Condition 3: There will be no length-6 cycles in the code’s graph in which four nodes are systematic nodes and two are parity nodes provided (bl1 ,j1 − bl2 ,j1 ) mod Q 6= (bl1 ,j2 − bl2 ±1,j2 ) mod Q
(15)
and (bl,j1 − b0,j1 ) mod Q 6= (bl,j2 − bL−1,j2 + 1)
mod Q,
(16)
where l1 , l2 , and l2 ± 1 ∈ [0, L) are distinct, l ∈ [0, L), and j1 , j2 ∈ [0, J), j1 6= j2 . There will be no length-6 cycles in the code’s graph in which two nodes are systematic nodes and four are parity nodes provided
(bl,j − b(l+2)
mod L,j )
where l ∈ [L − 2, L) and j ∈ [0, J).
Algorithm 1 Initialization: Initialize the parity part of H to ΠH2 . Generation of P: for j = 0 to J − 1 do
mod Q 6= 1,
(17)
9
for l = 0 to L − 1 do if bl,j is not masked REP: randomly select an integer x ∈ [0, Q) different from other already generated bl0 ,j and set bl,j = x. if any condition is unsatisfied if every possible x ∈ [0, Q) has been tested, then remove Condition 3. goto REP. else write circulant π x into the parity-check matrix. end (for l) end (for j)
Example 1: In this example, we present two S-IRA codes with low floors designed via Algorithm 1. The first is a rate-1/2 (2044, 1024) S-IRA code with L = J = 16, Q = 64, and wc = 5. The second is a rate-0.8 (5120, 4096) S-IRA code with L = 8, J = 32, Q = 128 and wc = 5. These codes were simulated in software and hardware [16] using close approximations of the sum-product algorithm and the results are presented in Fig. 3 and 4. For both bit-error rate (BER) and frame-error rate (FER), the software simulator counted errors only among the systematic bits, whereas the hardware counted errors among all bits. Further, the software simulator employed 50 decoder iterations whereas the hardware simulator employed 200. For the rate-1/2 code, we observe that the hardware curve reaches a near-zero slope at a BER near 10−9 . We attribute this to trapping sets [19] rather than a small dmin . Still, we note that the code’s floor is very low for its moderate length. For example, the level of its floor is the same as that of the (10000, 5000) code in [17]. For the rate-0.8 code, there appears to be no error floor down to an FER of about 10−9 so that this code has a large dmin as well as no structural problems. We note that only one codeword error occurred in approximately 730 million hardware-simulated codewords at an Eb /N0 of 3.67 dB. ¤ As seen in the above example, Algorithm 1 can lead to S-IRA codes with very low floors. However, as exposed in Example 2 below, achieving low floors in short, low-rate codes requires more effort. Floors are affected by minimum distance, girth, the approximate cycle extrinsic message degree (ACE) value to be defined below [17], and other factors. The S-IRA definition of the previous section addresses the issue of minimum distance by virtue of the row-column interleaver. In [12], the progressive edge growth (PEG) and ACE algorithms are applied to construct finite-length random and semi-random RA/IRA codes. Similarly, our second code design algorithm adds both girth and ACE conditioning, and is a variation of PEG algorithm in [18] proposed for unstructured IRA codes. We have adapted the algorithm of [18] to make it suitable for the quasi-cyclic structure in our S-IRA codes, but we have also added some minor improvements.
10
0
10
SW:BER SW:FER HW:BER HW:FER Shannon limit
−2
10
BER/FER
−4
10
−6
10
−8
10
−10
10
0
0.5
1
1.5 2 E /N (dB) b
2.5
3
3.5
0
Fig. 3: S-IRA (2044, 1024) codes. 0
10
SW:BER SW:FER HW:BER HW:FER Shannon limit
−2
10
BER/FER
−4
10
−6
10
−8
10
−10
10
2
2.5
3 E /N (dB) b
3.5
4
0
Fig. 4: S-IRA (5120, 4096) codes. Defined by [17], the ACE value, of a length 2dACE cycle is
P
i (di
− 2), where di is the degree of the ith variable node in
the cycle. The ACE algorithm proposed in [17] attempts to maximize the minimum stopping set size in the Tanner graph, by ensuring that cycles of length 2dACE or less have ACE ≥ η. Clearly a length-L cycle composed of only systematic nodes has a higher ACE value than a length-L cycle which includes also (degree-2) parity nodes. As in [18], we differentiate between these two cycle types and denote the girths for them by gsys and gall . In the computer-aided design of H, we add the missing 1 in the upper-right corner of H00 given in (12) to make it exactly quasi-cyclic; the added 1 may be removed after the design if desired. By making H00 quasi-cyclic, the algorithm complexity and speed are improved by a factor of Q compared with that for unstructured IRA codes. It is because for each Q-group of
11
variable nodes, only one variable node in the group need be condition tested during the matrix construction process. Further, in contrast with [18], rather than enforcing a single girth value across all columns, we assign an independent target girth value gall to each column group. The advantage of this modification is that if the girth value for one group of nodes cannot be attained in the design process, only this girth value need be decremented, keeping the girth targets for the other column groups unchanged. Thus, at the end of the design process gall will be a vector. This modification produces a better cycle distribution. Finally, in addition to the above girth conditioning, the ACE algorithm [17] is also applied in our condition testing.
Algorithm 2 Initialization: Initialize the parity part of H to the quasi-cyclic form. Generation of P: while P incomplete do s1: randomly select an unmasked position (l, j) in matrix P, l ∈ [0, L) and j ∈ [0, J), for which bl,j is yet undetermined; s2: randomly generate an integer x ∈ [0, Q) different from other already generated bl0 ,j ; s3: write the circulant π x in position (l, j) in matrix P; if all conditions (gsys , gall [j], and (dACE , η)) are satisfied, then continue; else if all x’s in [0, Q) have been tried, then decrease gall [j] by 2 and goto step s2; else clear current circulant π x , set x = (x + 1) mod Q and goto step s3; end (while)
Example 2: Several rate-1/2 (600, 300) S-IRA codes were constructed for comparison with the unstructured IRA (603, 301) code in [18]. We chose Q = 30, L = J = 10, gsys = 6, initial gall [j] = 10 (for 0 ≤ j < J), and different ACE parameters. The variable node degrees are chosen to be as close to those of the code in [18]. Thus, the variable node degree distributions of our code and of the one in [18], are λ(x) = 0.0005 + 0.3022x + 0.4851x3 + 0.2122x6 and λ(x) = 0.0005 + 0.2941x + 0.4709x3 + 0.2345x7 , respectively. The check node degree distribution of our code is ρ(x) = 0.0033x4 + 0.3967x5 + 0.6x6 . The results are presented in Fig. 5. The curve for the IRA code is copied from Fig. 7 in [18]. All codes, including the one from [18], are simulated with 80 iterations. We note that our best S-IRA code is almost as good as the code from [18], but has the advantage of structure. ¤
12
0
10
−2
10
BER/FER
−4
10
−6
10
−8
10
−10
10
1.5
FER:S−IRA−Alg 1 BER:S−IRA−Alg 1 FER:S−IRA−Alg 2, ACE(4,6) BER:S−IRA−Alg 2, ACE(4,6) FER:S−IRA−Alg 2, ACE(5,6) BER:S−IRA−Alg 2, ACE(5,6) FER:IRA, ref[18] BER:IRA, ref[18] 2
2.5
3 E /N [dB] b
3.5
4
0
Fig. 5: S-IRA (600, 300) codes. Example 3: In this example, we compare a rate-1/3 (3072, 1024) S-IRA code with wc = 6 to the (3072, 1024) “random permutation” RA code presented in [4]. The degree distributions for our S-IRA code are λ(x) = 0.000098+0.3998x+0.6001x5 and ρ(x) = 0.00039x3 + 0.9996x4 . The design parameters are Q = 64, L = 32, J = 16, gsys = 6, initial gall [j] = 10 (for 0 ≤ j < J), and ACE parameters (5, 8). The S-IRA code is simulated by a sum-product-algorithm (SPA) decoder and the RA code was simulated with a turbo-like decoder [4], both with 20 iterations. The performance comparison is presented in Fig. 6, where it is seen that the S-IRA code has no error floor for FER down to 10−6 and is generally far superior to the RA code. Our S-IRA code is also far superior to the rate-1/3 (2628, 876) quasi-cyclic RA code presented in [4]. ¤ 0
10
FER:S−IRA BER:S−IRA FER:RA BER:RA
−1
10
−2
10
−3
BER/FER
10
−4
10
−5
10
−6
10
−7
10
−8
10
0.5
1
1.5 E /N [dB] b
2
2.5
0
Fig. 6: S-IRA code vs. RA code with k = 1024 and rate=1/3.
Example 4: In this example, we design an S-IRA code with parameters similar to those of the rate-0.82 (4161, 3431)
13
unstructured eIRA code in Fig. 4 of [6]. Our S-IRA code has degree distributions λ(x) = 0.00007 + 0.1019x + 0.5970x2 + 0.1786x6 + 0.1225x7 and ρ(x) = 0.0013x17 + 0.3865x18 + 0.6123x19 . The degree distributions of the eIRA code in [6] are λ(x) = 0.00007 + 0.1014x + 0.5895x2 + 0.1829x6 + 0.1262x7 and ρ(x) = 0.3037x18 + 0.6963x19 . The S-IRA code has parameters: Q = 73, L = 10 , J = 47, gsys = 6, initial gall [j] = 8 (for 0 ≤ j < J), and ACE parameters (3, 6). The SPA decoder is used with 100 iterations. The performance is presented in Fig. 7. The other codes in Fig. 4 of [6] are also plotted in Fig. 7 for comparison. The results show that the S-IRA code performs as well as the eIRA code, and both outperform the other codes with similar parameters. ¤ Comparison of rate−0.82 LDPC codes (100 iter.)
−1
10
−2
10
−3
BER
10
−4
10
−5
10
−6
10
2.4
BER:MacKay BER:Finite Geometry BER:R&U Irregular BER:eIRA BER:S−IRA 2.6
2.8
3
3.2 3.4 E /N [dB] b
3.6
3.8
4
0
Fig. 7: Performance comparison of various LDPC codes with n = 4161 and rate=0.82.
V. S-IRA C ODE FAMILY D ESIGN Rate-compatible (RC) codes [20]-[22] have many applications in communication systems that undergo a range of operating SNRs, but require a single encoder/decoder structure. In a RC code family, the parity bits of higher-rate codes are embedded in those of lower-rate codes. In this section, we first consider an encoder/decoder-compatible family of codes comprising code rates 1/2, 2/3, and 4/5 (this is a family that is not RC but maintains encoder/decoder implementation advantages). We then consider a RC family with the same rates, achieved by puncturing. In contrast to [21] and [22], we are mostly concerned with the family performance in the error floor region, and focus less on the waterfall region. Specifically, our objective is that each code in the family has no error floor above FER = 10−6 . A. Algorithm I: Masking and Extending In [11], we proposed a design method that combines masking and “extension” techniques to design an S-IRA code family with fixed systematic block length k. In this method, the lower-rate codes are constructed by extending the rows of H1 of a
14
higher-rate code while masking certain circulant permutation matrices to maintain a targeted column weight. Thus, the codes in the family share some circulants, which make the decoder implementation more efficient. Although the family generated by this method is not exactly rate-compatible, the structure of each member code greatly simplifies the encoder/decoder compared with the random code family designed in [21] and [22]. In this paper, we improve the S-IRA code family design by applying Algorithm 2 of Section IV, but without ACE conditioning. We also changed the design order relative to that in [11]: from low-rate to high-rate. Specifically, after an initial low-rate design, higher-rate codes inherit circulants from lower rate codes. The performance of this family will be presented below. B. Algorithm II: Puncturing Although the easiest way to obtain a rate-compatible code family is by puncturing, it is not easy to design a good code family by simply puncturing. As explained in [18] and [22], the designer must be very careful because, when the percentage of check nodes affected by multiple punctured bits is large, decoder message-exchange becomes ineffective, leading to poor performance. It is also mentioned in [18] for IRA codes that zig-zag pattern of H2 permits a very simple puncturing pattern: the parity bits are punctured in an alternating pattern. In this subsection, we will explain how to design an S-IRA code family by puncturing. Moreover, both [18] and [22] suggest puncturing a rate-Rc mother code to obtain a rate-2Rc /(1 + Rc ) code, but we find that a larger range of rates is possible. As an illustrative design example, we choose as the mother code a rate-1/2 (2044, 1024) S-IRA code with parameters Q = 51, L = 20, J = 21, wc = 5, gsys = 6 and initial gall [j] = 16 (for 0 ≤ j < Q). We did not use ACE conditioning because our objective can be achieved without it. Because Q · J = 1071, the right-most 47 columns of the matrix P are deleted to make k = 1024. Note the highest rate we seek is 4/5, whereas 2Rc /(1 + Rc ) is only 2/3 since Rc = 1/2. Because the rate-4/5 code is the “most vulnerable” in the family, the mother code must be designed such that its own performance and that of its offspring rate-4/5 code is good. The puncturing pattern for the rate-4/5 is shown in Fig. 8(a), that is, one parity bit is transmitted out of every four. The accumulator subgraph and a simplified equivalent one are presented in Fig. 8 (b), where it is illustrated how each group of four check equations can be summed together and replaced by a single equation. Considering the block interleaver applied in (10), rows 0, 1, 2, and 3 of ΠT P are, respectively, the first rows in the first four row groups of P; rows 4, 5, 6 and 7 of ΠT P are, respectively, the first rows in the second four row groups of P; and so on. Thus, an equivalent parity-check matrix can be derived by summing every four row groups of the matrix given by (11). Because P has 20 row groups, there will be 5 row groups after summation, which can be considered to be the equivalent P matrix of rate-4/5 code. The puncturing pattern for the rate-2/3 code is such that one bit out of every two parity bits is transmitted and similar comments hold regarding the
15
equivalent P matrix. In order to make the family rate-compatible, the unpunctured bit is selected so that the parity bits in the rate-4/5 code are embedded in those of the rate-2/3 code. Systematic punctured S-IRA code u
u
block interleaver
P
puncture 0 0 0 1
+
p
D
(a) p0
. . . .
p1
. . . .
p2
p3 p3
(b)
Fig. 8: (a) A rate-4/5 punctured S-IRA code: 0=punctured, 1=transmitted. (b) Original and simplified illustrations of the parity subgraph for the rate-4/5 punctured S-IRA code.
Beside Algorithm II introduced above, we add a cycle test (CT) for the equivalent parity-check matrix of the rate 4/5 code to guarantee that it is free of length-4 cycles. As shown in the simulation-based example below, this additional CT rule improves the performance of rate 4/5 code at the higher SNR values without impairing the performance of the mother code. The example also shows that the CT test is not necessary for the rate-2/3 code.
C. Comparison and Results Fig. 9 shows the performance of the S-IRA code families designed using Algorithms I and II (with CT), with k = 1024 and rates of 1/2, 2/3 and 4/5. They are simulated using the approximate-min* decoder [23] with 50 iterations. The rate 2/3 and 4/5 codes in the Algorithm I family are slightly better than those in the Algorithm II family in the waterfall region. However, Algorithm II is much more flexible: (1) the decoder is essentially same for every member of the family and (2) any rate between 1/2 and 4/5 is easy to obtain by puncturing only, with performance no worse than that of the rate-4/5 code. To verify the improvement by using the cycle test for the equivalent parity-check matrix of rate-4/5 code, we also plot the curve for a rate-4/5 code without using the CT rule. The results show that the CT rule produces a gain of 0.15 dB at FER= 10−6 . We did not include the results of the other two rates since the performance for these two rates are the same with and without the CT rule.
16
0
S−IRA code families (k=1024, R=1/2, 2/3 and 4/5) by Algorithm I and II
10
−1
10
−2
10
−3
FER
10
−4
10
−5
10
−6
10
−7
10
FER−1/2(I) FER−2/3(I) FER−4/5(I) FER−1/2(II) FER−2/3(II) FER−4/5(II) FER−4/5(II no CT)
0.5
1
1.5
2
2.5 3 E /N [dB] b
3.5
4
4.5
5
0
Fig. 9: S-IRA code families with k = 1024 designed with Algorithm I and II. VI. ACKNOWLEDGMENTS The authors would like to thank Chris Jones of JPL for the hardware simulations of some of our codes, Gianluigi Liva of the University of Bologna for valuable discussions, and Shu Lin for teaching us about quasi-cyclic codes. R EFERENCES [1] R. G. Gallager, Low Density Parity Check Codes. Cambridge, MA: MIT Press, 1963. [2] D. MacKay and R. Neal, “Near Shannon limit performance of Low- Density Pairty-Check Codes,” Electronic Letters, 32:1645-1646, 1996. [3] D. Divsalar, H. Jin, and R. McEliece, “Coding theorems for Turbo-like codes,” in Proc. 36th Allerton Conf. on Communication, Control, and Computing, Monticello, IL, Sep., 1998. [4] R. Michael Tanner “On quasi-cyclic repeat-accumulate codes,” in Proc. 37th Allerton Conf. on Communication, Control, and Computing, Monticello, IL, Sep., 1999. [5] H. Jin, A. Khandekar, and R. McEliece, “Irregular repeat-accumulate codes,” in Proc. 2nd International Symposium on Turbo Codes, pp. 1-8, 2000. [6] M. Yang, W. E. Ryan and Y. Li, “Design of efficiently encodable moderate-length high-rate irregular LDPC codes,” IEEE Trans. on Communications, vol. 52, pp. 564-571, April 2004. [7] A. Abbasfar, D. Divsalar, and K, Yao, “Accumulate Repeat Accumulate codes,” in Proc. IEEE Int. Symp. of Information Theory, Chicago, IL, June 2004. [8] D. Divsalar, S. Dolinar, J. Thorpe, “Accumulate-repeat-accumulate-accumulate-codes,” in Proc. 60th IEEE Vehicular Technology Conference, Los Angeles, Sep. 26-29, 2004 [9] H. Jin, Personal communication, Jan. 2006. [10] M. M. Mansour, “High-performance decoders for regular and irregular repeat-accumulate codes,” in Proc. IEEE Global Telecomm. Conf., Nov. 29-Dec. 3, 2004, pp. 2583-2588. [11] Y. Zhang, William E. Ryan, and Y. Li, “Structured eIRA codes,” in Proc. 38th IEEE Asilomar Conf. on Signals, Systems, and Computers, Pacific Grove, CA, November 7-10, 2004. [12] S. Guemghar and G. Caire, “On the performance of finite length irregular repeat accumulate codes,” Proceedings of the 5th International ITG Conference on Source and Channel Coding, 14-16 Jan. 2005.
17
[13] J. Li, K. R. Narayanan, and C. N. Georghiades, “Generalized product accumulate codes: analysis and performance,” in Proc. IEEE Global Telecomm. Conf., Nov. 2001, pp. 25-29. [14] J. Xu, L. Chen, I. Djurdjevic, S. Lin and K. Abdel-Ghaffar, “Construction of regular and irregular LDPC codes: geometry decomposition and masking,” submitted to IEEE Trans. on Communications. [15] M. P. C. Fossorier, “Quasi-cyclic low-density parity-check codes from circulant permutation matrices,” IEEE Trans. Information Theory, vol. 50, pp. 1788-1793, Aug. 2004. [16] Hardware simulations performed by C. Jones of Jet Propulsion Laboratory. [17] T. Tian, C. Jones, J. Villasenor and R. D. Wesel, “Constructions of irregular LDPC codes with low error floors,” ICC 2003, Alaska, May 2003. [18] D. Libero, Sottile F. and Benedetto S., “Design of variable-rate irregular LDPC codes with low error floor,” ICC 2005, Seoul, South Korea, May 2005. [19] T. Richardson, “Error floors of LDPC codes,” in Proc. 41th Allerton Conf. on Comm., Control and Computing, October 2003. [20] J. Hagenauer, “Rate-compatible punctured convolutional codes (RCPC codes) and their applications,” IEEE Trans. Commun., vol. 36, pp. 389-400, Apr. 1998. [21] M. R. Yazdani and A. H. Banihashemi, “On construction of rate-compatible low-density parity-check codes,” IEEE Commmun. Letters, vol. 8, pp. 159-161, March 2004. [22] J. Li, and K. Narayanan, “Rate-compatible low density parity check codes for capacity-approaching ARQ scheme in packet data communications,” Int. Conf. on Comm. Internet and Info. Tech., Nov. 2002. [23] C. Jones, E. Vall´es, M. Smith, and J. Villasenor, “Approximate-MIN* constraint node updating for LDPC code decoding,” in Proc. IEEE Military Comm. Conf. (MILCOM), Oct. 2003.
Structured IRA Codes: Performance Analysis and Construction Yifei Zhang, Student Member, IEEE, and William E. Ryan, Senior Member, IEEE
Abstract In this paper, we present design techniques for structured irregular repeat-accumulate (S-IRA) codes with low error-rate floors. These S-IRA codes need not be quasi-cyclic, permitting flexibility in code dimension, length, and rate. We present a simple ensemble estimate of the level of the error-rate floor of finite-length IRA codes on the additive white Gaussian noise (AWGN) channel. This performance estimate provides guidance on the choice of IRA code column weights which yield low floors. We also present two design algorithms for S-IRA codes accompanied by software- and hardware-based performance results which demonstrate their low floors. Lastly, we present two design algorithms for multi-rate S-IRA code families implementable by a single encoder/decoder.
I. I NTRODUCTION Low-density parity-check (LDPC) codes were introduced by Gallager circa 1960 [1] and were re-discovered in 1996 by MacKay and Neal [2]. Since the work of MacKay and Neal, several LDPC code subclasses have been proposed, perhaps most notably, the accumulator-based LDPC codes. This subclass includes the repeat-accumulate (RA) codes [3], [4], the irregular repeat-accumulate (IRA) codes [5], the extended IRA (eIRA) codes [6], the accumulate-repeat-accumulate (ARA) codes [7], and the accumulate-repeat-accumulate-accumulate (ARAA) codes [8]. Accumulator-based codes offer the advantages of linearcomplexity encoding and low error-rate floors while sacrificing little in the decoding threshold [3], [5], [6], [7], [8]. We will not consider ARA or ARAA codes in this paper. An RA code is a serial concatenation of a repetition code and an accumulator through an interleaver: a length-k information block is repeated q times, scrambled by a qk × qk interleaver Π, and then encoded by a rate-1 accumulator having transfer function 1/(1 ⊕ D). When the information bits are transmitted along with the accumulator output, the result is a rate-1/(q + 1) systematic RA code; otherwise, it is a rate-1/q nonsystematic RA code. We emphasize systematic codes in this paper. The Tanner graph for the systematic RA code is given in Fig. 1(a). Note, as indicated in the figure for an RA code, each information bit This work was funded in part by NASA grants NAG5-13254 and NNG04GI06G and a gift from the INSIC EHDR program.
2
node connects to exactly q check nodes (db,i = q) and each check node connects to exactly one information bit node (dc,i = 1). Further, the number of parity bits m is given by m = qk. The systematic IRA codes generalize the systematic RA codes in that the repetition rate may differ across the k information bits. Further, according to Figure 1 in [5] and its associated discussion, m > k and the check nodes all connect to exactly a ≥ 1 information bit nodes. In the context of Fig. 1(a), this means that db,i varies with i and dc,i = a for some fixed a ≥ 1. However, as pointed out by the reviewers and H. Jin [9], the constraints m > k and dc,i = a are relaxed later in [5]. Thus, what have been called eIRA codes in [6] – db,i and dc,i are both variable and m need not be greater than k – can simply be called IRA codes and we shall do so hereafter.
Fig. 1: Structures of RA and IRA codes: (a) Tanner graph. (b) Systematic encoder.
The parity-check matrix for systematic RA and IRA codes has the form
H = [H1 H2 ],
(1)
3
where H2 is an m × m “dual-diagonal” square matrix, 1 1 H2 =
1 ..
. 1
1 1
1
.
(2)
For RA codes, H1 is a regular matrix having column weight q and row weight 1. For IRA codes, H1 has column weights {db,i } and row weights {dc,i }. The encoder of Fig. 1(b) will be useful to our ensemble analysis and is obtained by writing H1 as ΠT BT , where Π is a permutation matrix. In this paper, we extend Tanner’s quasi-cyclic RA code designs [4] to quasi-cyclic and near quasi-cyclic IRA code designs, which we simply call structured IRA (S-IRA) codes. We first present a simple ensemble estimate of the level of the errorrate floor of finite-length IRA codes on the AWGN channel. This performance estimate provides a guide to the selection of the design parameters {db,i } for IRA codes with low floors. We then present our definition of S-IRA codes, a construction algorithm, and simulation results. Lastly, we demonstrate the versatility of the S-IRA code class in that a multi-rate code family has been designed which may be implemented with a single encoder/decoder. These results also demonstrate that these codes are flexible in that designs with very specific information word lengths and code rates are possible, which is not the case for many other code classes, such as the ones in [4]. Related work may be found in [3], [4], [5], [6], [10], [11] and [12]. As indicated above, our contribution is the design of structured IRA codes with low floors for both high-rate and low-rate codes, in contrast to the works in [4] and [10] which focus on structured RA and IRA codes with low rates. Our analysis of the IRA code ensemble exploits the method of [3] which was used to prove the near-capacity capabilities of accumulator-based “turbo-like” codes, specifically, RA codes, on the AWGN channel. Our result provides insight into the design of IRA codes with low floors, in contrast to the work in [5] which deals with the design of good-threshold IRA codes using the Gaussian approximation. Furthermore, we focus on the structured IRA codes design as opposed to design of random and semi-random RA/IRA codes in [12]. The rest of this paper is organized as follows. Section II derives an estimate on the floor performance of a subset of IRA codes and presents some numerical examples for the binary-input AWGN channel. Sections III and IV introduce the S-IRA codes and two design algorithms. Section V presents two S-IRA code family design algorithms and example simulation results.
4
II. IRA C ODE E NSEMBLE P ERFORMANCE E STIMATE A. The Union Bound and Estimate Error floor estimation via the union bound is particularly valuable for signal-to-noise ratios (SNRs) that are beyond the reach of simulations. We have found it difficult to analyze truly irregular IRA codes and so we consider a subset of the full IRA ensemble. The IRA code ensemble we will analyze can be defined by parameter triple < k, m, wc >, where k is the number of information bit nodes, m is the number of check nodes, and wc is the column weight of H1 (db,i = wc ). We leave the row weight distribution {dc,i } of H1 unspecified. To simplify the analysis, we assume the wc 1’s in each column of H1 are uniformly distributed and the 1’s in any two rows have at most one position in common, i.e., there are no 4-cycles in the graph of H1 . Note that the ensemble we consider has finite length. The union bound on the bit error rate (BER) is given by
Pb
0
, Z=e
(3)
−Rc Eb /N0
where X
Dd =
w:w+h=d
w Aw,h . k
(4)
Here, Aw,h is the average input-output weight enumerator (IOWE) over the IRA code ensemble, where w and h are the Hamming weights of information block and parity block, respectively. To facilitate the analysis, we use the encoder structure in Fig. 1 (b) composed of the matrix B = HT1 Π−1 , the uniform interleaver Π, and the accumulator. Then, by analogy to serial turbo codes, the average IOWE of our IRA code ensemble can be written as
Aw,h =
X
(acc)
(1) Al,h Aw,l · ¡m¢ ,
(5)
l
l (1)
(acc)
where Aw,l is the average IOWE of the outer code corresponding to B and Al,h
is the IOWE of the inner accumulator code.
Per Fig. 1 (b), l and h are the accumulator input and output weights, respectively. The IOWE of the accumulator is given by [3]
µ (acc) Al,h
=
m−h bl/2c
¶µ
¶ h−1 . dl/2e − 1
(6)
The average IOWE of the outer code is a function of the input-output weight transfer probability (IOWTP) [13], Pw,l , as follows: (1)
Aw,l =
µ ¶ k Pw,l . w
(7)
5
The computation of the IOWTP Pw,l for all possible w and l is a difficult problem. Fortunately, from the analysis of turbolike codes, we know that the bound behavior at high SNRs is dominated by low-weight encoder inputs (i.e., small w). Hence, we consider the computation of an approximation of the union bound which exploits this fact. Suppose the input block has weight w = 1, the output block of the HT1 encoder, p1 , is precisely a row of HT1 , which means P1,l is 1 for l = wc and 0 for l 6= wc . For weight-2 inputs (w = 2), the weight of p1 will either be 2wc or 2(wc − 1) since no 4-cycles exist in HT1 . The IOWTP can be computed using
Nw,l Pw,l = P , Nw,l0 l
(8)
0
where Nw,l is the number of ways that p1 has weight l given weight-w inputs. In the case of w = 2, the two terms can be calculated by [11]
µ N2,2·wc =
and
m wc
¶µ
¶ m − wc /2 wc
µ N2,2·(wc −1) =
¶ µ ¶ m m − wc wc /2. wc wc − 1
The results in [11] show that P1,l and P2,l dominate the code ensemble performance in the error floor region. Thus, we omit all terms corresponding to w ≥ 3 in the following numerical examples.
B. Numerical Results We applied our performance estimate to the following IRA code < k, m, wc > ensembles, grouped so that codes of like parameters may be compared: 1) < 100, 100, 3 > and < 100, 100, 5 > 2) < 1000, 1000, 3 > and < 1600, 400, 3 > 3) < 100, 100, 3 > and < 1000, 1000, 3 > 4) < 3141, 730, 3 >, < 3141, 730, 4 >, and < 3141, 730, 5 >. For each ensemble in the first three groups, two codes are randomly selected (designed). In the last group, only one code is designed for each ensemble. The results of the above four comparison groups are shown in Fig. 2. The simulated bit error rates have solid curves and the error floor estimates have dashed curves. We observe the floor estimates are consistent with the simulation results for the randomly selected codes. Moreover, we notice that increasing wc (to at least 5) tends to lower the floor.
6
IRA ensembles
IRA ensembles vs
BER
−5
10
BER
−5
10
−10
10
−10
2
4 Eb/N0 [dB]
10
6
2
IRA ensembles vs
10
w =4
BER
wc=3
c
BER
10
6
IRA ensembles
−5
−5
4 Eb/N0 [dB]
wc=5
−10
10 −10
10
2
4 E /N [dB] b
6
2
3
0
4 E /N [dB] b
5
6
0
Fig. 2: Comparison of the performance of randomly selected IRA codes with ensemble estimates. III. S TRUCTURED IRA C ODES To specify the class of S-IRA codes, we first define the submatrix, P, of one version of the parity-check matrix as discussed below. The matrix P is an L × J array of Q × Q (Q can be any integer) circulant permutation submatrices, which can be represented by P=
b0,0
b0,1
...
b0,J−1
b1,0
b1,1
...
b1,J−1
.. . bL−1,0
bL−1,1
... bL−1,J−1
,
(9)
L×J
where bl,j ∈ { ∞, 0, 1, ..., Q − 1} is the exponent of the circulant permutation matrix π formed by cyclically shifting (mod Q) each row of a Q × Q identity matrix IQ to the right once. Thus, (9) is a short-hand notation in which each entry bl,j should be replaced by π bl,j . We define π ∞ to be the Q × Q zero matrix OQ which corresponds to the “masked out” entries of P [14]. These entries are recorded in a L × J binary masking matrix [14] which is chosen in accordance with a targeted degree distribution. Because of the structure of P, rows and columns occur in groups of size Q. We will refer to row group l,
7
0 ≤ l ≤ L, and column group j, 0 ≤ j ≤ J. Thus, the masking matrix determines the edge connections between row groups and column groups. It is easy to show that using matrices with circulant submatrices such as P in place of H1 will produce a poor LDPC code in the sense of minimum distance (consider weight-2 encoder inputs with adjacent ones). In view of this, to create the parity-check matrix for our S-IRA codes, instead of replacing H1 with P, we replace BT (see Fig. 1) with P so that H1 = ΠT P and
H = [ΠT P H2 ].
(10)
Our goal of explicitly adding the interleaver Π is to achieve a larger minimum codeword weight — at least for codewords corresponding to weight-2 inputs — via proper choice of the interleaver. We would simultaneously like to impart an organized structure in H, which constrains Π to be deterministic rather than (pseudo-)random. Thus, we choose Π so that row qL + l in row group q of H1 is row lQ + q in row group l of P, where 0 ≤ l < L and 0 ≤ q < Q. Observe that this choice of interleaver has the effect of spreading the 1’s in the word p1 for weight-2 inputs. The idea of spreading 1’s appears also in [4]. Note that an alternative parity-check matrix for a given S-IRA code is a row-permuted version of the one above:
H0 = Π−T H = [P ΠH2 ],
(11)
where we have used the fact that Π−T = Π. There is yet a third parity-check matrix form, first given by Tanner in the context of quasi-cyclic RA codes [4]:
H00 = [P
ΠH2 Π].
(12)
Observe that H00 describes essentially the same code as H0 , except for a re-ordering of the parity bits. We remark that, since H2 is not circulant, there is a minor difference between the form given by Tanner and that of H00 : a missing 1 in the upper-right corner of H00 . Further, S-IRA codes need not be quasi-cyclic for two reasons. First, we do not require the number of columns of H2 to be a multiple of Q. Second, we allow deletions of columns and rows of P so that m and k are not necessarily multiples of Q.
8
IV. C ODE D ESIGNS We discuss two S-IRA code design algorithms in this section and present selected performance results for each. In the first algorithm, we require the following three conditions. Condition 1: To avoid short cycles which have two nodes in H1 , we require that the bi,j ’s in one column of P be distinct. Condition 2: There will be no length-4 cycles within the systematic (H1 ) part of the code’s graph provided [15]
(bl1 ,j1 − bl1 ,j2 )
mod Q 6= (bl2 ,j1 − bl2 ,j2 ) mod Q,
(13)
where l1 , l2 ∈ [0, L), l1 6= l2 and j1 , j2 ∈ [0, J), j1 6= j2 . There will be no length-4 cycles across the H1 and H2 parts of H provided
bl,j 6= bl+1,j and (bL−1,j − b0,j ) mod Q 6= 1,
(14)
where l ∈ [0, L − 1), j ∈ [0, J). Condition 3: There will be no length-6 cycles in the code’s graph in which four nodes are systematic nodes and two are parity nodes provided (bl1 ,j1 − bl2 ,j1 ) mod Q 6= (bl1 ,j2 − bl2 ±1,j2 ) mod Q
(15)
and (bl,j1 − b0,j1 ) mod Q 6= (bl,j2 − bL−1,j2 + 1)
mod Q,
(16)
where l1 , l2 , and l2 ± 1 ∈ [0, L) are distinct, l ∈ [0, L), and j1 , j2 ∈ [0, J), j1 6= j2 . There will be no length-6 cycles in the code’s graph in which two nodes are systematic nodes and four are parity nodes provided
(bl,j − b(l+2)
mod L,j )
where l ∈ [L − 2, L) and j ∈ [0, J).
Algorithm 1 Initialization: Initialize the parity part of H to ΠH2 . Generation of P: for j = 0 to J − 1 do
mod Q 6= 1,
(17)
9
for l = 0 to L − 1 do if bl,j is not masked REP: randomly select an integer x ∈ [0, Q) different from other already generated bl0 ,j and set bl,j = x. if any condition is unsatisfied if every possible x ∈ [0, Q) has been tested, then remove Condition 3. goto REP. else write circulant π x into the parity-check matrix. end (for l) end (for j)
Example 1: In this example, we present two S-IRA codes with low floors designed via Algorithm 1. The first is a rate-1/2 (2044, 1024) S-IRA code with L = J = 16, Q = 64, and wc = 5. The second is a rate-0.8 (5120, 4096) S-IRA code with L = 8, J = 32, Q = 128 and wc = 5. These codes were simulated in software and hardware [16] using close approximations of the sum-product algorithm and the results are presented in Fig. 3 and 4. For both bit-error rate (BER) and frame-error rate (FER), the software simulator counted errors only among the systematic bits, whereas the hardware counted errors among all bits. Further, the software simulator employed 50 decoder iterations whereas the hardware simulator employed 200. For the rate-1/2 code, we observe that the hardware curve reaches a near-zero slope at a BER near 10−9 . We attribute this to trapping sets [19] rather than a small dmin . Still, we note that the code’s floor is very low for its moderate length. For example, the level of its floor is the same as that of the (10000, 5000) code in [17]. For the rate-0.8 code, there appears to be no error floor down to an FER of about 10−9 so that this code has a large dmin as well as no structural problems. We note that only one codeword error occurred in approximately 730 million hardware-simulated codewords at an Eb /N0 of 3.67 dB. ¤ As seen in the above example, Algorithm 1 can lead to S-IRA codes with very low floors. However, as exposed in Example 2 below, achieving low floors in short, low-rate codes requires more effort. Floors are affected by minimum distance, girth, the approximate cycle extrinsic message degree (ACE) value to be defined below [17], and other factors. The S-IRA definition of the previous section addresses the issue of minimum distance by virtue of the row-column interleaver. In [12], the progressive edge growth (PEG) and ACE algorithms are applied to construct finite-length random and semi-random RA/IRA codes. Similarly, our second code design algorithm adds both girth and ACE conditioning, and is a variation of PEG algorithm in [18] proposed for unstructured IRA codes. We have adapted the algorithm of [18] to make it suitable for the quasi-cyclic structure in our S-IRA codes, but we have also added some minor improvements.
10
0
10
SW:BER SW:FER HW:BER HW:FER Shannon limit
−2
10
BER/FER
−4
10
−6
10
−8
10
−10
10
0
0.5
1
1.5 2 E /N (dB) b
2.5
3
3.5
0
Fig. 3: S-IRA (2044, 1024) codes. 0
10
SW:BER SW:FER HW:BER HW:FER Shannon limit
−2
10
BER/FER
−4
10
−6
10
−8
10
−10
10
2
2.5
3 E /N (dB) b
3.5
4
0
Fig. 4: S-IRA (5120, 4096) codes. Defined by [17], the ACE value, of a length 2dACE cycle is
P
i (di
− 2), where di is the degree of the ith variable node in
the cycle. The ACE algorithm proposed in [17] attempts to maximize the minimum stopping set size in the Tanner graph, by ensuring that cycles of length 2dACE or less have ACE ≥ η. Clearly a length-L cycle composed of only systematic nodes has a higher ACE value than a length-L cycle which includes also (degree-2) parity nodes. As in [18], we differentiate between these two cycle types and denote the girths for them by gsys and gall . In the computer-aided design of H, we add the missing 1 in the upper-right corner of H00 given in (12) to make it exactly quasi-cyclic; the added 1 may be removed after the design if desired. By making H00 quasi-cyclic, the algorithm complexity and speed are improved by a factor of Q compared with that for unstructured IRA codes. It is because for each Q-group of
11
variable nodes, only one variable node in the group need be condition tested during the matrix construction process. Further, in contrast with [18], rather than enforcing a single girth value across all columns, we assign an independent target girth value gall to each column group. The advantage of this modification is that if the girth value for one group of nodes cannot be attained in the design process, only this girth value need be decremented, keeping the girth targets for the other column groups unchanged. Thus, at the end of the design process gall will be a vector. This modification produces a better cycle distribution. Finally, in addition to the above girth conditioning, the ACE algorithm [17] is also applied in our condition testing.
Algorithm 2 Initialization: Initialize the parity part of H to the quasi-cyclic form. Generation of P: while P incomplete do s1: randomly select an unmasked position (l, j) in matrix P, l ∈ [0, L) and j ∈ [0, J), for which bl,j is yet undetermined; s2: randomly generate an integer x ∈ [0, Q) different from other already generated bl0 ,j ; s3: write the circulant π x in position (l, j) in matrix P; if all conditions (gsys , gall [j], and (dACE , η)) are satisfied, then continue; else if all x’s in [0, Q) have been tried, then decrease gall [j] by 2 and goto step s2; else clear current circulant π x , set x = (x + 1) mod Q and goto step s3; end (while)
Example 2: Several rate-1/2 (600, 300) S-IRA codes were constructed for comparison with the unstructured IRA (603, 301) code in [18]. We chose Q = 30, L = J = 10, gsys = 6, initial gall [j] = 10 (for 0 ≤ j < J), and different ACE parameters. The variable node degrees are chosen to be as close to those of the code in [18]. Thus, the variable node degree distributions of our code and of the one in [18], are λ(x) = 0.0005 + 0.3022x + 0.4851x3 + 0.2122x6 and λ(x) = 0.0005 + 0.2941x + 0.4709x3 + 0.2345x7 , respectively. The check node degree distribution of our code is ρ(x) = 0.0033x4 + 0.3967x5 + 0.6x6 . The results are presented in Fig. 5. The curve for the IRA code is copied from Fig. 7 in [18]. All codes, including the one from [18], are simulated with 80 iterations. We note that our best S-IRA code is almost as good as the code from [18], but has the advantage of structure. ¤
12
0
10
−2
10
BER/FER
−4
10
−6
10
−8
10
−10
10
1.5
FER:S−IRA−Alg 1 BER:S−IRA−Alg 1 FER:S−IRA−Alg 2, ACE(4,6) BER:S−IRA−Alg 2, ACE(4,6) FER:S−IRA−Alg 2, ACE(5,6) BER:S−IRA−Alg 2, ACE(5,6) FER:IRA, ref[18] BER:IRA, ref[18] 2
2.5
3 E /N [dB] b
3.5
4
0
Fig. 5: S-IRA (600, 300) codes. Example 3: In this example, we compare a rate-1/3 (3072, 1024) S-IRA code with wc = 6 to the (3072, 1024) “random permutation” RA code presented in [4]. The degree distributions for our S-IRA code are λ(x) = 0.000098+0.3998x+0.6001x5 and ρ(x) = 0.00039x3 + 0.9996x4 . The design parameters are Q = 64, L = 32, J = 16, gsys = 6, initial gall [j] = 10 (for 0 ≤ j < J), and ACE parameters (5, 8). The S-IRA code is simulated by a sum-product-algorithm (SPA) decoder and the RA code was simulated with a turbo-like decoder [4], both with 20 iterations. The performance comparison is presented in Fig. 6, where it is seen that the S-IRA code has no error floor for FER down to 10−6 and is generally far superior to the RA code. Our S-IRA code is also far superior to the rate-1/3 (2628, 876) quasi-cyclic RA code presented in [4]. ¤ 0
10
FER:S−IRA BER:S−IRA FER:RA BER:RA
−1
10
−2
10
−3
BER/FER
10
−4
10
−5
10
−6
10
−7
10
−8
10
0.5
1
1.5 E /N [dB] b
2
2.5
0
Fig. 6: S-IRA code vs. RA code with k = 1024 and rate=1/3.
Example 4: In this example, we design an S-IRA code with parameters similar to those of the rate-0.82 (4161, 3431)
13
unstructured eIRA code in Fig. 4 of [6]. Our S-IRA code has degree distributions λ(x) = 0.00007 + 0.1019x + 0.5970x2 + 0.1786x6 + 0.1225x7 and ρ(x) = 0.0013x17 + 0.3865x18 + 0.6123x19 . The degree distributions of the eIRA code in [6] are λ(x) = 0.00007 + 0.1014x + 0.5895x2 + 0.1829x6 + 0.1262x7 and ρ(x) = 0.3037x18 + 0.6963x19 . The S-IRA code has parameters: Q = 73, L = 10 , J = 47, gsys = 6, initial gall [j] = 8 (for 0 ≤ j < J), and ACE parameters (3, 6). The SPA decoder is used with 100 iterations. The performance is presented in Fig. 7. The other codes in Fig. 4 of [6] are also plotted in Fig. 7 for comparison. The results show that the S-IRA code performs as well as the eIRA code, and both outperform the other codes with similar parameters. ¤ Comparison of rate−0.82 LDPC codes (100 iter.)
−1
10
−2
10
−3
BER
10
−4
10
−5
10
−6
10
2.4
BER:MacKay BER:Finite Geometry BER:R&U Irregular BER:eIRA BER:S−IRA 2.6
2.8
3
3.2 3.4 E /N [dB] b
3.6
3.8
4
0
Fig. 7: Performance comparison of various LDPC codes with n = 4161 and rate=0.82.
V. S-IRA C ODE FAMILY D ESIGN Rate-compatible (RC) codes [20]-[22] have many applications in communication systems that undergo a range of operating SNRs, but require a single encoder/decoder structure. In a RC code family, the parity bits of higher-rate codes are embedded in those of lower-rate codes. In this section, we first consider an encoder/decoder-compatible family of codes comprising code rates 1/2, 2/3, and 4/5 (this is a family that is not RC but maintains encoder/decoder implementation advantages). We then consider a RC family with the same rates, achieved by puncturing. In contrast to [21] and [22], we are mostly concerned with the family performance in the error floor region, and focus less on the waterfall region. Specifically, our objective is that each code in the family has no error floor above FER = 10−6 . A. Algorithm I: Masking and Extending In [11], we proposed a design method that combines masking and “extension” techniques to design an S-IRA code family with fixed systematic block length k. In this method, the lower-rate codes are constructed by extending the rows of H1 of a
14
higher-rate code while masking certain circulant permutation matrices to maintain a targeted column weight. Thus, the codes in the family share some circulants, which make the decoder implementation more efficient. Although the family generated by this method is not exactly rate-compatible, the structure of each member code greatly simplifies the encoder/decoder compared with the random code family designed in [21] and [22]. In this paper, we improve the S-IRA code family design by applying Algorithm 2 of Section IV, but without ACE conditioning. We also changed the design order relative to that in [11]: from low-rate to high-rate. Specifically, after an initial low-rate design, higher-rate codes inherit circulants from lower rate codes. The performance of this family will be presented below. B. Algorithm II: Puncturing Although the easiest way to obtain a rate-compatible code family is by puncturing, it is not easy to design a good code family by simply puncturing. As explained in [18] and [22], the designer must be very careful because, when the percentage of check nodes affected by multiple punctured bits is large, decoder message-exchange becomes ineffective, leading to poor performance. It is also mentioned in [18] for IRA codes that zig-zag pattern of H2 permits a very simple puncturing pattern: the parity bits are punctured in an alternating pattern. In this subsection, we will explain how to design an S-IRA code family by puncturing. Moreover, both [18] and [22] suggest puncturing a rate-Rc mother code to obtain a rate-2Rc /(1 + Rc ) code, but we find that a larger range of rates is possible. As an illustrative design example, we choose as the mother code a rate-1/2 (2044, 1024) S-IRA code with parameters Q = 51, L = 20, J = 21, wc = 5, gsys = 6 and initial gall [j] = 16 (for 0 ≤ j < Q). We did not use ACE conditioning because our objective can be achieved without it. Because Q · J = 1071, the right-most 47 columns of the matrix P are deleted to make k = 1024. Note the highest rate we seek is 4/5, whereas 2Rc /(1 + Rc ) is only 2/3 since Rc = 1/2. Because the rate-4/5 code is the “most vulnerable” in the family, the mother code must be designed such that its own performance and that of its offspring rate-4/5 code is good. The puncturing pattern for the rate-4/5 is shown in Fig. 8(a), that is, one parity bit is transmitted out of every four. The accumulator subgraph and a simplified equivalent one are presented in Fig. 8 (b), where it is illustrated how each group of four check equations can be summed together and replaced by a single equation. Considering the block interleaver applied in (10), rows 0, 1, 2, and 3 of ΠT P are, respectively, the first rows in the first four row groups of P; rows 4, 5, 6 and 7 of ΠT P are, respectively, the first rows in the second four row groups of P; and so on. Thus, an equivalent parity-check matrix can be derived by summing every four row groups of the matrix given by (11). Because P has 20 row groups, there will be 5 row groups after summation, which can be considered to be the equivalent P matrix of rate-4/5 code. The puncturing pattern for the rate-2/3 code is such that one bit out of every two parity bits is transmitted and similar comments hold regarding the
15
equivalent P matrix. In order to make the family rate-compatible, the unpunctured bit is selected so that the parity bits in the rate-4/5 code are embedded in those of the rate-2/3 code. Systematic punctured S-IRA code u
u
block interleaver
P
puncture 0 0 0 1
+
p
D
(a) p0
. . . .
p1
. . . .
p2
p3 p3
(b)
Fig. 8: (a) A rate-4/5 punctured S-IRA code: 0=punctured, 1=transmitted. (b) Original and simplified illustrations of the parity subgraph for the rate-4/5 punctured S-IRA code.
Beside Algorithm II introduced above, we add a cycle test (CT) for the equivalent parity-check matrix of the rate 4/5 code to guarantee that it is free of length-4 cycles. As shown in the simulation-based example below, this additional CT rule improves the performance of rate 4/5 code at the higher SNR values without impairing the performance of the mother code. The example also shows that the CT test is not necessary for the rate-2/3 code.
C. Comparison and Results Fig. 9 shows the performance of the S-IRA code families designed using Algorithms I and II (with CT), with k = 1024 and rates of 1/2, 2/3 and 4/5. They are simulated using the approximate-min* decoder [23] with 50 iterations. The rate 2/3 and 4/5 codes in the Algorithm I family are slightly better than those in the Algorithm II family in the waterfall region. However, Algorithm II is much more flexible: (1) the decoder is essentially same for every member of the family and (2) any rate between 1/2 and 4/5 is easy to obtain by puncturing only, with performance no worse than that of the rate-4/5 code. To verify the improvement by using the cycle test for the equivalent parity-check matrix of rate-4/5 code, we also plot the curve for a rate-4/5 code without using the CT rule. The results show that the CT rule produces a gain of 0.15 dB at FER= 10−6 . We did not include the results of the other two rates since the performance for these two rates are the same with and without the CT rule.
16
0
S−IRA code families (k=1024, R=1/2, 2/3 and 4/5) by Algorithm I and II
10
−1
10
−2
10
−3
FER
10
−4
10
−5
10
−6
10
−7
10
FER−1/2(I) FER−2/3(I) FER−4/5(I) FER−1/2(II) FER−2/3(II) FER−4/5(II) FER−4/5(II no CT)
0.5
1
1.5
2
2.5 3 E /N [dB] b
3.5
4
4.5
5
0
Fig. 9: S-IRA code families with k = 1024 designed with Algorithm I and II. VI. ACKNOWLEDGMENTS The authors would like to thank Chris Jones of JPL for the hardware simulations of some of our codes, Gianluigi Liva of the University of Bologna for valuable discussions, and Shu Lin for teaching us about quasi-cyclic codes. R EFERENCES [1] R. G. Gallager, Low Density Parity Check Codes. Cambridge, MA: MIT Press, 1963. [2] D. MacKay and R. Neal, “Near Shannon limit performance of Low- Density Pairty-Check Codes,” Electronic Letters, 32:1645-1646, 1996. [3] D. Divsalar, H. Jin, and R. McEliece, “Coding theorems for Turbo-like codes,” in Proc. 36th Allerton Conf. on Communication, Control, and Computing, Monticello, IL, Sep., 1998. [4] R. Michael Tanner “On quasi-cyclic repeat-accumulate codes,” in Proc. 37th Allerton Conf. on Communication, Control, and Computing, Monticello, IL, Sep., 1999. [5] H. Jin, A. Khandekar, and R. McEliece, “Irregular repeat-accumulate codes,” in Proc. 2nd International Symposium on Turbo Codes, pp. 1-8, 2000. [6] M. Yang, W. E. Ryan and Y. Li, “Design of efficiently encodable moderate-length high-rate irregular LDPC codes,” IEEE Trans. on Communications, vol. 52, pp. 564-571, April 2004. [7] A. Abbasfar, D. Divsalar, and K, Yao, “Accumulate Repeat Accumulate codes,” in Proc. IEEE Int. Symp. of Information Theory, Chicago, IL, June 2004. [8] D. Divsalar, S. Dolinar, J. Thorpe, “Accumulate-repeat-accumulate-accumulate-codes,” in Proc. 60th IEEE Vehicular Technology Conference, Los Angeles, Sep. 26-29, 2004 [9] H. Jin, Personal communication, Jan. 2006. [10] M. M. Mansour, “High-performance decoders for regular and irregular repeat-accumulate codes,” in Proc. IEEE Global Telecomm. Conf., Nov. 29-Dec. 3, 2004, pp. 2583-2588. [11] Y. Zhang, William E. Ryan, and Y. Li, “Structured eIRA codes,” in Proc. 38th IEEE Asilomar Conf. on Signals, Systems, and Computers, Pacific Grove, CA, November 7-10, 2004. [12] S. Guemghar and G. Caire, “On the performance of finite length irregular repeat accumulate codes,” Proceedings of the 5th International ITG Conference on Source and Channel Coding, 14-16 Jan. 2005.
17
[13] J. Li, K. R. Narayanan, and C. N. Georghiades, “Generalized product accumulate codes: analysis and performance,” in Proc. IEEE Global Telecomm. Conf., Nov. 2001, pp. 25-29. [14] J. Xu, L. Chen, I. Djurdjevic, S. Lin and K. Abdel-Ghaffar, “Construction of regular and irregular LDPC codes: geometry decomposition and masking,” submitted to IEEE Trans. on Communications. [15] M. P. C. Fossorier, “Quasi-cyclic low-density parity-check codes from circulant permutation matrices,” IEEE Trans. Information Theory, vol. 50, pp. 1788-1793, Aug. 2004. [16] Hardware simulations performed by C. Jones of Jet Propulsion Laboratory. [17] T. Tian, C. Jones, J. Villasenor and R. D. Wesel, “Constructions of irregular LDPC codes with low error floors,” ICC 2003, Alaska, May 2003. [18] D. Libero, Sottile F. and Benedetto S., “Design of variable-rate irregular LDPC codes with low error floor,” ICC 2005, Seoul, South Korea, May 2005. [19] T. Richardson, “Error floors of LDPC codes,” in Proc. 41th Allerton Conf. on Comm., Control and Computing, October 2003. [20] J. Hagenauer, “Rate-compatible punctured convolutional codes (RCPC codes) and their applications,” IEEE Trans. Commun., vol. 36, pp. 389-400, Apr. 1998. [21] M. R. Yazdani and A. H. Banihashemi, “On construction of rate-compatible low-density parity-check codes,” IEEE Commmun. Letters, vol. 8, pp. 159-161, March 2004. [22] J. Li, and K. Narayanan, “Rate-compatible low density parity check codes for capacity-approaching ARQ scheme in packet data communications,” Int. Conf. on Comm. Internet and Info. Tech., Nov. 2002. [23] C. Jones, E. Vall´es, M. Smith, and J. Villasenor, “Approximate-MIN* constraint node updating for LDPC code decoding,” in Proc. IEEE Military Comm. Conf. (MILCOM), Oct. 2003.
