Computing the Minimum Distance of Nonbinary ... - Semantic Scholar
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. XX, NO. X, FEBRUARY 2012
1
Computing the Minimum Distance of Nonbinary LDPC Codes Lei Liu, Jie Huang, Wuyang Zhou, and Shengli Zhou, Member, IEEE
Abstract—Finding the minimum distance of low-densityparity-check (LDPC) codes is an NP-hard problem. Different from all existing works that focus on binary LDPC codes, we in this paper aim to compute the minimum distance of nonbinary LDPC codes, motivated by the fact that operating in a large Galois field provides one important degree of freedom to achieve both good waterfall and error-floor performance. Our method is based on the existing nearest nonzero codeword search (NNCS) method, but several modifications are incorporated for nonbinary LDPC codes, including the modified error impulse pattern, the dithering method, and the nonbinary decoder. Numerical results on the estimated minimum distances show that a code’s minimum distance can be increased by careful selection of nonzero elements of the parity check matrix, or by increasing the mean column weight, or by increasing the size of the Galois field. These results support observations that have been made based on simulated performance in the literature. Finally, we provide an upper bound on the minimum distance for nonbinary quasi-cyclic LDPC codes.
I. I NTRODUCTION Binary low-density parity-check (LDPC) codes are excellent error-correcting codes [1]. The extension of LDPC codes to nonbinary Galois Field GF(q) was first investigated empirically by Davey and MacKay over the binary-input AWGN channel [2]. It was shown empirically that nonbinary LDPC codes can have better performance than binary irregular LDPC codes [2]. Operating on the binary field, LDPC cycle codes, whose parity check matrices have a fixed column weight of 2, have performance levelling off at very high block error rates (BLER). However, LDPC cycle codes over GF(q) can achieve near-Shannon-limit performance as q increases [3]. This is due to the fact that moving to a higher order field, say q ≥ 256, the code’s minimum Hamming distance can dramatically increase and the code’s Hamming distance spectrum can be shaped Manuscript received Apr 6, 2011, revised Nov 8, 2011, accepted Feb 12, 2012. The editor coordinating the review of this manuscript and approving it for publication was Dr. Faramarz Fekri. The work of L. Liu and W. Zhou is supported by NSFC (61172088), National programs for High Technology Research and Development (SS2012AA011702), the National Major Special Projects in Science and Technology of China under grant 2010ZX03003-001, 2010ZX03005-003, 2011ZX03003-003-04, and National key technology R&D program under Grant 2008BAH30B12. The work of J. Huang and S. Zhou is supported by the US Office of Naval Research grants N00014-07-1-0805 (YIP) and N00014-09-1-0704 (PECASE). L. Liu and W. Zhou are with the Wireless Information Network Lab, Department of Electrical Engineering and Information Science, University of Science and Technology of China, Hefei, China, 230027; (email: [email protected]; [email protected]). J. Huang and S. Zhou are with Department of Electrical and Computer Engineering, University of Connecticut, USA, 06269 (email: [email protected]; [email protected]). Digital Object Identifier 00.0000/TCOMM.2012.000000
to approach the optimal one [3]. Cycle codes over small to moderate Galois fields, (e.g., 4 ≤ q ≤ 64) suffer from performance loss due to a “tail” in the low weight regime of the distance spectrum [3]. In order to remedy this, we have proposed in [4] a code construction method that replaces an appropriate portion of columns in the parity check matrix of a cycle code by columns having weight t, where t = 3 or 4, to shape the code’s distance spectrum, i.e., increasing the code’s minimum distance and decreasing the multiplicities of low weight codewords. In other words, the proposed codes have mixed column weights of 2 and t in their parity check matrices. The proposed near-regular profile has been shown to be very effective to address the error-floor problem [4]. A. Finding the Minimum Distance of LDPC Codes The problem of finding the minimum distance of a linear code (including LDPC codes) was proved to be an NP-hard problem by Vardy [5]; in other words, there exists no known polynomial deterministic algorithm to compute the minimum distance of a particular, nontrivial LDPC code. An approximate randomized algorithm, called the nearest nonzero codeword search (NNCS) was proposed by Hu et al. [6] to tackle this problem for binary LDPC codes. The principle is to search codewords locally around the all-zero codeword perturbed by a minimum level of noise, anticipating that the resultant nearest nonzero codewords will most likely contain the minimum-Hamming-weight codeword. A branchand-cut algorithm was proposed to find the minimum distance of binary LDPC codes by studying two integer programming models [7]. For nonbinary LDPC codes, the upper bound of the minimum distance was derived by analyzing their binary images [8]. Instead of the search method, the minimum distance of binary quasi-cyclic (QC) LDPC codes can be also estimated based on the parity check matrix directly [9]. A necessary condition to reach an upper bound on the minimum distance for (3, dr )-regular QC LDPC codes is provided in [10], where dr is the row weight of the parity check matrix. B. Our Goal and Contributions Existing literature mostly focused on the computation of the minimum distance of binary codes. Instead, in this paper, we aim to find the minimum distance for nonbinary LDPC codes and quantify the benefits of nonbinary coding from the minimum distance perspective. The contributions of this paper are as follows.
c 2012 IEEE 0000–0000/00$00.00
2
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. XX, NO. X, FEBRUARY 2012
We generalize the NNCS method [6] to find minimum distance for nonbinary LDPC codes. We incorporate a local search based on the code’s structure to reduce the search space and embed a dither method to improve the search efficiency, necessary for nonbinary LDPC codes. This method searches the minimum distance of nonbinary codes directly, which achieves more accurate results than the method using the binary image of the code [8]. • Minimum distances for existing nonbinary LDPC codes are found and compared, revealing that the minimum distance can be increased by careful selection of nonzero elements, by increasing the mean column weight, or by increasing the size of the Galois field, of the codes’ parity check matrices. These observations validate the benefits of nonbinary coding. • An upper bound of the minimum distance for nonbinary QC-LDPC codes is presented, which may serve as a new design guideline in addition to girth. Unlike the result in [10], the bound is applicable to QC-LDPC codes with all-zero sub-matrices in the parity check matrix. The paper is organized as follows. Section II presents the improved NNCS approach for nonbinary LDPC codes. Section III collects numerical results. Section IV presents an upper bound on the minimum distance for nonbinary QC-LDPC codes, and conclusions are drawn in Section V. •
II. T HE I MPROVED NNCS M ETHOD FOR N ONBINARY LDPC C ODES The idea behind the nearest nonzero codeword search (NNCS) method [6] is to perturb the transmit vector corresponding to the all-zero codeword with random and/or deterministic noise according to an error impulse (EI) pattern, and to use a form of list decoding to obtain nonzero codewords that are closest to the all-zero codeword in terms of Hamming distance. The hope is that the corrupted vector is decoded just barely away from the all-zero codeword if the perturbing noise is minimized. This way, there is a high probability that the Hamming weight of the decoded nonzero codeword will reveal information on the minimum distance. Two key ingredients of the NNCS method are the EI pattern and the near maximum likelihood (ML) decoding algorithm. We adopt this idea from [6] and apply it to nonbinary LDPC codes. Several modifications are introduced to better suit nonbinary codes and to reduce the search complexity, as summarized below. • Dithering. For binary LDPC codes, one can assume that the all-zero codeword is transmitted for analysis, as the strict symmetry conditions, including channel symmetry, check node symmetry, and variable node symmetry as defined in [16, Section II.E], can be satisfied. For nonbinary LDPC decoders, the check node and variable node symmetry conditions may not be satisfied. Here, we adopt the dithering method, by adding a random codeword to the all-zero codeword for analysis. Dithering is a common randomization technique in lattice quantization for source coding [13]. Note that the dithered LDPC code is also referred as the coset GF(q) LDPC code [14], [15]. Using
a dithered codeword instead of the all-zero codeword has helped us to find codewords with lower weights1 . • Modified error impulse pattern. In [6], the error impulse (EI) pattern is generated randomly. This brute-force approach may lead to very high complexity to find lowweight codewords. In [17], Cole et al. proposed a method to determine the candidate set of EI bits by utilizing the fact that the parity check matrix of an LDPC code is sparse, and the error impulses are applied on multiple bits which form a smallest yet dominant trapping set or codeword. We extend the approach of [17] as the error impulse pattern for finding the minimum distance of nonbinary LDPC codes. • Nonbinary LDPC decoding. The decoding is a combination of the FFT-QSPA algorithm [18] and the orderstatistics-decoding [19]. In the list decoder, we permute the parity check matrix by comparing the information entropies instead of log likelihood ratio in the binary case. The codeword is obtained by solving linear equations in order to reduce the search time. Next, we describe the error impulse pattern and the nonbinary LDPC decoding. A. Modified Error Impulse Patterns Suppose that the nonbinary LDPC code has an m× n parity check matrix H. We transverse each of the n variable nodes in the code’s Tanner graph. For each variable node Vi , the error impulse patterns are generate as described follows. Suppose that variable node Vi is connected to di check nodes. Choose Vi as the root of a tree. Expand the tree to the first tier of variable nodes as shown in Fig. 1, which can be denoted as Vi1 , . . . , ViP , where P is number of variable nodes on the first tier. Different from the binary case, each edge has a nonbinary weight over GF(q) according to the nonzero element in the corresponding position of the H matrix. Denote the error impulse pattern as a vector e = [e1 , e2 , · · · , en ]. Extending the approach [17], nonzero elements in e shall be located on position i and some of the locations in i1 , · · · , iP , so that all di check equations that involve Vi are satisfied. We only consider two cases during the search. Denote the error impulse pattern associated with Vi as ei and the corresponding nonzero entry as α. In the first case as shown in Fig. 1. (a), consider that there is only one other variable node in error under each check node. Denote this variable node as Vj with an error impulse pattern ej and the corresponding nonzero entry as β, then ei · α + ej · β = 0.
(1)
In the second case as shown in Fig. 1. (b), consider that there are two other variable nodes in error under each check 1 There exist nonbinary LDPC codes that are geometrically uniform, see e.g., [21] and [22], where the geometrical uniformity implies symmetric properties such as a) the distance profiles from code sequences in C to all other code sequences are all the same, and b) all Voronoi regions of code sequences in C have the same shape [20]. Even for the geometrically uniform nonbinary LDPC codes coupled with suitable modulation such as M -PSK [21], the check node symmetry and the variable node symmetry as defined in [16, Section II.E] need to be verified in order to decide whether a dithered codeword is needed or not.
LIU et al.: COMPUTING THE MINIMUM DISTANCE OF NONBINARY LDPC CODES
3
the more reliable the symbol is.) The candidate codewords are generated as described in the following. • Perform semi-Gaussian elimination on the permuted parity-check matrix from left to right, yielding an approximate upper diagonal matrix as H1 H2 A H= , (4) 0 H3 B
•
Fig. 1.
Nonbinary error impulse patterns
node. Denote these two variable nodes as Vj and Vk and their associated error impulse patterns as ej and ek respectively, which have β and γ as the nonzero entries. It is possible that error impulses are formed through ei · α + ej · β + ek · γ = 0.
(2)
Denote the array of binary image of the error impulse pattern e as b = [b1 , . . . , bnp ]. The dithered codeword is modulated by BPSK modulation as [x1 , x2 , . . . , xnp ], by the rule of 0 → −1 and 1 → 1. The received vector is generated as follows: ( xn + wn , if bn = 0 yn = (3) xn + A + wn , if bn = 1 where A is a positive integer called the error impulse and wn is the additive white Gaussian noise of small variance. B. Nonbinary Decoder Prior to the decoding, the received symbol probabilities are re-mapped according to the transmitted dithering vector such that an all-zero codeword is assumed from the decoder point of view. For decoding we present a decoder combining the FFTQSPA algorithm [18] and the order-statistic-decoding (OSD) algorithm [19]. Basically, during each iteration of the FFTQSPA, which delivers the probability mass function (pmf) information for each symbol, the OSD algorithm is used to find tentative codewords that might satisfy the parity check equations. The OSD algorithm works as follows. Re-order the columns of the H matrix from left to right according to the reliability of each symbol in an increasing order. The reliability of each symbol is measured by the entropy of the pmf message delivered from the FFT-QSPA algorithm (The least the entropy,
where the matrix H1 is upper-triangular and contains independent columns. The dependent columns encountered during the Gaussian elimination are permuted right after H1 . Assume that H1 has N1 independent columns, and H2 has N2 dependent columns. The matrices A and B thus contain n − N1 − N2 columns. The value of N2 is set to be a small number, say N2 = 2 or 3 to keep the complexity low, as will be clear later. Denote a codeword with n symbols as s = [xT1 , xT2 , xT3 ]T , where x1 , x2 and x3 consist of N1 , N2 and n − N1 − N2 symbols, respectively. The n − N1 − N2 symbols in x3 are preloaded with hard decisions based on the pmf messages from the FFT-QSPA algorithm. Due to the restriction of Hs = 0, we have H1 x1 + H2 x2 = −Ax3 ,
H3 x2 = −Bx3 (5)
Denote r as the rank of the matrix H3 (which may not be square), and hence one can find r independent columns and N2 − r dependent columns. The entries of x2 corresponding to N2 − r dependent columns can be set freely, and hence there are q (N2 −r) solutions of x2 if the solutions exist. For each valid pair (x2 , x3 ), x1 is simply found by back substitution due to the upper-triangle structure of H1 . • Note that if (x1 , x2 , x3 ) is a valid codeword, so does α(x1 , x2 , x3 ) where α is an element form GF(q). Hence, for each codeword already obtained, we get additional codewords by multiplying it with each nonzero element from GF(q). In total, there would be about q (N2 −r+1) codewords if solution to (5) exists. Choose the non-all-zeros vector ˜s with lowest symbol weight or Hamming weight as the decoded codeword, and record the lowest Hamming weight obtained so far as the estimate of the minimum distance. Note that in addition to finding the minimum distance in terms of bits, we also record the minimum distance in terms of symbols. III. N UMERICAL R ESULTS This section contains numerical results obtained by the proposed method on both binary and nonbinary LDPC codes, whose multiplicities are defined as the number of codewords with certain bit weight. In simulations the value of N2 is set to be between 6 and 10 for binary LDPC codes whereas its value is set to be 2 or 3 for nonbinary LDPC codes. The found codewords with small weights are available online [23]. Test Case 1: Comparison With Existing Results on Binary LDPC Codes. We first apply the proposed method on the three codes listed in Table I of [6], whose minimum distances have been found by the original NNCS approach as follows.
4
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. XX, NO. X, FEBRUARY 2012
TABLE I E STIMATED MINIMUM DISTANCE OF NONBINARY CYCLE CODES WITH R AMANUJAN GRAPH X 3,7
0
10
Simulated performance Union bound −2
10
random 8 12 1
Semi-Determinate 12 15 2
Determinate 12 20 535
TABLE II E STIMATED MINIMUM DISTANCE OF SOME PEG CODES BY THE IMPROVED NNCS METHOD
i.i.d. fading channel
−4
10 Block Error Rate
code name minimum symbol weight minimum bit weight multiplicity
−6
10
AWGN channel −8
10
−10
10
GF(2) mean column weight minimum distance multiplicity GF(8) mean column weight minimum symbol weight minimum bit weight multiplicity GF(16) mean column weight minimum symbol weight minimum bit weight multiplicity
•
•
•
2.5 10 2
2.6 11 1
2.7 12 1
2.8 19 1
2.9 27 2
3.0 50 1
2.5 12 17 1
2.6 14 19 1
2.7 32 44 1
2.8 46 75 1
2.9 65 100 1
3.0 76 119 1
2.5 13 24 1
2.6 24 45 1
2.7 40 77 1
2.8 52 98 1
2.9 62 120 1
3.0 68 133 1
The MacKay (252,504) code: minimum distance 20 and multiplicity 2. The MacKay (504,1008) code: minimum distance 34 and multiplicity 1. The Margulis (p = 11) (1320,2640) code: minimum distance 40 and multiplicity 66.
With the proposed method using the modified error impulse pattern, we find identical minimum distance and multiplicity for the first and third cases. For the second case of MacKay (504, 1008) code, we further find one codeword with weight 30 [23], hence achieving a better result than the reported minimum distance of 34 [6]; note that the weight-30 codeword was also reported in [24], [25]. This verification using binary LDPC codes confirms that the proposed method is effective. Test Case 2: Results on Nonbinary Cycle Codes. Table I shows the found minimum distance of nonbinary cycle codes with Ramanujan graph X 3,7 , whose code rates are all 0.5 and symbol lengths are all 672 over GF(16), thus the corresponding block length is 2688. The definition and BLER performance of these codes can be found in [26]. This test case is to show the effect of selection of nonzero entries. Denote the code in Table I as “random” by selecting the nonzero entries of the H matrix randomly, while the nonzero elements are selected carefully in the semi-determinate and determinate codes. It can be seen that by nonzero elements selection the minimum distance gets increased considerably, leading to lower error-floor as shown in [26]. Test Case 3: Results on Binary and Nonbinary PEG Codes. Table II shows the comparison of minimum distances estimated by the improved NNCS method among PEG codes with different mean column weights, and constructed on different Galois fields. All PEG codes have the same bit length of 1008 and rate of 1/2. The symbol lengths are 336 and 252 for GF(8) and GF(16), respectively. The nonzero entries of
−12
10
−14
10
−2
0
2
4 6 E /N (dB) b
8
10
12
0
Fig. 2. Simulated performance versus the approximate union bound for the nonbinary cycle code with semi-determinate Ramanujan graph X 3,7 , in both AWGN and i.i.d. Rayleigh fading channels.
all H matrices for the GF(8) and GF(16) cases are selected randomly. Two observations are in order. • First, increasing the code’s mean column weight can increase the code’s minimum distance. • With the same mean column weight, the minimum distance increases as the finite field expands. Therefore, large minimum distance can be achieved with codes with small column weight over large Galois fields, a clear advantage of nonbinary LDPC coding. Test Case 4: The Approximate Union Bound The performance of coded system is commonly analyzed using the union bound [1], which can be used to obtain the upper bound on the error probability. The union bound on the BLER can be generalized as: ! r n X Eb 2wR (6) Aw Q Pe (C) ≤ N0 w=dmin
where Aw is the multiplicity of codes with bit weight of w, and R is the code rate. The union bound on BPSK modulation in i.i.d. Rayleigh fading channel is P e (C) ≤
n Aw 1 X w Eb 2 w=dmin 1 + R N0
(7)
With the profile of collected low weight codewords [23], Figs. 2 and 3 show the BLER performance and the approximate union bound of nonbinary cycle codes with determinate Ramanujan graph X 3,7 and the PEG code with mean column weight of 2.5 over GF(8), respectively, in both AWGN and i.i.d. Rayleigh fading channels. For decoding, we use the FFT-QSPA algorithm given in [18]. The maximum number of iterations is set to 80, and we run until 20 block errors are found or 2,000,000,000 blocks are transmitted. At high SNR, the simulated block error rate gets closer to the approximate union bound. This confirms that the minimum distance and the multiplicity obtained by the proposed method are close, if not identical, to the true values.
LIU et al.: COMPUTING THE MINIMUM DISTANCE OF NONBINARY LDPC CODES
Reference [28] generalized the approach in [27] to a βmultiplied circulant permutation matrix which translates to L|(q − 1) and λ = (q − 1)/L. More generally, here following [29] we select λ once L is given. Specifically, we choose the smallest λ for any given L such that λL = γ(q − 1) where γ is an integer. •
0
10
Simulated performance Union bound −2
10
i.i.d. fading channel
−4
Block Error Rate
10
−6
10
A. An Upper Bound on Minimum Distance
−8
10
AWGN channel −10
10
−12
10
−14
10
5
−2
0
2
4 6 Eb/N0 (dB)
8
10
12
Fig. 3. Simulated performance versus the approximate union bound for a PEG code over GF(8), in both AWGN and i.i.d. Rayleigh fading channels.
IV. E STIMATED U PPER B OUND OF M INIMUM D ISTANCE FOR N ONBINARY QC-LDPC C ODES Reference [9] shows that the minimum distance dmin of a binary (dc , dc + 1)-regular QC-LDPC code satisfies dmin ≤ (dc + 1)!, where dc is the column weight of the parity check matrix. An upper bound on the minimum distance of (dc , dr )regular QC LDPC codes was derived in [10], where dc and dr denote the column weight and the row weight of the parity check matrix, respectively. In this section, we follow the approach in [9] to present an upper bound on the minimum distance for nonbinary QC-LDPC codes, based only on the shift offset values in the parity check matrix. Note that the proposed upper bound is code-dependent. First, let us define a nonbinary QC-LDPC code. Its parity check matrix H of symbol length nL shall be written as ··· Pa0(n−1) Pa01 Pa00 .. .. .. .. H= (8) . . . . Pa(m−1)0
Pa(m−1)1
···
Pa(m−1)(n−1)
where Paij represents the L × L nonbinary circulant permutation matrix with a nonzero element over GF(q) at position r, (r + aij ) mod L , 0 ≤ r ≤ L − 1, and zero elsewhere. The offset value aij satisfies 0 ≤ aij ≤ L − 1. And Paij represents an L × L zero matrix when aij = ∞. In each nonbinary circulant permutation matrix, let ρij denote the nonzero element in the first row of Paij which can be drawn randomly from GF(q)\0, the nonzero elements for the remaining rows of Paij are obtained by multiplying the one in the row above it by αλ , where α is a primitive element of GF(q) and λ is an integer. However, generally speaking, the nonzero element in the first row is not equal to the nonzero element in the last row multiplied by αλ , and hence the codes are not qualified as quasi-cyclic codes. Two constructions have been provided to resolve this issue. • Reference [27] introduces a so-called α-multiplied circulant permutation matrix which translates to L = q − 1 and λ = 1.
Choose any m + 1 column-blocks of (8) to construct the following matrix a ··· Pai0 jm Pai0 j1 P i0 j0 .. .. .. .. ¯ = H . (9) . . . . aim−1 jm aim−1 j1 aim−1 j0 ··· P P P
For each column-block index h, h = j0 , j1 , · · · , jm , define an operator matrix Dh to be the ‘determinant’ (over GF(q)) ¯ by deleting column-block h. of the matrix obtained from H Thus, X (10) Pai0 j0 Pai1 j1 · · · Paim−1 jm Dh = (j0 j1 ···jm /h)
where (j0 j1 · · · jm /h) denotes the permutation of all (m + 1) elements except the h-th one. L vector x, define c¯ := For Tany length T (D0 x) , (D1 x)T , · · · , (Dm x)T . It can be easily ¯ c = 0. For example, the product of the first verified that H¯ ¯ row of H and ¯c is Pai0 j0 D0 x + Pai0 j1 D1 x + · · · + Pai0 jm Dm x Pai0 j0 ··· Pai0 jm Pai0 j1 Pai0 j0 ··· Pai0 jm Pai0 j1 = x = 0, (11) .. .. .. .. . . . a . P im−1 j0 Paim−1 j1 · · · Paim−1 jm
because the top two rows are identical. The same argument ¯ can be applied to other rows of H. Therefore ¯c can be easily extended to be a valid codeword by inserting zeros between subblocks. Since we aim to find an upper bound of minimum distance as low as possible, the following steps have been taken. • We combine any m + 1 column-blocks from n column¯ blocks to form H. • We use x as a weight-one vector of length L over GF(q), where different values and positions of the nonzero element in x are used. ¯ and x, we can find codewords For each combination of H through c¯, whose weights are found as w(¯c) = w(D0 x) + · · · + w(Dm x). We choose the lowest bit weight of all nonzero codewords as the upper bound of the code’s minimum distance given this parity check matrix. B. Numerical Example First, we apply the proposed algorithm on the binary codes with parity check matrices specified in [10, eqs. (10) and (11)]. The same results as in [10] are obtained, i.e., the upper bound on the minimum distance of the code [10, eq. (10)] is 20 and
6
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. XX, NO. X, FEBRUARY 2012
TABLE III E STIMATED MINIMUM WEIGHTS OF NONBINARY QC-LDPC CODES Symbol length Girth Minimum symbol weight Minimum bit weight results estimated by Minimum symbol weight Minimum bit weight
228 492 756 1596 6 8 10 12 10 14 20 24 12 22 24 24 the improved NNCS method 10 10 10 10 11 15 16 16
2496 12 24 24 10 16
the upper bound on the minimum distance of the code [10, eq. (11)] is 24. Second, let us consider the ‘Design 4’ codes listed in Table I of [29] with symbol length of 228, 492, 756, 1596 and 2496 over GF(8). Table III shows the estimated minimum weights of the ‘Design 4’ codes listed in Table I of [29], of all codes the mean column weight is 2.5 and the code rate is 0.5. Although the estimated upper bound of the minimum distance is larger than the result obtained by the improved NNCS method, it could be meaningful to guide the design of nonbinary QC-LDPC codes achieving large minimum distance, because compared to the NNCS method, we only need the parity check matrix instead of running the decoding process to estimate the minimum distance. V. C ONCLUSIONS In this paper, we improved the nearest nonzero codeword search (NNCS) method to directly compute the minimum distance of nonbinary LDPC codes, a topic that has not been investigated in the literature. Numerical results on the estimated minimum distances confirm the benefits of nonbinary LDPC coding, showing that a code’s minimum distance can be increased by careful selection of nonzero elements of the parity check matrix, or by increasing the mean column weight, or by increasing the size of the Galois field. An upper bound on the minimum distance was also presented for nonbinary quasi-cyclic LDPC codes. R EFERENCES [1] R. H. Morelos-Zaragoza, “The Art of Error Correcting Coding,” Boston, MA: Wiley, 2002. [2] M. C. Davey and D. MacKay, “Low-density parity-check codes over GF(q),” in IEEE Commun. Lett., vol. 2, June 1999. [3] X.-Y. Hu and E. Eleftheriou, “Binary representation of cycle Tanner-graph GF(2b ) codes,” in Proc. of ICC, June 2004. [4] J. Huang, S. Zhou, and P. Willett, “Nonbinary LDPC coding for multicarrier underwater acoustic communication,” in IEEE J. Select. Areas Commun., vol. 26, no. 9, pp. 1684–1696, Dec. 2008. [5] A. Vardy, “The intractability of computing the minimum distance of a code,” in IEEE Trans. Inf. Theory, vol. 43, Nov. 1997. [6] X. -Y. Hu , M. Fossorier and E. Eleftheriou, “On the computation of the minimum distance of low-density parity-check codes,” in Proc. of ICC, vol. 2, pp. 767-771, Jun. 2004. [7] A. Keha and T. Duman, “Minimum distance computation of LDPC codes using a branch and cut algorithm,” in IEEE Trans. Commun., vol. 58, no. 4, pp. 0090-6778, April 2010. [8] C. Poulliat, M. Fossorier, D. Declercq, “Design of Regular (2,dc)-LDPC Codes over GF(q) Using their Binary Images,” IEEE Trans. Commun., vol. 56, no. 10, pp.1626-1635, Oct. 2008. [9] D. J. C. MacKay and M. C. Davey, “Evaluation of Gallager Codes for short block length and high rate applications,” in IMA volumes in Mathematics and its Applications, vol. 123, ch. 5, pp. 113-130, 2001.
[10] M. Fossorier, “Quasi-Cyclic Low-Density Parity-Check Codes From Circulant Permutation Matrices”, in IEEE Trans. Inf. Theory, vol. 50, no. 8, pp. 1788-1793, Aug. 2004. [11] J. Hou, P. H. Siegel, L. B. Milstein, and H. D. Pfister, “Capacityapproaching bandwidth-efficient coded modulation schemes based on low-density parity-check codes”, in IEEE Trans. Inform. Theory, vol. 49, pp.2141-2155, Sep. 2003. [12] A. Ingber and M. Feder, “Parallel bit interleaved coded modulation”, in Proceedings of Annual Allerton Conference on Communications, Control, and Computing, Sep. 2010. [13] U. Erez and R. Zamir, “Achieving 1/2 log(1 + SNR) on the AWGN Channel with lattice encoding and decoding,” in IEEE Trans. Inf. Theory, vol. 50, no. 10, pp. 2293-2314, Oct. 2004. [14] A. Bennatan, D. Burshtein, “EXIT charts for non-binary LDPC codes over arbitrary discrete-memoryless channels”, in ISIT 2005, pp. 37-41, Sep. 2005. [15] A. Bennatan, D. Burshtein, “Design and analysis of nonbinary LDPC codes for arbitrary discrete-memoryless channels”, in IEEE Trans. Inform. Theory, vol. 52, no. 2, pp. 549-583, Feb. 2006. [16] T. Richardson and R. Urbanke, “The capacity of low-density parity check codes under message-passing decoding”, in IEEE Trans. Inform. Theory, vol. 47, no. 2, pp. 599-618, Feb. 2001. [17] C. A. Cole, S. G. Wilson, E. K. Hall, and T. R. Giallorenzi, “Analysis and design of moderate length regular LDPC codes with low error floors,” in Proc. of. on Information Sciences and Systems, Mar. 2006. [18] H. Song and J. R. Cruz, “Reduced-complexity decoding of Q-ary LDPC codes for magnetic recording,” in IEEE Trans. Magn., vol. 39, pp. 10811087, Mar. 2003. [19] M. P. C. Fossorier, “Iterative reliability-based decoding of low-density parity-check codes,” in IEEE J. Select. Areas Commun., vol. 19, pp. 908-917, May. 2001. [20] G. D. Forney, “Geometrically uniform codes,” in IEEE Trans. Inf. Theory, vol. 37, no. 5, pp. 1241C1260, Sep. 1991. [21] D. Sridhara, T. E. Fuja, “LDPC Codes Over Rings for PSK Modulation ,” in IEEE Trans. Inf. Theory, vol. 51, no. 9, pp. 3209-3220 , Sep. 2005. [22] E. Mo and M.A. Armand, “Structured LDPC codes over integer residue rings,” in EURASIP Journal on Wireless Communications and Networking: Special Issue on Advances in Error Control Coding Techniques, 2008. [23] Collected nonbinary LDPC codewords with small weights, http://mail.ustc.edu.cn/˜liul/ [24] M. Hirotomo, M. Mohri, M. Morii, “New Probabilistic Algorithm to Enhance the Reliability of Computed Weight Distribution of LDPC Codes”, in Proc. of ISITA2006, Seoul, Korea, 2006. [25] D. Declercq and M. Fossorier, “Improved Impulse Method to Evaluate the Low Weight Profile of Sparse Binary Linear Codes”, Proc. ISIT, Seoul, Korea, 2008. [26] J. Huang, S. Zhou, J. -K. Zhu, and P. Willett, “Group-theoretic analysis of Cayley-graph-based cycle GF(2p ) codes,” in IEEE Trans. Commun., vol. 57, no. 7, pp. 1560-1565, Jun. 2009. [27] S. Lin, S. Song, L. Lan, L. Zeng, and Y.-Y. Tai, “Constructions of nonbinary quasi-cyclic LDPC codes: A finite field approach,” in Proc. of Inform. Theory and Applications (ITA) Workshop 2006, UCSD, 2006. [28] R. -H. Peng and R. -R. Chen, “Design of nonbinary quasi-cyclic LDPC cycle codes,” in Proc. of ITW, Sept. 2-6 2007. [29] J. Huang, L. Liu, W. Zhou and S. Zhou, “Large-Girth nonbinary QCLDPC codes of various lengths,” in IEEE Trans. Commun., vol. 58, no. 12, pp. 3436-3447, Dec. 2010.
LIU et al.: COMPUTING THE MINIMUM DISTANCE OF NONBINARY LDPC CODES
Lei Liu received the B.S. degree in 2007 from the University of Science and Technology of China (USTC), Hefei, in communication engineering. He is currently working toward the Ph.D. degree with the Department of Electronic Engineering and Information Science, USTC. His research interest is in the areas of error control coding and information theory.
Jie Huang was born in Jiangling, Hubei, P. R. China on January 20, 1981. He received the B.S. degree in 2001 and the Ph. D. degree in 2006, from the University of Science and Technology of China (USTC), Hefei, both in electrical engineering and information science. He was a post-doctoral researcher from July 2007 to June 2009, working with the Department of Electrical and Computer Engineering (ECE) at the University of Connecticut (UCONN), Storrs. Now he is a research assistant professor with the ECE Department at UCONN. His general research interests lie in the areas of communications and signal processing, specifically error control coding theory and coded modulation system design. His recent focus is on signal processing, channel coding and network coding for underwater acoustic communications and underwater sensor networks. Mr. Huang has served as a reviewer for the IEEE Transactions on Communications, the IEEE Transactions on Information Theory, the IEEE Transactions on Signal Processing, and the IEEE Journal on Selected Areas in Communications.
Wuyang Zhou received B.Eng. and M.Eng. degrees from Xidian University, Xi’an, China, in 1993 and 1996, respectively, and the Ph.D. degree from the University of Science and Technology of China (USTC), Hefei, in 2000. He is currently a professor in the Department of Electronic Engineering and Information Science, USTC. His research interests include error control coding, wireless resource allocation, and wireless networking.
Shengli Zhou (M03) received the B.S. degree in 1995 and the M.Sc. degree in 1998, from the University of Science and Technology of China (USTC), Hefei, both in electrical engineering and information science. He received his Ph.D. degree in electrical engineering from the University of Minnesota (UMN), Minneapolis, in 2002. He was an assistant professor with the Department of Electrical and Computer Engineering at the University of Connecticut (UCONN), Storrs, 2003-2009, and now is an associate professor. He holds a United Technologies Corporation (UTC) Professorship in Engineering Innovation, 2008-2011. His general research interests lie in the areas of wireless communications and signal processing. His recent focus is on underwater acoustic communications and networking. He has served as an associate editor for IEEE Transactions on Wireless Communications from Feb. 2005 to Jan. 2007, and is now an associate editor for IEEE Transactions on Signal Processing. He received the 2007 ONR Young Investigator award and the 2007 Presidential Early Career Award for Scientists and Engineers (PECASE).
7
1
Computing the Minimum Distance of Nonbinary LDPC Codes Lei Liu, Jie Huang, Wuyang Zhou, and Shengli Zhou, Member, IEEE
Abstract—Finding the minimum distance of low-densityparity-check (LDPC) codes is an NP-hard problem. Different from all existing works that focus on binary LDPC codes, we in this paper aim to compute the minimum distance of nonbinary LDPC codes, motivated by the fact that operating in a large Galois field provides one important degree of freedom to achieve both good waterfall and error-floor performance. Our method is based on the existing nearest nonzero codeword search (NNCS) method, but several modifications are incorporated for nonbinary LDPC codes, including the modified error impulse pattern, the dithering method, and the nonbinary decoder. Numerical results on the estimated minimum distances show that a code’s minimum distance can be increased by careful selection of nonzero elements of the parity check matrix, or by increasing the mean column weight, or by increasing the size of the Galois field. These results support observations that have been made based on simulated performance in the literature. Finally, we provide an upper bound on the minimum distance for nonbinary quasi-cyclic LDPC codes.
I. I NTRODUCTION Binary low-density parity-check (LDPC) codes are excellent error-correcting codes [1]. The extension of LDPC codes to nonbinary Galois Field GF(q) was first investigated empirically by Davey and MacKay over the binary-input AWGN channel [2]. It was shown empirically that nonbinary LDPC codes can have better performance than binary irregular LDPC codes [2]. Operating on the binary field, LDPC cycle codes, whose parity check matrices have a fixed column weight of 2, have performance levelling off at very high block error rates (BLER). However, LDPC cycle codes over GF(q) can achieve near-Shannon-limit performance as q increases [3]. This is due to the fact that moving to a higher order field, say q ≥ 256, the code’s minimum Hamming distance can dramatically increase and the code’s Hamming distance spectrum can be shaped Manuscript received Apr 6, 2011, revised Nov 8, 2011, accepted Feb 12, 2012. The editor coordinating the review of this manuscript and approving it for publication was Dr. Faramarz Fekri. The work of L. Liu and W. Zhou is supported by NSFC (61172088), National programs for High Technology Research and Development (SS2012AA011702), the National Major Special Projects in Science and Technology of China under grant 2010ZX03003-001, 2010ZX03005-003, 2011ZX03003-003-04, and National key technology R&D program under Grant 2008BAH30B12. The work of J. Huang and S. Zhou is supported by the US Office of Naval Research grants N00014-07-1-0805 (YIP) and N00014-09-1-0704 (PECASE). L. Liu and W. Zhou are with the Wireless Information Network Lab, Department of Electrical Engineering and Information Science, University of Science and Technology of China, Hefei, China, 230027; (email: [email protected]; [email protected]). J. Huang and S. Zhou are with Department of Electrical and Computer Engineering, University of Connecticut, USA, 06269 (email: [email protected]; [email protected]). Digital Object Identifier 00.0000/TCOMM.2012.000000
to approach the optimal one [3]. Cycle codes over small to moderate Galois fields, (e.g., 4 ≤ q ≤ 64) suffer from performance loss due to a “tail” in the low weight regime of the distance spectrum [3]. In order to remedy this, we have proposed in [4] a code construction method that replaces an appropriate portion of columns in the parity check matrix of a cycle code by columns having weight t, where t = 3 or 4, to shape the code’s distance spectrum, i.e., increasing the code’s minimum distance and decreasing the multiplicities of low weight codewords. In other words, the proposed codes have mixed column weights of 2 and t in their parity check matrices. The proposed near-regular profile has been shown to be very effective to address the error-floor problem [4]. A. Finding the Minimum Distance of LDPC Codes The problem of finding the minimum distance of a linear code (including LDPC codes) was proved to be an NP-hard problem by Vardy [5]; in other words, there exists no known polynomial deterministic algorithm to compute the minimum distance of a particular, nontrivial LDPC code. An approximate randomized algorithm, called the nearest nonzero codeword search (NNCS) was proposed by Hu et al. [6] to tackle this problem for binary LDPC codes. The principle is to search codewords locally around the all-zero codeword perturbed by a minimum level of noise, anticipating that the resultant nearest nonzero codewords will most likely contain the minimum-Hamming-weight codeword. A branchand-cut algorithm was proposed to find the minimum distance of binary LDPC codes by studying two integer programming models [7]. For nonbinary LDPC codes, the upper bound of the minimum distance was derived by analyzing their binary images [8]. Instead of the search method, the minimum distance of binary quasi-cyclic (QC) LDPC codes can be also estimated based on the parity check matrix directly [9]. A necessary condition to reach an upper bound on the minimum distance for (3, dr )-regular QC LDPC codes is provided in [10], where dr is the row weight of the parity check matrix. B. Our Goal and Contributions Existing literature mostly focused on the computation of the minimum distance of binary codes. Instead, in this paper, we aim to find the minimum distance for nonbinary LDPC codes and quantify the benefits of nonbinary coding from the minimum distance perspective. The contributions of this paper are as follows.
c 2012 IEEE 0000–0000/00$00.00
2
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. XX, NO. X, FEBRUARY 2012
We generalize the NNCS method [6] to find minimum distance for nonbinary LDPC codes. We incorporate a local search based on the code’s structure to reduce the search space and embed a dither method to improve the search efficiency, necessary for nonbinary LDPC codes. This method searches the minimum distance of nonbinary codes directly, which achieves more accurate results than the method using the binary image of the code [8]. • Minimum distances for existing nonbinary LDPC codes are found and compared, revealing that the minimum distance can be increased by careful selection of nonzero elements, by increasing the mean column weight, or by increasing the size of the Galois field, of the codes’ parity check matrices. These observations validate the benefits of nonbinary coding. • An upper bound of the minimum distance for nonbinary QC-LDPC codes is presented, which may serve as a new design guideline in addition to girth. Unlike the result in [10], the bound is applicable to QC-LDPC codes with all-zero sub-matrices in the parity check matrix. The paper is organized as follows. Section II presents the improved NNCS approach for nonbinary LDPC codes. Section III collects numerical results. Section IV presents an upper bound on the minimum distance for nonbinary QC-LDPC codes, and conclusions are drawn in Section V. •
II. T HE I MPROVED NNCS M ETHOD FOR N ONBINARY LDPC C ODES The idea behind the nearest nonzero codeword search (NNCS) method [6] is to perturb the transmit vector corresponding to the all-zero codeword with random and/or deterministic noise according to an error impulse (EI) pattern, and to use a form of list decoding to obtain nonzero codewords that are closest to the all-zero codeword in terms of Hamming distance. The hope is that the corrupted vector is decoded just barely away from the all-zero codeword if the perturbing noise is minimized. This way, there is a high probability that the Hamming weight of the decoded nonzero codeword will reveal information on the minimum distance. Two key ingredients of the NNCS method are the EI pattern and the near maximum likelihood (ML) decoding algorithm. We adopt this idea from [6] and apply it to nonbinary LDPC codes. Several modifications are introduced to better suit nonbinary codes and to reduce the search complexity, as summarized below. • Dithering. For binary LDPC codes, one can assume that the all-zero codeword is transmitted for analysis, as the strict symmetry conditions, including channel symmetry, check node symmetry, and variable node symmetry as defined in [16, Section II.E], can be satisfied. For nonbinary LDPC decoders, the check node and variable node symmetry conditions may not be satisfied. Here, we adopt the dithering method, by adding a random codeword to the all-zero codeword for analysis. Dithering is a common randomization technique in lattice quantization for source coding [13]. Note that the dithered LDPC code is also referred as the coset GF(q) LDPC code [14], [15]. Using
a dithered codeword instead of the all-zero codeword has helped us to find codewords with lower weights1 . • Modified error impulse pattern. In [6], the error impulse (EI) pattern is generated randomly. This brute-force approach may lead to very high complexity to find lowweight codewords. In [17], Cole et al. proposed a method to determine the candidate set of EI bits by utilizing the fact that the parity check matrix of an LDPC code is sparse, and the error impulses are applied on multiple bits which form a smallest yet dominant trapping set or codeword. We extend the approach of [17] as the error impulse pattern for finding the minimum distance of nonbinary LDPC codes. • Nonbinary LDPC decoding. The decoding is a combination of the FFT-QSPA algorithm [18] and the orderstatistics-decoding [19]. In the list decoder, we permute the parity check matrix by comparing the information entropies instead of log likelihood ratio in the binary case. The codeword is obtained by solving linear equations in order to reduce the search time. Next, we describe the error impulse pattern and the nonbinary LDPC decoding. A. Modified Error Impulse Patterns Suppose that the nonbinary LDPC code has an m× n parity check matrix H. We transverse each of the n variable nodes in the code’s Tanner graph. For each variable node Vi , the error impulse patterns are generate as described follows. Suppose that variable node Vi is connected to di check nodes. Choose Vi as the root of a tree. Expand the tree to the first tier of variable nodes as shown in Fig. 1, which can be denoted as Vi1 , . . . , ViP , where P is number of variable nodes on the first tier. Different from the binary case, each edge has a nonbinary weight over GF(q) according to the nonzero element in the corresponding position of the H matrix. Denote the error impulse pattern as a vector e = [e1 , e2 , · · · , en ]. Extending the approach [17], nonzero elements in e shall be located on position i and some of the locations in i1 , · · · , iP , so that all di check equations that involve Vi are satisfied. We only consider two cases during the search. Denote the error impulse pattern associated with Vi as ei and the corresponding nonzero entry as α. In the first case as shown in Fig. 1. (a), consider that there is only one other variable node in error under each check node. Denote this variable node as Vj with an error impulse pattern ej and the corresponding nonzero entry as β, then ei · α + ej · β = 0.
(1)
In the second case as shown in Fig. 1. (b), consider that there are two other variable nodes in error under each check 1 There exist nonbinary LDPC codes that are geometrically uniform, see e.g., [21] and [22], where the geometrical uniformity implies symmetric properties such as a) the distance profiles from code sequences in C to all other code sequences are all the same, and b) all Voronoi regions of code sequences in C have the same shape [20]. Even for the geometrically uniform nonbinary LDPC codes coupled with suitable modulation such as M -PSK [21], the check node symmetry and the variable node symmetry as defined in [16, Section II.E] need to be verified in order to decide whether a dithered codeword is needed or not.
LIU et al.: COMPUTING THE MINIMUM DISTANCE OF NONBINARY LDPC CODES
3
the more reliable the symbol is.) The candidate codewords are generated as described in the following. • Perform semi-Gaussian elimination on the permuted parity-check matrix from left to right, yielding an approximate upper diagonal matrix as H1 H2 A H= , (4) 0 H3 B
•
Fig. 1.
Nonbinary error impulse patterns
node. Denote these two variable nodes as Vj and Vk and their associated error impulse patterns as ej and ek respectively, which have β and γ as the nonzero entries. It is possible that error impulses are formed through ei · α + ej · β + ek · γ = 0.
(2)
Denote the array of binary image of the error impulse pattern e as b = [b1 , . . . , bnp ]. The dithered codeword is modulated by BPSK modulation as [x1 , x2 , . . . , xnp ], by the rule of 0 → −1 and 1 → 1. The received vector is generated as follows: ( xn + wn , if bn = 0 yn = (3) xn + A + wn , if bn = 1 where A is a positive integer called the error impulse and wn is the additive white Gaussian noise of small variance. B. Nonbinary Decoder Prior to the decoding, the received symbol probabilities are re-mapped according to the transmitted dithering vector such that an all-zero codeword is assumed from the decoder point of view. For decoding we present a decoder combining the FFTQSPA algorithm [18] and the order-statistic-decoding (OSD) algorithm [19]. Basically, during each iteration of the FFTQSPA, which delivers the probability mass function (pmf) information for each symbol, the OSD algorithm is used to find tentative codewords that might satisfy the parity check equations. The OSD algorithm works as follows. Re-order the columns of the H matrix from left to right according to the reliability of each symbol in an increasing order. The reliability of each symbol is measured by the entropy of the pmf message delivered from the FFT-QSPA algorithm (The least the entropy,
where the matrix H1 is upper-triangular and contains independent columns. The dependent columns encountered during the Gaussian elimination are permuted right after H1 . Assume that H1 has N1 independent columns, and H2 has N2 dependent columns. The matrices A and B thus contain n − N1 − N2 columns. The value of N2 is set to be a small number, say N2 = 2 or 3 to keep the complexity low, as will be clear later. Denote a codeword with n symbols as s = [xT1 , xT2 , xT3 ]T , where x1 , x2 and x3 consist of N1 , N2 and n − N1 − N2 symbols, respectively. The n − N1 − N2 symbols in x3 are preloaded with hard decisions based on the pmf messages from the FFT-QSPA algorithm. Due to the restriction of Hs = 0, we have H1 x1 + H2 x2 = −Ax3 ,
H3 x2 = −Bx3 (5)
Denote r as the rank of the matrix H3 (which may not be square), and hence one can find r independent columns and N2 − r dependent columns. The entries of x2 corresponding to N2 − r dependent columns can be set freely, and hence there are q (N2 −r) solutions of x2 if the solutions exist. For each valid pair (x2 , x3 ), x1 is simply found by back substitution due to the upper-triangle structure of H1 . • Note that if (x1 , x2 , x3 ) is a valid codeword, so does α(x1 , x2 , x3 ) where α is an element form GF(q). Hence, for each codeword already obtained, we get additional codewords by multiplying it with each nonzero element from GF(q). In total, there would be about q (N2 −r+1) codewords if solution to (5) exists. Choose the non-all-zeros vector ˜s with lowest symbol weight or Hamming weight as the decoded codeword, and record the lowest Hamming weight obtained so far as the estimate of the minimum distance. Note that in addition to finding the minimum distance in terms of bits, we also record the minimum distance in terms of symbols. III. N UMERICAL R ESULTS This section contains numerical results obtained by the proposed method on both binary and nonbinary LDPC codes, whose multiplicities are defined as the number of codewords with certain bit weight. In simulations the value of N2 is set to be between 6 and 10 for binary LDPC codes whereas its value is set to be 2 or 3 for nonbinary LDPC codes. The found codewords with small weights are available online [23]. Test Case 1: Comparison With Existing Results on Binary LDPC Codes. We first apply the proposed method on the three codes listed in Table I of [6], whose minimum distances have been found by the original NNCS approach as follows.
4
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. XX, NO. X, FEBRUARY 2012
TABLE I E STIMATED MINIMUM DISTANCE OF NONBINARY CYCLE CODES WITH R AMANUJAN GRAPH X 3,7
0
10
Simulated performance Union bound −2
10
random 8 12 1
Semi-Determinate 12 15 2
Determinate 12 20 535
TABLE II E STIMATED MINIMUM DISTANCE OF SOME PEG CODES BY THE IMPROVED NNCS METHOD
i.i.d. fading channel
−4
10 Block Error Rate
code name minimum symbol weight minimum bit weight multiplicity
−6
10
AWGN channel −8
10
−10
10
GF(2) mean column weight minimum distance multiplicity GF(8) mean column weight minimum symbol weight minimum bit weight multiplicity GF(16) mean column weight minimum symbol weight minimum bit weight multiplicity
•
•
•
2.5 10 2
2.6 11 1
2.7 12 1
2.8 19 1
2.9 27 2
3.0 50 1
2.5 12 17 1
2.6 14 19 1
2.7 32 44 1
2.8 46 75 1
2.9 65 100 1
3.0 76 119 1
2.5 13 24 1
2.6 24 45 1
2.7 40 77 1
2.8 52 98 1
2.9 62 120 1
3.0 68 133 1
The MacKay (252,504) code: minimum distance 20 and multiplicity 2. The MacKay (504,1008) code: minimum distance 34 and multiplicity 1. The Margulis (p = 11) (1320,2640) code: minimum distance 40 and multiplicity 66.
With the proposed method using the modified error impulse pattern, we find identical minimum distance and multiplicity for the first and third cases. For the second case of MacKay (504, 1008) code, we further find one codeword with weight 30 [23], hence achieving a better result than the reported minimum distance of 34 [6]; note that the weight-30 codeword was also reported in [24], [25]. This verification using binary LDPC codes confirms that the proposed method is effective. Test Case 2: Results on Nonbinary Cycle Codes. Table I shows the found minimum distance of nonbinary cycle codes with Ramanujan graph X 3,7 , whose code rates are all 0.5 and symbol lengths are all 672 over GF(16), thus the corresponding block length is 2688. The definition and BLER performance of these codes can be found in [26]. This test case is to show the effect of selection of nonzero entries. Denote the code in Table I as “random” by selecting the nonzero entries of the H matrix randomly, while the nonzero elements are selected carefully in the semi-determinate and determinate codes. It can be seen that by nonzero elements selection the minimum distance gets increased considerably, leading to lower error-floor as shown in [26]. Test Case 3: Results on Binary and Nonbinary PEG Codes. Table II shows the comparison of minimum distances estimated by the improved NNCS method among PEG codes with different mean column weights, and constructed on different Galois fields. All PEG codes have the same bit length of 1008 and rate of 1/2. The symbol lengths are 336 and 252 for GF(8) and GF(16), respectively. The nonzero entries of
−12
10
−14
10
−2
0
2
4 6 E /N (dB) b
8
10
12
0
Fig. 2. Simulated performance versus the approximate union bound for the nonbinary cycle code with semi-determinate Ramanujan graph X 3,7 , in both AWGN and i.i.d. Rayleigh fading channels.
all H matrices for the GF(8) and GF(16) cases are selected randomly. Two observations are in order. • First, increasing the code’s mean column weight can increase the code’s minimum distance. • With the same mean column weight, the minimum distance increases as the finite field expands. Therefore, large minimum distance can be achieved with codes with small column weight over large Galois fields, a clear advantage of nonbinary LDPC coding. Test Case 4: The Approximate Union Bound The performance of coded system is commonly analyzed using the union bound [1], which can be used to obtain the upper bound on the error probability. The union bound on the BLER can be generalized as: ! r n X Eb 2wR (6) Aw Q Pe (C) ≤ N0 w=dmin
where Aw is the multiplicity of codes with bit weight of w, and R is the code rate. The union bound on BPSK modulation in i.i.d. Rayleigh fading channel is P e (C) ≤
n Aw 1 X w Eb 2 w=dmin 1 + R N0
(7)
With the profile of collected low weight codewords [23], Figs. 2 and 3 show the BLER performance and the approximate union bound of nonbinary cycle codes with determinate Ramanujan graph X 3,7 and the PEG code with mean column weight of 2.5 over GF(8), respectively, in both AWGN and i.i.d. Rayleigh fading channels. For decoding, we use the FFT-QSPA algorithm given in [18]. The maximum number of iterations is set to 80, and we run until 20 block errors are found or 2,000,000,000 blocks are transmitted. At high SNR, the simulated block error rate gets closer to the approximate union bound. This confirms that the minimum distance and the multiplicity obtained by the proposed method are close, if not identical, to the true values.
LIU et al.: COMPUTING THE MINIMUM DISTANCE OF NONBINARY LDPC CODES
Reference [28] generalized the approach in [27] to a βmultiplied circulant permutation matrix which translates to L|(q − 1) and λ = (q − 1)/L. More generally, here following [29] we select λ once L is given. Specifically, we choose the smallest λ for any given L such that λL = γ(q − 1) where γ is an integer. •
0
10
Simulated performance Union bound −2
10
i.i.d. fading channel
−4
Block Error Rate
10
−6
10
A. An Upper Bound on Minimum Distance
−8
10
AWGN channel −10
10
−12
10
−14
10
5
−2
0
2
4 6 Eb/N0 (dB)
8
10
12
Fig. 3. Simulated performance versus the approximate union bound for a PEG code over GF(8), in both AWGN and i.i.d. Rayleigh fading channels.
IV. E STIMATED U PPER B OUND OF M INIMUM D ISTANCE FOR N ONBINARY QC-LDPC C ODES Reference [9] shows that the minimum distance dmin of a binary (dc , dc + 1)-regular QC-LDPC code satisfies dmin ≤ (dc + 1)!, where dc is the column weight of the parity check matrix. An upper bound on the minimum distance of (dc , dr )regular QC LDPC codes was derived in [10], where dc and dr denote the column weight and the row weight of the parity check matrix, respectively. In this section, we follow the approach in [9] to present an upper bound on the minimum distance for nonbinary QC-LDPC codes, based only on the shift offset values in the parity check matrix. Note that the proposed upper bound is code-dependent. First, let us define a nonbinary QC-LDPC code. Its parity check matrix H of symbol length nL shall be written as ··· Pa0(n−1) Pa01 Pa00 .. .. .. .. H= (8) . . . . Pa(m−1)0
Pa(m−1)1
···
Pa(m−1)(n−1)
where Paij represents the L × L nonbinary circulant permutation matrix with a nonzero element over GF(q) at position r, (r + aij ) mod L , 0 ≤ r ≤ L − 1, and zero elsewhere. The offset value aij satisfies 0 ≤ aij ≤ L − 1. And Paij represents an L × L zero matrix when aij = ∞. In each nonbinary circulant permutation matrix, let ρij denote the nonzero element in the first row of Paij which can be drawn randomly from GF(q)\0, the nonzero elements for the remaining rows of Paij are obtained by multiplying the one in the row above it by αλ , where α is a primitive element of GF(q) and λ is an integer. However, generally speaking, the nonzero element in the first row is not equal to the nonzero element in the last row multiplied by αλ , and hence the codes are not qualified as quasi-cyclic codes. Two constructions have been provided to resolve this issue. • Reference [27] introduces a so-called α-multiplied circulant permutation matrix which translates to L = q − 1 and λ = 1.
Choose any m + 1 column-blocks of (8) to construct the following matrix a ··· Pai0 jm Pai0 j1 P i0 j0 .. .. .. .. ¯ = H . (9) . . . . aim−1 jm aim−1 j1 aim−1 j0 ··· P P P
For each column-block index h, h = j0 , j1 , · · · , jm , define an operator matrix Dh to be the ‘determinant’ (over GF(q)) ¯ by deleting column-block h. of the matrix obtained from H Thus, X (10) Pai0 j0 Pai1 j1 · · · Paim−1 jm Dh = (j0 j1 ···jm /h)
where (j0 j1 · · · jm /h) denotes the permutation of all (m + 1) elements except the h-th one. L vector x, define c¯ := For Tany length T (D0 x) , (D1 x)T , · · · , (Dm x)T . It can be easily ¯ c = 0. For example, the product of the first verified that H¯ ¯ row of H and ¯c is Pai0 j0 D0 x + Pai0 j1 D1 x + · · · + Pai0 jm Dm x Pai0 j0 ··· Pai0 jm Pai0 j1 Pai0 j0 ··· Pai0 jm Pai0 j1 = x = 0, (11) .. .. .. .. . . . a . P im−1 j0 Paim−1 j1 · · · Paim−1 jm
because the top two rows are identical. The same argument ¯ can be applied to other rows of H. Therefore ¯c can be easily extended to be a valid codeword by inserting zeros between subblocks. Since we aim to find an upper bound of minimum distance as low as possible, the following steps have been taken. • We combine any m + 1 column-blocks from n column¯ blocks to form H. • We use x as a weight-one vector of length L over GF(q), where different values and positions of the nonzero element in x are used. ¯ and x, we can find codewords For each combination of H through c¯, whose weights are found as w(¯c) = w(D0 x) + · · · + w(Dm x). We choose the lowest bit weight of all nonzero codewords as the upper bound of the code’s minimum distance given this parity check matrix. B. Numerical Example First, we apply the proposed algorithm on the binary codes with parity check matrices specified in [10, eqs. (10) and (11)]. The same results as in [10] are obtained, i.e., the upper bound on the minimum distance of the code [10, eq. (10)] is 20 and
6
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. XX, NO. X, FEBRUARY 2012
TABLE III E STIMATED MINIMUM WEIGHTS OF NONBINARY QC-LDPC CODES Symbol length Girth Minimum symbol weight Minimum bit weight results estimated by Minimum symbol weight Minimum bit weight
228 492 756 1596 6 8 10 12 10 14 20 24 12 22 24 24 the improved NNCS method 10 10 10 10 11 15 16 16
2496 12 24 24 10 16
the upper bound on the minimum distance of the code [10, eq. (11)] is 24. Second, let us consider the ‘Design 4’ codes listed in Table I of [29] with symbol length of 228, 492, 756, 1596 and 2496 over GF(8). Table III shows the estimated minimum weights of the ‘Design 4’ codes listed in Table I of [29], of all codes the mean column weight is 2.5 and the code rate is 0.5. Although the estimated upper bound of the minimum distance is larger than the result obtained by the improved NNCS method, it could be meaningful to guide the design of nonbinary QC-LDPC codes achieving large minimum distance, because compared to the NNCS method, we only need the parity check matrix instead of running the decoding process to estimate the minimum distance. V. C ONCLUSIONS In this paper, we improved the nearest nonzero codeword search (NNCS) method to directly compute the minimum distance of nonbinary LDPC codes, a topic that has not been investigated in the literature. Numerical results on the estimated minimum distances confirm the benefits of nonbinary LDPC coding, showing that a code’s minimum distance can be increased by careful selection of nonzero elements of the parity check matrix, or by increasing the mean column weight, or by increasing the size of the Galois field. An upper bound on the minimum distance was also presented for nonbinary quasi-cyclic LDPC codes. R EFERENCES [1] R. H. Morelos-Zaragoza, “The Art of Error Correcting Coding,” Boston, MA: Wiley, 2002. [2] M. C. Davey and D. MacKay, “Low-density parity-check codes over GF(q),” in IEEE Commun. Lett., vol. 2, June 1999. [3] X.-Y. Hu and E. Eleftheriou, “Binary representation of cycle Tanner-graph GF(2b ) codes,” in Proc. of ICC, June 2004. [4] J. Huang, S. Zhou, and P. Willett, “Nonbinary LDPC coding for multicarrier underwater acoustic communication,” in IEEE J. Select. Areas Commun., vol. 26, no. 9, pp. 1684–1696, Dec. 2008. [5] A. Vardy, “The intractability of computing the minimum distance of a code,” in IEEE Trans. Inf. Theory, vol. 43, Nov. 1997. [6] X. -Y. Hu , M. Fossorier and E. Eleftheriou, “On the computation of the minimum distance of low-density parity-check codes,” in Proc. of ICC, vol. 2, pp. 767-771, Jun. 2004. [7] A. Keha and T. Duman, “Minimum distance computation of LDPC codes using a branch and cut algorithm,” in IEEE Trans. Commun., vol. 58, no. 4, pp. 0090-6778, April 2010. [8] C. Poulliat, M. Fossorier, D. Declercq, “Design of Regular (2,dc)-LDPC Codes over GF(q) Using their Binary Images,” IEEE Trans. Commun., vol. 56, no. 10, pp.1626-1635, Oct. 2008. [9] D. J. C. MacKay and M. C. Davey, “Evaluation of Gallager Codes for short block length and high rate applications,” in IMA volumes in Mathematics and its Applications, vol. 123, ch. 5, pp. 113-130, 2001.
[10] M. Fossorier, “Quasi-Cyclic Low-Density Parity-Check Codes From Circulant Permutation Matrices”, in IEEE Trans. Inf. Theory, vol. 50, no. 8, pp. 1788-1793, Aug. 2004. [11] J. Hou, P. H. Siegel, L. B. Milstein, and H. D. Pfister, “Capacityapproaching bandwidth-efficient coded modulation schemes based on low-density parity-check codes”, in IEEE Trans. Inform. Theory, vol. 49, pp.2141-2155, Sep. 2003. [12] A. Ingber and M. Feder, “Parallel bit interleaved coded modulation”, in Proceedings of Annual Allerton Conference on Communications, Control, and Computing, Sep. 2010. [13] U. Erez and R. Zamir, “Achieving 1/2 log(1 + SNR) on the AWGN Channel with lattice encoding and decoding,” in IEEE Trans. Inf. Theory, vol. 50, no. 10, pp. 2293-2314, Oct. 2004. [14] A. Bennatan, D. Burshtein, “EXIT charts for non-binary LDPC codes over arbitrary discrete-memoryless channels”, in ISIT 2005, pp. 37-41, Sep. 2005. [15] A. Bennatan, D. Burshtein, “Design and analysis of nonbinary LDPC codes for arbitrary discrete-memoryless channels”, in IEEE Trans. Inform. Theory, vol. 52, no. 2, pp. 549-583, Feb. 2006. [16] T. Richardson and R. Urbanke, “The capacity of low-density parity check codes under message-passing decoding”, in IEEE Trans. Inform. Theory, vol. 47, no. 2, pp. 599-618, Feb. 2001. [17] C. A. Cole, S. G. Wilson, E. K. Hall, and T. R. Giallorenzi, “Analysis and design of moderate length regular LDPC codes with low error floors,” in Proc. of. on Information Sciences and Systems, Mar. 2006. [18] H. Song and J. R. Cruz, “Reduced-complexity decoding of Q-ary LDPC codes for magnetic recording,” in IEEE Trans. Magn., vol. 39, pp. 10811087, Mar. 2003. [19] M. P. C. Fossorier, “Iterative reliability-based decoding of low-density parity-check codes,” in IEEE J. Select. Areas Commun., vol. 19, pp. 908-917, May. 2001. [20] G. D. Forney, “Geometrically uniform codes,” in IEEE Trans. Inf. Theory, vol. 37, no. 5, pp. 1241C1260, Sep. 1991. [21] D. Sridhara, T. E. Fuja, “LDPC Codes Over Rings for PSK Modulation ,” in IEEE Trans. Inf. Theory, vol. 51, no. 9, pp. 3209-3220 , Sep. 2005. [22] E. Mo and M.A. Armand, “Structured LDPC codes over integer residue rings,” in EURASIP Journal on Wireless Communications and Networking: Special Issue on Advances in Error Control Coding Techniques, 2008. [23] Collected nonbinary LDPC codewords with small weights, http://mail.ustc.edu.cn/˜liul/ [24] M. Hirotomo, M. Mohri, M. Morii, “New Probabilistic Algorithm to Enhance the Reliability of Computed Weight Distribution of LDPC Codes”, in Proc. of ISITA2006, Seoul, Korea, 2006. [25] D. Declercq and M. Fossorier, “Improved Impulse Method to Evaluate the Low Weight Profile of Sparse Binary Linear Codes”, Proc. ISIT, Seoul, Korea, 2008. [26] J. Huang, S. Zhou, J. -K. Zhu, and P. Willett, “Group-theoretic analysis of Cayley-graph-based cycle GF(2p ) codes,” in IEEE Trans. Commun., vol. 57, no. 7, pp. 1560-1565, Jun. 2009. [27] S. Lin, S. Song, L. Lan, L. Zeng, and Y.-Y. Tai, “Constructions of nonbinary quasi-cyclic LDPC codes: A finite field approach,” in Proc. of Inform. Theory and Applications (ITA) Workshop 2006, UCSD, 2006. [28] R. -H. Peng and R. -R. Chen, “Design of nonbinary quasi-cyclic LDPC cycle codes,” in Proc. of ITW, Sept. 2-6 2007. [29] J. Huang, L. Liu, W. Zhou and S. Zhou, “Large-Girth nonbinary QCLDPC codes of various lengths,” in IEEE Trans. Commun., vol. 58, no. 12, pp. 3436-3447, Dec. 2010.
LIU et al.: COMPUTING THE MINIMUM DISTANCE OF NONBINARY LDPC CODES
Lei Liu received the B.S. degree in 2007 from the University of Science and Technology of China (USTC), Hefei, in communication engineering. He is currently working toward the Ph.D. degree with the Department of Electronic Engineering and Information Science, USTC. His research interest is in the areas of error control coding and information theory.
Jie Huang was born in Jiangling, Hubei, P. R. China on January 20, 1981. He received the B.S. degree in 2001 and the Ph. D. degree in 2006, from the University of Science and Technology of China (USTC), Hefei, both in electrical engineering and information science. He was a post-doctoral researcher from July 2007 to June 2009, working with the Department of Electrical and Computer Engineering (ECE) at the University of Connecticut (UCONN), Storrs. Now he is a research assistant professor with the ECE Department at UCONN. His general research interests lie in the areas of communications and signal processing, specifically error control coding theory and coded modulation system design. His recent focus is on signal processing, channel coding and network coding for underwater acoustic communications and underwater sensor networks. Mr. Huang has served as a reviewer for the IEEE Transactions on Communications, the IEEE Transactions on Information Theory, the IEEE Transactions on Signal Processing, and the IEEE Journal on Selected Areas in Communications.
Wuyang Zhou received B.Eng. and M.Eng. degrees from Xidian University, Xi’an, China, in 1993 and 1996, respectively, and the Ph.D. degree from the University of Science and Technology of China (USTC), Hefei, in 2000. He is currently a professor in the Department of Electronic Engineering and Information Science, USTC. His research interests include error control coding, wireless resource allocation, and wireless networking.
Shengli Zhou (M03) received the B.S. degree in 1995 and the M.Sc. degree in 1998, from the University of Science and Technology of China (USTC), Hefei, both in electrical engineering and information science. He received his Ph.D. degree in electrical engineering from the University of Minnesota (UMN), Minneapolis, in 2002. He was an assistant professor with the Department of Electrical and Computer Engineering at the University of Connecticut (UCONN), Storrs, 2003-2009, and now is an associate professor. He holds a United Technologies Corporation (UTC) Professorship in Engineering Innovation, 2008-2011. His general research interests lie in the areas of wireless communications and signal processing. His recent focus is on underwater acoustic communications and networking. He has served as an associate editor for IEEE Transactions on Wireless Communications from Feb. 2005 to Jan. 2007, and is now an associate editor for IEEE Transactions on Signal Processing. He received the 2007 ONR Young Investigator award and the 2007 Presidential Early Career Award for Scientists and Engineers (PECASE).
7
