278
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 4, NO. 1, JANUARY 2005
Estimating the PDF of the SIC-MMSE Equalizer Output and Its Applications in Designing LDPC Codes With Turbo Equalization Krishna R. Narayanan, Xiaodong Wang, and Guosen Yue
Abstract—We consider the analysis and design of low-density parity check (LDPC) codes for intersymbol interference (ISI) channels when used with soft interference cancellation plus linear minimum mean-square error filtering (SIC-MMSE) turbo equalization. We discuss techniques to compute the probability density function (pdf) of the extrinsic information at the output of the SIC-MMSE equalizer as a function of pdf of the input extrinsic information, channel impulse response, and the signal-to-noise ratio. For static ISI channels, we show that the output pdf can be modeled as symmetric Gaussian, and show that the mean can be evaluated without simulating the equalizer. For channels with long memory, we propose to use the unscented transform technique to compute the mean, which significantly reduces the computation required. Finally, for fading channels, we model the pdf by a mixture of symmetric Gaussian densities. Using these techniques, we are able to fairly accurately compute the thresholds for LDPC codes and design good irregular LDPC codes. Simulation results are in good agreement with the computed thresholds and the designed irregular LDPC codes outperform regular ones significantly. Index Terms—Low-density parity check (LDPC) codes, soft interference cancellation plus linear minimum mean-square error (SIC-MMSE), turbo equalization.
I. INTRODUCTION
T
HE design of turbo-like codes for intersymbol interference (ISI) channels has been an area of high interest since the results on turbo equalization appeared [4], [17]. Most of the results in this area are confined to the use of convolutional codes or parallel concatenated convolutional codes (PCCC). Recent results [19], [20] show that carefully designed irregular low-density parity check (LDPC) codes can outperform PCCC for long code lengths and provide near capacity performance on memoryless channels. It is then natural to attempt to design good LDPC code ensembles for ISI channels, which is the main focus of this paper. The main idea used in the design of LDPC codes is to employ the technique of density evolution [20], where the probability density function (pdf) of extrinsic information (messages passed Manuscript received April 30, 2003; revised September 11, 2003 and November 24, 2003; accepted November 27, 2003. The editor coordinating the review of this paper and approving it for publication is A. Svensson. This work was supported by the National Science Foundation under Grants CCR 0093020 and CCR 0207550. K. R. Narayanan and G. Yue are with the Department of Electrical Engineering, Texas A&M University, College Station, TX 77843 USA. X. Wang is with the Department of Electrical Engineering, Columbia University, New York, NY 10027 USA. Digital Object Identifier 10.1109/TWC.2004.840196
in the iterative procedure) is computed as a function of iteration and the given degree profiles for the LDPC code in order to com). Then, an optimization pute the thresholds (in SNR or procedure is used to find optimum degree profiles that result in the least thresholds (or, near capacity performance). It has been shown that for a small sacrifice in the resulting thresholds, the design procedure can be simplified by making the assumption that the messages (extrinsic information) at the output of the check nodes and the variable nodes have Gaussian distributions [3]. For ISI channels, the LDPC decoder will be used in conjunction with a soft–input–soft–output (SISO) equalizer. In order to extend the aforementioned technique to design good LDPC codes for ISI channels, we need a technique to characterize the pdf of the extrinsic information at the output of the equalizer as a function of the input pdf and channel signal-to-noise ratio. The exact SISO equalizer is based on the Bahl, Cocke, Jelinek, and Raviv (BCJR) algorithm [2]; however, it is not easy to characterize the output pdf for this equalizer. Usually one has to resort to lengthy Monte Carlo simulations to simulate the input extrinsic information and the channel [25]. Moreover, the complexity of the BCJR equalizer grows exponentially with the memory length of the channel and it may be impractical to use this equalizer when the channel memory is long. Nevertheless, this procedure has been used to compute thresholds for regular LDPC codes with the BCJR equalizer for ISI channels with short memory in [8] and [10], and to design LDPC codes in [25]. In this paper, we consider a SISO equalizer which is based on soft interference cancellation (SIC) and instantaneous linear MMSE filtering; a technique first proposed in [27] and considered for turbo equalization in [18], [23], and [24]. With this equalizer, the equalization complexity grows only quadratically with the channel memory length. We show how to characterize the input–output pdf of the extrinsic information using parametric models for this equalizer in both static and fading ISI channels, and use this to design good LDPC codes. The distinct features of this work include the following. 1) The input–output pdf of the equalizer is expressed in closed-form, and evaluated using parametric models. No simulation of the equalizer or the code is needed for static channels during the design process. 2) ISI channels with very long memory can be easily handled. 3) Code designed for fading channels can be simplified by approximating the equalizer output as a mixture of Gaussian densities (the pdf’s of the bit to check and
1536-1276/$20.00 © 2005 IEEE
NARAYANAN et al.: ESTIMATING THE PDF OF THE SIC-MMSE EQUALIZER OUTPUT
Fig. 1.
4)
279
LDPC coded ISI system.
check to bit messages in the LDPC decoder can be computed analytically; however, the equalizer needs to be simulated). The theoretical performance results match very well with the simulations for both regular and irregular codes, and for both static and fading channels. II. SYSTEM DESCRIPTIONS
A. System Model The system model under consideration is shown in Fig. 1. bits of the data sequence is encoded by A block of an error-correcting code. Any code that permits efficient soft output decoding can be used but we will restrict our attention to LDPC codes in this paper. Both regular LDPC codes and irregular LDPC codes are considered. An irregular LDPC code ensemble is assumed to be specified by two polynomials and , where are the fractions of edges in the bipartite graph associated with the LDPC code, that are connected to variable nodes of degree , are the fraction of edges that are connected to check and nodes of degree . Equivalently, the degree profiles can also be specified from the node perspective, i.e., two polynomials and , where and are the fractions of variable nodes of degree , and check nodes of degree , respectively. For a more detailed exposition, the reader is referred to [19] and [20]. is interleaved and the interleaved The encoder output sequence is modulated. The modulated sequence is then transmitted over the ISI channel. At the receiver, the received signal is sampled at the symbol rate. The combination of the ISI-channel and the sampler is equivalent to a discrete-time transversal filter (DTTF). Since the ISI channel can be time-varying, at each time instant , the DTTF is represented by a tap-delay line with complex weights , where and are the number of precursor and postcursor taps, as shown in Fig. 1. The sampled output at time instant can be expressed as
bits between a SISO equalizer and an LDPC decoder. In each turbo iteration, several inner iterations are performed within the LDPC decoder during which extrinsic messages are passed along the edges in the bipartite graph that represents the LDPC code. 1) Notation: All extrinsic messages (information) are in log-likelihood ratio (LLR) form and the variable is used to refer to extrinsic messages. The variable is used to denote the probability density function of the extrinsic information , and is used to denote the mean of . Superscript is used to denote quantities during the th round of inner decoding within the LDPC decoder and th stage of outer iteration between the LDPC decoder and the equalizer. For the quantities passed between the equalizer and the decoder, only one superscript , namely the turbo equalization iteration number, is used. A denotes quantities passed from the equalizer subscript to the LDPC decoder, and vice versa. Similarly, quantities passed between the bit nodes and the check nodes of the LDPC and , respectively. code are denoted by subscripts III. THE SIC-MMSE SOFT–INPUT SOFT–OUTPUT EQUALIZER It is well known that the exact SISO equalizer for computing the a posteriori bit probabilities is based on the BCJR algorithm [2], with a computational complexity that increases ex. In this section, ponentially with the channel memory we describe a suboptimal SISO equalization algorithm, whose . This technique is based on soft incomplexity is terference cancellation and instantaneous linear MMSE filtering (SIC-MMSE). We begin by rewriting (1) in the matrix form as shown in (2) found at the bottom of the next page, or, with
(3)
Based on the a priori LLR of the code bits provided by the , we first form soft estimates of LDPC decoder, the these code bits according to (4)
(1) Let us define the vector
as
where are samples of a complex Gaussian noise process which may be correlated. B. Receiver Structure The overall receiver is an iterative receiver, which performs turbo equalization by passing extrinsic messages on the code
(5) We then perform soft interference cancellation on
to obtain (6)
280
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 4, NO. 1, JANUARY 2005
Next, an instantaneous linear MMSE filter is applied to obtain
, to
Let us denote notes that
. Then by definition of , we have , where for matrices and , de is positive definite. Therefore
(7) (14)
is chosen to minimize the meanwhere the filter square error between the code bit and the filter output , i.e., This implies that
(15)
(8) where
in (11)
From the definition of (9)
and is a for the
-vector with all zero entries, except th entry, which is 1 (hence, is the th column of ). As in [27], in order to form the LLR of the code bit , we approximate the soft instantaneous MMSE filter output in (7) as Gaussian distributed, i.e., (10) Conditioned on the code bit given respectively by
, the mean and variance of
(16)
are
(11)
IV. DISTRIBUTION OF EQUALIZER EXTRINSIC MESSAGES IN STATIC CHANNELS
(12)
In this section, we assume that the channel is static, i.e., for all . We describe how to compute the pdf of the extrinsic LLRs at the output of the SISO equalizer, as a function of the pdf of the input a priori LLRs, the channel response and the noise , covariance . Note that when the noise is white, i.e., its variance is determined by , the ratio of energy per information bit to the power spectral density of the noise.
delivered by the
Therefore, the extrinsic information SISO equalizer is given by
A. Monte Carlo Evaluation Let us assume that the channel noise in (3) has independent real and imaginary components, then by (10), it follows that the real part of has the following distribution:
(13) Note further that in real-valued channels, we have ; and accordingly, . It can and hence, is positive as follows. be shown that
.. . .. .
.. . .. .
.. ..
.
..
. ..
.
(17)
.
.. . ..
.
.. .
.. .
.. .
.. .
(2)
NARAYANAN et al.: ESTIMATING THE PDF OF THE SIC-MMSE EQUALIZER OUTPUT
Then by (13), the extrinsic information has a Gaussian distribution with mean and variance given respectively by
281
denotes where is a scaling parameter and th column of the matrix . Then the mean and the covariance of can be approximated by (22)
(18) (23)
(19) Note that for real-valued channels, the distribution of the equalizer output extrinsic information is . Hence, the equalizer output extrinsic information has a , Gaussian distribution of the form , where for real-valued channels, with for complex-valued channels. Given the distribution and of the input code bit LLRs (a priori information), as well as the channel and noise statistics, we can compute via Monte Carlo evaluation. The a priori code bit LLR from the LDPC decoder is modeled using a mixture of symmetric Gaussian distributions, i.e., (20)
Note that such an approximation is very useful in evaluating the mean of the output extrinsic of the equalizer when the number of taps is large, especially in the design of good LDPC codes, where the evaluation of the mean at the output of the equalizer becomes the computational bottleneck (cf. Section VI). The UT-based algorithm for calculating the mean of the equalizer extrinsic message is summarized as follows. Let us identity denote as a be a matrix, and let matrix with all elements zeros, except for the th diagonal element, which is 1. Given the channel , the noise covariance , and the a priori code bit LLR distribution the following steps are performed. UT approximation of for static ISI channels . Set : For and
• Let where and are, respectively, the mean and the variance of the th component. Here, are the fractions of the bit nodes of degree and we assume that the pdf of the output extrinsic LLR at a node of degree is symmetric Gaussian with mean [3], [5], [22].
(24) (25)
B. Approximation via Unscented Transformation The above Monte Carlo method for calculating the mean output extrinsic of the equalizer is computationally involved. In order to reduce the complexity in code design, we can replace the Monte Carlo approach by the unscented transformation (UT) technique. We now provide a summary of the UT technique based on [7] and [26]. The UT technique is a method for calculating the statistics of a random variable which undergoes dimena nonlinear transformation. Consider passing an sional random vector through an arbitrary nonlinear function : to generate an dimensional random vector . Assume that has mean and covariance . To calculate the mean of using the UT, we first deterministically weighted samples so that select a set of they completely capture the true mean and covariance of . A selection scheme that satisfies this requirement is [7] and [26]
(26) (27) (28) • Compute (29) (30) and finally calculate
using
(31)
(21)
Note that the main computation involved in calculating the mean of the equalizer output is in the matrix inver-
282
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 4, NO. 1, JANUARY 2005
Fig. 2. Histogram for the SIC-MMSE equalizer extrinsic information in a 2-tap static ISI channel, and the symmetric Gaussian approximations by Monte Carlo evaluation and by unscented transform.
sion in (11). In the Monte Carlo apsamples, where typically proach, this is calculated for –5000, whereas in the UT approach, this is calcusamples. Typically (the lated for number of components in the input pdf, which is the same as the maximum left degree in the LDPC code profile), and hence, the computational savings offered by the UT approximation are evident. We now demonstrate the validity of the Gaussian assumption and the accuracy of the UT through the following example. Consider estimating the pdf of the extrinsic information at the output of the SIC-MMSE equalizer for a two-tap static channel with frequency response when and the pdf of the input a priori information to the equalizer is . Fig. 2 shows the histogram of the extrinsic information at the equalizer output obtained by simulating the channel and the given equalizer. A symmetric Gaussian pdf with a mean by (11) is also shown. It can be seen that the match is quite close, indicating that the underlying pdf is indeed symmetric Gaussian with a theoretical mean given by (11). The symmetric Gaussian pdf computed using the unscented transform is also shown in Fig. 2. It is observed that the UT approximation also closely matches the histogram (true pdf). In order to further show the accuracy of the proposed technique, we compute and plot the extrinsic information transfer (EXIT) curve [22] for the equalizer for the 2-tap channel with for an frequency response and an in Fig. 3. The EXIT chart computed by assuming that the equalizer output is Gaussian whose mean is evaluated using the UT approximation is shown. For comparison, the EXIT chart computed without making any assumptions on the pdf of the equalizer output (i.e., the EXIT and chart computed by first computing from Monte Carlo simulations) is also shown. It can be seen that the EXIT charts computed using the proposed approximation matches very well with the EXIT chart computed
Fig. 3. EXIT charts computed without any assumption on the equalizer output pdf and using unscented transform for 2-tap static ISI channel.
without making any approximations. Since EXIT charts have been shown to be quite useful and accurate in predicting the performance of iterative decoders, Fig. 3 shows that the proposed approximation will be accurate in predicting the performance of the SIC-MMSE equalizer in an iterative setup. V. DISTRIBUTION OF EQUALIZER EXTRINSIC MESSAGES IN FADING CHANNELS As noted before, conditioned on the channel realization at time , the pdf of the equalizer extrinsic message can be well approximated by a Gaussian distribution, i.e.,
(32) where is given by (11). In fading channels, is random and hence, the pdf of the equalizer output extrinsic message is given by (33) denotes the pdf of the equalizer output averNote that aged over , whereas is the equalizer output conditioned on the channel realization . In general, the pdf in (33) can not be well approximated as Gaussian. However, by using a as a mixdiscretized version of (33), we can approximate ture of symmetric Gaussian pdf’s. That is, we model the pdf of the equalizer output LLR as (34) Note that in the limit as , (34) can approximate (33) arbitrarily closely. However, here we are interested in using the pdf (34) with finite to closely approximate the pdf in (33). For fixed number of mixtures , based on the observations
NARAYANAN et al.: ESTIMATING THE PDF OF THE SIC-MMSE EQUALIZER OUTPUT
Fig. 4. Histogram for the SIC-MMSE equalizer extrinsic information in a 2-tap fading ISI channel, and the approximations by a single symmetric Gaussian pdf, and by a mixture of symmetric Gaussian pdfs obtained using the EM algorithm.
, the parameters can be estimated using the expectation-maximization (EM) algorithm. The minimum description length (MDL) principle can be used to select the optimal number of the components in a Gaussian mixture [12], [21]. We now demonstrate the efficacy of the mixture Gaussian modeling of the equalizer extrinsic information developed in this section through the following example. Consider a two-tap independently Rayleigh fading ISI channel and an when the input LLR distribution is . The histogram of the equalizer output extrinsic information obtained using Monte Carlo simulations is plotted in Fig. 4. The approximation of the pdf using a mixture of symmetric Gaussian distributions computed via the EM algorithm is also shown in the figure. Note that the two curves are almost indistinguishable, indicating that the approximation is very accurate. On the other hand, a symmetric Gaussian pdf which has the same mean as that of the histogram is also shown. It is seen that such a single symmetric Gaussian approximation of the extrinsic information distribution is quite inaccurate. This confirms that the extrinsic information delivered by the SIC-MMSE equalizer in fading channels cannot be assumed to be Gaussian, whereas a mixture of symmetric Gaussian pdf offers a good approximation. In this example, the codeword . The number of mixture components length is . given by the MDL criterion is
283
are passed in the iterative process as random variables. Then assuming the length of the codewords is infinite, we compute the pdf of the extrinsic messages at each iteration. For the case of turbo equalization considered here, we assume that the equalizer output is Gaussian for the static channel case. The mean is evaluated using Monte Carlo evaluation as discussed in Section IV. Note, however, that the equalizer does not have to be simulated. For the fading channel case, the pdf is modeled as a mixture of Gaussian densities. To obtain the parameters of the mixture, the equalizer is simulated and an EM algorithm is used to fit a model to the equalizer output. The pdf of the check to bit messages is assumed to be Gaussian and the pdf of the bit to check messages are assumed to be a mixture of Gaussian densities. Once the equalizer output pdf is determined, the mean of the check to bit messages and the parameters of the pdf of the bit to check messages can be obtained analytically. To deal with the memory of the ISI channel and yet, make the density evolution process simple, we assume that the all zeroes codeword is transmitted; however, the mapping from binary dois assumed to be main to the real domain random from one time instant to another. At the receiver, it is assumed that exact mapping sequence is known and, hence, this mapping can be undone before decoding. With this assumption, the sequence transmitted through the ISI channel is a sequence , however, as far as deof i.i.d. random variables coding or analyzing the LDPC code is concerned, the all-zeroes sequence can be assumed. It can be seen that this idea is similar to the Gallager coset codes considered in [8], for which the correctness of this procedure and a concentration theorem have been proved therein. Density Evolution With Gaussian Approximation: Denote . For a given , and , the following procedure is followed to compute the pdf of the extrinsic messages. , and [0:] Initialization: Set . [1:] Turbo equalization iterations: For [1-a:] Compute the pdf of the equalizer extrinsic messages: is computed for the given and using the appropriate procedure from Section IV (for static channels) or Section V (for fading channels), to obtain. Note must be replaced with that for fading channels, throughout this section
(35) VI. THRESHOLD COMPUTATION AND LDPC CODE DESIGN We first describe how to compute thresholds for a given irregand ]. Then, we ular LDPC code ensemble [(i.e., given and for a given rate of the discuss how to optimize code. A. Computing Thresholds The principal idea used here is that of density evolution [3], [11], [20] which is to treat the extrinsic messages that
[1-b:] Compute the pdf of the LDPC extrinsic messages: [1-b-i:] Iterate between bit node update and check node , the following hold. update: For • At a bit node of degree : In order to compute the pdf of the messages from the bit to check nodes at a node of degree , we will assume that the pdf of the incoming messages (from the check to bit) are
284
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 4, NO. 1, JANUARY 2005
Gaussian with mean
. Then
as , then we say that equalization. If the iterations have converged to the correct codeword. In practice, we fix the number of iterations to be and set a high threshold as an approximation to infinity. We fix and compute the means to see if it exceeds the an is reduced by a small threshold, and if it does, the quantity and the procedure is repeated. The threshold is for which excomputed as the minimum ceeds the set threshold (42) (36)
denotes convolution. Since a fraction of where the edges are connected to bit nodes of degree , the pdf of the extrinsic message passed from the bit nodes to the check nodes along an edge is
It should be noted that the above procedure can be used with can be computed for the particany equalizer if the pdf ular equalizer and if the output of the equalizer can be approximated as being mixture of symmetric Gaussian pdf’s. The advantage of using the SIC-MMSE equalizer is that this is indeed possible as explained in Sections IV and V. B. Design of LDPC Codes
(37) • At check node of degree : Assuming that the outgoing message has a Gaussian density, the mean of the outgoing messages at a check node of degree can be shown to be [3] (38) of the edges are connected to Since a fraction checks of degree , the mean of the extrinsic message passed from the check node to bit node (averaged over the degrees of the check nodes) is (39) Note that concentrated right degree are used in all the profiles in this paper (that is, the right degree is either or ). Hence, the right degree is not very irregular and so the assumption that the pdf of the check to bit messages is symmetric Gaussian instead of a mixture is reasonable. [1-b-ii:] Message passed back to the equalizer: At bit node of degree (40) Since a fraction
of the nodes have degree (41)
[2:] For a given lowed to compute
, the above procedure can be folat the end of iterations of turbo
The procedure for computing the thresholds for a given degree profile can be used in conjunction with an optimization procedure to design optimal LDPC codes for ISI and such that channels. The idea is to find optimal the threshold is minimized. Note that the rate of the LDPC code . If a rate of is required, is the optimization problem can be stated as follows. and such that we minimize subject to Find the following constraints 1) : • ; • [computed using (35)–(41)]. Since the optimization problem is complex (even with the Gaussian assumption due to the turbo equalization), a nonlinear optimization procedure called differential evolution [16], [19] has been used to solve the above optimization problem. The nonlinear optimization involves choosing several candidates for and and mutating them in a certain way and computing thresholds until the constraints can be met. Since this involves computing the thresholds for every mutation of the degree profile during the optimization, it is computationally demanding if the extrinsic LLRs are not assumed to be Gaussian mixtures. Without this assumption, the pdf’s have to be evaluated numerically within the LDPC decoder and using Monte Carlo in the equalizer. However, with this assumption, only the means of the components in the mixture need to be evaluated resulting in a significant reduction in complexity. VII. RESULTS Two-Tap Static ISI Channel: We first present results for a . 2-tap static ISI channel with impulse response The theoretical thresholds for a (3,6) rate 1/2 regular LDPC code, and the simulation results for a (3,6) rate 1/2 regular bits are shown in Fig. 5. LDPC code of length It is seen that the actual simulation results are within 0.15 dB of the theoretical thresholds, indicating that the Gaussian assumption and the characterization of the input-output pdf
NARAYANAN et al.: ESTIMATING THE PDF OF THE SIC-MMSE EQUALIZER OUTPUT
Fig. 5. Thresholds and simulation results for the (3,6) regular LDPC codes and for the optimum irregular LDPC codes in a 2-tap static ISI channel.
285
Fig. 6. Thresholds and simulation results for the (3,6) regular LDPC codes and for the optimum irregular LDPC codes in a 10-tap static ISI channel.
consider a 10-tap ISI channel with impulse response of the SIC-MMSE equalizer extrinsic information is quite accurate. For all the simulation results for the 2-tap channel, simulations were carried out until at least 25 block (codeword) errors were observed. Optimum degree profiles were computed for the same channel using the Gaussian assumption as discussed in Section IV-A and the technique discussed in Section VI. The optimum degree profile was
and . The resulting threshold is shown in Fig. 5. The performance of a randomly constructed LDPC code with the is also aforementioned degree profile of length shown in Fig. 5. It is seen that the performance is about 0.25 dB ; however, due to the large from the threshold at BER of percentage of weight-2 columns, there is a perceptible change in the slope. Therefore, we employed a special construction using the bit filling algorithm such that the weight-2 nodes did not participate in any cycles of length less than 50. The is shown simulation results for this code of length in Fig. 5. It is seen that there are no error floors at least until . The same construction with the longer BER of length can provide slightly better performance than reported here, making the gap between the theoretical thresholds and simulations even smaller. Nevertheless, the results presented here show that the irregular codes provide about 0.5 dB better performance than the regular codes. The thresholds plotted in Fig. 5 were computed by estimating the mean of the equalizer output using Monte Carlo simulations [actually simulating the equalizer, instead of using (18)]. We also computed the thresholds by estimating the mean at the output of the equalizer as described in Section IV-A. The two thresholds were in good agreement and, hence, are not plotted separately. Ten-Tap Static ISI Channel: Next we present some simulation results for a static channel with long memory to demonstrate the effectiveness of the approximation by the unscented transform (UT) discussed in Section IV-B. We
(the impulse response was generated randomly according to an exponential power delay profile). The theoretical thresholds for the (3,6) rate-1/2 regular LDPC code with maximum number of iterations between the equalizer and decoder, is shown in Fig. 6. Using the UT approximation, an irregular LDPC code was designed and the resulting was optimum degree profiles with
and . The threshold for the above degree profile and simulation results for a randomly constructed are shown in Fig. 6. It is LDPC code of length seen that the simulation results agree well with the theoretical thresholds, and that the irregular LDPC codes are about 0.8 dB better than the (3,6) regular LDPC code, indicating the usefulness of the proposed techniques for designing good LDPC codes. Due to the significantly large complexity in simulating this system, simulations were carried out until at least 15 block bits were errors were observed or a maximum of simulated. Again, the thresholds were computed both by simulating the equalizer at every iteration to compute the output mean and also by using (33)–(40) to estimate the mean. The two thresholds were very close to each other and, hence, they are not plotted separately in Fig. 6. This further corroborates that the UT provides a good approximation to the mean at the output of the equalizer. Two-Tap Fading ISI Channel: Finally, we consider a 2-tap fading channel where each tap is assumed to be zero-mean complex Gaussian random variable with variance 0.5. The tap coefficients are independent of each other and also independent from one time instant to another. The theoretical thresholds for a (3,6) rate-1/2 regular LDPC code and simulation results for a randomly constructed regular LDPC is shown in Fig. 7 (the recode of length ceiver assumes perfect channel state information). It is seen
286
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 4, NO. 1, JANUARY 2005
that at a BER of , the simulated BER performance is less than 0.15 dB away from the threshold, indicating that the threshold computation is fairly accurate. Then, we de, and rate-1/2 signed optimal degree profiles with for the 2-tap fading channel, and the resulting optimal degree profiles were and . Simulation results for a randomly constructed LDPC code with this degree profile for a length of is shown in Fig. 7. To obtain the BER results, simulations were carried out until at least 25 block errors were , the performance is about 0.25 observed. At a BER of dB away from the threshold. The irregular codes outperform the regular ones by about 0.5 dB. These results show that by using the EM algorithm, we can accurately model the extrinsic information as a mixture of Gaussian densities and use this to design good irregular LDPC codes.
VIII. CONCLUSION In this paper, we have shown how to characterize the pdf of the extrinsic information at the output of the SIC-MMSE equalizer as a function of the pdf of the input extrinsic information, for both static and channel impulse response, and the fading ISI channels. For static ISI channels, we have shown that the pdf can be assumed to be symmetric Gaussian, whereas for fading channels, the pdf can be approximated as a mixture of symmetric Gaussian densities. For channels with long memory, we have shown that the unscented transformation can be used to simplify the computation. By modeling the pdf of the extrinsic information as a mixture of Gaussian densities, we have shown how to compute the thresholds for a given irregular LDPC code degree profile and to design good irregular LDPC codes. In all cases, the computed thresholds match very well with simulations, validating the approximation. Irregular codes designed using these techniques outperform regular LDPC codes.
REFERENCES [1] D. Arnold and H.-A. Loeliger, “On the information rate of binary-input channels with memory,” in IEEE Int. Conf. Commun., vol. 9, Jun. 11–14 2001, pp. 2692–2695. [2] L. R. Bahl, J. Cocke, F. Jelinek, and J. Raviv, “Optimal decoding of linear codes for minimizing symbol error rate,” IEEE Trans. Inf. Theory, vol. 20, no. 2, pp. 284–287, Mar. 1974. [3] S. Y. Chung, T. Richardson, and R. Urbanke, “Analysis of sum-product decoding of low-density parity check codes using a gaussian approximation,” IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 657–670, Feb. 2001. [4] C. Douillard, A. Picart, P. Didier, M. Jezequel, C. Berrou, and A. Glavieux, “Iterative correction of intersymbol interference: Turbo-equalization,” Euro. Trans. Telecommun., vol. 6, no. 5, pp. 507–511, Sep.–Oct. 1995. [5] H. El Gamal and R. Hammons, “Analyzing the turbo decoder using the gaussian assumption,” IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 671–686, Feb. 2001. [6] J. Hagenauer, E. Offer, C. Meason, and M. Morz, “Decoding and equalization with analog nonlinear networks,” Euro. Trans. Telecommun., vol. 10, no. 6, pp. 1–21, Nov. 1999. [7] S. Julier, J. Uhlmann, and H. F. Durrant-Whyte, “A new method for the nonlinear transformation of means and covariances in filters and estimators,” IEEE Trans. Autom. Control, vol. 45, no. 3, pp. 477–482, Mar. 2000.
[8] A. Kavcic, X. Ma, and M. Mitzenmacher, “Binary intersymbol interference channels: Gallager codes, density evolution and code performance bounds,” IEEE Trans. Inf. Theory, vol. 49, no. 7, pp. 1636–1652, Jul. 2003. [9] E. L. Lehmann and G. Casella, Theory of Point Estimation, 2nd ed. New York: Springer-Verlag, 1998. [10] J. Li, K. R. Narayanan, C. N. Georghiades, and E. Kurtas, “Thresholds for iterative equalization of partial response channels using density evolution,” in Proc. IEEE Int. Symp. Inf. Theory, Jun., 24–29 2001, p. 73. [11] M. G. Luby, M. Mitzenmacher, M. A. Shokrollahi, and D. A. Spielman. Analysis of low density codes and improved designs using irregular graphs. presented at Proc. 30th ACM Symp. Theory Comput.. [Online]. Available: http://www.icsi.berkeley.edu/luby [12] P. McKenzie and M. Alder, “Selecting the optimal number of components for a Gaussian mixture model,” in Proc. IEEE Int. Symp. Inf. Theory, Jun.-Jul. 1994, p. 393. [13] G. J. McLachlan and T. Krishnan, The EM Algorithm and Extensions. New York: Wiley, 1997. [14] J. Nelson, R. Koetter, and A. Singer, “Evolution of prior information in SISO equalization,” in IEEE Int. Symp. Inf. Theory, Jun. 24–29, 2001, p. 74. [15] H. D. Pfister, J. B. Soriaga, and P. H. Siegel, “On the achievable information rates of finite state isi channels,” in IEEE Global Commun. Conf., vol. 5, Nov. 25–29, 2001, pp. 2992–2996. [16] K. Price and R. Storn, “Differential evolution—A simple and efficient heuristic for global optimization over continuous spaces,” J. Global Optimization, vol. 11, pp. 341–359, 1997. [17] D. Raphaeli and Y. Zarai, “Combined turbo equalization and turbo decoding,” IEEE Commun. Lett., vol. 2, no. 4, pp. 107–109, Apr. 1998. [18] D. Reynolds and X. Wang, “Low complexity turbo-equalization for diversity channels,” Signal Process., vol. 81, no. 5, pp. 989–995, May 2001. [19] T. Richardson, A. Shokrohalli, and R. Urbanke, “Design of capacity approaching irregular low density parity check codes,” IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 619–637, Feb. 2001. [20] T. Richardson and R. Urbanke, “Capacity of low density parity check codes under message passing decoding,” IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 599–618, Feb. 2001. [21] J. Rissanen, Stochastic Complexity in Statistical Inquiry, Singapore: World Scientific, 1989. [22] S. TenBrink, “Convergence behavior of iteratively decoded parallel concatenated codes,” IEEE Trans. Commun., vol. 49, no. 10, pp. 1727–1737, Oct. 2001. [23] M. Tüchler, R. Koetter, and A. Singer, “Turbo equalization: principles and new results,” IEEE Trans. Comm., vol. 50, no. 5, pp. 754–767, May 2002. [24] M. Tüchler, A. Singer, and R. Koetter, “Minimum mean squared error equalization using a priori information,” IEEE Trans. Signal Process., vol. 50, no. 3, pp. 673–683, Mar. 2002. [25] N. Varnica and A. Kavcic, “Optimized LDPC codes for partial response channels,” in Proc. IEEE Int. Symp. Inf. Theory, Jul. 2002, p. 197. [26] E. A. Wan and R. Van der Merwe, “The unscented Kalman filter for nonlinear estimation,” in Proc. IEEE Symp. 2000 Adaptive Syst. Signal Process. Commun. Control, Lake Louise, Canada, Oct., 1–4 2000, pp. 153–158. [27] X. Wang and H. V. Poor, “Iterative (turbo) soft interference cancellation and decoding for coded CDMA,” IEEE Trans. Commun., vol. 47, no. 7, pp. 1046–1061, Jul. 1999. Krishna R. Narayanan received the Ph.D. degree from the Georgia Institute of Technology, Atlanta, in December 1998. From December 1998 to September 2003, he was an Assistant Professor at Texas A&M University, College Station. Since September 2003, he has been an Associate Professor in the Department of Electrical Engineering at Texas A&M University, College Station. His research interests are mainly in the areas of coding, modulation, and receiver signal processing for wireless communications and digital magnetic recording. His current research focusses on concatenated coding and iterative decoding (turbo codes, low-density parity check codes), joint source-channel coding, space-time coding, and iterative receiver signal processing. Dr. Narayanan is currently an Associate Editor for the IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS and was on the Editorial Board of the IEEE TRANSACTIONS ON COMMUNICATIONS. He is the recipient of the National Science Foundation CAREER award in 2001 and the Outstanding faculty award from the Department of Electrical Engineering in 2002.
NARAYANAN et al.: ESTIMATING THE PDF OF THE SIC-MMSE EQUALIZER OUTPUT
Xiaodong Wang received the B.S. degree in electrical engineering and applied mathematics (with the highest honor) from Shanghai Jiao Tong University, Shanghai, China, in 1992, the M.S. degree in electrical and computer engineering from Purdue University, West Lafayette, IN, in 1995, and the Ph.D. degree in electrical engineering from Princeton University, Princeton, NJ, in 1998. From July 1998 to December 2001, he was an Assistant Professor in the Department of Electrical Engineering, Texas A&M University, College Station. In January 2002, he joined the Department of Electrical Engineering, Columbia University, NY, as an Assistant Professor. His research interests fall in the general areas of computing, signal processing, and communications. He has worked in the areas of digital communications, digital signal processing, parallel and distributed computing, nanoelectronics, and bioinformatics, and has published extensively in these areas. His current research interests include wireless communications, Monte Carlo-based statistical signal processing, and genomic signal processing. Dr. Wang currently serves as an Associate Editor for the IEEE TRANSACTIONS ON COMMUNICATIONS, the IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, the IEEE TRANSACTIONS ON SIGNAL PROCESSING, and the IEEE TRANSACTIONS ON INFORMATION THEORY. He received the 1999 National Science Foundation CAREER Award and the 2001 IEEE Communications Society and Information Theory Society Joint Paper Award
287
Guosen Yue received the B.S. degree in physics and the M.S. degree in electrical engineering from Nanjing University, Nanjing, China, in 1994 and 1997, respectively. He is currently working toward the Ph.D. degree in the Department of Electrical Engineering, Texas A&M University, College Station. His research interests are in the area of telecommunication and digital signal processing, primarily on the modulation and channel coding for wireless communications.