Improved Sum-Min Decoding for Irregular LDPC Codes

7 downloads 231 Views 137KB Size Report
without a loss of bit error performance, for irregular codes it is crucial to vary the post-processing from ... The Sum-Product Algorithm for decoding LDPC codes.
Improved Sum-Min Decoding for Irregular LDPC Codes Gottfried Lechner & Jossy Sayir Telecommunications Research Center Vienna (ftw.), Donaucitystr. 1, 1220 Vienna, Austria, {lechner|sayir}@ftw.at

Abstract An analytical approach to determining the optimal post-processing function for the minimum operation in the SumMin algorithm, previously derived for regular Low-Density Parity-Check (LDPC) codes, is extended to irregular LDPC codes. The optimal post-processing expression for the irregular case varies from one check node to another as well as from one iteration to the next. For practical use, it is necessary to approximate this optimal function. Unlike for the regular case, where one could use a unique post-processing function throughout the decoding process without a loss of bit error performance, for irregular codes it is crucial to vary the post-processing from one iteration to the next in order to achieve good performance. Using this approach, a bit error performance of 0.15dB from the Sum-Product decoder was demonstrated, corresponding to a 1dB improvement compared to the Sum-Min algorithm without post-processing.

1

Introduction

The Sum-Product Algorithm for decoding LDPC codes is hard to implement in practice, since it requires nonlinear functions and multiplications. The algorithm can be simplified by approximating the calculation at the check nodes using a simple minimum operation, resulting in the well-known Sum-Min algorithm (SMA). While the Sum-Min algorithm is less complex to implement, it requires approximately an additional 0.5dB of signal-to-noise ratio Eb /N0 to achieve the same bit error rate as the Sum-Product algorithm, when using a regular LDPC code for transmission over an additive white Gaussian noise (AWGN) channel with binary input. For irregular codes, the loss in performance can be up to 1.0dB. It has been observed by several authors that the performance of the Sum-Min Algorithm can be improved by “post-processing” the minimum. For example, the minimum can be multiplied by a scaling factor, or offset by a constant term. These approaches, known as normalized belief-propagation, offset beliefpropagation [1], [2] or applying an extrinsic scaling factor [3], have been developed heuristically. The optimal parameter, whether scaling factor or offset, is determined by simulation or by density evolution. In [4], we introduced an analytical framework for determining the optimal post-processing function, and found that it is neither a linear (i.e., a scaling factor) nor an affine (i.e., an offset) function but a general S-shaped non-linear function. This optimal post-processing function can be approximated by a linear or affine function, in each case resulting in the same values obtained heuristically in the papers cited above. This work was supported by the Kplus program.

In the present paper, we extend our analytical approach and derive the optimal post-processing function for irregular LDPC codes. Unlike for regular codes, where it was possible to implement the optimal postprocessing function efficiently using a look-up table, for irregular codes the optimal post-processing function is different for every node and for every iteration. Therefore, it is imperative to find easily implementable approximations of the optimal post-processing function. We begin in Section 2 by reviewing the approach derived in [4] and introduce our analytical framework for optimal post-processing. In Section 3, we present the optimal post-processing function for irregular codes and discuss possible simplifications. Finally, in Section 4, we show simulation results and discuss the cost and benefit of the approximation.

2

Post-Processing

In the Sum-Product algorithm, the message computed for each outgoing edge of a check node is an aposteriori probability distribution (i.e., a log-likelihood ratio) of a code digit, given the incoming messages to the check node, which are interpreted as independent observations of the other code digits involved in the corresponding parity-check equation. This interpretation is asymptotically accurate in the codeword length when using random interleavers, a fact often referred to as the independence assumption. When the check-node operation is replaced by a minimization, the resulting message sent along an outgoing edge is not a true log-likelihood ratio, even though its value may be closely correlated to the true loglikelihood ratio. In [4], we demonstrated that most of the performance loss of the Sum-Min algorithm is due to the misinterpretation of the check node outputs by

and φ+ (ζ), φ− (ζ) are defined as Z +∞ p(y)dy φ+ (ζ) =

the variable nodes. The EXIT curve for a check node in the Sum-Min algorithm differs only very slightly from the corresponding EXIT curve for the Sum-Product algorithm, whereas the actual performance trajectory of a simulated Sum-Min decoder was shown to miss the EXIT curves by a wide margin. This signals that the minimum contains almost all of the information originally contained in the input messages to the check nodes, but that the information is in a form that is not suitable for use by the variable nodes. EXIT functions plot the mutual information between the outgoing message and the code digit to be estimated. As such, they are invariant under bijective mappings of the outgoing messages. In other words, the EXIT chart is not fooled by the misrepresentation of information in the minimum and predicts that there exists a one-to-one transformation of the minimum that would allow the Sum-Min algorithm to perform almost as well as the Sum-Product algorithm. This one-to-one transformation of the minimum is what we call postprocessing. The aim of post-processing is to convert the information in the minimum into a form that will be interpreted correctly by the variable nodes. The variable nodes expect the messages from the check nodes to contain log-likelihood ratios. Therefore, we need to convert the minimum to a log-likelihood ratio. We can do this by reinterpreting the minimum as an observation of the code digit to be estimated, and computing the aposteriori probability distribution (i.e., log-likelihood ratio) given the minimum. If Xn is the code digit to be estimated and LSMA mn is the minimum computed by the Sum-Min algorithm without post-processing, then we want to calculate  p Xn = 0|LSMA mn Lmn = log , (1) p (Xn = 1|LSMA mn )

+|ζ|

φ− (ζ) =

Ψ+ (ζ; dc , p(·)) , Ψ− (ζ; dc , p(·))

−|ζ|

p(y)dy. −∞

The function depends on the check node degree dc and on the probability density function p(.) of the incoming messages. If we assume a Gaussian distribution for the incoming messages, the expression depends on the variance of the Gaussian, or, equivalently, on the a-priori mutual information at each iteration. It was shown in [4] that, while the optimal function does vary from iteration to iteration, it is sufficient to select the post-processing function at the narrowest point in the EXIT chart to guarantee the optimal bit error performance of the decoder. Varying the function from one iteration to the next would only serve to reduce the number of iterations, but not to improve the bit error performance. An example of a post-processing function for a regular (3,6) code is shown in Figure 1, along with its approximations by a scaling factor or an offset as in [1], [2], [3]. It is important to note that for the regular case, both the linear and offset approximations yield almost optimal performance and there is not much to be gained in implementing the optimal non-linear post-processing function. On the other hand, since it is sufficient to use a unique post-processing function throughout the iterations without loss in bit error performance, the optimal function can be implemented as a look-up table without a significant increase in complexity as compared to its approximations. 5

4

and transmit this log-likelihood ratio instead of the minimum on the outgoing edge of the check node. This expression, when evaluated analytically, gives a one-to-one mapping of the minimum to a log likelihood ratio. We consider this mapping to be the optimal post-processing operation, meaning that the resulting decoding performance with post-processing is exactly as predicted by the EXIT curves. In other words the decoder makes full use of the information contained in the minimum. The resulting expression for regular LDPC codes, which was derived in [4, Equation 3], is Lmn = f (ζ; dc , p(·)) = log

Z

3

2

L

1

0

−1

−2

−3

(2) −4

where Ψ+ (ζ; dc , p(·))

Ψ− (ζ; dc , p(·))

= [p(ζ) + p(−ζ)] [φ+ (ζ) + φ− (ζ)]

dc −2

+ [p(ζ) − p(−ζ)] [φ+ (ζ) − φ− (ζ)]

dc −2

= [p(ζ) + p(−ζ)] [φ+ (ζ)

−5 −5

−4

−3

−2

−1

0

1

2

3

4

5

SMA

L

Fig. 1. Post-processing function L = f (LSMA ; dc = 6, IAc = 0.5) (solid), linear approximation (dashed) and offset approximation d −2 + φ− (ζ)] c (dashed-dotted).

− [p(ζ) − p(−ζ)] [φ+ (ζ) − φ− (ζ)]

dc −2

,

3

Extension to irregular codes

Irregular LDPC codes can be designed to achieve a target bit error probability at rates much closer to capacity than regular codes, provided that the code length is large and that decoding is performed using the Sum-Product algorithm. However, irregular codes are more sensitive to the effects of replacing the check-node operation by a minimum, so much so that the loss of performance when using the Sum-Min approximation outweighs the benefits of using irregular codes. Again, as for regular codes, the EXIT chart of an irregular code does not predict a significant loss of performance for the Sum-Min approximation. Indeed, since irregular codes are often designed to be checkregular (i.e., the degree of all check nodes is the same), the EXIT functions for the check nodes for both algorithms are identical to the curves presented in [4] for regular codes, and the irregular codes only distinguish themselves by a different variable node EXIT function. This indicates that there is again a one-to-one mapping that can be applied to the minimum such that the performance loss of the Sum-Min approximation is negligeable, as predicted by the EXIT chart. So far the good news: for irregular as for regular LDPC codes, post-processing can help the Sum-Min algorithm achieve a bit-error performance only slightly worse than for the Sum-Product algorithm. There appears to be no difference in theory between the irregular and the regular case, except that we stand to gain much more by applying post-processing for irregular LDPC codes. Now for the bad news: as we will see, there are a two fundamental differences between the optimal postprocessing functions in the regular and in the irregular cases, so that optimal post-processing is not applicable in practice for irregular codes, and we will be forced to use approximations of the optimal post-processing functions. The two fundamental differences are as follows: • the model we used to derive the post-processing function for regular LDPC codes, whereby incoming messages were all assumed to have the same density, is not valid anymore. In the irregular case, incoming messages to a check node originate at variable nodes of different degrees, so that each message will potentially have a different probability density function. If we use a Gaussian approximation for the density, ideally, we should assume a different variance for each incoming message according to which type of variable node it originates in; • the argument used in the regular case, whereby it is possible to use one post-processing function throughout the iterations without a loss in bit error performance, by selecting the post-processing function corresponding to the narrowest point in the EXIT chart, does not apply to irregular LDPC codes. Indeed, irregular LDPC codes are designed

so that the EXIT functions match each other as closely as possible, resulting in a long narrow tunnel between the EXIT curves instead of a single narrowest position. Therefore, it is crucial to apply post-processing optimally (or near-optimally) throughout the iterations, because the decoding algorithm is liable to get stuck almost at any point in the iterative decoding process. We will now proceed to address each of these two differences between the regular and the irregular case.

3.1 Optimal Post-Processing for Unequal Message Densities The optimal post-processing function for regular LDPC codes, stated in Equation 2 and derived in [4], is valid for any message density function, but it specifically assumes that all incoming messages to a check node have the same density. For irregular LDPC codes, this is not a valid assumption anymore. All incoming messages to a check node may originate in variable nodes of different degrees and posess different message density functions. If we wish to determine the a-posteriori loglikelihood ratio Lmn given the minimum reinterpreted as an observation, as specified in Equation 1, for a check node with unequal incoming message densities, we obtain the same expression as in Equation 2, but with a modified expression for the functions Ψ+ and Ψ− . Let p(·) be the vector of the probability densitiy functions of all incoming messages to a check node, and pi (·) be the density function of the i-th incoming message, then we obtain Ψ± (ζ; dc , p(·)) =  dX dY c −1 c −1 [pi (ζ) + pi (−ζ)] [φi+ (ζ) + φi− (ζ)] i=1

± [pi (ζ) − pi (−ζ)]

j=1;j6=i

dY c −1

j=1;j6=i



[φi+ (ζ) − φi− (ζ)]

and the functions φi+ and φi− are the corresponding integrals as defined for regular codes. Calculating the exact optimal post-processing function for a check node with unequal input message density functions, even assuming Gaussian densities, is prohibitively complicated. It includes a multiplication of every combination of cross-products of distinct densities. In addition, it requires us to be able to determine the density (or the variance if we assume a Gaussian density) individually for each incoming message in function of the degree of the variable node that it originates from. Finally, this operation is likely to be different for every check node, as the composition of degrees of variable nodes that each check node is connected to will not be the same for every check node. We conclude that, although knowing the optimal post-processing function may help us design practical post-processing strategies to improve the performance

of the Sum-Min algorithm, it is not practical to implement optimal post-processing as is. This was different for regular codes, where all incoming message densities are the same and the expression above simplifies, so that we can compute the optimal post-processing function once and for all off-line and compute a look-up table that can easily be used in practice. Example 1: Assume an irregular LDPC code with variable node degree distribution λ(x) = 0.25x + 0.75x9 and check node degree dc = 10. If we compute the post-processing function for one output of the check node, g inputs are connected to a variable node of degree 2, and dc − 1 − g inputs are connected to a degree 10 variable node. The post-processing functions assuming a Gaussian message distribution are shown in Figure 2, in function of the parameter g and for IAc = 0.5 . 5 4 3 2 g=0 1

L

g=9 0

every check node, but depends on the variance of the incoming messages, or, equivalently, on the apriori mutual information. Therefore, it is necessary to compute a new post-processing function for every iteration. The argument used for regular LDPC codes, that the decoding trajectory only needs to attain the EXIT curves at the narrowest point in the EXIT chart if bit error performance is all we are interested in, does not extend to irregular LDPC codes. For irregular codes that are designed by optimizing the threshold, the EXIT curves are separated by a narrow tunnel, and failure to touch the EXIT curves at any point in that tunnel would result in the decoder getting stuck and the threshold would thus be affected. Therefore, whether we use non-linear post-processing functions or linear or offset approximations, it is imperative that we vary the post-processing from one iteration to the next. For the linear or offset approximation, this results in a sequence of scaling factors or offset terms, respectively. This sequence can be found by predicting the trajectory in the EXIT chart and computing a correction term for every iteration. The correction term can be predicted by simulation or by density evolution as in [1], [2], [3], or by direct approximation of the non-linear postprocessing function found by our method, using the expressions Z ∞h f (LSMA ; dc , IAc ) − α(dc , IAc ) = argmin α0

−1

0

αL

−2

−4

Fig. 2.

i2

· q(LSMA ; IAc )dLSMA .

for the optimal linear factor, or Z ∞h β(dc , IAc ) = argmin f (LSMA ; dc , IAc ) −

−3

−5 −5

−∞

SMA

β0

−4

−3

−2

−1

0 LSMA

1

2

3

4

5

Post-processing function for Example 1.

For practical use, we are constrained to compute an average incoming message density and calculate the optimal post-processing function using the same expression as for regular codes. It is not possible to measure the loss of performance due to this simplification, because we cannot even implement optimal postprocessing in an offline simulation due to its prohibitive complexity. However, it is important to keep in mind that any failure to achieve a performance predicted by EXIT analysis of Sum-Min decoding of irregular LDPC codes may be due in part to this simplification.

3.2 Varying the Post-Processing Function After simplifying optimal post-processing by assuming equal message densities and further simplifying by assuming Gaussian message densities, if we restrict ourselves to check-regular LDPC codes, we are left with a post-processing function that is identical for

sgn

−∞

LSMA mn



· max LSMA mn − β, 0

q(LSMA ; IAc )dLSMA .

 i2

·

for the optimal offset term, where f (.) is the nonlinear post-processing function and q(.) is the density function of the minimum before post-processing. The two approximations of the non-linear post-processing functions proposed here are weighted minimum mean squared error approximations. Post-processing is then performed using Lmn ≈ α(dc , IAc )LSMA mn for linear post-processing, and   Lmn ≈ sgn LSMA · max |LSMA mn mn | − β, 0 .

(3)

(4)

for affine post-processing. Note that we opted for a multiplicative linear factor, whereas in [1], [4], the minimum is divided by a constant factor. Apart from this distinction, we obtain the same optimal factor or offset as in the papers cited above, using our method of approximating the optimal non-linear post-processing function.

1

0.9

0.8

0.7 α

Since the minimum computed in the Sum-Min algorithm becomes close to the check node outputs computed by the Sum-Product algorithm for large messages, i.e. for large IAc , it can be observed that the values for α and β go to 1.0 or 0.0, respectively, as the number of iterations grows. This is consistent with the observations made in [5], where the authors proposed to set the offset to 0.0 after performing a fixed number of iterations, i.e. at high IAc .

0.6

0.5

1 variable nodes check nodes

0.4

0.9 0

0.8

0.7

Fig. 4.

0.6

10

20

30

40

50 iteration

60

70

90

100

Sequence of scaling factors for the irregular code.

0

10

SPA SMA scaled SMA

Ac Ev

I ,I

80

0.5 −1

10

0.4

0.3 −2

bit error rate

10

0.2

0.1

−3

10

0 0

0.1

0.2

0.3

0.4

0.5 IAv, IEc

0.6

0.7

0.8

0.9

1 −4

10

Fig. 3.

EXIT chart of the irregular LDPC code. 0.5

1

1.5 Eb/N0 [dB]

4

Simulation results

To verify our derivations, we performed bit error rate simulations of regular and irregular LDPC codes with the Sum-Product, the Sum-Min, and the Sum-Min algorithm with linear post-processing. Both codes have a rate of R = 0.5. The regular code has dv = 3 and dc = 6, and the irregular LDPC code has check node degree dc = 9 and a variable node degree distribution (edge perspective) as shown in the following table: degree 2 3 5 6 7 8 9 10 20 30

λ 0.212364 0.198528 0.008380 0.074687 0.014241 0.166518 0.009124 0.020019 0.000251 0.295888

The EXIT chart of the irregular code is shown in Figure 3. The block length was set to N = 105 in order to avoid effects that are caused by cycles in the decoding graph. The code was constructed using the progressive edge growth algorithm [6], [7]. All

Fig. 5.

Bit error rates for the regular LDPC code.

simulations were performed using binary phase shift keying over an AWGN channel, and the decoder was allowed to perform up to 100 iterations. Simulations for the regular LDPC code are shown in Figure 5. The Sum-Min algorithm with linear postprocessing uses a fixed scaling factor of α = 0.80 for every iteration. It can be seen that almost all of the performance loss can be recovered, and the gap to the Sum-Product algorithm is approximately 0.05dB. Note that the thresholds computed via EXIT charts for the Sum-Product algorithm and for the Sum-Min algorithms are 1.13 dB and 1.19 dB, respectively, so that any further improvement of the Sum-Min algorithm by using the optimal non-linear post-processing function instead of linear post-processing is likely to be negligeable. For the irregular LDPC code, simulation results are shown in Figure 6. The improved Sum-Min algorithm uses a sequence of scaling factors, shown in Figure 4. We observe that most of the loss in performance of the Sum-Min algorithm is recovered using variable linear post-processing, and the resulting gap to the SumProduct algorithm is approximately 0.15dB. This is in

2

0

10

SPA SMA variable scaling fixed scaling α=0.66 fixed scaling α=0.72

−1

10

−2

bit error rate

10

−3

10

−4

10

0.5

1

1.5 Eb/N0 [dB]

Fig. 6.

Bit error rates for the irregular LDPC code.

stark contrast to the Sum-Min algorithm without postprocessing, where a quick comparison of Figures 5 and 6 show that the peformance for the irregular code is even worse than the performance for the regular code, despite the much better threshold of the irregular code. This last comparison shows how crucial post-processing can be if one wishes to use irregular LDPC codes in practice. In order to show the importance of varying the scaling factor throughout the iterations, we compared the performance with a decoder that uses a constant scaling factor. The results are shown in Figure 6 for α = 0.66 and α = 0.72. It can be seen that for the lower value of α the waterfall region is close to that of variable scaling. However, the scaling is far to small for high values of IAc and the decoder is not able to converge to low bit error rates, resulting in a high error floor. For the higher value of α this error floor is lower but the price to pay is a larger threshold. Using varying values of α combines both good threshold and no observable error floor.

5

Conclusion

We discussed how the Sum-Min algorithm for decoding LDPC codes can be improved by means of postprocessing, by re-interpreting the minimum as an observation and computing the a-posteriori log-likelihood ratio of the code digits given this observation. We stated the optimal post-processing expression for irregular LDPC codes and concluded that, unlike for regular codes, optimal post-processing is not practically feasible for irregular codes. We opted for simplifying the problem by assuming equal densities for all incoming messages to a check node, but showed that it would be over-simplifying to also assume the same density at every iteration, as we had done in the regular case. We proceeded to implement linear post-processing with a variable scaling factor and were able to measure a decoding performance for the Sum-Min algorithm

2

within 0.15dB of the performance for the Sum-Product algorithm. Several methods have been proposed recently to improve the performance of the Sum-Min algorithm for irregular codes, notably by Fossorier in [8]. What the present paper shows is that optimal post-processing is not practical. Therefore, any engineering technique that approaches optimal post-processing using practical tools is a valid candidate for improving the Sum-Min algorithm in practice. In particular, the method proposed in [8], that combines linear post and pre-processing of the minimum, can be a valid approach, as is our approach of using variable linear post-processing factors. This despite the fact that in theory, it is sufficient to use post-processing to achieve optimal performance, and pre-processing would not be necessary if one were able to implement optimal post-processing.

References [1] Jinghu Chen and Marc P. C. Fossorier, “Near optimum universal belief propagation based decoding of low-density parity check codes,” IEEE Transactions on Communications, vol. 50, no. 3, pp. 406–414, March 2002. [2] Jinghu Chen and Marc P. C. Fossorier, “Density evolution for two improved BP-based decoding algorithms of LDPC codes,” IEEE Communications Letters, vol. 6, no. 5, pp. 208–210, May 2002. [3] Kienle F. and Wehn N., “Joint graph-decoder design of IRA codes on scalable architectures,” in Proc. ICASSP 2004, IEEE International Conference on Acoustics, Speech and Signal Processing, Montreal, Canada, 2004. [4] G. Lechner and J. Sayir, “Improved sum-min decoding of LDPC codes,” in Proc. ISITA 2004, International Symposium on Information Theory and its Applications, Parma, Italy, 2004. [5] J. Chen, R.M. Tanner, C. Jones, and Y. Li, “Improved min-sum decoding algorithms for irregular LDPC codes,” in Proc. 2005 IEEE International Symposium on Information Theory, Adelaide, Australia, 2005, pp. 449–453. [6] Xiao-Yu Hu, E. Eleftheriou, and D.M. Arnold, “Regular and irregular progressive edge-growth tanner graphs,” IEEE Transactions on Information Theory, vol. 51, no. 1, pp. 386–398, Jan 2005. [7] “Software for PEG code construction,” available at http://www.inference.phy.cam.ac.uk. [8] J. Zhang, M. Fossorier, D. Gu, and J. Zhang, “Improved min-sum decoding of LDPC codes using 2-dimensional normalization,” in Proc. Globecom 2005, IEEE Global Telecommunications Conference, St. Louis, Missouri, USA, 2005.