A Maximum-Likelihood Decoding Algorithm of LT Codes ... - CiteSeerX

1 downloads 0 Views 563KB Size Report
In this section, we go over the RSD based on Mackay's recursion in [11 ..... [11] DAVID J.C MACKAY, Information Theory, Inference and Learning. Algorithms ...
A Maximum-Likelihood Decoding Algorithm of LT Codes with a Small Fraction of Dense Rows. Ki-Moon Lee

Hayder Radha

Dept. of Mathematics Michigan State University E. Lasing, MI-48824, USA Email: [email protected]

Dept. of Electrical and Computer Engineering Michigan State University E. Lansing, MI-48824 Email: [email protected]

Abstract— We design a new form of the Maximum Likelihood Decoding Algorithm (MLDA) in [1] for LT codes [3] with a small fraction of dense rows. In particular, we design degree distributions using the Robust Soliton Distribution (RSD) for the proposed MLDA based decoding of LT codes. We also estimate the computational complexity of the proposed LT based MLDA. Simulation results, which show the viability of the proposed MLDA decoding of LT codes, are also presented.

I. I NTRODUCTION Let α = (α1 , . . . , αn ) be a given input symbol vector in (Fs2 )n . In the LT based encoding scheme, an encoder constantly generates syndrome βi ∈ Fs2 using a rowPn symbols d degree distribution ρ(x) = d=1 ρd x and α in the following manner: 1) a degree d is randomly chosen with probability ρd , then n a row  Hi ∈ F2 of degree d is chosen at random from the n possible choices; d 2) the encoder then transmits βi = Hi αT over a Binary Erasure Channel (BEC). The transmission stops when a receiver acquires a sufficient number of syndrome symbols. At the receiver end, if more than m = (1 + γ)n (∃γ > 0) βi ’s are received, an LT decoder recovers α by solving the consistent linear system HX T = β T ,

β = (β1 , . . . , βm ) T

(1)

where H consists of rows Hi , Hi α = βi . Assuming that rows of H follow the distribution ρ(x) such as the RSD in [3], H can be lower triangulated by means of permuting rows and columns of H, as in the Message Passing Algorithm (MPA) [4]. Thus α can be recovered by a forward substitution (FS) over the triangulated matrix. For short block lengths n, however, the success of the MPA is not guaranteed as γ → 0. Nonetheless, regardless of the MPA failure, (1) has a unique solution iff. Rank(H) = n. If Rank(H) = n, then (1) can be solved efficiently by the MLDA suggested by Burshtein and Miller in [1]. Their MLDA consists of four major routines: 1. ALTA: By the Approximate Lower Triangulation Algorithm (ALTA) [2], a pair of row and column permutations (P, Q) ¯ = P HQT = [ A B ], where B is is obtained with which H C D in a lower triangular form. Thus (1) is permuted as  T  T  βu A B XR ¯X ¯ T = P βT , ⇔ H (2) T = C D XR βlT ¯

¯ T = QX T = [XR , XR¯ ]T and P β T = [βu , βl ]T . where X h −1 i B 0 2. BSR: Multiplying by S = −DB , called Back−1 I Substitution of References (BSR), (2) is transformed to  T  T  A¯ I XR β¯u ¯ X ¯ T = SP β T , (3) = ⇔ [S H] T C¯ 0 XR¯ β¯lT

where C¯ = C − DA¯ and A¯ = B −1 A (see Fig. 1-(b)). 3. GE:XR is recovered by Gaussian Elimination (GE) on ¯ R = β¯T . CX l T ¯T ¯ 4. Final FS (FFS): XR¯ is recovered by XR ¯ = AXR + βu . The core of the MLDA is in the novel combination of the ALTA and the BSR by S that reduces (1) into a small system ¯ R = β¯T by means of a partial GE on H ¯ over the columns CX l B B ], the MLDA ¯ of the triangular block [ D ]. When H = [ D becomes the MPA. Of particular interest of the MLDA is in the ALTA on H, generated by the RSD. The prominence of Luby’s RSD is not just in the perfect triangulation of H by the MPA, but is also in the robustness of the RSD. A small perturbation on the RSD does not affect much the triangulation of H. Furthermore, even for short n and γ close to 0, H can be ¯ in (2) that has a very small [ A ]. Hence, permuted into an H C ¯ the GE on C is very efficient compared to the conventional GE on H. This advanced form of GE also can be found in [11, Ex-50.12] and the US patent [6]. Another MLDA using guessing strategies on XR at a bit-level was developed in [8]. ¯ Note that Rank(H) = Let q×r be the block dimension of C. ¯ ¯ n iff. Rank(C) = r. If Rank(C) = r, then the GE on C¯ is an LU -factorization [10, ch.7] that returns a r × r nonsingular ¯ where L and U is in a lower and sub-matrix C¯r = LU of C, upper triangular form, respectively (see Fig. 1-(c)). In fact, L−1 and U −1 can be obtained in a product form of elementary ¯ matrices, so that by the GE on C, T XR = (LU )−1 β¯rT ,

U −1 =

r Y

k=1

U (k) , L−1 =

1 Y

L(k) , (4)

k=r

where L(k) and U (k) are the elementary matrices formed by replacing the k th row of Ir×r with the k th row of L and U , respectively, and β¯r is a sub-vector of β¯l in (3) associated with ¯ < r, free variables in XR C¯r . Even the case when Rank(C) can be identified by the GE. Thus, (1) can be solved for α by the retransmission of input symbols of free variables only.

¯ in In this paper, first, without explicit construction of H (2), we interpret systems of the MLDA as an equivalent set of linear systems in a product form of elementary matrices via the permutations (P, Q). Let HQT = [HR ; HR¯ ], the rearrangement of columns of H associated with (XR , XR¯ ). We identify (2) from (1) via (P, Q). Then using S¯ = P T SP , we interpret (3) (in the BSR) as ¯ R ; HR¯ ]X ¯ T = Sβ ¯ T. S[H

(5)

¯ R = SH ¯ R . For each k, 1 ≤ k ≤ q, the k th row C¯k Let H ¯ ¯ ¯ of C is identical to the ith k row (HR )ik of HR for some ik . Hence, we perform the LU -factorization over the row ¯ R )i }q ¯ and U ¯ that set {(H and obtain m × m matrices L k k=1 are identical to an L and U via a permutation pair (σr , τr ), respectively, and whose inverses are also in a product form of elementary matrices similar to the ones in (4). Consequently, (3) and (4) together is equivalent to     ¯U ¯ )−1 · S¯ · (HQT ) · X ¯ T = (L ¯U ¯ )−1 S¯ · β T . (6) (L

Second, based on (6), we transform the original MLDA into a separated MLDA (S-MLDA) using two steps. 1) pre-decoding: 1a) compute the left-hand side of (6); 1b) discard the equations in (1) that become null after the T ¯ GE on SHQ (see null rows in the bottom of Fig. 1(c)). 2) post-decoding: 2a) compute the right-hand side β¯T = ¯U ¯ )−1 Sβ ¯ T in (6); 2b) recover XR from β¯ via (σr , τr ); (L 2c) recover each xk of XR¯ using sparser of the k th equations from (2) and (3) as below T T Bk XR ¯ = βk + Ak XR

or

T xi = β¯k + A¯k XR .

(7)

For a higher probability Pr(Rank(H) = n), small fractions of dense rows are required in ρ(x). Most of the dense rows, ¯ Thus, ahead of the however, become null after the GE on C. post-decoding step, all of redundant rows should be discarded to avoid symbol additions of β over those rows. As γ → 0, A¯ is not sparse in general and its column dimension r becomes larger. However, the top part of A¯ is more likely sparser than the top of [A; B]. In fact, many rows in the top of A¯ are null or with degree one. On the other hand, the bottom of A¯ is denser than the the bottom of [A; B]. Hence, the alternative recovery in (7) by comparing deg(A¯k ) and deg(Ak , Bk ) improves the efficiency in symbol additions significantly. Detailed routines of the S-MLDA can be found in [9]. The remainder of the paper is organized as follows. In Section II, we elaborate on the systems (5) and (6) from the perspective of basic linear algebra over F2 . In Section III, we design a degree distribution ρ(x) with a small fraction of dense rows for higher Pr(Rank(H) = n). In Section IV, we estimate the complexity of the S-MLDA in terms of the number of {sign, bit}-flips and symbol additions of β made by the predecoding and post-decoding, respectively. In Section V, we present our simulation results tested with LT codes for the following scenarios i) the performances of the codes in γ under the S-MLDA and the MPA; ii) number of symbol additions by the post-decoding; and iii) fractions of references r n . We then summarize the paper in Section VI.

XR

XR¯

XR

XR¯

XR

XR¯

A

B



I



I

D



C

¯ by ALTA (a) H

0

¯ by BSR (b) S H Fig. 1.

L

U

0

0 0

(c) LU by GE

¯ The MLDA on H

II. T HE S EPARATED MLDA We first outline several notations that are used in the remainder of the paper. For a given m × n matrix K over F2 , we denote kij , Ki , K j , and |K| as the (i, j)th entry, the ith row, the j th column, and the number of 1’s of K, respectively. We use the notation 1ij to indicate the (i, j)th 1 in K. With the block size m × n, let [m] = {1, 2, . . . , m} and [n] = {1, 2 . . . , n}. In the rest of the section, we verify (5) and (6) in a product form of elementary matrices. First, by flipping a known 1 into −1 through the diagonal extension of B, the ALTA in [2, p644] can be designed to obtain a set of ordered pairs (σl , τl ) in [m] × [n], such that (σl , τl ) = (i1 , jr+1 ) ≻ · · · ≻ (il , jr+l ),

n = l + r.

(8)

Then each bst in the triangular B is identical to his ,jr+t in ¯ and H via is ∈ σl and jr+t ∈ τl . With σl and τl , let (R, R) (T , T¯ ) be the disjoint pair of [n] and [m], respectively, where ¯ = τl = {jr+1 , . . . , jr+l }, R = {j1 , . . . , jr }, R T = σl = {i1 , . . . , il }, T¯ = {il+1 , . . . , im }. With the pairs, we set the permutations σ : [m] 7→ [m] by σ(ik ) = k and τ : [n] 7→ [n] by τ (jk ) = k. Then their matrix representations P and QT can be generated by permuting rows of Im×m and columns of In×n in the order of σ and ¯ let X ¯ = (XR , XR¯ ), τ , respectively. Associated with (R, R), where XR = (xj1 , . . . , xjr ) and XR¯ = (xjr+1 , . . . , xjr+l ). Then via (P, Q), (1) is equivalent to (2). Second, multiplying by B −1 , the top system of (2) is −1 T T ¯ T βu , then substituting the XR¯ brought into XR ¯ = AXR + B T ¯ T as in (3). This ¯ into the bottom yields CXR = B −1 βuT + Aβ l step is the BSR in [1], and is equivalent to the multiplication −1 B 0 B 0 ] in a ] of (2). Noting that S −1 = [ D by S = [ −DB −1 I Q1 I (k) lower triangular form, S can be expressed as S = k=l S , where each S (k) is the elementary matrix formed by replacing ¯ k . Therefore, S H ¯ in (3) is computed by the (Im×m )k with H ¯ := S (k) H, ¯ k = 1, 2, . . . , l. Rearranging columns iteration H of H associated with (XR , XR¯ ), let HQT = [HR ; HR¯ ] where HR = [H j1 ; . . . ; H jr ] and HR¯ = [H jr+1 ; . . . ; H jr+l ]. Then the iteration can be shifted on HR via P T . To see this, let S¯(k) = P T S (k) P , formed by replacing (Im×m )ik with H jr+k via the k th index pair (ik , jr+k ) in (σQ l , τl ). Now 1 let S¯ = P T SP . Then since P −1 = P T , S¯ = k=l S¯(k) .

Substituting the product into (5), (3) is now equivalent to 1 Y

¯T = S¯(k) [HR ; HR¯ ]X

k=l

1 Y

k=l

S¯(k) · β T .

(9)

¯ R¯ = P T [ I ], the computation of SH ¯ R alone is Since SH 0 T ¯ enough for SHQ , and is obtainable by the iteration HR := S¯(k) HR ,

k = 1, 2, . . . , l.

(10)

¯ R = SH ¯ R . Then C¯k = (H ¯ R )i via σ(ik ) = k, Let H k ∀ik ∈ T¯ . We perform the LU -factorization over the row set ¯ R )i }q with additional steps. At each pivoting round k: {(H k

k=1

1) When a pivot 1sk ,tk is chosen for the k th diagonal of LU , store the index pair (sk , tk ) into the set of pairs (σr , τr ). 2) For each pivoted 1s,tk whose row index s ∈ T¯ , rather ¯ R )s . than 0, flip it into −1s,tk in (H ¯ ¯ R )i }. 3) Then discard the row (HR )sk from {(H k ¯ If Rank(C) = r, then after the GE (σr , τr ) = (s1 , t1 ) ≻ · · · ≻ (sr , tr ) ⊂ T¯ × R.

(11)

Thus, for each sk ∈ σr and tj ∈ τr , the −1sk ,tj (1sk ,tj ) in ¯ R can be identified as the 1kj of an L (U respectively) in H ¯ < r, then free variables are precisely the xj ’s (4). If Rank(C) whose column index j ∈ R \ τr . ¯ T . We note that, each 1k,j in L(k) (or Lastly, let β¯T = Sβ U (k) ) of (4) corresponds to the symbol addition (β¯r )k := ¯ R )s (β¯r )k + (β¯r )j . Similarly, the −1sk ,tj or 1sk ,tj in (H k corresponds to the symbol addition β¯sk := β¯sk + β¯sj via the k th row index sk and the j th row index sj in σr . Therefore, each L(k) (or U (k) ) in (4) corresponds to the m×m ¯ (k) (or U ¯ (k) ), formed by placing those elementary matrix L −1sk ,tj (or 1sk ,tj respectively) as the 1sk ,sj in (Im×m )sk . ¯ −1 = Thus, products in (4) correspond to the products L Q1 the Q r (k) −1 (k) ¯ and U ¯ = ¯ via (σr , τr ). Consequently, k=r L k=1 U (3) and (4) together is equivalent to (as in (6)) ¯ −1 L ¯ −1 S[H ¯ R ; HR¯ ] · X ¯T = U ¯ −1 L ¯ −1 Sβ ¯ T. U

(12)

In the pre-decoding step in I, 1a) can be depicted as ALT A BSR ¯ GE ¯ ¯ H −−−−→ [HR ; HR¯ ], (σl , τl ) −−−→ H R −−→ LU , (σr , τr ). (10)

Then the redundant equations Hi X T = βi , ∀i ∈ T¯ \ σr are discarded in 1b). The right-hand side of (12) is computed by the post-decoding step as described in I. We note that in (7), ¯ R )i , via σ(ik ) = k, each A¯k , Ak , and Bk is identical to (H k (HR )ik , and (HR¯ )ik , respectively. III. LT D ISTRIBUTION D ESIGN WITH D ENSE F RACTIONS In this section, we go over the RSD based on Mackay’s recursion in [11, ch9]. We then supplement the RSD with a fraction of dense rows. A fine analysis of the RSD design in a continuous frame can be found in Shokrollahi’s works in [5]. Let us consider the diagonal extension of the ALTA on H (see [2, p644]). Starting from H(0) := H, at the tth diagonal extension step, let 1it ,jt be selected for the tth diagonal entry of B, and let H(t) be the residual matrix by discarding the

jt th column and the it th row from H(t−1). Now let ht (d) be the expected number of rows of degree d in H(t). Then at the (t + 1)th extension, using the arguments in [11, Ex. 50.2-4], ( 1 t (2) ) + 2·h d=1 (ht (1) − 1)(1 − n−t n−t , (13) ht+1 (d) = d t (d+1) ) + (d+1)·h , d > 1. ht (d)(1 − n−t n−t Expecting the ripple size ht (1) = S + 1 for all t, (13) asserts n−t ht (d) = d(d−1) + Sd for d > 1. Then at t = 0, ( S + 1, d=1 h0 (d) = . (14) n S + , d>1 d(d−1) d Pn Let m = d=1 h0 (d) ≈ n + S(1 + ln(n)) so that γ ≈ S(1 + ln(n))/n. Normalizing the {h0 (d)} gives the RSD as n X h0 (d) . (15) µd xd , µd = µ(x) = m d=1

Luby’s√RSD in [3] can be made from (14) with S = c ln(n/δ) n, h0 (d) = (14) for 1 ≤ d < n/S, h0 (d) = n/d(d − 1) + S ln(S/δ) for d = n/S, and h0 (d) = n/d(d − 1) for d > n/S. Theorem-17 in [3] guarantees that when H is randomly generated by the RSD in [3] with m ≈ n + S(ln(S/δ) + ln(n/S)), system (1) has a unique solution and can be solved by the MPA with probability at least 1 − δ. The success of the MPA depends on the constraint ht (1) ≥ 1 for all t ≤ n. Thus, γ may be increased for short n. For a stable ripple design, see [5, p21-22]. In contrast, the success ¯ = r, but it has heavier of the S-MLDA depends on Rank(C) ¯ However, the computational complexity due to the GE on C. A ] of H ¯ is very small for any γ > 0. Therefore, left block [ C the GE on C¯ in a bit-level may not be a major drawback to the efficiency of the S-MLDA. Let H be an m×n random matrix over F2 with a row-degree P d distribution ρ(x) = ρd xP . We first estimate its columnm degree distribution λ(x) = d=0 λd xd . With λ0 , the fraction of null columns of H, we then estimate a lower bound of |H| to keep λ0 small. Let ar be the average row-degree of H. Since each Hi of degree d follows the distribution fd (x) = (1 − d/n) + (d/n)x with Pr(hij Q = 1) = d/n and every row n is generated in random, λ(x) = d=1 fd (x)(1+γ)nρd , where (k) (1 + γ)n = m. Therefore, λk = λ (0)/k!, and in particular, n Y n(1+γ)ρd (1 − d/n) λ0 = ≈ e−ar (1+γ) . (16) d=1

For large n, the expected number of null columns of H would be estimated as ne−(1+γ)ar by (16). Thus, for a unique solution of (1) with a destined 0 < δ < 1, it should meet

ln(n/δ) . (17) 1+γ Notice that based on (17), we should have |H| ≥ n ln(n/δ). Therefore, fractions of dense rows are indispensable for ρ(x). With the distribution µ(x) in (15), we arrange a distribution ρ(x) that has 3 parts X X ρ(x) = ρd xd + ρd xd + ρn/2 xn/2 , (18) (nλ0 ≈ ne−(1+γ)ar ) < δ < 1 or

d∈D1

d∈D2

ar ≥

such that 1) ρd ≈ µd for d ∈ D1 = {1, . . . , 70}; 2) 0.005 ≤ ρd ≤ 0.01 for d ∈ D2 , and D2 consists of few d’s in 100 ≤ d ≤ 400; 3) 0.001 ≤ ρn/2 ≤ 0.005. The fraction ρn/2 is for higher Pr(Rank(H) = n), D1 is for P smaller nr , and D2 is aimed to hold D1 ∪D2 dρd ≥ ln(n/δ) 1+γ so ¯ that, hopefully, the [A, B] of H consists of sparse rows only. We now discuss the selection of an appropriate value for ρn/2 . Let H be an n × n random matrix generated by the fractions of ρ(x) in D1 ∪ D2 only. For a given V ∈ Fn2 , let V ⊥ = {X ∈ Fn2 |X · V = 0}. We note that Img(H) ⊂ V ⊥ iff. V ∈ Ker(H) and the number of solutions of (1) is exactly |Ker(H)|. Let us assume that η = dim(Ker(H)) is small, ˜ be which is feasible based on our simulations. Now let H an expansion of H supplemented with n(1 + γ)ρn/2 random ˜ ⊂ Ker(H), Rank(H) ˜ 0.007, derr = 0 by the MLDA. Even for 0 ≤ γ ≤ 0.007, although we do not provide the evidence, our simulations also exhibit that Rank(H) ≥ n−2. Therefore, a small increment in ρn/2 (or γ) based on (19) may increase Pr(Rank(H) = n) rapidly. Also notice that, by the MPA, derr > 0 for any γ < 0.08 and n. In Fig. 3, a black (red) curve ns = f (1 + γ) represents the number of symbol additions divided by n made by the post-decoding (by the non-separated MLDA). Similarly, a ¯ − d)/n and the green (blue) curve indicates ns = (|A| number of symbol additions made with the redundant symbols βi , i ∈ T¯ \ σr (recall the alternative recovery and the removal of redundant rows in I). When n = 5 · 103 and γ ≈ 0.01, for examples, the black point (1.01, 12) indicates that, approximately, 12 · 5 · 103 symbol additions is made by the post-decoding, and the red point (1.01, 39) corresponds to 39 · 5 · 103 symbol additions made by the non-separated MLDA. Similarly, the blue point (1.01, 15) and the green point ¯ (1.01, 11) corresponds to 15·5·103 and (|A|−d) = 11·5·103 redundant symbol additions, respectively. We observe that a black, blue, and a green ns is mainly contributed by sparse

We present the S-MLDA in Sections I and II. In III, we present a simple design of LT degree distributions with a small fraction of dense rows. We then estimate the complexity of the S-MLDA in IV, and present simulation results in terms of the performance and complexity in Section V. R EFERENCES [1] David Burshtein, Gadi Miller, An Efficient Maximum-Likelihood Decoding of LDPC Codes Over the Binary Erasure Channel, IEEE Trans. Inform. Theory, 50:2837-2844, 2004. [2] T. R ICHARDSON , R. U RBANKE, Efficient Encoding of Low-Density Parity-Check Codes, IEEE Trans. Inform. Theory, 47:638-656, 2001. [3] M. L UBY, LT Codes, 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. [4] M. L UBY, M. M ITZENMACHER , A. S HOKROLLAHI , D. S PIELMAN, Efficient Erasure Correcting Codes., IEEE Trans. Inform. Theory, 47:569584, 2001. [5] A. S HOKROLLAHI, Raptor codes, Digital Fountain, Inc., Tech. Rep. DF2003-06-001, June 2003. [6] A. Shokrollahi, S. Lassen, and R. Karp, Systems and Processes for Decoding Chain Reaction Codes Through Inactivation U.S. Patent 1,856,263, Feb. 15, 2005. [7] M AKATO M ATSUMOTO AND TAKUJI N ISHIMURA Mersenne Twister: A 623-Dimensionally Equidistributed Uniform Pseudo-Random Number Generator, ACM Trans. on Modeling and Computer Simulation, Vol. 8, No.1, Jan 1998, p3-30. [8] Hossein Pishro-Nik and Faramarz Fekri, On Decoding of Low-Density Parity-Check Codes Over the Binary Erasure Channel IEEE Trans. Inform. Theory, 50:439-454, Mar. 2004. [9] K IMOON L EE , H AYDER R ADHA, The Design of Maximum-Likelihood Decoding Algorithms of LDPC Codes over Binary Erasure Channels, Proceedings of 41st CISS, March 2007. [10] Steven Roman, Advanced Linear Algebra, Springer 2nd Edition 2005. [11] DAVID J.C M ACKAY, Information Theory, Inference and Learning Algorithms, Cambridge University Press 2003.