Soft-decision decoding using punctured codes

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 47, NO. 1, JANUARY 2001

59

Soft-Decision Decoding Using Punctured Codes Ilya Dumer, Member, IEEE

Abstract—Let a -ary linear ( )-code be used over a memoryless channel. We design a soft-decision decoding algorithm that tries to locate a few most probable error patterns on a shorter [ ]. First, we take cyclically consecutive positions length starting from any initial point. Then we cut the subinterval of length into two parts and examine most plausible error patterns on either part. To obtain codewords of a punctured )-code, we try to match the syndromes of both parts. Finally, ( )-code are re-encoded to find the designed codewords of an ( the most probable codeword on the full length . For any long linear code, the decoding error probability of this algorithm can be made arbitrarily close to the probability of its maximum-likelihood (ML) decoding given sufficiently large By optimizing , we prove that this near-ML decoding can be ( ) ( + ) error patterns. For achieved by using only most long linear codes, this optimization also gives about re-encoded codewords. As a result, we obtain the lowest complexity ) ( + ) order of ( known to date for near-ML decoding. For codes of rate 1 2, the new bound grows as a cubic root of the . For short blocks of length general trellis complexity min 63, the algorithm reduces complexity of trellis design by a few decimal orders. Index Terms—Complexity, maximum-likelihood (ML) decoding, sorting, splitting, syndromes, trellis.

I. INTRODUCTION

L

ET be a -ary input alphabet and be an output alphabet, which can be infinite. We then consider the input and output vectors . We use a -ary vectors of length to encode equiprobable linear code messages. The codewords are transmitted over a memoryless . Given an channel with transition probability (density) retrieves output , maximum-likelihood (ML) decoding . the codeword with the maximum posterior probability among This gives the minimum block error probability all decoding algorithms. Our main goal is to reduce ML-decoding complexity while allowing only a negligible increase in decoding error probability. as a benchmark for all other deTherefore, we use coding algorithms applied to code . Similarly, decoding complexity of our design will be compared with the upper bound on ML-decoding complexity, which was obtained in [2] and [15] by trellis design. Given any output , we first wish to restrict the list of inamong which the most probable codeword is being puts Manuscript received January 20, 2000. This work was supported by the NSF under Grant NCR-9703844. The material in this paper was presented in part at the 34th Annual Allerton Conference on Communication, Control and Computing, Monticello, IL, October 2–4, 1996. The author is with the College of Engineering, University of California, Riverside, CA 92521 USA (e-mail: [email protected]). Communicated by A. Barg, Associate Editor for Coding Theory. Publisher Item Identifier S 0018-9448(01)00460-6.

sought. More specifically, we consider the list of input vectors with maximum posterior probabilities among all inputs. In contrast to ML decoding, our decoding fails whenever this list does not inalgorithm finds the most probable clude codewords. Otherwise, codeword . Therefore, if otherwise. can fail to decode, its error probability can exceed Since of ML decoding. However, for the probability decoding performance deteriorates only by a negligible margin. Namely, the inequality [6] (1) holds for any (linear or nonlinear) code used over the so-called mapping channels. These channels arise when the output can be split into disjoint subsets of size alphabet in such a way that a conventional -ary symmetric channel is obtained for every . In particular, an additive white Gaussian noise (AWGN) channel with -PSK modulation is a mapping channel. In a more general setting [6], varies for different we can also assume that the output size can symmetric subchannels . Even in this case, algorithm -codes give near-ML decoding for any sequence of long of rate . Here we use the following definition. gives Definition 1: We say that a decoding algorithm near-ML decoding for an infinite sequence of codes if the decoding error probability converges to the probof ML decoding: as ability . -codes of rate To obtain near-ML decoding for long , we use inequality (1) and take , where . Note, however, that a brute-force technique inspects all most . In this probable inputs and gives complexity of order1 . case, we cannot reduce ML-decoding complexity Therefore, we wish to speed up our search by eliminating most . Given a sequence of codes of rate noncodewords from , we consider the upper bounds on complexity of the al. Our main result holds for most long linear codes gorithm of any rate , with an exception of a vanishing frac. This is summarized in the following tion of codes as theorem.

(n) has exponential order q

)=n ! as n ! 1. Similarly, q if

1We say that a function

if (log

0018–9448/01$10.00 © 2001 IEEE

lim(log )=n :

and write

=q

60


Theorem 2: For most long linear codes of rate , near-ML decoding algorithm has decoding complexity , where

STATE COMPLEXITY

#

TABLE I

OF THE ALGORITHM CODES OF LENGTH

63

9

FOR

BINARY

By taking slightly exceeding , we can reduce general of ML decoding to the order of complexity while increasing the output error probability only by an arbitrarily small fraction. In this way, we arrive at the following corollary. Corollary 3: Near-ML decoding can be executed for most -codes of rate with complexity order of long linear . , the new bound grows In particular, for long codes of rate as a cubic root of trellis complexity. For all rates , it surpasses and known for comthe two upper bounds plexity of near-ML decoding (see [6] and [7], respectively). For binary codes, it also falls below the lower bounds on trellis complexity from [13] and [16], provided that no long binary codes exceed the asymptotic Gilbert–Varshamov bound. Central to our proof of Theorem 2, will be the following statement. Theorem 4: Most long linear

-codes have (2)

of most probable inputs. Incodewords in each list equality (2) holds simultaneously over all outputs , with the exception of the fraction

of codes as

.

To prove Theorem 2, we consider a class of algorithms using a parameter . In doing so, we relegate decoding -codes. Each on a block of length to a few punctured punctured decoding will be similar to trellis design. In particular, we will seek the shortest subpaths on trellis diagrams of -synlength . Also, we label each trellis state by the drome associated with the arriving subpaths. However, for each -code, we use one more parameter and choose only best subpaths. Finally, we re-encode these subpaths to the full length and choose the shortest path. with convenIn this regard, we compare complexity of tional trellis design. For such a design, it is customary to consider state complexity that is the binary logarithm of the maximum number of trellis states passed at any time. We will see substantially reduces state complexity even on short that blocks. To unveil the advantages and shortcomings of the al, we consider two different implementations for a gorithm -code. It turns out that both implementations rebinary quire only about three thousand states in contrast to conventional trellis design whose complexity is upper-bounded by states. Similar reductions can also be obtained for other short codes. The results are summarized in Table I, while the details are relegated to Section VI and Appendix B. Here we list the parameters of Bose–Chaudhuri–Hocquenghem (BCH) codes of length . , we optimize and find the state complexity Given

of the algorithm . We first compare state complexity with the general upper bound valid for -code. Much better bounds were derived for BCH any codes (these bounds and further references are given in [10], [13], and [14]). For these codes, we use the upper and lower and from [13]. We see that for the medium bounds reduces general state complexity by three to rates, algorithm six decimal orders. We also reduce state complexity by two to three orders even if the trellis structure is optimized for specific BCH codes. In our decoding design, we will combine two versions of and , near-ML decoding. These algorithms, which we call , were developed in [6] and [7], respectively. For has complexity exponent the first algorithm and can be applied to most linear codes and all cyclic codes. The has complexity exponent for all second algorithm by linear codes. Now we achieve the lower order and . combining Similar technique was first employed in [8] for minimum-distance (MD) decoding that finds the closest codeword in the Hamming metric. Even for MD decoding, the combined algorequires more sophisticated arguments than those used rithm and . In particular, uses weight for its underpinnings and their cosets, as opposed to and spectra of codes , both of which do not rely on good code spectra. To develop , we first unveil how algogeneral soft-decision decoding , and their combination perform in the simrithms , plest case of MD decoding. This will allow us to set a detailed framework for general design and illuminate the problems that need to be addressed. II. BASIC TECHNIQUES FOR MD DECODING A. MD Decoding Let tered at . Let

be the ball of radius

cen-

denote its size (for brevity, parameter is dropped from the since is kept unchanged throughout the notation , algorithm seeks the codewords paper). Given among vectors closest to the output . These vectors of radius belong to the Hamming ball (3)

DUMER: SOFT-DECISION DECODING USING PUNCTURED CODES

Thus, algorithm needs to correct at most errors. According to (1), such a decoding at most doubles the decoding error prob. This important fact proved by ability of MD decoding [9] whose Evseev also led to a simple partition algorithm modification is described below. Note also that we obtain conif our decoding radius is increased by any dition errors enables integer growing with . Thus, correcting near-MD decoding for all long linear codes. Also, exact MD deby coding can be achieved for most long linear codes errors [5]. correcting and , we use the following noTo describe algorithms . tation. Let be a parity-check matrix of a linear code Given any subset of positions, we also consider a subvector and submatrix of size . We say that is an information subset, if any two different codewords disagree on this subset. Define a sliding cyclic window

as a subset of cyclically consecutive positions beginning with . Below, we execute decoding within any position of size . the ball B. Splitting and Re-Encoding: Algorithm Consider a cyclic code . Then each sliding window forms an information subset. Since at most errors occurred on the length , at least one window is corrupted by or fewer errors. In the first step of our decoding, we try to find such a window and choose any starting point . Given an information subset , we take each subvector in the second step. Then we re-encode as an information subblock. Every newly re-encoded codeword replaces the one chosen before if it is closer to . Finally, the closest codetrials. Obviously, the decoding word is chosen after all . It is readily verified from (3) is successful within the ball has exponential order of . In a more that this number general setting (see [9] and [6]), one can consider most long linear codes instead of cyclic codes considered above. Also, to achieve exact MD decoding, we can still examine the exponenerror patterns. So MD decoding also has tial order of . complexity order bounded by C. Matching the Halves: Algorithm For simplicity, let , , and be even numbers. In and the first step, we take any starting position split our block of length onto two sliding halves and . By shifting any starting point from to , we change the number of errors on and at most by one. Also, and replace each other when is increased by . , for which both halves Therefore, there exists a partition and are corrupted by or fewer errors. located In the second step, we test all possible subvectors , and all subvectors that bewithin the ball . In doing so, we calculate the long to the ball and . Then syndromes and . we form the records

61

Here the rightmost digits form the syndromes and are regarded as the most significant digits. The next symbol is called the subset indicator. This is used to separate the records from . The leftmost digits represent original subvectors. In the third step, we consider the joint set and sort its elements as natural numbers. Then we run through and examine each subset whose records have the same rightmost digits. We say that the records syndrome in the and from form a matching pair (they disagree in the gives indicator symbol ). Then each matching pair . While examining , we also the codeword closest to and subvector closest choose the subvector gives the closest codeword to . Obviously, this pair (if any exists) for a given . Finally, we choose the codeword closest to over all syndromes and all trials . As a result, we find the closest codeword in . Remark: The above procedure is readily modified for odd , , or (see [7]). We can also include distances and in our records and use them in the sorting proceand give the closest dures. For each , the first vectors in this case. pair and We now turn to decoding complexity. The sets have the size of the exponential order according , , which is done with to (3). Then we form the sets the same complexity order. In the next step, we need to sort out natural numbers. It is well known [1] that this can be done on and depth a parallel circuit (“network”) of size . The overall complexity obtained in trials has the . In a more general setting, we consider order of lightest error patterns. In this case, we get a similar . order D. Puncturing and Matching the Halves: Algorithm Again, consider a cyclic code . To decode within a ball , we now use an integer parameter . We first of length corrupted by wish to find a subblock or fewer errors. To find such a block, we run through all possible starting points in the first decoding step, similarly to the . For simplicity, consider the length and algorithm , both being even numbers. error weight In the second step, we presumably have at most errors on and try to our subvector . Now we use a punctured code “match the halves” similarly to the algorithm . As above, we to split and onto two use trials and . To enable full dehalves 2 , we take lightest error coding within a ball and . One can readily patterns from the balls . Therefore, we can substantially reduce verify that by taking . the former complexity In our third step, we run through the set and examine each . Here, however, we substantially diverge from the subset . Namely, we take each matching pair former algorithm . It will be proven later that for most long linear 2Note that each half can have one discontinuity. However, this fact is immaterial to our analysis.

62


-codes any subblock includes an information is unambiguously subset. In this case, each pair re-encoded to the full length . Finally, we choose the codeword closest to . At this point, we note that re-encoding does not preserve disthat is reltance properties. In particular, any vector can be placed sufficiently far from after atively close to re-encoding. It is for this reason that we take all matching pairs as opposed to the algorithm that was free from re-encoding. So, the new problem that we need to address in are decoding design, is how many codewords left for re-encoding in Step 3. that fall Note that Step 2 gives all the codewords from of size . It can be proven within the ball that for most linear codes the number of these codewords is upper-bounded as (4) regardless of an output and an interval . Using (4), we can readily verify that the overall complexity is upper-bounded by

To minimize

, we take and find the optimum . In this case, all the three terms—sorting , re-encoding complexity , and complexity overall complexity —have the same order

Here we omit the proof of the bound (4). In Section VIII, a similar bound is derived in a more general setting when a ball is replaced by an arbitrary list that inmost probable vectors. cludes Remark: In essence, the bound (4) specifies the number of or less in any coset . Thus, codewords of weight we need to upper-bound the weight spectra of cosets for all and . However, the weight spectra of cyclic codes are yet unknown (let alone the spectra of their cosets or punctured codes). It is for this reason that we cannot extend algorithm to all cyclic codes as we did with former algorithms and .

At this point, however, our algorithm will depart from its MD version, which seeks a light subvector corrupted by errors. In general (see Section IV for details), we cannot find a in our soft-decision setting even if we start light subvector with a light vector . However, it will turn out that for any , at least one subvector can be fully decom. Some technical complications posed into light subblocks vectors as we did notwithstanding, we still consider . Also, these vectors can be dein hard-decision algorithm . This will be signed with the same complexity order of done in Step 1 of our algorithm. In Step 2, we try to split any “relatively light” subblock onto two parts. Though is yet unknown, we will see that it error patterns taken on can be reconstructed from about either part. In general, however, these parts can have unequal lengths. Thus, we will generalize the splitting technique used above in the algorithm . In the final part of the paper, we will derive an upper bound on decoding complexity. The paper is organized as follows. In Sections III and IV, we consider enumeration properties of vectors and subvectors . We also design relatively light subblocks. This part closely fol. In Section V, we present lows [6] and constitutes Step 1 of further properties that allow us to execute Step 2 of our algo. In Secrithm. In Section VI, we present a general algorithm tions IV–VI we also proceed with two examples that give two code. different decoding algorithms for a binary Finally, in Sections VII, VIII, and Appendix A we estimate the decoding complexity. This estimate is mostly based on combinatorial properties of our codes. The main results are already stated in Theorems 2 and 4. Here we need to simultaneously es. To do this, timate a number of codewords in each list of all possible lists to a comwe reduce the huge set3 superlists. This reducparatively small subset of at most is contained in tion is done in such a way that each list at least one superlist. On the other hand, each superlist has the . This combisame exponential size upper-bounded by natorial construction is given in Appendix A. In Appendix B, for codes of length . we consider the algorithm III. ENUMERATION ON VECTORS AND SUBVECTORS A. Soft-Decision Weights .

Consider a general memoryless channel Here

E. Framework for General Design In soft-decision decoding, we need to consider arbitrary lists of most probable inputs. In this case, different positions of the received vector are not equally reliable. Therefore, the maximum Hamming weight taken in MD decoding cannot most probable error patterns. To cirbe used for choosing cumvent this problem, we rank all the input vectors according to their posterior probabilities . Then we coninto “soft-decision” weights vert probabilities . In essence, we enumerate vectors with respect and wish to consider the lightest vecto their weights according to their probabilities tors. We also rank subvectors .

Given an output with entries

, we consider a

matrix

(5) 3Even for q = 2, there exists about 2 lists 8(y; ). These lists are closely related to the threshold Boolean functions or, equivalently, to the hyperplanes that produce different cuttings of the unit cube in the Euclidean space . Surprisingly, the problem of counting these hyperplanes was first raised over 150 years ago, and was recently solved in [17].

R


63

defined over all input symbols and all positions . Given , define its “soft-decision weight” a vector

(6) taking their weights in

We then enumerate all vectors ascending order

(7) and with the same weight Given two vectors , we use lexicographic ordering, by considering vectors as -digital -ary numbers. In the sequel, the list of most probable vectors will be regarded of lightest inputs . Below we show that as the list depends on the complexity order of designing the list its size rather than on a specific matrix . More precisely, this . We also genercomplexity has almost a linear order of any length . Our final alize this design for subvectors whose size is exgoal is to design a few punctured lists . ponentially smaller than the size of the original list for . Given any , we use a brief notation B. Enumeration and its Complexity , let denote its number (rank) Given any vector in the ordered set (7). Similarly, we take any cyclic window and order all subvectors

list of possible paths of length . We of each path. also calculate the current weight Then in any further step , we leave only shortest paths . Given , we proceed with position by in the list to any path . appending any symbol . Note that here we Then we sort paths to obtain the list prior to the step . To exclude any path obtained by prove this, consider all paths to the list . Then each of these adding the lightest suffix paths is necessarily shorter than any path obtained by extending any path . Summarizing, the procedure uses steps. Each step sorts out entries and has complexity . Therefore, at most the following estimate holds. Lemma 6 ([6]): Any list . plexity

can be constructed with com-

Remark: Bound (9) also shows that at most

paths need to be sorted in any step instead of paths sorted with rank above. Indeed, consider a symbol among all possible symbols . To obtain shortest paths , we can add only to the first paths in step . Otherwise, is excluded from the due to its higher rank list

(8) C. “Good” Subvectors Here we also use lexicographic ordering for any two subvectors has unique with the same weights. Then any subvector in ordering (8). Consider also a partition number of our subset onto two disjoint subsets and . and of Similarly to (8), we consider the ranks and taken on subsets and . subvectors Lemma 5: Any vector

satisfies the inequality (9)

, we consider a subset of vectors Proof: Given such that and . has size . Second, is the Then the set , since any other vector has either a smaller last vector in , or the same weight and a weight smaller rank, due to the lexicographic ordering of subvectors with equal weights. Therefore, ordering (8) includes at least vectors placed ahead of . Our next step is to estimate complexity of constructing the list of lightest inputs on a subset . We show that this enu. Below meration can be done with complexity of order we use the tree-like diagrams from [6] that are similar to conven. First, tional trellises. For simplicity, we take , , we recursively build the full in any step

allows us to conAny implementation of the algorithm sider only lightest inputs. The list of these inputs can be conaccording to Lemma structed with complexity of order (needed for near-ML per6. However, the condition formance) makes this complexity excessively high. Therefore, we wish to reduce this complexity by re-encoding relatively short subblocks . Correspondingly, given vectors with ranks , we wish to bound the ranks . Our next step and is to specify such a relation between the numbers . Definition 7: Given a vector are good, if block and the subvector .

we say that a subhas rank

Note that any vector is good by definition. Also, recall that when errors occur in hard-decision decoding, we consider lightest error patterns. By taking errors on the length , we consider lightest subvectors. Therefore, our good subblocks generalize the lightest error patterns for an arbitrary soft-decision setting. exist The main issue, however, is whether good subblocks and . The following lemma and corollary show that for all , the answer this takes place whenever divides . In case is slightly different. To consider both cases, let a subset be deof the composed into disjoint consecutive subsets

64


lengths . Below we use notation for a specific decomposition used on the interval . includes at least one good Lemma 8: Any good subvector on any decomposition . subvector are not good. Proof: Assume that all subblocks and Then

This contradicts our assumption that

is a good subvector.

Now we assume that divides , and decompose the whole into consecutive subsets set of length . Then Lemma 8 leads to the following corollary. , then any vector Corollary 9: If of length for some one good subvector

contains at least .

all of length . Here and include a single block, while includes two blocks of length . We prove that these three suffice to find a relatively light subblock for any vector . is a good subblock, then we are done. OthIndeed, if is bad and the remaining subinterval erwise, is good according to Lemma 8. We proceed and suppose that is also bad. Then is good and we have two good subblocks of length that form the third decomposition . Note that in the hard-decision case we also need to consider only three sliding windows. and Example 2: Let algorithm to find decompositions , , , we take the continued fraction

. We use the Euclidean : , . In other words, (12)

IV. RELATIVELY LIGHT SUBVECTORS To consider arbitrary the following way.

and , we generalize Definition 7 in

is relatively light Definition 10: We say that the subvector if all the subvectors on a decomposition are good (10) The main result of this subsection is summarized below in and any Lemma 11. We show that for any vector , there exists an interval of length and its decom. Given position that form a relatively light subvector and , let . Then we use representation (11) to get an irreducible fraction . These parameters be of major importance in our design.

and

will

of at Lemma 11 [6]: For any and , there exists a set decompositions such that any vector most has a relatively light subvector on some decomposition . Lemma 11 is discussed in detail in [6] and [4], where the set is written out explicitly. Here we omit the lengthy proof. Instead, we consider two examples that illuminate the general outline. We also use these examples in the sequel to design two -code. different implementations for a binary be a decomposition of the subblock which Let consecutive subblocks of length starts in position with and is then followed by subblocks of length and so on, until subblocks of length complete the decomposition. For brevity, this decomposition will be denoted as

Example 1: Let positions

and ,

. Consider three decom, and ,

, Then we consider decompositions: , and . Todecompositions. First, consider . If at least tally, we have is good, we are done. Otherwise, all the one subblock of length are good accomplementary subblocks cording to Lemma 8. Then we need to add only one good subblock of length to obtain at least one relatively light subblock . If all subblocks are bad, then we proceed with . and bad subblocks Having good subblocks for all , we conclude that all subblocks of length are are bad. Then all good. Now suppose that all subblocks are good. The latter means that our vector has subblocks the lightest symbols in all 63 positions. Then all the subblocks are good. This contradiction implies that at least one is good. Therefore, we can use at least for subblock one . and . Outline of the General Design: Let Then we also apply the Euclidean algorithm starting with the and . Then entries

Here for all . Finally, . It is . In the general design dealso known [12] that form different types scribed in [6] all decompositions and can start only in positions that are multiples of . It is for of different decompothis reason that the overall number . Another important observation is sitions is bounded by that each decomposition includes at most subblocks . This are multiples of . follows from the fact that their lengths . Then the number of these subintervals cannot exceed of lightest vectors, we wish To proceed with the list to reconstruct their relatively light subblocks on the length . In , we construct good subblocks with complexity order case . Now we show that relatively light subblocks can be of recovered with the same complexity. Given and , we first of decompositions satisfying Lemma 11. For find the set , we need to consider each decomposition


65

only good subvectors . Therefore, we take lightest vectors on each subinterval . According to Lemma 6, these . Then vectors can be designed with complexity order of we form the product list

that begins in the left half of smaller decompositions

. Then we split

into two

(15) , then our splitting is completed. Namely, subvectors and belong to sublists and that include at most subvectors. Otherwise, our two decompositions (15) have different and . Then we consider the lengths, say new parameter

If (13) has size , since Note that our list Now Lemmas 6 and 11 give the following statement. and , there exist at most Lemma 12: For any lists such that includes at most vectors; • each sublist has at least one subvector • any some .

.

includes the shortest subpaths Remark: Each sublist constructed on cyclically consecutive positions. At this point, our trellis-like design is similar to tail-biting trellises. In general by their soft-decision setting, we replace good subblocks . Therefore, “relatively light” concatenations can include some longer paths that have the product list . Such a disparity with the hard-decision ranks above that is case is due to the inequality used in (9) instead of the Hamming weights, which are strictly additive. However, it is yet an open problem as to whether good subblocks exist for all , , and . V. TWO EVENLY CORRUPTED SUBBLOCKS For any vector , we now consider the set of lists defined in (13). Accordingly, at least one includes a relaof length . In this section, we wish to tively light subvector into two evenly corrupted parts. More precisely, split such and of size these parts will be taken from the sublists . Again, we depart from the hard-decision algorithm about and use subintervals and that can have unequal lengths. of one inWe first consider a decomposition are separated by some position . terval . Here and Lemma 13: For any subvector least one decomposition tions

(16)

on

and any , at satisfies two condi-

with running Next, we take any splitting through the interval . According to Lemma 13, at least one splitting satisfies conditions (14) for given . For any , we from the left part to the right part. relocate the subinterval As a result, we redefine our decompositions and (17) (18) We can also include our specific case (15) in general design (17), , and . As a result, we (18) by taking . consider the coupled decomposition is fully First, we note that any coupled decomposition and the defined by the original decomposition separating point . Therefore, there exists at most different decompositions, since takes on at most different positions. For simplicity, below we use the upper bound for the number of decompositions . Next, we see that there exists a decomposition such that subvector belongs to the sublist

(19) Here we take all the has the size

values of the symbol

. Therefore,

(14) Proof: Given any interval of length , consider its subinwith the same starting point and varying length . terval We then add the next position and consider the interval of length . For , we assume that and . Note that . The is a stepwise nondecreasing function latter means that of the length . Given any , we now choose as the right. Then . Also, most position, at which , according to (9). Therefore, both inequalities (14) hold. Now we take a relatively light subvector from the list (13), . Without which is split by decomposition . Our further splitloss of generality, we suppose that ting is done in two steps. First, we find the rightmost subblock

Also, the subvector

belongs to the sublist

(20) or less. In turn, this observation This list also has size shows that both lists are designed with the same complexity that we obtained in the hard-decision case. For order of , it can be shown that the lists have size bounded odd . Thus, we arrive at the following lemma. by and Lemma 14: For any coupled decompositions

, there exist at most such that for

66


some and

both subvectors and of size or less.

belong to the sublists

Remark: Lemma 13 is yet another deviation from the hardand taken in (14) are decision case. First, subvectors and are not necessarily good. As a result, subvectors not necessarily light. However, each subinterval in (17) and (18) includes only the lightest subvectors. Also, the product lists and still have the size used in hard-decision decoding. It is for these two reasons that we can apply Lemma 6 and keep . the former complexity order of VI. GENERAL ALGORITHM To present general algorithm known lemma (e.g. see [6]).

, we need the following well-

of length form Lemma 15: All sliding intervals if information subsets for most linear codes . Proof: Let be a generator matrix of code . An interval is not an information subset if submatrix has rank less than . Then we count the fraction of such (deficient) submatrices and show that for any . So, the fraction of mawith at least one deficient submatrix is bounded trices and vanishes for any used in the statefrom above by . ment. Thus, our lemma holds for most linear codes General Algorithm Preliminary Step: Given and , we choose the parameters and . Then we take a linear code that satisfies Lemma 15. In particular, one can choose any cyclic decompositions code. Given and , we find at most of length that satisfy Lemma 12. For each , we take the rightmost interval that begins on the left half of and find the corresponding parameter from (16). We take different positions, and furany separation point using into two parts and . Now we have at most ther split coupled decompositions defined in have dimen(17) and (18). Also, the punctured codes sion on any . Our decoding begins when we receive an output . Then we of any input symbol in each position define the weight and form the corresponding matrix . Decoding includes three steps. Step 1: “Splitting.” We choose any coupled decomposition found in the preliminary step. Step 2: “Designing of Plausible Candidates.” Given a decomposition , we construct the product lists and defined in (19) and (20). Recall that only the lightest subvectors are taken on each subinterval . From this point, our design follows its hard-decision counterpart presented in Section II. Namely, and we calculate the syndromes for all subvectors and . and Then we form the records . Step 3: “Matching and Re-Encoding.” We form the joint and sort its elements. Then we set

run through and find all matching pairs that agree in rightmost digits and disagree in the indicator symbol . Each matching pair yields . Each codeword is the codeword of length re-encoded onto the codeword . Then the weight is found. Finally, we choose the lightest codeword after all decompositions are considered. to a binary -code. We We now apply algorithm and obtain decoding error probability take that at most doubles the error probability of ML decoding. We and , complete Examples 1 and 2 by taking respectively. According to Lemma 14, we use at most subvectors for and subvectors for . This compares favorably with all other near-ML algorithms known to date. In particular, trellis complexity is upper-bounded by states while near-ML decoding from [6] requires states. Example 1 (Continued): We consider three decompositions: , , and of and are bad, in length . First suppose that both intervals consists of good halves and which case . On either half, we design lightest vectors of length using Lemma 6. Then we calculate their syndromes and sort and . Each matching codeword of the records is re-encoded to the full length and the closest codelength word is chosen. (or , our design is similar. In case of decompositions should include According to Lemma 13, both parts and vectors. This allows us to use 19 different decompositaking any . (Indeed, we cannot tions vectors on for . Nor can we do this construct for Then we follow the previous design peron formed on . Example 2 (Continued): We use decompositions , , and . Then each , with subvectors to be decomposition is split into , we decompose a considered on either part. When using with sliding point . single block into two parts Then we execute matching and re-encoding procedures. For , , and , we see that the first subblock includes the left half of the entire interval . This sub, with separation point block is split into two parts taking 12 rightmost positions on . On the right part , we take the product of multiple lists (20). Then we match the with the synsyndromes obtained on the left interval dromes found on the remaining positions. Finally, we proceed with re-encoding. Examples 1 and 2 (Completed): In both examples, we can consequently store and use the same trellis diagrams for difwith sliding point . In particferent decompositions , we ular, every time the separation point changes from to . Similarly, can proceed forward while constructing the list . This allows us to we proceed backward while constructing and . Therefore, we concuruse already presorted lists rently form two trellis diagrams obtained on length by moving


67

Fig. 1. Output bit error rate for the (63; 30) BCH code with decoding list of size T .

in two opposite directions. Our overhead includes sorting procedures performed on each step to reduce by half the number of shortest paths presorted in the previous step. The critical adpaths on each step instead vantage is that we use at most trellis paths used in conventional design. of up to Note that in the asymptotic setting, the number of coupled is immaterial. However, the decompositions above examples show that this number can be significant on short lengths. In particular, our design on the length uses only decompositions employed three decompositions instead of . On the other hand, we increase our lists only about for times. Therefore, Example 1 is a better choice. Similar improvements can be obtained for other codes of length . These are discussed in Appendix B. Example 1 becomes even more favorable when compared with near-ML decoding of [6]. The latter requires that lightest vectors be constructed on each of 63 different decompositions. It is also important that the practical design can be simplified further. For example, computer simulation has shown that the decoding error probability is left almost subvectors instead of examining unchanged if we use only or more of them. These results are shown in Fig. 1. VII. DECODING COMPLEXITY A. Preliminary Remarks To proceed with the asymptotic setting, we recall that our algorithm gives near-ML decoding if it recovers any codeword . The following lemma shows that does so. recovers each codeword Lemma 16: Algorithm from the list of most probable input vectors. . According to Proof: Consider any codeword Lemma 14, at least one coupled decomposition places both and into the sublists and . In turn, subvectors

and are matched in Step 3, since their combination belongs to the code . Finally, vector is obtained by re-encoding. To proceed with decoding complexity, we need to obtained upper-bound the number of codewords in Step 3. These codewords belong to the combined list obtained by linking the two parts and . into the three subintervals Recall that we split the interval and insert these intervals in our original decompo. To simplify the notation, below we call sition and respectively. Then these subintervals the combined list can be presented in the form (21) for (also, , , , Here . It is important that all the numbers are and . Also, fully defined by our original decomposition we will use the fact that (22) Now we use the restriction (11) and consider the asymptotic and . Here we keep the setting with fixed as . Then we consider all generator ratio and rank . The ensemble of codes matrices of size is defined by the uniform distribution taken on matrices . In total, we consider

codes. Finally, we define the maximum number of codewords obtained in Step 3 as

68


Recall that every list depends on the output (or the corresponding matrix ). Therefore, the above maximum is taken , decompositions of length , and matrices over all codes . Here we consider as a function of and will minimize later. Also, we use the fact that the lists have or less. size B. General Estimate Given parameters mate that depends on

of codewords re-encoded in Step 3. Also, the overall fraction of bad codes is defined only by Lemma 15, due to the eliminated in Lemma 22. For the immaterial fraction , it can be readily verified that this fraction optimum has exponentially small order of . For larger used in Theorem 2, the fraction of bad codes declines even faster. Now we need to prove bound (23) to complete our analysis. This is done in the remaining part of the paper.

and , we first give a general estiVIII. BOUNDS ON THE NUMBER OF CODEWORDS IN THE LISTS

.

Lemma 17: Algorithm

A. Fixed Output

has complexity

To find , we first fix an output and the corresponding -matrix . Given , we consider the maximum number of codewords where is the number of decompositions used in the first step; is full complexity of the second step; is re-encoding complexity of the third step. Proof: According to Lemma 14, the number of coupled . Since decompositions is upper-bounded by is fixed, . In Step 2, we form sublists and link (19) and (20) of size . them into the two lists includes only the shortest vectors and can be Each sublist , according to Lemma 6. designed with complexity Therefore, the overall complexity taken over all sublists is . The same complexity order upper-bounded by and . Therefore, is required to calculate all syndromes . Step 2 has complexity In Step 3, we sort out all the records of the combined list . Following Section II, we use a parallel circuit of size and depth . This gives complexity that is below . Then we run through the precodewords of the code . sorted set and find at most . We also take into Their re-encoding has complexity trials are executed in Steps 2 and 3. account that at most Proof of Theorem 2: Below in Lemma 22 we will prove that the upper bound

taken over all decompositions

and all codes

. We then show

fails to hold only that the upper bound for a small fraction of all codes that declines as rapidly as . We then wish to use the union bound while at the same time. To do this, we considering all real matrices by a comparatively small subset will replace the full set that includes only about matrices . For each matrix , we will also design a superlist that yet has the same as any original list . Howexponential size is fully contained in ever, we then prove that each list . This will allow us to use the at least one of superlists . As a result, the bound (23) fails union bound in the form of all codes. From to hold only for the vanishing fraction now on, we use the following parameters: (24) -codes, with the excepLemma 18: Most long linear of them, have tion of the fraction (25)

(23) holds for most long linear codes, with the exception of immate. By taking in Lemma rial fraction vanishing as 17, we readily obtain complexity

of lightest inputs. codewords in any list Proof: The proof is similar to that in [3, Proposition 2]. of linearly indeConsider all combinations . The number of these pendent vectors chosen from combinations is bounded as

where It is easy to verify that achieves its minimum if for any . Otherwise, we obtain a lower at . This proves Theorem 2. To value obtain Corollary 3, we choose a sequence of that tends to . In this case and . We conclude that the same upper bound can be used for the number of lightest subvectors taken in Step 2 and the number

(26) Since same order number of

, we verify that the lower bound has the as . We then find the -codes that include any specific combination . This number is


69

In the asymptotic equality we also use the fact that . belongs to Therefore, any given combination the fraction

B. Bounding the Number of Codewords: All Outputs Now we have an exponential order of codewords obtained for all decompositions . However, we still cannot for all matrices . Therefore, we use the union bound by a comparatively small subset wish to replace a huge set . In doing so, we also increase our subsets times, where (28)

of codes. Now we find the expected number of combinations taken over all codes from . This is,

Given

, we say that code is “bad” if it includes at least combinations. By the Chebyshev . Now we inequality, the fraction of these codes is includes fewer than vectors see that any remaining code , where from

Indeed, otherwise the number of possible combinations taken from exceeds . Here our arguments are similar to (26), since is an exponent in and . In turn, as . This shows that any that includes vectors is bad. Finally, note that gives the right-hand side of inequality (25) for the given from (24). This lemma is immediately generalized for all decomposi. tions and punctured codes Lemma 19: Most long linear equality

-codes satisfy the in-

(27) of codes. with the exception of the fraction Proof: Given a coupled decomposition and an output , we specify the list obtained in (21) and (22). This list or fewer vectors. Let denote the subset includes that give an -subcode of some dimension of codes , when punctured to the interval . Note that in the asymptotic setting, we have the inequality as . , we then apply the same arguments that were Given a list to obtain the fraction used in Lemma 18. We fix of bad codes. We also replace by and by . instead of . Then For brevity, we also use the term of bad -subcodes, which do not we find the fraction for all . satisfy inequality (27). We see that -subcode is Second, given and , we note that each obtained from the same number of the original codes . Therethat fail to satisfy (27) is fore, the fraction of bad codes for each . Now we have the also upper-bounded by of bad codes for any given decomfraction position . The proof is completed by using the union bound taken over decompositions .

that include lightest Namely, we consider the lists inputs taken with respect to . The main result of this step is the following theorem. matrices such that Theorem 20: There exists a set of is contained in at least one superlist . any list This theorem proved in Appendix A allows us to represent all by only superlists of the same exponential lists size . Now we combine Theorem 20 with Lemma 18. Corollary 21: Most long linear

codewords in each of the lists

-codes have

of

inputs, with the

of codes as . exception of the fraction , we increase times Proof: By taking any list from (25). On the other hand, the our upper bound on of bad codes is at most and still declines fraction . This proves our corollary for all lists . as Then the same fraction of codes is obtained for their enclosures . (sublists) In a similar way, we can consider any decomposition that subintervals. In doing so, we replace any sublist includes by a bigger list . The main problem becomes here is that the original product list times larger. Therefore, we take into account that is a , , and . The result constant as long as is the following lemma. Lemma 22: For give

, most linear

-codes of length

(29) in each of the lists punctured codewords of length , with the exception of the fraction of codes. decomposiProof: Given and above, we have intervals , where . Then tions . For any we take we apply Theorem 20 on any single interval . This allows us by superlists . Their to replace all lists on a single interval , and by , number is bounded by when all intervals are combined into . Original lists have size , according to (22). Then we consider obtained by linking the lists . the supersets These supersets have size

70


of bad codes as . This get the fraction . Then the same fraction proves our lemma for superlists . of codes can be taken for their enclosures

Lemma 23: All vectors have rescaled weight . includes only the vectors with Proof: The list and has size or more, according total weight includes lightest vectors. Thus, to (31). The list for any . .

IX. CONCLUDING REMARKS

let be the maximum weight of its vecFor any list and .) Then we define the closure tors. (This depends on

Given any superset , we can apply Lemma 19. Then . bound (29) fails to hold only for the fraction of codes supersets and decompositions, we still By taking all

In this paper, we study near-ML decoding algorithms. These algorithms achieve block error probability similar to that of ML ratio. We present a near-ML decoding decoding at any that decodes the original -code by comalgorithm paring a number of decoding results obtained on the punctured -codes. The algorithm employs trellis-like design and can be used for any linear code regardless of specific code structure. For most long linear codes, the algorithm achieves complexity along with decoding error probability order of of ML dethat is arbitrarily close to the error probability coding. Thus, we reduce up to three times the asymptotic comof trellis design. The algorithm also proplexity vides for significant complexity improvements even on the relatively short lengths. In particular, for binary linear codes of length , the algorithm uses fewer than three thousand states and reduces general trellis complexity by three to six decimal orders.

.

has size Lemma 24: Any closure and is contained in . includes only Proof: Subset vectors of maximum weight . For each vector from , we by the lightest replace the first (leftmost) symbol of “replaced” vectors, each vector symbol In this new set vectors . Then this set is obtained from at most has size

On the other hand,

belongs to our subset .

. Thus,

We now define the list with a given maximum weight . Then we study its size a function of (for brevity, we omit the argument ).

APPENDIX A UPPER BOUNDS ON THE NUMBER OF SUPERLISTS We consider all possible matrices . For each position , let where lightest entry. Consider the new weight

Below

, be the

Lemma 25: Consider a list has size the list

of size

as . Then (32)

Then all vectors

have their weights

equally changed. This shift keeps the same ordering of vectors . Therefore, we will consider only matrices in any list with zero entries . matrices Second, we further redefine in the following way. Given any threshold , let be in the column . Also, define the number of entries the subset of vectors

with all symbols of weight or less. Then the size of this set . We say that is a step if is for any . The latter means that at least one entry is equal to . In total, our function has at most steps. Given , we use the two adjacent steps

be the subset of vectors in , that Proof: Let in position . Let . The have weight includes the vectors with all complementary set . Therefore, this set is contained symbols of weight from (31), since is the weight closest in the set . Now we take any to from below. Then and replace the leftmost symbol with the weight by the lightest symbol . Similarly to the previous proof, we of “replaced” vectors has size see that the new set

On the other hand, , and (32) holds. Hence

for any

.

Proof of Theorem 20: Now we design a discrete set of at superlists , where is already defined most numbers in (28). We consider the set of

(30) Below we consider the rescaled entries any vector has rescaled weight our two adjacent steps and are now converted to Thus, we have the inequalities

. Then . Also, and . (31)

For any given matrix , let matrix be obtained to the closest number . by rounding down all entries in nonzero entries in can be rounded Since each of levels, there exist matrices . to any of


For brevity, we use a greater number . Now consider a list with the maximum weight and define its superlist

We wish to prove that any superlist includes at most vectors . Consider any and estimate its “un. First, note that , according rounded” weight includes only vectors to Lemma 23. Therefore, the list for all whose rounded weights satisfy inequalities cannot expositions . In this case, unrounded weights , according to our choice of the rounding set . Now ceed for all posiwe see that the rounding error does not exceed satisfies the inequality tions. Therefore, any

Then

. Now we use Lemma 25 to see that

On the other hand, is the closure of our original , since is the maximum weight in . list . Therefore, According to Lemma 24,

Now we see that any list is contained in the superlist . Finally, note that each superlist is fully defined by the corresponding matrix . Then the number of su, and the proof is completed. perlists is upper-bounded by APPENDIX B DECODING DESIGN FOR CODES OF LENGTH 1) We start with -code. Given , we need . We take . Then to choose near-optimum of length we use only six different blocks , where . If all six blocks are bad, we have good complementary blocks of length . These give us . For all seven one more decomposition in two decompositions, we split a subblock of length trellis paths in parts and seek the way we did in Examples 1 and 2. -code. Then and . 2) Consider now Here we can consider only six different decompositions using the following design. First, we take four different , where . If all blocks four decompositions are bad, then all complementary subof length are good. Given a blocks good subblock , we have at least one good half: either or . Then we add two decompoand . sitions This example is somewhat different from the previous does not include all cycliones, since decomposition cally consecutive positions. -code, and . Here we 3) For can use only five different decompositions. First, we take

71

three subblocks with . If these are bad, then the complementary subblocks of length are good. Following the previous into two example, we decompose the first subblock halves and . Then we complete our design by adding and . two decompositions -code, and . Here we first 4) For with . If take two subblocks both decompositions are bad, then both complementary are good. Subblock is then subblocks and of decomposed into three subintervals length . At least one of the subintervals is good. We add each subinterval to . As a result, we can use only five , , . decompositions: , , and -code, and . Here we 5) For with first take four subblocks . If all decompositions are bad, we have four complementary subblocks of length starting with the . All four are good, and form the interval last (fifth) decomposition. REFERENCES

O

[1] M. Ajtai, J. Komlós, and E. Szemeredi, “An (n log n) sorting network,” in Proc. 15th Annu. ACM Symp. Theory of Computing, 1983, pp. 1–9. [2] L. R. Bahl, J. Cocke, F. Jelinek, and J. Raviv, “Optimal decoding of linear codes for minimizing symbol error rate,” IEEE Trans. Inform. Theory, vol. IT-20, pp. 284–287, Mar. 1974. [3] A. M. Barg and I. I. Dumer, “The algorithm of cascade decoding with incomplete inspection of code vectors” (in Russian), Probl. Pered. Inform., vol. 22, no. 1, pp. 3–10, 1986. [4] A. Barg, “Complexity issues in coding theory,” in Handbook of Coding Theory, V. Pless and W. C. Huffman, Eds. Amsterdam, The Netherlands: Elsevier, 1998. [5] V. M. Blinovskii, “Lower asymptotic bound on the number of linear code words in a sphere of given radius in F ” (in Russian), Probl. Pered. Inform., vol. 23, no. 2, pp. 50–53, 1987. [6] I. Dumer, “Suboptimal decoding of linear codes. Partition technique,” IEEE Trans. Inform. Theory, vol. 42, pp. 1971–1986, Nov. 1996. , “Sort-and-match algorithm for soft-decision decoding,” IEEE [7] Trans. Inform. Theory, vol. 46, pp. 2333–2338, Sept. 1999. , “On minimum distance decoding of linear codes,” in Proc. 5th [8] Soviet–Swedish Workshop on Information Theory, Moscow, USSR, Jan. 1991, pp. 50–52. [9] G. S. Evseev, “On the complexity of decoding linear codes” (in Russian), Probl. Pered. Inform., vol. 19, no. 1, pp. 3–8, 1983. [10] T. Fujiwara, H. Yamamoto, T. Kasami, and S. Lin, “A trellis-based recursive maximum-likelihood decoding algorithm for binary linear block codes,” IEEE Trans. Inform. Theory, vol. 44, pp. 714–729, Mar. 1998. [11] R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968. [12] D. E. Knuth, The Art of Computer Programming. Reading, MA: Addison-Wesley, 1969, vol. 2, Seminumerical Algorithms. [13] A. Lafourcade and A. Vardy, “Lower bounds on trellis complexity of block codes,” IEEE Trans. Inform. Theory, vol. 41, pp. 1938–1952, Nov. 1995. [14] A. Vardy and Y. Be’ery, “Maximum-likelihood soft decoding of BCH codes,” IEEE Trans. Inform. Theory, vol. 40, pp. 546–554, Mar. 1994. [15] J. K. Wolf, “Efficient maximum likelihood decoding of linear codes using a trellis,” IEEE Trans. Inform. Theory, vol. IT-24, pp. 76–80, Jan. 1978. [16] V. V. Zyablov and V. R. Sidorenko, “Bounds on complexity of trellis decoding of linear codes” (in Russian), Probl. Pered. Inform., vol. 29, no. 3, pp. 3–9, 1993. [17] Yu. A. Zuev, “Combinatorial-probabilistic and geometrical methods in threshold logic” (in Russian), Discr. Math., vol. 3, no. 2, pp. 47–57, 1991.