Daniel J. Costello, Jr. · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: From Theory to Practice University of Notre Dame BIRS Workshop on

Daniel J. Costello, Jr.

Dept. of Electrical Engineering,

Spatially Coupled LDPC Codes:From Theory to Practice

University of Notre Dame

BIRS Workshop on Mathematical Coding Theory in Multimedia Streaming

Research Collaborators: David Mitchell,Michael Lentmaier, and Ali Pusane

D. J. Costello, Jr., “Spatial Coupling vs. Block Coding: A Comparison” / 39

Outline

LDPC Block Codes

Spatially Coupled LDPC Codes

Parity-check matrix and Tanner graph representations, minimumdistance bounds, iterative decoding thresholds, protograph-based constructions

Protograph representation, edge-spreading construction, termination

Iterative decoding thresholds, threshold saturation, minimum distance

Practical Considerations

1

Window decoding; performance, latency, and complexity comparisonsto LDPC block codes; rate-compatibility; implementation aspects


LDPC Block Codes

Definition by parity-check matrix: [Gallager, '62]

Code:(J,K)-regular LDPC

block code:

2

Bipartite graph representation: [Tanner, '81]

n = 20 variable nodes of degree J = 3

l = 15 check nodes of degree K = 4


LDPC Block Codes

Definition by parity-check matrix: [Gallager, '62]

Code:(J,K)-regular LDPC

block code:

2

Bipartite graph representation: [Tanner, '81]

n = 20 variable nodes of degree J = 3

l = 15 check nodes of degree K = 4

Graph-based codes can be decoded iteratively with low complexity by

exchanging messages in the graph using Belief Propagation (BP).


For an asymptotically good code ensemble, the minimum distance grows linearly with the block length n

Minimum Distance Growth Rates of

(J,K)-Regular LDPC Block Code Ensembles

3


where is called

the typical minimum

distance ratio, or

minimum distance

growth rate.




3

(J,K)-regular block codeensembles areasymptotically good, i.e.,


As the density of (J,K)-regular ensembles increases,approaches theGilbert-Varshamovbound.

where is called

the typical minimum

distance ratio, or

minimum distance

growth rate.




3

(J,K)-regular block codeensembles areasymptotically good, i.e.,


[RU01] T. J. Richardson, and R. Urbanke, “The capacity of low-density parity-check codes under message

passing decoding”, IEEE Transactions on Information Theory, vol. 47 no. 2, Feb. 2001.

AWGNC thresholdsBEC thresholds

Iterative decoding thresholds can be calculated for (J,K)-regular

LDPC block code ensembles using density evolution (DE).

Thresholds of (J,K)-regular LDPC

Block Code Ensembles

4




There exists a relatively large gap to capacity.






4




There exists a relatively large gap to capacity.






Iterative decoding thresholds get further from capacity as the graph

density increases.

4


Large LDPC codes can be obtained from a small base parity-checkmatrix B by replacing each nonzero entry in B with an M x M permutation matrix, where M is the lifting factor.

5

Protographs (Matrix Description)



length 6M = 24rate R = 1/2

Example: Irregular code with M = 4

5




length 6M = 24rate R = 1/2

Example: Irregular code with M = 4

5

Irregularcodes havevariable rowand columnweights(check nodeand variablenode degrees)



Protographs are often represented using a bipartite Tanner graph

Protographs (Graphical Description)

3 check nodes

6 variable nodes

[Tho05] J. Thorpe, “Low-Density Parity-Check (LDPC) codes constructed from

protographs”, Jet Propulsion Laboratory INP Progress Report, Vol. 42-154 Aug. 2003.

6


Protographs are often represented using a bipartite Tanner graph

Protographs (Graphical Description)

3 check nodes

6 variable nodes

The collection of all possible parity-check matrices with lifting factor M forms a code ensemble, where all the codes share a common structure

[Tho05] J. Thorpe, “Low-Density Parity-Check (LDPC) codes constructed from

protographs”, Jet Propulsion Laboratory INP Progress Report, Vol. 42-154 Aug. 2003.

6


.Quasi-cyclic (QC) LDPC codes are of great interest in practice, since

they have efficient encoder and decoder implementations

Quasi-Cyclic LDPC Codes

7




Example: protograph construction of a (2,3)-regular QC-LDPC block code


7




Example: protograph construction of a (2,3)-regular QC-LDPC block code

For QC codes, the permutationmatrices are shifted identities


7


[DDJA09] D. Divsalar, S. Dolinar, C. Jones, and K. Andrews, “Capacity-approaching protograph

codes”, IEEE Journal on Select Areas in Communications, vol. 27, no. 6 Aug. 2009.

Multi-Edge Protographs

Protographs can have repeated edges (corresponding to integer valuesgreater than one in B)

Note that this makes nosense without lifting

8






Repeated edges in aprotograph correspond tousing sums of permutationmatrices to form LDPC codeensembles


8






Repeated edges in aprotograph correspond tousing sums of permutationmatrices to form LDPC codeensembles

denser graphs!

can also be QC (using circulant matrices)!


8


2e variablenodes

Rate Threshold Capacity Distancegrowth

rate

1/2 0.628 0.187 0.01450

2/3 1.450 1.059 0.00582

3/4 2.005 1.628 0.00323

4/5 2.413 2.040 0.00207

5/6 2.733 2.362 0.00145

6/7 2.993 2.625 0.00108



'Good' Protographs

Ensemble average properties can be easily calculated from a protograph,thus simplifying the construction of 'good' code ensembles.

Iterative decoding thresholds close to capacity for irregular protographs

Minimum distance growing linearly with block length (asymptotically good) for regular and some irregular protographs

9


Outline




LDPC Block Codes


10




Spatially Coupled Protographs

...

Consider transmission of consecutive blocks (protograph representation):

... (3,6)-regularLDPC-BC

base matrix

11



...



base matrix

Blocks are spatially coupled (introducing memory) by spreading edges

over time:

11



...



base matrix

Blocks are spatially coupled (introducing memory) by spreading edges

over time:

Spreading constraint: ( has size )

11


Transmission of consecutive spatially coupled (SC) blocks results in aconvolutional protograph:


12




12




12



... ...


12



... ...

The bi-infinite convolutional protograph corresponds to a bi-infiniteconvolutional base matrix:


Constraint length:

Rate:

12


SC-LDPC Code Ensembles

An ensemble of (3,6)-regular SC-LDPC codes can be created from theconvolutional protograph by the graph lifting operation

13




13




Graph lifting: is an permutation matrix

13




Graph lifting: is an permutation matrix

If each permutation matrix is circulant, the codes are quasi-cyclic

13


Code rate:

Consider terminating to a (block code) base matrix of length Lbv:

Terminated Spatially Coupled Codes

14


Code rate:

For large L, RL approaches the unterminated code rate .



14


Example: (3,6)-regular base matrix , ms = 2, L = 4, R4 = 1/4

Code rate:



(check node degrees lowerat the ends)


14


Example: (3,6)-regular base matrix , ms = 2, L = 4, R4 = 1/4

Code rate:



(check node degrees lowerat the ends)


Codes can be lifted to different lengths and rates by varying M and L .

14


Wave-like Decoding of Terminated

Spatially Coupled Codes

Variable nodes all have the same degree as the underlying block code.

Check nodes with lower degrees (at the ends) improve the BP decoder.

...

13






10 20 30 40 50 60 70 80 90 10010-6

10-4

10-2

100

10 iterations

t

p

Evolution of message probabilities: (3,6)-regular SC-LDPC code (L = 100)

13







10 20 30 40 50 60 70 80 90 10010-6

10-4

10-2

100

20 iterations

t

p

13







10 20 30 40 50 60 70 80 90 10010-6

10-4

10-2

100

50 iterations

t

p

13







10 20 30 40 50 60 70 80 90 10010-6

10-4

10-2

100

100 iterations

t

p

13







10 20 30 40 50 60 70 80 90 10010-6

10-4

10-2

100

200 iterations

t

p

13







10 20 30 40 50 60 70 80 90 10010-6

10-4

10-2

100

300 iterations

t

p

13







10 20 30 40 50 60 70 80 90 10010-6

10-4

10-2

100

340 iterations

t

p

13







10 20 30 40 50 60 70 80 90 10010-6

10-4

10-2

100

340 iterations

t

p

Note: the fraction of lower degree nodes tends to zero as i.e., the codes are asymptotically regular.

13


Density evolution can be applied to the protograph-based ensembles

with [Sridharan et al. '04]:

Thresholds of Terminated


Example: BEC

16


L=4, gap=0.115





Example: BEC

16


L=4, gap=0.115





Example: BEC

L=10, gap=0.095

16


L=4, gap=0.115





Example: BEC

L=10, gap=0.095

(3,6)-regular block code:

16


BEC AWGN

Iterative decoding thresholds (protograph-based ensembles)

We observe a significant improvement in the thresholds of SC-LDPC codes compared to the associated LDPC block codes (LDPC-BCs) due to the lowerdegree check nodes at the ends of the graph and the wave-like decoding.

17



[LSCZ10] M. Lentmaier, A. Sridharan, D. J. Costello, Jr., and K.Sh. Zigangirov, “Iterative decoding threshold

analysis for LDPC convolutional codes,” IEEE Trans. Inf. Theory, 56:10, Oct. 2010.

SC


BEC AWGN

Iterative decoding thresholds (protograph-based ensembles)

We observe a significant improvement in the thresholds of SC-LDPC codes compared to the associated LDPC block codes (LDPC-BCs) due to the lowerdegree check nodes at the ends of the graph and the wave-like decoding.

In contrast to LDPC-BCs, the iterative decoding thresholds of SC-LDPC codes improve as the graph density increases.

17





SC


When symbols are perfectly known (BEC), all adjacent edges can be removedfrom the Tanner graph.

Why are Terminated Spatially Coupled

Thresholds Better?

...

18




Thresholds Better?

...

18




Thresholds Better?

...

18




Thresholds Better?

...

18




Thresholds Better?

...

18




Thresholds Better?

...

The threshold saturates (converges) to a fixed value numericallyindistinguishable from the maximum a posteriori (MAP) threshold of the (J, K)-regular LDPC-BC ensemble as [LSCZ10].



18




Thresholds Better?

...

The threshold saturates (converges) to a fixed value numericallyindistinguishable from the maximum a posteriori (MAP) threshold of the (J, K)-regular LDPC-BC ensemble as [LSCZ10].



For a more random-like ensemble, this has been proven analytically, first forthe BEC [KRU11], then for all BMS channels [KRU13].

[KRU11] S. Kudekar, T. J. Richardson and R. Urbanke, “Threshold saturation via spatial coupling: why

convolutional LDPC ensembles perform so well over the BEC”, IEEE Trans. on Inf. Theory, 57:2, 2011

[KRU13] S. Kudekar, T. J. Richardson and R. Urbanke, “Spatially coupled ensembles universally achievecapacity under belief propagation”, IEEE Trans. on Inf. Theory, 59:12, 2013.

18


Threshold Saturation (BEC)

0.32 0.34 0.36 0.38 0.4 0.42 0.44 0.46 0.48 0.5

10-4

10-2

100

epsilon

Bit e

rasure

rate

BP

threshold

MAP

threshold

(3,6)(3,6)

BC

19

BP = iterative (suboptimal) decoding thresholdMAP = (optimal) maximum a posteriori threshold



0.32 0.34 0.36 0.38 0.4 0.42 0.44 0.46 0.48 0.5

10-4

10-2

100

epsilon

Bit e

rasure

rate

(3,6)

MAP MAPBP BP

(3,6)

BC

19

(5,10)

BC




0.32 0.34 0.36 0.38 0.4 0.42 0.44 0.46 0.48 0.5

10-4

10-2

100

epsilon

Bit e

rasure

rate

(3,6)

BC

MAP MAPBP BP

SC-LDPC

codes

19

(5,10)

BC




0.32 0.34 0.36 0.38 0.4 0.42 0.44 0.46 0.48 0.5

10-4

10-2

100

epsilon

Bit e

rasure

rate

(3,6)

BC

MAP MAPBP BP

SC-LDPC

codes

19

(5,10)

BC


optimal decoding performance with a suboptimal iterative algorithm!


Threshold Saturation (AWGNC)

BPMAP

(3,6)-regularblock code

capacity

~0.5dB

20




BPMAP



BPMAP

capacity

~1.25dB

20




BPMAP



BPMAP

capacity

spatially coupledcodes

(3,6)(4,8)

20




optimal decoding performance with a suboptimal iterative algorithm!

BPMAP



BPMAP

capacity

spatially coupledcodes

(3,6)(4,8)

20



BEC Thresholds vs Distance Growth

By increasing J and K, we obtain capacity achieving (J,K)-regular SC-LDPC code ensembles with linear minimum distance growth.

21

-BC

(J,K)-regular SC-LDPC codes combine the best features of irregularand regular LDPC-BCs, i.e., capacity approaching thresholds and lineardistance growth.


AWGNC Thresholds vs. Distance Growth

[MLC10] D. G. M. Mitchell, M. Lentmaier and D. J. Costello, Jr., “AWGN Channel Analysis of Terminated

LDPC Convolutional Codes”, Proc. Information Theory and Applications Workshop, San Diego, Feb. 2011.

Similar results are obtained for the AWGNC

22


Distance Measures for SC-LDPC Codes

As the minimum distance growth rates of terminated SC-LDPCcode ensembles tend to zero. However, the free distance growth rates ofthe unterminated ensembles remain constant.

23

(3,6)-regular unterminated SC-LDPC free distance growth rates

(3,6)-regular terminated SC-LDPCminimum distance growth rates




23

For large L, thestrength ofunterminatedensembles scaleswith theconstraint length and isindependent of L.

independent of L

normalized by L






23

For large L, thestrength ofunterminatedensembles scaleswith theconstraint length and isindependent of L.

An appropriatedistance measurefor 'convolutional-like' terminatedensembles shouldbe independent of L.

independent of L

normalized by L




Outline

LDPC Block Codes





24



D. J. Costello, Jr., “Spatial Coupling vs. Block Coding: A Comparison” / 39 25

Decoding SC-LDPC Codes

SC-LDPC codes can be decoded with standard iterative decoding schedules.

me

ssage

s p

assed




Reliable messagesfrom the endspropagate throughthe graph toward thecenter as iterationsproceed.

me

ssage

s p

assed





me

ssage

s p

assed





me

ssage

s p

assed





The highly localized (convolutional) structure is well-suited for efficient

decoding schedules that reduce memory and latency requirements.

me

ssage

s p

assed





Sliding window decoding (WD) updates nodes onlywithin a localized windowand then the window shiftsacross the graph [Lentmaieret al '10, Iyengar et al '12].



me

ssage

s p

assed

width W








me

ssage

s p

assed

width W








me

ssage

s p

assed

width W


Window Decoding Performance

[LPF11] M. Lentmaier, M. M. Prenda, and G. Fettweis, “Efficient Message Passing Scheduling for

Terminated LDPC Convolutional Codes”, Proc. IEEE ISIT, St. Petersburg, Russia, July 2011.

Latencies:LDPC:SC-LDPC:

26






For equal liftingfactors, SC-LDPCcodes display alarge convolutionalgain at the cost ofincreased latency.

convolutional gain

26







For equal latency,SC-LDPC codesstill display asignificantperformance gain.

equallatency

26








equallatency

26








equallatency

26

Trade-off in M vs W


Required to achieve a BER of as a function of latency:

decreases as W (and thus thelatency) increases.

does not decreasesignificantly beyond acertain W

Equal Latency Comparison for

(3,6)-Regular LDPC Codes

BC

SC


27







BC

SC

SC


27







BC

SC

SC

SC


27







BC

SC

SC

SC

SC


27







BC

SC

SC

SC

SC

SC


27







BC

SC

SC

SC

SC

SC

SC


27







large improvescode performance.

large W improvesdecoder performance.

When choosingparameters:

BC

SC

SC

SC

SC

SC

SC


27


Complexity Tradeoffs

For equal latency, SC-LDPC codes display a performance gain comparedto the underlying LDPC-BCs

(including non-binary codes)




With standardstopping rules, thecomputationalcomplexity ishigher for SC-LDPC codes





With standardstopping rules, thecomputationalcomplexity ishigher for SC-LDPC codes

LDPC-BCs cannot achieve equalperformance byincreasing thenumber ofiterations



For equal performance (BER of 10-5 at 1.5dB), SC-LDPC codes display alarge reduction in latency compared to LDPC-BCs of similar complexity

Complexity/Latency Tradeoffs






With increasing(small) field sizes q,latency decreasesfor increasingcomplexity






For larger q, bothlatency and complexityincrease (for both SC-LDPC codes andLDPC-BCs)!






SC-LDPC codesover GF(4) offer agood balancebetween complexityand latency

For larger q, bothlatency and complexityincrease (for both SC-LDPC codes andLDPC-BCs)!


Rate-compatible Punctured Codes

A linear code with rate R is punctured by removing a set of p columnsfrom its generator matrix, reducing the codeword length from n to

x x

30




x x

A variety of code rates can be achieved using the same decoder bypuncturing different numbers of symbols (rate compatibility)

30




x x

A variety of code rates can be achieved using the same decoder bypuncturing different numbers of symbols (rate compatibility)

puncturing fraction:

punctured rate:

The code rate is increased by puncturing:

30


Randomly Punctured

SC-LDPC Codes (BEC)

M = 500, W=8Latency = 2MW = 8000

Randomly punctured SC-LDPC codes drawn from

Max its. = 10

31


Randomly Punctured

SC-LDPC Codes (BEC)

M = 500, W=8Latency = 2MW = 8000


Max its. = 10

Robust decodingperformance

Small gap tothreshold for allrates

No error floorsobserved

Gaps decreasewith increasing M

31


Robust decodingperformance

Small but slightlyincreasing gap tothreshold (~1-1.25dB at 10-5) forincreasing

Gaps decreasewith increasing M

32

M = 500, W=8Latency = 2MW = 8000Max its. = 10

No error floorsobserved

Randomly Punctured

SC-LDPC Codes (AWGNC)



Regular SC-LDPC Codes vs.

Irregular LDPC-BCs

Consider a comparison of a (3,6)-regular SC-LDPC code vs. anirregular-repeat-accumulate (IRA) LDPC-BC with optimizedprotograph taken from the WiMAX standard

33

Ex:



Irregular LDPC-BCs


The IRA LDPC-BC ensemble has rate R=0.5, BEC threshold , and AWGNC threshold dB.

33

Ex:



Irregular LDPC-BCs



We compare this to a (3,6)-regular SC-LDPC code ensemble withL=50, R=0.49, and thresholds and dB.

33

Ex:



Irregular LDPC-BCs



We compare this to a (3,6)-regular SC-LDPC code ensemble withL=50, R=0.49, and thresholds and dB.

For the SC-LDPC code, we choose W=6 and M=500 so that thelatency of both codes is 6000 bits. (Since a code symbol is present inW=6 'windows', we allow fewer iterations per position for the SC-LDPC window decoder.)

33

Ex:


Gaps tothreshold willreduce withincreasinglatency

34


Irregular LDPC-BCs



Theasymptoticallygood regularSC-LDPC codeshows no sign ofan error floor

34


Irregular LDPC-BCs



Theasymptoticallygood regularSC-LDPC codeshows no sign ofan error floor

The regular SC-LDPC codestructure hasimplementationadvantages

34


Irregular LDPC-BCs


Randomly Punctured LDPC Codes

Randompuncturing can beapplied to LDPCcodes to increasethe rate


Randomly Punctured LDPC Codes

Randompuncturing can beapplied to LDPCcodes to increasethe rate

Equal latencyperformancecomparisons areconsistent for higherrate codes

Regular SC-LDPCcodes displayrobust decodingperformancecompared toirregular LDPC-BCs


Spatial Coupling of Irregular Codes

We can also couple irregular codes to construct an irregular SC-LDPCcode ensemble. Consider the ARJA LDPC-BC protograph:

x

36



A spatially coupled version can be created by edge spreading:


x

36





x

36





x

36





x

36





x

36




... ...


x

36




... ...


x

36


AWGNC Thresholds vs Distance Growth

37

Irregular SC-LDPC code ensembles also display excellent asymptotic

properties

[MLC10] D. G. M. Mitchell, M. Lentmaier and D. J. Costello, Jr., “AWGN Channel Analysis of Terminated

LDPC Convolutional Codes”, Proc. Information Theory and Applications Workshop, San Diego, Feb. 2011.


Implementation Aspects

As a result of their capacity approaching performance and simplestructure, regular SC-LDPC codes may be attractive for future codingstandards. Several key features will require further investigation:

Hardware advantages of QC designs obtained by circulant liftings

Hardware advantages of the 'asymptotically-regular' structure

Design advantages of flexible frame length and flexible rateobtained by varying M, L, and/or puncturing

38


Implementation Aspects

As a result of their capacity approaching performance and simplestructure, regular SC-LDPC codes may be attractive for future codingstandards. Several key features will require further investigation:

Hardware advantages of QC designs obtained by circulant liftings

Hardware advantages of the 'asymptotically-regular' structure

Design advantages of flexible frame length and flexible rateobtained by varying M, L, and/or puncturing

Of particular importance for applications requiring extremely lowdecoded bit error rates (e.g., optical communication, data storage) isan investigation of error floor issues related to stopping sets,trapping sets, and absorbing sets.

38


Conclusions

Spatially coupled LDPC code ensembles achieve threshold saturation,i.e., their iterative decoding thresholds (for large L and M) approach theMAP decoding thresholds of the underlying LDPC block code ensembles.

The threshold saturation and linear minimum distance growth propertiesof (J,K)-regular SC-LDPC codes combine the best asymptotic featuresof both regular and irregular LDPC-BCs.

With window decoding, SC-LDPC codes also compare favorably toLDPC-BCs in the finite-length regime, providing flexible tradeoffsbetween BER performance, decoding latency, and decoder complexity.

39

SC-LDPC codes can be punctured to achieve robustly good performanceover a wide variety of code rates.

Documents

Daniel J. Costello, Jr. · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: From Theory to Practice University of Notre Dame BIRS Workshop on