Iterative Solutions Coded Modulation Library Theory of ... fileIterative Solutions Coded Modulation Library Theory of Operation Oct. 3, 2005 Matthew Valenti Associate Professor West

Iterative SolutionsCoded Modulation Library

Theory of Operation

Oct. 3, 2005

Matthew ValentiAssociate ProfessorWest Virginia UniversityMorgantown, WV [email protected]

10/4/2005Iterative Solutions Coded Modulation Library

2/106

Noisy Channel Coding TheoremClaude Shannon, “A mathematical theory of communication,” Bell Systems Technical Journal, 1948.Every channel has associated with it a capacity C.– Measured in bits per channel use (modulated symbol).

The channel capacity is an upper bound on information rate r.– There exists a code of rate r < C that achieves reliable communications.

• Reliable means an arbitrarily small error probability.


3/106

ContentsShannon capacity under modulation constraints.Bit interleaved coded modulation.Iterative demodulation and decoding– EXIT charts.

Turbo coding.– The SISO decoding algorithm.

LDPC codes– The message passing algorithm.


4/106

Computing Channel CapacityThe capacity is the mutual information between the channel’s input X and output Y maximized over all possible input distributions:

C I X Y

p x y p x yp x p y

dxdy

p x

p x

=

=RST

UVWzzmax ( ; )

max , log ( , )( ) ( )

( )

( )

k p

a f 2


5/106

Capacity of AWGNwith Unconstrained Input

Consider an AWGN channel with 1-dimensional input:– y = x + n– where n is Gaussian with variance No/2– x is a signal with average energy (variance) Es

The capacity in this channel is:

– where Eb is the energy per (information) bit.

This capacity is achieved by a Gaussian input x.– This is not a practical modulation.

C I X Y EN

rENp x

s

o

b

o

= = +FHGIKJ = +

FHG

IKJmax ( ; ) log log

( )k p 1

22 1 1

22 12 2


6/106

Capacity of AWGN withBPSK Constrained Input

If we only consider antipodal (BPSK) modulation, then

and the capacity is:

X s= ± E

C I X Y

I X Y

H Y H N

p y p y dy eN

p x

p x p

o

=

=

= −

= −

=

−∞

∞z

max ;

;

( ) ( )

log ( ) log

( )

( ): /

a fl qa f

a f b g

1 2

2 212

π

maximized whentwo signals are equally likely

This term must be integrated numerically with

p y p y p y p p y dY X N X N( ) ( ) ( ) ( ) ( )= ∗ = −−∞

∞z λ λ λ

Capacity of AWGN w/ 1-D Signaling

BPSK Capacity BoundSh

anno

n Ca

pacit

y Bo

und

It is theoreticallypossible to operatein this region.

It is theoreticallyimpossible to operatein this region.

1.0

Spe

ctra

l Effi

cien

cy

Cod

e R

ate

r

0.5

-2 -1 0 1 2 3 4 5 6 7 8 9 10

Eb/No in dB

Power Efficiency of StandardBinary Channel Codes

Mariner1969

Turbo Code1993

Galileo:BVD1992Galileo:LGA

1996

Pioneer1968-72

Voyager1977

OdenwalderConvolutionalCodes 1976

BPSK Capacity BoundSh

anno

n Ca

pacit

y Bo

und

UncodedBPSK

IS-951991

Iridium1998

510−=bParbitrarily lowBER:

1.0

Spe

ctra

l Effi

cien

cy

Cod

e R

ate

r

0.5LDPC Code

2001Chung, Forney,

Richardson, Urbanke

-2 -1 0 1 2 3 4 5 6 7 8 9 10

Eb/No in dB


9/106

M-ary Complex Modulationµ = log2 M bits are mapped to the symbol xk, which is chosen from the set S = {x1, x2, …, xM}– The symbol is multidimensional.– 2-D Examples: QPSK, M-PSK, QAM, APSK, HEX– M-D Example: FSK, block space-time codes (BSTC)

The signal y = hxk + n is received– h is a complex fading coefficient.– More generally (BSTC), Y = HX + N

Modulation implementation in the ISCML– Currently only 2-D modulation is supported.– The complex signal set S is created with the Create2D function.– Modulation is performed using the Mod2D function.


10/106

Log-likelihood of Received SymbolsLet p(xk|y) denote the probability that signal xk ∈S was transmitted given that y was received.Let f(xk|y) = Κ p(xk|y), where Κ is any multiplicative term that is constant for all xk.

When all symbols are equally likely, f(xk|y) ∝ f(y|xk) For each signal in S, the receiver computes f(y|xk)– This function depends on the modulation, channel, and receiver.– Implemented by the Demod2D function, which actually computes

log f(y|xk).

Assuming that all symbols are equally likely, the most likely symbol xk is found by making a hard decision on f(y|xk) or log f(y|xk).


11/106

Example: QAM over AWGN.Let y = x + n, where n is complex i.i.d. N(0,N0/2 ) and the average energy per symbol is E[|x|2] = Es

o

ks

kk

kk

kk

NxyE

xyxyf

xyxyf

xyxyp

2

2

2

2

2

2

2

2

2)(log

2exp)(

2exp

21)(

−−=

−−=

⎪⎭

⎪⎬⎫

⎪⎩

⎪⎨⎧ −−

=

⎪⎭

⎪⎬⎫

⎪⎩

⎪⎨⎧ −−

=

σ

σ

σπσ


12/106

Log-Likelihood of Symbol xk

The log-likelihood of symbol xk is found by:

{ }

[ ])|(logmax*)|(log

)|(logexplog)|(log

)|(log)|(log

)|()|(

log

)|()|(

log

)|(log

mSk

Smk

Smk

Sm

k

Sk

k

kk

ff

ff

ff

ff

pp

p

m

m

m

m

m

xyxy

xyxy

xyxy

xyxy

yxyx

yx

x

x

x

x

x

∈

∈

∈

∈

∈

−=

−=

−=

=

=

=Λ

∑

∑

∑

∑


13/106

The max* functionThe pairwise max* function is defined as:

– Where fc is a correction function that only depends on the magnitude of the difference between x and y.

For multiple arguments, it follows the recursive relationship:

[ ]{ }( )

( )xyfyx

xyyxyxyx

c −+=

−−++=

+=

),max(

exp1log),max()exp()exp(log),(max*

[ ]zyxzyx ),,(max*max*),,(max* =

The correction function

0 1 2 3 4 5 6 7 8 9 10-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

( )[ ])exp1log)( zzfc −+=

fc(|y-x|)

|y-x|

Simplifications

0 1 2 3 4 5 6 7 8 9 10-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

( )[ ])exp1log)( zzfc −+=

fc(|y-x|)

|y-x|


16/106

Capacity of Coded Modulation (CM)

Suppose we want to compute capacity of M-arymodulation – In each case, the input distribution is constrained, so there is no

need to maximize over p(x)– The capacity is merely the mutual information between channel

input and output.

The mutual information can be measured as the following expectation:

[ ] nats )|(loglog);( , yxnx kpMEYXICk

+==


17/106

Monte Carlo Calculation of the Capacity of Coded Modulation (CM)

The mutual information can be measured as the following expectation:

This expectation can be obtained through Monte Carlo simulation.

[ ][ ]

[ ]

[ ]bits

log(2)

bits log(2)

log

nats log

nats )|(loglog);(

,

,2

,

,

k

k

k

k

k

k

k

k

E

EM

EM

pMEYXIC

Λ+=

Λ+=

Λ+=

+==

nx

nx

nx

nx yx

µ

Simulation Block DiagramThis function is computed by the ISCML function Demod2D

This function is computed by the ISCML function Capacity

Modulator:Pick xk at randomfrom S

xk

nk

Noise Generator

Receiver:Compute log f(y|xk)for every xk ∈ S

Calculate:

After running many trials, calculate:

[ ])2log(

kEC

Λ+= µ

[ ])|(logmax*)|(log

mS

kk

ff

m

xyxy

x ∈−

=Λ

Benefits of Monte Carlo approach:-Allows high dimensional signals to be studied.-Can determine performance in fading.-Can study influence of receiver design.

-2 0 2 4 6 8 10 12 14 16 18 200

1

2

3

4

5

6

7

8

Eb/No in dB

Cap

acity

(bits

per

sym

bol)

2-D U

nconst

rained

Capacit

y

256QAM

64QAM

16QAM

16PSK

8PSK

QPSK

Capacity of 2-D modulation in AWGN

BPSK

Capacity of M-ary Noncoherent FSK in AWGNW. E. Stark, “Capacity and cutoff rate of noncoherent FSKwith nonselective Rician fading,” IEEE Trans. Commun., Nov. 1985.

M.C. Valenti and S. Cheng, “Iterative demodulation and decoding of turbo coded M-ary noncoherent orthogonal modulation,” to appear in IEEE JSAC, 2005.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

5

10

15

Rate R (symbol per channel use)

Min

imum

Eb/

No

(in d

B)

M=2

M=4

M=16

M=64

Noncoherent combining penalty

Ergodic Capacity (Fully interleaved)Assumes perfect fading amplitude estimates available to receiver

M=2

M=4

M=16

M=64

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

5

10

15

Rate R (symbol per channel use)

Min

imum

Eb/

No

(in d

B)

Capacity of M-ary Noncoherent FSK in Rayleigh Fading


22/106

BICMCoded modulation (CM) is required to attain the aforementioned capacity.– Channel coding and modulation handled jointly.– e.g. trellis coded modulation (Ungerboeck); coset codes (Forney)

Most off-the-shelf capacity approaching codes are binary.A pragmatic system would use a binary code followed by a bitwise interleaver and an M-ary modulator.– Bit Interleaved Coded Modulation (BICM); Caire 1998.

BinaryEncoder

BitwiseInterleaver

Binaryto M-arymapping

lu nc' nc kx


23/106

Transforming Symbol Log-Likehoods Into Bit LLRs

Like the CM receiver, the BICM receiver calculates log f (y|xk) for each signal in S.Furthermore, the BICM receiver needs to calculate the log-likelihood ratio of each code bit:

– where represents the set of symbols whose nth bit is a 1.

– and is the set of symbols whose nth bit is a 0.

( )( )

( )

( )

( )

( )

( )[ ] ( )[ ]kS

kS

Skk

Skk

Sk

Sk

n

nn

ff

xpp

xpp

p

p

cpcp

nknk

nk

nk

nk

nk

xyxy

xy

xy

yx

yx

yy

xx

x

x

x

x

logmax*logmax*

][|

][|log

|

|log

|0|1

log

)0()1(

)0(

)1(

)0(

)1(

∈∈

∈

∈

∈

∈

−=

====

=∑

∑

∑

∑λ

)1(

nS

)0(

nS


24/106

BICM CapacityBICM transforms the channel into µ parallel binary channels, and the capacity of the nth channel is:

[ ]

{ }[ ] nats )1(,0max*-)2log(

nats )|(

)|1(log,

)|()|0(

logmax*-)2log(

nats )|(

)|1(logexp

)|()|0(

logexplog-)2log(

nats )|(

)|1()|0(log)2log(

nats )|1()|0(

)|(log)2log(

nats )|(log)2log(

,

,

,

,

,

,

kc

c

k

k

k

kc

k

k

k

kc

k

kkc

kk

kc

kck

k

n

n

n

n

n

n

E

cpcp

cpcp

E

cpcp

cpcp

E

cpcpcp

E

cpcpcp

E

cpEC

λ−=

⎥⎦

⎤⎢⎣

⎡

⎭⎬⎫

⎩⎨⎧ ==

=

⎥⎦

⎤⎢⎣

⎡

⎭⎬⎫

⎩⎨⎧ =

+=

=

⎥⎦

⎤⎢⎣

⎡ =+=−=

⎥⎦

⎤⎢⎣

⎡=+=

+=

+=

n

n

n

n

n

n

yy

yy

yy

yy

yyy

yyy

y


25/106

BICM Capacity (Continued)Since capacity over parallel channels adds, the capacity of BICM is:

{ }[ ]{ }

{ }[ ]

{ }[ ] bits )1(,0max*)2log(

1

nats )1(,0max*)2log(

nats )1(,0max*-log(2)

1,

1,

1,

1

∑

∑

∑

∑

=

=

=

=

−−=

−−=

−=

=

µ

µ

µ

µ

λµ

λµ

λ

kk

cc

kk

cc

kk

cc

kk

k

n

k

n

k

n

E

E

E

CC

n

n

n


26/106

BICM CapacityAs with CM, this can be computed using a Monte Carlo integration.

Modulator:Pick xk at randomfrom S

xk

nk

Noise Generator

Receiver:Compute p(y|xk)for every xk ∈ S

For each bit, calculate:

After running many trials, calculate:

[ ])2log(

Λ+=

EC µUnlike CM, the capacity of BICMdepends on how bits are mapped to symbols

( )

( )∑

∑

∈

∈=

)0(

)1(

log

n

n

S

Sn p

p

x

x

xy

xyλ

For each symbol, calculate:

{ }[ ] bits )1(,0max*1

,∑=

−−=Λµ

λk

kc

ck

nE n

This function is computed by the ISCML function Somap

-10 -5 0 5 10 15 200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1C

ode

Rat

e R

(4-b

it sy

mbo

ls p

er c

hann

el u

se)

Minimum Es/No (in dB)

CM and BICM capacity for 16QAM in AWGN

CM M=16 QAM AWGNBICM M=16 QAM grayBICM M=16 QAM SPBICM M=16 QAM MSPBICM M=16 QAM AntigrayBICM M=16 QAM MSEW


28/106

BICM-IDThe conventional BICM receiver assumes that all bits in a symbol are equally likely:

However, if the receiver has estimates of the bit probabilities, it can use this to weight the symbol likelihoods.

This information is obtained from decoder feedback.– Bit Interleaved Coded Modulation with Iterative Demodulation– Li and Ritcey 1999.

( )

( )

( )

( )∑

∑

∑

∑

∈

∈

∈

∈ ==

)0(

)1(

)0(

)1(

log|

|log

n

n

n

n

S

S

S

Sn p

p

p

p

x

x

x

x

xy

xy

yx

yxλ

( )

( )∑

∑

∈

∈

=

=

=

)0(

)1(

)0|(

)1|(log

n

n

Sn

nS

n cpp

cpp

x

x

xxy

xxyλ


29/106

Mutual Information Transfer ChartNow consider a receiver that has a priori information about the code bits (from a soft output decoder).Assume the following:– The a priori information is in LLR form.– The a priori LLR’s are Gaussian distributed.– The LLR’s have mutual information Iv

Then the mutual information Iz at the output of the receiver can be measured through Monte Carlo Integration.– Iz vs. Iv is the Mutual Information Transfer Characteristic.– ten Brink 1999.

Generating Random a Priori Input

0 5 10 15 20 25 30 35 40 45 500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

variance

There is a one-to-one correspondencebetween the mutual informationand the variance of the Gaussiandistributed a priori input

Mut

ual I

nfor

mat

ion

Mutual Information Characteristic

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Iv

graySPMSPMSEWAntigray

16-QAMAWGN6.8 dB

I z

EXIT Chart

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Iv

graySPMSPMSEWAntigrayK=3 Conv code

16-QAMAWGN6.8 dBadding curve for a FEC codemakes this an extrinsic informationtransfer (EXIT) chart

I z

EXIT Chart for Space Time Block Code

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Iv

1 by 1 MSP2 by 1 Alamouti MSP2 by 1 Alamouti huangNr12 by 2 Alamouti MSP2 by 2 Alamouti huangNr2K=3 Conv code

16-QAM8 dBRayleigh fading

I z

BICM-ID for Noncoherent FSK

M.C. Valenti and S. Cheng, “Iterative demodulation and decoding of turbo coded M-ary noncoherent orthogonal modulation,” to appear in IEEE JSAC, 2005.

Review: Convolutional CodesA convolutional encoder encodes a stream of data.– The size of the code word is unbounded.

The encoder is a Finite Impulse Response (FIR) filter.– k binary inputs– n binary outputs– K -1 delay elements– Coefficients are either 1 or 0 – All operations over GF(2)

• Addition: XOR

Constraint Length K = 3

D D

Recursive Systematic Convolutional (RSC) Encoding

An RSC encoder is constructed from a standard convolutional encoder by feeding back one of the outputs.An RSC code is systematic.– The input bits appear directly in the

output.An RSC encoder is an Infinite Impulse Response (IIR) Filter. – An arbitrary input will cause a

“good” (high weight) output with high probability.

– Some inputs will cause “bad” (low weight) outputs.

D D

ix

ir

In the ISCML, convolutionalencoding is implemented withthe ConvEncode function.

The optional input nsc_flagdetermines if it is an RSC (flag = 0)or NSC (flag = 1) encoder.The default is for it to be an RSC encoder.


37/106

Parallel Concatenated Codeswith Nonuniform Interleaving

A stronger code can be created by encoding in parallel.A nonuniform interleaver scrambles the ordering of bits at the input of the second encoder.– Uses a pseudo-random interleaving pattern.

It is very unlikely that both encoders produce low weight code words.MUX increases code rate from 1/3 to 1/2.

RSC#1

RSC#2

NonuniformInterleaver

MUX

Input

ParityOutput

Systematic Outputix


38/106

Random Coding Interpretationof Turbo Codes

Random codes achieve the best performance.– Shannon showed that as n→∞, random codes achieve channel

capacity.

However, random codes are not feasible.– The code must contain enough structure so that decoding can be

realized with actual hardware.

Coding dilemma:– “All codes are good, except those that we can think of.”

With turbo codes:– The nonuniform interleaver adds apparent randomness to the

code.– Yet, they contain enough structure so that decoding is feasible.


39/106

Comparison of a Turbo Codeand a Convolutional Code

First consider a K=12 convolutional code.– dmin = 18– βd = 187 (output weight of all dmin paths)

Now consider the original turbo code.– C. Berrou, A. Glavieux, and P. Thitimasjshima, “Near Shannon limit error-

correcting coding and decoding: Turbo-codes,” in Proc. IEEE Int. Conf. on Commun., Geneva, Switzerland, May 1993, pp. 1064-1070.

– Same complexity as the K=12 convolutional code– Constraint length 5 RSC encoders– k = 65,536 bit interleaver– Minimum distance dmin = 6– ad = 3 minimum distance code words– Minimum distance code words have average information weight of only

f d = 2

Comparison of Minimum-distance Asymptotes

0.5 1 1.5 2 2.5 3 3.5 4

10-8

10-6

10-4

10-2

100

Eb/No in dB

BE

R

Convolutional Code CC free distance asymptoteTurbo Code TC free distance asymptote

Convolutional code:

Turbo code:

( ) ⎟⎟⎠

⎞⎜⎜⎝

⎛≈

o

bb N

EQP 18187

( ) ⎟⎟⎠

⎞⎜⎜⎝

⎛×≈ −

o

bb N

EQP 6102.9 5

187min =dc

18min =d

6min =d

6553623~~ minmin

min⋅

==kwac dd

d


41/106

The Turbo-PrincipleTurbo codes get their name because the decoder uses feedback, like a turbo engine.

0.5 1 1.5 210-7

10-6

10-5

10-4

10-3

10-2

10-1

100

Eb/No in dB

BE

R

1 iteration

2 iterations

3 iterations6 iterations

10 iterations

18 iterations

Performance as a Function of Number of Iterations

K = 5– constraint length

r = 1/2– code rate

L= 65,536 – interleaver size– number data bits

Log-MAP algorithm


43/106

Summary of Performance Factors and Tradeoffs

Latency vs. performance– Frame (interleaver) size L

Complexity vs. performance– Decoding algorithm– Number of iterations– Encoder constraint length K

Spectral efficiency vs. performance– Overall code rate r

Other factors– Interleaver design– Puncture pattern– Trellis termination

Tradeoff: BER Performance versus Frame Size (Latency)

0.5 1 1.5 2 2.510

-7

10-6

10-5

10-4

10-3

10-2

10-1

K = 5Rate r = 1/218 decoder iterationsAWGN Channel

0.5 1 1.5 2 2.5 310

-8

10-6

10-4

10-2

100

Eb/No in dB

BE

R

K=1024 K=4096 K=16384K=65536


45/106

Characteristics of Turbo CodesTurbo codes have extraordinary performance at low SNR.– Very close to the Shannon limit.– Due to a low multiplicity of low weight code words.

However, turbo codes have a BER “floor”.– This is due to their low minimum distance.

Performance improves for larger block sizes.– Larger block sizes mean more latency (delay).– However, larger block sizes are not more complex to decode.– The BER floor is lower for larger frame/interleaver sizes

The complexity of a constraint length KTC turbo code is the same as a K = KCC convolutional code, where:– KCC ≈ 2+KTC+ log2(number decoder iterations)

UMTS Turbo Encoder

From ETSI TS 125 212 v3.4.0 (2000-09)– UMTS Multiplexing and channel coding

Data is segmented into blocks of L bits. – where 40 ≤ L ≤ 5114

“Upper”RSC

Encoder

“Lower”RSC

EncoderInterleaver

Systematic Output

Xk

UninterleavedParity

Zk

InterleavedParity

Z’k

InterleavedInputX’k

InputXk

Output


47/106

UMTS Interleaver:Inserting Data into Matrix

Data is fed row-wise into a R by C matrix.– R = 5, 10, or 20.– 8 ≤ C ≤ 256– If L < RC then matrix is padded with dummy characters.

In the ISCML, the UMTS inteleaver is created by the function CreateUMTSInterleaverInterleaving and Deinterleaving are implemented by Interleave and Deinterleave

X1 X2 X3 X4 X5 X6 X7 X8

X9 X10 X11 X12 X13 X14 X15 X16

X17 X18 X19 X20 X21 X22 X23 X24

X25 X26 X27 X28 X29 X30 X31 X32

X33 X34 X35 X36 X37 X38 X39 X40


48/106

UMTS Interleaver:Intra-Row Permutations

Data is permuted within each row.– Permutation rules are rather complicated.– See spec for details.

X2 X6 X5 X7 X3 X4 X1 X8

X10 X12 X11 X15 X13 X14 X9 X16

X18 X22 X21 X23 X19 X20 X17 X24

X26 X28 X27 X31 X29 X30 X25 X32

X40 X36 X35 X39 X37 X38 X33 X34


49/106

UMTS Interleaver:Inter-Row Permutations

Rows are permuted.– If R = 5 or 10, the matrix is reflected about the middle row.– For R=20 the rule is more complicated and depends on L.

• See spec for R=20 case.

X40 X36 X35 X39 X37 X38 X33 X34

X26 X28 X27 X31 X29 X30 X25 X32

X18 X22 X21 X23 X19 X20 X17 X24

X10 X12 X11 X15 X13 X14 X9 X16

X2 X6 X5 X7 X3 X4 X1 X8


50/106

UMTS Interleaver:Reading Data From Matrix

Data is read from matrix column-wise.

Thus:– X’1 = X40 X’2 = X26 X’3 = X18 …– X’38 = X24 X’2 = X16 X’40 = X8

X40 X36 X35 X39 X37 X38 X33 X34

X26 X28 X27 X31 X29 X30 X25 X32

X18 X22 X21 X23 X19 X20 X17 X24

X10 X12 X11 X15 X13 X14 X9 X16

X2 X6 X5 X7 X3 X4 X1 X8


51/106

Recursive Systematic Convolutional(RSC) Encoder

Systematic Output(Upper Encoder Only)

D D D

Parity Output(Both Encoders)

Upper and lower encoders are identical:– Feedforward generator is 15 in octal.– Feedback generator is 13 in octal.


52/106

Trellis Termination

After the Lth input bit, a 3 bit tail is calculated.– The tail bit equals the fed back bit.– This guarantees that the registers get filled with zeros.

Each encoder has its own tail.– The tail bits and their parity bits are transmitted at the end.

D D D

XL+1 XL+2 XL+3

ZL+1 ZL+2 ZL+3


53/106

Output Stream FormatThe format of the output steam is:

X1 Z1 Z’1 X2 Z2 Z’2 … XL ZL Z’L XL+1 ZL+1 XL+2 ZL+2 XL+3 ZL+3 X’L+1 Z’L+1 X’L+2 Z’L+2 X’L+3 Z’L+3

3 tail bits forlower encoder

and their3 parity bits

3 tail bits forupper encoder

and their3 parity bits

L data bits andtheir associated2L parity bits

(total of 3L bits)

Total number of coded bits = 3L + 12

Code rate: r LL

=+

≈3 12

13


54/106

Channel Modeland LLRs

Channel gain: a– Rayleigh random variable if Rayleigh fading– a = 1 if AWGN channel

Noise– variance is:

BPSKModulator

{0,1} {-1,1}

a n2

2σ

a

y r

⎟⎟⎠

⎞⎜⎜⎝

⎛≈

⎟⎟⎠

⎞⎜⎜⎝

⎛=

o

b

o

b

NE

NEr 2

3

2

12σ


55/106

SISO-MAP Decoding Block

This block is implemented in the ISCML by the SisoDecode function

SISOMAP

Decoder

λu,oλu,i

λc,oλc,i

Inputs:– λu,i LLR’s of the data bits. This comes from the other decoder r.– λc,i LLR’s of the code bits. This comes from the channel observations r.

Two output streams:– λu,o LLR’s of the data bits. Passed to the other decoder.– λc,o LLR’s of the code bits. Not used by the other decoder.

Turbo Decoding Architecture

“Upper”MAP

Decoderr(Xk)r(Zk)

“Lower”MAP

Decoderr(Z’k)

Interleave

Deinnterleave

Demux

zerosDemux

$Xk

Initialization and timing:– Upper λu,i input is initialized to all zeros.– Upper decoder executes first, then lower decoder.

Performance as a Function of Number of Iterations

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 210-7

10-6

10-5

10-4

10-3

10-2

10-1

100

Eb/No in dB

BE

R

BER of 640 bit turbo code in AWGNL=640 bitsAWGN channel10 iterations

1 iteration

2 iterations

3 iterations

10 iterations


58/106

Log-MAP Algorithm:Overview

Log-MAP algorithm is MAP implemented in log-domain.– Multiplications become additions.– Additions become special “max*” operator (Jacobi logarithm)

Log-MAP is similar to the Viterbi algorithm.– Except “max” is replaced by “max*” in the ACS operation.

Processing:– Sweep through the trellis in forward direction using modified

Viterbi algorithm.– Sweep through the trellis in backward direction using modified

Viterbi algorithm.– Determine LLR for each trellis section.– Determine output extrinsic info for each trellis section.


59/106

The max* operator revisitedmax* must implement the following operation:

Ways to accomplish this:– C-function calls or large look-up-table.– (Piecewise) linear approximation.– Rough correction value.

– Max operator.

{ }( )( )

),(max*),max(

exp1ln),max(

yxxyfyx

xyyxz

c

=

−+=

−−++=

),max( yxz ≈

⎪⎩

⎪⎨⎧

≤

>+≈

5.1x-y f5.0

5.1x-y if0),max( yxz

log-MAP

constant-log-MAP

max-log-MAP

The Correction Function

0 1 2 3 4 5 6 7 8 9 1 0-0 .1

0

0 .1

0 .2

0 .3

0 .4

0 .5

0 .6

0 .7

Constant-log-MAP

log-MAP

|y-x|

fc(|y-x|)

dec_type option in SisoDecode=0 For linear-log-MAP (DEFAULT)= 1 For max-log-MAP algorithm = 2 For Constant-log-MAP algorithm= 3 For log-MAP, correction factor from

small nonuniform table and interpolation= 4 For log-MAP, correction factor

uses C function calls


61/106

The Trellis for UMTSDotted line = data 0Solid line = data 1Note that each node has one each of data 0 and 1 entering and leaving it.The branch from node Si to Sj has metric γij

S0 S0

S1

S2

S3

S4

S5

S6

S7

γ 10

ickk

ickk

iukkij jiZjiXjiX ,,,

21),(),(),( λλλγ ++=

γ 00

S1

S2

S3

data bit associatedwith branch Si →Sj The two code bits labeling

with branch Si →Sj

S4

S5

S6

S7


62/106

Forward RecursionA new metric must be calculated for each node in the trellis using:

where i1 and i2 are the two states connected to j.Start from the beginning of the trellis (i.e. the left edge).Initialize stage 0:αo = 0αi = -∞ for all i ≠ 0

α’0 α 0

α 1

α 2

α 3

α 4

α 5

α 6

α 7

γ 10

γ 00

α’1 α α γ α γj i i j i i j= + +max* ' , '1 1 2 2d i d io t

α’2

α’3

α’4

α’5

α’6

α’7


63/106

Backward RecursionA new metric must be calculated for each node in the trellis using:

where j1 and j2 are the two states connected to i.Start from the end of the trellis (i.e. the right edge).Initialize stage L+3:βo = 0βi = -∞ for all i ≠ 0

β0 β’0

β’1

β’2

β’3

β’4

β’5

β’6

β’7

γ 10

γ 00

β1 β β γ β γi j ij j ij= + +max* ' , ' 1 1 2 2d i d io t

β2

β3

β4

β5

β6

β7


64/106

Log-likelihood RatioThe likelihood of any one branch is:

The likelihood of data 1 is found by summing the likelihoods of the solid branches.The likelihood of data 0 is found by summing the likelihoods of the dashed branches.The log likelihood ratio (LLR) is:

α0 β0

β1

β2

β3

β4

β5

β6

β7

γ 10

γ 00

α γ βi ij j+ +

Λ XP XP Xk

k

k

S S X i ij j

S S X i ij j

i j k

i j k

b g

n sn s

===

FHG

IKJ

= + +

− + +

→ =

→ =

ln

max*

max*

:

:

10

1

0

α γ β

α γ β

α1

α2

α3

α4

α5

α6

α7


65/106

Extrinsic InformationThe extrinsic information is found by subtracting the corresponding input from the LLR output, i.e.

• λu,i (lower) = λu,o (upper) - λu,i (upper)• λu,i (upper) = λu,o (lower) - λu,i (lower)

It is necessary to subtract the information that is already available at the other decoder in order to prevent “positive feedback”.The extrinsic information is the amount of new information gained by the current decoder step.

Performance Comparison

0 0.5 1 1.5 2 2.5 310 -7

10 -6

10 -5

10 -4

10 -3

10 -2

10 -1

10 0

E b/N o in dB

BE

R

B E R o f 640 b it tu rbo c ode

m ax -log-M A P c ons tant-log-M A Plog-M A P

10 decoder iterations

Fading

AWGN


67/106

Practical ImplementationIssues

No need to store both alpha and beta in trellis.– Compute beta first and store in trellis.– Then compute alpha and derive LLR estimates at the same time.

• No need to store alpha trellis.The metrics keep getting larger and larger.– Floating point: loss of precision.– Fixed point: overflow.– Solution: normalize the metrics:

α α αi normalized ib g = − 0α α αi normalized i i ib g b g= − min or

β β βi normalized ib g = − 0β β βi normalized i i ib g b g= − min


68/106

Issues: Channel EstimationIn an AWGN Channel:– log-MAP and constant-log-MAP need an estimate of σ2

– Estimate of σ2 is not needed for max-log-MAP.– Constant-log-MAP is more vulnerable to bad estimates than log-

MAP.

In addition, for fading channels:– All three algorithms require estimates of the channel gain a.


69/106

Issues: Memory LimitationsStoring the entire beta trellis can require a significant amount of memory:– 8L states– For L=5114 there will have to be 40,912 stored values.– The values only need to be single-precision floats.

An alternative is to use a “sliding window” approach.


70/106

Issues: Halting the IterationsTurbo decoding progresses until a fixed number of iterations have completed.– However, the decoder will often converge early.– Can stop once it has converged (i.e. BER = 0).– Stopping early can greatly increase the throughput.

For a simulation, it is reasonable to automatically halt once the BER goes to zero.– Requires knowledge of the data.

For an actual system, a “blind” method for halting is desirable.– Blind --- don’t know the data.


71/106

Simple Method for HaltingA simple method is as follows:– After each decoder iteration, determine the smallest output LLR

value:

– Then compare this value to a threshold.– If the minimum LLR is more than the threshold, then halt:

• Halt if:

Λ Λmin min | |=≤ ≤1 k K kXb gm r

Λmin _> llr halt

BER of Different Values ofllr_halt in AWGN channel

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 210

-7

10-6

10-5

10-4

10-3

10-2

10-1

Eb/No in dB

BER

BER of 640 bit turbo code

llr halt = 1llr halt = 5llr halt = 10llr halt = 0

FER of Different Values ofllr_halt in AWGN channel

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 210

-5

10-4

10-3

10-2

10 -1

100

Eb/No in dB

FER

FER of 640 bit turbo code


Average Number of Iterations

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 21

2

3

4

5

6

7

8

9

10

Eb/No in dB

Num

ber o

f dec

oder

iter

atio

ns

Number of iterations for 640 bit turbo code



75/106

LDPC Code OverviewLow Density Parity Check (LDPC) codes are a class of linear block codes characterized by sparse parity check matrices H.Review of parity check matrices:– For a (n,k) code, H is a n-k by n matrix of ones and zeros.– A codeword c is valid iff cHT = 0– Each row of H specifies a parity check equation. The code bits in

positions where the row is one must sum (modulo-2) to zero.– In an LDPC code, only a few bits (~4 to 6) participate in each

parity check equation.

LDPC codes were originally invented by Robert Gallagerin the early 1960’s but were largely ignored until they were “rediscovered” in the mid-1990’s by MacKay.


76/106

Decoding LDPC CodesLike turbo codes, LDPC codes are iteratively decodedInstead of a trellis, LDPC codes use a “Tanner graph” – A Tanner graph consists of variable-nodes (corresponding to the

received code bits) and check-nodes (corresponding to the parity check equations).

– The H matrix determines the connections between v-nodes and c-nodes.– Messages iteratively flow back and forth between v-nodes and c-nodes.

In general, the per-iteration complexity of LDPC codes is less than it is for turbo codes.– e.g. the best LDPC code investigated here needs half as many additions

per iteration than the UMTS turbo code.However, many more iterations may be required (max≈100;avg≈30).– Thus, overall complexity can be higher than turbo.

More on the decoding algorithm and complexity later.


77/106

Detecting ErrorsWith LDPC codes, error detection comes for free.Errors can be detected by computing cHT =? 0– The probability of undetetected error is very small.

This can be used to dynamically halt the iterations.No outer error detecting code is needed.


78/106

Regular vs. Irregular LDPC codesAn LDPC code is regular if the rows and columns of H have uniform weight, i.e. all rows have the same number of ones and all columns have the same number of ones.– The codes of Gallager and MacKay were regular (or as close as

possible).– Although regular codes had impressive performance, they are still about 1

dB from capacity and generally perform worse than turbo codes.An LDPC code is irregular if the rows and columns have non-uniform weight.– Typically the rows have similar weight (e.g. 5 and 6) while the columns

have vastly different weight (e.g. from 2 to 15 or 20).– Around 2000, Richardson and Urbanke showed that long irregular LDPC

codes could come within ~0.1 dB of capacity.• One of their r=1/2 designs (with Chung) came within 0.04 dB of capacity.

– Irregular LDPC codes tend to outperform turbo codes for block lengths of about n>105


79/106

Encoding LDPC CodesA linear block code is encoded by performing the matrix multiplication c = uG, where u is the k-bit long message and G is a k by n generator matrix.A common method for finding G from H is to first make the code systematic by adding rows and exchanging columns to get the H matrix in the form H = [PT I].– Then G = [I P].– However, the result of the row reduction is a non-sparse P matrix.– The multiplication c=uG =[u uP] is now very complex, as the calculation

of uP is O(k(n-k))Solutions:– Richardson and Urbanke (2001) show how to transform the H matrix into

a (fairly) sparse (almost) lower triangular matrix. Then encoding complexity is approximately O(k).

– An alternative involving a sparse-matrix multiply followed by differential encoding has been proposed by Ryan, Yang, & Li….


80/106

Extended IRA Codes(Ryan, Yang, & Li)

Let H = [H1 H2] where H1 is sparse and

Then a systematic code can be generated with G = [I H1TH2

-T].It turns out that H2

-T is the generator matrix for an accumulate-code (differential encoder), and thus the encoder structure is simply:u u

uH1TH2

-T

Similar to Jin & McEliece’s Irregular Repeat Accumulate (IRA) codes.– Thus termed “Extended IRA Codes”

⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

=

⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

= −

1

1...

1...1

1...11

1...111

and

11

1...1

11

11

1

22THH

Multiplyby H1

T

D


81/106

Performance ComparisonWe now compare the performance of the maximum-length UMTS turbo code against four LDPC code designs.Code parameters– All codes are rate ⅓– The LDPC codes are length (n,k) = (15000, 5000)

• Up to 100 iterations of log-domain sum-product decoding• Code parameters are given on next slide

– The turbo code has length (n,k) = (15354,5114)• Up to 16 iterations of log-MAP decoding

BPSK modulationAWGN and fully-interleaved Rayleigh fadingEnough trials run to log 40 frame errors– Sometimes fewer trials were run for the last point (highest SNR).


82/106

LDPC Code ParametersCode 1: MacKay’s regular construction 2A– See: D.J.C. MacKay, “Good error-correcting codes based on very sparse

matrices,” IEEE Trans. Inform. Theory, March 1999.Code 2: Richardson & Urbanke irregular construction– See T. Richardson, M. Shokrollahi, and R. Urbanke, “Design of capacity-

approaching irregular low-density parity-check codes,” IEEE Trans. Inform. Theory, Feb. 2001.

Code 3: Improved irregular construction– Designed by Chris Jones using principles from T. Tian, C. Jones, J.D.

Villasenor, and R.D. Wesel, “Construction of irregular LDPC codes with low error floors,” in Proc. ICC 2003.

– Idea is to avoid small “stopping sets”Code 4: Extended IRA code– Designed by Michael Yang & Bill Ryan using principles from M. Yang and

W.E. Ryan, “Lowering the error-rate floors of moderate-length high-rate irregular LDPC codes,” to appear at ISIT 2003.


83/106

LDPC Degree DistributionsThe distribution of row-weights, or check-node degrees, is as follows:

The distribution of column-weights, or variable-node degrees, is:

454299876500054581354999100004

134321i

117816891518

19412065500056925844

2267223810000399999045828250002

114321i

Code number:1 = MacKay construction 2A2 = Richardson & Urbanke3 = Jones, Wesel, & Tian4 = Ryan’s Extended-IRA

BER in AWGN

Code #1:Mackay 2A

Code #4:IRA

Code #2:R&U

Code #3:JWT

turbo

BPSK/AWGN Capacity:-0.50 dB for r = 1/3

0 0.2 0.4 0.6 0.8 1 1.210

-7

10-6

10-5

10-4

10-3

10-2

10-1

Eb/No in dB

BE

R

FER in AWGN

Code #1:Mackay 2A

Code #4:IRA

Code #2:R&U

Code #3:JWT

turbo

0 0.2 0.4 0.6 0.8 1 1.2

10-4

10-3

10-2

10-1

100

Eb/No in dB

FER

Average Number of Iterations in AWGN

Code #1:Mackay 2A

Code #4:IRA

Code #2:R&U

Code #3:JWT

turbo

0 0.2 0.4 0.6 0.8 1 1.20

10

20

30

40

50

60

70

80

90

100

Eb/No in dB

Ave

rage

num

ber o

f ite

ratio

ns

BER in Fully-Interleaved Rayleigh Fading

Code #1:Mackay 2A

Code #4:IRA

Code #2:R&U

Code #3:JWT

turbo

BPSK/Fading Capacity:0.5 dB for r = 1/3

1 1.2 1.4 1.6 1.8 2 2.210 -6

10 -5

10 -4

10 -3

10 -2

10 -1

Eb/No in dB

BE

R

FER in Fully-InterleavedRayleigh Fading

1 1.2 1.4 1.6 1.8 2 2.210

-4

10-3

10-2

10-1

100

Eb/No in dB

Code #1:Mackay 2A

Code #4:IRA

Code #2:R&U

Code #3:JWT

turbo

FER

Average Number of IterationsRayleigh Fading

Code #1:Mackay 2A

Code #4:IRA

Code #2:R&U

Code #3:JWT

turbo

1 1.2 1.4 1.6 1.8 2 2.20

10

20

30

40

50

60

70

80

90

100

Eb/No in dB

Ave

rage

num

ber o

f ite

ratio

ns


90/106

Tanner GraphsA Tanner graph is a bipartite graph that describes the parity check matrix H.There are two classes of nodes:– Variable-nodes: Correspond to bits of the codeword or equivalently, to

columns of the parity check matrix.• There are n v-nodes.

– Check-nodes: Correspond to parity check equations or equivalently, to rows of the parity check matrix.

• There are m=n-k c-nodes.– Bipartite means that nodes of the same type cannot be connected (e.g. a

c-node cannot be connected to another c-node).The ith check node is connected to the jth variable node iff the (i,j)th

element of the parity check matrix is one, i.e. if hij =1.– All of the v-nodes connected to a particular c-node must sum (modulo-2)

to zero.


91/106

Example: Tanner Graphfor (7,4) Hamming Code

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

100110101010110010111

H

c-nodesf0 f1 f2

y0 y1 y2 y3 y4 y5 y6

v-nodes


92/106

Message Passing Decoding: Notation

Qi = LLR of the ith code bit (ultimate goal of algorithm)qij = extrinsic info to be passed from v-node i to c-node jrji = extrinsic info to be passed from c-node j to v-node iCi = {j: hji = 1} – This is the set of row location of the 1’s in the ith column.

Rj = {i: hji = 1} – This is the set of column location of the 1’s in the jth row

Rj\i= {i’: hji’=1}\{i}– The set of column locations of the 1’s in the jth row, excluding

location i.


93/106

More Notationαij = sign( qij )βij = | qij |φ(x) = -log tanh(x/2) = log( (ex+1)/(ex-1) )= φ-1(x)

0 1 2 3 4 5 60

1

2

3

4

5

6

x

φ(x)


94/106

Sum-Product Decoder (in Log-Domain)

Initialize:– qij = λi = 2Re{ai

*ri}/σ2 = channel LLR valueLoop over all i,j for which hij = 1– At each c-node, update the r messages:

– At each v-node update the q message and Q LLR:

– Make hard decision:

( )⎟⎟⎠

⎞⎜⎜⎝

⎛⎟⎟⎠

⎞⎜⎜⎝

⎛= ∑∏

∈∈ ijij Riji

Rijijir

\\ ''

'' βφφα

jiiij

Cjjiii

rQq

rQi

−=

+= ∑∈

λ

⎩⎨⎧ <

=otherwise0

0 if1ˆ i

i

Qc


95/106

Halting CriteriaAfter each iteration, halt if:

This is effective, because the probability of an undetectible decoding error is negligible.Otherwise, halt once the maximum number of iterations is reached.

0Hc =Tˆ


96/106

Min-Sum Decoder(Log-Domain)

Note that:

So we can replace the r message update formula with

This greatly reduces complexity, since now we don’t have to worry about computing the nonlinear φ function.Note that since α is just the sign of q, ∏α can be implemented by using XOR operations.

( ) ( )( ) ( )( ) jiijiijiiiji '''''''

' minminmax ββφφβφφβφφ ==≈⎟⎠

⎞⎜⎝

⎛∑

jiRiRi

jijiij

ij

r '''

'\

\

min βα∈

∈⎟⎟⎠

⎞⎜⎜⎝

⎛= ∏


97/106

Extrinsic-information ScalingAs with max-log-MAP decoding of turbo codes, min-sum decoding of LDPC codes produces an extrinsic information estimate which is biased.– In particular, rji is overly optimistic.

A significant performance improvement can be achieved by multiplying rji by a constant κ, where κ<1.

– See: J. Heo, “Analysis of scaling soft information on low density parity check code,” IEE Electronic Letters, 23rd Jan. 2003.

Here we compare the performance of sum-product, min-sum, and min-sum with extrinsic info scaling.– Experimentation shows that κ=0.9 gives best performance.

jiRiRi

jijiij

ij

r '''

'\

\

min βακ∈

∈⎟⎟⎠

⎞⎜⎜⎝

⎛= ∏

BER of Different Decoding Algorithms

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.810 -7

10 -6

10 -5

10 -4

10 -3

10 -2

10 -1

Eb/No in dB

Min-sum

Min-sumw/ extrinsic info scaling

Scale factor κ=0.9

Sum-product

Code #1:MacKay’s construction 2AAWGN channelBPSK modulation

BE

R

FER of Different Decoding Algorithms

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8

10 -4

10 -3

10 -2

10 -1

10 0

Eb/No in dB

Code #1:MacKay’s construction 2AAWGN channelBPSK modulation

Min-sum

Min-sumw/ extrinsic info scaling

Scale factor κ=0.9

FER

Sum-product


100/106

Complexity of Turbo DecodingConsider the complexity of an efficiently implemented max-log-MAP decoding of the UMTS turbo code.– **See: M.C. Valenti and J. Sun, “The UMTS turbo code and an efficient

decoder implementation suitable for software defined radios,” Int. J. Wireless Info. Networks, vol. 8, no. 4, pp. 203-216, Oct. 2001.

Forward Recursion.– For each stage of the trellis, compute 22 = 4 branch metrics.

• This only requires 1 addition**– Compute the pair of partial path metrics entering each node

• This requires 12 additions**– Select from among the two branches entering each node.

• This requires 8 pairwise comparisons.– Normalization

• Requires 7 subtractions** (subtraction has same complexity as addition)– Thus the forward recursion requires (where k is # info bits):

• 20k additions.• 8k comparisons.


101/106

Complexity of Turbo DecodingBackward Recursion– Complexity is identical to forward recursion

LLR Calculation:– Likelihood of one: Compare 8 branches*– Likelhood of zero: Compare 7 branches*– So, 13 pairwise comparisons per trellis section.– Subtract the two likelihoods– Thus LLR Calculation requires:

• k additions• 13k comparisons

Extra overhead:– Need to subtract input from output LLR to get extrinsic info, and

need to add extrinsic info to channel LLR.– Amounts to an extra 3k additions / iteration.**


102/106

Summary: Complexity of Turbo Decoding

Since each SISO decoder is executed twice per iteration, the per iteration decoding complexity of the UMTS turbo code is:– 44k additions / iteration– 39k pairwise comparisons / iteration– where k is the number of data bits

If log-MAP decoding is used instead of max-log-MAP, then the 39k pairwise comparisons are replaced with max* operators.This neglects the extra overhead of detecting the data bits, implementing an adaptive stopping criterion, or scaling the extrinsic information.


103/106

Complexity of Min-SumDecoding of LDPC Codes

Let mi be the number of rows of H with weight i.Let ni be the number of columns of H with weight i.Update r:

– For a c-node of degree i, there will be i(i-2) xors and i(i-2) pairwise comparisons.

– Thus overall complexity of the update r stage is:

Update Q:

– For v-node of degree i, this requires i additions.– So overall complexity of the update Q stage is:

jiRiRi

jijiij

ij

r '''

'\

\

min βα∈

∈⎟⎟⎠

⎞⎜⎜⎝

⎛= ∏

scomparison pairwise )2( and xors)2( ∑∑ −−i

ii

i iimiim

∑∈

+=iCj

jiii rQ λ

additions ∑ iini


104/106

Complexity of Min-SumDecoding of LDPC Codes

Update q:

– This is a single subtraction at every branch leaving a v-node, and thus the complexity is also:

And so the per iteration complexity of min-sum decoding of LDPC codes is:

This neglects the extra overhead of detecting the data bits, checking for errors, or scaling the extrinsic information.

jiiij rQq −=

additions ∑i

iin

additions2

scomparison pairwise)2(

xors)2(

∑∑∑

−

−

ii

ii

ii

in

iim

iim


105/106

Complexity of Sum-Product Decoding

If sum-product decoding is used, then

– Thus every comparison operation is replaced with an addition.• Complexity can be slightly reduced by first taking the sum of all φ( β)

and then subtracting off the particular βi’

– Also, there will now be two nonlinear φ function calls per iteration for every edge in the graph

• One for every β and another for every r.

( )⎟⎟⎠

⎞⎜⎜⎝

⎛⎟⎟⎠

⎞⎜⎜⎝

⎛= ∑∏

∈∈ ijij Riji

Rijijir

\\ ''

'' βφφα


106/106

Complexity ComparisonHere we compare the complexity of the (15000, 4996) turbo code with the four (15000,5000) LDPC codes.– Per iteration complexity.– Overall, assuming 5 turbo iterations and 30 LDPC iterations.

* For LDPC codes, one XOR is also required for every pairwise comparison.

Per iteration complexity Overall ComplexityCodeadditions comparisons* additions comparisons*

Turbo 219,824 194,844 1,099,120 974,220LDPC #1 (MacKay 2A) 80,000 80,000 2,400,000 2,400,000LDPC #2 (R&U) 119,974 239,883 3,599,220 7,196,490LDPC #3 (JWT) 109,084 190,878 3,272,520 5,726,340LDPC #4 (extended IRA) 79,998 114,995 2,399,940 3,449,850

Documents

Iterative Solutions Coded Modulation Library Theory of ... fileIterative Solutions Coded Modulation Library Theory of Operation Oct. 3, 2005 Matthew Valenti Associate Professor West