Local Theory of BER for LDPC Codes: Instantons on a Tree

Vladimir ChernyakDepartment of ChemistryWayne State University

In collaboration with:Misha Chertkov (LANL)Misha Stepanov (LANL)Bane Vasic (Arizona)

Special thanks:Fred Cohen (Rochester)

Outline

• Introduction and terminology: Linear Block and LDPC codes, parity checks and Tanner graphs

• Effective spin models for decoding: sMAP and BP approaches• Local and global structures of LDPC codes: The role of trees• Instanton (optimal fluctuation) approach to BER• Low SNR case: High-symmetry “local” instantons, Shannon

transition• High SNR case: Low-symmetry “global” instantons• From high to low SNR: Instantons with intermediate

symmetries• Towards non-tree instantons: High SNR case, quasi-

instantons and related painted structures• Summary and future plans

Linear block codes (parity check representation)

0

0

0

0

0

10

9

8

7

6

5

4

3

2

1

x

x

x

x

x

x

x

x

x

x

H mod 2

M

ii

ii xi

1

1,

1exp

M

Ni

,,1

,,1

Parity check matrix

- “spin” variables

Tanner graph

- set of constraints

Linear block codes and Tanner graphs

210 ,, XXXX

XX 22 201 XXX

NX 0 MX 2

20

1XXj

j nmX

jXj 1,

2,1 XCj

jX

1;0

jXjj

XX CCXX ~

2,ZMGLG

variable (bit) nodesconnections

checking nodes

words (spin representation)

code words

Equivalent codes (gauge invariance)

Gauge group

Effective spin models and decoding approaches

XC Xiiijj hhZm

0

exp)(tanh 1

j i

jiijj hm tanhtanhtanh 1

0

tanhtanh

)0(

)(1)1(

j

j i

ji

lij

lj h

j i

jiijj h tanhtanh 1

jjdj msgnsgn

Set of magnetic fields (measurement outcome) = log-likelihoods

sMAP decoding(gauge invariant)

auxiliary variables defined on connections“Approximate” gauge non-invariantschemes

Iterative belief propagation (BP) Belief propagation (BP) equation

All three schemes are equivalent in the loop-free case

Stat Mech interpretation was suggested byN. Sourlas (Nature ‘89)

Gallager ’63; Pearl ’88; MacKay ’99magnetization

=a-posteriori log-likelihoods

Post FEC bit-error rate (BER) and instantons

hFhmdhsP j

N

j

exp22/2

0

jj PdB

0

22122Xj

j shshF

0

hhj hmhF

h

0 hm j hFPj exp1

hFB j exp10

PDF of magnetizationProbability of a measurement outcome

Probability of a bit error Gaussian symmetric noise case

Instanton (optimal fluctuation) approach: PDF is dominatedby the most probable noise configuration (saddle point)

SNR

Lagrange factor

Geometry of Tanner graphs

XUj lj XXp :

gnm

XXj

j

12

2

2

2

20

gFXjpaXlaX 1

1;;

nn

mm j

nmYX

nMmN

,

Local structure: Each nodehas a tree “neighborhood”

Universal covering tree(similar to Riemann surfaces)

fundamental group free group with g generators

Gauss-Bonnet theorem (Euler characteristic)genus

local curvature

Graphs with constant curvature

The covering tree is universaland possesses high symmetry

Wiberg ’95Weiss ‘00

BP iterative algorithm and decoding tree

jpXlX 0;;0aaa

al

papa a

1

;0

01 tanhtanh

lX

a

aajj hm

Decoding tree for BP with thefixed number of iterations On a tree the auxiliary field can

be defined in variable nodes

the only in-bound nearest-neighborchecking nodeThe field that represents the

history of iterations

BP magnetization is represented by the fixed point of BPequation (that coincides with sMAP) on the decoding tree

Tree instantons

2

0

212

2

20 0

12

tanh2

1tanh

2

1;;

j

j jk

kkj

k

kk

j

ss

ss

hFQ

j ji

iijj

j

h tanhtanh 1

QB exp10

0;0

j

Q

02 ll

Local theory (no repetitions of magnetic fields)

shortest loop length (girth)

Express magnetic fields in terms of magnetization

Effective action for an instanton problem

High-symmetry low SNR local instantons

Symmetric phase: at any node on the tree depends

primarily on the generation (counted from the center)

112112

212

1

1

0

10

tanhtanh),(;tanhtanh)1(),(

2

),(),())1)(1((

2

)1(;

nn

ljj

l

j

jl

mssgmssg

sgsgnm

nmQ

);,(),(),()1(

),(),(),()1)(1(

1121

11

sgsgsgnm

sgsgsgnm

lnlml

jjjjj

01

for j=0,…,l-2

Symmetric instanton effective action

High symmetry: Shannon transition

201 ;, ssg jj

2sh j l Pcorresponds to the maximum of

),(1

),(

cc

ccc

sg

sg

Shannon’s

transition

Shannon’s transition is a localproperty of a code

cj

cj

ss

ss

;~;

Low-symmetry high SNR global instantons

lXA ;0Painted structure

00 A(i) Contains the tree center(ii) Together with a variable node containsall nearest-neighbor checking nodes(iii) Together with a checking node containsexactly one outbound nearest-neighborchecking node(iv) Minimal subgraph with these properties

02

0

;

;0

Ajsh

Ajh

j

j

High SNR instantons are associated with painted structures

Intermediate instantons with partially-broken symmetry

“0” “2”“1”

“4”“3”

Low SNR(high temp)

High SNR(low temp)

Symmetry is described by partially-painted structures

Instanton phases on a tree

m=4, n=5, l=4. Curves of different colors correspond to

the instantons/phases of different symmetries.

sg

msg

sgssk

sss

ms

nm

ccmcck

l

,1

tanhtanh)1(),(

,:11

121

Full numerical optimization (no symmetry assumed)

Area of a circle surroundingany variable node is proportional to the valueof the noise in the node.

m=2 n=3 l=3

NO MORE TREES

We apply the concept of the covering (decoding) tree

Wiberg ’95Weiss ‘00

For higher SNR instantons reflect theglobal geometry of the Tanner graph(loop structure)

How do instantons look in the high SNR limit?

High SNR instantons for LDPC codes: approximate BP equations

211

21 tanhtanhtanh

k

jjk

1010 sgn...

k

jjk

1010 sgn...

k ...1 10

k ...10

2s

Relevant “multiplication” operation

High SNR limit approximate formula

Reduced variables

Infinite SNR limit “multiplication” formulafor reduced variables

Min-sum

High SNR instantons: painted structure representation

lXA ;0

0;sgn Aj

ji

iij

j

02sm j

00

0

00Xi

ii

k

Akkk hnhsh

0

0

a

Caak

k

s

,A

0;0

2

1

00

22

Xiii

Xiii

i

j

hnhnhs

h

1

2

00

1

Xkk

Xkkjj nnnh

1

2

2

002

1

Xkk

Xkk nnnS

Painted structureDiscrete (Ising) variables

Expressions for magnetization

Quasi-instantons

High SNR instantons: pseudo-code word representation

lXC ;0,

10

lXaa

Xjjj

lXaaa

lXaaa

lXaa

Xjjj hhnhhhhn

;0;0;0;0 000000

22

jjj nnn0

0

Xj

jjhn

Successful (matched) competition of twopseudo-code words

Quasi-instanton relation

0BS

If B is a stopping set (graph)

02

0

;

;0

Bjsh

Bjh

j

j

Summary

• We have analyzed instantons for BER on trees• Depending on SNR BER is dominated by instantons of different

symmetry• Shannon transition for an LDPC code is determined by local

structure of the code (“curvature”)• For BP iterative decoding we have identified candidates that

dominate BER• Adiabatic expanding of instantons from high to lower SNR

Truth …

menu

mainslide