Polar codes – a new paradigm in communication
Alexander Barg
Dept of ECE/Inst. for Systems ResearchUniversity of Maryland, College Park, MD 20742
Dec. 12, 2012
Alexander Barg (UMD) Polar codes Dec. 12, 2012 1 / 63
Overview of the talk
• Introduction: Transmission over channels
• Binary Polar Codes
• Nonbinary Polar Codes
• Improved decoding: Approaching optimal performance
Alexander Barg (UMD) Polar codes Dec. 12, 2012 2 / 63
Introduction: Shannon theorem
Entropy
X - finite set, |X | = q
Elements of X can be labelled using log q bits
Uniform distrubtion PX (x) = 1q
log q = log 1PX (x)
bits suffice to identify x
Suppose PX = (PX (x), x ∈ X ) is a pmf
The expected number of bits is H(X ) =∑
x∈X PX (x) log 1PX (x)
H(X ) is called Shannon entropy
Alexander Barg (UMD) Polar codes Dec. 12, 2012 3 / 63
Introduction: Shannon theorem
Entropy
X - finite set, |X | = q
Elements of X can be labelled using log q bits
Uniform distrubtion PX (x) = 1q
log q = log 1PX (x)
bits suffice to identify x
Suppose PX = (PX (x), x ∈ X ) is a pmf
The expected number of bits is H(X ) =∑
x∈X PX (x) log 1PX (x)
H(X ) is called Shannon entropy
Alexander Barg (UMD) Polar codes Dec. 12, 2012 3 / 63
Introduction: Shannon theorem
Entropy
X - finite set, |X | = q
Elements of X can be labelled using log q bits
Uniform distrubtion PX (x) = 1q
log q = log 1PX (x)
bits suffice to identify x
Suppose PX = (PX (x), x ∈ X ) is a pmf
The expected number of bits is H(X ) =∑
x∈X PX (x) log 1PX (x)
H(X ) is called Shannon entropy
Alexander Barg (UMD) Polar codes Dec. 12, 2012 3 / 63
Introduction: Shannon theorem
Entropy
X - finite set, |X | = q
Elements of X can be labelled using log q bits
Uniform distrubtion PX (x) = 1q
log q = log 1PX (x)
bits suffice to identify x
Suppose PX = (PX (x), x ∈ X ) is a pmf
The expected number of bits is H(X ) =∑
x∈X PX (x) log 1PX (x)
H(X ) is called Shannon entropy
Alexander Barg (UMD) Polar codes Dec. 12, 2012 3 / 63
Introduction: Shannon theorem
Transmission of information
Let W : X → Y be a stochastic mapping, |Y| <∞W (y |x) = Pr(Y = y |X = x)Conditional entropy (residual uncertainty about X given Y )
H(X |Y ) = EXY log1
PX |Y (x |y)
Mutual information I(X ;Y ) := H(X )− H(X |Y )
Theorem (Shannon ’48)
There exists a subset D ⊂ X n such that its images in Yn under W canbe distinguished with high probability as long as |D| < 2nI(W ).
I(W ) = maxPX
I(X ;Y ) is called Channel Capacity
Alexander Barg (UMD) Polar codes Dec. 12, 2012 4 / 63
Introduction: Shannon theorem
Transmission of information
Let W : X → Y be a stochastic mapping, |Y| <∞W (y |x) = Pr(Y = y |X = x)Conditional entropy (residual uncertainty about X given Y )
H(X |Y ) = EXY log1
PX |Y (x |y)
Mutual information I(X ;Y ) := H(X )− H(X |Y )
Theorem (Shannon ’48)
There exists a subset D ⊂ X n such that its images in Yn under W canbe distinguished with high probability as long as |D| < 2nI(W ).
I(W ) = maxPX
I(X ;Y ) is called Channel Capacity
Alexander Barg (UMD) Polar codes Dec. 12, 2012 4 / 63
Introduction: Shannon theorem
Transmission of information
Let W : X → Y be a stochastic mapping, |Y| <∞W (y |x) = Pr(Y = y |X = x)Conditional entropy (residual uncertainty about X given Y )
H(X |Y ) = EXY log1
PX |Y (x |y)
Mutual information I(X ;Y ) := H(X )− H(X |Y )
Theorem (Shannon ’48)
There exists a subset D ⊂ X n such that its images in Yn under W canbe distinguished with high probability as long as |D| < 2nI(W ).
I(W ) = maxPX
I(X ;Y ) is called Channel Capacity
Alexander Barg (UMD) Polar codes Dec. 12, 2012 4 / 63
Introduction: Shannon theorem
Transmission of information
Let W : X → Y be a stochastic mapping, |Y| <∞W (y |x) = Pr(Y = y |X = x)Conditional entropy (residual uncertainty about X given Y )
H(X |Y ) = EXY log1
PX |Y (x |y)
Mutual information I(X ;Y ) := H(X )− H(X |Y ) = log q − H(X |Y )
Theorem (Shannon ’48)
There exists a subset D ⊂ X n such that its images in Yn under W canbe distinguished with high probability as long as |D| < 2nI(X ;Y ).
I(W ) = log q − H(X |Y ) (uniform) Channel Capacity
Alexander Barg (UMD) Polar codes Dec. 12, 2012 5 / 63
Polar codes Binary Polar Codes
Binary polar codes (Arikan ’09)Discrete memoryless channel W : X → Y with capacity I(W )
y11uW
Alexander Barg (UMD) Polar codes Dec. 12, 2012 6 / 63
Polar codes Binary Polar Codes
Binary polar codes (Arikan ’09)
Discrete memoryless channel W : X → Y with capacity I(W )
x1y1
Wu1
Wu x2 2 y2
2I(W ) = I(U1U2;Y1Y2) = I(U1;Y1Y2) + I(U2;Y1Y2|U1)
= I(U1;Y1Y2) + I(U2;Y1Y2U1)
W−(y1y2|u1) =12
1∑u2=0
W (y1|u1 ⊕ u2)W (y2|u2)
W+(y1y2,u1|u2) =12
W (y1|u1 ⊕ u2)W (y2|u2).
Alexander Barg (UMD) Polar codes Dec. 12, 2012 7 / 63
Polar codes Binary Polar Codes
Binary polar codes (Arikan ’09)
Discrete memoryless channel W : X → Y with capacity I(W )
x1y1
Wu1
Wu x2 2 y2
2I(W ) = I(U1U2;Y1Y2) = I(U1;Y1Y2) + I(U2;Y1Y2|U1)
= I(U1;Y1Y2) + I(U2;Y1Y2U1)
W−(y1y2|u1) =12
1∑u2=0
W (y1|u1 ⊕ u2)W (y2|u2)
W+(y1y2,u1|u2) =12
W (y1|u1 ⊕ u2)W (y2|u2).
Alexander Barg (UMD) Polar codes Dec. 12, 2012 7 / 63
Polar codes Binary Polar Codes
Binary polar codes (Arikan ’09)
Discrete memoryless channel W : X → Y with capacity I(W )
x1y1
Wu1
Wu x2 2 y2
2I(W ) = I(U1U2;Y1Y2) = I(U1;Y1Y2) + I(U2;Y1Y2|U1)
= I(U1;Y1Y2) + I(U2;Y1Y2U1)
W−(y1y2|u1) =12
1∑u2=0
W (y1|u1 ⊕ u2)W (y2|u2)
W+(y1y2,u1|u2) =12
W (y1|u1 ⊕ u2)W (y2|u2).
Alexander Barg (UMD) Polar codes Dec. 12, 2012 7 / 63
Polar codes Binary Polar Codes
Binary polar codes (Arikan ’09)
Discrete memoryless channel W : X → Y with capacity I(W )
x1y1
Wu1
Wu x2 2 y2
2I(W ) = I(U1;Y1Y2) + I(U2;Y1Y2,U1)
W−(y1y2|u1) =12
1∑u2=0
W (y1|u1 ⊕ u2)W (y2|u2)
W+(y1y2,u1|u2) =12
W (y1|u1 ⊕ u2)W (y2|u2)
I(W+) ≥ I(W ) ≥ I(W−)
Alexander Barg (UMD) Polar codes Dec. 12, 2012 8 / 63
Polar codes Binary Polar Codes
Binary polar codes (Arikan ’09)
Discrete memoryless channel W : X → Y with capacity I(W )
2y
y1
y3
4
x2
x3
x1
u2
u1
u3
u4
W
W
W
Wx y
4
W++,W+−,W−+,W−−
Alexander Barg (UMD) Polar codes Dec. 12, 2012 9 / 63
Polar codes Binary Polar Codes
Binary polar codes (Arikan ’09)
Discrete memoryless channel W : X → Y with capacity I(W )
2y
y1
y3
4
x 2
x 3
x 1
x 5
x 6
x 7
y5
y7
8
u 1
u 2
u 3
u 4
u 5
u 6
u 7
u 8
W
W
W
Wy
W
W
W
Wx y4
x
y6
8
W+++,W++−, . . . ,W−−−
Alexander Barg (UMD) Polar codes Dec. 12, 2012 10 / 63
Polar codes Binary Polar Codes
Binary polar codes, BEC(0.5)
I(W ) = 0.5
I(W−) = 0.25, I(W+) = 0.75
I(W−−) = 0.0625, I(W−+) = 0.4375, I(W+−) = 0.5625, I(W++) = 0.9375
Alexander Barg (UMD) Polar codes Dec. 12, 2012 11 / 63
Polar codes Binary Polar Codes
Binary polar codes, BEC(0.5)
I(W ) = 0.5
I(W−) = 0.25, I(W+) = 0.75
I(W−−) = 0.0625, I(W−+) = 0.4375, I(W+−) = 0.5625, I(W++) = 0.9375
Alexander Barg (UMD) Polar codes Dec. 12, 2012 11 / 63
Polar codes Binary Polar Codes
Binary polar codes, BEC(0.5)
I(W ) = 0.5
I(W−) = 0.25, I(W+) = 0.75
I(W−−) = 0.0625, I(W−+) = 0.4375, I(W+−) = 0.5625, I(W++) = 0.9375
Alexander Barg (UMD) Polar codes Dec. 12, 2012 11 / 63
Polar codes Binary Polar Codes
Polarization for BEC(0.5), N = 16
æ æ æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ æ æ
0 2 4 6 8 10 12 140.0
0.2
0.4
0.6
0.8
1.0
Channel Index
Cap
acity
Alexander Barg (UMD) Polar codes Dec. 12, 2012 12 / 63
Polar codes Binary Polar Codes
Polarization for BEC(0.5), N = 32
æ æ ææ
ææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
ææ
ææ
æ æ æ
0 5 10 15 20 25 300.0
0.2
0.4
0.6
0.8
1.0
Channel Index
Cap
acity
Alexander Barg (UMD) Polar codes Dec. 12, 2012 13 / 63
Polar codes Binary Polar Codes
Polarization for BEC(0.5), N = 64
æææææææ
æ
æææ
æ
æ
æ
æ
æ
æææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
ææ
æ
ææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææ
æ
æ
æ
æ
æ
æææ
æ
æææææææ
0 10 20 30 40 50 600.0
0.2
0.4
0.6
0.8
1.0
Channel Index
Cap
acity
Alexander Barg (UMD) Polar codes Dec. 12, 2012 14 / 63
Polar codes Binary Polar Codes
Polarization for BEC(0.5), N = 128
æææææææææææææææ
æ
æææææææ
æ
æææ
æ
æ
æ
æ
æ
æææææææ
æ
æææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææ
æææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææ
æ
æææææææ
æ
æ
æ
æ
æ
æææ
æ
æææææææ
æ
æææææææææææææææ
0 20 40 60 80 100 1200.0
0.2
0.4
0.6
0.8
1.0
Channel Index
Cap
acity
Alexander Barg (UMD) Polar codes Dec. 12, 2012 15 / 63
Polar codes Binary Polar Codes
Polarization for BEC(0.5), N = 256
æææææææææææææææææææææææææææææææ
æ
æææææææææææææææ
æ
æææææææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææææææææææææ
ææ
æ
æææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææ
æ
æ
æ
æ
æ
æææ
æ
æææææææ
æææææææ
æ
æææ
æ
æ
æ
æ
æ
æææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææ
æ
ææ
æææææææææææææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææææææ
æ
æææææææææææææææ
æ
æææææææææææææææææææææææææææææææ
0 50 100 150 200 2500.0
0.2
0.4
0.6
0.8
1.0
Channel Index
Cap
acity
Alexander Barg (UMD) Polar codes Dec. 12, 2012 16 / 63
Polar codes Binary Polar Codes
Polarization for BEC(0.5), N = 512
æææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææ
æ
æææææææææææææææææææææææææææææææ
æ
æææææææææææ
æ
æ
æ
æ
æ
æææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
ææ
æ
æææææææææææææææææææææææ
æ
æææ
æ
æ
æ
æ
æ
æææææææ
æ
æææ
æ
æ
æ
æ
æ
æææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææ
æææææææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææææ
æ
æ
æ
æ
æ
æææ
æ
æææææææ
æ
æææææææææææææææ
æææææææææææææææ
æ
æææææææ
æ
æææ
æ
æ
æ
æ
æ
æææææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææææææ
æææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææ
æ
æ
æ
æ
æ
æææ
æ
æææææææ
æ
æ
æ
æ
æ
æææ
æ
æææææææææææææææææææææææ
æ
ææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææ
æ
æ
æ
æ
æ
æææææææææææ
æ
æææææææææææææææææææææææææææææææ
æ
æææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææ
0 100 200 300 400 5000.0
0.2
0.4
0.6
0.8
1.0
Channel Index
Cap
acity
Alexander Barg (UMD) Polar codes Dec. 12, 2012 17 / 63
Polar codes Binary Polar Codes
Polarization for BEC(0.5), N = 1024
æææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææ
æ
æææææææææææææææææææææææææææææææææææææææææææææææææææææææææææ
æ
æ
æ
æ
æ
æææææææææææææææ
æ
æææææææ
æ
æææ
æ
æ
æ
æ
æ
æææææææ
æ
æææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææ
æææææææææææææææææææææææææææææææ
æ
æææææææææææææææ
æ
æææææææ
æ
æææ
æ
æ
æ
æ
æ
æææææææææææææææ
æ
æææææææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææ
æ
æææææææ
æææææææææææ
æ
æ
æ
æ
æ
æææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
ææ
æ
æææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææ
æ
æ
æ
æ
æ
æææ
æ
æææææææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææ
æ
ææ
æææææææææææææ
æ
æææææææææææææææææææææææææææææææ
æææææææææææææææææææææææææææææææ
æ
æææææææææææææ
ææ
æ
æææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææææææ
æ
æææ
æ
æ
æ
æ
æ
æææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææ
æ
ææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææ
æ
æ
æ
æ
æ
æææææææææææ
æææææææ
æ
æææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææææææ
æ
æææææææææææææææ
æ
æ
æ
æ
æ
æææ
æ
æææææææ
æ
æææææææææææææææ
æ
æææææææææææææææææææææææææææææææ
æææ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æ
æææ
æ
æææææææ
æ
æ
æ
æ
æ
æææ
æ
æææææææ
æ
æææææææææææææææ
æ
æ
æ
æ
æ
æææææææææææææææææææææææææææææææææææææææææææææææææææææææææææ
æ
æææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææ
0 200 400 600 800 10000.0
0.2
0.4
0.6
0.8
1.0
Channel Index
Cap
acity
Alexander Barg (UMD) Polar codes Dec. 12, 2012 18 / 63
Polar codes Binary Polar Codes
Binary polar codes: ordered bits (N = 24)
0.2 0.4 0.6 0.8 1.0
i
N
0.2
0.4
0.6
0.8
1.0
IHWNIiML
Alexander Barg (UMD) Polar codes Dec. 12, 2012 19 / 63
Polar codes Binary Polar Codes
Binary polar codes: ordered bits
0.2 0.4 0.6 0.8 1.0
i
N
0.2
0.4
0.6
0.8
1.0
IHWNIiML
Alexander Barg (UMD) Polar codes Dec. 12, 2012 20 / 63
Polar codes Binary Polar Codes
Binary polar codes: ordered bits
0.2 0.4 0.6 0.8 1.0
i
N
0.2
0.4
0.6
0.8
1.0
IHWNIiML
Alexander Barg (UMD) Polar codes Dec. 12, 2012 21 / 63
Polar codes Binary Polar Codes
Binary polar codes: ordered bits
0.2 0.4 0.6 0.8 1.0
i
N
0.2
0.4
0.6
0.8
1.0
IHWNIiML
Alexander Barg (UMD) Polar codes Dec. 12, 2012 22 / 63
Polar codes Binary Polar Codes
Binary polar codes: ordered bits
0.2 0.4 0.6 0.8 1.0
i
N
0.2
0.4
0.6
0.8
1.0
IHWNIiML
Alexander Barg (UMD) Polar codes Dec. 12, 2012 23 / 63
Polar codes Binary Polar Codes
Binary polar codes: ordered bits
0.2 0.4 0.6 0.8 1.0
i
N
0.2
0.4
0.6
0.8
1.0
IHWNIiML
Alexander Barg (UMD) Polar codes Dec. 12, 2012 24 / 63
Polar codes Binary Polar Codes
Binary polar codes: ordered bits
0.2 0.4 0.6 0.8 1.0
i
N
0.2
0.4
0.6
0.8
1.0
IHWNIiML
Alexander Barg (UMD) Polar codes Dec. 12, 2012 25 / 63
Polar codes Binary Polar Codes
Binary polar codes: ordered bits
0.2 0.4 0.6 0.8 1.0
i
N
0.2
0.4
0.6
0.8
1.0
IHWNIiML
Alexander Barg (UMD) Polar codes Dec. 12, 2012 26 / 63
Polar codes Binary Polar Codes
Binary polar codes: ordered bits
0.2 0.4 0.6 0.8 1.0
i
N
0.2
0.4
0.6
0.8
1.0
IHWNIiML
Alexander Barg (UMD) Polar codes Dec. 12, 2012 27 / 63
Polar codes Binary Polar Codes
Binary polar codes: ordered bits
0.2 0.4 0.6 0.8 1.0
i
N
0.2
0.4
0.6
0.8
1.0
IHWNIiML
Alexander Barg (UMD) Polar codes Dec. 12, 2012 28 / 63
Polar codes Binary Polar Codes
Binary polar codes: ordered bits, N = 215
0.2 0.4 0.6 0.8 1.0
i
N
0.2
0.4
0.6
0.8
1.0
IHWNIiML
Alexander Barg (UMD) Polar codes Dec. 12, 2012 29 / 63
Polar codes Binary Polar Codes
Binary polar codes: ordered bits
0.2 0.4 0.6 0.8 1.0
i
N
0.2
0.4
0.6
0.8
1.0
IHWNIiML
Alexander Barg (UMD) Polar codes Dec. 12, 2012 30 / 63
Polar codes Binary Polar Codes
Encoding mapC(Wi)
0.0039
0.1211
0.1914
0.6836
0.3164
0.8086
0.8789
0.9961
2y
y1
y3
4
x 2
x 3
x 1
x 5
x 6
x 7
y5
y7
8
u 1
u 2
u 3
u 4
u 5
u 6
u 7
u 8
W
W
W
Wy
W
W
W
Wx y4
x
y6
8
Alexander Barg (UMD) Polar codes Dec. 12, 2012 31 / 63
Polar codes Binary Polar Codes
Encoding mapC(Wi) rank
0.0039 8
0.1211 7
0.1914 6
0.6836 4
0.3164 5
0.8086 3
0.8789 2
0.9961 1
2y
y1
y3
4
x 2
x 3
x 1
x 5
x 6
x 7
y5
y7
8
u 1
u 2
u 3
u 4
u 5
u 6
u 7
u 8
W
W
W
Wy
W
W
W
Wx y4
x
y6
8
Alexander Barg (UMD) Polar codes Dec. 12, 2012 32 / 63
Polar codes Binary Polar Codes
Encoding mapC(Wi) rank
0.0039 8
0.1211 7
0.1914 6
0.6836 4
0.3164 5
0.8086 3
0.8789 2
0.9961 1 data
2y
y1
y3
4
x 2
x 3
x 1
x 5
x 6
x 7
y5
y7
8
u 1
u 2
u 3
u 4
u 5
u 6
u 7
u 8
W
W
W
Wy
W
W
W
Wx y4
x
y6
8
Alexander Barg (UMD) Polar codes Dec. 12, 2012 33 / 63
Polar codes Binary Polar Codes
Encoding mapC(Wi) rank
0.0039 8
0.1211 7
0.1914 6
0.6836 4
0.3164 5
0.8086 3
0.8789 2 data
0.9961 1 data
2y
y1
y3
4
x 2
x 3
x 1
x 5
x 6
x 7
y5
y7
8
u 1
u 2
u 3
u 4
u 5
u 6
u 7
u 8
W
W
W
Wy
W
W
W
Wx y4
x
y6
8
Alexander Barg (UMD) Polar codes Dec. 12, 2012 34 / 63
Polar codes Binary Polar Codes
Encoding mapC(Wi) rank
0.0039 8
0.1211 7
0.1914 6
0.6836 4
0.3164 5
0.8086 3 data
0.8789 2 data
0.9961 1 data
2y
y1
y3
4
x 2
x 3
x 1
x 5
x 6
x 7
y5
y7
8
u 1
u 2
u 3
u 4
u 5
u 6
u 7
u 8
W
W
W
Wy
W
W
W
Wx y4
x
y6
8
Alexander Barg (UMD) Polar codes Dec. 12, 2012 35 / 63
Polar codes Binary Polar Codes
Encoding mapC(Wi) rank
0.0039 8
0.1211 7
0.1914 6
0.6836 4 data
0.3164 5
0.8086 3 data
0.8789 2 data
0.9961 1 data
2y
y1
y3
4
x 2
x 3
x 1
x 5
x 6
x 7
y5
y7
8
u 1
u 2
u 3
u 4
u 5
u 6
u 7
u 8
W
W
W
Wy
W
W
W
Wx y4
x
y6
8
Alexander Barg (UMD) Polar codes Dec. 12, 2012 36 / 63
Polar codes Binary Polar Codes
Encoding mapC(Wi) rank
0.0039 8 frozen
0.1211 7 frozen
0.1914 6 frozen
0.6836 4 data
0.3164 5 frozen
0.8086 3 data
0.8789 2 data
0.9961 1 data
2y
y1
y3
4
x 2
x 3
x 1
x 5
x 6
x 7
y5
y7
8
u 1
u 2
u 3
u 4
u 5
u 6
u 7
u 8
W
W
W
Wy
W
W
W
Wx y4
x
y6
8
Alexander Barg (UMD) Polar codes Dec. 12, 2012 37 / 63
Polar codes Binary Polar Codes
Encoding mapC(Wi) rank
0.0039 8 0
0.1211 7 0
0.1914 6 0
0.6836 4 data
0.3164 5 0
0.8086 3 data
0.8789 2 data
0.9961 1 data
2y
y1
y3
4
x 2
x 3
x 1
x 5
x 6
x 7
y5
y7
8
u 1
u 2
u 3
u 4
u 5
u 6
u 7
u 8
W
W
W
Wy
W
W
W
Wx y4
x
y6
8
Alexander Barg (UMD) Polar codes Dec. 12, 2012 38 / 63
Polar codes Binary Polar Codes
Encoding mapC(Wi) rank
0.0039 8 0
0.1211 7 0
0.1914 6 0
0.6836 4 data
0.3164 5 0
0.8086 3 data
0.8789 2 data
0.9961 1 data
2y
y1
y3
4
x 2
x 3
x 1
x 5
x 6
x 7
y5
y7
8
u 1
u 2
u 3
u 4
u 5
u 6
u 7
u 8
W
W
W
Wy
W
W
W
Wx y4
x
y6
8
Alexander Barg (UMD) Polar codes Dec. 12, 2012 39 / 63
Polar codes Binary Polar Codes
Encoding map
(u1,u2)H2 = (x1, x2)t , where H2 =
(1 01 1
).
x41 = u4
1
1 0 0 01 1 0 01 0 1 01 1 1 1
2y
y1
y3
4
x2
x3
x1
u2
u1
u3
u4
W
W
W
Wx y
4
Generally xN1 = uN
1 HN , where HN = H⊗n2 ,N = 2n
The set {1, . . . ,N} contains NI(W ) indices such that I(W s) ≈ 1.
Alexander Barg (UMD) Polar codes Dec. 12, 2012 40 / 63
Polar codes Binary Polar Codes
Encoding map
(u1,u2)H2 = (x1, x2)t , where H2 =
(1 01 1
).
x41 = u4
1
1 0 0 01 1 0 01 0 1 01 1 1 1
2y
y1
y3
4
x2
x3
x1
u 2
u1
u3
u4
W
W
W
Wx y
4
Generally xN1 = uN
1 HN , where HN = H⊗n2 ,N = 2n
The set {1, . . . ,N} contains NI(W ) indices such that I(W s) ≈ 1.
Alexander Barg (UMD) Polar codes Dec. 12, 2012 40 / 63
Polar codes Binary Polar Codes
Encoding map
(u1,u2)H2 = (x1, x2)t , where H2 =
(1 01 1
).
x41 = u4
1
1 0 0 01 1 0 01 0 1 01 1 1 1
2y
y1
y3
4
x2
x3
x1
u 2
u1
u3
u4
W
W
W
Wx y
4
Generally xN1 = uN
1 HN , where HN = H⊗n2 ,N = 2n
The set {1, . . . ,N} contains NI(W ) indices such that I(W s) ≈ 1.
Alexander Barg (UMD) Polar codes Dec. 12, 2012 40 / 63
Polar codes Binary Polar Codes
Binary polar codes: Convergence
Why this works:
I(W−) + I(W+) = 2I(W )
After n steps we obtain 2n values In = {Ib,b ∈ {+,−}n}
Introduce a uniform distribution on In : Pn(Ib) = 2−n for all bConsider the random process In,n ≥ 1.
The sequence In forms a bounded martingale: E(In+1|Fn) = In
Ina.s.→ I∞
I∞ ∈ {0,1}, EI∞ = I(W )
Alexander Barg (UMD) Polar codes Dec. 12, 2012 41 / 63
Polar codes Binary Polar Codes
Binary polar codes: Convergence
Why this works:
I(W−) + I(W+) = 2I(W )
After n steps we obtain 2n values In = {Ib,b ∈ {+,−}n}
Introduce a uniform distribution on In : Pn(Ib) = 2−n for all bConsider the random process In,n ≥ 1.
The sequence In forms a bounded martingale: E(In+1|Fn) = In
Ina.s.→ I∞
I∞ ∈ {0,1}, EI∞ = I(W )
Alexander Barg (UMD) Polar codes Dec. 12, 2012 41 / 63
Polar codes Binary Polar Codes
Binary polar codes: Convergence
For any ε > 0
limn→∞
|{b ∈ {+,−}n : I(W b) ∈ (ε,1− ε)}|2n = 0.
Alexander Barg (UMD) Polar codes Dec. 12, 2012 42 / 63
Polar codes Binary Polar Codes
Decoding of polar codes
Let AN ⊂ {1, . . . ,N} be the set of bits used to transmit data
Successive cancellation (SC decoding, Arikan ’09)
ui =
{arg maxz∈{0,1}W (yN
1 , ui−11 |z) if i ∈ F c
N
0 if i ∈ FN .
Rate of polarization (Arikan-Telatar ’09)The decline rate of BER is given by O(2−
√N)
Alexander Barg (UMD) Polar codes Dec. 12, 2012 43 / 63
Polar codes Binary Polar Codes
Decoding of polar codes
Let AN ⊂ {1, . . . ,N} be the set of bits used to transmit data
Successive cancellation (SC decoding, Arikan ’09)
ui =
{arg maxz∈{0,1}W (yN
1 , ui−11 |z) if i ∈ F c
N
0 if i ∈ FN .
Rate of polarization (Arikan-Telatar ’09)The decline rate of BER is given by O(2−
√N)
Alexander Barg (UMD) Polar codes Dec. 12, 2012 43 / 63
Polar codes q-ary polar codes
q-ary polar codes, q = 2r
Arikan’s kernel: (x1, x2) = (u1,u2)H2 where H2 =(1 0
1 1
).
(x1, x2, . . . , xN) = (u1,u2, . . . ,uN)(H2)⊗n, N = 2n
(x1, x2, . . . , x8) = (u1, u2, . . . , u8)
1 0 0 0 0 0 0 01 1 0 0 0 0 0 01 0 1 0 0 0 0 01 1 1 1 0 0 0 01 0 0 0 1 0 0 01 1 0 0 1 1 0 01 0 1 0 1 0 1 01 1 1 1 1 1 1 1
Alexander Barg (UMD) Polar codes Dec. 12, 2012 44 / 63
Polar codes q-ary polar codes
q-ary polar codes, q = 2r
Arikan’s kernel: (x1, x2) = (u1,u2)H2 where H2 =(1 0
1 1
).
(x1, x2, . . . , xN) = (u1,u2, . . . ,uN)(H2)⊗n, N = 2n
(x1, x2, . . . , x8) = (u1, u2, . . . , u8)
1 0 0 0 0 0 0 01 1 0 0 0 0 0 01 0 1 0 0 0 0 01 1 1 1 0 0 0 01 0 0 0 1 0 0 01 1 0 0 1 1 0 01 0 1 0 1 0 1 01 1 1 1 1 1 1 1
Alexander Barg (UMD) Polar codes Dec. 12, 2012 44 / 63
Polar codes q-ary polar codes
q-ary polar codes, q = 2r
Arikan’s kernel: (x1, x2) = (u1,u2)H2 where H2 =(1 0
1 1
).
(x1, x2, . . . , xN) = (u1,u2, . . . ,uN)(H2)⊗n, N = 2n
(x1, x2, . . . , x8) = (u1, u2, . . . , u8)
1 0 0 0 0 0 0 01 1 0 0 0 0 0 01 0 1 0 0 0 0 01 1 1 1 0 0 0 01 0 0 0 1 0 0 01 1 0 0 1 1 0 01 0 1 0 1 0 1 01 1 1 1 1 1 1 1
Alexander Barg (UMD) Polar codes Dec. 12, 2012 44 / 63
Polar codes for q-ary alphabets, q = 2r
Polar codes for q-ary alphabets, q = 2r
Let Wn be the random channel at step n,
Pr(Wn = W B,B ∈ {+,−}n) = 2−n
In = I(Wn) – symmetric capacity
TheoremIn → I∞ a.e., where I∞ is supported on the set {0,1, . . . , r} andEI∞ = I(W ).
Alexander Barg (UMD) Polar codes Dec. 12, 2012 45 / 63
Polar codes for q-ary alphabets, q = 2r Extremal configurations
Extremal configurations
The virtual channels converge to one of r + 1 possibilities:
1 1 1 . . . 1 10 1 1 . . . 1 10 0 1 . . . 1 1...
......
0 0 0 . . . 0 10 0 0 . . . 0 0
Alexander Barg (UMD) Polar codes Dec. 12, 2012 46 / 63
Polar codes for q-ary alphabets, q = 2r Extremal configurations
Extremal configurations
Define the channel ”for the last k bits”:
W [k ](y |u) = 12r−k
∑x∈X :x r
r−k+1=u
W (y |x), u ∈ {0,1}k
Theorem
For any DMC W : X → Y the channels W (i)N polarize to one of the
r + 1 extremal configurations. Namely, let Vi = W (i)N and
πk ,N =|{i ∈ [N] : |I(Vi)− k | < δ ∧ |I(V [k ]
i )− k | < δ}|N
,
where δ > 0, then limN→∞ πk ,N = P(I∞ = k) for all k = 0,1, . . . , r .Consequently
r∑k=1
kπk → I(W ).
Alexander Barg (UMD) Polar codes Dec. 12, 2012 47 / 63
Polar codes for q-ary alphabets, q = 2r Extremal configurations
Extremal configurations: Example시트5
페이지 1
0 2048 4096 6144 8192 10240 12288 14336 163840
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
Channel Index
Cap
acity
A.B, W. Park, Polar codes for q-ary channels, q = 2r , IEEE Trans, Inform. Theory, in press, arXiv:1107.4965v3
Notes for polar codes: http://www.ece.umd.edu/∼abarg/627/polar.pdf
Alexander Barg (UMD) Polar codes Dec. 12, 2012 48 / 63
Polar codes for q-ary alphabets, q = 2r Extremal configurations
Extremal configurations: Example시트5
페이지 1
0 2048 4096 6144 8192 10240 12288 14336 163840
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
Channel Index
Cap
acity
A.B, W. Park, Polar codes for q-ary channels, q = 2r , IEEE Trans, Inform. Theory, in press, arXiv:1107.4965v3
Notes for polar codes: http://www.ece.umd.edu/∼abarg/627/polar.pdf
Alexander Barg (UMD) Polar codes Dec. 12, 2012 48 / 63
RM codes
Polar codes and RM codes
Polar codes are related to classical Reed-Muller codesRM(m, r) a code of length N = 2n, k =
∑ri=0
(mi
)data symbols, distance 2m−r
1000 0000 0000 00001100 0000 0000 00001010 0000 0000 00001111 0000 0000 00001000 1000 0000 00001100 1100 0000 00001010 1010 0000 00001111 1111 0000 00001000 0000 1000 00001100 0000 1100 00001010 0000 1010 00001111 0000 1111 00001000 1000 1000 10001100 1100 1100 11001010 1010 1010 10101111 1111 1111 1111
Alexander Barg (UMD) Polar codes Dec. 12, 2012 49 / 63
RM codes
Polar codes and RM codes
Polar codes are related to classical Reed-Muller codesRM(m, r) a code of length N = 2n, k =
∑ri=0
(mi
)data symbols, distance 2m−r
1000 0000 0000 00001100 0000 0000 00001010 0000 0000 00001111 0000 0000 00001000 1000 0000 00001100 1100 0000 00001010 1010 0000 00001111 1111 0000 00001000 0000 1000 00001100 0000 1100 00001010 0000 1010 00001111 0000 1111 00001000 1000 1000 10001100 1100 1100 11001010 1010 1010 10101111 1111 1111 1111
Alexander Barg (UMD) Polar codes Dec. 12, 2012 49 / 63
RM codes
Polar codes and RM codes
Polar codes are related to classical Reed-Muller codesRM(m, r) a code of length N = 2n, k =
∑ri=0
(mi
)data symbols, distance 2m−r
RM (0,4)
1000 0000 0000 00001100 0000 0000 00001010 0000 0000 00001111 0000 0000 00001000 1000 0000 00001100 1100 0000 00001010 1010 0000 00001111 1111 0000 00001000 0000 1000 00001100 0000 1100 00001010 0000 1010 00001111 0000 1111 00001000 1000 1000 10001100 1100 1100 11001010 1010 1010 10101111 1111 1111 1111
Polar
1000 0000 0000 00001100 0000 0000 00001010 0000 0000 00001111 0000 0000 00001000 1000 0000 00001100 1100 0000 00001010 1010 0000 00001111 1111 0000 00001000 0000 1000 00001100 0000 1100 00001010 0000 1010 00001111 0000 1111 00001000 1000 1000 10001100 1100 1100 11001010 1010 1010 10101111 1111 1111 1111
Alexander Barg (UMD) Polar codes Dec. 12, 2012 50 / 63
RM codes
Polar codes and RM codes
Polar codes are related to classical Reed-Muller codesRM(m, r) a code of length N = 2n, k =
∑ri=0
(mi
)data symbols, distance 2m−r
RM (1,4)
1000 0000 0000 00001100 0000 0000 00001010 0000 0000 00001111 0000 0000 00001000 1000 0000 00001100 1100 0000 00001010 1010 0000 00001111 1111 0000 00001000 0000 1000 00001100 0000 1100 00001010 0000 1010 00001111 0000 1111 00001000 1000 1000 10001100 1100 1100 11001010 1010 1010 10101111 1111 1111 1111
Polar
1000 0000 0000 00001100 0000 0000 00001010 0000 0000 00001111 0000 0000 00001000 1000 0000 00001100 1100 0000 00001010 1010 0000 00001111 1111 0000 00001000 0000 1000 00001100 0000 1100 00001010 0000 1010 00001111 0000 1111 00001000 1000 1000 10001100 1100 1100 11001010 1010 1010 10101111 1111 1111 1111
Alexander Barg (UMD) Polar codes Dec. 12, 2012 51 / 63
RM codes
Polar codes and RM codes
Polar codes are related to classical Reed-Muller codesRM(m, r) a code of length N = 2n, k =
∑ri=0
(mi
)data symbols, distance 2m−r
RM (2,4)
1000 0000 0000 00001100 0000 0000 00001010 0000 0000 00001111 0000 0000 00001000 1000 0000 00001100 1100 0000 00001010 1010 0000 00001111 1111 0000 00001000 0000 1000 00001100 0000 1100 00001010 0000 1010 00001111 0000 1111 00001000 1000 1000 10001100 1100 1100 11001010 1010 1010 10101111 1111 1111 1111
Polar
1000 0000 0000 00001100 0000 0000 00001010 0000 0000 00001111 0000 0000 00001000 1000 0000 00001100 1100 0000 00001010 1010 0000 00001111 1111 0000 00001000 0000 1000 00001100 0000 1100 00001010 0000 1010 00001111 0000 1111 00001000 1000 1000 10001100 1100 1100 11001010 1010 1010 10101111 1111 1111 1111
Alexander Barg (UMD) Polar codes Dec. 12, 2012 52 / 63
RM codes
Decoding of polar codes and RM codes
Goal: Fill the void for moderate block length: 200 ≤ N ≤ 2000
Problem: Relatively slow convergence
Enhancements of Decoding Algorithms:List decodingGradient-like decodingEasier logic by quantizing SC decoding
A.B. and I. Dumer (UC Riverside), work in progressI. Dumer, papers on decoding RM codes, IEEE Transactions on Information Theory,2006,2008.
Alexander Barg (UMD) Polar codes Dec. 12, 2012 53 / 63
RM codes
Decoding of polar codes and RM codes
Goal: Fill the void for moderate block length: 200 ≤ N ≤ 2000
Problem: Relatively slow convergence
Enhancements of Decoding Algorithms:List decodingGradient-like decodingEasier logic by quantizing SC decoding
A.B. and I. Dumer (UC Riverside), work in progressI. Dumer, papers on decoding RM codes, IEEE Transactions on Information Theory,2006,2008.
Alexander Barg (UMD) Polar codes Dec. 12, 2012 53 / 63
RM codes
Decoding of polar codes and RM codes
Goal: Fill the void for moderate block length: 200 ≤ N ≤ 2000
Problem: Relatively slow convergence
Enhancements of Decoding Algorithms:List decodingGradient-like decodingEasier logic by quantizing SC decoding
A.B. and I. Dumer (UC Riverside), work in progressI. Dumer, papers on decoding RM codes, IEEE Transactions on Information Theory,2006,2008.
Alexander Barg (UMD) Polar codes Dec. 12, 2012 53 / 63
RM codes
List decoding of polar codes
SC decoding:
ui = (arg maxz∈{0,1}
W (yN1 , u
i−11 |z)) · I{i∈F c
N}
Keep L = 2s most probable bit sequences (i1, . . . , is), start pruning thelist after that.
Alexander Barg (UMD) Polar codes Dec. 12, 2012 54 / 63
RM codes
List decoding of polar codes
SC decoding:
ui = (arg maxz∈{0,1}
W (yN1 , u
i−11 |z)) · I{i∈F c
N}
Keep L = 2s most probable bit sequences (i1, . . . , is), start pruning thelist after that.
Alexander Barg (UMD) Polar codes Dec. 12, 2012 54 / 63
RM codes
List decoding of polar codes
0 0.5 1 1.5 2 2.5 3 3.5 4 4.510−5
10−4
10−3
10−2
10−1
100
SNR (dB)
WER
Uncoded k = 93, L = 1 k = 78, L = 1 k = 78, L = 2 k = 78, L = 4 k = 78, L = 8 k = 78, L = 16k = 78, L = 32k = 78, ML
(256,78) Polar Code ⊂ (256,93) RM(8,3) code
Alexander Barg (UMD) Polar codes Dec. 12, 2012 55 / 63
RM codes
List decoding of polar codes
0 0.5 1 1.5 2 2.5 3 3.5 4 4.510
−5
10−4
10−3
10−2
10−1
100
SNR (dB)
WE
R
Uncoded k = 130, L = 1 k = 101, L = 1 k = 101, L = 2 k = 101, L = 4 k = 101, L = 16 k = 101, L = 64 k = 101, L = 1024k = 101, ML
(512,101) Polar Code ⊂ (512,130) RM(9,3) code
Alexander Barg (UMD) Polar codes Dec. 12, 2012 56 / 63
RM codes
List decoding of polar codes
0 1 2 3 4 5 6 7
10−5
10−4
10−3
10−2
10−1
100
SNR (dB)
WER
Uncodedk = 29, L(6) = 16k = 64, L(6) = 16k = 99, L(6) = 8
RM code of length N = 128L(∆)=list size required for BER withing 0.25 dB of ML performance
Alexander Barg (UMD) Polar codes Dec. 12, 2012 57 / 63
RM codes
List decoding of polar codes
0 1 2 3 4 5 6 7
10−5
10−4
10−3
10−2
10−1
100
SNR (dB)
WER
Uncodedk = 37, l(6) = 64, L(6) = 128k = 93, l(6) = 128, L(6) = 1024k = 163, l(6) = 128, L(6) = 1024k = 219, l(6) = 16, L(6) = 32
RM codes of length N = 256L(∆)=list size required for BER withing 0.25 dB of ML performance
Alexander Barg (UMD) Polar codes Dec. 12, 2012 58 / 63
RM codes
List decoding of polar codes
List decoding with CRC achieves state-of-the-art performance
Alexander Barg (UMD) Polar codes Dec. 12, 2012 59 / 63
RM codes
Quantization
Let ρi(W ) , LiN(y
N1 , u
i−11 ) = log W (i)
N (yN1 ,u
i−11 |0)
W (i)N (yN
1 ,ui−11 |1)
Recurstion step:
ρ(W−) = logeρ′+ρ′′ + 1
eρ′ + eρ′′, ρ(W+) = ρ′ + x · ρ′′
where x is the value of the decoded symbol.
Table-based approximation with only small loss of accuracy
Alexander Barg (UMD) Polar codes Dec. 12, 2012 60 / 63
RM codes
Quantization
Alexander Barg (UMD) Polar codes Dec. 12, 2012 61 / 63
RM codes
Gradient-like decoder
Alexander Barg (UMD) Polar codes Dec. 12, 2012 62 / 63
RM codes
Gradient-like decoder
Improvement for N = 210 – N = 212
Complexity increases by a factor of N
Alexander Barg (UMD) Polar codes Dec. 12, 2012 63 / 63