exercise in the previous class (1)
Define (one of) (15, 11) Hamming code:construct a parity check matrix, anddetermine the corresponding generator matrix
1
100011010011101010010101011011001001100110111000100011101111
11001010011010010101001111101101101101111111
I
generator matrix
parity check matrix
m = 4
24 – 1 nonzero vectors
exercise in the previous class (2)
Let C be a linear code with the following parity check matrix. Show that the minimum distance of C is 4.
2
.
10001110010011010010101100010111
Hweight 1: HuT = hi = 0...contradiction
weight 2: HuT = hi + hj= 0 ...contradiction
weight 3: HuT = hi + hi + hl = 0, but...column vectors have odd weight, and hi + hi has even weighthi + hi = hl never happens...contradiction
weight 4: we have 10001110C
previous class
Hamming codeone-bit error correcting, one of perfect codes
minimum Hamming distancesmallest distance between two codewordsgives the maximum number of correctable errors
easy but “rough” estimation of performance
3
today’s class
two independent topics:
performance evaluation using weight distributionslightly complicated ( ちょっと複雑 )precise and detailed estimation is possible
cyclic codea subclass of linear codesadvantages in realizations
4
is the minimum distance everything?
If the minimum Hamming distance of C is dmin, then C can correct (dmin – 1) / 2 or less bit errors.
5
C1 C2 C3
< =?Consider three codes with the same code length...
many neighborssmall # of neighbors
dmin dmin dmin
C2 seems better than C3...
1 codeword at distance 04 codewords at distance 33 codewords at distance 4
1 codeword with weight 04 codewords with weight 33 codewords with weight 4
weight distribution
6
C={000000,100111,010110,110001, 001011,101100,011101,111010}.
weight distribution
in linear codes, all codewords have the same distance distribution
3
4000000
distance distribution from 000000
from the view of 000000,
performance evaluation with weight distribution
assume...BSC with bit error probability pone-bit error correcting
send all-zero codeword and evaluate the probabilities of...correctly decodedwrongly decodederror detected
7
C={000000,100111,010110,110001, 001011,101100,011101,111010}.
r000000 decoder ?
receivedvector
1 codeword with weight 04 codewords with weight 33 codewords with weight 4
the domain of the received vector
the received vector rV6
V6 contains eight decoding territories
8
C={000000,100111,010110,110001, 001011,101100,011101,111010}.
correctly decoded; if r falls in the territory of 000000
wrongly decoded;if r falls in the territory of the other
codewordserror detected; if r does not fall in the territories
(not all vectors shown)
probability of correct decoding
correctly decoded if r belongs toT = {000000, 000001, 000010, 000100, 001000, 010000, 100000}
9
r is decoded correctly with probability Pc = (1 – p)6 + 6p(1-p)5
prob. that rT and wH(r) = 0
prob. that rT and wH(r) = 1
(1 – p)6
6p( 1 – p)5
(1 – p)6
p( 1 – p)5
probability of wrong decoding (1)
case1: r belongs to the territory of a weight-3 codeword 001011T = {001011, 001010, 001001, 001111, 000011, 011011, 101011}
10
weight 3weight 2
weight 4prob. that rT and wH(r) = 3
prob. that rT and wH(r) = 2
p3(1 – p)3
3p2( 1 – p)4
prob. that rT and wH(r) = 4 3p4( 1 – p)2
r is decoded to the wrong codeword 001011 with probability 3p2(1 – p)4 + p3(1 – p)3 + 3p4(1 – p)2
... the same discussion holds for other codewords with weight 3
probability of wrong decoding (2)
case2: r belongs to the territory of a weight-4 codeword 100111T = {100111, 100110, 100101, 100011, 101111, 110111, 000111}
11
prob. that rT and wH(r) = 4
prob. that rT and wH(r) = 3
p4(1 – p)2
4p3( 1 – p)3
prob. that rT and wH(r) = 5 2p5( 1 – p)
r is decoded to the wrong codeword 100111 with probability 4p3(1 – p)3 + p4(1 – p)2 + 2p5(1 – p)
... the same discussion holds for other codewords with weight 4
weight 4weight 3
weight 5
probability of wrong decoding (3)
wrongly decoded;Pe = 4×(falls in the territory of one of weight-3 codewords)
+ 3×(falls in the territory of one of weight-4 codewors) = 4×(3p2(1 – p)4 + p3(1 – p)3 + 3p4(1 – p)2 )
+ 3×(4p3(1 – p)3 + p4(1 – p)2 + 2p5(1 – p)) = 12p2(1 – p)4 + 16p3(1 – p)3 + 15p4(1 – p)2 + 6p5(1 – p)
error detected with probability Pd = 1 – Pc – Pe12
C={000000,100111,010110,110001, 001011,101100,011101,111010}.
1 codeword with weight 04 codewords with weight 33 codewords with weight 4
the probabilities in a graph
13
00.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
0.11
0.12
0.13
0.14
0.15
0.16
0.17
0.18
0.19
0.2
0.21
0.22
0.23
0.24
0.25
0.26
0.27
0.28
0.29
0.3
0.31
0.32
0.33
0.34
0.35
0.36
0.37
0.38
0.39
0.4
0.41
0.42
0.43
0.44
0.45
0.46
0.47
0.48
0.49
0.5
00.10.20.30.40.50.60.70.80.9
1PcPePd
bit error probability p
prob
abili
ties
comparison example
(7,4) Hamming code:0000000, 1000101, 0100111, 0010110,0010110, 1010011, 0110001, 1110100,0001011, 1001110, 0101100, 1101001,0011101, 1011000, 0111010, 1111111
(9, 4) 2D code:000000000, 000101011, 001001101, 001100110,010010011, 010111000, 011011110, 011110101,100010101, 100111110, 101011000, 101110011,110000110, 110101101, 111001011, 111100000
14
weight0347
#codewords1771
weight046
#codewords196
comparison of two codes
15
00.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
0.11
0.12
0.13
0.14
0.15
0.16
0.17
0.18
0.19
0.2
0.21
0.22
0.23
0.24
0.25
0.26
0.27
0.28
0.29
0.3
0.31
0.32
0.33
0.34
0.35
0.36
0.37
0.38
0.39
0.4
0.41
0.42
0.43
0.44
0.45
0.46
0.47
0.48
0.49
0.5
00.10.20.30.40.50.60.70.80.9
1 Hamm, PcHamm, PeHamm, Pd
Hamming code seems better, but...
“retransmission”
re-transmission of broken data is allowed in some systems
16
00.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
0.11
0.12
0.13
0.14
0.15
0.16
0.17
0.18
0.19
0.2
0.21
0.22
0.23
0.24
0.25
0.26
0.27
0.28
0.29
0.3
0.31
0.32
0.33
0.34
0.35
0.36
0.37
0.38
0.39
0.4
0.41
0.42
0.43
0.44
0.45
0.46
0.47
0.48
0.49
0.5
00.10.20.30.40.50.60.70.80.9
1 Hamm, PcHamm, PeHamm, Pd2D, Pc2D, Pe2D, Pd2D, Pc (retrans)
2D code + one-time re-transmission is better for p ≤ 0.28.
half the way...
two independent topics:
performance evaluation using weight distributionslightly complicated ( ちょっと複雑 )precise and detailed estimation is possible
cyclic codea subclass of linear codesadvantages in realizations
17
cyclic codes
Linear codes are characterized by matrix operations.Encoders/decoders are realized by combinatorial circuits.
simple but not scalable ... O(n2) for the code length n
cyclic codes ( 巡回符号 )a subclass of linear codesEncoders/decoders are realized by shift registers.
complex but scalable ... O(n) complexity
18
codes linear codes
cyclic codes
preliminary (1)
We represent binary vectors by binary polynomials ( 多項式 ).
19
11101 x4 + x3 + x2 + x + 1
addition of polynomials:(subtraction)
x4 + x3 + x2 + 1 x3 + x + 1+)
x4 + x2 + x
111010101110110
+)
multiplicationof polynomials:
x4 + x3 + x2 + 1 x3 + x + 1×)x4 + x3 + x2 + 1
x5 + x4 + x3 + xx7 + x6 + x5 + x3
x7 + x6 + x3 + x2+ x + 1 11001111
1110101011×)11101
1110111101
multiply by xm
= left-shift of m bits
preliminary (2)
20
division of polynomials:
x6 + x4x4 + x3 + x2 + 1 )x2 + x + 1
x6 + x5 + x4 + x2
x5 + x2
x5 + x4 + x3 + xx4 + x3 + x2 + xx4 + x3 + x2 + 1
x + 1
11101 ) 101000011101
1001011101
1111011101
11
111
addition = subtraction = XOR of coefficientsthe division circuit is easily implemented by a shift register
division circuit (1)
divide p(x) = x6 + x4 by q(x) = x4 + x3 + x2 + 1...
21
1 1 1 0 1
1 0 1 0 0 0 0
q(x)
p(x)
quotient( 商 ) remainder ( 剰余 )
1. store p(x) to the registers2. if MSB = 1, then AND gates are activated,
and registers are XOR’ed with q(x)3. left-shift, and go to step 1
division circuit (2)
22
0 1 0 0 1
1 1 1 0 1
0 0
11101 ) 101000011101010010
111011111011101
11
111
1 0 1 0 0
1 1 1 0 1
0 0
one-cycle of operations= one-step of the calculuscontents of the register= remainder of the division
definition of cyclic codes
How to construct an (n, k) cyclic code:length n, k information symbols, m = n – k parity symbols
Step 1: choose an order-m polynomial G(x) which divides xn + 1.Step 2: C = {multiples of G(x) with order < n}
Example: n = 7, m = 4:G(x) = x4 + x3 + x2 + 1 divides x7+1,because x7+1 = (x3 + x2 + 1) (x4 + x3 + x2 + 1).
G(x)...generator polynomial
23
1000000111101 )
111011101001
11101
1110111101
0
1101
G(x) x7+1
construction of a cyclic code
Step 1: choose an order-m polynomial G(x) which divides xn + 1.Step 2: C = {multiples of G(x) with order < n}
n = 7, m = 4: G(x) = x4 + x3 + x2 + 1
24
G(x)(x2+x+0)×
C=
00000000011101011101001001111110100110100110011101010011
G(x)(x2+x+1)×G(x)(x2+x+1)×G(x)(x2+x+1)×G(x)(x2+x+1)×G(x)(x2+x+1)×G(x)(x2+x+1)×G(x)(x2+x+1)×
=
properties of cyclic codes (1)
Lemma: a cyclic code is a linear code
proof: show that, for c1, c2C, we have c1+c2Cc1 C c1 = f1(x)G(x)c2 C c2 = f2(x)G(x)
c1+c2 = (f1(x)+f2(x)) G(x) C
25
properties of cyclic codes (2)
Lemma: if (an-1, an-2, ..., a0)C, then (an-2, ..., a0, an-1)C
proof:Let W(x) = an-1xn-1 + ... + a0 and W’(x) = an-2xn-1 + ... + a0x + an-1.
W(x) is a multiple of G(x)W’(x) = an-2xn-1 + ... + a0x + an-1
= an-1xn + an-2xn-1 + ... + a0x + an-1+ an-1xn
= xW(x) + an-1(xn + 1)
26
multiple of G(x) multiple of G(x)
W’(x) is a multiple of G(x), and (an-2, ..., a0, an-1)C.
A cyclic code C is closed for a cyclic shift.
three approaches for encoding
three approaches for an encoding procedure:matrix approach
use a generator matrix ... no advantage of cyclic codesmultiplication approach
codeword = (info. symbols) × G(x)the code not systematic ( p.24)
division approachslightly complicated (for human)make the code systematiceasily implemented by shift registers
27
encoding by division
1. represent information symbols by a polynomial A(x)2. divide A(x)xm by G(x), and let B(x) be the remainder3. let A(x)xm + B(x) be a codeword
encoding 011, with n = 7, k = 3, m = 4, G(x) = x4 + x3 + x2 + 1 1. A(x) = x + 12. A(x)x4 = x5 + x4 = x(x4 + x3 + x2 + 1) + (x3 + x), and B(x) = x3 + x3. A(x)x4 + B(x)= x5 + x4 + x3 + x, the codeword = 0111010
28
A(x)
D(x)G(x)
B(x)
x4
÷divisor
dividend
quotient
remainder
W(x) = A(x) x4 + B(x)
did we really make “encoding”?
simple question: Is A(x)xm + B(x) really a codeword? is A(x)xm + B(x) divided by
G(x)?
Yes, note that...B(x) is a remainder of A(x)xm
in a binary world, A(x)xp + B(x) = A(x)xp – B(x)“A(x)xp – B(x)” = “remove the remainder”
if A(x)xp – B(x) is divided by G(x), then there is no remainder...
29
example
n = 7, k = 3, m = 4, G(x) = x4 + x3 + x2 + 1
30
data000001010011100101110111
A(x)x2+x+0x2+x+1x2+x+1x2+x+1x2+x+1x2+x+1x2+x+1x2+x+1
A(x)x4
00000000010000010000001100001000000101000011000001110000
B(x)x3+x2+x+0x3+x2+x+1x3+x2+x+1x3+x2+x+1x3+x2+x+1x3+x2+x+1x3+x2+x+1x3+x2+x+1
A(x)xm + B(x) 00000000011101010011101110101001110101001111010011110100
systematic codeencoder ≈ division circuit O(n) < O(n2) of matrix operation
error “detection” of cyclic codes
error “detection” is easy for cyclic codesu C u (in a polynomial representation) is divided by G(x)
31
G(x)
= 0 ... no error 0 ... error÷divisor
dividend
quotient
remainderreceived u
one division circuit is used for encoding and error detection reduces the cost of realization
Cyclic Redundancy Check (CRC)... used in many communication systems
error “correction” of cyclic codes
general algorithm for all cyclic codeserror-trapping decoder [Kasami 61]
32
Tadao Kasami1930-2007
E. Berlekamp, 1940- J. L. Massey, 1934-
I. Reed, 1923- (left)andG. Solomon,1930-1996
special algorithms for special cyclic codesBerlekamp-Massey algorithm for...
BCH (Bose-Chaudhuri-Hocquenghem) codesReed-Solomon Codes
summary
performance evaluation using weight distributionslightly complicated ( ちょっと複雑 )precise and detailed estimation is possible
cyclic codea subclass of linear codesadvantages in realizations
33
exercise
Consider the following code C.determine the weight distribution of Ccompute the “three” probabilities (p. 7), and draw a graph
34
.
10001110010011010010101100010111
H
We want to construct a cyclic code with n = 7, k = 4, and m =3.confirm that G(x) = x3 + x2 + 1 can be a generator polynomialencode 0110decide if 0001100 is a correct codeword or not