exercise in the previous class (1)

exercise in the previous class (1)

Define (one of) (15, 11) Hamming code:construct a parity check matrix, anddetermine the corresponding generator matrix

1

100011010011101010010101011011001001100110111000100011101111

11001010011010010101001111101101101101111111

I

generator matrix

parity check matrix

m = 4

24 – 1 nonzero vectors

exercise in the previous class (2)

Let C be a linear code with the following parity check matrix. Show that the minimum distance of C is 4.

2

.

10001110010011010010101100010111

Hweight 1: HuT = hi = 0...contradiction

weight 2: HuT = hi + hj= 0 ...contradiction

weight 3: HuT = hi + hi + hl = 0, but...column vectors have odd weight, and hi + hi has even weighthi + hi = hl never happens...contradiction

weight 4: we have 10001110C

previous class

Hamming codeone-bit error correcting, one of perfect codes

minimum Hamming distancesmallest distance between two codewordsgives the maximum number of correctable errors

easy but “rough” estimation of performance

3

today’s class

two independent topics:

performance evaluation using weight distributionslightly complicated ( ちょっと複雑 )precise and detailed estimation is possible

cyclic codea subclass of linear codesadvantages in realizations

4

is the minimum distance everything?

If the minimum Hamming distance of C is dmin, then C can correct (dmin – 1) / 2 or less bit errors.

5

C1 C2 C3

< =?Consider three codes with the same code length...

many neighborssmall # of neighbors

dmin dmin dmin

C2 seems better than C3...

1 codeword at distance 04 codewords at distance 33 codewords at distance 4

1 codeword with weight 04 codewords with weight 33 codewords with weight 4

weight distribution

6

C={000000,100111,010110,110001, 001011,101100,011101,111010}.

weight distribution

in linear codes, all codewords have the same distance distribution

3

4000000

distance distribution from 000000

from the view of 000000,

performance evaluation with weight distribution

assume...BSC with bit error probability pone-bit error correcting

send all-zero codeword and evaluate the probabilities of...correctly decodedwrongly decodederror detected

7

C={000000,100111,010110,110001, 001011,101100,011101,111010}.

r000000 decoder ?

receivedvector


the domain of the received vector

the received vector rV6

V6 contains eight decoding territories

8

C={000000,100111,010110,110001, 001011,101100,011101,111010}.

correctly decoded; if r falls in the territory of 000000

wrongly decoded;if r falls in the territory of the other

codewordserror detected; if r does not fall in the territories

(not all vectors shown)

probability of correct decoding

correctly decoded if r belongs toT = {000000, 000001, 000010, 000100, 001000, 010000, 100000}

9

r is decoded correctly with probability Pc = (1 – p)6 + 6p(1-p)5

prob. that rT and wH(r) = 0


(1 – p)6

6p( 1 – p)5

(1 – p)6

p( 1 – p)5

probability of wrong decoding (1)

case1: r belongs to the territory of a weight-3 codeword 001011T = {001011, 001010, 001001, 001111, 000011, 011011, 101011}

10

weight 3weight 2

weight 4prob. that rT and wH(r) = 3


p3(1 – p)3

3p2( 1 – p)4

prob. that rT and wH(r) = 4 3p4( 1 – p)2

r is decoded to the wrong codeword 001011 with probability 3p2(1 – p)4 + p3(1 – p)3 + 3p4(1 – p)2

... the same discussion holds for other codewords with weight 3


case2: r belongs to the territory of a weight-4 codeword 100111T = {100111, 100110, 100101, 100011, 101111, 110111, 000111}

11



p4(1 – p)2

4p3( 1 – p)3

prob. that rT and wH(r) = 5 2p5( 1 – p)

r is decoded to the wrong codeword 100111 with probability 4p3(1 – p)3 + p4(1 – p)2 + 2p5(1 – p)

... the same discussion holds for other codewords with weight 4

weight 4weight 3

weight 5


wrongly decoded;Pe = 4×(falls in the territory of one of weight-3 codewords)

+ 3×(falls in the territory of one of weight-4 codewors) = 4×(3p2(1 – p)4 + p3(1 – p)3 + 3p4(1 – p)2 )

+ 3×(4p3(1 – p)3 + p4(1 – p)2 + 2p5(1 – p)) = 12p2(1 – p)4 + 16p3(1 – p)3 + 15p4(1 – p)2 + 6p5(1 – p)

error detected with probability Pd = 1 – Pc – Pe12

C={000000,100111,010110,110001, 001011,101100,011101,111010}.


the probabilities in a graph

13

00.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

0.11

0.12

0.13

0.14

0.15

0.16

0.17

0.18

0.19

0.2

0.21

0.22

0.23

0.24

0.25

0.26

0.27

0.28

0.29

0.3

0.31

0.32

0.33

0.34

0.35

0.36

0.37

0.38

0.39

0.4

0.41

0.42

0.43

0.44

0.45

0.46

0.47

0.48

0.49

0.5

00.10.20.30.40.50.60.70.80.9

1PcPePd

bit error probability p

prob

abili

ties

comparison example

(7,4) Hamming code:0000000, 1000101, 0100111, 0010110,0010110, 1010011, 0110001, 1110100,0001011, 1001110, 0101100, 1101001,0011101, 1011000, 0111010, 1111111

(9, 4) 2D code:000000000, 000101011, 001001101, 001100110,010010011, 010111000, 011011110, 011110101,100010101, 100111110, 101011000, 101110011,110000110, 110101101, 111001011, 111100000

14

weight0347

#codewords1771

weight046

#codewords196

comparison of two codes

15

00.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

0.11

0.12

0.13

0.14

0.15

0.16

0.17

0.18

0.19

0.2

0.21

0.22

0.23

0.24

0.25

0.26

0.27

0.28

0.29

0.3

0.31

0.32

0.33

0.34

0.35

0.36

0.37

0.38

0.39

0.4

0.41

0.42

0.43

0.44

0.45

0.46

0.47

0.48

0.49

0.5

00.10.20.30.40.50.60.70.80.9

1 Hamm, PcHamm, PeHamm, Pd

Hamming code seems better, but...

“retransmission”

re-transmission of broken data is allowed in some systems

16

00.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

0.11

0.12

0.13

0.14

0.15

0.16

0.17

0.18

0.19

0.2

0.21

0.22

0.23

0.24

0.25

0.26

0.27

0.28

0.29

0.3

0.31

0.32

0.33

0.34

0.35

0.36

0.37

0.38

0.39

0.4

0.41

0.42

0.43

0.44

0.45

0.46

0.47

0.48

0.49

0.5

00.10.20.30.40.50.60.70.80.9

1 Hamm, PcHamm, PeHamm, Pd2D, Pc2D, Pe2D, Pd2D, Pc (retrans)

2D code + one-time re-transmission is better for p ≤ 0.28.

half the way...

two independent topics:



17

cyclic codes

Linear codes are characterized by matrix operations.Encoders/decoders are realized by combinatorial circuits.

simple but not scalable ... O(n2) for the code length n

cyclic codes ( 巡回符号 )a subclass of linear codesEncoders/decoders are realized by shift registers.

complex but scalable ... O(n) complexity

18

codes linear codes

cyclic codes

preliminary (1)

We represent binary vectors by binary polynomials ( 多項式 ).

19

11101 x4 + x3 + x2 + x + 1

addition of polynomials:(subtraction)

x4 + x3 + x2 + 1 x3 + x + 1+)

x4 + x2 + x

111010101110110

+)

multiplicationof polynomials:

x4 + x3 + x2 + 1 x3 + x + 1×)x4 + x3 + x2 + 1

x5 + x4 + x3 + xx7 + x6 + x5 + x3

x7 + x6 + x3 + x2+ x + 1 11001111

1110101011×)11101

1110111101

multiply by xm

= left-shift of m bits

preliminary (2)

20

division of polynomials:

x6 + x4x4 + x3 + x2 + 1 )x2 + x + 1

x6 + x5 + x4 + x2

x5 + x2

x5 + x4 + x3 + xx4 + x3 + x2 + xx4 + x3 + x2 + 1

x + 1

11101 ) 101000011101

1001011101

1111011101

11

111

addition = subtraction = XOR of coefficientsthe division circuit is easily implemented by a shift register

division circuit (1)

divide p(x) = x6 + x4 by q(x) = x4 + x3 + x2 + 1...

21

1 1 1 0 1

1 0 1 0 0 0 0

q(x)

p(x)

quotient( 商 ) remainder ( 剰余 )

1. store p(x) to the registers2. if MSB = 1, then AND gates are activated,

and registers are XOR’ed with q(x)3. left-shift, and go to step 1

division circuit (2)

22

0 1 0 0 1

1 1 1 0 1

0 0

11101 ) 101000011101010010

111011111011101

11

111

1 0 1 0 0

1 1 1 0 1

0 0

one-cycle of operations= one-step of the calculuscontents of the register= remainder of the division

definition of cyclic codes

How to construct an (n, k) cyclic code:length n, k information symbols, m = n – k parity symbols

Step 1: choose an order-m polynomial G(x) which divides xn + 1.Step 2: C = {multiples of G(x) with order < n}

Example: n = 7, m = 4:G(x) = x4 + x3 + x2 + 1 divides x7+1,because x7+1 = (x3 + x2 + 1) (x4 + x3 + x2 + 1).

G(x)...generator polynomial

23

1000000111101 )

111011101001

11101

1110111101

0

1101

G(x) x7+1

construction of a cyclic code

Step 1: choose an order-m polynomial G(x) which divides xn + 1.Step 2: C = {multiples of G(x) with order < n}

n = 7, m = 4: G(x) = x4 + x3 + x2 + 1

24

G(x)(x2+x+0)×

C=

00000000011101011101001001111110100110100110011101010011

G(x)(x2+x+1)×G(x)(x2+x+1)×G(x)(x2+x+1)×G(x)(x2+x+1)×G(x)(x2+x+1)×G(x)(x2+x+1)×G(x)(x2+x+1)×

=

properties of cyclic codes (1)

Lemma: a cyclic code is a linear code

proof: show that, for c1, c2C, we have c1+c2Cc1 C c1 = f1(x)G(x)c2 C c2 = f2(x)G(x)

c1+c2 = (f1(x)+f2(x)) G(x) C

25

properties of cyclic codes (2)

Lemma: if (an-1, an-2, ..., a0)C, then (an-2, ..., a0, an-1)C

proof:Let W(x) = an-1xn-1 + ... + a0 and W’(x) = an-2xn-1 + ... + a0x + an-1.

W(x) is a multiple of G(x)W’(x) = an-2xn-1 + ... + a0x + an-1

= an-1xn + an-2xn-1 + ... + a0x + an-1+ an-1xn

= xW(x) + an-1(xn + 1)

26

multiple of G(x) multiple of G(x)

W’(x) is a multiple of G(x), and (an-2, ..., a0, an-1)C.

A cyclic code C is closed for a cyclic shift.

three approaches for encoding

three approaches for an encoding procedure:matrix approach

use a generator matrix ... no advantage of cyclic codesmultiplication approach

codeword = (info. symbols) × G(x)the code not systematic ( p.24)

division approachslightly complicated (for human)make the code systematiceasily implemented by shift registers

27

encoding by division

1. represent information symbols by a polynomial A(x)2. divide A(x)xm by G(x), and let B(x) be the remainder3. let A(x)xm + B(x) be a codeword

encoding 011, with n = 7, k = 3, m = 4, G(x) = x4 + x3 + x2 + 1 1. A(x) = x + 12. A(x)x4 = x5 + x4 = x(x4 + x3 + x2 + 1) + (x3 + x), and B(x) = x3 + x3. A(x)x4 + B(x)= x5 + x4 + x3 + x, the codeword = 0111010

28

A(x)

D(x)G(x)

B(x)

x4

÷divisor

dividend

quotient

remainder

W(x) = A(x) x4 + B(x)

did we really make “encoding”?

simple question: Is A(x)xm + B(x) really a codeword? is A(x)xm + B(x) divided by

G(x)?

Yes, note that...B(x) is a remainder of A(x)xm

in a binary world, A(x)xp + B(x) = A(x)xp – B(x)“A(x)xp – B(x)” = “remove the remainder”

if A(x)xp – B(x) is divided by G(x), then there is no remainder...

29

example

n = 7, k = 3, m = 4, G(x) = x4 + x3 + x2 + 1

30

data000001010011100101110111

A(x)x2+x+0x2+x+1x2+x+1x2+x+1x2+x+1x2+x+1x2+x+1x2+x+1

A(x)x4

00000000010000010000001100001000000101000011000001110000

B(x)x3+x2+x+0x3+x2+x+1x3+x2+x+1x3+x2+x+1x3+x2+x+1x3+x2+x+1x3+x2+x+1x3+x2+x+1

A(x)xm + B(x) 00000000011101010011101110101001110101001111010011110100

systematic codeencoder ≈ division circuit O(n) < O(n2) of matrix operation

error “detection” of cyclic codes

error “detection” is easy for cyclic codesu C u (in a polynomial representation) is divided by G(x)

31

G(x)

= 0 ... no error 0 ... error÷divisor

dividend

quotient

remainderreceived u

one division circuit is used for encoding and error detection reduces the cost of realization

Cyclic Redundancy Check (CRC)... used in many communication systems

error “correction” of cyclic codes

general algorithm for all cyclic codeserror-trapping decoder [Kasami 61]

32

Tadao Kasami1930-2007

E. Berlekamp, 1940- J. L. Massey, 1934-

I. Reed, 1923- (left)andG. Solomon,1930-1996

special algorithms for special cyclic codesBerlekamp-Massey algorithm for...

BCH (Bose-Chaudhuri-Hocquenghem) codesReed-Solomon Codes

summary



33

exercise

Consider the following code C.determine the weight distribution of Ccompute the “three” probabilities (p. 7), and draw a graph

34

.

10001110010011010010101100010111

H

We want to construct a cyclic code with n = 7, k = 4, and m =3.confirm that G(x) = x3 + x2 + 1 can be a generator polynomialencode 0110decide if 0001100 is a correct codeword or not

Documents

exercise in the previous class (1)