1

E.1

Error-control coding

A good communication system requires transmittinginformation from one end of the system at a rate and a levelof reliability and quality that are acceptable to a user at theother end.– Two key systems parameters are transmitted signal power

and channel bandwidth. In general,• Signal power ↑ error rate ↓• Bandwidth ↑ error rate ↓

– In practice, we often arrive at a modulation scheme andfind that it is not possible to provide acceptable dataquality.

– ∴ Error-control coding

E.2

Feed-Forward Error Correction (FEC)

The channel encoder in the transmitter accepts message bitsand adds redundancy according to a prescribed rule,thereby producing encoded data at a higher bit rate.

The channel decoder in the receiver exploits the redundancyto decide which message bits were actually transmitted.

2

E.3

Automatic-Repeat Request (ARQ)

FEC is used for both the detection and correction of errors.– Only a one-way link between the transmitter and receiver

is required.

ARC uses redundancy merely for the purpose of errordetection. Upon the detection of an error in a transmittedcode word, the receiver requests a repeat transmission of thecorrupted code word, which need a return path.– Used only on half-duplex or full-duplex links– widely used in computer-communication systems

E.4

Classification of channel coding

Block Codes– has no memory– adding redundant bits that are algebraically related to the

message bits.

Convolutional Codes– has memory– discrete time convolution of the input sequence with the

impulse response of the encoder.

3

E.5

Channel Coding Theorem

Channel Capacity bit/sec

If a discrete memoryless channel has capacity C bit/s and asource generates information at a rate less than C bit/s, thenthere exists a coding technique such that the output of thesource may be transmitted over the channel with anarbitrarily low probability of symbol error.

– This theorem gives a fundamental limit on the rate atwhich the transmission of reliable messages can takeplace over a discrete memoryless channel.

)/1(log 2 NSBC +=

E.6

Binary operations

Channel encoding and decoding involves the binaryarithmetic operations of modulo-2 addition andmultiplication for binary codes {0,1}:

0 + 0 = 0 0 0 = 00 + 1 = 1 0 1 = 01 + 0 = 1 1 0 = 01 + 1 = 0 1 1 = 1

Modulo-2 addition is the Exclusive-OR (XOR) operationModulo-2 multiplication is the AND operation

⋅⋅

⋅⋅

4

E.7

Linear Block Code

Linear – the modulo-2 sum of any codewords is another valid codeword.

A code of 8 possible codewords– ‘Linear’ means code 1 + code 2 = either code 1, 2, 3, 4,

5, 6, 7 or 8.– Binary code is linear (e.g. 101+111 =010)

8

2

76

54 3

1

E.8

Block

Block– the input message is divided into blocks of k bits and

coded into n-bit codewords.

message check bits

k bits

n bits

k bits(n-k) bits

message

Message block m Channelencoder

Code Block c

5

E.9

Message block m: Code block c: =

This is known as a systematic (n, k) linear block code.– A k-bit message is coded into a n-bit codeword consisted

of (n - k) parity-check bits and k message bits.

– Code rate (or Rate efficiency) =

[ ]110 −kmmm L[ ]110 −nccc LLLLL

[ ]110110 −−− kkn mmmbbb LL

[ ]mbM=

nk /

E.10

Encoding: Coefficient matrix P and Generator matrix G

Now, the n-bit code word is

Encoding

G Codeword cMessage m

11

0k-n

11-k-n

11

00

:

:

−−

−−

=

==

==

kn

kn

mc

mcbc

bcbc

6

E.11

The (n - k) parity-check bits are linear sum of the k messagebits, i.e.,

where

The coefficient are chosen in such a way that the rowsof the generator matrix G are linearly independent.

1,11100 ... −−+++= kikiii mpmpmpb

=otherwise0

on depends if1 jiij

mbp

ijp

E.12

In a matrix form,

where

and

mPb =

=

−−−−−

−−

−−

1,11,10,1

1,11110

1,00100

...

...

...

knkkk

kn

kn

ppp

pppppp

MMMMP

[ ][ ][ ]

mGIPmmmP

mbc

k

====

M

M

M (P is called coefficient matrix)

7

E.13

c = mG

(1 x n) (1 x k) (k x n)

where

G = k

n

⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅

−−−−−

−−

−−

1000

00100001

1,11,10,1

1,11110

1,00100

LML

M

M

M

LML

LML

knkkk

kn

kn

ppp

pppppp

E.14

Example 1

Consider a (6, 3) linear block code

=

m - 8 possible 3-bit messages

For m = [1 1 1]

c= = [0 0 0 1 1 1]

[ ]3IPG M=

100011010101001110

M

M

M

mGc =

[ ] 1 1 1

1 0 0 0 1 1 0 1 0 1 0 1 0 0 1 1 1 0

8

E.15

Example 1

Similarly, m c

0 0 0 0 0 0 0 0 00 0 1 1 1 0 0 0 10 1 0 1 0 1 0 1 00 1 1 0 1 1 0 1 11 0 0 0 1 1 1 0 01 0 1 1 0 1 1 0 11 1 0 1 1 0 1 1 01 1 1 0 0 0 1 1 1

Encoding: needs to store G or P

E.16

For a k-bit message block, there are possible messages:

k2

12

1

0

12

1

0

−−

=

kk c

cc

mGc

m

mm

MM

k bits and 2k possible combinations

n bits and 2k possible combinations(there are 2n-k - unused combinations)

9

E.17

Sum of codewords

To prove the sum of 2 codewords is another codeword, takecodeword ,

Since the mod-2 sum of and must result in anothermessage vector within the set of 2k message, say . So,

sum of two codewords give another codeword

ji cc +

Gmm

GmGmcc

ji

jiji

)( +=

+=+

im jmkm

kkji cGmGmm ==+ )(

kji ccc =+∴

E.18

Decoding: Parity-check matrix H

Let H denote an (n-k) x n Parity-check matrix, defined as is a (n - k) x k matrix

=

][ Tkn PIH M−= TP

⋅⋅⋅⋅⋅⋅⋅⋅⋅

⋅⋅⋅⋅⋅⋅⋅⋅⋅

−−−−−−−

−

−

−

1,11,11,0

2,11202

1,11101

0,11000

1000

010000100001

knkknkn

k

k

k

PPP

PPPPPPPPP

L

L

L

L

ML

M

M

M

ML

ML

ML

10

E.19

Property of H

Consider =

=

Thus,

Therefore, for any codeword c

and this equation is called parity-check equation

THG

−

k

TT

kn IPPI ][ M

[ ]0PP TT =+

0=THG )0( =⇒ TGH

cHT = mGHT= 0

E.20

Example

e.g., =

= =

TGH

221202

211101

201000

100 010 001

PPPPPPPPP

M

M

M

100010001221202

211101

201000

PPPPPPPPP

+++++++++

222212120202

212111110101

202010100000

PPPPPPPPPPPPPPPPPP

[ ]0

11

E.21

Summary

i. c = mG Encodingii. GHT = HGT = 0iii. cHT = mGHT= 0 Parity-check equation

Decoding

HT Null vector 0 Received code c

Encoding

G Codeword cMessage m

E.22

Syndrome s

Let r be the received codeword that results from sending thecodeword c overy a noisy channel.

Decoding

HT Syndrome s Received code r

Encoding

G Codeword cMessage m

12

E.23

Let be the received code word where e =

Define the Syndrome s as

No error: and r is a valid codeword∴ c = r

Error occurred:

ecr +=[ ]1210 −neeee L

=otherwise0

location at the occurrederror an if1 th

ii

e

TrHs =

0

)(

==

+=

+=

TT

TT

T

cHeHeHcHHec

Q

[ ] [ ]00000 =⇒= se L

[ ] [ ]00000 ≠⇒≠ se L

E.24

Example 2

Consider a (7, 4) Linear Block Code

For m = [1 0 1 1], c = mG = [0 0 1 1 0 1 1 ]No error, r = c, and s = cHT= [0 0 0]

=

=

110110010110100111001

1000110010010100100110001111

M

M

M

M

M

M

M

H

G

13

E.25

Let c suffered an error in the transmission such that thereceived vector r = [0 0 0 1 0 1 1] = [0 0 1 1 0 1 1] + [0 0 1 0 0 0 0] = c + e

s = cHT = [0 0 0 1 0 1 1] = [0 0 1] (= eHT )

Note: Error in the 3rd bit results in s corresponding to the 3rd

row of HT.

110101011111100010001

E.26

For this (7, 4) code, a single error in the ith bit would give sequal to the ith row of HT. The code is capable of correctinga single error.

Double errors: e = [0 1 1 0 0 0 0]

s = c = e = [0 1 1] ≠ [0] = 7th row of HTand is interpreted as an error in the 7th

bit of the received word.

– Error is detectable but not correctable.– More than 2 errors might give s = [0] and in that case the

errors are not detectable!

14

E.27

Syndrome decoding for multiple errors: Problem

In syndrome decoding, the receiver forms s = rHT.s = rHT = 0 implies transmitted codeword c = rs = rHT ≠ 0 Find the corresponding error pattern(s) that

gives rise to s and c = r + e

The process is a bit more complicated as was illustrated for adouble error in the (7, 4) code.

E.28

Consider the (7, 4) code with

G =

For r= [1 0 0 0 0 1 1] s = rHT = [1 0 0]

But s = rHT = eHT, let e =

1 0 0 0 1 1 0 0 1 0 0 1 0 1 0 0 1 0 0 1 1 0 0 0 1 1 1 1

M

M

M

M

[ ] 7654321 eeeeeee

15

E.29

s = =

3 equations and 7 unknowns. ∴ Unique solution for e isnot possible.

For s = [1 0 0], the possible error patterns are

[ ] 0 0 1 [ ]7654321 eeeeeee

1 0 00 1 00 0 11 1 01 0 10 1 11 1 1

7431

6421

5321

0 0 1

eeeeeeeeeeee

+++=+++=+++=⇒

MM

1110110000011100010010010000

2

7654321

4

eeeeeee

E.30

Error detection and error correction capabilities

Hamming weight of a code word c, w(c), is the number ofnonzero elements in the code vector. For example,

c = [11001010111] ⇒ w(c) = 7

Hamming distance, d(c1,c2), between a pair of code vectorsand that have the same number of elements is defined as thenumber of location in which their respective elements differ.Say,

c1=[1011001] , c2 =[0111101] ⇒ d(c1, c2) = 3

16

E.31

Minimum distance

Minimum distance, dmin, of a block code is the smallestdistance between any pair of code words in the code. Forexample,Codewords weight w(c)

000000 0 d(c1,c2)= 3 110001 3 d(c1,c2) = 3 101010 3 d(c1,c2) = 4 011011 4 : 011100 3 : 101101 4 Look for : 110110 4 the smallest distance d(c1,c2) 000111 3

E.32

Hamming weight and Minimum distance

TheoremMinimum distance of a linear block code is the smallestHamming weight of the nonzero code vectors in the code.

The smallest Hamming weight from the previous exampleexcept c= [0 0 0 0 0 0 0] is 3, therefore, dmin= 3

17

E.33

Proof– .The minimum distance is the same as the smallest

Hamming weight of the difference between any pair ofcode vector (i.e. )

– From the closure property of linear block code, the sum(or difference) of two code vectors is another codevector.(i.e. )

Note: The best strategy for the decoder is to pick the codevector closest to the received vector, that is, the one forwhich the Hamming distance d(ci,r) is the smallest.

{ } { })(min),(minmin iji wdd ccc ==

{ } { })(min),(min jiji wd cccc −=

{ } { })(min)(min iji ww ccc =−

E.34

TheoremA linear block code with a minimum distance dmin cancorrect up to [(dmin - 1) / 2] errors and detect up to (dmin–1)errors in each code word where [*] denotes the largestinteger no greater than *, i.e., [3.5] = 3. Thus, in theprevious example,

dmin = 3 ∴ [(3 - 1) / 2] = 1 (3 - 1) = 2

Therefore, this (7, 4) block code can correct 1 error anddetect up to 2 errors.

18

E.35

Interpretation:

Case I

error can be corrected

(t is the distance between received codeword r and thetransmitted codeword c)

12),( +≥ td ji cc

icjc

t r

E.36

Case II

error cannot be corrected but can be detected

1),( +≥ td ji cc

icjc

rt

t+1

19

E.37

Case III

error cannot be detected

1),( +< td ji cc

ic jc

r

E.38

Syndrome decoding for multiple errors: Solution

Consider a (n, k) codeThere are 2k possible codewords: c1, c2, c3,…, c2k

Received vector r has 2n possible values. Decoder has torelate r with one of the 2k valid code words.

Divide the 2n possible received codewords into 2k disjointsubsets in such a way that each subset contains only 1 codevector ci. Then the received codeword r is decoded into ci ifit is in the ith subset.

20

E.39

Construction of subsets: standard array

The subsets constitute is constructed as follows:1. The 2k code words are placed in a row with the all-zero codeword c1 as the left most element.2. An error pattern e2 is picked and placed under c1, and a

second row is formed by adding e2 to each of the remaining code vectors in the first row; it is importantthat the error pattern chosen as the first element in a rowhas not previously appeared in the standard array.

3. Step 2 is repeated until all the possible error patternshave been accounted for.

The array is called standard array.

E.40

The rows of the standard array are called co-sets and the firstelement in each row is called co-set leader. The co-setleaders are chosen to be the error patterns that are mostlikely to occur. In the case of a binary symmetric channel,the smallest the Hamming weight of an error pattern themore likely it is to occur. Accordingly, the standard arrayshould be constructed with each coset leader having theminimum Hamming weight in its coset.

1c 2c 3c … k2c

2e 2ec +2 2ec +3 … 2ec +k2

3e 32 ec + 33 ec + … 32ec +k

M M M … Mkn−2

e kn−+2

ec2 kn−+2

ec3 … knk −+2

ec2

21

E.41

Thus, the procedure for decoding a linear block code is

1. Compute s = rHT.

2. Within the coset characterized by the syndrome s,identify the coset leader (i.e., the error pattern with thelargest probability of occurrence); call it eo.

3. Compute c=r+ eo .

E.42

Example 3

Consider the (6, 3) linear code generated by the followingmatrix: .The standard array of this code is

coset leader

=

100011010101001110

G

000000 011100 101010 110001 110110 101101 011011 000111111100 001010 010001 010110 001101 111011 100111001100 111010 100001 100110 111101 001011 010111010100 100010 111001 111110 100101 010011 001111011000 101110 110101 110010 101001 011111 000011011110 101000 110011 110100 101111 011001 000101011101 101011 110000 110111 101100 011010 000110111000 001110 010101 010010 001001 111111 100011

100000010000001000000100

000010000001100100

22

E.43

Example 4

Construction of a single-error correction codeneed dmin = 3 (or more)

when single error occurs, the ith bit corresponds to the ith

row of HT

Therefore,i. Choose the rows of HT to be all distinct.

ii. Make sure that the first (n-k) rows of HT is In-k . iii. Do not use [000…0] for any row of HT.

E.44

Since there are (n-k) columns, we can have a possible of 2n-k

distinct rows. So, use the first rows for , then fill in the restof the matrix arbitrarily but make sure that all the rows aredifferent. Finally, make sure that none of the rows is [0 0 0 .. . 0].

TH =

−

PI kn =

n

k

kn

PPPP

PPPPPPPP

kn

knkkkk

kn

kn

−

−

−−−−−−

−−

−−

444444 3444444 21L

MM

L

L

L

MMM

L

L

1,1121,10,1

1,1121110

1,0020100

1000

00100001

23

E.45

Question: Given k, how can n be determined?

Now, n rows of HT must be distinct.

∴Given k, the minimum value of n is .

nkn ≥−⇒ − 12( ) ( )1log2 +≥− nkn

( )1log2 ++≥ nkn

( )1log2 ++ nk

E.46

Design a (n, 8) code with single error correction capability,

Minimum value of n that satisfies the equation is 12. Hence a(12,8) code.

)1(log + 8 2

+≥ nn

24

E.47

=

=∴−

P

IH

knT L

LLLL

11100111101001011001110001100011

1000010000100001

E.48

Check: For the 8-bit data m = [1 1 1 0 1 0 1 1] , the 12-bit codeword c is

[ ]

[ ]1101011101011 0 0 0 0 0 0 0 1 1 1 00 1 0 0 0 0 0 0 0 1 1 10 0 1 0 0 0 0 0 1 0 1 00 0 0 1 0 0 0 0 0 1 0 10 0 0 0 1 0 0 0 1 0 0 10 0 0 0 0 1 0 0 1 1 0 00 0 0 0 0 0 1 0 0 1 1 00 0 0 0 0 0 0 1 0 0 1 1

11010111

M

M

M

M

M

M

M

M

M

=

== mGc

25

E.49

Assume that the received bit is in error, i.e.,r = [ 1 0 1 0 0 1 1 0 1 0 1 1 ]

]1100[

01111110010110101001001101101100

0001001001001000

]111010011010[ =

==

LL

TrHs

E.50

s = [ 1 1 0 0 ] corresponds to the 5th row of .HT

∴the bit of r is in error and

c = r+e = [101001101011]+[000010000000] = [101011101011] and the message m is

[11101011]