ChannelCoding

Hafiz MalikDept. of Electrical & Computer Engineering

The University of Michigan-Dearborn hafiz@umich.edu

http://www-perosnal-engin.umd.umich.edu/~hafiz

Channel Coding • class of signal transformations designed to

improve communication performance by enabling the transmitted signals to better withstand channel distortions such as noise, interference, and fading.

• Channel coding can be divided into two major classes:

1. Waveform coding by signal design2. Structured sequences by adding redundancy

Waveform Coding– deals with transforming waveform into “better

waveform” robust to channel distortion hence improving detector performance.

• Examples:– Antipodal signaling– Orthogonal signaling– Bi-orthogonal signaling– M-ary signaling– Trellis-coded modulation

Structured Sequences– deals with transforming sequences into “better

sequences” by adding structured redundancy (or redundant bits). The redundant bits are used to detect and correct errors hence improves overall performance of the communication system.

• Examples:– Linear codes

• Hamming codes• BCH codes• Cyclic codes• Reed-Solomon codes

– Non-Linear codes• Convolution codes• Turbo codes

Waveform Coding

M-ary Communications• Send multiple, M, waveforms• Choose between one of M symbols instead of 1 or 0.• Waveforms differ by phase, amplitude, and/or

frequency• Advantage: Send more information at a time• Disadvantage: Harder to tell the signals apart or

more bandwidth needed

4-ary Amplitude• Each symbol sends 2 bits• Deciding which level is correct

gets harder due to fading and noise

• RCV needs better SNR to achieve accuracy

11

01

10

00

Orthogonal Signals

• Definition

• This means that the signals are perpendicular to each other in M-dim space

• For a correlative receiver, this means that each incoming signal can be compared with a model of the signal and the best match is the symbol that was sent

0 0

bT

i j

C i jp t p t dt

i j

Multi-Amplitude Shift Keying (MASK)

• Send multiple amplitudes to denote different signals

• Typical signal configuration:– +/- p(t)– +/- 3 p(t)

– +/- (M-1) p(t)

Multi-Phase• Binary Phase Shift Keying (BPSK)1: (t)= p(t) cos(ct)

0: (t)= p(t)cos(ct

• M-ary PSK

Re

Im

x x

2cosk cp t p t t k

M

Re

Im

x x

x x

x x

x

x

Quadrature Amplitude Modulation (QAM)

• Amplitude-phase shift keying (APK)

cos sin

cos

k k c k c

k c k

p t p t a t b t

p t r t

2 2 tan kk k k k

k

br a b

a

ri

i

Multitone Signaling (MFSK)

• M symbols transmitted by M orthogonal pulses of frequencies:

• Receiver is a bank of mixers, one at each frequency

• Higher M means wider bandwidth needed or tones are closer together

2 /k MN k T

M-ary Comments• As M increases, it is harder to make good

decisions, more power is used• But, more information is packed into a symbol

so data rates can be increased• Generally, higher data rates require more

power (shorter distances, better SNR) to get good results

How do we compare performance?• Symbols have different meanings, so what

does the probability of error, PE mean?

• If a detection error is made, then more than one bit is wrong

• DCS can be faster at the price of being less sensitive

15

Detect Error On Credit Card

16

Formula for detecting error

Let d2, d4, d6, d8, d10, d12, d14, d16 be all the even values in the credit card number.

Let d1, d3, d5, d7, d9, d11, d13, d15 be all the odd values in the credit card number.

Let n be the number of all the odd digits which have a value that exceeds four

Credit card has an error if the following is true:

(d1 + d3 + d5 + d7 + d9 + d11 + d13 + d15) x 2 + n +

(d2 + d4 + d6 + d8 + d10 + d12 + d14 + d16)

0 mod(10)

17

Detect Error On Credit Card

d1

d2 d3 … d15 d16

n = 3

18

Now the test

(4 + 4 + 8 + 1 + 3 + 5 + 7 + 9) = 41

(5 + 2 + 1 + 0 + 3 + 4 + 6 + 8) x 2 + 3 = 61

41 + 61 = 102 mod (10) = 2

3

19

Credit Card Summary

The test performed on the credit card number is called a parity check equation. The last digit is a function of the other digits in the credit card. This is how credit card numbers are generated by Visa and Mastercard. They start with an account number that is 15 digits long and use the parity check equation to find the value of the 16th digit.

“This method allows computers to detect 100% of single-position errors and about 98% of other common errors” (For All Practical Purposes p. 354).

Examples

• ISBN (international standard book number)• 0 – 20 – 1 – 36186 – 8

• UPC (universal product codes)– 12-digit sequence– 0 16000 66610 8

21

What is a code?

A code is defined as an n-tuple of q elements. Where q is any alphabet.

Ex. 1001 n=4, q={1,0}

Ex. 2389047298738904 n=16, q={0,1,2,3,4,5,6,7,8,9}

Ex. (a,b,c,d,e) n=5, q={a,b,c,d,e,…,y,z}

The most common code is when q={1,0}. This is known as a binary code.

22

The purposeA message can become distorted through a wide range of unpredictable errors.

• Humans• Equipment failure• Lighting interference• Scratches in a magnetic tape

23

Why error-correcting code?

To add redundancy to a message so the original message can be recovered if it has been garbled.

e.g. message = 10 code = 1010101010

24

Send a message

Message Encoder Channel Decoder Message

10 101010 noise 001010 10

25

Encoding

Naïve approach

Hamming codes

26

Take Naïve approach

Append the same message multiple times. Then take the value with the highest average.

Message:= 1001

Encode:= 1001100110011001

Channel:= 1001100100011001

Decode: = a1 = Average(1,1,0,1) = 1

a2 = Average(0,0,0,0) = 0 ... (a1,a2,a3,a4)

Message:= 1001

Naïve Solution: Example

Single CheckSum -• Truth table:

• General form:Data=[1 1 1 1]Message=[1 1 1 1 0]

• Repeats –Data = [1 1 1 1]Message= [1 1 1 1] [1 1 1 1] [1 1 1 1]

A B X-OR0 0 00 1 11 0 11 1 0

Why Naïve Solution is Inefficient?

ratio noise tosignal theis S/N

Capacity Channel raw isW

Capacity Channel is C

/1log2 NSWC

Shannon EfficiencyRepeat 3 times:•This divide W by 3•It divides overall capacity by at least a factor of 3x.

Single Checksum:•Allows an error to be detected but requires the message to be discarded and resent. •Each error reduces the channel capacity by at least a factor of 2 because of the thrown away message.

29

Hamming [7,4] Code

The seven is the number of digits that make the code.

E.g. 0100101

The four is the number of information digits in the code.

E.g. 0100101

30

Hamming [7,4] Encoding

Encoded with a generator matrix. All codes can be formed from row operations on matrix. The code generator matrix for this presentation is the following:

kknkk PIG

:

1111000

0110100

1010010

1100001

31

Hamming [7,4] Codes

1624 1000011010010100101100001111110011010101011001100011001101010100011001110100110010101111111011110000110010000000

12827 Codes

Possible codes

Multi-dimensional Codes

Code Space:

• 2-dimensional

• 5 element states

Circle packing makes more efficient use of the code-space

Cannon Balls

• http://wikisource.org/wiki/Cannonball_stacking• http://mathworld.wolfram.com/SpherePacking.html

Efficient Circle packing is the same as efficient 2-d code spacing

Efficient Sphere packing is the same as efficient 3-d code spacing

Efficient n-dimensional sphere packing is the same as n-code spacing

34

Minimum Weight Theorem

Definitions

Proof of Theorem

35

Definitions

The weight of a code is the number of nonzero components it contains.

e.g. wt(0010110) = 3

The minimum weight of Hamming codes is the weight of the smallest nonzero vector in the code.

e.g. d(G)= 3

1111000

0110100

1010010

1100001

G

36

Definitions

The distance between two codes u and v is the number of positions which differ

e.g. u=(1,0,0,0,0,1,1)

v=(0,1,0,0,1,0,1)

dist(u,v) = 4

Another definition of distance is wt(u – v) = dist(u,v).

37

Definitions

),(),(),( wvdistvudistwudist

),(),( uvdistvudist

0),( uudist

For any u, v, and w in a space V, the following three conditions hold:

38

Definitions

The sphere of radius r about a vector u is defined as:

}),(|{)( rvudistVvuSr

e.g. u=(1,0,0,0,0,1,1) (0,0,0,0,0,1,1)

(1,1,0,0,0,1,1)

(1,0,0,0,0,0,1)

(1,0,0,0,0,0,1)

(1,0,1,0,0,1,1)

(1,0,0,1,0,1,1) (1,0,0,0,1,1,1)

39

Minimum Weight Theorem

If d is the minimum weight of a code C, then C can correct t = [(d – 1)/2] or fewer errors, and conversely.

40

Proof

Want to prove that spheres of radius t = [(d – 1)/2] about codes are disjoint. Suppose for contradiction that they are not. Let u and w be distinct vectors in C, and assume that

)()( wSuSv tt

u wv

41

Proof

By triangle inequality

twvdistvudistwudist 2),(),(),(

u wv

Since spheres of radius t = [(d – 1)/2] so and this gives

But since

We have a contradiction. Showing the sphere of radius t about codes are disjoint.

42

Proof

12 dt12 dtdist(u,w)dist(u,v)dist(u,w)

dwuwtwudist )(),(

43

Result of Theorem

Since d(G) = 3 then for t = [(3 – 1)/2] = 1 or fewer errors, the received code is in a disjoint sphere about a unique code word.

1111000

0110100

1010010

1100001

G

Parity-Check Matrix• The parity check matrix is found by solving the generator matrix for

• For each generator matrix G, there exists an (n – k) x n matrix H, such that the rows of G are orthogonal to the rows of G; i.e.,

• where HT is the transpose of H, and 0 is an k x (n – k) all zeros matrix . • The matrix H is called the parity-check matrix, that can be used to decode

the received code words.

0TGH

3343

33

:

:

1001011

0101101

0011110

xx

T

IP

IP

H

0TGH

45

Channel Decoding

1. Brut force Approach: list all possible messages

2. Syndrome Decoding

46

Brute Force Approach:List all messages

This is done by generating a list of all the possible messages. For something small like the Hamming [7,4] codes the task is feasible, but for codes of greater length it is not. An example of a list is as follows:

Code words 1000011 0100101 0010110 … 0000011 0000101 0000110

Other 1000001 0100111 0010100Received 0010011 0001101 1010110Words 1100011 1100101 0110110

… … …

47

Brute Force Approach:List all messages

For example, if the received code was 0001101 then it would be decoded to 0100101 from the list.

Code words 1000011 0100101 0010110 … 0000011 0000101 0000110

Other 1000001 0100111 0010100Received 0010011 0001101 1010110Words 1100011 1100101 0110110

… … …

48

Syndrome DecodingConsider a transmitted code cm and y is the received sequence, y can be expressed as,

where e denotes binary error vector.

The decoder calculate product

(n – k)–dimensional vector s is called the syndrome of the error pattern. Or in other words, s contains the pattern of failure in parity checks.

ecy m

tyH

t

ttm

tm

t

eH

eHHc

Hec

yHs

Standard Array• Let us arrange the 2n n-tuples that represent all possible received

sequences in an array, such that first row contains all the codewords (2k) and first column contains all correctable error patterns. The standard array format of (k,n) code is as follows:

knkknknkn

knkknknkn

k

k

k

ececece

ececece

ececece

ececece

cccc

i

i

i

i

i

222212

1221212112

222212

121111

210

Standard Array• Note that codeword c0 is trivial codeword, that is, all-

zeros codeword. In addition, it is also error pattern of all-zeros.

• Each row in the standard array is called a coset.• An error pattern in the first column, called coset leader. • A coset leader is the vector with the minimum weight in

the coset.• Each coset consists of 2k n-tuples

there are 2n –k cosets.

51

Syndrome Decoding

The first step is to create a list of syndromes corresponding the coset leaders. The syndrome of each vector y is found by

THyysyn )(

When a code is received, the syndrome is computed and compared to the list of syndromes. Let the coset leader to the syndrome by e. Finally the code is decoded to x = y – e.

52

Syndrome example

1111000

0110100

1010010

1100001

G

1001011

0101101

0011110

H

Note that G=(I |P) and H = ( | I). TP

53

Syndrome example

Let x:= 1001100 be the original message

Message Encoder Channel Decoder Message

1001 1001100 noise 1000100 ?

Compute the syndrome of the received codeTHyysyn )(

1001011

0101101

0011110

H

1

0

1

0

0

0

1

1

1

1

54

Conclusion

A code of minimum weight d is called perfect if all the vectors in V are contained in the sphere of radius t = [(d – 1)/2] about the code-word.

The Hamming [7,4] code has eight vectors of sphere of radius one about each code-word, times sixteen unique codes. Therefore, the Hamming [7,4] code with minimum weight 3 is perfect since all the vectors (128) are contained in the sphere of radius 1.