Chapter Chapter 3131

INTRODUCTION TO INTRODUCTION TO ALGEBRAIC CODING ALGEBRAIC CODING

THEORYTHEORY

Motivation:Motivation:

• Suppose you wish to send a message to Suppose you wish to send a message to either execute or not a particular either execute or not a particular command; 1 to execute, 0 to not. The command; 1 to execute, 0 to not. The message is to be transmitted thousands of message is to be transmitted thousands of miles away and is susceptible to miles away and is susceptible to interference (noise) that could alter the interference (noise) that could alter the intended message. What could be done to intended message. What could be done to improve a safe delivery? improve a safe delivery?

SimpleSimple applyapply redundancyredundancy• Start with the idea of majority rule. So if you want Start with the idea of majority rule. So if you want

to send the message of 0, turn it into 10001. to send the message of 0, turn it into 10001. Whichever number appears the most is the Whichever number appears the most is the intended one.intended one.

• This involves the idea of independent error This involves the idea of independent error (maximum-likelihood decoding)(maximum-likelihood decoding)

• So if you send a sequence of length 500 and each So if you send a sequence of length 500 and each digit has a probability of error q=.01 (p=.99). SO digit has a probability of error q=.01 (p=.99). SO the chance that the message is sent error free the chance that the message is sent error free equals (.99)^500 or approx. .0066equals (.99)^500 or approx. .0066

Three fold repetition schemeThree fold repetition scheme

• 1001 -> 1110001111111001 -> 111000111111

• 1 ->1111 ->111

• Error can only occur if 001,010,100,000Error can only occur if 001,010,100,000• Q (error)= (.01)(.01)(.99)+(.01)(.99)(.01)+(.99)(.01)(.01)+Q (error)= (.01)(.01)(.99)+(.01)(.99)(.01)+(.99)(.01)(.01)+

(.01)(.01)(.01) ≈.000298(.01)(.01)(.01) ≈.000298

• P(no error) = (.99702)^500 ≈.86P(no error) = (.99702)^500 ≈.86

GoalGoal

• To devise reliable, efficient, and To devise reliable, efficient, and reasonably easy to implementreasonably easy to implement

• Three basic features: Three basic features: 1.set of messages1.set of messages2.method of encoding2.method of encoding 3. 3. method of decodingmethod of decoding

Example 1:Example 1:

Example 1 cont.Example 1 cont.

DefinitionDefinition: Linear Code: Linear Code

• An (n , k) An (n , k) linear code linear code over a field F is a k-over a field F is a k-dimensional subspace V of the vector dimensional subspace V of the vector spacespace

F = F + F +… +FF = F + F +… +Fn n copiescopies

OverOver F. F. TheThe membersmembers ofof V V areare calledcalled thethe codecode wordswords. . WhenWhen F is Z F is Z22 thethe codecode is is calledcalled binary. binary.

• The message consist of k digits and the The message consist of k digits and the redundancy of the remaining n - kredundancy of the remaining n - k

Example 4:Example 4:

• The set {0000,0121, 0212, 1022, 1110, The set {0000,0121, 0212, 1022, 1110, 1201, 2011, 2102, 2220} is a (4,2) linear 1201, 2011, 2102, 2220} is a (4,2) linear code over code over ZZ33; called a ; called a ternaryternary codecode. .

Definitions:Definitions:

The The Hamming distanceHamming distance between two vectors of a vector between two vectors of a vector space is the number of components in which they differ.space is the number of components in which they differ.

Denoted Denoted d(u,v).d(u,v).

The The Hamming weightHamming weight of a vector is the number of nonzero of a vector is the number of nonzero components of the vector. Denoted components of the vector. Denoted wt(u).wt(u).

The The Hamming weightHamming weight of a linear code is the minimum of a linear code is the minimum weight of any nonzero vector in the code.weight of any nonzero vector in the code.

ExampleExample

ss = 0010111, = 0010111, t t = 0101011, = 0101011, vv = 1101101 = 1101101

d(s,t)d(s,t) = 4, = 4, d(s,v)d(s,v) = 5, = 5, wt(s)wt(s) = 4, = 4, wt(v)wt(v) = 5 = 5

TheoremTheoremFor any vectors For any vectors uu,,vv, and , and ww, , d(u,v)d(u,v) ≤ ≤ d(u,w)d(u,w) + + d(w,v)d(w,v) and andd(u,v)d(u,v) = = wt(u – v).wt(u – v).

This theorem and the previous definitions give theThis theorem and the previous definitions give thefollowing.following.

TheoremTheoremIf the Hamming weight of a linear code is at least If the Hamming weight of a linear code is at least 2t + 12t + 1,,then the code can correct any to or fewer errors.then the code can correct any to or fewer errors.Alternatively, the same code can detect any Alternatively, the same code can detect any 2t2t or fewer or fewererrors.errors.

Note: A linear code with Hamming weight Note: A linear code with Hamming weight 2t + 12t + 1 can can detect detect tt

errors or errors or 2t2t errors, not both. errors, not both.

ExampleExample

Hamming (7,4) code revisited (Table on p523)Hamming (7,4) code revisited (Table on p523)

Hamming weight is 3 = 2 Hamming weight is 3 = 2 * 1 +1.* 1 +1.

This code can detect 1 error This code can detect 1 error oror 1 or 2 errors. 1 or 2 errors.

CHOSING THE G MATRIXCHOSING THE G MATRIX

Parity-check matrix decodingParity-check matrix decoding

• • For any received word w, compute wH.For any received word w, compute wH.

• If wH is the zero vector, assume no error was made.If wH is the zero vector, assume no error was made.

• If there exist one instance of a nonzero element sIf there exist one instance of a nonzero element s€€F and a F and a row i of H s.t. wH is s times row i, assume the sent word row i of H s.t. wH is s times row i, assume the sent word was w – (0…s…0) where s occurs in the ith component. If was w – (0…s…0) where s occurs in the ith component. If there is more than one such instance, do not decode.there is more than one such instance, do not decode.

• If wH does not fit into either category 2 or 3, at least two If wH does not fit into either category 2 or 3, at least two errors occurred; don’t decode.errors occurred; don’t decode.

ExampleExample

Hamming (7,4) codeHamming (7,4) codeThe generator matrix The generator matrix GG is is

11 00 00 00 11 11 00

00 11 00 00 11 00 11

00 00 11 00 11 11 11

00 00 00 11 00 11 11

The parity-check matrix H is11 11 00

11 00 11

11 11 11

00 11 11

11 00 00

00

0011

0000

11

Example (cont)Example (cont)Consider received word Consider received word vv = 0000110, = 0000110, vHvH = 110 = 110

110 is 1110 is 1stst row of row of HH and no other, so an error was made in and no other, so an error was made in the 1the 1stst position of position of vv..

So the received word is really 1000110So the received word is really 1000110

Lemma: Orthogonality RelationLemma: Orthogonality Relation

• Let C be an (n, k) linear code over F with Let C be an (n, k) linear code over F with generator matrix G and parity-check generator matrix G and parity-check matrix H. Then for any vector in F^n, we matrix H. Then for any vector in F^n, we have vH = 0 (the zero vector) iff v belongs have vH = 0 (the zero vector) iff v belongs to Cto C

TheoremTheorem 31.331.3 Parity-Check Matrix Parity-Check Matrix DecodingDecoding

Parity-check matrix decoding will correct any Parity-check matrix decoding will correct any single errorsingle error

if and only ifif and only if

the rows of the parity-check matrix are the rows of the parity-check matrix are nonzero and no one row is a scalar nonzero and no one row is a scalar multiple of any other.multiple of any other.

Coset DecodingCoset DecodingDo this by constructing a table called a standard array.Do this by constructing a table called a standard array.

• • 11stst row is the set row is the set CC of received code words beginning with of received code words beginning with the identity 0 . . . 0the identity 0 . . . 0

• • choose another element choose another element vv in in VV and consider the coset and consider the coset v + Cv + C

(where (where vv is not an element already listed in the table) is not an element already listed in the table)

• • among the elements of among the elements of v + Cv + C, choose one of minimum , choose one of minimum weight,weight,

call it call it v‘.v‘.

• • create the next row by placing vector create the next row by placing vector v‘ + cv‘ + c under the code under the code

word word cc

• • continue until all vectors in continue until all vectors in VV have been listed have been listed

(Note: an (Note: an (n, k)(n, k) linear code over a field with linear code over a field with qq elements will elements will have have

|v * c| = q^( n- k)|v * c| = q^( n- k) rows rows

• • 11stst column is called the coset leaders column is called the coset leaders

ExampleExampleConsider (6,3) binary linear codeConsider (6,3) binary linear codeC = {000000, 100110, 010101, 001011, 110011, C = {000000, 100110, 010101, 001011, 110011, 101101, 101101, 011110, 111000} 011110, 111000}

Theorem 31.4 Coset Decoding Is Theorem 31.4 Coset Decoding Is Nearest-Neighbor DecodingNearest-Neighbor Decoding

• In coset decoding, a received word w is In coset decoding, a received word w is decoded as a code word c such that d( w, decoded as a code word c such that d( w, c) is a minimumc) is a minimum

DefinitionDefinitionIf an If an (n, k)(n, k) linear code over F has a parity-check matrix linear code over F has a parity-check matrix HH, then, for any vector , then, for any vector uu in in F^nF^n, the vector , the vector uHuH is called is called the the syndromesyndrome of of uu..

TheoremTheoremLet Let CC be an be an (n, k,)(n, k,) linear code over linear code over FF with a parity-check matrix with a parity-check matrix

HH. Then, two vectors of . Then, two vectors of F^ nF^ n are in the same coset of are in the same coset of CC iff iff they they

have the same syndrome.have the same syndrome.

We can use syndromes for decoding any received word We can use syndromes for decoding any received word ww::

• • Calculate Calculate wHwH (syndrome) (syndrome)

• • Find coset leaderFind coset leader v v such that such that wHwH = = vHvH

• • Assume vector sent was Assume vector sent was w - vw - v

Example 11Example 11

• H = H =

• Coset leaders 000000 100000 010000 001000 000100 000010 000001 Coset leaders 000000 100000 010000 001000 000100 000010 000001 100001100001

• Syndromes 000 110 101 011 100 010 001 111 Syndromes 000 110 101 011 100 010 001 111 • For v = 101001, vH = 100. Since the coset leader 000100 has 100 as its For v = 101001, vH = 100. Since the coset leader 000100 has 100 as its

syndrome, v-000100 = 101101 was sent.syndrome, v-000100 = 101101 was sent.

11 11 00

11 00 11

00 11 11

11 00 00

00 11 00

00 00 11