Upload
others
View
9
Download
0
Embed Size (px)
Citation preview
Computer Mathematics
Week 6Coding systems and error detection
Department of Mechanical and Electrical System Engineering
last week
Mouse Keyboard GPU Audio
Input / OutputController
Universal Serial Bus PCI Bus
Central Processing Unit
addressbus
CU
PCIR
ALU
registers
PSR
DR
operationselect
incrementPC
AR
RandomAccessMemory
0
4
8
16
20
24
28
databus
HDD SSD Net
coding theory
� source coding
information theory concept
� information content
binary codes
� numbers
� text
variable-length codes
� UTF-8
compression
� Huffman’s algorithm
2
this week
Input / OutputController
Universal Serial Bus PCI Bus
Mouse GPU
Central Processing Unit
addressbus
CU
PCIR
ALU
registers
PSR
DR
operationselect
incrementPC
AR
RandomAccessMemory
0
4
8
16
20
24
28
databus
Keyboard HDD Audio SSD Net
coding theory
� channel coding
information theory concept
� Hamming distance
error detection
� motivation
� parity
� cyclic redundancy checks
error correction
� motivation
� block parity
� Hamming codes
3
channel coding — motivations
channel coding protects against data corruption
while being transmitted
� drop outs, dead zones
� electromagnetic interference... 1 0 ... ... 0 1 ...
transmitter receiver
while being stored
� device failure
� ionising radiation
0
00
0
0
00
0
0
0
0
1
1
1
1
1
1 1
1
1 1
1
1
1
0
4
channel coding — approach
add information to a message, allowing corruption to be detected
recover uncorrupted data by
� retransmission– receiver asks for the same data again
� redundancy– repetition: multiple copes of the same data are transmitted/stored, or– encoding: additional bits allow small errors to be identified and corrected
the medium can affect the choice of mechanism, for example
� networks: retransmission is always possible
� DVD/Blu-ray: retransmission is never possible
5
error detection using repetition
send each bit once: 0 or 1 0−→ −→ 1
� no error detection or correction
send each bit twice: 00 or 11 00−→ −→ 01 / 10
� single bit error detection
� no error correction
send each bit three times: 000 or 111 000−→ −→ 001 / 010 / 100
� double bit error detection
� single bit error correction
6
information theory concept
Hamming distance
� the number of bits of difference between two valid codes
valid Hamming error errorcodes distance detection correction
0, 1 1 0 bits 0 bits00, 11 2 1 0
000, 111 3 2 10000, 1111 4 3 100000, 11111 5 4 2
d d− 1 d−12
sending each bit twice (as 00 or 11)
� repeats the data verbatim (obviously), but also
� guarantees an even number of bits are transmitted per bit of information
we can generalise this last idea. . .
7
parity
parity
mathematics: the property of an integer with respect to being odd or even
computing: the state of being odd or even used to detect errors in binary-codeddata
the parity of a block of bits is
� even, if there is an even number of 1s
� odd, if there is an odd number of 1s
0000 even parity0001 odd parity0010 odd parity0011 even parity0100 odd parity
consider a single bit error in a block of bits
� either a 0 changes to a 1
� or a 1 changes to a 0 01010101 even parity01011101 odd parity
in either case, the number of 1s in the block changes by 1
� the parity changes (from odd to even, or from even to odd)
8
parity bits
idea: add a redundant bit to each block of data that forces it to have even parity; e.g.,
� if the block of data has even parity, add a 0 to the end
� if the block of data has odd parity, add a 1 to the end
(or you could choose to force odd parity, it doesn’t matter)
00000 even parity00011 even parity00101 even parity00110 even parity01001 even parity
↑ parity bitrecall: changing a single bit changes the parity
� the Hamming distance between valid codes is 2
⇒ parity can detect 1-bit errors (d− 1 = 1), but cannot correct errors (d−12 = 0)
advantages
� trivial to implement (add the bits, modulo 2)
� can trade between block size and reliability
disadvantages
� longer blocks have higher chance of undetectable double bit errors
� error recover requires retransmission let’s fix both of those. . .9
error detection for larger blocks — checksums
a checksum is a number calculated from the content of some data
� if the content changes, the checksum also changes
they can be used to ‘sign’ data, proving the content is authentic (or uncorrupted)
� on reception, verify that the checksum is correct
� also called ‘digital signatures’, or ‘digests’
popular checksums for digitally signing large blocks of data:
� md5 (e.g., the ‘md5’ program on Linux and Mac)
� sha256 (the ‘shasum’ program on Linux and Mac)
popular checksum for detecting changes in smaller blocks of data
� CRC-32 (e.g., the ‘sum’ program on Linux and Mac)
� CRC = Cyclic Redundancy Check
10
creating a cyclic redundancy check code
a n-bit CRC is the remainder after dividing the data by a large number
� a n+ 1 bit divisor is known to both sender and receiver
� the dividend consists of the original message data bits
� n additional bits, added to the right of the data, are initially all 0– this is where the remainder will appear
then a long division is performed; repeatedly:
� align the leftmost 1of the divisor with the leftmost 1of the dividend (data)
� ‘subtract’ the divisor from the dividend, using modulo-2 arithmetic on individualbits
(this will always change the leftmost 1 in the dividend to a 0)
until the dividend contains only 0s
the n additional bits to the right of the dividend now contain the ‘remainder’
� this is your n-bit CRC
� transmit it along with the message
11
verifying a cyclic redundancy check code
when the message is received
� repeat the CRC process described above, but
� initialise the n additional bits with the received CRC code
� when the algorithm finishes, if the data is undamaged, the n CRC bits will all be 0
using a 3-bit divisor 101, a 2-bit CRC for 10101110 is
transmitter10101110 0010100001110 00
10100000100 00
1010000000100
10100000000 01
−→
receiver10101110 0110100001110 01
10100000100 01
1010000000101
10100000000 00 ok
receiver (corrupted)10100110 0110100000110 01
1010000011101
10100000010 01
10 100000000 11 bad!
12
cyclic redundancy checks in communications
CRC-32 is a 32-bit cyclic redundancy check code
� the data validation code is a 32-bit checksum
� the checksum is based on a cyclic algorithm(a kind of long division, of the data by a 33-bit divisor)
� it is added to the data redundantly(it expands the message without adding new information)
CRCs are popular because they are
� simple to implement in hardware, including for serial streams of bits
� very good at detecting errors caused by electromagnetic noise
CRC-32 is used in Ethernet networks to detect corrupted packets
13
error correction — block parity
parity tells us if a word of data has an error
in a sequence of words
even parity bit ↓01010000 001100001 101110010 0011010010
? → 01100100 001111001 1
� we know which word is corrupted, but
� we do not know which bit in the word is corrupted
thinking two-dimensionally
� we know the row
� we do not know the column
to identify the column, use a parity word
even parity bit ↓01010000 001100001 101110010 0011010010
? → 01100100 001111001 1
even parity word→ 001001110↑?
� a group of parity bits working on columns, not rows
� identifies which column contains the corrupted bit
the parity bits provide lateral parity
the parity words provide longitudinal parity, for a few data words at a time
14
Hamming codes
block parity considers data as a 2-dimensional array of bits with (x, y) coordinates
if the bit at (x, y) is corrupted, we can identify it because
� its row’s lateral parity bit will be wrong, telling us y
� its column’s longitudinal parity bit will be wrong, telling us x
instead of (x, y) addressing within an array of bits, we could number them
� then use parity bits to identify the number of a bit that is corrupted
15
Hamming codesnumber the bits, starting from 1
any bit whose number is a power of 2 is a parity bit, pn (numbered 2n)
each parity bit pn checks all bits whose binary numbering includes 2n
� for example, bit 9 is numbered 10012 in binary, and so
� it is included in the parity calculations of p3 and p0 (because 23 + 20 = 9)
message: p0p1 0 p2 1 1 1 p3 0 1 0 0
bit number: 1 2 3 4 5 6 7 8 9 1 1 10 1 2
in binary: 1 0 1 0 1 0 1 0 1 0 1 0 ← 1 means this bit is included in p0
0 1 1 0 0 1 1 0 0 1 1 0 ← 1 means this bit is included in p1
0 0 0 1 1 1 1 0 0 0 0 1 ← 1 means this bit is included in p2
0 0 0 0 0 0 0 1 1 1 1 1 ← 1 means this bit is included in p3
encoded: 0 1 0 1 1 1 1 1 0 1 0 0
using even parity: p0 = 001100bits 135791
1
, p1 = 101110bits 236711
01
, p2 = 11110bits 45671
2
, p3 = 10100bits 89111
012
(the pattern can be extended to the right, for as many data and parity bits as needed)16
Hamming codes
example:
1 2 3 4 5 6 7 8 9 10 11 12
stored data: 0 1 0 1 1 1 1 1 0 1 0 0retrieved data: 0 1 0 1 1 0 1 1 0 1 0 0
check for correct (even) parities:
p0 = 001100bits 135791
1
ok
p1 = 100110bits 236711
01
wrong
p2 = 11010bits 45671
2
wrong
p3 = 10100bits 89111
012
ok
the incorrect parities are p2 and p1, so the corrupted bit is number 22 + 21 = 4+ 2 = 6
17
channel coding in action
redundant arrays of identical disks for server data storage
� repetition: RAID 1
� parity: RAID 3, RAID 5
ECC (error correcting code) RAM for server memory
� uses Hamming codes plus one additional parity bit
� 100% reliably performs single error correction, double error detection (SECDED)
� essential, if you care at all about losing 1 bit per hour per gigabyte of memory!
18
next week
Input / OutputController
Universal Serial Bus PCI Bus
Mouse GPU
Central Processing Unit
addressbus
CU
PCIR
ALU
registers
PSR
DR
operationselect
incrementPC
AR
RandomAccessMemory
0
4
8
16
20
24
28
databus
Keyboard HDD Audio SSD Net
the mathematics of logic circuits
� the foundation of all digital design
Boolean logic
� when 0 and 1 represent true and false
Boolean algebra
� Boolean functions
� canonical forms
simplification of Boolean expressions
� de Morgan’s laws
19
homework
practice generating and checking parity bits
� write a Python program to generate parity bits– simulate some errors in the data– detect the errors by checking the parity
practice block parity
� extend your Python program to correct single-bit errors
practice generating small CRC codes
� write a Python program to generate a CRC for strings
� using the same divisor, compare your result to somebody else’s result
ask about anything you do not understand
� from any of the classes so far this semester (or the lecture notes)
� it will be too late for you to try to catch up later!
� I am always happy to explain things differently and practice examples with you
20
glossary
channel coding — a form of encoding that adds redundant data to a message to allow detection and/orcorrection of message corruption.
checksum — a number that characterises the content of a block of data. The checksum is included withthe data during transmission, and verified during reception.
corruption — damage to a message during storage or communication, causing the values of bits tochange.
Cyclic Redundancy Check — a checksum based on modular division, especially good at detecting longsequences of corrupted bits as can occur in network communication.
error correction — identifying the specific bit that was corrupted, allowing it to be repaired.
error detection — identifying the presence of a corrupted bit, allowing retransmission of the message tobe requested.
even parity — a property of a block of data in which the number of 1s is even.
even parity bit — a bit added to a block of data that guarantees the data will have even parity.
Hamming code — a parity-based encoding of data that allows a corrupted bit to be identified andcorrected.
21
Hamming distance — the number of bits that must be changed to convert a valid message into anothervalid message.
lateral parity — parity that works ‘horizontally’ across a row of bits.
longitudinal parity — parity that works ‘vertically’ down a column of bits.
medium — anything that can carry a message. Media include copper wire for electrical messages, glassfibre for optical messages, space for electromagnetic radio messages, etc.
odd parity — a property of a block of data in which the number of 1s is odd.
odd parity bit — a bit added to a block of data that guarantees the data will have odd parity.
packet — in a computer network, a single unit of transmission (and retransmission, in the case of error).
parity — the quality of being even or odd.
redundant — data that is added to a message that does not increase the information content of themessage. Redundant data adds the ability to detect or correct errors within the message content.
repetition — transmission (or storage) of the same information multiple times.
retransmission — transmission of a message that has already been transmitted, but was found to containerrors when received.
22