seven-bitblocks

Information & Coding Theory

SourceEncoding

Source/Channel DecodingChannel/Source

Destin-ation

Symbols

s1,…,sq

InformationChannel(errors) Information

signal

Noise

signal+ noise

Example: Morse Code

A, …, Z Encoding

transmitter

keyer

telegraph wire

dots, dashesspaces

∙ ─ _

shortwave radio

Decoding

receiver

recognizer

A, …, Z

Example: ASCII Code

Character keyboard modem modemterminalscreen

characterTelephone

wireseven-bit

blocks

Symbols

s1,…,sq

Information Source – the symbols are undefined, and the “meaning” of the information being sent is not dealt with – only an abstract measure of the “amount” or “quantity.

Examples

text of various forms – reports, papers, memos, books, scientific data (numbers)

pictures of various forms – diagrams, art, photographic images, scientific data (e.g. from satellites)

sound of various forms – music, speech, noises, recorded sound, radio

animation of various forms – moving pictures, film, video tape, video camera, television

equations representing mathematical ideas or algorithms – two textual representation systems with graphical output: Tex & Mathematica

continuousanalog

waveforms and shapes

discretedigital

sampled and quantized

Analog vs. Digital

Examples of (apparently) analog sources of information:

pressure: + →

granularity – molecular

sound (amplitude versus time)

intensity: [0, 1] [0, 1] → +

granularity – crystalline

microphone → tape recorder → speaker

Examples of (apparently) digital sources of information:

Text

a′ ? B sequences of characters

different languages have different characters(often the typeface or writing style conveys additional meaning or information)

lenses → film (negative) → print (positive)

picture (amplitude versus space)

Examples of composites (some digital and analog)

color printing (as opposed to photography)

television intensity unquantized, temporally discretecolor spatially discrete

spatially discrete (dots)spectrum discrete (color separation)intensity discrete (dot or no dot)

random signal contains largest amount of information