11

Information Units of Measurement

Bit – The amount of information required to differentiate (decide) between two equally likely events.

Equally Likely ProbabilitiesH = log2 (N)

Unequal Likely ProbabilitiesH = log2 ( 1 / pi )

22

Equally Likely Examples

Paul Revere – Old North Church“One if by land, two if by sea”Alternatives – Land and Sea H = log2 (2) = 1 bit

Randomly chosen digit ( 0 – 9 ) H = log2 (10) = 3.3 bits

Randomly chosen letters ( A – Z ) H = log2 (26) = 4.7

33

Equally Likely Examples

Land = Light Sea = No LightNote: If three choices (not coming, land, sea) then H = log2 (3) = 1.6 bits Not Coming = No Light Land = 1 Light Sea = 2 Lights

Digit Bits Letter Letter Letter 0 0000 A 00001 K 01011 U 10101 1 0001 B 00010 L 01100 V 10110 2 0010 C 00011 M 01101 W 10111 3 0011 D 00100 N 01110 X 11000 4 0100 E 00101 O 01111 Y 11001 5 0101 F 00110 P 10000 Z 11010 6 0110 G 00111 Q 10001 7 0111 H 01000 R 10010 8 1000 I 01001 S 10011 9 1001 J 01010 T 10100

44

Information Measurements - continued

Average Information Conveyed by a series of events having different probabilities of occurrence.

H Average = pi x log2 (1 / pi)

Redundancy is the reduction in information due to unequal probabilities of occurrence.Redundancy ( % ) = ( 1 - H Average / H Maximum ) x 100 %

Bandwidth – The rate of information transmitted over a single channel, measured in bits per seconds (bps).

Sensitivity – Keenness or resolution of sensory system

55

Examples

RedundancyEnglish Language - Not all letters occur with samefrequency / probabilities and some often occurtogether as pairs th and qu . Redundancy 68 %

BandwidthHuman Hearing = 8000 bits per secondHuman Vision = 1000 bits per secondBoth of which are much faster than can be

processedand interrupted by our brains, hence serve as a

filter.

66

Signal Detection Theory (SDT)

Concept of Noise

Random variations of “non-signal” activity added toactual information relevant “signal”.

Noise is a condition imposed on the information signal.

Noise is a system phenomena that may originate in thetransmitter, receptor, or medium.

Random variation of noise levels is assumed to benormally distributed. (Gaussian – White Noise).

77

Signal Detection Theory - continued

Signal Processing Outcomes

Hit – Correct reception of true signal

Miss – Non-reception of true signal

False Alarm – Incorrect reception of false signal

Rejection – Correct rejection of false signal

88

Response Criterion

Likelihood of Observing SignalDepends on “Response Criterion Level”Say “Signal” or “No Signal”

Beta = Signal-to-Noise Ratio at a given criterion level = Response Bias

Sensitivity d’ = Keenness or resolution of sensory system

Response Criterion Level and Sensitivity are independent

99

Influences on Response CriterionSetting of Response Criterion Level depends on costs and benefits associated with possible outcomes.

Increase Response Criterion (shift towards right)Beta IncreasesSay “Signal” less oftenNumber of Hits Decreases / Number of False Alarms DecreasesNumber of Misses Increases“Conservative”

Decrease Response Criterion (shift towards left)Beta DecreasesSay “Signal” more oftenNumber of Hits Increases / Number of False Alarms Increases Number of Misses Decreases“Risky”

1010

Concept of Response Criterion

signal + noisedistribution

Intensity of “Sensory Activity”low high

Probabilityof

Occurrence

b

a

Say “signal”Say “no signal”

d’

noise onlydistribution

correctrejection

miss

hit

false alarm

Sensitivity = d’

Beta = b / a