Dr. Zhuanghua Shi

A brief introduction ofSignal Detection Theory

Changes of Schedule

• Due to NCP Day next Wednesday, Course C schedule changes as follows:

•

2

Date ContentDec. 12 Signal Detection TheoryDec. 19 --- (NCP Day)Jan. 09 Practical – Signal Detection AnalysisJan. 16 Exam: Theoretical part

A demo: Detection task

• There will be 4 x-ray photos of travel suitcases presented briefly. Your task is to identify if there is a gun inside. Please write down the number of the photos.

3

4

+

5

+

6

+

7

+

8

Results

Photo No. Gun No Gun

1 2 11

2 11 2

3 0 13

4 0 13

Gun No Gun

Yes 11 2

No 15 24

9

10

External environmentSensation

Prior knowledgeExpectation

Perception & Action

Bias

Sensitivity

Brain

Sensitivity and Criterion

• Performance is a function of TWO properties of the observer:• How well the observer perceives stimuli – or

sensitivity• And, how does the observer choose to respond –

or criterion• We need methods to assess the two independently• SIGNAL DETECTION THEORY

11

History of signal detection theory

• WWII, researchers concerned with maximization of correct detection responses and minimization of false detection responses• No way to optimize both

• First introduced by radar researchers (Marcum, 1960)• How to discriminate radar signal and noise

Signal and noise

0 50 100 150 200 250 300 350 400 450 500-1

-0.5

0

0.5

1

1.5

2

Signal Noise S + N = stimulus

0 50 100 150 200 250 300 350 400 450 500-1

-0.5

0

0.5

1

1.5

2

0 50 100 150 200 250 300 350 400 450 500-1

-0.5

0

0.5

1

1.5

2

0 50 100 150 200 250 300 350 400 450 500-1

-0.5

0

0.5

1

1.5

2

Noise0 50 100 150 200 250 300 350 400 450 500

-1

-0.5

0

0.5

1

1.5

2

Signal

Sensory signal and decision making

• Signals are always accompanied with background noise• luminance detection (background luminance)• target detection (against distractors)

• For any sensory inputs the brain need to interpret its meaning (e.g., signal or noise)

• Decision making (Criterion)• Simple two-alternative force choice• Above that criterion: Yes• Below that criterion: No

Confusion matrix – decoding the observer

• Different combinations of stimuli and responsesStimuli

Signal Noise

ResponseYes Hit False Alarm

No Miss Correct Rejection

What is noise?

External noise• interferes the signal

Internal noise• corrupts signal

representation

Signal and Noise: Statistical view

• Noisy signal representation, Rs:

!" = $ + &where S is signal, N is noise• How can we decode the signal from !"?

Distribution of signal representations

• If noise is white, then the received noisy signal will distribute as a normal distribution • yielding a normal distribution of signal

representations• The mean of this distribution would reflect the

signal

18

Mean signal, S

Signal representation

Prob

abili

ty

Distribution of noise

• Gaussian distribution of noise representation near 0• Usually we assume signal is positive, that is,

!" > !$

19

Mean noise, N

Noise representation

Prob

abili

ty

Decoding the signal

• Decision can be made based on the information from two distribution (e.g., likelihood)

20

N S

Sometime decision is hard!

21

N S

When you have to make a hard decision, flip a coin.

Why?Because when that coin is in the air…

You suddenly know what you’re hoping for.

22

Bias emerges !

• By comparing signal representation with a criterion

Decision requires a criterion

23

N SC

RS < C: Noise RS > C: Signal

• If the signal representation is far from the criterion, the decision is easy

• The closer it gets, the harder the decision

Easy and hard decisions

24

N SC

Potential S Potential N

Describing the S, N, and C

25

N SC

Correct rejectionsp(Rs < C|N)

False alarmsp(Rs > C|N)

N SC

Missesp(Rs < C|S)

Hitsp(Rs > C|S)

S N

Y H FA

N M CR

100% 100%

Describing the S, N and C

• p(H) + p(M) = 1 , so knowing p(H) is sufficient• p(FA) + p(CR) = 1 , so knowing p(FA) is sufficient• We can describe S, N, and C fully by considering only

hits and false alarms

26

S H

Y H FA

N M CR

100% 100%

Describing the S, N and C

• Overall decision difficulty is determined by the signal/noise relationship – sensitivity

• Actual performance also depends on decision criterion

27

N SC

Sensitivity

Sensitivity and decision criterion

• Sensitivity (Discriminability)• Measure of how close signal and noise are• Supposed to be a property only of the sensory process

• Decision Criterion (Response Bias)• Measure of what is considered noise and what signal• Susceptible to motivation, strategy, etc.

• Sensitivity and criterion are independent

28

Sensitivity and decision criterionHi

ghLo

w se

nsiti

vity

Unbiased criterion Biased

29

Computing sensitivity and criterion

• What are the measuring units of S and N?• Independently of the measuring scale (e.g., sound

frequency, line length, etc.) we can standardize the distribution by computing deviance scores

Real measure [x] Standard measure [z]

30

Normal distribution

• Assumption – N and S are normally distributed

31

Normal distribution

• For each probability, we can compute a z score

• p = 97.7%; z = 2• p = 2.3%; z = -2

32

Z score – a horizontal distance measure

• d’ = difference between means of S and N

Computing sensitivity

33

d’

d0 = µS � µN

d0 = z(H) + z(CR)

= Z(H)� Z(FA)

Effects of N and S variability

• If N and s variability differ, individual variability has to be computed

• Sensitivity is then da

34

d’( )

2

22ns

nsad

ss

µµ

+

-=

• Criterion influences overall frequency of YES responses:1. p(H) = 1, p(FA) = .84; d’ = 22. p(H) = .84, p(FA) = .16; d’ = 2 3. p(H) = .16, p(FA) = 0; d’ = 2

Computing decision criterion

35

1 2 3

Computing decision criterion

• Likelihood-ratio, β = p(C|S)/p(C|N)1. p(C1|S) = .004, p(C1|N) = .242, β = .018 – liberal criterion2. P(C2|S) = .242, p(C2|N) = .242, β = 1 – neutral criterion3. P(C3|S) = .242, p(C3|N) = .004, β = 55 – conservative

36

1 32Pr

obab

ility

Computing decision criterion

• Response bias, C = -(zH + zFA)/21. zH = 3, zFA = 1, C = -2 – liberal criterion2. zH = 1, zFA = -1, C = 0 – neutral criterion3. zH = -1, zFA = -3, C = 2 – conservative

37

1 32

Z score

Relating criterion and sensitivity – ROC curve

• Receiver operating characteristics – ROC curve• Relation between p(H) and p(FA) for varying response

criteria• One curve shows one sensitivity, d’

38

1 2 3

0; 0,16

0,16; 0,84

0,84; 1

0,00

0,25

0,50

0,75

1,00

0,00 0,25 0,50 0,75 1,00

p(H)

p(FA)

ROC and different sensitivities

• Increasing sensitivity increases the slope of ROC curve = switching between ROC curves

39

ROC and different criteria

• Changing criteria shifts the location at a single ROC curve

40

1 32

Sensitivity from ROC curve

• The area under ROC reflects sensitivity• Usually, we have only ONE p(H)/p(FA) data point• Problem – how to estimate sensitivity from a single

point?

41

Sensitivity from ROC curve

• Solutions:• Assume normal distribution

• Approximate the area

42

Criterion from ROC curve

• The slope of the ROC curve at the observed point

• b = 1 – neutral criterion• b < 1 – liberal criterion• b > 1 – conservative criterion

43

Two-alternative Discrimination task

• Two signals A and B• Yes responses are PA and

PB

• Assume equal variance,

!"′ = % &' + % &)

* = −(% &' − % &) )/2

44

7.1. THE TWO-ALTERNATIVE DISCRIMINATION TASK 115

Figure 7.1: The signal-detection model for two-alternative discrimination,shaded areas correspond to correct responses.

The

more attractive to treat the discrimination stimuli symmetrically. Insteadof speaking of hits and false alarms, a better representation is to use theproportion of correct responses made to each stimulus. Denote the twosignals by A and B and the responses by A and B. The probabilities ofcorrect responses of each type are given by the conditional probabilitiesPA = P(A|A) and PB = P(B|B). Denote the corresponding proportions bytheir lowercase equivalents, PA and PB- Figure 7.1 shows the distributionalmodel of the task. The decision axis is represented by a single dimension,and observing a stimulus gives a value along this axis. The distribution ofthe observations depends on the which stimulus was presented. A decisioncriterion divides the continuum into response regions associated with thetwo responses. The shaded areas in the figure correspond to the probabili-ties PA and PB of correctly identifying the two types of stimulus.

The decision model of Figure 7.1 also implies the existence of a discrim-ination operating characteristic. By varying the position of the decisioncriterion, performance can be varied from an emphasis on A responses,with PA 1 and PB 0, to an emphasis on B responses, with PA 0 andPB 1. The relationship of the two probabilities is shown in Figure 7.2.This discrimination operating characteristic is reversed left to right fromthe detection operating characteristics because the horizontal axis plotscorrect responses instead of errors. Its interpretation is the same. In par-ticular, the area under the curve has the same meaning as a measure of thedistinguishability of the signals.

Suppose further that the distributions are Gaussian, with means uA anduB and common variance o2. Because the decision continuum cannot bedirectly observed, exact values for uA, uB, o2 , and the decision criterion

From Wickens (2002)

2-Alternative Force-Choice (2AFC) task

• 2AFC task can be treated as detection of the difference between the two• Suppose Noise (N), and Signal (N+S) both are normal distributed• The variance of the difference of the two has double! (See below)• !" = ! $% > $' = ! $% − $' > 0 ~ +(-%, 2)

45

179

6.3.4.2 Calculation of d ! for Biased 2AFC Let us consider the situation in which the two alternatives are presented sequen-

tially, i.e., 2IFC. Figure 6.9 plots the distribution of differences in sensory magni-tude between those in the fi rst interval (X1) and those in the second interval (X2), i.e., the distribution of X1 ! X2. Note that there are now two distributions, one for signal present in the fi rst interval and one for signal present in the second interval. The two distributions will be separated by 2 d " and have σ s of ! 2 (see above). If the observer is biased towards responding to one interval more than the other, then their criterion C will be non-zero. The observer’s decision rule is “ 1 ” (fi rst interval) if X1 ! X2 # C, and “ 2 ” (second interval) if X1 ! X2 $ C. As explained in Section A, the key to calculating d " for a biased 2AFC task is to classify the responses in terms of hits and false alarms, where a “ 1 ” response is scored as a hit when the signal is in the fi rst interval and a false alarm when the signal is in the second interval.

One can see from Figure 6.9 that:

z pH d C( ) ( )% " ! / 2 (6.17)

and

z pF d C( ) ( )% ! " ! / 2 (6.18)

6.3. SECTION B: THEORY 179

0

0.1

0.2

0.3

0.4

−4 −3 −2 −1 0 1 2 3 4 5 60

0.1

0.2

0.3

0.4

Sensory magnitude (z units)

d"

σ = 1

σ = √2

NS + N

(S + N)−N

Pro

babi

lity

dens

ity

FIGURE 6.8 Graphical illustration of how d " can be calculated for an unbiased 2AFC task. Top: distributions of noise alone ( N ) and signal-plus-noise ( S & N ) separated by d " . Bottom: distribution of the difference between the two distributions: ( S & N ) ! N . Note the different σ s for the upper and lower distributions. The z values along the abscissa are normalized to the σ of the two distributions in the top, not bottom panel. See text for further details.

!" = 1(23

2)

2′ = 5 !" 2

Stimuli intensity and sensitivity

• Detection rate is prop. to the sensitivity• Yet, it includes biases• Strictly speaking, a null stimulus (noise alone) should be used, so we

can distinguish the sensitivity from response bias

46

pashler-44093 book December 18, 2001 10:12

Models with Simpler Assumptions (or None at All?) 59

Table 2.8 Evaluation of Sensitivity and Bias in Two-Response Classification

Number ofTask Measure Proportions Conclusion

Psychometric function—detection Absolute threshold 1 BiasPsychometric function—discrimination PSE 1 Bias

JND 2 SensitivitySpeech classification Boundary location 1 Bias

Trading relation 2 Change in bias?Sensitivity?

False memory Difference between 2 Sensitivity“yes” rates for twotypes of lures

0

0.2

0.4

0.6

0.8

1.0

1 2 3 4 5 6 7

P(“

yes”

)

Stimulus number

0

1.0

2.0

3.0

4.0

1 2 3 4 5 6 7

Cum

ulat

ive

d!

Stimulus number

(a) (b)

Figure 2.10 A psychometric function for detection.NOTE: Stimulus 1 is Noise, and the other stimuli are increasing nonzero levels of intensity. In panel (a),P(“yes”) is plotted, in panel (b), the z score of this value is plotted. The plot in panel (b) can also beinterpreted as portraying values of cumulative d ′.

the noise-alone stimulus was not immedi-ately appreciated, but from an SDT perspec-tive it is obviously necessary to distinguishsensitivity from response bias. The observersets a criterion along the decision axis, re-sponding “yes” to points above and “no”to points below it. Traditionally, the datummost often abstracted from a psychometricfunction ( frequency-of-seeing curve, in vi-sion) is the (absolute) threshold, the stimu-lus value corresponding to some fixed per-formance level such as 50% “yes.” This isjust a bias measure, as it depends on a singleproportion.

In a discrimination context, a null stimu-lus is not used, and it is the difference thresh-old whose value is sought.2 Historical prece-dence (Fechner is responsible, according toJones, 1974) and lasting influence belongto the method of constant stimuli: A stan-dard stimulus (usually drawn from the middleof the stimulus range) is presented on eachtrial, and the observer labels each comparison

2Two meanings of threshold must be distinguished. Ear-lier I presented examples of threshold theories that as-sumed discrete representations, but here the term is usedto refer to the weakest stimulus that can be detected, ordiscriminated, and has no theoretical implications.

Handbook of Experimental Psychology, vol. 4

Summary

• Performance in psychophysical paradigm depends both on sensitivity and response/decision criteria

• Similar to speed-accuracy trade-off, changes in criteria can mimic changes in sensitivity

• It is necessary to develop measures that independently index one and the other

• SDT helps us estimate sensitivity and decision criterion independently

47

Additional literature

• Stevens' Handbook of Experimental Psychology, 3rd Edition, see vol. 4

• Psychophysics: A Practical Introduction, Kingdom & Prins.

48