Representativeness, Similarity, & Base Rate Neglect Psychology 466: Judgment & Decision Making Instructor: John Miyamoto 10/27/2015: Lecture 05-1 Note:

Representativeness, Similarity,

& Base Rate Neglect

Psychology 466: Judgment & Decision Making

Instructor: John Miyamoto 10/27/2015: Lecture 05-1

Note: This Powerpoint presentation may contain macros that I wrote to help me create the slides. The macros aren’t needed to view the slides. You can disable or delete the macros without any change to the presentation.

Outline

• Base-rate neglect:What is base-rate neglect?

How is base-rate neglect related to Bayesian reasoning

• Three types of evidence that people tend to ignore base-rate information.

• Similarity Heuristic: Base-rate neglect occurs because people often substitute a judgment of similarity for a judgment of probability (attribute substitution).

Psych 466, Miyamoto, Aut '15 2Representatitiveness Heuristic is an Example of Attribute Substitution

Representativeness Heuristic as Attribute Substitution

• Attribute Substitution: When trying to evaluate a hard-to-judge attribute, people often substitute a judgment of a related attribute that is easy-to-judge.

♦ E.g., when trying to judge a probability, people may substitute a judgment of availability or similarity.

♦ This is a common mental habit. People are often unaware that they are making this substitution.

Psych 466, Miyamoto, Aut '15 3Same Slide with Addition of Another Bulllet Point

Representativeness Heuristic as Attribute Substitution

• Attribute Substitution: When trying to evaluate a hard-to-judge attribute, people often substitute a judgment of a related attribute that is easy-to-judge.

• Similarity Thesis: People substitute a judgment of similarity for a judgment of probability.

♦ The conjunction fallacy – a consequence of similarity-based reasoning

♦ Insensitivity to sample size

♦ Insensitivity to regression effects

♦ Base rate neglect

♦ Misperceptions of randomness

Psych 466, Miyamoto, Aut '15 4

Discussed Last Thursday .

Discuss Today .

Background to Study of Base Rate Neglect

Background to Study of Base Rate Neglect

• What is "base-rate neglect"?♦ "Base-rate": Another name for the prior probability of an individual event.

♦ "Base-rate neglect" has occurred if a person's judgments of probability are insufficiently influenced by variations in base-rate across different problems.

Remainder of this Lecture:

• Bayesian Reasoning – what is it? What is a base-rate?

• Medical example of Bayesian reasoning

• Experimental evidence that people do not generally follow Bayes Rule when reasoning about probabilities

Psych 466, Miyamoto, Aut '15 5What is Bayesian Reasoning?

6Psych 466, Miyamoto, Aut '15

What is Bayesian Reasoning?

Bayesian Hypothesis: People reason as if uncertainty is measured by

subjective probability.• Possibly this is a normative claim.• Possibly this is a descriptive claim.

Bayesian theory accomplishes two things:

1. Tells us how our existing beliefs should adjust to new information. Tells us how our probabilities should change as we acquire new information.

2. Tells us how to adopt internally consistent gambling strategies. Tells us how to make optimal decisions relative to our values (utilities) and our subjective probabilities.

Bayes Rule - What Is It? Why Is It Important?

Bayes Rule

Bayes Rule – What Is It? Why Is It Important?


• Reverend Thomas Bayes, 1702 – 1761

British clergyman & mathematician

• Bayes Rule is fundamentally important to:o Bayesian statisticso Bayesian decision theoryo Bayesian models in psychology

P Data|Hypothesis P(Hypothesis)P(Hypothesis|Data) =

P(Data)

1

n

i ii

P(Data) P Data | Hypothesis P Hypothesis

Explanation of Bayes Rule

Bayes Rule – Explanation (Look at Handout)


P Data | Hypothesis P(Hypothesis)P(Hypothesis |Data)

P(Data)=

Derivation of the Odds Form of Bayes Rule

NormalizingConstant

Likelihood of the Data

Posterior Probability of the

Hypothesis

Prior Probability

of the Hypothesis

a.k.a. base rate of the hypothesis

Psych 466, Miyamoto, Aut '159

Bayes Rule – Odds Form

P D|H P(H)P H|D =

P D

P H| D

P( | D)~ H

Bayes Rule for H given D

Bayes Rule for not-H given D

Odds Form of Bayes Rule

Explanation of Odds form of Bayes Rule

P D|H P(H)= g

P D| H P(H)

P D|~ H P(~ H)P ~ H|D =

P D

H = a hypothesis, e.g., H = hypothesis that the patient has cancer

= the negation of the hypothesis,

e.g., ~H = hypothesis that the patient does not have cancer

D = the data, e.g., D = + test result for a cancer test

Bayes Rule (Odds Form)


P H| D

P(~ H | D)

~ H

Memorable Form of the Bayes Rule (Odds Version)

P D | H P(H)

P D | ~ H P(~ H)

Posterior OddsLikelihood Ratio(diagnosticity)

Prior Odds

H = a hypothesis, e.g., H = hypothesis that the patient has cancer

= the negation of the hypothesis,

e.g., ~H = hypothesis that the patient does not have cancer

D = the data, e.g., D = + test result for a cancer test

Bayes Rule (Odds Form)


P H| D

P(~ H | D)

~ H

Interpretation of Medical Test Result

P D | H P(H)

P D | ~ H P(~ H)

Posterior Odds = (Likelihood Ratio) x (Prior Odds)


Three Types of Empirical Violations of Bayes Rule

• Given statistical information about prior probabilities and likelihoods,

people's judgments of posterior probability

seriously violate Bayes Rule.

o Example: Physicians judgments of P( +Cancer | +Test Result) are much too high

• Judging probability in the context of widely-known base rates. o Preview of finding: Judgments ignore well-known base rates.

• Judging probability when base rates are experimentally

manipulated between subjects.

o Preview of finding: Judgments are approximately the same in conditions where the base rate is low or where it is high.

Medical Diagnosis Problem

Next .


Bayesian Analysis of a Medical Test Result (Look at Handout)

QUESTION: A physician knows from past experience in his practice that 1% of his patients have cancer (of a specific type) and 99% of his patients do not have the cancer. He also knows the probabilities of a positive test result (+ result) given cancer and given no cancer. These probabilities are:

P(+ test | Cancer) = .792 and P(+ test | no cancer) = .096

Suppose Mr. X has a positive test result. What is the probability that Mr. X has cancer?

• P(Cancer) = Prior probability of cancer = .01

• P(No Cancer) = Prior probability of no cancer = 1 - P(Cancer) = .99

Frequency-Based Solution to this problem

Have class write down

their answers


Frequency-Based Analysis of the Medical Test Result

• Imagine that we have data for 1,000 patients.

• A 1% cancer rate means: 990 do not have cancer, and 10 have cancer (on the average)

• 8 of 10 patients with cancer will have a + test result (8 0.792*10).

95 of 990 patients without cancer will have a + test result (95 0.092*990).

• 103 = 8 + 95 : Out of 103 patients with + test results,

8 have cancer (8/103 = 0.077) and 95 do not have cancer (95/103 = 0.92).

• So if Mr. X has a + test result, he is much more likely to not have cancer than to have cancer.

Math Analysis of the Medical Test Problem


Mathematical Analysis of the Medical Test Result

Mr. X has a + test result.

What is the probability that Mr. X has cancer?

P(+ test | Cancer) = .792 and P(+ test | no cancer) = .096

P(Cancer) = Prior probability of cancer = .01

P(No Cancer) = Prior probability of no cancer

= 1 - P(Cancer) = .99

P(Cancer | + test) = 1 / (12 + 1) = 0.077David Eddy: Fallacious Reasoning by Physicians re Interpretation of Test Result

P cancer | test P test | cancer P(cancer)P no cancer | test P test | no cancer P(no cancer)

0 792 0 010 0833 1 12

0 096 0 99

. .. /

. .

Bayes Rule


Fallacious Reasoning by Physicians

• Correct answer given the data (based on Eq. (4)):

P(Cancer | + Result) = (.792)(.01)/(.103) = .077

Notice: The test is very diagnostic but still P(cancer | + result)

is low because the base rate is low.

• David Eddy found that about 95% of physicians think that

P(cancer | +result) is about 75% in this case

(very close to the 79% likelihood of a + result given cancer).

Physicians sometimes overlook base rates!

Bad Advice from a Medical Textbook


Fallacious Probabilistic Reasoning in Medicine

• DeGowin & DeGowin. Bedside diagnostic examination. "When a patient

consults his physician with an undiagnosed disease, neither he nor the

doctor knows whether it is rare until the diagnosis is finally made.

Statistical methods can only be applied to a population of thousands. The

individual either has a rare disease or doesn't have it; the relative incidence

of two diseases is completely irrelevant to the problem of making his

diagnosis."

• Eddy documents many examples in medical literature of confusion between

P(cancer | +result) and P(+ result | cancer).

General Characteristics of Bayesian Inference

DeGowin & DeGowin say: Ignore the prior odds!


General Characteristics of Bayesian Inference

• The decision maker (DM) is willing to specify the prior probability of the

hypotheses of interest.

• DM can specify the likelihood of the data given each hypothesis.

• Using Bayes Rule, we infer the probability of the hypotheses given the data.

• Main Point of Cognitive Critique:

People tend to ignore base rates.

People do not use Bayes Rule to update their probabilities.


Given Information







• Judging probability in the context of widely-known base rates. o Preview of finding: Judgments ignore well-known base rates.




Medical Diagnosis Problem

Previous Topic .

Next Topic .


Problem #4: Tom W (K&T, 1973)

PERSONALITY DESCRIPTION OF TOM W:

Tom W. is currently a graduate student at the University of Washington. Tom W.

is of high intelligence, although lacking in true creativity. He has a need for

order and clarity, and for neat and tidy systems in which every detail finds its

appropriate place. His writing is rather dull and mechanical, occasionally

enlivened by somewhat corny puns and by flashes of imagination of the sci-fi

type. He has a strong drive for competence. He seems to have little feel and

little sympathy for other people and does not enjoy interacting with others. Self-

centered, he nonetheless has a deep moral sense.

Response Sheet


Analysis of Problem #4: Tom W (cont.)

Between-subjects design. Separate groups of subjects judged (i) base rate, (ii) similarity ranks, (iii) probability ranks.

Estimated Similarity ProbabilityBase Rate Rank Rank Grad specialization

_____ _____ _____ Bus AD

_____ _____ _____ Computer Science

_____ _____ _____ Engineering

_____ _____ _____ Humanities

_____ _____ _____ Law

_____ _____ _____ Library Science

_____ _____ _____ Medicine

_____ _____ _____ Physical Science

_____ _____ _____ Social Science

Same Slide without Emphasis Rectangles

Condition 3Condition 2Condition 1


Analysis of Problem #4: Tom W (cont.)

Between-subjects design. Separate groups of subjects judged (i) base rate, (ii) similarity ranks, (iii) probability ranks.

Estimated Similarity ProbabilityBase Rate Rank Rank Grad specialization

_____ _____ _____ Bus AD

_____ _____ _____ Computer Science

_____ _____ _____ Engineering

_____ _____ _____ Humanities

_____ _____ _____ Law

_____ _____ _____ Library Science

_____ _____ _____ Medicine

_____ _____ _____ Physical Science

_____ _____ _____ Social Science

Use Bayes Rule to Analyze Tom W Problem

Relative Prevalence of Different Grad Specializations

People fail to take this into account.

P D | H P(H)P D | ~ H P(~ H)

Tom W & Bayes Rule


P H|D

P(~ H|D)

D = Data = the personality description of Tom W

H = Hypothesis that Tom W is a ____ student, e.g., an engineering student

Results for Tom W

Probability of Grad Specialization H

Given Tom W's Description

Similarity of H to Di.e., similarity of stereotype H

to Tom W's Description


Tom W Problem - Results

Correlation between judged base rate and likelihood rank = -.63.

Correlation between similarity rank and likelihood rank = +.97.

"Likelihood" is just another word for "probability" and here it refers to the posterior probability of each of the grad specializations.

Repeat: Tom W & Bayes Rule


People fail to take this into account!

Tom W & Bayes Rule




Conclusions re Tom W





P H|D

P(~ H|D)

P D | H P(H)P D | ~ H P(~ H)

26

Similarity heuristic is one part of the representativeness heuristic.

• The judgment of probability ought to be base in part on a judgment of similarity. (Similarity determines the diagnosticity of the evidence.)

• Focusing on similarity should not cause one to overlook the relevance of base rate.

Psych 466, Miyamoto, Aut '15

Similarity Heuristic Predicts Tom W Results

Repeat: Bayes Rule Odds Form with Annotation for Components

People substitute a judgment of similarity, ....

How much is Tom like the stereotype of a computer science student?

... for a judgment of probability

What is the probability that Tom is a computer science student?


People fail to take this into account!

Tom W & Bayes Rule




Return to ThreeTypes of Empirical Violations of Bayes Rule





P H|D

P(~ H|D)

P D | H P(H)P D | ~ H P(~ H)


Tuesday, October 27, 2015: The Lecture Ended Here







• Judging probability in the context of widely-known base rates. o Tom W: Judgments ignore well-known base rates.




Lawyer/Engineer Problem

Previous Topic .

Next Topic .

Documents

Representativeness, Similarity, & Base Rate Neglect Psychology 466: Judgment & Decision Making Instructor: John Miyamoto 10/27/2015: Lecture 05-1 Note: