Bayes’ Theorem

Bayes’ TheoremIn this section, we look at how we can use informationabout conditional probabilities to calculate the reverseconditional probabilities such as in the example below. Wealready know how to solve these problems with treediagrams. Bayes’ theorem just states the associatedalgebraic formula.

Example Suppose that a factory has two machines,Machine A and Machine B, both producing jPhone touchscreens. Forty percent of their touch screens come fromMachine A and 60% of their touch screens come fromMachine B. Ten percent of the touch screens produced byMachine A are defective and five percent of the touchscreens from Machine B are defective. If I randomly choosea touch screen from those produced by both machines andfind that it is defective, what is the probability that it camefrom machine A?

Bayes’ TheoremIn this section, we look at how we can use informationabout conditional probabilities to calculate the reverseconditional probabilities such as in the example below. Wealready know how to solve these problems with treediagrams. Bayes’ theorem just states the associatedalgebraic formula.

Example Suppose that a factory has two machines,Machine A and Machine B, both producing jPhone touchscreens. Forty percent of their touch screens come fromMachine A and 60% of their touch screens come fromMachine B. Ten percent of the touch screens produced byMachine A are defective and five percent of the touchscreens from Machine B are defective. If I randomly choosea touch screen from those produced by both machines andfind that it is defective, what is the probability that it camefrom machine A?

Bayes’ Theorem

We can draw a tree diagram representing the informationwe are given. If we choose a touch screen at random fromthose produced in the factory, we let MA be the event thatit came from Machine A and let MB be the event that itcame from Machine B. We let D denote the event that thetouch screen is defective and let ND denote the event thatit is not defective. Fill in the appropriate probabilities onthe tree diagram on the left on the next page.

Bayes’ Theorem

xxxxxxxxxND

CCCCCCCC

{{{{{{{{

FFFFFFFF D

Bayes’ Theorem

We can now calculate P(MA

∣∣D)=

P(MA ∩D)

∣∣MA)·P(MA) + P

∣∣MB)·P(MB)

. Note the

event D is shown in red above and the event MA ∩D isshown in blue.

0.1rrrrrrrrrrr

0.6 JJJJJJJJJJJ

0.4ttttttttttt

MB0.05

0.95 LLLLLLLLLL D

Bayes’ TheoremD

0.1rrrrrrrrrrr

0.6 JJJJJJJJJJJ

0.4ttttttttttt

MB0.05

0.95 LLLLLLLLLL D

∣∣D)=

P(MA ∩D)

∣∣MA)·P(MA) + P

∣∣MB)·P(MB)

0.4 · 0.1

0.4 · 0.1 + 0.6 · 0.05=

0.04 + 0.03=

Bayes’ Theorem

Let E1 and E2 be mutually exclusive events (E1 ∩ E2 = ∅)whose union is the sample space, i.e. E1 ∪ E2 = S. Let Fbe an event in S for which P(F ) 6= 0. Then

∣∣F)=

P(E1 ∩ F )

P(F )=

P(E1 ∩ F )

P(E1 ∩ F ) + P(E2 ∩ F )=

P(E1)P(F

∣∣E1

)P(E1)P

∣∣E1

)+ P(E2)P

∣∣E2

Note that if we cross-classify outcomes in the sample spaceaccording to whether they belong to E1 or E2 and whetherthey belong to F or F ′, we get a tree diagram as abovefrom which we can calculate the probabilities.

Bayes’ Theorem

||||||||F ′

@@@@@@@@

~~~~~~~~

BBBBBBBB F

P(F| E )

Predictive value of diagnostic testThe above analysis allows us to gain insight a commonlymisunderstood point about the accuracy of tests fordiseases and drugs. The predictive value of a diagnostictest does not depend entirely on the sensitivity of the test.It also depend on the prevalence of the disease. Manypeople when asked the following question “If a swimmerfails a drug test that is known to be 95 percentaccurate(whether they have drugs in their system or not),how likely is it that he/she is really guilty?” will answer 95percent, but of course you know that you need moreinformation in order to answer the question. Check out thefollowing article on the subject:

Doctors flunk quiz on screening-test math

If you are more legally inclined, here is a discussion ofBayes Theorem as it applies to criminal trials.Let’s see what happens in a few examples.

If you are more legally inclined, here is a discussion ofBayes Theorem as it applies to criminal trials.

Let’s see what happens in a few examples.

If you are more legally inclined, here is a discussion ofBayes Theorem as it applies to criminal trials.Let’s see what happens in a few examples.

Predictive value of diagnostic test

Example Suppose, for example a test for the HIV virus is95% accurate.The test gives a positive result for 95% ofthose taking the test who are HIV positive. Also the testgives a negative result for 95% of those taking the test whoare not HIV positive.

(a) According to a recent estimate, approximately onemillion people in the U.S. are HIV positive. The populationof the U.S. is approximately 308 million. Suppose a randomU.S. resident takes the aids test and tests positive, what isthe probability that the person is infected given that theyhave tested positive, That is what is P

(I∣∣P)

(We let P denote the event that a person chosen at randomfrom the population tests positive, we let I denote theevent that a person chosen at random is infected.)

Predictive value of diagnostic testP

lllllllllllllllllNP

QQQQQQQQQQQQQQQQQ

mmmmmmmmmmmmmmmm

RRRRRRRRRRRRRRRRRR NP

0.05lllllllllllllllll

0.95NP

308QQQQQQQQQQQQQQQQQ

308mmmmmmmmmmmmmmmm

0.95 RRRRRRRRRRRRRRRRRR NP

P(I∣∣P)≈

1308· 0.95

307308· 0.05 + 1

308· 0.95

0.00308441558442

0.04983766233766 + 0.00308441558442=

0.00308441558442

0.05292207792208≈ 0.0583 = 5.83%

Predictive value of diagnostic test(b) In country X, forty percent of the residents are HIVpositive. Suppose a random resident of Country X takesthe aids test and tests positive, what is the probability thatthe person is infected given that they have tested positive,That is what is P

(I∣∣P)

lllllllllllllllllNP

QQQQQQQQQQQQQQQQQ

mmmmmmmmmmmmmmmm

0.95NP

0.4 QQQQQQQQQQQQQQQQQ

0.6mmmmmmmmmmmmmmmm

P(I∣∣P)

=0.4 · 0.95

0.6 · 0.05 + 0.4 · 0.95=

0.03 + 0.38=

0.41≈

0.9268 ≈ 93%

(I∣∣P)

lllllllllllllllllNP

QQQQQQQQQQQQQQQQQ

mmmmmmmmmmmmmmmm

0.95NP

0.6mmmmmmmmmmmmmmmm

P(I∣∣P)

=0.4 · 0.95

0.6 · 0.05 + 0.4 · 0.95=

0.03 + 0.38=

0.41≈

0.9268 ≈ 93%

(I∣∣P)

lllllllllllllllllNP

QQQQQQQQQQQQQQQQQ

mmmmmmmmmmmmmmmm

0.95NP

0.6mmmmmmmmmmmmmmmm

P(I∣∣P)

=0.4 · 0.95

0.6 · 0.05 + 0.4 · 0.95=

0.03 + 0.38=

0.41≈

0.9268 ≈ 93%

(I∣∣P)

lllllllllllllllllNP

QQQQQQQQQQQQQQQQQ

mmmmmmmmmmmmmmmm

0.95NP

0.6mmmmmmmmmmmmmmmm

P(I∣∣P)

=0.4 · 0.95

0.6 · 0.05 + 0.4 · 0.95=

0.03 + 0.38=

0.41≈

0.9268 ≈ 93%

Predictive value of diagnostic testExample A test for Lyme disease is 60% accurate when aperson has the disease and 99% accurate when a persondoes not have the disease. In Country Y, 0.01% of thepopulation has Lyme disease. What is the probability thata person chosen randomly from the population who testpositive for the disease actually has the disease?

lllllllllllllllllNP

QQQQQQQQQQQQQQQQQ

mmmmmmmmmmmmmmmm

0.99NP

0.9999mmmmmmmmmmmmmmmm

P(I∣∣P)

= 0.0001 · 0.06

0.9999 · 0.01 + 0.0001 · 0.06=

0.00006

0.00006 + 0.009999=

0.00006

0.010059≈

0.006.

Predictive value of diagnostic testExample A test for Lyme disease is 60% accurate when aperson has the disease and 99% accurate when a persondoes not have the disease. In Country Y, 0.01% of thepopulation has Lyme disease. What is the probability thata person chosen randomly from the population who testpositive for the disease actually has the disease?

lllllllllllllllllNP

QQQQQQQQQQQQQQQQQ

mmmmmmmmmmmmmmmm

0.99NP

0.9999mmmmmmmmmmmmmmmm

P(I∣∣P)

= 0.0001 · 0.06

0.9999 · 0.01 + 0.0001 · 0.06=

0.00006

0.00006 + 0.009999=

0.00006

0.010059≈

0.006.

A legal example

A crime has been committed and the only evidence is ablood spatter that could only have come from theperpetrator. The chance of a random individual having thesame blood type as that of the spatter is 10%. Joe hasbeen arrested and charged. The trial goes as follows.Prosecutor: Since there is only a 10% chance that Joe’sblood would match, there is a 90% chance that Joe did it.That’s good enough for me.Defence Lawyer: There are two hundred people in theneighborhood who could have done the crime. Twenty ofthem will have the same blood type as the sample. Hence

the chances that Joe did it are1

20= 5% so there is a 95%

chance that Joe is innocent. That’s good enough for me.

The ghost of the Reverend Thomas Bayes: You’re allnuts!

./figures/guilty.{ps,eps} not found (or no BBox)If P(I) = x and so P(G) = 1− x then

P(I∣∣M)

=0.1 · x

0.1 · x + 1 · (1− x)=

1− 0.9x

The ghost of the Reverend Thomas Bayes: You’re allnuts!./figures/guilty.{ps,eps} not found (or no BBox)If P(I) = x and so P(G) = 1− x then

P(I∣∣M)

=0.1 · x

0.1 · x + 1 · (1− x)=

1− 0.9x

If P(I) = x and so P(G) = 1− x then

P(I∣∣M)

=0.1 · x

0.1 · x + 1 · (1− x)=

1− 0.9x

If you think that Joe is guilty, then x = 0 and after seeingthe evidence you still think Joe is guilty, P

(I∣∣M)

= 0.If you think that Joe is innocent, then x = 1 and afterseeing the evidence you still think Joe is innocent,P

(I∣∣M)

= 1.If you think that Joe there is a 40% change that Joe isguilty, then x = 0.4 and after seeing the evidenceP

(I∣∣M)

= 0.0625.If you think that the police just searched a blood typedatabase until they came up with a name in theneighborhood, then you should probably start with the

defense lawyer’s idea that x = P (I) =19

20= 95%. Now after

seeing the evidence P(I∣∣M)

=0.095

1− 0.855=

0.145= 0.66.

P(I∣∣M)

=0.1 · x

0.1 · x + 1 · (1− x)=

1− 0.9xIf you think that Joe is guilty, then x = 0 and after seeingthe evidence you still think Joe is guilty, P

(I∣∣M)

If you think that Joe is innocent, then x = 1 and afterseeing the evidence you still think Joe is innocent,P

(I∣∣M)

20= 95%. Now after

=0.095

1− 0.855=

0.145= 0.66.

P(I∣∣M)

=0.1 · x

0.1 · x + 1 · (1− x)=

(I∣∣M)

If you think that Joe there is a 40% change that Joe isguilty, then x = 0.4 and after seeing the evidenceP

(I∣∣M)

20= 95%. Now after

=0.095

1− 0.855=

0.145= 0.66.

P(I∣∣M)

=0.1 · x

0.1 · x + 1 · (1− x)=

(I∣∣M)

= 0.0625.

If you think that the police just searched a blood typedatabase until they came up with a name in theneighborhood, then you should probably start with the

20= 95%. Now after

=0.095

1− 0.855=

0.145= 0.66.

P(I∣∣M)

=0.1 · x

0.1 · x + 1 · (1− x)=

(I∣∣M)

20= 95%. Now after

=0.095

1− 0.855=

0.145= 0.66.

General Bayes’ Theorem

Let E1, E2, . . . , En be (pairwise) mutually exclusive eventssuch that E1 ∪ E2 ∪ · · · ∪ En = S, where S denotes thesample space. Let F be an event such that P(F ) 6= 0, Then

∣∣F)=

P(E1 ∩ F )

P(F )=

P(E1 ∩ F )

P(E1 ∩ F ) + P(E2 ∩ F ) + · · ·+ P(En ∩ F )=

P(E1)P(F

∣∣E1

)P(E1)P

∣∣E1

)+ P(E2)P

∣∣E2

)+ · · ·+ P(En)P

∣∣En

Example A pile of 8 playing cards has 4 aces, 2 kings and2 queens. A second pile of 8 playing cards has 1 ace, 4kings and 3 queens. You conduct an experiment in whichyou randomly choose a card from the first pile and place iton the second pile. The second pile is then shuffled and yourandomly choose a card from the second pile. If the carddrawn from the second deck was an ace, what is theprobability that the first card was also an ace?

General Bayes’ TheoremLet A be the event that you draw an ace, K the event thatyou draw a king and Q be the event that you draw a queen.

mmmmmmmmmmmmmmmm

QQQQQQQQQQQQQQQQ K2

66666666666666666666666666

��K

mmmmmmmmmmmmmmmm

QQQQQQQQQQQQQQQQ K2

QQQQQQQQQQQQQQQQ

mmmmmmmmmmmmmmmmK2

In the first round there are 4 + 2 + 2 = 8 cards so theprobabilities in the first round are

mmmmmmmmmmmmmmmm

QQQQQQQQQQQQQQQQ K2

666666666666666666666666660.25

��K

mmmmmmmmmmmmmmmm

QQQQQQQQQQQQQQQQ K2

QQQQQQQQQQQQQQQQ

mmmmmmmmmmmmmmmmK2

General Bayes’ TheoremIn the second round there are 1 + 4 + 3 + 1 = 9 cards andthe probabilities are different at the various nodes. If youdraw an ace in round 1 the cards are 2 aces, 4 kings and 3queens so we get

mmmmmmmmmmmmmmmm49

QQQQQQQQQQQQQQQQ K2

666666666666666666666666660.25

��K

mmmmmmmmmmmmmmmm

QQQQQQQQQQQQQQQQ K2

QQQQQQQQQQQQQQQQ

mmmmmmmmmmmmmmmmK2

General Bayes’ TheoremIf you draw a king in round 1 the cards are 1 ace, 5 kingsand 3 queens so we get

mmmmmmmmmmmmmmmm49

QQQQQQQQQQQQQQQQ K2

666666666666666666666666660.25

��K

mmmmmmmmmmmmmmmm

QQQQQQQQQQQQQQQQ K2

QQQQQQQQQQQQQQQQ

mmmmmmmmmmmmmmmmK2

If you draw a queen in round 1 the cards are 1 ace, 4 kingsand 4 queens so we get

mmmmmmmmmmmmmmmm49

QQQQQQQQQQQQQQQQ K2

666666666666666666666666660.25

��K

mmmmmmmmmmmmmmmm

QQQQQQQQQQQQQQQQ K2

49 QQQQQQQQQQQQQQQQ

mmmmmmmmmmmmmmmmK2

General Bayes’ Theorem A2

mmmmmmmmmmmmmmmm49

QQQQQQQQQQQQQQQQ K2

666666666666666666666666660.25

��K

mmmmmmmmmmmmmmmm

QQQQQQQQQQQQQQQQ K2

49 QQQQQQQQQQQQQQQQ

mmmmmmmmmmmmmmmmK2

The question asks for

∣∣A2)

=P(A ∩ A2)

P(A2)=

12· 2

912· 2

Bayes’ Theorem - University of Notre Dameapilking/Math10120/Lectures/Solutions/Topic13.pdf ·...

Documents

Bayes Theorem and Naive BayesClassifier

Understanding bayes theorem

Tree Diagrams & Bayes Theorem

PROBABILITY AND BAYES THEOREM

Bayes’ Theorem Bayes’ Theorem Proof - Computer Scienceevans/poker/wp-content/uploads/2011/02/class3.pdf · • One-line Proof of Bayes’ Theorem • Inductive Learning Home Game

GEOGG121: Bayes’ Theorem & Monte Carlo · •Intro to Bayes’ Theorem –Science and scientific thinking –Probability & Bayes Theorem –why is it important? –Frequentistsv

Bayes Theorem & Share Trading on the ASX

Bayes Theorem, Independence, and Discrete Random Variables

Bayes Theorem

Yudkowsky Bayes' Theorem

Decision Trees & Bayes Theorem 2016

On Bayes’ Theorem and the Inverse Bernoulli Theorem Bayes’ Theorem and the Inverse Bernoulli Theorem A. I. ... number of cases by experiment ... to the discussion by I. Hacking

Introduction - WordPress.com · Introduction y Naive Bayes is a simple probabilistic classifier based on applying Bayes' theorem. y Bayes theorem is the one that has assumptions on

Bayes Theorem and Concept Learning (6.3) - cedar.buffalo.edusrihari/CSE574/ChapBL/ChapBL.Part2.pdf · Bayes Theorem and Concept Learning (6.3) • Bayes theorem allows calculating

Conditional Probability, Total Probability Theorem and Bayes

Bayes' Theorem - University of Michigan Health System

Lecture 3: Bayesian parameter estimates I. › ... › Lecture3.pdf · Lecture 3: Bayesian parameter estimates I. Bayes’ theorem. ... Bayes Theorem Thomas Bayes 1702-1761 (same

Bayes’ Theorem - ggn.dronacharya.info

Conditional Probability and Bayes Theorem