2303 Probability Bayes F12

INTRODUCTION TO INTRODUCTION TO PROBABILITYPROBABILITY

andandBAYES THEOREMBAYES THEOREM

Introduction to probabilityIntroduction to probability

• Basics of probability• Picturing probability: Venn diagrams, Tree

diagrams• Probability rules

– Complement rule

– Addition rules

– Multiplication rules

– Conditional probability

– Independence

– Reversing the conditioning: Bayes’ Theorem

Dealing with Random PhenomenaDealing with Random Phenomena

• A random phenomenon is a situation in which we know what outcomes could happen, but we don’t know which particular outcome did or will happen.

• When dealing with probability, we will be dealing with many random phenomena

ProbabilityProbability

ProbabilityProbability refers to the chance that a particular refers to the chance that a particular event will occur.event will occur.

The probability of an event will be a value in the range 0.00 to 1.00.

A value of 0.00 means the event will not occur (impossibility).

A probability of 1.00 means the event will occur (certainty).

Anything between 0.00 and 1.00 reflects the uncertainty of the event occurring.

Events and Sample Space:Events and Sample Space:ExperimentExperiment

An experimentexperiment is a process that produces a single outcome whose result cannot be

predicted with certainty.

Events and Sample Space:Events and Sample Space:Elementary EventsElementary Events

Elementary eventsElementary events are the most rudimentary outcomes resulting from a

simple experiment.

Events and Sample SpaceEvents and Sample Space

The sample space (S)sample space (S) is the collection of all elementary outcomes that can result from

an experiment.

Events and Sample SpaceEvents and Sample Space

An eventevent is a collection of elementary events.

Example 1:Example 1:Sample space and eventsSample space and events

• An experiment consists of randomly drawing three components from a batch production and checking whether they are defective or not.

• Let’s define the events:A: acceptable component

D: defective component

• Determine the sample space S for this experiment (tree diagram)

A

D

Example 1Example 1Sample space and eventsSample space and events

1st drawing

A

D

A

D

A

D

1st drawing 2nd drawing


A

D

A

D

A

D

D

A

A

D

A

D

A

D

1st drawing 2nd drawing 3rd drawing


A

D

A

D

A

D

D

A

A

D

A

D

A

D

Sample space (S)

AAA

AAD

ADA

ADD

DAA

DAD

DDA

DDD

Eig

ht

ele

men

tary

e

ven

ts1st drawing 2nd drawing 3rd drawing Possible outcomes


• B and C are two events defined as follows:

B: three components are acceptableB = {(AAA)}

• C: at least one component is acceptableC = {(ADD),(DAD),(DDA),(AAD),(ADA),(DAA),(AAA)}


Picturing ProbabilitiesPicturing Probabilities

• Tree diagrams can be used to define the sample space of an experiment

• Also, the most common kind of picture to make is called a Venn diagram.

• We will see Venn diagrams in practice shortly…

Mutually Exclusive Events (or Mutually Exclusive Events (or Disjoint Events)Disjoint Events)

Two events are mutually exclusivemutually exclusive if the occurrence of one event precludes the

occurrence of another event.

AS

B

• B: Three components are acceptable

B = {(AAA)}• Let E: at least one component is defective

E = {(AAD),(ADA),(DDA),(ADD),(DAA),(DAD),(DDD)}

• B and E are disjoint or mutually exclusive events

Example 1 (cont.)Example 1 (cont.)Two disjoint eventsTwo disjoint events

Trial, Experiment

Sample space (S) = all possible outcomes

Trial, Experiment


A combination of outcomes

Event (Ei)

Elementary: one outcome of S

A B

A B

Trial, Experiment


A combination of outcomes

Event (Ei)

Elementary: one outcome of S

Disjoint = No outcomes in common Non disjoint = can occur together, =

Mutually exclusive outcomes in common

SA SB

Four Types of ProbabilityFour Types of Probability

Marginal

The probability of A occurring

Union

The probability of A or B occurring

Joint

The probability of A and B occurring

Conditional

The probability of A occurring given that B has occurred

BA BA

B

A

)(AP )( BAP )( BAP )|( BAP

Independent and Dependent Independent and Dependent EventsEvents

Two events are independent independent if the occurrence of one event in no

way influences the probability of the occurrence of the other event.

See later Probability Rules 4 and 4a

Being independent is

a totally different thing

that being mutually exclusive

Independent and Dependent Independent and Dependent EventsEvents

Two events are dependent dependent if the occurrence of one event impacts the

probability of the other event occurring.

See later Probability Rules 4 and 4a

Assessing ProbabilitiesAssessing Probabilities

• Theoretical: Number of Possibilities• Relative Frequency• Subjective

Theoretical Probability Theoretical Probability AssessmentAssessment

Theoretical Probability AssessmentTheoretical Probability Assessment refers to the method of determining probability based on the ratio of the

number of ways the event of interest can occur to the total number of ways any event can occur when the

individual elementary events are equally likely.

THEORETICAL PROBABILITY MEASUREMENTTHEORETICAL PROBABILITY MEASUREMENT

Theoretical Probability Theoretical Probability AssessmentAssessment

events elementary ofnumber Total

occurcan E waysofNumber )P(E i

i

• What is the probability that three components are acceptable?

B: three components are acceptable– B = {(AAA)}

– P(B) = 1/8

• What is the probability that at least one component is acceptable?

C: at least one component is acceptable– C = {(ADD),(DAD),(DDA),(AAD),(ADA),(DAA),(AAA)}

– P(C) = 7/8

Example 1 (cont.)Example 1 (cont.)Theoretical probability assessmentTheoretical probability assessment

Relative Frequency of Relative Frequency of OccurrenceOccurrence

Relative Frequency of OccurrenceRelative Frequency of Occurrence refers to a method that defines probability as the number of times an

event occurs, divided by the total number of times an experiment is performed in a large number of trials.


RELATIVE FREQUENCY OF OCCURRENCERELATIVE FREQUENCY OF OCCURRENCE

where:

Ei = the event of interest

RF(Ei) = the relative frequency of Ei occurring

n = number of trials

noccurs E timesof Number

RF(E ii )


Relative Frequency of OccurrenceRelative Frequency of OccurrenceExample 2Example 2

On a production line products are testedthroughout a day.Out of 10000 products, 52 are found to bedefective.

Probability of defective product = 52/10000= 0.52%

Law of Large Numbers (LLN)Law of Large Numbers (LLN)

• If the events are independent, then as the number of trials increases, the long-run relative frequency of an event gets closer and closer to a single value.

Subjective Probability Subjective Probability AssessmentAssessment

Subjective Probability AssessmentSubjective Probability Assessment refers to the method that defines probability of an event as

reflecting a decision maker’s state of mind regarding the chances that the particular event

will occur.

Partner in consulting company puts in a bid fora contract, and subjectively assesses the chanceof getting the contract as 70%.

The Rules of ProbabilityThe Rules of Probability

PROBABILITY RULE 1PROBABILITY RULE 1

For any event Ei

0.0 P(Ei) 1.0 for all i

The Complement of an EventThe Complement of an Event

The complement complement of an event E is the collection of all possible elementary

events not contained in event E. The complement of event E is represented by

E or EC or E’

The complement of E is the same as “not E”



)(1)( EPECP

E EC

Sample Space

• We pick a component at random from a production line

• The inspection reveals that a component can be good (G) or defective (D)

• Two disjoint outcomes are possible : S = {G,D}• G = DC and D = GC

• If P(G) = 0.90 determine P(D) = ?P(D) = P(S) - P(G) = 1 - P(G) = 1 – 0.90 = 0.10

Example 3:Example 3:The complement ruleThe complement rule



Addition or “OR” rule for any two events E1 and E2:

P(E1 or E2) = P(E1) + P(E2) - P(E1 and E2)

E1 E2

2E 1E and


SPECIAL CASE OF THE “OR” RULE SPECIAL CASE OF THE “OR” RULE

“OR” rule for mutually exclusive (disjoint) events E1 and E2:

P(E1 or E2) = P(E1) + P(E2)

• Let A and B two disjoint events of S, with P(A) = 0.4 and P(B) = 0.45, we have:

Example 4Example 4Addition rule for two disjoint eventsAddition rule for two disjoint events

0.4=0-0.4=B) andP(A -P(A)=)B andP(A

0.15=0.85-1=

B)or P(A -1= B)or P(A =)B and P(A

0.85=0.45+0.4=P(B)+P(A)=B)or P(A

0=B) andP(A

0.6=0.4-1=P(A)-1=)P(A

C

CCC

C

A SB

• 400 students were recently interviewed concerning the newspapers they read.

• The study revealed that:– A: 165 read « The Citizen"

– B: 240 read « The National Post"

– A and B: 90 read both

Example 5Example 5Addition rule for any two eventsAddition rule for any two events

A B

A B


• The study revealed that:– A: 165 read « The Citizen"– B: 240 read « The National Post"– A and B: 90 read both

• What is the probability that a student reads “The Citizen”; “The National Post”


A B

A B


• The study revealed that:– A: 165 read « The Citizen"

– B: 240 read « The National Post"

– A and B: 90 read both

• What is the probability that a student reads “The Citizen”; “The National Post”


A B

A B

0.60400

240P(B) and 0.4125

400

165P(A)


• How many students read only The Citizen (A); only the National Post (B); None of them?

A B

B: Nat. Post BC:(Nat. Post)C

A : The Citizen 90 165

AC: (The Citiz.)C

240 400

A B

A B


• How many students read only The Citizen (A); only the National Post (B); None of them?

A B

B: Nat. Post BC:(Nat. Post)C

A : The Citizen 90 75 165

AC: (The Citiz.)C 150 85 235

240 160 400

A B

A B

• Illustrating using Venn diagram:

75

S85

90 150


A B

A B

• What is the probability that a student reads either of these two newspapers? (Think OR)


A B

A B

78.75%=400

315=

400

90-

400

240+

400

165=

B) andP(A -P(B)+P(A)=B)or P(A

Conditional ProbabilityConditional Probability

Conditional probabilityConditional probability refers to the probability that an event will occur given

that some other event has already happened.



Conditional probability for any two events E1 , E2:

0)(

)(

)()|(

2

2

2121

EP

EP

EandEPEEP

Definition of IndependenceDefinition of Independence

• Independence of two events means that the outcome of one event does not influence the probability of the other.

• With our new notation for conditional probabilities, we can now formalize this definition:– Events A and B are independent whenever

P(B|A) = P(B).

(Likewise, events A and B are independent whenever P(A|B) = P(A).)

• A: The consumer bought the product

• B: The consumer saw the commercial on the TV

A: bought Not_A: did'nt buy Total

B: saw the comm. 5 25 30Not_B: did'nt see the commercial 15 55 70Total 20 80 100

Example 6 Example 6 Conditional probability and Conditional probability and

independenceindependence

• A: The consumer bought the product

• B: The consumer saw the commercial on the TV

A: bought Not_A: did'nt buy Total

B: saw the comm. 5 25 30Not_B: did'nt see the commercial 15 55 70Total 20 80 100

30%=10030=P(B) and 20%

10020P(A)

Example 6 Example 6 Conditional probabilityConditional probability

• What is the probability that the consumer bought the product (A) and saw the commercial (B)?

%5100

5) and ( BAP



• What is the probability that the consumer bought the product (A) and saw the commercial (B)?

%5100

5) and ( BAP

• What is the probability that the consumer bought the product (A) given that he saw the commercial (B)?

%67.16100/30

100/5

)(

) and ()/(

BP

BAPBAP


PROBABILITY RULE 4aPROBABILITY RULE 4a

Multiplication or “AND” rule for any two events

E1 and E2:

)|()()(

)|()()(

21212

12121

EEPEPEandEP

and

EEPEPEandEP

Rule 4 revisited


SPECIAL CASE OF THE “AND” RULESPECIAL CASE OF THE “AND” RULE

Multiplication or “and” rule for two independent events

E1 , E2:

)()()( 2121 EPEPEandEP

Recall: Definition of independence: Two events are independent if:P(A|B)=P(A)

Independent IS NOT THE SAME Independent IS NOT THE SAME AS Mutually ExclusiveAS Mutually Exclusive

Independent Check whether P(B|A) = P(B)OrCheck whether P(A|B) = P(A)OrCheck whether P(A and B) = P(A) * P(B)

Disjoint (mutually exclusive)

Check whether P(A and B) = 0OrCheck whether the events A and B overlap in the sample space diagram.OrCheck whether both events can occur together

• Are the events A and B independent?– We have to prove that:

– Step 1:

– Step 2:

– Step 3:

Thus, A and B are not independent

Example 6 (cont.) Example 6 (cont.) IndependenceIndependence

P(A B) = P(A) P(B)

P(A B) = 5%

5% 6%%660.0)3.0()2.0()()( BPAP

Tree DiagramsTree Diagrams

• A tree diagram helps us think through conditional probabilities by showing sequences of events as paths that look like branches of a tree.

Example 7Example 7

• A construction Company has a 60% chance of obtaining an important contract. If this first contract is obtained, the company has a 2/3 chance of getting a second contract.

• If it doesn't obtain the first contract, its chances of obtaining the second one fall to 30%.

• What is the probability that the company obtains the second contract?

• Let:– A: The company obtains the first contract– B: The company obtains the second contract

• We know the following:– P(A) = 0.6 thus P(AC) = 0.4– P(B/A) = 2/3 thus P(BC/A) = 1/3– P(B/AC) = 0.3 thus P(BC/AC) = 0.7

• We have to find P(B) given:– P(A), and– The conditional probabilities

Example 7Example 7

Example 7Example 7

SA A’

B

P(B) = ?

P B P A B P A B( ) ( ) ( ' )

P(A)=0.6

P(A')=0.4

Example 7Example 7

P(A)=0.6

P(A')=0.4

P(B/A)=2/3

P(B'/A)=1/3

P(B/A')=0.3

P(B'/A')=0.7

Example 7Example 7

P(A)=0,6

P(A')=0,4

P(B/A)=2/3

P(B'/A)=1/3

P(B/A')=0.3

P(B'/A')=0.7

P A B P A P B A( ) ( ) ( / )

P A B' P A P B'/A( ) ( ) ( )

P A B P A P B A( ' ) ( ' ) ( / ' )

P A B' P A P B'/A( ' ) ( ' ) ( ' )

Example 7Example 7

P(A)=0,6

P(A')=0,4

P(B/A)=2/3

P(B'/A)=1/3

P(B/A')=0.3

P(B'/A')=0.7

P A B P A P B A( ) ( ) ( / )

P A B' P A P B'/A( ) ( ) ( )

P A B P A P B A( ' ) ( ' ) ( / ' )

P A B' P A P B'/A( ' ) ( ' ) ( ' )

=(.6)(2/3)=.4

=(.6)(1/3)=.2

=(.4)(.3)=.12

=(.4)(.7)=.28

Example 7Example 7

Example 7Example 7

SA A’

B

%5252.012.040.0

)'/()'()/()(

)'()()(

ABPAPABPAP

BAPBAPBP

40.0)( BAP 12.0)'( BAP

Let A be an event of S and E1, E2, ..., En, a set of mutually exclusive and exhaustive events of S

GeneralizationGeneralization

E1

A

E3

E5

E4

En

E2A A E A E

A E A En

( ) ( )

( ) ( )1 2

3

P Eii

n

( ) 1

1

Let A be an event of S and E1, E2, ..., En, a set of mutually exclusive and collectively exhaustive events of S

:

P A P A E P A E P A E n( ) ( ) ( ) ( ) 1 2

A A E A E

A E A En

( ) ( )

( ) ( )1 2

3

GeneralizationGeneralization

E1

A

E3

E5

E4

En

E2

P Eii

n

( ) 1

1

Let A be an event of S and E1, E2, ..., En, a set of mutually exclusive and collectively exhaustive events of S

P A P A E P A E P A E n( ) ( ) ( ) ( ) 1 2

)E/A(P)E(P

)E/A(P)E(P)E/A(P)E(P)A(P

nn

2211

)E/A(P)E(P)A(P i

n

1ii

Generalization: Probability rule 10Generalization: Probability rule 10

ExampleExample 8 8

• A manufacturing company produces mechanical components using 3 machines: M1, M2 and M3 in the following proportions: 50% from M1, 30% from M2 and 20% from M3.

• The probability of having a defective component given that it was produced by machine M1, M2 and M3 are 3%, 4% and 5% respectively.

• We pick a component at random from the components produced during the day.

• What is the probability that the component is defective?

• We want to know the probability of the event:– A: The component is defective

• Let:– E1: The picked component comes from M1

– E2: The picked component comes from M2

– E3: The picked component comes from M3

• E1, E2 and E3 are mutually exclusive

ExampleExample 8 8

• We don’t have the probability of A (directly), but we have the following probabilities:– P(E1) = 0.5 P(E2) = 0.3 P(E3) = 0.2

– P(A/E1) = 0.03 P(A/E2) = 0.04 P(A/ E3) = 0.05

ExampleExample 8 8

P Eii

n

( ) 1

1

S

A

E1 E2 E3

P(A) = ?

• We obtain the probability of A using:

ExampleExample 8 8

P A P E P A Eii

n

i( ) ( ) ( / )

1

%7.3037.0

)05.02.0()04.03.0()03.05.0()(

AP

• Thus, the probability to have a defective component produced during the day equals 3.7%

Reversing the ConditioningReversing the Conditioning

• Reversing the conditioning of two events is rarely intuitive.

• Suppose we want to know P(A|B), but we know only P(A), P(B), and P(B|A).

• We also know P(A and B), since

P(A and B) = P(A) x P(B|A)

• From this information, we can find P(A|B):

(A and B)(A|B)(B)

PPP

Reversing the Conditioning (cont.)Reversing the Conditioning (cont.)

• When we reverse the probability from the conditional probability that you’re originally given, you are actually using Bayes’ Rule (or Theorem) .

Let A be an event of S and E1, E2, ..., En, a set of mutually exclusive and exhaustive events of S, To obtain P(Ej / A):

P E AP E A

P Aj

j( / )

( )

( )

Bayes’ TheoremBayes’ Theorem

Let A be an event of S and E1, E2, ..., En,, a set of mutually exclusive and exhaustive events of S, To obtain P(Ej / A):

P E AP E A

P Aj

j( / )

( )

( )

P E A P E P A Ej j j( ) ( ) ( / )



P E AP E A

P Aj

j( / )

( )

( )

P E A P E P A Ej j j( ) ( ) ( / )

P A P E P A Eii

n

i( ) ( ) ( / )

1



P E AP E A

P Aj

j( / )

( )

( )

P E A P E P A Ej j j( ) ( ) ( / )

P A P E P A Eii

n

i( ) ( ) ( / )

1

P E AP E P A E

P E P A Ej

j j

ii

n

i

( / )( ) ( / )

( ) ( / )

1


• In the previous example (ex. 8), we knew P(A/Ei), i.e., the probability that a component is defective given that it is produced by machine Mi (event Ei)

• If we now pick a component at random from the production of the day, and we notice that it is defective (A), what is the probability that the component was produced by machine M3 (event E3)?

Example 9Example 9

Bayes’ RuleBayes’ Rule

• We are interested in determining the probability that the picked component comes from M3 (event E3) given that the component is defective, i.e., P(E3/A).

• We know the following:

– P(E1) = 0.5 P(E2) = 0.3 P(E3) = 0.2

– P(A/E1) = 0.03 P(A/E2) = 0.04 P(A/ E3) = 0.05

Example 9Example 9


• Using Bayes’ theorem, we have:

Example 9Example 9


P E AP E P A E

P E P A Ej

j j

ii

n

i

( / )( ) ( / )

( ) ( / )

1P(A)

%2727.0037.0

01.0

)05.02.0()04.03.0()03.05.0(

05.02.0)/( 3

AEP

What Can Go Wrong?What Can Go Wrong?

• In most situations where we want to find a probability, we’ll use the rules in combination.

• Beware of probabilities that don’t add up to 1.• Don’t add probabilities of events if they’re not

disjoint.• Don’t multiply probabilities of events if they’re

not independent.• Don’t confuse disjoint and independent—disjoint

events can’t be independent.

• Using Bayes’ theorem, we have:

Example 9Example 9


P E AP E P A E

P E P A Ej

j j

ii

n

i

( / )( ) ( / )

( ) ( / )

1P(A)

• Thus, once we know that the component is defective, the probability that it comes from the machine M3 is revised upward from 20% to 27%

%2727.0037.0

01.0

)05.02.0()04.03.0()03.05.0(

05.02.0)/( 3

AEP

Bayes TheoremBayes TheoremWhen to Use itWhen to Use it

• I know P(X|Y)

• I want to know P(Y|X)

• I have an initial estimate of P(Y) (Prior prob.)

• I do a test and find that X is true

• I update my initial estimate to get P(Y|X) (Posterior prob.)

Ethics in ActionEthics in ActionHair Salon: Facials or Massages?Hair Salon: Facials or Massages?

• Survey of salons offering facials and massages– 50% of customers wanting hair styling also want

facials

– 90% of customers want hair styling or massages

• Therefore we should offer massages.• Ethical Issue

– Item A: Probabilities not comparable (conditional/OR)

• Ethical Solution– Report all details of survey.

Documents

2303 Probability Bayes F12