Lect02-2019-Conditional Prob Idp-n - Kasetsart University

Department of Computer Engineering, Faculty of Engineering,Kasetsart University, THAILAND

1st Semester 2019 (July – Nov)

Assoc. Prof. Anan Phonphoem, Ph.D.

Department of Computer Engineering, Faculty of Engineering,Kasetsart University, THAILAND

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 3

P[☺]is a function that maps event

in the sample space to real numberFrom experiment: Roll a diceOutcomes:

number = 1,2,3,4,5,6Sample space:

S = {1,2,3,…,6}Event examples:

E1 = {number < 3} = {1,2}E2 = {number is odd} = {1,3,5}

P[E1] = 2/6 = 1/3P[E2] = 3/6 = 1/2

Axiom 1: For any event A, P[A] ³ 0

Axiom 2: P[S] = 1

Axiom 3: For events A1, A2,…, Anof mutually exclusive events P[A1ÈA2È…ÈAn] = P[A1]+P[A2]+…+P[An]

4Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty

• If we know P[A] before an experiment• P[A] » 1• Advanced knowledge à almost certainly occur

• P[A] » 0• Advanced knowledge à almost certainly not occur

• P[A] » ½• Advanced knowledge à maybe occur

• P[A] is a priori probability of A

5Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty

• In practice, it maybe impossible to find the precise outcome of an experiment • However, if we know that Event B has

occurred• Probability of A when B occurs can be described

(the outcome of Event A is in set B) • Still don’t know P[A]

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 6

• Notation: P[A|B]• “Probability of A given B”• The condition probability of the event A

given the occurrence of the event B • Definition:

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 7

P[A|B] = P[AB]P[B]

• P[B] > 0

B

AA Ç B

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 8

S

A

BP[A|B] = P[AB]

P[B]

P[A|S] = P[AS]P[S]P[A]

1=

= P[A]

S

A

B

A

A Ç BB

A Ç B

S

A

BBA Ç B

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 9

Assume: all teams are equally likely to win the game

What is the probability that France will be the champion ?

132

What is the probability that Germany will be the champion, given that Sweden and Mexico are withdrawn ?

130

or 116

Let B1, B2,…,Bn be mutually exclusive events whose union equals sample space S à partition of S

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 10

B1 B2B3

B4 Bn…A

Si = 1

nP[A Ç Bi]P[A] =

For any event AA = AÇS = AÇ(B1È B2È…ÈBn) P[A] = P[AÇB1] + P[AÇB2] +…+ P[AÇBn]

Theorem:

• Let B1, B2,…,Bn be mutually exclusive events whose union equals sample space S• P[Bi] > 0

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 11

B1 B2B3

B4 Bn…A

P[A] = P[AÇB1] + P[AÇB2] +…P[A] = P[A|B1]P[B1] + P[A|B2]P[B2] +…

Si = 1

nP[A Ç Bi]P[A] = Theorem:

Si = 1

nP[A|Bi]P[Bi]P[A] = Theorem:

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 12

P[B|A] = P[BA]P[A]P[A|B]P[B]

P[A]=

P[A|B] = P[AB]P[B]

P[B|A]Theorem: P[A|B]P[B]P[A]=

www.pr-owl.org/basics/probability.php

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 14

Definition: Event A and B are independent iffP[AB] = P[A]P[B]

P[A|B] = P[AB]P[B]

= P[A]P[B]P[B]

P[B|A] = P[B]

P[A|B] = P[A]

P[A] = 0.3P[A|B] = 0.3

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 15

No matter event B occurs or not, event A is not affected

16

Independent Disjoint

P[AB] ¹ 0 P[AB] = 0

P[AÇB] = P[A]*P[B] P[AÈB] = P[A]+P[B]

BA

BA

Note: Independent = Disjoint iff P[A]=0 or P[B]=0

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 17

• 3 traffic lights, observe a sequence of lights• Mapping real world à (Simple) Model

1 2 3

redorgreen

• The sequence is equally likely• R1 = The first light was red• R2 = The second light was red• G2 = The second light was green• Are Event R2 and G2 independent?• Are Event R1 and R2 independent?

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 18

1 2 3

• 3 traffic lights, observe a sequence of lights

• Sample Space:• S = {rrr, rrg, rgr, rgg, grr, grg, ggr, ggg}

• Are Event R2 and G2 independent?

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 19

rrrrrggrrgrg

rgrrggggrggg

• P[R2]= P[{rrr, rrg, grr, grg}] = 4/8 = ½• P[G2]= P[{rgr, rgg, ggr, ggg}] = ½• P[R2G2] = 0• P[R2]P[G2] = (½)* (½) = ¼àR2 and G2 are not independentàR2 and G2 are disjoint

rrrrrggrrgrg

R2

rgrrggggrggg

G2

1 2 3

• P[R1]= P[{rrr, rrg ,rgr, rgg}] = ½• P[R2]= P[{rrr, rrg ,grr, grg}] = ½• P[R1R2] = P[{rrr ,rrg}] = 2/8 = 1/4• P[R1]P[R2] = (½) * (½) = ¼àR1 and R2 are independentàR1 and R2 are not disjoint

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 20

rrrrrggrrgrg

rgrrggggrggg

rrrrrggrrgrg

R2

R1rrrrrg

rgrrgg

• Are Event R1 and R2 independent?

21

Definition: Event A1,A2 and A3 are independent iff1) A1 and A2 are independent2) A2 and A3 are independent3) A1 and A3 are independent4) P[A1ÇA2ÇA3] = P[A1] P[A2] P[A3]

Definition: Event A and B are independent iffP[AB] = P[A] P[B] Is it sufficient ?

P[ABC] = P[A] P[B] P[C]

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty

NO !

Note:

• Given: P[A] = P[B] = P[C] = 1/5• P[AB] = P[AC] = P[BC] = P[ABC] = 1/25• Independence in pairs (number 1-3) may not

independent

• Only number 4) is insufficient to guarantee the independence

• Ex.: One of the event is Null

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 22

• Assume that the event of separate experiments are independent• Example:• Assume that outcome of a coin toss is independent of

the outcomes of all prior and all subsequent coin tosses• P[H] = P[T] = ½• Toss a coin 3 times• S = • P[HTH] = 1/8• P[HTH] = P[H] P[T] P[H] = 1/2*1/2*1/2 = 1/8

{HHH,HHT,HTH,HTT,THH,THT,TTH,TTT}

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 23

HTH

• Experiment: in sequence sub-experiments à sub-experiments

• Each sub-experiment may depend on the previous one• Represented by a Tree Diagram• Model Conditional Prob. à Sequential Experiment

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 25

Leaf1 Outcome (node)

Outcomes ofthe completeExperiment (Leaf)

BranchProb. value

• Timing coordination of 2 traffic lights• P[the 2nd light is the same color as the 1st light] = 0.7• Assume 1st light is equally likely to be green or red

• Find P[The 2nd light is green] ?• Find P[wait for at least one light] ?

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 26

27

• P[G1] = P[R1] = 0.5

0.5

0.5

0.7

0.3

0.3

0.7

G1

R1

G2

G2

R2

R2

G1G2 = 0.35

G1R2 = 0.15

R1G2 = 0.15

R1R2 = 0.35

• P[G2G1] = P[G2|G1]P[G1] = (0.7)(0.5) = 0.35

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty

28

P[The 2nd light is green] ?

P[G2] = P[G2G1] + P[G2R1] = 0.35 + 0.15 = 0.5

P[G2] = P[G2|G1]P[G1] + P[G2|R1]P[R1]

0.5

0.5

0.7

0.3

0.3

0.7

G1

R1

G2

G2

R2

R2

G1G2 = 0.35

G1R2 = 0.15

R1G2 = 0.15

R1R2 = 0.35

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty

29

P[wait for at least one light] ?

W = {G1R2 È R1G2 È R1R2}

P[W] = P[G1R2] + P[R1G2] + P[R1R2] = 0.15 + 0.15 + 0.35 = 0.65

0.5

0.5

0.7

0.3

0.3

0.7

G1

R1

G2

G2

R2

R2

G1G2 = 0.35

G1R2 = 0.15

R1G2 = 0.15

R1R2 = 0.35

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 30

Japan

W1

L1

Colombia

D10.1

0.4

0.5

Senegal

W2

L2

D2

0.3

0.1

0.6

Poland

W3

L3

D3

W3

L3

D3

W3

L3

D3

0.6

0.1

0.30.6

0.1

0.30.6

0.1

0.3

W2

L2

D2

W2

L2

D2

0.3

0.1

0.6

0.3

0.1

0.6

W1W2W3 = 0.4 * 0.3 * 0.6 = 0.072W1W2D3 = 0.4 * 0.3 * 0.1 = 0.012W1W2L3 = 0.4 * 0.3 * 0.3 = 0.036

32

If experiment A has n possible outcomes,and experiment B has k possible outcomes,

->Then there are nk possible outcomes when you perform both experiments

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty

33

1st draw: select 1 out of 52 -> 52 outcomes2nddraw: select 1 out of 51 (one card has been drawn)

-> 51 outcomes3rddraw: select 1 out of 50 -> 50 outcomes

Total outcomes = (52)(51)(50)

Example: Shuffle a deck and select 3 cards in order.How many outcomes?

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 34

(n-k)!(n-k)!

(n)k = n!(n-k)!

n(n-1)(n-2)…(n-k+1) (n-k)(n-k-1)…(1)(n-k)!

=

(n)k = n(n-1)(n-2)…(n-k+1)= n(n-1)(n-2)…(n-k+1)

Theorem: The number of k-permutations, (n)k , (ordered sequence) of n distinguishable objects is

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 35

Shuriken

Short Sword

Spear of Longinus

Gloves

Astral Spear

Example: You are allowed to choose only three from six items (in ordered sequence).

Death sickle

A

B

C

How many possible outcomes?

(n)k = n!(n-k)!

(6)3 =6!

(6-3)!6 x 5 x 4 x 3!

3!= = 120

Different

36

Theorem: Given n distinguishable objects, There are nk ways to choose with replacement a sample of k objects

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 37

Example: You are allowed to choose only three from six items (with replacement).

Shuriken

Short Sword

Spear of Longinus

Gloves

Astral Spear

Death sickle

B

How many possible outcomes?

A

C

nk = 6 x 6 x 6 = 216

38

Theorem: The number of ways to choose k objects out of n distinguishable objects is

( )nk =

(n)kk!

n!k!(n-k)!=

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty

39

2 subexperiments: ( ) then (k)k

( ). (k)k = (n)k

nknk

63( ) = 20

No order

(3)3 = 6

Order(6)3 = 120Order

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty

Shuriken

Short Sword

Spear of Longinus

Gloves

Astral Spear

Death sickle

1 2 3

40

• Perform repeated trials• p = a success probability• (1-p) = a failure probability• Each trial is independent• Sk,n = the event that k successes in n trials

( )nk

P[Sk,n] = pk(1-p)n-k

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty

• 3 trials with 2 successes• 000 001 010 011 100 101 110 111• How many way to choose 2 out of 3

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 41

• What is the probability of success for each way ?• p2 * (1-p)

( )32

P[S2,3] = p2(1-p)3-2

( )nk= = = 3( )3

2

• Example: In the first round of a programming contest, probability that a program will pass the test is 0.8 .

• From 10 candidates, what is the probability that x candidates will pass?

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 42

10x( )P[Ax,10] = (0.8)x(1-0.8)10-x

P[A8,10] = (45)(0.1678)(0.04) = 0.3

Solution:A = {program pass the test}, P[A] = 0.8Testing a program is an independent trial

And what is P[x = 8]?

43

Let probability that a computer works = pSeries: P[A] = P[A1A2] = p2

Parallel: P[B] = ?

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty

P[B] = 1 – P[Bc] = 1 – P[B1

cB2c]

= 1 – (1 – p)2

ParallelSeries

• Probability meaning• Sample space, Event, Outcome• Set Theory• Probability measurement• Conditional Probability• Independence• Sequential experiments -> tree diagram• Counting Methods• Independent Trials

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 44