ReviewCh2-Basic Concept Probability

8/3/2019 ReviewCh2-Basic Concept Probability

1/92

6/7/2011

1

241-460 Introduction to Queueing

Netw orks : Engineering Approach

Probability TheoryProbability Theory

Assoc. Prof. Thossaporn Kamolphiwong

Centre for Network Research (CNR)

Department of Computer Engineering, Faculty of Engineering

r nce o ong a n vers y, a anEmail : [email protected]

Outline

Random Experiments

Type of sample space

Event Set Theory

Chapter 2 : Basic Concepts of Probability Theory

2


2/92

6/7/2011

2

Specifying Random Experiments

A random experiment is an experiment in

unpredictable fashion when theexperiment is repeated under the samecondition


3

Example of Random Experiments

Experiment E1 : Toss a coin three times andnote the sequence of heads and tails

Experiment E2 : Toss a coin three times andnote the number of heads

Experiment E3 : Count the number of voicepackets containing only silence procedure from agroup of N speakers in a 10-ms period

xper men : oc o n orma on stransmitted repeatedly over a noisy channel untilan error-free block arrives at the receiver. Countthe number of transmissions required.


4


3/92

6/7/2011

3

(Continue)

Experiment E5 : Measure the time between twomessa e arrives at a messa e center

Experiment E6 : Determine the value of avoltage waveform at time t1 and t2

Experiment E7 : Pick a number at randombetween zero and one.

between zero and one, then pick a number Yatrandom between zero andX


5

(Continue)

Note:

xper men an xper men are esame procedure but they are different

observations different Experiments


6


4/92

6/7/2011

4

Set and P robability

Outcome or sample pointany possible observation is an element or point in Sample Space

Sample SpaceS

The set of all possible outcomes

Set notation, tables, diagrams, intervals of the

rea ne, reg ons o e p aneFinite, Countably infinite, Uncountably infinite


7

Sample Space

Sample spaces corresponding to theex erim ent

S1 = {HHH, HHT, HTH, THH, TTH, THT, HTT, TTT}S2 = {0, 1, 2, 3}

S3 = {0, 1, 2, ,N}

S4 = {0, 1, 2, }

S5 = {t: t> 0} = [0, )

S6 = {(v1, v2) : - < v1 < and- < v2 < }


8


5/92

6/7/2011

5

(Continue)

S7 = {x : 0


6/92

6/7/2011

6

Event

Event

Event Sample Space


11

Event examples

Event examples

E1 : The three tosses gives the same outcome

A1 = {HHH, TTT}E2 : The three tosses gives # of head equals #

of tails

A2 = {}=

E8 : The two numbers differ by less than 1/10

A8 = {(x,y) : (x,y) S8 and |xy| < 1/10}


12


7/92

6/7/2011

7

Example

Flip four coins, 0.5 baht(red), 1 baht(yellow), 5baht white and 10 baht silver .

Examine the coins in order (0.5 baht, 1 baht, 5baht, 10 baht) and observe whether each coinshows a head (h) or a tail (t).

What is the sample space?

LetBi = {outcomes with i heads}


13

Solution

Outcome

r y w s , r y w s , r y w s , r y w s , r y w s ,

(hrtyhwts), (hrtytwhs), (hrtytwts), (trhyhwhs), (trhyhwts),

(trhytwhs), (trhytwts), (trtyhwhs), (trtyhwts), (trtytwhs),

(trtytwts)


14


8/92

6/7/2011

8

(Continue)

There are 16 members of the sample space

S = { (hrhyhwhs), (hrhyhwts), (hrhytwhs), (hrhytwts),

(hrtyhwhs), (hrtyhwts), (hrtytwhs), (hrtytwts),

(trhyhwhs), (trhyhwts), (trhytwhs), (trhytwts),

(trtyhwhs), (trtyhwts), (trtytwhs), (trtytwts) }


15

(Continue)

EventB0 = { (trtytwts) }

=1 r y w s , r y w s , r y w s , r y w s

EventB2 = { (hrhytwts), (hrtyhwts), (hrtytwhs), (trtyhwhs),

(trhytwhs), (trhytwhs)} EventB3 = {(hrhyhwts),(hrhytwhs),(hrtyhwhs), (trhyhwhs)

}

EventB4 = { (hrhyhwhs) }

Event B = {B0,B1,B2, B3, B4} is Event space


16


9/92

6/7/2011

9

Event Space

Event Space

Collective exhaustive

S = {1,2,3,4,5,6}

A = {2,4,6}

= , , Mutually exclusive


17

Applying Set Theory to P robability

Probability is a number that describes a

Set Algebra Probability

Set Event

Universal set Sample space

Element Outcome


18


10/92

6/7/2011

10

Set Theory

The mathematical basis of probability is the theoryof sets

Set is a collection of things

Things that together make up the set areelement

= x x = , , , ,= {1, 4, 9, 16, 25}


19

Set Operation

Union

A B : the set of outcomes either inA or inB orboth

Intersection

A B : the set of outcomes in bothA andB IfA B = , thenA andB are mutually exclusive

Complement

A = The set of all outcomes not inA Difference

A - B contains all elements of A that are not elementsofB


20


11/92

6/7/2011

11

A

Venn diagram: Set Operation

A BSS A BA

B

SA B = S


BA21

(Continue)

SA -BS

A BA B

A


22


12/92

6/7/2011

12

Mutually exclusive

Mutually exclusive AS

Ai Aj = for i jA B = Disjoint

B

A8SA5

A1 A2 An = SChapter 2 : Basic Concepts of Probability Theory

A1A7A6

A2

A4

A1

A3

23

Properties of set operation

Commutative Properties:

A B = B A and A B = B A

Associative Properties:

A (B C) = (A B) C and

A (B C) = (A B) C

Distributive Properties:

=

A (B C) = (A B) (A C)

De Morgans law:

(A B) = A B and(A B) = A B


24


13/92

6/7/2011

13

(Continue)

Union and intersection operation can bere eated

n

n

k

k AAAAA 3211

n

n

k AAAAA 321


25

Ax ioms of Probability

Axiom 1 : For any event A, P[A] 0

Axiom 2 : P[S] = 1

Axiom 3 :If A B = , then P[A B] = P[A] + P[B]

Axiom 4 :If A1, A2, is a sequence of events such that

Ai Aj, = for all i j, then


...2111

APAPAPAPk

k

k

k26


14/92

6/7/2011

14

Some Consequences of The

Axioms The probability measure P[] satisfies

=

Corollary 2 : P[A] < 1

Corollary 3 : P[] = 0

Corollary 4 : IfA1,A2, ,An are pairwise mutually

exclusive, then


27

(Continue)

Corollary 5 : P[A B] = P[A] + P[B] P[A B]

A BSB

A


28


15/92

6/7/2011

15

(Continue)

Corollary 6 :


29

(Continue)

Corollary 7 : IfA B, then P[A] < P[B]

A BA

B

S


30


16/92

6/7/2011

16

Example

A company has a model of telephone usage. Itclassifies all calls as either long (L), if they last more

, .whether calls carry voice (V), data (D) or fax (F).This model implies an experiment in which theprocedure is to monitor a call and the observationconsists of the type of call, V, D, or F, and thelength,L or B. The corresponding table entry is theprobability of the outcome. The table is

V D F

L 0.3 0.12 0.15B 0.2 0.08 0.15


31

Solution

V D F

B 0.2 0.08 0.15

From the table we can read that the probability of

a brief data call is

P[BD] = 0.08.

The probability of a long call is

P[L] = P[LV] + P[LD] + P[LF] = 0.57


32


17/92

6/7/2011

17

Types of Sample Space

Discrete Sample Space


33

Discrete Sample Space

Finite sample space : S = {a1, a2, , an }

All distinct events are mutually exclusive

Event ''2'1 ,...,, maaaA

'''


''2'121

...

,...,,

m

m

aPaPaP

aaaPAP

34


18/92

6/7/2011

18

(Continue)

Countable infinite, S = {b1, b2, }

All distinct events are mutually exclusive

Event

''

,..., '2'1 bbB


...21 bPbPBP

35

(Continue)

Sample space has n elements,

S = {a1, a2, , an }

Probability assignment is equally likelyoutcomes


naPaPaP n

1}][{}][{}][{ 21

36


19/92

6/7/2011

19

Example

Selecting a ball from urn containing 10 identicalballs numbered 0 1 9.

A = number of ball selected is odd

B = number of ball selected is a multiple of 3

C = number of ball selected is less than 5

Find

P[A B C]


37

Solution

S = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}

= , , , ,

B = {3, 6, 9}

C= {0, 1, 2, 3, 4} Assume that outcome are equally likely

5}]9[{}]7[{}]5[{}]3[{}]1[{][ PPPPPAP


10

3}]9[{}]6[{}]3[{][ PPPBP

10

10

5}]4[{}3[{}2[{}]1[{}]0[{][ PPPPPCP

38


20/92

6/7/2011

20

(Continue)

2

9,3][ PBAP 2

3,1][ PCAP

10

13][ PCBP

10

13][ PCBAP

][][][][ BAPBPAPBAP


][][][

][][][][][

CBAPCBPCAP

BAPCPBPAPCBAP

39

(Continue)

6235][][][][ BAPBPAPBAP

91122535

][][][

][][][][][

CBAPCBPCAP

BAPCPBPAPCBAP


1010101010101010

40


21/92


22/92

6/7/2011

22

(Continue)

Relative frequency

= = j =j , , ,

P[j toss till first head] = (1/2)j j = 1, 2, 3,

Heads Tails

n = 100 trials

N 50

50

25

100/8


N2 25

N3 100/8

N4 100/1643

Continuous Sample Spaces

Outcomes are numbers.

line, rectangular regions in the plane


44


23/92

6/7/2011

23

Example

Pick two numberx andy at random between

zero and one.

Find the probability of the following events:

A = {x > 0.5},

B = {y > 0.5}

C= {x >y}


45

Solution

y

1

y

1

Sx

10x

10

x>1/2

(a) Sample space (b)Event(x > )

P A = 1 2

y y


(c)Event(y > ) (d)Event(x >y)1

y > 1/2

x0

P[B] = 1/2

x10

x >yP[C] = 1/2

46


24/92

6/7/2011

24

Example

Suppose that the lifetime of a computer memorychi is measured and we find that the

proportion of chips whose lifetime exceeds tdecreases exponentially at a rate .

Find the probability of arbitrary intervals in S.


47

Solution

Probability that chips lifetime exceeds tdecreases

e-t

,0for, tetP t

S = (0,)1

0.5

Lifetime


t time

P S = P 0, = 1

48


25/92

6/7/2011

25

(Continue)

LetA : event of chips lifetime in interval (r,s)

P[(r, )] = P[(r,s]] + P[(s, )]

r s time( ](

P[A] = P[(r,s]] = P[(r, )] - P[(s, )] = e-r e-s


49

Computing probabilities usingCounting M ethods

The outcome of finite sample can be assumed tobe e ui robable

Calculation of probabilities reduces to countingthe number of outcome in an event


50


26/92

6/7/2011

26

Basic principle of Counting rules

Addition rule


51

Addition Rule

Addition Rule

1 2

Let eventEdescribe the situation where either

eventE1 or eventE2 will occur. The number of times event Ewill occur can be

given by the expression:

n(E) =n(E1) +n(E2)


52


27/92

6/7/2011

27

Example

Set of numbers S = {1, 2, 3, 4, 5, 6, 7, 8, 9}

1

E2 = choosing an even number from S.

Find n(E) when

E= choosing an odd or an even number from S;

n(E) = n(E1) + n(E2) = 5 + 4


53

Multiplication Rule

Multiplication Rule

If one event can occur in m ways and another

event can occur in n, independently of eachother, then there are mn ways in which bothevents can occur.


54


28/92

6/7/2011

28

Example

Multiple-choice test has 2 questionsst ,

answers and 2nd question the student select oneof 5 possible answers.

What is the total number of ways of answering

Total number of ways =4x5


55

(Continue)

Multiple-choice test has 2 questions

ipossible answers.

What is the total number of ways of answeringthe entire test?


56


29/92

6/7/2011

29

Solution

xi : question number i

i

b1

x1a1 a2 1n

a

(a1,b1) (a2,b1)

(a ,b ) (a ,b )

1,1 ban

,bab

Total number of

ways of answering

the test = n1n2


2,1 nba

x2

2nb

21, nn ba

1

57

Counting Method

The number of distinct ordered k-tuples (x1,,xk)

i i

element is

Number of distinct ordered k-tuples = n1n2nk


58


30/92

6/7/2011

30

Example

There are two subexperiments. The first .

outcomes,Hand T. The second

subexperiment is Roll a die. It has sixoutcomes, 1, 2, , 6. The experiment,

Flip a coin and roll a die, has 2x6 = 12

ou comes:(H,1), (H,2), (H,3), (H,4), (H,5), (H,6)

(T,1), (T,2), (T,3), (T,4), (T,5), (T,6)


59

Sampling

Sampling

Without Replacement

With Ordering Without Ordering

With replacement Without Rep lacement

RR -

RB RB

BB -

BR BR Chapter 2 : Basic Concepts of Probability Theory

60


31/92

6/7/2011

31

Sampling w ith Replacement and

w ith OrderingChoose kobject from a setA that has n

distinct ob ects with re lacement andordering

The experiment produces an ordered k-tuple,

(x1,,xk) wherexi A and i = 1,, k


Number of distinct ordered k-tuples = nk

61

Example

An urn contains five ball number 1 to 5

How many distinct ordered pairs are possible? What is the probability that the two draws yield

the same number?


62


32/92

6/7/2011

32

Solution

Ordered pairs for sampling with replacement

, , , , ,

(2,1) (2,2) (2,3) (2,4) (2,5)

(3,1) (3,2) (3,3) (3,4) (3,5)

(4,1) (4,2) (4,3) (4,4) (4,5)

(5,1) (5,2) (5,3) (5,4) (5,5)

(a) Total distinct ordered pairs are 52 = 25

(b) Probability that two draws yield the samenumber is 5/25 = 0.2


63

Example

A laptop computer has PCMCIA expansion cardslots A and B. Each slot can be filled with eithera modem card (m), a SCSI interface (i), or aGPS card (g). From the set {m, i, g} of possible

cards, what is the set of possible ways to fill thetwo slots when we sample with replacement?Inother words, What is the probability that bothslots hold the same type of card?


64


33/92

6/7/2011

33

Solution

Letxy : the outcome that card type x is used inslotA and card t e is used in slotB. The

possible outcomes are

S = {mm, mi, mg, im, ii, ig, gm, gi, gg}

e num er o poss e ou comes s n ne

The probability that both slots hold the same typeof card is 3/9


65

Sampling w ithout Replacement andw ith Ordering

Choose kobjects in succession withoutreplacement formA ofn distinct object. k< n.

The number of possible outcomes in the first draw

is n1 = n; the number of possible outcomes inthe second draw is n2 = n -1, all n object exceptthe one selected in the first draw; and so on, up

k -


Number of distinct ordered k-tuples

= n(n-1)(n k +1)66


34/92

6/7/2011

34

Example

A college club has 25 members and is electingfour officers res v secretar treasurer

How many ways can the officers be filled?

Order matters Bill as pres and Bob as vice-presis different from Bob as pres and Bill as vice-pres

,= 25!/21!

25 possibilities for the first officer, 24 for thesecond, 23 for the third, 22 for the fourth


67

Permutations

Sampling without replacement with k< n

Use the multiplication rule

Number of ways

!nP n

is called the number of permutation ofk

objects out ofnChapter 2 : Basic Concepts of Probability Theory

!kn

n

kP68


35/92

6/7/2011

35

Sampling w ithout Replacement and

w ithout Ordering

Pickkobject from a set ofn distinct objects .

The number of ways to choose kobjects out ofn

distinguishable objects is

= n n-1 n k +1 /k!


69

Example

A college club has 25 members and is sendingfour students to meetin

How many ways can the students be chosen?

Order does not matter sending Bill and Bob isthe same as sending Bob and Bill

# ways = 25x24x23x22/(4x3x2x1)

= , = x

Permutations divided by 4!, the number of waysto mix around the four chosen


70


36/92

6/7/2011

36

Combinations

Sampling without replacement with k< n

Use the multiplication rule

Number of ways

!!!

knk

n

k

nCnk

is call the number ofcombinations ofk

objects out ofn distinguishable objects, and is

also called the binomial coefficientChapter 2 : Basic Concepts of Probability Theory

n

kC

71

When P and When C

Permutations

Change the order its a different outcome

CombinationsOrder does not matter

Change the order its the same outcome


72


37/92

6/7/2011

37

Permutations and Combination

A batch of 50 items contains 10 defective items.Su ose 12 items are selected at random andtested.

How many ways can the items be chosen sothat exactly 5 of the items tested are defective?

A more complicated permutation / combinationproblem


73

Solution

5 defective items from the batch of 10

7 nondefective items from the batch of 40

25212345/678910105 C

34353637383940407

C


560,643,18

74


38/92

6/7/2011

38

Permutations and Combination

# of ways to selecting 5 defective and 7

nondefective items from the batch of 50 =

252x18,643,560 = 4,698,177,120

# of ways each 12 items can be ordered = 12!

nondefective items with ordering from the batch

of 50 = 4,698,177,120 x12!


75

Partitioning

Combinations how to choose k out of n

The ones that are chosen

The ones that are not chosen

Two results from this observation

#1 : the number of ways of partitioning n n -

#2 :


n

kC

n

kn

n

k CC 76


39/92

6/7/2011

39

Multinom ial coefficient

How about partitioning n items into more than

two rou s?

Partition n items into m groups containing k1 ,k2, , km objects (where k1 + k2 + + km = n)

This is a more general version of thecombinations problem

This is called themultinomial coefficientChapter 2 : Basic Concepts of Probability Theory

!!...!!!...!

ways#2121 mm

kkkn

kkkn

77

Example

Number of distinctB1

n = 9

B3

B2

223

56789!2!3!4

!9

2,3,4

9


k1 = 4 k3 = 2k2 = 3

m = 3

1260

78


40/92

6/7/2011

40

Example

A six-sided die is tossed 12 times.

from the set {1, 2, 3, 4, 5, 6}) have eachnumber appearing exactly twice?

400,484,72

!12

!2!2!2!2!2!2

!126

a s e pro a y o o a n ng suc asequence?


3

12

6

104.3336,782,176,2

400,484,7

6

2!12 79

Example

For five subexperiments with sample, ,

sequences are there in which 0 appears 2

times and 1 appears 3 times?


80


41/92

6/7/2011

41

Solution

The set of five-letter words with 0 appearing twiceand 1 a earin three times is

{00111, 01011, 01101, 01110, 10011, 10101, 10110,

11001, 11010, 11100}

ere are exac y suc wor s

Multinomial coefficient


10!3!2

!5

3,2

5

81

Sampling w ith Replacement andw ithout Ordering

Pickkobjects from set ofn distinct with

Number of different ways of picking kobjectsfrom a set ofn distinct objects with


1

11

n

kn

k

kn

82


42/92

6/7/2011

42

Example

Consider the following program segment, wherea b c are inte er variables

for a = 1 to 20 do

for b = 1 to a do

for c = 1 to b do

print(a *b + c)

How many times is the print statement executein this program segment?


83

(Continue)

The selections ofa, b, and c where the print

statement is executed satisfies the condition 1


43/92

6/7/2011

43

Conditional Probability

Interested in determining whether two events,

A andB are related in sense that knowledgeB,

alters the likelihood of occurrence ofA

P A|B : Probabilit of A iven B


85

(Continue)

Conditional Probability

the occurrence of the event B is

S

BPBAP

BAP

|

,P[B] > 0


AA B

86


44/92

6/7/2011

44

(Continue)

Probability as Relative frequency

B

BA

n

n

BAP

Beventintimeofnumber

BAeventintimeofnumber|


nn

nn

BAPB

BA

|

87

Correlated Events

some definitions describing the conditionalrobabilities of correlated events.

P[AB] orP[AB] : Joint Probability The probability that both events A andB occur

P[A|B] : Condition Probability

The probability of event A, given that some other

eventB has occurred

P[B] : Probability of eventB


88


45/92

6/7/2011

45

(Continue)

BAPBAP

BP


P[A B] = P[A|B]P[B]= P[AB]

P[B A] = P[B|A]P[A]= P[BA]=

89

Example

Communication systems can be modeled :User inputs a 0 or a 1 into system,

Receiver makes a decision about what was the inputto the system, based on the signal it received.

Suppose thatUser sends 0s with probability 1-p and 1s with

probabilityp,

Receiver makes random decision errors withprobability .


R0

R1

1-

1-

T01-p

T1

p

90


46/92

6/7/2011

46

(Continue)

R01-

T0

1-p

Given P(R0|T0) = (1- ), P(R1|T0) = ,

P(R1|T1) = (1- ), P(R0|T1) =

R11-

T1

p

Find P(R0)

P(R0) = P(R0/T0)P(T0) + P(R0/T1)P(T1)

= (1- )(1-p) + p = 1 - -p + 2p


91

(Continue)

For i = 0, 1, let

i ,

Rj be the event receiver decision wasj.

Find probabilities P[Ti Rj] for i =0,1 andj = 0,1.

0 0 =

P[T0 R1] = ?

P[T1 R0] = ?

P[T1 R1] = ?


92


47/92

6/7/2011

47

10

Solution

Input into binary channelTransmitter

0 1 0 1Output form binary channel

Receiver

1-

- p

1-

(1-p)(1-) (1-p) p p(1-)

P[T0 R0] = (1 - )(1 p)


P[T0 R1] = (1 p)

P[T1 R0] = p P[T1 R1] = (1 - )p93

Example

Ball is selected from urn containing

,

Two white balls, numbered 3 and 4

EventsA : black ball selected

B : even-numbered ball selected

: num er o a s grea er an

Find P[A|B] and P[A|C]


94


48/92

6/7/2011

48

Solution

S = {(1,b), (2,b) , (3,w) , (4,w)}

A = {(1,b), (2,b)}

B = {(2,b), (4,w)}

C = {(3,w), (4,w)}

5.05.0

25.0

BP

BAPBAP

00 CAPCAP P[AB] = P[(2,b)]

P[AC] = P[] = 0


.

95

Example

An urn contains

three white ball

Two balls are selected at random withoutreplacement and the sequence of colors in

.

Find the probability that both balls are black.


96


49/92

6/7/2011

49

Solution

0

W1

5

2

B1

B2 W2

5

3

4

1

4

34

2

4

2

u come o rs raw

B2 W2

1 2

Outcome of second draw


101

103

10

3

10

1

5

2

4

121 BBP

103

212 | BPBBP97

Total Probability

LetB1, B2, , Bn be mutually exclusive event

= 1 2 n B1, B2, , Bn form a partition of S


98


50/92

6/7/2011

50

(Continue)

SB1

B3 Bn-1

A = A S =A (B1 B2 Bn)

= (A B1) (A B2) (A Bn)

A

B2Bn

P[A] = P[A B1] + P[A B2] + + P[A Bn]

= P[A|B1]P[B1]+P[A|B2]P[B2]+ + P[A|Bn]P[Bn]


99

Example

An urn contains

three white ball

Two balls are selected at random withoutreplacement and the sequence of colors in

.

Find the probability of the event that the

second ball is white.


100


51/92

6/7/2011

51

Solution

B1

0

W1

1B2 W2 B2 W2

5 5

4

1

4

34

2

4

2

10

1

10

3

10

3

2

10

3

P[W2] = P[W2|B1]P[B1] + P[W2|W1]P[W1]


5

3

10

3

10

3

5

3

2

1

5

2

4

3

101

Example

A manufacturing process produces a mix of ood memor chi s and bad memor chi s.The lifetime of good chips follows theexponential law with rate of failure . The

lifetime of bad chips also follows the exponentiallaw, but the rate of failure is 1000. Suppose

that the fraction of good chips is 1 p and of

bad chips,p.

Find the probability that a randomly selected

chip is still functioning after tsecond.


102


52/92

6/7/2011

52

(Continue)

Let1Lifetime

B : Chip is bad

C: Chip still

functioning aftert

second

e-1000t

0

e-t

2 4 6 8 10

0.2

0.4

0.6

.

timet


BPBCPGPGCPCP ||

pBCPpGCP |1| tt peep 10001 103

Example

Three machines B1, B2, and B3 for making 1 kresistors.

B1 produces 80% of resistors within 50 of thenominal value.


Percentage for machineB3 is 60%.

, 1 , 2produces 4000 resistors, and B3 produces 3000resistors.


104


53/92

6/7/2011

53

Solution

What is the probability that the company ships aresistor that is within50 of the nominalvalues?


105

(Continue)

LetA = {resistor is within 50 of the nominalvalue


P[A|B1] = 0.8,


P[A|B2] = 0.9,

Percentage for machineB3 is 60%.

P[A|B3] = 0.6


106


54/92

6/7/2011

54

(Continue)

Each hour,B1 produces 3000 resistors,B2 produces4000 resistors, andB3 produces 3000 resistors.

Total product = 3000 + 4000 + 3000 = 10,000

P[B1] = 3000/10,000 = 0.3,

P[B2] = 0.4,

3 = .


107

(Continue)

A = {resistor is within 50 of the nominal value}

S

B1

B3 B2

A

P[A] = P[A|B1]P[B1] + P[A|B2]P[B2] + P[A|B3]P[B3]

= (0.8)(0.3) + (0.9)(0.4) + (0.6)(0.3) = 0.78


108


55/92

6/7/2011

55

Bayes Rule

Situations

Advance in formation aboutP[A|B]


Need to calculateP[B|A]

109

(Continue)

LetB1, B2, , Bn be a partition of a sample space S.

prob.condition,|

AP

ABPABP

BAPABP


APBPBAP

ABP|

|

110


56/92

6/7/2011

56

Example of Bayes Rule

Communication system

0.92R0

R1

0.95

T00.45

T1

0.55

Given : P(R0|T0)

Want to know : P(T0|R0)


111

Example: Communication System

Communication system

R0

R1

.

0.95

T00.45

T1

0.55

Given P(R0|T0) = 0.92;


P(R1|T1) = 0.95

P(T0) = 0.45; P(T1) = 0.55112


57/92

6/7/2011

57

(Continue)

Given P R0 T0 = 0.92

P(R1|T1) = 0.95

P(T0) = 0.45

P(T1) = 0.55

Find P(R0), P(R1), P(T1|R1), P(T0|R0), P(Error)

Find which input is more probable given that thereceiver has output a 1


113

Solution

P(R0) = P(R0/T0)P(T0) + P(R0/T1)P(T1)

= 0.92x0.45 + 0.05x0.55 = 0.4415

P(R1) = P(R1/T1)P(T1) + P(R1/T0)P(T0)

= 0.95x0.55 + 0.08x0.45 = 0.5585P(R1) = 1 P(R0)

rror = 0 1 1 + 1 0 0= 0.05x0.55 + 0.08x0.45 = 0.0635


114


58/92

6/7/2011

58

(Continue)

P(T1/R1) = P(R1/T1)P(T1)/P(R1)= 0.95x0.55/0.5585

= 0.9355

P(T0/R0) = P(R0/T0)P(T0)/P(R0)= 0.98x0.45/0.4415

= 0.9988


115

(Continue)

P[T0 |R1] = P[T0R1]/P[R1] = [R1|T0]P[T0]/P[R1]

T1 is more probable given that

the receiver has output a 1

= 0.08x0.45/0.5585 = 0.0645

P[T1 |R1] = P[T1R1]/P[R1] = P[R1|T1]P[T1]/P[R1]

= 0.95x0.55x/0.5585 = 0.9355


116


59/92

6/7/2011

59

Independence of Events

Two Independent Events

][][

][][

][

][]|[ AP

BP

BPAP

BP

BAPBAP

If P[A B] = P[A]P[B] thenA andB are independent.

Since


Implies that P[A |B] = P[A],

P[B |A] = P[B]

Implies that P[A] 0 and P[B] 0117

Independent & Disjoint

Two eventsA andBP[A] 0, P[B] 0, P[A B] = 0A andB cannot be independent.

AS

A

S

Independent Disjoint


B

P[AB] 0P[AB] = P[A]P[B] P[AB] = 0118


60/92

6/7/2011

60

Example

A short-circuit tester has a red light (r) to indicate

to indicate that there is no short circuit. Consider

an experiment consisting of a sequence of three

tests. Suppose that for the three lights, each

outcome (a sequence of three lights, each either

. 2that the second light was red and G2 that the

second light was green independent?Are the

eventsR1 andR2 independent?


119

(Continue)

A short-circuit tester

green light (g) : no short circuit.

Consider a sequence of three tests.

The sequence of three lights is equally likely.

Are R2 and G2 independent?

Are R1 andR2 independent?


120


61/92

6/7/2011

61

Solution

Circuit 3Circuit 2Circuit 1

R1 = the first circuit is red

R2 = the second circuit is red

G2 = the second circuit is red


Outcomerrr rrg rgr rgg grr grg ggr ggg121

(Continue)


, , , , , , ,

P[R2] = 4/8 = 1/2

P[G2] = 4/8 =1/2

srrr rgr

R2 G2

R2 G2 =

P[R2G2] = 0


grr

grgggr

ggg

122


62/92

6/7/2011

62

(Continue)


P[R2]P[G2] = ()() =

P[R2G2] = 0

P[R2G2] P[R2]P[G2]

R2 and G2 are not independent

R2 and G2 are disjoint


123

(Continue)

Are R1 and R2 independent?

, , , , , , ,

P[R1] = 4/8 = 1/2

P[R2] = 4/8 =1/2

srrr

rrg

rgr

rgg

R1

R1 R2 = {rrr, rrg}

P[R1R2] = 2/8 = 1/4


grr

grg

ggr

ggg

R2124


63/92

6/7/2011

63

Solution

Are R1 and R2 independent?

P[R1]P[R2] = ()() =

P[R1R2] =

P[R1R2] = P[R1]P[R2]

R1 andR2 are independent


125

Example

Two numberx andy are selected at random

between zero and one

A = {x > 0.5}B = {y > 0.5}

C= x >

AreA andB independent?

AreA and Cindependent?


126


64/92

6/7/2011

64

Solution

y

1

y

1B

A

1x

0 0x

1

P[A B] = P[A] =

P[B] =

P[A C] = + 1/8 = 3/8

P[A] =

P[C] =

AC


=P[A B] = P[A]P[B ] =

Event A and B are

independent

=P[A B] P[A]P[B ]

Event A and C are not

independent127

3 Independent Events

Event A, B and C are independent if the

triplet of events is equal to the product of the

probabilities of the individual events

P A B C = P A P B P C


128


65/92

6/7/2011

65

Example

Two numberx andy are selected at random

between zero and one

A = {x > 0.5}

B = {y > 0.5}

C= x >

AreA andB and Cindependent?


129

Solution

y

1A = {x > 0.5}

P[A ] =

P[B ] = P[A B C] = P[A]P[B]P[C]

A

1 x0C

B = {y > 0.5}

C= {x >y}


P C =

P[A B C] = 1/8P[A]P[B]P[C] = ()()()

= 1/8

Event A and B and

C are independent

130


66/92

6/7/2011

66

Example

LetB = {y > }

=

F= {x < andy < } {x > andy > }

AreB,D and Fare independent?


131

Solution

y

1

y

1F

y

1B

0x

1

D

D = {x < }

0 x1

F

F= {x}

1x

0

B = {y > }

=

P[D] =

P[F] =


132


67/92

6/7/2011

67

Solution

B = {y > }

y

1

P[B] = P[B D F] P[B]P[D]P[F]

D = {x < }

F= {x}

1x

0

DF

P[F] =

P[B D F] = 0

P[B]P[D]P[F] = ()()() =1/8


ree even s are no n epen en

133

Sequential Experiments

Sequential experiment consist of a

Sequences of Independent Experiment

Sequences of Dependent Experiment


134


68/92

6/7/2011

68

Sequences of Independent

Experiments

Experiment subexperiments subexperiments

A1,A2,,An are subexperiments

IfA1,A2,,An are independent

Thus

P[A1 A2 An] = P[A1]P[A2] P[An]


135

Example

Suppose that 10 numbers are selected atrandom from the interval 0 1 . Find theprobability that the first 5 numbers are less than and the last 5 numbers are greater than .


136


69/92

6/7/2011

69

Solution

Let x1, x2, , x10 : the sequence of 10 numbers

Ak= {xk< } for k= 1,,5

Ak= {xk> } for k= 6,,10

0 1

P[A1 A2 A10] = P[A1]P[A2] P[A10]

= ()5()5


137

Bernoulli trial

Bernoulli trial

two possible outcomes, "success with probability

p and "failure with probability q = 1 -p.

In practice it refers to a single experiment whichcan have one of two possible outcomes.

Toss a coin

Head (success)

Tail (fail)


138


70/92

6/7/2011

70

Binomial Experiment

Abinomial experiment is one that possessesthe followin ro erties:

The experiment consists ofn repeated trials;

Each trial results in an outcome that may beclassified as a successor a failure(hence thename binomial

The probability of a success, denoted byp,

remains constant from trial to trial and repeatedtrials are independent.


139

Binomial P robability Law

k : the number of successes inn independent

The probabilities of k successes in n independent

repetitions is :

knkn pp

nkP

1


n = the number of trials

k= 0, 1, 2, ... np = the probability of success in a single trial140


71/92

6/7/2011

71

(Continue)

nn

n ppk

kP

)1()(

)!(!

!

knk

n

k

n

ro a ty o ksuccessesin n trials

Binomial coefficient


n

kn Ck

nkN

)( Picking kpositions out ofn

for the success141

Example

Coin is tossed three time

heads isp

Let k: the number of head in three trials

Find the probability of k success in head.


142


72/92

6/7/2011

72

Solution

Sequence of heads and tail is

P HHH = P H P H P H = 3

P[{HHT}] = P[{H}]P[{H}]P[{T}] =p2(1-p)

P[{HTH}] = P[{H}]P[{T}]P[{H}] =p2(1-p)

P[{THH}] = P[{T}]P[{H}]P[{H}] =p2(1-p)

P[{TTH}] = P[{T}]P[{T}]P[{H}] =p (1-p)2

P[{THT}] = P[{T}]P[{H}]P[{T}] =p (1-p)2

P[{HTT}] = P[{H}]P[{T}]P[{T}] =p (1-p)2

P[{TTT}] = P[{T}]P[{T}]P[{T}] = (1-p)3


143

Solution

P3(0) = P[k= 0] = P[{TTT}] = (1 p)3

P3(1) = P[k= 1] = P[{TTH, THT, HTT}] = 3p(1 p)2

3303 110

30 pppP


13 1311

1 ppppP

144


73/92

6/7/2011

73

(Continue)

P3(2) = P[k= 2] = P[{TTH, THT, HTT}] = 3p2(1 p)

P3(3) = P[k= 3] = P[{HHH}] =p3

ppppP

131

2

32

212

3


3033 13

33 pppP

145

Example

Communication system transmits binaryinformation over channel that introduced

random bit errors with probability = 10-3

Transmitter transmits each information bit threetimes

Receiver take majority vote of received bit todecide on that the transmitted bit was

Find the probability that the receiver will makean incorrect decision


146


74/92

6/7/2011

74

Solution

Let k: number of errors that the receiver will

6

0312999.0001.0

3

3999.0001.0

2

32

kP


147

Example

To communicate one bit of information reliably,

symbol five times.

zero is transmitted as 00000

one is 11111.

The receiver detects the correct information ifthree or more binar s mbols are receivedcorrectly.

What is the information error probability P[E], ifthe binary symbol error probability is q = 0.1?


148


75/92

6/7/2011

75

Solution

k: # of errors that the receiver will make correct

The probability of a successes isp

p = 1- q = 0.9

The information error probability P[E]


00856.02

5

1

5

0

53245

555

qppqq

149

Multinomial Experiment

Amultinomial experiment has the followingro erties:

The experiment consists ofnrepeated trials.

Each trial has a discrete number of possibleoutcomes.

On any given trial, the probability that aparticular outcome will occur is constant.

The trials are independent; that is, the outcomeon one trial does not affect the outcome on othertrials.


150


76/92

6/7/2011

76

Example

Toss two dice three times, and record the outcomeon each toss. This is a multinomial ex erimentbecause:

The experiment consists of repeated trials. We tossthe dice three times.

Each trial can result in a discrete number of outcomes- 2 through 12.

T e pro a i ity o any outcome is constant; it oesnot change from one toss to the next.

The trials are independent; that is, getting aparticular outcome on one trial does not affect theoutcome on other trials.


151

Multinomial Probability Law

A multinomial experiment consists ofn trials, and eachtrial can result in any ofkpossible outcomes:B ,B ,. . . ,BM. Suppose that each possible outcome canoccur with probabilitiesp1,p2, . . . , pk.

The probability that B1 occurs k1 times, B2 occurs k2 times, .

. . , and BMoccurs km times is

where n = k1 + k2 + . . . + kM.


MkMkk

M

M pppkkk

nkkkP ...

!!...!

!,...,, 21 21

21

21

152


77/92

6/7/2011

77

(Continue)

Mkkkn!

21

M

M

Mkkk

...!!...!

,...,, 2121

21

Multinomial Coefficient


153

Example

Pick 10 telephone numbers at random from atele hone book and note the last di it in each ofthe numbers. What is the probability that weobtain each of the integers from 0 to 9 only

once?

Solution


410 106.31.0!1!...1!1

!10

154


78/92

6/7/2011

78

Geometric Probability Law

Geometric Probability Law

occurrence of the first success

Letm : # of trials carried out until the occurrence of

the first success

,...2,11...1''' mAAAAPmP

m

Where p :probability of success for the Bernoulli trial.

Ai : event success in ith trial


155

(Continue)

P(m) = (1-p)m-1p Geometric probability

= q - p q = -p

1

1

1 m

m

mqpmP

= -


=x

n

i

i

ni

i

m

qpqpmP001

lim

156


79/92

6/7/2011

79

(Continue)

n

i

iqx0

x = 1 + q + q2 ++ qn

qx = q + q2 ++ qn + qn+1

(1-q)x = 1 - qn+1

qx

n

1 1

q

qq

nn

i

i

1

1 1

0


11

1limlim

1

11

q

qpqpmP

n

n

n

m

i

nm

157

(Continue)

K= # of Bernoulli trial

1

11

...Km

mKK

qppqpqKmP


01

1

i

iK

Km

mqqq

i = m 1 - k158


80/92

6/7/2011

80

(Continue)

qqq

nnii

111

The probability that more than Ktrial :

Kq

pqKmP

1

1

qqnini 00


Where p = 1 q

Kq

159

Example

ComputerA sends a message to computer B.

WhenB detects an error

RequestA to retransmit

Probability of a message transmission error isq = 0.1

What is the probability that a message needs tobe transmitted more than two times?


160


81/92

6/7/2011

81

Solution

Probability ofmth success is

P(m) = qm-1p m = 1,2,

The probability that a message needs to betransmitted more than two times


161

(Continue)

1st p

2nd The probability that a

Trial more than 2 times

3rd q2p

mth qm-1p

message nee s o etransmitted more than

two times

P[m > 2] = q2 = 10-2

1=p + qp + q2p+ q3p +

q2p+ q3p + = 1 (p + qp) = 1 p qp

= q q(1 - q) = q q + q2 = q2


162


82/92

6/7/2011

82

Reliabil ity Problem

Independent trials can be used to describereliabilit roblem calculate the robabilit

that a particular operation succeeds

Operation consist ofn components

Each component succeeds withp, independent ofany other component.

Components in series

Components in parallel


163

Type of Operation

Components in series Components in parallel

W1 W3W2

W1

W2


P[W] = P[W1W2Wn]

= p xp xxp

= pn

nn

p

WWWPWP

1

... ''2'

1

3

164


83/92

6/7/2011

83

Sequential Experiments

Sequential experiment consist of a

Sequences of Independent Experiment



165


The outcome of a given experiment determineswhich subex eriment is erformed next.

It can be represented by a tree diagram.

0 0 0 0h 1

0

1 1

0 0


1 1 1 1t

0 0 0111

1 2 3 4166


84/92

6/7/2011

84

Example

Sequential experiment involves repeatedly drawing aball from one of two urns, noting the number on theball and replacing the ball in its urn. The urn fromwhich the first draw is made is selected at randomby flipping a fair coin. Urn 0 is used if the outcomeis heads and urn 1 if the outcome is tails. Thereafterthe urn used in a subexperiment corresponds to the

subexperiment.


167

(Continue)

0

0h 01

00

1

00

1

0

0011

11

0

11


1t

1

01

1 2 3 4

1

01

1

01

168


85/92

6/7/2011

85

Probabil ity of Dependent

Experiment Compute the probability of a particular sequence

of outcomes sa s s s where ,

s0 is result from 1st outcome

s1 is result from 2nd outcome

s2 is result from 3rd outcome

P[{s0}{s1}{s2}] = ??


169

(Continue)

P[{s0}{s1}{s2}] = ??

LetA = {s2},B = {s1}{s0}

P[{s0}{s1}{s2}] = P[AB]P[{s0}{s1}{s2}] = P[{s2}{s1}{s0}] = P[AB]

Since P[A B] = P[A|B]P[B]

P[{s0}{s1}{s2}] = P[{s2}|{s1}{s0}]P[{s1}{s0}]

= P[{s2}|{s1}{s0}]P[{s1}|{s0}]P[{s0}]


170


86/92

6/7/2011

86

(Continue)

P[{s0}{s1}{s2}] = P[{s2}|{s1}{s0}]P[{s1}|{s0}]P[{s0}]

P[{sn}|{sn-1}{s1}{s0}] = ??

depends on only on {sn-1}]

P[{sn}|{sn-1} {s1} {s0}]{s0}] = P[{sn}|{sn-1}]


ar ov a n

171

(Continue)

P[{s0}{s1}{s2}] = P[{s2}|{s1}{s0}]P[{s1}{s0}]

= 2 1 0 1 0 0

ThereforeP[{s0}{s1}{s2}] = P[{s2}|{s1}]P[{s1}|{s0}]P[{s0}]

{s1}

P[s0s1s2] = P[s2|s1]P[s1|s0]P[s0]


172


87/92

6/7/2011

87

(Continue)

P[s0s1sn] = P[sn|sn-1]P[sn-1|sn-2]P[s1|s0]P[s0]


173

Example

10 1

0h 010

0

010

0

010

0

001 10 1

Find the probability of the sequence 0011


1t

1 11 2 3 4

11 11

174


88/92

6/7/2011

88

(Continue)

The probability of the sequence 0011

Meaning

P[0011] = P[{s0}{s1}{s2}{s3}]

= P[{0}{0}{1}{1}]


P[0011] = P[1|1] P[1|0] P[0|0]P[0]

= P[{s3}|{s2}] P[{s2}|{s1}]P[{s1}|{s0}]P[{s0}]175

0h

(Continue)

01

00

1

00

1

0

31

1t

1

0

11 2 3 4

32

32

32

1

0

11

0

1

001


1 1

6

1

65 1 1

31 3

1

61

61

65

65

1

11

0

11

176


89/92

6/7/2011

89

(Continue)

P[0011] = P[1|1]P[1|0]P[0|0]P[0]

2 2 2

=

31

0

1

0

1

31

3

61

65

0

1

0

1

31

3 3

61

61

65

65

h

1t

1

1

0

11 2 3 4

1

1

0

1 1

1

0

1

P 1 0 = 1/3

P[0|0] = 2/3


54

5

2

1

3

2

3

1

6

50011

P

P[0] = = P[1]

177

Example

Suppose traffic engineers have coordinated the timingof two traffic lights to encourage a run of greenlights. In particular, the timing was designed so thatwith probability 0.8 a driver will find the second light

to have the same color as the first. Assuming thefirst light is equally likely to be red or green, what isthe probability P[G2] that the second light is green?

, ,at least one light? Lastly, what is P[G1|R2], the

conditional probability of a green first light given ared second light?


178


90/92

6/7/2011

90

Solution

Given P[G2|G1] = P[R2|R1] = 0.8

= =1 1 .

0.5

0.8

0.2

P[G1G2] = 0.4

P[G1R2] = 0.1

G1

R2

G2


0.5

0.8

0.2

P[R1R2] = 0.4

P[R1G2] = 0.1

R1

R2

G2

179

(Continue)

What is the probability P[G2] that the second

li ht is reen?

0.5

0.8

0.2

0.2

P[G1G2] = 0.4

P[G1R2] = 0.1

P[R1G2] = 0.1

G1

R2

G2

G2

P[G2] = P[G1G2] + P[R1G2] = 0.4 + 0.1 = 0.5


.

0.8 P[R1R2] = 0.4

R1

R2

180


91/92

6/7/2011

91

(Continue)

What is the probability P[W] that you wait for at

least one li ht?

0.5

0.5

0.8

0.2

0.2

P[G1G2] = 0.4

P[G1R2] = 0.1

P[R1G2] = 0.1R1

G1

R2

G2

G2

W = {G1R2 R1G2 R1R2}

P[W] = P[R1G2]+P[G1R2]+P[R1R2] = 0.1+0.1+0.4

= 0.6Chapter 2 : Basic Concepts of Probability Theory

. P[R1R2] = 0.42

181

(Continue)

What is P[G1|R2], the conditional probability of a

reen first li ht iven a red second li ht?

0.5

0.8

0.2

0.2

P[G1G2] = 0.4

P[G1R2] = 0.1

P[R1G2] = 0.1

G1

R2

G2

G2


2.05.0

1.0|

2

2121

RP

RGPRGP

.

0.8 P[R1R2] = 0.4

R1

R2

182


92/92

6/7/2011

References

1. Alberto Leon-Garcia, Probability and RandomProcesses for Electrical En ineerin Addision-Wesley Publishing, 1994

2. Roy D. Yates, David J. Goodman, Probabilityand Stochastic Processes: A FriendlyIntroduction for Electrical and ComputerEngineering, 2nd, John Wiley & Sons, Inc, 2005

3. Jay L. Devore, Probability and Statistics forEngineering and the Sciences, 3rd edition,Brooks/Cole Publishing Company, USA, 1991.


183

Documents

ReviewCh2-Basic Concept Probability