Probability Theory

Rahul Jain

Probabilistic Experiment

• A Probabilistic Experiment is a situation in which– More than one thing can happen– The outcome is potentially uncertain

When we speak of the probability ofsomething happening, we are referring tothe likelihood—or chances—of ithappening. Do we have a better chanceof it occurring or do we have a betterchance of it not occurring?

The Sample Space

• The Sample Space of a probabilistic experiment E is the set of all possible outcomes of E.

The Sample Space

Examples:E1 = Toss a coin, observe whether it is a Head (H)

or a Tail (T)1 = {H, T}

Experiments, Outcomes, Events

Experiment: a process of measurement or observation

Trial: a single performance of an experiment

Outcome (sample point): the result from a trial

Sample space (S): the set of all possible outcomes

Events: the subset of S (outcomes = simple events)

Examples:Rolling a dice. S = {1,2,3,4,5,6}Events: A={1,3,5}; B={5,6} etc.Simple events are {1},{2},{3},{4},{5},{6}

Unions and Intersections of Eventsthe union of A and B: consists of all points that are in A or B

A [ B

1

A \ B

2

the intersection of A and B: consists of all points that are in A and B

Venn diagramA

B

Unions and Intersections: example

Event A = { 1, 2, 3}

Event B = { 2, 4, 6}

Example:

A [ B

1

A \ B

2

= {1, 2, 3, 4, 6}

= {2}

Mutually Exclusive (or Disjoint)

A \ B =;

3

If A and B have no points in common

A

B

A and B are mutually exclusive (or disjoint)

empty set (set with no elements)

Event A = { 1, 3, 5}

Event B = { 2, 4, 6}A \ B =;

3

Example:

Complements of Events

S

A

complement of A: consists of all the points of S that are not in A

Ac

A [ Ac=S

5

A \ Ac=;

4

The AND Rule of Probability• The probability of 2 independent events both happening is

the product of their individual probabilities.• Called the AND rule because “this event happens AND that

event happens”.• For example, what is the probability of rolling a 2 on one die

and a 2 on a second die? For each event, the probability is 1/6, so the probability of both happening is 1/6 x 1/6 = 1/36.

• Note that the events have to be independent: they can’t affect each other’s probability of occurring. An example of non-independence: you have a hat with a red ball and a green ball in it. The probability of drawing out the red ball is 1/2, same as the chance of drawing a green ball. However, once you draw the red ball out, the chance of getting another red ball is 0 and the chance of a green ball is 1.

The OR Rule of Probability

• The probability that either one of 2 different events will occur is the sum of their separate probabilities.

• For example, the chance of rolling either a 2 or a 3 on a die is 1/6 + 1/6 = 1/3.

NOT Rule

• The chance of an event not happening is 1 minus the chance of it happening.

• For example, the chance of not getting a 2 on a die is 1 - 1/6 = 5/6.

• This rule can be very useful. Sometimes complicated problems are greatly simplified by examining them backwards.

Combining the Rules• More complicated situations involve combining the AND and

OR rules.• It is very important to keep track of the individuals involved

and not allow them to be confused. This is the source of most people’s problems with probability.

• What is the chance of rolling 2 dice and getting a 2 and a 5? The trick is, there are 2 ways to accomplish this: a 2 on die A and a 5 on die B, or a 5 on die A and a 2 on die B. Each possibility has a 1/36 chance of occurring, and you want either one or the other of the 2 events, so the final probabilty is 1/36 + 1/36 = 2/36 = 1/18.

Getting a 7 on Two Dice• There are 6 different ways of

getting two dice to sum to 7:• In each case, the probability of

getting the required number on a single die is 1/6.

• To get both numbers (so they add to 7), the probability uses the AND rule: 1/6 x 1/6 = 1/36.

• To sum up the 6 possibilities, use the OR rule: only 1 of the 6 events can occur, but you don’t care which one.

• 6/36 = 1/6

die A die B prob1 6 1/362 5 1/363 4 1/364 3 1/365 2 1/366 1 1/36total 6/36

Probability P(A) = probability of an event A

Example: In rolling a fair die, what is the probability of A being an even number.

Sample space S = {1, 2, 3, 4, 5, 6}

Event A = { 2, 4, 6} P(A) = 36

12

=

P(S) = 1

P(A) = Number of point in ANumber of point in S

Conditional probability

The probability of an event B under the condition that an event A occurs

P (BjA) = P (A \ B)P (A)

1

P (A \ B) =P (BjA)P (A)

1

=P(AjB)P (B)

2

conditional probability of B given A

Independent Events

P (A \ B) =P (A)P (B)

1

Events A and B are called independent events, if

Example Problem: 22.3 (prob.5) Three screws are drawn at random from a lot of 100 screws, 10 of which are defective. Find the probability of the event that all 3 screws drawn are non-defective, assuming that we drawn: (a) with replacement, (b) without replacement

(a)90100¢

90100¢

90100=72:9%

1

90100¢

8999¢

8898=72:65%

2

(b) (apply conditional probability)

P (A \ B) =P (BjA)P (A)

1

Permutations

Three letters a,b,c:6 possible permutations: abc, acb, bac, bca, cab, cba

3! = 3*2*1 = 6

The number of permutations of n different things taken all at a time is:

n! = n*(n-1)*…*2*1 Read “n factorial”

n possible ways

…

Permutation: an ordered arrangement of a set of objects

Permutations (contd.)

If n given things can be divided into c classes of alike things differing from class to class, then the number of permutations of these things taken all at a time is:

Three letters a,a,c:

3 possible permutations: aac, aca, caa 3!2! = 3

n1 + n2 +…+ nc = n

…a a a n1 = 3

n! n1! n2! nc!…

3! permutations of aaa are viewed to be one

Permutations (contd.)The number of different permutations of n different things taken k at a time without repetitions is:

…1 2 3 … k

n n-1 n-2 … n-k+1

n(n ¡ 1)(n ¡ 2)¢¢¢(n ¡ k+1) = n!(n ¡ k)!

1

1 2 3 … k

n n n … n

…

with repetitions

nk

1

Combinations Permutation: “order” is essentialCombination: “order” is not essential

a,b,c and b,c,a are different permutation, but they are the same combination.

Example:

The number of different permutations of n different things taken k at a time without repetitions is:

…1 2 3 … k

n n-1 n-2 … n-k+1n(n ¡ 1)(n ¡ 2)¢¢¢(n ¡ k+1) = n!

(n ¡ k)!

1

The number of different combinations of n different things taken k at a time without repetitions is: n!

k!(n ¡ k)!

1

(k! permutations are corresponding to 1 combination)

Combinations (contd.)

Ãnk

!= n!k!(n ¡ k)!

1

=n(n ¡ 1)(n ¡ 2)¢¢¢(n ¡ k+1)1¢2¢¢¢k

2

The number of different combinations of n different things taken k at a time without repetitions is:

n!k!(n ¡ k)!

1

The number of different combinations of n different things taken k at a time with repetitions is:

Ãn+k ¡ 1

k

!

1

= (n+k ¡ 1)!k!(n ¡ 1)!

2