Chapter 1

Fundamentals of Applied

Probability by Al Drake

Experiment Any non-deterministic process. For example:

Model of an Experiment A simplified description of an experiment. For example: - the number on the top face

Sample Space All possible outcomes of an experiment.

“The precise statement of an appropriate sample space, resulting from the detailed description of a model of an experiment, will do much to resolve common difficulties [when solving problems].

In this book we shall literally live in sample space.” (pg .6)

Visualizing the Sample Space

(1)(2) (3) (4) (5)

(6)

sample points (S1, S2, etc.)

universe (U)

Different ways of listingexperimental outcomes:

(1)(2) (3) (4) (5)

(6)

(divisible by 2) (odd)(even)

(divisible by 3)

(5)

Sample Space All possible outcomes of an experiment.

Sample points must be: - finest grain no 2 distinguishable outcomes share a point

- mutually exclusive no outcome maps to more than 1 point

- collectively exhaustive all possible outcomes map to some point

O1

O2

S1

S1

S2O1

??O1

Different ways of listingexperimental outcomes:

(1)(2) (3) (4) (5)

(6)

(divisible by 2) (odd)(even)

(divisible by 3)

(5)

Event Defined on a sample space. A grouping of 1 or more sample points.

(1)

(2)

(3)

(4)

(5)

(6)

{even}{< 3}

{5}

Warning: This is subtly unintuitive.

Event Space A set of 1 or more events that covers all outcomes of an experiment.

Different ways of definingevents:

(1)

(2)

(3)

(4)

(5)

(6)

(1)

(2)

(3)

(4)

(5)

(6)

(1)

(2)

(3)

(4)

(5)

(6)

{even}

{odd}

{2} {4} {6}

{1} {3} {5}

{5}

{divisible by 2}

{divisible by 3}

Sample Event Space A set of 1 or more events that covers all outcomes of an experiment.

Event points must be: - mutually exclusive no outcome maps to more than 1 event

- collectively exhaustive all possible outcomes map to some event

Different ways of definingevents:

(1)

(2)

(3)

(4)

(5)

(6)

(1)

(2)

(3)

(4)

(5)

(6)

(1)

(2)

(3)

(4)

(5)

(6)

{even}

{odd}

{2} {4} {6}

{1} {3} {5}

{5}

{divisible by 2}

{divisible by 3}

Operations on events

A = {< 3}

(1)

(2)

(3)

(4)

(5)

(6)

A’

complement

Operations on events

A = {< 3}

(1)

(2)

(3)

(4)

(5)

(6)

AB

intersection

B = {odd}

Operations on events

A = {< 3}

(1)

(2)

(3)

(4)

(5)

(6)

A + B

union

B = {odd}

Venn Diagram – picture of events in the universal set

Axioms

Theorems

Proving a Theorem: CD = DC

Start with axiom 1: A+B = B+A

Warning: A + B is not addition, AB is not multiplication A + B = A + B + C does not mean C = φ

A B

C

How do you establish the sample space?

Sequential Coordinate System

How do you establish the sample space?

Probability A number assigned to an event which represents the relative likelihood that event will occur when the experiment is performed.

For event A, its probability is P(A)

Axioms of Probability

Theorems

Warning: There are 2 types of + operator here

Conditional Probability

Conditional Probability

A

B

shift to conditional probability space

B

A

A

B

0.1 0.4 0

0.30.1

0.1

P(A|B) ?

Sequential sample spaces revisited

Independence Two events A and B, defined on a sample space, are independent if knowledge as to whether the experimental outcome had attribute A would not affect our measure of the likelihood that the experimental outcome also had attribute B.

Formally: P(A|B) = P(A)

P(AB) = P(A)*P(B)

NOTE: A may be φ

Mutual Independence

For example, if you have 5 events defined, any subset of these events has the property: P(A1 A2 A5) = P(A1)P(A2)P(A5)

Mutual Independence Pair-wise independence doesn’t meanmutual independence.

If we know:P(A1 A2) = P(A1)P(A2)P(A1 A3) = P(A1)P(A3)P(A2 A3) = P(A1)P(A3)

This does not imply mutual independence:P(A1 A2 A3) = P(A1)P(A2)P(A3)

Conditional Independence

Bayes Theorem

If a set of events A1, A2, A3… form an event space:

Permutations, Combinations

Bonus Problem

1 2 331