Why probability?

It’s cool

Machine learning

A = {< 3}

(1)

(2)

(3)

(4)

(5)

(6)

C = {odd}

class SampleSpace{ List<SamplePoints> allSamplePoints;}

class ModelOfExperiment{ SamplePoint doExperiment(); SampleSpace getSampleSpace(); //can generate this using tree or //coordinate system}

class SamplePoint{ String description; List<Event> eventsIAmAMemberOf; float probability;}

class Event{ String description; List<SamplePoint> samplePointsIContain; float computeProbability();//sum over sample pts}

Event complement(Event e)

Event intersection(Event A, Event B)

Event union(Event A, Event B)

Conditional Probability P(A | B) = P(AB)/P(B) Event condProb(Event A, Event B)Independence P(A | B) = P(A), or P(AB) = P(A)P(B) boolean areIndependent(Event A, Event B)Conditional Independence P(AB | C) = P(A|C)P(B|C) boolean areCondInd(Event A, Event B, Event C)Bayes Theorem, Combinations, Permutations

class EventSpace{ List<Event> specialSetOfEvents;}

B = {even}

Random Variables

Chapter 2

Random Variable Numerical attribute of an experimental outcome.

Discrete Random Variable Continuous Random Variable

Example Experiment: Flip a coin 3 times.

h = total # of headsr = length of longest run (eg. 2 tails in a row)

Relationship btw. random variables and events

HHHTTT

HHTTHH

TTHHTT

THT HTH

TTT HTT

TTHTHT

THH

HHTHTH

HHH

Probability Mass Function (PMF) For discrete random variables: PMF = ph(h0) = probability that the experimental outcome will have h = h0

ph(0) = 1/8 ph(1) = 3/8 ph(2) = 3/8 ph(3) = 1/8

Example:

Example (cont.)

Graph of PMF

0

1/8

2/8

3/8

4/8

0 1 2 3

# of heads

pro

bab

ilit

y

p(h)

Compound Probability Mass Function (PMF)

Example:

Independence x, y are independent iff: For all x0, y0: P(x0y0) = P(x0)*P(y0)

P(x0|y0) = P(x0y0) / P(y0)Conditional Probability

Conditional Independence x, y are conditionally independent iff: For all x0, y0: P(x0y0 | A) = P(x0|A)*P(y0|A)

Functions defined on random variables: A function on random variable(s) creates a new random variable:

Examples:

w = f(h, r) = h*r = {0, 1, 2, 4, 9}

v = f(h) = h2 = {0, 1, 4, 9}

Expectation Weighted average of all possible outcomes.

E[x] = ∑ [ x0 px (x0) ]

E[g(x)] = ∑ [ g(x0) px(x0) ]

E[w] = E[g(x,y)] = ∑ [ ∑ [ g(x0, y0) px,y(x0, y0) ]]

Variance Measures the spread of the PMF around the expected value. σx

2 = ∑ [ (x0 – E[x])2 p(x0) ] = E[ (x – E[x])2 ]

Continuous Sample Spaces, Event Spaces

Examples: Height, Weight…

Experiments with infinitely many possible outcomes.

-3 -2 -1 0 1 2 3

Cumulative Density Function (CDF)

Function px≤ (x0) such that:

Cumulative Density Function (CDF)

Properties:

0

1/8

2/8

3/8

4/8

5/8

6/8

7/8

1

0 1 2 3

# of heads

pro

bab

ilit

y

p(h)

0

1/8

2/8

3/8

4/8

5/8

6/8

7/8

1

0 1 2 3

ruler

pro

bab

ilit

y

p(h)

Probability Density Function (PDF)

Function f(x) such that:

event space

Unit-Impulse Function

What is the derivative of f(x):

What is ∫-∞ f ’(x) dx ?x0

3

3

PMF & CDF

0

1/8

2/8

3/8

4/8

5/8

6/8

7/8

1

# of heads

pro

bab

ilit

y

p(h)

0

1/8

2/8

3/8

4/8

5/8

6/8

7/8

1

months

pro

bab

ility

CDF

PDF

0

1/8

2/8

3/8

4/8

5/8

6/8

7/8

1

months

pro

bab

ility

0

1/8

2/8

3/8

4/8

5/8

6/8

7/8

1

# of heads

pro

bab

ilit

y

p(h)

PDF

Compound Probability Density Function

Conditional Probability

Conditional PDF

Independence

Expectation