Probability Theory Presentation 05

8/8/2019 Probability Theory Presentation 05

1/53

BST 401 Probability Theory

Xing Qiu Ha Youn Lee

Department of Biostatistics and Computational BiologyUniversity of Rochester

September, 16

Qiu, Lee BST 401
http://goforward/http://find/http://goback/


2/53

Outline

1 Extension of Measures

Qiu, Lee BST 401


3/53

Motivations

To show that a measure defined on an algebra F0 (not

necessarily a -algebra) can be extended to a measure on

F = (F0), the -algebra generated by F0.

Remark: F is generated by F0 is just another way of

saying F is the minimal -algebra containing F0.

It serves as a bridge between finite step set operations

(pertain to an algebra) and infinite step set operations

(e.g., taking set limit) that are essential to -algebras.

As a consequence, we can use finite steps (valid within thealgebra) to approximate measures of members of the

-algebra.

Qiu, Lee BST 401


4/53

Motivations










-algebra.

Qiu, Lee BST 401


5/53

Motivations










-algebra.

Qiu, Lee BST 401


6/53

Motivations










-algebra.

Qiu, Lee BST 401


7/53

Other Similar Ideas

Classical example of rational number approximation to realnumbers.

Tolerance of small errors is of crucial importance.

Essentially we are replacing the real thing with an

imperfect finitely constructed replacement.

For rational/real number approximation, it means that we

replace an irrational number aby a rational number an a,such that the discrepancy between the two measured by

f(x), dist(f(an), f(a)) ,is less than some pre-specified .

For measure extension: we replace an arbitrary Borel set Aby An A, An F0, which is constructed from intervals byfinitely many set operations, such that the discrepancy

between the two measured by (), (AnA) ,is less thansome pre-specified .

Qiu, Lee BST 401


8/53

Other Similar Ideas










Qiu, Lee BST 401


9/53

Other Similar Ideas










Qiu, Lee BST 401


10/53

Other Similar Ideas










Qiu, Lee BST 401


11/53

Discountinuous Examples

Without continuity there is no prediction. The Dirichletfunction 1Q(x).

(Exercise 1.4, Pg 438:) does not satisfy countable

additivity, so it is discontinuous, therefore the extension to

(A) does not exist.

A technical term, -finiteness, is also important. It basically

says () can be infinite, but it can not be too infinite. Wemust be able to break into countably many subsets An,each with finite measure.

Most measures we will study in this course will be -finite.Some infinite dimensional spaces have natural measures

that are not -finite, but they are not useful for probability

theory.

Qiu, Lee BST 401


12/53





(A) does not exist.





theory.

Qiu, Lee BST 401


13/53





(A) does not exist.





theory.

Qiu, Lee BST 401


14/53





(A) does not exist.





theory.

Qiu, Lee BST 401


15/53

Approximations in the real world

Finite step approximation is more than approximation. It

is pretty much the only way we, as animals equipped with

finite step logic calculation ability, can deal with the real

world, which is infinitely complex.

Philosophical implications. Almost all engineering solutions

assumes continuity of the real world. Think: why you even

dare to drive a car? Predictability.

No perfect predictability in this world. In fact there is no

perfect measurement of any sort: time, length, force,thickness of your car, smoothness of the road, etc.

Qiu, Lee BST 401


16/53











Qiu, Lee BST 401


17/53











Qiu, Lee BST 401


18/53

Measure extension (I)

Extending a probability measures 0 onF

0, the algebragenerated by intervals, to (F0), the Borel -algebra.

First, extend F0 to G = F0 {limits of increasing sequenceof sets}

Its pretty easy to check there exist a set function defined

on this larger collection:For those A F0, define 1(A) = 0(A).For those B that can only be expressed as a limit ofincreasing sequence of A1,A2, . . . in F0, define1(B) = limn 0(An). Why this is a good definition?

It satisfies:Non-negativity,1() = 0., 1() = 1,Finite additivity, and a prototypeof countable additivity.

Qiu, Lee BST 401


19/53








Qiu, Lee BST 401


20/53








Qiu, Lee BST 401


21/53








Qiu, Lee BST 401


22/53








Qiu, Lee BST 401


23/53








Qiu, Lee BST 401


24/53








Qiu, Lee BST 401


25/53








Qiu, Lee BST 401


26/53

Measure extension(II)

You may think if we could just extend G to G, which include

all limits of interval-sequences then it would be a -algebra

and we are done. After all, thats precisely how we extend

Q to R, right?

Unfortunately, this is not the case. G is indeed closed

under -union/intersection and complement for F0, but it isnot close for such operations acting on those non-F0members!

The analogy for real numbers would be: is the limit of a

sequence of real numbers (which are the limits of a

sequence of rational numbers) again a real number?

limn

lim

mqnm

R? (1)

Qiu, Lee BST 401


27/53





Q to R, right?






limn

lim

mqnm

R? (1)

Qiu, Lee BST 401


28/53





Q to R, right?






limn

lim

mqnm

R? (1)

Qiu, Lee BST 401


29/53

Measure extension (III)

As for the Borel sets, we want to know whether

limn

lim

mAnm

G, Anm G. (2)

The answer for the real number analogy is (fortunately)

yes.

The answer for the Borel sets is no.

You can checkout the wikipedia entry Borel set, and pay

attention to the first section, Generating the Borelalgebra.

Qiu, Lee BST 401


30/53



limn

lim

mAnm

G, Anm G. (2)

The answer for the real number analogy is (fortunately)

yes.




Qiu, Lee BST 401


31/53



limn

lim

mAnm

G, Anm G. (2)

The answer for the real number analogy is (fortunately)yes.




Qiu, Lee BST 401


32/53



limn

lim

mAnm

G, Anm G. (2)

The answer for the real number analogy is (fortunately)yes.




Qiu, Lee BST 401


33/53

Measure extension (IV)

Next, we extend 1 to , a set function that is almost a

measure defined on 2, the maximum -algebra of :

For G G, (G) = 1(G).For every A ,

(A) = inf {1(G) : G G, G A} . (3)

Essentially, it tries to define measure of an arbitrary set A

by surrounding it with many larger sets, and let these sets

shrink to their infimum. So is called an outer measure,

which defines upper bounds of an arbitrary set.

Qiu, Lee BST 401


34/53





(A) = inf {1(G) : G G, G A} . (3)





Qiu, Lee BST 401


35/53





(A) = inf {1(G) : G G, G A} . (3)





Qiu, Lee BST 401

M i (IV)


36/53





(A) = inf {1(G) : G G, G A} . (3)





Qiu, Lee BST 401

P ti f t


37/53

Properties of an outer measure

Properties of , also serves as general definitions of outer

measure:Non-negativity, () = 0, () = 1. (All inherited from1, which in turn inherited from 0.

A B implies (A) (B). (monotonicity)

Countable sub-additivity

(n=1An) n=1

(An). (4)

And yes, there are indeed some F 2, such that:

(F) < (F1) + (F2), F = F1 + F2, F1 F2 = . (5)

Qiu, Lee BST 401

P ti f t


38/53






(n=1An) n=1

(An). (4)


(F) < (F1) + (F2), F = F1 + F2, F1 F2 = . (5)

Qiu, Lee BST 401

P ti f t


39/53






(n=1An) n=1

(An). (4)


(F) < (F1) + (F2), F = F1 + F2, F1 F2 = . (5)

Qiu, Lee BST 401



40/53






(n=1An) n=1

(An). (4)


(F) < (F1) + (F2), F = F1 + F2, F1 F2 = . (5)

Qiu, Lee BST 401

The existence part of the extension theorem


41/53


The outer measure is defined on the maximum

-algebra of (which is good), but in general it does nothave countable additivity (which is bad).

A workaround: restrict to a collection F, so a set

E F

satisfy:

(F) = (F E F1

) + (F Ec F2

), for all F . (6)

Equation (6) sometimes serve as the definition ofmeasurable sets. It implies countable additivity of on

F.

Qiu, Lee BST 401



42/53





E F

satisfy:

(F) = (F E F1

) + (F Ec F2

), for all F . (6)


F.

Qiu, Lee BST 401



43/53





E F

satisfy:

(F) = (F E F1

) + (F Ec F2

), for all F . (6)


F.

Qiu, Lee BST 401

(Cont)


44/53

(Cont )

Lemma (2.4), pg 447 proves that F is indeed a -algebra.

So restricted on F is a true probability measure!

If you prefer, you can further restrict

to F = (F0).In fact, F and F are very similar: sets in F can always

be expressed as AN, where A F is the main part, N isa null set ((N) = 0).

Qiu, Lee BST 401

(Cont)


45/53

(Cont )






Qiu, Lee BST 401

(Cont)


46/53

(Cont )






Qiu, Lee BST 401

Carathodory extension theorem


47/53


Theorem (1.1), pg. 438.

It generalized the above extension theorem to the case of

-finite measures.It claims this extended measure is unique. In other words,

you cant have two measures as extensions of 0, and they

disagree on some set A F.

Qiu, Lee BST 401



48/53


Theorem (1.1), pg. 438.





Qiu, Lee BST 401



49/53


Theorem (1.1), pg. 438.





Qiu, Lee BST 401

Measure approximation


50/53


Definition: set difference operation:

AB= (A Bc) (Ac B).

Assume is -finite on F0.

For any A F and a given > 0, there exists a set B F0such that (AB) < .

This theorem is almost the - technique in probability

theory.

Qiu, Lee BST 401



51/53

easu e app o at o


AB= (A Bc) (Ac B).




theory.

Qiu, Lee BST 401



52/53

pp


AB= (A Bc) (Ac B).




theory.

Qiu, Lee BST 401



53/53

pp


AB= (A Bc) (Ac B).




theory.

Qiu, Lee BST 401

Documents

Probability Theory Presentation 05