CL 202 Fundamentals Handout Spring2016

8/18/2019 CL 202 Fundamentals Handout Spring2016

1/43

20-01-2016

1

CL 202: Introduction to Data Analysis

Fundamentals of Probabilityand Random Variables

Mani Bhushan and Sachin Patawardhan

Department of Chemical Engineering

I.I.T. Bombay

1/20/2016 Fundamentals 1

Automation LabIIT Bombay

Outine

Sample Space

Borel Field and Probability Measure

Probability Space

Computing Probabilities

Concept of a Random Variable

Discrete and Continuous Random Variables

Properties of Random Variables

Appendix: Conditional Probability and Independence



2/43

20-01-2016

2


Note

The material in this presentation is composed frommultiple sources. The references are listed at theend. If you are looking for one reference text thatcontains almost every concept covered here then

refer to the following standard textbook:

Papoulis, A. and Pillai, S. U., Probability, RandomVariables and Stochastic Processes, (4’th Ed.),

MacGraw-Hill International, 2002.



Probability (Maybeck, 1979)


Intuitive approach to define probabilities of events ofinterest in terms of the relative frequencies of occurrence

If the event A is observed to occur N(A) times ina total of N trials, then P(A) is defined by

provided that this limit in fact exists.

Although this is a conceptually appealing basis for probabilitytheory, it does not allow precise treatment of many problems

and issues of direct importance.

Modern probability theory is more rigorously basedon an axiomatic definition of the probability.

N

A N

N A P

)(lim)(


3/43

20-01-2016

3


Sample Space (Maybeck, 1979)


)(i.e.

experimenttheofoutcomeelementarysingle:

conductedexperimenttheofoutcomes

possibleallcontainingspacesamplelfundamenta:

S

S

S S Ai.e. ofsubsetaisAeventsuchEach

.experimenttheofoutcomesof

setspecificainterest,ofeventspecifica:A

,

.Aifi.e.A,ofelementanis

outcomeobservedtheifoccurtosaidisAeventAn


Sample Space

Discrete Sample space: consists of a finite or

countably infinite number of elements/outcomes

Examples : (1) Coin toss or roll of a die experiments,

(2) set of manufacturing defects in a device

Continuous Sample Space: consists of

uncountable number of elements

Examples : (1) Values measurement noise can take in a

sensor, (2) Yield of a reaction, (3) Monthly profit of a

company



4/43

20-01-2016

4


Field (Papoulous and Pillai, 2002)


FF

FF

F

FField

B A B Ab

A Aa

then

andthen

ifthatsuchsetsofclassempty-nonaisfieldA

,)(

)(

)(

)(event"impossible"theand

)(event"certain"thecontainsfieldThe(b)

thenIf(a)

thatshownbecanit,propertiestheseUsing

A A

A AS

B A B A B A

B A A,B

FF

FF

________

Example: Consider experiment of rolling a die once 6,5,4,3,2,1S asspacesampledefinecanWe


Borel Field (Papoulous and Pillai, 2002)


field.Borelacalledisthen

tobelongalsosetstheseofonsintersecti

andunionstheIf.fieldinsetsofsequence

infiniteanis....,......Suppose)(

B

B

B

BFieldBorel

n A A A 21

A probability measure is defined only on a Borel field.

field.abetoqualifiesclassThe

asdefinedsets,ofclassaconsiderNow

(contd)Example

F

F

F

S ,6,4,6,4,2,5,3,2,1,5,3,1},6,5,4,3,1{},2{,

,


5/43

20-01-2016

5


Example 1: Rolling of a Die (Jazwinski, 1970)


Consider experiment of rolling a die (with facesnumbered from 1 to 6) once

ways.multipleindefinedbecanfieldBorelaNow,

asspacesample)(or yprobabilitdefinecanWe

6,5,4,3,2,11 S

asdefinedbecanfieldBorelthen(even),and(odd)

eventsonlyonbettingininterestedareweIf:2Case

2

2

1,6,4,2,5,3,1, S B

B

)11

1

ofsetpower(i.e. ofsubsets

allofsetastakenbecanfieldBorel:1Case

S S

B


Example 1: Rolling of a Die


field.BorelaasqualifynotdoessetThus

.andthougheven

:Note

asdefinedsetaconsiderNow

F

F

F

F

F

11

1

5,3,2,1

5,3,2,15,3,1}2{

,6,4,2,5,3,1},2{,

S S

S

Example 2: Error in temperature measurement

This is an example of a continuous RV.Theoretically, the measurement error can take any

real value.

line.reali.e.spaceSample RS


6/43

20-01-2016

6


Example 2: Measurement Errors


formtheofeventsofconsistswhich

Field,BorelthetobelongsetsfollowingtheThen,

eethatsuchRonpointsbeeandeLet 2121,

.

B

1

212

*

1

1

*

11

*

1

21

222

2

111

1

],(

),(],(,:

],(,:

],(,:

e

ee A A

ee Re A

ee

e Re A

e

e Re A

e

toequaliserror:4Event

andbetweenliesError:3Event

toequalor thanlessiserror:2Event

toequalorthanlessiserror:1Event


Axioms of Probability


N.infinitecountablyandfiniteallfor

)P(A)AP(thenofelements

exclusivemutuallyordisjointareIf3.

1)P( 2.

allfor0)P(1.

:thatsuch) (ofmemberaiswhicheachto

),value,aassignsthatfieldBorelondefined

functionvalued-scalarrealabetodefinedis

measure) yprobabilit(orfunction yprobabilitThe

ii

N

i

N

i

N

i

ii

i

,...A ,A A

S

A

A A

A P

P

11

21

(

(.)

B

BA

BB

B

i


7/43

20-01-2016

7


Note (Papoulous and Pillai, 2002)


It may be noted that, the axioms of probability areso chosen that the resulting theory gives asatisfactory representation of the physical world.Probabilities as used in real problems must,therefore, be compatible with the axioms. Using thefrequency interpretation of probability, it can beshown that they do.

P(A)+P(B) N

B N A N

N

B A N B A P

B A B A

B N A N B A N B A=N S N S =S P

N> A N A P

)()()()(

),()()()(1)(

0)(0)(

Hence

both.notbut occursorthenoccurs,if

becausethen},{=If3..hencetrial;everyatoccursbecause2.

.and0because1.


Example: Dart Board


1 A

2 A

7 A

Sample Space (S): Set of allPoints on the Dart Boardseteachtovalue

yprobabilitaassignto

easyRelatively:sets

exclusivemutually

ordisjointare

Cones ,... A ,... ,A A 721 ,

Event of interest: Concentric Circles


8/43

20-01-2016

8


Probability Space


lly.axiomaticadefinedallfunction, yprobabilitthe

andfield,Borelunderlyingthespace,sampletheof

P),,(tripletthebyDefined :space yProbabilit BS

Example 1: Rolling of a Die (contd.)

1B

B

ineventanyof yprobabilitfindcanweThen

forsetweIf

forsetsdisjointConsider ofsetpowerfieldBorel:1Case 11

6,..,2,16/1)(

.6,..,2,1}{

1

i A P

ii AS

i

i

6,5,4,3,2,11 S spacesampleConsider




asdefinedisfieldBoreland

(even),and(odd)eventsonlyon

bettingininterestedareweIf:2Case

2

2

1,6,4,2,5,3,1, S B

B

.)111 space yprobabilitaforms(tripletThus, ,P ,S B

space yprobabilitaformsTriplet

withand

and

Define

)(

0,1)(0)(

6,4,25,3,1

221

122

22

,P ,S

p,qq pS P P

q P p P

B


9/43

20-01-2016

9




RofsubsetsallofSetfieldBorel

numbers.realallofseti.e.definecan We:3Case

3

2

B

RS

PP

P

33

3

110:;01:

)6/1(43.59.1:

space yprobabilitaformsTriplet )( 332 ,P ,S B

)(;)(

contains}a],(-which1,2,...,6pts.of{No.×)(=])((

6153.5:6125.2:

613

/ P / P

/ ,a- P


Points to Note


S A A

A A

implyNOT doesPr

implyNOT doesPr

1

0

.experiemntphysicalthebydeterminedNOT

andspecified''bemustP)B,,(Triplet S

unique.NOT isexperiment givenaforspace yprobabilitaofDefinition


10/43

20-01-2016

10




disjoint.NOT arethatineventstwobeandLet B B A

)()( I B A P B A P A P

B A B A A

........

disjoint.arewhich

writecanWe

Non-Disjoint Events

).,, P S B(sapce yprobabilitaConsider

)(1/)()(/)(

)(1)(1)()()(

,

).()(,

A P A P A P A P A

A P A P A P A P S P

A AS A A

A P A P A

eventofOdds

writecanwedisjoint,are)(andfacttheusingThen,

thanfindtoeasierisitcases,someineventanGiven B




)()()( B A P B P A P B A P

B A P

havewe(I),using(II)fromgEliminatin

)()( II B A P B P B A P

B A B

B B A B B A B A

.......

disjoint.arewhich

writecanweSimilarly,

Non-Disjoint Events (contd.)


11/43

20-01-2016

11


Example (Ross, 2009)


What percentage of males smoke neither cigars norcigarettes?

Event A: a randomly chosen male is a cigarette smokerEvent B: a randomly chosen male a cigar smoker.

Thus the probability that the person is not a smokeris .7, implying that 70 percent of American malessmoke neither cigarettes nor cigars.

A total of 28 percent of American males smokecigarettes, 7 percent smoke cigars, and 5 percent smokeboth cigars and cigarettes.

3.005.007.028.0

)()()()(

B A P B P A P B A P


Product of Sample Spaces


In statistics one usually does not consider asingle experiment, but that the same experiment

is performed several times.

experimentoriginaltheofspacesampletheof

copyaisforwhere

isspacesampleingcorrespondthethen

times,experimentanperformweWhen

n , . . . ,iS

S S S S

n

i

n

1

....321

S

i

nn

n

p

p p p P

yprobabilithaseachwhere

outcomecombinedtheof yProbabilit

i

1

1

.....),....,,(

),....,,(

212

2


12/43

20-01-2016

12


Example: Repeated Coin Toss


1,0

1)()(

,

q p

pqT P p H P

T H S

and Let

experimenttosscoinwithassociatedspaceSample

T H T H T H

n

,......,, S

timesexperimenttosscointheperformweWhen

on.soand

ThenwhensituationtheConsider

)1(..),,(

),,(

.3

2

3

p p pq p H T H P

p H H H P

n

Dekking et al., 2005


Example: Infinite Outcomes


upturnsheadfirsttheuntilrepeatedlycoinaToss

outcomesmanyinfinitelywithExperiment

,......3,2,1)

S

H ofoccurrence(i.e.

successfirstthehavetotakesittossesofNo.

experimenttheofOutcome

1

2

)1(),,....,,()(

..........

)1(),,()3(

3

n p p H T T T P n P

ni

p p H T T P P

i

havewe,For

havewe,For


13/43

20-01-2016

13


Example Infinite Outcomes


events.exclusivemutuallyordisjointare

i.e.eventsthatnotedbemayIt

),.....,,....,,),......(,,(),,(),(

,......,21

H T T T H T T H T H

n , ,

1)1(1

1

......)1()1(1

....)1(....)1(

....)(....)2()1()(

2

1

p p

p p p

p p p p p

n P P P S P

n

thatfollowsit y,probabilitofaxiomsfromThus,


Random Variable


Example 1: A die with 6 faces painted with 6different colors

Problem 1: A sample space associated with a randomphenomenon need not consist of elements that are

not numbers.

Example 2: Candidates appearing in an election heldin a constituency

How to perform numerical calculationsinvolving such sample spaces?


14/43

20-01-2016

14


Random Variable


To understand the reactor behavior, these

multiple random phenomenon have to beconsidered simultaneously.

Problem 2: We often have to consider multiple randomphenomena simultaneously. Even if the sample spacesassociated with these random phenomenon consists of

numbers, their ranges can be widely different.

Example: Temperature, pressure and feedconcentration fluctuations in a chemical reactor.

How can we treat such problems through aunified mathematical framework?


Random Variable


The concept of a random variable is introducedbecause, we need a mapping from the sample space

to the set of real numbers for carrying outquantitative analysis through

a unified mathematical framework.

It is possible to define a transformationsuch that we can perform all the calculations using

a generic sample spaceand

a generic Borel fielddefined using the generic sample space?


15/43

20-01-2016

15


Random Variable (Maybeck, 1979)


.fieldBoreltheofelementanis

R)(linerealtheonvalueanyfor

})x(:{A

formtheofAseteverythatsuch,)x(asdenoted

,inpointeachtovaluesclarrealaassignswhich

mapping'orfunction'pointvalued-realais

)x(variablerandomscalarA

B

xa

x

S

fieldBorelassociatedan

andspace,sampleaGiven

.

,

B

S


Random Variable


.function yprobabilitathrough

definedbecaniesprobabilitthe

whichforineventsare

in],(-formtheofintervalsopen-half

ofimagesinversethatsuch

intospace,samplethefrommappingais

variablerandomscalarA

(.)

,

P

R x

RS

B

)particularaforassumesfunctionthisthatvaluethe(i.e.

variablerandomtheofnrealizatioa

(mapping)variableRandom:)x(

:


16/43

20-01-2016

16


Advantage


RS

S

R spacesamplegeneric''the

withworkingstartcanwe

sayspace,sampleoriginalanon

xvariablerandomadefineweOnce

,

)(

Rofintervals-suballofconsisting

FieldBorelGeneric

FieldBorelOriginal

)(

RB

B

],(,: x R x x A

S A R

xwithxx

ineventelementarygenericA


The Generic Borel Field


values.pointandopen)halfclosed,(open,intervalsfinitetoleads

Asetstheofonsintersectiandunions,s,complementTaking i

.onproblem yprobabilitadescribingininterestofof

intervals-suballvirtuallyofcomposedis,Bfield,BorelThe

R R

R

R

x x x x

BField,BorelthetobelongsetsfollowingtheThen,

thatsuchRonpointsbeandLet .2121

],(

),(],(,

],(

],(

212

*

1

1*

11*1

222

111

x x A A

x x R x A

x x A

x x A

x

x

x


17/43

20-01-2016

17


Advantage


problems.allfor

space yprobabilitwithworkwe on,nowfromThus,

x ))(( x ,F R, RB

FunctiononDistributi yProbabilit

where

formtheofeventsallfor

definedisfunction, yprobabilitgenericA

x

x

:)(

10

]),(()(

],(

(.),

x F

x P x F

x A

P


Properties of Distribution Function


00

)(0

lim)()(

0

lim)(

for

and

Notation

xxxx x F x F x F x F

0)(1)( F F and

1Property

).()(

,)(

2121 x F x F x x

x x F

xx

x

thenifi.e.,

offunctiondecreasing-nonais

2Property


18/43

20-01-2016

18


Properties of Distribution Function


.0)(0)( 00 x x x F x F allforthenIf

3Property

xx

).()(

)(

x F x F

x F

xx

x

i.e.,right,thefromcontinuousisfunctionThe

4Property

).()()( x F x F x P xxx

5Property

)()()( 1221 x F x F x x P xxx

6Property


Points to Note


.)()()(

)(

x x F x F x F

x F

allfor

i.e.,,continuousisfunctionondistributithe

iftypecontunuouscalledisvariablerandomA

VariableRandomContinuous

xxx

x

RV.type-discretebetosaidisxthen

type),stepconstant,(piecewiseiesdiscontuit jump

ofnumberfiniteaforexceptconstantisIf

VariableRandomDiscrete

x )( x F

It is possible to encounter a situation wherea random variable is mixed type.


19/43

20-01-2016

19


Example: Coin Toss


1),(0,1,)(,)(

,,,,

q pq pqT P p H P

S T H T H S ; ;

tripletwithexperimenttosscointheConsider

B

R R

T H

BisfieldBorelassociatedandisspacesampleNew

xandx

asRVdiscreteadefinecanWe

0)(1)(

x

xq

x

x F

11

10

00

)(

for

for

for

functionondistributiNew

x


Probability of Other Events


)1)(

)(1]),((1),(

),(}

S P

x F x P x P

x x

yprobabilitofaxiomthefromfollows(This

seti.e.{xeventof yProbabilit

x

)()(],(

],(],(],(

],(],(

],(],(],(],(}

1221

2112

211

2112

2121

x F x F x x P

x x P x P x P

x x x

x x x x x x x x

xx

y)probabilitofaxiomsfrom(follows

disjointareandSets:Note

seti.e.x{eventof yProbabilit


20/43

20-01-2016

20


Example 1: Rolling a Die (contd.)


Consider experiment of rolling a die (with 6 facespainted with 6 different colors) once.

654321 C,C,C,C,C,CspacesampleOriginal 4S

.

.........,,

,

,

4

S

S

andtoadditionin

onsoand

C,C,CCC,C

C,C,C,CC,C

formtheofsetscontainsspace,sample

originaltheondefined,sayfield,BorelA

542631

654321

B




RS

i

R

spacesampleNew

x(C

RVdiscreteadefinecanWe

i 10)

on.soand

wherexA

or

wherexA

formtheofintervalsareinEvents

inAeventanusingdescribedbenowcan fieldBoreltheinsayevent,An

21212121 ,,],[~

],(~

~,

x x R x x x x x x

R x x x

A

R

R

B

BB4


21/43

20-01-2016

21




1,2,3iifonlyx(Cbecause

inC,C,CEventin35}{xEvent

i

321

35)

4BB R

6)

ix(Coutcomenoistherebecause

inEventin6}{xEvent 4BB R

2,3,4iforonly46.8x(C17.5because

inC,C,CEventin46.8}x{17.5Event

i

432

)

4BB R

2,3,4iforonly45x(C20because

inC,C,CEventin45}x{20Event

i

432

)

4BB R




28)

4

i

x(Cthatsuchoutcomenoistherebecause

inEventin28}{xEvent BB R

5iifonlyx(Cbecause

inCEventin50}{xEvent

i

5

50)

4BB R

4

4

BB

B

B

ineventassametheNOT isinEvent(2)

ineventsMultiple

tocorrespondcanineventAn(1)

Note

R

R

4BB inEventin80}{xEvent S R


22/43

20-01-2016

22


Probability Measured


contains],(-which

6010,20,...,pts.ofNo.×)(=])((

onmeasure yprobabilitequivalentAn

x x

/ ,x- P x F

R

61)(

B

6,...,2,16/1)(

4

iC P i for

onMeasure yProbabilitA B

points)6atonly

itiesdiscontinu(with

Rentireoverdefined

functioncontinuousais Unlike

:Note

x )(

),(

x F

C P i




function.staircaseais

10i)x(CRVdiscreteoffunctiononDistributi i

1}{

0}{

44

4

4

4

4

BB

BB

BB

BB

BB

inin82}x{-(82)

inin6}x{-(6)6

12inC,Cin29.5}xP{-(29.5)

6

13inC,C,Cin31.9}xP{-(31.9)

6

13inC,C,Cin38}x{-(38)

particularIn

x

x

21x

321x

321x

S P P F

P P F

P F

P F

P P F

R

R

R

R

R


23/43

20-01-2016

23


Probability Mass Function


.


24/43

20-01-2016

24




function.deltaDiractherepresentswhere

for asdrepresentebecanRVdiscretea

forfunctionmass yprobabilitthe

calculus,throughtreatmentfacilitateTo

x

)(

)()(

i

i

ii

a x

x-a x p x f

The definition of probability mass function onthe previous slide is intuitively appealing.However, it does not facilitate treatment

through calculus.




)ofpropertiesthefromfollows(This

equationintegralfollowingthe

throughfunctionondistributi yprobabilit

torelatedisfunctionmass yprobabilitThe

x

xx

)(

)()(

)()()(

,

i

xai

i

i

x

ii

x

i

ii

x

a x

a f d a p

d a pd f x F

i


25/43

20-01-2016

25


Example: Data Transmission


There is a chance that a bit transmitted through adigital transmission channel is received in error.

Define discrete RV, x, equal to the number of bitsin error in the next four bits transmitted.

S

, , , ,S

ofsetPowerfieldBorelOriginal

spacesampleOriginal

43210

000104003603

048602291601656100

54

32

.== P .==

.== P .== P .== P

S

)( ,)P(

)( ,)( ,)(

toreferencewithDefinediesProbabilit

1


Example: Data Transmission


R

i

R

ii

B

fieldBorelnewandspacesampleNew

forx

RVdiscreteadefineusLet

5,4,3,2,11)(

000104003603

048602291601656100

.= f .= f

.= f .= f .= f

)( ,)(

)( ,)( ,)(

FunctionMass yProbabilit

xx

xxx

2194770

1065610

00

)(

x.

x.

x

x F

for

for

for

x

x

x.

x.

x F

41

4399990

3299630

)(

for

for

for

x

Probability/Cumulative Distribution Function


26/43

20-01-2016

26


Example 2: Telephone Call


][)(SpaceSampleOriginal

].[intervaltimeduring

randomatoccurscalltelephoneA

,T S

,T

0

0

],0[

][ 21

T

,t t

intervaloverdefined

intervals-suballofSet

FieldBorelOriginal

B

T t t T

t t t t P

21

1221 0)( anyfor

onFunction yProbabilit

B




.,

],(

B

B

inevent,certainthewithassociatedis

where,x,ineventAn

S

T aa R

][when)x(

asRVcontinuousaDefine

,T 0

)0

],(],0[

T t

t t

RS

R

R

(for xEvent Event

)(spacesampleNew

BB

.

0],(

B

B

ineventimpossiblethewithassociatedis

where,x,ineventAn

t t R


27/43

20-01-2016

27




)()()(

)()()()()(

2112

211

2112

t t P t P t P

t t t t t t t

xxx

thatfollowsit y,probabilitofaxiomsthefrom

disjoint,arexandxeventsand xxxSince

t T

T t t/T

t -

t P t F

if

if

if

x Define x1

0)(

00

)()(

)(

],0[,

21

2121

t t

t t T t t

xintervalincallaofoccurrenceof

yprobabilittheoutfindtowantweSuppose

thatsuchconsiderNow




xThus,

xxx

xx ,)()()(

)()()(

121221

1221

T

t t t F t F t t P

t P t P t t P

002

)(

0

21

asx

thennumber,smallaiswhere andsupposeNow,

Note

T t t P

t t t t

This is an example of a continuous random variable.Which attributes qualify a RV to be called continuous?


28/43

20-01-2016

28


Continuous Random Variable


1)(0)()(

)1......()()(

:)(

dx x f x x f x f

dx x f ba P

ba Ra,b

R R x f

b

a

XXX

X

X

andsatisfymust

x

withanyforthatsuch

functiondensityaexiststhereif

continuouscalledis x,variable,randomA

a

dx x f a P a F

,:R F

)()()(

]10[(.)

Xx

bydefinedfunctionais x,RV,continuousaoffunctiononDistributi


Probability Density Function


)( x f xTypical

][

],[)(

a,b

ba x f

inliewill

xRVthethat yprobabilit

infunctiondensity

yprobabilittheunderArea

x


29/43

20-01-2016

29


Note (Dekking et al., 2005)


If the interval gets progressively smaller, thenthe probability will tend to zero.

0)(,0

)()(

0

a P

dx x f aa P

,>

a

a

thatfollowsitAs

x

havewesmall yarbitrarilanyFor

X

)()()()(


30/43

20-01-2016

30


Uniform Distribution


b x

b xaaba x

x f

x

0

)/(10

)(

)(

x

x getwe,FatingDifferenti

The distribution is notdifferentiable at a and at b.

Uniform Density Function

xb

b xaab x/

a x-

x F

if

if

if

x

1

))((

0

)(

Generalization of PDF appearing in Telephone Call example


Example: Chemical Reactor

1/20/2016Fundamentals 60

Consider steady state operation ofa Continuously Stirred Tank

Reactor (CSTR)

rateflowfeedinlet

rateflowvolumetriceffluent

VolumeReactor

q

V

vesseltheinparticle

aoftimeresidencex

interestofVariableRandom

Assumption: perfect mixing,particle's position is uniformlydistributed over the volume


31/43

20-01-2016

31




t/nt

,t

t t

lengthequalofeach

intervalssmallnintodividedbe][intervaltheLet

large.toonotiswhenintervalConsider

0

],0[

reactor?theinstaydoeslongHow:Question

atCSTRtheenteredvolume,elementalanSuppose

v

t v

0,

.)stillisvolumetotaltheandduringreactorthe

leavesamountequalstate,steadyatissystemthe(Since

intervalanduringreactor theenterswhichelementvolumeaConsider

V t

t t qv

.




)()

islengthofintervalstheofany

duringvesseltheleavesthat yprobabilitthe

mixed,wellbetoassumedisreactortheSince

)/(//( 1 nV qt V t qV v p P

t n

v

reactor)theinstays( reactor)theleaves(

ofconsistsspacesampletheeachduringThus,

2

1

v failure

v success

S t

,

.sexperimenttosscoinrepeatedthetosimilarelyqualitativ

isintervaleachinhappenswhatthatassumeusLet t


32/43

20-01-2016

32




n

n

nV

qt --p P t P

nt

t

v

1)1(),....,,()(

0

,

222 x

byedapproximatwell,largeforis,timeuptoleastatvesseltheinstillisatCSTRtheenteredwhich

volume,elementalanthat yprobabilittheThus,

,......3,2,1

S

n

1ofoccurrencefirstthe

havetotakesit)tosses""(orintervalsofNo.

:)n(assumptiospaceSample

p P

t n

v

1( 2)

islengthofintervalstheofany

duringvesseltheinstaysthat yprobabilittheThus,




V

qt

nV

qt -

nt P

n

n

exp1

1lim

)(x

lettingBy

0exp

00

)(

0exp1

00)()(

t V

qt

V

qt

t f

t V

qt t

t P t F

for

for

isfunctiondensity yprobabilitassociatedand

for for

x

equalsxoffunctionondistributithethatfollowsIt

x

x

on.distributilexponentiatheofexampleanisThis


33/43

20-01-2016

33


Exponential Distribution


)(t F x)(t f x

t t

Probability Density Function Probability Distribution Function


Exponential Distribution


for

for

functiondensity yProbabilit

for

for x

functionondistributi yProbabilit

x

x

0exp

00)(

0exp1

00)()(

t t

t t f

t t

t t P t F

Useful in describing probabilitiesassociated with many engineering problems

For example, x = lifetime of an equipment/component(i.e. time before which the equipment/component fails)


34/43

20-01-2016

34


Points to Note


A discrete RV does not have a probability density functionand

continuous RV does not have a probability mass function.

However, both have a distribution function and

dx x f a P a P a F )(],()()( xxx

exists.derivativethewhen

thatcalculusintegralthe

fromfollowsitRV,discreteorcontinuousaFor

xx

dx

xdF x f

)()(


Points to Note


A continuous RV is defined using the integralequation (1) on slide 51. The RV definition doesnot require the density function to becontinuous and differentiable at all points on R.Thus, a valid probability density function may

have discontinuities at isolated points on thereal line.

Histogram of a continuous RV can be viewed asan approximation of the probability densityfunction. The relative frequency is an estimateof the probability that a measurement falls inthe interval.


35/43

20-01-2016

35


Histogram


A typical histogram of a continuous Random Variable [1]


Summary

The modern axiomatic definition of the probabilityfacilitates rigorous mathematical treatment ofthe random phenomenon.

A probability space consists of the triplet (i)

sample space, (ii) a Borel field defined on thesample space and (iii) a probability measuredefined on each event in the Borel field.

The concept of a random variable is introducedbecause, we need a mapping from the sample spaceto the set of real numbers for carrying outquantitative analysis through a unifiedmathematical framework.



36/43

20-01-2016

36


Summary


problems.allfor

space yprobabilitwithworkwe

mappingvariablerandomadefiningAfter

x ))(( x ,F R, RB

problems.allfor

where formtheofeventsallfor

definedisfunction, yprobabilitgenerica

Moreover

x 10]),(()(],(

(.),

x P x F

x A

P



AppendixConditional Probabilities

and Independence(Dekking et al., 2005)


37/43

20-01-2016

37


Conditional Probability


IndependenceIf the conditional probability of A equals what theprobability of A was before, then events A and B

are called independent .

.

).,,(

B

B

ineventstworepresentBandALet

sapce yprobabilitaConsider P S

Conditional ProbabilityKnowing that an event B has occurred sometimes

forces us to reassess the probability of event A. Thenew probability is the conditional probability.




0.provided

asdefinedisoccurredhasBeventgiven

Aeventof yprobabilitlconditionaThe

)(

)(

)()|(

B P

B P

B A P B A P

1

)(

)(

)(

)()(

)(

)(

)(

)()|()|(

)(

)()|(

B P

B P

B P

B A B A P

B P

B A P

B P

B A P B A P B A P

B P

B A P B A P

Note


38/43

20-01-2016

38




NoteThe conditional probability function

satisfies all the axioms of probability, and,thus, is a valid probability function in itself.

)()|()()|()( A P A B P B P B A P B A P

BandAeventsanyforruletionMultiplica

)()|()()|(

)()()(

)()(

,

B P B A P B P B A P

B A P B A P A P

B A B A A

A

Aeventof yprobabilitthe

asexpressedbecanwhicheventConsider


Example: Mad Cow Disease


Consider a test in which a cow is tested to determineinfection with the “mad cow disease.” It is known that,using the specified test, an infected cow has a 70%chance of testing positive, and a healthy cow just 10%. Itis also known that 2% cows are infected. Find probabilitythat an arbitrary cow tests positive.

Note: As no test is 100% accurate, most tests have theproblem of false positives and false negatives. A falsepositive means that according to the test the cow isinfected, but in actuality it is not. A false negative means aninfected cow is not detected by the test.

1.0)|(7.0)|( BT P BT P and

positivecomesTest:T Event

infectediscowpickedrandomlyA:BEvent


39/43

20-01-2016

39




)(

98.0)(02.0)(

T P

B P B P

B BS

findtoisProblem

:Note

112.098.01.002.07.0

)()|()()|()(

)()()(

)()(

B P BT P B P BT P T P

BT P BT P T P

BT BT T

Since

This is an application of the law of total probability.


Law of Total Probability


)()|(......

)()|()()|(

)().....()()(

.

2211

21

21

21

mm

m

m

m

C P C A P

C P C A P C P C A P

C A P C A P C A P A P

S C . . .C C

C , . . . , , C C

:asexpressedbecan

Aeventarbitraryanof yprobabilittheThen,

thatsuch

eventsdisjointareSuppose

Computing a probability through conditioning onseveral disjoint events that make up the whole

sample space


40/43

20-01-2016

40


Law of Total Probability


The law of total probability (illustration for m = 5).


Mad Cow Example (contd.)


A more pertinent question about the mad cowdisease test is the following:

Suppose a cow tests positive; what is theprobability it really has the mad cow disease?

?)|( T B P iswhatterms,almathematicIn

125.098.01.002.07.0

02.07.0

)()|()()|(

)()|(

)()(

)(

)(

)()|(

B P BT P B P BT P

B P BT P

BT P BT P

BT

T P

BT T B P

(Dekking et al., 2005)


41/43

20-01-2016

41




InterpretationIf we know nothing about a cow, we would say

that there is a 2% chance it is infected.However, if we know it tested positive, then we

can say there is a 12.5% chance the cow isinfected.

century.18ththeinBayesThomasclergyman

EnglishbyderivedRuleBayes'ofnapplicatio

anisusingFinding )|()|( BT P T B P


Bayes’ Rule


)(

)()|(

)()|(......)()|(

)()|()|(

.

11

21

21

A P

C P C A P

C P C A P C P C A P

C P C A P AC P

C

S C . . .C C

C , . . . , , C C

ii

mm

iii

i

m

m

:asexpressedbecanA,eventarbitrary

angiven,of yprobabilitlconditionatheThen,

thatsuch

eventsdisjointareSuppose


42/43

20-01-2016

42


Independence


)()( A P A|B P

B A

ifoftindependencallediseventAn

)()(1)(1)( A P A P A|B P |B A P

B A B A

oftindependenoftindependen:1Result

)()()()()(

)()()(

B P A P B P A|B P B A P

B A

B P A P B A P B A

oftindependenisif

rule,tionmultiplicatheofnapplicatioBy

oftindependen:2Result


Independence


)()(

)()(

)(

)()|( B P

A P

B P A P

A P

B A P A B P

A B B A

oftindependenoftindependen:3Result

true.arethemofallholds,statementstheseofoneIf

byreplacedbemayandbyreplacedbemay

followingtheofone justproveto

sufficesittindependenareandthatshowTo

.)(

)()()(

)()|(

)()(

B B A A

B P A P B A P

B P A B P

A P A|B P

B A

dependent.calledaretheyt,independennotareeventstwoIf


43/43

20-01-2016


Independence Multiple Events


formula.thethroughoutscomplement

theirbyreplacedareeventstheof

numberanywhenholdsalsostatementthisand

iftindependencalledareeventAn

n

nn

n

,. . . , A A

A P A P A P A A A P

A A A

1

2121

21

)()....()()....(

,...,,

tindependenareandthatimplyNOT doesit

thent,independenareandand

tindependenareandIf :Note

C A

C B

B A


References

1. Papoulis, A. Probability, Random Variables and StochasticProcesses, MacGraw-Hill International, 1991.

2. Dekking, F.M., Kraaikamp, C., Lopuhaa, H.P., Meester, L. E., AModern Introduction to Probability and Statistics:Understanding Why and How, Springer, 2005.

3. Montgomery, D. C. and G. C. Runger, Applied Statistics andProbability for Engineers, John Wiley and Sons, 2004.

4. Ross, S. M., Introduction to Probability and Statistics forEngineers and Scientists, Elsevier, 4th Edition, 2009.

5. Maybeck, P. S., Stochastic models, Estimation, and Control:Volume 1, Academic Press, 1979.

6. Jazwinski, A. H., Stochastic Processes and Filtering Theory,Academic Press, 1970.


Documents

CL 202 Fundamentals Handout Spring2016