Basic statistics notes

Lecture 10Reasoning: Random VariablesA.VinciarelliArtificial Intelligence 4 (AI4)School of Computing ScienceUniversity of Glasgow (UK)

Artificial IntelligenceSchool of Computing Science

Slide 2 of 27

Lecture 10: Reference

This lecture corresponds to Appendix A of thefollowing textbook:

Machine Learning for Audio, Image and VideoAnalysisF.Camastra and A.Vinciarelliwww.dcs.gla.ac.uk/∼vincia/textbook.pdf


Slide 3 of 27

The Intelligent Agent

Agent Environm

entSensors

Actuators Actions

Percepts

I Program and reasoning


Slide 4 of 27

Outline

Random Variables

Mean, Variance, Covariance


Slide 5 of 27

Outline

Random Variables



Slide 6 of 27

Random Variables

ξ = ξ(ω)

I A variable ξ is said to be a random variablewhen its values depend on the events ω of asample space Ω

I A variable ξ that takes the value of thenumber printed on the face of a dice is arandom variable


Slide 7 of 27

Probability Distribution

I Random variables are associated tofunctions called Probability Distributions thatprovide the probability of ξ falling between xi

and xj (with xi ≤ xj)I Probability distributions are different

depending on whether the random variableis discrete or continuous


Slide 8 of 27

Discrete Random Variables

P(xi ≤ ξ ≤ xj) =

xj∑ξ=xi

pξ(ξ)

I The values of a discrete variable belong to afinite set X = x1, x2, . . . , xN

I It is possible to know the probability that ξtakes a specific value xk : P(ξ = xk) = pξ(xk)

I pξ(ξ) is the probability distribution


Slide 9 of 27

Cumulative Distribution (I)

F (x) = P(ξ ≤ x) =x∑

ξ=−∞

pξ(ξ)

I The cumulative distribution F (x) providesthe probability that ξ ≤ x

I When ξ /∈ X , p(ξ) = 0

[email protected]

Texto tecleado

Hay un error. La xi dentro de los parentesis deberia ser Xi. Es el índice del sumatorio. La xi que va justo después de la p es para indicar que variable aleatoria.

[email protected]

Texto tecleado

F(3.7) = P(xi <= 3.7) = p xi (1) + p xi (2) + p xi (3)


Slide 10 of 27

Cumulative Distribution (II)

F (∞) =∞∑

ξ=−∞

pξ(ξ) =N∑

k=1

pξ(xk) = 1

I The probability of ξ being less than∞ is theprobability that ξ takes any of the values in X

I When ξ /∈ X , p(ξ) = 0


Slide 11 of 27

Continuous Random Variables

P(x1 ≤ ξ ≤ x2) =

∫ x2

x1

pξ(x)dx

I The values of a continuous variable belongto a continuous interval [a,b]

I It is possible to know the probability pξ(x)dxthat ξ falls between x and x + dx

I pξ(x) is the probability density function

[email protected]

Texto tecleado

x + dx is a value infintitely close to x.


Slide 12 of 27


F (x) = P(ξ ≤ x) =

∫ x

−∞pξ(x ′)dx ′

I The cumulative distribution F (x) providesthe probability that ξ ≤ x

I When ξ < a or ξ > b, p(ξ) = 0

[email protected]

Texto tecleado

x prime is the integration variable


Slide 13 of 27


F (∞) =

∫ ∞

−∞pξ(x)dx =

∫ b

apξ(x)dx = 1

I The probability of ξ being less than∞ is theprobability that ξ takes any of the values in[a,b]

I When ξ < a or ξ > b, p(ξ) = 0


Slide 14 of 27

Discrete Joint Probability Distribution (I)

P(ξ1 = x , ξ2 = y) = pξ1ξ2(x , y)

I The joint probability distribution pξ1ξ2(x , y)provides the probability that ξ = x and ξ = y

I ξ1 ∈ Xx1, . . . , xN and ξ2 ∈ Y = y1, . . . , yMare discrete random variables


Slide 15 of 27

Discrete Joint Probability Distribution (II)

P(xi ≤ ξ1 ≤ xj , yk ≤ ξ2 ≤ yl) =

xj ,yl∑ξ1=xi ,ξ2=yk

pξ1ξ2(ξ1, ξ2)

I The joint probability distribution allows one toestimate the probability that xi ≤ ξ1 ≤ xj andyk ≤ ξ2 ≤ yl , with xi ≤ xj and yk ≤ yl

[email protected]

Texto tecleado

xi1 y xi2 dentro del parentesis son los indices del sumatorio x: de i a j y: de k a l


Slide 16 of 27

Discrete Joint Cumulative Distribution

F (x , y) = P(ξ1 ≤ x , ξ2 ≤ y) =

x ,y∑ξ1=−∞,ξ2=−∞

pξ1ξ2(ξ1, ξ2)

I The joint cumulative distribution allows oneto estimate the probability that ξ1 ≤ x andξ2 ≤ y , with F (∞,∞) = 1


Slide 17 of 27

Joint Probability Density Function

P(x ≤ ξ1 ≤ x+dx , y ≤ ξ2 ≤ y+dy) = pξ1,ξ2(x , y)dxdy

I The values of ξ1 and ξ2 belong to continuousintervals: ξ1 ∈ [a,b] and ξ2 ∈ [c,d ]

I The joint Probability Density Functionpξ(ξ1, ξ2) allows one to estimateP(x ≤ ξ1 ≤ x + dx , y ≤ ξ2 ≤ y + dy)


Slide 18 of 27


F (x , y) = P(ξ1 ≤ x , ξ2 ≤ y)∫ x

−∞

∫ y

−∞pξ(x ′, y ′)dx ′dy ′

I The cumulative distribution F (x , y) providesthe probability that ξ1 ≤ x and ξ2 ≤ y

I When ξ1 /∈ [a,b] or ξ2 /∈ [c,d ], p(ξ1, ξ2) = 0


Slide 19 of 27


F (∞,∞) =

∫ ∞

−∞

∫ ∞

−∞pξ(x ′, y ′)dx ′dy ′ = 1

I The cumulative distribution F (∞,∞)provides the probability that ξ1 and ξ2 takeany of the values in [a.b] and [c,d ],respectively.

I When ξ1 /∈ [a,b] or ξ2 /∈ [c,d ], p(ξ1, ξ2) = 0


Slide 20 of 27

Outline

Random Variables



Slide 21 of 27

Mean Value (Discrete)

E(ξ) =∞∑

x=−∞xpξ(x) =

N∑k=1

xkpξ(xk)

I E(ξ) is the Mathematical Expectation orMean Value of pξ(ξ)

I The mean E(ξ) is not necessarily inX = x1, . . . , xN


Slide 22 of 27

Mean Value (Continuous)

E(ξ) =

∫ ∞

−∞xpξ(x)dx =

∫ b

axpξ(x)dx

I E(ξ) is the Mathematical Expectation orMean Value of pξ(ξ)


Slide 23 of 27

Variance (Discrete)

σ2ξ =

∞∑x=−∞

[x−E(ξ)]2pξ(x) =N∑

k=1

[xk−E(ξ)]2pξ(xk)

I σ2ξ is the variance or dispersion of pξ(ξ)

I The variance is the mean of [ξ − E(ξ)]2, thesquare of the difference between thevariable and the mean


Slide 24 of 27

Variance (Continuous)

σ2ξ =

∫ ∞

−∞[x − E(ξ)]2pξ(x)dx =∫ b

a[x − E(ξ)]2pξ(x)dx

I σ2ξ is the variance or dispersion of pξ(ξ)

I The variance is the mean of [ξ − E(ξ)]2, thesquare of the difference between thevariable and the mean


Slide 25 of 27

Covariance

σξ1ξ2 = E[ξ1 − E(ξ1)][ξ2 − E(ξ2)]

I The covariance is a measurement of howmuch two variables covariate, i.e., tend tojointly increase or jointly decrease

[email protected]

Resaltar


Slide 26 of 27

Covariance (Discrete)

σξ1ξ2 =∞∑

ξ′1=−∞

∞∑ξ′2=−∞

[ξ′1−E(ξ′1)][ξ′2−E(ξ′2)]pξ1ξ2(ξ′1, ξ′2)



Slide 27 of 27

Covariance (Continuous)

σξ1ξ2 =∫ ∞

−∞

∫ ∞

−∞[ξ′1−E(ξ′1)][ξ′2−E(ξ′2)]pξ1ξ2(ξ′1, ξ

′2)dξ′1dξ′2


Documents

Basic statistics notes