ch_06

1

Chapter 6: The Multivariate Normal

Distribution and Copulas

Daniel B. Rowe, Ph.D.

Department of Mathematics,

Statistics, and Computer Science

Copyright 2015 by D.B. Rowe

Marquette University MSCS6020

2 2

Agenda 6.1 The Multivariate Normal 6.2 Generating a Multivariate Normal Random Vector


D.B. Rowe

3 3

6.1 The Multivariate Normal-Univariate We can obtain a random variable x that has a general

normal distribution with mean μ and variance σ2 via the

transformation

.

The PDF of x can be obtained by

where z(x) is z written in terms of x and J(∙) is the Jacobian

of the transformation.


D.B. Rowe

2( | , ) ( ( )) | ( ) | f x f z x J z x

x z

4 4

6.1 The Multivariate A bivariate (2D) PDF of two continuous random variables depending upon parameters θ satisfies 1) 2) .


D.B. Rowe

1 2( , )x x

1 2 1 20 ( , | ), ( , ) f x x x x

1 2

1 2 1 2( , | ) 1 x x

f x x dx dx

Normal-Bivariate

5 5

6.1 The Multivariate Normal-Bivariate

Let be a 2-dimensional (or p-dimensional)

random variable with PDF of x being f(x|θ), then


D.B. Rowe

1

2

xx

x

2×1

Marginal means. 1 1

2 2

( | )( | )

( | )

E xE x

E x

2×1

21 1 2 1 12

21 2 2 12 2

var( | ) cov( , | )var( | )

cov( , | ) var( | )

x x xx

x x x

2×2

Vectors are lower case, matrices are upper case.

Marginal variances.

6 6

6.1 The Multivariate Normal-Univariate Recall: If z~N(0,1), then

We can obtain a random variable x that has a

general normal distribution with mean μ and

variance σ2 via the transformation x = σ z + μ.


D.B. Rowe

2 11 1

( 0)(1) ( 0)1/2 1/22 21

( ) 2 (1)2

z z z

f z e e

Note the way I

have written this.

mean mean

variance variance

1×1 1×1 1×1 1×1

7 7

6.1 The Multivariate Normal-Univariate The PDF of x can be obtained by

where z=z(x) and J(∙) is the Jacobian of the transformation.

The PDF of x is

which can be written as


D.B. Rowe

21

2 21( | , )

2

x

f x e

1×1

2 11

( )( ) ( )1/21/22 2 2( | , ) 2x x

f x e

2( | , ) ( ( )) | ( ) | f x f z x J z x( )

( )dz x

J z xdx

Note the way I

have written this.

1×1

8 8

6.1 The Multivariate Normal-Bivariate Given two continuous random variables , we write

them as a 2-dimensional vector , and this vector

has the PDF .

If z1 and z2 are independent, then

.


D.B. Rowe

( | )Zf z

1 2( , )z z

1

2

zz

z

2×1

1 21 1 2 2( | ) ( | ) ( | )Z Z Zf z f z f z

vector

vector

9 9

6.1 The Multivariate Normal-Bivariate Let z1 and z2 be iid N(0,1) random variables. Then,

has PDF

.

With vector z, this can be rewritten as

.


D.B. Rowe

1

2

zz

z

2×1

1 2 1 2

2 21 2

, 1 2 1 2

1( )

2

( , ) ( ) ( )

1

2

Z Z Z Z

z z

f z z f z f z

e

1'

21

( )2

z z

Zf z e

1×1 1×1

2×1

10 10

6.1 The Multivariate Normal-Bivariate This can also be written as

and we write that .

That is, the 2-dimensional random vector z has a

mean vector of zero and identity variance-covariance matrix.


D.B. Rowe

1

2

zz

z

2×1

12

1'

2

1( 0) '( ) ( 0)

2/2 1/2 22

1( )

2

(2 ) | |

z z

Z

z I z

f z e

I e

2×1

mean vector mean vector

covariance matrix covariance matrix

mean vector covariance matrix

2I is 2×2

Identity matrix

2×1

2~ (0, )z N I

11 11

6.1 The Multivariate Normal-Bivariate This means that


D.B. Rowe

1

2

zz

z

2×1

12

1( 0) '( ) ( 0)

2/2 1/2 22( ) (2 ) | |

z I z

Zf z I e

1 1

2 2

( ) 0( )

( ) 0

E zE z

E z

21 1 2 1 12

21 2 2 12 2

var( ) cov( , ) 1 0var( )

cov( , ) var( ) 0 1

z z zz

z z z

2×1

2×2

0

2I

2×2 Identity matrix

12 12


If , then

We can obtain a random variable x that has a general

normal distribution with mean vector μ and variance-

covariance matrix Σ via the transformation

where Σ = A A′, is a factorization (i.e. Cholesky or Eigen).


D.B. Rowe

x A z

2~ (0, )z N I2×1

12

1( 0) '( ) ( 0)

2/2 1/2 22( ) (2 ) | |

z I z

f z I e

2×1

2×1

2×1 2×1 2×1 2×2 2×2

2×2 2×2 2×2

13 13


If a random variable x has a normal distribution with

mean vector μ and variance-covariance matrix Σ, then

and we write . The covariance matrix Σ, has to

be of full rank (there is an inverse in PDF).


D.B. Rowe

2×1

11( ) ' ( )

/2 1/2 2( | , ) (2 ) | |x x

pf x e

2×1

2×1 ~ ( , )x N

2×2

mean vector mean vector

covariance matrix

covariance matrix

mean vector covariance matrix

, px

0

2p

set of pos

def matrices

2×2

2×1 2×2

make sure you know what this means

14 14

6.1 The Multivariate Normal-Bivariate Let’s take a closer look at this bivariate transformation.

We can solve for z1 and z2 in terms of x1 and x2.


D.B. Rowe

1 11 12 1 1

2 21 22 2 2

x a a z

x a a z

1 11 1 12 2 1x a z a z

2 21 1 22 2 2x a z a z

2×1 2×1 2×2 2×1

1×1 1×1 1×1 1×1

x A z

15 15

6.1 The Multivariate Normal-Bivariate This will give us


D.B. Rowe

1z A x

1 11 1 1 12 2 2( ) ( )z b x b x

x A z

A invertible

22 121

21 1111 22 12 21

1 a aA

a aa a a a

2 21 1 1 22 2 2( ) ( )z b x b x

11 121

21 22

b bB A

b b

1 11 12 1 1

2 21 22 2 2

z b b x

z b b x

2×1 2×2 2×1 2×1

2×1 2×2 2×1

2×1 2×2 2×1

2×2

2×2 2×2

16 16


Continuing on, this leads to

i.e. with , the vector derivative is .


D.B. Rowe

z B x

1 11 1 1 12 2 2( ) ( )z b x b x

1 1 2 1 1 2

1 2 11 12

1 2 1 2

21 222 1 2 2 1 2

1 2

( , ) ( , )

( , , )( , ) ( , )

z x x z x x

x x b bJ z z x x B

b bz x x z x x

x x

2 11 1 1 12 2 2( ) ( )z b x b x

zJ B

x

2×1 2×2

2×2

17 17

6.1 The Multivariate Normal-Bivariate The distribution of vector variable x (joint of x1 and x2) is


D.B. Rowe

2×1

110 '( ) 0

2/2 1/2 2( | , ) (2 ) | |pB x I B x

X pf x I e B

( | ) ( ( )) | ( ) |X Zf x f z x J z x

11( 0) '( ) ( 0)

/2 1/2 2( ) (2 ) | |pz I z

p

pf z I e

z B x

zJ B

x

11'

/2 1/2 2( | , ) (2 ) | |x x

p

Xf x e

, px

0

2| | | || ' | | |A A A 'AA 1/2| | | |A 1B A1/2| | | |B

2×2

18 18

6.1 The Multivariate Normal-Bivariate This form may be more familiar


D.B. Rowe

12 1 2cov( , )x x 12 1 2/ ( )

1 20, 0, 1 1

2 2

1 1 1 1 2 2 2 2

2

1 1 2 2

12

(1 )

x x x xQ

1

2 2 21 2 1 2 1 2 2

1 2

1( , | , , , )

2 1

Q

Xf x x e

19 19

6.1 The Multivariate Normal-Bivariate Theorem:

If x is a 2-D (or p-D) random variable from f(x|μ,Σ), with

then we form where dimensions match

and A full column rank (A: r×p, r≤p), then

and .


D.B. Rowe

( | , )E x

var( | , )x

y A x

( | , , , )E y A A var( | , , ) 'y A A A

p×1

p×p

r×1 r×p p×1 r×1

r×p p×p p×r r×p p×1 r×1

think of p=2

r×p

20 20

6.2 Generating a Multivariate Normal Random Vector Recall: Let u1~uniform(0,1) and u2~uniform(0,1).

Let and

then and .

This means , , and are independent.


D.B. Rowe

1 1 22ln( ) cos(2 ) z u u 2 1 22ln( ) sin(2 ) z u u2 21 2

1( )

21 1, 2( )

z z

u z z e 22 1 2

1

1( , ) atan

2

zu z z

z

1 2 1 2, 1 2 , 1 1 2 2 1 2 1 2 1 2( , | ) ( , ), ( , ) | | ( , , ) | Z Z U Uf z z f u z z u z z J u u z z

2 21 2

1 2

1 1

2 2, 1 2

1 1( , )

2 2

z z

Z Zf z z e e

1 ~ (0,1)z N 2 ~ (0,1)z N 1z 2z

21 21

6.2 Generating a Multivariate Normal Random

Obtain 2-D standard normal variates by transforming

two independent standard uniform random variates u1 and u2.


D.B. Rowe

Bivariate Normal Bivariate Normal

First half of 106 standard normal variates as z1's and second half as z2's. Produce 5×105 statistically independent z's.

1

2

zz

z

2×1

Vector

6.2 Generating a Multivariate Normal Random

Multiplied 5×105 simulated z’s by A and added μ.

The y's are now N(μ,Σ=AA′).

-5 0 5 100

0.5

1

1.5

2

2.5

3

3.5x 10

4

y1

-5 0 5 10 150

0.5

1

1.5

2

2.5

3

3.5x 10

4

y2

22

3 2

2 4

3

5

y A z

Cholesky

0.58

Bivariate Normal Marginal

Normal

Marginal

Normal

cross

section

1.7321 0

1.1547 1.6330A

D.B. Rowe


A=chol(Sigma)’

2×2 2×1

2×1 2×1 2×1 2×2 2×2

2×1 2×2

1y

2y

23 23

Homework 5:

Chapter 6: # 4, 5, 6.

* Assume Marquette Undergrads heights have

μh=67 in and σh=2 in while their weights have

μw=150 lbs and σw=4 lbs with ρ=.75.

a) Present an algorithm for generating random height &

weights using both Cholesky and Eigen factorizations.

b) Generate 10,000 using both factorizations.

Compute summary means, variances, covariance and

correlation. Make 2D & 3D histograms.


D.B. Rowe

24 24


D.B. Rowe

Rowe, 2003

25 25


D.B. Rowe

2 .75 2 4

.75 2 4 4

11 2 1.4142a

2

22 214 1.3229a a

21

.75 2 41.5

2a

Sigma=[2,sqrt(2)*sqrt(4)*.75;sqrt(2)*sqrt(4)*.75,4];

a11=sqrt(2)

a21=.75*sqrt(2*4)/sqrt(2)

a22=sqrt(4-a21^2)

A=[a11,0;a21,a22] 1.4142 0

1.5000 1.3229A

Documents

ch_06