HW1_Group_3

8/3/2019 HW1_Group_3

1/24

MAE 600: Homework 1Due on February 2nd, 2012

Instructor: Professor P. Singla

Priyanshu Agarwal (Contribution: 50 %)

Suren Kumar (Contribution: 50 %)

8/3/2019 HW1_Group_3

2/24

Contents

Problem 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Problem 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Problem 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Problem 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

(a) E[x] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11(b) E

(x x) (x x)T

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

(c)E(y y) (y y)

T . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14Problem 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Problem 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

1

8/3/2019 HW1_Group_3

3/24

Priyanshu Agarwal, Suren Kumar MAE 600 : Homework 1

Problem 1

Prove that Gaussian distribution function p(x) = 12

e(x)2

22 is a probability density function.

Solution 1: To prove that p(x) is a probability density function we need to prove,

1. p(x) 02.

p(x)dx = 1

Proving (1) is trivial because exponential raised to any power will result in a positive quantity and variance

is a positive quantity.

p(x)dx =

1

2e

(x)2

22 dx (1)

=1

2

e(x)2

22 dx (2)

Let y = x2

, which implies dy = dx2

. Substituting y and dy in Equation 2, we get

p(x)dx =1

ey2

dy (3)

Integral in Equation 3 is in form of a gaussian integral. Consider e

y2dy,

ey2

dy2

=

ey2

dy

ez2

dz (4)

=

e(y2+z2)dzdy (5)

=

e(y2+z2)dA (6)

where z is a dummy variable and dA = dzdy is the area of a small element in Y Z plane. PhysicallyEquation 6 refers to area of complete Y Z plane. This area could also be obtained in terms of polarcoordinates (r, ) when r [0, ), [0, 2]. Original coordinates (y, z) can be expressed in terms of polarcoordinates as

y = r cos()

z = r sin()

y2 + z2 = r2

Also in terms of radial coordinates, area of a small strip can be written as dA = rdrd. Equation 6 can nowbe written as

ey2

dy2

=

20

0

rer2

drd

=

20

12

er20

d

Problem 1 continued on next page. . . Page 2 of 23

8/3/2019 HW1_Group_3

4/24

Priyanshu Agarwal, Suren Kumar MAE 600 : Homework 1 Problem 1 (continued)

=

20

1

2d

=1

2|20

=

This implies that ey2dy = . By substituting this result in Equation 3, we get,

p(x)dx = 1

Hence, Guassian density function is a valid probability density function.

Page 3 of 23

8/3/2019 HW1_Group_3

5/24

Priyanshu Agarwal, Suren Kumar MAE 600 : Homework 1 Problem 1

Problem 2

The number =1

3E(x )3 is called the skewness of random variable, x. Show that Poisson distribution

has =1

.

Solution: 2 The probability mass function for Poisson distribution of a discrete random variable is givenby

p(x, ) =xe

x!(7)

where x Z (set of non-negative integers) and is a positive real number.The mean of x is given by

= E[x]

=

x=0

xxe

x!

= ex=0

x x

x!

= e

0 + + 22

2!+ 3

3

3!+ ...

= e

1 +

1!+

2

2!+ ...

From Taylor series expansion of e

e = 1 +

1!+

2

2!+ ...

So,

= ee

= (8)

Now, the variance of x is given by

2 = E

(x )2

= Ex2 + 2 2x

= Ex2 + E2 E[2x] ( E[x + y] = E[x] + E[y])= Ex2 + 2 2E[x] ( E[c] = c, E[cx] = cE[x], where c is a constant)= Ex2 + 2 22 ( E[x] = )

2 = Ex2 2 (9)From Eqn.s 8 and 9, we get

2 = Ex2 2 (10)Problem 2 continued on next page. . . Page 4 of 23

8/3/2019 HW1_Group_3

6/24


Now by definition E[x2] is given by

E[x2] =

x=0x2p(x, )

=x=0

x2xe

x!

= ex=0

x2x

x!

= e

0 + 12 + 222

2!+ 32

3

3!+ ...

E[x2] = e

1 + 2

1!+ 3

2

2!+ ...

(11)

Now, using the Taylor series expansion of e, we get

e = 1 +

1!+

2

2!+ ...

Multiplying both the sides by , we get

e = +2

1!+

3

2!+

4

3!+ ...

Differentiating both sides w.r.t , we get

e

+ e

= 1 + 2

1! + 3

2

2! + 4

3

3! + ...

( + 1) e = 1 + 2

1!+ 3

2

2!+ 4

3

3!+ ... (12)

From Eqn.s 11 and 12, we get

E[x2] = e ( + 1) eE[x2] = ( + 1) (13)

From Eqn. 10 and 13, we get

2 = ( + 1) 22 =

=

(14)

Now E[(x )3] is given by


8/3/2019 HW1_Group_3

7/24


E

(x )3

= Ex3 3 3x (x )= Ex3 E3 3Ex2 + E3x2

E(x )3

= Ex3 3 3Ex2 + 33 ( E[x] = , E[c] = c, E[cx] = cE[x], where c is a constant)(15)

From Eqn.s 8, 13, and 15, we get

E

(x )3

= Ex3 3 3Ex2 + 33 (16)Now by definition E[x3] is given by

Ex3 = x=0

x3p(x, )

=x=0

x3 xe

x!

= ex=0

x3x

x!

= e

0 + 13 + 232

2!+ 33

3

3!+ ...

E[x3] = e

1 + 22

1!+ 32

2

2!+ ...

(17)

Now, to evaluate the series in Eqn. 17, multiply Eqn. 12 on both sides by

( + 1) e = + 22

1!+ 3

3

2!+ 4

4

3!+ ..

Differentiating both sides w.r.t , we get

( + 1) e +

e + e + e

= + 22

1!+ 3

3

2!+ 4

4

3!+ ..

2 + 3 + 1

e = + 2

2

1!+ 3

3

2!+ 4

4

3!+ .. (18)

From Eqn.s 17 and 18, we get

E[x3] = ee 2 + 3 + 1E[x3] = 3 + 32 + (19)

Now, from Eqn.s 13, 19 and 16, we get


8/3/2019 HW1_Group_3

8/24


E

(x )3

= 3 + 32 + 3 3.. ( + 1) + 33

= 3 + 32 + 3 33 32 + 33

E(x )3

= (20)Now, skewness of a random variable is defined as

=1

3E(x )3 (21)

So, from Eqn.s 14, 20 and 21, we get

=

32

=1

(22)

From Eqn. 8 and 22, we get

=1

Hence, skewness of random variable with Poisson distribution is =1

.

Page 7 of 23

8/3/2019 HW1_Group_3

9/24


Problem 3

Show that the equal probability contours for a multi-variate Gaussian density function will be hyper-ellipsoid

surfaces. Also, draw equal probability contours (without using contour command in MATLAB) for a nor-

mally distributed 2-dimensional random vector with zero mean and following covariance matrix:

P =

4 1

1 4

Solution 3: N-dimensional gaussian density function is given by

p(x) =1

(2)n

2 || 12 exp(1

2[x ]T1[x ])

where is NN covariance matrix and is n-dimensional mean vector. Equal probability density contourscan be obtained by finding xp(x) = k, where k is a constant. To prove that equal probability densitycontours for a multi-variate gaussian is hyper-ellipsoid surface,

k =1

(2)n

2 || 12 exp(1

2 [x ]T1[x ])

k(2)n

2 || 12 = exp(12

[x ]T1[x ])

Taking log on both sides, we get

ln(k(2)n

2 || 12 ) = (12

[x ]T1[x ]) (23)2ln(k(2)n2 || 12 ) = [x ]T1[x ] (24)

LHS of Equation 24 is a constant and futhermore let it be equal to k1.

k1

= [x ]T

1

[x ] (25)Equation 25 is equation of a general hyper-ellipsoid in N-dimensional space given that is a positive

definite matrix. It can be further demonstrated using the eigen decomposition of the positive definite matrix

1 = VDVT, where D is the diagonal matrix containing eigenvalues and V is the orthonormal matrix

containing eigenvectors on its columns. Also the diagonal entries of D are positive. V is in form of a rotation

matrix. Let Y be a new vector defined as Y = VT[x ] . Substituting this result in Equation 25,

k1 = YTDY (26)

1 =i=Ni=1

i

k1yi2 (27)

Since each eigenvalue of a positive definited symmetric matrix is greater than zero, Equation 27 is an hyper-ellipsoid in n-dimensional space which is rotated with respect to original space. So it will be hyperellipsoid

in the original space as well.


8/3/2019 HW1_Group_3

10/24


For the current case after substituting and in Equation 25, we get

k1 =1

15

x y

4 11 4

x

y

(28)

15k1 = 4x2 2xy + 4y2 (29)

Equation 27 can be compared with equation of a general ellipse centered at ( x, y) = (0, 0), Ax2 + Bxy + Cy2,to get the angle of inclination with x-axis as tan 2 = B

CA . In current case the angle with x-axis is 450

degrees. Applying the rotation and getting the ellipse equation in rotated coordinates ( x1, y1),we get

x = x1 cos() y1 sin()x =

x12

y12

y = x1 sin() + y1 cos()

y =x1

2+

y12

15k1 = 3x21 + 5y21

Equation of ellipse in (x1, y1) coordinates in parametric form can be written as (x1, y1) = (5k1 cos(), 3k1 sin())as is varied from 0 to 2 Listing 1 shows a MATLAB script that was used to generate Figure 1.

6 4 2 0 2 4 66

4

2

0

2

4

6

X axis

Ya

xis

Equal Probability density Contours for Guassian Pdf

p(x) = 0.001

p(x) = 0.005

p(x) = 0.01

Figure 1: The plot shows that for large values of k the function f(k) converges to the value of 2

Listing 1: MATLAB code Used to Generate Figure 1

% Drawing a hyperellipsoid using equal probability contour of density

c l c ;c le ar a l l ; c lo se a l l ;


8/3/2019 HW1_Group_3

11/24


% Equal Probability density contour value

k = [0.001 0.005 0.01];

5 types = {-rs,--bo,:k+};

detE = 15;

fo r i = 1 :length(k)

k1 = -2*lo g(k(i)*(2*pi )*(detE(0.5)))*detE;

% Getting 360 datapoints to plot a ellipse10 data = zeros(360,2);

counter = 0;

fo r psi=0:pi /360:2*pi

x1 = sqrt(k1/3)*s in (psi);

y1 = sqrt(k1/5)*co s(psi);

15 counter = counter+1;

data(counter,:)=[(x1-y1)/sqrt(2),(x1+y1)/sqrt(2)];

end

%Plot the data

plot(data(:,1),data(:,2),types{i});

20 hold on;

end

xlabel(X axis \rightarrow);ylabel(Y axis \rightarrow);

t i t l e (Equal Probability density Contours for Guassian Pdf);

25 legend(p(x) = 0.001,p(x) = 0.005,p(x) = 0.01);

Page 10 of 23

8/3/2019 HW1_Group_3

12/24


Problem 4

Consider the following linear model:

y = Hx + (30)

where, y Rm, x Rn and is a Gaussian white noise vector with zero mean and covariance R. Giventhat x =

HTR1H

1HTR1y. Derive the relations for the following:

(a) E[x]

Solution: 4(a)

Using the given expression for x, we get

E[x] = E

HTR1H

1

HTR1y

=

HTR1H1 HTR1E[y] ( E[cx] = cE[x], where c is a constant)

Substituting the expression for y from the given linear model (Eqn. 30)

E[x] = HTR1H1 HTR1E[Hx + ]=

HTR1H1

HTR1 (HE[x] + E[])=

HTR1H1

HTR1HE[x] ( noise is zero mean, so E[] = 0)=

HTR1H1

HTR1Hx ( x is the true value which is treated as a constant, so E[x] = x)E[x] = x H

TR1H1

HTR1H = I (31)


8/3/2019 HW1_Group_3

13/24

Priyanshu Agarwal, Suren Kumar MAE 600 : Homework 1 Problem 4 [(b) E

(x x) (x x)T

]

(b) E

(x x) (x x)T

Solution: 4(b)

E(x x) (x x)T

= E(x x) xT xT

= ExxT xx xxT + xxT= ExxT ExxT ExxT + ExxT= xxT xExT E[x] xT + ExxT (32)( x is the true value which is treated as a constant, so E[x] = x)

also, E[xxT] = E[x]xT, E[xxT] = xE[xT]

Now from Eqn.s 31 and 36, we get

E

(x x) (x x)T

= xxT xxT xxT + E

xxT

E(x x) (x x)T = xxT + ExxT (33)Now, let us consider ExxT and use the expression given for x.

ExxT = EHTR1H1 HTR1y HTR1H1 HTR1yT

ExxT = EHTR1H1 HTR1yyTRTH HTRTHTExxT = HTR1H1 HTR1EyyTRTH HTRTHT (34)

Now, let us consider EyyT and use the linear model Eqn. 30.EyyT = E(Hx + ) (Hx + )T

= E(Hx + ) xTHT + T= EHxxTHT + HxT + xTHT + T= HxxTHT + HxET + E[] xTHT + ET( x is the true value which is treated as a constant, so E[x] = x)

also, E[HxxTHT] = HxxTHT, EHxT = HxET , E[xxT] = xE[xT], ExTHT = E[] xTHTEyy

T

= HxxTHT + R ( noise is zero mean, so E[] = 0) (35)

Now, from Eqn.s 34 and 35, we get

ExxT = HTR1H1 HTR1 HxxTHT + RRTH HTRTHT

Problem 4 [(b) E

(x x) (x x)T

] continued on next page. . . Page 12 of 23

8/3/2019 HW1_Group_3

14/24


ExxT = HTR1H1 HTR1HxxTHTRTH HTRTHT +HTR1H

1HTR1RRTH

HTRTH

T= xxT +

HTR1H

1HTRTH

HTRTH

T

HTR1H1

HTR1H = I

= xxT + HTR1H1 HTR1H HTR1H1 RT = RExxT = xxT + HTR1H1 (36)

Now, from Eqn.s 33 and 36, we get

E

(x x) (x x)T

= xxT + xxT + HTR1H1E

(x x) (x x)T

=

HTR1H1

(37)


8/3/2019 HW1_Group_3

15/24

Priyanshu Agarwal, Suren Kumar MAE 600 : Homework 1 Problem 4 [(c) E

(y y) (y y)T

]

(c) E

(y y) (y y)T

Solution: 4(c)

Here y = Hx

E

(y y) (y y)T

= E(y y) yT yT= EyyT yyT yyT + yyT= EyyT EyyT EyyT + EyyT (38)

Now, from Eqn.s 35, 38 and the given linear model (Eqn. 30), we get

E

(y y) (y y)T

= EyyT EyyT EyyT + EyyT= HxxTHT + R E(Hx + ) x

THT EHx (Hx + )T

+ EHxxTHT

= HxxTHT + R EHxxTHT + xTHT EHxxTHT + HxT + HExxTHTE

(y y) (y y)T

= HxxTHT + R HxExTHT ExTHT HE[x] xTHT HExT + HExxTHT(39)

( x is the true value which is treated as a constant, so E[x] = x)

Now, from Eqn.s 31, 34 and 39 and the given expression for x, we get

E

(y y) (y y)T

= HxxTHT + R HxExTHT ExTHT HE[x] xTHT HExT + HExxTHT= HxxTHT + R

HxxTHT

EyTRTH HTR1HTHT Hxx

THT

HE

HTR1H

1HTR1yT

+ HxxTHT + H

HTR1H

1HT

= HxxTHT + R HxxTHT EyTRTH HTR1HT HT HxxTHTH

HTR1H1

HTR1EyT + HxxTHT + H HTR1H1 HT(40)

Now, again using the given linear model (Eqn. 30), we get

E

(y y) (y y)T

= HxxTHT + R HxxTHT E

(Hx + )

T

RTH

HTR1H

T

HT HxxTHT

H

HTR1H1 HTR1E(Hx + ) T + HxxTHT + H HTR1H1 HT

= R ExTHT + TRTH HTR1HT HTH

HTR1H1

HTR1EHxT + T + H HTR1H1 HT= R E[] xTHT ETRTH HTR1HT HT

H

HTR1H1

HTR1

HxET + ET + H HTR1H1 HT

Problem 4 [(c) E(y y) (y y)T

] continued on next page. . . Page 14 of 23

8/3/2019 HW1_Group_3

16/24


= R RRTH HTR1HT HT H HTR1H1 HTR1R + H HTR1H1 HT( noise is zero mean, so E[] = 0)

= R

RRTH HTR1HT HT

H HTR1H1 HT + H HTR1H1 HT( noise is zero mean, so E[] = 0)

= R RRTH HTR1HT HT RT = R1

= R H HTR1HT HT RT = R1

= R H HTR1H1 HT= R HTHTR1HH11= 0

Page 15 of 23

8/3/2019 HW1_Group_3

17/24


Problem 5

Let us consider the problem of choosing an appropriate radar system for precise position tracking of an

airplane. You are supposed to estimate the range of an airplane which must have an error variance of less

than or equal to 50m2 using two radar stations measurements:

y1 = r + 1

y2 = r + 2

where r is the true range of the aircraft. It is given that one radar station has been built such that 1 is

zero-mean Gaussian noise with variance of 100m2. The other Radar station has not yet been built, and 2can be assumed to be zero-mean Gaussian noise with variance of Rm2, where R is a design parameter. Since

accurate radar station cost money, hence your objective is to find the maximum value of R that is acceptable.

Furthermore assume that a priori knowledge of r indicates that it is well modeled as zero-mean Gaussian

and have a variance of 1km2. Once again, find the maximum value of R that is acceptable and discuss how

the a priori knowledge affect your solution for R. With that R find the equations for an estimator that

incorporates both y1 and y2 simultaneously. Also find the equations for an estimator that incorporates y1

and y2 recursively. Show that these are the same estimates with the same variance.

Solution : Consider a combined model for two observationy1y2

=

1

1

r +

12

(41)

Y = Hr + , x = r (42)

For this problem, there is no prior information available and hence we choose maximum likelihood solution as

the best estimate, which is given by x = (HTR1H)1

HTR1y. R is covariance matrix of the system which

is given by E(T) because has zero mean. Also both the observations and their errors are uncorrelated.Hence R can be determined as

R =

E(1T1 ) 00 E(2T2 )

R =

100 0

0 R

Error covariance (which will be variance in current case because their is only one true quantity range ) is

given by

E((x x)(x x)T) = E((x (HTR1H)1HTR1Y)(x (HTR1H)1HTR1Y)T)

Substituting Y from Equation 42, and noting that (HT

R1H)1

(HT

R1H) = I, we get

E((x x)(x x)T) = (HTR1H)1HTR1E(T)(HTR1)T(HTR1H)T

E(T) = RE((x x)(x x)T) = (HTR1H)1

E((x x)(x x)T) = (1 1 1100 00 1

R

1

1

)1


8/3/2019 HW1_Group_3

18/24


E((x x)(x x)T) = ( 1100

+1

R)1

As a design requirement we are given that its error variance should be less than or equal to 50 m2.

(1

100+

1

R)1 50

1

100+

1

R 1

501

R 1

100

R 100

Hence the maximum acceptable value of R is 100m2.

In case of apriori information, we can use the maximum a posteriori solution , in which case the esti-

mate is given by x = (HTR1H + Q1)1

(HTR1Y + Q1xa). In current case, the prior is given as zero

mean (xa = 0) and with variance of 1km2. Error covariance (which will be variance in current case because

their is only one true quantity range ) is given by

E((x x)(x x)T) = E((x (HTR1H + Q1)1HTR1Y)(x (HTR1H + Q1)1HTR1Y)T)(x x) = x (HTR1H + Q1)1(HTR1Y)

= x (HTR1H + Q1)1(HTR1Hx + HTR1)= x (HTR1H + Q1)1((HTR1H + Q1)x Q1x + HTR1)= (HTR1H + Q1)

1Q1x (HTR1H + Q1)1HTR1)

E((x x)(x x)T) = E((HTR1H + Q1)1Q1x (HTR1H + Q1)1HTR1)(HTR1H + Q1)

1Q1x (HTR1H + Q1)1HTR1)T)

Noting that x, are uncorrelated we can simplify this expression further by using the linear property ofexpectation operator.

E((x x)(x x)T) = (HTR1H + Q1)1E(Q1xxTQT + HTR1TRTH)(HTR1H + Q1)T

Noting that E(xxT) = Q (as given by the prior with zero mean and Q covariance), E()T = R (by definition)and x, are uncorrelated, we can further simplify this expression as

E((x x)(x x)T) = (HTR1H + Q1)1(QT + HTRTH)(HTR1H + Q1)T

= (HTR1H + Q1)1

= (1 1 1100 0

01R 1

1 + 1

106

)1


8/3/2019 HW1_Group_3

19/24


As a design requirement we are given that final error variance should be less than or equal to 50 m2.

(1

100+

1

R+

1

106)1 50

1

100+

1

R+

1

106 1

50

1R

9999106

R 100.010001

Hence the maximum acceptable value of R is 100.01m2. We can see from the results that even when prior

has comparatively high variance, error variance after including it is reduced and this allows us to choose a

greater R design value. So unless the variance of prior information is , it always improves the covarianceof estimate. In this case, the prior information allowed us to choose a bigger value of variance for one of our

observation to result in same error variance.

To design an estimator incorporating y1, y2 simultaneously, lets assume the form of estimator as x = MY +n.

For this estimator to be unbiased (

E(x) = x), we get the constraints as MH = I, n = 0. Designing the

estimation to have minimum variance, we get

E((x x)(x x)T) = E(xxT) E(xxT)E(T) = RE(xT) = E(Tx) = 0 x, are uncorrelatedE(xxT) = MHE(xxT)HTMT + MRMT

= E(xxT) + MRMTE((x x)(x x)T) = MRMT

This is a constrained optimization problem which can be solved using lagrange multipliers. Loss function

can be formed as

J =1

2T r(MRMT) + T r((I MH))

MJ = MR THT = 0M = THTR1

J = I MH = 0T = (HTR1H)1

M = (HTR1H)1HTR1

Error variance of this estimator is given by

E((x

x)(x

x)T) = MRMT = (HTR1H)1. Substituting


8/3/2019 HW1_Group_3

20/24


values for current case

H =

1

1

R = 100 0

0 RE((x x)(x x)T) = (HTR1H)1

= (1

R+

1

100)1

= 50m2

Hence error variance of sequential estimator is 50m2.

To design an estimator that incorporates y1, y2 sequentially, consider the information given by first

observation as prior for second observation. The form of our system in now Y = r + 2 with H matrix being

equal to 1 and R matrix being equal to

E(22

T). A priori estimate of true state x is available as xa = x + w

and associated a priori error covariance matrix is given by E(wwT) = Q. Since in first observation equation1 has zero mean ,w also has zero mean. Also 1 and 2 are uncorrelated which implies that E(wT2 ) = 0Consider the form of estimator as x = MY + Nxa + n. For perfect observation (2 = 0), we have our

observation model as Y = Hx. If we also assume perfect a priori estimates, (xa = x, w = 0), our estimator

should give us perfect results. Using these conditions, we get

x = MHx + Nx + n

x = (MH + N)x + n

This gives us two constraints MH + N = I and n = 0. Thus our desired estimator has the form x =

MY + Nxa. Error in the estimate is given by

x x = MY + Nxa x= MHx + M + Nx + Nw x= (MH + N)x + M + Nw x= M + Nw

Our aim is to minimize the variance of error given by

E((x x)(x x)T) = E((M + Nw)(M + Nw)T)= ME(T)MT + NE(wwT)NT= MRMT + NQNT

Variance of error needs to be minimized and the final problem can be formed as a lagrange multiplier based


8/3/2019 HW1_Group_3

21/24


minimization due to presence of constraints. Loss function can be formulated as

J =1

2T r(MRMT + NQNT) + T r((I MH + N))

MJ = MR THT = 0M = THTR1

NJ = NQ T = 0N = TQ1

J = I MH + N = 0T = (HTR1H + Q1)1

M = (HTR1H + Q1)1HTR1

N = (HTR1H + Q1)1Q1

Error variance estimate is given by E((x x)(x x)T) = MRMT + NQNT. Substituting values for currentcase where H = 1, R = R, Q = 100m2,we get

M = (1

R+

1

100)1

1

R

N = (1

R+

1

100)1

1

100

MRMT + NQNT = (1

R+

1

100)2(

1

R+

1

100)

= (1

R+

1

100)1

R = 100

E((x x)(x x)T) = 50m2

This implies that error variance of both sequential and combined estimator is same.

Page 20 of 23

8/3/2019 HW1_Group_3

22/24


Problem 6

Let us consider the problem of estimation of a constant vector x Rn based upon m measurements givenby the following model:

y = Hx + (43)

Let x and be modeled as jointly Gaussian, zero-mean vectors with

E[xxT] = X, E[T] = R, E[xT] = S (44)

Your task is to find the conditional mean E[x|y] and corresponding conditional error covariance.

Solution: 6

x and are given to be jointly normal with zero-mean. Since, any linear combination of jointly Gaussian

random variables is also jointly Gaussian, x and y are also jointly Gaussian.

The mean of y using the given model (Eqn. 43) is given by

E[y] = E[Hx + ]= HE[x] + E[]= 0 ( E[x] = 0, E[] = 0)

So, the joint distribution of x and y can be written as

p(x, y) =1

(2)m+n2 det(P)

12

e1

2[xT yT]P1

x

y

(45)

where

P =

E[xxT] E[xyT]E[yxT] E[yyT]

, E[xyT] = E[yxT]T

Now, let us evaluate E[xyT] using the given model (Eqn. 43)

E[xyT] = E[x (Hx + )T]= ExxTHT + xT= ExxTHT + ExT= XHT + S (46)

Now, evaluate E[yyT] again using the given model (Eqn. 43)

E[yyT] = E[(Hx + ) (Hx + )T]= E[HxxTHT + HxT + xTHT + T]= HE[xxT]HT + HE[xT] + E[xT]HT + E[T]= HXHT + HS + STHT + R (47)


8/3/2019 HW1_Group_3

23/24


So, P is given by

P =

X XHT + S

XHT + S

T

HXHT + HS + STHT + R

,

Now, let P is given by

P =

Px Pxy

PxyT Py

,

So, P1 using the block matrix inverse from matrix cookbook can be written as

P1 =

Px PxyPy1PxyT

1

Px PxyPy1PxyT1

PxyPy1

Py1PxyT

Px PxyPy1PxyT1

Py1 + Py

1PxyT

Px PxyPy1PxyT1

PxyPy1

,

Now,

[xT yT]

Px PxyPy1PxyT

1

Px PxyPy1PxyT1

PxyPy1

Py1PxyT

Px PxyPy1PxyT1

Py1 + Py

1PxyT

Px PxyPy1PxyT1

PxyPy1

x

y

= [xT

Px PxyPy1PxyT1

yTPy1PxyT

Px PxyPy1PxyT1

xT Px PxyPy1PTxy1 PxyPx1 + yT

Py1 + Py

1PxyT

Px PxyPy1PxyT1

PxyPy1

]

x

y

= xT

Px PxyPy1PxyT

1

x yTPy1PxyT

Px PxyPy1PxyT1

x

xT Px PxyPy1PTxy1 PxyPx1y + yTPy1 + Py1PxyT Px PxyPy1PxyT1 PxyPy1y= yTPy

1y + xT

Px PxyPy1PxyT1

x yTPy1PxyT

Px PxyPy1PxyT1

x

xT Px PxyPy1PTxy1 PxyPx1y + yTPy1PxyT Px PxyPy1PxyT1 PxyPy1y= yTPy

1y + xT

Px PxyPy1PxyT1

x PxyPy1y

yTPy1Pxy

T

Px PxyPy1PxyT1

x PxyPy1y

= yTPy1y +

xT yTPy1PTxy

Px PxyPy1PxyT1

x PxyPy1y

= y

T

Py1

y +

x PxyPy1

yT

Px PxyPy1

PxyT1 x PxyPy1y

So, the joint distribution of x and y becomes

p(x, y) =1

(2)m+n2 det(P)

12

e1

2

yTPy

1y +

x PxyPy1yT

Px PxyPy1PxyT1

x PxyPy1y


8/3/2019 HW1_Group_3

24/24


det(P) = det(Py)det(Px PxyPy1PxyT) (48)

p(x, y) =1

(2)m+n2 det(Py)

12 det(Px PxyPy1PxyT) 12

e1

2

yTPy

1y +

x PxyPy1yT

Px PxyPy1PxyT1

x PxyPy1y

(49)

Now,

p(x|y) = p(x, y)p(y)

= 1

(2)n

2 det(Px PxyPy1PxyT) 12e

1

2

x PxyPy1

yT

Px PxyPy1

PxyT1

x PxyPy1

y

(50)

Hence, the mean and covariance of the conditional distribution is given by

E[x|y] = PxyPy1yE[xxT|y] = Px PxyPy1PxyT

where

Px = XPxy = XH

T + S

Py = HXHT + HS + STHT + R

Documents

HW1_Group_3