Advanced data assimilation methods:Extended Kalman Filter (EKF)

Ensemble Kalman Filter (EnKF)And

Advanced Kalman Filter TechniquesDave Kuhl, Hong Li and Eugenia

Kalnay

University of Maryland

10:00-12:30 CSS 2428

November 12, 2008

Introduction

• Introduction• Data Assimilation Review:

– OI-Scaler– OI Multi-variate

• Kalman Filters– Extended Kalman Filter– Ensemble Kalman Filter– Advanced Ensemble Kalman Filters

• LETKF computer lab

OI Assimilation for a Scaler (sect. 5.3)

Tem

pera

ture

t

Tb

Ta

To

Optimal weight to minimize the analysis error is:!

Ta

= Tb

+W (To"T

b)

!

W =(" b

)2

(" b)2

+ (" o)2

time

σb

σo

σa

!

(" a)2

= (1#W )(" b)2

Sequential Assimilation of a Scalar (sect. 5.3.3)

model

timet1 t2 t3

Tem

pera

ture

Tb2

Ta2

To2

Tb1

Ta1

To1 Tb

3

Ta3

To3

!

Ti

a= T

i

b+W (T

i

o"T

i

b)

!

Ti+1

b= M[T

i

a]

!

("i

a)2

= (1#W )("i

b)2

Include a model (M) to advance the time step:

Multivariate Data Assimilation

xa = analysis model state

xb = background model state

W = weighting matrix

yo = observations

H = observation operator (model->obs)

x

timet1 t2 t3

xb1

xa1

yo1

xb2

xa2

yo2

xb3

xa3

yo3

!

xi

a= x

i

b+W(y

i

o"Hx

i

b)

!

xi+1

b= M[x

i

a]

Multivariate OI Assimilation (sect. 5.4.1)

Pa=A=analysis error covariance Matrix

Pb=B=background error covariance Matrix

Po=R=observation error covariance Matrix

H=Linear observation operator

x

timet1 t2 t3

xb1

xa1

yo1

xb2

xa2

yo2

xb3

xa3

yo3 Po=R

Pa=A

Pb=B

!

xi

a= x

i

b+W(y

i

o"Hx

i

b)

!

xi+1

b= M[x

i

a]

!

W = BHT[HBH

T+R]

"1

!

Pa

= [I"WH]B

Multivariate OI Assimilation (sect. 5.4.1)

•But where do you get B?

•B is statistically pre-estimated, andconstant in time in it’s practicalimplementations.

x

timet1 t2 t3

xb1

xa1

yo1

xb2

xa2

yo2

xb3

xa3

yo3 Po=R

Pa=A

Pb=B

!

xi

a= x

i

b+W(y

i

o"Hx

i

b)

!

xi+1

b= M[x

i

a]

!

W = BHT[HBH

T+R]

"1

!

Pa

= [I"WH]B

TrueAtmos

xt

!

B = "xb # ("xb )T

!

"xb = xb# x

t

Background Error Covariance Issue

• The spatial variation of the errors is highly time dependent.• With a constant (in time) Background Error Covariance (OI and 3D-

Var) the assimilation cannot take advantage of the “errors of theday”

“Errors of the day” computed with the Lorenz 3 variable model:compare with rms (constant) error

!z

2= (z

b" z

t)2

=0.08

“Errors of the day” obtained in the reanalysis(figs 5.6.1 and 5.6.2 in the book)

Note that the mean error went down from 10m to 8m because ofimproved observing system, but the “errors of the day” (on a synoptic timescale) are still large.

In 3D-Var/OI not only the amplitude, but also the structure of B is fixedwith time

Flow independent error covariance

In OI(or 3D-Var), the scalar error correlation between two points in thesame horizontal surface is assumed homogeneous and isotropic. (p162 inthe book)

!!!!!

"

#

$$$$$

%

&

=

2

,2,1

,2

2

22,1

,12,1

2

1

...covcov

.........

covcov

cov...cov

nnn

n

n

B

'

'

'

If we observe only Washington, D.C,we can get estimate for Richmond andPhiladelphia corrections through the

error correlation (off-diagonal term in B).

Typical 3D-Var error covariance

In OI(or 3D-Var), the error correlation between two mass points in thesame horizontal surface is assumed homogeneous and isotropic.(p162in the book)

!!!!!

"

#

$$$$$

%

&

=

2

,2,1

,2

2

22,1

,12,1

2

1

...covcov

.........

covcov

cov...cov

nnn

n

n

B

'

'

'

Background ~106-8 d.o.f.

Suppose we have a 6hr forecast (background) and new observationsSuppose we have a 6hr forecast (background) and new observations

The 3D-Var Analysis doesn’t know about the errors of the day

Observations ~105-7 d.o.f.

BR

Kalman Filter Data Assimilation

x

timet1 t2 t3

xb

xa

yo

• In Kalman Filter Data Assimilation theBackground Error Covariance (B) is advanced intime along with the analysis.

• In Extended Kalman filtering a linear tangentmodel (L) that transforms the initial perturbationat time ti to a final perturbation at time ti+1

Extended Kalman Filter Data (sect. 5.6.1)

x

timet1 t2 t3

xb

xa

yo

K=Kalman Gain Matrix (weighting Matrix)

Q=Covariance Matrix of the Noise

L=Linear Tangent Model

!

xi

a= x

i

b+K

i(y

i

o"Hx

i

b)

!

xi+1

b= M[x

i

a]

!

Ki= B

iH

T[HB

iH

T+R]

"1

!

Pi

a= [I"K

iH]B

i

!

Bi+1 = P

i+1

b= L

iPi

aLi

T+Q

i

Background ~106-8 d.o.f.

Errors of the day: they lieon a low-dim attractor

With Ensemble Kalman Filtering we get perturbations pointingWith Ensemble Kalman Filtering we get perturbations pointingto the directions of the to the directions of the ““errors of the dayerrors of the day””

3D-Var Analysis: doesn’t know about the errors of the day

Observations ~105-7 d.o.f.

Background ~106-8 d.o.f.

Errors of the day: they lieon a low-dim attractor

Ensemble Kalman Filter Analysis:correction computed in the low dimensemble space

Ensemble Kalman Filtering is efficient because Ensemble Kalman Filtering is efficient because matrix operations are performed in the low-dimensional matrix operations are performed in the low-dimensional

space of the ensemble perturbationsspace of the ensemble perturbations

3D-Var Analysis: doesn’t know about the errors of the day

Observations ~105-7 d.o.f.

Background ~106-8 d.o.f.

Errors of the day: they lieon the low-dim attractor

Observations ~105-7 d.o.f.

After the EnKF computes the analysis and the analysis error covarianceAfter the EnKF computes the analysis and the analysis error covarianceAA, the new ensemble initial perturbations are computed:, the new ensemble initial perturbations are computed:1

1

k

T

i i

i

! !+

=

=" a a A

i!a

These perturbations represent the analysis error covariance and areused as initial perturbations for thenext ensemble forecast

Flow-dependence – a simple example (Miyoshi,2004)

This is not appropriateThis does reflects the flow-dependence.

There is a cold front inour area…

What happens in thiscase?

An Example of the analysis corrections from3D-Var (Kalnay, 2004)

An Example of the analysis corrections fromEnKF (Kalnay, 2004)

Extended Kalman Filter (sect. 5.6.1)

x

timet1 t2 t3

xb

xa

yo

• Updating the forecast error covariance matrix (B) ensures that theanalysis takes into account the “errors of the day.”

• Unfortunately the EKF filter is exceeding expensive since the lineartangent model (L) has size n, n=the number of degrees of freedom.For a modern model ~10^6.

• So updating B is equivalent to performing O(n) model integrations.

Ensemble Kalman Filter Data Assimilation

x

timet1 t2 t3

xb

xa

yo

• An ensemble of of model forecasts are made from an ensemble ofinitial conditions created from the analysis and analysis errorcovariance

• How the ensemble of analysis initial conditions (xai=1-K) is created

(ensemble update) differentiates the different forms of the ensembleKalman Filter

• The number of ensembles is usually of the order of 10-100– How can such a small number of ensembles capture appropriately the

error covariance?

Ensemble Kalman Filter (sect. 5.6.2)

x

timet1 t2 t3

xb

xa

yo

• The Ensemble Kalman Filter only provides an estimate of thebackground error covariance.

• Before with EKF:

• One form of EnKF:

!

Bi+1 = P

i+1

b= L

iPi

aLi

T+Q

i

!

Bi+1 = Pi+1

b "1

K #1(xk

f # x f)

k$1

K

% (xk

f # x f)

T

K=# ensembles

xf1

xfK

xf=mean ofForecastedensembles

Ensemble Kalman Filter (sect. 5.6.2)

• The ensemble Kalman filtering covariance is estimated from only alimited sample of ensemble members (K~O(10-100)), comparedwith a much larger number of degrees of freedom of the model(n~O(10^7)), so it is rank deficient (K<<n).

• “errors of day” are the instabilities of the background flow. Stronginstabilities have a few dominant shapes (perturbations lie in a low-dimensional subspace).

• It makes sense to assume that large errors are in similarly low-dimensional spaces that can be represented by a low order EnKF.

Ensemble Kalman Filter: Ensemble Update• Again how do we update the ensemble: How do we get xa

i?• Two basic techniques

1. Perturbed Observations Method– The Analysis Ensemble is formed by assimilating random perturbations added to the

observations2. Ensemble Square Root Filter Method

– The Analysis Ensemble is formed by transforming the forecast ensembleperturbations through a transform matrix.

xbxa

yoRandom

perturbationsadded to obs

Perturbed Observation Method

xbxa

yoAssimilateto meanAnalysis

EnSRF Method

Form ensemblethrough

transformationmatrix

xbxa xbxa

Ensemble Update: two approaches1. Perturbed Observations method:An “ensemble of data assimilations”

It has been proven that an observationalensemble is required (e.g., Burgers et al.1998). Otherwise

is not satisfied.

Random perturbations are added to theobservations to obtain observations foreach independent cycle

However, it introduces a source of samplingerrors when perturbing observations.

xi

a

(k )= x

i

b

(k )+K

i(y

i

o

(k )! Hx

i

b

(k ))

Pi

a

n!n = [I "KiH]P

i

b

xi

b

(k )= Mx

i!1

a

(k )

Ki= P

i

bH

T[HP

i

bH

T+ R]

!1

Pi

b !1

K "1(x

k

b

k=1

K

# " xb )(xk

b " xb )T

noiseo

ik

o

i+= yy

)(

xbxa

yoRandom

perturbationsadded to obs

Perturbed Observation Method

xbxa

Ensemble Update: two approaches

2. Ensemble square root filter(EnSRF) Observations are assimilated to

update only the ensemble mean.

Assume analysis ensembleperturbations can be formed bytransforming the forecast ensembleperturbations through a transformmatrix

xi

a= x

i

b+K

i(y

i

o! Hx

i

b)

xi

b= Mx

i!1

a

Ki= P

i

bH

T[HP

i

bH

T+ R]

!1

Pi

b !1

K "1(x

k

b

k=1

K

# " xb )(xk

b " xb )T

xi

a= x

i

b+K

i(y

i

o! Hx

i

b)

xbxa

yoAssimilateto meanAnalysis

EnSRF Method

Form ensemblethrough

transformationmatrix

xbxa

!

xi(k )

a= x

i

a+X

i(k )

a

!

Xi(k )

a= T

iX

i(k )

b

Ensemble Update: two approaches2. Ensemble square root

filter (EnSRF) Basically we know what Pa is

from the assimilation and wewant to form an ensemble whichsatisfies that result.

Different Flavors of EnSRF: EnSRF (Andrews 1968,

Whitaker and Hamill 2002) EAKF (Anderson 2001) ETKF (Bishop et al 2001) LETKF (Hunt 2005)

xbxa

yoAssimilateto meanAnalysis

EnSRF Method

Form ensemblethrough

transformationmatrix

xbxa

X

a= (I ! %KH)X

b, %K = "K

Xa= AX

b

Xa= X

bT

2/1)~( abaPXX =

LETKF Data AssimilationLETKF is roughly based upon the ETKF assimilation

however model space is broken into local regionswhere assimilation occurs simultaneously

Provides computational efficiency on parallelcomputer systems. Each local region can becomputed separately and independently.

Summary steps of LETKF

Ensemble analysis

Observations

GCM

Obs.operator

Ensembleforecast

Ensemble forecastat obs. locations

LETKF

1) Global 6 hr ensemble forecaststarting from the analysis ensemble

2) Choose the observations used foreach grid point3) Compute the matrices of forecastperturbations in ensemble space Xb

4) Compute the matrices of forecastperturbations in observation space Yb

5) Compute Pb in ensemble spacespace and its symmetric square root

6) Compute wa, the k vector ofperturbation weights

7) Compute the local grid pointanalysis and analysis perturbations.

8) Gather the new global analysisensemble. Go to 1

LETKF algorithm summary, Hunt 2006Forecast step (done globally): advance the ensemble 6 hours, global model size n

xn

b(i )= M (x

n!1

a(i )) i = 1,...,k

Analysis step (done locally). Local model dimension m, locally used obs s

Xb= X ! x

bY

b= H (X) ! H (x

b) " HX

b

in model space (mxm)

Ensemble analysis perturbations in model space (mxk)

in ensemble space (kxk)

Analysis increments in ensemble space (kx1)

xn

a= x

n

a+ X

bw

n

a Analysis (mean) in model space (mx1)

We finally gather all the analysis and analysis perturbations from each grid pointand construct the new global analysis ensemble (nxk) and go to next forecast step

xn

a= x

n

a+ X

a Analysis ensemble in model space (mxk)

(mxk) (sxk)

%Pa = (k !1)I + HXb( )

T

R!1HX

b"#

$%

!1

babTaaTaXPXXXP

~==

2/1)~( abaPXX =

)(~ 1 b

n

o

n

bTaa

nyyRYPw !=

!

Issues with the Kalman Filter: Inflationx

timet1 t2 t3

xb

xa

yo

• As was discussed last time without any help the Kalman filter tendsto calculate decreased error with each time step eventually leadingto disregarding any new observations.

• A tuned error covariance inflation factor must be used to balance outthe decrease in error from each assimilation time step– Too small of factor: Converges on model, Disregards Observations– Too large of factor: Disregards model, Converges on Observations

t4

LETKF Computer Lab

• Set-up the Lorenz-96 model– Create Control forecast– Create Ensemble of forecasts

• Set-up fake observations:– Create Observations by perturbing the Control forecast

• Set-up LETKF (without localization) to assimilateobservations

• Investigate the impact of different variables on results:– Number of observations– Observation Error– Size of the ensemble– Inflation factor

References and thanks:Ott, Hunt, Szunyogh, Zimin, Kostelich, Corazza, Kalnay, Patil, Yorke, 2004: Local

Ensemble Kalman Filtering, Tellus, 56A,415–428.

Kalnay, 2003: Atmospheric modeling, data assimilation and predictability,Cambridge University Press, 341 pp. (3rd printing)

Hunt, Kalnay, Kostelich, Ott, Szunyogh, Patil, Yorke, Zimin, 2004: Four-dimensional ensemble Kalman filtering. Tellus 56A, 273–277.

Szunyogh, Kostelich, Gyarmati, Hunt, Ott, Zimin, Kalnay, Patil, Yorke, 2005:Assessing a local ensemble Kalman filter: Perfect model experiments with theNCEP global model. Tellus, 57A. 528-545.

Hunt, 2006: Efficient Data Assimilation for Spatiotemporal Chaos: a LocalEnsemble Transform Kalman Filter. Physica D., submitted.

Whitaker and Hamill, 2002: Ensemble Data Assimilation Without PerturbedObservations. Monthly Weather Review, 130, 1913-1924.