29
Approximating Kalman ltering in large dynamical systems Harri Auvinen, Heikki Haario and Tuomo Kauranne Department of Applied Mathematics Lappeenranta University of Technology [email protected] 4 January 2006 – Typeset by FoilT E X

kauranne

Embed Size (px)

Citation preview

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 1/29

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 2/29

Outline

Á      Introduction to data assimilation

Á     Kalman filter

Á     Extended Kalman filter

Á     3D-Var

Á     

4D-VarÁ      Variational Kalman filter

Á     Simulated assimilation results

Á     Conclusions

– Typeset by FoilTEX – 1

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 3/29

Introduction to data assimilation

The formulation of the general data assimilation (state estimation)problem for discrete time steps t = 1, 2, 3,...,n contains an evolution orprediction equation and an observation equation:

x(t) = Mt(x(t− 1)) +Et (1)

y(t) = Kt(x(t)) + et, (2)

whereMt is a model of the evolution operator, xt is the state of the processat time t and Et is a vector valued stochastic process.

The second equation connects the state estimate x(t) to themeasurements y(t), with their associated observation errors et, by wayof an observation operator Kt.

– Typeset by FoilTEX – 2

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 4/29

Popular assimilation methods:

Á      Kalman filter

– Extended Kalman filter (EKF)– Fast Kalman filter (FKF)– Reduced Rank Kalman filter (RRKF)– Ensemble Kalman filter (EnKF)

Á      3D-Var

Á      4D-Var (used in operational weather forecasting)

– Typeset by FoilTEX – 3

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 5/29

Basic linear Kalman filter

The basic Kalman filter operates on a linear version of the generalassimilation problem, equations (1) and (2):

x(t) = Mtx(t− 1) + Et (3)

y(t) = Ktx(t) + et, (4)

where Mt is the linear evolution operator and similarly Kt is the linearobservation operator. The Kalman filter operates sequentially in time.

– Typeset by FoilTEX – 4

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 6/29

Kalman filter algorithm (1/2)

Let xest(t − 1) be an estimate of state x(t − 1) and Sest(t − 1) bethe corresponding error covariance matrix of the estimate. At time t theevolution operator is used to produce an a priori  estimate xa(t) and itscovariance Sa(t):

xa(t) = Mtxest(t− 1) (5)

Sa(t) = MtSest(t− 1)MT

t + SEt, (6)

where SEt is the covariance of the prediction error Et.

– Typeset by FoilTEX – 5

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 7/29

Kalman filter algorithm (2/2)

The next step is to combine xa(t) with the observations y(t) made attime t to construct an updated estimate of the state and its covariance:

Gt = Sa(t)KT

t (KtSa(t)KT

t + Set)−1 (7)

xest(t) = xa(t) + Gt(y(t)−Ktxa(t)) (8)

Sest(t) = Sa(t)−GtKtSa(t), (9)

where Gt is the Kalman gain matrix, which is functionally identical to themaximum a posteriori  estimator.

In a more general case, when the evolution model and/or the observationmodel is non-linear, the Extended Kalman Filter (EKF) is required.

– Typeset by FoilTEX – 6

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 8/29

Extended Kalman filter algorithm (1/3)

The filter uses the full non-linear evolution model equation (1) to producean a priori  estimate: xa(t) = Mt(xest(t− 1)) .

In order to obtain the corresponding covariance Sa(t) of the a priori 

information, the prediction model is linearized about xest(t− 1):

Mt = ∂ Mt(xest(t− 1))∂ x

(10)

Sa(t) = MtSest(t− 1)MT

t + SEt. (11)

The linearization in equation (10) is produced by taking finite differences

between evolved small state vector perturbations, which is acomputationally very  expensive operation for large models, unless an

– Typeset by FoilTEX – 7

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 9/29

adjoint code to the model M has been produced. In both cases, integratingall the columns in M forward in time in equation (11) is expensive.

– Typeset by FoilTEX – 8

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 10/29

Extended Kalman filter algorithm (2/3)

The observation operator is linearized at the time of the observation aboutthe a priori  estimate xa(t) in order to obtain Kt, which is then used forcalculating the gain matrix:

Kt =∂ Kt(xa(t))

∂ x(12)

Gt = Sa(t)KT

t (KtSa(t)KT

t + Set)−1. (13)

– Typeset by FoilTEX – 9

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 11/29

Extended Kalman filter algorithm (3/3)

After this, the full non-linear observation operator is used to update xa(t)and this is then used to produce a current state estimate and thecorresponding error estimate:

xest(t) = xa(t) + Gt(y(t)−Kt(xa(t))). (14)

Sest(t) = Sa(t)−GtKtSa(t). (15)

If the linearization of the observation operator at xa(t) is not good enoughto construct xest(t), it is necessary to carry out some iterations of the lastfour equations.

– Typeset by FoilTEX – 10

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 12/29

Three-dimensional variational method (3D-Var)

The idea of 3D-Var method is to avoid the computation of the gaincompletely by looking for the analysis as an approximate solution to theequivalent minimization problem defined by the cost function:

J (xt) = (xa(t)− xt)T(Sa(t))−1(xa(t)− xt) (16)

+ (yt −K(xt))TSe−1t (yt −K(xt)), (17)

where K is the nonlinear observation operator.

Opposite to the EKF method, the solution is retrieved iteratively byperforming several evaluations of the cost function and of its gradientusing a suitable descent algorithm. The approximation lies in the fact thatonly a small number of iterations are performed.

– Typeset by FoilTEX – 11

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 13/29

Illustration of the variational cost-function minimization

– Typeset by FoilTEX – 12

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 14/29

The four-dimensional variational method (4D-Var)

4D-Var is a generalization of the less expensive 3D-Var method and it alsosupersedes earlier statistical interpolation methods, which do not take intoaccount model dynamics. In its general form, it is defined as theminimization of the following cost function:

J (x0) = J b + J o (18)

= (xb − x0)TSa(t0)−1(xb − x0) (19)

+T 

t=0

(y(t)−Kt(Mt(x0)))TSe−1t (y(t)−Kt(Mt(x0))).(20)

4D-Var method assumes that the model is perfect over the assimilationperiod T . This leads to the so-called strong constraint formalism.

– Typeset by FoilTEX – 13

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 15/29

Illustration of the 4D-Var assimilation

– Typeset by FoilTEX – 14

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 16/29

Approximating EKF

Á      (Strong constraint) 4DVAR:

Á     S est(t) is a static background error matrix S a(t0) and there is no modelerror

S est(t) ≡ S a(t0)

– Typeset by FoilTEX – 15

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 17/29

Approximating EKF

Á      Reduced Rank Kalman Filtering RRKF:Á     

RRKF behaves like the Extended Kalman Filter in a low dimensionalsubspace determined at the beginning of the assimilation window

S est(

t) =

M tLT  S F T 

F I LM T 

t

Á      L is the orthogonal matrix that transforms model variables into controlvariables. S is the model error covariance matrix in the chosenk-dimensional subspace of the control space and F  is its

cross-correlation matrix with the complement subspace

– Typeset by FoilTEX – 16

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 18/29

Approximating EKF

Á      Fast Kalman Filtering FKF:

Á     FKF produces results numerically identical to the full Kalman filter EKFbut has one order of magnitude lower complexity

Á     FKF complexity is still superlinear, but it suits medium sized systemswell

– Typeset by FoilTEX – 17

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 19/29

Approximating EKF

Á      Ensemble Kalman Filtering EnKF:

Á     A low dimensional approximation En − S est to S a(t0) is propagated bythe full nonlinear model and there is no model error

En− S est(t) = si∈V  

(M(si)(t)−M(si)(t))(M(si)(t)−M(si)(t))T 

Á      V  is a sample of vectors, such as a multinormal sample modulated by alow rank approximation to the analysis error covariance matrix S a(t0) atinitial time

– Typeset by FoilTEX – 18

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 20/29

Variational Kalman filter

Á      The idea of the VKF method is to replace the most time consumingpart of the Extended Kalman Filter (EKF) method, the updating of theprediction error covariance matrix Sest(t− 1) equation (11) in order toobtain Sa(t), by a low rank update that only requires a few dozenmodel integrations.

Á     The Variational Kalman Filter can be viewed as a predictor correctorscheme in the Cartesian sum of the state and covariance spaces. In thescheme, a 3D-Var method is used to find both the state estimate andthe error covariance matrix.

Á     Dominant Hessian singular vectors at time t + 1 are approximated byevolving the search directions from time t.

Á     

The rank of the error covariance approximation can be regulated.

– Typeset by FoilTEX – 19

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 21/29

Variational Kalman filter algorithm (1/3)

The VKF method uses the full non-linear prediction equation (1) toconstruct an a priori  estimate from the previous state estimate:

xa(t) = Mt(xest(t− 1)). (21)

The corresponding approximated covariance Sa(t) of the a priori 

information is available from previous time step of VKF method.

In order to avoid the computation of the Kalman gain we perform a3D-Var optimization with equivalent cost function. As a result of theoptimization we get the state estimate xest(t) for present time t.

– Typeset by FoilTEX – 20

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 22/29

Variational Kalman filter algorithm (2/3)

The error estimate Sest(t) of the state estimate follows from the formula:

Sest(t) = 2(Hess(t))−1, (22)

where the matrix Hess(t) can be approximated by using search directionsof the optimization process. During the optimization another task is done:the full non-linear evolution model equation (1) is used to transfer searchdirections to next time step t + 1. These evolved search directions are thenused to update the approximation of the present covariance Sa(t) in orderto approximate Sa(t + 1).

– Typeset by FoilTEX – 21

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 23/29

Variational Kalman filter algorithm (3/3)

In practice the updates are carried out by using the limited memoryBroyden-Fletcher-Goldfarb-Shanno (L-BFGS) formula in order to maintainan upper limit on the rank of the Hessian approximation used in theL-BFGS Quasi-Newton method. After a 3D-Var optimization we have anupdated covariance Sa(t) which is then used to produce:

Sa(t + 1) = Sa(t) + SEt. (23)

At each iteration of the optimization method the evolved search directionsand the evolved gradients of  J  are stored and the approximation of theSa(t) is updated as the optimization method updates it’s own

approximation of the Hessian J .

– Typeset by FoilTEX – 22

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 24/29

Simulated assimilation results

Á      Simple linear evolution model

Á     100 observation times

Á     100 grid points

Á     data is obtained at the last 2 grid points in each set of 5 (i.e. theobserved grid point indexes were 4, 5, 9, 10, 14, 15, 19, 20, 24, 25,....)

– Typeset by FoilTEX – 23

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 25/29

Simulated assimilation results (1/4)

0 10 20 30 40 50 60 70 80 90 100−10

−5

0

5

10

15

Relative error: 100*(EKF−True)./True

   R  e   l  a   t   i  v

  e  e  r  r  o  r   [   %   ]

time t

– Typeset by FoilTEX – 24

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 26/29

Simulated assimilation results (2/4)

0 10 20 30 40 50 60 70 80 90 100−10

−5

0

5

10

15

Relative error: EKF vs. VKF

   R  e   l  a   t   i  v

  e  e  r  r  o  r   [   %   ]

time t

Red lines: EKF

Blue lines: VKF

– Typeset by FoilTEX – 25

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 27/29

Simulated assimilation results (3/4)

0 10 20 30 40 50 60 70 80 90 100−20

−15

−10

−5

0

5

10

15

20

Relative error: 100*(EKF−True)./True

   R  e   l  a   t   i  v

  e  e  r  r  o  r   [   %   ]

time t

– Typeset by FoilTEX – 26

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 28/29

Simulated assimilation results (4/4)

0 10 20 30 40 50 60 70 80 90 100−20

−15

−10

−5

0

5

10

15

20

Relative error: EKF vs. VKF

   R  e   l  a   t   i  v  e  e  r  r  o  r   [   %   ]

time t

Red lines: EKF

Blue lines: VKF

– Typeset by FoilTEX – 27

8/3/2019 kauranne

http://slidepdf.com/reader/full/kauranne 29/29

Conclusions

Á      EKF is computationally unfeasible for large dynamical systems

Á     FKF is a good, exact supplement to EKF for medium sized systems

Á     VKF is a good approximation to EKF for large systems

Á     It allows model error to be incorporated

Á     

It has a dynamic error covariance matrixÁ      It retains the linear complexity of 4DVAR

– Typeset by FoilTEX – 28