A comparison of hybrid ensemble transform Kalman filter(ETKF)-3DVAR

and ensemble square root filter (EnSRF) analysis schemes

Xuguang WangNOAA/ESRL/PSD, Boulder, CO

Thomas M. HamillJeffrey S. Whitaker

NOAA/ESRL/PSD, Boulder, COCraig H. Bishop

NRL, Monterey, CA

Why hybrid ETKF-3DVAR ?

• Hybrid ETKF-3DVAR is expected to be less expensive than EnSRF, and still can benefit from ensemble-estimated error statistics.

• The hybrid may be more robust for small ensemble size, since can adjust the amount of static vs. ensemble covariance used.

• The hybrid can be conveniently adapted to the existing variational framework.

member 1 forecast

member 2 forecast

member 3 forecast

perturbation 1

perturbation 2

perturbation 3

ensemble mean ETKF-3DVAR update mean

ETKF update perturbations

updated mean

updated perturbation 3

updated perturbation 2

updated perturbation 1

member 1 analysis

member 2 analysis

member 3 analysis

member 1 forecast

member 2 forecast

member 3 forecast

data assimilation forecast

Hybrid ETKF-3DVAR

Hybrid ETKF-3DVAR updates mean

yxyxxxxx aTababTba HHJ 11

2

1

2

1RP

Pb 1 Pe B , 0 ≤ ≤ 1

Background error covariance is approximated by a linear combination of the sample covariance matrix of the ETKF forecast ensemble and the static covariance matrix.

yxxyxxvvvv bTbTT HHJ 12211 2

1

2

1

2

1R

2

2/1

12/1 1 vvx ePB

Can be conveniently adapted into the operational 3DVAR through augmentation of control variables (Lorenc 2003; Buehner 2005; Wang et al. 2006).

B static background error covariance weighting coefficient eP ensemble covariance 1v standard 3DVAR control variables

Pb hybrid covariance 2v extended control variables

ETKF updates perturbations• ETKF transforms forecast perturbations into analysis

perturbations by

where is chosen by trying to solve the Kalman filter error covariance update equation, with forecast error covariance approximated by ensemble covariance.

• Latest formula for (Wang et al. 2004;2006, MWR)

• Computationally inexpensive for ensemble size of o(100), since transformation fully in perturbation subspace.

TCIΓCT 2/1

TXX ba

bXaX

T

T

C and Γcontain eigenvectors and eigenvalues of the KK (ens. size)

matrix )1/(1 KbTTb HXRHX ; is the estimated fraction of forecast error variance projected onto ensemble subspace

Experiment design

• Observations362 Interface and surface Exner functions taken at equally spaced locations

Observation values are T31 truth plus random noise drawn from normal distribution

Assimilated every 24h

• Numerical model Dry 2-layer spectral PE model run at T31;

Model state consists of vorticity, divergence and layer thickness of Exner function

Error doubling time is 3.78 days at T31

Perfect model assumption

(Hamill and Whitaker MWR, 2005)

Experiment design

In this experiment, the hybrid updates the mean using

bTbTbba xyxx HRHHPHP 1

TTeTb BHHPHP 1

TTeTb HBHHHPHHP 1

whose solution is equivalent to that if solved variationally under our experiment design.

The static error covariance model is constructed iteratively from a large sample of 24h fcst. errors.

• Update of mean (OI) and formulation of B

First estimate BHT and HBHT is constructed from 250 24h fcst. errors with covariance localization (iteratively)

Run a huge number of DA cycles (7000 > number of model dimension) with BHT and HBHT

Construct BHT and HBHT with this huge sample of 24h forecast errors

RMS analysis errors: 50 member

• Improved accuracy of EnSRF over 3DVAR can be mostly achieved by the hybrid.

• Covariance localization applied on the ETKF ensemble when updating the mean (but not applied when updating the perturbations) improved the analyses of the hybrid.

RMS analysis errors: 20 member

• Both 20mem hybrid and EnSRF worse than 50mem, but still better than 3DVAR

• Hybrid nearly as accurate (KE, 2 norms) or even better (surface norm) compared to EnSRF

RMS analysis errors: 5 member

• EnSRF experienced filter divergence for all localization scales tried.

• Hybrid was still more accurate than the 3DVAR.

• Hybrid is more robust in the presence of small ensemble size.

EnSRF filter divergence

Comparison of flow dependent background error covariance models

(Hybrid 50 mem. result) (EnSRF 50 mem. result)

Initial-condition balance

(50 mem. result)

• Analysis is more imbalanced with more severe localization.

• Analyses of the hybrid with the smallest rms error are more balanced than those of the EnSRF, especially for small ensemble size (not shown).

Spread-skill relationships

(20 mem. result)

• Overall average of the spread is approximately equal to overall average of rms error for both EnSRF and Hybrid.

• Abilities to distinguish analyses of different error variances are similar for EnSRF and Hybrid.

Summary

• The hybrid analyses achieved similar improved accuracy of the EnSRF over 3DVAR.

• The hybrid was more robust when ensemble size was small.

• The hybrid analyses were more balanced than the EnSRF analyses, especially when ensemble size was small.

• The ETKF ensemble variance was as skillful as the EnSRF.

• The hybrid can be conveniently adapted into the existing operational 3DVAR framework.

• The hybrid is expected to be less expensive than the EnSRF in operational settings.

Preliminary results with resolution model error

• T127 run as truth; imperfect model run at T31

• 200-member ensembles

• Additive model error parameterization

• comparable rms errors for hybrid and EnSRF

BUFRyo

OBS PROC

yo

VARXb Xa

ETKFPb

mean

gen_be

VAR da_ntmax=0

sum

Xf1

….

XfK

H(Xf1)

….

H(XfK)

δXf1

….

δXfK

Xa1

….

XaK

WRF

Hybrid ETKF-3DVAR for WRF(collaborated with Dale Barker and Chris Snyder)

• α-control variable method (Lorenc 2003) to incorporate ensemble in WRF-VAR

Statistics of iteratively constructed B

rms analysis errors

balance

Ensemble square-root filter(Whitaker and Hamill,MWR,2002)

es

b PP ρ

1 RHHPHPK TbTb

bba H xyxx K

• Background error covariance is estimated from ensemble with covariance localization.

• Mean state is corrected to new observations, weighted by the Kalman gain.

• Reduced Kalman gain is calculated to update ensemble perturbations.

• Observations are serially processed. So cost scales with the number of observations.

KRHHP

RK

1

1~

Tb

bba H ''' ~xxx K

sρ correlation matrix

K Kalman gain matrix K~

reduced Kalman gain matrix a'x analysis ensemble perturbation b'x forecast ensemble perturbation

• Covariance localization can produce imbalanced initial conditions.

EnSRF EnSRF

obs1 obs2

EnSRF

obs3

Maximal perturbation growth

• Find linear combination coefficients b to maximize

• Maximal growth in the ETKF ensemble perturbation subspace is faster than that in the EnSRF ensemble perturbation subspace.

bb

bbaTaT

bTbT

SXX

SXX