Soil Moisture Data Assimilation in the SHEELS Land Surface Model

Clay BlankenshipUSRA

Special thanks to: Bill Crosson, Jon Case

Top ofcanopy

Bare soil energy fluxes

Surface runoff Groundheat flux

Sensible LatentShortwaveLongwave Precipitation

Interceptionby canopy

Canopy energy fluxes

Radiative Fluxes

Upper zone

Root zone

Bottom zone

Diffusion/drainage Heat exchange

SoilLayers

Wind

ThroughfallSensible Latent

Infiltration

Sub-surfacelateral flow

Top ofcanopy

Bare soil energy fluxes

Surface runoff Groundheat flux

Sensible LatentShortwaveLongwave Precipitation

Interceptionby canopy

Canopy energy fluxes

Radiative Fluxes

Upper zone

Root zone

Bottom zone

Diffusion/drainage Heat exchange

SoilLayers

Wind

ThroughfallSensible Latent

Infiltration

Sub-surfacelateral flow

Overview

•Data assimilation and retrieval background

•1dvar, 3dvar, and Kalman Filters

•Soil Moisture Data Assimilation

•SHEELS LSM

•AMSR-E data

•Results

•Some results from profile retrievals with MIRS

Data Assimilation (and Variational Retrievals)

Given observations {y} and background state {xb}, estimate most likely state {x} taking into account errors in both xb and y.

 

€

P(x | y) =P(y | x)P(x)

P(y)~

exp −1

2y − H (x)( )

TR−1 y − H (x)( )

⎧ ⎨ ⎩

⎫ ⎬ ⎭exp −

1

2x − xb( )

TPb

−1 x − xb( ) ⎧ ⎨ ⎩

⎫ ⎬ ⎭

Obs-Calc Obs

{ {

Retrieved-Bkgd State

x: geophysical statexb: background (a priori) of statey: observationsH(x): forward operator converting x into observation spacePb: background error covarianceR: obs+forward model error covariance

•In “data assimilation” you are using observations to improve a model. In “profile retrievals” you always have some assumed background state (maybe climatology).

P(UT fan|orange shirt)=P(orange shirt|UT fan)P(UT fan) P(orange shirt)

Profile Retrieval (1d)

Soil Moisture (1d)

NWP

(3d or 4d)

Model Variable Column vector of T, q, hydrometeors

Column vector of soil q [and T]

3D (or 4D) grid of T, q, u, v, clouds…

Observations {TB} from satellite instrument(s)

{TB} or retrieved near-surface water content

{TB} from multiple satellites, raobs, surface obs, etc.

Typical size of state vector

500 (or 5 to 15 in EOF space)

5-20 10^6 to 10^8

Typical size of observation vector

5-20 1-3 up to 10^6

Retrieval and Data Assimilation Applications

Data Assimilation (and Variational Retrievals)

Given observations {y} and background state {xb}, estimate most likely state {x} taking into account errors in both xb and y.

 

To maximixe this probability, minimize -2 times its log:

€

P(x | y) =P(y | x)P(x)

P(y)~

exp −1

2y − H (x)( )

TR−1 y − H (x)( )

⎧ ⎨ ⎩

⎫ ⎬ ⎭exp −

1

2x − xb( )

TPb

−1 x − xb( ) ⎧ ⎨ ⎩

⎫ ⎬ ⎭

O+F Error Cov Bkgd Error Cov

Obs-Calc TB

{ {

Retrieved-Bkgd Atmosphere

€

J(x) = −2* ln P({x} |{y})( ) = y − H (x)( )TR −1 y − H (x)( ) + x − xb( )

TPb

−1 x − xb( )

Minimizing the cost function:

Given initial guess x=xb, linearize and solve for new estimate of x. Skipping the math, the solution can be expressed as: 

The “Gain” represents a weighted average of bk and ob errors

The Jacobian K relates the y terms to x termsand can include interpolation, averaging or integration,radiative transfer, etc.

€

xa = xb + PbKT (KPbK

T + R)−1(y − H (xb ))

DA Solution

€

J(x) = y − H (x)( )TR −1 y − H (x)( ) + x − xb( )

TPb

−1 x − xb( )

€

K ij ≡∂y i∂x j

Bkgd Gain Innovation€

Pb(Pb + R)

simplified:

1DVar/3DVar 4DVar Kalman Filter Ensemble Kalman Filter

Background Error

Covariance Matrix

(normally fixed)

Defined at t0. Flow dependent based on model adjoint.

Computed based on linearized model dynamics.

Estimated from ensemble covariance

Gain Calculation

Equivalent to KF with fixed Gaussian background errors

Equivalent to EnKF assuming perfect model, Gaussian bk error

Most general method in terms of error covariance

Applies innovations at proper time

Data Assimilation Methods

Hybrid methods use ensemble spread to define background covariance in 3dvar.

€

PbKT(KPbK

T + R)−1

€

PbKT(KPbK

T + R)−1

€

QK T(KQK T + R)−1

€

PbMTK T(KMPbM

TK T + R)−1

Soil Moisture Data Assimilation in the SHEELS Land Surface Model

SHEELS – Simulator for Hydrology and Energy Exchange at the Land Surface

SHEELS

• Distributed land surface hydrology model provides soil state(T,q), fluxes of energy and moisture• Heritage: 1980’s Biosphere-Atmosphere Transfer Scheme (BATS)• Can run off-line or coupled with meteorological model• Flexible vertical layer configuration designed to facilitate microwave data assimilation• Described in Martinez et al. (2001), Crosson et al. (2002)

Top ofcanopy

Bare soil energy fluxes

Surface runoff Groundheat flux

Sensible LatentShortwaveLongwave Precipitation

Interceptionby canopy

Canopy energy fluxes

Radiative Fluxes

Upper zone

Root zone

Bottom zone

Diffusion/drainage Heat exchange

SoilLayers

Wind

ThroughfallSensible Latent

Infiltration

Sub-surfacelateral flow

Top ofcanopy

Bare soil energy fluxes

Surface runoff Groundheat flux

Sensible LatentShortwaveLongwave Precipitation

Interceptionby canopy

Canopy energy fluxes

Radiative Fluxes

Upper zone

Root zone

Bottom zone

Diffusion/drainage Heat exchange

SoilLayers

Wind

ThroughfallSensible Latent

Infiltration

Sub-surfacelateral flow

SHEELS InputSHEELS Input

Required static variables:Soil type (STATSGO): Landcover (U of Md):Saturated hydraulic conductivity Canopy heightSaturated matric potential Fractional vegetation coverSoil wilting point Minimum stomatal resistanceSoil porosity Root depth

Reflectance propertiesSeasonal:Leaf area index Topography (GTOPO30):

Surface elevation and slopeTime-dependent input (forcing):• Wind speed (NLDAS)• Air temperature• Relative humidity• Atmospheric pressure• Downwelling solar radiation• Downwelling longwave radiation

• Rainfall (Stage IV)

SHEELS Input

STATESSoil surface and canopy temperaturesSoil temperature Soil water/ice content Depth of water on canopy Ponded waterSnow temperature, depth, and density

FLUXESSurface latent and sensible heat fluxesGround heat fluxNet radiation flux

Evapotranspiration InfiltrationRunoff

SHEELS Output

Land Information System (LIS)A modeling and data assimilation system with the capability to run several different LSMs, from GSFC’s Hydrological Sciences Branch. It is very customizable with the ability to swap out LSMs, forcing datasets, etc.

LSMS VIC, Noah, CLM, Catchment,SiB2, Hyssib Base Forcings ECMWF, GDAS, NLDAS... Supplemental Forcings TRMM 3B42, Agrrad, Cmap, Cmorph, Stg4... Parameters Landcover, soils, greenness, albedo, LAI, topography, tbot Data Assimilation algorithm, observation, perturbation method

Fractional soil moisture (water+ice) Soil Temperature

NebraskaJAN-JUL 2003

Example Depth-Time Sections

14

SHEELS output time seriesVolumetric soil moisture, 1 Mar 2011 - 21 Apr 2011

Salinas, California

Root zone

0-10cm

Total column (10m)

SHEELS output time series

Precip, Soil Water, ET

AMSR-E (Advanced Microwave Scanning Radiometer for EOS)

NASA Aqua satellite with AMSR-E instrument

AMSR-E retrieved soil moisture for

August 2, 2008 over the SE US  

AMSR-EConically scanning passive microwave radiometer on NASA Aqua polar orbiterMeasures polarized brightness temperatures at 6 frequencies from 6.9 to 89.0 GHzWe use the Level 3 retrieved soil moisture product (Njoku et al. 2003) resampled to a 25-km grid. (Obs twice daily.)Stated accuracy is .06 m3/m3. (Typical range is .05 to .40)Algorithm minimizes differences between the observed brightness temperatures and those generated using a forward radiative transfer model. Due to extensive radio frequency interference in the 6.9 GHZ channel, 10.7 and 18.7 GHz observations are used for soil moisture estimation.  

Background and Observed Variables

SHEELS state variables (x) include temperature and fractional water content in each of 14 layers •6 in the top 10 cm•6 root zone (up to 1.5m) •2 deep layers to 10m.

The observation* (y) (retrieved soil moisture) is the volumetric water content (cm3/cm3) near the surface (exponentially weighted by depth)

Mostly top layer but must account for porosity to convert from FWC to VWC

*Actually a retrieval or estimate but in the context of data assimilation it’s an observation.

q1

q2q3

q4

q5

q6

q7

q8

q9

€

J(x) = y − H (x)( )TR −1 y − H (x)( ) + x − xb( )

TPb

−1 x − xb( )

•The ensemble consists of N model state fields.

•The mean of the N ensemble states is used to define the state vector estimate.

•The spread of an ensemble of N model ‘trajectories’ is used to estimate the error covariances. The full non-linear dynamic equations are used to propagate each ensemble member forward in time, thus determining the trajectories. This is in contrast to the traditional Kalman filter in which linearized model dynamics are used to propagate error covariances.

•When observations are available, each ensemble member is updated based on the difference between the observation and the model state, weighted by the Kalman gain (as in the EKF).

•Random error is added to the observation based on assumed noise characteristics; this ensures that the variance of the updated ensemble matches the true estimation error covariances (Burgers et al., 1998, Mon. Wea. Rev.)

•Propagation of error covariance matrix is more stable than in the traditional Kalman filter, especially if there are strong non-linearities in the model.

Ensemble Kalman Filter in LIS

Domain

Micronet Validation•First validation attempt vs. ARS Micronet (Little Washita River, OK)

•Large bias and variability due to sampling error and soil properties

•Ran artificial experiments with 0.5x and 1.5x rain forcing, validated against Micronet and “truth” run.

•Large errors remain, but DA runs (dashed) tend to converge

•Square-wave error maybe due to day-night bias differences

QuickTime™ and a decompressor

are needed to see this picture.

Bias Correction (CDF Matching)

The dynamic range of AMSR-E observed soil moisture is small relative to that of the model.A correction (right) is applied to convert the observation into a model-equivalent value. A Cumulative Distribution Function (CDF)-matching technique is used here. This is similar in purpose to the bias corrections usually applied to satellite observations in NWP models. Simulations made without the proper correction showed a pronounced dry bias.

0.0 0.1 0.2 0.3 0.4 AMSR-E Observed Soil Moisture

CD

F-C

orr

ecte

d S

oil

Mo

istu

re

0.5

0.4

0.3

0.2

0.1

0.0

AMSR-E Bias Correction

Landcover-dependent CDF Correction

Impact of Land Use CDF Correction

Difference in fractional soil moisture immediately following

assimilation of AMSR-E data at 8 UTC on June 7, 2003: Land Use

CDF minus Uniform CDF simulation. The spatial pattern

reflects the land use type distribution and illustrates the impact of the Land Use CDF

correction on soil moisture data

assimilation.

“Truth”Stage IV

Precipitation Forcing ControlNo Data

Assimilation

Data Assimilation (Combined Bias

Correction)

22Z 24 Jun 2003

The data assimilation adds soil water, particularly in the eastern part of the domain and the Texas panhandle.

No Rain Experiment

“Truth”Stage IV

Precipitation Forcing ControlNo Data

Assimilation

Data Assimilation (Combined Bias

Correction)

09Z 22 Jun 2003

The data assimilation is able to reduce the water content from the false rainfall in Kansas while increasing it in the Texas panhandle.

False Rain Experiment

Time series of top layer soil water in SE Nebraska (point marked by diamond at left). The 3 DA runs are able to match the “true” run closely throughout much of the experiment, although they overestimate soil moisture from about June 7-14.

No Rain Experiment

False Rain Experiment

Time series of top layer (1.6 cm) soil water fraction from 5 runs (false rain scenario) in Texas panhandle (100.5W, 35N). The DA run with the combined bias correction (red line) shows transitions that are less abrupt (reduced amplitude) during data assimilation steps, compared to the other DA runs (blue/green).

No Rain Run

Control (No DA)

Uniform BC Vegetation BC

Combined BC

Bias -0.161 -0.042 -0.041 -0.042

Std. Dev. 0.128 0.130 0.129 0.130

RMS 0.206 0.137 0.136 0.137

Correlation (r2)

0.381 0.444 0.454 0.454

False Rain Run

Control (No DA)

Uniform BC Vegetation BC

Combined BC

Bias -0.015 -0.013 -0.013 -0.014

Std. Dev. 0.171 0.149 0.148 0.148

RMS 0.172 0.150 0.149 0.149

Correlation (r2)

0.090 0.375 0.384 0.391

Results (Statistics)

Summary• AMSR-E Soil Moisture Estimates assimilated into SHEELS LSM using EnKF• Bias correction is necessary for good results.

– Must be regionally and seasonally appropriate. – Landcover- and day/night-dependent corrections implemented.

• Day/night correction alleviates square wave pattern in biases• With hiqh quality forcing data, validation vs. ground truth is difficult

– Variability of rainfall and soil properties within a FOV confound this.– Could use anomaly correlations for verification

• DA improves modeled soil moisture in simulations with intentionally poor rain forcing

• Potentially of greatest use in areas without high-quality continuous rainfall data (radar, gauge networks).– An advantage over microwave (polar orbiting) rainfall observations: It

can see the effects of rainfall after it happens (alleviates sampling issue)

Possible Future Work

• Validation against ground stations by anomaly correlation

• Test impact on a coupled weather forecast model.

• Implement for SMOS and SMAP.– AMSR-E .06 cm3/cm3 accuracy – SMOS (ESA, 2009) .04 cm3/cm3

– SMAP (NASA, 2014) .04 cm3/cm3

1DVAR in MIRS

NOAA STAR algorithm/software package

•1DVAR physically-based retrieval algorithm based on OI theory

•Simultaneous retrieval of temperature, humidity, and hydrometeor profiles in EOF space

•Water vapor and hydrometeors are in logarithmic coordinates

•Assumes local linearity and gaussian pdf of state variable

•Applicable to any sensor combining imaging /sounding capabilities.

•Experimental products of cloud / precipitation liquid and ice have potential for refinement into contributions to GPM.

•Emissivity spectrum is part of the retrieved state vector enabling more direct response to precipitation over land.

Microwave Integrated Retrieval System (MIRS)

MIRS VariablesBackground+3 EOF’s

GraupelWater Vapor

Cloud Liquid

Rain

covariance error model forward : E

covariance error background : B

Xs on based estimates model forward : Y(X)

radiances channel measured : Y

quantity retrieved of background : X

state) sfc or atmos r,hydrometeo (e.g.

quantity lgeophysica retrieved : X

XYYEXYY2

1XXBXX

2

1 J(x)

m

o

m1Tmo

1To

priori a

bservedmeasured/omatchingnotradiancesmodeled

forpenalty

estimatebackgroundfromdeparting forpenalty

4444444 34444444 2144444 344444 21

⎥⎦⎤

⎢⎣⎡ −−+⎥⎦

⎤⎢⎣⎡ −−= −− ))((xx))(()(xx)(

MIRS 1DVAR

TS Lee (Sep 2, 2011)

•Retrievals from AMSU have consistently smaller Rain/Ice amounts compared to TMI

•Are these differences due to sensors or algorithms?

•Test whether improved first guess and background/covariance constraints can give more consistent results

•TRMM and GPM (will) have high resolution measurements but AMSU’s are important for sampling

Intercomparison of AMSU and TRMM Rain Retrievals

MIRS Experiments

Sep 2009 Study

MIRS Graupel

W.Pac. SE US C. Africa

MIRS Rain

• More frequent large ice consistent with high lightning frequency over Africa.

• Strong convection, dry environment.

MIRS Results (Graupel and Rain)