Interfaces EO data with Atmospheric and Land Surface Model: Progress

report

Liang Feng, Paul Palmer

Reference OSSE system High-resolution model CO:CO2 ratios

Project Tasks

I. Progress overview

Overall Aim: Developing a reference OSSE simulation for community researchers to evaluate impacts of space-borne atmospheric composition measurements on surface flux estimates.

Outputs:A. OSSE framework with flexible modules and libraries.

Status: Revised and ready for beta-test.

B. One complete example OSSE system for 1) simulating OCO-like XCO2 observations; and 2) estimating regional CO2 fluxes by assimilating XCO2 observations using an Ensemble Kalman Filter (EnKF).

Status: close to be finished .

C. Detailed documents. Status: beta version due in Dec, 2012.

D. Visualization Tools. Status: to be started.

Observing System Simulation Experiment Tool

II. OSSE framework.

III. Example OSSE system

Today we will focus on:

II. OSSE framework

Data flow and directory structureMain modulesStyles

Obs+random errors

Flux Forecasts

Obs operator

Forecast(3-D concentrations)

CTM

Pri

or

+ e

rror Poste

riori +

erro

r

Observation

Simulation

EnKF assimilatio

n

ETKF Model ObsEnsemble

Ensemble forecasts(3-D fields)

Surface flux Ensemble CTM

Obs operator

Model world: Object, process, measurements, and estimates.

Measure

Transport

Top-down estimate

Atmosphere

Observation

Surface flux

Flux modelinventory

Met files

Disk IO

observation

ctm

lib

surface_flux

atmosphere

etkf

instrument

We have hence constructed a directory structure for OSSE tools as:

Main modules

Module name Description

etkf_m.py Calculate a posteriori ; gain matrix (K); transform matrix (T) , by using ETKF algorithm.

state_vectort_c.py Class for state vector and its projection to surface fluxes: a). construct, manage, and update state vectors and their perturbation ensembles. b). maintain a (virtual) projection to surface fluxes.

construct_error_cor_m.py Construct (spatial) error correlations from regional flux perturbation (basis) functions

contruct_state_vector_m.py Interface for user to construct state vector from flux perturbation (basis) functions

1. etkf

Module name Description

ctm_grid_c.py Class for tempo-spatial grid for modelling atmospheric compositions.

ctm_field_c.py Class for gridded model data of single atmospheric tracer. It also contains horizontal and vertical interpolation functions etc.

ctm_slice_c.py Class for tracer profiles along sampling track etc.

ctm_world_c.py General container for tracer data ( field or slice). It provides: a) member management (add, group, remove, update, search, retrieve etc) ; b) (virtual) disk IO.

Io_file_c.py Class for helping to access CTM outputs, it helps to :1) group file names for parameters to be read. 2) finalize extensions for file names.

ctm_file_read_m.py Interface for reading data from CTM outputs

ctm_file_write_m.py Interface for writing data to be read by CTM.

2. atmosphere

Module name Description

surface_grid_c.py Class for tempo-spatial grid for surface fluxes.

surface_flux_c.py Class for gridded surface flux. It also contains functions for: a) disk IO; b) re-gridding.

flux_pb_c.py Class to contain perturbation (basis) flux maps. It has a virtual disk IO for flux files used to force CTM.

region_map_c.py Class for generating multi-layer region maps according to user’s choices.

gen_region_pb_flux_m.py construct class flux_pb_c from multi-layer regional maps and the map for flux errors over the world.

3. surface_flux

Module name Description

observation_c.py Class for observations. It also contains functions to a) read (write) observations from/to disk files; b) filter and weight observations for assimilation.

ctm_observation_c.py Class for sampling model outputs at observation time and locations.

ens_observation_c.py Class for sampling ensemble model outputs at observation time and locations.

4. observation

Module name Description

ctm_config_c.py Class to generate configuration file to control CTM forecast simulations.

ctm_restartfile_c.py Class to (re-) construct CTM restart files.

5. ctm

Module name Description

sat_orbit_c.py Class for generating (reading ) satellite orbits.

sat_avk_c.py Class for generating satellite averaging kernels from pre-calculated tables.

sat_oerr_c.py Class for generating satellite observation errors from pre-calculated tables.

check_cloud_m.py Functions for checking the probabilities of cloud contaminations along satellite observation tracks from given cloud cover (probability) map and the viewspot penalty function

check_aerosol_m.py Function for calculating AOD along satellite tracks from given AOD probability maps.

6. instrument

Module name Description

time_m.py Functions for time conversions.

gp_axis_c.py General class for axis. It has functions for indexing and interpolating etc

error_message_m.py Record and print error or warning message

fort_lib.f90 Functions for (faster and parallelized) array operations, and equation solving.

7. lib

Styles

1. Simple, short and stable

2. Object - and task –orientation.

Only use stable python features and well-established packages

Example: ctm_field_c.py: class ctm_field: #members

obj_id:<integer>: id of the class instance. name:<string>: name of the tracerunit: <string>: unit of the datagrid: <ctm_grid_c>: grid. data:<array>: gridded dataid:<integer>: id of the tracer. category: <string>: category of the tracergroup: <string>: tracer grouptime_tag: <string>: time step for the data __attr_dict:<dictonary>: dictionary of the attribute

# functions (12 in total) def get_profiles(self, lon_lst, lat_lst, time_lst=None):

For example: by writing less than 30 lines, we can use class region_map to generate a complicated map for estimating regional fluxes at different spatial resolutions.

3. Easy to use

4. Easy to change/expand

Example: In ctm_world_c.py, efforts have been made to minimize the efforts needed to communicate with user’s CTM via disk files. The user now can read their model input/outputs by overriding member function read_ctm_to_fields in class ctm_world, or by overriding the interface module ctm_file_read_m.py.

III. Example OSSE system

Aim: use modules in OSSE framework to build a complete OSSE system for OCO-like measurements (Feng et al, 2009).

The system uses GEOS-Chem (which is, however, not part of the delivered package) as the chemistry transport model.

Obs ETKF

Flux Forecasts(climatology)

Obs operator

forecast

GEOS-Chem

Model ObsEnsemble

Ensemble forecasts

Surface CO2 Ensemble GEOS-Chem

Obs operator

Pri

or

+ e

rror Poste

rior +

erro

r

1) Observation simulation

2) Data Assimilatio

n(+Perturbations)

Observation Simulation module

A. Surface Fluxes We use CO2 flux climatology as ‘the true fluxes’. We provide interfaces to store them into binary files for

GEOS-Chem CTM to read.

B. Chemistry Transport Model Functions in ctm_restart_c.py and ctm_config_c.py are

changed (overridden) to generate input.geos and restart files for GEOS-Chem forward simulations.

C. Sampling• We override IO interfaces so that we can use class

ctm_world to read and sample 3D model fields along the satellite orbits.

• We used cloud and aerosol climatology files and the default algorithm as the inputs to screen cloud and aerosols.

D. Vertical integration/conversion• A standard module is used to convert model vertical profiles

to satellite retrievals, by applying the averaged kernels read from pre-calculated tables for OCO-like instrument.

Example: Simulating OCO XCO2 1) Sampled along Aqua

orbitsGEOS-Chem transport model: Biosphere (CASA), Biomass (GFED), Fossil (+Bio) fuel (ODIAC), Ocean (Takahashi)

2) Scenes with cloud or AOD > 0.3 removed

Jan

1-day

3) Averaging kernels applied

Pre

ssu

re [

hPa

]Glint mode

Averaging kernels

Top-down Optimal surface flux estimation

x: regional surface fluxesyobs: measurements of atmospheric concentrations. H: Jacobian (CTM model).

)( fobs

fa HxyKxx

K=PfHT(HPfHT+R)-1 – Kalman gain matrix.Pf: a priori uncertainty matrix R: observation error matrix

Posteriori Priori gain observation model

Data Assimilation Module (Ensemble Kalman Filter)

Ensemble Approach

Key features: (Feng et al., 2009 ; 2011)• Represent a-priori uncertainties by an ensemble of flux

perturbations. )T

• Use a lag windows to limit computational costs. Any emission will only be constrained by observations within a following limited time period. After that period, it is considered to be well-known.

• Project the ensemble of perturbations together with prior estimates into the observation space using CTM.

• Use Ensemble Transform Kalman Filter (ETKF) to determine posterior fluxes, and the associated uncertainties from digesting observations.

A. State vector and ensemble representation of its uncertainties

),,(),,(),,( 0 tyxBFctyxFtyxF l

ll

Surface fluxes Prior estimates of surface fluxesBasis functions for pulse-like flux perturbations,

:coefficients to be estimated: State vector: , , , …, ]; Perturbations: , , , …, ]Error covariance: Full or partial representation: , , , …, ],

• We use region_map_c.py to define 144 global regions.

• We use gen_region_pb_flux_m.py to generate BFs from biospheric GPP map and the multi-layer map for the 144 regions.

B. Ensemble forecasts

• We override IO functions in pb_flux_c.py to generate the ensemble of flux perturbations according to the defined basis functions and pre-defined uncertainties, and save them in GEOS-Chem bpch2 format.

• We also override functions in ctm_config_c.py, and ctm_restart_c.py to support the GEOS-Chem tagged runs for projecting flux perturbation ensemble to the ensemble of atmospheric tracer concentrations.

C. Projection of ensemble forecasts to the observation space.

GEOS-Chem outputs are sampled by ctm_world_c.py with over-ridden IO functions.

D. Inversion algorithm:Ensemble Transform Kalman Filter (ETKF)

• A posteriori and the associated uncertainties are simultaneously calculated using SVD (or sparse-matrix LU) technique.

+R]-1

TTTT=[1+ -1

• Corresponding updates are made to the ensemble of model 3D concentrations to retain the contributions of fluxes outside the assimilation (time) windows (Feng et al, 2009)

Algorithm validation: Comparison with the LSCE 4d-var system.

In the comparison experiments, we have assimilated the same ACOS-GOSAT B210 XCO2 retrievals (real data) over 2009 and 2010, by using 2 different approaches: EnKF (UoE) and 4d-var (LSCE) (Feng et al, 2013).

VI. Summary

• OSSE system will be issued for beta test soon.

• Whole package, including detailed documents will be delivered in next march.

INTERNAL DEADLINES

1) Deliver revised code and documentation to Paul end of November 2012. This is before the Reading meeting. I will check the code and documentation. Return to LF a week later.

2) Circulate to Silvia, Annemarie, and Siegfried the week of the 10th December 2012. Start beta testing.

3) End of beta testing period at the end of January 2013.

4) Debug, edit, package, and produce online material by the week of the 21st February 2013.

5) Final edits and release the end of February.

We are about two weeks behind the internal deadline, but months ahead the project schedule.