Interfaces EO data with Atmospheric and Land Surface Model: Progress

report 2

Liang Feng, Paul Palmer

A. Reference OSSE systemB. High-resolution modelC. CO:CO2 ratios

Project Tasks

I. Progress overviewOverall Aim: Developing a reference OSSE simulation for community researchers to evaluate impacts of space-borne atmospheric composition measurements on surface flux estimates. Outputs:

1). OSSE framework with flexiable module and libraries. Status: Done.

2). One complete example OSSE system for 1) simulating OCO-like XCO2 observations; and 2) estimating regional CO2 fluxes by assimilating XCO2 observations using an Ensemble Kalman Filter (EnKF).

Status: Done . 3. Detailed documents. Done. 4. EOF and Visualization Tools. Done.

Observing System Simulation Experiment Tool

II. Download and Installation Codes and documents:

1) Packages for PyOSSE and EOF visualizations can be downloaded from http://xweb.geos.ed.ac.uk/~lfeng 2) Installation guides has also been provided, covering how to:

• unzip archive; • set python search path;• build shared libraries;• test codes.

3) documents and references can be found on-line or from local directory.

Data• Being required mostly by the example OSSE system.• Covering climatology for cloud, aerosol PDF , satellite orbit,

instrument averaging kernel, and GEOS-Chem model outputs. • Being stored as ASCII text files, netCDF files and BPCH files in

www.esa-da.org/data/otool_data.tar.gz.

III. PyOSSE framework

Data flow and directory structureClasses IO ModulesConfiguration files

Obs+random errors

Flux Forecasts

Obs operator

Forecast(3-D concentrations)

CTM

Prio

r + e

rror Posteriori +

error

Observation Simulation EnKF

assimilation

ETKF Model ObsEnsemble

Ensemble forecasts(3-D fields)

Surface flux Ensemble CTM

Obs operator

Data flow and directory structure

observation (sample/convolve)

instrument (ORB, AVK. CLD …)

example (OSSE for OCO)

surface (flux ensemble)

atmosphere (CTM outputs)

util (lib)

enkf (stv and etkf solvers)

PyOSSE package has over 90 python/fortran modules, stored in 8 subdirectories:

Classes:

We have defined about 40 python classes as containers for data and functions of model objects

Measure

Transport

Top-down estimate

Atmosphere

Observation

Surface flux

Flux modelinventory

Met files

class ens_stat_cl: Class for state vector used to assimilate observations Members: (total of 44) ------------------------------------------------------------- 1. attr_dict:<dict>: attributes 2. step_tag_lst:<list>: ID for run step # variables for current assimilation window 3. wnd_mean_x0:<array, (wnd_nx)>: prior coefficient values for lag window 4. wnd_mean_x:<array, (wnd_nx)>: posterior coefficient values for lag window 5. wnd_dx:<array, (wnd_nx, wnd_ne)>: perturbations ensemble for coefficients 6. wnd_inc_m:<array, (wnd_ne>: increment matrix for ETKF data assimilation 7. wnd_xtm: <array, (wnd_nx, wnd_ne)>: transform matrix for ETKF data assimilation 8. wnd_xinc: <array, (wnd_nx)>: increment for ETKF data assimilation # current analysis increments 9. inc_m:<array, (wnd_nx)>: increment matrix for current step 10. xtm: <array, (wnd_nx, wnd_ne)>: transform matrix for current step 11. xinc: <array, (wnd_nx)>: increment for ETKF data assimilation

Functions (total of 14) ========================================================== 1. __init__: initialization 6. construct_tcor_matrix: Create temporal correlation matrix 7. add_new_x_to_window: Add new apriori to assimilation window 10. do_assim: assimilate observations and update state

Example: class for state vector

Configurable IO modules• PyOSSE relays on many pre-defined data sets such as satellite orbit,

instrument sensitivity, and cloud PDF etc. • PyOSSE needs mechanism to exchange data with CTM. • Data is usually given in different formats, and often needs pre-

processing. • Configurable IO modules are designed as bridges between host

classes and data files.• These modules are easy to be re-configured or even be replaced.

Var-listVar-dictfdescfopenfclosefwritefread

Def-Var-ListDef-Var-dict Create-fdesc

OpenfileReadfilewritefileClosefile

.

data

Host class IO module Disk files

Configuration filesWe have developed a new type of text-based menu files to:

1. Numeric or string parameters as function inputs (traditional).2. Object as parameters such as list, functions (new) 3. Object creater (new)

#MENU (cloud)#================================================name|cloud path|$DATAPATH$/clim_dat/flnm|meancloud_frac_XMONTHX_landsea.datdict|__load:$MDPATH$.cloud_file_m:cld_varname_dictfopen|__load:$MDPATH$.cloud_file_m:open_cloud_filefread|__load:$MDPATH$.cloud_file_m:read_cloud_filefclose|__load:$MDPATH$.cloud_file_m:close_cloud_filefget|__load:$MDPATH$.cloud_file_m:get_cloud_datakeywords|__dict:Nilfclass|__load:$MDPATH$.cloud_m:cloud_cl#MEND

Example: vob_def.cfg for observation simulations allows users to choose classes for satellite orbit, averaging kernel, cloud/AOD PDF , and sampling etc

#MENU (data random sampling)#=======================================================name|sample# cloud sampling fcld_sample|__load:$MDPATH$.cloud_m:cloud_samplefcld_penalty|Nonefcld_keywords|__dict:Nil

# AODfaod_sample|__load:$MDPATH$.aod_m:aod_samplefaod_uplimit|0.3faod_keywords|__dict:Nil

# landcoverflc_sample|__load:$MDPATH$.landcover_m:sample_lcflc_stype_lst|__load:$MDPATH$.ak_file_m:ak_lc_stype_lstflc_keywords|__dict:Nil

#MEND

Menu term (sample) in vob_def.cfg

Menu term (averaging kernel ) in vob_def.cfg

#MENU (averaging kernel)#===================================================== name|ak path|$DATAPATH$/oco/oco_ak/flnm|aknorm_XSURFACEX.XEXTXviewmode_dict|__load:$MDPATH$.ak_file_m:def_ak_viewmode_dictstype_dict|__load:$MDPATH$.ak_file_m:def_ak_surf_type_dictfopen|__load:$MDPATH$.ak_file_m:open_ak_filefread|__load:$MDPATH$.ak_file_m:read_ak_filefclose|__load:$MDPATH$.ak_file_m:close_ak_filefget|__load:$MDPATH$.ak_file_m:get_ak_datakeywords|__dict:Nilfclass|__load:$MDPATH$.ak_m:ak_cl#MEND……

IV. OSSE Example system

Observation simulation Inversion

Observation simulation (example/obs_simulation):

1. Generate dummy observationby using gen_dummy_obs.py (and vod_def.cfg)

2. Generate model observationby sampling actual model outputs (gen_sat_obs.py and sob_def.cfg)

#MENU (class for model profile)#================================================name|fc_prof# model output path|$DATAPATH$/gc_std/# file name (empty to use defaults in )flnm|'ts_satelliteXEXTX.XYYYYXXMMXXDDX.bpch'fdiaginfo|'diaginfoXEXTX.dat' ftracerinfo|'tracerinfoXEXTX.dat' vname|single_profileext|'.ST001.EN0001-EN0002'# output access fopen|__load:ESA.util.gc_ts_file_m:open_ts_filefread|__load:ESA.util.gc_ts_file_m:setup_daily_profilefget|__load:ESA.util.gc_ts_file_m:get_mod_gpfpres|__load:ESA.util.gc_ts_file_m:get_mod_presfclass|__load:ESA.atmosphere.ts_slice_m:gc_slice_cl

#MEND

GEOS-Chem outputs can easily be replaced by user’s CTM simulations by changing sob_def.cfg

Inversion (example/obs_simulation): 1. Settings: default settings are defined in osse_def.cfg, configuring classes for state vector, flux, perturbation ensemble, prior flux, observations, model forward simulation, and model outputs sampling etc.

#MENU (CTM for single tracer run)name|ctmrunpath|$FCRUN_ROOT$datapath|$FCRUN_DATAPATH$tra_st|1tra_end|2finput_gen|__load:ESA.example.enkf_oco.input_geos_gen:create_new_input_fileext|ST001.EN0001-EN0002runtype|3runscript|rungeos.shfclass|__load:ESA.example.enkf_oco.run_geos_chem:geos_chem_cl#MEND

Example: CTM defined in osse_def.cfg

2) Results:

We have a rich set of outputs for inversion results as well as diagnostic data

For example: Posterior fluxes

For example: Posterior model XCO2

V. EOF and visualization

The pyeof module

Visualization package

pyeof• Pyeof defines a class eof_cl to find EOFs for temporal and spatial

data sets. • It uses SVD technique to find eigenvalues and eigenstate for

covariance matrix. • It is easy to use, and also provides functions for visualization of

EOFs and PCs

EOF1

Visualization

• We have developed a wxpython-based GUI application for quick visualizations including 2D plots and animations.

• It also has a python shell to use a rich set of existing visualization tools.

• We have also provided functions for common data processing.

VI Conclusion OSSE tool and visualization package are ready for public

release.

This framework is simple and flexible, due to the adopted algorithm (EnKF), and the way to implement it (classes, configurable IO modules, and enhanced menu functions …).

It has been successfully used to digest GOSAT observations.

We are making further developments.

B. Hi-resolution modeling

I. Motivation Interpret and digest future satellite (like OCO-2) observation of

atmospheric CO2 concentrations: Model errors (transport and representation errors) have significant

impacts on top-down estimates of surface fluxes. Higher model resolution usually means better transport modelling, and

smaller representation errors.

Provide benchmarks for coarser simulations or nested model simulations.

Itself is one by-product of our efforts to develop EnKF approach based on nested GEOS-Chem transport model.

In-cooperate other process-based models (such as plume model) into GEOS-Chem (Gonzi et al, University of Edinburgh).

Due to the finer temporal and spatial resolutions, It is more sensitive to

the details of ‘small scale’ processes.

II. Challenges:

Availability of High-resolution emission inventories

• 1x1 monthly fossil fuel emissions (Oda et al).

• 1x1.25 3-hourly CASA biospheric CO2 exchanges (R. Kawa , J. Collatz, and D. Liu)

• 1x1.25 daily biomass burning (R. Kawa , J. Collatz, and D. Liu)

• 4x5 monthly oceanic surface CO2 flux (Takahashi et al)

Programming issues

1. Memory:

In particular, stack size is limited by most of fortran compiler. Modifications have been made over GEOS-Chem v8.02 to limit the use of common block and avoid static arrays in modules. 2. Parallelisation:

Mainly we have to reschedule parallelisation in tpcore modules, and change functions in dao_mod.f .

3. Minor issues: Mainly they are associated with diag outputs.

Results

Comparison with standard GEOS-Chem 4x5 simulations

CO2 /ppm (ML1-8) CO2/ppm (ML1-8)

Range: 374 – 415 ppm Range: 377 ppm – 403 ppm

Comparison with nested simulation

Global 0.5x0.666 Nested 0.5x0.666

Nested model being used in UK GAUGE project

CO2 /ppm (ML1-4) CO2 /ppm (ML1-4)

Comparison with GEOS-Chem drived by ECMWF winds

GEOS-5 winds ECMWF winds

ECMWF Met fields may have stronger vertical transport across boundary layer.

CO2 /ppm (ML1-8) CO2 /ppm (ML1-8)

Comparison to averaged aircraft profiles

TABATINGO SANTAREM

RIO BRANCO ALTA FLORESTA

N = 11 N = 7

N = 11 N = 11

--- GEOS-5 PBL

Summary Native resolution simulation provides:

a) More realistic descriptions of observed variations

b) Benchmark for lower-resolution model

c) Framework for including detailed process models

Our comparisons also reveal some possible model errors.

We are now working on flux inversions for selected regions • We have done some OSSEs for Astrium Project • We are working on digesting ICOS data.

Spare slides

Top-down Optimal surface flux estimation

x: regional surface fluxesyobs: measurements of atmospheric concentrations. H: Jacobian (CTM model).

)( fobs

fa HxyKxx

K=PfHT(HPfHT+R)-1 – Kalman gain matrix.Pf: a priori uncertainty matrix R: observation error matrix

Posteriori Priori gain observation model

Data Assimilation Module (Ensemble Kalman Filter)

Ensemble Approach

Key features: (Feng et al., 2009 ; 2011)• Represent a-priori uncertainties by an ensemble of flux

perturbations. )T

• Use a lag windows to limit computational costs. Any emission will only be constrained by observations within a following limited time period. After that period, it is considered to be well-known.

• Project the ensemble of perturbations together with prior estimates into the observation space using CTM.

• Use Ensemble Transform Kalman Filter (ETKF) to determine posterior fluxes, and the associated uncertainties from digesting observations.

A. State vector and ensemble representation of its uncertainties

),,(),,(),,( 0 tyxBFctyxFtyxF l

ll

Surface fluxes Prior estimates of surface fluxesBasis functions for pulse-like flux perturbations,

:coefficients to be estimated: State vector: , , , …, ]; Perturbations: , , , …, ]Error covariance: Full or partial representation: , , , …, ],

We use region_map_c.py to defined as 8-day surface CO2 flux perturbations over 144 global regions.

We use gen_region_pb_flux_m.py to generate BFs from biospheric GPP map and multi-layer maps for 144 regions.

B. Ensemble forecasts

• We change IO function in pb_flux_c.py to generate the ensemble of flux perturbations according to the defined basis functions and pre-defined uncertainties.

• Functions in ctm_config_c.py and ctm_restart_c.py are override to support GEOS-Chem tagged runs to project flux perturbation ensemble to the ensemble of atmospheric tracer concentrations.

C. Projection of ensemble forecasts to the observation space (ctm_world_c.py)

D. Inversion algorithm:Ensemble Transform Kalman Filter (ETKF)

• A posteriori and the associated uncertainties are simultaneously calculated using SVD (or sparse-matrix LU) technique.

+R]-1

TTTT=[1+ -1

• Corresponding updates are made to the ensemble of model 3D concentrations to retain the contributions of fluxes outside the assimilation (time) windows (Feng et al, 2009)

Algorithm testing: Comparison with the LSCE 4d-var system

In the comparison experiments, we have assimilated the same ACOS-GOSAT B210 XCO2 retrievals over 2009 and 2010, but using different approaches: EnKF (UoE) and 4d-var (LSCE)