24
CORDEX archiving: documentation, ToU, archives, ESGF Grigory Nikulin Swedish Meteorological and Hydrological Institut

CORDEX archiving: documentation, ToU , archives, ESGF

  • Upload
    darrin

  • View
    60

  • Download
    0

Embed Size (px)

DESCRIPTION

CORDEX archiving: documentation, ToU , archives, ESGF. Grigory Nikulin Swedish Meteorological and Hydrological Institute. CORDEX-ESGF team. publication of CORDEX simulations on ESGF is an joint effort of several European groups with coordination by the IS-ENES2 FP7 project. - PowerPoint PPT Presentation

Citation preview

Page 1: CORDEX archiving: documentation,  ToU , archives, ESGF

CORDEX archiving:documentation, ToU, archives, ESGF

Grigory NikulinSwedish Meteorological and Hydrological Institute

Page 2: CORDEX archiving: documentation,  ToU , archives, ESGF

CORDEX-ESGF team

• Martin Juckes and Stephen Pascoe (BADC, UK)• Ole B. Christensen (DMI, Denmark)• Stephanie Legutke and Stephan Kindermann (DKRZ, Germany)• Sebastien Denvil (IPSL, France)• Grigory Nikulin and Michael Kolax (SMHI, Sweden)• Prashanth Dwarakanath and Torgny Faxén (NSC-LIU, Sweden)• and many more ….

publication of CORDEX simulations on ESGF is an joint effort of several European groups with coordination by the IS-ENES2 FP7 project

Page 3: CORDEX archiving: documentation,  ToU , archives, ESGF

CORDEX archive specifications• authors: Ole B. Christensen (DMI), Bill Gutowski (ISU), Grigory Nikulin

(SMHI) and Stephanie Legutke (DKRZ)• principal maintainer: Stephanie Legutke (DKRZ)• 3 main versions during 2009-2014: original one (2009-2010), pre-ESGF

(2010-2011) and with ESGF-related modifications (2012-2014)• still not approved by SAT• place:

• CORDEX Data Management Wiki (DKRZ): working area, earlier versions, both *doc, *pdf https://madwiki.dkrz.de/farm/CORDEXDataManagement

• ENES CORDEX page, the latest version *pdf https://verc.enes.org/data/projects/cordex

• DMI CORDEX page, the latest version *pdf, http://cordex.dmi.dk/

Page 4: CORDEX archiving: documentation,  ToU , archives, ESGF

CORDEX Variable Requirement Table• variable names, attributes, frequencies etc. • authors: Stephanie Legutke (DKRZ), Ole B. Christensen (DMI) and Grigory

Nikulin (SMHI) • responsible: Stephanie Legutke (DKRZ)• no ocean variables (coupled RCMs in Arctic- and MED-CORDEX) ?• vegetation and biogeochemical variables (dynamical vegetation) ?• more pressure levels for cross sections (now 850, 500 and 200 mb) • place:

• CORDEX Data Management Wiki (DKRZ): working area, earlier versions, both *doc, *xsl https://madwiki.dkrz.de/farm/CORDEXDataManagement

• ENES CORDEX page, the latest version *pdf https://verc.enes.org/data/projects/cordex

• DMI CORDEX page, the latest version *pdf, http://cordex.dmi.dk/

Page 5: CORDEX archiving: documentation,  ToU , archives, ESGF

CORDEX MIP Tables• the same information as in the CORDEX VR table: variable names,

attributes etc. • text files used by Climate Model Output Rewriter (CMOR)• author: Sebastien Denvil (IPSL, France), Jamie Kettleborough (MOHC, UK)• place: PCMDI

• http://www2-pcmdi.llnl.gov/cmor/tables/copy4_of_cmip5-tables/

Page 6: CORDEX archiving: documentation,  ToU , archives, ESGF

CORDEX Terms of Use (1)• author : Martin Juckes (BADC), Colin Jones(MOHC/SMHI), Ole B.

Christensen (DMI) , Michael Lautenschlage (DKRZ) • based on the CMIP5 ToU• approved by the CORDEX SAT (May 2013)• place:

• DMI CORDEX page, (*pdf), http://cordex.dmi.dk/

Page 7: CORDEX archiving: documentation,  ToU , archives, ESGF

• the CORDEX ToU are mainly based on the CMIP5 ToU• both define two groups: “non-commercial research and educational

purposes” and “unrestricted use“ (+ commercial) • all users have to select one of two groups (ESGF)• ToU are defined by individual CORDEX RCM groups (ESGF) and by the

CORDEX domain archives (the same for all RCMs) • the situation is more complex compared to CMIP5 (a combination of two

ToU)• an unrestricted RCM downscales a non-commercial AOGCM: RCM

output is non-commercial by default but there are exemptions (not reflected in the CORDEX ToU )

• Martin Juckes (BADC) has been contacting the non-commercial CMIP5 modelling groups (MIROC, MRI and NICAM (aqua planet) in May 2014):

• MIROC5 is non-commercial but dynamically downscaled data driven by MIROC5 does not inherit the restrictions - a derived product (not documented at moment)

• MRI are hoping that the removal of restrictions on the use of their data will be approved this summer.

• still not so clear with Empirical-Statistical Downscaling (ESD): many different methodologies of different complexity

CORDEX Terms of Use (2)

Page 8: CORDEX archiving: documentation,  ToU , archives, ESGF

CORDEX ESGF coordination issues (1)• excel spreadsheet with information about modelling centres: institution

names (short and long), RCM names, ToU, contacts etc. • collected by sending requests to all known RCM groups• author: Stephanie Legutke (DKRZ) + input: Grigory Nikulin (SMHI)• responsible: Stephanie Legutke (DKRZ)• place:

• IS-ENES: https://github.com/IS-ENES-Data/cordex-doc (*xlsx)• CORDEX Data Management Wiki (DKRZ)

Page 9: CORDEX archiving: documentation,  ToU , archives, ESGF

CORDEX ESGF coordination issues (2)• a set of scripts have been developed (Stephan Kindermann, DKRZ)• conversion of the COREDX ESGF issues excel spreadsheet to different

formats: text lists, html• html table with ToU for different modelling groups as example

Page 10: CORDEX archiving: documentation,  ToU , archives, ESGF

RCM and GCM name lists (CVs)• confirmed RCM names and all CMIP5 GCM names (technical docs for QC)• authors: Ole B. Christensen (RCM) and Martin Juckes (GCM)• contribution (RCM): Stephanie Legutke (DKRZ), Grigory Nikulin (SMHI)• place: http://cordex.dmi.dk/ (text)

Page 11: CORDEX archiving: documentation,  ToU , archives, ESGF

Quality Control

• The CEDA CORDEX Compliance Checker (Martin Juckes, BADC): file names, attributes etc.

http://proj.badc.rl.ac.uk/exarch/wiki/PackageFCC• The DKRZ Quality Checker (Heinz-Dieter Hollweg, DKRZ): file names,

attributes + missing time steps, wrong time, suspicious max/min values ..

https://redmine.dkrz.de/collaboration/projects/cordex/wiki/DKRZ_QC_Tool and a link from the DMI CORDEX page

• list below - what has to be checked before publication on ESGF• authors: Stephanie Legutke (DKRZ) with input from other groups• place: ENES CORDEX page https://verc.enes.org/data/projects/cordex DMI CORDEX page http://cordex.dmi.dk/

Page 12: CORDEX archiving: documentation,  ToU , archives, ESGF

IS-ENES2 Datanode Administrator’s Reference Manual

• clear instructions how to publish CORDEX simulations on ESGF (consistent across all IS-ENES2 ESGF datanodes)

• author: Prashanth Dwarakanath (NSC-LIU)• place: IS-ENES GitHub

• https://github.com/IS-ENES-Data/datanode-mgr-doc

Page 13: CORDEX archiving: documentation,  ToU , archives, ESGF

CORDEX archives (not ESGF)Med-CORDEX (MED):• http://www.medcordex.eu/ users should be approved ToU – “non-commercial only” for all simulations (web)

East Asia (EAS) • http://cordex-ea.climate.go.kr/ register and download ToU – “non-commercial only” for all simulations (pdf doc)

South Asia (WAS) • http://cccr.tropmet.res.in/cordex/ users should be approved ToU – “non-commercial only” for all simulations (web)

CCCma (Canada) • http://www.cccma.ec.gc.ca/data/canrcm/CanRCM4/

register and downloadToU – “unrestricted”only CanRCM4: AFR, ARC, EUR, NAM, both 0.22 and 0.44deg

Page 14: CORDEX archiving: documentation,  ToU , archives, ESGF

CORDEX-ESGF in operation • CORDEX-ESGF is in operation since mid September 2013 • now ESGF provides the same interface for both global (CMIP5)

and regional (CORDEX) climate simulations• 542 users in the CORDEX-ESGF group (12 May 2014)

Page 15: CORDEX archiving: documentation,  ToU , archives, ESGF

CORDEX ESGF archive

At moment (May 2014) full CORDEX support only on the European ESGF index nodes:• esg-dn1.nsc.liu.se (SMHI-NSC, Sweden) • cordexesg.dmi.dk (DMI, Denmark)• esgf-data.dkrz.de (DKRZ, Germany)• esgf-index1.ceda.ac.uk (BADC, UK)• esgf-node.ipsl.fr (IPSL, France)

• It doesn’t matter from which of the index nodes CORDEX simulations can be accessed, a federative system, no central archive, links lead to the same files

• All CORDEX simulations are visible on almost all ESGF index nodes [ thanks to Prashanth Dwarakanath (NSC-LIU) ]• Not all ESGF index nodes support the full CORDEX search: the

CORDEX specific search facets “Domain”, “Driving model”, “Downscaling realisation”

Page 16: CORDEX archiving: documentation,  ToU , archives, ESGF

CORDEX ESGF datanodes (12 May 2014) Datanode Number of

DatasetsNumber of CORDEX domains

esg-dn1.nsc.liu.se SMHI, NCS-LIU, Sweden

27804 AFR-44, ARC-44, EUR-11, EUR-44, MNA-44, MNA-22, NAM-44, SAM-44, WAS-44

cordexesg.dmi.dkDMI, Denmark

5394 AFR-44, ANT-44, EUR-11, EUR-44, NAM-44

carbon.dkrz.deDKRZ, Germany

3324 AFR-44, EUR-11, EUR-44, WAS-44

esgf-data1.ceda.ac.ukBADC, UK

1004 AFR-44, AUS-44, EUR-44, EAS-44

data.meteo.unican.esUniv. Cantabria, Spain

228 EUR-22, EUR-44

esgf.extra.cea.frCEA, France

196 EUR-11, EUR-44

noresg.norstore.uio.noNorStore, Norway

- published EUR-44 and EAS-44 earlier, not available now

• DHMZ (Croatia), UQAM (Canada) and CNRM (France) are waiting at DMI (QC) • one dataset: one variable, experiment, domain, frequency• full CORDEX output for one simulation (Core+Tier1): 235 datasets

Page 17: CORDEX archiving: documentation,  ToU , archives, ESGF

CORDEX Domains on ESGF (12 May 2014)

• 10 CORDEX domains are available (different number of simulations) • a RCM group publishes simulations on its own datanode (if available)• where to publish is decided within the CORDEX domains: ESGF non-ESGF EUR-11 to DKRZ and EUR-44 to DMI (Euro-CORDEX)

CORDEX Domain Number of Datasets Number of groups

Europe (44, 22, 11) 6091/113/3894 8/1/5

Africa (44) 6234 4

North America (44) 1752 2

Arctic (44) 2718 1

MENA (44, 22) 1551/700 1/1

South Asia (44) 830 2

South America (44) 564 1

Antarctica (44) 604 1

Australia (44) 171 1

East Asia (44) 171 1

Page 18: CORDEX archiving: documentation,  ToU , archives, ESGF

RCM groups on ESGF (12 May 2014)CORDEX Domain Number of Datasets Number of modeling groups

EUR-44, -22, -11 6091/113/3894 8/1/5 (7 RCMs)

AFR-44 6234 4

NAM-44 1752 2

ARC-44 2718 1

MNA-44, -22 1551/700 1/1

WAS-44 830 2

SAM-44 564 1

ANT-44 604 1

AUS-44 171 1

EAS-44 171 1

• post-processing is more complex than expected (QC, a few iterations)• not only for new RCM groups using WRF and RegCM • WRF NetCDF Extract&Join – a common tool for WRF (still in development) http://www.meteo.unican.es/wiki/cordexwrf/SoftwareTools/WrfncXnj• RegCM - ?

Page 19: CORDEX archiving: documentation,  ToU , archives, ESGF

What is actually available on ESGF ?• an automatically generated list with all CORDEX simulations visible on

ESGF (Stephan Kindermann, DKRZ)• an example, can be adjusted

Page 20: CORDEX archiving: documentation,  ToU , archives, ESGF

Euro-CORDEX downloads (EUR-44)

Most popular variables: precipitation, mean, maximum, minimum temperature and wind

Page 21: CORDEX archiving: documentation,  ToU , archives, ESGF

Euro-CORDEX downloads (EUR-11)

Most popular variables: precipitation (favourite), mean, minimum, maximum temperature and specific humidity

Page 22: CORDEX archiving: documentation,  ToU , archives, ESGF

Africa-CORDEX downloads (AFR-44)

Most popular variables: precipitation, minimum, mean, maximum temperature and wind

Page 23: CORDEX archiving: documentation,  ToU , archives, ESGF

MENA-CORDEX downloads (MNA-44)

Most popular variables: minimum temperature, precipitation, mean, maximum temperature and wind

• User needs: minimum temperature is important for agriculture

Page 24: CORDEX archiving: documentation,  ToU , archives, ESGF

Downloads per CORDEX domain

EUR-11 – hi-res Euro-CORDEX is a clear favourite