Eric Guilyardi (LOCEAN/IPSL and Univ. Reading)

and the Metafor team

Common Metadata for Climate Modelling Digital Repositories

IS-ENES kick-off meetingParis, 30-31 March 2009

Outline• What is Metafor• What we want to do – objectives• How we are doing it• What we have done– Standards development– “Common Information Model” (CIM)– Controlled vocabulary

• What we are planning to do– CMIP5 metadata

• Links with IS-ENES

Where Metafor came from• PRISM project (FP5 2001-2004), ENES• PRISM Sustained Initiative (PSI)

– Code coupling and I/O– Integration and modelling environments – Data processing and management – Meta-data standards (key !) – Computing issues

MetaforIS-ENES

IS-ENESIS-ENES

IS-ENES, Metafor

ENES coordination

Facts and Figures

11 partnersEU contribution of 2.2M€Started March 2008, duration 3 years

• NCAS, University of Reading, UK (Coordinator)• BADC, Science and Technology Facilities Council, UK• CERFACS, France• Models and Data, Max Planck Institute for Meteorology,

Germany• Institute Pierre-Simon Laplace, CNRS, France• University of Manchester, UK• Met Office, UK• Administratia Nationala de Meterologie, Romania• Météo France, CNRM, France• CLIMPACT, France• CICS, Princeton University, USA

INFRA-2007-1.2.1 Scientific Digital Repositories

Metafor objectivesCreate a standard metadata Common Information Model (CIM) to describe climate data and the models and experiments that produced those data

• Allows essential data, model and experiment distinctions to be understood

• Builds on existing metadata standards used internationally in climate (CF, CDML, CSML, Curator, NMM, FLUME, etc.)

• Uses existing format and framework (XML, RDF, etc.)

Metafor objectivesDevelop, deploy, and evaluate a prototype infrastructure that will allow key data and models to be discovered and compared between distributed digital repositories

• single sign-on services to populate and manipulate, the CIM metadata

• services exploit NDG CSML to provide a common Geographic Markup Language interface to climate data

• centralized CIM content harvested from individual repositories using OAI-PMH (Open Archives Initiative Protocol for Metadata Harvesting).

Metafor Work plan

Metafor activities and work packages (WP) map onto the I3 structure.

Project management, training and dissemination are organised in WP1 and WP7.

Metafor standards activity

Aim - metadata encompassing the entire modelling process

Guiding Principles for metadata

• integration of existing standards (ISO, climate modeling community,...)• flexibility to support emerging standards both from within Metafor as

well as from the broad community• maintaining the “separation of concerns” (modularity)• providing clear governance policies

CIM development strategy

Describe climate data and the models and experiments that produce it:• Premise :

• Domain analysis is captured in a formal model preceding, and driving as much as possible, implementations

• Argument : • Analysis can be done at the right level of abstraction• Applications derived from a conceptual view are more robust than software developed directly on a specific technology• Platform independent representation supports multiple platforms • Revisions flow through to all platforms automatically

• Goal :• One normative artifact – the UML model• Derived XSD generated automatically

Conceptual Model

Climate ModellingAn activity using a software to produce data to be archived in a repository.

UML

Application Model

Application Model

XSD

RDF

Instance @ PCMDI

XML

An essential aim of Metafor is that the conceptual model is not changed by the manner in which it is used or applied.

Instance @ IPSL

Instance @ BADC

e.g. CIM

e.g. CMIP5

CIM v1.0schematicview

Software pkg

Grids pkg

Data pkg

Activity pkg

CIM v 1.0 available on the Metafor website at: http://metaforclimate.eu/trac/browser/CIM

Using the CIM to support CMIP5

• The CMIP5 experimental archives will be ~500TB of model run data

• We need to be able to capture all the details of these experiments (and the component models and platforms used) to allow users of the archive to differentiate between the experiments and the models.

• To do this, Metafor has been tasked by WGCM/CMIP to produce a questionnaire to capture the model metadata.

Controlled vocabulary for “activity/numerical experiment”

Controlled vocabulary for “activity/boundary condition and forcing”

+ flow of specifications

Controlled Vocabulary for Software

QuickTime™ and a decompressor

are needed to see this picture.

Current work: CMIP5 questionnaire

Target date: July 09

Intensive community testing planned

Projects coordinated from the design stage via ENES Metafor will support use of CIM by IS-ENES IS-ENES will promote use of CIM and use/support CIM tools

developed in Metafor (SA2, JRA1, JRA4) Strong links via many common PIs (and advisory board) 2 years in common

IS-ENES & METAFOR

MetaforIS-ENES

2008 2009 2010 2011 2012

Metafor highlights after one year

• CIM development strategy proposed, including conceptual level and meta-model

• CIM v1.0 delivered, freely available at http://metaforclimate.eu/trac/browser/CIM

• Strong international collaboration and links established with USA colleagues in Curator/ESG/PCMDI

• Leading the CMIP5 metadata collection• Very active group of experts and inclusive mail list (~100/month)• Community buy-in growing – future wide-range dissemination

planned to tie in with CMIP5 questionnaire and AR5

The open standard developed in Metafor will play a catalytic role in the way next generation climate data repositories, such as IPCC AR5, are organised, preserved and accessed.

More on: http://metaforclimate.eu

Project structure

ECEC

WP2WP2(Univ. Reading)(Univ. Reading)

Project CoordinatorProject Coordinator(Univ. Reading)(Univ. Reading)

Project ManagerProject Manager

WP6WP6(MPI-M M&D)(MPI-M M&D)

WP4WP4(BADC)(BADC)

WP5WP5(Met Office)(Met Office)

WP3WP3(IPSL)(IPSL)

SME interactionsSME interactionsand disseminationand dissemination

activitiesactivities

Project ExecutiveProject ExecutiveScientific and Scientific and

Technical Advisory Technical Advisory BoardBoard

ESM communityESM community

Metafor PartnersMetafor Partners

Project ExecutiveProject ExecutiveScientific and Scientific and

Technical Advisory Technical Advisory BoardBoard

Metafor standards definition

International Emerging Community

Discovery metadataISO 19139ISO 19115 Climate Modelling

gridspec - model discretisationSensor ML - observationsNMM - model descriptionCERA2 - data management

DataCF for netcdf

Metafor will coordinate the filling of the metadata gaps, mapping to different standards, aggregating the metadata and, if necessary, creating new standards.

Metadata

Standards