Upload
jesse-allen
View
219
Download
0
Embed Size (px)
Citation preview
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
V. Balaji / GFDL, Cecelia DeLuca / NCAR, Chris Hill / MIT, Max Suarez, Arlindo da Silva / NASA GSFCEGS - AGUNice, France
Architecture of the Earth System Modeling Framework
NSIPP Seasonal ForecastNCAR/LANL CCSM
NCEP Forecast
GFDL FMS SuiteMITgcm
NASA GSFC PSAS
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
Outline
• ESMF Project Overview• Related Projects and Interactions• ESMF Design Overview• ESMF Design Principles• Timeline and Status
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
Technological TrendsIn climate research and NWP...
increased emphasis on detailed representation of individual physical processes; requires many teams of specialists to contribute components to an overall modeling system.
In computing technology... increase in hardware and software complexity in high-performance computing, as we shift toward the use of scalable computing architectures and open community software standards.
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
ESMF Project Overview
GOAL: To increase software reuse, interoperability, ease of use and performance portability in climate, weather, and data assimilation applications
PRODUCTS: • Coupling superstructure and utility infrastructure software • Synthetic code suite for validation and demonstration• Set of 15 ESMF-compliant applications (including CCSM, WRF, GFDL models;
MIT, NCEP and NASA data assimilation systems)• Set of 8 interoperability experiments
RESOURCES: $10.1M over 3 years from NASA Earth Science Technology Office
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
ESMF CollaboratorsNSF NCARTim Killeen, PIByron BovilleCecelia DeLucaRoberta JohnsonJohn MichalakesAl KellieJeff AndersonDavid NeckelsEarl Schwab Nancy CollinsJon WolfeSilverio VasquezDennis FlaniganRobbie Staufer
DOE LANLPhil Jones
NOAA NCEPStephen LordMark IredellMike YoungWeiyu YangJohn Derber
MITJohn Marshall, PIChris Hill
NASA DAOArlindo da Silva, PILeonid ZaslavskyWill SawyerCarlos Cruz
University of MichiganQuentin Stout
NOAA GFDLAnts LeetmaaV. BalajiRobert HallbergShep Smithline
NASA NSIPPMax Suarez Michele RieneckerChristian KeppenneAtanas Trayanov
DOE ANLRob JacobJay Larson
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
ESMF Interoperability Demonstrations
COUPLED CONFIGURATION NEW SCIENCE ENABLED
GFDL B-grid atm / MITgcm ocn Global biogeochemistry (CO2, O2), SI timescales.
GFDL MOM4 / NCEP forecast NCEP seasonal forecasting system.
NSIPP ocean / LANL CICE Sea ice model for extension of SI system to centennial time scales.
NSIPP atm / DAO analysis Assimilated initial state for SI.
DAO analysis / NCEP model Intercomparison of systems for NASA/NOAA joint center for satellite data assimilation.
DAO CAM-fv / NCEP analysis Intercomparison of systems for NASA/NOAA joint center for satellite data assimilation.
NCAR CAM Eul / MITgcm ocn Improved climate predictive capability: climate sensitivity to large component interchange, optimized initial conditions.
NCEP WRF / GFDL MOM4 Development of hurricane prediction capability.
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
Outline
• ESMF Project Overview• Related Projects and Interactions• ESMF Design Overview• ESMF Design Principles• Timeline and Status
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
Larson/ANLDeLuca/NCAR-SCDJones/LANLStout/U MichKilleen/NCARDrake/ORNLBoville/NCAR-CGDMichalakes/NCAR-MMMSuarez/NASA GoddardBalaji/NOAA GFDL
CCA
WRF
SciDAC
CCSM
GEMSFMSSWMF
PRISM
ESMF
Related Projects
ESMF Earth System Modeling FrameworkCCA DOE Common Component ArchitectureSciDAC DOE/NSF CCSM SciDAC ProjectGEMS Goddard Earth Modeling SystemFMS GFDL Flexible Modeling SystemSWMF Space Weather Modeling FrameworkWRF Weather Research and Forecast ModelCCSM Community Climate System ModelPRISM Program for Int. Earth System Modeling
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
ESMF and PRISM
• A European Earth system modeling infrastructure project, started December 2001
• Funded by the European Commission ($4.8M)
• Involves current state-of-the-art atmosphere, ocean, sea-ice, atmospheric chemistry, land-surface and ocean-biogeochemistry models
• 22 partners: leading climate researchers and computer vendors, includes MPI, KNMI, UK Met Office, CERFACS, ECMWF, DMI.
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
ESMF and PRISM, cont.
• Working together to supplement CF convention for physical field names and quantities
• Component interface database developed by ESMF stores fields in model import and export states
• V. Balaji (GFDL) is ESMF liaison to PRISM
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
ESMF and CCA
• Common Component Architecture (CCA) is creating a minimal interface and sets of tools for linking high performance components. CCA can be used to implement frameworks and standards developed in specific domains (such as ESMF).
• DOE funded through SciDAC.
• Collaborators include LANL, ANL, LLNL, ORNL, Sandia, University of Tennessee, and many more. Ongoing ESMF collaboration with CCA/LANL on language interoperability.
• Shujia Zhou / Arlindo da Silva (NASA GSFC) currently prototyping an ESMF API using CCA tool CCAFFEINE; working towards CAM/DAO PSAS coupling using ESMF and CCA.
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
Outline
• ESMF Project Overview• Related Projects and Interactions• ESMF Design Overview• ESMF Design Principles• Timeline and Status
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
Architecture
External Libraries
Low Level Utilities
Fields and Grids Layer
Model Layer
Coupling Layer
ESMF Infrastructure
User Code
ESMF Superstructure
BLAS, MPI, NetCDF, …
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
Field: Halo( ), Import( ), Export( ) + I/O
ESMF Components
2. ESMF provides a toolkit that components use to
i. increase interoperability
ii. improve performance portability
iii. abstract common services
Component: Init( ), Run( ), Checkpoint( )
Grid: Regrid( ), Transpose( ) + Metrics
Layout, PE List, Machine Model
Application Component
Gridded Components Coupler Components
1. ESMF provides an environment for assembling geophysical components into an application.
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
General Features
• ESMF will be usable by models written in F90/C/C++• ESMF will be usable by models requiring adjoint capability• ESMF will be usable by models requiring shared or distributed memory
parallelism semantics• ESMF will support SPMD and MPMD coupling• ESMF will support several I/O formats, including GRIB/BUFR, netCDF,
HDF• ESMF will have uniform syntax across platforms• ESMF will target a broad range of platforms, from major centers
commodity hardware
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
Outline
• ESMF Project Overview• Related Projects and Interactions• ESMF Design Overview• ESMF Design Principles• Timeline and Status
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
Design Principles:Scalable ApplicationsSince each ESMF application is also a component, entire ESMF applications may be treated as Gridded Components and nested within larger applications.
climate_comp
ocn_comp atm_comp
ocn2atm_coupler
atm_phys
phys2dyn_coupler
atm_dynPE
Example: atmospheric application containing multiplecoupled components within a larger climate application
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
Design Principles:Local CommunicationAll inter-component communication within ESMF is local.
climate_comp
ocn_comp atm_comp
atm_phys
phys2dyn_coupler
atm_dynPE
atm2ocn _coupler
This means:
Coupler Components must be defined on the union of the PEs of all the components that they couple.
In this example, in order to send data from the ocean component to the atmosphere, the Coupler mediates the send.
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
Design Principles:ModularityGridded Components don’t have access to the internals of other Gridded Components. Gridded Components may
- pass their States through their argument list or
- receive methods, through their argument list, for interacting with other Components (Transforms).
coupler
ocn_componentcall ESMF_StateXform(ex_state, &
xform)
atm_componentcall ESMF_StateXform(xform, &
im_state)
transformcall ESMF_CompRun(atm, xform)call ESMF_CompRun(ocn, xform)
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICHDesign Principles:Uniform Communication APIThe same programming interface is used for shared memory, distributed memory, and combinations of both.
Machine model = abstraction of machine architecture (num_nodes, num_pes_per_node)
DE = a decomposition element - may be virtual, thread or MPI process
Layout = an arrangement of DEs, in which dimensions requiring faster communication may be specified and resources arranged accordingly
A 2 x 6 Layout of 4 3-processor nodes
The data in a Grid is decomposed into the number of chunks specified in the Layout.
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
ESMF Class Structure
MachineModel
Layout
DistGridPhysGrid
State
Bundle
GriddedComponent
Array
Utilities: TimeMgr, LogErr, I/O etc.
F90
Superstructure
Infrastructure
Field
Grid
Regrid
Comm
Data Communications
More workPrototype exists
Route
CouplerComponent
C++
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
Outline
• ESMF Project Overview• Related Projects and Interactions• ESMF Design Overview• ESMF Design Principles• Timeline and Status
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
Timeline
May 2002 Draft Developer’s Guide and Requirements Document completed; Community Requirements Meeting and review held in D.C.
July 2002 ESMF VAlidation (EVA) suite assembled
August 2002 Architecture Document: major classes and their relationships; Implementation Report: language strategy and programming model; Software Build and Test Plan: sequencing and validation
May 15 2003 First API and software release, Community Meeting at GFDL
July 2003 First 3 interoperability experiments completed
April 2004 Second API and software release, Community Meeting
July 2004 All interoperability experiments completed
Jan 2005 Final releaseCurrently planning for ongoing funding and extensions
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
May 2003 Release
Focus for May 2003 ESMF release is on developing sufficient infrastructure and superstructure to achieve the initial set of interoperability experiments.
These are:– FMS B-grid atmosphere coupled to MITgcm ocean– CAM atmosphere coupled to NCEP analysis– NSIPP atmosphere coupled to DAO analysis
8 April 2003
NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH
More information
2nd ESMF Community Meeting at GFDL May 15, 2003
ESMF website: http://www.esmf.ucar.eduOn the website:• Information about the Community Meeting• ESMF documents • ESMF browsable source code repository • On-line forms for community input and catalogued responses• Milestone schedule, project archives and history, related projects• Management plan, board and team contacts• More …