NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH ESMF Infrastructure Layer 2 nd...

NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH

ESMF Infrastructure Layer

2nd ESMF Community Meeting, GFDL, May 15th 2003

http://www.esmf.ucar.edu, esmf@ucar.edu

Talk Outline

• What is the infrastructure layer– What is it for– What it contains

• ESMF infrastructure in this release – ESMF version 1.0 function call† examples (code!!!)

(†aka API or methods)

• Next steps….• Infrastructure Utility Example

Infrastructure Layer

1. A standard software platform for enabling interoperability (developing couplers, ensuring performance portability).

2. Set of reusable software for Earth science applications. Streamlined development for researchers.

NCARAtmosphere

GFDLOcean NSIPP

NCARAtmosphere

GFDLOcean NSIPP

MySea Ice

Infrastructure Layer Scope

SupportSupport for• Physical Grids• Regridding• Decomposition/composition• Communication• Calendar and Time• I/O• Logging and Profiling

ESMF Infrastructure

User Code

ESMF Superstructure

2. ESMF provides a toolkit that components use toi. ensure interoperabilityii. abstract common services

Location Within ESMFLocation Within ESMF

Component: run(), checkpoint()

Field: halo(), import(), export() + I/O

Grid: regrid(), transpose() + Metrics

Layout, PEList, Machine Model

Application ComponentGridded

ComponentsCoupler

Components

1. ESMF provides an environment for assembling components.

INFRASTRUCTURE

LAYERLAYER

Talk Outline

• What is the infrastructure layer– What is it for– What it contains

• Next steps….• Infrastructure utility example

Infrastructure Internal Organization

MachineModel

DELayout

DistGridPhysGrid

Bundle

Time, Alarm, Calendar, LogErr, I/O, Attributes

Regrid

Data Communications

Two tiers

Class hierarchy for data and communications

ESMF Infrastructure Fields and Grids

Field Contains array, grid, array to grid Contains array, grid, array to grid mapping and metadatamapping and metadata

Array Holds actual data e.g. temperature, wind Holds actual data e.g. temperature, wind speed etc…speed etc…

Grid Contains physical and distributed gridContains physical and distributed grid

Physical Grid Grid metrics, e.g. lat-lon coords, Grid metrics, e.g. lat-lon coords, spacings, areas, volumes etc..spacings, areas, volumes etc..

Distributed Grid Parallel decomposition informationParallel decomposition information

Creating an ESMF Field

1. Fortran Array

2. Array Attach

3. Field Attach

real, dimension(100,100) :: arr

esArr = ESMF_ArrayCreate(arr,…)

ESMF_Array

ESMF_FieldAttachArray(esFld,esArr,…)

ESMF_Fieldmetadata

ESMF_Grid

Creating an ESMF Grid

ESMF_Grid

PhysGrid

DistGrid

call ESMF_PhysGridCreate…(…) call ESMF_DistGridCreate…(…)

Field and Grid Together :grid = ESMF_GridCreate(…,layout,…) :field_u = ESMF_FieldCreate(grid, array) :

1. Create field distributed over a set of decomposition elements (DE’s).

2. Domain decomposition determined by DELayout, layout.3. Each object (grid and field_u) has internal representation.4. Other parts of the infrastructure layer use the internal

representation e.g.• Regrid() – interpolation/extrapolation + redistribute over DE’s• Redistribution() - general data rearrangement over DE’s• Halo() – specialized redistribution

ESMF_Array

ESMF_Fieldmetadata

ESMF_Grid

Regrid• Function mapping field’s array to a different physical and distributed grid.

• RegridCreate() – creates a Regrid structure to be used/re-used

regrid = ESMF_RegridCreate(src_field, dst_field, method, [name], [rc])

Source, Dest field can be empty of field data (RegridCreate() uses grid metrics)– PhysGrid, DistGrid info used for setting up regrid– Resulting regrid can be used for other fields sharing same Grid

• Method specifies interpolation algorithm. For example, bilinear, b-spline, etc…

},,{},,{ DPADPA

Regrid Interface (cont)• RegridRun() – performs actual regridding call ESMF_RegridRun(src_field, dst_field, regrid,[rc])

– Communication and interpolation handled transparently.

• RegridDestroy() – frees up memory call ESMF_RegridDestroy(regrid,[rc])

src_field

dst_field

Redistribution• No interpolation or extrapolation• Maps between distributed grids, field’s array and physical grid are the

same i.e.

• Example: layout() created two distributed grids one decomposed in X and one decomposed in Y.

– Redistribution() function maps array data between the distributions (a transpose/corner turn)

– Communication handled transparently

},,{},,{ DPADPA

src_field dst_field

Halo • Fields have distributed index space of

– Exclusive {E}, compute {C} and local {L} region where

• Halo() fills points not in {E} or {C} from remote {E} e.g

{C}{L}

call ESMF_FieldHalo(field_foo, status)

DE3 DE4

NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICHMore functions in reference manual e.g

Bundle and Location Stream

• ESMF_Bundle– collection of fields on the same grid

• ESMF_LocationStream– Like a field but…..– unstructured index space with an associated

physical grid space– useful for observations e.g. radiosonde, floats

• Functions for create(), regrid(), redistribute(), halo() etc…

ESMF Infrastructure Utilities• Clock• Alarm• Calendar• I/O• Logging• Profiling• Attribute• Machine modeland comms

Ensures consistent time between components

Provides field level I/O in standard forms – netCDF, binary, HDF, GRIB, Bufr

Consistent monitoring and messaging

Consistent parameter handling

Hardware and system software hiding. Platform customizable

Time• Standard type for any component

• Calendar (support for range of calendars)

! initialize stop time to 13May2003, 2:00 pmcall ESMF_TimeInit(inject_stop_time, & YR=int(2003,kind=ESMF_IKIND_I8), & MM=off_month, DD=off_day, H=off_hour, M=off_min, & S=int(0,kind=ESMF_IKIND_I8), & cal=gregorianCalendar, rc=rc)

do while (currTime .le. inject_stop_time ) : call ESMF_ClockAdvance(localclock, rc=rc) call ESMF_ClockGetCurrTime(localclock, currtime, rc)end

call ESMF_CalendarInit(gregorianCalendar, ESMF_CAL_GREGORIAN, rc)

I/O• Field level binary, netCDF, HDF, GRIB, bufr

– Currently I/O piped through 1 PE call ESMF_FieldAllGather(field_u, outarray, status) if (de_id .eq. 0) then write(filename, 20) "U_velocity", file_no call ESMF_ArrayWrite(outarray, filename=filename,

rc=status) endif call ESMF_ArrayDestroy(outarray, status)

– Current system fixed textual.

Attributes• Flexible parameter specs, configuration e.g. file

of parameters

Internal Classes

• Machine model– Captures system attributes, CPU, mem,

connectivity graph– Useful for defining decomposition, load-balance,

performance predictions.

• Comms– Communication driver, allows bindings to MPI,

shared memory, vendor system libraries

NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICHComms Performance Test

Right mix (green) on Compaq gives x2 realized bandwidth (in large message limit)

Talk Outline• What is the infrastructure layer

– What is it for– What it contains

• Next steps….• Infrastructure utility example

Regrid Next Steps• Support for all ESMF Grids• Support for regridding methods:BilinearBicubic1st-order Conservative2nd-order ConservativeRasterized ConservativeNearest-neighbor distance-weighted averageSpectral transforms1-d interpolations (splines)Index-space (shifts, stencils)Adjoints of many above

Halo Next Steps• Support for periodicity• More general haloing

– in tandem with distributed grid evolution

• Adjoint forms

Distributed Grid • Regular 2d already supported• Next steps

– Generalized 1d decomposition– Extend support for 2d and quasi regular

decomposition – Spectral grid decompositions

• Larger set of metrics, grids• High level routines for rapid definition of common grids.

Physical Grid Next Steps

I/O Next Steps• Broaden format set, binary, netCDF, HDF, GRIB,

bufr• Improve parallelization

• Full support for alarms• Broader functionality

Time/Logging/Profiling Next Steps

Summary

• ESMF Infrastructure Layer– Comprehensive class structure available in

version 1.0– Over the coming year significant extension of

functionality will take place.– Feedback and comments on version 1.0 welcome

http://www.esmf.ucar.edu

ESMF Infrastructure Utility detailed example

Earl Schwab, ESMF Core Team, NCAR

Time Manager

What is Time Manager?

• Clock for time simulation• Time representation• Time calculator• Time comparisons• Time queries• F90 API, C++ implementation

Clock for time simulation

type(ESMF_Clock) :: clock

call ESMF_ClockInit(clock, timeStep, startTime, stopTime, rc=rc)

do while (.not.ESMF_ClockIsStopTime(clock, rc))

! Do application work

call ESMF_ClockAdvance(clock, rc=rc)

end do

Clock (cont.)

• Clock queries/commands

call ESMF_ClockGetCurrTime(clock, currTime, rc)call ESMF_ClockSetCurrTime(clock, currTime, rc)call ESMF_ClockGetTimeStep(clock, timeStep, rc)call ESMF_ClockSetTimeStep(clock, timeStep, rc)call ESMF_ClockGetAdvanceCount(clock, advanceCount, rc)call ESMF_ClockGetStartTime(clock, startTime, rc)call ESMF_ClockGetStopTime(clock, stopTime, rc)call ESMF_ClockGetPrevTime(clock, prevTime, rc)call ESMF_ClockSyncToWallClock(clock, rc)

Time Representation

type(ESMF_Calendar) :: calendar1

call ESMF_CalendarInit(calendar1, ESMF_CAL_GREGORIAN, rc)

- ESMF_CAL_GREGORIAN (3/1/-4800 to 10/29/292,277,019,914)

- ESMF_CAL_JULIAN (+/- 106,751,991,167,300 days)

- ESMF_CAL_NOLEAP

- ESMF_CAL_360DAY

Time Representation (cont.)

type(ESMF_Time) :: time1call ESMF_TimeInit(time1, YR=int( 2003 ,kind=ESMF_IKIND_I8), &

MM= 5 , DD= 15 , H= 15 , cal=calendar1, rc=rc)

- YR, MM, DD, H, M, S F90 optional- D, H, M, S arguments

type(ESMF_TimeInterval) :: timeInterval1call ESMF_TimeIntervalInit(timeInterval1,

D=int( 90 ,kind=ESMF_IKIND_I8), rc=rc)

- D, H, M, S F90 optional arguments

Time Calculator

• Time differencing• Time increment/decrement by a time interval• Time interval arithmetic (+, -, *, /)

Time Calculator (cont.)

call ESMF_TimeInit(time1, YR=int( 2003 ,kind=ESMF_IKIND_I8), & MM= 5 , DD= 15 , cal=gregorianCalendar, rc=rc)

call ESMF_TimeInit(time2, YR=int( 2003 ,kind=ESMF_IKIND_I8), & MM= 3 , DD= 26 , cal=gregorianCalendar, rc=rc)

call ESMF_TimeIntervalInit(timeInterval1, D=int( 90 ,kind=ESMF_IKIND_I8, rc=rc)

timeInterval2 = time2 - time1time1 = time1 + timeInterval1 ! Uses F90 overloaded timeInterval3 = timeInterval1 * 2 ! operators

double precision :: ratioratio = timeInterval1 / timeInterval2

Time Comparisons

>, <, >=, <=, ==, <> F90 overloaded operators between any 2 times or time intervals

if (time1 < time2) then…

end if

if (timeInterval1 .ge. timeInterval2) then…

end if

Time Queries

call ESMF_TimeGet(time1, YR=yr, MM=mm, DD=dd, H=h, M=m, S=s, rc=rc)call ESMF_TimeIntervalGet(timeInterval1, D=d, H=h, M=m, S=s)

call ESMF_TimeGetDayOfYear(time1, dayOfYear, rc) ! double or integercall ESMF_TimeGetDayOfMonth(time1, dayOfMonth, rc)call ESMF_TimeGetDayOfWeek(time1, dayOfWeek, rc)call ESMF_TimeGetMidMonth(time1, midMonth, rc)

call ESMF_TimeGetString(time1, string, rc) ! 2003-05-14T12:20:19 (ISO 8601)call ESMF_TimeIntervalGetString(timeInterval1, string, rc) ! P1DT12H0M0S (ISO)

call ESMF_TimeGetRealTime(time1, rc)

More information

• ESMF User’s Guide• ESMF Reference Manual• ESMF Requirements Document• Time Manager F90 API & examples source code

esmf_1_0_0_r/src/Infrastructure/TimeMgr/interface

esmf_1_0_0_r/src/Infrastructure/TimeMgr/examples

NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH ESMF Infrastructure Layer 2 nd...

Documents

LANL Personal Portfolio

ESMF/Curator Status

Lanl History

Training telescopes - LANL

LANL Emergency Planning and Preparedness OverviewLANL’s National Security Mission and Strategies Unique LANL Operational Considerations LANL Emergency Management Capabilities LANL

Www.esmf.ucar.edu Strategic Plan Implementation Cecelia DeLuca/NCAR (ESMF) December 17, 2008 ESMF Board/Interagency Meeting

ENSO Diversity - GFDL Extranet

climate modeling with GFDL FV3 - European Centre for ... · Variable-resolution weather and climate modeling with GFDL FV3 Lucas Harris for S-J Lin and the GFDL FV3 Team 18th Workshop

Climate Modeling at GFDL - NASAGeophysical Fluid Dynamics Laboratory Princeton, New Jersey Climate Modeling at GFDL Leo Donner GFDL/NOAA, Princeton University CERES Science Team, Princeton,

Main text ESMF 4 March 2016 update · ມັງກອນ ກອບການຄຸ້ມຄອງ (esmf) ດ້ານສັງຄົມ ແລະ ສິ່ງແວດລ້ອມ

NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH 15 May 2003 Cecelia DeLuca / NCAR 2 nd ESMF Community Meeting Princeton, NJ NSIPP Seasonal

Web arch gfdl

documents.worldbank.orgdocuments.worldbank.org/.../Report-ESMF-Terra-Segura-Final-Oct… · Web viewMITADERESMF for Mozland. ESMF for Mozland . MITADER. ESMF for Mozland . MITADER

ESMF Draft Eng

NOAA/ GFDL State-of-the-Science · PDF fileFrontiers in Climate and Earth System Modeling: ... Director, NOAA/ GFDL NOAA/ GFDL State-of-the-Science . May 20, 2013 NOAA/GFDL STATE -OF

ISTI Overview - LANL

Users Guide - LANL

NSF NCAR | NASA GSFC | DOE LANL ANL | NOAA NCEP GFDL | MIT | U MICH Arlindo da Silva, NASA/GSFC/GMAO The Earth System Modeling Framework

NSF NCAR / NASA GSFC / DOE LANL ANL / NOAA NCEP GFDL / MIT / U MICH May 15, 2003 Nancy Collins, NCAR 2nd Community Meeting, Princeton, NJ Earth System

A missing sink for radicals Jingqiu Mao (Princeton/GFDL) With Songmiao Fan (GFDL), Daniel Jacob (Harvard), Larry Horowitz (GFDL) and Vaishali Naik (GFDL)