Overview of ESMF in the Community Climate System Model (CCSM) Erik Kluzek NCAR -- CCSM Software Engineering Group (CSEG) Erik Kluzek NCAR -- CCSM Software

Overview of ESMF in the Community Climate System Model (CCSM)

Overview of ESMF in the Community Climate System Model (CCSM)

Erik KluzekNCAR -- CCSM Software

Engineering Group (CSEG)

Erik KluzekNCAR -- CCSM Software

Engineering Group (CSEG)

July 21, 2005 Erik Kluzek -- NCARErik Kluzek -- NCAR 22

Outline

Introduction to CCSM Projects using ESMF

ESMF Stage-1 Code Development Stage-1 Requirements and Overall Design

Simple pseudo-code examples of Stage-1 design

River Transport Model Project Data-7 Model Project ESMF Infrastructure in CCSM Conclusion

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.


Introduction to CCSM Projects using ESMF

1. ESMF Stage-1 Code Development -- Implement a sequential driver using ESMF that replaces stand-alone CAM.

2. River Transport Model (RTM) Re-Gridding -- Use ESMF to make the CCSM river-routing model distributed.

3. Data-7 Model Project -- Rewrite of CCSM data-models designed with the ability to use ESMF superstructure from the get-go.

4. ESMF utilities in CCSM -- Plan on using ESMF utilities such as: time-manager, ConfigAtt, LogErr and I/O.




ESMF Stage-1 Code Development

Goal: Implement a single-executable sequential version of CCSM that replicates the answers from stand-alone CAM.

Process: Developed and approved evaluation plan, project requirements and design documents. Staged implementation, frequent iterative design/review.

Status: Currently modifying CAM toward desired product with staged commits on development version.

Motivation: Initially work from a simpler non-concurrent version of CCSM, that can show replication of answers on different configurations of tasks/threads using a more widely used model than CCSM.




ESMF Stage-1 Overall Design

New SPMD, sequential CCSM: Replace stand-alone CAM with a single-executable fully sequential implementation of CCSM (which we currently do not have).

Reduce code-duplication: Will reduce code duplication between CAM and CCSM.

Plug and Play: Active, data, and dead components can be interchanged in the system.

Easy to make driver concurrent for Stage-2: After Stage-1 a concurrent system will be developed that only involves changing the hub and driver components. The underlying separable model components won’t change.




Overview of ESMF Stage-1 Project Requirements

Keep backwards compatibility with both CCSM and all stand-alone CAM configurations.

Hub-spoke design of CCSM used with five components: atmosphere, land, ocean, sea-ice, and hub.

List of fields exchanged -- same as CCSM3. Amount of model code modified should be kept to

the minimum. Top level driver referred to as Sequential CCSM. Capability to easily swap different types of

components in (active, data or dead). Confine ESMF to a thin wrapper layer. Sequential and concurrent mode should share as

much code as possible.




Example of ESMF Layer

M o du l e: a t m _Gr i dC o m p – ESM F i n te r f ace l a y er t o C A M_C o m p Me t h o ds: a t m _reg i s t er, a t m _ i n i t , a t m _ ru n 1 , a t m _ r u n 2, a t m _ fi n a l

M o du l e: C A M_C o m p – N o n - ESM F l a y er Me t h o ds: ca m _ i n i t , ca m _ru n 1 , ca m _ru n 2 , ca m _ fi na l




Top Level Sequential CCSM

Module: ccsm_seq_gridcomp Data: atm_,ice_,ocn_, lnd_(import/export/gc/cpl) Methods: register, init, run, final Module: seq_hub_gridcomp Methods: register, setup, init, final and run (redistribution, merging, history output, flux calculations)

Module: atm_gridcomp Methods: register, init, run1, run2, final

Module: lnd_gridcomp Methods: register, init, run1, run2, final

Module: ice_gridcomp Methods: register, init, run1, run2, final

Module: ocn_gridcomp Methods: register, init, run1, run2, final

Module: camhub_comp Methods: setup, init, run, final

Module: cam_comp Methods: init, run1, run2, final

Module: clm_comp Methods: init, run1, run2, final

Module: camcsim_comp Methods: init, run1, run2, final

Module: camdom_comp Methods: init, run1, run2, final

Module: camsrfexch_types Types: atm2srf_t, srf2atm_t Methods: alloc, dealloc

Module: ocnhub_cplcomp

Module: atmhub_cplcomp

Module: icehub_cplcomp

Module: lndhub_cplcomp




Top level Plug and Play Sequential DriverDo while( .not. ESMF_IsStopTime( sync_clock ) ) call ESMF_CplCompRun( cpl=atm2hub_cpl, import=hub_export, export=atm_import ,& clock=sync_clock, phase=1 ) ! Run Atmosphere phase 1 call ESMF_GridCompRun( gc=atm_gc, in=atm_import, out=atm_export, sync_clock, phase=1 ) call ESMF_CplCompRun( cpl=atm2hub_cpl, in=atm_export, out=hub_import , phase=2 & clock=sync_clock ) ! Run hub to prepare land import call ESMF_GridCompRun( gc=hub_gc, in=hub_import, out=hub_export , phase=1,

clock=sync_clock ) call ESMF_CplCompRun( cpl=hub2lnd_cpl, in=hub_export out=lnd_import , phase=3,& clock=sync_clock ) ! Run Land call ESMF_GridCompRun( in=lnd_import, out=lnd_export , phase=1, clock=sync_clock ).. Similarly for ice, and ocean and second phase run-methods.End do




Plug and Play ESMF Component

Module Atm_gridcompUse cam_comp, only: cam_input_t, cam_output_t, cam_init, cam_run1,

cam_run2, cam_finalContains

! Register methods with ESMFPublic atm_register!! Following are called only through ESMF!Public :: atm_init, atm_run1, atm_run2, atm_final! Convert between ESMF import/export states into CAM input/output derived

typesPrivate :: convert_import2camin, convert_output2export.End module




Model Component non-ESMF Layer

Module cam_compPublic data types :: cam_input_t, cam_output_tContains

Subroutine cam_alloc( cam_in, cam_out )

Subroutine cam_init( cam_in, cam_out )

Subroutine cam_run1( cam_in, cam_out )

Subroutine cam_run2( cam_in, cam_out )

Subroutine cam_finalize( cam_in, cam_out )

End module cam_comp






River Transport Model Re-gridding

Prototyping the use of ESMF re-gridding infrastructure for the eventual creation of a high-resolution parallel biogeochemical river transport model.

Found and resolved problems in ESMF for conservative re-grid on widely differing grid scales.




Data-7 Model Project

Total re-write of CCSM data-model components.

Requirements for utilization of ESMF superstructure (initial, run, final phases) is being done from the ground up.

We will prototype the new system with ESMF superstructure and concurrency in the late fall.




Use of ESMF Infrastructure in CCSM

Currently use prototype ESMF time-manager in two CCSM components, we will update to ESMF2 in the fall independent of Stage-1 evaluation plan.

CCSM is also interested in the use of ESMF ConfigAtt and LogErr utilities in CCSM.




Conclusions CCSM has several active projects

utilizing ESMF: Stage-1, RTM, Data-7, and Infrastructure.

The Stage-1 Project is taking considerable time and resources, but will payoff in having a well-designed flexible and extensible system built around ESMF.

The Data-7 project will be able to evaluate ESMF concurrency.

The RTM project has helped to find and resolve issues with ESMF re-gridding.

Documents

Overview of ESMF in the Community Climate System Model (CCSM) Erik Kluzek NCAR -- CCSM Software Engineering Group (CSEG) Erik Kluzek NCAR -- CCSM Software