CESM/ESMF Progress CESM/ESMF Progress ReportReport

Mariana Vertenstein

NCAR Earth System LaboratoryCESM Software Engineering Group (CSEG)

NCAR is sponsored by the National Science Foundation

CESM CommunityCESM Community Registered CESM Release Users

• 4240 Total registered users• 1110 New users in past 12 months

DiscussCESM Forums Bulletin Board http://bb.cgd.ucar.edu

• 1793 Total Registered users• 14 Top level forums with 77 sub-forums• 5229 Total posts (topics + comments) • 800 posts since March 2014

Registered CESM Developers• 290 Total registered users – vetted annually

CESM Architecture (CPL7)CESM Architecture (CPL7) Single executable 7 geophysical components / 1 coupler (hub)

– atm, land, river, land-ice, sea-ice, ocean, wave– run on potentially different grids – exchange boundary data with each other

Hub and Spoke architecture– components communicate with each other ONLY a

coupler (hub and spoke architecture)– components/hub can run on unique PES

concurrently) or overlapping PES (sequentially) or hybrid layout

– components can be prognostic or prescribed -data Model Driver

– runs on union of all component/hub PES – coordinates time evolution of system

CESM Architecture (cont) CESM Architecture (cont)

processors

tim

e CPL (regridding, merging)

CAM

CLM

CICE

Driver (controls time evolution)

POP

Sequential Layout

processors

Hybrid Sequential/Concurrent Layout

CAM

CLM CICE

POP

Driver

CPL

CPL

Driver (controls time evolution)

Concurrent Layout

CAM CLM CICE POP

processors

tim

e

CESM/ESMF Offline CESM/ESMF Offline RegriddingRegridding

ESMF offline regridding has – completely changed the ability of CESM to

run on new grids – both structured and unstructured

– Enabled new science! CESM has now introduced new tool chain

based on the ESMF offline regridding– capability has been introduced as part of the

CESM releases in top level directory What took days (even weeks) now takes

hours to create

Recent ExampleRecent Example Problem: With CAM spectral element dycore, ocean

is having problems spinning up from Levitus (much slower compared to using CAM finite volume dycore)

Want to examine this further by Want to examine this further by creating new polar refined gridcreating new polar refined grid

Created new atm/landgrid and mapping files and was able toobtain new coupledmodel run on this grid in < 2 weeks!

Result showed that grid was not not problem

ESMF online regridding - ESMF online regridding - why?why?

As more regionally refined grids are introduced (e.g. MPAS, SE) – need to– minimize the number of mapping files that are

needed – simplify and streamline workflow for generating

new user grid configurations Will be a requirement for run-time adaptive

mesh refinement ESMF is the only tool that currently delivers

this capability Status: Prototype implementation has been

done and is being updated for newest coupler

NUOPC/HYCOM in ESMFNUOPC/HYCOM in ESMFGoal

– Introduce HYCOM into CESM as an alternate ocean component via the NUOPC Layer.

Initial work – Redesigned the CESM driver so that it

easily accommodate both NUOPC-based components and components that use native CESM datatypes, from the Model Coupling Toolkit (MCT)

Why Redesign Driver/Hub Why Redesign Driver/Hub Implementation?Implementation?

driver code hard-coded to contain MCT types one large routine (~6K lines) with functionality

often repeated difficult to understand and MODIFY flow

through system difficult to implement use of alternative

coupling architecture (e.g. ESMF) at the driver/hub

Redesign DetailsRedesign Details Introduced a new abstraction layer between driver

and components - driver now has no references to either MCT or ESMF types – Driver LOC reduced from 6k -> 3k and much easier to

understand and modify flow Much easier to incorporate ESMF/NUOPC driver

components Permits ESMF and MCT data structures to share

memory where possible - allows significant backwards compatibility

Redesign work has been brought into CESM trunk (development code base) with both ESMF and MCT data types