View
215
Download
0
Category
Preview:
Citation preview
CSEG UpdateCSEG Update
Mariana VertensteinCCSM Software Engineering
Group
Mariana VertensteinCCSM Software Engineering
Group
22
Improvements to CCSM scriptsImprovements to CCSM scripts
Addition of timing tool that provides automated information to help determine load balance, throughput and cost of a run.
Improvements made to CCSM script infrastructure Provide the ability to define new “component
sets/modes”. and as command line options. Makes it easier to run and test new CCSM science (such as addition of component biogeochemistry and atmospheric chemistry).
Simplified user specification of non-default task/thread settings.
Changes improve extensibility and robustness of scripts.
Addition of timing tool that provides automated information to help determine load balance, throughput and cost of a run.
Improvements made to CCSM script infrastructure Provide the ability to define new “component
sets/modes”. and as command line options. Makes it easier to run and test new CCSM science (such as addition of component biogeochemistry and atmospheric chemistry).
Simplified user specification of non-default task/thread settings.
Changes improve extensibility and robustness of scripts.
33
Improvements to CCSM scripts (cont)
Improvements to CCSM scripts (cont)
CCSM test framework rewritten using new functionality. Introduced new testing functionality (e.g. auto-
promotion test). Introduced new “test suites” that now permit different
testing levels:development, pre-tag, post-tag, monthlycurrently in process of defining contributions to each
test suite Benefits:
Regression testing will be easier to perform.Straightforward addition of tests for new CCSM
scenarios. Easier testing of new scenarios (e.g. CLM-CN with
CO2 exchange).More frequent testing of new science.More robust CCSM code base.
CCSM test framework rewritten using new functionality. Introduced new testing functionality (e.g. auto-
promotion test). Introduced new “test suites” that now permit different
testing levels:development, pre-tag, post-tag, monthlycurrently in process of defining contributions to each
test suite Benefits:
Regression testing will be easier to perform.Straightforward addition of tests for new CCSM
scenarios. Easier testing of new scenarios (e.g. CLM-CN with
CO2 exchange).More frequent testing of new science.More robust CCSM code base.
44
Porting and PerformancePorting and Performance ORNL Cray X1 (phoenix)
Both release and development code base have been ported. CAM FV dycore ported in development code base. Porting this dycore in CCSM proved to be very challenging.
30 years/day on 200 procs (ignorning performance fluctuations)
13 years/day on 108 procs on NCAR bluevista 400 year FV control simulation carried out
ORNL Cray XT3 (jaguar) Port of development code base is starting now. Release
code base is running - will require a validation. NCAR IBM (bluevista)
Major new OS upgrade will permit over-subscription of nodes. Provides an opportunity for major performance improvements - but will also result in significantly larger parameter space for performance optimization.
ORNL Cray X1 (phoenix) Both release and development code base have been
ported. CAM FV dycore ported in development code base. Porting this dycore in CCSM proved to be very challenging.
30 years/day on 200 procs (ignorning performance fluctuations)
13 years/day on 108 procs on NCAR bluevista 400 year FV control simulation carried out
ORNL Cray XT3 (jaguar) Port of development code base is starting now. Release
code base is running - will require a validation. NCAR IBM (bluevista)
Major new OS upgrade will permit over-subscription of nodes. Provides an opportunity for major performance improvements - but will also result in significantly larger parameter space for performance optimization.
44
55
Porting and Performance (future)Porting and Performance (future)
NCAR IBM BluegeneWill require single executable
concurrent systemWill require parallel I/O (NetCDF and
binary)
NCAR IBM BluegeneWill require single executable
concurrent systemWill require parallel I/O (NetCDF and
binary)
66
Data Model ProjectData Model ProjectSerial version rewrite of all data models
(datm7, dlnd7, dice7, docn7) completed!datm7 can now duplicate stand-alone CLM
functionality (in serial mode).docn7 now has both DOM and SOM
functionality (SOM scientific verification is close to complete).
All data components now have same functionality.Can perform spatial interpolation from input
data resolution to model resolution.Can data cycle over subset of years.
Serial version rewrite of all data models (datm7, dlnd7, dice7, docn7) completed!datm7 can now duplicate stand-alone CLM
functionality (in serial mode).docn7 now has both DOM and SOM
functionality (SOM scientific verification is close to complete).
All data components now have same functionality.Can perform spatial interpolation from input
data resolution to model resolution.Can data cycle over subset of years.
77
Data Model Project (what is next)
Data Model Project (what is next)
Parallelize the new serial data models.Incorporate parallel data models into
currently developed sequential CCSM.Replace component specific data models
with CCSM data models. As examples:Replace SOM/DOM in CAM with DOCN7Replace forcing atm driver in CLM with DATM7
Use of only one set of data models will result in more consistent science and remove existing code duplication.
Parallelize the new serial data models.Incorporate parallel data models into
currently developed sequential CCSM.Replace component specific data models
with CCSM data models. As examples:Replace SOM/DOM in CAM with DOCN7Replace forcing atm driver in CLM with DATM7
Use of only one set of data models will result in more consistent science and remove existing code duplication.
77
88
Single Executable Concurrent CCSM
Single Executable Concurrent CCSM
CSEG is leveraging the work that Helen He and Chris Ding (SciDAC) have done in order to create a single executable concurrent implementation of CCSM development code.
A somewhat different implementation will be produced to satisfy current CCSM requirements.
Aim to have development CCSM tag (including all CCSM components) by mid-July.
Aim to have a CCSM3.0 release based version released by later in the summer.
Existence of single executable system should improve CCSM portability and debugging.
CSEG is leveraging the work that Helen He and Chris Ding (SciDAC) have done in order to create a single executable concurrent implementation of CCSM development code.
A somewhat different implementation will be produced to satisfy current CCSM requirements.
Aim to have development CCSM tag (including all CCSM components) by mid-July.
Aim to have a CCSM3.0 release based version released by later in the summer.
Existence of single executable system should improve CCSM portability and debugging.
99
Single Executable Sequential CCSM
Single Executable Sequential CCSM
Major progress made in replacing stand-alone CAM with “pseudo-sequential CCSM”.
Both ESMF and MCT will be examined as coupling frameworks. We are committed to creating a sequential interoperable CCSM utilizing the ESMF framework.
ESMF superstructure coding for Stage 1 Evaluation Plan is beginning now. Superstructure design is already in place. Estimated completion date is end of summer. Plan is to incorporate ESMF coupling framework on CAM trunk upon successful completion of Stage 1 evaluation effort.
Major progress made in replacing stand-alone CAM with “pseudo-sequential CCSM”.
Both ESMF and MCT will be examined as coupling frameworks. We are committed to creating a sequential interoperable CCSM utilizing the ESMF framework.
ESMF superstructure coding for Stage 1 Evaluation Plan is beginning now. Superstructure design is already in place. Estimated completion date is end of summer. Plan is to incorporate ESMF coupling framework on CAM trunk upon successful completion of Stage 1 evaluation effort.
1010
Original stand-alone CAM architecture (CAM3.0)
Original stand-alone CAM architecture (CAM3.0)
CAM Driver
Physics
CLM CAM-ICE CAM-OCN
Dynamics
Note that surface models are invoked from within CAM
physics - not from a top level driver
Pseudo-Sequential CCSM Top level Architecture
Pseudo-Sequential CCSM Top level Architecture
Application Driver(FD)
CLM CAM-ICE CAM-OCNCAM
Phys Dyn
Couplers (FD)
ATM merger(FD)
OCN merger(FD)
LND merger(FD)
ICE merger(FD)
Thin coupling layer (FD)
Thin coupling layer (FD)
Thin coupling layer (FD)
Thin coupling layer (FD)
Introduce top level framework dependent
(FD) application driver to replace CAM driver
Introduce toplevel ESMF clock tocoordinate time evolution of all
components
Introduce new flexible andextensible “thin”
coupling layer design
1212
Sequential CCSM (cont)Sequential CCSM (cont)
New top level based application driver. Independent of CAM data structures Time evolution based upon ESMF general time
management utilities Utilizes CCSM share code for reading input Initial implementation: MCT Near term implementation: ESMF
New surface coupling layer. Initial implementation: MCT Near term implementation: ESMF
New directory structure for sequential CCSM. Applies to both MCT and ESMF
Inter-component domain checking in coupling layer. Initial implementation: MCT Near term implementation: ESMF
New top level based application driver. Independent of CAM data structures Time evolution based upon ESMF general time
management utilities Utilizes CCSM share code for reading input Initial implementation: MCT Near term implementation: ESMF
New surface coupling layer. Initial implementation: MCT Near term implementation: ESMF
New directory structure for sequential CCSM. Applies to both MCT and ESMF
Inter-component domain checking in coupling layer. Initial implementation: MCT Near term implementation: ESMF
1313
CAM Update CAM Update Implemented support for non lat-lon grids (Jim Edwards
and Pat Worley). Non lat-lon support added to Boundary data and aerosol data interpolation code.Physics/Dyamics coupling layer. Only aqua-planet mode is currently supported.
Significant FV dycore interface refactoring implemented:Only data on the XY decomposition outside the
portable dycore is now exposed.Added dynamics import/export states, and dynamics
component module with init, run, final methods. CAM testing framework extended to add new platforms
(e.g. phoenix) and to perform overnight regression testing.
New features added to CAM tropospheric MOZART.
Implemented support for non lat-lon grids (Jim Edwards and Pat Worley). Non lat-lon support added to Boundary data and aerosol data interpolation code.Physics/Dyamics coupling layer. Only aqua-planet mode is currently supported.
Significant FV dycore interface refactoring implemented:Only data on the XY decomposition outside the
portable dycore is now exposed.Added dynamics import/export states, and dynamics
component module with init, run, final methods. CAM testing framework extended to add new platforms
(e.g. phoenix) and to perform overnight regression testing.
New features added to CAM tropospheric MOZART.
1414
CAM Update (what is next)CAM Update (what is next) Refactoring of CAM to run with new dycores.
Incorporation of HOMME dycore into CAM.Refactoring CAM’s history module to
encompass non lat-lon output.Ability to run pseudo-sequential CCSM (not
just aqua-planet) with non lat-lon CAM. Incorporation of parallel I/O into CAM.
Replace all binary I/O with NetCDF I/O.Creation of new tool to generate CAM
namelists (tool should easily extend to other models if desired).
Refactoring of CAM to run with new dycores.Incorporation of HOMME dycore into CAM.Refactoring CAM’s history module to
encompass non lat-lon output.Ability to run pseudo-sequential CCSM (not
just aqua-planet) with non lat-lon CAM. Incorporation of parallel I/O into CAM.
Replace all binary I/O with NetCDF I/O.Creation of new tool to generate CAM
namelists (tool should easily extend to other models if desired).
1515
CAM Update (what is next) CAM Update (what is next)
SCAM (Single Column Atmosphere Model) refactoring Goal is to produce a more flexible and maintainable
interfaces. , Adoption of new components
Incorporation of new ice sheet model (GLIMMER).Primary work will be done by Bill Lipscomb.GLIMMER has already been incorporated into CAM
build system.Replacement of CAM-CSIM with CICE and
CAM-SOM/DOM with DOCN7.
SCAM (Single Column Atmosphere Model) refactoring Goal is to produce a more flexible and maintainable
interfaces. , Adoption of new components
Incorporation of new ice sheet model (GLIMMER).Primary work will be done by Bill Lipscomb.GLIMMER has already been incorporated into CAM
build system.Replacement of CAM-CSIM with CICE and
CAM-SOM/DOM with DOCN7.
1616
CLM UpdateCLM UpdateFinemesh grids have been implemented in
CLM. This gives CLM the new capability to run on its own independent grid. The implementation follows the scheme of Hahmann and Dickinson.CLM still couples to CAM or CCSM via the CAM coarse
grid. Mapping is done within the CLM code base.The implementation does not change answers when
the CLM fine grid is identical to the CAM grid.Stand-alone CAM runs have been run with a T42
coarse grid and a half degree finemesh grid. Results are encouraging.
New downscaling and upscaling interactions are being implemented.
Finemesh grids have been implemented in CLM. This gives CLM the new capability to run on its own independent grid. The implementation follows the scheme of Hahmann and Dickinson.CLM still couples to CAM or CCSM via the CAM coarse
grid. Mapping is done within the CLM code base.The implementation does not change answers when
the CLM fine grid is identical to the CAM grid.Stand-alone CAM runs have been run with a T42
coarse grid and a half degree finemesh grid. Results are encouraging.
New downscaling and upscaling interactions are being implemented.
1717
CLM Update (what is next)CLM Update (what is next) A nested grid capability is being added to CLM, so the
model can be run at spatially varied resolution to optimize cost.
A new prognostic canopy air space scheme is being implemented. Scientific formulation is being finalized. Software implementation (Forrest Hoffman) will start as
soon as the scientific formulation is completed. CLM offline testing framework will be rewritten in order
to test the numerous new features that are being introduced into the system.
Urban code will be incorporated into the main development line of development.
A nested grid capability is being added to CLM, so the model can be run at spatially varied resolution to optimize cost.
A new prognostic canopy air space scheme is being implemented. Scientific formulation is being finalized. Software implementation (Forrest Hoffman) will start as
soon as the scientific formulation is completed. CLM offline testing framework will be rewritten in order
to test the numerous new features that are being introduced into the system.
Urban code will be incorporated into the main development line of development.
1818
POP2 UpdatePOP2 UpdateCompleted incorporation of major CCSM
features into the LANL pop2.1.alpha code. CCSM POP2 code is now in CCSM SVN repository and current CCSM scripts.
Added New tracer advection scheme (1D Lax-
Wendroff with 1D flux limiters).Near-surface eddy flux, enhanced deep
mixing. Support for new 1-degree grid with
Galapagos Islands topography (gx21v4).
Completed incorporation of major CCSM features into the LANL pop2.1.alpha code. CCSM POP2 code is now in CCSM SVN repository and current CCSM scripts.
Added New tracer advection scheme (1D Lax-
Wendroff with 1D flux limiters).Near-surface eddy flux, enhanced deep
mixing. Support for new 1-degree grid with
Galapagos Islands topography (gx21v4).
1919
POP2 Update (what is next)
POP2 Update (what is next)
Inclusion of a more efficient barotropic solver designed by John Dennis.
Reduced equatorial viscosity.A new vertical grid.Systematic exploration of the above
features. Plans for incorporating the ecosystem
model into POP2 will be developed during the CCSM workshop.
Inclusion of a more efficient barotropic solver designed by John Dennis.
Reduced equatorial viscosity.A new vertical grid.Systematic exploration of the above
features. Plans for incorporating the ecosystem
model into POP2 will be developed during the CCSM workshop.
2020
CICE UpdateCICE UpdatePCWG has decided to incorporate CICE as the
new standard CCSM ice model. New name will be “Community Ice CodE”.
Testing has been done using CICE 3.1 to establish main differences with CSIM 5.0.
CICE 3.1 changes answers significantly, but still within the realm of the same climate.
CICE 3.14 has additional bug fixes and a few new physical parameterizations. This is the version that will be adopt after more testing is completed.
Goal is to incorporate CICE 4 (with new data structures) into CCSM and into pseudo-sequential CCSM.
PCWG has decided to incorporate CICE as the new standard CCSM ice model. New name will be “Community Ice CodE”.
Testing has been done using CICE 3.1 to establish main differences with CSIM 5.0.
CICE 3.1 changes answers significantly, but still within the realm of the same climate.
CICE 3.14 has additional bug fixes and a few new physical parameterizations. This is the version that will be adopt after more testing is completed.
Goal is to incorporate CICE 4 (with new data structures) into CCSM and into pseudo-sequential CCSM.
Recommended