An overview of the Unified Forecast System (UFS)

1 / 23DTC UFS SRW workshop, Sep. 21, 2021

An overview of the Unified Forecast System (UFS)Hendrik Tolman

For the UFS Steering Committeehttps://UFSCommunity.org

Dr. Ir. Hendrik L. TolmanSenior Advisor for Advanced Modeling SystemOffice of Science and Technology IntegrationNational Weather Service / NOAA

[email protected]

https://ufscommunity.org/


BLUFNOAA moving to Unified Forecast System (UFS) approach● Simplification of Production Suite (nowcast to seasonal)● Accelerate R2O (research and operations with same tools)● Broaden the base (government, academia and industry)

This has moved from an idea to a reality● Releases and operational implementations

➤ Medium Range Weather and Short Range Weather App.➤ Hurricane Analysis and Forecast System ➤ Prototype coupled system

u NG-GODAS 40 year reanalysis


About the UFS The Unified Forecast System (UFS) is a comprehensive, community-developed Earth modeling system, designed as both a research tool and as the basis for NOAA’s operational forecasts.

Planning and evidence-based decision-making support improving research and operations transitions and community engagement.

UFS is configurable into multiple applications that span local to global domains and predictive time scales from less than an hour to more than a year.

UFS is a unified system because the applications within it share science components and software infrastructure.

UFS is a paradigm shift that will enable NOAA to simplify the NCEP Production Suite, to accelerate use of leading research, and to produce more accurate forecasts for the U.S. and its partners.

Purpose

Scope

Governance

Design

Impact

“System” in UFS = code + governance + community


UFS in a nutshell

Milestones:● Buy in at AA level (UMC)● MoA with NCAR● Community Modeling● Research and Ops.● UFS R2O project● Recently:

➤ NOAA Modeling Board➤ EPIC

NPS Modeling System

Current Version

Q1 FY 20

Q2 FY 20

Q3FY 20

Q4 FY 20

Q1 FY 21

Q2 FY 21

Q3FY 22

Q4FY 22

Q1 FY 23

Q2FY 23

Q3FY 23

Q4FY 23

Q1 FY 24

Q2FY 24

Q3FY 24

Q4FY 24

UFS Application

Global Weather & Global Analysis

GFS/ GDASv15

Global Waves GWMv3Global Weather Ensembles GEFSv11Global Wave Ensembles GWESv3Global Aerosols NGAC v2Short-Range Regional Ensembles SREFv7

Global Ocean & Sea-Ice RTOFSv1.2 RTOFSv2 RTOFSv3Global Ocean Analysis GODASv2 GODASv3

Seasonal ClimateCDAS/ CFSv2

SFSv1 UFS Seasonal

Regional Hurricane 1 HWRFv12 HWRFv13Regional Hurricane 2 HMONv2 HMONv3Regional High Resolution CAM 1

HiRes Window v7

Regional High Resolution CAM 2

NAM nests/ Fire Wxv4

Regional High Resolution CAM 3

RAPv4/ HRRRv3

RAPv5/ HRRRv4

Regional HiRes CAM Ensemble HREFv2

HREFv3

Regional Mesoscale Weather NAMv4

Regional Air Quality CMAQv5 CMAQv6

Regional Surface Weather Analysis

RTMA/ URMA v2.7

RTMA/ URMA v2.8

3DRTMA/URMAv3

Atmospheric Transport & Dispersion HySPLITv7

HySPLITv8

HySPLITv9

UFS Air Quality & Dispersion

Coastal & Regional Waves NWPSv1.2

NWPS v1.3

NWPS v1.4 RWPSv1 UFS Coastal

Great Lakes GLWUv3.4 GLWUv4 GLWUv5 UFS LakesRegional Hydrology NWMv2 NWMv3 NWMv4 UFS Hydrology

Space Weather 1 WAM/IPEv1Space Weather 2 ENLILv1

HAFSv1 HAFSv2

RRFSv2

HAFSv3

UFS Short-Range Regional HiRes

CAM & Regional Air Quality

RRFSv1

WAMv2

Q3FY 21 - Q2FY22MORATORIUM

UFS Space Weather

GEFSv12

UFS Medium Range & Sub-

Seasonal

UFS Marine & Cryosphere

GFSv16

UFS Hurricane

GFSv17/ GEFSv13

3. Community-friendly workflowCIME - CROW unification, CIME Case Control System 4. Hierarchical model development capabilitiesExtensions of CIME data models, unit, and system testing5. Forecast Verification: Comparison to ObservationsExtension of METplus

6. Software Repository ManagementNCAR manage_externals tool

7. User / Developer SupportDTC and CESM Capabilities

1. Coupling componentsNew ESMF/NUOPC mediator (CMEPS/NEMS)2. Interoperable atmospheric physicsCCPP & CPF frameworks


Theory or Practice ?

What makes the UFS a reality:● Graduate Student Test (GST)

➤ Install an application in a day● Supported code releases

➤ Medium Range Weather Application releases (2)➤ Short Range Weather Application release➤ Access to full code base, even if there is no ‘release” yet

u HAFS, Coupled prototype S2S model➤ Infrastructure, components, tools …➤ Link to JEDI for Data Assimilation


R2O AccelerationPre-UFS● WW3 and HWRF examples● Factor 3-5 acceleration of R2O process

➤ Same code in research and operations➤ Shared test environment➤ Leveraging of external investments / Larger team

UFS:● A little early to assess● …. but GFS beating ECMWF for a full week in March had not

happenened in a long time .....


Release Schedule (tentative)(Planned) releases / implementations● Medium-range Weather (MRW) App 1.0.0, March 2020

➤ FV3 based, Interoperable atmospheric physics and land surface supported with Common Community Physics Package (CCPP)

● GEFSv12.0, UFS based implementation September 2020➤ FV3 based, coupled waves, aerosols

● Medium-range Weather (MRW) App 1.1.0, October 2020➤ Updates from graduate student test responses, build systems, documentation, chgres

● GFSv16, operational implementation March 2021➤ Updated atmospheric physics

● Short-range Weather (SRW) App 1.0, March 2021➤ FV3 Limited Area Model before estimated 2023-2024 implementation

● MRW App v2.0.0, TBD, tentatively “next one up”➤ Possibly fully coupled Seasonal / S2S prototype, well before tent. 2024-2025 impl.

● Many components, e.g., MET, CCPP, as well as first JEDI-FV3 release in Nov. 2020.


Other UFS foci

RRFS development● Other presentations in this workshop

HAFS development● Tentative replacement of HWRF● Tentative framework for Warn on Forecast System

Following pages:● Coupled Prototype S2S model● Workflow ● Formalizing the Innovation to Operations process


S2S Prototype layout (p1)Atmosphere● FV3 dynamical core● GFS Physics with GFDL microphysics● CCPP physics driver ● C384 (~25km), 64 levels

Ocean● MOM6 Modular Ocean Model● ¼ degree tripolar grid, 75 hybrid levels ● OM4 Set up [Adcroft, 2019]

Waves (not active in P1)● WAVEWATCH III● ½ degree regular lat/lon grid ● ST4 Physics [Ardhuin, 2010]

Ice● CICE5 Los Alamos Sea Ice

Model ● ¼ degree tripolar grid (same as

ocean) ● 5 thickness categories● No Mushy thermodynamics

Driver/Mediator● NEMS driver● NEMS mediator

https://github.com/ufs-community/ufs-s2s-model

Prototype slides courtesy Jessica Meixner, EMC

https://agupubs.onlinelibrary.wiley.com/doi/epdf/10.1029/2019MS001726

https://journals.ametsoc.org/doi/pdf/10.1175/2010JPO4324.1

https://github.com/ufs-community/ufs-s2s-model


S2S Prototype experimentsInitial Conditions

FV3GFS MOM6 CICE5 WW3

UFS_p1 CFSR CFSR CFSR n/a

UFS_p2 CFSR CPC 3Dvar CFSR n/a

UFS_p3.1 CFSR CPC 3Dvar

CPC ice analysis n/a

UFS_p4 CFSR CPC 3Dvar

CPC ice analysis

Generated with CFS forcing

● 35 day free forecasts ● April 2011 to March 2018

○ Initialized from the 1st and 15th of each month

○ 7 years, 168 forecasts○ Covers El Niño and La

Niña events○ Years of low ice extent


S2S Prototype experimentsü Prototype 5:

• CICE6 ice model • Fractional grid for atm

ü Prototype 6: • GFSv16 physics with 127L • CMEPS Mediator

✘ Prototype 7: • GEFS reanalysis for IC• Noah-MP Land model• Mushy ice thermo • uGWD, NSST• MERRA2 Aerosol clim.

✘ Prototype 8: • Marine JEDI DA for IC• Physics tuning https://github.com/ufs-community/ufs-weather-model

https://github.com/ufs-community/ufs-weather-model


Sea Ice Extent (P3)

Arct

ic

Data Source : NOAA/NSIDC Climate Data Record of Passive Microwave Sea Ice Concentration, Version 3 (https://nsidc.org/data/g02202/versions/3)

backup

https://nsidc.org/data/g02202/versions/3


Next Generation Global Ocean Data Assimilation System (NG-GODAS)● Mainly driven by EMC/CPC/JCSDA-JEDI-SOCA teams

➤ DA system: JEDI Sea Ice Ocean Coupled Assimilation System (SOCA)➤ UFS DATM-MOM6-CICE6 DA 40 year reanalysis production run (1979-2019)➤ Comprehensive JEDI-based marine observation database➤ NSIDC L3 sea ice concentration data: 1979-2003 ➤ EMC L2 sea ice concentration data: 2003-2019

● Improved sea ice extents in both hemispheres: compared against observation● Leads to better estimation of total ice volumes: in decadal time scales● Consistent and reasonable analysis statistics maintained for the 40 year run:

observation-minus-model/background/analysis

NG-GODAS 40 Year Reanalysis


NG-GODAS 40 Year ReanalysisSea ice DA statistics: observation minus model (free run, analysis background, and analysis)

● Demonstrates the JEDI-based DA capability for the sea ice ocean coupled model system: UFS DATM-MOM6-CICE6

backup


WorkflowBasic Principles:● UFS

➤ Unified, not Unitary, balance of focus and diversity➤ Modular tools versus “one size fits all”

● Start from (functional) requirements, not solutions➤ Software engineering perspective➤ System engineering perspective

Outcomes of discussion:● One size does not fit all (latency versus resource use)● “Matrix” of elements, with modular library approach


Systems engineeringCritical efficiency aspects vary per application● For rapidity cycling systems, latency is likely to be the critical

factor➤ RRFS with DA step ?

● For slowly cycling systems, efficient use of assigned resources is likely the key efficiency factor ➤ GFS with single DA step ?➤ “Offline” product generation vs. model run !

XImplications of this for Unification:● Use shared common elements to generate a small set of

workflows

Workflow


The

“Lib

rary

”

Input processing Initialization /DA Model run Product Generation

Program(code config.)

code / repository management standards same for all

Obs processing IODA, restart files, etc.

Stand alone DA configuration

Stand alone model config. UPP, ensemble

processing tools, etc. Integrated DA / Model config.

“script” Standard execution environment for each element above, engineered to allow for stringing individual elements together

Run config. Standardized naming conventions for configuration of all “scripts”Automate combining configurations for scripts when used together

Functional scheduler

Workflows tailored for application / experimentWorkflow is a sequence of “library” elements

New capabilities are introduced as library elementsUFS selects community OS Scheduler

OS schedulerNCO adopts UFS scheduler, or EMC “translates” between schedulers

The “matrix view” of the workflow is a simplification to allow visualization. In order to implement this high-level vision, the next step is to develop a comprehensive analysis of all tasks (both

functional and developmental) associated with it


Organizing R2O / I2O

This is the next step following the Organizing Research to Operations report from 2018● We are developing

➤ a three-part report,u Assumptions, Best Practices, Requirements and

Constraintsu Describing the processu Roles, Responsibilities and Expectations

➤ With a shared preamble, and ➤ access to many existing pieces of documentation

https://docs.google.com/document/d/1KaMpf4sRdM67_4z113Ytpn3_1evdxggQpBHBWKVTxkY/edit%23heading=h.x83lrzhhzr4v


Part I, content

This is not a conventional report, but rather a gathering place of basic principles that have been discussed for yearsContent:● Purpose and scope● Assumptions, and requirements and constraints● Best practices● Managing risk● Desired outcomes


Part II, content Use case Describes Stage / Gate

1.Implementation in operations at NCO

Describe the general process of transitioning an implementation from EMC to NCO

4,5

2.GFS v16 implementation (2021)

Example of an actual implementation, including test plans and metrics.

(3),4,5

3.GEFS v12 implementation (2020)

Example of an actual implementation, including test plans and metrics. Showing differences in plans for different global model implementations.

(3),4,5

4.The annual HWRF operational upgrade cycle

Non-global operational implementation, showing differences with previous 2 use cases. Showing process with overlapping development and implementation cycles.

3,4,5

5.NGGPS selection of the FV3 dynamical core

Example of a “revolutionary” change, focusing on the selection of the new dycore

2,3

6.Prototyping coupled models

Incremental testing and development of the prototype coupled global UFS application

(2),3

7.GEFS v12 reanalysis and reforecast production (2020)

Example of a non-tradition “implementation” with high relevance for operational applications

Not defined / relevant

8.Statistical post processing Example of implementation that does not include the a core model upgrade, and therefore may not follow all steps in example 1.

3,4,(5)

9.A lower RL example Generic description of what is expected with lower maturity projects

1,2,(3)

10.Common Community Physics Package (CCPP)

Example of a new infrastructure introduced in the UFS, with different definition of Stage 5

2,3,5

11.NOPP project on operational wind wave model improvement

Example of a project outside of the UFS that resulted in operational implementations at NOAA through UFS shared code (pre-UFS)

All

Item Subject Status and Action Items

5.2.1 Supported code releases Some Applications of the UFS have been formally released to the public with appropriate support (e.g., MRW App 1.0 and SRW App 1.0). The UFS community needs to:1.Expand this to providing releases for all Applications.2.Address scheduling and resourcing releases.

5.2.2 Graduate Student Tests (GSTs)

GSTs have been used to test, illustrate and document the quality of initial releases of Applications. The UFS community needs to1.Formalize its approach to using GSTs.2.Address resourcing GSTs

https://ufscommunity.org/news/medrangeweatherapp/

https://ufscommunity.org/news/srwa/


Part III, content

Schema'cadaptedfrom‘DescribingR2OInterface’byUFS-SC,SIPWGs

ForecastPrio

ri,es

UFS-SCInformsForecastPriori,estoProgramOffices

TypesofR2OTransi0ons:AsoutlinedbyUFSSCandSIPWGs

Systemsleveltransi,ons-AmajoraspectoftheUFSisselectedfromacommunity-basedmodelrepository.Rela0velyinfrequentevent-Occurswhenanewapplica0onisbroughtin-oraspartofalong-termstrategytoaddressabasicmodelshortcoming.E.g.,Introduc0onofFV3dynamicalcore.Timelinefortransi0on-~5yearsApplica,onleveltransi,ons-Significantchangestoamodelcomponent-E.g.,advancingAtmosphericmodelPhysics.Timeline-monthsto2years

Incrementalleveltransi,ons-Morefrequent.Targetnarrowchangestoanexis0ngopera0onalsystem.Canbescien0fic,technical,and/orengineeringimprovements.E.g.,Inves0ga0onofsensi0vityoftheforecasttogrid-scalemixingparameteriza0ons.Rela0velyless0meconsuming.

T2O

ModelbiasesCustomerrequestsScienceques,ons

Use case Describes Stage / Gate

1.Inside NOAA:(UFS-R2O project)

The present processes inside of NOAA (the traditional NOAA Funnel)

All

2.NOAA FOs(JTTI)

NOAA funding a broader community to help develop the UFS

(2), 3, (4)

3.External Contributions(WW3 unstructured grids)

A project that started outside of NOAA that was adopted by NOAA operations at later Stages

(3),4,5

4.Bypassing NOAA(NOPP wave model development)

Work that is done with the UFS or component models that does not directly touch opun NOAA

N/A


Example Stage 1 Stage 2 Stage 3 Stage 4 Stage 5

1: Inside NOAA NOAA senior leadership makes final decisions with input from Subject Matter Experts and Stakeholders1.Focus on NOAA Subject Matter Experts (SMEs) and Stakeholders (e.g., MEG evaluations)2.Input from broader community (focus on stakeholders)3.Driven by metrics and test plans with community input4.Both for individual projects and prioritization between proposed / available project

2: NOAA FOs (R2O) 1.Projects: focus is on well-defined individual projects, with relatively short period of performance, reducing gatekeeping to establishing final RL (suggested by PI, confirmed by POC of receiving office)2.Prioritization: NOAA defines scope of projects in FO

3.Prioritization: proposals are peer reviewed with peers from the broad community

3: External contributions 1.Presently outside of the scope of the UFS, with respect to resourcing, prioritization and validation of Gates passed.2.In many cases relying on UFS / component model code management.

Same as in Example 1.

4: Bypassing NOAA Generally the same as in Example 3 Generic benefits such as code stability and efficiency and minor tools are achieved without formal governance, but depend on code management.

Part III, content Example Stage 1 Stage 2 Stage 3 Stage 4 Stage 5

1: Inside NOAA

2: NOAA FOs (R2O)

3: External contributions

4: Bypassing NOAAUFS–SC as “clearinghouse” for assigning Stages and managing

Gates for the broad UFS comunity

Table 5.2.1: Stage focus of Funnel examples; Darker colors identify stronger focus. Green identifies NOAA

focus, Yellow identifies external to NOAA focus.


Documents

An overview of the Unified Forecast System (UFS)