1
Thomas Karl Director National Oceanic and Atmospheric Administration
National Climatic Data Center
AMS 88th Annual MeetingTown Hall Meeting January 21,2008
NOAA Reanalysis
User Needs
AMS Town Hall Meeting
2
Outline
Purpose of MeetingOverview of NOAA Reanalysis EffortsClimate Forecast System Reanalysis and
Reforecast (CFSRR):– Data Availability Plans– Data Access Optimization
User Input and Open Discussion
3
Has the proposed file organization and groupings reflect current user needs and expectations?
Which data need to be placed on-line given e.g., 150TB of disk? Are we on the right track? Do we need to re-group data sets to
create needed products – such as initialization files, analysis, or by forecast projections?
Can some other mechanism (i.e. LDM) provide some CFSRR output to users in near real-time, shifting the load from the archive?
NCDC and NCEP are committed to provide the highest resolution if possible. Tonight’s discussion is a focus on user needs tempered by available resources.
Tonight’s discussion is a focus on user needs
Open Forum and User Input
4
New NOAA Reanalysis Projects
1) Historical SFC Reanalysis (Compo et al.,)– 1850 to present. ~60TB
2) Post WW-II Reanalysis (NCEP)– 1944 to present. ~235TB
3) Climate Forecast System Reanalysis and Reforecast (CFSRR) Project (Saha et al.)– 1978-2008 Reanalysis and Seasonal Reforecast:
915TB (Grib1)
Development of a Reanalysis Clearinghouse– Capability for an “on-going analysis of the climate
system”
5
CFSRR Overview (Saha, EMC/NCEP) Coupled Climate Forecast System (CFS) Reanalysis and Reforecast
(CFSRR) (Atm - Ocn - Land – SeaIce) has two parts:
– Reanalysis: 31-year (1979-2009) T384L64 (~32 km and 64 hybrid
layers for the atmosphere, 0.50 and 40 levels for the ocean, 4 soil
levels)
– Reforecast: 28-year (1982-2009) T126L64 (~32 km and 64 hybrid
layers for the atmosphere, 0.50 and 40 levels for the ocean, 4 soil
levels)
– 6 hourly Reforecast for 1 year
NCDC and NCEP are seeking community input for archive and access
priorities to this massive dataset
Approximately 915 Terabytes in GRIB1.
Note: this presentation assumes conversion is needed to Grib1 as many user
applications cannot yet deal with GRIB2 directly (including NCDC).
6
Proposed CFS Reanalysis Data
CFS High Resolution Initial Conditions: 72 TB CFS Low Resolution Initial Conditions: 35 TBFull Data Ingest : 46TB (satellite radiances,
RAOB, in-situ, etc.)Pgbh Pressure Grib Files 0.5 x 0.5 Hourly: 91TB Flxf T382 Gaussian Hourly: 25TBOcnh 0.5 x 0.5 Hourly: 18TBDiabf 1.0 x 1.0 Hourly: 27TB Ipvh 0.5 x 0.5 Hourly: 17TBMonthly: 1TB
TOTAL: 332 TB
7
1. Siganl : 3-D Hybrid Analysis
2. Sfcanl : Surface Analysis
3. Ocnanl : 3D Ocean Analysis
4. Pgbh : 3-D Pressure level data
5. Flxf : SFC fluxes, radiative fluxes, precip.
6. Ocnh : 3-D Ocean data
7. Ipvh : 3-D Isentropic level data
8. Diabf : Diabatic Heating, Moistening rates, etc.
9. Egyh : Energetics, u’ v’, TKE, etc.
Proposed CFS Reanalysis Hi-Resolution File Level Organization
T382L64 + Ocean (1/4 x ½) 6 hourly = 72TB T126L64 + Ocean (1/2 x 1) 6 hourly = 35 TB
0.5 x 0.5 degree Hourly= 91TB
T382 Gaussian Hrly= 25TB
0.5 x 0.5 degree Hourly= 18TB
1.0 x 1.0 6-Hourly: 27TB
0.5 x 0.5 degree Hourly: 17TB
0.5 x 0.5 Monthly: 1 TB
8
CFS 28-Year Reforecast Data1982-2009
6-hourly Pressure Grib: 412TB6-hourly Ocean : 102TB6-hourly Ipv: 78TB Monthly and Time-series: 3TB
1.0 x 1.0 degree for first 6 months of forecast
2.5 x 2.5 for next 6 months of forecast
TOTAL: 595 TB
9
National Academies Recommendation
National Research Council, Board of Atmospheric Sciences and Climate (BASC):
“Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather
and Climate Forecasts”
Recommendation 3.4:
The NOAA National Operational Model Archive and
Distribution System (NOMADS) should be maintained and
extended to include (a) long-term archives of global and regional
ensemble forecasting systems and their native resolution, and
(b) re-forecast datasets to facilitate post-processing”
10
Open Discussion
Proposed Data and System Access Priorities
11
1) Basic ftp and http scripting services to the highest resolution data though the NCDC tape archive services.
2) NOMADS on-line sub-setting services for most requested data (tonight’s discussion).
3) Advanced GIS and other Web Based Portal Services as resources and time permit
Issue: Data access may be throttled based on number of concurrent users, I/O restraints, and communications bandwidth at NCDC.
CFSRR System Access Prioritization 1
12
Reanalysis Data Access Prioritization 1
Highest Resolution CFSRR data provided– Low Res users can subset or use Post WW-II– Access assisted by user feedback as to file
level organization (this meeting!)Historical “20th Century” SFC Reanalysis
– 60TB (Grib and HDF) FY10 Post WW-II U/A Reanalysis
– Low resolution 6-hourly (235TB Grib-1)– When funded
13
FEEDBACK and QUESTIONS
CFSRR PI
nomads.ncdc.noaa.gov
OPEN DISCUSSION
14
Backup Slides
15
• To overcome a deficiency in model data access, some of the Nations top scientists are actively engaged in a grass-roots framework to share data and research findings over the Internet.
• NCDC, NCEP and GFDL initiated the NOAA Operational Model Archive and Distribution System.
• NOMADS is a distributed data services pilot for format independent access to climate and weather models and data.
Overview
16
• foster research within the geo-science communities (ocean, weather, and climate) to study multiple earth systems using collections of distributed data;
• promote model evaluation and community feedback;
• develop institutional partnerships and access via distributed open standard technologies.
• Establish a unified climate and weather model archive providing format independent access to retrospective models;
NOMADS GoalsNOMADS Goals
The NOAA Operational Model Archive and Distribution System
17
Direct Client Access
NCEP Dual Ingest & QC
Portals
GDS and TDS
Live Access Server
GrADS, Ferret, MatLab, IDL, IDV, Web browsers or any OPeNDAP enabled client
• Project ACCESS: NASA,
GMU, GMU & OPeNDAP• Project GALEON Unidata• NOMADS Web-Plotter,
http, ftp & binary subsetting• GIS access (WCS, W*S)
NCDC Archive
CEOS-Grid
NOAA-wide LAS“sister-servers”
Exploratory Grid Projects w/ Globus
Collaboration Focus
The NOMADS System Design
Multiple paths to format independent data access
18
NCDC HDSS Access System(HAS = NCDC Archive)
NOMADSModel data
CLASSSatellite
NEXRADRadar CDO, GIS
ServicesIn-situ
Data requestfrom accessservice—eg,NOMADS
Data delivery to access service—eg,NEXRAD
A very simplified diagram of data flow from NCDC‘s HDSS tape robotics system & incoming sources
Other incomingdata
MajorAccessServices
NOAA Customers
Recommended
