Data Support for Climate Research from NCAR

Steven WorleyDoug SchusterJoey Comeaux

4th ACRE Workshop, EC FP7 EURO4M, ERA-CLIM, AHRC Historic WeatherKNMI, De Bilt, The Netherlands, 21-23 September 2011

Topics

Research Data Archive at NCARData Service CommunitiesData Content

Reanalyses ObservationsAnalyses

Future

Research Data Archive (RDA) at NCAR

Purpose - support climate & weather research by curating, organizing, documenting, and creating access to research datasets

To extent possible, all data and services are free and open

Take Away Metrics Established in 1960s 600+ datasets, 4M files, 600 TB 7000 unique users annually Many datasets routinely updated

daily, weekly, monthly

Research Data Archive (RDA) at NCAR

Core Data Categories

Operational and Reanalysis model outputs

Meteorological and Oceanographic Observations

Remote Sensing Observations

Topography/Bathymetry, Vegetation, Land Use

RDA Infrastructure Components, w/o Network

DAV

SC

GLADERDA Web

HPSS

Data Service Communities

Self-help Users

DAV

SC

GLADE RDA Web

HPSS

Self-help Users

GLADE, 155 TB, 125 DatasetsHPSS, 600 TB, All (600+) DatasetsAutomated Services move data from HPSS to GLADE as needed

File Flow

DAV

GLADE RDA Web

HPSS

Assisted User

Assisted Users, One-off Requests

SC

Assistance:- Data-selection interfaces and

automated processing- Data processed by RDA staff

Computational Service

DAV

SC

GLADE RDA Web

HPSS

NCAR Users

NCAR Users – two pathways Reference Only

1 1

1

2 2

2

Data Content

Reanalyses Observations Analyses

Reanalyses

10 Major Collections, providers NCEP, ECMWF, JMA, NOAA-CIRES Start Year: aligned with satellite era, exceptions: NNR, ERA-40, 20CR

ERA-I recently pre-pended to 1979, not in RDA now End Year: Most are ongoing Resolution:

Highest shown here – lower resolutions also available Increasing resolutions => more storage and need for subsetting

E.g. NCEP CFSR v1 is 80TB, online

Reanalyses

Format Native GRIB1 or GRIB2 Conversion – limited capability now

Subsetting By variable/parameter – only newest ones By date – newest ones on specific days, some capability from file organization By vertical level – only newest ones Geospatial – not available

Reanalyses Usage Examples (CFSR, 20CR)

Observations

5 notable long-term collections: ICOADS, ISPD, UADB, WMSSC, NCEP Surf. Ops. Start Year: push back to the 1600-1700’s End Year: Mostly ongoing Format: Many Native, conversion to ASCII and one to netCDF Subsetting: Generally, complete temporally and geo-spatially Truth – This is really time consuming work

Observations Examples: ICOADS, ISPD, WMSSC

Start Date: ~January 2010 ICOADS, 30% requesting subsetting

Monthly updates have increased routine file access ISPD, 50% users subset, probably for ASCII outputWMSSC, many users, small data volume

Observations: UADB UpdateBasic Features ~ 2,500 Total Stations from 30+ sources Pibals, 1920-2011 Raobs, 1943-2011

< 1

1

5

10

15

20

25

30

35

40

45

> 50

0 50 100 150 200 250 300 350 400

# of Stations by Period of Record Length

# of Stations

Perio

d of

Rec

ord > 700 stations with 30+ years

Observations: UADB Update

Status Homogeneous units and format Updated monthly with NCEP operational data Interface access; two sources 1973-2011

http://dss.ucar.edu/datasets/ds370.0 All data available by requestGoals Complete access to CHUAN

“Corrected" stations, ingested, undergoing validation Improve station history metadata

Form longer time series, upon user selection Extend interface availability to earliest records

Complete QC research and implement Add aircraft data Coordinate with NCDC, bilateral completeness

Analyses

Surface: SST, SLP, ocean fluxes; Ocean 3D: T&S, and V Start Year: push back to 1850’s at best End Year: Mostly ongoing Format: a mixture Subsetting: not extensive

Future

Complete ERA-I back to 1979 Add products from ERA-CLIM Project? CFSR-lite

NOAA, 1949(?)-> current, T126Retirement of NCEP/NCAR Reanalysis

Arctic System Reanalysis, OSU/UA/NCAR Initially, 11 years

Improvements to UADB More subsetting and more format conversion Expansion into interoperable web access

Wrap up

Research Data Archive at NCAR Well designed for climate research support

Data Service CommunitiesConvenient for many worldwide

Data Content: Reanalyses, Observations, AnalysesSubstantial with increasingly easy access options

FuturePlans to grow content, and improve current archives

http://dss.ucar.edu