NPOESS EDRs vs. Climate Data Records (CDRs)

NRC April 23, 20071

NPOESS EDRs vs. Climate Data Records (CDRs)

John J. Bates, ChiefRemote Sensing Applications Division

NOAA’s National Climatic Data Center

NRC April 23, 20072

Outline

What is a CRD and why should NOAA produce CDRs?

Steps Toward Operational Production of Climate Data Records – Scientific Data Stewardship– Prioritization– Production– Productivity– Organization

Conclusion

NRC April 23, 20073

NOAA’s Scientific Data Stewardship rooted in NRC dialogue and reports

NOAA/NRC SDS leads

– Bates

– Goldberg

Scientific Data Stewardship

NRC April 23, 20074

NRC – NOAA Response

With the transition of the U.S. Global Change Research Program into the Climate Change Science Program (CCSP), NOAA was identified as the lead U.S. Agency on climate.

“NOAA’s new climate mandate is fundamentally different from its traditional weather forecasting mandate and raises a new set of challenges owing to the varied uses of climate data, the complexities of data generation, and the difficulties in sustaining the program indefinitely.” (NRC)

In response to this change and the planned transition of NASA research climate observing missions to the NPOESS mission, the Scientific Data Stewardship Project was proposed to begin the operational production of climate data records.

NRC April 23, 20075

Key Elements of a Successful CDR Program

CDR Organizational Elements

• High-level leadership council

• Advisory council to represent climate research community and other stakeholders

• Fundamental Climate Data Record (FCDR) Teams

• Thematic Climate Data Record (TCDR) Teams

CDR Organizational Elements

• High-level leadership council

• Advisory council to represent climate research community and other stakeholders

• Fundamental Climate Data Record (FCDR) Teams

• Thematic Climate Data Record (TCDR) Teams

CDR Generation Elements

• High accuracy and stability of FCDRs

• Pre-launch characterization of sensors and lifetime monitoring

• Thorough calibration of sensors

• Well-defined criteria for TCDR selection

• Stakeholder involvement and feedback for TCDRs

• Well-defined criteria for TCDR validation

• Use of in-situ data for validation

CDR Generation Elements

• High accuracy and stability of FCDRs

• Pre-launch characterization of sensors and lifetime monitoring

• Thorough calibration of sensors

• Well-defined criteria for TCDR selection

• Stakeholder involvement and feedback for TCDRs

• Well-defined criteria for TCDR validation

• Use of in-situ data for validation

Sustaining CDR Elements

• Available resources for reprocessing CDRs as new information becomes available

• Provisions for feedback from scientific community

• Long-term commitment of resources for generation and archiving of CDRs and associated data

Sustaining CDR Elements

• Available resources for reprocessing CDRs as new information becomes available

• Provisions for feedback from scientific community

• Long-term commitment of resources for generation and archiving of CDRs and associated data

Fundamental Climate Data Record (FCDR): Time series of calibrated signals for a family of sensors together with the ancillary data used to calibrate them.

Thematic Climate Data Record (TCDR): Geophysical variables derived from FCDRs, often generated by blending satellite observations, in-situ data, and model output.

A Climate Data Record (CDR) is a time series of measurements of sufficient length, consistency, and continuity to determine climate variability and change

NRC April 23, 20076

Sensor DataRecords (SDRs)

Data (Direct & Remotely Sensed)

Fundamental Climate Data

Records (FCDRs)

Thematic Climate Data Records

(TCDRs)

Climate Data Records or Homogenized Time Series

Homogenization and Inter-Calibration

Time-tagged Geo-Referenced & Calibrated

Converted to Bio-Geophysical

Variables

EnvironmentalData Records

(EDRs)

Converted to Bio-Geophysical

Variables

What is a CDR vs. EDR?Climate Data Records

NRC April 23, 20077

Only CDRs Provide the Accurate Rate of Global Warming

We require Intercalibrated CDRs in order to merge multiple, legacy observing systems

(Red is EDR, Blue is CDR)

NRC April 23, 20078

Is There an El Niño?Expect the Unexpected

Volcanic eruptions contaminate infrared-only SSTs. Lesson is that we Volcanic eruptions contaminate infrared-only SSTs. Lesson is that we must correct and reprocess EDRs to CDRs for critical climate variablesmust correct and reprocess EDRs to CDRs for critical climate variables

NRC April 23, 20079

CDRs Seamlessly Transition Over Major Technology Upgrades: AVHRR to MODIS

NRC April 23, 200710

Operational Climate Data Records –Prioritization, Production, & Productivity


Brief History of Prioritization

Mid-June 2006 CCSP Prioritization Workshop– Included WG Representatives– Had difficulty in balancing satellite measurement

priorities with in-situ priorities – and priorities of different communities

Initial Development of a Hypertext Alternative– Workshop constrained by 2-D spreadsheets– Hypertext (Web-based) navigation approach

provides better balance between complexity and understandability

– Led to a representation showing how Essential Climate Variables interact withScience IssuesSocietal Benefits


Prioritization Approach

CCSP and Other Groups have identified ~40 “Essential Climate Variables” (ECVs), e.g.– Total Solar Irradiance– Snow Cover– Sea Level

Agencies involved in CCSP work have identified– Thematic Areas where data may provide societal benefit– Scientific Questions where data is needed for answering research

questions Prioritization aided by weighting contribution of ECV data in

both Thematic Areas and Scientific Questions Started with Spreadsheet – then have been extending

weighting approach to database with web interface Creates a priority according to connection and breadth of

impact– Trying to avoid disciplinary myopia– Trying to ensure considering both scientific interest and societal

benefits Adding additional information based on availability of current

and past data sets


Societal benefits of the Global Earth Observations System of Systems (GEOSS) emphasize the tradition of applied climatology but using 21st century systems

Key Science Intergovernmental Panel on Climate Change (IPCC) Issues address the policy-relevant scientific research topics of highest priority for Climate and Global Change studies

Prioritizing Observations and Climate Data Records – Societal Benefits and Key Science

Issues


Prioritization - Future Work

Extend Input Opportunities– Creating web site with extensive information on

satellites, instruments, in-situ data sources, and data products

Need Deeper and More Rigorous Approach to Prioritization– Can we quantify the economic value of climate

data?– Can we quantify the required “Signal to Noise”

for climate data on a rigorous basis?Rigorous error budgets – and identification of physical

basis for error estimatesObjective approaches to weighting influence of

different variables on climate change (e.g. OSSEs)


NOAA CDR Production - Climate Central Requirements

– For reprocessing, SDS requires an IDPS-like system, (Climate Central) to process:

– SDS interdependent with CLASS, e.g., large data set I/O

– CLASS and SDS mutually dependent

– Requires full production engineering

NOAA/CLASS Users

ACQUIRE RDRs

IMPROVED SDRsMULTISATELLITE

FCDRs

PROCESS TCDRs

PROVIDEASSESSMENTSIMPROVEMENTS

NOAASDS

CLIMATECENTRAL

Products and Services

Provide FCDRs and TCDRs

RDR SDR Multi-satellite FCDRs & TCDRs


Production Example - Sea Surface Temperature

Linkages & processes needed to

create daily SST product


Satellite Climate In-Situ Model

NOAA POES NASA AMSR

GTS Buoy & Ships

NCEP Sea Ice

Production - Sea Surface Temperature

Observations



Data Management / Processing

All data management and processing performed at NCDC



Analysis / Examples of Expected Uses


Hurricane intensity historically estimated from “best track” data, in spite of its inherent temporal heterogeneities

An objective reanalysis of homogeneous 23-year satellite data set produced by NCDC

U Wisconsin developed an objective analysis algorithm to work with NCDC data.

“UW/NCDC” intensities have little temporal bias

Productivity - Hurricane Intensity Reanalysis

~169,000 images~2,000 tropical cyclones


Climate Working

Group

NOAA Science Advisory

Board


CLASS Working Group


Program Management

Research Climate Data Science Teams

FCDR Teams

Observations Scoping

Requirement Systems

C2D2 NCDC ORA

FY06EDSM $2.5M

C2D2 $1 to $1.5MGovernance and Management Structure

TCDR Teams

R&D Products and

Services Theme Areas

NOAA

(5-10%)

External

(1-2%)

NOAA, Other

Agencies External Scientific

(~90%)

NOAA Climate Program & Climate Board

NOAA Observing System Council

Operational CDR Generation & Data Mgmt.

CDR Generation

NOAA

Other Agencies

Universities

Private Sector

CLASS

NOAA IT Infrastructure

operators

Research to

Operations

Currently exists FY05 FY06

NOAA’s Scientific Data Stewardship Program

21


Conclusions

NPOESS EDRs CDRs, but are a helpful first step

The foundation of operational production of satellite CDRs has begun within NOAA

Prioritization, production, productivity, and organization have been scoped and analogs begun using current data sets

Resource requirements for SDS are now being captured in the NPOESS climate remanifesting exercises

Documents

NPOESS EDRs vs. Climate Data Records (CDRs)