Integrating ocean climate observations Perspectives for consideration by the COSC Steve Hankin, NOAA/PMEL

Integrating ocean climate Integrating ocean climate observationsobservations

Perspectives for consideration by Perspectives for consideration by the COSCthe COSC

Steve Hankin, NOAA/PMELSteve Hankin, NOAA/PMEL

Oct '10Oct '10 OCO Annual ReviewOCO Annual Review 22

What do we mean byWhat do we mean by

integrated data integrated data

managementmanagement,,

anyway?anyway?iTunesiTunes®® as a metaphor ... as a metaphor ...(songs (songs datasets) datasets)


Mel Briscoe, ONR program manager,Mel Briscoe, ONR program manager,speaking about integrated data access:speaking about integrated data access:

““I’m not worried about I’m not worried about the research the research community. They will community. They will figure out a way to get figure out a way to get the data they need. the data they need. ””

A comment on the A comment on the culture of scientists …culture of scientists …


Informal guidance to the DMAC Steering Informal guidance to the DMAC Steering Committee (Ocean.US 2002):Committee (Ocean.US 2002):

““The financial industry has managed a The financial industry has managed a successful integration of its data. What successful integration of its data. What can we learn from them in order to can we learn from them in order to integrate ocean data?integrate ocean data?””

Lets look at this from a cultural Lets look at this from a cultural perspective. perspective. (irony aside )(irony aside )


Imagine a world of finance organized Imagine a world of finance organized as the world of ocean science is …as the world of ocean science is …

Many small, independent banksMany small, independent banks

Bank managers often cede financial data Bank managers often cede financial data management planning to department heads:management planning to department heads:– loan dept, mortgage dept, checking/savings loan dept, mortgage dept, checking/savings

dept., investments, …dept., investments, …


What habits for exchanging financial What habits for exchanging financial data would evolve?data would evolve?

Ad hoc means would dominate initially Ad hoc means would dominate initially (Excel spreadsheets and ASCII tables)(Excel spreadsheets and ASCII tables)

Some departments would develop Some departments would develop protocols for exchange with counterparts protocols for exchange with counterparts at other banksat other banks


How would we improve integration of How would we improve integration of data in such a world?data in such a world?

Difficult.Difficult.

Departments have developed systems that work Departments have developed systems that work adequatelyadequately

Making changes would be disruptive to Making changes would be disruptive to achieving short term financial (read “scientific”) achieving short term financial (read “scientific”) goals.goals.

Ref: The Innovator’s Dilemma, Clayton M. Christensen


How banking reality made integration easierHow banking reality made integration easier

A big bank chain needs integration A big bank chain needs integration internallyinternally– Creates an IT divisions to achieve itCreates an IT divisions to achieve it

Integration Integration betweenbetween big chains becomes easier: big chains becomes easier:– have a “culture of integration”have a “culture of integration”

management sees gainsmanagement sees gains

technical IT staff share outlookstechnical IT staff share outlooks

users rely upon itusers rely upon it


Technology alone is not the solution;Technology alone is not the solution;it is just one part of a strategy. it is just one part of a strategy.


What is the state of our “culture of What is the state of our “culture of integration” today?integration” today?

If we’re serious about developing “Climate If we’re serious about developing “Climate Services” then integration is no longer Services” then integration is no longer optional.optional.


The state of the ocean/climateThe state of the ocean/climate“culture of integration”“culture of integration”

Weak linkages across data life cycle:Weak linkages across data life cycle:– realtime delivery systemsrealtime delivery systems– scientific data assembly centersscientific data assembly centers– archivesarchives

Weak linkages across data typesWeak linkages across data types– platforms: ships platforms: ships vv. profilers . profilers vv. drifters . drifters vv. .

moorings, …moorings, …– modeling modeling vv. obs, …. obs, …– ““science data” science data” vv. end user data (geo-. end user data (geo-

political, commerce, …)political, commerce, …)


A cross-platform culture of integration A cross-platform culture of integration is developing …is developing …

Argo and OceanSites are converging on Argo and OceanSites are converging on interoperable formats using netCDF-CFinteroperable formats using netCDF-CF

But clear opportunities remain untapped:But clear opportunities remain untapped:

– Argo successes have not become the basis of Argo successes have not become the basis of a strategy for other profile data -- XBTs, a strategy for other profile data -- XBTs, CTDs, …. CTDs, ….

– OceanSites successes have not become a OceanSites successes have not become a basis for tide gaugesbasis for tide gauges

– drifters, cruise tracks, gliders, CPR, etc. do drifters, cruise tracks, gliders, CPR, etc. do not exploit opportunities to share standards not exploit opportunities to share standards


0.0. Applause for adding a data integration Applause for adding a data integration hat on the COSC ! hat on the COSC ! (Ken Casey) (Ken Casey)

1.1. Be a voice for data integration in Be a voice for data integration in strategic discussions strategic discussions (e.g. NOAA strategic (e.g. NOAA strategic plan, NOAA matrix management)plan, NOAA matrix management)

2.2. Coach management in creating a Coach management in creating a “culture of integration”“culture of integration”

3.3. Promote specific integration goals for Promote specific integration goals for metadata and data accessmetadata and data access

How can the COSC help to How can the COSC help to foster a foster a “culture of integration”?“culture of integration”?


2. Coach management on creating a2. Coach management on creating a“culture of integration”“culture of integration”

Funding opportunities that encourage Funding opportunities that encourage scientists and software developers to work scientists and software developers to work togethertogether

Give greater weight to contributions of data Give greater weight to contributions of data and data systems – not just of publications and data systems – not just of publications

“… the need for new partnerships and collaborations among domain scientists […] and data scientists [as well as] infrastructure development to support digital data; and the need for sustainable economic models to support [it]” NSB, 2007


3. Metadata integration3. Metadata integration

RT metadata is incompleteRT metadata is incompleteby its nature (QC, fall rate, …)by its nature (QC, fall rate, …)

strive for “good enuf” RT metadata (BUFR 2012)strive for “good enuf” RT metadata (BUFR 2012)

Metadata servers are here to stayMetadata servers are here to stay

the sensor’s ID is the the sensor’s ID is the “=““=“ that creates a solvable that creates a solvable equationequation


3. Metadata integration3. Metadata integration

Define the servicesDefine the services

Service(ID) Service(ID) theMetadata theMetadata

See OceanSites examples. See OceanSites examples.

Extend the success to Argo,Extend the success to Argo,

tide gauges, XBTs, drifters, …tide gauges, XBTs, drifters, …

For help with the metadata alphabet soupFor help with the metadata alphabet soup (ISO 19115, (ISO 19115,

SensorML, FGDC, CSW, )SensorML, FGDC, CSW, ) look to IT partners: NDBC and look to IT partners: NDBC and

Ted Habermann.Ted Habermann.


3. Integrated Data Access3. Integrated Data Access

Evolving …. Don’t look for a single, dominant Evolving …. Don’t look for a single, dominant

service any time soon.service any time soon.

Support multiple access techniques.Support multiple access techniques.

Prioritize based upon which services arePrioritize based upon which services are– popular with userspopular with users

ANDAND– illustrate a clear visionillustrate a clear vision


3. Integrated Data Access3. Integrated Data Access

A clear priority: A clear priority: netCDF-CF-OPeNDAP-THREDDSnetCDF-CF-OPeNDAP-THREDDS

OceanSites and Argo converging.OceanSites and Argo converging.

Still needing plans: tide gauges, XBT’s, drifters, Still needing plans: tide gauges, XBT’s, drifters, hydrographic,…hydrographic,…

Manage the progress !Manage the progress !

For help look to OSMCFor help look to OSMCand …and …

Oct '10Oct '10 OCO Annual ReviewOCO Annual Review 2020Oct. 2010Oct. 2010NOAA/UAF-gridNOAA/UAF-grid

The NOAA UAF project (Unified Access Framework):

OCO data assembly centers: make the data available

UAF: connect it to the scientists’ favorite “iTunes” (w/ volunteer energy)


ConclusionConclusion

““Management of data should have a status in Management of data should have a status in

a science organization that is as fundamental a science organization that is as fundamental

and unquestioned as management of and unquestioned as management of

personnel and management of budgets.personnel and management of budgets.““

“NOAA is, at its foundation, an environmental information generating organization …” NGSP 2010


Thank youThank you

Documents

Integrating ocean climate observations Perspectives for consideration by the COSC Steve Hankin, NOAA/PMEL