Reconstituting the Ocean: Reconstituting the Ocean: a tale from U.S. JGOFSa tale from U.S. JGOFS

Cyndy Chandler (MCG, WHOI)Cyndy Chandler (MCG, WHOI)U.S. JGOFS Data Management OfficeU.S. JGOFS Data Management Office

and Ocean Carbon and Biogeochemistry Coordination and and Ocean Carbon and Biogeochemistry Coordination and Data Management OfficeData Management Office

10 August 200510 August 2005

MBARI-MMI Marine Metadata WorkshopMBARI-MMI Marine Metadata Workshop

Boulder, COBoulder, CO

10 August 200510 August 2005 U.S. JGOFS DMOU.S. JGOFS DMO 22 of 17 of 17

QuestioQuestion:n:

What do users want from metadata What do users want from metadata anyway?anyway?

This presentation will focus mostly on usage This presentation will focus mostly on usage metadata.metadata.

10 August 200510 August 2005 U.S. JGOFS DMOU.S. JGOFS DMO 33 of 17 of 17

IntroductionIntroduction• What is U.S. JGOFS?What is U.S. JGOFS?

Ocean reconstitution explainedOcean reconstitution explained• merged products merged products • metadatametadata

Topics in today’s Topics in today’s presentation:presentation:

10 August 200510 August 2005 U.S. JGOFS DMOU.S. JGOFS DMO 44 of 17 of 17

What is U.S. JGOFS?What is U.S. JGOFS?

part of multinational JGOFSpart of multinational JGOFS U.S. Global Change Research Program (US GCRP), U.S. Global Change Research Program (US GCRP),

Scientific Committee on Oceanic Research (SCOR), and Scientific Committee on Oceanic Research (SCOR), and International Geosphere-Biosphere Programme (IGBP) International Geosphere-Biosphere Programme (IGBP)

long term (U.S. 1989-2005)long term (U.S. 1989-2005) multidisciplinary (bio, chem, PO, geology)multidisciplinary (bio, chem, PO, geology) process studies (U.S. 1989-1998), time-process studies (U.S. 1989-1998), time-

series, global surveys, synthesis and series, global surveys, synthesis and modeling, data managementmodeling, data management

investigate ocean carbon fluxinvestigate ocean carbon flux

U.S. Joint Global Ocean Flux U.S. Joint Global Ocean Flux StudyStudy

10 August 200510 August 2005 U.S. JGOFS DMOU.S. JGOFS DMO 55 of 17 of 17

Lessons Lessons Learned . . .Learned . . .metadata is of critical importance metadata is of critical importance

accurate, complete, available with data

monitor emerging standards

define minimum metadata requirements

standards-compliant solutions where possible

complete metadata record enables reuse of data

metadata assembly is time consuming, but is the key to enabling secondary (reuse) of data

10 August 200510 August 2005 U.S. JGOFS DMOU.S. JGOFS DMO 66 of 17 of 17

metadatametadata

documentation documentation providing providing

answers to the answers to the who, what, who, what,

where, when, where, when, why and how why and how

questions questions

10 August 200510 August 2005 U.S. JGOFS DMOU.S. JGOFS DMO 77 of 17 of 17

Lessons Lessons Learned . . .Learned . . .

quality assurance is an ongoing process

intense QA process during initial acquisition and ingestion into data system

problems discovered as data are utilized by others

◊ insufficient or inaccurate metadata

process of data product synthesis becomes a valuable diagnostic tool for improving data quality

10 August 200510 August 2005 U.S. JGOFS DMOU.S. JGOFS DMO 88 of 17 of 17

Data SynthesisData Synthesis

merged data productsmerged data products

combine all data from the same combine all data from the same sampling device for a defined sampling device for a defined geospatial area and time rangegeospatial area and time range

“attempt to “attempt to put the ocean back put the ocean back togethertogether””

10 August 200510 August 2005 U.S. JGOFS DMOU.S. JGOFS DMO 99 of 17 of 17

U.S. JGOFS Process Study U.S. JGOFS Process Study AreasAreas

10 August 200510 August 2005 U.S. JGOFS DMOU.S. JGOFS DMO 1010 of 17 of 17

CTD photosCTD photos

let the let the frenzy frenzy begin!begin!

10 August 200510 August 2005 U.S. JGOFS DMOU.S. JGOFS DMO 1111 of 17 of 17

Arabian Arabian SeaSea

Niskin Niskin Bottle Bottle

DataData 73 data 73 data

entitiesentitiesfromfrom

6 cruises 6 cruises in 1995in 1995

data data contributed contributed

by 46 by 46 different PIsdifferent PIs

10 August 200510 August 2005 U.S. JGOFS DMOU.S. JGOFS DMO 1212 of 17 of 17

Results of Data SynthesisResults of Data Synthesis merged products for bottle and CTD datamerged products for bottle and CTD data bottle data products include many bottle data products include many

parameters: parameters: Arabian Sea > 110Arabian Sea > 110

North Atlantic > 80North Atlantic > 80

Equatorial Pacific > 110Equatorial Pacific > 110

Southern Ocean > 170Southern Ocean > 170

10 August 200510 August 2005 U.S. JGOFS DMOU.S. JGOFS DMO 1313 of 17 of 17

The Merging ProcessThe Merging Process

establish rulesestablish rules• which data entities to combine?which data entities to combine?

common sampling devicecommon sampling device

• what determines the columns and rows?what determines the columns and rows? could not have merged this many could not have merged this many

different data entities without different data entities without sufficient metadatasufficient metadata

10 August 200510 August 2005 U.S. JGOFS DMOU.S. JGOFS DMO 1414 of 17 of 17

Columns and RowsColumns and Rows

each column contains data from a each column contains data from a different parameter (variable) different parameter (variable) parameter names are uniquely defined by parameter names are uniquely defined by

long name (definition of term), units long name (definition of term), units (encoding) and methodology(encoding) and methodology

rows are uniquely defined by X,Y,Z rows are uniquely defined by X,Y,Z and Tand Tusually this could be done by matching usually this could be done by matching

time, longitude, latitude, depth, station, time, longitude, latitude, depth, station, cast and bottle numbercast and bottle number

10 August 200510 August 2005 U.S. JGOFS DMOU.S. JGOFS DMO 1515 of 17 of 17

Property DepthProperty Depth

LAS LAS v6v6

Property-Property-DepthDepth

Arabian SeaArabian SeaNitrateNitrate

from merged from merged product product

generated by generated by DMO from DMO from in situin situ Niskin bottle data Niskin bottle data

10 August 200510 August 2005 U.S. JGOFS DMOU.S. JGOFS DMO 1616 of 17 of 17

The ChallengeThe Challenge

the originating PI doesn’t need an the originating PI doesn’t need an explicit, written record of metadata explicit, written record of metadata during the initial analysis of their during the initial analysis of their data data

the metadata supplied with a data the metadata supplied with a data contribution is often insufficient to contribution is often insufficient to support secondary use of datasupport secondary use of data

10 August 200510 August 2005 U.S. JGOFS DMOU.S. JGOFS DMO 1717 of 17 of 17

SummarySummaryWhat do users want from metadata anyway?What do users want from metadata anyway?

users require sufficient information to:users require sufficient information to: determine data of interestdetermine data of interest determine quality of data sourcedetermine quality of data source answer basic meta questions: who, answer basic meta questions: who,

what, where, when, how (and what, where, when, how (and sometimes why)sometimes why)

10 August 200510 August 200509 09 August 2005August 2005

U.S. JGOFS DMOU.S. JGOFS DMO 1818 of 17 of 17

U.S. JGOFS web siteU.S. JGOFS web sitehttp://usjgofs.whoi.ehttp://usjgofs.whoi.edudu