Marine Data Interoperability:What Will It Take?

John Graybeal

(MBARI)

Marine Metadata Interoperability Project

MMI, MBARI, & Moi

• MMI = Marine Metadata Interoperability– Community-based project– Almost 2 years old, 3(+) to go– Not owned (or driven) by…

• MBARI = Monterey Bay Aquarium Research Institute– For 5 years now (after NASA Ames)– Data systems, ces sont moi

Survey Question

What is the biggest problem with metadata in your domain?

Lessons from WHOI

Getting A Book: The Way It Used To Be

QuickTime™ and aTIFF (LZW) decompressorare needed to see this picture. QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

I need to special order a book.What book do you want?What book do you want?

It’s by Rachel Carson, Silent Spring.Yes, we can order that. It’s 23.95.Yes, we can order that. It’s 23.95.

When will you have it?Maybe Tuesday, but if it’s not in stock, not for a month.Maybe Tuesday, but if it’s not in stock, not for a month.

Can you send it to me?No, you’ll need to pick it up.No, you’ll need to pick it up.

OK, can I pay by phone?Yes, we’ll need your credit card and address.Yes, we’ll need your credit card and address.

OK, here’s my card info and address…OK, we’ll call you when it’s in.OK, we’ll call you when it’s in.

Getting A Book: The Way It Is Now

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Doing Science: The Way It Has Been

Hey Francisco, you still running that mooring in Monterey?Yeah, John, what do you need?Yeah, John, what do you need?

Is it collecting current data?Yeah, you looking for profiles?Yeah, you looking for profiles?

Yup, are you doing that with an ADCP?Yes, we’ll probably start profiling soon, too.Yes, we’ll probably start profiling soon, too.

No, don’t need that. How do I get the data?You want the raw data or QCd? Real-time?.You want the raw data or QCd? Real-time?.

Oh, the QC’d, delayed mode data, definitely, last 3 years worth.That’d be off our dods site, here’s the address..That’d be off our dods site, here’s the address..

Documentation come with that?We got some, I’ll have to get one of my techs to send it to you…We got some, I’ll have to get one of my techs to send it to you…

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Doing Science: The Way It Is and Will Be

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

Doing Science: Another Option

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Or…

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

And Furthermore…

Digital Data Center

GlobalGlobal

CoastalCoastal

RegionalRegional

3Observatories25 Sites100s Instrument types1000s Instruments 10000sParticipants

Tie It Together

ORION Conceptual Network Topology

Imagine a Trip…

From Technician…

… To Scientist

Looks Easy

But What About…?

QuickTime™ and aTIFF (LZW) decompressorare needed to see this picture.

What Will It Take?

(from ORION CI Concept of Operations)

What will it take for a WHOI researcher to access all the data from MBARI, NDBC, *and* ORION, assess the data that is useful for his or her project, bring it into an analysis program, and produce the first visualization all in a single day, and still have time to call one of the data producers and ask about the details of the latest quality control process?

What Else It Will Take  

Communities Creating Marine Metadata Interoperability

Communities Creating Marine Metadata Interoperability

Why? Because the problem is too large, too broad, and too heterogeneous to be solved by fiat.

How? Bring the communities together using information resources. Identify needs not addressed, and create teams from the community to address them. Create opportunities to develop shared solutions.

What’s needed for marine metadata interoperability?

We need community agreement in several areas.

• The way to transfer the metadata:Transport Protocol

• What data to transfer: Content Standard

• What the contents mean: Vocabulary

Transport Protocol

Content Standard

Vocabulary

We have some of these—so where’s the problem? (1)

EML

ISO

DCMI

FGDC

MarineXML

GML

ADL

NetCDF

ASCII

Content

Protocol

ESML

OPeNDAP

REST

SOAPZ39.50

DFDL

THREDDS LAS

WxS

HTTP

HDF

TIF

JPEG

Metadata Data

Coards/CF

We have some of these—so where’s the problem? (2)

Search for sea water temp data

Water Temperature

sea_water_temperature

TEMP

BODC

GCMD

CF

Only matches

Building a digital community—around metadata

• Find people who see the problems and want to work together to fix them

• Get funding to support basic infrastructure, distributed resources, technical leadership

Building a digital community—around metadata

• Create a community center

Discovering common ground

Use community center as a starting point.• One place with all the information• References• Reviews, comments• News, events• Content providerA meeting place for people, projects, andorganizations.

Identifying technical gaps—Vocabularies

• Few formal marinevocabularies exist

• There was no agreed format or location

• Most weren’t comprehensive or detailed• There was no way to connect them• There was no architecture to nurture them

– Like children, they need encouragement to grow healthy and strong, and to play well with others

Water Temperature

sea_water_temperature

TEMP

BODC

GCMD

CF

Vocabularies:Discovery and harmonization

DTDDTD

Comma Comma Separated Separated

ValuesValues

HTMLHTML

Tab Tab Separated Separated

ValuesValues

Relational Relational DatabaseDatabase

XML/XSDXML/XSD

RDFRDF

OWLOWL

Vocabularies: Harmonization with Voc2OWL

Relating Vocabularies: Mapping tool: Got VINE?

• Vocabulary INtegration Environment

Quick Example: Mapped Terms

Demonstrating Systems: Semantic Interoperability

Our Guides

• Robert Arko, LDEO• Julie Bosch, NOAA• Taco de Bruin, • Ben Domenico, Unidata • Karen Stocks, SDSC• Steve Hankin, NOAA -

Ocean.US/DMAC

• Mark Musen, Stanford Univ• Michael Parke, Univ of Hawaii• Lola Olsen, NASA Goddard• Bob Weller, WHOI• Dawn Wright, Oregon State

University

Steering Committee

Executive Committee

• John Graybeal, MBARI. (PI)• Philip Bogden, SURA/SCOOP

• Stephen Miller, SIO.• Francisco Chavez, MBARI.

• Stephanie Watson, Texas A&M

MMI:Your Handy Reference Guide

MMI: http://marinemetadata.orgHelp Line: ask@marinemetadata.orgPost an event: addthisevent@marinemetadata.orgPost an item: addcontent@marinemetadata.orgGet info re site: info@marinemetata.orgWorkshop: http://marinemetadata.org/workshop05Ontologies: http://marinemetadata.org/nsTerm Search: http://mmi.mbari.org:9600/mmi2/search.jspDemo: http://marinemetadata.org/demo