Briefing on the Integrated Ocean Observing System (IOOS) and the

Data Management and Communication System

Steven WorleyNCAR

19 August 2005

Objectives• What is IOOS?• What is DMAC?• How might OHH participate?

The Integrated Ocean Observing System (IOOS)

Oceans & Coasts Component of the U.S. Integrated Earth Observation System (IEOS) & the International GEOSS

NOAANavyNASANSF

USACE

USGSMMSEPA

USCGDOE

An Interagency Collaborationfor the Public Good

http://www.ocean.us

IOOS GoalsIntegration for Broad

Benefit

1. Detect and Predict Change 2. Mitigate effects of natural hazards3. Improve safety and efficiency of marine ops4. Improve national security5. Reduce public health risks6. Protect and restore marine ecosystems7. Sustain marine resources

Society oriented goals

E.G. Florida Fish and Wildlife Research InstituteRed tides off southwest Florida, Aug. 17

Reported system effects:• Expanding area since

last week• Benthic mortalities• Fish kills• Discolored water• Turtle mortalities• Human respiratory

irritation

http://www.floridamarine.org/

E.G. Florida Fish and Wildlife Research InstituteRed tides off southwest Florida, Aug. 17

Actions:• Airplane over

flights – not good weather – more flights needed

• Ocean sampling cruise 10-12 August

E.G. Florida Fish and Wildlife Research InstituteRed tides off southwest Florida, Aug. 17

Observation and what ifs?• Respiratory irritation greater with westerly wind• What if Hi-res wind predictions were available?• What if satellite sensed data, not impacted by

weather, was easily used? • What if real-time ocean circulation models

were run to analyze and predict?• What if ocean models with T, O2, nutrients, red

tide samples, were easy to run?

Contention:

We have the ability to do these estimates

IOOS strives to enable them and address the goals previously shown

The Political Environment

• Commission on Ocean Policy Report

• Executive Order Ocean Action Plan

• Legislation Senate (S. 361 -

recently passed) House (H.R. 1489,

1584)

IOOSTwo Interdependent Components

Global Ocean Component

Coastal OceanComponent

NationalBackbone

RegionalCoastal Observing

Systems

IOOS Coastal Ocean Component

Regional Coastal SystemsRegional Coastal Systems

• Largely non-federal (may be Largely non-federal (may be federally funded)federally funded)

• Regional priorities & needsRegional priorities & needs• Greater resolutionGreater resolution• More variablesMore variables

National BackboneNational Backbone

• Largely federalLargely federal• Satellite remote sensingSatellite remote sensing• Reference, fixed-site stationsReference, fixed-site stations• Links to global componentLinks to global component• Fewer variablesFewer variables

Both use IOOS data standards and exchange protocols

The Global Component

An International Collaboration

IOOS is the U.S. contribution to international IOOS is the U.S. contribution to international GOOSGOOSand the Oceans & Coasts component of IEOS & and the Oceans & Coasts component of IEOS & GEOSSGEOSS

IOOS Three Subsystems

1. Measurements

• Remote and in situ observations

• Telemetry

2. Data Management and Communications (DMAC)

• Integration

3. Data analysis and modeling

• Data synthesis, evaluation

• Change detection and prediction

IOOS Education Component

Ocean.US Education Workshop, 22-24 March 2004, Charleston, SC

Result: Workshop Report

Promoting Lifelong Ocean Education: Using the Integrated Ocean Observing System (IOOS) to Shape Tomorrow’s Earth Stewards and Science and Technology Work Force.

Muriel Cole Ocean.US liaison to OHH

• Hear more at IOOS Human Health Workshop, January 2006

DMAC is the component that enables the diverse

sources of data to be accessed in a consistent

manner.DMAC is composed of

data standards, communications

protocols, software tools, archiving and community

agreements.

Why so much focus on DMAC?

Why is DMAC the top priority?

• DMAC is the “I” in IOOS

• The weak link in the chain • Huge payoff from investment

Data Supplier

Data User

1.Do no harm

Data Suppliers

SatelliteMulti-beam

Geo-chemical

2.Expand access

3.Increase efficiency

DMAC Three Guiding Principles

DM

ACExisting Protocol

The DMAC Plan is a framework for interoperability

among independent, heterogeneous systems, large

and small• Organizations will manage their data

independently• Standards – enable interoperability

– Data discovery and delivery to applications• New - “Web services” standards

– communication between applications– E.g. Browsers that initiate Google searches

The heart of integration is StandardsExample: Surface Marine Buoys

• “Work horse” for coastal and open ocean observations

• International, National and Regional use

• Surface , near-surface and subsurface observations

What “Simple Standards” buy you today…

• Meeting present “standards”• A “ticket” onto GTS • Real-time provides utility for

national forecasts & warnings operations

• “Accepted” QA /QC

• Immediate payoffs– Large user base– Improved local storm detection,

intensity and track forecasts– Beach rip tide, boating safety and

trip planning, & others

But today’s standards are too narrow. Inefficiencies and lost opportunities …

• E.g. Delayed mode not standardized • Data not uniformly

• archived• QCed

• Data not uniformly available for • climate assessment• commercial applications• research (esp. interdisciplinary)

• Commonalities are not exploited (e.g. a time series)– Variables: winds, waves, temperature,

salinity, biota, chemicals, fish landings– Platforms: surface marine buoys,

terrestrial stream gauges, tide gauges– Products: forecasts, satellite products,

coral bleaching (paleo), El Nino index

IOOS Data Management andCommunications

Subsystem

Users

Archive Centers

Modeling

On-line Browse

Products

ForecastsMaps

RegionalData

ManagementSystems

InternationalData

ManagementSystems

Terrestrial and AtmosphericData Management

SystemsMetadata, Data Discovery

and Data TransportStandards and Protocols

Ships

Satellites

Primary DataAssembly & QC

Hand Measurements

Moorings

Floats

IOOS Data Management andCommunications Subsystem

Users

Archive Centers

Modeling

On-line Browse

Products

ForecastsMaps

RegionalData

ManagementSystems

InternationalData

ManagementSystems

Terrestrial and AtmosphericData Management

SystemsMetadata, Data Discovery

and Data TransportStandards and Protocols

Satellites

Ships

Primary DataAssembly & QC

Hand Measurements

Moorings

Floats

The DMAC Plan – a community effort

• Brought together• 6 Federal agencies• 6 Universities• 4 Regional/State agencies• 3 Private sector orgs.

• Steering Committee,2 Outreach Teams, 4 Expert Teams

• Data Facilities Outreach• User Outreach

• Data Discovery & Metadata

• Data Transport• Data Archive & Access• Applications & Products

The DMAC Plan – a community effort• Four levels of review

1. Selected specialists2. Public workshop3. Two public reviews

• 150 reviewers• 6 Federal Agencies• 22 Universities & Institutes• 13 Private Sector• 13 Regional/State agencies• 5 International

• Presented at numerous meetings• ASLO• AGU• AMS• JCOMM, WMO• IODE

The DMAC Plan – What does it contain?

• Articulates the vision, requirements,and technological approaches.

• Recommends next steps, estimates costs and timeframes

• A roadmap, not a specification. No attempt to map tasks to specific organizations

The DMAC Plan – first order guidance

COOS in RAs advise about IOOS NOW

DMAC Steering Committee designated initial guidance

• Report standards and standards processes presently used to Ocean.US

• Use FGDC (ISO 19115) metadata standards if starting out• Use community accepted standards otherwise

• Create metadata in XML-schema with a style sheet• Gridded data – install server and provide access through

OPeNDAP• Use RDBMS

• OPeNDAP enabled• Enterprise GIS, OGC compliant

• Data providers – ensure irreplaceable data are permanently archived

• Archive Centers – structure archives to be accessible and DMAC compliant

• Security – until IOOS has data and network security guidelines use those prescribed by like federal centers

• Use NDBC services for mooring data

Many more, and will be updated, see http://www.ocean.us

Next Steps: Work on DMAC Plan Recommendations

• New DMAC-Steering Team• Formed in April 2005 • Open, fair, balance

representation• About 25 members

• Objective: Minimally functioning DMAC framework – in the near-term

Advancing the DMAC PlanWorking Structure under the Steering Team

**Expert Teams:Metadata & Discovery Metadata & Discovery Standards Process Standards Process Archive Archive Transport & Access Transport & Access

Caucuses:Private Sector International Modeling

Working Groups:IT SecurityOn-Line BrowseSystems Engineering

Status: All but two WGs have SOWs and ETs are being formed.

**Possible areas of OHH Participation

More on participation

Expert Teams:Metadata & Discovery Metadata & Discovery Standards Process Standards Process Archive Archive Transport & Access Transport & Access

How to participateHow to participate• Contact, Contact,

rosalind.e.cohen@noaa.govrosalind.e.cohen@noaa.gov• Fit into an ETFit into an ET• Review SOW, online soonReview SOW, online soon

E.G. SOW for the Archive Expert Team

Statement of need brief,• Forge cooperation between multiple centers

to insure:– Data flow integrity– Publish data for discovery and access– Stewardship and long-term curation

• DMAC metadata and data transport methods are critical – cross communication with other ETs

• Preserve data through life cycles of storage systems

SOW for Archive ET, continued

Membership:• All US agencies with IOOS data

responsibilities• International counterparts• Representative from National Archives

(NARA)– Long term strategies and preservation policies

• Members must have knowledge and have authority to advance ET goals at the organization level

• Approx. 10 members

SOW for Archive ET, continued

Short term Tasks (2005-2006):• Review and refine DMAC Plan with report to

DMAC ST• Define archive critical metadata – provide to

Metadata ET• Assess, inventory, and report on IOOS

relevant archives– Access status– Metadata status– Identify preservation weakness– Identify gaps in irreplaceable data archives

SOW for Archive ET, continued

Long term Tasks (2007-2008):• Demonstrate, test bed project, merging

metadata from multiple ACs into DMAC data discovery portal

• Use DMAC meta data standards and transport protocols for two-way data flow

– Receipt and delivery of data

• Establish comprehensive metrics – system wide

• Composite data polices from all agencies– Participate with governing committees to form a

IOOS data policy

• Many other goals outlined in DMAC plan!

SOW for Archive ET, continued

Schedule:• Form, to extent possible the Archive ET,

October 2005. Current members:» Bob Keeley (MEDS/Canada)» John Lever (Navy/CNMOC)» Landry Bernard (NOAA/NDBC)» Steven Worley (NCAR/NSF)

• Validate membership before DMAC ST, Nov. 2005

• Hold first AET meeting, early 2006• Use, web forums, email, and twice annual

meetings to address Short Term tasks

Conclusions:Conclusions:•IOOS and DMAC have begunIOOS and DMAC have begun

•Much growing to doMuch growing to do•Must keep up with rapid Must keep up with rapid developments at COOS and developments at COOS and RAsRAs

•Fostering broad participation is Fostering broad participation is the challengethe challenge•Technology is tracking the Technology is tracking the need wellneed well•Our success will be measured Our success will be measured by how well we fulfill the by how well we fulfill the societal needssocietal needs

Questions?Questions?

http://www.ocean.us