NOAA Office of Ocean ExplorationEnd-to-End Data Management:

A Success Story

NOAA Tech ConferenceNovember 2005

Susan GottfriedNational Coastal Data Development Center

The Data Management Challenge

• Standardize diverse data collection and management efforts

• Handle multiple types of data and formats

• Automatically generate required metadata

• Make data discoverable and accessible

Post-cruise Reports & Publications

Dive Videos and Digital/Still imagery and annotations

Scientific Instrumentation Raw/Processed Data Files

Geological/Biological/Water Samples

Expeditions Information System

Website archives, weblogs, situation reports, etc.

ShipTracks & Underway Data

Dive and non-dive activities & locations

Research Proposals

Database & Documents

Data files

Video

Physical samples

Integrated Product Team (IPT)

NOAA’s Office of Ocean Exploration (OE)Expedition Information System (EIS)

National Undersea Research Center (NURC)Management Information System (MIS)

National Coastal Data Development Center (NCDDC)Metadata Enterprise Resource Management Aid (MERMAid)

National Oceanographic Data Center (NODC)

National Geophysical Data Center (NGDC)

NOAA Central Library (NCL)

College of Charleston

Mitretek Systems, Inc.

Phase II Data Flow

MIS(Access Data

Project, SQL Server)

Master EIS (Filemaker

Pro 6)

Research Proposals & Cruise Plans export/

initialize

import/update

ingest

Runtime EIS

generate

reference

record

Access db

Public and Protected Data Discovery and Access

Metadata Generation and Archive

catalog

Phase III Data Flow

reference

record

CIMS Web-Instance (cruise

specific)

CIMS Web-Instance (cruise

specific)

CIMSPortal

CIMS Data Repository

Public and Protected Data Discovery and Access

Metadata Generation and Archive

Cruise Information Management System (CIMS)

• Proposal Management– Researchers submit and manage their proposals– OE Proposal managers use for decision making

• Cruise Planning– Common interface for cruise participants– Standard cruise plan document

• At-Sea Data Collection– Logical, efficient UI guides data manager – Synchs to main project data management component

automatically

• Project Data Management– Secure, controlled access to project data, post-cruise– Meets OE’s mission to eventually make data accessible to

public

CIMS Technology

•Web Based

•Cost Effective

•Readily Accessible

•Strong Programming Community and Support

•Built-in Security Features

Zope/Plone/Python

CIMS User Interface

Metadata Generation and Archive

MERMAid

Create/Validate

NOAA Central Library

NODC Archive

NGDC Archive

FGDC formatted metadata

FGDC metadata bundled with

oceanographic data sets

FGDC metadata bundled with geophysical

data sets

FGDC translated via MarcXML to

Marc21, bundled with video,

imagery, products and documents

FGDC records ingested for cruise,

project, dives, scientific activities Online Metadata

Catalog for Public Discovery

CIMS

Data Discovery and Accesshttp:\\www.ncddc.noaa.gov\ocean-exploration\Home

Collaborative Tools

Video Data Management Systemwww.lib.noaa.gov

• Developed by NOAA Central Library for OE• Developed digital video workflow requirements

– from acquisition through archiving• Provides NOAALINC (online library catalog)

access to OE data and information– Digital video clips - Maps– Cruise reports - Images– Raw data - Web Site

• Establishing original materials archive

• Participates in outreach and collaboration

Data Discovery and Access

Data Discovery via Metadata SearchData Discovery and Access

Metadata management facilitates data access– Points of Access

• National Spatial Data Infrastructure

• NCDDC Metadata Catalog

• NOAALinc

• Geospatial One-Stop (GOS)

– Data download available• Open data format

Digital Atlas GIS

• Internet mapping of OE data collection activities• Integrates OE data with other NOAA and non-NOAA data• Integrates geospatial data with other data products • Developed in accordance with NOAA E-GIS characteristics

Data Discovery and Access

Documents

Data

Collections

Contact Information

OE Deputy Director:John McDonough 301-713-9444 x113

John.McDonough@noaa.gov

Federal Program Manager (NCDDC): Sharon Mesick 228-688-2256

Sharon.Mesick@noaa.gov

Project Manager (NCDDC): Susan Gottfried 228-688-2459

Susan.Gottfried@noaa.gov

CIMS Software Engineer (NCDDC):Stacy Ladnier 228-688-1878

Stacy.Ladnier@noaa.gov