NOAA Metadata UpdateTed Habermann

NOAA EDMC Documentation Directive

This Procedural Directive establishes

1) a metadata content standard (International Standards Organization [ISO] 19115 Parts 1 and 2) and

2) a recommended representation standard (ISO 19139)

for documenting NOAA’s environmental data and information.

Roles and Responsibilities

Background Information

Building the Foundation

Develop and implement common metadata management tools

Support training specifically targeted at improving NOAA’s data documentation

Initiate teams to work on “special documentation problems” that cross Line and Staff Offices

Promote and highlight good examples

Encourage and support participation in the ISO and Open Geospatial Consortium (OGC)

Use rubrics to establish a baseline and monitor progress

NOAA Line/Staff Offices and Programs must build on

a strong foundation of support across all of NOAA

Line / Staff Office & Program Responsibilities

Data Collectors/Providers

Data Users

Data Stewards

Standards Experts

1. Identify Documentation Expertise

3. Create and Improve Metadata

2. Assess current state of documentation

4. Publish/Preserve Metadata

Metadata for DiscoveryMetadata for UseMetadata and Documentation for UnderstandingDocumentation of CollectionsDocumentation of DatasetsDocumentation of Services

NOAA Data Centers / Archives

Building the Foundation

The GOES-R Program recently expanded the scope of their metadata model from just ISO to ISO plus netCDF.

This creates an integrated metadata framework that includes metadata in granules (files) as well as the ISO standards.

This framework is important across NOAA and for many of our partners (NASA).

Initiate teams to work on “special documentation problems” that cross Line and Staff Offices

Building the Foundation

The ISO 19115 metadata standard adopted by NOAA is currently undergoing a significant revision that addresses many requirements from NOAA and the netCDF-CF community that includes many NOAA data providers and partners.

These changes will be adopted as part of the Draft International Standard which is about to be approved by ISO (Nov. 2011)

Encourage and support participation in the ISO and Open Geospatial Consortium (OGC)

Building the Foundation

ISO metadata standard trainings were held during June in Silver Spring as part of the annual EDMC Data Management Workshop and during October as part of the FGDC ISO Metadata Summit

The GEO-IDE Wiki includes over 200 pages of information about the ISO Standards.

Support training specifically targeted at improving NOAA’s data documentation

Building the Foundation

A set of tools for translating metadata collections from FGDC to ISO have been developed and tested by NODC and NGDC.

Tools for evaluating the completeness of ISO metadata that include links to the GEO-IDE wiki have been developed and tested at NGDC.

Develop and implement common metadata management tools

Use rubrics to establish a baseline and monitor progress

What Goes Where?

Spiral 1:Initial ContentSpiral 1:Initial Content

Metadata ContentIndependent of standardMetadata ContentIndependent of standard

StandardGuidance /

Implementation

StandardGuidance /

Implementation

Check BackWith UsersCheck BackWith Users

Spiral 2-N: Scientific QuestionsNew RequirementsNew Use Cases

Check BackWith Data Collectors/

Providers

Check BackWith Data Collectors/

Providers

Spiral Development / Training

IdentificationIdTitleAbstractResource DateTopic CategoryTheme KeywordMetadata ContactScience Contact

IdentificationIdTitleAbstractResource DateTopic CategoryTheme KeywordMetadata ContactScience Contact

ExtentGeospatial Bounding BoxTemporal Start/EndVertical Min/MaxPlace Keywords

ExtentGeospatial Bounding BoxTemporal Start/EndVertical Min/MaxPlace Keywords

DistributionDistributor ContactOnline ResourceDistribution FormatData Center KeywordsBrowse Graphic

DistributionDistributor ContactOnline ResourceDistribution FormatData Center KeywordsBrowse Graphic

Text SearchesPurposeExtent DescriptionLineage StatementProject Keywords

Text SearchesPurposeExtent DescriptionLineage StatementProject Keywords

Quality/LineageSourcesProcess StepsQuality Reports / Coverages

Quality/LineageSourcesProcess StepsQuality Reports / Coverages

Acquisition InformationInstrumentPlatformInstrument KeywordsPlatform Keywords

Acquisition InformationInstrumentPlatformInstrument KeywordsPlatform Keywords

Potential Spirals

Content InformationAttribute TypeAttribute NamesAttribute DefinitionsAttribute Units

Content InformationAttribute TypeAttribute NamesAttribute DefinitionsAttribute Units

DiscoveryDiscovery UnderstandingUnderstanding

ConnectionOnlineResource:NameDescriptionFunctionApplication Schema

ConnectionOnlineResource:NameDescriptionFunctionApplication Schema

Spiral None 0-33% 34-66% 67-99% All

Identification

Connection

Extent

Distribution

Description

Content

Lineage

AcquisitionInformation

Metadata Evaluation for Dataset XDate: 2010/10/24

Spirals and Rubrics

Progress

GOALS

Each record has a score that can be calculated using a consistent, standard tool

Spiral Development / Training: Rubrics

Questions?

Identify documentation expertise1. Establish Data Stewardship Teams to facilitate documentation creation and

improvement for appropriate organizational units or around programmatic needs.

2. Data Stewardship Teams should include the following expertise/skills: • Data Collectors/Providers• Data Users• Data Stewards• Standards Experts

Line/Staff Office Responsibilities

Assess current state of documentation1. Identify existing sources of documentation (Data Collectors/Providers)2. Classify existing documentation into following categories (Data Stewards and

Standards Experts): Metadata for Discovery Metadata for Use Metadata and Documentation for Understanding Documentation of Collections Documentation of Datasets Documentation of Services

3. Identify high-priority targets for improvement (All members of Data Stewardship Team)

4. Highlight best practices, successful teams and individuals (Line and Staff Office Management)

Line/Staff Office Responsibilities

Create and Improve Metadata

1. Translate/transform existing metadata into the recommended representation (ISO 19139)

2. Create metadata for undocumented data and information3. Use spiral approach for improving metadata

Line/Staff Office Responsibilities

Publish/Preserve Metadata (Data Stewards)

1. Publish metadata record in new or existing Web-Accessible Folders or using a standard catalog service. This will make it possible to connect metadata records to various discovery portals using standard services

2. Work with NOAA Archives to ensure that documentation and metadata will be preserved for the long-term

Line/Staff Office Responsibilities

Cross-NOAA Responsibilities1. Encourage and support participation in the ISO and Open Geospatial Consortium

(OGC) standards development and evolution processes 2. Develop and implement common metadata management tools including

mechanisms for evaluating the completeness and quality of data documentation• Utilize rubrics to establish the baseline and monitor progress.• Engage users in providing feedback on data documentation efforts and

opportunities.3. Promote and highlight good examples of documentation and the individuals

involved in their creation. 4. Support training specifically targeted at improving NOAA’s data documentation.5. Initiate teams to work on “special documentation problems” that cross Line and

Staff Offices

EDMC Responsibilities1. Review this Procedural Directive twice a year to evaluate effectiveness and

monitor progress.2. Work with the CIO Council and the NOAA Observing Systems Council to implement

and monitor progress on the Cross-NOAA responsibilities listed above.3. Encourage and support partnerships with external organizations in the process of

migration of metadata from FGDC to ISO Standards.

Measure Progress

1. An increase in the amount of NOAA environmental data and information that is well documented and discoverable via national and international discovery portals.

2. Improvement in the quality of Line and Staff Office and Program data documentation processes.