(NSG) GEOSPATIAL METADATA DESK SIDE · Web viewGEOSPATIAL METADATA DESK SIDE REFERENCE (NGMDSR) VERSION 1.0 August 23, 2007 This page intentionally left blank Table of Contents Foreword

NSG Geospatial Metadata Desk Side Reference

August 23, 2007

NATIONAL SYSTEM FOR GEOSPATIAL INTELLIGENCE (NSG)

GEOSPATIAL METADATADESK SIDE REFERENCE

(NGMDSR)

VERSION 1.0

August 23, 2007


1


August 23, 2007

This page intentionally left blank


2


August 23, 2007

Table of Contents

Foreword Page 5

Preface Page 6

Executive Summary Page 7

Part 1 – Introduction and Purpose Page 9

1. Introduction – The Importance and Use of Geospatial Metadata Page 101.1 Purpose – The Metadata Guidance Document Page 111.2 Background – The Criticality of Geospatial Metadata within the NSG Page 121.3 Intelligence Community Data Management Committee (DMC) Page 141.4 The Role of NGA/NCGIS and Standardization within the NSG Page 161.5 Standards Enforcement Page 181.6 Registries Page 20

Part 2 – GWG Metadata Focus Group Page 21

2. GWG Metadata Focus Group (MFG) Page 222.1 Levels of Metadata Page 232.2 NSG Geospatial Metadata Profile Page 282.3 Geospatial Metadata Drivers Page 312.4 On-going Geospatial Metadata Activities Page 33

Part 3 – Geospatial Metadata Issues and Exploitation Page 41

3. Metadata Page 423.1 General Discussion of Metadata Page 423.2. Geospatial Metadata Management Page 423.3 Geospatial Metadata Management and Architecture Page 433.4 Other Metadata Categories Page 443.5 Testing of Metadata Requirements Page 463.6 Geospatial Profiles and Access Tools Page 473.7 Metadata Management and Responsibilities Page 483.8 Classes of General Metadata Page 493.9 Types of Specific Metadata Page 503.10 Profiles of Standards Page 51 3.11 Specific WEB Services Page 523.12 User Requirements Page 523.13 Encoding Page 533.14 XML Schema reuse capabilities Page 54

3.15 Crosswalk Management and Entity Mapping Page 54

NSG Geospatial Metadata Desk Side Reference3


August 23, 2007

APPENDICES:

Appendix A – Terms of Reference Page 61Appendix B – Related Standards Information Page 63Appendix C – Legislation and Guidance on Standards Page 65

Appendix D – Emerging Standards Page 71 Appendix E – ISO 19115 Normative and Informative References Page 73 Appendix F – DoD IT Standards Registry (DISR) Definitions Page 79

Appendix G – List of Acronyms Page 81

Review Information and Comment Forms Page 83


4


August 23, 2007

Foreword

This document has been developed to be used as a general reference for Geospatial Metadata users and developers to consult for guidance and helpful information pertaining to questions and issues that have been raised frequently in various fora within the geospatial community It is hoped that this document will, after continuous update, serve as a single reference to satisfy the needs of the community by addressing those many common questions and issues. The document is written in an informative manner, allowing for some interpretation by the reader, and yet points the reader to various normative agencies’ mandated requirements (DOD Metadata Working Group, Intelligence Community Data Management Committee, etc.) and driving standards (ISO 19115, OGC specifications, etc.).

There are numerous ongoing metadata activities and issues that overlap and affect interoperability of GEOINT across the National System for Geospatial Intelligence (NSG) (explained later in this document). Coordination of all these metadata activities is critical and is being facilitated by a GEOINT Standards Working Group (GWG) Metadata Focus Working Group (MFG) which was formed in 2005 and helps leverage NSG resources to coordinate these profiling and harmonization efforts.

This Geospatial Metadata Desk Side Reference allows the Metadata Focus Group of the GWG to provide a handy companion to all NSG Geospatial Metadata users and developers in a timely and sufficient manner.

There are several sections of this document that remain incomplete. There are also other sections being considered for future incorporation. Many of these sections require input from those members of the Geospatial Community of Interest to share their profiling and general experience with the rest of the community.

It is planned that this Desk Side Reference be updated periodically, perhaps semi-annually, to let the community as a whole share their experiences with the remainder of the Geospatial community. This reference will be as good and helpful to the community as what the community invests in its development and applicability.

Please send your questions and comments to:

Norman C. AndersenNational Geospatial Standards AgencyNational Center for Geospatial Intelligence StandardsMetadata Standards officeTelephone 703-814-4565

[email protected]


5

mailto:[email protected]


August 23, 2007

PREFACE

Metadata and Its Importance

Metadata is first data about data that describes the content and the appropriate detail about a data resource. More than that, metadata describes the richness of information behind data collected from any source—when the data was collected, who owns the information, who can access the information, and when the data becomes no longer of value and should be retired or archived. Of prime importance is that by providing such contextual and background material, metadata enables automated discovery of information resources. As such, metadata is a key enabler of data and system interoperability. The usage of metadata is being expanded to include finding resources such as geospatial systems and data assets, web services, standards and other specific tagged information. In the current movement to an increasingly net-centric world, the speed, agility and interoperability required cannot be met without appropriate metadata.

Metadata is becoming even more critical as the volumes of archived data, imagery and other geospatial assets resources continue to grow at rapid rates. One example is the airborne community’s imagery libraries with their rapidly increasing volume of data. Human search capabilities could not possibly access and evaluate such copious amounts of data in any range of normal operational time constraints.

Without metadata many searches would have to be conducted with Google-like tools with much effort spent trying to determine if the data or other resource meets the specific need. Without metadata, a consumer could not efficiently and effectively discover and retrieve the information that contains the characteristics and content necessary to support a required mission. Metadata instills data accountability and limits data liability. This is especially true for the National System for Geospatial-Intelligence (NSG) in the use of geospatial intelligence (GEOINT) data. Providing metadata according to applicable standards, including harmonization between and among required metadata standards, further increases the utility and value of data and discovery services.

It is critical that the work and development of metadata and metadata standards for NGA and GEOINT data and other informational resources be funded in the current move to an increasingly net-centric environment to provide the critical high level of customer support required of NGA.


6


August 23, 2007

EXECUTIVE SUMMARY

The National System of Geospatial Intelligence (NSG) Geospatial Metadata Deskside Reference (NGMDSR) addresses the need as identified in the Preface for clear and concise guidance in the use of geospatial standards as metadata within the geospatial intelligence community. The document covers a wide array of topics from metadata drivers through metadata management to current issues in the field.

The three-part guide is intended as an answer to many of the frequently asked questions from users of geospatial standards compiled over the years, as well as an informative text to consolidate in one place the references, informal practices, and guiding principles of using metadata within the geospatial community. The first Chapter discusses the framework within which geospatial standards function and are derived. Chapter Two introduces the Metadata Focus Group which is the source of the document and highlights the many activities of the Focus Group and its sub-groups. The third part, by far the longest, contains many theoretical and practical aspects of using and managing geospatial metadata within the community. The many appendices include useful information on additional related standards, emerging standards, and the DISR, the Defense registry for information technology standards.

The document is intended to evolve as the geospatial and the standards communities evolve and as net-centric development and guidance continues. Please send feedback to the National Center for Geospatial Intelligence Standards (NCGIS) at [email protected] use the feedback form after Appendix G.


7



August 23, 2007



8


August 23, 2007

Part 1 – Introduction and Purpose


9


August 23, 2007

Part 1 – Introduction and Purpose

1.0 Introduction – The Importance and Use of Geospatial Metadata

The importance of metadata is that it enables discovery of information and is a key enabler of data and system interoperability. Metadata is data about data that describes the content and the appropriate use of a data resource. It is particularly important to have metadata for data resources that have no obvious meaning outside of the context within which they were collected. For example, an unlabeled image is useless without knowledge of when, where, and how it was taken. Metadata are critical to the search, use, re-use, and quality assessment of data. Incorrect or non-existent metadata constrain the sharing and exploitation of GEOINT, resulting in negative impacts to the NSG customer community (explained later).

Metadata is a key component of the geospatial data set. It carries critical information as to the dataset purpose, location, content, and lineage. Geospatial software and analysts increasingly rely on metadata to ingest, display, and manage data. Perhaps most significantly, metadata is the consumer information needed by a rapidly growing geospatial data market to locate available geospatial data resources and assess their fitness for a particular use. Metadata instills data accountability and limits data liability. The timely capture of metadata is fundamental to the quality of the data set as a whole.

In order for a data resource to be discovered, assessed for its fitness or use, retrieved, and then exchanged in the current Joint and Coalition operating environment, it must comply with existing International Organization for Standards (ISO) documents such as ISO 19115 (Geographic Information - Metadata) that mandates the structure and content of metadata for geospatial information. But simple compliance with standards is not the only issue concerning metadata. The implementation of the standards that allow a good deal of flexibility and customization must be harmonized across an enterprise to ensure that all data within that enterprise can be easily discovered and accurately exchanged. Improperly implemented metadata is worse than no metadata at all because it presents a false sense of security while prohibiting or significantly limiting a consumer’s proper use and exploitation of resources, or even the awareness that resources exist. Continuing support of ISO metadata extension efforts is necessary to ensure that NSG requirements are addressed in emerging international standards.

One of the things that ISO 19115 brings to metadata organization is that it documents a number of things in the metadata instance that have previously been documented in a product specification. For example, a given specification could have a requirement that all geographic coordinates be reported using World Geodetic System 1984 (WGS84). Since the coordinate reference system in this example is fixed (never changing) and specified by the product specification, it is not necessary to report it in the metadata. For this reason, the legacy metadata reports a geographic extent without the Coordinate Reference System (CRS) (since it is set by the product specification to WGS84).

Metadata is becoming even more critical as the volumes of archived data and imagery, such as in the airborne community’s imagery libraries, continue to grow to enormous volumes – such that humans could not possibly access and evaluate all this volume of data. Without good metadata,


10


August 23, 2007

a consumer could not search and retrieve the data that contains the characteristics and content necessary to support a required mission. Metadata instills data accountability and limits data liability. The timely capture of metadata is also fundamental to the quality of the data resource as a whole.

Examples of metadata critical to identify and describe GEOINT data include the title and date/time of the resource, a narrative description of the resource, the language used within the resource, lineage information about the events or source data used in constructing the resource, its security classification and handling procedures, the entity primarily responsible for making the content of the resource, the quality of the content of the resource, the geographic area that the resource covers, the method used to spatially represent the geographic information within the resource, among others.

Metadata can be the description of the data structures used to contain the data (structural metadata) or descriptive information about the data (discovery metadata). “Structural metadata” defines how information is organized. For maintenance of meaning and administration over time, metadata is usually stored in a metadata registry. A metadata registry contains descriptions of the organization or description of data or data assets, but not the data itself. Formal registration of metadata provides for different versions to be maintained over time, a feature that permits reference in the future without loss of meaningful context and details.

“Discovery metadata” describes the aspects of data that define the catalogue data for searching. A number of general purpose standards exist for cataloguing data. ISO 19115 is a standard for cataloging geospatial data. Geospatial metadata is required to support the cataloguing of more specialized information, specifically, information that is associated with a terrestrial location. Cataloguing geospatial data for efficient search and discovery requires geospatial metadata that can capture the complex, descriptive information about the data (for example, coordinate reference system used, positional accuracy, and topological consistency) to be organized in a common manner, using a common vocabulary.

Geographic Information Systems (GIS) form a distinct class of information systems through their unique requirements for collecting, converting, storing, retrieving, processing, analyzing, creating, and displaying geographic data. The generic nature of GIS – organizing information by location – is interdisciplinary and not specific to any application. Adequate and sufficient metadata is critical for the robust exploitation of a GIS.

1.1 Purpose – The Metadata Guidance Document

This document has been developed to be used as a general reference for Geospatial Metadata users and developers to consult for guidance and helpful information pertaining to questions and issues that have been raised frequently in various fora within the geospatial community. Furthermore, it is intended for those individuals or organizations who desire to develop their metadata activities within the harmonization and interoperability requirements for Geospatial Metadata development. Due to the numerous questions concerning metadata from various working groups, this document is intended to address, at a high level, some of these questions.


11


August 23, 2007

1.2 Background – The Criticality of Geospatial Metadata within the NSG

The various entities that comprise the NSG are depicted in Figures 1-1, 1-2 and 1-3. The NSG is defined as the integration of technology, policies, capabilities, and doctrine necessary to conduct geospatial intelligence in a multi-intelligence environment. It includes the DoD and non-DoD components of the Intelligence Community (IC), including, where appropriate, coalition and Federal civil agency partners.

Of particular importance to the National System for Geospatial-Intelligence (NSG) is the use of geospatial intelligence (GEOINT) data. In the same manner as previously described, an accurate and descriptive set of geospatial metadata is a key component of the geospatial dataset that is needed by a rapidly growing geospatial data market within the NSG to locate available geospatial data resources and assess their fitness for a particular use, allowing the appropriate use and exploitation of that geospatial data.

As data moves into the net-centric environment with data being served and packaged on the fly according to a user’s requirements (and possibly from heterogeneous sources) the idea of metadata being hidden in a product specification becomes very unattractive. This is because the flexibility provided in the net-centric environment may mean that data, for example, is reported using multiple CRSs that must be known to the application so that it can make the appropriate translations into a CRS for display of the information.


12


August 23, 2007

Figure 1-1. NSG Configuration

Figure 1-1. The NSG

Geospatial Metadata profiles for differing NSG environments (refer to Figure 1-1., above) provide standardized extensions that apply to special user communities, while harmonization activities help to maintain a high degree of interoperability across these communities (Extensions are unique to the organization or activity). Since metadata activities and issues overlap and affect interoperability of GEOINT across the NSG, coordination of all metadata activities is critical. Under the GWG, a Geospatial Metadata Focus Group (MFG) was developed to help leverage NSG resources to coordinate these profiling and harmonization efforts. The challenge to the MFG will be to profile and extend existing Geospatial Metadata activities to define a metadata implementation for GEOINT from disparate data stores, to include legacy product holdings.

To be successful, the NSG geospatial metadata standardization work must begin with the profiling of all intelligence related metadata standards (see 1.3) with appropriate extensions to account for geospatial intelligence. In addition to sensor and product specific profiles, there will also be several profiles that define metadata for cataloguing, search and retrieval, and exploitation. These disparate profiles must then be connected (i.e., harmonized, or mapped) to enhance interoperability by providing a common base for describing, translating, validating and accessing metadata about geographic, imagery and gridded data between Communities of Interest (COI). The profiles and their mappings must be made available to the COI through a net-centric, enterprise-wide, metadata support infrastructure required to support the capture, storage and exchange of metadata thorough out the entire enterprise.


13

2

National System for Geospatial Intelligence (NSG)

NSG Doctrine, Policies,Standards, and Guidance


August 23, 2007

Figure 1-2. Intelligence Community (IC) Organizational Structure

1.3 Intelligence Community Data Management Committee (IC DMC)

The Intelligence Community (IC) (Figure 1-2) is comprised of key entities of the NSG. The IC, notably the IC Data Management Committee (DMC), has purview over several intelligence related standards which, as stated earlier, need to be profiled with appropriate extensions by NSG community organizations to account for their own geospatial intelligence mission and needs.

1.3.1 Introduction – A Metadata Driver

The IC DMC is the body that has replaced the Intelligence Community Metadata Working Group (IC MWG), and has taken over the metadata standards that describe the production of finished intelligence (FINTEL). These are the intelligence products that are created by intelligence analysts and disseminated to the consumers of intelligence. It is imperative that the GEOINT community ensure that the appropriate metadata is in the raw intelligence data so that the analysts can put it in the FINTEL.

There are three standards this body is in charge of:

IC Metadata Standard for Publications (IC MSP)

IC Metadata Standard for Information Security Markings (IC ISM)

IC Standard for Core Metadata (IC Core)



August 23, 2007

1.3.2 IC Metadata Standard for Publications (IC MSP)

The IC Metadata Standard for Publications (IC MSP) is a set of Document Type Definitions (DTDs) and corresponding eXtensible Markup Language (XML) schemas for generic publication types, with accompanying documentation in the form of a data element dictionary and an implementation guide. The standard has been developed as a cooperative effort in response to requests by numerous organizations within the IC to have an IC-wide mandated XML model to support interoperability of intelligence content across producers and consumers of FINTEL within the Community.

The six generic models are Article, Report, Analytical Packet, Correspondence, Briefing, and Basic Object. These generic models differ in complexity and applicability. The models are sufficiently general in nature that, when combined with appropriate formatting information, they can be used to create most common IC product styles.

Each of the publication models makes use of a set of common content models in the form of XML entities, notations, attributes and elements. The common content models are what make IC MSP conducive to interoperability and information reuse. Documents produced in XML using IC MSP must contain the appropriate metadata in order to be compliant with the standard.

1.3.3 IC Metadata Standard for Information Security Markings (IC ISM)

The Information Security Marking Standard defines the XML attributes that are used to describe classification level, SCI controls, dissemination controls, the presence of foreign government information, et al. The standard also provides XML declarations for three parameter entities that can be used to associate the attributes with any desired XML element. The Data Element Dictionary (DED) publication defines the attributes; the Implementation Guide describes the parameter entities.

The DED contains only unclassified definitions of the security-marking data elements. The Implementation Guide provides usage instructions and methods for incorporating the security-marking data elements into Extensible Markup Language (XML) document type definitions (DTDs) and World Wide Web Consortium XML Schemas

1.3.4 IC Standard for Core Metadata (IC SCM)

The IC Standard for Core Metadata (SCM) defines logical core metadata elements for resources posted to Community, collateral, and organizational shared spaces. Core metadata is the information about intelligence resources that is common across intelligence disciplines, domains and data formats.

The IC Discovery Metadata Core (DMC) has included only rudimentary geographic reference elements and attributes in the IC SCM and is relying on the NSG to develop “a change request to replace ‘Geographic Reference’ with a set of elements drawn from International Standard ISO 19139.”


15


August 23, 2007

Continuing support of all ongoing Geospatial Metadata extension efforts are necessary to ensure that NSG requirements are addressed in emerging national security systems (NSS), IC, and international standards. Because all of the metadata activities and issues overlap and affect interoperability of GEOINT across the NSG, coordination of all metadata activities is critical.

JFCOMRole as Joint Force

Integrator; work with Services to identify areas for

collaboration on joint GEOINT issues

NGAProvide guidance on

NSG, explain NGA transformation efforts, identify requirements, resource requirements

resulting from NGA, establishes standards,

etc

ServicesIdentify requirements, ensure

Service equities represented in development of architecture,

CONOPS, etc., align Service PORs with joint goals

STRATCOM Support operational warfighter; work to integrate National

and Theater tasking processes, advocate

joint goals.

DISAAssess impact upon organic

warfighter comms; how best to carry capability forward

net centric architecture

Joint StaffResponsible for requirements

certification, IT interoperability, future

concepts/DOTMLPF/policy changes, etc

IC AgenciesPower user of GEOINT and potential

consumer of value-added data;

COCOMsIdentify GEOINT battlefield requirements.

Identify joint interoperable framework.

Collaborative solution to

provide GEOINT to/from tactical

level

Key Players

Figure 1-3. NSG Participants

As future systems and technologies are advanced, the evolution of GEOINT standards must keep pace ensuring optimum interoperability. The growing need for GEOINT collaboration across government and national communities requires evolving from the use of government and proprietary standards toward the use of voluntary consensus standards that support standards-based commercial off-the-shelf (SCOTS) solutions. At the same time, the sophistication of GEOINT customers and technologies is changing the emphasis from building static products and their accompanying specifications to operating within data- and net-centric environments.

1.4 The Role of NGA/NCGIS and Standardization within the NSG

Metadata issues cut across all standardization areas, resulting in a large number of metadata related activities and a heavy reliance on the NGA National Center for Geospatial Intelligence


16


August 23, 2007

Standards (NCGIS) for guidance and expertise to help identify relevant standards (especially those that refer to schemas, coding/mapping, etc.) and develop the necessary metadata data tables and schemas. While no specific metadata projects are considered unfunded, resource issues will affect the level of participation by the NCGIS in NSG community profiling and harmonization activities.

As the functional manager for GEOINT, the National Geospatial Intelligence Agency (NGA) has designated the NCGIS to set, implement, and actively advocate for GEOINT standards and standards management processes and policies that promote interoperability and operational efficiency across the NSG community. The NCGIS ensures a standards-based approach in developing enterprise-wide system architectures that are essential to interoperability:

among the traditional Military Service and Command consumers of GEOINT

for numerous operational plans that include international coalition partners and engage our domestic counterparts

across the GEOINT enterprise that includes imagery, imagery intelligence and geospatial information produced and provided by NGA, as well as in collaboration with other nations, co-production partners and the private sector

throughout the myriad components of tools, equipment, training and people that constitute the NSG

Several metadata standardization efforts are underway both within the NSG and the broader international geospatial community, for which NGA is a key leader and stakeholder. Within this context, NGA leads and participates in efforts to develop and harmonize geospatial metadata standards within the Intelligence Community (IC), NATO, the international community within the ISO, as well as within industry and certain federal agencies. NGA’s primary focus in these activities is to ensure that NSG requirements are addressed in emerging international metadata standards.

NGA also works to develop profiles of existing standards. Profiles of metadata standards, such as ISO 19115, and the XML implementing standard ISO 19139, customize the standards to add more detailed technical content and enhance the discovery and exchange of data for certain organizations and uses. NGA’s metadata harmonization efforts design a consistent implementation and use of these profiles across the NSG. Without such efforts, the implementations of these standards and profiles would significantly inhibit and impede interoperability for data and systems as defined by the data interoperability strategies published by ASD/NII and the requirements of CJCSI 6212.01D and other applicable DoD directives and instructions.

A recent and significant activity is NGA’s sponsoring an effort to develop a new ISO standard defining metadata for sensors, which will be ISO 19130 (Geographic Information-Sensor data model for imagery and gridded data). This effort is of great importance to the airborne


17


August 23, 2007

community, and will facilitate improved data interoperability, and shorter timelines for the data from the sensor platforms to exploitation stations.

NGA is also aggressively pursuing the development, adoption in organized standards bodies, and harmonized implementation of metadata standards across the NSG. The main goal for all this effort is to support the interoperable and discovery needs of the NSG in today’s and tomorrows systems.

The NCGIS will be working with the NGA/Acquisitions and NGA/Production Directorates to define a metadata model for geospatial information, imagery, and imagery intelligence data. The existing ISO 19115 standard is heavily geospatial-based and lacks imagery fields; therefore, the emphasis has been geospatial. This problem is being addressed within ISO/Technical Committee (TC) 211 as ISO 19115-2, Geographic Information – Metadata – Part 2: Extensions for imagery and gridded data.

1.5 Standards Enforcement

Since January 2005, the GWG has provided community leadership and management of GEOINT standardization activities for the NSG.

The GWG is responsible for all activities assigned by the Information Technology Steering Committee (ITSC), including making recommendations for GEOINT standards to be placed into the DoD IT Standards Registry (DISR) operated by the Defense Information Systems Agency (DISA) and for coordinated advice and policy recommendations on GEOINT standards issues related to the DISR. See Annex F for information about the DISR. In its coordinating and advisory role, GWG activities extend to all aspects of GEOINT standardization, including issues related to GEOINT standards identification, adoption, promulgation, implementation, compliance, and education.

The GWG focuses on GEOINT standards that enable interoperability in net-and data-centric environments and standards that support enabling technologies, data architectures, and software tools. The GWG is responsible for the GEOINT standards in three DISR Service areas: Geospatial, Motion Imagery, and Still Imagery. GEOINT is the exploitation and analysis of imagery and geospatial information to describe, assess, and visually depict physical features and geographically referenced activities on the earth. GEOINT consists of imagery, imagery intelligence and geospatial information. GEOINT standards are those that enable the discovery, access, use, integration, dissemination, exchange, and exploitation of GEOINT and include, but are not limited to:

Still and motion imagery and gridded data content, format, and compression

Application Schemas for geospatial feature encoding, data dictionaries, and catalogues

Geospatial portrayal

Metadata for geospatial intelligence

Geospatial intelligence reporting


18


August 23, 2007

Information transfer, exchange, and architecture of GEOINT services

Sensor models for GEOINT services, production, and applications

The GWG supports the ITSC in the configuration management of GEOINT standards within the DISR. Additionally, the GWG provides a standards-focused forum that the NSG community can use as a means to exchange and communicate issues regarding GEOINT standards requirements, development, implementation, and conformance. The GWG recommends GEOINT standards for data, systems, and their interfaces to ensure interoperability with DoD and non-DoD systems.

Because they are designed to support interoperable systems, the testing for interoperability and systems adherence to approved GEOINT community standards has been assigned to the DISA Joint Interoperability Test Command (JITC). Figure 1-4 depicts the various directives and instructions that drive the mission of the JITC. NGA is the lead organization for enforcement of U.S. agencies in the use and compliance with the approved standards.

3

DoDD 4630.5DoDD 4630.5IT and NSS interoperability IT and NSS interoperability

shall be verified early, shall be verified early, and with sufficient frequency and with sufficient frequency throughout a system's life throughout a system's life ……

Joint InteroperabilityDirectives & Instructions

DoDI 4630.8All IT and NSS … must be tested for interoperability

before fielding ... and certified by DISA (JITC).

CJCSI 6212.01DCJCSI 6212.01DAll IT and NSS must be All IT and NSS must be

evaluated and certified for evaluated and certified for Joint interoperability by Joint interoperability by DISADISA

(JITC).(JITC).

DoD 5000 seriesDoD 5000 seriesFor IT systems, including NSS, For IT systems, including NSS,

... ... JITCJITC shall provide system shall provide system interoperability test certification interoperability test certification memoranda ... throughout the memoranda ... throughout the

system lifesystem life--cycle andcycle andregardless of ACAT.regardless of ACAT.

DoDD 5105.19DoDD 5105.19DISADISA shall ensureshall ensure

endend--toto--endendinteroperability.interoperability.

CJCSI 3170.01ECJCSI 3170.01EEstablishes JCIDS Establishes JCIDS

w/ NRw/ NR--KPP for KPP for CDD and CPD.CDD and CPD.

DoD 4630/5000Interoperability is

“the ability to provide and accept data, information, materiel, and services ...

...includes both the technical exchange of information and the end-to-end

operational effectiveness ofthat exchange, as required for

mission accomplishment.”

Figure 1-4. DISA/JITC Directives and support capabilities

Under the GWG, NGA has formed the Metadata Focus Group (MFG) which serves as a NSG community wide forum to leverage NSG resources to coordinate these metadata standardization efforts. The MFG membership includes many distinct metadata working groups, agencies and organizations. The membership consists of over 120 (and growing) individuals at this time, who represent the US DoD, IC, Federal agencies, coalition partners, and the private geospatial industry including the Open Geospatial Consortium (OGC) member companies.


19


August 23, 2007

1.6 Registries

Register or Registry has often been interchanged with the terms catalogue and repository. While the use and definition of these terms still appears to be evolving, the term Registry is most often used to describe the place where a consistent stable store of information about services, assets, or resources is located, along with the formal process of registration to administer the metadata over time. The Registration capability may be an automated feature of the metadata registry, or registration may be accomplished via process. In any case, a registry provides a means by which metadata may be captured and maintained over time. Should the asset’s metadata change at some point (for example, if there are changes in the applicable standard), the old format, its effective dates and all other metadata are still maintained in case the information is needed in future operations. In the geospatial community a registry is generally where services level metadata is placed to facilitate the discovery and use of specific geo-processing services. See sections 2.4.8 and 2.4.8.2 in this document for more detailed information about registries and their relevance within the NSG.


20


August 23, 2007

Part 2 – GWG Geospatial Metadata Focus Group


21


August 23, 2007

Part 2 – GWG Geospatial Metadata Focus Group

2.0 GEOINT Standards Working Group (GWG) Metadata Focus Group (MFG)

The Geospatial Metadata Focus Group (MFG), mentioned earlier in Part 1, was established in 2005 and its purpose is to serve as a community-based forum to advocate for information technology (IT) standardization activities related to the standardization of metadata for GEOINT data. The MFG serves as a technical advisory group to the GWG and as a coordinating body for the GEOINT community to address all aspects of the metadata of GEOINT.

The mission of the MFG is to serve as geospatial intelligence (GEOINT) community forum to identify requirements for metadata of GEOINT data, identify and resolve standardization and interoperability issues relating to the development of metadata for geospatial intelligence information and data, and as a conduit for information and coordination relating to GEOINT metadata activities within the community.

The MFG is designed to assist all groups with compliancy and harmonization of their respective activities. The group also serves as the GEOINT community forum for metadata-related standardization and interoperability issues, and exchange of information. This includes the US DoD, Intelligence Community, Federal agencies, coalition partners, and the GIS industry.

The group currently participates in the development, review, and maintenance of multi-national, international, and national geospatial metadata standards. These include: NATO Standardization Agreements (STANAG), ISO Standards, National and Federal Geospatial Metadata Standards. These standards are the drivers that will assist in providing interoperability and provides for implementation activities. “Drivers” as referred to in this document, are the force behind numerous metadata collection, management, archival, and verification tools and instructions, i.e. metadata standards that NGA and other DoD/IC organizations have been either mandated to use or have chosen to use. The results of the work mentioned above provide input directly into the mentioned standards activities.

The GWG MFG deals with those aspects of the standardization of GEOINT relating to Imagery and geospatial metadata. Metadata is used by a large population of people other than the producer of the metadata. Usually, it is created by someone and used by someone else. Standardizing the use of metadata will provide creators with appropriate information to characterize the data. Furthermore, standardizing the use of metadata will enable users to effectively implement the metadata to facilitate the most efficient method to discover and access the associated data.

The MFG coordinates activity with the various recognized organizations and will assume a leading role in the development of standardizing metadata in their respective communities.


22


August 23, 2007

2.1 Levels of Metadata

The focus of MFG activities are to support the various metadata profiling and harmonization efforts across the NSG community and to influence current ISO standards development work to support NSG metadata requirements. To achieve this, metadata is developed at several different levels, and the MFG has developed recommended core metadata sets analogous to these levels for use in the NSG community.

Figure 2-1. Major Levels and Flow of Metadata for NSG Geospatial Data Types

In Figure 2-1 above, the “Green Boxes” represent those mandatory elements and entities needed to be compliant with the various levels of existing and driving metadata requirements. The “Tan Boxes” represent those mandatory and optional elements and entities which are either suggested or provided to enhance a specific operational requirement. The “Blue Boxes” represent those mandatory and optional elements and entities which may be required in some security and data quality contexts. (Note: Modeling and Simulation is under consideration.)


23


August 23, 2007

2.1.1 All Datasets Level – The highest level of metadata is to be used by all DOD Agencies, the Intelligence Community, and other Organizations, whether said activity describes geospatial data, sensor data, or raster data within their realm. This Metadata is the initial driver for interoperability across all activities and establishes at this level a minimum set of mandatory and recommended core metadata elements and entities. All datasets are driven by the work of the DoD Metadata Working Group (DoD Metadata Specification (DDMS)) and the Intelligence Community (IC) metadata standards. 2.1.2 All Geospatial Datasets Level – This level of metadata is the minimum core for Geospatial Metadata datasets regardless of the specific geospatial subject (i.e., vector, imagery and sensor). It is designed to assist in discovery and retrieval. The “All Geospatial Datasets” metadata level includes the DoD and IC metadata standards and is supplemented with the ISO 19115 Geospatial Metadata Standard. The recommended core metadata set analogous to this level is referred to as the “NSG Common Core”, “NSG Recommended Core”, or sometimes as the “NSG Minimum Core”.

2.1.3 Specific Geospatial Datasets Level – That minimum core for Geospatial Metadata datasets that address the specific Geospatial requirements for vector, raster (imagery) or sensor geospatial data, and expanded using ISO 19115.

Figure 2-1a. Vector, Raster, and Sensor Geospatial Metadata Level Examples



August 23, 2007

2.1.4 Organizational Extensions – Depicted as “Yellow Boxes” in Figure 2-1a, the deepest level of metadata is the level of metadata unique only to a specific product, agency or activity and not used by any other activity. It is incumbent upon the specific agencies and activities to produce their own extensions since, obviously, only that activity is familiar with their own requirements and can best determine that set of metadata for their own mission and use.

Figure 2-1b. Breakdown Flow of the Sensor Geospatial Metadata Level

The sensor metadata comes from many varied sensor types and formats. The primary example of sensor metadata requirements is that which is required for sensor modeling to mathematically explain the variables required for the Rigorous Sensor Model (RSM). Each individual sensor requires its own RSM. For the purpose of identifying the elements needed to access and discover information derived from a particular sensor, conditional and optional metadata elements are recommended to ascertain specific identifying information.


25


August 23, 2007

In the “Sensor Schematic” in Figure 2-1b above, the “M” box (green) is the mandatory core set of elements needed to access and discover certain information derived from a specific sensor type. The next “O” (meaning “Optional” elements) box colored tan is the “recommended core” set of metadata elements needed to identify a specific sensor and determine the usefulness of the information. These elements describe the specific sensor parameters and the resultant data sets to be used for various specific purposes.

The next set of boxes in the chart (yellow) break down the sensor specific types into the “active” and “passive” sensors. There are additional sensor types that may need to be added in future iterations of this document. The ones that have been described to date include those sensors that are non-scanning/non-imaging and those that are imaging sensors. Motion imagery sensors, hand-held sensors, acoustical sensors, and hydrographic sensors may need additional explanations and their own descriptions within the “operational extensions” described in this profile. The examples provided below the yellow boxes are not meant to be all-inclusive. They are merely examples of the operational capabilities of a sampling of sensors and are meant to assist in determining metadata elements necessary to describe certain exploitation capabilities.

Figure 2-1c. Breakdown Flow of the Raster (Imagery) Geospatial Metadata Level


26


August 23, 2007

The raster core set of metadata elements in Figure 2-1c are used to identify not only imagery data but non-imagery matrix sets of data as well. Examples of matrix data are the terrain elevation models and elevation data sets being used extensively by many members of the NSG. To define the mandatory core (top green box in the chart), certain access and discovery capabilities had to be derived and limitations placed upon the total raster suite of information. Additional elements will be defined within the quality, life-cycle, and textual data.

For the geospatial raster core of elements, the division of information was best described using imagery and the products derived from that imagery or associated by-products of the imagery-derived data. The second level recommended core of metadata elements (tan box marked “O”) helps in establishing the set of information needed to define the specific images and the exploited datasets as a result of using that imagery as a source. The organizational extensions (level three, the yellow boxes) break the imagery further into the primary and secondary image capabilities as well as the products and associated library activities.

These operational extensions (third level) are not meant to be all inclusive and/or all encompassing of either the imagery or non-imagery portions of the raster data set elements. They are examples of the imagery sets used as source and the resultant products and services provided as a result of the exploitation of the imagery.


27


August 23, 2007

Figure 2-1d. Breakdown Flow of the Vector Geospatial Metadata Level

The vector core set of metadata elements in Figure 2-1d is primarily derived from the use of the vector information rather from the exploitation of the vector source. Vector data, by its nature, is derived information from a set of known elements as identified within the imagery (raster) source data. It is, however, a valuable asset to the NSG member organizations and is used by the war-fighter to identify various valuable pieces of intelligence information. The core set of mandatory vector elements (in-work) will be derived from the access and discovery requirements used in the data warehouse, distribution activity, and product derivation processes (top-level in the chart – green).

The recommended core set (tan box – in-work) of elements further identify the vector information based upon the feature elements themselves and the products produced as a result of identifying those features and compiling them into a recognizable and usable view. The recommended “core of elements” is used to identify the vector data itself based upon the data dictionary and derived feature descriptions and upon the storage and retrieval process requirements. As additional “operational extensions” for the vector core metadata elements are identified, the attributes and storage information requirements will be refined.

2.2 National System for Geospatial Intelligence Geospatial Metadata Profile

The Metadata Focus Group has developed a National System for Geospatial Intelligence Geospatial Metadata Profile document (for data discovery and retrieval) which primarily provides the detailed metadata in the form of data dictionaries and associated XML schemas for the concept metadata levels outlined above. Those data dictionaries provide the detailed information behind each metadata component area for each of the data type metadata sets and their corresponding detailed XML schemas. That detailed metadata represents the metadata elements recommended for use by NGA, but are general enough so that they could apply to other sets of data as well. The NSG is strongly encouraged to consult that document.

The NSG profile document establishes and defines a systematic approach to managing, organizing, and disseminating standards to the development population. It also establishes and maintains agreement between the customer and the project team on changing standards. The profile document was developed within the Recommended Core Sub-group. However, see section 2.4.2 below for more detailed discussions regarding each of the core metadata sets that were developed within that Sub-group for the purpose of their inclusion in the NSG Metadata Profile document.

2.2.1 MFG Community


28


August 23, 2007

Figures 2-2a and 2-2b categorize the organizations participating within the MFG and also represent those organizations and activities that would likely have unique enough requirements to warrant their development of their own unique organizational metadata extensions. The figures also assign these organizations to a level of participation within the MFG.

Geospatial MetadataHarmonization Activities

-Advanced Geospatial Intelligence Metadata Profile Working Group (AGI MPWG)- American National Standards Institute/International Committee for Information Technology Standards – Geographics Committee (ANSI/INCITS L1) - Central Intelligence Agency (CIA)- Defense Information Systems Agency (DISA)- Dept Homeland Security Community of Interest Metadata Working Group (DHS COI MWG)- Federal Geographic Data Committee (FGDC)- Geospatial Intelligence Standards Working Group (GWG) and Relevant Focus Groups- Modeling & Simulation Community of Interest Metadata & Mediation Working Groups (M&S COI MWG)- Motion Imagery Standards Board Metadata Working Group (MISB MWG)- National Imagery Transmission Format Standard Technical Board (NTB)- National Security Agency (NSA)- NGA Engineering Data Working Group Metadata Focus Group (EDWG MFG)- Open Geospatial Consortium (OGC)

Figure 2-2a - Metadata Focus group (MFG) Partners

“Partners”, within the context of the MFG, in Figure 2-2a are organizations that conduct metadata-related activities and develop metadata materials for use within their own organizations which they share with the GWG MFG for review/coordination as well as provide input to other GWG MFG-facilitated activities.


29


August 23, 2007

Geospatial MetadataHarmonization Activities

- Distributed Common Ground System Multi-Service Execution Team Metadata Working Group (DCGS MET MWG)- DIA MASINT Standards Management & XML CCB

- Digital Geospatial Information Metadata Working Group (DGIWG MWG)

- JITC XML Multi-Function Lab

- Multinational Geospatial Co-Production Group Technical Group (MGCP TG)

- NATO Joint ISR Capability Group Metadata Harmonization Technical Support Team (JISRCG MH TST)- GeoScout

- NGA Enterprise Engineering (EE)

- NGA Image Product Libraries Access Standardization Working Group (IPL ASWG)

- National Reconnaissance Agency (NRO) & NRO IMINT Labs (ILABS)

Figure 2-2b - Metadata Focus group (MFG) Contributors

“Contributors”, within the context of the MFG, in Figure 2-2b are organizations that conduct metadata-related activities within their own organizations and are involved in the GWG MFG on a monitoring/observation basis to investigate and determine an appropriate area for their own contribution.

In addition to these specific activities, the MFG also pursues standards that enhance the ability to manage and preserve the integrity of GEOINT, including the development of standards profiles, registers of geospatial information, sponsoring of consortia test-beds and standards development activities, and monitoring the viability and exploitation of commercial standards, all with a focus on the return-on-investment in standards as measured at the enterprise level.

2.3 Geospatial Metadata Drivers



August 23, 2007

Figure 2-3 outlines the various geospatial metadata input directives and levels of management. These “drivers” are the force behind numerous metadata collection, management, archival, and verification tools and instructions, i.e. metadata standards that NGA and other DoD/IC organizations have been either mandated to use or have chosen to use. Much of the information collected, stored, and disseminated is driven by the organizations and directives from the data managers and requirements offices.

Geospatial Metadata Drivers

DoD MWG- Defense Discovery Metadata Specification (DDMS)

IC DMC- IC Metadata Standard for Information Security Marking (ISM) is an XML model for capturing the

components of CAPCO security marking instructions. - IC Metadata Standard for Publications (MSP) is a set of XML models for intelligence reporting, which

includes resource metadata, inline content tagging, and portion-level information security markings. - IC Metadata Standard for HTML Documents (HTML) defines resource metadata tags to identify an

information object.

ISO/TC211- ISO 19101-2 Reference Model for Imagery- ISO 19115:2003 Geospatial Metadata- ISO 19139 XML Schema Implementation of ISO 19115:2003- ISO 19115-2 Extensions for Imagery and Gridded Data- ISO 19130 Sensor Data Model for Imagery and Gridded Data

ISO/IEC JTC1 SC24- ISO/IEC 12087-5 Basic Image Interchange Format (BIIF) - ISO/IEC 18023-18026, 18041-18042 SEDRIS-based standards

ISO/IEC JTC1 SC29- ISO/IEC 15444JPEG 2000

Figure 2-3. Geospatial Metadata Drivers

Throughout the course of time, several metadata standards have been developed to meet broad community needs, while many organization-specific metadata sets have been developed to extend, subset and customize these standards to specific needs. This has resulted in many


31


August 23, 2007

metadata sets that needed to be harmonized as they have had to come together by necessity and be interoperable. The use of crosswalks is employed in this harmonization process by mapping equivalencies between metadata sets and metadata standards. (See the section of Part 2 of this document that details the work of the GWG MFG Metadata Registries and Crosswalks (RX) sub-focus group and the future of harmonization work.) There are currently twelve different Geospatial Metadata crosswalk activities on-going within the seventeen distinct metadata working groups that comprise the MFG membership. These include: standards crosswalks (ISO, DDMS, ICMWG, etc.), product crosswalks, and crosswalks for Image Product Libraries (IPL), vector data, imagery data, and sensor data. These crosswalks are being developed to assist in interoperability and harmonization across these activities and to also promote visibility and coordination between organizations. The results of these crosswalks will lead to minimum core metadata sets and data dictionaries for imagery, sensor, and vector metadata for the GEOINT community.

One of the deficiencies in current intelligence production process is that Geospatial Metadata production commonly occurs as an added task at the end of a process. The approach is both cumbersome and questionable as individuals attempt to recall data development methods and specific values to include in metadata tags. Ensuring that metadata is present in the underlying reported information and that it is preserved during the entire intelligence production cycle will reduce the analysts work load and ensure a more accurate intelligence product.

As the concept of geospatial data documentation, or metadata, is introduced to organizations, efforts generally focus on the documentation of existing geospatial data resources. As a result, metadata are captured after the data development process is complete. Unfortunately, most organizations continue this approach as they document new and evolving geospatial data resources. Metadata production outside of the data development process is both cumbersome and the results can lack integrity. Those producing the metadata must attempt to re-create stages of data development and recall specific values. The resulting metadata is often inaccurate and/or incomplete.

Most organizations have standing geospatial data development methodologies. While some methodologies are more formal than others, the integration of new processes and technologies is always disruptive because:

metadata standards are too extensive and difficult to implement

metadata production requires time and other resources

there are few immediate and tangible benefits and fewer incentives to produce metadata

the lack of guidance for the use, collection, and importance of metadata

One objective of these harmonization efforts is to address the obstacles identified above by streamlining metadata creation into the workflow process and providing guidance in the development of policies and procedures that will encourage and enforce metadata production.

2.4 Metadata Focus Group (MFG) On-going Activities



August 23, 2007

Current topical areas of focus for the Metadata Focus Group include:

Domain Space Handling

Minimum Geospatial Common/ Recommended Metadata Core

Taxonomy / semantics / Ontologies

Quality Metadata

Feature Metadata

Metadata Implementation

Symbology Metadata

Metadata Registries and Crosswalk Management

2.4.1 Domain Namespace Handling

This sub-group is developing a strategy for identifying and using namespaces to enhance interoperability between user communities. In addition, a set of guidelines will be developed for the implementation of this strategy to ensure that geospatial-enabled information can be shared. NGA’s Namespace Manager (NSM) will register all significant data dictionaries, Universal Modeling Language (UML) models and XML schemas relevant to geospatial intelligence in the DoD Metadata Registry (MDR—http://metadata.dod.mil). The NSM is the single point of contact for registration as well as for Governance policies and procedures. The DOD MDR Concept of Operations document outlines the NSM’s responsibilities. The DoD MDR is the “One Stop” Shop for publication and subscription for DoD XML metadata. The MDR contains data services infrastructure for Net-Centric Enterprise Services (NCES), which promotes information sharing, interoperability, and software reuse in a secure, reliable, global environment. At the time of publication (August 2007), DISA maintains registration for more than 90,000 data elements, 3300 XML schemas, 700 web services and 175 taxonomies in the registry.

2.4.2 NSG Recommended Minimum Geospatial Core(s)

This sub-group has developed a table of metadata elements which is the recommended set for use by NGA, and likely for use or consideration by other DoD Agencies and Activities. It is a listing of Mandatory and Recommended metadata elements in the form of a core data dictionary. The elements are intended to describe geospatial datasets, but are general enough so that they could apply to other sets of data as well. These Mandatory and Recommended core sections are drawn from the IC Metadata Standard, DDMS, Dublin Core, and is supplemented by ISO 19115 for geospatial metadata. The Mandatory section includes those metadata elements for compliancy with ISO 19115, and the recommended core contains conditional and optional elements from ISO 19115. The Group is also addressing minimum core sets for Vector Core, Imagery (Raster) Core, and Sensor Core.


33


August 23, 2007

2.4.2.1. NSG Recommended Geospatial Core Data DictionaryThis data dictionary of metadata elements is the recommended set for use by NGA, developed by the GWG/MFG. It is a listing of Mandatory and Recommended metadata elements in the form of data dictionary. The elements are intended to describe geospatial datasets, but are general enough so that they could apply to other sets of data as well. Figure 2-4 lists an excerpt of this data dictionary.

These Mandatory and Recommended sections are drawn from the Intelligence Community (IC) Metadata Standard, Department of Defense Discovery Metadata Specification (DDMS), Dublin Core, and is supplemented by ISO 19115:2003 – Metadata for geospatial metadata. The Mandatory section includes those metadata elements for compliancy with ISO 19115, and the recommended core contains conditional and optional elements from ISO 19115.

This is only a minimum list, and is intended to be extended. The following three items (See Figure 2-4) are examples of subject-specific extensions to this list. Each of the extended lists (Imagery, Sensor, and Vector) includes additional metadata fields for describing more specific resources. Additional lists of extended metadata will be proposed as the program matures.

Geospatial MetadataCore Dictionary

ISO 19115Dublin CoreIC CoreDDMS

Free TextCharacter String

Mandatorybrief narrative summary of the content of the resource(s)

resource abstract3


DateDateMandatoryreference date for the cited resource

resource reference date

2


Free TextCharacter String

Mandatoryname by which the cited resource is known

resource title1

SourceDomainData TypeConditionalityDefinitionNameRow#

Figure 2-4. Metadata Dictionaries In-work



August 23, 2007

2.4.2.2 Imagery Core (In Development)This list of metadata elements is intended to compliment the NSG Recommended Core list of elements and provide additional information to facilitate the discovery and retrieval of Imagery-specific datasets and resources. The elements in the list are drawn from the common elements in the Image Product Libraries (IPL) and from ISO 19115-2 – Metadata for imagery and gridded data, which is nearing completion

2.4.2.3 Sensor Core (In Development)This list of metadata elements is intended to compliment the NSG Recommended Core list of elements, and provide additional information to facilitate the discovery and retrieval of Sensor-specific datasets and resources. The elements in the list are drawn from the work of the GWG Community Sensor Model Working group (CSMWG) and from ISO 19130 – Sensor model for imagery and gridded data, which is in development.

The Sensor Core consists of a set of generic metadata elements used to describe the collection sensor of various items of interest. Sensor metadata can be further defined for any specific sensor if required for predetermined parameters. There are numerous metadata elements that can be used to support the requirement for an accurate, easy to use sensor model. For instance, one would assume that the “principal point” is the point at which the lens axis intersected the center of the collection array. But that is not always the case. Due to lens distortions, imperfections, and design flaws, there may be an offset. Thus for an accurate sensor model to have an accurate center point, offsets must be applied in the x and y direction. The same offset may be applied to data quality, source applicability, and exploitation capabilities.

2.4.2.4 Vector Core (In Development)The Vector Core metadata elements are intended to compliment the NSG Recommended Core list of elements and provide additional information to facilitate the discovery and retrieval of Vector-specific datasets and resources. The elements in this list are drawn from the work on the Multinational Geospatial Co-production Program (MGCP) Metadata Profile of ISO 19115. The Vector Core metadata elements provide information in four broad dataset descriptive areas: data identification, dataset content, source data information (lineage), and data quality. In addition, the Vector Core metadata set contains elements used to document the metadata itself; drawing on the elements defined in the set of Recommended Core elements. The Vector Core metadata set represents the minimum set of metadata elements required to document geospatial-intelligence vector datasets. Because of this, some elements which are optional in the Recommended Core (such as the coordinate reference system) are mandatory in the Vector Core.

In the Vector Core, the data identification metadata provides an identifier for the dataset and documents the producer, geospatial extent, security classification, and any commercial restrictions placed on the data.

Vector datasets contain information about geographic features. Each feature in the dataset is described by a feature type documented in one or more feature catalogues. The content


35


August 23, 2007

information in the minimum Vector Core metadata identifies the feature catalogue (or catalogues) containing the type definitions for the features in the dataset. The feature catalogue is required to validate the dataset data and for the exploitation of the dataset.

Lineage information identifies the source material used in the creation of a vector dataset. Source material can be imagery or other remote sensing data, other vector geospatial datasets, and non-geospatial datasets. Identification of the source information is required in many cases to determine the suitability of the data for a particular use. The Vector Core metadata elements require that, as a minimum, the oldest and newest sources used in creating the dataset be identified.

A key determiner of a dataset’s suitability for use is the quality (or accuracy) of the geospatial values contained in the dataset. Specifically, the Vector Core metadata set requires that the horizontal accuracy of the data in the dataset be identified. If the dataset contains vertical data, the vertical accuracy shall also be provided.

2.4.3 Geospatial Metadata Taxonomies, Semantics, and Ontologies

This sub-group will identify the taxonomy requirements for discovery metadata catalog content extensions and the taxonomy requirements for discovery metadata content extensions. The DoD Discovery Metadata Specification (DDMS) identifies the requirements for a discovery metadata catalog core augmented with content extensions as needed by each Community of Interest (COI). More detailed taxonomy guidance is required for discovery metadata catalog content extensions.

2.4.4 Quality Metadata

The goal of this sub-group is to recommend to the GEOINT Standards Working Group, Metadata Focus Group (MFG) a standard protocol (format, process) for documenting the quality of geospatial data in GEOINT metadata. The GWG MFG may then determine next steps for incorporating the recommendations into practice across the NSG, and to identify key relevant international, national, and/or and federal standards impacting the process by which geospatial data quality are assessed and documented, and identify any outstanding or conflicting issues for future resolution.

The most difficult, yet most important, characteristic of metadata for geospatial data is reporting data quality. The dynamics related to reporting geospatial data quality present a variety of diverse challenges for geospatial feature utilization. When trying to determine the fitness-of-use of geospatial data for a particular mission, a consistent and standard set of detailed information is useful, if not necessary, to validate the selection of data that meets end-user criteria or application requirements. As more accurate geospatial data is developed, maintained, and shared throughout the DoD, the ability to determine data suitability through current reporting mechanisms is severely deficient and needs enhancement.


36


August 23, 2007

The Quality Metadata Sub-Focus Working Group (QMSWG) recognizes the limitations/absence of existing data quality reporting protocols. It must focus on a wide array of data quality characteristics and provide relevant instructions that enable data stewards to collect and convey more robust quality guidance and facilitate end-user quality awareness. This group will leverage existing ISO standards, best practice implementations related to quality reporting, and recommend a framework for data quality assurance planning.

As geographic data become institutionalized across DoD War fighter, Intelligence, and Business Mission Areas as a decision support mechanism, the rationale for collecting specific geospatial details becomes increasingly crucial. However, in order to support the concept of quality reporting in a broad universe of varying geospatial implementations, there is a need to apply the assessment of data quality towards known mission support requirements and/or product specifications. Quality reporting can be achieved by extending existing standards and developing additional guidance for mission specific data (MSD) stewards to craft their own quality assurance plans that can be easily implemented and sustained.

Data quality has quickly developed as the key dynamic in influencing utilization, and as such, deserves focused attention in order to support the reuse of this asset by secondary unanticipated users.

2.4.5 Feature Metadata

Feature and attribute level metadata will become increasingly more critical as data providers move from “product” based storage to a more feature centric set of data holdings. More and more metadata will be required at the feature and attribute level, but as is the case with all metadata, the collection and population of pertinent information can be time consuming. Advances in software and automation processes for metadata collection will be critical to the future viability of feature and attribute level metadata collection.

The Feature Metadata Sub-Group will, understanding the constraints of today’s production environments, concentrate on the development of a core set of feature level metadata. In order to accomplish this, the group is in the process of developing a set of use cases for vector feature instance metadata. This is the metadata that is use to describe the feature (for example, positional accuracy of the geometry) and the feature’s properties (i.e. method used to determine the positional accuracy). Because there are potentially hundreds of metadata elements that could be used in describing geospatial data, this sub-group has decided to examine the metadata requirements for a set of specific use cases or scenarios. It is expected that these use cases will lead to core metadata profiles that identify the metadata needed by users in these scenario areas.

A number of questions remain to be answered concerning the responsibility of the data producer versus the data consumer when it comes to feature level metadata. For instance, at the dataset level, metadata is made available by the data provider concerning producer such as point of contact, address, etc. When addressing feature level metadata, is it required that the provider identify the producer information for each feature or does that responsibility fall to the data consumer and their client software to populate information concerning from where data was received? The Feature Metadata Sub-Group will address these issues and others. It is


37


August 23, 2007

anticipated that at the core level feature metadata will concentrate on information that facilitates discovery and retrieval of feature data from a feature centric database through a net centric environment. Effective discovery and retrieval will require many facets of metadata specifically tied to user requirements and use case scenarios. For example, it is anticipated a user will ask questions such as “what feature data can be found over my area of interest, that is current to specific dates of interest, that has the positional accuracy (or other data quality components) I need”. The level of detail in feature level metadata tagging must be sufficient to answer such questions.

A secondary level of feature metadata may be required to provide answers to the quality of the metadata itself. Described as meta-metadata this will address items such as providing definition and explanation of procedures and or formulas used in determining data quality results.

A third set of feature, and in some cases attribute, level metadata will be needed to address security and release restrictions. It is anticipated that where required certain features and/or certain attributes will only be made available if proper credentials are presented. Software solutions will be required to address, for example, the issue of PKI certification and access to data holdings. Metadata at the feature and/or attribute level for security restraints will be critical to enabling any software solution.

Feature and attribute level metadata will use ISO 19115 with extensions as required. Extensions to ISO 19115 for security and release restrictions as well as code lists will be required. Additional extensions to the feature level metadata profile may be required to support legacy products and predefined fitness for use. Standard metadata elements across the Federal network may be achieved by coordinating with other initiatives such as the FGDC and efforts developing the DoD Discovery Metadata Specification.

2.4.6 Metadata Implementation

Creating a metadata implementation involves determining the metadata information requirements for the target user community. ISO 19115 provides a framework for organizing geospatial metadata and can be used with the user community information requirements to create an implementation profile of the metadata standard. By profiling an accepted standard, exchanging information within the user community and, if required, with user’s outside of the target community can be facilitated. However, in order to affect interchange, an encoding of the information profile must also be specified. This sub-group will address these issues and provide guidance to the user communities.

2.4.7 Symbology Metadata

Symbology consists of a set of graphical symbols and the portrayal rules to visualize these symbols in a manner meaningful to the user. Portrayal rules can vary by community and are therefore maintained separately from the symbol itself. Symbology is addressed by ISO Standard 19115 on metadata and ISO Standard 19117 on Portrayal. Organizations such as the Open Geospatial Consortium (OGC) have developed detailed web service based specifications for symbol encoding, symbol management, and symbol portrayal based on


38


August 23, 2007

these ISO standards. This effort will identify how symbol dataset metadata will reference these and other portrayal architecture specifications.

This sub-group will attempt to identify the metadata requirements for symbology and create a Recommended Metadata Core for this domain.

2.4.8 Metadata Registries and Crosswalks (RX)

Registries and Crosswalks (RX) is a combined subgroup dealing with the overlapping areas of Registries and Crosswalk Management, including harmonization and mapping activities.

Because standards have in the past developed independently of each other, they are often specified differently using specialized terminology, methodology and processes. Besides providing a useful map between similar items in different systems or standards, performing new crosswalks and studying existing work is a way to highlight terminological issues and address their resolution. One way to cut down on the “clutter” of having too many inter-related crosswalks and mappings is to map each specification or profile against a common standard. When elements of multiple specifications are mapped to a common standard, the inter-relationships between those specifications can be more easily identified. This also allows the common standard to function as an “exchange standard” between all the specifications mapped to it. This is the approach being taken with the MFG to map the many GEOINT metadata specifications and profiles, including NATO STANAGs, to a common standard, namely ISO 19115.

Metadata harmonization activities between DCGS and the NGA have been taking place since early FY 2005. Starting in FY 2006, NGA in coordination with USD(I), established a Metadata Harmonization activity consisting of a small team of Domain, XML, and Data Modeling technical experts representing each Service and NGA to harmonize data elements associated with the data assets available within the NGA and DCGS enterprises. This harmonization effort focuses on discovery metadata and is consistent with the Net-centric Data Strategy to make data visible, accessible and understandable. The results of the harmonization efforts will be registered in the DoD MDR.

The Registries part of the sub-group deals with Registration and Administration practices and policies. The Registries sub-group interacts with the Namespace and Domain Handling Group to establish and normalize practices and procedures.


39


August 23, 2007

Part 3 – Geospatial Metadata Issues and Exploitation


40


August 23, 2007

Part 3 – Geospatial Metadata Issues and Exploitation

3.0 Metadata

This portion of the Metadata Desk Side Reference will provide general and specific information about the requirement for capturing, archiving, disseminating, populating, and using the information referred to in this document as “metadata”. Hopefully, when complete, Part 3 will be most useful and helpful for the explanation of how and why metadata is important.

3.1 General discussion of Metadata – data about data

Metadata has traditionally been used in a variety of ways to increase the utility of certain other datasets. The assets contained in these datasets are extremely valuable for missions and goals of the community at large. Certain groups within that community may or may not know of the existence of certain assets (Data!). Traditionally, DoD organizations have used data administration and management as the primary tool to define the structure, relationship, content, use, and exploitation techniques for the databases under their control. Through various “software applications”, the DoD organizations have authorized access to this information even though other coalition partners were not allowed access. These management techniques and controls tended to prevent interoperability rather than enhance it.

Metadata requires enhanced, consistent definitions to properly describe the elements or fields that need to be maintained. These consistent definitions will support planning for new systems, new system development, and enhanced system operational capabilities. Metadata needs to describe key attributes and “define” the data elements that can be used in the discovery and access processes. There are vocabulary elements that define a “language” or “consistent meaning” to certain elements that can, at times, have multiple meanings. Very similar are the “taxonomic” data elements that will consistently name the fields for use throughout the community. There are “discovery” metadata elements that are used for remote site discovery of information contained in a central database of related datasets. There are “structural” metadata fields that define the composition and relationships of various related elements. All of these metadata types are closely related and can be utilized to maintain control, understanding, and access to the database without the management authority losing its overall guidance. Various other storage and processing capabilities are used throughout the industry. Registries, catalogues, and shared responsibilities are used to store, discover, provide access, and generally support interoperability.

3.2 Geospatial Metadata – Management

In this instance, geospatial metadata is a key element in sharing information concerning the collection processes, location, exploitation capabilities, quality, and mission value of data contained in multiple database structures. The maintenance of geospatial metadata elements requires consistent definitions and authoritative centralized control of the elements for future use by disassociated groups and organizations. To aid in the management of metadata, a metadata


41


August 23, 2007

registry is often utilized. A registry is a file (often a separate system) that contains vital information about the elements, schemas, or systems that describe the structure, format, definitions, location and occasionally the content of the data. Certain values or terms are allowable at any given time. These values are sometimes controlled by “language” or “vocabulary” metadata type environment. Registries can also define the “relationships” between metadata elements. These relationships can be dependencies, value-added instances, or optional data choices that rely on external influences.

Registries do not necessarily contain the actual data values but can simply store the format required for the metadata element that may contain the information needed to interpret the data. Allowable contents are maintained and described to the user, producer and developer depending upon the intended use. For instance, “keyword” may be a metadata element that is required for discovery of an item in a dataset. However, the information contained in the “keyword” element may not be defined as part of the metadata element but is contained in the “registry” of allowable values. This situation supports the access and discovery requirements as well as the production capabilities.

3.2.1 Metadata Architecture documentation

The NSG Architecture Compliance Document – The National System for Geospatial-Intelligence (NSG) Architecture Compliance (NAC) document, referred to as the “NAC”, serves new and Legacy/Heritage (L/H) systems that compose the NSG enterprise. The NAC defines the set of architecture principles and provides implementation direction for complying with the architectural principles and the Technical Architecture Views for systems that are delivered by the National Geospatial-Intelligence Agency (NGA). The body of the NAC and its Appendices and Addendums are to be used by NGA and by non-NGA organizations that develop systems to interoperate with NGA-developed systems.

The NAC applies to new systems inserted into the NSG. A “new system” is defined as a system provided by NGA. NGA L/H segments and associated data will be modified to be compliant with the metadata, interface service standards, and interoperability requirements described in the NAC on a case-by-case basis dependent on system functionality and projected retirement date. New systems that are operational pilots must be compliant with the NAC. (Note: Additional details on the NAC and a link to the NAC site will be added to a future edition.)

3.3 Architecture Implications

By utilizing tools such as the NAC, metadata architecture can be useful to provide aspects of metadata that are invaluable to the producer/customer. Principles of architecture to maintain as much as possible are:

a. Make metadata visible—Users and applications can discover the existence of data assets through catalogs, registries, and other search services. All data assets (intelligence, non-intelligence, raw, and processed) are advertised or “made visible” by providing metadata, which describes the data asset. Data storage and access need to be


42


August 23, 2007

generalized to be system function independent and not singly optimized to support access or update by one given system function or system.

b. Treat data as an asset — Data is an asset that has value to users and to the enterprise and is managed accordingly. Data is a valuable corporate resource; it has measurable value. The purpose of data is to support users and drive problem solving and decision-support. Accurate, timely data is critical to accurate timely solutions and therefore must be managed carefully. Data is the foundation of the NSG so we must carefully manage data to know where it is, to rely upon its accuracy and to obtain it when and where it is needed.

c. Make data accessible—Users and applications post data to a “shared space.” Posting data implies that (1) descriptive information about the asset (metadata) has been provided to a catalog that is visible to the Enterprise, and (2) the data is stored such that users and applications in the Enterprise can access it. Data assets are made available to any user or application except when limited by policy, regulation, or security. Data is posted before processing, so that the raw data can be made available for more than a single processing step and new processing can be more easily added.

d. Make data understandable—Users and applications can comprehend the data, both structurally and semantically, and readily determine how the metadata may be used for their specific needs.

e. Establish data as an Enterprise Resource1. Critical data is maintained at a single central place; copies may be

transferred elsewhere as required to support functional and performance requirements.

2. Data content conforms to enterprise standards with respect to naming, meaning, referential integrity, and other validation criteria, as well as format and tagging for exchange purposes.

3. Data content is labeled using enterprise standards for metadata and security labels.

4. Synchronization of metadata is supported by enterprise tools, on an enterprise-managed schedule.

5. Access privileges are defined according to enterprise policy. 6. Data are discoverable and searchable using enterprise discovery

and retrieval tools. 7. Data are also available to authorized users for “native” access via

file system access, COTS-provided APIs, query languages or similar tools. 8. Data base administration is provided as a centralized service.

3.4 Other Metadata Categories

With the proper controls and management tools, there is the opportunity to eliminate redundancies and ambiguities in geospatial metadata. However, when attempting to separate metadata into various categories, there will almost always be some overlap due to the nature of


43


August 23, 2007

the product and element derivations. The following are examples of some of the various categories of metadata not already discussed.

3.4.1 Dublin Core

The Dublin core set of metadata elements is the basic set of core metadata elements captured in an ISO standard: ISO/IEC 15836. The Dublin core was established by an international consortium for use with other ISO standards. The core is the basis for Intelligence Community metadata requirements and the mandatory set of metadata for DDMS/XML schema implementations.

3.4.2 Imagery Product Library metadata

Another set of metadata fields that is extremely useful to the customer of geospatial information is the imagery and geospatial product library descriptive metadata elements. These libraries are used primarily for specific information that requires a geo-location and description to ensure proper identification. Metadata is one of the key pieces of information required to identify, locate, get access to, and distribute the items in the library in a rapid, accurate method. When the production entity (agency, organization, or corporation) releases a product for use by the customer, it is automatically distributed to the manager of that particular geographic area of the earth’s surface. The item is then placed into the library for access by the customer requiring the information to be exploited in some manner. If the metadata contents are not properly populated, the piece of information may not be identifiable or not be accessible to the customer during his enquiry process. Therefore, critical information may not be included in the decision making process that requires all types and pieces of information to be evaluated.

As noted in the previous section, some overlap of metadata is going to occur depending on the informational requirements for tracking, quality, identification, and location. In this case, “sensor name” is the same piece of information captured as metadata in the Sensor Modeling. This is an example of the importance of populating and capturing metadata accurately and consistently. If the usefulness (quality, exploitation capability) of the dataset is dependent on the name of the sensor that collected the information, it becomes imperative for interoperability that the metadata is consistently captured.

3.4.3 MASINT Data Sharing and Metadata requirements

MASINT developers may need to point to the intelligence data dictionary/schemas/models, however, everything that will provide a consolidated list of MASINT Sensors and MASINT collection requirements (which both include signatures) is not contained in MASINT community XML standard docs. To fully describe the processes and procedures (at least to an extent) in addition to our standard docs, the following would be needed:

1) the MASINT tags developed for the original NTSDS and MRS metadata requirements (collection metadata tags in our common data storage instance),

2) the tags coming out of the SAR study DO (sensor specific),


44


August 23, 2007

3) the tags that NSP believes they need for data sets/signatures,

4) the tags specifically used in the MASINT products (the style-sheets). A compilation of these metadata elements may be a start for the data sharing requirement between the “---INT” systems.

As of today, there is limited interoperability and data-sharing requirements across the intelligence systems world. Each “---INT” system has its own set of elements required to access the independent data library (or primary data base). There are preliminary discussions of sharing information between MASINT, SIGINT, IMINT, and HUMINT but currently have not been assigned for process consideration.

3.5 Testing of metadata requirements

3.5.1 JITC/ILABS

The testing requirement for metadata has never been fully enforced nor required. The best that has been available is a “strong recommendation” for testing. There are several reasons for this lack of testing enforcement of metadata. First and foremost, is the inability of the testers and testing software to verify the correctness of the metadata with respect to the data itself. Since many Agencies have been involved with and dedicated to the production of “products”, those organizations are the sole verification authority. If the product is an “elevation matrix”, the test organization can verify that the data is populated correctly and that the metadata fields (elements) have been populated with data that fits the “format”. The content can not be verified unless the production organization is willing to release the source, the exploitation capabilities, and the quality requirements for that dataset.

3.5.2 JFCOM Test Laboratory

The Joint Systems Integration Command (JSIC) aims to improve warfighters' ability to plan and execute operations by:

• driving resolution of command and control (C2) interoperability problems

• providing unbiased evaluations of existing and emerging C2 capabilities, and

• rapidly integrating technology solutions Located in Suffolk, VA and assigned to the U. S. Joint Forces Command (USJFCOM).

JSIC combines the best of infrastructure, methods, and people in a culture of continuous learning. JSIC efforts to resolve interoperability issues have included a series of Joint Intelligence Interoperability Board (JIIB) Joint System Baseline Assessments (JSBA) to assess the ability of existing and prototype command and control (C2) and intelligence systems to support data identification, exchange, and retrieval, and to execute network enabled operations at the joint task force (JTF) level.

JSIC technology assessments are conducted using a recognized and repeatable methodology that measures maturity, jointness, and warfighter utility. Assessments also identify potential costs and impacts on doctrine, organization, training, material, leadership, personnel and facilities.


45


August 23, 2007

Using the assessment results, JSIC provides objective recommendations through USJFCOM to the Joint Requirements Oversight Council and other C2 program decision makers on the effectiveness and implementation of actual C2 systems. This proven and successful process acts as a forcing function for technology insertion of new capabilities. JSIC efforts to provide short-term interim solutions for existing C2 problems include the recently-deployed C2 On-The-Move system, a prototype vehicle-mounted system to provide mobile communications for commanders during operational engagements/deployments, and Wireless for the Warfighter, providing long-range wireless access to deployed JTFs.

JSIC personnel represent all four military services, plus government civilians and contractor support. These personnel, combined with strong support from both industry and academia, form an unbiased organization that provides objectives and feasible recommendations to meet joint needs.

Activated on 17 Dec 1996, the Joint C4ISR Battle Center (JBC) began as a Chairman of the Joint Chiefs of Staff-controlled activity. In 1998, the JBC was realigned under then-U.S. Atlantic Command, now U.S. Joint Forces Command. The JBC became JSIC on 9 Sep 2004. JSIC directly supports all the combatant commands by validating selected current and proposed warfighter C2 systems. This process identifies systems that clearly demonstrate joint utility or identifies non-interoperable systems for elimination.

3.6 Geospatial Profiles and Access Tools

3.6.1 Geospatial Knowledge Base

The Geospatial Intelligence Knowledge Base (EL-GKB or GKB) system element provides data access and data management of NSG data. GKB provides data management capabilities and federated access capability for the enterprise. These capabilities include the ability to access, insert, update, delete, index, ingest, import, and export mission, mission management, corporate, and infrastructure data.

3.6.2 Federated Access

The Federated Access capability provides functionality for users to discover and retrieve GEOINT and other Multi-INT data from the mission data management capability, other internal NGA data sources, and external sources.

3.6.3 Resource Tasking and Marketplace

The Resource Tasking and Marketplace (EL-RTM or RTM) system element of data management (including metadata) provides the ability to receive and manage user requirements related to GEOINT and Multi-INT information needs, and broker products and services to satisfy user requirements. This enables the establishment of a geospatial intelligence marketplace by bringing together users and suppliers of information and services, defining roles and responsibilities of participants, and providing an ordering and order tracking mechanism. The


46


August 23, 2007

RTM system element is provided through software only and leverages the processing and storage provided by system engineering and the data management capabilities of GKB.

3.6.4 Global Metadata Catalogue (GMC)

This section will be included in a future edition of the Metadata Desk-top Reference Guide as the GMC becomes mature. There have been several requests for a document such as a Global Metadata Catalogue or a similar descriptive document detailing the metadata requirements for various implementations across the NSG.

3.7 Metadata Management and Responsibilities

3.7.1 Records Management Metadata (i.e. DoD Directive 5015.2/NGA Records Management docs)

Records management is a requirement established by DoD Directive 5015.2. This directive requires that all information used or produced by an Agency as “federal records” be documented in order to identify, locate, retrieve and manage the information. The requirements of this directive are generalized for all federal records, but must be met for metadata describing geospatial data as well. It is incumbent upon the developers of any metadata implementation to ensure that the records management requirements are met in accordance with Directive 5015.2.[NOTE: Need to determine how this applies to web services. For example, a web-centric environment could produce datasets dynamically by serving out features (in this case vector features, imagery features, and gridded features such as elevation models) from different datasets. From the user’s standpoint, it might be useful to have a metadata instance presented that describes the contents of this dynamically created “dataset;” however, the requirements of 5015.2 need to be examined to determine if this dynamically created dataset also needs to be recorded from a records management point of view.]

3.7.2 Date Metadata: Date Metadata is the class of metadata elements that captures life-cycle date information associated with products.

1. Publication Date 2. Date Posted 3. Cutoff Date 4. Date Valid Until (review cycle? Or obsolescence date)

3.7.3 Terms of Use Metadata: “Terms of Use Metadata” is the class of metadata elements that indicate whether legal rights or restrictions are associated with products.

1. Privacy Act Indicator 2. Vital Record Indicator 3. Copyright Indicator

3.7.4 Electronic Metadata: Electronic Metadata is the class of metadata elements that provide information needed to electronically process and consume products.

1. Application Title



August 23, 2007

2. Data Format 3. Uniform Resource Locator

3.7.5 Representation Classes: The list of representation classes is as follows: 1. Absolute URL 2. Activity Name 3. Classified Sensor 4. Code 5. Date 6. Date/time 7. Product Line Name 8. Qualified Data Activity Date/time 9. Qualified Data Activity Date/time Range

3.7.6 Repeatable Metadata Fields: A data element only needs to repeat if there are multiple values that need to be captured. The data elements that can support repeatable data include:

1. SCI Controls2. FGI – Open Source 3. Dissemination Controls 4. Releasability 5. Document Identifier 6. Country 7. Subject Code 8. Intelligence Discipline Type 9. Point of Contact 10. Intelligence Sub-description11. Tasking Requirement 12. Sensor 13. Product Line 14. Date-time Point 15. Analysis Tool 16. Activity Recording 17. Keyword 18. Non-Intelligence Community Markings

3.8 Classes of General Metadata

3.8.1 Introduction

The basic definition of metadata is “data about data”, so any time you have descriptive information about something you effectively have metadata. For that reason metadata comes in many different forms and flavors. One person’s metadata is another person’s data. Different developers approach data differently, so their implementations can be drastically different in what they call metadata.


48


August 23, 2007

This can often lead to confusion as to what exactly metadata is, and how it should be used. The following definitions are an attempt to explain the terminology that will be used in this document.

3.8.2 Metadata Metadata: type of metadata that describes other metadata. A metadata record can be treated as a resource (or data), in which case it would have its own metadata. Just like data is updated and maintained, so must metadata be updated and maintained. The information documenting when the metadata is updated and maintained is an example of meta-metadata, as it is information about the metadata record.

3.8.3 Security Metadata: class of metadata elements required to fully specify the classification and controls through the entire information lifecycle.

1. Classification 2. Owner/Producer 3. SCI Controls 4. Special-Access-Required Program Identifier 5. Foreign Government Information 6. Dissemination Controls 7. Releasability 8. Non-Intelligence Community Markings 9. Declassification Manual Review Indicator 10. Declassify On

3.8.4 Product Metadata: class of metadata elements that provides identification, content description, and publisher information about products.

1. Title 2. Document Identifier 3. Summary 4. Keyword 5. Country 6. Subject Code 7. Agency 8. Point of Contact 9. Intelligence Discipline Type10. Product Line

3.9 Types of Specific Metadata

Extension Metadata (MASINT): Extension Metadata is the class of optional metadata elements that may be used to specify additional metadata for cataloging and discovery in a Community-standard way. All of the extension elements listed below originated with the MASINT community. However, these elements have applicability to more than one intelligence discipline.

1. Intelligence Sub-discipline Type 2. Tasking Requirement


49


August 23, 2007

3. Sensor 4. Report Phase 5. Date-time Range 6. Date-time Point 7. Analysis Tool 8. Activity Recording

3.10 Profiles of Standards

3.10.1 Introduction

Most metadata standards are generalized for broad applicability. As can be seen by the various types of metadata canvassed in this document so far, implementing a complete set of metadata could result in a metadata instance rivaling the size of the data itself. The reality of data production is that there is a finite amount of time available to document the dataset produced; however, it is important from a regulatory as well as usability standpoint that the dataset be properly documented with adequate metadata.

For the interchange of information, it is important that the parties in the exchange agree on the organization and encoding of the data. The organization and encoding of the metadata exchanged is just as important. Profiles are developed to meet the requirements for a particular use within a particular user community. In order to simplify production and allow the data produced to be able to serve the broadest set of user communities, it is important for different user communities to reuse profiles and extensions to standards whenever possible.

In order to be adopted for inclusion in the DISR, the profile must be documented. According to DoD Manual 4120.24, there are several forms of documentation that could be used. An appropriate format must be chosen in the profile and it should provide guidance on the use of the format in the documentation of a metadata profile.

Profiling/Extending Metadata – Form of profiling documentationReference: DoD Manual 4120.24 – see C.5.2.7-10, i.e. Defense Specification, Guide Specification, Defense Standard, Defense Handbook

The NSG metadata standard will be a profile of the ISO 19139 and 19139 Part 2 (metadata implementation specifications for geospatial information and imagery respectively) with appropriate extensions to account for geospatial intelligence. These comprehensive implementation specifications provide conformant profiles (defined in UML) and XML schemas of the ISO 19115 and 19115 Part 2. These profiles are meant to enhance interoperability by providing a common base for describing, validating and accessing metadata about geographic, imagery and gridded data between coalitions while maintaining a high degree of interoperability with other communities complying with the ISO 19139 and 19139 Part 2 technical specifications. For example, a DGIWG profile is required to more strictly enforce rules required for the NATO military coalition environment as well as to standardize extensions that apply only to this special user community. A design schema is required to detail the infrastructure required


50


August 23, 2007

to support the capture and storage of metadata. The design schema may contain several profiles that define metadata for cataloguing, search and retrieval, and exploitation.

3.10.2 Standards “Compliant” versus “Conformance”

There is still some open thoughts on the difference between “being compliant” to a standard and “conforming” to a standard. A developer can conform to a standard but not be interoperable to the fullest extent. The other developer can be compliant to the standard but not fulfill all of the contractual obligations as specified by the customer. There has always existed a fine line between the two ideas. There has been no finite definition for either term when discussing standards, metadata adherence, or product orientation. In the past several years, the most difficult aspect of data and metadata processing has been the testing for compliancy to standards. As stated above, to test for compliancy is to test for format, not content. There is no way of knowing, for compliancy testing, if the data is good, bad or indifferent. However, the format and structure can be tested thoroughly provided a good test set and test procedure has been written and implemented.

How can compliance be confirmed with geospatial datasets? With feature data? With dynamically generated datasets (i.e. features returned from a query to a Web Features Service)?

What it means to have standards compliant metadata:- Benefits of compliancy- Consequences of non-compliancy- Compliance with Records Management Requirements (5015.2)- Harmonization- Data transformation (legacy-to-minimum core)

3.10.3 Profiling/Extending Metadata – In work.

3.10.4 Management Responsibilities for general metadata controls

This section is in work for enforcement procedures and management responsibilities. The DISR Online mandates certain IT standards and the procedures for control are fairly well established. However, the relationships to the multiple standards and standards organizations are being developed as part of the NAC.

3.11 Web Services

Web services provide the capability to serve datasets, both static datasets akin to the products of today and dynamically-generated datasets built on demand in response to the query of a data consumer. The metadata requirements for web services cover at least two areas: metadata describing the services and metadata describing the data. The service metadata provides information about what types of services are available to a data consumer. The data metadata provides information about the data that is available and the data that was returned in response to a user’s query.


51


August 23, 2007

For a service that serves traditional datasets, the metadata returned in response to a user’s query identifies the data that matches the user’s query parameters. This metadata may be an aggregated set of metadata for all matching datasets, or could be the metadata for each of the datasets matching the user’s query. This metadata can then be used to further evaluate if the datasets identified are suitable for the user’s purpose and to locate the dataset for retrieval.

For a service that serves dynamically-generated datasets, the information returned from the service could be thought of as a product. A single instance of metadata is returned describing the dataset produced with information provided to the user for locating and acquiring the dataset.

3.11.1 Data Representation – WEB specific

3.11.2 MD TAGS vs. Metadata

3.12 User Requirements

Metadata for geospatial data is complex and can consist of many individual elements. For example, the ISO 19115 metadata standard is composed of more than 300 elements. In addition, there are multiple levels of use for metadata, among these are:

a. Discovery – searching catalogues in order to locate datab. Exploration – determining if a subject dataset is suitable for the user’s applicationc. Exploitation – information needed in addition to the data in order to make use of the

dataset

The metadata sets needed by a data consumer need to be evaluated to determine the complexity level required by a user. The NSG Recommended Core (aka NSG Minimum Core), discussed in Part 2, provides a set of metadata elements that meet the broad user requirements identified for the GEOINT community. The requirements addressed by the NSG Recommended Core are aimed primarily at cataloguing and discovery for information across the NSG. For a particular user community, the information required by data consumers for exploration (evaluation) and exploitation also needs to be identified. The additional information elements (or more specific information elements) can often be identified from existing legacy systems currently in use by the target community.

3.13 Encoding

ISO 19139 provides a standard XML schema implementation of the ISO 19115 metadata standard and provides a common language for exchange of metadata. XML is a good encoding for common exchange because it is an open standard and open source tools are available from multiple sources for parsing, validating and transforming XML encodings. Additional issues need to be addressed:

a. Legacy encodings – Support is needed to transform from XML to legacy encodings while legacy XML-ignorant software is still in use. There is also a requirement to


52


August 23, 2007

transform legacy encoding formats into XML encodings. In the arena of tactical message handling, the Common Message Format (CMF) being developed by the Integrated Broadcast Services (IBS) program office is pioneering this two-way transformation process.

b. Tactical encoding requirements – The ISO 19139 XML schemas are very overhead intensive in order to provide a simple mapping to the ISO 19115 standard. Tactical data links are very sensitive to excessive overhead, so a compact encoding for metadata is required. IBS is developing a binary format for CMF that addresses this issue for tactical intelligence data links and broadcasts. In addition, XML responds well to simple data compression algorithms.

c. Encoding extensions – User community requirements will inevitably require extensions to the standard. Guidance should be developed to provide a consistent mechanism for development of extensions.

3.14 XML Schema reuse capabilities -- The XSD Concept

As mentioned in Part 2, Item N, Metadata Registries, one of the primary methods of utilizing the XML technology and capability is the “XSD – XML Schema definition” language and it’s associated concepts.

Using XSD to define the encoding of metadata exchanged by a target community allows for the reuse of existing schema (such as the schema defined by ISO 191139). In addition, XSD allows a community to define community-specific elements to meet their own specialized requirements. The reuse of XML schema definitions provides a more consistent encoding for information exchange. XSDs defined within their own namespace reduce the risk of naming conflicts when several schemas from different developers are brought together. The base XML schemas developed for general purposes (such as the geospatial metadata XML schemas) form one part of the encoding scheme for information interchange. For example, a cataloguing service may return metadata describing a number of datasets that meet the requirements of a query. The metadata can be encoded using ISO 19139 but will be encapsulated in the response structure (also encoded using XML) defined for the cataloguing service. The obvious advantage of using a single encoding for geospatial metadata is that the information can be decoded by a receiving system once it is aware of the way in which the metadata is encapsulated.

3.15 Crosswalk Management and Entity Mapping

3.15.1 Crosswalk Management Basics

A crosswalk is a record of the decisions made about the definitions of data elements from different standards, representations, or models and a record of the mapping of these data elements between standards


53


August 23, 2007

Crosswalks and mappings are useful tools that contribute significantly to various metadata efforts. They help the harmonization effort by showing where standards have points of commonality and where they diverge. They help to show conformance by showing where a specification or profile is conformant to a standard.

There are many standards and specifications, so there are many crosswalks. To make matters worse, each of the standards and specifications have their own level of abstraction, so what is elemental to one standard may be an aggregate entity in another. This can lead to a confusing environment where everything is mapped to everything else at varying levels of abstraction. That sort of environment is not the most useful environment.

One way to cut down on the “clutter” of having too many inter-related crosswalks and mappings is to map everything against a common standard. When elements of multiple specifications are mapped to a common standard, the inter-relations between those specifications are drawn out as the individual elements are mapped to a common element. This also allows that common standard to function as an “exchange standard” between all the specifications mapped to it. However, experience has shown us that the exchange standard, or hub, can not be developed in the abstract, but must be created by accretion of the elements as each new standard, or spoke, is added to the wheel. This makes it essential that the information about the entities, elements, attributes, notations (domains) be kept in the form of data, and not as data structures.

3.15.2 A Scalable Approach to Crosswalk Management

The approach to data harmonization that works at all scales is based on reducing things to the essence of data modeling, as shown in Figure 3-1. When implemented in a relational database, this is a three table solution. So far in modeling the structure of databases, DTDs, standards, etc., only two tables, attributes and relationships have been needed. The relationship modifier table had not been invoked, although some of the fields in the relationship table could be encoded as modifiers in a more normalized form of the database.


54


August 23, 2007

20

The Essence of Data Models

Named Attribute of SomethingNamed Attribute of Something

Named Attribute of SomethingNamed Attribute of Something

Named Relationship Between AttributesNamed Relationship Between Attributes

Modifiers to Named RelationshipModifiers to Named Relationship

Named Attribute of RelationshipNamed Attribute of Relationship

Named Attribute of ModifierNamed Attribute of Modifier

Attribute A

Attribute B

Figure 3-1. The Essence of Data Models

Figure 3-2 shows the power of this method in harmonizing across disparate data models. Down at the bottom of the screen shot one sees that the TGTID field in the NITF 2.0 Image Sub-header identifies three separate metadata elements in the IC MSP Information Pool DTD. So the internals of a single field in an image transmission data header have been linked to data elements in a finished intelligence product.

Note that the selected record in TBL_ATTRIBUTES is number 39131 out of 39517. It would require more that one spreadsheet to represent these attributes in Excel workbooks.


55


August 23, 2007

Figure 3-2. Crosswalk between NITF 2.0 TGTID and IC-MSP InfoPool DTD

It is also important to note that the attributes and relationships from the IC MSP Information Pool DTD module were not entered by hand, but were entered into the JADB by a portion of the JADB that reads text files according to parsing instructions that are also contained in the JADB as attributes and relationships. In the case of an XML DTD, we manually entered the Extended Bacus-Naur Form (EBNF) productions from the XML standard, as shown in Figure 3-3. The EBNF production named “extSubset” is the production that represents an external XML DTD file.


56


August 23, 2007

Figure 3-3. Partial View of XML 1.0 Standard EBNF Production Tree


57


August 23, 2007

For each of the productions, parsing actions were defined that would take place when a production was matched to the incoming DTD. Shows the way this is done for the Name of an Attribute Definition. Guidance on how to read the screen in Figure 3-4 is listed below:

The “AttDef Has One Name – Entity Name” data tells the parser that there is one and only one instance of a Name at this portion of a valid DTD.

The “Attribute B Units or Modifier: Relation.Defines – Defines” data tells the parser that new relationship row generated for this attribute name should say that the parent attribute, in this case the element to which that attributes apply, “Defines” the named attribute.

The “A_title: EBNF.AttDef” data tells the parser that the attribute to be produced for this new parsing level should have the data type of EBNF.AttDef.

The “A_ControlType: Start New Parsing Level & Data Type” creates a new relationship-attribute pair with the “A” side of the relationship being the attribute from the previous production level, and the “B” side of the relationship being the attribute to be produced by this and subsequent productions at this level.

The “B_Title: XML_attribute: data tells the parser to assign “XML_attribute” to the “type” field of the attribute to be created for this level.

The “B_ControlType: Text to Attribute Code at Current Level” data tells the parser to put the text from the incoming DTD returned by the “Name – Entity Name” production as the “code” field of the attribute to be produced for this level.


58


August 23, 2007

Figure 3-4. Parsing Actions for Matching the Name in an Attribute Definition

Once all the data that represents an XML DTD is set up properly, one simply points the JADB data-driven parsing function at an XML DTD and new attributes and relationships are added to the database that accurately and completely represent the subject DTD. Furthermore, the JADB data-driven parser can use the data entered about the DTD to parse XML documents based on that particular DTD. If, as a separate effort, the DTD is mapped to another data format, then a data-driven translator should be able to be created that will convert from the incoming XML document to the other format.

In summary, this approach allows the representation of any kind of data structure or format in a single database and links any part of any structure to any part of any other structure. It allows standards to be encoded so that they can actually do what they were intended to do, for example having the XML Language specification actually parse XML DTDs and Documents.


59


August 23, 2007

Appendix A

Terms of Reference

EDITORS NOTE:“Terms of reference portion”, Appendix A, is under development


60


August 23, 2007



61


August 23, 2007

Appendix B

Related Standards Information

Related Standards Information

The GWG Metadata Focus Group (GWG – MFG) is committed to using voluntary consensus standards, such as those developed through ISO and ANSI, consistent with OMB Circular A-119. The GWG – MFG also tracks other standards activities that may not be considered as voluntary consensus standards.

Metadata Standards, an Overview

There are many metadata standards, developed by different organizations for different purposes. The purpose of this Overview is to present a list of metadata standards and specifications that are directly involved with geospatial metadata.

The standards are categorized and presented in a manner intended to explain and familiarize the reader with those standards, to facilitate understanding and eventual implementation of those standards.


62


August 23, 2007



63


August 23, 2007

APPENDIX C

Legislation and Guidance on Standards

Note: This appendix may not be totally complete. If there are additional documents and/or organizations that should be included or deleted, please specifically submit comments appropriately using the comment form at the end of this document.

Public Law 104 – 113Sec. 12) Directs NIST to: (1) coordinate Federal, State, and local technical standards and conformity assessment activities; and (2) report to the Congress regarding implementation within 90 days of enactment of this Act. Directs Federal agencies and departments to use technical standards developed by voluntary consensus bodies, unless impractical or inconsistent with applicable law. (Requires an explanatory report to the Office of Management and Budget.)http://thomas.loc.gov/cgi-bin/bdquerytr/z?d104:HR02196:|TOM:/bss/d104query.html|

OMB Circular A-16 – Coordination of Surveying, Mapping, and Related Spatial Data Activities

This revised Circular August 2002) describes the responsibilities of Federal agencies with respect to coordination of those Federal surveying, mapping, and related spatial data activities described in paragraph 2, below. (Spatial data are geographically referenced features that are described by geographic positions and attributes in an analog and/or computer-readable (digital) form.) It rescinds and replaces Circular No. A-l6 dated 1990 which replaced the May 6, l967 version. The revised Exhibits will remain in effect until replaced pursuant to paragraph 4 of this Circular. http://www.whitehouse.gov/omb/circulars/a016/a016.html

OMB Circular A-119 (Revised 1998), on Federal Participation in the Development and Use of Voluntary Consensus Standards and in Conformity Assessment Activities

Revised OMB CircularA-119 establishes policies on Federal use and development of voluntary consensus standards and on conformity assessment activities. Pub. L. 104-113, the “National Technology Transfer and Advancement Act of 1995,” codified existing policies in A-119, established reporting requirements, and authorized the National Institute of Standards and Technology to coordinate conformity assessment activities of the agencies. OMB is issuing this revision of the Circular in order to make the terminology of the Circular consistent with the National Technology Transfer and Advancement Act of 1995, to issue guidance to the agencies on making their reports to OMB, to direct the Secretary of Commerce to issue policy guidance for conformity assessment, and to make changes for clarity.http://www.whitehouse.gov/omb/circulars/a119/a119.html


64

http://www.whitehouse.gov/omb/circulars/a119/a119.html

http://www.whitehouse.gov/omb/circulars/a016/a016.html

http://thomas.loc.gov/cgi-bin/bdquerytr/z?d104:HR02196:%7CTOM:/bss/d104query.html%7C


August 23, 2007

Department of Defense (DoD)

DoD Directive 5105.60, National Imagery and Mapping Agency, 11 November 1996This Directive: (1) Establishes the NIMA within the Department of Defense consistent with the authorities and duties of the Secretary of Defense and the Director of Central Intelligence (DCI) under Title 10, United States Code; Title 50, United States Code; Executive Order 12333, December 4, 1981; and National Imagery and Mapping Agency Act of 1996. (2) Prescribes the NIMA’s mission, organization, responsibilities, functions, relationships, and authorities, pursuant to the authority vested in the Secretary of Defense by Section 113 and Chapters 8 and 22 of Title 10, United States Code, and in accordance with Title 10, United States Code; Title 50, United States Code; National Imagery and Mapping Agency Act of 1996.http://www.dtic.mil/whs/directives/corres/html/510560.htm

DoD Instruction 4120.24, Defense Standardization Program, 18 June 1998This Instruction implements a Defense Standardization Program (DSP) as required by Sections 2451 and 2452 of title 10, United States Code, assigns responsibilities, and prescribes procedures; Designates the Director, Defense Logistics Agency (DLA), as the DoD Executive Agent for the DSP; Authorizes continued publication of DoD 4120.24-M, “Defense Standardization Program (DSP) Policies and Procedures,” current edition, authorized by this Instruction, consistent with DoD 5025.1-M, “DoD Directives System Procedures,” August 1994, authorized by DoD Directive 5025.1, June 24, 1994, to provide guidance and procedures for the DSP. http://www.dtic.mil/whs/directives/corres/html/412024.htm

DoD 4120.3 – M, Defense Standardization Program Policies and Procedures, July 1993This Manual establishes the policies and procedures to achieve the standardization objectives required by Pub. L. No. 82-436 (1952), DoD Directive 5000.1, and DoD Instruction 5000.2.http://www.dtic.mil/whs/directives/corres/html/41203m.htm

DoD Instruction 4120.24, Defense Standardization Program, 18 June 1998

DoD Instruction 4630.8, Procedures for Compatibility, Interoperability, and Integration of Command, Control, Communications, and Intelligence (C#I) Systems, 18 November 1992

This Instruction assigns responsibilities, and prescribes procedures to achieve compatibility and Interoperability of a consolidated, DoD-wide, global C3I infrastructure.

http://www.dtic.mil/whs/directives/corres/html/46308.htm

National Imagery and Mapping Agency

NIMA PD 2000, Policy Directive for International and Foreign Affairs, 3 April 1998


65


http://www.dtic.mil/whs/directives/corres/html/41203m.htm




August 23, 2007

National Technology Transfer and Advancement Act (NTTAA) Toolkit for New Standards Executives

NGA Record Metadata Standard Electronic Records Management (DRAFT)

NGA Technical Standard and Guide Electronic Records Management (DRAFT)

NIST - National Institute of Standards and Technology

NIST developed the NTTAA Plan for Implementation in accordance with the provisions of Public Law 104-113. The completed plan was transmitted to Congress on June 7, 1996. It envisaged broad cooperation among governmental agencies and between government and the private sector in developing and implementing strategic standards and conformity assessment policies. As one might expect, the plan has evolved since this initial draft. Still, this document may serve as a primer in gaining a basic understanding of the basis for Congressional passage of the Act and NIST’s subsequent role in its implementation.http://ts.nist.gov/ts/htdocs/210/nttaa/toolkit.htm NISO – National Institute of Standards OrganizationThis organization is the representative to the development of numerous documents and standards often including the Federal Information Processing Standards (FIPS) – many FIPS have been withdrawn when there are equivalent voluntary consensus standards. http://www.niso.gov/

American National Standards Institute (ANSI)

National Standards Strategy of the United States http://www.ansi.org/Public/nss.html

NSSN - A National Resource for Global Standards – AKA the Standards Mallhttp://www.nssn.org/

International Committee for Information Technology Standards (INCITS) – ANSI-accredited standards development organization INCITS Technical Committee L1, Geographic Information Systems (password protected) – NGA, FGDC and various member agencies are represented on L1 INCITS, as NCITShttp://gsvaresa07.er.usgs.gov/QuickPlace/ncits_l1/Main.nsf (password protected)

Open Geospatial Consortium (OGC)Industry consortium aimed at growing interoperability for technologies involving spatial information and location. Many NSG members are principal members of the Open Geospatial Consortium (OGC). The OGC News features interesting articles on the relationship among standards activities.


66

http://gsvaresa07.er.usgs.gov/QuickPlace/ncits_l1/Main.nsf

http://www.nssn.org/

http://www.ansi.org/Public/nss.html

http://www.niso.gov/

http://ts.nist.gov/ts/htdocs/210/nttaa/toolkit.htm


August 23, 2007

http://www.opengis.org/ Object Management Group (OMG)A not-for-profit consortium that produces and maintains computer industry specifications for interoperable enterprise applications, including specifications for Unified Modeling Language (UML), the conceptual schema language used for ISO TC 211 standards. http://www.omg.org/

Joint Steering Group on Spatial Standardization and Related InteroperabilityThis group of representatives of Spatial Standards Setting Organizations (including ISO and OGC) share information in order to manage coordination activities better. http://www.jspspatial.org/

International Organization for Standardization (ISO) ISO Technical Committee (TC) 211, Geographic Information/Geomatics – ANSI accredits the U.S. Technical Advisory Group (TAG) to ISO TC 211http://www.isotc211.org/

Common Terrorism Information Sharing Standards (CTISS)

The standards group previously referred to as CISS (Common Information Sharing Standards) and IISS (Intelligence Information Sharing Standards) has been renamed to CTISS (Common Terrorism Information Sharing Standards).

1.6.1 There are six Common Terrorism Information Sharing Standards (CTISS) proposed by the Office of the DNI in response to Executive Orders 13356 and 13388 and pursuant to the Interim Implementation Plan for the Information Sharing Environment.

a. CTISS for Resource Metadata: Metadata Element Set – this is a superset of International Standard ISO 15836, The Dublin Core Metadata Element Set and defines logical elements in an implementation-neutral form.

b. CTISS for Resource Metadata: Application Profile for Discovery – adapted from the Defense Discovery Metadata Specification (DDMS). This adaptation incorporates the IC ISM approach to tagging security marking abbreviations. This standard is available for use when applicable, but it is not mandated, since its applicability within the IC is being investigated as part of the IC Enterprise Architecture activity.

c. CTISS for Information Security Markings: XML Implementation – adapted from the IC Metadata Standard for Information Security Markings, based on CAPCO. This standard will supplant the IC ISM, and is to be adopted by all applicable departments and agencies.


67

http://www.isotc211.org/

http://www.jspspatial.org/

http://www.omg.org/

http://www.opengis.org/


August 23, 2007

d. CTISS for Tearline Applications: Messaging Implementation – adapted from the Technical Standard for Automated Processing and Cross-Domain Dissemination of Tearlines. This standard deals with text (untagged) documents.

e. CTISS for Tearline Applications: XML Implementation – adapted from the model that approved by the ICML Panel in October 2004. This standard is for metadata-enabled (XML) documents.

f. Terrorist Watchlist Person Data Exchange Standard – adapted from the IC Terrorist Watchlist Person Data Exchange Standard (TWPDES)

g. IISS Global Justice Model - adapted from the Global Justice XML Data Model. This model is being promulgated by the DOJ's Office of Justice Programs throughout the law enforcement community down to the state and local levels. TWPDES data can be used to feed GJXDM and vice versa.


68


August 23, 2007



69


August 23, 2007

APPENDIX D

Emerging Standards

ISO 19101-2 Geographic Information – Reference Model – Part 2: ImageryTechnical specification that will define a reference model for standardization for geographic imagery and will be independent of any application development method or technology implementation approach.

ISO 19115-2 Geographic Information – Metadata – Part 2: Extensions for Imagery and Gridded DataThis standard is complimentary to and will extend ISO 19115:2003 Geographic information – Metadata by defining additional metadata elements to support the description of geographic imagery including gridded data. Together with ISO 19130 Geographic Imagery – Sensor and Data Models for Imagery and Gridded Data, it will support the documentation of the collection process and post-collection processing used in the creation, enhancement and maintenance of geographic imagery datasets. With ISO, these extensions will provide a comprehensive model for the specification of metadata appropriate for cataloguing, discovery, determining suitability for use and exploitation of geospatial imagery.

ISO 19123 Geographic Information – Schema for Coverage Geometry FunctionsWill define a standard conceptual schema for describing the spatial and geometric characteristics of coverage’s to increase the ability of geographic information to be shared among applications. ISO 19130 Geographic Imagery – Sensor and Data Models for Imagery and Gridded Data – (In work) Will specify the content of sensor models describing the physical and geometrical properties of each kind of remote sensing instrument that produces imagery.


70


August 23, 2007



71


August 23, 2007

A PPENDIX E

ISO 19115 – Geographic information - Metadata Normative and Informative References (those standards used in the development of the standard)

ISO 639 (all parts), Code for the representation of names of languages

ISO 646, Information technology – ISO 7-bit character set for information exchange

ISO 690:1996, Documentation — Bibliographic references — Content, form and structure

ISO 3166 (all parts), Codes for the representation of names of countries and their subdivisions

ISO 4217:1995, Codes for the representation of currencies and funds ISO 8859 (parts 1 to 15), Information technology ― 8 bit single byte coded

graphic character sets ISO 8601:1987, Data elements and interchange formats — Information

interchange — Representation of dates and times

ISO 8879, Information processing ― Text and office systems ― Standard Generalized Markup Language (SGML)

ISO/IEC 10646-1, Information technology ― Universal Multiple-Octet Coded Character Set (UCS) ― Part 1: Architecture and Basic Multilingual Plane

ISO/IEC 11179 (all parts), Information technology ― Specification and standardization of data elements

ISO 11180, Postal addressing

ISO/TS 19103: -Geographic information – Conceptual schema language

ISO/TS 19104: -Geographic information – Terminology

ISO 19106 – Geographic information – Profiles

ISO 19107 – Geographic information – Spatial schema

ISO 19108: 2002 – Geographic information – Temporal schema

ISO 19109 – Geographic information – Rules for application schema

ISO 19110 – Geographic information – Methodology for feature cataloguing


72


August 23, 2007

ISO 19111 – Geographic information – Spatial referencing by coordinates

ISO 19112 – Geographic information – Spatial referencing by geographic identifiers

ISO 19113: 2002 – Geographic information – Quality principles

ISO 19114 – Geographic information – Quality evaluation procedures

ISO/TS 19116: Geographic information – Positioning Services

ISO 19117 – Geographic information – Portrayal

ISO 19118 – Geographic information – Encoding

ISO/TS 19119: Geographic information – Services

ISO/TS 19123: Geographic information – Schema for coverage geometry and functions

ISO 23950:1998, Information and documentation — Information retrieval (Z39.50) — Application service definition and protocol specification

ISO/TR 19121: 2000:1998, Geographic information- Imagery and gridded data

IETF RFC 1738, Uniform Resource Locators (URL)

IETF RFC 2056, Uniform Resource Locators for Z39.50

Content Standard for Geospatial Metadata Federal Geographic Data Committee, June 8, 1994, Washington DC. This US Federal standard was initially used for the first version of ISO 19115 and the list reflects those standards referred to in the original development by the Federal Geographic Data Committee, ensuring compliancy with all.

American Congress on Surveying and Mapping and American Society of Civil Engineering, 1978, Definitions of surveying and associated terms: Falls Church, Virginia, American Congress on Surveying and Mapping.

American National Standards Institute, 1975, Representations of universal time, local time differentials, and United States time zone reference for information interchange (ANSI X3.51-1975): New York, American National Standards Institute.


73


August 23, 2007

American National Standards Institute, 1986, Representation for calendar date and ordinal date for information interchange (ANSI X3.30-1985): New York, American National Standards Institute.

American National Standards Institute, 1986, Representations of local time of day for information interchanges (ANSI X3.43-1986): New York, American National Standards Institute.

American National Standards Institute, 1986, Representations of Geographic Point Locations for Information Interchange (ANSI X3.61-1986): New York, American National Standards Institute.

American National Standards Institute, 1990, Dictionary for information systems (ANSI X3.172-1990): New York, American National Standards Institute.

Anglo-American Committee on Cataloguing of Cartographic Materials, 1982, Cartographic materials: A manual of interpretation for AACR2: Chicago, American Library Association.

ASTM Section D18.01.05, various dates, Spatial metadata Content Standard for geographic information systems, catalogs, and data exchange (drafts).

Clark, Suzanne, Larsgaard, Mary, and Teague, Cynthia, 1992, Cartographic citations: A style guide: Chicago, American Library Association, Map and Geography Roundtable.

Cogan, Christopher, and Edwards, Thomas, Jr., 1994 (February), Metadata standards for Gap analysis: Moscow, Idaho, Idaho Cooperative Fish and Wildlife Research Unit, University of Idaho.

Department of Commerce, 1975, Tide and Current Glossary, Special Publication 228: Washington, Department of Commerce, National Oceanic and Atmospheric Administration, National Ocean Survey.

Department of Commerce, 1989 (January), State Plane Coordinate System of 1983 (National Oceanic and Atmospheric Administration Manual NOS NGS 5): Silver Spring, Maryland, Department of Commerce, National Oceanic and Atmospheric Administration, National Ocean Service, Coast and Geodetic Survey.

Department of Commerce, 1992, Spatial Data Transfer Standard (SDTS) (Federal Information Processing Standard 173): Washington, Department of Commerce, National Institute of Standards and Technology.


74


August 23, 2007

Department of Defense, 1981, Glossary of mapping, charting, and geodetic terms (4th ed.): Washington, Department of Defense, National Imagery and Mapping Agency.

Department of Defense, 1990, Military specification ARC Digitized Raster Graphics (ADRG) (MIL-A-89007): Philadelphia, Department of Defense, and Defense Printing Service Detachment Office.

Department of Defense, 1992, Vector Product Format (MIL-STD-600006): Philadelphia, Department of Defense, And Defense Printing Service Detachment Office.

Dodd, Susan, 1982, Cataloging machine-readable files. Chicago, American Library Association.

Hansen, Wallace, 1991, Suggestions to authors of the reports of the United States Geological Survey (7th ed.): Washington, U.S. Government Printing Office.

Li, Xia, and Crane, Nancy, 1993, Electronic style: A guide to citing electronic information: Westport, Connecticut, Meckler Publishing.

Network Development and MARC Standards Office, 1988, USMARC format for bibliographic data: Washington, Library of Congress, and Cataloging Distribution Service.

Network Development and MARC Standards Office, 1988, USMARC code list for realtors, sources, and description conventions: Washington, Library of Congress, and Cataloging Distribution Service.

(no author), 1994, The Government Information Locator Service (GILS): Report to the Information Infrastructure Task Force (May 2, 1994).

Patriaes, Karen, 1991 (April), National Library of Medicine recommended formats for bibliographic citations: Bethesda, Maryland, U.S. Department of Health and Human Services, Public Health Service, National Institutes of Health, National Library of Medicine.

Snyder, John, 1987, Map projections: A working manual (U.S. Geological Survey Professional Paper 1395): Washington, U.S. Government Printing Office.

Westbrook, J. H., and Grattidge, W., 1991, A glossary of terms relating to data, data capture, data manipulation, and data bases: CODATA Bulletin, v. 23, no. 1-2.


75


August 23, 2007

Other Metadata Standards consulted and made compliant with ISO 19115 – Geographic information – Metadata and the Federal Geographic Data Committee’s Content Standard for Geospatial Metadata:

ASTM Section D18.01.05, various dates, spatial metadata content Standards for geographic information systems, catalogs, and data exchange

Dublin Metadata Core and Z39.50, February 1998 •Spatial Data Transfer Standard (SDTS), FIPS 173, 1992

ANZLIC Working Group on Metadata: Core Metadata Elements Guidelines Draft 7, Australia and New Zealand Land Information Council, November 1995

Canadian Directory Information Describing Digital Geo-referenced Data Sets, Canadian General Standards Board, July 1994, Ottawa.

Standard for Geographic Information - Metadata, European Committee for Standardization (CEN), September 1996, Brussels.

Digital Geographic Information Exchange Standard (Digest), Version 1.2. Digital Geographic Information Working Group, January 1994,

IHO Transfer Standard for Digital Hydrographic Data, International Hydrographic Bureau, October 1995, Monaco

–Spatial Data Transfer Standard (SDTS), US Department of Commerce, August 1992, Gaithersburg, MD

Application Profile for the Government Information Locator Service (GILS), US Department of Commerce, December 1994, Gaithersburg, MD

Cartographic materials: A manual of interpretation for AACR2, Anglo-American Committee on Cataloguing of Cartographic materials, 1982, Chicago

USMARC Format for Bibliographic Data, US Library of Congress, 1988, Washington, DC


76


August 23, 2007



77


August 23, 2007

APPENDIX F

Department of Defense Information Technology Standards Registry (DISR) Definitions

The DISR provides the minimal set of rules governing the arrangement, interaction, and interdependence of system parts or elements, whose purpose is to ensure that a conformant system satisfies a specified set of requirements. It defines the service areas, interfaces, standards (DISR elements), and standards profiles applicable to all DoD systems. Use of the DISR is mandated for the development and acquisition of new or modified fielded IT and NSS systems throughout the Department of Defense. The DISR replaced the Joint Technical Architecture. Standards in the DoD IT Standards Registry (DISR) move through the standards lifecycle across a timeline from emerging to mandated to retired. These are defined as follows: Emerging may be implemented, but shall not be used in lieu of a mandated standard. An emerging standard is expected to be elevated to mandatory status within three years. Use of an emerging standard in a TV-1 requires a waiver and a Technology Insertion Risk Assessment. In general, emerging standards should be placed in the TV-2. Mandated standards provide interoperability and net-centric services across the DoD enterprise. They are the minimum set of essential standards for the acquisition of all DoD systems that produce, use, or exchange information and, when implemented, facilitate the flow of information in support of the warfighter. These standards are required for the management, development, and acquisition of new or improved systems throughout the DoD. Mandated Sunset standards have been identified and approved by ASD(NII) to be tagged for retirement. Usually a pre-defined event and date are associated with the standard the sunset tag. In addition, a replacement standard is frequently identified. An X in the “sunset” column in a standards profile identifies this sunset status. Retired standards should not be used in a new or upgraded system. All retired standards citations remain in the DoD IT Standards Registry (DISR). However, when selected for inclusion in a Technical Standards View (TV), a retired standard citation requires a waiver and a Technology Insertion Risk Assessment.


78


August 23, 2007



79


August 23, 2007

APPENDIX G

List of Acronyms

COI Community of interestCRS Coordinate Reference SystemCSMWG Community Sensor Model Working GroupDCGS Distributed Common Ground SystemDDMS Dept of Defense Discovery Metadata SpecificationDED Data Element DictionaryDISA Defense Information Systems AgencyDISR DoD IT Standards RegistryDoD Department of DefenseFINTEL Finished IntelligenceGEOINT Geospatial IntelligenceGIS Geospatial Information SystemGML Generalized Markup LanguageGWG Geospatial Intelligence Standards Working GroupHUMINT Human IntelligenceIC Intelligence CommunityIC DMC Intelligence Community Data Management CommitteeIC ISM Intelligence Community Metadata Standard for Information

Security MarkingsIC MSP Intelligence Community Metadata Standard for PublicationsIMINT Imagery IntelligenceIPL Image Product LibraryISO International Organization for StandardsITSC Information Technology Steering CommitteeJFCOM Joint Forces CommandJITC Joint Interoperability Test Command JSIC Joint Systems Integration CommandJTF Joint Forces CommandMASINT Measurement and Signatures IntelligenceMFG Metadata Focus GroupMGCP Multinational Geospatial Co-production ProgramNAC National System for Geospatial-Intelligence (NSG) Architecture

Compliance documentNCGIS National Center for Geospatial Intelligence StandardsNGA National Geospatial Intelligence AgencyNSG National System for Geospatial-IntelligenceQMSWG Quality Metadata Sub-Focus Working GroupSCOTS Standards-Based Commercial Off-the-Shelf SIGINT Signals IntelligenceTC Technical CommitteeXML eXtensible Markup Language


80


August 23, 2007

This page left intentionally blank


81


August 23, 2007

METADATA DESK SIDE REFERENCE GUIDE

Review information and comment forms

The following information is provided for your guidance and for your use as a reviewer/analyst of the Metadata Desktop Reference Guide. All comments are welcome and appreciated.

Please try to provide correct information if you disagree with what is being presented. If you believe information is incorrect, inadequate or insufficient, please provide the correct version. If you so not suggest replacement information, we may have to disregard the comment.

If you have suggestions for making the document more presentable, more usable, and/or a better reference guide, please make those suggestions as clearly as you can.

Please review the other comments that we have received as they may answer your own comment or question. If you agree with the other reviewers comment, please indicate “Agree” or Disagree” in the appropriate column.

Thank you all very much for your interest in this document. It is intended for your use to help in the understanding of “METADATA” and its value.

Please use this comment form for suggestions, comments, corrections, additions, and other helpful pieces of information. This will allow the review team to organize, compare, and generally maintain control of the comments.

Name of Person Completing Form:

Organization: Date: Standard Version and Date:

Contact information of reviewer:

Please return comments to: [email protected] or [email protected]


82




August 23, 2007

COMMENT INSTRUCTIONSGeospatial Metadata Desk Side Reference GuideGeospatial Metadata Desk Side Reference Guide

Instructions for Submitting Comments

When providing comments on the Geospatial Metadata Desk Side Reference GuideGeospatial Metadata Desk Side Reference Guide, please use the provided electronic spreadsheet and structure your comments as follows:

Paragraph/Subparagraph: The identification of the paragraph and subparagraph of the section the comment addresses

The paragraph and subparagraph number are located along the left margin of the document at the beginning of each section, in the format paragraph, subparagraph (for example, 1.4).

**If the sub-paragraph is not identifiable by numeric value, please get as specific as you can as to the location, i.e., Paragraph 3.4, 4th subparagraph, 3rd line.

Figure/Table/Line #: The identification of the figure, table, or line number of the section the comment addresses

Type of Comment: Comments fall in one of three categories:

General (G) comments that address overall issues Technical (T) comments that address the technical accuracy of sections or

sub-sections of the document Editorial (E) comments that are editorial in nature, for example improper

sentence structure or typographical errorsComment: the reviewer’s identification of the problem or question with the

documentThe comment should be as specific as possible and, if appropriate, include the text

in question. Comments will not be considered if they are overly general, for example, “I don’t like this section”.

Proposed Change: the reviewer’s proposed change to the documentThe proposed changes should be as specific as possible to ensure the question or

problem is adjudicated correctly. Comments will not be considered without a specific proposed change, including suggested wording.

Note: Type of Comment, Comment, and Proposed Change columns must include entries to be considered. Either the Paragraph/Subparagraph or Table/Figure/Line # column must contain entries.Note: Extensive comments may be transferred electronically as attachments to this document. References to attachments can be made in the Proposed Change column.


83

DRAFTUNCLASSIFIED

Geospatial Metadata Desk Side ReferenceAugust 23, 2007

COMMENT FORMGeospatial Metadata Desk Side ReferenceGeospatial Metadata Desk Side Reference

Comment Submission Form

Page numberParagraph/ Subparagraph

Sentence #

Name/Organization

Figure/ Table/ Line #

Type of Comment:S=SuggestedR = RecommendE=Editorial

Comment Proposed Change

DRAFT

UNCLASSIFIEDGeospatial Metadata Desk Side Reference

84

DRAFTUNCLASSIFIED

Geospatial Metadata Desk Side ReferenceAugust 23, 2007

THIS PAGE INTENTIONALY LEFT BLANK

DRAFTUNCLASSIFIED

Geospatial Metadata Desk Side Reference

85