Current Applications of the OAIS Model

David Giaretta

OutlineConformance – what does it meanExamples

NSSDCCDPPLotar

Conformance – what does it mean?Section 1.4: A conforming OAIS archive implementation shall:

Support the model of information described in section 2.2• But OAIS RM does not define or require

any particular method of implementation

Fulfill the responsibilities listed in section 3.1• OAIS RM does not specify mechanisms

Roadmap identifies areas suitable for OAIS-related standards

Negotiates and accepts Information Packages from information producersObtains sufficient control to ensure long-term preservationDetermines which communities (designated) need to be able to understand the preserved informationEnsures the information to be preserved is independently understandable to the Designated CommunitiesFollows documented policies and procedures which ensure the information is preserved against all reasonable contingenciesMakes the preserved information available to the Designated Communities in forms understandable to those communities

OAIS Responsibilities

NSSDC - DIOnAS Data migration project DIOnASUsing OAIS conceptsProducing AIP’sTransforming existing data to Canonical format from VMS operating system specific format

Maintaining metadata to specify VMS-specific features

AIP

Based on ISO 12175 (SFDU packaging)ADID is an internationally object-type-unique identifier

Points to additional metadataThis metadata may initially be very simple BUT can be enhanced later

SFDU concept

NSSDC - IMAGE satellite dataNSSDC provided software to IMAGE project to convert their “Level-0.5” data products into AIPs

Original data is tar files containing “Universal Data Format (UDF) files.

Consumers will be provided with the UDF files plus software to read these

This may change as UDF falls out of use

IMAGE satellite data

Negotiates and accepts Information Packages from information producersObtains sufficient control to ensure long-term preservationDetermines which communities (designated) need to be able to understand the preserved informationEnsures the information to be preserved is independently understandable to the Designated CommunitiesFollows documented policies and procedures which ensure the information is preserved against all reasonable contingenciesMakes the preserved information available to the Designated Communities in forms understandable to those communities

OAIS Responsibilities

CDPP CDPP created in 1998

about 10 missions (WIND, ARCAD, …), 30 experiments, 100 data sets

OAIS typeInformation objects and type of objectsDomain dictionary (has already been run in on many missions)Modeling of delivered data

Standards and tools (examples)Data description in EAST is required from the producers

Tools are provided to create these

CDPP

Standards Tools

Basic standard for number coding

CCSDS EAST (syntactic description)

CCSDS DEDSL (semantic description)

OASIS (syntactic and semantic description)

PVL (delivery files) PVL parser and generator

CDPP data deliveryExample of data delivery model

RepresentationInformation

Data Objects

PDISourceContextReference

EASTStructuresheet

Catalogs

Data EntityDictionary

DED File

Descriptive docs

DIF

Data files

Mission, experiment descriptive files

MissionExperiments

Data set (main, auxiliary data, images)

EASTThe EAST language was designed to meet three objectives in order to enable :

a complete description of any given data format, without any 'ephemeral' references to any short-life file management systems, access to the values of the data by means of generic tools which do not require writing of even a single line of specific code for the data to be read, formatting the data on their storage or exchange medium while guaranteeing a structure that matches their description.

The EAST language is an international standard (CCSDS and ISO).Tools based on this standard have been developed. They can be used to help players throughout the "data life cycle".

EASTCNES point of view : EAST success will be achieved by using free tools. The user by using the OASIS tool doesn ’t have to learn a new language.

package logical_PRODUCT_X_SUN_FCSTC000 istype PRODUCT_X_NAME is (PRODUCT_X) ;for PRODUCT_X_NAME'size use 72 ;type AGENCY_ID is (CNES, NASA, USER_ESA, BNSC, CCSDS) ;for AGENCY_ID'size use 8 ;type NUMBER_TYPE is range 0 .. 9999 ;for NUMBER_TYPE'size use 32 ;type PRODUCT_ID_TYPE is record NAME : PRODUCT_X_NAME ; PRODUCER : AGENCY_ID ; NUMBER : NUMBER_TYPE ;end record ;for PRODUCT_ID_TYPE use record NAME at 0 * WORD_16_BITS range 0 .. 71 ; PRODUCER at 4 * WORD_16_BITS range 8 .. 15 ; NUMBER at 5 * WORD_16_BITS range 0 .. 31 ;end record ;for PRODUCT_ID_TYPE'size use 112 ;

Some lines of EAST

Snapshot d ’écran OASIS

Data Entity Dictionary Specification

Language Design Goals of DEDSL

Able to carry the definitions of data elements for various purposes

• Establish a discipline/domain dictionary independent of any particular data product usage

• Establish a product level dictionary that can be specialized from a domain level dictionary or created independently

Support registration of dictionaries as whole entitiesProvides standard attributes for use in the description of scientific data productsAttribute set easily extendable by any Data Entity Dictionary creatorSupport both human understanding and automated validation and processingHarmonized with ISO 11179-3 data element efforts

DEDSL ConceptsOne Abstract syntax with multiple concrete syntaxes defined for different languages and environmentsThe Abstract Syntax does not specify the schema or physical design of a Data Entity Dictionary implementationThe concrete syntax uses the constructs of the underlying language to implement the abstract modelEntities are described by the values of a set of standard Attribute values which may be extended by specifying user defined attributesAttributes are defined by a set of Descriptor values A Data Entity Dictionary consists of:

1 set of Dictionary Entity attribute definition0 or more User-defined attribute definitions1 or more Data Entity Definitions

A Data Entity Dictionary may be transferred using any agreed DEDSL concrete syntax specification.

Dictionary AttributesProvide information that applies to the entire Data Entity

Dictionary:IDENTIFYING

*Dictionary_name

RELATIONALExternal_dictionary_reference

DEFINITIONALDictionary_definition

REPRESENTATIONAL*Text_field_character_setCase_sensitivity default=not_case_sensitive

*Language

ADMINISTRATIVE*DEDSL_versionDictionary_versionDictionary_identifier

*= mandatory

HTML pages are inserted into MS WORD documents

Input Output of OASIS

EAST

DATA

Syntaxic Description

INTERPRETER :Library for readingand validating data

DATA GENERATOR :Library for writing data

(generation of data for simulation)

DEDPVL

SemanticDescription

DEDXML

XML translator

XSL ProcessorDEDDTD HTML

Users documentation

Data Description

Main CDPP difficulties

Non-availability of the Producer, or inadequate available resources

Diversity of experiments and data producedLate intervention of the Archive in the data production processAbsence of standard archiving structuring format (in Plasma Physics)No imposed schedule

Negotiates and accepts Information Packages from information producersObtains sufficient control to ensure long-term preservationDetermines which communities (designated) need to be able to understand the preserved informationEnsures the information to be preserved is independently understandable to the Designated CommunitiesFollows documented policies and procedures which ensure the information is preserved against all reasonable contingenciesMakes the preserved information available to the Designated Communities in forms understandable to those communities

OAIS Responsibilities

LOTAR

Long Term Archiving and Retrieval and Product Data within the Aerospace Industry

AIRBUSMTUEADSFairchild DornierPROSTEP AGProSTEP iViP Association

Requirements

Very large number of 3-D CAD modelsVarious formats – software dependent

• “quasi-safe for about 10 years”

Not safe enough for Legal and product liability

Legal:Ability to verify the conformity of a part with its documentationSystem has to enable the user to assure the provisions of a law regarding data security and protection of data privacy over the life cycle of the archivesPossibility to audit the process of archiving and retrieval

SecurityEnsure the information is understandable by the Designated Community without the assistance of the information producers……

Approach

Use vendor neutral ISO 10303 (STEP)Use OAIS RM

Document examines all the functional entities in great detailRecognises that Designated Community may be broadened

Negotiates and accepts Information Packages from information producersObtains sufficient control to ensure long-term preservationDetermines which communities (designated) need to be able to understand the preserved informationEnsures the information to be preserved is independently understandable to the Designated CommunitiesFollows documented policies and procedures which ensure the information is preserved against all reasonable contingenciesMakes the preserved information available to the Designated Communities in forms understandable to those communities

OAIS Responsibilities