An Overview of Selected ISO Standards Applicable to Digital

Archives

Science Archives in the 21st Century

25 April 2007

Donald Sawyer - NASA/GSFC/NSSDC

Lou Reich - CSC (NASA/GSFC)

John Garrett - Raytheon (NASA/GSFC/NSSDC)

Overview• NSSDC Standards Participation: Historical Perspective

• Selected ISO standards for Archives– Reference Model of an Archive and its Information

– Checklist of Activities between Data Providers and Archives

– Packaging Data and Metadata with an XML Manifest

– Describing Data and Sending it to an Archive– Ensuring Archives can be Trusted to Preserve Information

• Wrap-up Views

NSSDC Standards Participation:

Historical Perspective• NSSDC formed NASA/Science Office of Standards and Technology (NOST) in 1988– Created and supported a formal NOST standards development process– NOST FITS Technical Committee formed by Astrophysics discipline– In 1999, NOST Definition of FITS: NOST 100-2.0

• NSSDC began participation in Consultative Committee for Space Data Systems (CCSDS) standards development in 1986– Data packaging standard (SFDU): ISO 12175

– Data description registration authority procedures: ISO 13764– Data description packaging for registration: ISO 15395– PVL data tagging language: ISO 14961– Data description language EAST specification: ISO 15889– Data entity dictionary specification: ISO 21961, 21962, 22643

• NSSDC began formally leading Space Physics Archive Search and Extract (SPASE) development for Heliophysics in 2003– Version 1.1.0 is available for data producers at

http://www.spase-group.org/data/doc/spase-1_1_0.pdf

Reference Model of an Archive

• Reference Model for an Open Archival Information System (OAIS): ISO 14721– Standardizes terms, concepts, and basic responsibilities

– Addresses functions, information models, migration, federation

– Basis for this workshop’s Framework document– Extremely widely adopted

• Recognized and used throughout virtually all national archives and digital libraries, and many science data repositories, commercial organizations

• NSSDC uses it for internal and external communication

NSSDC Uses OAIS Concepts

Archive Information Packages Provenance InformationArchival Information Collections Context InformationArchive Information Units Reference InformationSubmission Information Packages Fixity InformationContent Information Designated CommunityRepresentation Information

Checklist of Activities between Data Providers and

Archives•Producer-Archive Interface Methodology Abstract Standard (PAIMAS): ISO 20652

Data ready to archive

Preliminary Agreement

DictionaryFormal modelSubmission Agreement

Transferred object files

Validation agreement

Ph

ase

obje

ctiv

e

Preliminary Phase

Formal Definition Phase

Transfer Phase

ValidationPhase

Anomalies

Validate

the

transfe

rred

objects

Define t

he inform

ation

to be a

rchived

Develo

p agre

emen

t (dat

a to

be deli

vere

d, com

plemen

tary

elem

ents,

sched

ule)

Actual t

ransfe

r of t

he

objec

ts in

SIPs

Methodology: Preliminary Phase Sub-

phases and ActionsFirst contact

Preliminary definition,feasibility and assessment

Establishment of apreliminary agreement

Information to be archived, Quantification, Legal andcontractual aspects, permanent impact on the Archive,Summary of costs, etc.

Id Preliminary phase: quantification Involves

P-19 Estimate the data volume to be transmitted to the Archive Producer

P-20 Assess the permanent data volume to store Archive

P-21 Assess the storage capability need for the ingest process Archive

P-22 Assess the associated costs Archive

Action table

Description

Packaging Data and Metadata with an XML

Manifest• XML Formatted Data Unit (XFDU): CCSDS 661.0-R-1– Undergoing formal Space Agency review– ISO review to follow– Supports single file implementation (e.g., zip)

• Contains or points to other files• Supports OAIS or custom metadata categorization, mime types, checksums, and transformations such as compression and encryption.

• Validation API supports a range of structural and semantic validation capabilities

• Prototype java software library available from NASA – Prototype C++ API available from ESA

• Current scalability testing to 50 Gbytes

XDFU Environment View

XFDU Manifest Logical View

Describing Data and Sending it to an

Archive• Producer-Archive Interface Specification (PAIS): CCSDS/ISO Data Archive Ingest WG draft

– Formal Space Agency review late summer– ISO review 2008– Standardized and extensible attributes to describe data objects prior to transfer to archive

– Maps data and PAIS information to XFDU packaging standard for transfer•Will leverage and extend XFDU software library

PAIS Process Flow

PAIS Formal Descriptions

Ensuring Archives can be Trusted to Preserve

Information• Digital Repository Audit and Certification: CCSDS/ISO BOF working group– Looking to become chartered CCSDS/ISO Working Group

– Build on work completed in other organizations, particularly:• Research Libraries Group/National Archives and Records Administration’s “Trustworthy Repositories Audit & Certification: Criteria and Checklist (TRAC)”

• Network of Expertise in long-term STORage Working Group’s “Catalogue of Criteria for Trusted Digital Repositories”

– Subject of subsequent presentation at this workshop

Wrap-up Views• Difficult to pre-judge the success of any particular standards effort– May take several years to ‘catch on’ or be superceded• However, OAIS reference model ‘caught on’ in draft stage - large need and good timing!

– Significant educational component associated with any standards effort• Another form of research

• Contact presenter for more information– Donald.Sawyer@nasa.gov