North Carolina Geospatial Data Archiving Project (NCGDAP)JISC/NDIIPP Joint Digital Preservation Workshop May 2006Presented by: Rob Farrell, Steve Morris, and Jim TuttleIssueLocal government agency data resources usually lack metadata.

SolutionGather available metadata details from GIS inventories and capture technical and administrative details during the data transfer process.

Future ScenariosThe NC One Map metadata outreach program is expanding state-wide. The new RAMONA survey tool includes the rapid development of basic metadata elements online.Project Phases OverviewReview existing inventories of state, regional, and local agency data holdings

Target data for acquisition, focusing on GIS vector data, digital orthophotography, digital maps, and georeferenced tabular data

Determine data acquisition frequency for continuously updated data layers such as land parcels and municipal boundaries

Determine optimal data formats for preservation and develop plans for archiving unique spatial data formatsAcquire state, local, and regional agency data as well as federal agency products of local scope

Capture, process, and normalize existing FGDC metadata

Use the Metadata Encoding and Transfer Standard (METS) to form preservation metadata records that include descriptive, administrative, and technical components

Explore Open Geospatial Consortium specifications and technologies for potential use in automated identification and capture of dataWork within the NC One Map framework of inventories, data sharing agreements, and technical best practicesExplore state, local, and federal partnerships in developing strategies for digital preservation infrastructure developmentAdvisory Committee formed from elements of NC Geographic Information Coordinating Council subcommitteesData AcquisitionDigital Repository DevelopmentData Selection and IdentificationPartnership BuildingIngest data into DSpace open source digital repository software for bit-level preservation

Develop redundant archive, with offsite storage

Implement format migration strategy for at-risk data resources

Explore longer-term archival approaches with state, federal, and local partners

Explore options for exposure of time-versioned and archived content through access and discovery toolsNCGDAP Context: Long-Term Risks to Geospatial Data Media Failure or Obsolescence - old or unreadable tapes

Format Obsolescence - how long will formats be supported?

Absence of Metadata - insufficient documentation to support use of the data

Naming and Coding Inconsistency - data layer names, attributes, and coding schemes vary across agencies

Permanent Access Not Available - data distribution focuses on current copies

Shift Towards Web Services-based Distribution - threat to formation of secondary archives (libraries, etc.)The North Carolina State University Libraries and the North Carolina Center for Geographic Information and Analysis have partnered on a three-year project to collect and preserve at-risk digital geospatial data resources from state and local government agencies. This project draws on a cooperative agreement with the Library of Congress in connection with NDIIPP as one of eight Collaborative Collection Development Partnerships. Although this venture will focus solely on North Carolina, it is expected to serve as a demonstration project for other states. The project is closely tied to NC OneMap, a combined state, federal, and local initiative that is focused on facilitating seamless access to geospatial data across all levels of government. One of the stated goals of NC OneMap is that Historic and temporal data will be maintained and available."Project IntroductionMitigating Community Practice Risks 3 examplesProject Introduction, Context, and Phases OverviewMitigating Technological Risks 3 examplesIssueMost geospatial data sets are multi-file objects involving multiple formats.

SolutionGroup files as single logical item during ingest workflow. Use METS for distribution packaging.

Future ScenariosThe GeoDRM initiative may stimulate geospatial industry attention to the issue of content packaging.A Single Vector Layer=Layer Components

Geometry file Index file Attribute file Metadata file Coordinate System file Spatial Index filesSpatial DatabaseRaster DataVector DataTopologicalRelationshipsXML DatabaseExportTIFF ImagesFeature Class LayersPotential Ingest ObjectsIssueSpatial databases have become an increasingly common data management environment. These complex systems include components that are difficult to preserve.

SolutionArchive data layer exports, raster exports, XML representation of the database components, and the original spatial database.

Future ScenariosSpatial database technical directions being pursued by vendors may make it more feasible to preserve spatial databases in their entirety. IssueThere is no open, widely-supported format for geospatial vector data.

SolutionSettle for now on the shapefile as an openly documented, easily-archived, and widely supported format.

Future ScenariosInitial discussions regarding an archival profile of the Geography Markup Language (GML) are underway, involving the Open Geospatial Consortium, ESRI, and NARA. IssueMetadata may occur in html, xml, text, or pdf and may adhere to a broad content standard such as ISO 19115 or FGDC, an industry profile, or a locally defined scheme.

SolutionNormalize the format to an XML representation of a commonly utilized profile of the FGDC content standard and retain original metadata record.

Future ScenariosThe finalization and adoption of the North American Profile of the ISO 19139 metadata encoding standard will promote industry-wide consistency in metadata records. IssueMany geospatial data layers are volatile in nature, and may be updated daily or weekly.

SolutionCapture snapshots of data layers at defined intervals and encourage maintenance of snapshots by data custodians.

Future ScenariosPreservation partners will conduct a survey to determine optimal data capture frequency for each framework data layer. State Archives retention schedule processes may provide a means to formalize the capture process.Parcel delineation in northernRaleighMarch 1997Parcel delineation in northernRaleighFebruary 2005Metadata Encodings and FormatsResults from 2003 North Carolina local government survey