NCPMA Fall Meeting October 11, 2006

GIS Data Preservation:Partnership with Library of Congress Steve MorrisNorth Carolina State University Libraries

2

Today’s geospatial data as tomorrow’s cultural heritage

Future uses of data are difficult to anticipate (as with Sanborn Maps).

3

Temporal Data Supports Decision Making

Land use change analysisReal Estate trend analysisSite selection (past uses?)

Forecasting

Parcel Boundary Changes 2001-2004

North Raleigh, NC

4

Time series – Ortho imageryVicinity of Raleigh-Durham International Airport 1993-2002

5

Digital Preservation Points of Failure

Data is not saved, or …can’t be found, or …media is obsolete, or …media is corrupt, or …format is obsolete, or …file is corrupt, or …

meaning is lost

Solutions:

MigrationEmulationEncapsulationXML

6

Risks to Geospatial Data

Producer focus on current dataData overwrite as common practice

Future support of data formats in question

No open, supported format for vector data

Shift to web services-based accessData becoming more ephemeral

Inadequate or nonexistent metadataImpedes discovery and use

Increasing use of spatial databases for data management

The whole is greater than the sum of the parts

7

How would you describe your current geospatial archive?

Last week’s set of nightly tape backups

Several boxes of CD’s and DVD’s

Bob’s hard drive

A collection of files in our “GIS Folder”

A stand-alone spatial database

The data back-end for our internet mapping application

An enterprise GIS

8

NC Geospatial Data Archiving Project (NCGDAP)

Partnership between university library (NCSU) and state agency (NCCGIA), with Library of Congress under the National Digital Information Infrastructure and Preservation Program (NDIIPP)One of 8 initial NDIIPP partnershipsFocus on state and local geospatial content in North Carolina (state demonstration)Tied to NC OneMap initiative, which provides for seamless access to data, metadata, and inventoriesObjective: engage existing state/federal geospatial data infrastructures in preservation

Serve as catalyst for discussion within industry

9

10

Geospatial data types: Aerial imagery

85+ NC counties with orthophotos1-5 flights per county30-300 gb per flight

11

Geospatial data types: Vector & tabular

Economic, infrastructure, and ethnographic data

12

Geospatial data types: Cartographic project files

Counterpart to the map is not just the dataset but also models, symbolization, classification, annotation, etc.

13

Project Technical Approaches

Receive data as is – variety of distribution methodsMigration of some at-risk formatsMetadata remediation, standardization, and synchronizationDistilling complex objects into repository ingest items (not easy)Build a digital repository (catalyst for discussion)Develop a repository ingest workflow (learning experience)

Some unsustainable activities – for learning experience

14

Project Cultural/Organizational Approaches

Engage data producer community and spatial data infrastructure through outreach and engagement; influence practiceSell the problem to software vendors and standards developmentFind overlap with more compelling business problems: disaster preparedness, business continuity, road building, etc.Start a discussion about roles at the local, state, and federal level

Current use and data sharing requirements – not archiving needs – drive improved preservability of content and improvement of metadata

15

Challenge: Coordinated Content Transfer

How to allow one data snapshot to be accessible by multiple agencies – more compelling use cases than preservation can put the data in motion (business continuity, disaster preparedness, etc.)Other activities? (DHS, WGRT, State Archives, Census, etc.)Question: Capture frequency of data snapshot?

Survey just completed to identify local government best practices, consumer agencies needs

16

NC Frequency of Capture Survey

Survey objective:Document current practices for obtaining archival snapshots of county/municipal geospatial vector data layersSeek guidance about frequency of capture

Survey topics:General questions about data archiving practiceSpecific questions about parcels, street centerlines, jurisdictional boundaries, and zoning

Survey subjects:All 100 counties and 25 municipalities58% response rateSurvey conducted September 2006

17

Survey Results: Overview

Two-thirds of responding agencies create and retain periodic snapshotsLong-term retention more common in counties with larger populationsStorage environments vary, with servers and CD-ROMs most commonOffsite storage (or both onsite and offsite) is used by nearly half of the respondentsPopularity of historic images has resulted in scanning and geo-referencing of hardcopy aerial photos among one-third of the respondents

18

Local Business Rules and Uses Driving Temporal Snapshotting

Information technology policy (20%)Records retention policy (18%)Tax administration rules (25%)Land use change analysis (11%)Resolution of legal issues (18%)Historic mapping (56%)Other (30%)

19

Frequency of Capture: Parcel Data

42%

9%9%

14%

13%

13% Annually

Every 6 Months

Quarterly

Monthly

Weekly or Daily

Not Saved

20

Parcel Data Archival Data Format*

Shapefile (76%)Geodatabase (36%)Arc Coverage (29%)Arc Interchange (7%)Other (10%)

* Respondents were allowed to select multiple formats

52% of respondents indicated that a format conversion was carried out in creating the archival snapshot

21

Parcel Attribute Data Handling

61%23%

16%Attributes Attachedto Vector Data

Attributes Stored inSeparate Table

Attributes NotStored

22

Digital Conversion of Hardcopy Resources

Historic hardcopy maps:Scanned only (15.5%)Scanned and georeferenced (9.9%)

Aerial photos:Scanned only (8.5%)Scanned and georeferenced (26.8%)

None (54.9%)

23

Questions?

Contact:

Steve MorrisHead, Digital Library InitiativesNCSU LibrariesSteven_Morris@ncsu.edu

Web site: http://www.lib.ncsu.edu/ncgdap/