Content and Practice: Background to the NC Geospatial Data Archiving Project Steve Morris NCSU Libraries

Content and Practice:

Background to the NC Geospatial Data Archiving Project

Steve Morris NCSU Libraries

Note: Percentages based on the actual number of respondents to each question 2

Project Context

Partnership between university library (NCSU) and state agency (NCCGIA)Focus on state and local geospatial content in North Carolina (state demonstration)Addressing the NC OneMap objective “Historic and temporal data will be maintained and availableOne of eight projects in the first NDIIPP funding round: “Building a Network of Partners”


Targeted Content

Resource TypesGIS dataDigital orthophotography Digital mapsTabular data (e.g. assessment data)

Content ProducersMostly state, local, regional agenciesSome university, not-for-profit, commercialSelected local federal projects


Value of Content

National State Geographic Information Council estimate for NC: $162 million (2003)Local content also forms basis of many state/federal data resourcesValue over time:

Greatest demand for current dataIncreasing (yet still small) demand for older versions of dataOlder versions impossible to replace once lost


Large scale, current, attributed vector data


High resolution digital orthophotography


County Digital Orthophotography Specifics

Imagery Type

21.0%

76.0%

1.5% 1.5%

Color-InfraredTrue ColorBlack & WhiteNot Sure

Maintenance Frequency

3.0%

1.5%

10.6%

7.6%

1.6%

36.4%

2.0%3.0%

10.6%

1.7%

Daily Annually Every 2 Yrs. Every 3 Yrs.Every 4 Yrs. Every 5 Yrs. As Funds Allow OtherNot Sure Not Maintained

Data Meet LRMP Specifications?

80.0%

3.1%

16.9%

Yes

No

Not Sure

Source: NC OneMap Data Inventory 2004


Value as part of cultural heritage


Data Quantity

County Digital Orthophotos88 counties with, estimated 154 flights by 2006Estimated 30 gb/flight – 4.6 TB total

County, City, COG Vector DataVariable mix of layers; some continuous update92 of 100 counties with GIS systems51 municipalities with GIS systems

State Agency Data1993 and 1998 statewide orthos – 800 gbTerabytes of vector data and other imagery17-20 TB of LIDAR data


NC OneMap Initial Data Layers Produced by Cities and Counties

0%

20%

40%

60%

80%

Ortho Cadastral Roads Municipal Bnd.County Bnd. ETJs Surface Waters ElevationLand Use Airports Schools UniversitiesHospitals Storm Surge Police Stations Fire StationsLandfills Watersheds Wetlands Hazardous Disposal SitesBuilding Footprints Future Land Use Water Lines Sewer Lines



Content Organization (tiling, etc.)

State agency data Vector data: statewide, river basin, quarter quadrangles, countiesImagery: quarter quadrangles, local images

County dataVector data: county, tax map unitsOrthophotos: tax map units, county mosaicsIncreasing: spatial databases (SDE, PostGIS, etc.)

Municipal dataVector data: city, tax map units


Metadata

FGDC CGDSM implemented by major state GIS agencies starting in 1994NC CGIA Metadata Outreach: regional workshops, phone support, training materialsAdoption

Some adoption by county agencies (21 of 92 county GIS systems as of Spring 2004)Some adoption by municipal agencies and COGS (13 of 51 municipal GIS systems by Spring 2004)Rare adoption by private, university, NGO’s


Local Agency Geospatial Metadata

0%

10%

20%

30%

40%

50%

60%

70%

FGDC-CompliantInternally DefinedData DictionaryNone/ Not Sure



Versioning and Updating

OrthophotosCounty digital orthophotos reflown every 2-7 yearsStatewide digital orthophoto plan: every 5 years (alternating B&W and color infrared)

Vector DataState agency vector data: some static, some periodically updated, relatively fewer continuously updatedCounty/City/COG vector data: many data layers continuously or periodically updatedOld versions supplanted, exist on relatively inaccessible backups


Time series – vector dataParcel Boundary Changes 2001-2004, North Raleigh, NC


Content Use

Much sharing between agencies (all levels)Municipal agencies use county data (orthos, centerlines, parcels) as base layersState and federal agencies use local data to improve their own data (linework adoption/improvement or attribute conflation)Local agencies use LIDAR (control points, etc.)Problems with different data structures, naming, coding schemes (e.g. parcels)NC OneMap framework data standards in development


Local Applications Where GIS Is Used

0%

20%

40%

60%

80%

100%

Economic Development Emergency Management

GIS/ Mapping Planning/ Community Development

Police/ Public Safety Public Works

Utilities Water/ Waste Water



Knowledge Required for Use

Most data published in widely used formatsVector: shapefile, coverage, geodatabase, MIFImage: TIFF, MrSID, BIL

LIDAR data: range of early/middle/late stage products requiring different skillsSemantic issues: different naming schemes across counties; need data dictionaries and ancillary documentationEarly stage image products (e.g. pre-rectification orthos) require greater knowledge, but are not widely available


Associated Applications/Web Services

Largely compatible with predominate commercial software packagesException: several counties using Understanding Systems GIS softwareNC OneMap viewer system: Cascading map server based on drawing on local/state/federal WMS sourcesGeocoding function on the NC OneMap map viewer uses the commercial geocoding web service


GIS Software Used

0%

10%

20%

30%

40%

50%

60%

70%

ArcGIS (ESRI) ArcInfo (ESRI) ArcView 8.x (ESRI)

ArcView 3.x (ESRI) ArcIMS (ESRI) GenaMap

IMAGINE Intergraph MapInfo

Understanding Systems Other Not Sure










Data/Metadata Workflow: Community

DataOrthophoto work contracted out to commercial firmsSome vector data contracted out (notably parcels)Most other vector data produced in-houseEarly, middle, late, and late-late stage products

MetadataMetadata published by producer, with NC Metadata Outreach Program supportMetadata published to NC NSDI clearinghouse, Geospatial One-Stop, and NC OneMap


GIS Data Management

0%

10%

20%

30%

40%

50%

60%

70%

Centrally ManagedSplitNot Sure



County Street Centerline Specifics

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Street Centerline Attributes

Road Names Route NumbersRoad Type Road DescriptionRoad Direction Street AddressesAddress Ranges ZIP CodesNot Sure

0%

10%

20%

30%

40%

50%

60%

Source of Street Centerlines

Photo Interpretation Digitize from MapsCommercial Product GPSCadastral OtherNot Sure



County Cadastral Specifics

0%

10%20%

30%

40%

50%60%

70%

80%

Methods Used to Create Data

COGO Photo InterpretationDigitize from Maps GPSOther Not Sure

0%

20%

40%

60%

80%

100%

Cadastral Attributes

Owner Parcel AddressOwner Address Construction DateOwnership Type ZoningLand Use Deed/ Book/ PageAcreage Land Value# of Structures Building ValueAssessed Value None


County Surface Water Specifics

0%

5%

10%

15%

20%

25%

30%

Source of Data

CGIA 1:24kCGIA 1:100kUSGS 1:24k DLGUSGS 1:24k DRGPrivate ContractorDelineated from Locally-Produced OrthoElevation DataOtherNot Sure

0%

10%

20%

30%

40%

50%

60%

70%

Surface Water Attributes

Stream Name Stream ClassStream Order Stream TypeOther Not SureNone


Data/Metadata Workflow: Project

Data identified through a mix of eight different existing inventory processesNC OneMap Inventory starting from 2004Data acquired through a variety of means

Download, WAN, CD/DVD, external drives, tapesWill experiment with OGC technologies for identification and transfer of data

Ingest existing metadata and augmentNormalize existing FDGC and create minimal if noneWrap descriptive, adminstrative, technical metadata into METS records; links to services


Data/Metadata Workflow: Project

Ingest into Dspace open source digital repository software

Explore issues surrounding ingest of geospatial content into generic, widely available repository environmentsMaintain independence of preservation object from ingest object of a given repository environment

Explore re-ingest into a different repository environment – avoid “imprinting” on a single environmentExplore a mix of federal/state/local options for longer-term archiving


Is Content Being Archived Now?

Records retention requirements don’t necessarily ensure preservationA wide range in practice – in general custodian and consumer energies are focus on current dataWhere archiving is occurring “permanent access” is generally not supportedA variety of different state/regional/local approaches (focus of project site visits)


Integrating with NGDA?

Community: Would need to be relatively effortless and inexpensive

local GIS shops are small, strapped for staff timelong-term access is a low priorityneed automation, web services–based archive development (and GeoDRM?)Rights issues must be cleared; need to address concerns about disclaimer provision, liability, etc.

Project: Interest in exploring re-ingest into a separate repository environment


Intellectual Property Rights Issues

Subject to Public Records LawPublic record: no privacy issues …… but records for some individuals may be filteredDisclaimer viewing important (liability)Restrictions on commercial reuse – desire for downstream control of data

Great deal of variability in access/use policyTrust between agencies is important; interpersonal and interorganizational relationships a key part of the infrastructure


Obtaining Local GIS DataRestrict Access to Data?

29.6%

48.0%

18.4%

3.9%

Yes No Sometimes Not Sure

Restrict Redistribution of Data?

48.7%

35.5%

11.2%

4.6%


Charge for Data?

53.9%

28.9%

15.8%

1.3%




NC OneMap Data Sharing Agreements

NCCGIA working to clarify legal issues surrounding redistributionStriking MOAs with local agencies as part of NC OneMap framework for open access

One of stipulations: “…AGREE that the data shared under this agreement may be further redistributed with applicable metadata by either agency WITHOUT FEES in the public domain and without restriction, unless otherwise noted herein and/or unless otherwise subject to public laws of governing authorities …

As of 2004 MOAs distributed to 55 counties in draft form, 24 signing with option for redistribution


Transition Plan for Content

NCSU/CGIA NDIIPP ProjectProject timeline: 3 years, 2004-2007Commitment to retain data 5 years after project termination

State/Regional/Local AgenciesVariableNC OneMap and other initiatives evolving Role of State Archives?


Questions?

Contact:

Steve Morris

Head of Digital Library Initiatives

NCSU Libraries

[email protected]

Phone: (919) 515-1361

Documents

Content and Practice: Background to the NC Geospatial Data Archiving Project Steve Morris NCSU Libraries