Tim Loesch Minnesota Department of Natural Resources€¦ · Tim Loesch Minnesota Department of...

Tim Loesch

Minnesota Department of Natural Resources

Agenda

Status of State Project

Where to get data

What types of data are available

How to download and retrieve information

What to do after you download the data

Examples of LiDAR Use

• Remote sensing technology similar to

radar, but uses laser light instead of

radio waves

• Combined with GPS information a

point cloud, a collection of X,Y,Z

points (millions per square mile!)

• Precision and number of points

depends on collect specifications

• Scanners can be ground-based,

airborne, or spaceborne

LiDAR Data in Minnesota

Available for large areas

52,000 square miles

in the Public Domain

High quality

High degree of consistency

Acquisition Initiatives

Red River Valley – International Water Institute

Minnesota High Resolution Elevation Mapping Project

Several County Collects

Blue Earth, Chisago, Crow Wing, McLeod, Pine, Rice, Stearns, Wright

Carver, Dakota, Goodhue, Ramsey, Scott

Older data, proprietary

Re-acquired as part of MN HREM

LiDAR was used in the Wild Rice Watershed (Red River Valley) to

accurately locate, size and align the structure, to estimate ring dike material

quantities and total costs, and to meet permitting/mitigation requirements.

DESIGNING A RING DIKE

DESIGNING A RING DIKE

The result during the 2009 spring floods

Minnesota Elevation Mapping Project

Started with the USGS LiDAR Base Specifications V13

Draft Published February 22, 2010

Specifies minimum specifications for Federal and Federally funded LiDAR projects

Collection

Data Processing and Handling

Hydro-Flattening Requirements

Deliverables

2/21/2012

Data Deliverables

All data are delivered in tiles based on 1/16th of a USGS 1:24,000 scale quadrangle

Roughly 3.25 square miles

DNR_QQQ_ID – Tile name field

A good balance between file size and area

Average LAS file size = 291 mb

Average GDB file size = 5.6 mb

2/21/2012

Tile scheme

16 tiles per 24K quadrangle

Data must fill tile – no partials

2/21/2012

File Geodatabase Contents

Contour_Data – Feature Dataset Contours – 2’ contours (3D Polylines)

Terrain_Data – Feature Dataset Bare_Earth_Points – bare earth points from LAS files (3D

Multipoint)

Hydro_Breaklines – Edge of water breaklines (3d Polygons)

DEM01 – one meter DEM

Buildings – Building outlines generated from LAS

• Products from the state collects include:

• LAS (Open standard format), classified

• Bare Earth points

• Rough building footprints

• Hydro Breaklines

• 1-meter DEM

• 2’ contours

• 3-meter DEM

• hillshades

Hydro Breaklines

• Building footprints

• Not required by state collect contract to be perfect

• Gives relative proximity and quantity

• Sometimes tall vegetation in flat areas is

erroneously depicted as building

In ArcCatalog

LAS Files One per tile Full Point Cloud

All returns all points

Classifications 1 - Processed but unclassified 2 - Bare earth ground 4 – Vegetation 6 - Buildings 7 - Noise (low or high) 8 – Mass Points 9 - Water 10 – Breakline Proximity 14 – Bridge Decks

2/21/2012

All Points

Bare Earth Ground

Vegetation

Buildings

Mass Points

Water

What about existing data?

Projected to UTM Zone 15

Vertical Units converted to meters

Re-Tiled to 1/16 tile

Reproject 3d shapefiles

Scale 3D shapefile from feet to meters

Recreate DEMs, Contours

Published on ftp site

County Data Bundles

Data have been assembled into county based bundles

Mosaic all tiles that touch county

Ease of use for county and other staff

Assists with QA/QC

Additional derived products

ArcGIS 9.3 File Geodatabases

County Bundle File Geodatabase Contents

ELEVATION_DATA.GDB • Buildings – Building polygons • Contours – Two foot contours • DEM01 – One meter DEM Mosaic • DEM01HS – Hillshade of DEM01 • DEM03 – Three meter DEM (rescaled DEM01) • DEM03HS – Hillshade of DEM03 • DEM03Slope_degree – Slope of DEM03 in degrees • DEM03Slope_percent – Slope of DEM03 in percent • Hydro_Breaklines – Edge of water breaklines • qaqc – table of tiles and statistics for data • Tile_index –index of tiles for county • Validation_points – survey shots used for QA/QC

In ArcCatalog

Data dissemination is a challenge

Data volume is high

26,000 square miles = 1 tb of data

Variety of data representations

Contours, DEMs, LAS, etc

Plethora of software formats

CAD, GIS, Graphics

Projections….?

Lots of questions…

County Area (sq

miles)

Mosaic LAS Geodatabase Total

Douglas 720 9.17gb 56gb 21.5gb 86.67

Faribault 721 5.21gb 76.5gb 16.5gb 98.21

Jackson 719 11.6gb 75.8gb 18.2gb 105.6

LeSueur 473 3.87gb 55.6gb 10gb 69.47

Martin 729 7.5gb 91.5gb 17.3gb 116.3

Nobles 722 9.2gb 71.3gb 20gb 100.5

Pope 717 9.4gb 70gb 22.8 102.2

Rock 483 6.67gb 51.5gb 12.5gb 70.67

Waseca 432 3.27gb 41.5gb 7.91gb 52.68

LiDAR allowed staff

to quickly assess the

entire project area

and select

preliminary sites.

On-site surveying

determined which

sites were optimum.

Once selected, staff

compared cut-and-

fill estimates using

the computer model

and on-site

surveying (the

figures were

remarkably similar,

within 3-7 percent).

Where to get Data DNR ftp access

Server name - lidar.dnr.state.mn.us

Pros Data is available

Data is highly structured and consistent

Can download vast quantities of data

Cons Not entirely intuitive

Must use ftp Client

Must learn storage structures

Not entirely fast

FTP Site contents

\data – all LiDAR related data found here

\documentation – documentation and instructions for site navigation

\examples –small lidar example datasets

\tools – tools for using LiDAR and Elevation data

\training – training materials

\readme_first.rtf – site documentation in text format

\watchme_first.swf – site documentation in video format

Mosaiced Data Packets are found at:

\data\county\<COUNTY_NAME>

Elevation_data.gdb

Tiled Data Packets are found at:

\data\raw\<COUNTY_NAME>

Geodatabase Folder

LAZ Folder

Accessing LAS Data

LAS files are stored as compressed LAZ files

5142-25-15.LAS -> 5142-25-15.LAZ

Lossless Compression averages 90%

This is a BIG deal

Uses LASTOOLS LASZIP.EXE

Command Line tool

Compression and Uncompression

LASTOOLS

An easy to use, lightweight and efficient set of tools for processing LiDAR data in LAS format

http://www.cs.unc.edu/~isenburg/lastools/

Not a free set of tools but some parts ARE free

License information can be found at:

http://www.cs.unc.edu/~isenburg/lastools/download/LICENSE.txt

LASZIP compression software IS Free as is LAS2LAS

Scenario –

YOU – are a technician who has to obtain LAS data for a Project

Project is in LeSueur County at the intersection of State Highways 99 and 112

Convert data to ASCII X,Y,Z format for input into CAD

How do I get there?

Using an FTP Client 1) Log into lidar.dnr.state.mn.us

1) Anonymous username and password

2) Change folders to: 1) \data\raw\county\leseuer

3) Download and View TILE_INDEX_MAP.PDF

4) Identify Tiles to Download

5) Change folders to 1) \data\raw\county\leseuer\laz

2) Find and download tiles of interest

3) Unzip to LAS Format

Need Tile 4342-20-08

Download LAZ via FTP

Unzip using LASZIP tool

LASTOOL Capabilities

LAS2TXT to convert LAS files to X,Y,Z ASCII format

LAS2LAS to re-project data

Clip Data to circles, squares and rectangles

Filter data based on Classifications

Class 2 (bare earth) is often too much information

Class 8 (Mass Point) is a subset of Class 2

Can find tips for LASTOOLS at: \tools\lastools\LAS_file_processing_using LASTOOLS.pdf

Export to CAD as ASCII x,y,z file

Other CAD Options

AutoCAD Civil3D can now read LAS files

Classes 2, 8 make up bare earth

Some users prefer to use Raster data to build surfaces

Don’t use Contours!

Geodatabases – if you are working with ArcGIS

Once you’ve found the Tiles you need…

Download from the county GEODATABASE folder

4342-20-08.gdb

Ready to use in ArcGIS

If you want County Mosaic Data

Go to the county folder of interest

\data\county\lesueur

Download entire folder

ELEVATION_DATA.GDB

Validation Report

Ready to use in ArcGIS

Future Plans Simplify downloads using browser based front

Provide options for: Project data

Output formats (CAD, GIS)

Output representations (contours, TIN, DEM, Points)

Subset data (clipping and filtering)

Expect availability in 2012

More Web Services Check out:

http://arcgis.dnr.state.mn.us/ArcGIS/rest/services

Future Plans

Web and Smartphone LiDAR Viewer

DEM, Contours, Terrain, Slope, Air Photos, Base maps

User Defined Profile and Point Elevations

PDF output

A great example….

International Water Institute LiDAR Portal

http://gis.rrbdin.org/lidardownload/index.html

Pros

Visual access via browser

Intuitive interface that is easy to use

Select and download

Six representations

E-mail instructions for downloading

Training Opportunities Conservation Applications of LiDAR

University of MN, DNR, BWSR

Funded by LCCMR

Nominal fee <$40

Six Class Series Basics of Using LiDAR Data

Terrain Analysis

Hydrologic Applications

Engineering Applications

Forestry and Ecological Applications

Wetland Mapping

Training Opportunities

http://wrc.umn.edu/randpe/agandwq/tsp/lidar/

Or search on Conservation Applications of LiDAR

Locations

Winona – March 27, 28, 29

East Grand Forks – April 10, 11, 12, 13

Morris – July 25, 26; August 8

Cloquet – April 26, 27

Minneapolis, May 15

More sessions are being planned

http://www.mngeo.state.mn.us/committee/elevation/mn_elev_mapping.html