UAS to Accuracies…how good are they...

Mike Zoltek, PLS, CP, CFedS, GISPSenior Project Manager

Mike.Zoltek@Woolpert.com

UAS to Accuracies…how good are they really?

Portsmouth Bypass• Woolpert was contracted to

provide aerial mapping for the ~16-linear-mile Portsmouth Bypass, which is currently being built as part of a $429 million construction project.

• Create a viable and repeatable solution for construction monitoring

Portsmouth Bypass Corridor – Client needs

• Provide data to support the payment of contractor fees based upon cut-fill quantities

• Obtain aerial imagery at 2 cm GSD to provide a cohesive snapshot of site conditions (imagery captured in a single day!)

Portsmouth Bypass Corridor - Deliverables

• LandXML file of 1-foot contours

• 2 cm imagery

Portsmouth Bypass Corridor

Woolpert proposed solution…

Portsmouth Bypass CorridorCollect 2-centimeter GSD aerial imagery...with Woolpert’s Renaissance Surrogate UAS capture system…

Portsmouth Bypass Corridor

Utilize 2 cm GSD imagery to create an auto-correlated point cloud

Portsmouth Bypass CorridorCreate a DSM surface from point cloud

UAS

Portsmouth Bypass CorridorExport point cloud a LAS file of entire project corridor.UAS

Portsmouth Bypass Corridor

Use traditional Lidar processing workflow to create desired deliverable

UAS

RENAISSANCETM What is it?• Combines the benefits of manned

capture and UAS workflows

• Driven by Affordability• Consumer Grade Camera• Single-Engine Aircraft• Pilot-Only Operation• Fly places where UAS can’t (FAA rules)• Computer-Vision Processing (Semi-

automated)

RENAISSANCETM What is it?

Biggest Benefits• Bypass all FAA UAS

restrictions!• Extended flight times

• Self-contained “POD”• Mounted to bottom

of a Cessna 172

Portsmouth Bypass Corridor

Workflow Details

Portsmouth Bypass Corridor

Step 1 -• Define project limits (Cad

files provided by client)• Note: Local coordinate

system 70 foot shift from state plane!

UAS

Portsmouth Bypass Corridor

Step 2 -• Create flight plan to collect

2 cm GSD imagery in project coordinate system

• Export flight plan and shift back to “real world” coordinate system

• Plane navigates with GPS, not the contractor coordinate system!

UAS

Portsmouth Bypass Corridor

Step 3 -• Plan Ground Control Points

required to support accuracy requirements for project (0.20 foot vertical RMSE)

• Plan & Provide data to contractor’s surveyors in project coordinate systems

UAS

Portsmouth Bypass Corridor

Step 4 -• Coordinate with contractor’s surveyor to set targets prior to flight• Make sure they are maintained and ready for flightStep 5 -• Capture the imageryStep 6 -• Process the data in Pix4D & create deliverables

UAS

Portsmouth Bypass Corridor

• Processing in Pix4D…

UAS

Pix4D Processing• Analytical Aerial Triangulation (AT)UAS

Pix4D Processing• Comparison of initial and final adjusted camera positions

UAS

Pix4D Processing• Number of overlapping images covering AOI

• 4+ needed for best solution

UAS

Pix4D ProcessingUAS• Digital Surface Model creation to support ortho-rectification

Pix4D Processing• AT Results – North Block summary

• Observations & Camera calibration results

>13 million 2D Observations

~4 million 3D Observations

Portsmouth Bypass Corridor

What worked…• Flight Planning• Target coordination with the contractor• Imagery Acquisition• Creation of Auto correlated point cloud (LAS files)

UAS

Portsmouth Bypass Corridor• What didn’t work…• Creation of Imagery tiles

• 2 cm tile @ 5,000 x 5,000 pixel tiles (Pix4D limitation)• Equates to 100 meter x 100 meter tiles…that is a lot of tiles!

Portsmouth Bypass Corridor• What didn’t work…

• SOLUTION – decimate imagery to 5cm pixel size• 6.25x file size savings!

Portsmouth Bypass Corridor• What didn’t work…Creation of Autocad LandXML database

• File size - could not be loaded in the field on a survey crew’s laptop

Portsmouth Bypass Corridor• What didn’t work…

• SOLUTION – provide 3 foot grid in ASCII xyz format• Still would not work, single file still too large!

Portsmouth Bypass Corridor

• What didn’t work…• SOLUTION – Break up ASCII files in to maximum

20 mb file size each!• Woolpert “script”

1825131.200 328536.466 542.174

1825134.186 328536.466 542.193

1825137.171 328536.466 542.702

1825125.229 328533.480 542.276

1825128.214 328533.480 541.945

1825131.200 328533.480 542.164

1825134.186 328533.480 541.954

1825137.171 328533.480 542.164

1825119.258 328530.495 541.363

1825122.243 328530.495 541.696

1825125.229 328530.495 541.861

1825128.214 328530.495 541.857

1825131.200 328530.495 541.741

1825134.186 328530.495 542.368

1825137.171 328530.495 542.249

1825140.157 328530.495 542.726

1825110.301 328527.509 540.745

1825113.287 328527.509 540.740

1825116.272 328527.509 541.073

1825119.258 328527.509 541.213

1825122.243 328527.509 541.422

1825125.229 328527.509 541.875

1825128.214 328527.509 541.817

1825131.200 328527.509 541.833

Portsmouth Bypass Corridor – Flight #2

• Adjusted flying plans (fly higher) to capture 5 cm imagery• Less flight lines = more efficient

capture!• Less image processing time =

faster turn-round time

• Added ground control to strengthen AT solution and account for higher flying height

• Limited deliverables to 5 cm image tiles and 20 mb ASCII files of the 3-foot gridded DSM

UAS

Example Images...

Example Point Clouds…

Portsmouth Bypass Corridor – Point Clouds

Portsmouth Bypass Corridor – Point Clouds

Portsmouth Bypass Corridor – Point Clouds

Portsmouth Bypass Corridor – Point Clouds

Portsmouth Bypass Corridor – Point Clouds

Portsmouth Bypass Corridor – Point Clouds

Portsmouth Bypass Corridor

Significant observations…• No other system (other than traditional photogrammetry) could

collect a site of this size and complexity in a single day• Zero impact an active construction site• No other production methodology could provide 2-3 week turn-

around time for all deliverables

Accuracy results

• Independent client checks:

• “The conventional topo matched up well with the Dec flight, +/-0.2ft.”

• “Everyone on our end was pleased with the data provided from the Dec flight.”

• “Tell your crew I appreciate all their work and efforts”

UAS

How did we know this would work?

Dr. Qassim AbdullahSenior Geospatial Scientist and Associate

Woolpert, Inc.

Woolpert’s UAS and Renaissance Program: Products Accuracy

Report

TRB2017 Annual Conference Washington, D.C. January 8, 2017

• County Line Road Corridor:• 1.3 miles in Dayton, OH• 2-cm Orthos / 40 to 700 pts/m2 point clouds

Accuracy Verification: Controls Layout

1.3 m

iles

B C D EF G

How Accurate are the Renaissance-derived Products?

A B C D E F G0 4 6 8 10 21 38

38 34 32 30 28 17 04.47 0.23 0.16 0.18 0.13 0.05 0.051.89 0.26 0.20 0.14 0.14 0.07 0.054.86 0.35 0.26 0.23 0.19 0.08 0.06

13.51 0.54 0.71 0.40 0.35 0.26 0.178.40 0.60 0.45 0.39 0.34 0.14 0.11

26.49 1.05 1.40 0.78 0.69 0.52 0.34

Accuracy Term

Horizontal Accuracy at 95% (ft.)Vertical Accuracy at 95% (ft.)

RMSE Elev. (ft.)

Processing Scenario

Number of Control PointsNumber of Check Points

RMSE E (ft.)RMSE N (ft.)

Radial RMSE N,E (ft.)HorizontalVertical

The winner: Pair of GCPs every 500 to 700 ft.

along the route…similar to MMS!

name # GCPs RMSE (z) @ GCP

Std Dev (z) @ GCP

RMSE (z) @ QC

Std Dev (z) @ QC

“B” 0 n/a n/a 13.61 ft 2.14 ft

name # GCPs RMSE (z) @ GCP

Std Dev (z) @ GCP

RMSE (z) @ QC

Std Dev (z) @ QC

“C” 4 0.01 ft 0.01 ft 0.58 ft 0.58 ft

name # GCPs RMSE (z) @ GCP

Std Dev (z) @ GCP

RMSE (z) @ QC

Std Dev (z) @ QC

“D” 6 0.02 ft 0.02 ft 0.76 ft 0.75 ft

name # GCPs RMSE (z) @ GCP

Std Dev (z) @ GCP

RMSE (z) @ QC

Std Dev (z) @ QC

“E” 8 0.02 ft 0.02 ft 0.44 ft 0.44 ft

name # GCPs RMSE (z) @ GCP

Std Dev (z) @ GCP

RMSE (z) @ QC

Std Dev (z) @ QC

“F” 10 0.04 ft 0.03 ft 0.47 ft 0.46 ft

name # GCPs RMSE (z) @ GCP

Std Dev (z) @ GCP

RMSE (z) @ QC

Std Dev (z) @ QC

“A” 38 0.13 ft 0.12 ft n/a n/a

Pix4D Processing• AT Results – North Block summary

• Observations & Camera calibration results

Pix4D Processing• AT Results – Run A - no GCPs Held

• Observations & Camera calibration results

Pix4D Processing• AT Results – Run F – all GCPs Held

• Observations & Camera calibration results

Pix4D ProcessingComparison to a metric camera:

4 years between RCD30 (Digital Camera) USGS calibrations = only 2-3 microns difference in focal length!

Takeaways…

• UAS Camera systems are not as “metric” (a.k.a. Stable) as traditional photogrammetric cameras

• Software (e.g. self-cal) can only do so much to compensate for the instability of a camera system

• The variation of self-cal computed focal lengths would indicate there is instability in the camera itself

Takeaways…

• The software appears to be putting the “error budget” in the focal legth

• The variation indicates that the errors may be non-normalized• You cannot use a random point sampling to

accurately asses the accuracy of your data (e.g. Traditional NSSDA 95% accuracy assessment methodology)

However…• UAS products (e.g. auto-correlated point clouds) can support

many needs for high resolution imagery

• Auto-correlated point cloud derivative products can meet many horizontal accuracy requirements for planimetric feature locations

• Auto-correlated point cloud derivative products do not meetmost 3D (a.k.a. vertical) most design grade mapping needs

• Auto correlated point cloud solutions appear to achieve • 2-3 pixel RMSE horizontal accuracies • 4-8 pixel RMSE vertical accuracies

• (Vertical RMSE errors can be larger than ½ foot with 2 cm GSD imagery)

Topics for further discussion…

• When does using a UAS make sense?• How to properly set customer expectations• Where can you fly (FAA restrictions)• Insurance requirements - Liability issues!• Training – FAA Certifications

UAS

Thank you!

Questions?