University of Illinois at Urbana-Champaign

GPS-LiDAR Sensor Fusion

Aided by 3D City Models for UAVs

Akshay Shetty and Grace Xingxin Gao

SCPNT, November 2017

University of Illinois at Urbana-Champaign

1

1

Positioning in Urban Areas

• GPS signals blocked or reflected

• Additional sensors: LiDAR, cameras, etc

University of Illinois at Urbana-Champaign

2

2

LiDAR State Estimation Challenge

• Surrounding features affect accuracy

• Need to characterize covariance accordingly

Start

End

[Google Earth][https://github.com/ethz-asl/ethzasl_icp_mapping]

University of Illinois at Urbana-Champaign

3

3

State Estimation Covariance

Adequate features Poor features Lack of features

[Google Earth] [Google Earth][Google Earth]

University of Illinois at Urbana-Champaign

4

4

Approach

• Deep sensor fusion

− Characterize LiDAR-based position covariance based

on features

• Eliminate NLOS satellites

− Use 3D city model to detect and eliminate NLOS GPS

satellites

University of Illinois at Urbana-Champaign

5

5

Outline

• Approach:

− Overall Architecture

− 3D City Model

− LiDAR-based State Estimation and Covariance

− GPS Measurement Model

• Experimental Setup and Results

• Summary

• Future Work: Deep Learning

University of Illinois at Urbana-Champaign

6

6

Overall Architecture

University of Illinois at Urbana-Champaign

7

7

3D City Model

• Illinois Geospatial Data Clearinghouse provides

top-view point cloud [https://clearinghouse.isgs.illinois.edu]

• OpenStreetMap provides building footprint

information [www.openstreetmap.com]

Building wall

information from

OpenStreetMap

Top-view point

cloud from

geospatial data

University of Illinois at Urbana-Champaign

8

8

LiDAR Odometry

• Use Iterative Closest Point (ICP) algorithm

• Match consecutive point clouds to estimate

incremental motion

Reference Point Cloud

Input Point Cloud

ICP

University of Illinois at Urbana-Champaign

9

9

LiDAR – 3D City Model

• Use ICP algorithm

• Match LiDAR point cloud with 3D city model

Before Matching

After Matching

ICP

University of Illinois at Urbana-Champaign

10

10

LiDAR – 3D City Model

True Position

Initial Positions

Final Positions

Feature

distribution

Position

accuracy

Adequate High

Poor Low

[Google Earth][Google Earth]

[Google Earth] [Google Earth]

University of Illinois at Urbana-Champaign

11

11

LiDAR Point Cloud Features

• LiDAR-based position covariance as function of

features

• Extract feature points based on curvature values [Zhang et al., 2014]

LiDAR Point Cloud

Surface Points

Edge Points

University of Illinois at Urbana-Champaign

12

12

Surface Feature Points

Covariance Ellipsoid

Surface Normal

Orthonormal Basis

University of Illinois at Urbana-Champaign

13

13

Edge Feature Points

Covariance Ellipsoid

Edge Direction

Orthonormal Basis

University of Illinois at Urbana-Champaign

14

14

Combined Position Covariance

LiDAR-based position covariance:

Covariance Ellipsoid

LiDAR Point Cloud

Surface Points

Edge Points

University of Illinois at Urbana-Champaign

15

15

GPS Measurement Model

• Pseudorange measurement:

• Double-difference measurement:

• Measurement covariance:

Clock biasesSpeed of light Atmospheric errors Measurement noise

University of Illinois at Urbana-Champaign

16

Eliminate satellites blocked by 3D city model

16

Non-line-of-sight (NLOS) Satellites

University of Illinois at Urbana-Champaign

17

17

Outline

• Approach:

− Overall Architecture

− 3D City Model

− LiDAR-based State Estimation and Covariance

− GPS Measurement Model

• Experimental Setup and Results

• Summary

• Future Work: Deep Learning

University of Illinois at Urbana-Champaign

18

18

Experimental Setup

Custom-built iBQR UAV

LiDAR

GPS

Antenna

Onboard

Computer

GPS

Receiver

IMU

University of Illinois at Urbana-Champaign

19

19

Results: Individual Measurements

GPS unweighted least

squares estimate

contains large errors

LiDAR odometry drifts

over time, due to poor

distribution of features in

some sections

LiDAR – 3D city model

matching contains errors

where ICP might

converge to local minima

University of Illinois at Urbana-Champaign

20

20

Results: Sensor Fusion

Our covariance model v/s fixed covariance model

Our algorithm matches

true path more accurately

compared to a fixed

covariance model

University of Illinois at Urbana-Champaign

21

21

Outline

• Approach:

− Overall Architecture

− 3D City Model

− LiDAR-based State Estimation and Covariance

− GPS Measurement Model

• Experimental Setup and Results

• Summary

• Future Work: Deep Learning

University of Illinois at Urbana-Champaign

22

22

Summary

• Proposed a deep sensor fusion architecture for

GPS and LiDAR

• Implemented a novel method to characterize

LiDAR-based position covariance

• Applied a 3D city model to eliminate NLOS

satellites

• Validated improvement in positioning accuracy

using proposed technique

University of Illinois at Urbana-Champaign

23

23

Outline

• Approach:

− Overall Architecture

− 3D City Model

− LiDAR-based State Estimation and Covariance

− GPS Measurement Model

• Experimental Setup and Results

• Summary

• Future Work: Deep Learning

University of Illinois at Urbana-Champaign

24

24

Deep Learning for Sensor Fusion

Develop deep learning for different components

University of Illinois at Urbana-Champaign

25

25

Deep Learning Dataset for GPS

• Experimental vehicle with 10 GPS receivers

• Collected data near San Francisco: downtown,

underground, open areas, etc.

• Intermediate measurements such as

pseudoranges, carrier phases, SNR

• High-grade IMU for ground truth

University of Illinois at Urbana-Champaign

26

26

Deep Learning Dataset for LiDAR

Simulations in Unity Game Engine

University of Illinois at Urbana-Champaign

27

Acknowledgement

We would like to thank Kalmanje Krishnakumar and his group at NASA

Ames for supporting this work under the grant NNX17AC13G

27

Thank you!