Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Local-to-Global Point Cloud Registration Using a Viewpoint
Dictionary
David Avidar
MSc Research under the supervision of
Prof. David Malah and Dr. Meir Barzohar
02/07/18
Omek Consortium
Signal and Image
Processing Lab
SIPL Annual Event 2018
What is a point cloud?
• Point cloud: a set of points given in a common coordinate system 𝑥
𝑦
𝑧
2
What is a point cloud?
• Point cloud: a set of points given in a common coordinate system
• Typically represented as a list of coordinates:
𝑥1 𝑦1 𝑧1⋮ ⋮ ⋮𝑥𝑁 𝑦𝑁 𝑧𝑁
• Acquisition methods: Structure-from-Motion, stereo
cameras Laser sensors (e.g., LiDAR) IR cameras, mostly indoors (e.g.,
Kinect, RealSense)
𝑥𝑦
𝑧
3
Problem definition
• Alignment between:- a local cloud (e.g., terrestrial LiDAR)
- and a global cloud (e.g., airborne LiDAR)
5
Problem definition
• Alignment between:- a local cloud (e.g., terrestrial LiDAR)
- and a global cloud (e.g., airborne LiDAR)
6DoF:Translation
+Rotation
6
Motivation
• High-resolution urban modeling Airborne LiDAR –
coarse overall structure Terrestrial LiDAR –
fine details
• Accurate localization in large scale urban environment Better reliability than GPS No dependency on
inertial sensors
7
Motivation
• High-resolution urban modeling Airborne LiDAR –
coarse overall structure Terrestrial LiDAR –
fine details
• Accurate localization in large scale urban environment Better reliability than GPS No dependency on
inertial sensors
8
Motivation
• High-resolution urban modeling Airborne LiDAR –
coarse overall structure Terrestrial LiDAR –
fine details
• Accurate localization in large scale urban environments Better reliability than GPS No dependency on
inertial sensors
9
Motivation
• High-resolution urban modeling Airborne LiDAR –
coarse overall structure Terrestrial LiDAR –
fine details
• Accurate localization in large scale urban environments Better reliability than GPS No dependency on inertial
sensors for position and orientation (Yaw,Pitch,Roll)
10
Challenges
Figure: Williams et al., 2013
• Different point density distributions: Airborne: high density on
horizontal surfaces Terrestrial: high density on
vertical surfaces
• Different types of occlusion Airborne: mostly vertical
occlusion Terrestrial: mostly horizontal
occlusion
• Different typical noise levels Airborne: 𝜎 ≈ 15𝑐𝑚 @ 1𝑘𝑚 Terrestrial: 𝜎 ≈ 0.5𝑐𝑚 @ 100𝑚
11
Challenges
Figure: Williams et al., 2013
• Different point density distributions: Airborne: high density on
horizontal surfaces Terrestrial: high density on
vertical surfaces
• Different types of occlusion Airborne: mostly vertical
occlusion Terrestrial: mostly horizontal
occlusion
• Different typical noise levels Airborne: 𝜎 ≈ 15𝑐𝑚 @ 1𝑘𝑚 Terrestrial: 𝜎 ≈ 0.5𝑐𝑚 @ 100𝑚
12
Challenges
Figure: Williams et al., 2013
• Different point density distributions: Airborne: high density on
horizontal surfaces Terrestrial: high density on
vertical surfaces
• Different types of occlusion Airborne: mostly vertical
occlusion Terrestrial: mostly horizontal
occlusion
• Different typical noise levels Airborne: 𝜎 ≈ 15𝑐𝑚 @ 1𝑘𝑚 Terrestrial: 𝜎 ≈ 0.5𝑐𝑚 @ 100𝑚
13
Outline
• Problem definition and motivation
• Prior art - keypoint-based registration
• Registration using a viewpoint dictionary
• Summary
14
Keypoint-based registration - 1
1. Keypoint detection Intrinsic Shape Signature (ISS)
(Zhong, 2009)
3D Harris (Sipiran et al., 2011)
3D SIFT (Rusu et al., 2011)
15
Keypoint-based registration - 1
1. Keypoint detection Intrinsic Shape Signature (ISS)
(Zhong, 2009)
3D Harris (Sipiran et al., 2011)
3D SIFT (Rusu et al., 2011)
2. 3D descriptor computation Spin-Images (Johnson, 1997)
Fast Point Feature Histogram (FPFH) (Rusu et al., 2009)
Rotational Projection Statistics (RoPS) (Guo et al., 2013)
16
Keypoint-based registration - 1
1. Keypoint detection Intrinsic Shape Signature (ISS)
(Zhong, 2009)
3D Harris (Sipiran et al., 2011)
3D SIFT (Rusu et al., 2011)
2. 3D descriptor computation Spin-Images (Johnson, 1997)
Fast Point Feature Histogram (FPFH) (Rusu et al., 2009)
Rotational Projection Statistics (RoPS) (Guo et al., 2013)
17
3. Descriptor matching k-d tree (Bentley, 1975), FLANN (Muja and Lowe, 2009)
Keypoint-based registration - 2
4. Coarse registration RANdom SAmple Consensus
(Fischler and Bolles, 1981) or variants
18
Keypoint-based registration - 2
4. Coarse registration RANdom SAmple Consensus
(Fischler and Bolles, 1981) or variants
19
Keypoint-based registration - 2
4. Coarse registration RANdom SAmple Consensus
(Fischler and Bolles, 1981) or variants
5. Registration refinement Iterative Closest Point
(Besl and McKay, 1992) (Chen and Medioni, 1991)
Many other variations…
20
Keypoint-based registration - 2
4. Coarse registration RANdom SAmple Consensus
(Fischler and Bolles, 1981) or variants
5. Registration refinement Iterative Closest Point
(Besl and McKay, 1992) (Chen and Medioni, 1991)
Many other variations…
Find nearest neighbors (NN)
From 𝑃 to 𝑄
Find 𝑹, 𝒕 thatminimize MSE
𝑀𝑆𝐸 =1
𝑁
𝑖=1
𝑁
𝑹𝒑𝑖 + 𝒕 − 𝒒𝑁𝑁 𝒑𝑖
2
21
Keypoint-based registration - 3
• Limitations: Captures only local information (sensitive to large
differences in point density)
Lack of salient features in mostly flat scenes
Ambiguity in repetitive scenes
Found to be unreliable for terrestrial Vs. airborneregistration
22
Outline
• Problem definition and motivation
• Prior art - keypoint-based registration
• Registration using a viewpoint dictionary
• Summary
23
Registration using a viewpoint dictionary
• Main concepts: Large-scale global cloud
Characterize each viewpoint using the geometric information of its environment
Local-to-global registration
Viewpoint-based dictionary
24
Viewpoint dictionary search
(scene matching)
Block diagram
“Local-to-global point cloud registration using a dictionary of viewpoint descriptors”, ICCV 2017, Venice, Italy
25
Block diagram
26
Block diagram
27
• Define a viewpoint grid over the global cloud
• Create a descriptor for each viewpoint - panoramic range-image
Viewpoint dictionary creation
Viewpoint grid
Dictionary range-image
Dictionary range-image
28
Local viewpoint descriptor creation
Viewpoint grid, Local sensor
• Create a panoramic range-image of the local cloud
Local range-image
Dictionary range-image
29
Block diagram
30
• If GPS location is available, define dictionary search area around it
• Otherwise, search over entire grid
Candidate viewpoint selection - 1
Viewpoint grid, Search area, GPS
31
• Match local and dictionary range-images using phase correlation Select a small subset of candidate grid viewpoints (e.g.,
10 with largest phase correlation peaks)
Candidate viewpoint selection - 2
Phase correlation
32
Candidate viewpoint selection - 2
• Match local and dictionary range-images using phase correlation
Phase correlation
33
• Main steps:• Compute 2D DFT of both images𝑮𝑎 𝑢, 𝑣 = ℱ 𝑔𝑎 𝑥, 𝑦 𝑮𝑏 𝑢, 𝑣 = ℱ 𝑔𝑏 𝑥, 𝑦
• Compute normalized cross-power spectrum
𝑅 𝑢, 𝑣 =𝑮𝑎∗𝑮𝑏
𝑮𝑎∗𝑮𝑏
• Find the phase correlation using inverse Fourier transform𝑟 𝑥, 𝑦 = ℱ−1 𝑅
• Recover image shift - find peak in phase correlation Δ𝑥, Δ𝑦 = 𝑎𝑟𝑔𝑚𝑎𝑥𝑥,𝑦 𝑟 𝑥, 𝑦
ℱ 𝑔𝑎 ⋆ 𝑔𝑏⋆ = cyclic cross-
correlation
Phase correlation
34
𝑔𝑎 𝑥, 𝑦 𝑔𝑏 𝑥, 𝑦
Cyclic cross-correlation𝑔𝑎 ⋆ 𝑔𝑏
Phase correlation 𝑟 𝑥, 𝑦
Images: Foroosh et al., 2002
Phase correlation - example
35
Candidate viewpoint selection - 3
• Match local and dictionary range-images using phase correlation Select a small subset of candidate grid viewpoints (e.g.,
10 with largest phase correlation peaks)
• Grid viewpoints
• Ground truth
• 10 candidates with largest phase correlation peaks
36
Block diagram
37
Block diagram
38
RMSE verification
• Select a smaller subset of candidate viewpoints with lowest RMSE between the local and global clouds (e.g., lowest 3)
• Grid viewpoints
• Ground truth
• 10 candidates with largest phase correlation peaks
• 3 candidates with lowest RMSE
39
Block diagram
40
Registration refinement
• Refine using ICP and select registration with lowest RMSE
• Ground truth sensor location
41
Registration refinement
• Refine using ICP and select registration with lowest RMSE
• Ground truth sensor location
42
Registration refinement
• Refine using ICP and select registration with lowest RMSE
• Ground truth sensor location
43
Test dataset (Vancouver)
• Global cloud area: ~1𝑘𝑚2
• # Local clouds: 104
44
Registration results - 1
RegistrationRefinement
Method
Localization error [m]
Mean STD Max
Using GPS (30m search radius)
0.40 0.20 1.13
No GPS (search entire grid)
0.44 0.27 1.94
RRE [deg]
Mean STD Max
0.67 0.30 1.73
0.78 0.39 1.96
Runtime [sec]*
Mean STD Max
2.0 0.4 3.7
14.5 1.2 19.4
* Run on PC (i7-5820K CPU @ 3.30 GHz), MATLAB implementation
(Registration time per local cloud)
RRE: Relative Rotation Error
𝑹𝑹𝑬 = 𝒀𝒂𝒘 𝒆𝒓𝒓𝒐𝒓 +𝑷𝒊𝒕𝒄𝒉 𝒆𝒓𝒓𝒐𝒓 +𝑹𝒐𝒍𝒍 𝒆𝒓𝒓𝒐𝒓
45
Registration results - 2
Our registration
46
Registration results - 3
Ground Truth
47
Comparison to keypoint-based registration methods
48
Registration method # Tested local clouds
# Local clouds with localization
error < 3m
Avg. runtime per local cloud [sec]
Mean STD
FPFH (Rusu et al., 2009) 24 6 (25%) 53.7 23.1
RoPS (Guo et al., 2015) 104 8 (7.7%) 88.7 23.7
Viewpoint dictionary (proposed)
104 104 (100%) 2.0 0.4
• Using the proposed method, the maximal localization error over all 104 clouds was 1.13m The mean error was 0.40m (STD=0.20m)
Outline
• Problem definition and motivation
• Prior art - keypoint-based registration
• Registration using a viewpoint dictionary
• Summary
49
Summary• Proposed a viewpoint-dictionary-based point
cloud registration method
50
Achieved significant improvement in registration performance over state-of-the-art keypoint-based methods (FPFH, RoPS)
Viewpoint grid, Search area, GPS
• Take-home message: When matching two very
different types of 3D data (e.g., terrestrial vs. airborne LiDAR), better to use viewpoint descriptors than local features
02/07/18
Omek Consortium
Signal and Image
Processing Lab
SIPL Annual Event 2018
Thank You!Questions?