Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
Using multiple optical images3D surface reconstrunction
20205309 Chungsu Jang
2020 12.10
●Introduction
●Related Works
•Approach & Experiment
●Conclusion
●Q&A
Contents
DSM(Digital Surface Model)
Korea Daejeon, WorldView3 Satellite
Test area 1
Test area 2
Analysis Data characteristics3D dense points obtained using satellite imagery and SGM(Semi Global Mapping) matching technique
Test area 1
Test area 2
Area 1(Daejeon) : Disparity Maps
Area 2(Daejeon) : Disparity Maps
Refinementblock_size = 15
➢ All Census & MI data : noise vertical range ~ 2 m, Many miss-matching points
Refinement
Problem Derivation through Analysis Data characteristics
●Introduction
●Related Works
•Approach & Experiment
●Conclusion
●Q&A
Contents
Related Works
●Urban 3D reconstruction– Urban Semantic 3D reconstruction from Multiveiw Satellite Imagery, CVP
R 2019
Related Works
• Geo referencing –mapping image pixels to global coordinates, photometry
GtoI()
ItoG()
Ref: Introduction to Photogrammetry and Remote sensing, 1st Edition
R : RPC(The rational polynomial coefficient)
Related Works
• Semi Global Matching(SGM)– Algorithm and architecture of disparity estimation with mini-Census
adaptive support weight, IEEE Transactions on Circuits, 2006
Related Works
• Point cloud set matching– ICP(Iterative Closest Point)
• A Method for Registration of 3-D Shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992
–Least-squares Height Difference(LHD) Matching• Three-dimensional absolute orientation of stereo models using digital elevation models,
PHOTOGRAMMETRIC ENGINEERING AND REMOTE SENSING, 1988
●Introduction
●Related Works
•Approach & Experiment
●Conclusion
●Q&A
Contents
For 3D building shape restoration , approach
High-rise Building Reconstructionusing Warped Images
No way to recover these missing buildings?
Projected points from image # 1
Alignment and color information in the building
Left P.C.
Left image
1 4
Building
Object space
Plane
segmented
from LiDAR
2
3
Right P.C.
Right image
1 4
Building
Object space
Plane
segmented
from LiDAR
2
3
Projected points from image # 2
Warped Image
When knowing information about the floor plan of the building
Alignment and color information in the building
Projected points from image # 1
Left P.C.
Left image
1 4
Building
Object space
Plane
segmented
from LiDAR
2
3
Two warped image comparison /
Inside and outside the building Similarity
Warped Image
fl frelbl dl erbr dr
LiDAR plane
PCL PCR
A
B
C
D E
Ground
Rooftop
Wall
Hig
h s
imila
rity
Lo
w s
imila
rity
Hig
h s
imila
rity
F
Hig
h s
imila
rity
Left warped image
Right warped image
g(bl) g(dl) g(el) g(fl)
g(br) g(dr) g(er) g(fr)
Left satellite image
Right satellite image
Left warped image
Building rooftop plane
Right warped image
➢ Similarity measure between two warped images and building height estimation
High-rise Building Reconstructionusing Warped Images
Similarity Check: Hamming distance
19
Proposed Workflow – 1st track approach
Two warped image generation
Panchromatic Image (Right)
Min < Height< Max
Similarity Calculation (Hamming Dist.)
Best Height Estimation
Panchromatic Image (Left)
DBM generation
Yes
No
Check the min. costRecord costs
for each height
Best Views
Three-dimensional geometric condition
Convergence Angle : 25~45
Bisector Elevation Angle : > 60
Satellite Azimuth Angle : <70
Satellite Azimuth Angle : <15
Similarity Measure Map
Left warped image
Right warped image
High similarity
Low similarity
Similarity Check
➢ High similarity measure at the correct height
➢ Low similarity measure at the incorrect height
H : 81m H : 107mH : 99m
Test results
Best estimated Building Height : 99m at the min. cost
※ Reference height is extracted by intersection after manually measuring the same feature from left and right satellite images
Test results
Reference Height (m)
Estimated Height(m)
diff (m)
‘ㄷ’type building 72.61 72.50 0.11
‘ㅁ’type building 99.29 99.00 0.29
Apartment 100.75 101.00 0.25
High building 233.45 233.50 0.05
Low builidng– 1 70.47 70.00 0.47
Low building– 2 70.71 71.00 0.29
24
Using Warped image Outline and plane extraction method
Warped image (Right)
Line matching
Warped image (Left)
Best height
Line extraction (Left) Line extraction (Right)
Line merge (Left) Line merge (Right)
Line intersection
Space partitioning
Cumulative min cost image generation
Space Selection
Space Union
DBM
Cost check for each partition
Linear Information Extraction and Merge
Line detection Line merge
Line detection Line merge
Left warped image at the Best height
Right warped imageat the Best height
Best height estimation using similarity
measure
Matching, Intersection, Space partitioning and Building area selection
Line matching Line intersection Space Partitioning
Space SelectionSpace Union DBM
Test results (two satellite images, Redundancy)
ㄷtype
ㅁtype
Hig
h
Space partitioning Space selection Space unionImage
Space partitioning Space selection Space unionImage
Space partitioning Space selection Space unionImage
Apartm
ent
Low
-
1
Low
-
2
Space partitioning Space selection Space unionImage
Space partitioning Space selection Space unionImage
Space partitioning Space selection Space unionImage
Test results (two satellite images, Redundancy)
DBM
ㄷtype
ㅁtype
Hig
h
DBM
DBM
Apartm
ent
Low
-1Low
-
2
DBM
DBM
DBM
Test results (two satellite images, Redundancy)
Test results (Improved DSM Generation)
DSM from point cloud
Build
ing 1
Build
ing 2
Build
ing 3
DSM from point cloud
DSM from point cloud
Improved DSM
Improved DSM
Improved DSM
Test results (Improved DSM Generation)
Build
ing 4
Build
ing 5
DSM from point cloud Improved DSM
DSM from point cloud Improved DSM
Sensor Fusion
IKONOS-2
Test area
7340 7340
7341 7341
Epipolar Image SGM Disparity Map
WV-3
Sensor Fusion
7340 7341
DEM
IKONOS-2 WV-3
Sensor Fusion
Point Cloud Matching : Use 12,322 points
RMSE = 23.4m; Max = 89.1m
LHD Matching (reference)
Sensor Fusion
3D matching
(7340+7341)
Point Cloud
Matching
Mosaic
(7340+7341)
●Introduction
●Related Works
•Approach & Experiment
●Conclusion
●Q&A
Contents
Conclusion
➢ Suggest to generate 3d surface model from several sat
ellite image
➢ Show you the generated 3d model surface, visually
➢ Limit : Completeness? / Accuracy?
●Introduction
●Related Works
•Approach & Experiment
●Conclusion
●Q&A
Contents
Thank youQ & A