Upload
hugh-leslie-carroll
View
217
Download
0
Tags:
Embed Size (px)
Citation preview
Allen HansonChris Jaynes
Mauricio MarengoniEdward RisemanHoward Schultz
Frank Stolle
University of Massachusetts3D Building Reconstruction
With: Keith Hoepfner and Heddi Plumb
Ascona 2001
Talk Outline
Introduction: RADIUS and Ascender IOverview of Ascender IIGeometric Reconstruction of RooftopsExperimentsConclusions
Lab URL: http://vis-www.cs.umass.eduAPGD URL: http://vis-www.cs.umass.edu/projects/ascender/ascender.html
Research Goal(s)
EO, Digital Elevation Maps
IFSAR, Multi/Hyperspectral
3D Geometric Site Model Reconstruction
With multiple objects,
Using multiple strategies,
From multiple images
RADIUS: Ascender I
System for Automatic Site Modelling
Multiple Images Geometric Models
2D polygon detection line extraction corner detection perceptual grouping
epipolar matchingmulti-image triangulation
geometric constraints precise photogrammetry
extrusion to ground planeprojective texture mappingCVIU 72(2) 1998
Ascender Works
Extensive evaluation Good detection but high false alarms Good geometric accuracy
Ground Truth Ascender Results
.
10
20
30
40
50
60
70
80
90
73 Ground Truth Polygons
100
110
713
711
927525
Combined, 95%
Combined, 89%
.1 .2 .3 .4 .5 .6 .7 .8 .9
Detection Rates
Ascender Doesn’t Work
Delivered to NEL (NIMA) Applied to classified imagery Performance not as expected
High failure rate
(buildings not detected) High false positive rate
WHY? Imagery didn’tconform to design constraints!
No Buildings Detected!
Some Observations
IU Algorithms work within correct context constrained contexts constrained object classes
Domain knowledge provides constraints from domain, partial results, strategies
Many strategies, only correct ones used selective application correct parameters fuse results from individual strategies into
complete reconstruction
Key to Success
A system which:
applies the right strategy
at the right time
in the right place
with the right constraints
Ascender II Overview
Dealing with Context: Ascender II
Goals 3D Geometric Site Model Reconstruction Focus: More complex building structures Context sensitive control strategies for applying algorithms
Multiple strategies Multiple images Multiple sensors
Knowledge Representation, Control and Inferencing layered Bayesian networks
EO, Digital Elevation Maps
IFSAR, Multi/Hyperspectral
Observations
Buildings = Location + Geometry
Reconstruction = Localization + 3D Recovery
3D Recovery is a two-stage process
Where are the buildings? What is their structure?
Ascender II Overview
Bayes NetworkControl SystemCP
CPCP
CP
Ascender II Knowledge BaseAvailable Data
ImageryElevation dataSite Model
ASCENDER ISystem
andStrategies
Ascender IIRCDESocket I/O
CP: Control Policy
IU Algorithms work within correct context. Domain knowledge provides constraints. Choose good strategy from many alternatives.
Basic Principles
Ascender II Components
Visual Subsystem Library of IU Algorithms Geometric Database (data, models) Display/user interface through RCDE
Knowledge Base Control system Belief networks in Hugin Reasoning mechanisms over regions of discourse
Regions of Discourse Subset of available data Image regions, a model, partial interpretation
Knowledge Representation
PossibleValuesH1={j,..,k}
H1
Variable of interest
Schema/Belief Node
Evidence Policy
H3
H4
H2
First level of classification hierarchy
Second level (object subclass)
Combination of belief networks and visual schemas, implemented in Hugin.
Encodes: Domain Knowledge Acquired Site Knowledge Control Mechanism
Allows both diagnostic and causal inference.
Hierarchical topology allows simpler networks which reduces propagation time and controls complexity.
Evidence policy defines how IU strategies gather relevant evidence according to context.
Algorithms to Evidence
Define the context in which it can run currently straightforward context only disallows an algorithm too simple in the long run
Method for deriving certainty value needed to update knowledge network
Given an IU algorithm....
Algorithms
Algorithms operate on regions of discourse Ascender I Surface Fits Line Extraction and Count Junction Count Average Junction Contrast Region Height Region Height to Width ratio Edge Terminals ……………...
3D Geometric Rooftop Reconstruction
Fusing DEM and Optical Data
Registered Optical and Elevation Reconstruct Rooftops (and buildings)
Optical Image (Fort Hood) Elevation Data (Fort Hood)
Technical Overview
Registered Optical and DEM
Parameterized Model Library
Site Model Visualization
Model Indexing
Elevation Surface and Histogram
Mod
el F
it: O
ptim
izat
ion
Surface Primitive Library
51 Models in 8 Parameterized Classes
Gable
Ascender + Terrest Results
Experiments
Fort HoodAvenches
Fort Benning
Experiments
Approach Site model from Ascender I with no filtering and loose polygon acceptance criteria => high detection rate but lots of errors (false positives) Classify Ascender I regions into
{building, parking lot, open field, unknown} Classify buildings into
{multi-level building, single-level building} Then into {flat, peak, round, flat-peak, other} Site model from Ascender II after classification/reconstruction
Degraded Ascender I ModelFort Hood
22 True Positive Buildings1 Incorrect Model (A) (multi-level reconstructed as single level)
No Knowledge and Loose Polygon Acceptance Criteria
A
20 False Positive Buildings
Bayes Nets for Experiments
Planar Fit
Evidence Policy: none
Multi-LevelSingle
Building
TJunctions
GoodBad
YesNo
Region
BuildingOpen FieldParking LotComplexUnknown
Evidence Policy: none
Height
Evidence Policy:multi-EO: triangulateHeight()DEM: medianHeight()
LowMediumHigh
Planar Fit
GoodBad
Evidence Policy:
DEM: robustFit(plane)
WidthLine
Count
LJunctions
NumberT Junc.
ContrastT Junc.
Evidence Policy:EO: junctions(T)
<5>5None
<50>50Zero
Evidence Policy: Evidence Policy:EO: junction_count(T) EO: junction_contr(T)
First level of classification hierarchy
Second level (object subclass)
Level 1
Level 2
Next Slide
Level 3 of Network
Rooftop
FlatPeakRoundFlat-peakOther
Evidence Policy: none
Planar Fit
GoodBad
Evidence Policy:
DEM: robustFit(plane)
Level 3
Middle>
CornerYesNo
SideLine
YesNo
CenterLine
YesNo
ShadowFormat
PrismRoundFlat
Multi-level
YesNo
Evidence Policy: none
Planar Fit A
GoodBad
Evidence Policy:
DEM: robustFit(plane)
Level 3
Height A>
Height BYesNo
CenterLineRatio
YesNo
Planar Fit B
GoodBad
Evidence Policy:
DEM: robustFit(plane)
Multiple calls, one for each section
Visual Subsystem:Algorithms and Evidence Policies
Stereo-OpticalDEM
DEM MedianHeight
Multi-OpticalEpipolar Align.
Region-EdgeTriangulate Hgt.
USGS DTM DTM at regioncenter
HeightP{}
xF{x}
P{}
Height
Evidence policies associate algorithms with belief.
Function P maps request for evidence from knowledge base to algorithm.
F(x) maps algorithm result to belief value.
Example Algorithms: Parallel Lines T junctions DEM-Median-Height Region-Edge-Triangulate-Height Planar Fit Error
Experiments
Detection rates per Object Class (Semantic Accuracy) Three-Dimensional Accuracy
For each DataSet For each Object Class (buildings)
How do network priors influence performance? Uniform (no knowledge) versus Estimated Priors Adjustment of Priors for each dataset versus uniform
Three Datasets Fort Benning MOUT Fort Hood Facility Avenches Boat Yard
Priors and Operators Used
Network Priors by Dataset
Operators Applied by Dataset
Site Name Building ParkingLot
OpenField
SingleVehicle
Other
Fort Hood 0.35 0.2 0.2 0.15 0.1Avenches 0.32 0.25 0.38 0.03 0.02Fort Benning 0.35 0.2 0.2 0.15 0.1
SiteName
Total(avg.)
Building ParkingLot
OpenField
SingleVehicle
FortHood
3.9 4.25 4.00 3.29 0.0
Avenches 4.7 5.22 5.00 4.0 0.0FortBenning
2.9 3.00 0.0 0.0 2.0
SkipF.Hood
Ascender II ClassificationContext Sensitive Strategies
43 Regions, 4 Misclassified, 1 Unknown
88% Accuracy
Ascender II Reconstruction
3 False Positives23 True Positive Buildings0 Incorrect Models (single, multi level)3 New Functional Areas
Site Reconstruction:Fort Hood
Reconstruction Accuracy:Fort Hood
Object Class TruePositive
FalsePositive
FalseNegative
Completeness Correctness
Building (General) 23 3 10 70.0% 88.5%Peak Roof 0 0 1 0% 0%Flat Roof 22 2 9 71.0% 91.6%Cylinder Roof 0 1 0 0% 0%
Detection Rates by Object Class
Error MetricIV Planimetric 0.69IV Altimetric 0.52IV Absolute 0.864Centerline 0.491PercentageOverlap
98.4%
Median Accuracy Measures , Building Classes
SkipAvenches
ISPRS Avenches Dataset
Two Overlapping Optical Images (60% sideward)
Corresponding DEM Calibration provided by ETH 200x220 meter “Boatyard” Ground Sample Dist: ~ 0.13 meters Reference model provided by ETH
Regions and Classifications:Avenches
Parking Lot
Peak Bldg.
Peak Bldg.
Peak Bldg.
Peak Bldg.
Open Field
Flat Bldg.
Flat Bldg.
Flat Bldg.
Flat Bldg.
Flat Bldg.
Peak Bldg.
Site Reconstruction:Avenches
Skip Benning
Fort Benning MOUT Site
Two overlapping downlooking optical images DEM computed from image pair using Terrest Camera resection (SRI) Reference Model (SRI) GSD, optical view, 0.31 meters
Optical DEM
Original Image Ascender I Polygons
Polygons from SAR
+
Final Polygons
MOUT Input Polygons
MOUT Site: Classification
Object Class True False False Positive Positive Negative
Building (General) 19 1 0Peak Roof 14 1 0Flat Roof 5 0 0Single Car 1 1 2
Flat roof building detectedas peak.
Low walled entrance (open field)Detected as single vehicle.
Missed detection of two vehicles.
Overall classification accuracy: 91%
Recovery of 4 attached buildings.
Model Indexing:MOUT Region 8
Constructed Surface Mesh Corresponding Gaussian Sphere
PlaneFlat PeakPeak (15)Peak (35)
.939
.819
.742
.601
Correlation Scores
MOUT Site: Accuracy
Detection Rates, Full Evaluation
Detection Rates, FOA Regions Only
Object Class TruePositive
FalsePositive
FalseNegative
Completeness Correctness
Building(General)
18 1 4 81.8% 94.7%
Peak Roof 14 1 0 100% 93.3%Flat Roof 5 0 0 100% 100%Single Car 1 1 2 33.3% 50%Overall Rates 20 2 2 90.1% 90.1%
Object Class TruePositive
FalsePositive
FalseNegative
Completeness Correctness
Building(General)
18 1 0 100.0% 94.7%
Peak Roof 14 1 0 100% 93.3%Flat Roof 5 0 0 100% 100%Single Car 1 1 0 100.0% 50%Overall Rates 20 2 0 90.1% 90.1%
MOUT Site: Reconstruction
MOUT Reconstruction
SkipKodak
Examples on Three Image Chips
Hospital
FineArts
Industrial
N
Building Localization
Optical Lines 2D PolygonsBuilding FP Hypotheses
+Final Building
Location Hypotheses
+ Classifier:TreesBuildingsGround
Building FP Hypotheses
Elevation BlobsBuildings (st. lines)Trees (no st. lines)
v Building FP Hypotheses
Reg
istr
atio
n
FP = Footprint
Building Localization: Polygons
Group
Focus (optional)
Graph Polys
GeometricRecovery
Classification
Canny Edges Grouped Lines Hypotheses
Building FootprintOn Range Data
Essentially Ascender I 2D Polygon Detection Algorithm with additional external inputs providing additional constraints.
Building Localization: Blobs
Bald Earth Model
Thresholded using bald earth
modelFiltered using
Long Lines and Region Size
First Threshold
Lines from Optical
Range Data
Localization Results
Hospital
Industrial
Fine Arts
Final Building Models
Conclusions
Vision works when properly constrained and focused Still a lot of work to do on:
Basic Algorithms Knowledge Representations Representing and Using Context and Constraints Inferencing and Causal Reasoning System Architectures
Conclusions, Part 2
No shortage of interesting research problems!
Lab URL: http://vis-www.cs.umass.edu
3D Geometric Rooftop Reconstruction
Model-Based Indexing
Use DEM and focus-of-attention to recognize rooftop shape.
Building rooftops can be modeled as the union of a small set of
parameterized surfaces.
Variety arises from different combinations and parameterizations.
Index into a library of rooftop models.
Provide initial parameters for final reconstruction (robust fitting)
Regions classified as single-building are then passed to the Model-
Indexing algorithm for recognition.
Library of models is filtered based on the distribution
at the single-building node.
DEM + Optical 2: Complex Roofs
Registered Optical and Elevation Reconstruct Rooftops (and buildings)
Optical Image (Fort Hood) Elevation Data (Fort Hood)
Technical Overview
Registered Optical and DEM
Parameterized Model Library
Site Model Visualization
Model Indexing
Elevation Surface and Histogram
Mod
el F
it: O
ptim
izat
ion
Model-Based Indexing
Use DEM and focus-of-attention to recognize rooftop shape.
Building rooftops can be modeled as the union of a small set of
parameterized surfaces.
Variety arises from different combinations and parameterizations.
Index into a library of rooftop models.
Provide initial parameters for final reconstruction (robust fitting)
Regions classified as single-building are then passed to the Model-
Indexing algorithm for recognition.
Library of models is filtered based on the distribution
at the single-building node.
Visual Subsystem:Algorithms and Evidence
Policies
Evidence policies associate algorithms with belief.
Function P maps request for evidence from knowledge base to algorithm.
F(x) maps algorithm result to belief value.
Example Algorithms: Parallel Lines T junctions DEM-Median-Height Region-Edge-Triangulate-Height Planar Fit Error
Experimental Method
Use “reference model” of each site as basis for evaluation
Centerline Distance to measure the difference between two arbitrary
polygons/wireframes
tessellate target and reference, take average of minimum distance for all
points between target and reference, and reference to target.
measures: size, shape, location error
Detection: C-dist(T,R) < 0.2*cube-root(volume(T))
Planimetric, Altimetric, and Intervertex error
Voxel Overlap
Completeness = 100 x (True Positives)/(True Positives + False
Negatives)
Correctness = 100 x (True Positives)/(True Positives + False Positives)
Algorithm Library
Library contains both lightweight and heavyweight processes
Algorithms expected to perform only under constrained conditions
Algorithms contain descriptions of context under which they can be applied