Data-Driven Shape Analysis--- Geometry Reconstruction

Qi-xing HuangStanford University

Geometry Reconstruction

• Occlusion

• Material

• Accuracy/Resolution/Range

• Speed

Issues

Data-Driven Reconstruction

Combine data + priors (from existing shapes)

• Structure-aware

– Data from the same shape (enforced by structures)

– Rich literature in the computer vision community

• Global priors

– Find similar shapes

Two topics

Structure-aware

Context-based Surface Completion [SIG’04]

Approach

Symmetry aware [SIG’08]

Symmetry aware [SIG’10]

Symmetry aware [SIG’ 10]

Algorithm Steps

Detection Registration Consolidation Filtering/sampling

Mapping

Semi-automatic Repetition Detection

• Cluster descriptors

• Descriptors configuration

from user query

• Automatic partial

matching [Bokeloh et al. 2009]

Algorithm Steps

Detection Registration Consolidation Filtering/sampling

Mapping

Segmentation and Registration

• Coarse alignment using ICP

– Based on planes and edges (RANSAC)

– Weighed by plane confidence

• Estimate plane confidence

– Area

– Homogeneity

– Point anisotropy

0 1

Algorithm Steps

Detection Registration Consolidation Filtering/sampling

Mapping

Consolidation

• Off-plane: work on planes

– Cluster planes

– Choose representative planes

– Project points to

representatives

• In-plane: work on edges in

each plane

Clustering

Consolidation

Weighted L1 median consolidation

Algorithm Steps

Detection Registration Consolidation Filtering/sampling

Mapping

Filtering of lines based on major axis directions

Partition plane into rectangles

Classify to inliers, outliers

Remove outliers, upsample

Filtering and Upsampling

Results: cylinders consolidation

Combining Images and Shapes [ICCV’11]

Detecting symmetries in the image domain

Structure from motion [TOG’14]

Detect symmetries in 3D

Symmetries are enforced in SFM

Coupled Structure-from-Motion and 3D Symmetry Detection for Urban Facades, TOG’14

Global Shape Prior

Example-Based Scan Completion

Example-Based 3D Scan Completion, SGP’05

Examples

Final Model

Context Models

Deformed Models

Search-Classify [TOG’12]

A Search-Classify Approach for Cluttered Indoor Scene Understanding, TOG’12

Two interleaved problems

What are the objects?

Where are the objects?

• Search

–Propagate /

accumulate

patches

• Classify

–Query

classifier to

detect object

Approach

Overview

Training

Search-Classify

Point-cloud Features– Height-size ratio of BBox

– Aspect ratio of each layer

– Bottom-top, mid-top size ratio

– Change in COM along horizontal slabs

Bh

BdBw

Search-Classify• Starts from seeds

– Random patch triplets

– Remove seeds with low confidence

• Accumulating neighbor patches

– Highest classification confidence

• Stop condition

– Steep decrease in classification confidence

0.65 0.92 0.93 0.88

Seed

Refinement via template fitting• Segmented - classified objects problems

– Overlap, outliers, ambiguities etc.

• Refinement

– Outliers = patches with large distance

Results

Contextual information is no considered

Part-based Shape Reconstruction [TOG’12]

No suitable model!shape retrieval

Structure Recovery by Part Assembly TOG’12

Recover the structure by part assembly

Algorithm Overview

Candidate Parts Selection Structure Composition Part Conjoining

……

Algorithm Overview

Candidate Parts Selection Structure Composition Part Conjoining

……

Algorithm Overview

Candidate Parts Selection Structure Composition Part Conjoining

……

Algorithm Overview

Candidate Parts Selection Structure Composition Part Conjoining

……

Results: Chairs• 70 repository models, 11 part categories

Results: Tables• 61 repository models, 4 part categories

Results: Bicycles

• 38 repository models, 9 part categories

Results: Airplanes

• 70 repository models, 6 part categories

Future directions

• Grammar-based

– Understand the variability of a class of shapes [Hao et al. 14]

• Scene reconstruction/Understanding

• Data-driven dynamic geometry reconstruction

Future Directions