Scene Reconstruction

Seminar presented by Anton Jigalin

Advanced Topics in Computer Vision (048921)

2

Scene Reconstruction

Scene Visualization:

Photo Tourism: 3D photo browser

Path Finder: 3D navigation interface

References: Scene Reconstruction and Visualization from Internet Photo Collections. N. Snavely. 2008.

3

Scene Reconstruction

4

The goal

Automatically reconstruct 3D geometry from large collections of photos from the Internet

Motivation

5

New way to organize photo collections Experience of being at a place

6

Related works

Image based modeling

Structure from motion (SfM)

Moviemaps

7

Challenges

Unknown cameras position

Unknown internal camera parameters

Correspondence problem

Visualization

Algorithmic complexity

Running the complete pipeline on a 3.80GHz Intel Xeon machine with 4GB of core memory. The keypoint detection and matching phases were run in parallel on 10 such machines.

8

Results

9

How was this result achieved?

10

Finding correspondence

Feature detection - SIFT

Feature matching - RANSAC

Epipolar constraint

11

Structure from Motion Algorithm

12

Initial two-frame reconstruction

Intermediate stage Final reconstruction

Structure from Motion Algorithm

13

Failure modes

Repeating textures

Insufficient overlap or texture

Cascading errors

Bad initialization

14

Connectivity graphs

15

Scene Visualization:

Photo Tourism: 3D photo browser

16

Photo Tourism: Main components

Rendering engine

Navigation interface

Tools for scene annotation

17

Transitions between photographs

18

Navigation interface

Object-based navigation

Stabilized slideshow

19

Scene annotation

Annotate regions

Register new photographs

20

Scene Visualization:

Path Finder: 3D navigation interface

21

Best possible controls

The goals of 3D navigation:

• Exploration• Search• Object inspection

Principles for 3D scene navigation:

• Controls are scene and task dependent

• Constrained and automatic navigation

• User’s position and motion should be clear

22

Pathfinder: Main components

Viewpoint scoring

Navigation controls for a scene

Rendering engine

User interface

25

Scene-specific controls: Panoramas and Orbits

Panoramas

Orbits

26

Path planning

High-quality rendering

Shortest path

Piecewise linear path

Path smoothing

Connectivity graph

29

Limitations

Non circular orbits

User needs

Appearance compensation

Larger scenes

(a) Pisa Doumo

(b) Stonehenge

30

Conclusion

Structure from motion

Photo Tourism - 3D photo browser

Path Finder - 3D navigation system

Reconstructing Rome

Capturing appearance

New photo-sharing community

Future work

31

Thank you