1

3D reconstruction3D reconstruction from un from uncalibrated imagescalibrated images

Young Ki Baik

CV Lab

2

ContentsContents

Introduce Basic geometrical theory Overview – 3D reconstruction

Conditions for 3D reconstruction and Solution Correspondence Camera parameters and motion

Results Experimental results and demonstration

Future Works

3

Introduction(1)Introduction(1)

3D point

Camera

3D object

Camera

Mapping to images

mapping

Image plane

Camera system for obtaining images

4

Introduction(2)Introduction(2)

3D point

Camera

3D object

Camera

3D reconstruction from images Point correspondence

Camera parameter and motion

3D reconstruction system to make 3D object

5

3D reconstruction3D reconstruction from uncalibrated i from uncalibrated imagesmages Overview

Image Sequence

Feature Extraction/ Matching

Relating Image

Projective Reconstructi

on

Auto-Calibration

Dense Matching

3D Model Building

6

Conditions for Conditions for 3D 3D reconstructionreconstruction Correspondence

Feature extraction Harris corner method SIFT method

Scale Invariant Feature Transform

Initial feature matching Template matching (Image base descriptor) Descriptor (SIFT-d, PCA-d, SIFT-d+PCA-d, …)

Feature matching RANdom SAmple Consensus

To eliminate outlier

7

Conditions for Conditions for 3D 3D reconstructionreconstruction Correspondence

Guide matching To get more correspondence Using previous features and Geometry information

xxxxFxFx ,,,Cost SADdd T

Difference Absolute of Sum:) (

distanceEuclidean :) (

pair Matching:

matrix lFundamenta :

constant:,

SAD

d

xx,

F

Geometry based distance value

using fundamental

matrix

Correlation based cost

value

About 2 times more correspondence

8

Conditions for Conditions for 3D 3D reconstructionreconstruction Camera parameter and motion (Using Self-calibration)

Dual Absolute Conic Hartley ’94 / Hartley ’99, David Nistér IJCV 2004 ( + cheirality sol

ution )

Dual Absolute Quadric Triggs’97 M.Pollefeys et al. PAMI’98, ECCV 2002, IJCV 2004

Dual Absolute Quadric

M. Pollefeys

9

Conditions for Conditions for 3D 3D reconstructionreconstruction Constraints for self-calibration

Constant internal parameter Fixed camera

K1 = K2 = …

Known internal parameter Rectangular pixel : s = 0 Square pixel : s = 0, fx = fy

Principle point known : ( ux , uy ) = image center

1yy

xx

uf

usf

K

10

Experiments and resultsExperiments and results

Result using rig Rig

Calibration using vanishing point

DAQ (using weighted linear equation)

100

0.652765.700

5.187650.725 766.67

100

0.8635-752.2830

13.6090 752.283

100

0763.2190

00763.219

Using the calibration rig information

Using the manual

vanishing points input

Self-calibration result using rig correspondence

only

Self-calibration result is similar to the method using calibration rig.

11

Experiments and resultsExperiments and results

Real scene test Assuming that self-calibration works well

12

Experiments and resultsExperiments and results

Manual input to check self-calibration results Points : Correspondence information Line : Connection information

13

Experiments and resultsExperiments and results Test 1 (Pinball machine : 3 images)

Fig.1 Fig.2 Fig.3

5476 5609 8530

Fig.1-2 Fig.2-3 Fig.1-2-3

Initial

match

146 196 41

RANSAC 104 124

Guide

match

230 160 67

RANSAC 281 202

Key points

Match

14

Experiments and resultsExperiments and results Test 2 (Mask : 3 images)

Fig.1 Fig.2 Fig.3

1837 1420 1888

Fig.1-2 Fig.2-3 Fig.1-2-3

Initial

match

102 158 4

RANSAC 35 100

Guide

match

150 258 23

RANSAC 78 186

Key points

Match

15

Experiments and resultsExperiments and results Test 3 (Building : 6 images)

Fig.1 Fig.2 Fig.6

959 1064 1177

Fig.1-2 Fig.2-3 Fig.1~6

Initial

match

386 377 30

RANSAC 227 254

Guide

match

465 484 35

RANSAC 308 309

Key points

Match

16

Experiments and resultsExperiments and results Test 4 (House : 5 images)

Fig.1 Fig.2 Fig.5

3013 3084 2873

Fig.1-2 Fig.2-3 Fig.1~5

Initial

match

1023 973 15

RANSAC 656 716

Guide

match

1186 1216 54

RANSAC 911 909

Key points

Match

17

Future worksFuture works

Quasi-Dense matching technique and reconstruction To get more reliable results

Full side 3D reconstruction Using attaching algorithm

Bundle adjustment algorithm To reduce error

Full 3D reconstruction system Dense matching and 3D modeling