Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab

Correspondence-Free Determinationof the Affine Fundamental Matrix

2007. 2. 6 (Tue)Young Ki Baik, Computer Vision Lab.

2

Correspondence-Free Determination of the Affine fundamental Matrix

• References• Correspondence-Free Determination of the

Affine Fundamental Matrix• Stefan Lehmann et. al. PAMI 2007

• Radon-based Structure from Motion Without Correspondences

• Ameesh Makadia et. al. CVPR 2005

• Robust Fundamental Matrix Determination without Correspondences

• Stefan Lehmann et. al. APRS 2005

3


• Contents• The conventional method of SfM

• Features of the proposed method

• Theory of the proposed algorithm

• Experimental results

• Discussion

4

• Conventional SfM

Image Sequence

Feature Extraction/ Matching

Relating Image

Projective Reconstructi

on

Auto-Calibration

Dense Matching

3D Model Building


5


• The Problem of conventional SfM

• The high sensitivity of fundamental matrix

• Noise and outlier correspondences in feature data severely affect the precision of the fundamental matrix

• Incomplete 3D reconstruction

6


• The Key Feature• Correspondence-free

• Finding Correspondence (X)

• Illumination changes-free (?)• Intensity value (X)• Position of features (O)

• Limitation• Occlusion ? (X)

• Affine camera only!!

7


• Parallel projection• Orthographic projection

8


• Mathematical Model• Assumption

• We have 3-dimensional N features. • The 3D feature space is represented by,

N

nnnn zzyyxxzyxf

13 ,,,,

locations feature Individual : ,, nnn zyx

function delta Dirac :

9


• Mathematical Model• Assumption

• Parallel projection model determines the 2D feature projections along the lines that are running parallel to the view axis (z-axis) of the camera.

• The model considers a continuous projection plane with infinite extent.

• The corresponding 2D projection data is…

N

nnn yyxxdzzyxfyxf

132 ,,,,

R

10


• Mathematical Model• Fourier spectra

• The Fourier spectra of and can be denoted as

N

n

zyxj

zyxj

N

n

yxjyxj

nnn

nn

e

dxdydzezyxfF

edxdyeyxfF

1

33

122

,,,,

, ,

3

2

R

R

yxf ,2

componentsfrequency 3D : ,,

zyxf ,,3

theoremslice-projection The

0,,, 32 FF

11


• Mathematical Model• 2-view case

• The 3D correspondence feature point

• Relation between images

• The 3D frequency vector

TT zyxzyx ,,,,, PP

matrixon translatiandrotation 3D eousnonhomogen:, tR

tRPP

T ,,Δ

12


• Mathematical Model• 2-view case

• Relation between 3D spectrums

ΔRΔ Δt Tj FeFT

33

ΔPΔTjeF 3

tRPP

The equation shows that rotation R also establishes the transformation between corresponding frequency indices in the 3D Fourier spaces of the original and the transformed spectrum or scene.

13


• Mathematical Model• Matching line

• The magnitudes of two spectra along these lines will be identical, while the phases will show a linear offset dependent upon the translational component of transformation.

0,,, 32 FF

• The proposed method is to detect these matching lines.

14


• Mathematical Model• Matching line angle pair

• Angle pair of the matching lines with respect to the axes of the frequency spectra F and F’, respectively.

,

,

15


• Mathematical Model• Analysis of the transformation parameters

• as the corresponding frequency locations along the matching lines of the spectrum F of the first and the spectrum F’ of the second set of 2D features, respectively.

• It follows that,

,,,

000 ,, zyxt ,, 2200 FeF yxj

16


• Mathematical Model• Analysis of the transformation parameters

000 ,, zyxt ,, 2200 FeF yxj

sin,cos

sin,cos

sin,cossin,cos 22 vvFevvF vj

sincos 00 yx

sin,cos21 vvFvF

17


• Mathematical Model• Derivation of a 3D rotation matrix

zxz RRRR

100

0cossin

0sincos

zR

cossin0

sincos0

001

xR

100

0cossin

0sincos

zR

angleunknown :

18


• Estimation of the fundamental matrix• By using 3D rotation matrix, we can obtain the

relation between 2D projection point (x’,y’) of a 3D feature (x,y,z) with translation.

0sinsin

coscossinsincos

sincossincoscos

xz

y

xx

0sincos

coscoscossinsin

sincoscoscossin

yz

y

xy

19


• Estimation of the fundamental matrix• In the orthographic projection case, all epipolar

lines are parallel.

• Then we can denote the epipolar line of 2D feature point (x,y) as

00]sincossinsin[

]coscoscossinsin

coscossinsincos[

]sincoscoscossin

sincossincoscos[

qypxzqp

yq

p

xq

pc

v)(x,on depends ccyqxp

20


• Estimation of the fundamental matrix

00]sincossinsin[

]coscoscossinsin

coscossinsincos[

]sincoscoscossin

sincossincoscos[

qypxzqp

yq

p

xq

pc

sin,cos qpcyqxp

yxyx sincossincos

21


• Estimation of the fundamental matrix

yxyx sincossincos

edc

b

a

F 00

00

0 edycxybxa

sincos

sin00

cos00

F ?,,

22


• Estimation of matching line angle• For the practical purpose, corresponding discrete

spectra should be defined as follows.

fkFc

fkFb

k

k

2

2

1

1 resolutionfrequency :f

resolutionfrequency circular :2 f

N

nnnk

N

nnnk

yxjkc

yxjkb

1

1

sincosexp

sincosexp

23


• Estimation of matching line angle• The final object function

• Discrete Fourier-Mellin transformation method• To find out the matching line (According to the well known shift theorem of the FT,

a shift in the space domain corresponds to a phase shift in the frequency domain.)

22

2

1

,,2 maxarg

cb

ecbd

N

k

jkkk

24


• Overall flow

25


• Experimental result• test images : telephoto lens

• Feature points : Harris corner detection method

• Ideal epipolar lines are the horizontal lines.

• The proposed method shows us good result relative to conventional methods.

26

Camera Calibration Methods for Wide Angle view

• Discussion• Key feature

• Correspondence-free method for obtaining the fundamental matrix is presented.

Matching line exists between the Fourier transformed data.

• Limitation• Proposed method

Considers only affine projection model Does not treat occlusion problem

• Future work• Applying projective projection model

Documents

Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab