Download ppt - Robot Vision SS 2008 Matthias Rüther 1 ROBOT VISION Lesson 2: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2008 Matthias Rüther 1

ROBOT VISION Lesson 2: Projective Geometry

Matthias Rüther

Slides courtesy of Marc Pollefeys Department of Computer Science

University of North Carolina, Chapel Hill


Content

Linear Algebra: basic vector and matrix operations

Background: Projective geometry (2D, 3D), Parameter estimation, Algorithm evaluation.

Single View: Camera model, Calibration, Single View Geometry.

Two Views: Epipolar Geometry, 3D reconstruction, Computing F, Computing structure, Plane and homographies.

Three Views: Trifocal Tensor, Computing T.

More Views: N-Linearities, Multiple view reconstruction, Bundle adjustment, auto-calibration, Dynamic SfM, Cheirality, Duality


Vector Basics

nT

n

xxxv

x

x

x

v ...,... 21

2

1

vx2

x1

nnnn xx

xx

xx

vv

xx

xx

xx

vv

'

...

'

'

',

'

...

'

'

' 22

11

22

11

vv‘

v+v‘

n

ii

T xvvv1

2

wvyxyxyxwv Tnn ...2211

coswvwv v

w

Φ


Vector Spaces

Linear combination– Given vectors v1, ..., vk and scalars c1, ..., ck, the vector

v c1v1 c2v2 . . . ckvk

is obtained as a linear combination of the vectors v1, ..., vk

– Example: all vectors in R3 are linear combinations of i (1, 0, 0),

j (0, 1, 0), and k (0, 0, 1)

v (a, b, c) a(1, 0, 0) b(0, 1, 0) c(0, 0, 1)

Spanning– S=(v1, v2, . . . , vk ) spans a space W if every vector in W can be written as a

linear combination of the vectors in S

Linear Indepedence– v1, ..., vk is linearly indepedent if

c1v1 c2v2 . . . ck vk 0 implies c1 c2 . . . ck 0


Vector Spaces

Basis– S=(v1, ..., vk) is a basis for a vector space W if

• vj are linearly indepedent

• S spans W

! S need not be orthogonal !

Dimension– All bases for W have the same number of vectors.

– The dimension of W is the size of its sets of basis vectors.

Coordinates Relative to a Basis– If S=(v1, ..., vn) is a basis for a vector space W, then any v W has a

unique vector expansion:

nn

n

iii vxvxvxvxv

...22111


Example:– In R3 the vectors

relative to and

relative to are identical.

Vector Spaces

1

1

1

v

3

2

1

w

1

0

0

,

0

1

0

,

0

0

1

S

0

1

1

,

0

1

2

,

1

2

2

'S


Matrix Basics

Addition/SubtractionC A B implies cij aij bij

C A B implies cij aij bij

MultiplicationCmq = Amn*Bnq implies

Determinants– det(A)=0 A is singular A is not invertible

– det(A-I) is characteristic polynomial, roots are the eigenvalues

m

kkjikij bac

1

122122112221

1211 )det(, aaaaAaa

aaA


Matrix Basics

Inverse– AA1=A1A=I– Exists, iff det(A) != 0, A is not singular– Properties:

Pseudo-InverseA(AT A)1AT

A+A=I

RankMaximum number of linearly independent columns or rows.– If A is mn, rank(A) <= min(m,n)– If A is nn, rank(A) = n iff A nonsingular (invertible, det(A)!=0)– If A is nn, rank(A) < n iff A singular

TT AA

ABAB

AA

11

111

1

)(

)(

)det(

1)det(


Solving Linear Equation Systems

Common problem: solve the equation system Amnx=b

– If m>n, the system is over-determined

– If m<n, the system is under-determined

– If b=0, the system is homogeneous

– The system has either no solutions, exactly one or infinitely many

– Methods for solving Amnx=b include Gauss elimination, LU factorization, computing pseudo-inverse using SVD etc.

Conditions for solutions of Ax=b (A is square)– If A is invertible, Ax=b has exactly one solution.

– A is invertible Ax=0 has only trivial solution

– If rank(A|b)>rank(A), the system has no solution

– If rank(A)<n, and rank(A|b)=rank(A), the system has infinite number of solutions

– Ax=0 has non-trivial solutions iff rank(A)<n


Points, lines & conics

Transformations & invariants

1D projective geometry and

the Cross-ratio

Projective 2D Geometry


Euclidean 1D vector space:

– Geometric Primitives: points

– Representation: 1D vector: p = (x)

– Transformations:• Translation: p‘ = (x+t)

• Scaling: p‘ = (s*x)

• Translation and Scaling: p‘ = (s*x+t)

1D Projective Geometry

ℝ1

p1 p2 p3 p4x

p1 p2 p3 p4 x

p‘1 p‘2 p‘3 p‘4 x

p‘1 p‘2 p‘3 p‘4 x

dp1p2


1D Projective space:


ℙ1

p1 p2 p3 p4

x

w

W=1

Representation: 1D vector: p‘ = (x‘ w‘)T, where p = (x) = (x‘/w‘)

e.g. p‘ = (2 1)T = (4 2)T = (1 0,5)T are all equivalent to p = (2)



1D Projective space:

– Geometric Primitives: points

– Representation: 1D vector: p = (x w)T

– Transformations:• Translation: p‘ = T * p1DOF

• Scaling: p‘ = S * p 1DOF

• Translation and Scaling: p‘ = M * p, where M=S*T

2DOF

• Projective Mapping: p‘ = P * p, where P is a 2x2 Matrix 3DOF

10

1 tT

sS

/10

01

s

tM

/10

1



Translation:

x

w

W=1

p‘1 p‘2 p‘3 p‘4

pt

p

10

1'



Scaling:

x

w

W=1

p‘1 p‘2 p‘3 p‘4

ps

p

/10

01'



Projective Mapping:

x

w

W=1

p‘1p‘2

p‘3p‘4

pa

aap

1'

3

21

Analogy to central projective camera!



Invariants: – transformations form a hierarchy, some geometric features remain unchanged

under the transformations

– Translation (isometry)• Length, overall scale

– Translation and Scaling (similarity)• Ratio of lengths (d12 : d23)

– Projective Mapping (projectivity)• Cross ratio (ratio of ratios):

d12*d34 = d13*d24


Homogeneous coordinates

0 cbyax Ta,b,c

0,0)()( kkcykbxka TT a,b,cka,b,c ~

Homogeneous representation of lines

equivalence class of vectors, any vector is representative

Set of all equivalence classes in R3(0,0,0)T forms P2

Homogeneous representation of points

0 cbyax Ta,b,cl Tyx,x on if and only if

0l 11 x,y,a,b,cx,y, T 0,1,,~1,, kyxkyx TT

The point x lies on the line l if and only if xTl=lTx=0

Homogeneous coordinates

Inhomogeneous coordinates Tyx,

T321 ,, xxx but only 2DOF


Points from lines and vice-versa

l'lx

Intersections of lines

The intersection of two lines and is l l'

Line joining two points

The line through two points and is x'xl x x'

Example

1x

1y


Ideal points and the line at infinity

T0,,l'l ab

Intersections of parallel lines

TTand ',,l' ,,l cbacba

Example

1x 2x

Ideal points T0,, 21 xx

Line at infinity T1,0,0l

l22 RP

tangent vector

normal direction

ab , ba,

Note that in P2 there is no distinction

between ideal points and others


A model for the projective plane

exactly one line through two points

exaclty one point at intersection of two lines


Duality

x l

0xl T0lx T

l'lx x'xl

Duality principle:

To any theorem of 2-dimensional projective geometry there corresponds a dual theorem, which may be derived by interchanging the role of points and lines in the original theorem


Conics

Curve described by 2nd-degree equation in the plane

022 feydxcybxyax

0233231

2221

21 fxxexxdxcxxbxax

3

2

3

1 , xxyx

xx or homogenized

0xx CT

or in matrix form

fed

ecb

dba

2/2/

2/2/

2/2/

Cwith

fedcba :::::5DOF:


Five points define a conic

For each point the conic passes through

022 feydxcyybxax iiiiii

or

2 2, , , , ,1 0i i i i i ix x y y x y c Tfedcba ,,,,,c

0

1

1

1

1

1

552555

25

442444

24

332333

23

222222

22

112111

21

c

yxyyxx

yxyyxx

yxyyxx

yxyyxx

yxyyxx

stacking constraints yields


Tangent lines to conics

The line l tangent to C at point x on C is given by l=Cx

lx

C


Dual conics

0ll * CTA line tangent to the conic C satisfies

Dual conics = line conics = conic envelopes

1* CCIn general (C full rank):


Degenerate conics

A conic is degenerate if matrix C is not of full rank

TT mllm C

e.g. two lines (rank 2)

e.g. repeated line (rank 1)

TllC

l

l

m

Degenerate line conics: 2 points (rank 2), double point (rank1)

CC **Note that for degenerate conics


Projective transformations

A projectivity is an invertible mapping h from P2 to itself such that three points x1,x2,x3 lie on the same line if and

only if h(x1),h(x2),h(x3) do.

Definition:

A mapping h:P2P2 is a projectivity if and only if there exist a non-singular 3x3 matrix H such that for any point in P2 reprented by a vector x it is true that h(x)=Hx

Theorem:

Definition: Projective transformation

3

2

1

333231

232221

131211

3

2

1

'

'

'

x

x

x

hhh

hhh

hhh

x

x

x

xx' Hor

8DOF

projectivity=collineation=projective transformation=homography


Mapping between planes

central projection may be expressed by x’=Hx(application of theorem)


Removing projective distortion

333231

131211

3

1

'

''

hyhxh

hyhxh

x

xx

333231

232221

3

2

'

''

hyhxh

hyhxh

x

xy

131211333231' hyhxhhyhxhx 232221333231' hyhxhhyhxhy

select four points in a plane with known coordinates

(linear in hij)

(2 constraints/point, 8DOF 4 points needed)

Remark: no calibration at all necessary, better ways to compute (see later)


More examples


Transformation of lines and conics

Transformation for lines

ll' -TH

Transformation for conics-1-TCHHC '

Transformation for dual conicsTHHCC **'

xx' HFor a point transformation


A hierarchy of transformations

Projective linear group

Affine group (last row (0,0,1))

Euclidean group (upper left 2x2 orthogonal)

Oriented Euclidean group (upper left 2x2 det 1)

Alternative, characterize transformation in terms of elements or quantities that are preserved or invariant

e.g. Euclidean transformations leave distances unchanged


Class I: Isometries

(iso=same, metric=measure)

1100

cossin

sincos

1

'

'

y

x

t

t

y

x

y

x

1

11

orientation preserving:orientation reversing:

x0

xx'

1

tT

RHE IRR T

special cases: pure rotation, pure translation

3DOF (1 rotation, 2 translation)

Invariants: length, angle, area


Class II: Similarities

(isometry + scale)

1100

cossin

sincos

1

'

'

y

x

tss

tss

y

x

y

x

x0

xx'

1

tT

RH

sS IRR T

also know as equi-form (shape preserving)

metric structure = structure up to similarity (in literature)

4DOF (1 scale, 1 rotation, 2 translation)

Invariants: ratios of length, angle, ratios of areas, parallel lines


Class III: Affine transformations

11001

'

'

2221

1211

y

x

taa

taa

y

x

y

x

x0

xx'

1

tT

AH A

non-isotropic scaling! (2DOF: scale ratio and orientation)

6DOF (2 scale, 2 rotation, 2 translation)

Invariants: parallel lines, ratios of parallel lengths,

ratios of areas

DRRRA

2

1

0

0

D


Class VI: Projective transformations

xv

xx'

vP T

tAH

Action non-homogeneous over the plane

8DOF (2 scale, 2 rotation, 2 translation, 2 line at infinity)

Invariants: cross-ratio of four points on a line

(ratio of ratio)

T21,v vv


Action of affinities and projectivities on line at infinity

2211

2

1

2

1

0v

xvxvx

xx

x

vAA

T

t

000 2

1

2

1

x

xx

x

vAA

T

t

Line at infinity becomes finite,

allows to observe vanishing points, horizon,

Line at infinity stays at infinity,

but points move along line


Overview of transformations

1002221

1211

y

x

taa

taa

1002221

1211

y

x

tsrsr

tsrsr

333231

232221

131211

hhh

hhh

hhh

1002221

1211

y

x

trr

trr

Projective8dof

Affine6dof

Similarity4dof

Euclidean3dof

Concurrency, collinearity, order of contact (intersection, tangency, inflection, etc.), cross ratio

Parallellism, ratio of areas, ratio of lengths on parallel lines (e.g midpoints), linear combinations of vectors (centroids). The line at infinity l∞

Ratios of lengths, angles.The circular points I,J

lengths, areas.