30
Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer Science University of North Carolina, Chapel Hill

Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Embed Size (px)

Citation preview

Page 1: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 1

ROBOT VISION Lesson 3: Projective Geometry

Matthias Rüther

Slides courtesy of Marc Pollefeys Department of Computer Science

University of North Carolina, Chapel Hill

Page 2: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 2

Content

Projective geometry in 3D– Points

– Planes

– Lines

– Points from planes, planes from points

– Hierarchy of transformations / invariants

Parameter Estimation– Problem: intersecting lines -> SVD

– Problem: rectification

– Normalization

– Robust Estimation

Algorithm Evaluation– Error measures

Page 3: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 3

Projective 3D Geometry

Points, lines, planes and quadrics

Transformations

Page 4: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 4

3D points

TT

1 ,,,1,,,X4

3

4

2

4

1 ZYXX

X

X

X

X

X

in R3

04 X

TZYX ,,

in P3

XX' H (4x4-1=15 dof)

projective transformation

3D point

T4321 ,,,X XXXX

Page 5: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 5

Planes

0ππππ 4321 ZYX

0ππππ 44332211 XXXX

0Xπ T

Dual: points ↔ planes, lines ↔ lines

3D plane

0X~

.n d T321 π,π,πn TZYX ,,X~

14 Xd4π

Euclidean representation

n/d

XX' Hππ' -TH

Transformation

Page 6: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 6

Planes from points

X

X

X

3

2

1

T

T

T

2341DX T123124134234 ,,,π DDDD

0det

4342414

3332313

2322212

1312111

XXXX

XXXX

XXXX

XXXX

0πX 0πX 0,πX π 321 TTT andfromSolve

(solve as right nullspace of )π

T

T

T

3

2

1

X

X

X

0XXX Xdet 321

Or implicitly from coplanarity condition

124313422341 DXDXDX 01234124313422341 DXDXDXDX 13422341 DXDX

Page 7: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 7

Points from planes

0X

π

π

π

3

2

1

T

T

T

xX M 321 XXXM

0π MT

I

pM

Tdcba ,,,π T

a

d

a

c

a

bp ,,

0Xπ 0Xπ 0,Xπ X 321 TTT andfromSolve

(solve as right nullspace of )X

T

T

T

3

2

1

π

π

π

Representing a plane by its span

Page 8: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 8

Lines

T

T

B

AW μBλA

T

T

Q

PW* μQλP

22** 0WWWW TT

0001

1000W

0010

0100W*

Example: X-axis

(4dof)

Page 9: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 9

Points, lines and planes

TX

WM 0π M

W*

M 0X M

W

X

*W

π

Page 10: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 10

Plücker matrices

jijiij ABBAl TT BAABL

Plücker matrix (4x4 skew-symmetric homogeneous matrix)

1. L has rank 22. 4dof3. generalization of4. L independent of choice A and B5. Transformation

24* 0LW T

yxl

THLHL'

0001

0000

0000

1000

1000

0

0

0

1

0001

1

0

0

0

L T

Example: x-axis

Page 11: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 11

Plücker matrices

TT QPPQL*

Dual Plücker matrix L*

-1TLHHL -'*

*12

*13

*14

*23

*42

*34344223141312 :::::::::: llllllllllll

XLπ *

LπX

Correspondence

Join and incidence

0XL*

(plane through point and line)

(point on line)

(intersection point of plane and line)

(line in plane)0Lπ

0π,L,L 21 (coplanar lines)

Page 12: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 12

Plücker line coordinates

TL 344223141312 ,,,,, llllll 5P

0

000001

000010

000100

001000

010000

100000

34

42

23

14

13

12

344223141312

l

l

l

l

l

l

llllll

B,A,BA,ˆ, LL

LLLLT ˆ|Kˆ

ˆˆˆˆˆˆB,AB,A,det 123413421423231442133412

llllllllllll

0231442133412 llllll on Klein quadric 0K LLT

Page 13: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 13

Plücker line coordinates

0B,AB,A,detˆ| LL

0Q,PQ,P,detˆ| LL

0BPAQBQAPˆ| TTTTLL

0| LL (Plücker internal constraint)

(two lines intersect)

(two lines intersect)

(two lines intersect)

Page 14: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 14

Estimation

2D Problem: Lines l1, l2, l3 intersect in a point

l1

l2

l3

?? Point of intersection xx

13333

2

1

3

2

1

0

0

0

0

0

0

xx

T

T

T

T

T

T

x

l

l

l

xl

xl

xl

How many solutions exist?

When does a solution exist?

Geometric interpretation?

Solve equation system:

Page 15: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 15

Estimation

Practical 2D Problem: measured lines l1, l2, l3, … lm approximately intersect in a point x

l1

l2

l3

?? Point of intersection xx

Find a solution that minimizes |Lx|.

lm

Find an approximate solution to the system:

13

2

1

0...

00

...

mxmx

Tm

T

T

Lxx

l

ll

Page 16: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 16

Singular Value Decomposition (SVD)

SVD: algorithm that decomposes Amxn (mn) to

Tnxnmxnmxn nxn

VDUA

Properties:

– U has orthogonal columns: UTU = I– U is norm-preserving: |Ux| = |x|– D is diagonal, diagonal elements are the singular values– V is orthogonal: VTV = I

Page 17: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 17

Singular Value Decomposition (SVD)

Solve Ax=0 minimize |Ax| subject to |x|=1:

xVxxDVxUDV

xUDVAx

TTT

T

,

minmin

minimize |DVTx| subject to |VTx|=1

TT xV 0...0,1,0,...,0

1 is in the same row as the smallest singular value in D.X is the column of V corresponding to the smallest Singular Value.

Page 18: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 18

Robust Estimation

Practical 2D Problem: SVD obtains an optimal least squares solution:

l1

l2

l3

x

Problem with outliers:

l1

l2

l3

x

l4

Page 19: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 19

RANdom SAmple Consensus

RANSAC algorithm: given a set of lines l1,…,lm, find an intersection point x such that |lkx| is minimal for k=1..m, subject to the condition that none of the valid lines deviates from x by more than t.

l1

l2

l3

x

l4

t

lm

Page 20: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 20

RANSAC

ObjectiveRobust fit of model to data set S which contains outliers

Algorithm

(i) Randomly select a sample of s data points from S and instantiate the model from this subset.

(ii) Determine the set of data points Si which are within a distance threshold t of the model. The set Si is the consensus set of samples and defines the inliers of S.

(iii) If the subset of Si is greater than some threshold T, re-estimate the model using all the points in Si and terminate

(iv) If the size of Si is less than T, select a new subset and repeat the above.

(v) After N trials the largest consensus set Si is selected, and the model is re-estimated using all the points in the subset Si

Page 21: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 21

Parameter estimation

2D homographyGiven a set of (xi,xi’), compute H (xi’=Hxi)

3D to 2D camera projectionGiven a set of (Xi,xi), compute P (xi=PXi)

Fundamental matrixGiven a set of (xi,xi’), compute F (xi’TFxi=0

x1

x2

x3

x4

x1‘ x2‘

x3‘x4‘?H?

Page 22: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 22

Number of measurements required

At least as many independent equations as degrees of freedom required

Example:

Hxx'

11

λ

333231

232221

131211

y

x

hhh

hhh

hhh

y

x

2 independent equations / point8 degrees of freedom

4x2≥8

Page 23: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 23

Approximate solutions

Minimal solution4 points yield an exact solution for H

More points– No exact solution, because measurements are inexact (“noise”)

– Search for “best” according to some cost function

– Algebraic or geometric/statistical cost

Page 24: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 24

Gold Standard algorithm

Cost function that is optimal for some assumptions

Computational algorithm that minimizes it is called “Gold Standard” algorithm

Other algorithms can then be compared to it

Page 25: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 25

Direct Linear Transformation(DLT)

ii Hxx 0Hxx ii

i

i

i

i

xh

xh

xh

Hx3

2

1

T

T

T

iiii

iiii

iiii

ii

yx

xw

wy

xhxh

xhxh

xhxh

Hxx12

31

23

TT

TT

TT

0

h

h

h

0xx

x0x

xx0

3

2

1

TTT

TTT

TTT

iiii

iiii

iiii

xy

xw

yw

Tiiii wyx ,,x

0hA i

Page 26: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 26

Direct Linear Transformation(DLT)

Equations are linear in h

0

h

h

h

0xx

x0x

xx0

3

2

1

TTT

TTT

TTT

iiii

iiii

iiii

xy

xw

yw

0hA i Only 2 out of 3 are linearly independent

(indeed, 2 eq/pt)

0

h

h

h

x0x

xx0

3

2

1

TTT

TTT

iiii

iiii

xw

yw

(only drop third row if wi’≠0)

Holds for any homogeneous representation, e.g. (xi’,yi’,1)

Page 27: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 27

Direct Linear Transformation(DLT)

Solving for H

0Ah 0h

A

A

A

A

4

3

2

1

size A is 8x9 or 12x9, but rank 8

Trivial solution is h=09T is not interesting

1-D null-space yields solution of interestpick for example the one with 1h

Page 28: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 28

Direct Linear Transformation(DLT)

Over-determined solution

No exact solution because of inexact measurementi.e. “noise”

0Ah 0h

A

A

A

n

2

1

Find approximate solution- Additional constraint needed to avoid 0, e.g.

- not possible, so minimize

1h Ah0Ah

Page 29: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 29

DLT algorithm

ObjectiveGiven n≥4 2D to 2D point correspondences {xi↔xi’}, determine the 2D homography matrix H such that xi’=Hxi

Algorithm

(i) For each correspondence xi ↔xi’ compute Ai. Usually only two first rows needed.

(ii) Assemble n 2x9 matrices Ai into a single 2nx9 matrix A

(iii) Obtain SVD of A. Solution for h is last column of V

(iv) Determine H from h

Page 30: Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer

Robot Vision SS 2005 Matthias Rüther 30

Inhomogeneous solution

'

'h~

''000'''

'''''000

ii

ii

iiiiiiiiii

iiiiiiiiii

xw

yw

xyxxwwwywx

yyyxwwwywx

Since h can only be computed up to scale, pick hj=1, e.g. h9=1, and solve for 8-vector

h~

Solve using Gaussian elimination (4 points) or using linear least-squares (more than 4 points)

However, if h9=0 this approach fails also poor results if h9 close to zeroTherefore, not recommendedNote h9=H33=0 if origin is mapped to infinity

0

1

0

0

H100Hxl 0

T