Reconnaissance d’objets et vision artificielle Jean Ponce (ponce@di.ens.fr) ponce Equipe-projet...

Reconnaissance d’objetset vision artificielle

Jean Ponce (ponce@di.ens.fr)http://www.di.ens.fr/~ponce

Equipe-projet WILLOWENS/INRIA/CNRS UMR 8548Laboratoire d’Informatique

Ecole Normale Supérieure, Paris

Outline

• Motivation: Making mosaics• Perspetive and weak perspective• Coordinates changes• Intrinsic and extrensic parameters• Affine registration• Projective registration

Feature-based alignment outline

Feature-based alignment outline

Extract features

Feature-based alignment outline

Extract features

Compute putative matches

Feature-based alignment outline

Extract features

Compute putative matches

Loop:• Hypothesize transformation T (small group of putative

matches that are related by T)

Feature-based alignment outline

Extract features

Compute putative matches

Loop:• Hypothesize transformation T (small group of putative

matches that are related by T)• Verify transformation (search for other matches consistent

with T)

Feature-based alignment outline

Extract features

Compute putative matches

Loop:• Hypothesize transformation T (small group of putative

matches that are related by T)• Verify transformation (search for other matches consistent

with T)

2D transformation models

Similarity(translation, scale, rotation)

Affine

Projective(homography)

Why these transformations ???

Pinhole perspective equation

z

yfy

z

xfx

''

''NOTE: z is always negative..

Affine models: Weak perspective projection

0

'where

'

'

z

fm

myy

mxx

is the magnification.

When the scene relief is small compared its distance from theCamera, m can be taken constant: weak perspective projection.

Affine models: Orthographic projection

yy

xx

'

' When the camera is at a(roughly constant) distancefrom the scene, take m=1.

Analytical camera geometry

Coordinate Changes: Pure Translations

OBP = OBOA + OAP , BP = AP + BOA

Coordinate Changes: Pure Rotations

BABABA

BABABA

BABABABAR

kkkjki

jkjjji

ikijii

...

...

...

TB

A

TB

A

TB

A

k

j

i

ABA

BA

B kji

Coordinate Changes: Rotations about

the z Axis

100

0cossin

0sincos

RBA

A rotation matrix is characterized by the following properties:

• Its inverse is equal to its transpose, and

• its determinant is equal to 1.

Or equivalently:

• Its rows (or columns) form a right-handedorthonormal coordinate system.

Coordinate changes: pure rotations

PRP

z

y

x

z

y

x

OP

ABA

B

B

B

B

BBBA

A

A

AAA

kjikji

Coordinate Changes: Rigid Transformations

ABAB

AB OPRP

11111

PT

OPRPORP ABA

ABAB

AA

TA

BBA

B

0

Pinhole perspective equation

z

yfy

z

xfx

''

''NOTE: z is always negative..

The intrinsic parameters of a camera

Normalized imagecoordinates

Physical image coordinates

Units:k,l : pixel/m

f : m

, a b : pixel

The intrinsic parameters of a camera

Calibration matrix

The perspectiveprojection equation

The extrinsic parameters of a camera

Perspective projections induce projective transformations between planes

Weak-perspective projection

Paraperspective projection

Affine cameras

Orthographic projection

Parallel projection

More affine cameras

Weak-perspective projection model

r

(p and P are in homogeneous coordinates)

p = A P + b (neither p nor P is in hom. coordinates)

p = M P (P is in homogeneous coordinates)

Affine projections induce affine transformations from planes onto their images.

Affine transformationsAn affine transformation maps a parallelogram onto

another parallelogram

11001

'

'

22221

11211

v

u

baa

baa

v

u

Fitting an affine transformationAssume we know the correspondences, how do we get

the transformation?

2

1

43

21

t

t

y

x

mm

mm

y

x

i

i

i

i

i

i

ii

ii

y

x

t

t

m

m

m

m

yx

yx

2

1

4

3

2

1

1000

0100

),( ii yx ),( ii yx

Fitting an affine transformation

Linear system with six unknowns

Each match gives us two linearly independent equations: need at least three to solve for the transformation parameters

i

i

ii

ii

y

x

t

t

m

m

m

m

yx

yx

2

1

4

3

2

1

1000

0100

Beyond affine transformationsWhat is the transformation between two views of a

planar surface?

What is the transformation between images from two cameras that share the same center?

Perspective projections induce projective transformations between planes

Beyond affine transformationsHomography: plane projective transformation

(transformation taking a quad to another arbitrary quad)

Fitting a homographyRecall: homogenenous coordinates

Converting to homogenenousimage coordinates

Converting from homogenenousimage coordinates

Fitting a homographyRecall: homogenenous coordinates

Equation for homography:

Converting to homogenenousimage coordinates

Converting from homogenenousimage coordinates

11 333231

232221

131211

y

x

hhh

hhh

hhh

y

x

Fitting a homographyEquation for homography:

iT

T

T

ii x

h

h

h

xHx

3

2

1

11 333231

232221

131211

i

i

i

i

y

x

hhh

hhh

hhh

y

x

0 ii xHx

iT

iiT

i

iT

iiT

iT

iT

i

ii

yx

x

y

xhxh

xhxh

xhxh

xHx

12

31

23

0

0

0

0

3

2

1

h

h

h

xx

xx

xx

TTii

Tii

Tii

TTi

Tii

Ti

T

xy

x

y

3 equations, only 2 linearly independent

9 entries, 8 degrees of freedom(scale is arbitrary)

Direct linear transform

H has 8 degrees of freedom (9 parameters, but scale is arbitrary)

One match gives us two linearly independent equationsFour matches needed for a minimal solution (null space

of 8x9 matrix)More than four: homogeneous least squares

0

0

0

0

0

3

2

1111

111

h

h

h

xx

xx

xx

xx

Tnn

TTn

Tnn

Tn

T

TTT

TTT

x

y

x

y

0hA

Application: Panorama stitching

Images courtesy of A. Zisserman.