Embed Size (px)
Citation preview
Reconnaissance d’objetset vision artificielle
Jean Ponce ([email protected])http://www.di.ens.fr/~ponce
Equipe-projet WILLOWENS/INRIA/CNRS UMR 8548Laboratoire d’Informatique
Ecole Normale Supérieure, Paris
Outline
• Motivation: Making mosaics• Perspetive and weak perspective• Coordinates changes• Intrinsic and extrensic parameters• Affine registration• Projective registration
Feature-based alignment outline
Feature-based alignment outline
Extract features
Feature-based alignment outline
Extract features
Compute putative matches
Feature-based alignment outline
Extract features
Compute putative matches
Loop:• Hypothesize transformation T (small group of putative
matches that are related by T)
Feature-based alignment outline
Extract features
Compute putative matches
Loop:• Hypothesize transformation T (small group of putative
matches that are related by T)• Verify transformation (search for other matches consistent
with T)
Feature-based alignment outline
Extract features
Compute putative matches
Loop:• Hypothesize transformation T (small group of putative
matches that are related by T)• Verify transformation (search for other matches consistent
with T)
2D transformation models
Similarity(translation, scale, rotation)
Affine
Projective(homography)
Why these transformations ???
Pinhole perspective equation
z
yfy
z
xfx
''
''NOTE: z is always negative..
Affine models: Weak perspective projection
0
'where
'
'
z
fm
myy
mxx
is the magnification.
When the scene relief is small compared its distance from theCamera, m can be taken constant: weak perspective projection.
Affine models: Orthographic projection
yy
xx
'
' When the camera is at a(roughly constant) distancefrom the scene, take m=1.
Analytical camera geometry
Coordinate Changes: Pure Translations
OBP = OBOA + OAP , BP = AP + BOA
Coordinate Changes: Pure Rotations
BABABA
BABABA
BABABABAR
kkkjki
jkjjji
ikijii
...
...
...
TB
A
TB
A
TB
A
k
j
i
ABA
BA
B kji
Coordinate Changes: Rotations about
the z Axis
100
0cossin
0sincos
RBA
A rotation matrix is characterized by the following properties:
• Its inverse is equal to its transpose, and
• its determinant is equal to 1.
Or equivalently:
• Its rows (or columns) form a right-handedorthonormal coordinate system.
Coordinate changes: pure rotations
PRP
z
y
x
z
y
x
OP
ABA
B
B
B
B
BBBA
A
A
AAA
kjikji
Coordinate Changes: Rigid Transformations
ABAB
AB OPRP
11111
PT
OPRPORP ABA
ABAB
AA
TA
BBA
B
0
Pinhole perspective equation
z
yfy
z
xfx
''
''NOTE: z is always negative..
The intrinsic parameters of a camera
Normalized imagecoordinates
Physical image coordinates
Units:k,l : pixel/m
f : m
, a b : pixel
The intrinsic parameters of a camera
Calibration matrix
The perspectiveprojection equation
The extrinsic parameters of a camera
Perspective projections induce projective transformations between planes
Weak-perspective projection
Paraperspective projection
Affine cameras
Orthographic projection
Parallel projection
More affine cameras
Weak-perspective projection model
r
(p and P are in homogeneous coordinates)
p = A P + b (neither p nor P is in hom. coordinates)
p = M P (P is in homogeneous coordinates)
Affine projections induce affine transformations from planes onto their images.
Affine transformationsAn affine transformation maps a parallelogram onto
another parallelogram
11001
'
'
22221
11211
v
u
baa
baa
v
u
Fitting an affine transformationAssume we know the correspondences, how do we get
the transformation?
2
1
43
21
t
t
y
x
mm
mm
y
x
i
i
i
i
i
i
ii
ii
y
x
t
t
m
m
m
m
yx
yx
2
1
4
3
2
1
1000
0100
),( ii yx ),( ii yx
Fitting an affine transformation
Linear system with six unknowns
Each match gives us two linearly independent equations: need at least three to solve for the transformation parameters
i
i
ii
ii
y
x
t
t
m
m
m
m
yx
yx
2
1
4
3
2
1
1000
0100
Beyond affine transformationsWhat is the transformation between two views of a
planar surface?
What is the transformation between images from two cameras that share the same center?
Perspective projections induce projective transformations between planes
Beyond affine transformationsHomography: plane projective transformation
(transformation taking a quad to another arbitrary quad)
Fitting a homographyRecall: homogenenous coordinates
Converting to homogenenousimage coordinates
Converting from homogenenousimage coordinates
Fitting a homographyRecall: homogenenous coordinates
Equation for homography:
Converting to homogenenousimage coordinates
Converting from homogenenousimage coordinates
11 333231
232221
131211
y
x
hhh
hhh
hhh
y
x
Fitting a homographyEquation for homography:
iT
T
T
ii x
h
h
h
xHx
3
2
1
11 333231
232221
131211
i
i
i
i
y
x
hhh
hhh
hhh
y
x
0 ii xHx
iT
iiT
i
iT
iiT
iT
iT
i
ii
yx
x
y
xhxh
xhxh
xhxh
xHx
12
31
23
0
0
0
0
3
2
1
h
h
h
xx
xx
xx
TTii
Tii
Tii
TTi
Tii
Ti
T
xy
x
y
3 equations, only 2 linearly independent
9 entries, 8 degrees of freedom(scale is arbitrary)
Direct linear transform
H has 8 degrees of freedom (9 parameters, but scale is arbitrary)
One match gives us two linearly independent equationsFour matches needed for a minimal solution (null space
of 8x9 matrix)More than four: homogeneous least squares
0
0
0
0
0
3
2
1111
111
h
h
h
xx
xx
xx
xx
Tnn
TTn
Tnn
Tn
T
TTT
TTT
x
y
x
y
0hA
Application: Panorama stitching
Images courtesy of A. Zisserman.
