Upload
zukun
View
634
Download
0
Tags:
Embed Size (px)
Citation preview
CSE486, Penn StateRobert Collins
Lecture 16:
Planar Homographies
CSE486, Penn StateRobert Collins Motivation:
Points on Planar Surface
x
y
CSE486, Penn StateRobert Collins
Review : Forward Projection
UVW
XYZ
xy
uv
WorldCoords
CameraCoords
FilmCoords
PixelCoords
Mext Mproj Maff
MintUVW
XYZ
uv
MUVW
uv
34
24
14
33
23
13
31
22
12
31
21
11
m
m
m
m
m
m
m
m
m
m
m
m
CSE486, Penn StateRobert Collins
World to Camera Transformation
XX
YY
ZZ
PC
UU
VV
WWPW
Translate by - C (align origins)
CRotate toalign axes
R
PC = R ( PW - C ) = R PW + T
CSE486, Penn StateRobert Collins
Perspective Matrix Equation(Camera Coordinates)
PM p ⋅= int
10
0
0
1
0
0
0
0
0
0
'
'
'
=
Z
Y
X
f
f
z
y
x
C
Z
Yfy
Z
Xfx
=
=
CSE486, Penn StateRobert Collins
Film to Pixel Coords
10
0
0
1
0
0
0
0
0
0
'
'
'
=
Z
Y
X
f
f
z
y
x
100
a13a12a11
a23a22a21
w’
v’
u’
2D affine transformation from filmcoords (x,y) to pixel coordinates (u,v):
u = Mint PC = Maff Mproj PC
MprojMaff
CSE486, Penn StateRobert Collins
Projection of Points on Planar Surface
x
y
Point on plane
Rotation +Translation
Perspectiveprojection
Film coordinates
CSE486, Penn StateRobert Collins
Projection of Planar Points
CSE486, Penn StateRobert Collins
Projection of Planar Points (cont)
Homography H(planar projectivetransformation)
CSE486, Penn StateRobert Collins
Projection of Planar Points (cont)
Homography H(planar projectivetransformation)
Punchline: For planar surfaces, 3D to 2D perspectiveprojection reduces to a 2D to 2D transformation.
Punchline2: This transformation is INVERTIBLE!
CSE486, Penn StateRobert Collins
Special Case : Frontal Plane
What if the planar surface is perpendicular tothe optic axis (Z axis of camera coord system)?
Then world rotation matrix simplies:
CSE486, Penn StateRobert Collins
Frontal Plane
So the homography for a frontal plane simplies:
Similarity Transformation!
CSE486, Penn StateRobert Collins
Convert to Pixel Coords
u
v
100
a13a12a11a23a22a21
pixelsInternal cameraparameters
CSE486, Penn StateRobert Collins
Planar Projection Diagram
Here’s wheretransformationgroups get useful!
CSE486, Penn StateRobert Collins
General Planar Projection
H1
A1
H
CSE486, Penn StateRobert Collins
General Planar Projection
H1
A1
H-1
CSE486, Penn StateRobert Collins
Frontal Plane Projection
S1
A1
A
CSE486, Penn StateRobert Collins
Frontal Plane Projection
S1
A1
A-1
CSE486, Penn StateRobert Collins
General Planar Projection
H2H1
H
CSE486, Penn StateRobert Collins
Summary: Planar Projection
x
y
Point on planeRotation + Translation
Perspectiveprojection
Pixel coordsu
v
Internalparams
Homography
CSE486, Penn StateRobert Collins Applying Homographies to Remove
Perspective Distortion
from Hartley & Zisserman
4 point correspondences suffice forthe planar building facade
CSE486, Penn StateRobert Collins Homographies for
Bird’s-eye Views
from Hartley & Zisserman
CSE486, Penn StateRobert Collins
Homographies for Mosaicing
from Hartley & Zisserman
CSE486, Penn StateRobert Collins
Two Practical Issues
How to estimate the homography given four or more point correspondences (will derive L.S. solution now)
How to (un)warp image pixel values to produce a new picture (last class)
CSE486, Penn StateRobert Collins
Estimating a Homography
Matrix Form:
Equations:
CSE486, Penn StateRobert Collins
Degrees of Freedom?
There are 9 numbers h11,…,h33 , so are there 9 DOF?
No. Note that we can multiply all hij by nonzero kwithout changing the equations:
CSE486, Penn StateRobert Collins
Enforcing 8 DOFOne approach: Set h33 = 1.
Second approach: Impose unit vector constraint
Subject to theconstraint:
CSE486, Penn StateRobert Collins
L.S. using Algebraic Distance
Setting h33 = 1
Multiplying through by denominator
Rearrange
CSE486, Penn StateRobert Collins
Algebraic Distance, h33=1 (cont)
Point 1
additionalpoints
2N x 8 8 x 1 2N x 1
Point 4
Point 3
Point 2
CSE486, Penn StateRobert Collins
Algebraic Distance, h33=1 (cont)
A h = b2Nx8 8x1 2Nx1Linear
equations
Solve: AT A h = AT b 2Nx8 8x1 2Nx18x2N 8x2N
(AT A) h = (AT b)8x18x8 8x1
h = (AT A) (AT b)-1
Matlab: h = A \ b
CSE486, Penn StateRobert Collins
Caution: Numeric Conditioning
R.Hartley: “In Defense of the Eight Point Algorithm”
Observation: Linear estimation of projective transformation parameters from point correspondences often suffer from poor“conditioning” of the matrices involves. This means the solutionis sensitive to noise in the points (even if there are no outliers).
To get better answers, precondition the matrices by performinga normalization of each point set by:• translating center of mass to the origin • scaling so that average distance of points from origin is sqrt(2).• do this normalization to each point set independently
CSE486, Penn StateRobert Collins
Hartley’s PreConditioning
H?PointSet1 PointSet2
Scale so averagepoint dist is sqrt(2)S1 S2
Translatecenter of massto origin
T1 T2
Estimate Hnorm
H = T2-1 S2
-1 Hnorm S1 T1
CSE486, Penn StateRobert Collins
A More General Approach
What might be wrong with setting h33 = 1?
If h33 actually = 0, we can’t get the right answer.
CSE486, Penn StateRobert Collins
Algebraic Distance, ||h||=1
||h|| = 1
Multiplying through by denominator
Rearrange
= 0
= 0
CSE486, Penn StateRobert Collins
Algebraic Distance, ||h||=1 (cont)
additionalpoints
2N x 9 9 x 1 2N x 1
4 POINTS
CSE486, Penn StateRobert Collins
Algebraic Distance, ||h||=1 (cont)
A h = 02Nx9 9x1 2Nx1Homogeneous
equations
Solve:
SVD of ATA = U D UT
Let h be the column of U (unit eigenvector) associated with the smallest eigenvalue in D. (if only 4 points, that eigenvalue will be 0)
AT A h = AT 02Nx9 9x1 2Nx19x2N 9x2N
(AT A) h = 09x19x9 9x1