Embed Size (px)
Citation preview
Lecture 8 - !!!
Fei-Fei Li!
Lecture 8: Camera Calibra0on
Professor Fei-‐Fei Li Stanford Vision Lab
2-‐Oct-‐12 1
Lecture 8 - !!!
Fei-Fei Li!
What we will learn today?
• Review camera parameters • Affine camera model (Problem Set 2 (Q4)) • Camera calibra0on • Vanishing points and lines (Problem Set 2 (Q1))
2-‐Oct-‐12 2
Reading: • [FP] Chapter 3 • [HZ] Chapter 7, 8.6
Lecture 8 - !!!
Fei-Fei Li!
What we will learn today?
• Review camera parameters • Affine camera model • Camera calibra0on • Vanishing points and lines
2-‐Oct-‐12 3
Reading: • [FP] Chapter 3 • [HZ] Chapter 7, 8.6
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 4
Projec've camera f
Oc
f = focal length
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 5
Projec've camera
x
y
xc
yc
C=[uo, vo]
f
Oc
f = focal length uo, vo = offset
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 6
Projec've camera f
Oc
Units: k,l [pixel/m] f [m]
[pixel] ,α βNon-‐square pixels
f = focal length uo, vo = offset
→ non-‐square pixels ,α β
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 7
Projec've camera
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
10100000
zyx
vus
P' o
o
β
α
f
Oc
K has 5 degrees of freedom!
Pc
P’
f = focal length uo, vo = offset
→ non-‐square pixels ,α βθ = skew angle
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 8
Projec've camera f
Oc
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡ −
=ʹ′
1zyx
01000v0
0ucot
P o
o
sinθβ
θαα
Pc
P’
f = focal length uo, vo = offset
→ non-‐square pixels ,α βθ = skew angle K has 5 degrees of freedom!
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 9
Projec've camera f
Oc
Pc
Ow
iw
kw
jw R,T
P’
f = focal length uo, vo = offset
→ non-‐square pixels ,α βθ = skew angle R,T = rota0on, transla0on
wPTR
P4410×
⎥⎦
⎤⎢⎣
⎡=
cORT ~−=
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 10
Projec've camera
f = focal length uo, vo = offset
→ non-‐square pixels ,α β
f
Oc
P
Ow
iw
kw
jw
wPMP =ʹ′
[ ] wPTRK=Internal (intrinsic) parameters
External (extrinsic) parameters
θ = skew angle R,T = rota0on, transla0on
P’
R,T
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 11
Projec've camera
wPMP =ʹ′ [ ] wPTRK=Internal (intrinsic) parameters
External (extrinsic) parameters
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 12
Projec've camera
wPMP =ʹ′ [ ] wPTRK=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡ −
=
100v0
ucot
K o
o
sinθβ
θαα
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=T3
T2
T1
Rrrr
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
z
y
x
ttt
T
43×
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 13
Goal of calibra'on
wPMP =ʹ′ [ ] wPTRK=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡ −
=
100v0
ucot
K o
o
sinθβ
θαα
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=T3
T2
T1
Rrrr
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
z
y
x
ttt
T
43×
Es0mate intrinsic and extrinsic parameters from 1 or mul0ple images
Lecture 8 - !!!
Fei-Fei Li!
What we will learn today?
• Review camera parameters • Affine camera model (Problem Set 2 (Q4)) • Camera calibra0on • Vanishing points and lines
2-‐Oct-‐12 14
Reading: • [FP] Chapter 3 • [HZ] Chapter 7, 8.6
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 15
Weak perspec've projec'on
Rela0ve scene depth is small compared to its distance from the camera
= magnifica0on ⎩⎨⎧
−=
−=
myymxx
''
0
'wherezfm −=
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 16
Orthographic (affine) projec'on
Distance from center of projec0on to image plane is infinite
⎩⎨⎧
=
=
y'yx'x
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 17
Affine cameras
[ ]PTRKP ='
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
100000s
K y
x
α
α
⎥⎦
⎤⎢⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=10TR
100000100001
KM
Affine case
Parallel projec0on matrix
⎥⎦
⎤⎢⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=10TR
010000100001
KM⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
100y0xs
K oy
ox
α
αProjec0ve case Compared to
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 18
Remember….
Projec0vi0es: ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎦
⎤⎢⎣
⎡=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
1yx
H1yx
bvtA
1'y'x
p
Affini0es: ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎦
⎤⎢⎣
⎡=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
1yx
H1yx
10tA
1'y'x
a
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 19
[ ]PTRKP ='
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
1000000
y
x
K α
α
⎥⎦
⎤⎢⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=10TR
100000100001
KM
⎥⎦
⎤⎢⎣
⎡=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=×
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
×=10bA
1000]affine44[
100000100001
]affine33[ 2232221
1131211
baaabaaa
M
⎥⎦
⎤⎢⎣
⎡=+=⎟⎟
⎠
⎞⎜⎜⎝
⎛+⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
⎥⎦
⎤⎢⎣
⎡=⎟⎟
⎠
⎞⎜⎜⎝
⎛=
1'
2
1
232221
131211 PMP
bb
ZYX
aaaaaa
yx
P EucbA
[ ]bAMMEuc ==
We can obtain a more compact formula0on than:
Affine cameras
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 20
Affine cameras
P P’
P’
;1
' ⎥⎦
⎤⎢⎣
⎡=+=⎟⎟
⎠
⎞⎜⎜⎝
⎛=
PbAP M
vu
P [ ]bAM =
M = camera matrix
[non-‐homogeneous image coordinates]
To recap:
This nota0on is useful when we’ll discuss affine structure from mo0on
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 21
Affine cameras
• Weak perspec0ve much simpler math. – Accurate when object is small and distant. – Most useful for recogni0on.
• Pinhole perspec0ve much more accurate for scenes. – Used in structure from mo0on.
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 22
The Kangxi Emperor's Southern Inspec7on Tour (1691-‐1698) By Wang Hui You tube video – click here
Weak perspec've projec'on -‐ examples
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 23
Weak perspec've projec'on -‐ examples
Qingming Fes7val by the Riverside Zhang Zeduan ~900 AD
Lecture 8 - !!!
Fei-Fei Li!
What we will learn today?
• Review camera parameters • Affine camera model • Camera calibra0on • Vanishing points and lines
2-‐Oct-‐12 24
Reading: • [FP] Chapter 3 • [HZ] Chapter 7, 8.6
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 25
Calibra'on Problem
• P1… Pn with known posi0ons in [Ow, iw, jw, kw] • p1, … pn known posi0ons in the image Goal: compute intrinsic and extrinsic parameters
jC
Calibra0on rig
Lecture 8 - !!!
Fei-Fei Li!
Remember the “digital Michelangelo project”?
2-‐Oct-‐12 26
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 27
Calibra'on Problem
jC
Calibra0on rig
How many correspondences do we need? • M has 11 unknown • We need 11 equa0ons • 6 correspondences would do it
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 28
Calibra'on Problem
image jC
Calibra0on rig
In prac0ce: user may need to look at the image and select the n>=6 correspondences
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 29
Calibra'on Problem
jC
ii PMp → ⎥⎦
⎤⎢⎣
⎡=→
i
ii v
up
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
3
2
1
Mmmm
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
⋅
⋅⋅
⋅
=
i
i
i
i
PPPP
3
2
3
1
mmmm
in pixels
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 30
Calibra'on Problem
i3
i1i P
Pumm
=
i2i3i P)P(v mm =→
i1i3i P)P(u mm =→
i3
i2i P
Pvmm
=
⎥⎦
⎤⎢⎣
⎡
i
i
vu
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
=
i3
i2
i3
i1
PPPP
mmmm
0)( 23 =−→ iii PPv mm
0)( 13 =−→ iii PPu mm
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 31
Calibra'on Problem
…
…
0)( 12131 =− PPv mm
0)( 11131 =− PPu mm
0)( 23 =− iii PPv mm
0)( 13 =− iii PPu mm
0)( 23 =− nnn PPv mm
0)( 13 =− nnn PPu mm
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 32
Block Matrix Mul0plica0on
⎥⎦
⎤⎢⎣
⎡=⎥
⎦
⎤⎢⎣
⎡=
2221
1211
2221
1211
BBBB
BAAAA
A
What is AB ?
⎥⎦
⎤⎢⎣
⎡
++
++=
2222122121221121
2212121121121111
BABABABABABABABA
AB
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 33
Calibra'on Problem
2n x 12 12x1
1x4
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
=T3
T2
T1def
mmm
m
4x1
…
Homogenous linear system
known unknown
0)( 12131 =+− PPv mm
0)( 11131 =+− PPu mm
0)( 23 =+− nnn PPv mm
0)( 13 =+− nnn PPu mm
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 34
Homogeneous M x N Linear Systems
A x 0 =
Rectangular system (M>N)
• 0 is always a solu0on
Minimize |Ax|2
under the constraint |x|2 =1
M=number of equa0ons N=number of unknown
• To find non-‐zero solu0on
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 35
How do we solve this homogenous linear system?
Calibra'on Problem
Singular Value Decomposi0on (SVD)
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 36
Calibra'on Problem
1212T
121212n2 VDU ×××
Last column of V gives m
M iPM ip→
Compute SVD decomposi0on of P
Why? See page 593 of Hartley & Zisserman
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 37
Extrac'ng camera parameters
A
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=T3
T2
T1
Aaaa
[ ]TRK=
3
1a±
=ρ
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
3
2
1
bbb
b
Es0mated values
)(u 212
o aa ⋅= ρ)(v 32
2o aa ⋅= ρ
( ) ( )3231
3231cosaaaaaaaa
×⋅×
×⋅×=θ
Intrinsic
b ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡ −
=
100v0
ucot
K o
o
sinθβ
θαα
ρ
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 38
Theorem (Faugeras, 1993) [ ] [ ] ][ bATKRKTRKM ===
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
3
2
1
aaa
A
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
1000 y
x
ccs
K β
α
lf;kf
=
=
β
α
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 39
Extrac'ng camera parameters
A
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=T3
T2
T1
Aaaa
[ ]TRK=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
3
2
1
bbb
b
Es0mated values
Intrinsic θρα sin31
2 aa ×=
θρβ sin322 aa ×=
b
f
ρ
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 40
Extrac'ng camera parameters
Extrinsic ( )
32
321 aa
aar×
×=
33
1a
r ±=
132 rrr ×= b1KT −= ρ
A
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=T3
T2
T1
Aaaa
[ ]TRK=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
3
2
1
bbb
b
Es0mated values
bρ
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 41
Calibra'on Demo Camera Calibra7on Toolbox for Matlab J. Bouguet – [1998-‐2000]
hxp://www.vision.caltech.edu/bouguetj/calib_doc/index.html#examples
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 42
Calibra'on Demo
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 43
Calibra'on Demo
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 44
Calibra'on Demo
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 45
Calibra'on Demo
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 46
Calibra'on Demo
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 47
Calibra'on Demo
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 48
Calibra'on Demo
Lecture 8 - !!!
Fei-Fei Li!
What we will learn today?
• Review camera parameters • Affine camera model • Camera calibra0on • Vanishing points and lines (Problem Set 2 (Q1))
2-‐Oct-‐12 49
Reading: • [FP] Chapter 3 • [HZ] Chapter 7, 8.6
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 50
Proper'es of Projec'on • Points project to points • Lines project to lines
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 51
Proper'es of Projec'on Vanishing point • Angles are not preserved
• Parallel lines meet
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 52
Lines in a 2D plane
0cbyax =++
-c/b
-a/b
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
cba
l
If x = [ x1, x2]T ∈ l 0cba
1xx T
2
1
=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
l
x
y
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 53
Lines in a 2D plane
Intersec0ng lines
llx ʹ′×= l
lʹ′
Proof
lll ⊥ʹ′×lll ʹ′⊥ʹ′×
0l)ll( =⋅ʹ′×→0l)ll( =ʹ′⋅ʹ′×→
lx∈→
x
lx ʹ′∈→
→ x is the intersec0ng point
x
y
x
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 54
Points at infinity (ideal points)
0x,xxx
x 3
3
2
1
≠
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
cba
l
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
ʹ′
=ʹ′
cba
l
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−ʹ′−=ʹ′×→
0ab
)cc(llLet’s intersect two parallel lines:
Agree with the general idea of two lines intersec0ng at infinity
l
lʹ′
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=∞
02
1
xx
x
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 55
Lines at infinity ∞lSet of ideal points lies on a line called the line at infinity How does it look like?
∞l
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=∞
100
l
T
2
1
0xx
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡0
100=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡Indeed:
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 56
Projec've projec'ons of lines at infinity (2D)
lHl T−=ʹ′
⎥⎦
⎤⎢⎣
⎡=
bvtA
H
?lH T =∞−
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎦
⎤⎢⎣
⎡=
−
btt
100
bvtA
y
xT
is it a line at infinity?
…no!
?lH TA =∞−
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎦
⎤⎢⎣
⎡
−=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎦
⎤⎢⎣
⎡=
−
−−
100
100
10
100
10 TT
TT
AtAtA
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 57
horizon
-‐ Recognize the horizon line -‐ Measure if the 2 lines meet at the horizon -‐ if yes, these 2 lines are //
• Recogni0on helps reconstruc0on! • Humans have learnt this
Are these two lines parallel or not?
∞−= lHl T
hor
Projec've projec'ons of lines at infinity (2D)
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 58
Vanishing points (= ideal points in 2D)
[ ]TRKM =
dv K=
d=direc0on of the line
d
C
v Vanishing points = points where parallel lines intersect in 3D
Image of a vanishing point =
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 59
Horizon
• Sets of parallel lines on the same plane lead to collinear vanishing points [The line is called the horizon for that plane]
horizon
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 60
C
n
horizTK ln =
horizl
Horizon
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 61
Criminisi & Zisserman, 99
Applica'on These transforma0ons are used in single view metrology
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 62
Applica'on these transforma0ons are used in single view metrology
Criminisi & Zisserman, 99
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 63
La Trinita' (1426) Firenze, Santa Maria Novella; by Masaccio (1401-‐1428)
Criminisi & Zisserman, 99
Applica'on these transforma0ons are used in single view metrology
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 64
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 65
Hoiem et al, 05…
Applica'on these transforma0ons are used in single view metrology
hxp://www.cs.uiuc.edu/homes/dhoiem/projects/so|ware.html
Lecture 8 - !!!
Fei-Fei Li!
A software: Make3D “Convert your image into 3d model”
Saxena, Sun, Ng, 05…
hxp://make3d.stanford.edu/
hxp://make3d.stanford.edu/images/view3D/185 hxp://make3d.stanford.edu/images/view3D/931?noforward=true hxp://make3d.stanford.edu/images/view3D/108
Applica'on these transforma0ons are used in single view metrology
2-‐Oct-‐12 66
Lecture 8 - !!!
Fei-Fei Li!
What we have learned today
• Review camera parameters • Affine camera model (Problem Set 2 (Q4)) • Camera calibra0on • Vanishing points and lines (Problem Set 2 (Q1))
2-‐Oct-‐12 67
Reading: • [FP] Chapter 3 • [HZ] Chapter 7, 8.6
Lecture 8 - !!!
Fei-Fei Li!
Supplementary Materials
2-‐Oct-‐12 68
Lecture 8 - !!!
Fei-Fei Li!
Degeneracy and distor0on in real-‐world camera calibra0on
2-‐Oct-‐12 69
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 70
Degenerate cases
• Pi’s cannot lie on the same plane! • Points cannot lie on the intersec0on curve of two quadric surfaces
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 71
Radial Distor'on
No distortion
Pin cushion
Barrel
– Caused by imperfect lenses – Devia0ons are most no0ceable for rays that pass through the edge of the lens
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 72
Radial Distor'on
ii
ii p
vu
PM
100
00
00
1
1
=⎥⎦
⎤⎢⎣
⎡→
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
λ
λ
d
v
vucvbuad 222 ++=
u
∑±==
3
1p
2ppdκ1λ
Polynomial func0on
Distor0on coefficient
To model radial behavior
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 73
Radial Distor'on
⎥⎦
⎤⎢⎣
⎡=
i
ii v
up
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
3
2
1
Qqqq
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
=
i3
i2
i3
i1
PPPP
qqqq
ii
ii p
vu
PM
100
00
00
1
1
=⎥⎦
⎤⎢⎣
⎡→
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
λ
λ
Q
⎩⎨⎧
=
=
PPvPPu
2i3i
i1i3i
qqqq
Non-‐linear system of equa0ons
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 74
General Calibra'on Problem
)(PfX =
measurement parameter
f( ) is nonlinear
-‐ Newton Method -‐ Levenberg-‐Marquardt Algorithm
• Itera0ve, starts from ini0al solu0on • May be slow if ini0al solu0on far from real solu0on • Es0mated solu0on may be func0on of the ini0al solu0on • Newton requires the computa0on of J, H • Levenberg-‐Marquardt doesn’t require the computa0on of H
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 75
General Calibra'on Problem
A possible algorithm 1. Solve linear part of the system to find approximated solu0on 2. Use this solu0on as ini0al condi0on for the full system 3. Solve full system (including distor0on) using Newton or L.M.
)(PfX =
measurement parameter
f( ) is nonlinear
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 76
General Calibra'on Problem
)(PfX =
measurement parameter
f( ) is nonlinear
Typical assump0ons for compu0ng ini0al condi0on :
-‐ zero-‐skew, square pixel -‐ uo, vo = known center of the image -‐ no distor0on
Just es0mate f and R, T
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 77
Tsai’s calibra'on technique 1. Es0mate m1 and m2 first:
⎥⎦
⎤⎢⎣
⎡=
i
ii v
up
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
=
i3
i2
i3
i1
PPPP
1
mmmm
λHow to do that?
d
v
u Hint: slopevu
i
i =
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 78
Tsai’s calibra'on technique 1. Es0mate m1 and m2 first:
⎥⎦
⎤⎢⎣
⎡=
i
ii v
up
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
=
i3
i2
i3
i1
PPPP
1
mmmm
λ
0)()( 121111 =− PuPv mm0)()( 21 =− iiii PuPv mm
0)()( 21 =− nnnn PuPv mm
…
0Q =n ⎥⎦
⎤⎢⎣
⎡=
2
1
mm
n
i
i
i
i
i
i
i
i
PP
PPPP
vu
2
1
3
2
3
1
)()()()(
mm
mmmm
==
Lecture 8 - !!!
Fei-Fei Li! 2-‐Oct-‐12 79
Tsai’s calibra'on technique 2. Once that m1 and m2 are es0mated, es0mate m3:
⎥⎦
⎤⎢⎣
⎡=
i
ii v
up
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
=
i3
i2
i3
i1
PPPP
1
mmmm
λ
3m is non linear func0on of 1m 2m λ
There are some degenerate configura0ons for which m1 and m2 cannot be computed
