© 2005 Yusuf Akgul Gebze Institute of Technology Department of Computer Engineering Computer Vision Geometric Camera Calibration

© 2005 Yusuf Akgul

Gebze Institute of TechnologyDepartment of Computer

Engineering

Computer Vision

Geometric Camera Calibration

© 2005 Yusuf Akgul

Geometric Camera Calibration• Geometric camera calibration is the estimation

of camera parameters (intrinsic, extrinsic, or both)

• Why do we need camera calibration?

• Most vision techniques need camera parameters but there are a few that don’t need the parameters.

© 2005 Yusuf Akgul

Camera Calibration• We need to link the

known world coordinates with the corresponding image coordinates.

• To be able to know the world coordinates, we need an object whose measurements are known, which are called calibration patterns.

© 2005 Yusuf Akgul

Camera Calibration

Is it enough to solve just the above matrix to do the camera calibration?

© 2005 Yusuf Akgul

Camera Calibration

We will see two ways of calibration

• First method: Direct camera calibration estimates the intrinsic and extrinsic camera parameters directly.

• Second method: The projection matrix is estimated first and then the camera parameters.

So which one should we prefer?

© 2005 Yusuf Akgul

Direct Camera Calibration

Extrinsic parameters to recover are

• R, 3x3 rotation matrix

• T, 3D translation vector.

© 2005 Yusuf Akgul


© 2005 Yusuf Akgul


If we ignore radial distortions, then we have

•Focal length

•Pixel size in x and y

•Image center x and y

5 dependent parameters

© 2005 Yusuf Akgul

Direct Camera calibrationIntrinsic camera parameters

© 2005 Yusuf Akgul


© 2005 Yusuf Akgul

Direct Camera CalibrationTwo main steps

• Assume the image center is known, estimate all the remaining parameters. We will assume (ox, oy) is (0,0)

• Estimate the image center later.

© 2005 Yusuf Akgul


Denominators are the same! So, for each

pair, we can write

© 2005 Yusuf Akgul


• If we write N equations like this one, we would have a homogenous linear system of equations of the form

),,,,,,,(

),,,,,,,(

131211232221

87654321

xy TrrrTrrr

vvvvvvvv

© 2005 Yusuf Akgul


How do we solve v?

© 2005 Yusuf Akgul

Direct Camera Calibration• Homogeneous System of N Linear Equations

– Given N corresponding pairs {(Xi, Yi,, Zi) <-> (xi, yi) }, i=1,2,…N

– 8 unknowns v = (v1,…,v8)T, 7 independent parameters

• The system has a nontrivial solution (up to a scale) – IF N >= 7 and N points are not coplanar => Rank

(A) = 7– Can be determined from the Singular Value

Decomposition (SVD) of A

© 2005 Yusuf Akgul

• Singular Value Decomposition:– Any mxn matrix can be written as the product of

three matrices TUDVA

Singular values i are fully determined by A D is diagonal: dij =0 if ij; dii = i (i=1,2,…,n)

1 2 … N 0

Both U and V are not unique Columns of each are mutual orthogonal vectors

nnnn

n

n

n

mmmm

m

m

mnmm

n

n

vvv

vvv

vvv

uuu

uuu

uuu

aaa

aaa

aaa

21

22212

121112

1

21

22221

11211

21

22221

11211

00

0

0

00

V1U1

© 2005 Yusuf Akgul

Singular Value Decomposition• 1. Singularity and Condition Number

– nxn A is nonsingular IFF all singular values are nonzero– Condition number : degree of singularity of A

• A is ill-conditioned if 1/C is comparable to the arithmetic precision of your machine; almost singular

• 2. Rank of a square matrix A– Rank (A) = number of nonzero singular values

• 3. Inverse of a square Matrix– If A is nonsingular– In general, the pseudo-inverse of A

• 4. Eigenvalues and Eigenvectors– Eigenvalues of both ATA and AAT are si2 (si > 0)– The columns of U are the eigenvectors of AAT (mxm)– The columns of V are the eigenvectors of ATA (nxn)

TUDVA

nC /1

TUVDA 10

iiT

iuuAA 2

iiT

ivAvA 2

© 2005 Yusuf Akgul

• Least Squares– Solve a system of m equations for n unknowns x(m >=

n)– A is a mxn matrix of the coefficients – b (0) is the m-D vector of the data – Solution:

– How to solve: compute the pseudo-inverse of ATA by SVD

• (ATA)+ is more likely to coincide with (ATA)-1 given m > n• Always a good idea to look at the condition number of ATA

bAx

bAAxA TT bAAAx TT )(

nxn matrix Pseudo-inverse

Singular Value Decomposition

© 2005 Yusuf Akgul

• Homogeneous System– m equations for n unknowns x(m >= n-1)– Rank (A) = n-1 (by looking at the SVD of A)– A non-trivial solution (up to a arbitrary scale) by SVD:– Simply proportional to the eigenvector corresponding

to the only zero eigenvalue of ATA (nxn matrix)

• Note:– All the other eigenvalues are positive because

Rank (A)=n-1

– In practice, the eigenvector (i.e. vn) corresponding to the minimum eigenvalue of ATA, i.e. n

2

0Ax

iiT

ivAvA 2

Singular Value Decomposition

© 2005 Yusuf Akgul

Singular Value Decomposition• Problem Statements

– Numerical estimate of a matrix A whose entries are not independent

– Errors introduced by noise alter the estimate to Â

• Enforcing Constraints by SVD– Take orthogonal matrix A as an example

– Find the closest matrix to Â, which satisfies the constraints exactly

• SVD of Â • Observation: D = I (all the singular values are 1) if A is

orthogonal• Solution: changing the singular values to those expected

TUDVA ˆ

© 2005 Yusuf Akgul

• Homogeneous System of N Linear Equations – Given N corresponding pairs {(Xi, Yi,, Zi) <-> (xi, yi) }, i=1,2,…

N– 8 unknowns v = (v1,…,v8)T, 7 independent parameters

• The system has a nontrivial solution (up to a scale) – IF N >= 7 and N points are not coplanar => Rank (A) = 7– Can be determined from the SVD of A– Rows of VT: eigenvectors {ei} of ATA– Solution: the 8th row e8 corresponding to the only zero singular

value 8=0

• Practical Consideration– The errors in localizing image and world points may make the

rank of A to be maximum (8)– In this case select the eigenvector corresponding to the

smallest eigenvalue.

0Av

TUDVA

8ev c


© 2005 Yusuf Akgul

• Equations for scale factor and aspect ratio

• How do we find the scale factor and alpha?• How do we find other parameters?

),,,,,,,( 131211232221 xy TrrrTrrr v v1 v2 v3 v4 v5 v6 v7 v8

Direct Camera CalibrationHow do we find the scale factor and alpha?

© 2005 Yusuf Akgul

• Equations for scale factor and aspect ratio

• Knowledge: R is an orthogonal matrix

• Second row (i=2):

• First row (i=1)

),,,,,,,( 131211232221 xy TrrrTrrr v

T

T

T

ij

rrr

rrr

rrr

r

3

2

1

333231

232221

131211

33R

R

R

R

ji if 0

ji if 1RRT

1222232221

rrr2

322

21|| vvv

1222131211

rrr 27

26

25|| vvv

||

v1 v2 v3 v4 v5 v6 v7 v8

Direct Camera CalibrationHow do we find the scale factor and alpha?

© 2005 Yusuf Akgul

Rotation R and Translation TRotation R and Translation T• Equations for first 2 rows of R and T gave us and ||

• First 2 rows of R and T can be found up to a common sign s (+ or -)

• The third row of the rotation matrix by vector product

• Remaining Questions :

– How to find the sign s?– Is R orthogonal?– How to find Tz and fx, fy?

),,,,,,,( || 131211232221 xy TrrrTrrrs v

yxTT sTsTss ,,, 21 RR

TTTTT ss 21213 RRRRR

T

T

T

ij

rrr

rrr

rrr

r

3

2

1

333231

232221

131211

33R

R

R

R


© 2005 Yusuf Akgul

Find the sign sFind the sign s

• Facts:– fx > 0– Zc >0– x known– Xw,Yw,Zw known

• Solution Check the sign of Xc Should be opposite to

x

zwww

ywwwyy

zwww

xwwwxx

TZrYrXr

TZrYrXrf

Zc

Ycfy

TZrYrXr

TZrYrXrf

Zc

Xcfx

333231

232221

333231

131211

Zw

Xw

Yw

Y

X

Z

xyO

PwP

p

xim

yim

(xim,yim)


© 2005 Yusuf Akgul

Rotation R : OrthogonalityRotation R : Orthogonality• Question:

– First 2 rows of R are calculated without using the mutual orthogonal constraint

• Solution: – Use SVD of estimate R

TTTTT ss 21213 RRRRR

T

T

T

ij

rrr

rrr

rrr

r

3

2

1

333231

232221

131211

33

R

R

R

R

?ˆˆ IRR T

TUDVR ˆ TUIVR

Replace the diagonal matrix D with the 3x3 identity matrix


© 2005 Yusuf Akgul


Find Tz, Fx and FyFind Tz, Fx and Fy• Solution

– Solve the system of N linear equations with two unknown

• Tx, fx

– Least Square method

– SVD method to find inverse

zwww

xwwwx TZrYrXr

TZrYrXrfx

333231

131211

bA

x

z

f

T

)()( 333231131211 wwwxxwwwz ZrYrXrxfTZrYrXrxT

ai2ai1 bi

bAAA TT

x

z

f

T 1)(ˆ

ˆ

© 2005 Yusuf Akgul

1. Measure N 3D coordinates (Xi, Yi,Zi)2. Locate their corresponding image

points (xi,yi) - Edge, Corner, Hough3. Build matrix A of a homogeneous

system Av = 0 4. Compute SVD of A , solution v5. Determine aspect ratio and scale ||6. Recover the first two rows of R and

the first two components of T up to a sign

7. Determine sign s of by checking the projection equation

8. Compute the 3rd row of R by vector product, and enforce orthogonality constraint by SVD

9. Solve Tz and fx using Least Square and SVD, then fy = fx /

Yw

Xw

Zw

Direct Camera Calibration Algorithm

© 2005 Yusuf Akgul

Estimating the Image CenterEstimating the Image Center• Vanishing points:

– Due to perspective projection, all parallel lines in 3D space appear to meet in a point on the image - the vanishing point, which is the common intersection of all the image lines

VP1

VP2


© 2005 Yusuf Akgul


Estimating the Image Center


• Orthocenter Theorem:

– Input: three mutually orthogonal sets of parallel lines in an image

– T: a triangle on the image plane defined by the three vanishing points

– Image center = orthocenter of triangle T

– Orthocenter of a triangle is the common intersection of the three altitudes VP1

VP2

VP3

h3

h1

h1

(ox,oy)

© 2005 Yusuf Akgul

VP1

VP2

VP3

h3

h1

h1

(ox,oy) ?




Wrong aspect ratio would produce wrong image center.

© 2005 Yusuf Akgul

HomeworkRead the Projection Matrix

Based Camera Calibration In Your Book.

Documents

© 2005 Yusuf Akgul Gebze Institute of Technology Department of Computer Engineering Computer Vision Geometric Camera Calibration