Camera Calibration to ARToolkitkowon.dongseo.ac.kr/~lbg/web_lecture/imageprocessing/... ·...

Camera Calibration

to ARToolkit

2012.12.

lbg@dongseo.ac.kr

Agenda

11/26/2012 lbg@dongseo.ac.kr 2

Homogeneous Coordinates

Projective Transformation

Pinhole Camera Model

Camera Calibrations

Zhengyou Zhang

http://www.cs.unc.edu/~marc/

Homogeneous Coordinates

lbg@dongseo.ac.kr

Homogeneous coordinates

0 cbyax Ta,b,c

0,0)()( kkcykbxka TTa,b,cka,b,c ~

Homogeneous representation of lines

equivalence class of vectors, any vector is representative

Set of all equivalence classes in R3(0,0,0)T forms P2

Homogeneous representation of points

0 cbyax Ta,b,cl x , ,1x yTon if and only if

0l 11 x,y,a,b,cx,y,T 0,1,,~1,, kyxkyx

The point x lies on the line l if and only if xTl=lTx=0

Homogeneous coordinates

Inhomogeneous coordinates Tyx,

, ,x y zT

but only 2DOF

Points from lines and vice-versa

Intersections of lines

The intersection of two lines and is l l'

Line joining two points

The line through two points and is x'xl x x'

Example

Ideal points and the line at infinity

T0,,l'l ab

Intersections of parallel lines

TTand ',,l' ,,l cbacba

Example

Ideal points T0,, 21 xx

Line at infinity T1,0,0l

tangent vector

normal direction

Note that in P2 there is no distinction

between ideal points and others

A model for the projective plane

exactly one line through two points

exactly one point at intersection of two lines

Duality

0xl T0lx T

l'lx x'xl

Duality principle:

To any theorem of 2-dimensional projective geometry

there corresponds a dual theorem, which may be

derived by interchanging the role of points and lines in

the original theorem

Projective Transformations

lbg@dongseo.ac.kr

Projective 2D Geometry

Projective transformations

A projectivity is an invertible mapping h from P2 to itself

such that three points x1,x2,x3 lie on the same line if and

only if h(x1),h(x2),h(x3) do.

Definition:

A mapping h:P2P2 is a projectivity if and only if there

exist a non-singular 3x3 matrix H such that for any point

in P2 reprented by a vector x it is true that h(x)=Hx

Theorem:

Definition: Projective transformation

333231

232221

131211

xx' Hor

projectivity=collineation=projective transformation=homography

Mapping between planes

central projection may be expressed by x’=Hx

(application of theorem)

Removing projective distortion

333231

131211

333231

232221

131211333231' hyhxhhyhxhx

232221333231' hyhxhhyhxhy

select four points in a plane with know coordinates

(linear in hij)

(2 constraints/point, 8DOF 4 points needed)

http://www.youtube.com/watch?v=NOU1lXR4JKo

http://www.youtube.com/watch?v=-tuS3GbHtV8

http://www.youtube.com/watch?v=i7woG0pqFjs

http://www.youtube.com/watch?v=zFMOtzpkgQo

More Examples

Transformation for lines

ll' -TH

xx' HFor a point transformation

Isometries

cossin

sincos

orientation preserving:

orientation reversing:

RHE IRR T

special cases: pure rotation, pure translation

3DOF (1 rotation, 2 translation)

Invariants: length, angle, area

Similarities

cossin

sincos

sS IRR T

also know as equi-form (shape preserving)

metric structure = structure up to similarity (in literature)

4DOF (1 scale, 1 rotation, 2 translation)

Invariants: ratios of length, angle, ratios of areas, parallel lines

Affine transformations

non-isotropic scaling! (2DOF: scale ratio and orientation)

6DOF (2 scale, 2 rotation, 2 translation)

Invariants: parallel lines, ratios of parallel lengths, ratios of areas

Projective transformations

Action non-homogeneous over the plane

8DOF (2 scale, 2 rotation, 2 translation, 2 line at infinity)

Invariants: cross-ratio of four points on a line (ratio of ratio)

T21,v vv

sPAS TTTT v

t AIKRHHHH

Ttv RKA s

K 1det Kupper-triangular, decomposition unique (if chosen s>0)

Overview Transformations

333231

232221

131211

Projective

Affine

Similarity

Euclidean

Concurrency, collinearity,

order of contact (intersection,

tangency, inflection, etc.),

cross ratio

Parallellism, ratio of areas,

ratio of lengths on parallel

lines (e.g midpoints), linear

combinations of vectors

(centroids).

The line at infinity l∞

Ratios of lengths, angles.

The circular points I,J

lengths, areas.

Line at infinity

Projective Transformation : Line at infinity becomes finite,

allows to observe vanishing points, horizon,

Affine Transformation : Line at infinity stays at infinity

The line at infinity

21 vvl

211 llv

432 llv

The line at infinity l is a fixed line under a projective transformation H if and only if H is an affinity

Recovery of affine properties from images

Projective transformation Affine rectification

Affine transformation

Affine Rectification

21 vvl

211 llv

432 llv

Projective 3D geometry

tAProjective

Affine

Similarity

Euclidean

Intersection and tangency

Parallellism of planes,

Volume ratios, centroids,

The plane at infinity π∞

The absolute conic Ω∞

Volume

lbg@dongseo.ac.kr

TT ZfYZfXZYX )/,/(),,(

camX0|IKx

Principal point offset Calibration Matrix

Internal Camera Parameters

Camera rotation and translation

RRXcam

X camX0|IKx

t|RKP C~

Camera Parameter Matrix P

CCD Cameras

Correcting Radial Distortion of Cameras

Calibration Process

Correcting Radial Distortion of Camera with

Rotation Angle

( )(1 )cos

( )(1 )sin

u x d x d d d

d y d d d

x c x c k r k r k r

y c k r k r k r

( )(1 )sin

( )(1 )cos

u y d x d d d

d y d d d

y c x c k r k r k r

y c k r k r k r

m x m y mx

m x m y

m x m y my

m x m y

Camera Calibration

lbg@dongseo.ac.kr

Camera Models

“An empirical evaluation of factors influencing camera calibration accuracy using three publicly

available techniques”,Wei Sun and Jeremy R. Cooperstock

Machine Vision and Applications Volume 17, Number 1 (2006), 51-67,

Roger Y. Tsai

"An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision", Roger Y. Tsai,

Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Miami Beach, FL,

1986, pages 364-374,

Zhengyou Zhang

Z. Zhang. Flexible Camera Calibration By Viewing a Plane From Unknown Orientations. International Conference on Computer Vision, Corfu, Greece, pages 666-673, September 1999.

Basic Equations

Solving Camera Calibration

Zhengyou Zhang

http://research.microsoft.com/en-us/um/people/zhang/

Camera Calibration Toolbox

http://www.vision.caltech.edu/bouguetj/calib_doc/

Camera Calibration

Calibrating a Stereo System

Robust Multi-camera Calibration

http://graphics.stanford.edu/~vaibhav/projects/calib-

cs205/cs205.html

Multi-Camera Self Calibration

http://cmp.felk.cvut.cz/~Esvoboda/

Multi-Camera Self Calibration

ARToolKit Camera Calibration

http://www.hitl.washington.edu/artoolkit/

OpenCV

http://opencv.willowgarage.com/documentation/

camera_calibration_and_3d_reconstruction.html

http://www.youtube.com/watch?v=DrXIQfQHFv0

http://www.youtube.com/watch?v=O6PEFJhsCEo

OpenCV 2.1 Camera Calibration

http://opencv.willowgarage.com/documentation/cpp/camera_calibration_and_3d_reconstruction.html

http://mmlab.disi.unitn.it/wiki/index.php/Camera_Calibration_Tool_in_OpenCV

xcorrected = x + [2P1xy + P2(r2 + 2x

ycorrected = y + [P1(r2 + 2y

2) + 2P2xy]

Using the Polynomial Distortion Model to Correct Tangential Distortion The polynomial distortion model uses two parameters, P1 and P2, to characterize tangential distortion.

The distortion model for tangential distortion can be represented as:

http://zone.ni.com/reference/en-XX/help/372916L-01/nivisionconcepts/spatial_calibration_indepth/

xcorrected = x(1+ K1r2+ K2r

4 + K3r

6 + Knr

(n × 2))

ycorrected = y(1+ K1r2+ K2r

4 + K3r

6 + Knr

(n × 2))

Brown, D. C. Decentric distortion of lenses.

Journal of Photogrammetric Engineering and Remote Sensing, 32(3), 444-462, 1966.

http://www.nowpublishers.com/product.aspx?product=CGV&doi=0600000001&section=x1-10r1

calibrateCamera

solvePnP

findChessboardCorners

drawChessboardCorners

rodrigues2

http://dprg.geomatics.ucalgary.ca/system/files/AKAM_ENGO667_CH_1Handouts.pdf

Camera Calibration

lbg@dongseo.ac.kr

Capture chessboard image

Chessboard no of row and col

10 : no of calibration images

Auto : automatically add images after 3*150 milliseconds

Start : start calibration

Calibration parameters

Focal length f_x, f_y

Center c_x, c_y

Radial : k1, k2, k3(check box)

Tangential : p1, p2

Und : Undistorted live image

Debug images

Filenames

Capture072309511500

month day hour min sec index

Corner072309511500

Image0723095115.xml

Object0723095115.xml

Distortion0723095115.xml

Intrinsics0723095115.xml

Image & Object xml

<?xml version="1.0"?>

<opencv_storage>

<data>

2.25150803e+002 1.92720139e+002 2.42047302e+002 1.88396301e+002

2.59520660e+002 1.83857040e+002 2.77741913e+002 1.79775314e+002

2.96598541e+002 1.75898483e+002 3.15514801e+002 1.72283661e+002

<opencv_storage>

<data>

0. 1.37500000e+001 0. 2.75000000e+000 1.37500000e+001 0.

5.50000000e+000 1.37500000e+001 0. 8.25000000e+000 1.37500000e+001

0. 11. 1.37500000e+001 0. 1.37500000e+001 1.37500000e+001 0.

Distortion & Intrinsics xml

<opencv_storage>

<data>

-4.46110874e-001 2.08227739e-001 9.08007473e-003 -3.17067979e-003</data></Distortion0723095127>

</opencv_storage>

<opencv_storage>

<data>

2.66292694e+002 0. 3.24350769e+002 0. 2.58133820e+002

2.33235382e+002 0. 0. 1.</data></Intrinsics0723095127>

</opencv_storage>

Camera Calibration to ARToolkitkowon.dongseo.ac.kr/~lbg/web_lecture/imageprocessing/... ·...

Documents

2011 LBG Canada Annual Benchmarking Report

LBG Reports PM08

TCS North America - LBG Canada

Mathematical Methods in Computer Graphics, Duksung Univ. Nov. 1. 2006, lbg@dongseo.ac.kr Mathematical Methods in Computer Graphics Byung-Gook Lee Dongseo

LLOYDS BANKING GROUP (LBG) OPEN BANKING SANDBOX …...LBG Sandbox Developers Guide Lloyds Banking Group Page 3 of 18 1 INTRODUCTION 1.1 OVERVIEW The Lloyds Banking Group (LBG) Sandbox

LBG benchmarking report - Amazon S3 · 2015. 3. 5. · Michelle Commandeur, ANZ Chair, Steering Group LBG Australia /New Zealand Introduction to the LBG Australia / New Zealand Benchmarking

LBG 08 UK (Dec-10).pdf

Current openings in lbg july 2016 linked in

28 Computer Graphics, Image processing and computer Vision Division of Computer Information Engineering Lee Byung Gook, Dongseo Univ., Busan, Korea lbg@dongseo.ac.kr

Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121

An Efficient Codebook Initialization Approach For Lbg Algorithm

New York Rising Community Reconstruction Plan South Valley ... · Louis Berger Group (LBG); Jennifer Gonzales, LBG; Deana Rhodeside, Rhodeside & Harwell (RHI); Jenny Koch, RHI; Amy

Prescolite Litebox Downlight Catalog LBG-1 1994

ARTICLE IN PRESSkowon.dongseo.ac.kr/~lbg/web_lecture/grapprog/20082/... · 2015-02-06 · Subdivision surfaces for CAD—an overview Weiyin Ma* Department of Manufacturing Engineering

2010 LBG Interim Results

LBG MONDAY DRAFT last

Icon-LBG 2013 - Artikel Universitas Bandar Lampung [UBL]

ARTICLE IN PRESSkowon.dongseo.ac.kr/~lbg/web_lecture/grapprog/20122/... · 2015-02-06 · Subdivision surfaces refer to a class of modelling schemes that deﬁne an object through

Dongseokowon.dongseo.ac.kr/~lbg/web_lecture/grapprog/20082/JoeWarren.pdfCon ten ts In tro duction Sub division metho ds for uniform Bsplines Degree zero Bsplines Higher degree Bsplines

Single-Source Shortest Pathskowon.dongseo.ac.kr/~lbg/web_lecture/algorithm/... · Single-source shortest paths can be defined if there are not any negative-weight cyclesreachable