Upload
yaswanth
View
233
Download
0
Embed Size (px)
Citation preview
7/25/2019 Camera Geometry and Single View Geometry
1/36
1-1Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
Chapter 3
Camera Geometry and
Single View GeometryVersion 11.05.2015
7/25/2019 Camera Geometry and Single View Geometry
2/36
1-2Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
Content
3.1 Geometrical Imaging Process
3.1.1 Introduction
3.1.2 Extrinsic Model3.1.3 Extrinsic ModelMatlab Example
3.1.4 Intrinsic ModelProjection
3.1.5 Intrinsic ModelAffine Transformation
3.1.6 Summary
3.2 Pinhole Camera Model
3.2.1 The Geometrical Model of a Pinhole Camera
3.2.2 The Geometrical Model of a Pinhole CameraMatlab
Example
3.2.3 Intrinsic ModelCamera Calibration Matrix K
3.2.4 Extrinsic Model
3.2.5 Extrinsic ModelMatlab Example
3.3 Projective Camera Model
3.3.1 Intrinsic ModelCamera Calibration Matrix K3.3.2 The Projection Matrix P
3.3.2 The Projection Matrix PMatlab Example
3.3.3 The Projection Matrix PExample Basler acA630-
100gc
7/25/2019 Camera Geometry and Single View Geometry
3/36
1-3Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.1 Geometrical Imaging Process
An imaging device or camera usually maps an observed
3D scene to a 2D image by projection.
Several projection principles are known and vitally
characterize a camera: parallel projection, perspective
projection, orthographic projection, oblique projection,
Projection principles can be mathematically approximated
by an appropriate model which is commonly known as
camera modelorprojection model.
Most commonly known imaging devices follows the
perspective projection principle and is known as pinhole
camera model
3.1.1 Introduction
Principle Point / Camera Center / Optical Center / Pinhole
Sensor Plane/Image Plane
Focal length f
Camera (box)
Optical axis/principle axis
7/25/2019 Camera Geometry and Single View Geometry
4/36
1-4Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.1 Geometrical Imaging Process
Mathematically the image that has been projected upside-
down can be flipped using the principle point
3.1.1 Introduction
Focal length f
Sensor Plane/Image Plane
Principle Plane
7/25/2019 Camera Geometry and Single View Geometry
5/36
1-5Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.1 Geometrical Imaging Process
The principle of the pinhole camera was already
discovered by Aristoteles (384-322 B.C.).
First steps with the pinhole camera was taken by the Arab
Alhazen in 980 as he investigates the human eye.
At the end of the 13th century astronomics used the
pinhole camera to observe sunspots.
3.1.1 Introduction
7/25/2019 Camera Geometry and Single View Geometry
6/36
1-6Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.1 Geometrical Imaging Process
The first published picture of a pinhole camera obscurais apparently a drawing in Gemma Frisius' De Radio Astronomica et
Geometrica(1545).
The very term camera obscura("dark room") was coined by Johannes Kepler (15711630).
The first photography was taken by a pinhole camera in 1816 by Joseph Nicphore Nipcecht. Unfortunately he wasnt
able to fix it. But a few months later he presents the first photo of the world, a view out of his studyroom.
3.1.1 Introduction
7/25/2019 Camera Geometry and Single View Geometry
7/361-7Professur Digital- und Schaltungstechnik
Prof. Dr.-Ing. Gangolf HirtzComputer Vis ion
Version vom 11.05.2015
3.1 Geometrical Imaging Process
The geometrical relationship between 3D scene points
Xwrldand the resulting 2D image points Ximgis often quite
complex due to the physical construction of the imaging
device.
By employing a finite amount of numerical parameters,
the actual process of projection can be replaced by an
appropriate mathematical relationship of bearable
computational cost.
3.1.1 Introduction
Observed scene
Camera
World coordinate system
wrldX
Image
imgX
zwrld
xwrldywrld
3D scene point
2D
Image point
Geometrical imaging process
wrld
wrld
wrld
wrld
wrld
x
y
z
w
X
img
img img
img
x
y
w
X
7/25/2019 Camera Geometry and Single View Geometry
8/361-8Professur Digital- und Schaltungstechnik
Prof. Dr.-Ing. Gangolf HirtzComputer Vis ion
Version vom 11.05.2015
3.1 Geometrical Imaging Process
The overall imaging process that transfers a scene point
Xwrld to an image point Ximg successively utilizes an
extrinsicand an intrinsic model
3.1.1 Introduction
Observed scene
Camera
wrldX imgX
zwrld
xwrldywrld
Extrinsic model Intrinsic model
xcamzcam
ycam
Camera coordinate system
yimg
ximg
Image coordinate system
camX
wrld
wrld
wrld
wrld
wrld
x
y
z
w
X
img
img img
img
x
y
w
X
cam
cam
cam
cam
cam
x
y
z
w
X
7/25/2019 Camera Geometry and Single View Geometry
9/361-9Professur Digital- und Schaltungstechnik
Prof. Dr.-Ing. Gangolf HirtzComputer Vis ion
Version vom 11.05.2015
3.1 Geometrical Imaging Process
Transformation from world points Xwrldto camera points Xcamin multiple camera environments
Incorporates the distinct geometrical relationship Hiof each camera with respect to WCS
3.1.2 Extrinsic Model
Observed scene
Camera
wrldX
zwrld
xwrldywrld
xcamzcam
Camera
xcam
zcam
cam,0X
cam,1X
0 0R C
1 1R C
i i ii
1
R R CH0
1
cam,0 0 wrld
1
cam,1 1 wrld
X H X
X H X
1H
0H
Ci origin of the camera coordinate
frame provided in world frame
coordinates
Ri 3x3 rotation matrix representing the
orientation of the cameracoordinate frame with respect to
the world frame
R results out of three angles
discribing a sequence of three
elementary rotations around x, y, z-
axis respectively (each performed
clockwise)
7/25/2019 Camera Geometry and Single View Geometry
10/361-10Professur Digital- und Schaltungstechnik
Prof. Dr.-Ing. Gangolf HirtzComputer Vis ion
Version vom 11.05.2015
3.1 Geometrical Imaging Process
Example: Optical sensor network to observe a complex indoor environment (e.g. domestic home environment)
3.1.3 Extrinsic Model - Matlab Example
WCS
Camera 1
Camera 2
Camera 3
7/25/2019 Camera Geometry and Single View Geometry
11/36
1-11Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.1 Geometrical Imaging Process
3.1.3 Extrinsic Model - Matlab Example
0001 clear all; close all; clc;
0002
0003 %% Extrinsic Camera Parameter0004 % Camera centres
0005 C1 = [ 2.6607; 0.1006; 2.4089]; % Smart Sensor 605
0006 C2 = [-1.6157; 2.7995; 2.4119]; % Smart Sensor 601
0007 C3 = [-1.8482; -1.7443; 2.3955]; % Smart Sensor 602
0008
0009 % Rotation matrices
0010 R1 = [ -0.7770 0.6286 0.03390011 0.3719 0.5018 -0.7809
0012 -0.5079 -0.5942 -0.6237];
0013
0014 R2 = [ -0.5233 -0.8521 0.0080
0015 -0.5388 0.3236 -0.7778
0016 0.6602 -0.4113 -0.6285];
0017
0018 R3 = [ -0.4861 -0.8670 0.1097
0019 -0.6689 0.2884 -0.6852
0020 0.5624 -0.4065 -0.7201];
0021
7/25/2019 Camera Geometry and Single View Geometry
12/36
1-12Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.1 Geometrical Imaging Process
3.1.3 Extrinsic Model - Matlab Example
0022 % Translation vectors
0023 T1 = -R1*C1;
0024 T2 = -R2*C2;0025 T3 = -R3*C3;
0026
0027 %% Setup Homographies
0028 H1 = [R1 T1; 0 0 0 1];
0029 H2 = [R2 T2; 0 0 0 1];
0030 H3 = [R3 T3; 0 0 0 1];
00310032 %% Plot WCS
0033 h1 = figure(1); hold on; grid on; axis equal;
0034 xlabel('x [m]'); ylabel('y [m]'); zlabel('z [m]'); view(70, 20);
0035 axis([-3 +3 -3 +3 0 3]);
0036
0037 i = [1; 0; 0];
0038 j = [0; 1; 0];
0039 k = [0; 0; 1];
0040 o = [0; 0; 0];
0041
0042 plot3([o(1) i(1)], [o(2) i(2)], [o(3) i(3)], 'r-', 'MarkerSize', 5);
0043 plot3([o(1) j(1)], [o(2) j(2)], [o(3) j(3)], 'g-', 'MarkerSize', 5);
0044 plot3([o(1) k(1)], [o(2) k(2)], [o(3) k(3)], 'b-', 'MarkerSize', 5);
7/25/2019 Camera Geometry and Single View Geometry
13/36
1-13Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.1 Geometrical Imaging Process
3.1.3 Extrinsic Model - Matlab Example
0045
0046 %% Plot camera coordinate systems
0047 i1 = H1^-1*[i; 1];0048 j1 = H1^-1*[j; 1];
0049 k1 = H1^-1*[k; 1];
0050 o1 = H1^-1*[o; 1];
0051
0052 plot3([o1(1) i1(1)], [o1(2) i1(2)], [o1(3) i1(3)], 'r-', 'MarkerSize', 1);
0053 plot3([o1(1) j1(1)], [o1(2) j1(2)], [o1(3) j1(3)], 'g-', 'MarkerSize', 1);
0054 plot3([o1(1) k1(1)], [o1(2) k1(2)], [o1(3) k1(3)], 'b-', 'MarkerSize', 1);0055
0056 i2 = H2^-1*[i; 1];
0057 j2 = H2^-1*[j; 1];
0058 k2 = H2^-1*[k; 1];
0059 o2 = H2^-1*[o; 1];
0060
0061 plot3([o2(1) i2(1)], [o2(2) i2(2)], [o2(3) i2(3)], 'r-', 'MarkerSize', 1);
0062 plot3([o2(1) j2(1)], [o2(2) j2(2)], [o2(3) j2(3)], 'g-', 'MarkerSize', 1);
0063 plot3([o2(1) k2(1)], [o2(2) k2(2)], [o2(3) k2(3)], 'b-', 'MarkerSize', 1);
7/25/2019 Camera Geometry and Single View Geometry
14/36
1-14Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.1 Geometrical Imaging Process
3.1.3 Extrinsic Model - Matlab Example
0064
0065 i3 = H3^-1*[i; 1];
0066 j3 = H3^-1*[j; 1];
0067 k3 = H3^-1*[k; 1];0068 o3 = H3^-1*[o; 1];
0069
0070 plot3([o3(1) i3(1)], [o3(2) i3(2)], [o3(3) i3(3)], 'r-', 'MarkerSize', 1);
0071 plot3([o3(1) j3(1)], [o3(2) j3(2)], [o3(3) j3(3)], 'g-', 'MarkerSize', 1);
0072 plot3([o3(1) k3(1)], [o3(2) k3(2)], [o3(3) k3(3)], 'b-', 'MarkerSize', 1);
1H
0H
2H
Observed Indoor Scene
Floor (z == 0 m)
WCS
7/25/2019 Camera Geometry and Single View Geometry
15/36
1-15Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.1 Geometrical Imaging Process
The Projection process from Xcam to normalized sensor
coordinates Xnorm is modelled by a projection function F
The cameras sensor plane is assumed to have unit
length from the principle point
3.1.4 Intrinsic Model - Projection
Observed scene
ynorm
xnorm
xcam
ycamcamX
zcam
Normalized sensor coordinate system
normXF
Camera centre/Principle point
Camera sensor plane
Focal length f normalizedto unit length
Projection
norm
norm norm
1
x
y
X
cam
cam
camcam
cam
x
y
z
w
X
7/25/2019 Camera Geometry and Single View Geometry
16/36
1-16Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.1 Geometrical Imaging Process
The camera coordinate Xcam is considered as incident
light ray(,,)
The radial projection of an incoming light ray (,,)
onto a virtual image plane with a distance of f= 1 to the
projection center can be modelled with the radial
projection function F():
3.1.4 Intrinsic Model - Projection
Observed scene
ynorm
xnorm
xcam
ycamcam X
zcam
elevation
azimuth
()
cam
cam cam
cam
x
y
z
X
norm
norm r
norm
r
( ) ( )
coswithsin
xF
y
X u
u
normXF
distance
ur
7/25/2019 Camera Geometry and Single View Geometry
17/36
1-17Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.1 Geometrical Imaging Process
Projection
Type
Function F Properties and Applications
Perspective
projection
= tan Applied in conventional
cameras that follow the pinhole
principle with <
Stereographic
projection = 2 tan
Commonly used for the
generation of stellar maps and
photography. angles are
preserved (equal angles) and
circles are imaged without
distortions (equal circles).
Equidistanceprojection
= It is the most popular model fordescribing fisheye projection
lenses. Incidence angles of rays
are imaged linearly.
Equisolid
angle
projection
= 2 sin
2
Each pixel corresponds to one
distinct solid angle which makes
the image look like the reflection
by a specular ball.
Orthogonal
projection = sin This camera principle models
the projection onto a sphere.
The projection radius is
increasing with increasing
incidence angle
until reachespi/2. Than it is
declining again.
3.1.4 Intrinsic Model - Projection
7/25/2019 Camera Geometry and Single View Geometry
18/36
1-18Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.1 Geometrical Imaging Process
3.1.4 Intrinsic Model - Projection
Equiangular Projection
Applied by e.g. fish-eye cameras
FOV > 180
Perspective Projection
Most commonly applied projection method
Limited field of view (FOV) to approx. 120()
= tan =
7/25/2019 Camera Geometry and Single View Geometry
19/36
1-19Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.1 Geometrical Imaging Process
3.1.5 Intrinsic Model Affine Transformation
The transformation from normalized sensor coordinates to
image coordinates is finally done by means of an affine
transformation matrixK
Kis called camera calibration matrix
imgX
Affine
Transformation
yimg
ximg
Image coordinate system
img
img img
img
x
y
w
X
norm
norm norm
1
x
y
X
ynorm
xnormnormX
Normalized sensor coordinate system
K
img norm X K X
7/25/2019 Camera Geometry and Single View Geometry
20/36
1-20Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.1 Geometrical Imaging Process
3.1.6 Summary
Geometrical imaging process is characterized by an
extrinsic and intrinsic model
Intrinsic model (camera model) is split into different
projection models Fand an affine transformation K
Relevant camera models are
Pinehole camera model
Projective camera model
7/25/2019 Camera Geometry and Single View Geometry
21/36
1-21Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
Central projection is mapping a point Xcamin 3D space to
a point Ximgin 2D space:
Using similar triangles one verifies that:
Using homogeneous coordinates, one can put that
transformation in a matrix expression:
The 3x4 transformation matrix used is called a camera
projection matrixP
Pcan be decomposed in the following manner:
The Diagonal Matrix is called the camera calibration
matrixK
3.2 Pinhole Camera Model
3.2.1 The Geometrical Model of a Pinhole Camera
x
object point in
3D world frameimage point in 2D
image frame
imaging process
cam cam cam img img, , ,TT
x y z x y
f
fcam
z
cam cam/f y z
cam cam cam cam cam cam cam, , / , /T T
x y z f x z f y z
cam
img camcam
img cam
camcam cam
cam
normalized sensor coordinatestan
01 1
0
1 1 01
xx f x f
yy f y f
zz zz
img cam
X P X
diag( , ,1) |f f P I 0
( , ,1)diag f f K
camX
camXimgX
7/25/2019 Camera Geometry and Single View Geometry
22/36
1-22Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.2 Pinhole Camera Model
3.2.2 The Geometrical Model of a Pinhole Camera - Matlab Example
0001 %% Camera Matrix
0002 % Intrinsic Camera Parameter
0003 f = 4; % Focal Length
00040005 % Setup Projection matrix
0006 P = diag([f f 1])*[eye(3) zeros(3, 1)];
0007
0008 %% Project World Point
0009 Xw = [4; 5; 9]; % Object Point (World Frame)
0010 % X = 4, Y = 5, Z = 9
0011 Xw_h = [Xw; 1]; % Transform to homogeneous Coordinates
0012 Xc_h = P*Xw_h; % Project!
00130014 Xc = 1/Xw(3)*Xc_h;% Image Point (Camera Frame)
0015
0016 %% Plot Principal Point
0017 figure_01 = figure; hold on; grid on;
0018 plot3(0, 0, 0, 'm.', 'Markersize', 20);
0019
0020 %% Draw Image Plane
0021 [X, Y, Z] = meshgrid(-5:0.1:5, -5:0.1:5, -5:0.1:5);
0022
0023 data = 0*X+0*Y+1*Z-f; % Normal from for a plane
0024 p = patch(isosurface(X, Y, Z, data, 0)); hold on;
0025 set(p, 'FaceColor', 'red', 'EdgeColor', 'none')
0026 alpha(0.5); view(17,24);
0027 axis([-5 5 -5 5 0 10]);
0028
7/25/2019 Camera Geometry and Single View Geometry
23/36
1-23Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.2 Pinhole Camera Model
3.2.2 The Geometrical Model of a Pinhole Camera - Matlab Example
0029 %% Draw Object and Image Point
0030 plot3(Xw(1), Xw(2), Xw(3), 'b*', 'Markersize', 10);
0031 plot3(Xc(1), Xc(2), f, 'b*', 'Markersize', 5);
0032 plot3([0 Xw(1)], [0 Xw(2)], [0 Xw(3)], 'b--');0033
0034 %% Add some text
0035 text(0.5, 0.5, 0,...
0036 'Principal Point', 'FontSize', 10,...
0037 'FontWeight', 'bold', 'Color', 'magenta');
0038 text(Xw(1) - 3.0, Xw(2) - 3.0, Xw(3),...0039 'Object Point', 'FontSize', 10, 'FontWeight', 'bold', 'Color', 'blue');
0041 text(Xc(1) + 0.5, Xc(2) + 0.5, f,...
0042 'Image Point', 'FontSize', 10,...0043 'FontWeight', 'bold', 'Color', 'blue');
0044 text(-2, -2, f,...
0045 'Image Plane', 'FontSize', 10,...
0046 'FontWeight', 'bold', 'Color', 'red');
3 2 C
7/25/2019 Camera Geometry and Single View Geometry
24/36
1-24Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
It is common for images to have only positive coordinates
(image coordinates start at the upper left corner of the
image), so a further adaption has to be done
The principal point (cx, cy) is the offset with respect to the
image frame and enhances the camera calibration matrix
Kis an upper triangle matrix with 3 dof
A camera is denoted as acalibrated cameraso soon as
Kis known
The pinhole camera model is a ideal one assuming that
image coordinates are Euclidean coordinates having
equal scales in both axial directions
3.2 Pinhole Camera Model
3.2.3 Intrinsic Model - Camera Calibration Matrix K
img img img imgx yx x c y y c
1
x
y
f cf c
K
imgX
Image coordinate system
yimg
ximg
yimg
ximg
Principal Point (cx, cy)
3 2 Pi h l C M d l
7/25/2019 Camera Geometry and Single View Geometry
25/36
1-25Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
Every single cameras position C is described by a
rotation Rand a translation Twith respect to the world
coordinate system
A Point Xwrld can be translated into a point Xcamby the
following mathematical relationship
Homogeneous coordinates let us use this relation in a
more compact matrix form
The overall imaging process is now as follows:
3.2 Pinhole Camera Model
3.2.4 Extrinsic Model
With the new relationship one can determine the image
coordinates of a point given in world coordinates as
follows:
Pcan be computed from point correspondences Xwrld
Ximg
cam wrld X R X C
cam wrld1
R RCX X
img wrld with | X P X P KR I C
img cam
img cam
img wrld
img wrld
Translation vector
img wrld
|
|1
|
|
T
X P X
X K I 0 X
R RCX K I 0 X
X K R RC X
X K R I C X
1H
0H
2H
WCS
3 2 Pi h l C M d l
7/25/2019 Camera Geometry and Single View Geometry
26/36
1-26Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.2 Pinhole Camera Model
3.2.5 Extrinsic Model Matlab Example
0001 %% Camera Matrix
0002 % Intrinsic Camera Parameter
0003 f = 2; % Focal Length
0004 K = diag([f f 1]); % Intrinsic camera Parameter0005
0006 % Extrinsic Camera Parameter
0007 C1 = [-3; +4; -3]; % Camera Centre Cam 1
0008 C2 = [3; -4; 1]; % Camera Centre Cam 2
0009 C3 = [3; -5; -5]; % Camera Centre Cam 2
0010
0011 % Rotate Cam 1 by 0, Cam2 by 45 and Cam3 by 900012 % arround the x-axis
0013 R1 = [1.0000 0 00014 0 1.0000 0
0015 0 0 1.0000];
0016 R2 = [1.0000 0 0
0017 0 -0.7071 0.7071
0018 0 0.7071 0.7071];
0019 R3 = [1.0000 0 0
0020 0 0 1.0000
0021 0 1.0000 0];
00220023 t1 = -R1*C1; % Translation vector with respect to R and C
0024 t2 = -R2*C2; % Translation vector with respect to R and C
0025 t3 = -R3*C3; % Translation vector with respect to R and C
0026
3 2 Pi h l C M d l
7/25/2019 Camera Geometry and Single View Geometry
27/36
1-27Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.2 Pinhole Camera Model
3.2.5 Extrinsic Model Matlab Example
0027 % Setup Projection matrix
0028 P1 = K*[R1 t1];
0029 P2 = K*[R2 t2];
0030 P3 = K*[R3 t3];0031
0032 %% Plot Principal Points
0033 figure_01 = figure; hold on; grid on;
0034 plot3(C1(1), C1(2), C1(3), 'r.', 'Markersize', 20);
0035 plot3(C2(1), C2(2), C2(3), 'g.', 'Markersize', 20);
0036 plot3(C3(1), C3(2), C3(3), 'b.', 'Markersize', 20);
0037
0038 %% Draw Image Plane Camera 1
0039 [X, Y, Z] = meshgrid( (C1(1)-3):0.1:(C1(1)+3),...0040 (C1(2)-3):0.1:(C1(2)+3),...
0041 (C1(3)-3):0.1:(C1(3)+3));
0042 plane = P1(3, :);0043 data = plane(1)*X + plane(2)*Y + plane(3)*Z + (plane(4)-f)*1;
0044 p = patch(isosurface(X, Y, Z, data, 0)); hold on;
0045 set(p, 'FaceColor', 'red', 'EdgeColor', 'none');
0046 %% Draw Image Plane Camera 2
0047 [X, Y, Z] = meshgrid( (C2(1)-3.0):0.1:(C2(1)+3.0),...
0048 (C2(2)-4.2):0.1:(C2(2)+4.2),...0049 (C2(3)-4.2):0.1:(C2(3)+4.2));
0050 plane = P2(3, :);
0051 data = plane(1)*X + plane(2)*Y + plane(3)*Z + (plane(4)-f)*1;
0052 p = patch(isosurface(X, Y, Z, data, 0)); hold on;
0053 set(p, 'FaceColor', 'green', 'EdgeColor', 'none');
0054
3 2 Pi h l C M d l
7/25/2019 Camera Geometry and Single View Geometry
28/36
1-28Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.2 Pinhole Camera Model
3.2.5 Extrinsic Model Matlab Example
0055 %% Draw Image Plane Camera 3
0056 [X, Y, Z] = meshgrid( (C3(1)-3):0.1:(C3(1)+3),...
0057 (C3(2)-3):0.1:(C3(2)+3),...
0058 (C3(3)-3):0.1:(C3(3)+3));0059 plane = P3(3, :);
0060 data = plane(1)*X + plane(2)*Y + plane(3)*Z + (plane(4)-f)*1;
0061 p = patch(isosurface(X, Y, Z, data, 0)); hold on;
0062 set(p, 'FaceColor', 'blue', 'EdgeColor', 'none');
0063
0064
0065 alpha(0.5); view(140, 35);
0066 axis([C(1)-10 C(1)+10 C(2)-10 C(2)+10 C(3)-10 C(3)+10]);
3 3 Projecti e Camera Model
7/25/2019 Camera Geometry and Single View Geometry
29/36
1-29Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
In the case of CCD cameras there is the possibility to
have non-square pixels and hence having unequal scale
factors for both image directions.
Imagine a pixel density(no. of pixels per unit length) of
mx(x-direction) and my(y-direction), the calibration matrix
Kchanges to
wherexandyrepresent the focal length with respect
to the pixel dimensions in both directions.
Although it is very seldom for some rare cases it is
necessary to introduce the skew factor s, defining the
geometrical dependency of the y-dimension in pixel
frame from the x-axis:
A camera having Pas described above is known as finite
projective camera.and has 11 degrees of freedom
3.3 Projective Camera Model
3.3.1 Intrinsic Model - Camera Calibration Matrix K
Aspect Ratio
The ratio ofxandy is often calledaspect ratioand is
very close to 1.
mxno. of pixel per m along the x-axis, hor. pixel density,
[px/m]
myno. of pixel per m along the y-axis,vert. pixel density,
[px/m],
Shear angle between the image axes; is encoded
within s
(x,y) focal length, with respect to pixel dimensions
with , ,
1
x x
y y x x y y
s c
c f m f m s f
K
s>0 s=0
x/y = =1x/y =
7/25/2019 Camera Geometry and Single View Geometry
30/36
1-30Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
The imaging process of a camera is completely described
by the projection matrix P
The projection matrix P is a 3x4 matrix performing the
transformation from projective space in 3D to a 2D plane
Note that Phas only eleven degrees of freedom, though
it has twelf components. In other words, P is determined
up to an unknown scale factor
Pcan be decomposed in the following manner
3.3 Projective Camera Model
3.3.2 The Projection Matrix P
img wrld X P X
1
1 11 12 13 14
2
2 21 22 23 24
3
3 31 32 33 34
4
xx p p p p
xx p p p p
xx p p p p
x
| | | |
| | | |
1R
2R 1C 2C 3C 4C
3R
p
P p p p p p
p
3 3 Projective Camera Model
7/25/2019 Camera Geometry and Single View Geometry
31/36
1-31Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
Rows
RowsofPcan be seen as planes in the projective space
P3 (Principal Plane, Planes orthogonal to x-, y-axis
containing principal point and origin of image coordinatesystem)
The line formed by the intersection of the rows p2Rand
p3R connects the principal point and the origin of the
image coordinate system
Columns
Columns ofPdenote special points within the projection
plane
p1C, p2C, p3Care the projections of the axis directions of
x-, y- and z-axis.
p4C is the projection of the origin of the world frame.
Projection Centre/Principal Point
3.3 Projective Camera Model
3.3.2 The Projection Matrix P
P C 0
3 3 Projective Camera Model
7/25/2019 Camera Geometry and Single View Geometry
32/36
1-32Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.3 Projective Camera Model
3.3.3 The Projection Matrix P - Matlab Example
0001 %% Camera Matrix
0002 % Intrinsic Camera Parameter
0003 f = 2; % Focal Length
0004 K = diag([f f 1]); % Intrinsic camera Parameter0005 K(1, 3) = 0;
0006 K(2, 3) = 0;
0007
0008 % Extrinsic Camera Parameter
0009 C = [0; 0; 0]; % Camera Centre
0010 R = [1 0 0
0011 0 0 1
0012 0 1 0]; % We change x and y coordinates
0013 t = -R*C; % Translation vector with respect to R and C
00140015 % Setup Projection matrix
0016 P = K*[R t];
0017
0018 %% Graphical Sketch
0019 figure_01 = figure; hold on; grid on;0020 plot3(C(1), C(2), C(3), 'm.', 'Markersize', 20);
0021
0022 %% Draw all Planes encoded in P
0023 [X, Y, Z] = meshgrid( (C(1)-5):0.1:(C(1)+5),...
0024 (C(2)-0):0.1:(C(2)+5),...0025 (C(3)-5):0.1:(C(3)+5));
0026
0027 % The x-axis Plane is the 1rd row of P!
0028 plane = P(1, :);
0029 data = plane(1)*X + plane(2)*Y + plane(3)*Z + plane(4)*1;
0030 p = patch(isosurface(X, Y, Z, data, 0)); hold on;
0031 set(p, 'FaceColor', 'yellow', 'EdgeColor', 'none');
00320033 % The y-axis Plane is the 2rd row of P!
0034 plane = P(2, :);
0035 data = plane(1)*X + plane(2)*Y + plane(3)*Z + plane(4)*1;
0036 p = patch(isosurface(X, Y, Z, data, 0)); hold on;
0037 set(p, 'FaceColor', 'green', 'EdgeColor', 'none');
0038
0039 % The Principal Plane is the 3rd row of P!
0040 plane = P(3, :);
0041 data = plane(1)*X + plane(2)*Y + plane(3)*Z + plane(4)*1;
0042 p = patch(isosurface(X, Y, Z, data, 0)); hold on;
0043 set(p, 'FaceColor', 'red', 'EdgeColor', 'none');
0044
0045 % The Image Plane is the Principal Plane with an offset of f!
0046 plane = P(3, :);
0047 data = plane(1)*X + plane(2)*Y + plane(3)*Z + (plane(4)-f)*1;
0048 p = patch(isosurface(X, Y, Z, data, 0)); hold on;0049 set(p, 'FaceColor', 'blue', 'EdgeColor', 'none');
0050
0051 alpha(0.5); view(110, 15);
0052 axis([C(1)-5 C(1)+5 C(2)-0 C(2)+5 C(3)-5 C(3)+5]);0053
0054 % Save Image
0055 print(figure_01, '-r600', '-dtiff',...
0056 '..\Images\16_Example_Projection_Matrix_01.tif');
0057
0058 % Convert document to rtf
0059 highlight('m05_Example_Camera_Matrix.m', 'rtf', ...
0060 'm05_Example_Camera_Matrix.rtf')
P =
2 0 0 0
0 0 2 0
0 1 0 0
3 3 Projective Camera Model
7/25/2019 Camera Geometry and Single View Geometry
33/36
1-33Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.3 Projective Camera Model
3.3.3 The Projection Matrix P - Matlab Example
0001 %% Camera Matrix
0002 % Intrinsic Camera Parameter
0003 f = 2; % Focal Length
0004 K = diag([f f 1]); % Intrinsic camera Parameter0005 K(1, 3) = 3;
0006 K(2, 3) = 2;
0007
0008 % Extrinsic Camera Parameter
0009 C = [3; 2; 1]; % Camera Centre
0010 R = [1 0 0
0011 0 0 1
0012 0 1 0]; % We change x and y coordinates
0013 t = -R*C; % Translation vector with respect to R and C
00140015 % Setup Projection matrix
0016 P = K*[R t];
0017
0018 %% Graphical Sketch
0019 figure_01 = figure; hold on; grid on;0020 plot3(C(1), C(2), C(3), 'm.', 'Markersize', 20);
0021
0022 %% Draw all Planes encoded in P
0023 [X, Y, Z] = meshgrid( (C(1)-5):0.1:(C(1)+5),...
0024 (C(2)-0):0.1:(C(2)+5),...0025 (C(3)-5):0.1:(C(3)+5));
0026
0027 % The x-axis Plane is the 1rd row of P!
0028 plane = P(1, :);
0029 data = plane(1)*X + plane(2)*Y + plane(3)*Z + plane(4)*1;
0030 p = patch(isosurface(X, Y, Z, data, 0)); hold on;
0031 set(p, 'FaceColor', 'yellow', 'EdgeColor', 'none');
00320033 % The y-axis Plane is the 2rd row of P!
0034 plane = P(2, :);
0035 data = plane(1)*X + plane(2)*Y + plane(3)*Z + plane(4)*1;
0036 p = patch(isosurface(X, Y, Z, data, 0)); hold on;
0037 set(p, 'FaceColor', 'green', 'EdgeColor', 'none');
0038
0039 % The Principal Plane is the 3rd row of P!
0040 plane = P(3, :);
0041 data = plane(1)*X + plane(2)*Y + plane(3)*Z + plane(4)*1;
0042 p = patch(isosurface(X, Y, Z, data, 0)); hold on;
0043 set(p, 'FaceColor', 'red', 'EdgeColor', 'none');
0044
0045 % The Image Plane is the Principal Plane with an offset of f!
0046 plane = P(3, :);
0047 data = plane(1)*X + plane(2)*Y + plane(3)*Z + (plane(4)-f)*1;
0048 p = patch(isosurface(X, Y, Z, data, 0)); hold on;0049 set(p, 'FaceColor', 'blue', 'EdgeColor', 'none');
0050
0051 alpha(0.5); view(110, 15);
0052 axis([C(1)-5 C(1)+5 C(2)-0 C(2)+5 C(3)-5 C(3)+5]);0053
0054 % Save Image
0055 print(figure_01, '-r600', '-dtiff',...
0056 '..\Images\16_Example_Projection_Matrix_01.tif');
0057
0058 % Convert document to rtf
0059 highlight('m05_Example_Camera_Matrix.m', 'rtf', ...
0060 'm05_Example_Camera_Matrix.rtf')
P =
2 3 0 -12
0 2 2 -6
0 1 0 -2
3 3 Projective Camera Model
7/25/2019 Camera Geometry and Single View Geometry
34/36
1-34Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.3 Projective Camera Model
3.3.4 The Projection Matrix P - Example Basler acA640-100gc
3 3 Projective Camera Model
7/25/2019 Camera Geometry and Single View Geometry
35/36
1-35Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz
Computer Vis ionVersion vom 11.05.2015
3.3 Projective Camera Model
3.3.4 The Projection Matrix P - Example Basler acA640-100gc
0001 %% Example Projection Matrix (Computed)
0002
0003 fx = 4.5e-3; % Focal Length 4.5mm
0004 fy = 4.5e-3; % Focal Length 4.5mm
0005 mx = 1/6.5e-6; % no of pixels per mm
0006 my = 1/6.5e-6; % no of pixels per mm
0007 x0 = 640/2;
0008 y0 = 480/2;
0009 ax = fx*mx;
0010 ay = fy*my;
0011 s = 0; % skew factor
00120013 K = [ ax s x0;
0014 0 ay y0;
0015 0 0 1 ];
0016
0017 C = [0; 0; 0];
0018 R = eye(3);
0019 t = -R*C;
0020
0021 P = K*[R t];0022 disp(P);
P (estimated by calibration procedure)
8.275244814e+02 0 3.2435919943e+02
0
0 8.2706873781e+02 2.4445479377e+020
0 0 1 0
P (computed)
692,3076923076 0 320 0
0 692,3076923076 240 0
0 0 1 0
Literature
7/25/2019 Camera Geometry and Single View Geometry
36/36
Professur Digital und Schaltungstechnik Computer Vis ion
Literature
[1]Richard Hartley, Andrew Zisserman. Multiple View Geometry in
computer vision. Cambridge university press, 2003