Camera Geometry and Single View Geometry

Embed Size (px)

Citation preview

  • 7/25/2019 Camera Geometry and Single View Geometry

    1/36

    1-1Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    Chapter 3

    Camera Geometry and

    Single View GeometryVersion 11.05.2015

  • 7/25/2019 Camera Geometry and Single View Geometry

    2/36

    1-2Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    Content

    3.1 Geometrical Imaging Process

    3.1.1 Introduction

    3.1.2 Extrinsic Model3.1.3 Extrinsic ModelMatlab Example

    3.1.4 Intrinsic ModelProjection

    3.1.5 Intrinsic ModelAffine Transformation

    3.1.6 Summary

    3.2 Pinhole Camera Model

    3.2.1 The Geometrical Model of a Pinhole Camera

    3.2.2 The Geometrical Model of a Pinhole CameraMatlab

    Example

    3.2.3 Intrinsic ModelCamera Calibration Matrix K

    3.2.4 Extrinsic Model

    3.2.5 Extrinsic ModelMatlab Example

    3.3 Projective Camera Model

    3.3.1 Intrinsic ModelCamera Calibration Matrix K3.3.2 The Projection Matrix P

    3.3.2 The Projection Matrix PMatlab Example

    3.3.3 The Projection Matrix PExample Basler acA630-

    100gc

  • 7/25/2019 Camera Geometry and Single View Geometry

    3/36

    1-3Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.1 Geometrical Imaging Process

    An imaging device or camera usually maps an observed

    3D scene to a 2D image by projection.

    Several projection principles are known and vitally

    characterize a camera: parallel projection, perspective

    projection, orthographic projection, oblique projection,

    Projection principles can be mathematically approximated

    by an appropriate model which is commonly known as

    camera modelorprojection model.

    Most commonly known imaging devices follows the

    perspective projection principle and is known as pinhole

    camera model

    3.1.1 Introduction

    Principle Point / Camera Center / Optical Center / Pinhole

    Sensor Plane/Image Plane

    Focal length f

    Camera (box)

    Optical axis/principle axis

  • 7/25/2019 Camera Geometry and Single View Geometry

    4/36

    1-4Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.1 Geometrical Imaging Process

    Mathematically the image that has been projected upside-

    down can be flipped using the principle point

    3.1.1 Introduction

    Focal length f

    Sensor Plane/Image Plane

    Principle Plane

  • 7/25/2019 Camera Geometry and Single View Geometry

    5/36

    1-5Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.1 Geometrical Imaging Process

    The principle of the pinhole camera was already

    discovered by Aristoteles (384-322 B.C.).

    First steps with the pinhole camera was taken by the Arab

    Alhazen in 980 as he investigates the human eye.

    At the end of the 13th century astronomics used the

    pinhole camera to observe sunspots.

    3.1.1 Introduction

  • 7/25/2019 Camera Geometry and Single View Geometry

    6/36

    1-6Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.1 Geometrical Imaging Process

    The first published picture of a pinhole camera obscurais apparently a drawing in Gemma Frisius' De Radio Astronomica et

    Geometrica(1545).

    The very term camera obscura("dark room") was coined by Johannes Kepler (15711630).

    The first photography was taken by a pinhole camera in 1816 by Joseph Nicphore Nipcecht. Unfortunately he wasnt

    able to fix it. But a few months later he presents the first photo of the world, a view out of his studyroom.

    3.1.1 Introduction

  • 7/25/2019 Camera Geometry and Single View Geometry

    7/361-7Professur Digital- und Schaltungstechnik

    Prof. Dr.-Ing. Gangolf HirtzComputer Vis ion

    Version vom 11.05.2015

    3.1 Geometrical Imaging Process

    The geometrical relationship between 3D scene points

    Xwrldand the resulting 2D image points Ximgis often quite

    complex due to the physical construction of the imaging

    device.

    By employing a finite amount of numerical parameters,

    the actual process of projection can be replaced by an

    appropriate mathematical relationship of bearable

    computational cost.

    3.1.1 Introduction

    Observed scene

    Camera

    World coordinate system

    wrldX

    Image

    imgX

    zwrld

    xwrldywrld

    3D scene point

    2D

    Image point

    Geometrical imaging process

    wrld

    wrld

    wrld

    wrld

    wrld

    x

    y

    z

    w

    X

    img

    img img

    img

    x

    y

    w

    X

  • 7/25/2019 Camera Geometry and Single View Geometry

    8/361-8Professur Digital- und Schaltungstechnik

    Prof. Dr.-Ing. Gangolf HirtzComputer Vis ion

    Version vom 11.05.2015

    3.1 Geometrical Imaging Process

    The overall imaging process that transfers a scene point

    Xwrld to an image point Ximg successively utilizes an

    extrinsicand an intrinsic model

    3.1.1 Introduction

    Observed scene

    Camera

    wrldX imgX

    zwrld

    xwrldywrld

    Extrinsic model Intrinsic model

    xcamzcam

    ycam

    Camera coordinate system

    yimg

    ximg

    Image coordinate system

    camX

    wrld

    wrld

    wrld

    wrld

    wrld

    x

    y

    z

    w

    X

    img

    img img

    img

    x

    y

    w

    X

    cam

    cam

    cam

    cam

    cam

    x

    y

    z

    w

    X

  • 7/25/2019 Camera Geometry and Single View Geometry

    9/361-9Professur Digital- und Schaltungstechnik

    Prof. Dr.-Ing. Gangolf HirtzComputer Vis ion

    Version vom 11.05.2015

    3.1 Geometrical Imaging Process

    Transformation from world points Xwrldto camera points Xcamin multiple camera environments

    Incorporates the distinct geometrical relationship Hiof each camera with respect to WCS

    3.1.2 Extrinsic Model

    Observed scene

    Camera

    wrldX

    zwrld

    xwrldywrld

    xcamzcam

    Camera

    xcam

    zcam

    cam,0X

    cam,1X

    0 0R C

    1 1R C

    i i ii

    1

    R R CH0

    1

    cam,0 0 wrld

    1

    cam,1 1 wrld

    X H X

    X H X

    1H

    0H

    Ci origin of the camera coordinate

    frame provided in world frame

    coordinates

    Ri 3x3 rotation matrix representing the

    orientation of the cameracoordinate frame with respect to

    the world frame

    R results out of three angles

    discribing a sequence of three

    elementary rotations around x, y, z-

    axis respectively (each performed

    clockwise)

  • 7/25/2019 Camera Geometry and Single View Geometry

    10/361-10Professur Digital- und Schaltungstechnik

    Prof. Dr.-Ing. Gangolf HirtzComputer Vis ion

    Version vom 11.05.2015

    3.1 Geometrical Imaging Process

    Example: Optical sensor network to observe a complex indoor environment (e.g. domestic home environment)

    3.1.3 Extrinsic Model - Matlab Example

    WCS

    Camera 1

    Camera 2

    Camera 3

  • 7/25/2019 Camera Geometry and Single View Geometry

    11/36

    1-11Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.1 Geometrical Imaging Process

    3.1.3 Extrinsic Model - Matlab Example

    0001 clear all; close all; clc;

    0002

    0003 %% Extrinsic Camera Parameter0004 % Camera centres

    0005 C1 = [ 2.6607; 0.1006; 2.4089]; % Smart Sensor 605

    0006 C2 = [-1.6157; 2.7995; 2.4119]; % Smart Sensor 601

    0007 C3 = [-1.8482; -1.7443; 2.3955]; % Smart Sensor 602

    0008

    0009 % Rotation matrices

    0010 R1 = [ -0.7770 0.6286 0.03390011 0.3719 0.5018 -0.7809

    0012 -0.5079 -0.5942 -0.6237];

    0013

    0014 R2 = [ -0.5233 -0.8521 0.0080

    0015 -0.5388 0.3236 -0.7778

    0016 0.6602 -0.4113 -0.6285];

    0017

    0018 R3 = [ -0.4861 -0.8670 0.1097

    0019 -0.6689 0.2884 -0.6852

    0020 0.5624 -0.4065 -0.7201];

    0021

  • 7/25/2019 Camera Geometry and Single View Geometry

    12/36

    1-12Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.1 Geometrical Imaging Process

    3.1.3 Extrinsic Model - Matlab Example

    0022 % Translation vectors

    0023 T1 = -R1*C1;

    0024 T2 = -R2*C2;0025 T3 = -R3*C3;

    0026

    0027 %% Setup Homographies

    0028 H1 = [R1 T1; 0 0 0 1];

    0029 H2 = [R2 T2; 0 0 0 1];

    0030 H3 = [R3 T3; 0 0 0 1];

    00310032 %% Plot WCS

    0033 h1 = figure(1); hold on; grid on; axis equal;

    0034 xlabel('x [m]'); ylabel('y [m]'); zlabel('z [m]'); view(70, 20);

    0035 axis([-3 +3 -3 +3 0 3]);

    0036

    0037 i = [1; 0; 0];

    0038 j = [0; 1; 0];

    0039 k = [0; 0; 1];

    0040 o = [0; 0; 0];

    0041

    0042 plot3([o(1) i(1)], [o(2) i(2)], [o(3) i(3)], 'r-', 'MarkerSize', 5);

    0043 plot3([o(1) j(1)], [o(2) j(2)], [o(3) j(3)], 'g-', 'MarkerSize', 5);

    0044 plot3([o(1) k(1)], [o(2) k(2)], [o(3) k(3)], 'b-', 'MarkerSize', 5);

  • 7/25/2019 Camera Geometry and Single View Geometry

    13/36

    1-13Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.1 Geometrical Imaging Process

    3.1.3 Extrinsic Model - Matlab Example

    0045

    0046 %% Plot camera coordinate systems

    0047 i1 = H1^-1*[i; 1];0048 j1 = H1^-1*[j; 1];

    0049 k1 = H1^-1*[k; 1];

    0050 o1 = H1^-1*[o; 1];

    0051

    0052 plot3([o1(1) i1(1)], [o1(2) i1(2)], [o1(3) i1(3)], 'r-', 'MarkerSize', 1);

    0053 plot3([o1(1) j1(1)], [o1(2) j1(2)], [o1(3) j1(3)], 'g-', 'MarkerSize', 1);

    0054 plot3([o1(1) k1(1)], [o1(2) k1(2)], [o1(3) k1(3)], 'b-', 'MarkerSize', 1);0055

    0056 i2 = H2^-1*[i; 1];

    0057 j2 = H2^-1*[j; 1];

    0058 k2 = H2^-1*[k; 1];

    0059 o2 = H2^-1*[o; 1];

    0060

    0061 plot3([o2(1) i2(1)], [o2(2) i2(2)], [o2(3) i2(3)], 'r-', 'MarkerSize', 1);

    0062 plot3([o2(1) j2(1)], [o2(2) j2(2)], [o2(3) j2(3)], 'g-', 'MarkerSize', 1);

    0063 plot3([o2(1) k2(1)], [o2(2) k2(2)], [o2(3) k2(3)], 'b-', 'MarkerSize', 1);

  • 7/25/2019 Camera Geometry and Single View Geometry

    14/36

    1-14Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.1 Geometrical Imaging Process

    3.1.3 Extrinsic Model - Matlab Example

    0064

    0065 i3 = H3^-1*[i; 1];

    0066 j3 = H3^-1*[j; 1];

    0067 k3 = H3^-1*[k; 1];0068 o3 = H3^-1*[o; 1];

    0069

    0070 plot3([o3(1) i3(1)], [o3(2) i3(2)], [o3(3) i3(3)], 'r-', 'MarkerSize', 1);

    0071 plot3([o3(1) j3(1)], [o3(2) j3(2)], [o3(3) j3(3)], 'g-', 'MarkerSize', 1);

    0072 plot3([o3(1) k3(1)], [o3(2) k3(2)], [o3(3) k3(3)], 'b-', 'MarkerSize', 1);

    1H

    0H

    2H

    Observed Indoor Scene

    Floor (z == 0 m)

    WCS

  • 7/25/2019 Camera Geometry and Single View Geometry

    15/36

    1-15Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.1 Geometrical Imaging Process

    The Projection process from Xcam to normalized sensor

    coordinates Xnorm is modelled by a projection function F

    The cameras sensor plane is assumed to have unit

    length from the principle point

    3.1.4 Intrinsic Model - Projection

    Observed scene

    ynorm

    xnorm

    xcam

    ycamcamX

    zcam

    Normalized sensor coordinate system

    normXF

    Camera centre/Principle point

    Camera sensor plane

    Focal length f normalizedto unit length

    Projection

    norm

    norm norm

    1

    x

    y

    X

    cam

    cam

    camcam

    cam

    x

    y

    z

    w

    X

  • 7/25/2019 Camera Geometry and Single View Geometry

    16/36

    1-16Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.1 Geometrical Imaging Process

    The camera coordinate Xcam is considered as incident

    light ray(,,)

    The radial projection of an incoming light ray (,,)

    onto a virtual image plane with a distance of f= 1 to the

    projection center can be modelled with the radial

    projection function F():

    3.1.4 Intrinsic Model - Projection

    Observed scene

    ynorm

    xnorm

    xcam

    ycamcam X

    zcam

    elevation

    azimuth

    ()

    cam

    cam cam

    cam

    x

    y

    z

    X

    norm

    norm r

    norm

    r

    ( ) ( )

    coswithsin

    xF

    y

    X u

    u

    normXF

    distance

    ur

  • 7/25/2019 Camera Geometry and Single View Geometry

    17/36

    1-17Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.1 Geometrical Imaging Process

    Projection

    Type

    Function F Properties and Applications

    Perspective

    projection

    = tan Applied in conventional

    cameras that follow the pinhole

    principle with <

    Stereographic

    projection = 2 tan

    Commonly used for the

    generation of stellar maps and

    photography. angles are

    preserved (equal angles) and

    circles are imaged without

    distortions (equal circles).

    Equidistanceprojection

    = It is the most popular model fordescribing fisheye projection

    lenses. Incidence angles of rays

    are imaged linearly.

    Equisolid

    angle

    projection

    = 2 sin

    2

    Each pixel corresponds to one

    distinct solid angle which makes

    the image look like the reflection

    by a specular ball.

    Orthogonal

    projection = sin This camera principle models

    the projection onto a sphere.

    The projection radius is

    increasing with increasing

    incidence angle

    until reachespi/2. Than it is

    declining again.

    3.1.4 Intrinsic Model - Projection

  • 7/25/2019 Camera Geometry and Single View Geometry

    18/36

    1-18Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.1 Geometrical Imaging Process

    3.1.4 Intrinsic Model - Projection

    Equiangular Projection

    Applied by e.g. fish-eye cameras

    FOV > 180

    Perspective Projection

    Most commonly applied projection method

    Limited field of view (FOV) to approx. 120()

    = tan =

  • 7/25/2019 Camera Geometry and Single View Geometry

    19/36

    1-19Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.1 Geometrical Imaging Process

    3.1.5 Intrinsic Model Affine Transformation

    The transformation from normalized sensor coordinates to

    image coordinates is finally done by means of an affine

    transformation matrixK

    Kis called camera calibration matrix

    imgX

    Affine

    Transformation

    yimg

    ximg

    Image coordinate system

    img

    img img

    img

    x

    y

    w

    X

    norm

    norm norm

    1

    x

    y

    X

    ynorm

    xnormnormX

    Normalized sensor coordinate system

    K

    img norm X K X

  • 7/25/2019 Camera Geometry and Single View Geometry

    20/36

    1-20Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.1 Geometrical Imaging Process

    3.1.6 Summary

    Geometrical imaging process is characterized by an

    extrinsic and intrinsic model

    Intrinsic model (camera model) is split into different

    projection models Fand an affine transformation K

    Relevant camera models are

    Pinehole camera model

    Projective camera model

  • 7/25/2019 Camera Geometry and Single View Geometry

    21/36

    1-21Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    Central projection is mapping a point Xcamin 3D space to

    a point Ximgin 2D space:

    Using similar triangles one verifies that:

    Using homogeneous coordinates, one can put that

    transformation in a matrix expression:

    The 3x4 transformation matrix used is called a camera

    projection matrixP

    Pcan be decomposed in the following manner:

    The Diagonal Matrix is called the camera calibration

    matrixK

    3.2 Pinhole Camera Model

    3.2.1 The Geometrical Model of a Pinhole Camera

    x

    object point in

    3D world frameimage point in 2D

    image frame

    imaging process

    cam cam cam img img, , ,TT

    x y z x y

    f

    fcam

    z

    cam cam/f y z

    cam cam cam cam cam cam cam, , / , /T T

    x y z f x z f y z

    cam

    img camcam

    img cam

    camcam cam

    cam

    normalized sensor coordinatestan

    01 1

    0

    1 1 01

    xx f x f

    yy f y f

    zz zz

    img cam

    X P X

    diag( , ,1) |f f P I 0

    ( , ,1)diag f f K

    camX

    camXimgX

  • 7/25/2019 Camera Geometry and Single View Geometry

    22/36

    1-22Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.2 Pinhole Camera Model

    3.2.2 The Geometrical Model of a Pinhole Camera - Matlab Example

    0001 %% Camera Matrix

    0002 % Intrinsic Camera Parameter

    0003 f = 4; % Focal Length

    00040005 % Setup Projection matrix

    0006 P = diag([f f 1])*[eye(3) zeros(3, 1)];

    0007

    0008 %% Project World Point

    0009 Xw = [4; 5; 9]; % Object Point (World Frame)

    0010 % X = 4, Y = 5, Z = 9

    0011 Xw_h = [Xw; 1]; % Transform to homogeneous Coordinates

    0012 Xc_h = P*Xw_h; % Project!

    00130014 Xc = 1/Xw(3)*Xc_h;% Image Point (Camera Frame)

    0015

    0016 %% Plot Principal Point

    0017 figure_01 = figure; hold on; grid on;

    0018 plot3(0, 0, 0, 'm.', 'Markersize', 20);

    0019

    0020 %% Draw Image Plane

    0021 [X, Y, Z] = meshgrid(-5:0.1:5, -5:0.1:5, -5:0.1:5);

    0022

    0023 data = 0*X+0*Y+1*Z-f; % Normal from for a plane

    0024 p = patch(isosurface(X, Y, Z, data, 0)); hold on;

    0025 set(p, 'FaceColor', 'red', 'EdgeColor', 'none')

    0026 alpha(0.5); view(17,24);

    0027 axis([-5 5 -5 5 0 10]);

    0028

  • 7/25/2019 Camera Geometry and Single View Geometry

    23/36

    1-23Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.2 Pinhole Camera Model

    3.2.2 The Geometrical Model of a Pinhole Camera - Matlab Example

    0029 %% Draw Object and Image Point

    0030 plot3(Xw(1), Xw(2), Xw(3), 'b*', 'Markersize', 10);

    0031 plot3(Xc(1), Xc(2), f, 'b*', 'Markersize', 5);

    0032 plot3([0 Xw(1)], [0 Xw(2)], [0 Xw(3)], 'b--');0033

    0034 %% Add some text

    0035 text(0.5, 0.5, 0,...

    0036 'Principal Point', 'FontSize', 10,...

    0037 'FontWeight', 'bold', 'Color', 'magenta');

    0038 text(Xw(1) - 3.0, Xw(2) - 3.0, Xw(3),...0039 'Object Point', 'FontSize', 10, 'FontWeight', 'bold', 'Color', 'blue');

    0041 text(Xc(1) + 0.5, Xc(2) + 0.5, f,...

    0042 'Image Point', 'FontSize', 10,...0043 'FontWeight', 'bold', 'Color', 'blue');

    0044 text(-2, -2, f,...

    0045 'Image Plane', 'FontSize', 10,...

    0046 'FontWeight', 'bold', 'Color', 'red');

    3 2 C

  • 7/25/2019 Camera Geometry and Single View Geometry

    24/36

    1-24Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    It is common for images to have only positive coordinates

    (image coordinates start at the upper left corner of the

    image), so a further adaption has to be done

    The principal point (cx, cy) is the offset with respect to the

    image frame and enhances the camera calibration matrix

    Kis an upper triangle matrix with 3 dof

    A camera is denoted as acalibrated cameraso soon as

    Kis known

    The pinhole camera model is a ideal one assuming that

    image coordinates are Euclidean coordinates having

    equal scales in both axial directions

    3.2 Pinhole Camera Model

    3.2.3 Intrinsic Model - Camera Calibration Matrix K

    img img img imgx yx x c y y c

    1

    x

    y

    f cf c

    K

    imgX

    Image coordinate system

    yimg

    ximg

    yimg

    ximg

    Principal Point (cx, cy)

    3 2 Pi h l C M d l

  • 7/25/2019 Camera Geometry and Single View Geometry

    25/36

    1-25Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    Every single cameras position C is described by a

    rotation Rand a translation Twith respect to the world

    coordinate system

    A Point Xwrld can be translated into a point Xcamby the

    following mathematical relationship

    Homogeneous coordinates let us use this relation in a

    more compact matrix form

    The overall imaging process is now as follows:

    3.2 Pinhole Camera Model

    3.2.4 Extrinsic Model

    With the new relationship one can determine the image

    coordinates of a point given in world coordinates as

    follows:

    Pcan be computed from point correspondences Xwrld

    Ximg

    cam wrld X R X C

    cam wrld1

    R RCX X

    img wrld with | X P X P KR I C

    img cam

    img cam

    img wrld

    img wrld

    Translation vector

    img wrld

    |

    |1

    |

    |

    T

    X P X

    X K I 0 X

    R RCX K I 0 X

    X K R RC X

    X K R I C X

    1H

    0H

    2H

    WCS

    3 2 Pi h l C M d l

  • 7/25/2019 Camera Geometry and Single View Geometry

    26/36

    1-26Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.2 Pinhole Camera Model

    3.2.5 Extrinsic Model Matlab Example

    0001 %% Camera Matrix

    0002 % Intrinsic Camera Parameter

    0003 f = 2; % Focal Length

    0004 K = diag([f f 1]); % Intrinsic camera Parameter0005

    0006 % Extrinsic Camera Parameter

    0007 C1 = [-3; +4; -3]; % Camera Centre Cam 1

    0008 C2 = [3; -4; 1]; % Camera Centre Cam 2

    0009 C3 = [3; -5; -5]; % Camera Centre Cam 2

    0010

    0011 % Rotate Cam 1 by 0, Cam2 by 45 and Cam3 by 900012 % arround the x-axis

    0013 R1 = [1.0000 0 00014 0 1.0000 0

    0015 0 0 1.0000];

    0016 R2 = [1.0000 0 0

    0017 0 -0.7071 0.7071

    0018 0 0.7071 0.7071];

    0019 R3 = [1.0000 0 0

    0020 0 0 1.0000

    0021 0 1.0000 0];

    00220023 t1 = -R1*C1; % Translation vector with respect to R and C

    0024 t2 = -R2*C2; % Translation vector with respect to R and C

    0025 t3 = -R3*C3; % Translation vector with respect to R and C

    0026

    3 2 Pi h l C M d l

  • 7/25/2019 Camera Geometry and Single View Geometry

    27/36

    1-27Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.2 Pinhole Camera Model

    3.2.5 Extrinsic Model Matlab Example

    0027 % Setup Projection matrix

    0028 P1 = K*[R1 t1];

    0029 P2 = K*[R2 t2];

    0030 P3 = K*[R3 t3];0031

    0032 %% Plot Principal Points

    0033 figure_01 = figure; hold on; grid on;

    0034 plot3(C1(1), C1(2), C1(3), 'r.', 'Markersize', 20);

    0035 plot3(C2(1), C2(2), C2(3), 'g.', 'Markersize', 20);

    0036 plot3(C3(1), C3(2), C3(3), 'b.', 'Markersize', 20);

    0037

    0038 %% Draw Image Plane Camera 1

    0039 [X, Y, Z] = meshgrid( (C1(1)-3):0.1:(C1(1)+3),...0040 (C1(2)-3):0.1:(C1(2)+3),...

    0041 (C1(3)-3):0.1:(C1(3)+3));

    0042 plane = P1(3, :);0043 data = plane(1)*X + plane(2)*Y + plane(3)*Z + (plane(4)-f)*1;

    0044 p = patch(isosurface(X, Y, Z, data, 0)); hold on;

    0045 set(p, 'FaceColor', 'red', 'EdgeColor', 'none');

    0046 %% Draw Image Plane Camera 2

    0047 [X, Y, Z] = meshgrid( (C2(1)-3.0):0.1:(C2(1)+3.0),...

    0048 (C2(2)-4.2):0.1:(C2(2)+4.2),...0049 (C2(3)-4.2):0.1:(C2(3)+4.2));

    0050 plane = P2(3, :);

    0051 data = plane(1)*X + plane(2)*Y + plane(3)*Z + (plane(4)-f)*1;

    0052 p = patch(isosurface(X, Y, Z, data, 0)); hold on;

    0053 set(p, 'FaceColor', 'green', 'EdgeColor', 'none');

    0054

    3 2 Pi h l C M d l

  • 7/25/2019 Camera Geometry and Single View Geometry

    28/36

    1-28Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.2 Pinhole Camera Model

    3.2.5 Extrinsic Model Matlab Example

    0055 %% Draw Image Plane Camera 3

    0056 [X, Y, Z] = meshgrid( (C3(1)-3):0.1:(C3(1)+3),...

    0057 (C3(2)-3):0.1:(C3(2)+3),...

    0058 (C3(3)-3):0.1:(C3(3)+3));0059 plane = P3(3, :);

    0060 data = plane(1)*X + plane(2)*Y + plane(3)*Z + (plane(4)-f)*1;

    0061 p = patch(isosurface(X, Y, Z, data, 0)); hold on;

    0062 set(p, 'FaceColor', 'blue', 'EdgeColor', 'none');

    0063

    0064

    0065 alpha(0.5); view(140, 35);

    0066 axis([C(1)-10 C(1)+10 C(2)-10 C(2)+10 C(3)-10 C(3)+10]);

    3 3 Projecti e Camera Model

  • 7/25/2019 Camera Geometry and Single View Geometry

    29/36

    1-29Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    In the case of CCD cameras there is the possibility to

    have non-square pixels and hence having unequal scale

    factors for both image directions.

    Imagine a pixel density(no. of pixels per unit length) of

    mx(x-direction) and my(y-direction), the calibration matrix

    Kchanges to

    wherexandyrepresent the focal length with respect

    to the pixel dimensions in both directions.

    Although it is very seldom for some rare cases it is

    necessary to introduce the skew factor s, defining the

    geometrical dependency of the y-dimension in pixel

    frame from the x-axis:

    A camera having Pas described above is known as finite

    projective camera.and has 11 degrees of freedom

    3.3 Projective Camera Model

    3.3.1 Intrinsic Model - Camera Calibration Matrix K

    Aspect Ratio

    The ratio ofxandy is often calledaspect ratioand is

    very close to 1.

    mxno. of pixel per m along the x-axis, hor. pixel density,

    [px/m]

    myno. of pixel per m along the y-axis,vert. pixel density,

    [px/m],

    Shear angle between the image axes; is encoded

    within s

    (x,y) focal length, with respect to pixel dimensions

    with , ,

    1

    x x

    y y x x y y

    s c

    c f m f m s f

    K

    s>0 s=0

    x/y = =1x/y =

  • 7/25/2019 Camera Geometry and Single View Geometry

    30/36

    1-30Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    The imaging process of a camera is completely described

    by the projection matrix P

    The projection matrix P is a 3x4 matrix performing the

    transformation from projective space in 3D to a 2D plane

    Note that Phas only eleven degrees of freedom, though

    it has twelf components. In other words, P is determined

    up to an unknown scale factor

    Pcan be decomposed in the following manner

    3.3 Projective Camera Model

    3.3.2 The Projection Matrix P

    img wrld X P X

    1

    1 11 12 13 14

    2

    2 21 22 23 24

    3

    3 31 32 33 34

    4

    xx p p p p

    xx p p p p

    xx p p p p

    x

    | | | |

    | | | |

    1R

    2R 1C 2C 3C 4C

    3R

    p

    P p p p p p

    p

    3 3 Projective Camera Model

  • 7/25/2019 Camera Geometry and Single View Geometry

    31/36

    1-31Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    Rows

    RowsofPcan be seen as planes in the projective space

    P3 (Principal Plane, Planes orthogonal to x-, y-axis

    containing principal point and origin of image coordinatesystem)

    The line formed by the intersection of the rows p2Rand

    p3R connects the principal point and the origin of the

    image coordinate system

    Columns

    Columns ofPdenote special points within the projection

    plane

    p1C, p2C, p3Care the projections of the axis directions of

    x-, y- and z-axis.

    p4C is the projection of the origin of the world frame.

    Projection Centre/Principal Point

    3.3 Projective Camera Model

    3.3.2 The Projection Matrix P

    P C 0

    3 3 Projective Camera Model

  • 7/25/2019 Camera Geometry and Single View Geometry

    32/36

    1-32Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.3 Projective Camera Model

    3.3.3 The Projection Matrix P - Matlab Example

    0001 %% Camera Matrix

    0002 % Intrinsic Camera Parameter

    0003 f = 2; % Focal Length

    0004 K = diag([f f 1]); % Intrinsic camera Parameter0005 K(1, 3) = 0;

    0006 K(2, 3) = 0;

    0007

    0008 % Extrinsic Camera Parameter

    0009 C = [0; 0; 0]; % Camera Centre

    0010 R = [1 0 0

    0011 0 0 1

    0012 0 1 0]; % We change x and y coordinates

    0013 t = -R*C; % Translation vector with respect to R and C

    00140015 % Setup Projection matrix

    0016 P = K*[R t];

    0017

    0018 %% Graphical Sketch

    0019 figure_01 = figure; hold on; grid on;0020 plot3(C(1), C(2), C(3), 'm.', 'Markersize', 20);

    0021

    0022 %% Draw all Planes encoded in P

    0023 [X, Y, Z] = meshgrid( (C(1)-5):0.1:(C(1)+5),...

    0024 (C(2)-0):0.1:(C(2)+5),...0025 (C(3)-5):0.1:(C(3)+5));

    0026

    0027 % The x-axis Plane is the 1rd row of P!

    0028 plane = P(1, :);

    0029 data = plane(1)*X + plane(2)*Y + plane(3)*Z + plane(4)*1;

    0030 p = patch(isosurface(X, Y, Z, data, 0)); hold on;

    0031 set(p, 'FaceColor', 'yellow', 'EdgeColor', 'none');

    00320033 % The y-axis Plane is the 2rd row of P!

    0034 plane = P(2, :);

    0035 data = plane(1)*X + plane(2)*Y + plane(3)*Z + plane(4)*1;

    0036 p = patch(isosurface(X, Y, Z, data, 0)); hold on;

    0037 set(p, 'FaceColor', 'green', 'EdgeColor', 'none');

    0038

    0039 % The Principal Plane is the 3rd row of P!

    0040 plane = P(3, :);

    0041 data = plane(1)*X + plane(2)*Y + plane(3)*Z + plane(4)*1;

    0042 p = patch(isosurface(X, Y, Z, data, 0)); hold on;

    0043 set(p, 'FaceColor', 'red', 'EdgeColor', 'none');

    0044

    0045 % The Image Plane is the Principal Plane with an offset of f!

    0046 plane = P(3, :);

    0047 data = plane(1)*X + plane(2)*Y + plane(3)*Z + (plane(4)-f)*1;

    0048 p = patch(isosurface(X, Y, Z, data, 0)); hold on;0049 set(p, 'FaceColor', 'blue', 'EdgeColor', 'none');

    0050

    0051 alpha(0.5); view(110, 15);

    0052 axis([C(1)-5 C(1)+5 C(2)-0 C(2)+5 C(3)-5 C(3)+5]);0053

    0054 % Save Image

    0055 print(figure_01, '-r600', '-dtiff',...

    0056 '..\Images\16_Example_Projection_Matrix_01.tif');

    0057

    0058 % Convert document to rtf

    0059 highlight('m05_Example_Camera_Matrix.m', 'rtf', ...

    0060 'm05_Example_Camera_Matrix.rtf')

    P =

    2 0 0 0

    0 0 2 0

    0 1 0 0

    3 3 Projective Camera Model

  • 7/25/2019 Camera Geometry and Single View Geometry

    33/36

    1-33Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.3 Projective Camera Model

    3.3.3 The Projection Matrix P - Matlab Example

    0001 %% Camera Matrix

    0002 % Intrinsic Camera Parameter

    0003 f = 2; % Focal Length

    0004 K = diag([f f 1]); % Intrinsic camera Parameter0005 K(1, 3) = 3;

    0006 K(2, 3) = 2;

    0007

    0008 % Extrinsic Camera Parameter

    0009 C = [3; 2; 1]; % Camera Centre

    0010 R = [1 0 0

    0011 0 0 1

    0012 0 1 0]; % We change x and y coordinates

    0013 t = -R*C; % Translation vector with respect to R and C

    00140015 % Setup Projection matrix

    0016 P = K*[R t];

    0017

    0018 %% Graphical Sketch

    0019 figure_01 = figure; hold on; grid on;0020 plot3(C(1), C(2), C(3), 'm.', 'Markersize', 20);

    0021

    0022 %% Draw all Planes encoded in P

    0023 [X, Y, Z] = meshgrid( (C(1)-5):0.1:(C(1)+5),...

    0024 (C(2)-0):0.1:(C(2)+5),...0025 (C(3)-5):0.1:(C(3)+5));

    0026

    0027 % The x-axis Plane is the 1rd row of P!

    0028 plane = P(1, :);

    0029 data = plane(1)*X + plane(2)*Y + plane(3)*Z + plane(4)*1;

    0030 p = patch(isosurface(X, Y, Z, data, 0)); hold on;

    0031 set(p, 'FaceColor', 'yellow', 'EdgeColor', 'none');

    00320033 % The y-axis Plane is the 2rd row of P!

    0034 plane = P(2, :);

    0035 data = plane(1)*X + plane(2)*Y + plane(3)*Z + plane(4)*1;

    0036 p = patch(isosurface(X, Y, Z, data, 0)); hold on;

    0037 set(p, 'FaceColor', 'green', 'EdgeColor', 'none');

    0038

    0039 % The Principal Plane is the 3rd row of P!

    0040 plane = P(3, :);

    0041 data = plane(1)*X + plane(2)*Y + plane(3)*Z + plane(4)*1;

    0042 p = patch(isosurface(X, Y, Z, data, 0)); hold on;

    0043 set(p, 'FaceColor', 'red', 'EdgeColor', 'none');

    0044

    0045 % The Image Plane is the Principal Plane with an offset of f!

    0046 plane = P(3, :);

    0047 data = plane(1)*X + plane(2)*Y + plane(3)*Z + (plane(4)-f)*1;

    0048 p = patch(isosurface(X, Y, Z, data, 0)); hold on;0049 set(p, 'FaceColor', 'blue', 'EdgeColor', 'none');

    0050

    0051 alpha(0.5); view(110, 15);

    0052 axis([C(1)-5 C(1)+5 C(2)-0 C(2)+5 C(3)-5 C(3)+5]);0053

    0054 % Save Image

    0055 print(figure_01, '-r600', '-dtiff',...

    0056 '..\Images\16_Example_Projection_Matrix_01.tif');

    0057

    0058 % Convert document to rtf

    0059 highlight('m05_Example_Camera_Matrix.m', 'rtf', ...

    0060 'm05_Example_Camera_Matrix.rtf')

    P =

    2 3 0 -12

    0 2 2 -6

    0 1 0 -2

    3 3 Projective Camera Model

  • 7/25/2019 Camera Geometry and Single View Geometry

    34/36

    1-34Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.3 Projective Camera Model

    3.3.4 The Projection Matrix P - Example Basler acA640-100gc

    3 3 Projective Camera Model

  • 7/25/2019 Camera Geometry and Single View Geometry

    35/36

    1-35Professur Digital- und SchaltungstechnikProf. Dr.-Ing. Gangolf Hirtz

    Computer Vis ionVersion vom 11.05.2015

    3.3 Projective Camera Model

    3.3.4 The Projection Matrix P - Example Basler acA640-100gc

    0001 %% Example Projection Matrix (Computed)

    0002

    0003 fx = 4.5e-3; % Focal Length 4.5mm

    0004 fy = 4.5e-3; % Focal Length 4.5mm

    0005 mx = 1/6.5e-6; % no of pixels per mm

    0006 my = 1/6.5e-6; % no of pixels per mm

    0007 x0 = 640/2;

    0008 y0 = 480/2;

    0009 ax = fx*mx;

    0010 ay = fy*my;

    0011 s = 0; % skew factor

    00120013 K = [ ax s x0;

    0014 0 ay y0;

    0015 0 0 1 ];

    0016

    0017 C = [0; 0; 0];

    0018 R = eye(3);

    0019 t = -R*C;

    0020

    0021 P = K*[R t];0022 disp(P);

    P (estimated by calibration procedure)

    8.275244814e+02 0 3.2435919943e+02

    0

    0 8.2706873781e+02 2.4445479377e+020

    0 0 1 0

    P (computed)

    692,3076923076 0 320 0

    0 692,3076923076 240 0

    0 0 1 0

    Literature

  • 7/25/2019 Camera Geometry and Single View Geometry

    36/36

    Professur Digital und Schaltungstechnik Computer Vis ion

    Literature

    [1]Richard Hartley, Andrew Zisserman. Multiple View Geometry in

    computer vision. Cambridge university press, 2003