Cameras, Central Projection, Binocular Vision1rklette/CCV-Auckland/pdfs/A08-Cameras.pdf · 5 Binocular Vision 26/32. 1826 and BeforeDigital CamerasCamera PropertiesCentral ProjectionBinocular

1826 and Before Digital Cameras Camera Properties Central Projection Binocular Vision

Cameras, Central Projection, Binocular Vision1

Lecture 08

See Section 6.1 inReinhard Klette: Concise Computer Vision

Springer-Verlag, London, 2014

ccv.wordpress.fos.auckland.ac.nz

1See last slide for copyright information.1 / 32

ccv.wordpress.fos.auckland.ac.nz


Agenda

1 1826 and Before

2 Digital Cameras

3 Camera Properties

4 Central Projection

5 Binocular Vision

2 / 32


The First Photograph: Projection + Recording

First photograph: 1826 by N. Niepce (1765 – 1833) at Le Gras, France

Eight hours of exposure time, captured on 20 × 25 cm oil-treated bitumen3 / 32


Camera Obscura: Projection Only

Illustration of principle: projected image, but no recording

Was known for thousands of years (e.g. about 2500 years ago in China)16th century: Better quality by inserting a lens into projection hole

4 / 32


Pinhole Camera

Light rays pass through the pinhole and create a top-down projection

[Image by Pbroks13 in the public domain]5 / 32


Agenda

1 1826 and Before

2 Digital Cameras

3 Camera Properties


5 Binocular Vision

6 / 32


Recording Today: Matrix Sensors

Digital camera uses one or several matrix sensors for recording an image

Edges of individual sensor cells (phototransistors) are 1.4µm to 20µm

Produced in charge-coupled device (CCD) or complementary metal-oxidesemiconductor (CMOS) technology

First digital camera: Sony’s Mavica in 19817 / 32


Bayer Pattern

A filter in front of a single-matrix CCD or CMOS sensor

Named after its inventor Bryce Bayer at Eastman Kodak

A mosaic of 2 × 2 identical filter patterns:two sensor cells for Green, one for Red, and one for Blue

8 / 32


An Alternative to Array Sensors: Line Sensors

Just one row of sensor cells (e.g. in a flatbed scanner)Below, left: CCD sensor line for RGB color image recording

Continuous recording creates an “infinite” sequence of scanned rows

Examples of Applications

Industrial inspection (e.g. above a conveyor belt)Aerial recording (see above: three line sensors in one airplane)Panoramic imaging by rotating a line sensor

9 / 32


Agenda

1 1826 and Before

2 Digital Cameras

3 Camera Properties


5 Binocular Vision

10 / 32


Computer Vision Cameras I

The use of high-quality cameras simplifies computer vision solutions

Important properties:

1 Color accuracy

2 Reduced lens distortion

3 Ideal aspect ratio

4 High spatial image resolution (also called high-definition)

5 Large bit depth

6 A high dynamic range (i.e. value accuracy in dark regions of an imageas well as in bright regions of the same image)

7 High speed of frame transfer

11 / 32


Example for High-Speed Recording at 1,000 pps

“Did the mannequin’s head hit the steering wheel?”

Analysis of a car crash test at Daimler A.G. in 200612 / 32


Computer Vision Cameras II

Connected to a computer via a video port or a frame grabber

Software for frame capture or camera control; e.g. for

1 Time synchronization (e.g. for 1,000 pps in example above)

2 Panning

3 Tilting or

4 Zooming

Software for camera calibration; e.g. for

1 Geometric calibration of multi-sensor system or

2 Photometric calibration of sensitivity of individual sensor cells

13 / 32


Digital Video

Two options: Recording of still images or of video data

For a given camera, spatial times temporal resolution is typically a constant

Example:

A camera captures 7,680 × 4,320 (i.e. 33 Mpixel) at 60 fps

Thus: Records 1.99 Gigapixels per second

Possibly also supports to record 2,560 × 1,440 (i.e. 3.7 Mpixel) at 540 fps

Also 1.99 Gigapixels per second

14 / 32


Interlaced or Progressive Video

Interlaced videoScans subsequent frames either at odd or even lines of the image sensor

Half-frames defined by either odd (left) or even (right) row indices

Progressive videoEach frame contains the entire imageProvides the appropriate input for video analysis

15 / 32


Image Resolution and Bit Depth

Aspect Ratio. Each phototransistor is an a× b rectangular cellIdeally, the aspect ratio a/b should be equal to 1 (i.e. square cells)

Megapixel (Mpixel). Number of sensor elements

Example: 4 Mpixel camera (≈ 4, 000, 000 pixel) in some image formatWithout further mentioning, the number of pixels means “color pixel”1991: Kodak offered its DCS-100 with a 1.3 Mpixel sensor array

Sensor Noise and Bit Depth. More pixels: Smaller sensor area per pixelThus less light per sensor area and a worse signal-to-noise ratio (SNR)

Common goal: More than just 8 bits per pixel value in one channelE.g. 16 bits per pixel in a gray-level image for motion or stereo analysis

16 / 32


Color Accuracy

Color checker: A chart of squares showing different gray-levels or colorvalues

Selected window in the red patch and histograms for R, G, B channels

17 / 32


Lens Distortion

Optic lenses contribute radial lens distortion to the projection process

Barrel transform or pincushion transform

Left to right: Barrel transform, ideal rectangular image, pincushiontransform, and projective and lens distortion combined in one image

18 / 32


Linearity of a Camera

Cameras often designed in a way that they correspond to perceivedbrightness in the human eye, which in non-linear

For image analysis purposes we either turn off the non-linearity of createdvalues, or, if not possible, it might be desirable to know a correctionfunction for mapping captured intensities into linearly distributedintensities

Gray-level bar going linearly up from value 0 to value Gmax

19 / 32


Agenda

1 1826 and Before

2 Digital Cameras

3 Camera Properties


5 Binocular Vision

20 / 32


Model of a Pinhole Camera

Theoretical model for light projection through a small holeDiameter of the hole is assumed to be “very close” to zeroThe hole is the projection center

f

Z

P

Yy

x

X

Z =f

Z

Optic axis

Projection center

s

s

s

s

s

u

u

α

W

Left: Sketch of an existing pinhole camera (“shoebox camera”)Point P projected onto an image plane at distance f behind the hole

Right: Model of a pinhole camera,Image (width W , viewing angle α) between world and projection center

21 / 32


3D Sensor Coordinates, Image Plane, Focal Length

3D Sensor Coordinates

In figure above: Right-hand XsYsZs camera coordinate systemSubscript “s” comes from “sensor” (also, e.g., laser range-finder, or radar)

Zs -axis points into the world; is the optic axis

Image Plane

This model excludes the consideration of radial distortionThus: undistorted projected points in image plane with coordinates xu, yu

Focal Length

Distance f between xuyu plane and projection center is the focal length

22 / 32


Central Projection

Zsf

xu

Xs

p=(xu,yu)

P=(Xs,Ys,Zs)

Xs

fxu

Zs

Left: Central projection in the XsZs plane for focal length f

Right: Illustration of ray theorem for xu to Xs and f to Zs

23 / 32


Central Projection Equations

XsYsZs camera coordinates represent points in the 3D world

Visible point P = (Xs ,Ys ,Zs) mapped into p = (xu, yu) in the image plane

Ray theorem of elementary geometry

f to Zs is the same as xu to Xs

f to Zs is the same as yu to Ys

xu =fXs

Zsyu =

fYs

Zs

By knowing xu and yu we cannot recover all three values Xs , Ys , Zs

24 / 32


The Principal Point

Optic axis intersects the image somewhere close to its center

xy image coordinate system: Coordinate origin in the upper left corner

Principal Point

Intersection point (cx , cy ) of optic axis with image plane in xy coordinates

(x , y) = (xu + cx , yu + cy ) = (fXs

Zs+ cx ,

fYs

Zs+ cy )

Pixel location (x , y) in 2D xy image coordinateshas 3D camera coordinates (x − cx , y − cy , f ) in XsYsZs system

Camera calibration has to provide cx , cy , and f (and more)

25 / 32


Agenda

1 1826 and Before

2 Digital Cameras

3 Camera Properties


5 Binocular Vision

26 / 32


Two-Camera Systems

3D geometry of a scene can be measured by using more than one cameraStereo vision or binocular vision: use of two or more cameras

Two Examples of Two-Camera Systems: For Car or Quadcopter

Left: A stereo camera rig on a suction pad with indicated base distance b

Right: Stereo camera system integrated into a quadcopter27 / 32


Base Distance

Camera calibrationneeds to ensure that we have virtually two identical camera

Base distance bthe translational distance between projection centers of both cameras

Also to be calibrated

Figure on page before:

Suction pad: Base distance of about 500 mmQuadcopter: Base distance of 110 mm

28 / 32


Result of Camera Calibration

Two virtually-identical cameras perfectly aligned as illustrated below

Optic axis of left camera Optic axis of right camera

P=(X,Y,Z)

Row y Row y

Left image Right image

Base distance b

xuL xuR

We describe each camera by using the model of a pinhole camera

29 / 32


Canonical Stereo Geometry

XsYsZs camera coordinate system for the left camera

Projection center of the left camera is at (0, 0, 0)

Projection center of the right camera is at (b, 0, 0)

We have

1 Two coplanar images of identical size Ncols × Nrows

2 Parallel optic axes

3 An identical effective focal length f

4 Collinear image rows (i.e., row y in one image is collinear with row yin the second image)

30 / 32


Central Projection in Both Cameras

A visible 3D point P = (Xs ,Ys ,Zs) in the XsYsZs coordinate system ofthe left camera is mapped into undistorted image points

puL = (xuL, yuL) = (f · Xs

Zs,f · Ys

Zs)

puR = (xuR , yuR) = (f · (Xs − b)

Zs,f · Ys

Zs)

in the left and right image plane, respectively

Those two equations are used for stereo vision:

3 input parameter xuL, xuR , and yuL = yuR (same row)3 parameters Xs , Ys , and Zs to be computed

31 / 32


Copyright Information

This slide show was prepared by Reinhard Klettewith kind permission from Springer Science+Business Media B.V.

The slide show can be used freely for presentations.However, all the material is copyrighted.

R. Klette. Concise Computer Vision.c©Springer-Verlag, London, 2014.

In case of citation: just cite the book, that’s fine.

32 / 32

http://www.cs.auckland.ac.nz/~rklette

Documents

Cameras, Central Projection, Binocular Vision1rklette/CCV-Auckland/pdfs/A08-Cameras.pdf · 5 Binocular Vision 26/32. 1826 and BeforeDigital CamerasCamera PropertiesCentral ProjectionBinocular