CPSC 425: Computer Vision

Instructor: Fred Tungftung@cs.ubc.ca

Department of Computer ScienceUniversity of British Columbia

Lecture Notes 2015/2016 Term 2

1 / 55

Menu January 7, 2016

Topics:Image FormationCameras and Lenses

Reading:Today: Forsyth & Ponce (2nd ed.) 1.1.1–1.1.3Next: Forsyth & Ponce (2nd ed.) 4.1, 4.5

Reminders:Complete Assignment 1 by Tuesday, January 12www: http://www.cs.ubc.ca/~ftung/cpsc425/piazza: https://piazza.com/ubc.ca/winterterm22015/cpsc425/

2 / 55

Today’s “Fun” Example: Eye in Sink Illusion

“Tried taking a picture of a sink draining, wound up with a picture of aneye instead”

Photo credit: reddit user Liammm

3 / 55

Today’s “Fun” Example: Eye in Sink Illusion

“Tried taking a picture of a sink draining, wound up with a picture of aneye instead”

Photo credit: reddit user Liammm

4 / 55

Lecture 1: Re-cap

What we see depends on:— object shape— surface material— illumination— viewpoint

Visual perception also is influenced by:— familiarity— context— expectation

5 / 55

Lecture 1: Re-cap

Computer vision technologies have moved from research labs intocommercial products and services. Examples cited include:— broadcast television sports— electronic games (Microsoft Kinect)— real-time language translation— image search— smart infrastructure

6 / 55

Framework for Today’s Topic

Problem: Obtain information about the 3D world

Key Idea(s): Treat a “camera” as a scientific instrument for obtainingmeasurements of the 3D world

Alternatives:— Treat “images” as 2D entities only— Treat “images” as just another kind of “big data”

Theory: Optics (geometry and radiometry)

Practical Detail(s): Cameras and lenses

“Gotchas:”— interpretation of 3D world can be ambiguous— role of human perception

7 / 55

Overview: Image Formation, Cameras and Lenses

Goal: to understand how images are formed

Camera obscura dates from 16th century (and earlier)

Basic abstraction is the pinhole camera

Cameras and lenses maintain the abstraction

The human eye functions very much like a camera

8 / 55

Camera Obscura (Latin for ‘dark chamber’)

Reinerus Gemma-Frisius observed an eclipse of the sun at Louvain onJanuary 24, 1544. He used this illustration in his book, “De RadioAstronomica et Geometrica,” 1545. It is thought to be the firstpublished illustration of a camera obscura.

Credit: John H., Hammond, “The Camera Obscura, A Chronicle”

9 / 55

First Photograph on RecordLa table servie

Credit: Nicéphore Niepce, 1822

10 / 55

Pinhole CameraA pinhole camera is a box with a small hole in it

11 / 55

Image Formation

Forsyth & Ponce (2nd ed.) Figure 1.1

Credit: US Navy, Basic Optics and Optical Instruments. Dover, 1969

12 / 55

Pinhole Camera (Simplified)

x’

x

zf’

imageplane

pinhole object

13 / 55

Pinhole Camera (Simplified) (cont’d)

x’

x

zf’

imageplane

pinhole object

f’

x’

imageplane

14 / 55

Perspective EffectsFar objects appear smaller than close ones

Size is inversely proportional to distance

15 / 55

Perspective Effects (cont’d)Parallel lines meet

16 / 55

Vanishing Points

17 / 55

Vanishing Points

Slide credit: David Jacobs

18 / 55

Vanishing Points (cont’d)

Each set of parallel lines meets at a different point— the vanishing point for this direction

Sets of parallel lines on the same plane lead to collinear vanishingpoints— the line is called the horizon for that plane

Good ways to spot faked images— scale and perspective don’t work— vanishing points behave badly

19 / 55

20 / 55

Perspective Projection

21 / 55

Perspective Projection

Forsyth & Ponce (1st ed.) Figure 1.4

3D object point, P[x , y , z], projects to 2D image point P ′[x ′, y ′] where

x ′ = f ′xz

y ′ = f ′yz

22 / 55

Weak Perspective

23 / 55

Weak Perspective

Forsyth & Ponce Figure 1.5 (1st ed.)

3D object point, P[x , y , z] in Π0, projects to 2D image point P ′[x ′, y ′]where

x ′ = m xy ′ = m y

and m =f ′

z024 / 55

Orthographic Projection

Forsyth & Ponce (1st ed.) Figure 1.6

3D object point, P[x , y , z], projects to 2D image point P ′[x ′, y ′] where

x ′ = xy ′ = y

25 / 55

Summary of Projection Equations3D world point, P[x , y , z], projects to 2D image point P ′[x ′, y ′] where

Perspectivex ′ = f ′

xz

y ′ = f ′yz

Weak Perspectivex ′ = m x

y ′ = m ym =

f ′

z0

Orthographicx ′ = x

y ′ = y

26 / 55

Projection Models: Pros and Cons

Weak perspective (including orthographic) has simplermathematics— Accurate when object is small and/or distant— Useful for recognition

Perspective more accurate for real scenes— Useful in structure from motion

When maximum accuracy required, it is necessary to modeladditional details of the particular camera— Use perspective projection with other calibration

parameters (e.g., radial lens distortion)

27 / 55

Why Not a Pinhole Camera?

Credit: E. Hecht. “Optics,” Addison-Wesley, 1987

28 / 55

Why Not a Pinhole Camera (cont’d)?

If pinhole is too big then many directions are averaged, blurringthe image

If pinhole is too small then diffraction becomes a factor, alsoblurring the image

Generally, pinhole cameras are dark, because only a very smallset of rays from a particular scene point hits the image plane

Equivalently, pinhole cameras are slow, because only a very smallamount of light from a particular scene point hits the image planeper unit time

29 / 55

Snell’s Law

n1 sinα1 = n2 sinα2

30 / 55

The Reason for Lenses

31 / 55

Pinhole Model (Simplified) with Lens

x’

x

z

imageplane

objectlens

z’

32 / 55

Thin Lens Equation

33 / 55

Thin Lens Equation

Forsyth & Ponce (1st ed.) Figure 1.9

1z ′ − 1

z=

1f

34 / 55

Aside: Depth From Focus

Figure credit: H. Jin and P. Favaro, 2002

35 / 55

Focal Length

imageplane

lens

f

Another way of looking at the focal length of a lens. The incomingrays, parallel to the optical axis, converge to a single point adistance f behind the lens. This is where we want to place theimage plane.

36 / 55

Out–Of–Focus

imageplane

lens

f

The image plane is in the wrong place, either slightly closer thanthe required focal length, f , or slightly further than the requiredfocal length, f .

37 / 55

Spherical Aberration

Forsyth & Ponce (1st ed.) Figure 1.12a

38 / 55

Compound Lens Systems

A modern camera lens maycontain multiple compo-nents, including asphericalelements

39 / 55

VignettingVignetting in a two-lens system

Forsyth & Ponce (2nd ed.) Figure 1.12

The shaded part of the beam never reaches the second lens40 / 55

Vignetting

Image credit: Cambridge in Colour

41 / 55

Other (Possibly Significant) Lens Effects

Chromatic aberration— Index of refraction depends on wavelength, λ, of light— Light of different colours follows different paths— Therefore, not all colours can be in equal focus

42 / 55

Chromatic Aberration

Image credit: Trevor Darrell

43 / 55

Other (Possibly Significant) Lens Effects

Chromatic aberration— Index of refraction depends on wavelength, λ, of light— Light of different colours follows different paths— Therefore, not all colours can be in equal focus

Scattering at the lens surface— Some light is reflected at each lens surface

There are other geometric phenomena/distortions— pincushion distortion— barrel distortion— etc

44 / 55

Lens Distortion

Image credit: Fig. 2.13 in Szeliski

45 / 55

Human Eye

Image credit: https://www.nei.nih.gov/health/eyediagram

46 / 55

Human Eye

The eye has an iris (like a camera)

Focusing is done by changing shape of lens

When the eye is properly focused, light from an object outside theeye is imaged on the retina.

The retina contains light receptors called rods and cones

47 / 55

Human Eye

Rods— 75 to 150 million— Not involved in colour vision— Sensitive to low levels of illumination— Capable of responding to a single photon, but yield relativelypoor spatial detailCones— 6 to 7 million— Highly sensitive to colour— Active only at higher levels of illumination but yield higherresolution

48 / 55

Human Eye

Density of rods and cones

Image credit: Gonzalez and Woods (3rd ed.) Fig. 2.2

49 / 55

Human Eye: Illumination SensitivityA classic experiment to study the sensitivity of the human visionsystem to different illumination levels:

A subject looks at a uniformly illuminated field— typically a diffuser such as opaque glass that is illuminatedfrom behind by a light source whose brightness can be variedAn increment of illumination is added in the form of ashort-duration flashThe subject states whether or not there is a perceivable change

Image credit: Gonzalez and Woods (3rd ed.) Fig. 2.5

50 / 55

Human Eye: Illumination Sensitivity

Image credit: Gonzalez and Woods (3rd ed.) Fig. 2.5

The ratio ∆Ic/I, where ∆Ic is the increment of illumination that isenough to be perceivable 50% of the time, is known as the WeberratioA small value for ∆Ic/I means a small change in illumination isdiscernable - high illumination sensitivity.

51 / 55

Human Eye: Illumination SensitivityA typical plot of the Weber ratio as a function of brightness:

Image credit: Gonzalez and Woods (3rd ed.) Fig. 2.6

Shows that illumination sensitivity is poor at low levels ofillumination and improves significantly as the backgroundillumination increasesWhy two branches?

52 / 55

Human Eye: Simultaneous Contrast

Image credit: Gonzalez and Woods (3rd ed.) Fig. 2.8

Finally, it’s worth noting that human-perceived brightness is not asimple function of the intensityAll the center squares have the same intensity, but appear to theeye to become darker as the background becomes lighter

53 / 55

Summary

We discussed a “physics-based” approach to image formation.Basic abstraction is the pinhole camera.Lenses overcome limitations of the pinhole model while trying topreserve it as a useful abstractionProjection equations: perspective, weak perspective, orthographicThin lens equationSome “aberrations and distortions” persist— e.g. spherical aberration, vignettingThe human eye functions much like a camera

54 / 55

Reminders:Complete Assignment 1 by Tuesday, January 12www: http://www.cs.ubc.ca/~ftung/cpsc425/piazza: https://piazza.com/ubc.ca/winterterm22015/cpsc425/

55 / 55