Embed Size (px)
Citation preview
CPSC 425: Computer Vision
Instructor: Jim [email protected]
Department of Computer ScienceUniversity of British Columbia
Lecture Notes 2016/2017 Term 2
1 / 55
Menu January 17, 2017
Topics:Image FormationCameras and Lenses
Reading:
Today: Forsyth & Ponce (2nd ed.) 1.1.1–1.1.3Next: Forsyth & Ponce (2nd ed.) 4.1, 4.5
Reminders:Complete Assignment 1 by Tuesday, January 12www: http://www.cs.ubc.ca/~ftung/cpsc425/
piazza: https://piazza.com/ubc.ca/winterterm22015/cpsc425/
2 / 55
Today’s “Fun” Example: Eye in Sink Illusion
“Tried taking a picture of a sink draining, wound up with a picture of aneye instead”
Photo credit: reddit user Liammm
3 / 55
Today’s “Fun” Example: Eye in Sink Illusion
“Tried taking a picture of a sink draining, wound up with a picture of aneye instead”
Photo credit: reddit user Liammm
4 / 55
Lecture 1: Re-cap
What we see depends on:— object shape— surface material— illumination— viewpoint
Visual perception also is influenced by:— familiarity— context— expectation
5 / 55
Lecture 1: Re-cap
Computer vision technologies have moved from research labs intocommercial products and services. Examples cited include:— broadcast television sports— electronic games (Microsoft Kinect)— real-time language translation— image search— smart infrastructure
6 / 55
Framework for Today’s Topic
Problem: Obtain information about the 3D world
Key Idea(s): Treat a “camera” as a scientific instrument for obtainingmeasurements of the 3D world
Alternatives:— Treat “images” as 2D entities only— Treat “images” as just another kind of “big data”
Theory: Optics (geometry and radiometry)
Practical Detail(s): Cameras and lenses
“Gotchas:”— interpretation of 3D world can be ambiguous— role of human perception
7 / 55
Overview: Image Formation, Cameras and Lenses
Goal: to understand how images are formed
Camera obscura dates from 16th century (and earlier)
Basic abstraction is the pinhole camera
Cameras and lenses maintain the abstraction
The human eye functions very much like a camera
8 / 55
Camera Obscura (Latin for ‘dark chamber’)
Reinerus Gemma-Frisius observed an eclipse of the sun at Louvain onJanuary 24, 1544. He used this illustration in his book, “De RadioAstronomica et Geometrica,” 1545. It is thought to be the firstpublished illustration of a camera obscura.
Credit: John H., Hammond, “The Camera Obscura, A Chronicle”
9 / 55
First Photograph on RecordLa table servie
Credit: Nicéphore Niepce, 1822
10 / 55
Pinhole CameraA pinhole camera is a box with a small hole in it
Forsyth & Ponce (2nd ed.) Figure 1.2
11 / 55
Image Formation
Forsyth & Ponce (2nd ed.) Figure 1.1
Credit: US Navy, Basic Optics and Optical Instruments. Dover, 1969
12 / 55
Pinhole Camera (Simplified)
x’
x
zf’
imageplane
pinhole object
13 / 55
Pinhole Camera (Simplified) (cont’d)
x’
x
zf’
imageplane
pinhole object
f’
x’
imageplane
14 / 55
Perspective EffectsFar objects appear smaller than close ones
Forsyth & Ponce (1st ed.) Figure 1.3a
Size is inversely proportional to distance15 / 55
Perspective Effects (cont’d)Parallel lines meet
Forsyth & Ponce (1st ed.) Figure 1.3b
16 / 55
Vanishing Points
17 / 55
Vanishing Points
Slide credit: David Jacobs
18 / 55
Vanishing Points (cont’d)
Each set of parallel lines meets at a different point— the vanishing point for this direction
Sets of parallel lines on the same plane lead to collinear vanishingpoints— the line is called the horizon for that plane
Good ways to spot faked images— scale and perspective don’t work— vanishing points behave badly
19 / 55
20 / 55
Perspective Projection
21 / 55
Perspective Projection
Forsyth & Ponce (1st ed.) Figure 1.4
3D object point, P[x , y , z], projects to 2D image point P ′[x ′, y ′] where
x ′ = f ′ xz
y ′ = f ′ yz
22 / 55
Weak Perspective
23 / 55
Weak Perspective
Forsyth & Ponce Figure 1.5 (1st ed.)
3D object point, P[x , y , z] in Π0, projects to 2D image point P ′[x ′, y ′]where
x ′ = m xy ′ = m y
and m =f ′
z024 / 55
Orthographic Projection
Forsyth & Ponce (1st ed.) Figure 1.6
3D object point, P[x , y , z], projects to 2D image point P ′[x ′, y ′] where
x ′ = xy ′ = y
25 / 55
Summary of Projection Equations3D world point, P[x , y , z], projects to 2D image point P ′[x ′, y ′] where
Perspectivex ′ = f ′ x
z
y ′ = f ′ yz
Weak Perspectivex ′ = m x
y ′ = m ym =
f ′
z0
Orthographicx ′ = x
y ′ = y
26 / 55
Projection Models: Pros and Cons
Weak perspective (including orthographic) has simplermathematics— Accurate when object is small and/or distant— Useful for recognition
Perspective more accurate for real scenes— Useful in structure from motion
When maximum accuracy required, it is necessary to modeladditional details of the particular camera— Use perspective projection with other calibration
parameters (e.g., radial lens distortion)
27 / 55
Why Not a Pinhole Camera?
Credit: E. Hecht. “Optics,” Addison-Wesley, 1987
28 / 55
Why Not a Pinhole Camera (cont’d)?
If pinhole is too big then many directions are averaged, blurringthe image
If pinhole is too small then diffraction becomes a factor, alsoblurring the image
Generally, pinhole cameras are dark, because only a very smallset of rays from a particular scene point hits the image plane
Equivalently, pinhole cameras are slow, because only a very smallamount of light from a particular scene point hits the image planeper unit time
29 / 55
Snell’s Law
Forsyth & Ponce (2nd ed.) Figure 1.7
n1 sin α1 = n2 sin α2
30 / 55
The Reason for Lenses
31 / 55
Pinhole Model (Simplified) with Lens
x’
x
z
imageplane
objectlens
z’
32 / 55
Thin Lens Equation
33 / 55
Thin Lens Equation
Forsyth & Ponce (1st ed.) Figure 1.9
1z ′ −
1z
=1f
34 / 55
Aside: Depth From Focus
Figure credit: H. Jin and P. Favaro, 2002
35 / 55
Focal Length
imageplane
lens
f
Another way of looking at the focal length of a lens. The incomingrays, parallel to the optical axis, converge to a single point adistance f behind the lens. This is where we want to place theimage plane.
36 / 55
Out–Of–Focus
imageplane
lens
f
The image plane is in the wrong place, either slightly closer thanthe required focal length, f , or slightly further than the requiredfocal length, f .
37 / 55
Spherical Aberration
Forsyth & Ponce (1st ed.) Figure 1.12a
38 / 55
Compound Lens Systems
A modern camera lens maycontain multiple compo-nents, including asphericalelements
39 / 55
VignettingVignetting in a two-lens system
Forsyth & Ponce (2nd ed.) Figure 1.12
The shaded part of the beam never reaches the second lens40 / 55
Vignetting
Image credit: Cambridge in Colour
41 / 55
Other (Possibly Significant) Lens Effects
Chromatic aberration— Index of refraction depends on wavelength, λ, of light— Light of different colours follows different paths— Therefore, not all colours can be in equal focus
42 / 55
Chromatic Aberration
Image credit: Trevor Darrell
43 / 55
Other (Possibly Significant) Lens Effects
Chromatic aberration— Index of refraction depends on wavelength, λ, of light— Light of different colours follows different paths— Therefore, not all colours can be in equal focus
Scattering at the lens surface— Some light is reflected at each lens surface
There are other geometric phenomena/distortions— pincushion distortion— barrel distortion— etc
44 / 55
Lens Distortion
Image credit: Fig. 2.13 in Szeliski
45 / 55
Human Eye
Image credit: https://www.nei.nih.gov/health/eyediagram
46 / 55
Human Eye
The eye has an iris (like a camera)
Focusing is done by changing shape of lens
When the eye is properly focused, light from an object outside theeye is imaged on the retina.
The retina contains light receptors called rods and cones
47 / 55
Human Eye
Rods— 75 to 150 million— Not involved in colour vision— Sensitive to low levels of illumination— Capable of responding to a single photon, but yield relativelypoor spatial detail
Cones— 6 to 7 million— Highly sensitive to colour— Active only at higher levels of illumination but yield higherresolution
48 / 55
Human Eye
Density of rods and cones
Image credit: Gonzalez and Woods (3rd ed.) Fig. 2.2
49 / 55
Human Eye: Illumination SensitivityA classic experiment to study the sensitivity of the human visionsystem to different illumination levels:
A subject looks at a uniformly illuminated field— typically a diffuser such as opaque glass that is illuminatedfrom behind by a light source whose brightness can be varied
An increment of illumination is added in the form of ashort-duration flash
The subject states whether or not there is a perceivable change
Image credit: Gonzalez and Woods (3rd ed.) Fig. 2.5
50 / 55
Human Eye: Illumination Sensitivity
Image credit: Gonzalez and Woods (3rd ed.) Fig. 2.5
The ratio ∆Ic/I, where ∆Ic is the increment of illumination that isenough to be perceivable 50% of the time, is known as the Weberratio
A small value for ∆Ic/I means a small change in illumination isdiscernable - high illumination sensitivity.
51 / 55
Human Eye: Illumination Sensitivity
A typical plot of the Weber ratio as a function of brightness:
Image credit: Gonzalez and Woods (3rd ed.) Fig. 2.6
Shows that illumination sensitivity is poor at low levels ofillumination and improves significantly as the backgroundillumination increases
Why two branches?
52 / 55
Human Eye: Simultaneous Contrast
Image credit: Gonzalez and Woods (3rd ed.) Fig. 2.8
Finally, it’s worth noting that human-perceived brightness is not asimple function of the intensity
All the center squares have the same intensity, but appear to theeye to become darker as the background becomes lighter
53 / 55
Summary
We discussed a “physics-based” approach to image formation.Basic abstraction is the pinhole camera.
Lenses overcome limitations of the pinhole model while trying topreserve it as a useful abstraction
Projection equations: perspective, weak perspective, orthographic
Thin lens equation
Some “aberrations and distortions” persist— e.g. spherical aberration, vignetting
The human eye functions much like a camera
54 / 55
Reminders:Complete Assignment 1 by Tuesday, January 12www: http://www.cs.ubc.ca/~ftung/cpsc425/
piazza: https://piazza.com/ubc.ca/winterterm22015/cpsc425/
55 / 55
