CS4501: Introduction to Computer Vision Projective

CS4501: Introduction to Computer VisionProjective Geometry and Light

Various slides from previous courses by: D.A. Forsyth (Berkeley / UIUC), I. Kokkinos (Ecole Centrale / UCL). S. Lazebnik (UNC / UIUC), S. Seitz (MSR / Facebook), J. Hays (Brown / Georgia Tech), A. Berg (Stony Brook / UNC), D. Samaras (Stony Brook) . J. M. Frahm (UNC), V. Ordonez (UVA).

Announcing Office Hours

Masum(mb2vj at virginia.edu)

Office Hours: Tuesdays 3 to 4pm (ET)

Anshuman(as9rw at virginia.edu)

Office Hours: Wednesdays 2 to 3pm (ET)

Fridays 1 to 2pm (ET)

Vicente(vicente at virginia.edu)

Office Hours: Mondays 3 to 5pm (ET)

Last Class

What is a camera? Who invented cameras?Photography – Life of a Photograph - from Light to PixelsShutter Speed / Aperture Size / ISO sensitivity

• Recap from Last Class on Shutter Speed / Aperture / ISO sensitivity• Camera Parameters• Brief Introduction to Projective Geometry (Computer Graphics)• Light Models (Diffuse Light vs Specular Light)

Today’s Class

Life of a Photograph

Slide by Steve Seitz

How to Shoot Photos in Manual?

•Shutter speed•Aperture size•Focus / Auto-focus (Yes, you can shoot in

manual and also probably should focus in manual)

•ISO sensitivity

Fast Shutter Speed

http://www.photographymad.com/pages/view/shutter-speed-a-beginners-guide

Doesn’t allow much light Allows a lot of light

Slow Shutter Speed

Large Aperture SizeSmall Aperture Size

ISO sensitivity – Should be small ideally

https://www.exposureguide.com/iso-sensitivity/

Projection: world coordinatesàimage coordinates

Camera Center (0, 0, 0)

úúú

û

ù

êêê

ë

é=ZYX

P.

.. f Z

Y

úû

ùêë

é=VU

p

.V

U

If X = 2, Y = 3, Z = 5, and f = 2What are U and V?

Projection: world coordinatesàimage coordinates

Camera Center (0, 0, 0)

úúú

û

ù

êêê

ë

é=ZYX

P.

.. f Z

Y

úû

ùêë

é=VU

p

.V

U

ZfXU *-=

ZfYV *-=

52*2-=U

52*3-=V

Projection: world coordinatesàimage coordinates

Camera Center

(tx, ty, tz)

úúú

û

ù

êêê

ë

é=ZYX

P.

.. f Z Y

úû

ùêë

é=vu

p

.Optical Center (u0, v0)

v

u

Homogeneous coordinates vs Cartesian coordinatesConversion

Converting to homogeneous coordinates

homogeneous image coordinates

homogeneous scene coordinates

Converting from homogeneous coordinates

Invariant to scaling

Point in Cartesian is ray in Homogeneous

úû

ùêë

é=ú

û

ùêë

éÞ

úúú

û

ù

êêê

ë

é=

úúú

û

ù

êêê

ë

é

wywx

kwkykwkx

kwkykx

wyx

k

Homogeneous Coordinates

Cartesian Coordinates

Homogeneous coordinates vs Cartesian coordinates

Projection: world coordinatesàimage coordinates

Camera Center (0, 0, 0)

úúú

û

ù

êêê

ë

é=ZYX

P.

.. f Z

Y

úû

ùêë

é=VU

p

.V

U

ZfXU *-=

ZfYV *-=

52*2-=U

52*3-=V

[ ]X0IKx =úúúú

û

ù

êêêê

ë

é

úúú

û

ù

êêê

ë

é=

úúú

û

ù

êêê

ë

é

10100000000

1zyx

ff

vu

w

K

Slide Credit: Savarese

Projection matrix: from World to Image Coordinates

Intrinsic Assumptions• Unit aspect ratio• Optical center at (0,0)• No skew

Extrinsic Assumptions• No rotation• Camera at (0,0,0)

X

x

Remove assumption: known optical center

[ ]X0IKx =úúúú

û

ù

êêêê

ë

é

úúú

û

ù

êêê

ë

é=

úúú

û

ù

êêê

ë

é

101000000

10

0

zyx

vfuf

vu

w

Intrinsic Assumptions• Unit aspect ratio• No skew

Extrinsic Assumptions• No rotation• Camera at (0,0,0)

Remove assumption: square pixels

[ ]X0IKx =úúúú

û

ù

êêêê

ë

é

úúú

û

ù

êêê

ë

é=

úúú

û

ù

êêê

ë

é

101000000

10

0

zyx

vu

vu

w ba

Intrinsic Assumptions• No skew

Extrinsic Assumptions• No rotation• Camera at (0,0,0)

Remove assumption: non-skewed pixels

[ ]X0IKx =úúúú

û

ù

êêêê

ë

é

úúú

û

ù

êêê

ë

é=

úúú

û

ù

êêê

ë

é

10100000

10

0

zyx

vus

vu

w ba

Intrinsic Assumptions Extrinsic Assumptions• No rotation• Camera at (0,0,0)

Note: different books use different notation for parameters

Oriented and Translated Camera

Ow

iw

kw

jw

t

R

X

x

Allow camera translation

[ ]XtIKx =úúúú

û

ù

êêêê

ë

é

úúú

û

ù

êêê

ë

é

úúú

û

ù

êêê

ë

é=

úúú

û

ù

êêê

ë

é

1100010001

1000

0

10

0

zyx

ttt

vu

vu

w

z

y

x

ba

Intrinsic Assumptions Extrinsic Assumptions• No rotation

3D Rotation of PointsRotation around the coordinate axes, counter-clockwise:

úúú

û

ù

êêê

ë

é -=

úúú

û

ù

êêê

ë

é

-=

úúú

û

ù

êêê

ë

é-=

1000cossin0sincos

)(

cos0sin010

sin0cos)(

cossin0sincos0001

)(

gggg

g

bb

bbb

aaaaa

z

y

x

R

R

R

p

p’ g

y

z

Slide Credit: Savarese

Allow camera rotation

[ ]XtRKx =

úúúú

û

ù

êêêê

ë

é

úúú

û

ù

êêê

ë

é

úúú

û

ù

êêê

ë

é=

úúú

û

ù

êêê

ë

é

1100

01 333231

232221

131211

0

0

zyx

trrrtrrrtrrr

vus

vu

w

z

y

x

ba

Slide Credit: Savarese

Projection matrix (Word Coordinates to Image Coordinates)

[ ]XtRKx = x: Image Coordinates: (u,v,1)K: Intrinsic Matrix (3x3)R: Rotation (3x3) t: Translation (3x1)X: World Coordinates: (X,Y,Z,1)

Ow

iw

kw

jwR,t

X

x

Intrinsic Camera Properties: K

Extrinsic Camera Properties: [R t]

Degrees of freedom

[ ]XtRKx =

úúúú

û

ù

êêêê

ë

é

úúú

û

ù

êêê

ë

é

úúú

û

ù

êêê

ë

é=

úúú

û

ù

êêê

ë

é

1100

01 333231

232221

131211

0

0

zyx

trrrtrrrtrrr

vus

vu

w

z

y

x

ba

5 6

Assignment – World Coordinates to Image Coordinates

What the camera can “see”assuming this is a wireframe

object

(0,0,0)

(0,0,3)

(1,1,4)

Things to Remember for Quiz

• Pinhole camera model• Focal length in the pinhole camera model• Shutter Time / Aperture / ISO• Homogeneous Coordinates• Extrinsic Camera Properties and Intrinsic Camera Properties• Describe mathematically (and intuitively) the conversion process

from World Coordinates to Image Coordinates

Light

• What determines the color of a pixel?

Figure from Szeliski

BRDF (Bidirectional reflectance distribution function)

Slide by Aaron Bobick

BRDF (Bidirectional reflectance distribution function)

Slide by Aaron Bobick

Reflection

Slide by Aaron Bobick

Phong Reflection Model

Slide by Aaron Bobick

Phong Reflection Model

https://en.wikipedia.org/wiki/Phong_reflection_model

Phong Reflection Model - Recap

https://en.wikipedia.org/wiki/Phong_reflection_model

Phong Reflection Model - Recap

https://en.wikipedia.org/wiki/Phong_reflection_model

Phong Reflection Model - Recap

https://en.wikipedia.org/wiki/Phong_reflection_model

Phong Reflection Model - Recap

https://en.wikipedia.org/wiki/Phong_reflection_model

Phong’s Shading / Illumination Model

• Originally from Vietnam / PhD from Utah, Professor at Utah, and later Stanford.

• Died at age 32 from leukemia

• Phong’s professor Ivan Sutherland went on to win the Turing Award (Nobel Prize in CS) for lifelong contributions to Computer Graphics

Same ideas used in Computer Graphics

• Ray Tracing• Radiosity• Photon Mapping

Reflection

Slide by Aaron Bobick

Diffuse Reflection – Lambertian Surface / BRDF

Slide by Aaron Bobick

• Light intensity does not depend on the outgoing direction. Only incoming.

• It is independent of where the viewer stands.

• Smooth surface, not glossy. Can think of any examples?

Slide by Aaron Bobick

The other extreme – Only Specular Reflection

Slide by Aaron Bobick

Problem in Computer Vision: Intrinsic Image Decomposition

Given this Extract this

Images by Marc Serra

Given this Extract this

Images by Aaron Bobick

Problem in Computer Vision: Shape from Shading

A photon’s life choices

• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection

λ

light source

?

Slide by James Hays

A photon’s life choices

• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection

λ

light source

Slide by James Hays

A photon’s life choices

• Absorption• Diffuse Reflection• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection

λ

light source

Slide by James Hays

A photon’s life choices

• Absorption• Diffusion• Specular Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection

λ

light source

Slide by James Hays

A photon’s life choices

• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection

λ

light source

Slide by James Hays

A photon’s life choices

• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection

λ

light source

Slide by James Hays

A photon’s life choices

• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection

λ1

light source

λ2

Slide by James Hays

A photon’s life choices

• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection

λ

light source

Slide by James Hays

A photon’s life choices

• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection

t=1

light source

t=n

Slide by James Hays

A photon’s life choices

• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection

λ

light source

(Specular Interreflection)

Slide by James Hays

The Eye

• The human eye is a camera! (not quite)• Iris - colored annulus with radial muscles• Pupil - the hole (aperture) whose size is controlled by the iris• What’s the “film”?

– photoreceptor cells (rods and cones) in the retina

Slide by Steve Seitz

Next Class: Image Processing and Image Filters

Questions?

57