Photometric Image Formation

CSE 559: Computer VisionGuest Lecturer: Austin Abrams

Images/Demo from Steve Seitz, Wikipedia

How are images made?

• One half: geometric vision– “how the pixel projected onto the image”

• Today: photometric vision (aka radiometric)– “how the pixel got its color”

Vision and Graphics

Properties of a sceneImage

Computer Graphics

Vision

Image Formation Approach

• Come up with a model for how the scene was created

• Given images, find the most likely properties that fit that model

Diffuse Surfaces

Brightness of a pixel depends on:• object color• lighting direction• surface normal

But NOT view direction!

Lambertian Cosine Law

• The intensity of an observed diffuse object is proportional to the cosine of the angle between the normal and lighting direction

= ρ L N

I = ρ cos θ

= ρ |L||N| cos θL Nθ

=

L N = L N

= x

I = ρ L N

Recovering Albedo and Normals

• Can you decompose a single image into its albedo and normal images?

=

x

x

x

Photometric Stereo

• Given multiple images taken with varying illumination, recover albedo and normals.– take pictures in dark room with varying

illumination.– estimate lighting directions L.– recover albedo and normals.

Side note 1: How to get the lighting direction?

• Put a shiny sphere in the scene• Sphere’s geometry (normals) are known• Find specular highlight

Side-note 2: Why “Stereo”?

Surface normals provide constraints on depth differences

Photometric Stereo

• If L is known, and albedo is grayscale this is a linear problem.

I = ρ(L N) = ρ (Lx Nx + Ly Ny + Lz Nz ) = Lx Nxρ + Ly Nyρ + Lz Nzρ = Lx a + Ly b + Lz c

Lx1 Ly1 Lz1

Lx2 Ly2 Lz2

Lx3 Ly3 Lz3

…

Lxn Lyn Lzn

I1I2I3…In

abc

=

I = ρ(L N) = Lx a + Ly b + Lz c

Then:ρ = sqrt(a2 + b2 + c2)N = (a,b,c) / ρ

For each pixel:

Demo

When does this model fail?

I ≠ ρ (L N)

Attached shadows

I = ρ max(L N, 0)

L N > 0

L N = 0

L N < 0

Cast Shadows, Ambient Light

I = ρ (S L N + a) S = 0 or 1

Radiometric Camera Calibration

• Pixel intensities are usually not proportional to the energy that hit the CCD

RAW image Published image

Radiometric Camera Calibration

f

RAW

Published

Radiometric Camera Calibration

Observed = f(RAW)

(Grossberg and Nayar)

f -1 (Observed) = RAW

Radiometric Camera Calibration

• How do you model f -1?

f -1(x) = xγ

f -1(x) = c0 + c1x + c2x2 + c3x3 + …

f -1(x) = f0(x) + f1(x) c1 + f2(x)c2 + …

mean camera curve basis camera curves

Radiometric Camera Calibration

I = f (ρ (S L N + a))

Adding exposure:

I = f (e ρ (S L N + a))

Heliometric StereoGiven lots of images from a stable webcam,

use lighting from the sun to recover:

I = f (e ρ (S L N + a))

Heliometric Stereo

Heliometric Stereo

Heliometric Stereo