1

Algorithms compared

• Bicubic Interpolation• Mitra's Directional Filter• Fuzzy Logic Filter•Vector Quantization• VISTA

2

Bicubic spline Altamira VISTA

3

Bicubic spline Altamira VISTA

4

User preference test results

“The observer data indicates that six of the observers rankedFreeman’s algorithm as the most preferred of the five testedalgorithms. However the other two observers rank Freeman’s algorithmas the least preferred of all the algorithms….

Freeman’s algorithm produces prints which are by far the sharpestout of the five algorithms. However, this sharpness comes at a priceof artifacts (spurious detail that is not present in the originalscene). Apparently the two observers who did not prefer Freeman’salgorithm had strong objections to the artifacts. The other observersapparently placed high priority on the high level of sharpness in theimages created by Freeman’s algorithm.”

5

6

7

Training images

8

Training image

9

Processed image

10

Inference in Markov Random Fields

Gibbs sampling, simulated annealingIterated conditional modes (ICM)Belief propagation

Application examples:super-resolutionmotion analysisshading/reflectance separation

Graph cutsVariational methods

11

Motion applicationimage patches

image

scene

scene patches

12

What behavior should we see in a motion algorithm?

• Aperture problem

• Resolution through propagation of information

• Figure/ground discrimination

13

The aperture problem

14

The aperture problem

15

Motion analysis: related work

• Markov network– Luettgen, Karl, Willsky and collaborators.

• Neural network or learning-based– Nowlan & T. J. Senjowski; Sereno.

• Optical flow analysis– Weiss & Adelson; Darrell & Pentland; Ju,

Black & Jepson; Simoncelli; Grzywacz & Yuille; Hildreth; Horn & Schunk; etc.

16

Motion estimation results (maxima of scene probability distributions displayed)

Initial guesses only show motion at edges.

Iterations 0 and 1

Inference:

Image data

17

Motion estimation results

Figure/ground still unresolved here.

(maxima of scene probability distributions displayed)

Iterations 2 and 3

18

Motion estimation results

Final result compares well with vector quantized true (uniform) velocities.

(maxima of scene probability distributions displayed)

Iterations 4 and 5

19

Inference in Markov Random Fields

Gibbs sampling, simulated annealingIterated conditional modes (ICM)Belief propagation

Application examples:super-resolutionmotion analysisshading/reflectance separation

Graph cutsVariational methods

Forming an Image

Surface (Height Map)

Illuminate the surface to get:

The shading image is the interaction of the shapeof the surface and the illumination

Shading Image

Painting the Surface

Scene

Add a reflectance pattern to the surface. Points inside the squares should reflect less light

Image

Problem

What if we want to do shape from shading?

Need some mechanism of informing theshape from shading algorithm about thereflectance pattern.

Image Height Map

Represent Scene As

These types of images are known as intrinsic images (Barrow and Tenenbaum)

=

Shading ImageImage Reflectance Image

Not Just for Shape from Shading

• Ability to reason about shading and reflectance independently is necessary for most image understanding tasks– Material recognition– Image segmentation

• Intrinsic images are a convenient representation of the scene– Less complex than fully reconstructing the scene

Goal

Image Shading Image Reflectance Image