Shape and Shading

Koenderink & Van DornChapter 72 of The Visual Neurosciences

Introduction

• Observers are interested in – Geometry of objects– Material identification– Light field: the primary and secondary

sources of radiation and how radiation pervades space

• What we have: an image• What we are interested in: the scene

Introduction

• Current topic deals with only the static, achromatic, monocular observer

• Ignoring – Observer motion– Object motion– Color– Binocular – Material properties

Introduction

• We want to know how object shape can be determined from monochrome pictures.

The light field

• Light model: rays (particle model)• Radiators

– Primary: luminous– Secondary

• Reflecting• Scattering

The light field

• Volume (ray) density of radiation: the length of all rays crossing a volume, divided by that volume.

• Or the density of photons crossing that volume during a given time interval.

The light field

• Net flux vector:• Number of photons crossing a unit area

per unit time.• Calculated as proportional to cos(),

where is the angle between a surface patch normal and the beam direction.

• Such a definition is appropriate only for small patches, of course.

The light field

• The light passing thru any such circular patch defines a tube.

• Such tubes may be curved.

The light field

• One might also consider the rays of light passing thru a point.

• There are zero such rays.• However, we can define something

called radiance.• <see handbook>

The light field

• Radiant flux: the total energy emitted by a source, or thru some area.

• Irradiance: the radiant flux per unit area (radiant flux density). W/m2

• Radiance: the flux density per solid unit angle. (W/m2)/sr, where sr = A/r2. This is effectively the energy passing thru the solid angle of one steradian.

The light field

• In these terms, the light field is simply the radiance distribution.

• I.e., for any point, we can consider the rays of light that impinges upon it from all directions.

The light field

• Rays– Origins: primary radiators– End: black surfaces

• Air, smoke, fog, etc. may also absorb ray energy but such effects are ignored in this treatment.

Objects in the light field

• Photons are scattered at object surfaces.• Probability of any photon reflecting is a function

of the incident angle and the viewing (or exit) direction.

• The scattered radiance from a point on a surface, to a particular viewing direction, divided by the irradiance from a given direction, of the surface is called the bidirectional reflectance distribution function (BRDF).

• BRDF depends on 4 angular parameters.

Objects in the light field

• BRDFs tend to be complex functions of the 4 angles.

• Many psychophysical studies and computer models ignore this fact.

• A perfectly white surface has a BRDF of 1/ (integral)

Objects in the light field

• Illumination of a matt sphere by directional light– Shading (attitude effect)– Cast shadow– Body shadow

• Disc appearance from the direction of illumination: disc with dark edge

Objects in the light field

• Natural illumination environments– Combination of directional light (solar) and

diffuse. – Approximately uniform hemispherical

diffuse beam.• Very little body shadow• Ground side will be darker (vignetting)

– Ganzfield• Whiteout

Photometric effects

• Levels of scale– Whole scene– Object– Texture

• Incomplete description• Some averaging

– Subtexture• Material properties• Incorporated into BRDF

Photometric effects• Context

– Whole scene• Object

– Texture» Subtexture

• Examples– Shape from shading affected by contours– Shape from shading with illumination direction information

• Scene cues– Specularities– Shadow directions– Degree of diffusion in shadows

Photometric effects

• Texture and illumination cues• Directional illumination

– Texture most apparent near body shadow• Diffuse illumination

– Texture due to cracks and pits– Darkness here not due to attitudinal

mechanisms but vignetting mechanisms.

Photometric effects

• Specularities• Light may be scattered by underlying

substrates for some angles and reflected for others.

• Disappear in diffuse light fields• Directional illumination combined with surface

texture can produce arrays of specularities– Example: shining ripples on a lake– Orange vs. tomato

Photometric effects

• Inter-reflections • Light bounces from one object onto

others.• Ties scene together and indicates

relations between objects.

Photometric effects

• Deviations from Lambertian– Backscatter– Asperity scattering– Translucence

Photometric effects

• Backscatter– Light projected onto textures tends to be reflected

back more to the source.– Why? Shadows are not visible to the source.– Viewed from other angles, shadows decrease

average luminance.– Attitudinal shading is decreased, making object

look flatter.

Photometric effects• Asperity scattering

– Reflections from hair tips– Common on plants and animals– Furs: hairs are ~parallel with surface provide distinct patterns of

specularities.– Asperity occurs when hairs stick straight up.– More light is reflected to the viewer when there are more tips per

unit view angle (as near contours).– Again: Attitudinal shading is decreased, making object look flatter.– Can create light edges.– Together with Lambertian shading, explains

• The existence of odd order filters in V1• The use of line drawings in art.

Photometric effects• Translucence

– Most materials are translucent at the micro scale.– Thus, BRDF concepts do not exactly apply.– Light is not reflected from a point but enters at one point and

exits at a slightly different point.– However, if we consider BRDF as reflection from small

discs, then micro translucence and microstructures within a surface can explain a BRDF.

The structure of pictures• The radiance distribution from any point determines

the set of possible picture that can be taken from that point.

• However, cameras have their own properties.– Dynamic range– Resolution– MTF– Iimited field of view

• Computer screens, film and printers also have their own limitations on dynamic range, resolution etc.

• Not mentioned: luminance response functions

Shape from shading• Definition: invariant under a group of

transformations.• Equivalently: what’s in the scene, rather than

what’s in the image.• In this analysis, viewpoint and other

parameters are held constant.• Illumination varies.

Shape from shading• Concave / convex ambiguities.• Illumination direction / shape confound• Other cues can help resolve these problems.• Only bumps share inter-reflections with the parent

plane.• Only dents have internal inter-reflections.• Many such details may be missing in computer

graphic implementations, affecting the associated psychophysics.

Shape from shading• Cues in collimated light fields

– Body shadow– Textural quality of the body shadow edge– Structure of specularities– Structure of contour edge (serrated or not)

• Orange vs tomato ( same macro shape )

Shape from shading• Cues in diffuse light fields

– Overall contour shape– Textural effects due to vignetting– Shading

• Due to vignetting not attitudinal

• Lambertian is not realistic– Vignetting is not taken into account.– Inter-reflections are not taken into account.

Shape from shading• Shape from Lambertian shading

– Illumination is assumed directional and the same at all points in the scene

– Light returned determines attitude – Surface normals can be considered as unit vectors on the unit sphere

(Gauss maps)

• Gauss map examples– Plane: point on sphere, degenerate– Cylinders and cones: curves on sphere, degenerate– All others: 2D patches on sphere, 1 to 1 locally

• Isophotes– Lines of equal luminance– Correspond to circles on the Gauss sphere

Shape from shading• The nature of non 1 to 1 maps

– Sphere is multiply covered– Think of a cloth with folds– Folds give rise to critical points (min, max, saddle) in the

irradiance field– Folds of the Gauss map correspond to inflections of the

surface, parabolic curves – Such parabolic curves bound the convex, concave and

saddle regions of the surface– The pattern of such parabolic curves can act as a

description of the shape.

Psychophysical results• Little definitive• Extreme stimulus reduction

– Studying images with only one type of cue (say shading)– Doesn’t provoke strong impressions of shape– Examples

• Silhouettes• Line drawings• Indication of shadows and lit areas

– The more cues available, the more subjects agree – Is the real problem interaction?

• Lack of realism– Published work with computer graphics using unknown or un-

described algorithms

Open problems• “Majority of literature is irrelevant due to incomplete description of

the stimuli, extreme stimuli reduction, or invalid paradigms.”• Stimuli must be complex / natural• Control experiments by using complex / natural images and by

varying only a single cue.• Some stimuli are not generic

– Ellipsoids have isophotes and shadow boundaries are planar curves.– No other shapes do.– Degenerate Gauss maps

• Veridicality– Observer’s response is due to two factors

• Idiosyncratic differences• Cues

– These are often lumped together.

Open problems• How to draw theories from computer vision models

for use in human research?– This is difficult because a computer might return a surface

mesh.– What is the human equivalent?

• What about shading in art?– It differs from actual shading yet we understand it.

• Shape impressions of observers need to be recorded rather than simple yes / no forced choices.