Dr Lindsay MacDonald, 3DIMPact Research Group, Faculty of Engineering, UCL

Image Sets under Directional Lighting UCL Centre for Digital HumanitiesSeminar – 11th March 2015

Dr Lindsay MacDonald, 3DIMPact Research Group,Department of Civil, Environmental and Geomatic Engineering, UCL 1

Image Sets under Directional LightingA Richer Representation of Cultural Heritage Objects

Lindsay MacDonald

Department of Civil, Environmental and Geomatic EngineeringUniversity College London

Digital Humanities Seminar, March 2015

Engelbach, R. (1934)A foundation scene of the second dynasty.

The Journal of Egyptian Archaeology,Vol. 20, pp.183‐184.



The power of raking light to reveal surface detail…

http://www.factumfoundation.org/pag/208/Making‐the‐Facsimile

The UCL Dome

1 metre hemisphere

64 flash lights

Nikon D200 camera

In use since 2006



Layout of lamps

• 64 lights in total

• 5 horizontal tiers

• Aim to distribute lights approximately uniformly over the hemisphere

A

ED

C

B

16

15

14

13

1211

10

9

8

7

6

54

3

2

1

Plan

Elevation

Nikon D200Fixed mounting



Flash board

Flash board

Top view (outside) Bottom view (inside)



Hinged superstructure

Object placedon baseboard



Imaging cultural artefacts

Petrie Museum, UCL Egyptian funerary cone c.1200 BC

Refers to a priest Nefer‐Iahand the moon‐god Lah, and a priestess Hemet‐Netjer and the god Amun.



Column 2To the ka‐spirit of the high priest of Iah, Nefer‐

Column 1Iah true of voice, revered, at peace.

Column 3Lady of the house, chantress of Amun, singer of Mut,

Column 4Hemet‐netjer, true of voice, at peace, beloved.

Translated by Prof Stephen Quirke, UCL Institute of Archaeology

80°



60°

40°



20°

5°



















Increasingangle ofelevation

Set of 64 images

100x100 pixel detail

Image sets from the dome

1. Visualisation by interactive movement of a virtual light source

2. 3D reconstruction of the object surface

3. Modelling of specular highlights from the surface

Each image is illuminated from a different direction.

All images are in pixel register.

A richer representation than normal photography!

Three ways of using dome image sets



Part 1 – Interactive visualisation

Imagine moving a candle around to different positions over a surface

Appearance of surface relief changes as light is moved: the illusion of 3D

Intensity distribution at one pixel

• Vector of 64 values

• Low values similar to cosine (Lambertian)

• Few high values near specular direction



Azimuthal equidistant projection

• Projection of hemisphere onto plane

• Useful properties:

All points on map are at proportionately correct distances from centre

All points on map are at correct azimuth (direction) from the centre point

Polar plot of distribution

• Plot intensities of pixel for 64 lamps.

• Position in plane corresponds to direction vector of lamp.

• Specular values stand out above others.



Polynomial texture mapping

• A form of the bidirectional texture function (BTF) model, simplified by holding exitant direction constant with reflected angle always toward fixed camera:

• Assuming a Lambertian surface, the reconstruction function is separable, with a constant colour per pixel modulated by an angle‐dependent luminance factor L:

Malzbender, T., Gelb, D. and Wolters, H. (2001)‘Polynomial Texture Maps’, Proc. ACM SIGGRAPH, 28, 519‐528.

, , Θ ,Φ , ,

Θ ,Φ , , , , ,

u, v ‐ texture coordinatesa0 - a5 ‐ fitted coefficients stored in texture maplu, lv ‐ projection of light direction into texture plane

5v4u3vu22

v12

u0vu alalallalala),lL(u,v;l

lu

lv

Light direction parametrisation



1

1

0

5

1

0

111111

111111

000000

1

1

1

NvNuNvNuN2vN

2uN

vuvu2v

2u

vuvu2v

2u

L

L

L

a

a

a

llllll

llllll

llllll

For each pixel, given N light sources and observed intensities L0 … LN-1 (reflected from the object), compute best fit for the six parameters (a0‐a5) using Singular Value Decomposition (SVD).

Fitting PTM to image data

Inverse matrix calculated only once

Polar plot of PTM distribution

• PTM values for 64 lamps calculated by biquadratic function.

• Models cosine distribution well

• Cannot model the specular peak



Demonstration of PTM viewer

Softwaredeveloped

by Tom Malzbender at HP Labs

in 2001

Circular area represents hemisphere for the movement of virtual light source

Hemispherical harmonics1/2

6/ cos cos cos3/2 2cos 1

6/ sin cos cos30/ cos 2 cos cos

30/ cos 1 2cos cos cos

5/2 1 6 cos cos

30/ sin 1 2cos cos cos

30/ sin 2 cos cos

140/ cos 3 cos cos

210/ cos 2 1 2cos cos cos

84/ cos cos cos 1 5 cos cos

7/2 12cos 1 30cos 20cos

84/ sin cos cos 1 5 cos cos

210/ sin 2 1 2cos cos cos

140/ sin 3 cos cos

atan2 ,

acos 1

Change of variables:

Azimuth

Co‐latitude

First order

Second order

Third order

Co‐latitude



Hemispherical harmonics

First 16 modes plotted on polar plane

Green = positiveBlue = negative

Modelling intensity distributions

PTMHSH

Order 1

HSHOrder 2

HSHOrder 3



Angular resolution

Spacing of lamps in dome sets limit on fineness of detail that can be resolved in an angular sense.

Range of angles for all neighbouring lamps is 12–28 with median 20.

Reflection transform imaging

Generalisation of PTM with enhancements:

• Basis functions (spherical harmonics)

• High dynamic range (HDR) imaging

• Virtual dome (movable light source)

• Highlight‐based calibration (spherical targets)

• More flexible file format *.rti

Specular highlight on blacksphere enables position oflight source to be determined



RTI in action

Photographer Elizabeth Minor and assistant Kierstin Sakai during RTI capture,with raking light angle (PAHMA) – Hearst Museum, Berkeley, CA

Applications of RTI

Coins Rock art Cuneiform tablets

Fossils Byzantine glasstesseræ

Marble friezes



Advantages of PTM/RTI in cultural heritage

• Non‐contact acquisition

• Convincing illusion of 3D shape

• Interactive visualisation

• Better discernment of surface detail than physical examination

• No data loss due to shadows and specular highlights

• Simple and achievable image processing pipeline

• Higher resolution on object surface than with 3D scanners

Marble capitalMuseum of San Matteo

Factors affecting quality of PTM/RTI

• Spatial resolution of images

• SNR and dynamic range

• Number of light sources

• Fit of basis functions to actual surface reflectance distribution

• Spectral resolution, i.e. ability to reconstruct reflectance spectrum

The Antikythera MechanismFreeth et al (2006) Nature



Part 2 – Reconstructing height of surface

Very few surfaces are perfectly planar.

We are interested in 2½D surfaces, i.e. flat with relief.

Try to make a digital terrain map (DTM)

Use principle of ‘shape from shading’, aka photometric stereo.

Surface normals

• A normal N to a surface S at point P is a a vector perpendicular to the tangentplane touching surface at P.

• For a set of points satisfying S x,y,z 0, a normal at x,y,z on surface is gradient formed by partial first derivatives wrt each axis:

• Where surface is defined by z S x,y , the normal is:

, , , ,T

N

S

P

, , 1T

, , 1 T



Photometric stereo

For a Lambertian surface, from which incident light is scattered equally in all directions, the luminance of reflected light is given by vector dot product:

∙ | | cos

∙

Three equations are needed to solve system, by illuminating the surface in successive images from three lighting directions with incident vectors L1,L2,L3 :

L1L2

L3

Test object – Chopin terracotta

Tier 1 lamps – lowest Tier 5 lamps – highest



Effect of specular values on photometric stereo

V

N

SN’

P

Normal distorted away from view vector Computed normal for spherical surface

Finding normals by avoiding specular values

Intensity vs lamp number at one pixel Intensity values sorted into ascending order

Choose subset with slope similar to cosine



Normals and albedo

False colour: X in R, Y in G, Z in B

Gradients

Slopes in X and Y directions are given by partial derivatives of height wrt X and Y:

Compute intensity gradients from normals:



Simple summation of gradients

Cumulative sums of P gradientsin two directions across horizontal midline.

Cross‐sections of P and Q gradients across horizontal midline.

Horizontal and vertical image summations

Summation along rows Summation down columns



Measuring height at a few points

Height measuring gauge

Difference from reference height

Mean of two summations Error range from 0.96 to 7.48 mm



Profile of reconstruction

The clay disc on which Chopin’s head is moulded is tilted on a diagonal axis from upper left to lower right. Thus summation of gradients has

stretched the scale of the relief, while compressing the height of the disc.

View from south‐east at zero elevation, parallel to the X‐Y plane.

Height reconstruction by Fourier transform

Technique yields 3D surface that is continuous and is recognisably Chopin, but is distorted over the whole area with the height greatly amplified. Also there is a

false undulation of base with period of approximately one cycle over whole width.

Frankot R.T. and Chellappa R. (1988) A method for enforcing integrability in shape from shading algorithms,IEEE Trans. on Pattern Analysis & Machine Intelligence 10(4):439‐451.



Replacing inaccurate low frequencies

Smooth surface of hump produced by interpolation of measured points.

Log(power) distribution of spatial frequencies of hump gradients.

Log(power) distribution of spatial frequencies of photometric gradients.

The low spatial frequencies of gradients from Frankot‐Chellappa integration can be replaced by corresponding frequencies from hump.

Linear combination of spatial frequencies

Blended over a radial distance in the range 1.5 to 4.0 cycles/width by a linear interpolation (lerp) function.

Blending functions α and 1‐α. Oblique view of reconstruction



Difference from reference height

Elevation view Error range from ‐1.31 to +1.92 mm

Using a laser scanner

Arius 3D colour laser scanner Rendering from point cloud



Conclusions on height reconstruction

• Photometric stereo provides excellent normals but lacks overall scale.

• Scale can be obtained from a few discrete height measurements.

• Results are much higher in visual quality than laser scan with texture map.

Many applications inproducing surrogates.

Scarab of steatite with gold bandPetrie Museum UC11365

CloudComparematching of point clouds from Arius 3D scanner and reconstruction from dome image set.

Part 3 – Specular reflectance distribution

The world is not filled with Lambertian surfaces!

All real objects have some gloss or sheen.

Specular (from Latin speculum) is mirror‐like reflection.

Aim to model surface reflection as sum of diffuse body colour plus specular component.



Roman medallionTest object

64 images

8x8 mosaic

Detail 200x200

Pixel sampled from below eye



Variety of colour in a single pixel

Same camera

Same object

Same illumination

Same point

Same pixel

Same scaling

64 incident light directions

Intensity distribution at one pixel

• Vector of 64 values

• Low values similar to cosine (Lambertian)

• Few high values near specular direction



Finding surface normal from distribution

• Sort 64‐value distribution into ascending order

• Lowest values in shadow

• High values near specular

• Middle values are good approximation of cosine

• Use regression over selected vectors

Plotting 3D vector distribution at pixel

• V is view vector

• N is normal vector

• S is specular vector

• Lamp vectors shown in red are excluded.

• Lamp vectors shown in blue are selected to calculate normal by regression.

VN

S



Albedo Normal

Calculate ‘specular quotient’

• Ratio at each pixel of actual intensity/diffuse

• ~1 for matte areas

• >>1 for shiny areas



Plot ‘specular quotient’ vs angle

• High values near specular peak.

• Falling to asymptote of 1 with increasing radial angle from peak.

• Plotted here for 3x3 pixel cell, giving 9x64 = 576 values.

Fitting angular distribution of specular intensity

• Various functions for BRDF models in computer graphics.

• Lorentzian function chosen for its broad flanks:

11 ⁄



Fitting angular distribution of specular intensity

• First fit linear flank.

• Then fit Lorentzian function for values above flank.

• Four parameters in combined model:

1 ⁄

Polar plot of Lorentz distribution

• Diffuse values for 64 lamps calculated by cosine (blue).

• Specular values for 64 lamps calculated by Lorentzian (red).

• Sum is good match to the intensity data.



Image components of specular model

Specularamplitude

Specularwidth

Flankslope

Flankoffset

Photographic image Modelled image



• To obtain visual realism the directionality of the lighting must be considered.

• The Lorentzian function provides a good basis for modelling the specular component of reflectance.

• With a continuous function of angle, views can be interpolated between the original photographs.

Conclusions on specular rendering

Overall conclusions

Sets of images with structured light provide a much richer representation than a single image

1. Interactive visualisation and rendering

2. 3D reconstruction of the object surface

3. Modelling of specular highlights

There are many applications in cultural heritage for digitising and display of objects that are flattish with surface relief:

– coins, medals, fossils, rock art, incised tablets, bas reliefs, engravings, canvas paintings, etc.

Islamic handbag, c.1310, Mosul, IraqCourtauld Gallery, London

Technology

Dr Lindsay MacDonald, 3DIMPact Research Group, Faculty of Engineering, UCL