Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Surface Radiance Correction for Shape-from-Shading
Hossein Ragheb1 and Edwin R. Hancock2
1Department of Computer Engineering, Bu-Ali Sina University,
Hamedan, Iran, PO Box 65175-4161, [email protected]
2Department of Computer Science, University of York, York,
YO1 5DD, UK., [email protected]
Abstract
It is well known that many surfaces exhibit reflectance that is not well modelled by
Lambert’s law. This is the case not only for surfaces that are rough or shiny, but also
those that are matte and composed of materials that are particle suspensions. As a
result, standard Lambertian shape-from-shading methods can not be applied directly
to the analysis of rough and shiny surfaces. In order to overcome this difficulty, in this
paper, we consider how to reconstruct the Lambertian component for rough and shiny
surfaces when the object is illuminated in the viewing direction (retroreflection). To do
this we make use of the diffuse reflectance models described by Oren and Nayar, and
by Wolff. Our experiments with synthetic and real-world data reveal the effectiveness
of the correction method, leading to improved surface normal and height recovery.
1 Introduction
The recovery of surface shape from shading patterns has been a topic of sustained activity
in the computer vision literature for almost three decades. Some of the earliest work in
computer vision is reported in the PhD theses by Horn [1] and by Krakauer [2]. Early
developments in the field are well documented by Horn in the chapter “Height and Gradient
from Shading” which appears in the collection of papers in [3]. In addition to being limited
to surfaces of constant albedo illuminated by a parallel beam, the early work on shape-
from-shading (SFS) suffered from a number of technical problems. These include issues
of numerical instability [4], the need to deal with multiple light sources [5], and how to
initialize the surface normal directions [6]. During the 1990’s there was significant progress
in the field aimed at overcoming these problems. For instance, Kimmel and Bruckstein [7]
showed how the apparatus of level-set theory could be used to solve the differential equation
underpinning SFS, Oliensis and Dupuis [8] developed a provably correct method for surface
height recovery that involves propagation in the direction of maximum surface slope, Forsyth
and Zisserman [9] considered the problem of interreflection, Brooks et al. [10] investigated
the effects of ambiguous shading patterns, and, Kozera and Klette [4] have concentrated on
issues of numerical stability. An up-to-date review, that discusses a number of methods in
detail and which provides a comparative study is presented by Zhang et al. [11].
With hindsight, when measured against current achievements, some of the earliest work
on the subject was ambitious in its goals, allowing for both general reflectance models and
perspective geometry. Although the bulk of the recent work in SFS restricts itself to a
Lambertian model, the need for non-Lambertian reflectance models has been appreciated
from the outset. For instance, in work from photoclinometry Rindfleisch [12] has used the
non-Lambertian reflectance models of Hapke, Minnaert and Fesenkov [13, 14, 15] to recover
the slopes of lunar maria, and the reflectance map of Horn [16] is not restricted to Lambertian
reflectance. One area where non-Lambertian reflectance models are of critical importance,
and have been used with some success, is in radar SFS. This is a process of recovering
surface gradient and height from radar returns, and has proved to be an important tool
in the analysis of Megallan radar images of Venus. Here Frankot and Chellappa [17] have
used an empirical radar reflectance model in conjunction with a Fourier domain method for
reconstructing height under integrability constraints. In a recent paper, Bors et al. [18] have
developed a statistical method for radar SFS, using a digital elevation map to estimate the
radar reflectance function. However, one of the problems that can arise with radar-SFS is
the steepness of the radar beam which leads to range overlap.
1.1 Non-Lambertian Reflectance
Recently, there has been considerable interest in the computer vision literature in developing
reflectance models that can account for departures from Lambert’s law [19, 20, 21]. According
to Lambert’s law, the observed brightness at a point on the surface does not depend on the
viewing direction, and is proportional to the cosine of the incidence angle of the illuminant.
2
However, there are many situations where this simple reflectance model is violated. The
most familiar example is at the locations of specularities for shiny surfaces. Perhaps the
most widely used of these is the Phong model [22]. However, there are also situations in
which Lambert’s law is violated for non-specular reflectance. This is the case for matte
surfaces that consist of a suspension of reflecting particles in a transparent medium, those
that are rough (i.e. those in which there is a significant variation in local surface relief) and
those that are shiny (i.e. those where there is significant microfacet reflection). For rough
and shiny surfaces these effects are particularly marked at the occluding boundary where the
surface is highly inclined to the viewer direction. In the case of shiny surfaces the departure is
due to backscattering from microfacets which protrude above the surface. For rough surfaces
it is due to the foreshortening of surface cavities. For surfaces of intermediate roughness,
the diffuse scattering process is particularly complicated since it is a combination of effects
produced both by internal scattering and by the external roughness conditions. As a result
there may be limb brightening due to coherent scattering from microfacets that protrude
from the surface. The rough surface may also have cavities, and the angular distribution
of the cavity walls may also cause increases in limb brightness [23]. There are also surfaces
composed of materials that are particle suspensions in a transparent medium. These surfaces
while neither shiny, nor rough exhibit non-Lambertian reflectance that is best accounted for
by a “paint-model” [24]. Examples include semi-translucent plastics and surfaces to which
paint has been applied.
There have been many attempts to develop more complex light reflectance models for these
different types of surface. For instance, paint-models assume that the surface is composed of
a suspension of transparent refractive particles in a medium of different refractive index [2].
The Torrance-Sparrow model [25] assumes that the surface is composed of specular facets
whose orientation is governed by an angular distribution. Surface texture may be described
using a similar model in which the planar microfacets have an arbitrary reflectance function
[13, 14, 15]. Here the reflectance function could be Lambertian, and the departures from
Lambert’s law are governed by the angular distribution of the microfacets. However, an
alternative approach is to describe the surface using a roughness model based on a topo-
graphic relief distribution [26]. Although there are different definitions of surface roughness,
the parameter of the relief distribution is usually taken to be the root-mean-square height
deviation about the mean. The quantity can be obtained directly from surface-profile mea-
3
surements, or it can be calculated from scattering measurements provided that a theoretical
light scattering model is to hand. As we will see later, Oren and Nayar [23] adopt a different
measure of roughness based on the distribution of cavity wall slope angles. When estimating
the surface roughness for slightly-rough surfaces, it is standard practice to use the so called
TIS (Total Integrated Scattering) ratio measurement. However, the estimates returned by
this method are reliable only when the wavelength is long relative to the roughness. In fact
the TIS method forms the basis for a U.S. standard for effective surface roughness. As noted
by Bennett and Mattsson [26], there is no unique measure of surface roughness.
There are a number of ways in which rough surface reflectance can be modelled. For
instance, in the graphics community some purely empirical models have been developed
with the goal of developing computationally efficient tools for the purposes of performing
realistic surface synthesis [27, 28]. At the other extreme, in physics the study of rough
scattering has attracted considerable attention and has lead to the development of accurate
models using wave scattering theory and based on physically meaningful parameters that
can be directly measured [29, 30]. However, these models are not tractable for problems of
surface analysis in computer vision. As an example, the Beckmann formulation of Kirchhoff
scatter theory [29] fails to handle wide-angle scattering and large angles of incidence. The
reason for this is that Kirchhoff theory is based on a single scatter model and hence does
not account for effects such as multiple scattering or self shadowing, which occur at large
values of surface slope. Vernold and Harvey [30] have recently developed a modification of
the Beckmann model and have claimed that it gives excellent agreement with experimental
scattering data from rough surfaces at both large angles of incidence and at large scatter
angles. We have recently developed a technique for estimating the surface slope of very-
rough surfaces using this model [31]. Finally, if a model proves to be altogether too elusive,
then one approach is to acquire empirical estimates of the bidirectional reflectance function
(BRDF) from data. The BRDF is defined to be the ratio of the outgoing surface radiance to
the incoming light-source irradiance per unit solid angle. Measuring the BRDF is hence an
expensive and time consuming process since it has four degrees of freedom corresponding to
the zenith and azimuth angles of the light-source and the viewer. In a recent paper, Dana
et al. [19] have tabulated the BRDF’s of a number of different types of rough surface.
A compromise between the empirical and physics-based methods is offered by the phe-
nomenological approach adopted by Wolff [32], and, by Oren and Nayar [23] which attempts
4
to incorporate simple physics. These models attempt to account for departures from Lam-
bertian reflectance due to surface roughness or subsurface scattering in a parametrically
efficient manner. Each model modifies Lambert’s cosine law in a different way. Oren and
Nayar [23] address the problem of rough reflectance from matte surfaces. Their model ac-
counts for the angular distribution of surface cavity walls. When restricted to the case of
retroreflection (when the surface is illuminated in the viewing direction), the model adds to
Lambert’s cosine law a term that depends on the squared sine of the incidence angle. This
term results in brightening at the occluding object boundary. Wolff, on the other hand,
has a physically deeper model for diffuse reflectance from shiny but slightly rough surfaces.
The model uses an angle dependent Fresnel term to account for the refractive attenuation
of incident light at the surface-air boundary [32, 33]. This Fresnel term modifies in a multi-
plicative way the Lambertian cosine model. The effect is to depress the surface radiance for
near-normal incidence. Recently, Wolff et al. [34] have suggested a combination of these two
methodologies to model reflectance from surfaces with intermediate roughness. For the case
of retroreflection, the combined model includes both multiplication of the cosine law by a
Fresnel term, and the addition of a sine-squared term. Hence, there can be both an increase
in radiance at the limbs and a reduction in radiance at near-normal incidence.
1.2 Contribution
In this paper we are interested in whether these non-Lambertian reflectance models can
be used in conjunction with SFS. Of course the classical way in which to deal with non-
Lambertian surfaces is to follow Horn and construct a reflectance map as input to SFS
[16]. However, here we adopt a different approach, and explore whether we can use the
reflectance models to extract corrected Lambertian radiance from non-Lambertian surfaces.
This allows us to apply simple SFS algorithms, without the need to construct the reflectance
map. Although this approach allows non-Lambertian SFS to be conveniently modularized,
we do not argue that it is to be preferred to the use of the reflectance map.
The present work is also motivated by the fact that several of the most successful existing
methods for SFS are restricted in applicability to Lambertian reflectance and can not be
adapted easily to work with the reflectance map. Hence, our correction process can be used
as a simple tool to preprocess the input image data for such methods. It applies when
the viewer and light-source directions are the same. This is not an ideal configuration for
5
the perception of relief because of the lack of shadows. Several authors have investigated
different backscattering mechanisms. For instance, Orlova [35] found that the moon’s surface
reflects more light in the direction of the source (sun) than in other directions. Hapke et al.
[36] have discussed the retroreflection or opposition effect. Because the correction process
delivers corrected information about the angle of incidence at every surface location, this
allows the Lambertian SFS method to recover improved estimates of surface normal direction
and hence surface height. In other words, our correction process allows existing Lambertian
SFS methods to be applied to a greater variety of surfaces. It is also worth noting that
Lambertian correction can be useful to assist in visualization, since the limb brightening
due to roughness can result in further loss of perceived surface relief. Our experiments
with synthetic data confirm that the correction process results in improved surface normal
directions when a Lambertian SFS method is used.
The outline of this paper is as follows. Section 2 discusses the three different reflectance
models used in our analysis and explains how they may be simplified for the case of retrore-
flection. In Section 3 we describe how the simplified reflectance models outlined in Section
2 may be used to extract the corrected Lambertian component. Section 4 details experi-
ments on synthetic and real-world images. Finally, Section 5 offers conclusions and identifies
avenues for future research.
2 Diffuse Reflectance Models
Lambert’s law is the simplest model for diffuse reflectance from a surface. It has been widely
used in computer vision to recover shape from shading patterns. If θ is the angle of incidence
for the illuminant on the surface, the normalized radiance is IL(θ) = cos θ. There are a large
number of situations in which there are significant departures from Lambert’s law. This is
the case close to the occluding object boundary for all illumination directions. For illuminant
incidence angles larger than 50◦ relative to the viewer direction, there are also significant
departures from Lambert’s law both near the occluding boundary and over a large portion
of the object bounded by the shadow boundary [34]. However, it should be stressed that
the basis of Lambert’s law is purely phenomenological, and the reflectance function does
not attempt to model the physical processes of light scattering. The method does, however,
provide a surprisingly good account of reflectance from certain surfaces (packed Magnesium
6
Oxide for instance). Attempts have been made to justify the method using physics, and
these include the use of the distribution of black-body radiation from glowing particles.
In this paper we investigate whether the reflectance models described by Wolff [32] and
by Oren and Nayar [23] can be used as alternatives to Lambert’s law for rough and glossy
surfaces. In particular we investigate whether the two models can be used to perform radiance
correction with the aim of recovering the Lambertian component from the measured image
brightness. Unfortunately, since these non-Lambertian reflectance models depend on both
the incidence and reflectance angles they prove to be intractable for arbitrary geometry.
To overcome this problem, we restrict our attention to the retroreflection case where the
difference between the light-source direction ~L and the viewing direction ~V is small, i.e.
~L ≈ ~V . Although this is a restrictive assumption, it still results in methods which are of
practical utility. Few would view SFS as a tool for analyzing objects in unstructured scenes
and uncontrolled lighting. Instead, it may prove useful as a means by which to acquire
surface shape under controlled lighting conditions. The method described in this paper is
one that can be used to recover corrected Lambertian component under such conditions.
For instance, Zhang and Shah [5] have mentioned that when only one input image is used,
in order to recover the shape as completely as possible, the image has to be taken with
careful light-source placement to illuminate most of the object. Moreover, it is important
to stress that the restriction does not weaken or approximate the contributions from the
non-Lambertian component.
2.1 Oren-Nayar Model for Rough Surfaces
Oren and Nayar have developed a diffuse reflectance model for rough surfaces [23]. They
have used the roughness model proposed by Torrance and Sparrow [25] that assumes the
surface is composed of extended symmetric V-shaped cavities. Each cavity consists of two
planar facets. The width of each facet is assumed to be small compared to its length. The
roughness of the surface is specified using a probability distribution function for the facet
slopes. Further, it is assumed that the facet area is large compared to the wavelength of the
incident light, and hence, geometrical optics can be used to derive the reflectance model.
Finally, each facet is assumed to follow the Lambert’s law. In its more complex form,
the model takes into account detailed geometrical effects such as shadowing, masking and
interreflections between points on the surface. However, here we use their basic qualitative
7
reflectance model in which interreflections are ignored to simplify matters. It is interesting
to note that for the illumination conditions used in our method, both the more detailed and
the basic forms of the model are identical. According to this model (Fig. 1.a), for a point on
a surface with a roughness parameter σ, if the incident irradiance in direction (θi, φi) relative
to the surface normal is Li, then the surface radiance Lr in the reflection direction (θr, φr) is
Lr(θi, θr, φr − φi, σ) =ρ
πLi cos(θi){A + B max [0, cos(φr − φi)] sin(α) tan(β)} (1)
The parameters A and B, which are dimensionless, are only dependent on the surface
roughness σ which measures the distribution of cavity wall slope angles, and is hence
measured in degrees or radians (rd). Specifically A = 1.0 − 0.5σ2/(σ2 + 0.33) and B =
0.45σ2/(σ2 + 0.09). Also, the angles α and β are the maximum and minimum of the values
of the zenith angles θi and θr, i.e. α = max[θi, θr] and β = min[θi, θr]. It is important to
note that the model reduces to the Lambertian case when σ = 0. Finally, the parameter ρ
represents the surface albedo (reflectivity) which is assumed to be constant.
(a)0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6
0
0.2
0.4
0.6
0.8
1
Incidence angle (radians)
Rad
ianc
e
0.05
0.2
0.4
0.6 0.8
1.0
(b)
Figure 1: (a) Typical geometry for the local tangent plane PN of the surface normal ~N ; (b) surface
radiance versus θi using the Oren-Nayar model (Eq. 2) for σ = 0.05, 0.2, 0.4, 0.6, 0.8, 1.0 rd.
Retroreflection: Here we use a simplified version of the Oren-Nayar model for the pur-
poses of radiance correction, i.e. the process of recovering the Lambertian component. This
method is applied to the case of retroreflection where the light-source and viewing directions
are nearly identical (~L ≈ ~V ). Under such conditions θr = θi = θ and φi = φr, and so we
can approximate the Oren-Nayar model by making the substitutions cos(φr − φi) = 1 and
α = β = θ. As a result, the simplified radiance (for ρ = 1) is
ID(θ, σ) = A cos(θ) + B sin2(θ) (2)
8
Hence, the correction to Lambert’s law is additive and proportional to sin2 θ. This term is
greatest at the occluding boundary, and hence results in limb brightening. Fig. 1.b shows
the radiance versus the incidence angle for different values of the roughness parameter σ in
Eq. (2). The effect of increasing the roughness is to make the reflectance function flatter
with incidence angle. Hence, the contrast between the limb and the remainder of the object
is reduced. It is important to note that when φi = φr, the more detailed form of the model
of Oren-Nayar appearing in [23] is identical to the basic form given by Eq. (1), since when
cos(φr − φi) = 1 effects of the interreflection term vanishes.
2.2 Wolff Model for Smooth Surfaces
Wolff has developed a physically motivated model for reflectance from smooth surfaces [32,
34]. The model accounts for subsurface refraction using a Fresnel attenuation factor, which
modifies a Lambertian reflectance function in a multiplicative way. According to this model,
if Li is the incident irradiance at angle θi relative to the surface normal through solid angle
dω, then the reflected radiance at angle θr is
Lr(θi, θr, n) = %Li cos(θi)[1 − F (θi, n)]{1 − F (sin−1[(sin θr)/n], 1/n)}dω (3)
The attenuation factor, 0 ≤ F (αi, n) ≤ 1.0, is governed by the Fresnel function
F (αi, r) =1
2
sin2(αi − αt)
sin2(αi + αt)
[
1 +cos2(αi + αt)
cos2(αi − αt)
]
(4)
where αt is the transmission angle of light into the surface which is given by Snell’s law:
r = (sin αi)/(sin αt) ⇒ αt = sin−1[(sin αi)/r] (5)
where n is the index of refraction of the dielectric medium. Using Fresnel attenuation and
Snell refraction, the model accounts for how incident light and the distribution of subsurface
scattered light behaves at a smooth air-dielectric surface boundary. The Wolff model deviates
from the Lambertian form (i.e. cos θi) when the Fresnel reflection terms become significant.
Almost all commonly found dielectric materials have an index of refraction, n, in the range
of 1.4 to 2.0. For instance, for glass or silicon n ≈ 1.5, for ceramic n ≈ 1.7 and for titanium
oxide or diamond n ≈ 2.0. As a result the Fresnel function is weakly dependent upon the
index of refraction for most dielectrics. When light is transmitted from air into a dielectric
r = n and the incidence angle is αi = θi. However, when transmission is from a dielectric
9
into air, then r = 1/n and αi = sin−1[(sin θr)/n]. Here, we do not consider the more complex
case, which can be used to model paint, in which transparent particles of one dielectric are
immersed in a carrier of another dielectric with a different index of refraction. The role of the
scaling factor % is similar to that of the surface albedo parameter in Lambert’s law. For very
precise results, one should compute the value of % using the quite complex procedure given
in [34]. However, the value of % is very nearly constant over most incidence and reflectance
angles [34]. Hence, in this paper we treat % as a constant.
Retroreflection: We can also develop a simplified version of the Wolff model when
~L ≈ ~V . When θr = θi = θ the two Fresnel terms are identical and the radiance simplifies to
ID(θ, n) = cos(θ)[1 − F (θ, n)]2 (6)
The Fresnel term has the effect of depressing the radiance for near-normal incidence. In Fig.
2.a we plot the radiance versus incidence angle using Eq. (6) for different values of n. The
greater the index of refraction, the greater the reduction in the near-normal reflectance.
The Fresnel term F (θ, n) given by Eqs. (4) and (5) is shown in Fig. 2.b by a solid curve.
There are a number of features of the curve that deserve comment, since they are pertinent to
the correction process. The first of these is that the Fresnel term is approximately constant
and given by Fc(n) = (1 − n)2/(1 + n)2 until a shoulder is reached. At the shoulder, it
increases rapidly to unity when θ = π/2. We can approximate the shoulder by a straight-
line whose slope is given by the gradient of the Fresnel term at θ = π/2. To first order,
the equation of this line is Fs(θ, n) = 1 + 2(θ − π/2)(n2 + 1)/√
n2 − 1. The position of the
shoulder can be approximated by the intersection of the two lines. For small values of n− 1,
the position of the shoulder is approximately given by θs = π/2− (√
n − 1/2√
2). The larger
the refractive index of the material n, the larger the value Fc(n). As a result, the effect of
the correction process is simply to scale the Lambertian reflectance by a constant amount
(1 − Fc)2 until the shoulder is reached. We show this behavior in Fig. 2.b by plotting the
term [1 − F (θ, n)]2 versus incidence angle as a dashed curve.
2.3 Combined Model of Wolff-Nayar-Oren
Wolff et al. [34] have commented on the fact that the two diffuse reflectance models, i.e.
the Oren-Nayar model for rough surfaces [23] and the Wolff model for smooth surfaces [32],
are complementary in their applicability to surfaces with different roughness properties.
10
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.60
0.2
0.4
0.6
0.8
0.9
Incidence angle (radians)
Rad
ianc
e1.5
1.71.9
2.0(a)
0 10 20 30 40 50 60 70 80 900
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Fres
nel t
erm
( −
) and
the
Wol
ff te
rm (
− −
)
Incidence angle (degrees)
(b)
Figure 2: (a) Surface radiance versus θi using the Wolff model (Eq. 6) for n = 1.5, 1.7, 1.9, 2.0; (b)
Fresnel term (Eqs. 4-5; solid) and the term [1 − F (θ, n)]2 (dashed) versus θi for n = 1.4.
The Oren-Nayar model relies on the assumptions that the surface is composed of V-groove
microfacets and that the reflectance from each microfacet is Lambertian. The Wolff model,
on the other hand, assumes that there is subsurface optical scattering and air-dielectric
boundary conditions. These two effects are modelled by the Fresnel scattering term [34].
Hence, they [34] have suggested a methodology for how these two models can be combined, by
making the assumption that each V-groove microfacet reflects according to the Wolff model
(Eq. 3). Based on this assumption, a simple approximation to the diffuse reflectance model
that accounts for the observed empirical data over a broader physical range of roughness
conditions is obtained by replacing A in Eq. (1) with the quantity
C = A[1 − F (θi, n)]{1 − F (sin−1[sin θr/n], 1/n)} (7)
for an appropriate range of σ. This modification of the reflectance model is important
since radiance from surfaces with intermediate roughness exhibits a combination of effects
produced by both internal scattering and external roughness conditions.
Retroreflection: We can also simplify the combined diffuse reflectance model to the case
where ~L ≈ ~V . The simplified radiance is
ID(θ, σ, n) = A[1 − F (θ, n)]2 cos(θ) + B sin2(θ) (8)
Hence, the Fresnel term reduces the contribution from the Lambertian component at near-
normal incidence. The sin2 θ term is not affected, and results in limb brightening. In Fig.
3 we show the radiance versus the incidence angle using the combined model. The different
11
curves are for different roughness values in the range [0.01,0.3]. As the roughness increases,
then so the proportion of radiance at large incidence angles increases, and the departure
from the Lambertian case becomes more marked. For small values of σ the combined model
becomes identical to the Wolff model (Fig. 2.a). For large values of σ, on the other hand,
it does not behave exactly as the Oren-Nayar model. The combined model has an inflection
point close to θ = π/2, whereas the Oren-Nayar model does not exhibit this feature.
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.60
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Incidence angle (radians)
Rad
ianc
e
0.010.1
0.150.20.250.3
Figure 3: Surface radiance versus θi using the combined model (Eq. 8) for n = 1.7 and σ =
0.01, 0.1, 0.15, 0.2, 0.25, 0.3 rd.
3 Radiance Correction Process
Our overall aim is to use the diffuse reflectance models discussed in the previous sections
of this paper to perform radiance correction and hence recover the Lambertian component.
With corrected Lambertian images to hand, we can apply conventional SFS techniques to
recover surface shape information. There are of course many algorithms which can be used
for this purpose. However, here we use our recently developed SFS method outlined in [37].
This method is an extension of the Worthington and Hancock algorithm [6]. The surface
normals are constrained to fall on a cone whose axis points in the light-source direction
and whose apex angle is the inverse cosine of the corrected Lambertian component. In this
paper, we restrict our attention to the retroreflection case where the viewer and light-source
directions are co-incident. This means that the apex angle of the cone and the zenith angle
of the surface normal are identical. The zenith angle is the inverse cosine of the corrected
Lambertian radiance, i.e. θ = cos−1(IL). The position of the surface normal on the cone, i.e.
12
its azimuth angle φ, is free to vary. We initialize the azimuth angles of the surface normals
so that they point in the direction of the local image gradient. The surface normal directions
are iteratively updated by smoothing their directions and reprojecting the smoothed surface
normal directions onto the nearest position on the relevant cone. The smoothing process
adopted here involves aligning the directions of the local Darboux frames using the field
of principal curvature directions. Hence, we circumvent the problem of solving an under-
constrained inverse reflectance equation to recover surface normal direction. It is this inverse
problem that renders the non-Lambertian reflectance models intractable. By utilizing the
field of principal curvature directions we recover more accurate surface normals both at
parabolic locations on the surface and at the locations of surface discontinuities.
3.1 Closed Form Solution
The Oren-Nayar model can be inverted to recover the cosine of the incidence angle, and
hence the effective Lambertian radiance as follows. Suppose that at each image pixel (i, j)
the diffuse (non-specular) radiance ID consists of two components. The first of these is
a Lambertian component A cos θ. The second is the non-Lambertian component B sin2 θ
which takes on its maximum value where θ = π/2, i.e. close to the occluding boundary.
To perform radiance correction, we proceed as follows. At every pixel location, we use Eq.
(2) to estimate IL = cos θ using the measured radiance and solving the quadratic equation
ID = B + A cos θ − B cos2 θ for cos θ. The solution is
cos θ =A
2B
1 ±(
1 − 4B
A2(ID − B)
)
1
2
(9)
Since the Lambertian component is proportional to cos θ, this provides a means by which we
can perform direct radiance correction and remove the effects of rough surface reflectance.
The main problem with using Eq. (9) to estimate the Lambertian component is that
there are sometimes two possible solutions for each value of ID. This problem is illustrated
in Fig. 4.a which shows ID as a function of cos θ (for σ = 0.9rd); here the solutions are
found where the horizontal line corresponding to the radiance ID intersects the parabolic
curve. The value of cos θ must fall in the interval [0, 1], and hence we can rule-out solutions
that fall outside this interval. When cos θ = 0 then ID = B, and when cos θ = 1 then
ID = A. Moreover, the maximum point of the curve occurs at cos θ = A/2B which gives
ImaxD = B + A2/4B2. Fig. 4.b shows A and B as a function of σ. It is clear from the plot
13
0 10.35
Cosine of incidence angle
Sur
face
rad
ianc
e
(A−B)/B A/(2B)
B
A
B+(A/2B)(A/2B)
(a)
0 0.25 0.622 1 1.25 1.50
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Roughness parameter (radians)
A (s
olid
) and
B (d
ashe
d)
A
B
(b)
Figure 4: Inverse solution for the Oren-Nayar model using Eq. (9): (a) surface radiance versus
cos θ for σ = 0.9rd; (b) parameters A and B versus roughness parameter σ.
that A > B for all values of σ. Moreover, as σ becomes large then A tends to 0.5 and B
tends to 0.45. Provided that B < ID < A, then there is a single solution, and this lies in the
interval 0 < cos θ < (A−B)/B. When A < ID < ImaxD , then there are two possible solutions
placed symmetrically about cos θ = A/2B. When the quantity A/2B > 1 then only one of
the solutions gives a meaningful value of cos θ, and this is the one associated with the minus
sign in Eq. (9). The condition A/2B > 1 is satisfied when σ < 0.622rd; this is a rather large
value and corresponds to the extremes of roughness encountered in our experiments. Hence,
multiple solutions are only encountered when σ > 0.622rd for A < ID < ImaxD . This case
can be seen in Fig. 4.a where σ = 0.9rd. As the roughness parameter becomes large, then
A/2B approaches its limiting value of 5/9. Also, we note that when σ becomes small, i.e.
B becomes small then cos θ = (ID −B)/B. Finally, it is clear from Fig. 4.a that there is no
closed form solution for values of ID < B.
3.2 Lookup Table Solution
Although the problem of obtaining the corrected Lambertian radiance IL = cos θ in terms
of non-Lambertian diffuse radiance ID using the Oren-Nayar model is analytically tractable
(Section 3.1), the corresponding problems for the Wolff model and the combined model are
analytically intractable. For all three models, we therefore adopt a lookup table approach as
a practical alternative. To do this we tabulate cos θ as a function of the computed radiance
from each model. Since the Lambertian component is proportional to cos θ, this allows us to
14
perform Lambertian correction using the measured radiance values. In practice, the larger
the number of incidence angles tabulated (0 ≤ θ ≤ π/2), the more precise the correction.
For the Oren-Nayar model, using an approximate value of the surface roughness σ = σ0,
the lookup table is computed from Eq. (2). For the Wolff model, the radiance values are
almost identical for most dielectrics. This applies when the index of refraction n is in the
range (1.4 ≤ n ≤ 2.0). To give some idea of the physical significance of this restriction, glass
has a refractive index of about 1.5 while diamond has a refractive index which is even larger
than 2. Hence, we can use the center-range value (i.e. n = 1.7) when we compute the lookup
table using Eq. (6). Finally, for the combined model, the lookup table can be computed
using a value for the index of refraction and a value for the surface roughness using Eq. (8).
By assuming that ~L ≈ ~V , all three models can be made amenable to our lookup table
approach. The method is usable in practice since, like Lambert’s law, the brightness decreases
monotonically with increasing incidence angle (as shown in Figs. 1-3). As a result the
reflectance functions appearing in Eqs. 2, 6 and 8 are injective and invertible. Each measured
brightness value I0 is related to a single value of incidence angle θ0 and hence to a single
corrected Lambertian component cos(θ0).
4 Experiments
In this section we provide an experimental investigation of the correction process. We com-
mence with a study on synthetic data aimed at providing ground-truth, and then furnish
some results for real-world objects. In both cases the objects under study are illuminated in
the viewing direction.
4.1 Synthetic Data
In this section, we explore the use of the correction process on synthetic images with known
ground-truth surface normals. Here we experiment with cylindrical and spherical surfaces.
4.1.1 Cylindrical Surface
In Fig. 5.a we show the ground-truth elevation data for a cylindrical surface. Fig. 5.b shows
the corresponding Lambertian image. Using the ground-truth data, we generate three non-
Lambertian images using the Wolff (n = 2.0), the Oren-Nayar (σ = 0.52rd) and the combined
15
model (n = 1.7 and σ = 0.2rd). These images are shown in the top row of Fig. 6. We then
apply the Lambertian SFS method to these images and reconstruct the corresponding height
maps using the Frankot and Chellappa method [17], and the results are shown in the bottom
row of Fig. 6. Comparing the recovered height maps with the original surface in Fig. 5.a,
it is clear that there are significant shape difference, and these are attributable to errors in
the surface normal directions that result if SFS is applied to the uncorrected images. In Fig.
7 the top row shows the corrected Lambertian images when (from-left-to-right) the Wolff,
Oren-Nayar, and combined models are used. The lookup table parameters are identical to
those used in Fig. 6. In the second row of Fig. 7 we show the surfaces recovered from the
corrected images. The shape of the recovered surfaces is greatly improved.
Figure 5: (a) The ground-truth height map; (b) the ground-truth Lambertian image.
Figure 6: Top row: Images rendered using the Wolff (left), Oren-Nayar (middle) and combined
(right) models; bottom row: the corresponding height maps reconstructed using the surface normals
obtained by applying Lambertian SFS to the non-Lambertian images (in the top row).
In Fig. 8 we show plots of the brightness across transverse sections of the cylinders
shown in Figs. 6 and 7. Here the solid curves correspond to the initial uncorrected non-
Lambertian images, while the dashed curves correspond to the corrected Lambertian images.
16
Because of the large surface roughness value used in the Oren-Nayar model, the solid curve
(uncorrected) in Fig. 8.b exhibits the largest departure from the dashed curve (corrected).
In Fig. 8.a, it is important to note that despite relatively small differences between the
reflectance predictions by the Wolff model and Lambert’s law, the effect on the recovered
surface height is significant. This experiment illustrates the effectiveness of the correction
process and its usefulness for surface shape recovery.
Figure 7: Top row: corrected Lambertian images (by applying the table lookup method to the
images shown in Fig. 6) using the (a) Wolff, (b) Oren-Nayar and (c) combined models (from-
left-to-right); bottom row: the corresponding height maps reconstructed using the surface normals
obtained by applying Lambertian SFS to the corrected images (in the top row).
0 50 100 150 200 2500
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
(a)
0 50 100 150 200 2500
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
(b)
0 50 100 150 200 2500
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
(c)
Figure 8: Transverse brightness plots corresponding to the non-Lambertian (solid) and the cor-
rected Lambertian (dashed) images using the (a) Wolff, (b) Oren-Nayar and (c) combined models.
4.1.2 Spherical Surface
The surface studied here consists of a large elliptical dome surrounded by four smaller ones.
In Fig. 9.a we show the elevation or height data for the synthetic surface, while Fig. 9.b
shows the field of ground-truth surface normals. In Fig. 10, we show the result of rendering
17
the synthetic surface using the four reflectance models studied in this paper. From left to
right, the panels show the images obtained using the Lambertian model, the Wolff model
(n = 1.7), the Oren-Nayar model (σ = 20◦), and the combined model (n = 1.7 and σ = 20◦).
In Fig. 11, we illustrate the effect of applying the correction process to the synthetic
images. In the top row the panels from left to right show the corrected Lambertian images
obtained by applying the table lookup method to the synthetic images generated using the
Wolff model, the Oren-Nayar model and the combined model. In the bottom row, the panels
show the difference between the corrected images and the original images (the darker the
image location, the greater the difference). In all three cases, the effects of the correction
process are greatest near the occluding limb of the object. The values of the parameters used
for constructing the lookup tables are different from those used when creating the images of
Fig. 10. Our aim in doing this is to illustrate that even using imprecise values of the surface
parameters, the method gives good results. The lookup table parameters are n = 2.0 in the
Wolff, σ = 10◦ in the Oren-Nayar, and, n = 1.7 and σ = 10◦ in the combined model.
To show that the corrected images are closer to the ground-truth Lambetian image than
the uncorrected ones, in Fig. 12 we compare the corresponding difference images. The top
row shows the differences between the ground-truth Lambetian image and the normalized
uncorrected images of Fig. 10, whereas the bottom row shows the differences with the nor-
malized corrected images of Fig. 11. From left-to-right the columns show the results obtained
with the Wolff, Oren-Nayar, and combined models. Using the Wolff model the corrected im-
ages show negligible differences, while using either the Oren-Nayar or the combined model,
the differences are smaller but non-negligible. These two models are reasonably sensitive
to the roughness parameter used. Hence with imprecise values of the roughness parameter,
although the corrected radiance is closer to the Lambertian radiance, small differences are
still expected. However, the Wolff model does not have a roughness parameter and is not
very sensitive to the index of refraction chosen.
In Fig. 13 we illustrate the effect of the correction process on the arc-cosine (ACS) er-
ror for the estimated surface normals. Each panel in the plot shows the image of ACS =
cos−1( ~NG. ~NE), where ~NG is the unit ground-truth surface normal and ~NE is the estimated
unit surface normal at the corresponding image location. To improve visualization, we show
negative images of the ACS error (the darker the point, the larger the error). The panels
in the top row show the ACS error when the SFS algorithm is applied to the uncorrected
18
0 20 40 60 80 100 1200
20
40
60
80
100
120
Figure 9: Synthetic surface: elevation data (left); field of surface normal directions (right).
Figure 10: Images of the surface of Fig. 9 rendered (from-left-to-right) using the models of
Lambert, Wolff (n = 1.7), Oren-Nayar (σ = 20◦), and Wolff-Nayar-Oren (n = 1.7 and σ = 20◦).
Figure 11: Top row: corrected images corresponding to the non-Lambertian images of Fig. 10
using the Wolff, Oren-Nayar, and combined models (from-left-to-right); bottom row: differences
between uncorrected and corrected images.
images generated with the Wolff, Oren-Nayar and combined models (left-to-right). In the
second row, we show the ACS error plots obtained when the SFS algorithm is applied to the
corrected images. It is clear from the plots that the greatest error occurs at the cusp between
the surfaces, where the SFS algorithm results in some oversmoothing of the field of surface
normals. It is important to note that the problem of shape recovery at intersections between
surfaces with singular points and discontinuities cannot be overcome just by correcting re-
flectance properties. For points of the surface which do not exhibit such discontinuities,
the errors corresponding to the corrected images are smaller than those corresponding to
19
Figure 12: Differences between the ground-truth Lambetian image and the uncorrected images of
Fig. 10 (top row), and the corrected images of Fig. 11 (bottom row), with the Wolff, Oren-Nayar,
and combined models (from-left-to-right).
the uncorrected images. Also, while the errors are greatest at the object boundary before
correction, the distribution in error across the surface is relatively uniform after correction.
Figure 13: ACS error images for the ground-truth and the recovered surface normals corresponding
to the uncorrected images of Fig. 10 (top row) and the corrected images of Fig. 11 (bottom row).
To analyze the behavior of the method in more detail, we compute the total-mean-square
(TMS) error between the field of ground-truth unit surface normals ~NG and the field of
estimated unit surface normals ~NE. We compute the TMS error using the formula TMS =
Σi,j{( ~NGij (x) − ~NE
ij (x))2 + ( ~NGij (y) − ~NE
ij (y))2 + ( ~NGij (z) − ~NE
ij (z))2} where x, y and z denote
the vector components and i, j denote the image pixel location. In Fig. 14 we show the TMS
error between the estimated and ground-truth surface normals as a function of iteration
number. In each plot, the dashed curve corresponds to the uncorrected images, while the
20
solid curve corresponds to the corrected images. In each case the error is significantly lower
after the correction. For all cases we use 30 iterations in the SFS algorithm. To give some
idea of the scale of the TMS error, here we compute the maximum error. This maximum error
occurs when every estimated surface normal is in the opposite direction to the corresponding
ground-truth direction. For the surface used in this experiment (Fig. 9) TMSmax = 50292.
0 5 10 15 20 25 30550
600
650
700
750
800
850
Iteration
Nor
mal
Err
or
0 5 10 15 20 25 30400
500
600
700
800
900
1000
1100
1200
Iteration
Nor
mal
Err
or
0 5 10 15 20 25 30400
500
600
700
800
900
1000
1100
Iteration
Nor
mal
Err
or
Figure 14: TMS surface normal error versus iteration number when applying Lambertian SFS
method [37] to the uncorrected (dashed) and corrected (solid) images shown in Figs. 10-11 using
the Wolff, Oren-Nayar and combined models (from-left-to-right).
4.2 Real-World Data
Our experiments with real-world data are divided into four parts. We commence by illus-
trating the effect of the correction process. Second, we analyze the effects of the correction
process on the surface normal directions. Third, we show results on surface height recovery.
Finally, we present a study on cylindrical surfaces which allow the results of the correction
process to be compared with ground-truth.
The real-world images used in our experiments have been captured using an Olympus 10E
camera. Each surface has been imaged under controlled lighting conditions in a darkroom.
The objects have been illuminated using a single collimated tungsten light-source. To exper-
iment under the conditions where ~L ≈ ~V , the light-source and camera are slightly displaced
in the vertical plane. For dielectric shiny surfaces (e.g. porcelain objects) both specular and
diffuse components are present. The specular component must therefore be removed before
we can perform our radiance correction on the diffuse component. One reason for this is
that local specularities may be misidentified as high curvature surface features. To subtract
the specular component we use the method that we have reported earlier in [38]. However,
for matte dielectric surfaces (e.g. terra-cotta objects) the diffuse component is the dominant
21
one. Hence, specularity subtraction is not necessary. The images (raw images in the case of
matte objects and those to which specularity subtraction has been applied for shiny objects)
used as input to the correction process is referred to as “diffuse” images.
4.2.1 Results of Radiance Correction
In this section we show the results of performing surface radiance correction by applying
table look-up with the Oren-Nayar, Wolff and combined models. Since, the differences in
brightness resulting from the correction process are small, for the purposes of visualization
we magnify the differences by a factor K (typically K = 10). In the difference images, the
darker the pixel, the larger the difference value.
In Figs. 15 we show some results for three rough terra-cotta objects. In the top row of
Fig. 15, we show the original (diffuse) images. In the second row, we show the recovered
Lambertian images obtained by lookup table using the Oren-Nayar model. The third row
shows the images of the differences between the original images and the recovered images.
Finally, in the fourth row we show transverse brightness sections across approximately cylin-
drical regions of the objects. Here the solid curve is the original brightness while the broken
curve is the corrected brightness. Note that the broken curve is lower than the solid curve.
Although the correction process has an effect at almost every location on the surface, the
differences are most marked where the inclination of the object surface is steepest.
In Figs. 16, we turn our attention to shiny porcelain objects. Here we must first perform
specularity subtraction. Hence, in the second row of Fig. 16 we show the result of performing
specularity removal using the method described in [38]. These are the diffuse images used as
inputs to the correction process. The third row shows the specular component reconstructed
using the Torrance-Sparrow model [25, 38]. In the fourth row we show the recovered Lamber-
tian images obtained using the Wolff model. The fifth row shows the images of the differences
between the recovered Lambertian component and the diffuse component. When comparing
these results to those obtained using the Oren-Nayar model (Fig. 15), it is clear that the
radiance correction is greatest for the Wolff model. Also, the differences obtained using the
Wolff model are sharper than those obtained using the Oren-Nayar model. Finally, the sixth
row shows transverse brightness sections across cylindrical surface regions. Here the broken
curve (corrected) is higher than the solid curve (diffuse).
In Figs. 17, we show the results for two objects with smooth and rough surfaces when
22
Figure 15: Radiance correction using the Oren-Nayar model for terra-cotta objects: raw images
(top); corrected images (second); difference images (third); transverse brightness sections (bottom).
the combined model is used for radiance correction. The first row in Fig. 17 shows the
original images (diffuse component for the porcelain urn) while the second row in each
figure shows the recovered Lambertian components. The third row shows the differences
between the recovered Lambertian images and the diffuse images. Finally, the bottom row
shows transverse brightness sections. Here, for the locations close to the occluding object
boundary, the broken curve (corrected) is higher than the solid curve (diffuse). For other
surface locations the pattern is reversed. The main features to note from the difference
images are as follows. First, the correction on the limbs of the objects is most marked for
the porcelain urn which has a smooth surface. However, the amount of limb correction for the
terra-cotta bear is significantly smaller. Second, the amount of rough reflectance correction
is greatest for the terra-cotta bear, and, is least marked for the porcelain urn. Note that the
surface of the urn is not perfectly smooth and that there is a small roughness component.
23
Figure 16: Recovering Lambertian images (fourth) by removing specularities (third) from the raw
images (top) [38], and, applying the correction process using the Wolff model to the separated diffuse
images (second) of porcelain objects; differences (fifth); transverse brightness sections (bottom).
4.2.2 Surface Normal Directions
In this section we turn our attention to the surface normal directions recovered using our
Lambertian SFS algorithm [37]. In Fig. 18 we show the fields of surface normals, or needle-
maps, extracted from the images shown in Figs. 15-17. In the left-hand column of Fig. 18,
we show the needle-maps obtained when the Oren-Nayar model is applied (Fig. 15, terra-
cotta teapot), the middle column those obtained when the Wolff model is used (Fig. 16,
24
Figure 17: Radiance correction using the combined model for a smooth and a rough object: diffuse
images (top); corrected images (second); differences (third); brightness sections (bottom).
porcelain bear), and the right-hand column those obtained when the combined model is used
(Fig. 17, porcelain urn). The results in the top row correspond to the “diffuse” images, and
those in the middle row correspond to the corrected Lambertian images. We also investigate
the differences in the fields of surface normals extracted from the “diffuse” images and the
corrected images. In the bottom row we show the field of surface normal differences. The
most significant differences occur at the steepest locations on the surface. In addition, it
should be noted that the needle-maps reflect the fine detail of the surfaces.
We now focus in more detail on the effects of the correction process on the surface normal
directions. In qualitative terms, when the image becomes brighter then the apex (opening)
angle of the Lambertian cone becomes smaller. The reason for this is that the Lambertian
SFS method constrains the surface normals to fall on a cone whose apex angle is the arc-
cosine of the brightness. The axis of the cone points in the light-source direction, and since
~L = ~V , the apex angle of the cone is equal to the zenith angle θ. Hence the zenith angle θL =
cos−1(IL) of the surface normal decreases. When the image becomes darker, on the other
25
0 50 100 150 2000
50
100
150
200
0 20 40 60 80 100 120 140 160 180 2000
20
40
60
80
100
120
140
160
180
200
0 20 40 60 80 100 120 140 160 180 2000
20
40
60
80
100
120
140
160
180
200
0 50 100 150 2000
50
100
150
200
0 20 40 60 80 100 120 140 160 180 2000
20
40
60
80
100
120
140
160
180
200
0 20 40 60 80 100 120 140 160 180 2000
20
40
60
80
100
120
140
160
180
200
0 50 100 150 2000
50
100
150
200
0 20 40 60 80 100 120 140 160 180 2000
20
40
60
80
100
120
140
160
180
200
0 20 40 60 80 100 120 140 160 180 2000
20
40
60
80
100
120
140
160
180
200
Figure 18: Needle maps obtained by applying the Lambertian SFS method [37] to the uncorrected
(top) and the corrected (center) images shown in Figs. 15, 16 and 17; difference (bottom).
hand, then the zenith angle increases. Fig. 19 explores the effect of the correction process
on the azimuth and zenith angles for the terra-cotta bear using the combined model. The
left-hand plot shows the zenith angle θL = cos−1(IL) extracted from the corrected radiance,
as a function of the zenith angle estimated from the uncorrected diffuse image radiance,
i.e. θr = cos−1(ID). The center panel shows a plot of the azimuth angle estimated from the
corrected radiance, as a function of the azimuth angle estimated from the uncorrected diffuse
radiance. Since the normal is constrained to fall on a cone (as described above), in the zenith
angle plot there is no scatter and the data points trace out a curve. The Lambertian zenith
angle is greater than the diffuse zenith angle for small and intermediate angles. For large
angles, however, the pattern is reversed. The azimuth angle plot, on the other hand, does
exhibit scatter, but there is a clear regression line. The reason for this is that the surface
normals are free to rotate about the cones subject to smoothness constraints. To investigate
the stability of the surface normals under changes in azimuth angle, in the right-hand panel
26
0 10 20 30 40 50 60 70 80 900
10
20
30
40
50
60
70
80
90
−200 −150 −100 −50 0 50 100 150 200−200
−150
−100
−50
0
50
100
150
200
0 10 20 30 40 50 60 70 80 90−200
−150
−100
−50
0
50
100
150
200
Figure 19: Scatter plots for the incidence zenith (left) and azimuth (center) angles (in degrees)
and azimuth difference angles (right) extracted from the surface normals for the terra-cotta bear.
of Fig. 19 we show the difference in azimuth angle extracted from the “diffuse” and corrected
images as function of the zenith angle extracted from the corrected image. It is clear that the
large azimuth angle differences occur at small zenith angles. Hence, these are associated with
small changes in surface normal direction since the opening angle of the cone is small. The
largest differences in azimuth angle occur at intermediate zenith angles. This means that
the combined model results in significant differences in surface normal direction. Although
this effect is mainly attributable to the larger differences in corrected radiance, it may also
reflect problems associated with the curvature dependant smoothing employed in the SFS
algorithm. This uses the principal curvature direction to adjust the azimuth angle of the
surface normal. When the surface is umbilic (e.g. at spherical locations) or hyperbolic (e.g.
at saddles) there are singularities in the field of principal curvature directions and these in
turn may lead to unstable azimuth angle estimates. Note also that SFS results in small
differences in azimuth angle near object limbs. The reason for this is that the boundary
condition constrains surface normal to be perpendicular to the occluding boundary.
4.2.3 Height Map Reconstruction
In Fig. 20 we show the results of applying the Frankot and Chellappa algorithm [17] to the
surface normals extracted from three real-world images (shown in Fig. 18). The different
columns show the results for the porcelain bear, the porcelain urn and the terra-cotta teapot.
The top row of the figure shows the surface height maps recovered from the input “diffuse”
images (the top row of Fig. 18). The second row shows the results obtained from the
corrected images (the middle row of Fig. 18); we have used the Wolff model for the bear, the
27
combined model for the urn and the Oren-Nayar model for the teapot. The third row shows
the height difference between the surfaces recovered from the input “diffuse” and corrected
images. For comparison, the bottom row shows height data for the objects obtained using
a Polhemus Fastscan Cobra range sensor. The surfaces recovered are detailed and appear
to reflect the shapes of the underlying object very well. Much of the surface structure is
recovered, and sometimes this is better than that delivered by the range sensor. The reason
for this is that the range sensor relies on the stereoscopic reconstruction of depth from a laser
stripe. This stripe is easily detected on matte surfaces, but the reconstruction does not work
well on shiny surfaces since it is confused by reflections. The height differences between the
surfaces extracted from the input “diffuse” and corrected images are greatest at the object
boundaries.
4.2.4 Comparison with Ground Truth
In this section we compare the results of the correction process with ground-truth data. To
do this we use a cylindrical object, which allow the evaluation of results. We have aligned the
cylinder so its axis of symmetry is vertical (i.e. aligned with the y-axis). If d is the measured
diameter of the cylinder on the image, x0 is the horizontal coordinate of the symmetry axis
and x is the measured horizontal coordinate of the point on the cylinder, then the predicted
unit surface normal at the measured point is ~Ncyl(x) = (cos γ, 0, sin γ)T where cos γ =
2(x − xo)/d. The predicted Lambertian component on the cylinder is Icyl(x) = 2(x − xo)/d.
In the top row of Fig. 21, the left-hand panel shows the raw image of a strawboard cylinder.
We show the corrected images obtained using the Oren-Nayar model with σ = 0.1rd in the
center panel and with σ = 0.12rd in the right-hand panel. In the bottom row of the figure,
we show the difference (K = 5) between the raw and corrected images in the top row. Here,
the darker a point on a difference image, the higher the difference value. The effect of using
the larger roughness value is to increase the difference, especially near the boundary of the
cylinder. Fig. 22 illustrates the difference (K = 1) between the corrected Lambertian image
and the predictions delivered by the fit to the cylinder geometry. In the left-hand panel of
Fig. 22 we show the Lambertian component predicted by the cylinder fit. The remaining
panels in Fig. 22, from left to right, show the difference between the cylinder prediction and
the images in the top row of Fig. 21, i.e. the raw image, and the Lambertian corrections
obtained with σ = 0.1rd and σ = 0.12rd. The greatest difference occurs for the raw image.
28
Figure 20: Height maps reconstructed using the Frankot-Chellappa method [17] for surface nor-
mals corresponding to input “diffuse” images (top row) and corrected Lambertian images (second
row) using Wolff (left column), combined (middle column) and Oren-Nayar (right column) models;
difference between height maps (third row); elevation data using a range finder (bottom row).
In Fig. 23.a, we compare the recovered Lambertian radiance using the Oren-Nayar model
with the prediction of Lambert’s law for a cross section on the strawboard half-cylinder.
The plots show the normalized radiance as a function of the incidence angle γ computed
29
Figure 21: Top row: (a) raw image of a strawboard cylinder, corrected images using the Oren-
Nayar model for (b) σ = 0.1 and (c) σ = 0.12rd (left-to-right); bottom row: magnified differences
between the raw and corrected images.
Figure 22: From-left-to-right: ground-truth Lambertian image for a fitted cylinder, and, its abso-
lute differences with the raw image and the two corrected images of Fig. 21, respectively.
from the fit of the cylinder. In the plot, the solid curve is the measured radiance, the dash-
dot curve is the prediction of Lambert’s law, the dashed curve is the recovered Lambertian
radiance with σ = 0.1rd and the dotted curve that with σ = 0.12rd. The closer the corrected
Lambertian radiance to the Lambertian curve, the more successful the correction process.
Here, it is the dotted curve which is the closest to the Lambertian curve. Hence, the Oren-
Nayar model with σ = 0.12 rd results in a better correction than that with σ = 0.1rd. It
is worth mentioning that for higher values of the roughness parameter, i.e. σ > 0.12rd, the
difference between the corrected radiance and the ground-truth increases.
In Fig. 23.b, we show plots of the total error between the recovered surface normals and
the surface normals predicted by the cylinder fit, as a function of the iteration number for
the SFS algorithm. The solid curve is for the raw image, the dashed curve for the corrected
image obtained using the Oren-Nayar model with σ = 0.1rd, and the dash-dot curve that
obtained with σ = 0.12rd. It is the dash-dot curve that corresponds to the lowest error.
30
0 10 20 30 40 50 60 70 80 900
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Incidence angle (degrees)
Rad
ianc
e
Lambertian
Corrected (O−N, 0.1)
Corrected (O−N, 0.12)
Measured radiance
60 90 120 150160
180
200
220
Iteration
Nor
mal
Err
or
Raw image
Corrected (O−N, 0.1)
Corrected (O−N, 0.12)
Figure 23: Left: normalized raw radiance for the strawboard cylinder versus θi compared to the
corrected radiance using the Oren-Nayar model (σ = 0.1 and σ = 0.12rd) and Lambert’s law; right:
TMS error versus iteration number obtained by applying SFS to the images of Fig. 21.
5 Conclusions
In this paper, we have investigated how to perform radiance correction so that a Lambertian
shape-from-shading (SFS) method can be applied to both shiny and rough objects. We have
analyzed three diffuse reflectance models. The Wolff model is for shiny surfaces, the Oren-
Nayar model is for rough surfaces and the Wolff-Nayar-Oren model combines rough and
smooth scattering effects. There are a number of obstacles to the correction process. Firstly,
the models depend on both the direction of the incident light and the viewer. Second, even
when these two directions are identical, then the Wolff model and the Wolff-Nayar-Oren
model are not tractable because of the multiplicative nature of the Fresnel correction. To
overcome these problems, we construct a lookup table which allows the cosine of the incidence
angle, and hence the Lambertian component, to be recovered from the measured brightness.
It is interesting to note that in the case of the Oren-Nayar model, which involves an additive
rather than multiplicative correction of Lambert’s law, then the radiance correction can be
performed by inverting the radiance equation. To apply the reflectance models and perform
Lambertian correction, we need to know the surface roughness parameter and the index of
refraction appearing in the models. Here, we use approximate values of these parameters for
similar surfaces obtained from tabulations in the literature [23, 34].
We have derived simplified formulas for each model for the case of retroreflection to perform
radiance correction using table lookup. These simplified models are applicable when the angle
31
between the light-source and viewing directions is small. This condition is easily achievable
in practice. Under these conditions the simplified models are only dependent on the angle of
incidence. Finally, we experiment with a variety of real-world images of surfaces with different
scales of roughness. Here we show the results for our radiance correction process using the
three reflectance models. These results demonstrate that for surfaces with intermediate
roughness the results delivered by the separate rough or smooth reflectance models are poor.
However, the combined model of Wolff-Nayar-Oren provides reliable reflectance predictions
for a wide range of surfaces which can be characterized as having intermediate roughness.
We have also investigated the effect of the correction process on the surface normal directions
recovered using SFS. Our experiments show that different models can be used to correct the
radiance provided that the appropriate roughness conditions apply, and that the correction
process leads to improved surface normal estimates. Hence, we argue for the use of such
phenomenological reflectance models in applications including photometric stereo and surface
inspection.
Finally, it is important of note that the reflectance models studied in this paper are phe-
nomenological. Hence, the parameters are not directly related to the physical properties of
the surface. There is a considerable body of literature concerned with the detailed physical
modelling of rough surfaces. This work builds on the Kirchhoff theory [29] which has been
developed by Beckmann [29] and by Vernold and Harvey [30]. In the computer vision litera-
ture there have been some attempts to exploit this body of work for reflectance and texture
modelling [20]. This more thorough physics-based approach provides a means by which the
reflectance from surfaces of diverse roughness may be modelled. In a recent paper we have
used the Kirchhoff theory to develop reflectance models which can be used for rough surface
analysis tasks in computer vision. The main thrust of this work has been to use the the-
ory to develop ways of estimating surface roughness parameters [39]. The resulting models
are considerably more complicated than the phenomenological ones used here. However, we
have compared them to BRDF measurements, and they agree well with the data. In fact for
surfaces with intermediate roughness they outperform the phenomenological models studied
here. It is also worth noting that the wave scattering theory can also be used to perform
radiance correction, and [39] also explores this as a relatively minor component part of its
experimental evaluation.
32
References
[1] B.K.P. Horn, Shape from Shading: A Method for Obtaining the Shape of a Smooth Opaque
Object from One View, PhD Thesis, Department of Electrical Engineering, MIT, 1970.
[2] K.J. Krakauer, Computer Analysis of Visual Properties of Curved Objects, PhD Thesis, MIT
AI-TR, 19710.
[3] B.K.P. Horn and M.J. Brooks, Shape from Shading, MIT Press, Cambridge Massachusetts,
1989.
[4] R. Kozera and R. Klette, ”Finite Difference Based Algorithms for Linear Shape from Shading,”
Machine Graphics and Vision, 1997, pp. 157201.
[5] R. Zhang and M. Shah, “Iterative Shape Recovery from Multiple Images,” Image and Vision
Computing, vol. 15, 1997, pp. 801-814.
[6] P.L. Worthington and E.R. Hancock, “New Constraints on Data-closeness and Needle-map
consistency for SFS,” IEEE Trans. Pat. Anal. Mach. Int., vol. 21, no. 11, 1999, pp. 1250-1267.
[7] R. Kimmel and A.M. Bruckstein, “Tracking Level-sets by Level-sets: A Method for Solving
the Shape from Shading Problem”, Computer Vision and Image Understanding, vol. 62, no. 1,
1995, pp. 47-58.
[8] J. Oliensis and P. Dupuis, “A Global Algorithm for Shape from Shading”, Proc. IEEE Int’l
Conf. Comp. Vis., 1993, pp. 692-701.
[9] D.A. Forsyth and A. Zisserman, “Reflections on Shading”, IEEE Trans. Pat. Anal. Mach. Int.,
vol. 13, no. 7, 1991, pp. 671-679.
[10] M.J. Brooks, W. Chojnacki and R. Kozera, “Impossible and Ambiguous Shading Patterns”,
Int’l. J. Comp. Vis. vol. 7, no. 2, 1992, pp. 119-126.
[11] R. Zhang, P. Tsai, J.E. Cryer and M. Shah, “Shape from Shading: A Survey,” IEEE Trans.
Patt. Anal. Mach. Intel., vol. 21, no. 8, 1999, pp. 690-706.
[12] T. Rindfleisch, “Photometric Method for Lunar Topography,” Photogrammetric Engineering,
vol. 32, no. 2, 1966, pp. 262-277.
[13] B.W. Hapke, “A Theoretical Photometric Function for the Lunar Surface,” J. Geophysical
Research, vol. 68, no. 15, 1963, pp. 4571-4586.
33
[14] M. Minnaert, Photometry of the Moon, Chapter 6 in Volume 3 of Planets and Sattelites: the
Solar System, University of Chicago Press, IL, 1961, pp. 213-248.
[15] V. Fesenkov, Photometry of the Moon, in Physics and Astronomy of the Moon, New York,
Academic Press, 1962, pp. 99-130.
[16] B.K.P. Horn, “Hill Shading and the Reflectance Map,” Proceedings of the IEEE, vol. 69, no.
1, 1981, pp. 14-47.
[17] R.T. Frankot and R. Chellappa, “A Method for Enforcing Integrability in Shape from Shading
Algorithms,” IEEE Trans. Patt. Anal. Mach. Intel., vol. 10, no. 4, 1988, pp. 439-451.
[18] A.G. Bors, E.R. Hancock and R.C. Wilson, “Terrain Analysis Using Radar Shape-from-
Shading,” IEEE Trans. Patt. Anal. Mach. Intel., vol. 25, no. 8, 2003, pp. 974-992.
[19] K. Dana, S.K. Nayar, B.V. Ginneken and J.J. Koenderink, “Reflectance and Texture of Real-
World Surfaces,” Proc. IEEE Conf. Comp. Vis. Patt. Recog., 1997, pp. 151-157.
[20] B.V. Ginneken, M. Stavridi and J.J. Koenderink, “Diffuse and Specular Reflectance from
Rough Surfaces,” Applied Optics, vol. 37, no. 1, 1998, pp. 130-139.
[21] S. Magda, D.J. Kriegman, T.E. Zickler and P.N. Belhumeur, “Beyond Lambert: Reconstruct-
ing Surfaces with Arbitrary BRDFs,” Proc. IEEE Int’l. Conf. Comp. Vis., 2001, pp. 391-399.
[22] B.T. Phong, “Illumination for Computer Generated Pictures,” Communications ACM, vol.
18, 1975, pp. 311-317.
[23] M. Oren and S.K. Nayar, “Generalization of the Lambertian Model and Implications for
Machine Vision,” Int’l. J. Comp. Vis., vol. 14, no. 3, 1995, pp. 227-251.
[24] P. Kubelka, “New Contribution to the Optics of Intensely Light-Scattering Materials: Part I”,
J. Opt. Soc. Am., vol. 38, no. 5, 1948, pp. 448-457.
[25] K. Torrance and E. Sparrow, “Theory for Off-Specular Reflection from Roughened Surfaces,”
J. Opt. Soc. Am., vol. 57, 1967, pp. 1105-1114.
[26] J.M. Bennett and L. Mattsson, Introduction to Surface Roughness and Scattering, Optical
Society of America, Washington, D.C., 1999.
[27] J. Blin, “Models of Light Reflection for Computer Synthesized Pictures,” Computer Graphics,
vol. 11, no. 2, 1977, pp. 192-198.
34
[28] R.L. Cook and K.E. Torrance, “A Reflectance Model for Computer Graphics,” Computer
Graphics, vol. 15, no. 3, 1981, pp. 307-316.
[29] P. Beckmann and A. Spizzichino, The Scattering of Electromagnetic Waves from Rough Sur-
faces, Pergamon, New York, 1963.
[30] C.L. Vernold and J.E. Harvey, “A Modified Beckmann-Kirchoff Scattering Theory for Non-
paraxial Angles,” Scattering and Surface Roughness, Proc. SPIE, vol. 3426, 1998, pp. 51-56.
[31] H. Ragheb and E.R. Hancock, “Estimating Surface Characteristics using Physical Reflectance
Models,” Proc. IEEE Conf. Comp. Vis. Patt. Recog., vol. 2, 2003, pp. 177-184.
[32] L.B. Wolff “Diffuse Reflectance Model for Smooth Dielectric Surfaces,” J. Opt. Soc. Am. A,
vol. 11, no. 11, 1994, pp. 2956-2968.
[33] L.B. Wolff, “On The Relative Brightness of Specular and Diffuse Reflection,” Proc. IEEE
Conf. Comp. Vis. Patt. Recog., 1994, pp. 369-376.
[34] L.B. Wolff, S.K. Nayar and M. Oren, “Improved Diffuse Reflection Models for Computer
Vision,” Int’l. J. Comp. Vis., vol. 30, no. 1, 1998, pp. 55-71.
[35] N.S. Orlova, “Photometric Releif of the Lunar Surface,” Astron. Z, 1956, vol. 33, no. 1, pp.
93-100.
[36] B.W. Hapke, R.M. Nelson and W.D. Smythe, “The Opposition Effect of the Moon: The
Contribution of Coherent Backscatter,” Science, vol. 260, no. 23, 1993, pp. 509-511.
[37] H. Ragheb and E.R. Hancock, “Darboux Smoothing for Shape-from-Shading,” Pattern Recog-
nition Letters, vol. 24, nos. 1-3, 2003, pp. 579-595.
[38] H. Ragheb and E.R. Hancock, “A Probabilistic Framework for Specular Shape-from-Shading,”
Pattern Recognition, vol 36, no. 2, 2003, pp. 407-427.
[39] H. Ragheb and E.R. Hancock, “The Modified Beckmann-Kirchhoff Scatter Theory for Rough
Surface Analysis,” submitted to IEEE Trans. Pat. Anal. Mach. Int.
35