A Survey of Inverse Rendering Problemsima.udg.es/~dagush/papers/surveyInvLighting.pdf“sur 200 pag Volume 22 (2003), number 4 pp. 663–687 COMPUTER GRAPHICS forum A Survey of Inverse

“sur200pag

✐

✐ ✐

✐

✐ ✐

Volume 22 (2003), number 4 pp. 663–687 COMPUTER GRAPHICS forum

A Survey of Inverse Rendering Problems

Gustavo Patow and Xavier Pueyo

Grup de Grafics de GironaInstitut d’Informatica i Aplicacions

Universitat de Girona,Campus de Montilivi,

E-17003 Girona, [email protected], [email protected]

AbstractInverse rendering problems usually represent extremely complex and costly processes, but their importance inmany research areas is well known. In particular, they are of extreme importance in lighting engineering, wherepotentially costly mistakes usually make it unfeasible to test design decisions on a model. In this survey we presentthe main ideas behind these kinds of problems, characterize them, and summarize work developed in the area,revealing problems that remain unsolved and possible areas of further research.

Keywords: computer graphics, inverse problems, global illumination, lighting design, luminaire design.

ACM CSS: I.3.6 Computer Graphics Methodology and Techniques I.3.7 Computer Graphics—Three-Dimensional Graphics and Realism I.4.1 Image Processing and Computer Vision Digitization and Image CaptureI.4.7 Image Processing and Computer Vision Feature Measurement I.4.8 Image Processing and Computer VisionScene Analysis

1. Introduction

Inverse problems are usually of an extreme complexityand are emerging as an important research topic for thegraphics community due to their interest in a wide rangeof fields including lighting engineering and lighting design.Although progress in rendering to date has mainly focusedon improving the accuracy of the physical simulationof light transport and developing algorithms with betterperformance, some attention has been paid to the problemsrelated to inverse analysis, leading recently to interestingresults.

In computer graphics this sort of problem is notcompletely new: the problem of inverse kinematics has beenwidely applied for animation [61], and an excellent surveyon inverse placement of cameras, curves and objects foranimation can be found in [27] with references therein.

We can say that inverse problems infer parameters ofa system from observed or desired data which definetheir behavior, in contrast to direct problems which, given

all the parameters, simulate the effects. Traditional directproblems in lighting involve the computation of the radiancedistribution in an a priori, completely known environment(geometry and materials). These problems can be proven tobe well posed [23]. The inverse rendering problems lack atleast one of the Hadamard [19] criteria for being well posed:the solution does not depend continuously on the data, whichmeans that small errors in measurements may cause largeerrors in the solution (see [23]).

Inverse lighting problems refer to all the problems where,as opposed to what happens with traditional direct lightingproblems, several aspects of the scene are unknown. Onecommon characteristic of this kind of problem is that,in general, we know in advance the desired illuminationat some surfaces of the scene (their final appearance).Therefore, the algorithm has to work backwards to establishthe missing parameters. Such a tool is of extreme importancein lighting engineering, and animators and lighting expertsfor the film industry would also benefit highly from it.

c© The Eurographics Association and Blackwell Publishing Ltd2003. Published by Blackwell Publishing, 9600 Garsington Road,Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148,USA. 663

“sur200pag

✐

✐ ✐

✐

✐ ✐

664 G. Patow and X. Pueyo / Inverse Rendering Problems

Inverse illumination problems are intimately related to thecomputer vision field. The specific problems investigatedinclude shape from shading [20], direction of the luminaryfrom images and identifying surface characteristics from animage or a sequence of images. Fortunately, in a computergraphics system the viewing parameters and the exact scenegeometry (or at least part of it) are known and thereforemany problems become easier to solve. Our situation alsodiffers from the one in computer vision in that we wantthe user to control the illumination on a surface. As such,we expect the user to provide options and feedback into thesystem and thus solve ambiguities when they arise. Thus, theaim of this paper is leaning towards the Computer Graphicsfield, but the interested reader is referred elsewhere [20] forrelated developments in the Computer Vision field.

The paper is organized as follows: in Section 2 thetheoretical background is given and a classification based onthe rendering equation is presented. Next, in Section 3 theapproaches developed for each possible problem, accordingto the previous classification, are explained, characterizedand analyzed. In Section 6 the conclusions and open linesof research are presented.

2. Theoretical Background

Global illumination is related to transport theory and canbe viewed as a special case of it [8, 56]. The behavior oftransported light is characterized by the properties of theparticles (photons) when traversing the environment. Globalillumination’s most fundamental magnitude is radianceL(r, ω) which is defined as the power radiated at a givenpoint r in a given direction ω per unit of projected areaperpendicular to that direction per unit solid angle for a givenfrequency (Watt m−2 sr−1).

The boundary conditions of the integral form of thetransport equation are expressed as

L(r, ω) = Le(r, ω) +∫Si

fr (r, ω, ωi )L(r, ωi ) cos θdωi

(1)

for points r in surfaces, being fr the bidirectional reflection(and/or transmission) distribution function (BRDF), θ theangle between the surface normal at r and ω, Si thehemisphere of incoming directions with respect to r and ωian incoming direction.

This classical governing equation can be concisely ex-pressed as a linear operator equation [1, 2]. First, define thelocal reflection operator K by

(K h)(r, ω) ≡∫Si

k(r; ω′ → ω)h(r, ω′)dµ(ω′)

which accounts for the scattering of incident radiant energy.Here h is a field radiance function, corresponding to all

incident light. The K operator maps the incident lightdistribution onto the corresponding exiting light distributionthat results from one local reflection.

Next, we can define the field radiance operator G, thattransforms an exiting light distribution into the incidentlight distribution that results from surfaces illuminating oneanother:

(Gh)(r, ω) ≡{

h(p(r; −ω), ω) when ν(r, ω) < ∞0 otherwise

where ν(r, ω) is the visible surface function and isdefined [2] as ν(r, ω) ≡ in f {x > 0 : r + xω ∈Surfaces in the environment}.

Defining these operators we can factor out the implicitfunction r(r, ω) from the integral equation 1 and we maywrite:

L = Le + K GL . (2)

Following the outlines in Stephen Marschner’s Ph.D.thesis introduction [36], we can classify the different paperson inverse lighting problems according to which of thequantities of the above equation is unknown:

Direct problems are those which, given known valuesfor Le, K and G, solve for L . But, if we have someknowledge of L , we can pose different kinds of inverselighting problems.

If Le is unknown, and K , G and L or part of it, are known,we have a problem of inverse lighting: given a photograph orany other information that covers part of L , and a completemodel of the scene (K and G), find the emittances (Le) ofthe luminaries illuminating the scene.

If K is unknown, and G, Le and part of L are known,we must solve for information about K . This problem can,in general, be called inverse reflectometry, and a particularcase is the one called image-based reflectometry in [36],where images are used as input to the information about L .As described there, since K includes information about thevariance of the reflectance both spatially and directionally,this can be a very difficult problem since it can be a verycomplex function. Depending on the constraints imposed onthe problem, we can subdivide it into the inverse texturemeasurement (constraints on the directional variation), or theinverse BRDF measurement (spatial uniformity is assumed).In Table 1 we present the different papers surveyed in ourwork classified according to this scheme.

Finally, if G is unknown, we have an inverse geometryproblem. For an in-depth survey on those problems, refer to[46].

Equation (1) can be regarded in a signal processingframework [51] under the restrictions of distant illumination,

c© The Eurographics Association and Blackwell Publishing Ltd 2003

“sur200pag

✐

✐ ✐

✐

✐ ✐

G. Patow and X. Pueyo / Inverse Rendering Problems 665

Table 1: Papers reviewed in this work, classified according toequation 2.

Inverse Lighting (Le) [52], [37], [36], [7], [55], [35][23],[14], [12], [32], [15], [38] [40],[51], [59],[58],[57] [42],[47],[18]

Inverse Reflectometry(K )

[9], [3], [14], [12], [62], [32], [31],[26], [28], [44], [41], [13], [34],[45, 43], [36, 39], [54], [60], [10], [63],[36], [51]

Combined Problems(Le and K )

[29], [51], [58], [59]

no inter-reflections, isotropic BRDFs and known geometryand camera parameters. So, the reflected light field integral isregarded as the convolution of two signals: the bidirectionalreflectance function and the incident lighting; i.e. by filteringthe illumination using the BRDF. Inverse rendering cansimply be viewed as a deconvolution of the two signals.This framework [51] let the authors conclude that BRDFrecovery is well conditioned (in a mathematical sense) whenlighting contains high frequencies (e.g., directional sources)and is ill-conditioned for soft lighting. Alternatively, inverselighting is well conditioned for BRDFs with high-frequencycomponents (specular peaks) and ill-conditioned for diffusesurfaces.

Another factor to take into account is whether the differentpapers treat the full global illumination equation, Equation(2), or a simpler local-illumination version based on asimplification of the illuminating equation, considering onlypoint light sources and without considering inter-reflections.It is also important to mention the treatment of visibilityin the different approaches reviewed: when computing theradiance with the above equations, the visibility problemconsists of detecting if there are any blockers between thesource and the surface being illuminated, and not addingtheir contribution in that case. The same is true for thepaths from the surface to the eye or the region wherethe final radiance computations are needed. Most of thereviewed papers omit this treatment, arriving at solutions notapplicable in real-life situations.

Other kinds of methods are based on a treatment ofthe problem in its global form, considering the inter-reflections of light on the whole scene, that is, in thecontext of global illumination. Most of them work with somekind of projection space where they project the solution,transforming the integro-differential problem of computingthe illumination into a matricial one.

DistributionRadiosity

Reflector

BRDF

?^

K

G

Figure 1: Diagram of Inverse Lighting Problems, wherethe problem is to characterize the illumination (Le) on ascene, either by finding the emissivities of already positionedsources, or by finding their locations in the scene

Table 2: Classification of inverse lighting problems according towhether they compute surface emittances or light source position-ing. Also, the different works can be organized with respect to thetreatment they give to Equation (1): general, radiosity, Monte Carloand local.

Emittances Positioning

General [52],[37], [36] [7], [55], [35]

Radiosity [23], [14], [12], [32], [15], [38]

Monte Carlo [40]

Local [51], [59], [58], [57] [42],[47], [18]

3. Inverse Lighting Problems (ILP)

The inverse lighting problems are those problems where theunknown is the lighting of the scene (see figure 1 where avery simple scene is depicted with an unknown light source).These problems can be further classified into problems ofinverse emittances and inverse light positioning. In the for-mer the unknowns are the emittances of a given subset ofsurfaces of the scene. In the second the problem is to findthe locations of the light sources (or luminaries) in orderto achieve a desired illumination. Another possible classi-fication of ILP problems naturally arises when consideringthe treatment each one gives to Equation (1), where the dif-ferent approaches could be posed in a way such that anylighting algorithm would fit it. These are the classical F.E.radiosity setting (with BRDFs and radiance constant all overeach patch surface), an inverse Monte Carlo framework oreven by posing them as local illumination problems. Table 2presents a classification of inverse lighting problems papersaccording to these two criteria.


“sur200pag

✐

✐ ✐

✐

✐ ✐


3.1. Emittance Problem (EP)

As stated above, these problems deal with obtaining theemittance of a subset of the patches, the light sources. Thedifferent works dealing with this type of problem can begrouped according to the restrictions they introduce to solveto Equation (1) in the following classes:

• General formulations. We consider n distinct lightsources illuminating a scene, each of them beingcharacterized by a function �i , which represents theisolated contribution of the i-th light source, with unitintensity, on the environment. Thus, we could describethe illumination in a scene by a linear combinationof the form: � = ∑n

i ui �i , where ui is the non-negative weight of the i-th light on the environment. Byassuming a linear relationship � with some measuredintensity values α j in the scene or in a screen, we arriveat an expression of the form

α j = �(�) =n∑i

ui �(�i ) (3)

which can be regarded as a typical least squaresproblem. As we can see, the problem formulated thisway is rendering independent, since any approximationfor Equation (1) can be used without changing theformulation.

• Radiosity-based formulations. Here the general prob-lem is reduced to a radiosity setting by making the fol-lowing approximation to Equation (1): a purely diffuseBRDF for the surfaces (patches), that is constant all overeach one. In this case, Equation (1) is reduced to theform:

Bi = Lei + ρi∑

j

Fi j B j (4)

where Bi is the i-th patch radiosity, Lei its emittance, ρiits reflectivity (diffuse BRDF) and Fi j is the form factorfrom element i to element j . From there, re-writingit in the form of Equation (3) is trivial, as shown inSection 3.1.2. In general, in those problems the patchescan be grouped according to whether their radiosities Biand their emittances Ei are known or not, resulting ina system of equations with some Ei and some Bi asunknowns.

• Monte Carlo formulations. An Inverse Monte Carlomethod is proposed and proceeds by firing a set of raysfrom the surfaces with known properties towards pointson surfaces with unknown properties, and gatheringillumination information from surfaces of the first typeto the surfaces of the second type.

• Local Illumination formulations. A simple localillumination model is used instead of a global one. So,Equation (1) is actually not used.

Marschner and Greenberg [37, 36] studied the ill-conditioning of the inverse lighting problem in the caseof diffuse surfaces, and later Ramamoorthi and Hanrahan[51] presented a signal-processing framework for inverserendering, showing that inverse lighting is well conditionedonly when the BFDR has high-frequency components(sharp specularities), and is ill-conditioned for diffusesurfaces. They show that, for the special case of a mirrorBRDF, the lighting coefficients of a spherical harmonicsdecomposition of the Lighting correspond in a very directway to the reflected light field, thus being a well conditionedinverse problem. Instead, for Lambertian objects thelighting recovery is ill-conditioned for frequencies abovethe second order in a spherical harmonics decomposition(low frequencies). For Phong BRDFs, it is shown thatinverse lighting calculations are well conditioned only upto order of the square root of the shininess, while for theTorrance–Sparrow micro-facet model it is well conditionedonly for frequencies up to order of the inverse of theroughness.

3.1.1. General EP

One of the first approaches to the emittance problemwas by Chris Schoeneman et al. [52], where the userdefines the light features by “spraying” color onto surfaces.Later, Stephen Marschner and Donald Greenberg [37, 36]presented their re-lighting system, which, from a photographand a 3D surface model of the object pictured (and amodel of the camera used to take the picture), estimatesthe directional distribution of the incident light. In the firstcase, a modified Gauss-Seidel iteration was implemented tosolve Equation (3), in a way such that, at each iteration, thenegative values for the weights (light intensities) are clippedto zero. In this technique, the “sprayed” colors are the αi . Asmentioned above, the system is ill-conditioned if the BRDFused is too diffuse, so the authors decided to add a first orderlinear regularization, and to use a generalized singular valuedecomposition (SVD) to allow the regularization parameterto be adjusted interactively. Here, the αi are the observedpixel values on the photographs.

In the case of Schoeneman et al. , interactivity wasachieved by accounting only for direct illumination in theirfinal implementation, but the method is independent of theillumination algorithm used. Marschner and Greenberg’sapproach has the advantage of using a photograph asobjective, rather than manually user-defined objectives, andof using a generic, scene-independent set of basis lightsources. Unfortunately, this leads to a system which ismore ill-conditioned than the system that comes from aset of focused light sources, thus requiring a regularizationprocedure.


“sur200pag

✐

✐ ✐

✐

✐ ✐


3.1.2. Radiosity-based EP

The formulations presented in this subsection are closelybased on the radiosity approximations of Equation (1):assuming a purely diffuse BRDF for the surfaces (patches),and assuming it is constant all over them. In this case,Equation (1) reduces to the known

Bi = Lei + ρi∑

j

Fi j B j

where Bi is the i-th patch radiosity, Ei its emittance, ρiits reflectivity (diffuse BRDF) and Fi j is the form factorfrom element i to element j . Re-writing this equation inthe form of Equation (3) can be done trivially by defininga matrix Mi j = δi j − ρi Fi j which allows us to write

Bi = ∑j Le j M−1

i j .

A remarkable work with these sorts of inverse problemswas developed by Harutunian et al. [23], done in the contextof radiative heat transfer. The authors observed that theresulting set of equations for the inverse problem is ill-conditioned, and thus the need to resort to the ModifiedTruncated SVD [25] (MTSVD) matrix inversion method tocompute Lei . Working with the same approach (linearizingthe system to solve it and MTSVD or TSVD to invert it dueto its ill-conditioning), Franca et al. [15] and Morales et al.[38] solved the inverse problem of source emissivities, thistime in the presence of participating media. In the latter twocases, the problem requires the introduction of a system ofequations resulting from the discretization of the mediuminto volume elements in order to solve the correspondingpartial differential equations (PDE).

Computer Augmented Reality enables users to mix realand virtual worlds. As such, it requires the precise char-acterization of the geometry, source-emittances and surfacereflectances of the real scene through a given set of pho-tographs. To compute these emittances, Fournier, Gunawanand Romanzin [14] also base their formulation on the ra-diosity approximation, directly fitting the element emissionsto the observed values (αi ). Drettakis, Robert and Boug-noux [12] improved the work of Fournier et al. by setting upa hierarchical radiosity system. Loscos et al. [32] approxi-mately reconstructed real scene geometry from photographstaken from several different viewpoints. There, indirect illu-mination is computed with a hierarchical radiosity system asbefore, while the direct component is calculated separatelyusing ray-casting on a per pixel basis.

3.1.3. Inverse Monte Carlo EP

As is well known, radiative heat transfer problems are equiv-alent to lighting problems. We summarize here an interestingwork on inverse radiative heat transfer, finding the tempera-tures of emitting surfaces, carried out by Masahito Ogumaand John Howell [40]. The authors developed an Inverse

Monte Carlo method. In this paper, surfaces are perfect lam-bertian reflectors, although the generalization to more gen-eral BRDFs seems straightforward. The method starts byletting users choose a set of points (calculation points) onthe surfaces where they are interested in finding the tem-perature (light) distribution. Then, it randomly chooses a setof points at the surfaces with completely known properties,and for each of those points casts a given number Ns of rays.Although never stated explicitly in the paper, the points seemto be chosen following a uniform distribution, and the raysare fired by subdividing the hemisphere of directions aboveeach surface point in Ns equal intervals, and firing one rayfor each interval. These rays fly to points at the surfaceswith unknown temperature (emittance). For each ray, thealgorithm finds the expected energy it should carry. Whenenough rays have arrived, each calculation point on an “un-known” surface is assigned a temperature that is a weighted-sum of the temperatures of the rays. The weights are a set ofposition-related coefficients, because the exiting point of theincident intensity is often not a calculation point. Finally,the whole process is iterated until the radiance distributionon “known” surfaces led by the calculated temperature dis-tribution of “unknown” surfaces satisfies the required heatflux distribution of “known” surfaces within a user-providedthreshold.

3.1.4. Local Illumination EP

Based on the features presented in the introduction ofSection 3.1, Ramamoorthi and Hanrahan [51] proposed analgorithm that only recovers frequencies below a cutoffof the order of the inverse of the roughness. Thus, twopossible ways are shown: solving a linear least-squaressystem for the lighting coefficients like Equation (3), orsubtracting the diffuse component and using the resultingmirror-like object to recover a high-resolution angular-spaceversion of the illumination. In the second case, a two-stepprocess is presented, where the first phase estimates thediffuse components of the reflected field from the estimatedillumination frequency parameters, and the second phasedoes it the other way round to achieve sharper results.

Sato et al. [59] use the radiance information insideshadows to estimate the illumination distribution of a sceneas a collection of imaginary point light sources uniformlydistributed over the scene. To do this, it is necessary for theBRDF to be Lambertian, and to solve a system like Equation(3). This work was later improved by the authors [58], seesection 5, but the introduction of regularization using user-weighted penalty terms was required in both works, andthe computational complexity limited the formulations toa coarse discretization of the sphere. Instead, the methodproposed by Ramamoorthi and Hanrahan [51] requiredno explicit regularization and yielded results with bettersharpness and overall quality than the approaches of Satoet al. On the other hand, the methods proposed by Sato et al.


“sur200pag

✐

✐ ✐

✐

✐ ✐


are easier to extend to concave surfaces. Finally, Sato et al.[57] proposed the use of two omni-directional stereo imagesto construct a geometric model of the scene: extractingcommon feature points, generating a triangular mesh andfinally mapping the radiance over the mesh. The radiance ofthe whole scene was constructed from a sequence of omni-directional high dynamic range radiance images [11] andmapped onto the constructed geometric model.

3.2. Light Source Positioning Problem (LSPP)

In this subsection we review the papers which perform com-putations on the position and orientation of light sources. Asmentioned above, these works can also be classified accord-ing to the treatment they give to Equation (1).

• General formulations. These methods try to find lu-minary locations and/or orientations without relying onany particular illumination algorithm. In general, the lu-minary position/orientation is chosen as an optimizablevariable of a certain objective function, which, in turn,is optimized with a problem-independent optimizationalgorithm like Stimulated Annealing [49, 48].

• Radiosity-based formulations. Problems which tryto locate light sources in the context of the classicradiosity approximation have not been presented yet,but we believe this is possible. Unfortunately, thiscase corresponds to purely diffuse BRDFs, which showthemselves as producers of a severe ill-conditioningin lighting-characterization problems [50]. The overallmethod would assume small polygonal emitters anduse the knowledge of the illumination they produce tocompute their form factors (Fi j in Equation (4)). Fromthis knowledge we should find the desired vertices asdegrees of freedom of an optimization problem.

• Monte Carlo formulations. Although, to the best ofour knowledge there are no works using this kind offormulation, we believe that this is a feasible approach,too. By firing rays from the illumination-constrainedareas of the scene, it is possible to gather informationand qualify areas in the scene space which may containthe sources, and by further refinement, either automaticor interactive, finding their exact location within acertain threshold.

• Local illumination formulations. These formulationsare based on the simplification of Equation (1) to takeinto account only local illumination, and using theobservation of this local illumination (highlights andshadows) to position the corresponding light sources.Thus, instead of Equation (1), we could introduce theexpression for a point light source [8],

L(r, ω) = Le(r, ω)+ �

4π |r − rs |2 fr (r, ω, ωs) cos θsdωi

where θs is the angle between the normal and the lightsource direction. When using shadow information forpositioning the light sources, points on the shadowboundary must be determined and joined to theircorresponding blocking silhouette in order to get areliable direction. When enough pairs are defined, aleast squares procedure can be performed. On the otherhand, when using highlights for source locations, pointson the desired highlight maximum and its boundary(defined to be the line where the highlight falls below agiven threshold) must be given. This procedure stronglydepends on the BRDF chosen, in general a simplePhong [5] formula.

3.2.1. General LSPP Formulations

Costa et al. [7] implemented an automatic method tosearch for the best placement of luminaries, as well as theirrelative intensities. The method implements a preprocessingstep where the user-defined requirements (called inverseluminaries, IL) are considered as sources of unit importance,which propagate through the environment as the dual ofradiance [17]. This allows using any global illuminationengine to run the simulation backwards, from the InverseLuminaries to a user-defined set of surfaces where theimportance distribution is computed. Basically, what ispresented is a validating preprocess step which tries tofind incompatibilities between design goals and alreadyplaced design elements (light sources), followed by acalculation step which attempts to find the best placementand orientation for the light sources by optimizing auser-defined objective function. The minimization ofthe objective function is performed with the SimulatedAnnealing algorithm [49, 48]. The objective functionis given to the system by means of a script languagespecially developed, which allows the specialized userto define the lighting goals to achieve (positive ILs),the illumination constraints to avoid (negative ILs, likehaving too much glare into a virtual character’s face), andthe geometric constraints the user might impose on thelocation or direction of the sources. Although this methodof scripting the design goals seems very promising, a higherabstraction level should be achieved in order to allow thenon-programming-skilled designers to be able to use thepresented tools.

Instead, Shacked and Lichinski [55] presented anapproach to lighting design based on the optimization ofan objective function which is a perception-based imagequality function. This function was designed to yieldcompressible images of 3D scenes, trying to effectivelycommunicate information about shapes, materials andspatial relationships. Their current implementation wasbased in an OpenGL rendering engine and a local steepestdescent optimization scheme, although the authors statedthat local minima were found to be quite satisfactory if


“sur200pag

✐

✐ ✐

✐

✐ ✐


initial values were chosen wisely [53]. The main differencewith [7] is the choice of the perception-based optimizationfunction, as well as the use of a local optimization methodvs. the global algorithm used before.

An entirely different approach for exploring the space oflighting designs was presented by Marks et al. [35]. In aframework named Design Galleries, they try to optimallydisperse the space of solution images in terms of perceptualquality, and allow the user to browse and combine them toachieve a desired solution.This is clearly not an automaticprocess, since user input is required.

3.2.2. Local Illumination LSPP Formulations

One of the first works on inverse problems in the context oflocal illumination was done by Poulin and Fournier, [42].They proposed using the highlights and shadows on thescene’s objects in the modeling of the light sources. Inthe case of highlights, the authors considered the specularterm of Phong [5] shading as expressed by Blinn [4]. Byletting the user manually point at the desired maximumintensity of this highlight, they were able to analyticallyoptimize this expression and determine the light direction.By determining another point on the surface, the userspecifies where the specular term reaches a fixed threshold,and thus the roughness exponent can be computed. The useof further restrictions, like the light being on a given plane,is suggested for other cases. Observe that this method givesonly directional light sources, that is, point light sourcesat infinity. For a general point light source, the shadowvolume [16] generated by it must be used [45]. In orderto specify the direction of a directional light one simplychooses two arbitrary distinct points in the scene, the secondbeing along the shadow cast by the first one. For extendedlinear or polygonal (planar) light sources, new point lightsources that define the vertices of the light source are needed.For general extended light sources a divide and conquerstrategy was presented: if both the light and the object beingshaded are divided into convex elements, the whole shadowis the union in 3D of all the shadow convex hulls.

Poulin, Ratib, and Jacques [47] find the position of pointlight sources by sketches of shadows or highlights, andextended light sources are positioned by sketches of umbraor penumbra. The user introduces the sketches as continuousstrokes of points that are immediately transformed to 3D[22], where the sketched points are considered to be allenclosed by the real shadow of the object, and similarlyfor a highlight. The method starts by considering each pointforming the sketch of the shadow and defining for it the coneof possible positions for the light (Figure 2). The volumewhere the light can be is the intersection of all these cones:if the volume is infinite, a directional light is computed;otherwise a point light is used. The problem is presentedas a constrained optimization problem by defining as anobjective function the distance between the sketched points

Stroke

Valid positionsfor light source

Sketched point

Blocker

Cone

Figure 2: Elements for the computation of the position of alight source from a sketch of a shadow.

and the light source, and maximizing it. The constraintsdefined are that the point light source must lie inside allcones, that the light must stay on the same side as the normalvector at the sketched point, and that the light position ison the right side of the half-cone for this sketched point,oriented along the axis that joins this point and the centerof the occluder. The initial position for the solver is chosenas a small distance above the occluder surface aligned withthe center of mass of the sketched points. For extended lightsources, the user sketches the umbra or penumbra, relaxingthe inclusion condition to force all points to lie withinall cones simultaneously in the case of umbras, and forpenumbras the condition is that at least one point of the lightmust belong to each cone (a test that the intersection of thecone and the light is not null must be made). When sketchinghighlights, a point on a surface is considered within ahighlight if the evaluation of Phong specular function at thispoint is higher than a certain threshold τ . It could be thatthe cones do not intersect at all, due to the curvature ofthe surface. Then, the roughness coefficient n (the exponentin Phong specular term) is lowered, broadening the cone,until an intersection is possible. The main difference with thepreviously mentioned research [42] is that the input method,sketching shadows and highlights, is far superior in termsof user interaction and greatly improves upon the design ofsimple illumination in a computer graphics scene.

A different approach was taken by Guillou [18], who de-veloped a local illumination-based method to determine theposition of n light sources. His method starts by comput-ing m directional light sources from user-defined regionsof a known, purely diffuse scene. Then, these m directionalsources are grouped into sets and each set is used to estimatethe position of a point light source by trying to optimizean intersection point from the directions of the distant lightsources. Finally, the light position and photometric parame-ters are found by minimizing (Levenberg–Marquardt numer-ical minimization method) a least squares error, using theprevious estimations as a starting point. As can be seen, this


“sur200pag

✐

✐ ✐

✐

✐ ✐


method is strongly local-illumination-based, since it relieson an estimation step that closely follows this assumption.Also, this distant light estimation step can only serve as areference as long as the user-defined regions are cleverlychosen, since it does not take into account the superpositionof illumination from the different sources at the points wherethe estimations are computed. With respect to the previousworks mentioned, it has the clear advantage of working si-multaneously with n point light sources in a diffuse environ-ment.

3.3. Conclusions on ILP

Analyzing the problems concerning EP (Subsection 3.1) wesee that:

• Most of the papers deal with perfectly diffuse BRDFs.The more relevant exceptions to this rule are [52], [37],[36] and [51], that deal with linear combinations ofphotographs and present algorithms that are independentof the BRDF used because they only depend on theobtained image, not on the method to compute it.([51] assumes a local illumination model to deconvoluteillumination from the known BRDF.)

• The assumption that the BRDF is constant on thepatch surface is found in most of the papers, too. Theexceptions to this rule are [40] and [52], [37], [36],mainly because the first paper presents an algorithm thatsamples on the surfaces pointwise and the others areBRDF independent.

• Only Franca et al. [15] and Morales et al. [38] solved theinverse problem of source emissivities in the presenceof participating media. The rest of the reviewed worksonly deal with non-participating media. It is importantto note the added cost: in this case, the problem requiresthe introduction of a system of equations resulting fromthe discretization into volume elements of the mediumin order to solve the corresponding PDE.

• Among the reviewed papers, only [52], [37], [36]and [29] present viewing-dependent goals, especiallythe first three because they deal with input given byphotographs, while the last one can also be used withview-independent goals.

• It is also important to point out that, although themethod proposed by Schoeneman et al. [52] achievedinteractivity by accounting only for direct illuminationin their final implementation, it is independent ofthe illumination algorithm used. The same happenswith Marschner and Greenberg’s approach [37, 36],which has the advantage of using a photograph asobjective, rather than manually user-defined objectivesas before, and of using a generic scene-independentset of basis light sources. Unfortunately, this leads to amore ill-conditioned system than the system that comes

from a set of focused light sources, thus requiring aregularization procedure.

• Sato et al. [59, 58] required the introduction ofregularization using user-weighted penalty terms, andthe computational complexity limited their formulationsto a coarse discretization of the sphere. Instead, themethod proposed by Ramamoorthi and Hanrahan[51] required no explicit regularization and yieldedresults with better sharpness and overall quality thanthe approaches of Sato et al. On the other hand, themethods proposed by Sato et al. are easier to extend toconcave surfaces.

With respect to LSPP (Subsection 3.2), it is very evidentthat two ways of attacking this problem, that is radiosity-based formulations and Monte Carlo formulations, havenot been studied in the literature. Nevertheless, in ouropinion, these ways are feasible in spite of the possibleill-conditioning they might present (which would surelybe alleviated by the use of some sort of regularizationprocedure).

In the reviewed articles we can also notice that:

• It is remarkable that [7], [55] and [35] are the onlyglobal illumination-based approaches among all thereviewed papers for this kind of problem. Contrary towhat happens in inverse reflectometry problems, allstudied approaches rely on some sort of optimizationprocedure to achieve their results. This is so because ofthe high complexity of the problem faced, since findingabsolute locations of light sources or types (and othercharacteristics) of luminaries involves using indirectlymeasured information. The choice of the optimizationmethod is, to our knowledge, quite arbitrary and thepapers present several different approaches to this point.It is clear that the best optimization method to use is stillan open research area.

• Although the method by Costa et al. [7] for scriptingthe design of goals seems very promising, a higherabstraction level should be achieved in order to allowthe non-programming-skilled designers to be able to usethe presented tools.

• On the other hand, the main difference between Shackedand Lichinski [55] and Costa et al. [7] is the choice in thefirst case of the perception-based optimization function,as well as the usage of a local optimization method vs.the global algorithm used before.

• With respect to the work done by Marks et al. [35], itcan be clearly seen that this process does not involve anautomatic optimization at all, the user being responsiblefor all the decisions towards the final result.

• The main difference of the work done by Poulin, Ratiband Jacques [47] with the work by Pouin and Fournier


“sur200pag

✐

✐ ✐

✐

✐ ✐


Table 3: Classification of inverse reflectometry problems organizedwith respect the treatment they give to Equation (1): general,radiosity, Monte Carlo and local.

General [9], [3]

Radiosity [14], [12], [62], [32], [31]

Texture-based [26], [28], [44], [41], [13], [34]

Local Angular-based [45, 43], [36, 39], [54], [60], [10]

General [63], [36], [51]

[42] is that the input method, sketching shadows andhighlights, is far superior in terms of user interaction andgreatly improves upon the design of simple illuminationin a computer graphics scene.

• Finally, for Guillou [18] we can say that it has theclear advantage of working simultaneously with n pointlight sources in a diffuse environment, whilst the othermethods presented in the same subsection do not.

4. Inverse Reflectometry Problems (IRP)

Inverse reflectometry problems are those where K in equa-tion 2 is unknown, and G, Le and part of L are known (Fig-ure 3). Thus, we must solve for information about K . Themethods studied in this section can be classified according tothe illumination approach used. As above, this can be eitherlocal-based, general global illumination, Monte Carlo-basedor radiosity-based, see Table 3.

• General formulations. These methods try to findreflectance properties without relying on any particularillumination algorithm.

• Radiosity-based formulations. As stated in section 3.1,here the general problem is reduced to a radiosity settingby making the following approximation to Equation (1):a purely diffuse BRDF for the surfaces (patches), that isconstant all over each one. In this case, Equation (1) isreduced to the form:

Bi = Lei + ρi∑

j

Fi j B j

where Bi is the i-th patch radiosity, Lei its emittance, ρiits reflectivity (diffuse BRDF) and Fi j is the form factorfrom element i to element j . If we know Bi and Lei forevery surface, and with form factors Fi j known if thegeometry is known, finding the reflectances is reducedto:

ρi = (Bi − Lei )/∑

j

Fi j B j .

Source

DistributionRadiosity

Reflector

BRD

K^

Le

^G

Figure 3: Diagram for inverse reflectometry problems.

The most common way of knowing Bi and Lei is byusing an image of each surface taken by a camera, andretrieving the information from there. But if we workwithout an image for each surface, some heuristics mustbe used. The most common approach is to start froman initial estimate of the average reflectivity ρ∗ andestimating the surface reflectance as

ρ∗i = B∗

iB∗

A× ρ∗

where B∗i is the average intensity of the pixels recovered

from the camera observation and B∗A is the ambient

radiosity defined as

B∗A =

∑xy pxy

Nρ∗

where the sum is performed over the x×y = N pixels ofthe image with intensities pxy . The procedure is iterateduntil some satisfactory threshold is achieved.

• Monte Carlo formulations. To the best of ourknowledge, there is no research using this kind offormulation, but we strongly believe that this is afeasible and sensible approach: Simply fire rays fromthe camera or the surfaces with known properties andcontinue its path until hitting a surface with unknownBRDF. Then, using the gathered information wecould estimate the reflectance parameters. Also, abidirectional approach could be used, given knownlighting conditions. In any case, it is clear that BRDFrecovery is feasible, but certainly will have to deal withthe inherent variance problems Monte Carlo methodspresent, which can only get worse for an inverseproblem of this kind.

• Local Illumination formulations. These formulationsare based on the simplification of Equation (1) to takeinto account only local illumination, and using theobservation of this local illumination to obtain valuesfor the fr (BRDF) coefficients.


“sur200pag

✐

✐ ✐

✐

✐ ✐


Ramamoorthi and Hanrahan [51] have studied the inversereflectometry problem under a signal processing frameworkand arrived at the conclusion that BRDF recovery is feasible(well conditioned in the mathematical sense) when theknown lighting contains high frequencies like directionalsources, and is ill-conditioned for soft lighting.

4.1. General Global Illumination based IRP

Paul Debevec, in [9], introduces the concept of a light-basedmodel, a representation of a scene that consists of radianceinformation, possibly with specific reference to light leavingthe surfaces, but not necessarily containing BRDF informa-tion. He presents a method that uses the measured scene’sradiances and global illumination in order to add new ob-jects to light-based models with correct lighting. The light-based model is constructed from an approximate geometricmodel of the scene and by using a light probe to measurethe incident illumination at the location of the synthetic ob-jects. To do that he divides the scene into three main re-gions: the distant scene, represented with an environmentmap; the local (or near) scene which is going to photomet-rically interact with the synthetic objects and whose geom-etry must be well known; and synthetic objects. To estimatethe local scene BRDF, he assumes a reflectance model (e.g.diffuse, specular, ...) with approximate initial values, anditeratively computes the global illumination solution for thelocal scene with the current parameters with respect to theobserved lighting configurations. By comparing the appear-ance of the rendered local scene to the actual appearance, hedecides whether to continue iterating with adjusted parame-ters, or not. In the case of purely diffuse reflectors, the nextestimate of the reflectances is the ratio from the resultingradiance to the observed value. Any other case is left asfuture work, manually estimating the specular coefficientsfor the non-diffuse objects in his test scenes.

Boivin and Gagalowicz developed [3] a reflectance re-covery algorithm that starts with a pure Lambertian modeland successively tries more complex BRDF models until afit between the original image and its synthetic reproduc-tion is achieved. For the simpler models (diffuse, perfectand almost-perfect specular), an iterative correction is ap-plied based on the object image to synthetic image ratio ofthe previous iteration, while in more sophisticated models aSimplex method is used. If the whole hierarchy of modelsfails to provide a good fit, the method proceeds to a plaintexture extraction, using methods from any of the examplesin section 4.2. This method has the clear advantage over theprevious one of being able to work with non-diffuse BRDFswithout requiring a manual user intervention.

4.2. Radiosity-based IRP

Computer augmented reality also requires the computationof reflectances from images. All research works have a

preprocess stage were the scene geometry is approximatelyreconstructed with photogrammetric techniques. Thework from Fournier et al. [14], assumes reflectance isconstant across each patch and uses a heuristic methodthat assigns each patch a reflectance that is an averagereflectivity multiplied by the ratio between the elementradiosity (computed as the average of all the visible pixelsit contains) and an average radiosity. Average values arecomputed directly from the images, weighting the obtainedvalues with the respective areas of the patches. In [12],textures of arbitrary resolution are extracted by de-warping(extracting the textures from the images by reversing theperspective-introduced distortion) the original image andbringing it back to the plane of the previously built polygon.Unfortunately, their method sacrifices the usage of dynamiccameras and real scenes to gain speed, and the quality of theobtained images is slightly degradated due to the use of apolygon-projection method instead of ray tracing.

Yu et al. [62] have presented an inverse radiositymethod to account for mutual illumination in estimatingspatially varying diffuse and piecewise constant specularproperties within a room from a sparse set of photographs.Their technique is based on the usage of a low-parameterreflectance model (metals and plastics treated differently,see below) allowing the diffuse component to vary freelyover surfaces while assuming non-diffuse characteristicsremain constant across particular regions. As input themethod receives a geometric model of the scene and a setof calibrated, high dynamic range photographs [11] takenwith known direct illumination. The algorithm proceedsby hierarchically partitioning the scene into a polygonalmesh and, by using image-based rendering techniques, itcomputes an estimate of both the radiance and irradianceof each patch. Using the known geometry and light sourcepositions, it computes the estimate placement of the specularhighlights falling inside the radiance images, and runsan iterative optimization procedure to recover the diffuseand specular reflectance parameters of each region. Theseresults are used to update the hierarchical system. Then,the estimation-update procedure for the BRDF parametersis repeated. The iteration is performed several times toobtain the final solution of the BRDFs for all surfaces.Two different formulas are used for isotropic or anisotropicBRDFs (Ward’s model), and it is suggested that metals betreated differently from plastics: for plastics, they considerthe specular coefficient constant and the diffuse one variable,while for metals they do the opposite. The selection betweenboth models was performed through a simple test: a metallicsurface has its specular reflectance larger than the estimateddiffuse component.

In [32] Loscos et al. a different approach for reflectancerecovery is presented: a set of images from a fixed viewpointbut with controlled, varying illumination (no shadows) iscombined with confidence weight factors that represent the


“sur200pag

✐

✐ ✐

✐

✐ ✐


visibility with respect to the light source for each pixel inthe images. The resulting textures are de-warped as before.Later, this was extended [31] and an algorithm for interac-tive re-lighting was introduced, based on a preprocessingstep that reconstructs geometry and creates un-occluded il-lumination textures (thus taking into account the effect ofshadows). Unfortunately, only diffuse surfaces can be con-sidered. The creation of these un-occluded textures has twosteps: firstly they add the light that was blocked in a hier-archical radiosity solution, and then a heuristic correctionis applied. This correction is computed by finding an ap-propriate un-occluded reference patch which will give anindication of the desired color, and computing a modulationfactor consisting of the ratio of form-factors of each patch tothe light source. In [30] this method was improved by addinga low-cost photometric calibration method which improvesthe reflectance estimate of real scenes. This was achievedby adapting a high-dynamic range image creation to a low-cost camera, and an iterative approach to correct reflectanceestimation using a radiosity algorithm for indirect light cal-culation. Unlike previous work, it allows for a restricted setof BRDFs (purely diffuse) to be recovered, but works withthe simplest capture process since it does not need user-controlled specific lighting. Most important, this last pieceof research does not attempt to perform a reflectance estima-tion, since it uses a simple texture modulation for display.Also, when compared with the work by Yu et al. [62], wesee that the last one is far from interactive, due basically tothe generality of the algorithm used for the light propagation(despite the fact that their method is specifically tailored tothe radiosity setting), but has the advantage of handling anyviewpoint in the environment.

4.3. Local Illumination Based IRP

Although a BRDF is a function fr (r, ω, ωi ) where r is apoint on a surface and ω and ωi the outgoing and incomingdirections, for classification purposes it is convenient tosubdivide the different methods that attempt to recover fras texture-based, angular-based and general BRDF-based.Texture-based are those that only consider spatial variationsin r disregarding angular variations and assuming a constantangular behavior, generally purely Lambertian. Instead,Angular-based methods consider the BRDF as a function ofω and ωi only, disregarding spatial variations.

4.3.1. Texture-based BRDF Recovery

In 1991, Ikeuchi and Sato [26], using one intensity andone range map (z-buffer), and assuming constant materialregions over an object, were able to obtain estimates ofdiffuse and specular parameters for a specifically tailoredversion of the Torrance–Sparrow BRDF (by means of aniterative least-squares fitting algorithm). Later, Kay andCaelli [28] extended this approach without requiring BRDFto be constant over object regions, estimating the parameters

at each point on the object. They applied a photometricstereo method to a range map and a number of intensitymaps, and inverted the illumination model at each pointon the object. (For non-highlighted regions, they used alinear least-squares method while for highlight regions anonlinear separable least-squares method with regularizationwas used.) They were able to recover highly texturedsurfaces by using enough intensity maps to recover diffuseand specular parameters at each point. If enough intensitymaps were not available, they showed how to recover theBRDF when the specular component is assumed to varysmoothly over the material.

Poulin et al. [44] describe an interactive system toreconstruct 3D geometry and extract textures from a setof photographs. The authors describe a three step process:A least-squares problem is first solved for the cameraparameters, and then for the 3D geometry. Once a satisfying3D model is recovered, its color textures are extractedby sampling the re-projected texels in the correspondingimages. All the textures associated with a polygon are fittedto each other, and the corresponding colors are combinedaccording to a set of custom criteria in order to form a uniquetexture. For each texel, the size in pixels of its projectionin the images is used as an indication of the quality of theextracted color.

Ofek et al. [41] present a method which deals withthe problem of recovering and blending textures fromdifferent images, but also discusses the problem of removinghighlights and reflections. Multi-resolution textures arestored in a quad-tree data structure, which is filled by arecursive level-of-detail projection algorithm of the imageto the texture space, followed by a push–pull procedureto propagate information all along the tree. The algorithmthus calculates an approximation of the view-independentcolor for each texel, and calculates the average of theprojected texture area for every texture pixel that is near thementioned estimation. The main advantage of this algorithmis its ability to account for the different sampling rates thatresult from different views of the surface. Unfortunately,they do not attempt to model the surface reflectance, butuse texture maps to record the diffuse component of theradiance reflected under the lighting conditions at the timethe photographs were taken.

Dana et al. [13] applied reflectometry techniques to thedomain of textured objects by using a spectrophotometerto carefully measure spectral BRDFs without separatingdiffuse or specular components. Of course, this becomesimpractical for complex BRDFs due to the high storagecosts, and requires the lighting to be totally known for theimages.

Lensch et al. [34] presented an image-based measuringmethod that robustly detects the different materials ofobjects, organizes them in clusters of similar materials


“sur200pag

✐

✐ ✐

✐

✐ ✐


and fits an average BRDF to each of them (Levenberg–Marquardt method initialized by an average BRDF).Although their method is BRDF-independent, they useda Lafortune [33] model as target BRDF. In order tomodel local changes, they projected the measured data foreach surface point into a basis formed by the recoveredBRDFs, leading to a spatially varying representation.To do this they introduced the concept of a lumitexel, adata structure that stores all geometric and photometricinformation for a surface point. The general idea behindthe BRDF classification algorithm is to start with a clustercontaining all the samples and recursively subdivide it,classifying the lumitexels according to the error they havewith respect to each cluster-computed BRDF. This methodhas the advantage of not requiring a specific BRDF or ahomogeneous material, as most of the previously mentionedapproaches, but requires full knowledge of the objectgeometry, lights and camera.

4.3.2. Angular-based BRDF Recovery

Poulin and Forunier [45, 43] deal with the problem ofdetermining the characteristics of surface materials (someparameters of their BRDF) by using a painting paradigmwhere the user simply paints color points on a surface.The system attempts to find the best values for the surfacecharacteristics such that the points will retain their assignedcolor in the final rendering. Depending on the number ofconstraints (color points) given by the user, the problemcan either be a non-linear constrained optimization one(when there are less color points than variables to find out)or a weighted least-squares fitting problem with penaltyfunctions to constrain the values of surface parameters(otherwise). For the former, each color point is consideredas a volume in the 3D color space of acceptable colors,introducing two inequality constraints in each direction(and no value can be negative). For the non-linear, least-squares fitting problem, the authors use penalty functionsto introduce the constraints. Also, the system is modifiedto assign different weights depending on the location of thecolor points, e.g. at the dark side of an object the ambientterm dominates.

Another inverse reflectometry problem posed byMarschner [36, 39] was called image-based BRDFmeasurement, which presents a system that measuresreflectance quickly without special equipment. Themethod works by taking a series of photographs of acurved object, each image capturing light reflected bydifferently oriented parts of the surface. The photographsare analyzed to determine the BRDF by using a curvedtest sample with known shape, an imaging detector andautomated photogrammetry to measure the camera position,light source location and sample placement. At first, thegeometric calibration stage uses machine-readable targetswith embedded identification codes placed near the sample

to allow photogrammetric techniques [36] to be able tolocate those samples. The information derived in this stage isthe position of the light source, the camera location for eachmeasurement and the location of the sample. The next step,radiometric calibration, obtains the relationship betweenthe radiance reflected to the camera and the irradiancedue to the source. An important assumption done in thosemeasurements is the approximation of the source as a singlepoint, which is correct when the source is small compared tothe distance to the sample. To get the absolute magnitude ofthe BRDF correctly, they measured the intensity of the lightsource relative to the camera sensitivity by photographinga diffuse white reference sample in a known position. Thelast step, data processing, is performed by the de-renderer,which uses standard rendering techniques [39] to findthe intersection point of each pixel’s viewing ray withthe sample surface and to compute the radiance from thesource. To obtain the desired BRDF value, the de-rendererdivides the pixel’s measured radiance by the irradiance.The de-renderer’s output is a list of BRDF samples, eachincluding the incident direction, the exitant direction andthe value for that configuration.

Debevec et al. [10] recovered a two-parameter BRDFmodel for the human skin with color space analysistechniques from a set of photographs with varyingillumination. To do that, they assumed that the specularcomponent was the same color as the incident light,while the diffuse one was obtained in a two-step manner:first, by fitting a Lambertian lobe to obtain the surfacenormal, and then finding the parameters in their model byfitting them to the observed chromaticities in the originalun-separated reflectance function. Previously, Sato et al.[60] had presented a similar algorithm to retrieve the shapeand the BRDF of convex objects by using a turntable anda single point source. The main difference between thesealgorithms for BRDF recovery is that, in the second step inSato et al. , the diffuse lobe is fitted to the diffuse term in amodified Torrance–Sparrow model. Their method required120 color images and 12 range maps to compute the BRDFparameters. This work is an extension of a method [54]that recovered a simplified Torrance–Sparrow reflectionmodel for an isolated object from a sequence of rangeimages and a reconstructed 3D model, constraining thecamera parameters and light source position. (They used theLevenberg–Marquardt numerical minimization method.)This way, they were able to separate the diffuse andspecular components and recover the uniform reflectanceof the surface. Unlike Marschner et al. [36, 39], Sato etal. sacrificed the generality of measuring a full BRDFat each surface point and instead used a single-formulamodel of specular and diffuse reflectance to extrapolate theappearance at novel viewpoints.


“sur200pag

✐

✐ ✐

✐

✐ ✐


4.3.3. General BRDF Recovery

Yu and Malik [63] presented an approach to produce photo-realistic computer renderings of real outdoor architecturalscenes (building facades) under novel lighting conditions.Their system uses a small set of photographs as input,along with a geometric model of the scene generated withphotogrammetric techniques. The input photographs aretaken with a hand-held CCD camera and converted intoradiance images [11]. They defined two pseudo-BRDFs,one corresponding to the spectral distribution of the sun(modeled as a parallel light source) and one correspondingto the integrated light from sky (they fit a sky modelto a set of calibrated photographs) and the environment(modeled through a low-resolution spherical environmentmap). The incident radiance is obtained from the sun, skyand environment, while the outgoing diffuse radiance istaken from the photographs in directions away from specularreflection. Each face of a building must appear in at least twophotographs, one with direct illumination from the sun andthe other without it. Each polygon in the original geometricmodel is first triangulated and a dense grid is set up oneach triangle in order to capture the spatial variations inthe pseudo-BRDF. The specular lobes are recovered withan empirical model [33]. The sky and environment aredivided into small pieces and the vector flux is pluggedfrom each piece into the specular model, thus resulting in aleast-square minimization problem. The authors assume thateach visibility-blocking surface in the model has the samespecular lobe (except for windows which are left for furtherinvestigation).

Together with the re-lighting system [37] described in sec-tion 3, Marschner’s thesis presents two inverse reflectometryproblems, the first one being Photographic Texture Measure-ment [36]. Its purpose is to construct a representation ofthe spatially varying parameters of the BRDF in equation2, based on samples provided by a set of photographs ofthe object (each with known lighting and camera position).For that reason, a BRDF with a small number of parame-ters is chosen. In particular, for many of the examples, theychoose a pure lambertian BRDF. The algorithm first gath-ers all the observations of radiance reflected from a partic-ular surface point by sampling all the user-provided pho-tographs in which the surface point is visible and illumi-nated. From these measurements, and using the known ge-ometry of the surface, the incident and exitant directionsand the reflectance value are computed for each observation.These samples are used at that point to estimate parametersof the BRDF model. To obtain an estimate of the Lambertiancomponent of surface reflection, for example, it requires aleast-squares fitting of the values, with weights that dependon the incident and exitant directions, i.e. values with viewor illumination directions near the surface normal are morereliable than samples that are near-grazing, and samples thatinclude significant contribution from specular reflection are

less reliable than those that do not. The specular part of theBRDF is handled by combining measurements from differ-ent points, based on the supposition that some parametersof the BRDF are spatially constant while others may vary.Sample points are chosen at the vertices of an optimizedtriangulation of the object’s surface, performing a linear in-terpolation for the points in between.

Ramamoorthi and Hanrahan [51] estimate low-parameterBRDFs using a three-component model of the reflectedlight field: a diffuse component, specularities from theslowly varying lighting and specular highlights from thefast-varying lighting component (obtained from a sphericalharmonics decomposition). It is shown that two loopsestimate the parameters of a simplified Torrance–SparrowBRDF: the outer one, through a simplex algorithm adjuststhe non-linear parameters, while the inner loop performsa linear optimization of the diffuse and specular weightsof the formula. For spatially varying BRDFs, a loop overthe different surface points is added and the mentionedalgorithm is repeated for each location. The main differencewith previous work, and especially the ones able to workwith outdoor scenes, is that it does not assume a simpleparametric model for skylight like Yu and Malik [63],nor does it requires highly controlled lighting conditions(generally by careful active positioning of a singlesource) like Marschner et al. [37]. Also, Ramamoorthiand Hanrahan were the first to solve the IRP for generalillumination (irradiance), without requiring simple BRDFsor low-resolution textures, as in the previously mentionedwork.

4.4. Conclusions on IRP

The first important thing to notice is that, to the best of ourknowledge, there are no efforts to solve these problems withMonte Carlo-based methods, although they could be wellsuited for this kind of problem.

In general, we can make a few observations:

• In the category of General Inverse Reflectometry Prob-lems, we can observe that the method used by Boivinand Gagalowicz [3] presents the clear advantage overother image-based methods (like Debevec et al. [9], Yuet al. [62], Loscos et al. [32], Fournier et al. [14] andDrettakis et al. [12]) in that it uses the area covered bythe projection of an object in the real image or imagesto determine its reflectance, and thus avoids producinglarge errors for small objects, since this method usesa feedback through the comparison between real andsynthetic images (significatively reducing bias). Also, itdoes not need a specular highlight for each surface toappear in at least one image, as Yu et al. [62] do. Onthe other hand, observe that the method by Yu et al.[62] and the one by Boivin and Gagalowicz [3] work on


“sur200pag

✐

✐ ✐

✐

✐ ✐


full scenes, while Debevec [9] only estimated materialproperties on a part of the scene.

• With respect to radiosity-based approaches, it isimportant to note that in [12] textures of arbitraryresolution are extracted, but unfortunately, their methodsacrifices the usage of dynamic cameras and realscenes to gain speed, and the quality of the obtainedimages is slightly de-gradated due to the use of apolygon-projection method instead of ray tracing.Unlike other previous research, Loscos et al. [32] allowfor a restricted set of BRDFs (purely diffuse) to berecovered, but work with the simplest capture processsince user-controlled specific lighting is not needed.Most important, this last work does not attempt toperform a reflectance estimation, since it uses a simpletexture modulation for display. Also, in comparison, thework by Yu et al. [62] is far from interactive, basicallybecause of the generality of the algorithm used for thelight propagation (RADIANCE) despite the fact that themethod is specifically tailored to the radiosity setting.However, it has instead the clear advantage of handlingany viewpoint in the environment.

• In the Local-based approaches subsection, we observethat the method by Poulin et al. [44] and the oneby Poulin and Fournier [42] are the only interactivemethods that allow some sort of texture recovery.Instead, Ofek et al. [41] do not attempt to do real-timeprocessing, but to compute high quality multi-resolutiontextures from image sequences.The work by Dana et al. [13] has the problem ofbecoming impractical for complex BRDFs due to itshigh storage costs, and requires the lighting to betotally known for the images. On the other hand, themethod by Lensch et al. [34] has the advantage of notrequiring a particular BRDF or a homogeneous material,like most of the previously mentioned approaches, butrequires full knowledge of the object geometry, lightsand camera.The main difference between the works by Debevec etal. [10] and Sato et al. [60] for BRDF recovery is that, inthe second step in the Sato et al. [60] work, the diffusepart is fitted to the diffuse term in a modified Torrance–Sparrow model. Unlike Marschner et al. [36, 39], Satoet al. sacrificed the generality of measuring a full BRDFat each surface point and used a model of specularand diffuse reflectance to extrapolate the appearance ofnovel viewpoints.Finally, the main advantage over previous methods ofthe work by Ramamoorthi and Hanrahan [51] is that itdoes not assume a simple parametric model for skylight,nor does it require highly controlled lighting conditions.Also, Ramamoorthi and Hanrahan were the first to solvethe IRP for general illumination (irradiance), withoutrequiring simple BRDFs or low resolution textures, asin the previously mentioned work. Thus, we see that

Table 4: Inverse reflectometry papers that use images from differentpoints of view (PoV) as input.

Several PoV Single PoV

Local [54], [60], [41], [28], [26], [43], [45], [36],[63], [13], [10], [34] [39], [44], [51]

General [9] [3]

Radiosity [62], [32], [31] [12], [14]

they presented a general solution based on a convolutionprinciple that enables a reasonably easy method forBRDF, texture and/or lighting recovery.

Also, we can make some further comparisons on methodson different sections by noticing that:

• We can see that [9] [3] [62] [14] [12] [3] [32] [30] [31]are papers that use the full global illumination equations,while the rest of the reviewed papers in this section useonly the simplified local versions.

• We also see that [44, 10, 60] and the works on computeraugmented reality present an approach that does not relyon any optimization procedure, taking the measurementsdirectly from the user’s input.

• The same happens with [36, 39], which takethe information directly from the user-providedphotographs, without needing an intermediateoptimization process to get the results.

• Instead, [62] uses a hybrid approach, measuring thediffuse component of the BRDF and resorting to anotherapproach for the other parameters in the BRDF used.

• One of the most important things to notice is that mostof the papers are based on small-parameter BRDFs, ingeneral of two main types:

– The scene can be decomposed in regions witharbitrary variation of the diffuse reflectance.

– The scene must be decomposed in regions withoutspatial variation of the reflectance, but with thepossibility of using a BRDF function with higherdimensionality (more complex glossi behaviour).

• The papers that solve this sort of problems and useimages as input can be further classified [3] accordingto whether they used multiple points of view or a singlepoint of view for the images in their computations. Thisclassification can be found in Table 4.


“sur200pag

✐

✐ ✐

✐

✐ ✐


5. Combined Inverse Lighting and InverseReflectometry Problems (CILRP)

Combined problems are those where K and Le are unknownin equation 2, but G and part of L are known. The methodsstudied in this section also can be further classified accordingto the illumination approach used: As in Sections 3 and 4,this can be either local-based, general global illumination,Monte Carlo-based or radiosity-based.

• General formulations. These methods try to findreflectance properties and lighting conditions withoutrelying on any particular illumination algorithm. Thereare no works using this sort of approach, probablydue to the high complexity of the optimization of bothproblems at the same time.

• Radiosity-based formulations. Here the general prob-lem is reduced to a radiosity setting by making the sameapproximations to Equation (1) as the ones describedin Section 3: a purely diffuse BRDF for the surfaces(patches), that is constant all over each one.

• Monte Carlo formulations. Once again and to thebest of our knowledge, there are no works using thiskind of formulation, but the solution would be tosimply fire rays from the camera or the surfaces withknown properties and continue its path until hitting asurface with unknown BRDF or emittance, and usingthe gathered information to estimate the parameters.Also, a bidirectional approach could be thought of,too, where rays are fired as before and from the lightsources with unknown emittance, and using the gatheredinformation to recover the missing information. In anycase, it is clear that simultaneous recovery of bothBRDF and lighting conditions is feasible, but certainlythe inherent variance problems presented by the MonteCarlo methods will have to be dealt with.

• Local Illumination formulations. These formulationsare based on a local illumination formulation to obtainvalues for the lighting parameters and the BRDF coeffi-cients.

In the work by Ramamoorthi and Hanrahan [51], thecombined problem was studied: inverse lighting and inversereflectometry problem in a signal processing framework, andit was concluded that, up to a global scale, the reflectedlight field can be separated into the lighting and the BRDF,provided that the appropriate coefficients of the reflectedlight field, in a spherical harmonics representation, do notvanish. It is important to notice that this factorization can bedone up to a global scaling factor.

5.1. Radiosity-based CILRP

John Kawai, James Painter and Michael Cohen [29] usethe radiosity to minimize the global energy of the scene,

instead of the mean-squared difference between the desiredradiosity values and current values at the patches as before.This global energy is given by the area-weighted sum ofthe element radiosities, plus a user-defined weighted sumof physical terms and human perception based terms. Thephysical terms include:

• radiosities;

• emissions;

• directionality and distribution of the light sources;

• patch reflectances;

while the human perception based terms are a quantificationof the subjective impression of clearness, pleasantness orprivacy based on the scene’s brightness. Constraints areimposed by the user to the objective function as explicitweighted penalty terms. The resulting unconstrainedproblem is solved by the Broyden–Fletcher–Goldfarb–Shanno (BFGS) method. In order to speed up computations,a hierarchical radiosity (HR) solution was used [24] tocompute the initial baseline rendering and they reusethe computed links to propagate the increments usedby a finite difference scheme to find an approximationto the derivatives of the radiosities with respect to theunknowns. This is possible since these derivatives obey thesame equation as the ordinary radiosity problem. In thementioned user-defined weighted sum of physical terms,they were able to include a variable element reflectivityterm (constant over the patch surface), thus allowing thecombined optimization of both emissivities and reflectivitiesin the same process. The partial derivative of the radiositieswith respect to the reflectivities was computed by “shooting”the un-shot radiosity �ρ due to the change in reflectivity,Bi �ρ.

5.2. Local Illumination-based CILRP

Sato et al. [59] use the radiance information inside shadowsto derive the illumination distribution of a real scene whenthe BRDF is Lambertian, recovering the diffuse coefficientup to a scaling factor, which is obtained from the cameracalibration (see Section 4). This work was later improved[58] by using a non-uniform, adaptive discretization ofthe directions of illumination. Also, the need to know thereflectance properties of the shaded surface was not longerrequired, but the limitations of using only distant lighting, nointereflections, uniform reflectance and known camera andobject shape remain unchanged. The algorithm proceeds bytwo nested loops, the outer one estimating radiance valuesof imaginary directional light sources (a linearized set ofequations of the influences of the light sources vs. the pixelsof the shadow surface) and the inner one estimating thereflectance parameters of the surface in shadows (Powell’smethod on the RMS difference of the actual pixel value andthe estimated, local-illumination only value).


“sur200pag

✐

✐ ✐

✐

✐ ✐


Ramamoorthi and Hanrahan [51] presented a remarkablework from the perspective of signal processing, requiringa single manually specified directional source to recoverthe roughness of the surface. Absolute reflectance cannotbe found from this method, so an ad-hoc relationshipbetween the diffuse and specular weights is established. Thealgorithm consists of two nested loops, the outer one beingan inverse BRDF-problem (see above, section 4) while theinner one is an estimation of the lighting with known BRDFparameters (see section 3). This work can be consideredan extension of the previous one [58], since it does notrequire shadow information and presents improved methodsfor estimating the illumination. It also addresses a moregeneral setting, being able to work with spatially varyingmaterials.

5.3. Conclusions on CILRP

One of the most important things to notice in this sectionis that there are only a few approaches that deal with bothproblems, illumination and BRDF recovery at the same time.In particular, radiosity-based solutions only include the workby John Kawai, James Painter and Michael Cohen [29], butit is important to notice that their work presents a fairlyefficient solution given the diffuse-surface and constant-patch-radiosity approximations involved.

On the other hand, local illumination-based CILRP isincluded in two works, with the one by Ramamoorthi andHanrahan [51] being able to be considered as an extension ofSato et al. [59, 58] because it does not require a classificationof the scene into shadow regions and it works in a moregeneral setting, and is even able to deal with spatiallyvarying materials.

Finally, it is important to note that no General CILRPand no Monte Carlo-based CILRP were presented, mostlybecause of the inherent high complexity involved, of specialimportance in the former. However, we think that MonteCarlo-based solutions are a sensible way of trying to solvethese combined problems, despite the high variance inherentin the application of those methods to inverse problems.

6. Conclusions and Open Problems

Here we will summarize the conclusions of the surveyedwork, to show common problems, characterize approachesto solutions and suggest open issues. We can organize thestudied methods depending on the type of illuminationmodel (general, radiosity, Monte Carlo and local) they useand according to the sort of problem they solve (Tables 2and 3).

In Tables 5 and 6 we summarize the papers on Inverseemittance and inverse reflectance respectively. We then showthe main features of each method: type of approach used

to solve it (direct or indirect) and numerical methods toimplement it. These mathematical aspects are summarizedin the Appendix.

As we can see in the tables, there are three kinds ofapproaches used to solve the different problems faced:

• Indirect-solving approaches, or optimization-based ap-proaches, where the solution is obtained by finding theminimum (or maximum) of an adequately defined objec-tive function. These methods generally require solvingthe direct problem at least once per iteration [52], [29],[37, 36], [43], [36], [62].

• Direct-solving approaches, where the goal is to findmethods to invert the equation without solving the directproblem at any time. Among those, we can mentionthe matrix-based approaches, where the goal is to inverta highly ill-conditioned algebraic system of equationsresulting from a finite element approximation of theunderlying equations [23, 15, 38]. Another approach[40] implements an Inverse Monte Carlo method to findthe emissivities for a set of surfaces.

• Mixed, that use a combination of the two above-mentioned approaches. The only examples reviewedwith this kind of approach are [54], [60] and [63].

Analyzing the inverse emittance problems, with the aidof Table 5, we see that most of the papers that work onglobal illumination formulations deal with purely lambertianBRDFs, with only a few exceptions. ([37], [36] and [51],that deal with combinations of photographs and presentalgorithms that are independent of the BRDF used). Onlytwo works pay any attention to the problem of ParticipatingMedia, [23] and [38].

With respect to the inverse reflectometry problems, wesee in Table 3 that [9] and [3] are papers that present anillumination-independent formulation, while the rest of thereviewed papers in this section use only radiosity or thesimplified local version. We also see that [44] presents adirect approach, and together with [36], [39], [14], [12], [32],[41] and [13] are the purely direct approaches among thosesurveyed. On the other hand, [62], [54] and [60] use hybridapproaches, using a direct measurement for one of the BRDFcomponents (generally the diffuse component of the BRDF)and resorting to an indirect approach for the other parametersin the BRDF used. This assumes that the BRDF is separableinto different parts. The most important thing to notice is thatall papers are based on BRDFs of two main types:

• The scene can be decomposed into regions with arbi-trary variation of the diffuse reflectance (directional re-flectance properties remain constant on each area).

• The scene must be decomposed into regions withoutspatial variation of the reflectance while using a BRDFfunction with higher dimensionality.


“sur200pag

✐

✐ ✐

✐

✐ ✐


In general, we see that indirect methods use some sort ofleast squares for the optimization process, with the exceptionbeing [62] that uses the Nelder-Mead with Simulated An-nealing for this part. From the reviewed results, it seems thatthe Levenberg–Marquardt numerical minimization methodis one of the best suited for non-linear BRDF fitting.


“sur200pag

✐

✐ ✐

✐

✐ ✐


Tabl

e5:

Cla

ssifi

cati

onof

inve

rse

rend

erin

gm

etho

ds:

inve

rse

ligh

ting

.(N

LC

O=

non-

line

arco

nstr

aine

dop

tim

izat

ion.

MT

SVD

=m

odifi

edtr

unca

ted

sing

ular

valu

ede

com

posi

tion

.L

S=

leas

t-sq

uare

s.C

LS

=co

nstr

aine

dL

S).

Wor

kTy

peof

appr

oach

App

roxi

mat

ions

Met

hod

used

[52]

Indi

rect

Lam

bert

ian

surf

aces

.Fin

alim

pl:l

ocal

illum

.C

LS

[40]

Dir

ect

Lam

bert

ian

surf

aces

.Uni

form

spat

iala

ndan

gula

rsu

bdiv

.In

vers

eM

onte

Car

lo

[23]

Dir

ect

Lam

bert

ian

surf

aces

+R

adio

sity

cons

tant

onpa

tche

s.T

runc

atio

nof

sing

ular

valu

es.

MT

SVD

[15]

Dir

ect+

Fini

teV

olum

em

etho

dfo

rm

edia

+R

adio

sity

cons

tant

onpa

tche

s+

lam

bert

ian

surf

aces

and

isot

ropi

cm

edia

.

Dis

cret

izat

ion.

Tru

ncat

ion

ofsi

ngul

arva

lues

.T

SVD

[38]

Dir

ect+

Fini

teV

olum

em

etho

dfo

rm

edia

Fini

teE

lem

ents

.P1

Sphe

rica

lHar

mon

ics

App

roxi

mat

ion.

Rad

iosi

tyco

nsta

nton

patc

hes

+la

mbe

rtia

nsu

rfac

esan

dis

otro

pic

med

ia.

MT

SVD

[37]

,[36

]In

dire

ct(p

reco

mpu

ted

rend

erin

gsfo

rea

chlig

htso

urce

)D

ista

ntlig

htso

urce

son

ly.L

inea

rre

latio

nshi

pbe

twee

nlig

htso

urce

san

dth

em

appi

ngto

imag

eva

lues

.L

inea

r1s

tord

erre

gula

rize

dL

east

Squa

res

met

hod

with

Gen

eral

ized

SVD

for

acce

lera

tion

[12]

,[32

],[1

4]In

dire

ctD

iffu

seSu

rfac

esw

ithco

nsta

ntra

dios

ity.K

now

nE

mitt

ing

Patc

hes

with

know

nge

omet

ry.

Lea

stSq

uare

s.[1

2]an

d[3

2]bu

ilda

hier

arch

ical

radi

osity

syst

emto

spee

dup

com

puta

tions

.

[42]

Indi

rect

Blin

n’s

illum

inat

ion

mod

el.

Ana

lytic

alO

ptim

izat

ion

[47]

Indi

rect

none

NL

CO

[7]

Indi

rect

none

SA(w

ithpo

ssib

lefin

allo

cals

earc

hst

ep)

[58]

,[59

]D

irec

tD

ista

ntso

urce

s,N

oin

terr

eflec

tions

,uni

form

refle

ctan

ce,

cam

era

&kn

own

geom

CL

S

[57]

Dir

ect

Com

mon

feat

ures

inst

ereo

pair

,no

inte

rrefl

ectio

ns,

unif

orm

refle

ctan

ce,c

amer

a&

know

nge

omC

LS

[55]

Indi

rect

Non

e(a

lthou

ghim

plem

ente

das

loca

lillu

m)

Stee

pest

desc

ent

[35]

Indi

rect

Non

eSe

lect

ions

disp

ersi

onan

dm

anua

luse

rse

lect

ion

[18]

Indi

rect

Kno

wn

geom

etry

and

diff

use

refle

ctan

ces

Lev

enbe

rg–M

arqu

ardt

[51]

1stI

ndir

ect,

2nd

Dir

ect

Loc

alill

um.S

impl

ified

BR

DF

(dif

f+

spec

ular

)1)

Lin

ear

LS

with

cuto

ffre

gula

riza

tion.

2)Su

bstr

actd

iff

com

pof

BR

DF

and

reco

ver

angu

lar-

base

dill

umre

sulti

ngfr

omsp

ecul

arB

RD

F.


“sur200pag

✐

✐ ✐

✐

✐ ✐


Tabl

e6:

Cla

ssifi

cati

onof

inve

rse

rend

erin

gm

etho

ds:

inve

rse

tefle

ctom

etry

.NL

CO

=no

n-li

near

cons

trai

ned

opti

miz

atio

n.L

S=

leas

t-sq

uare

s.SA

=si

mul

ates

anne

alin

g.N

LC

LS

=no

n-li

near

cons

trai

ned

leas

t-sq

uare

s.N

LL

S=

non-

line

arle

ast-

squa

res.

Wor

kTy

peof

appr

oach

App

roxi

mat

ions

Met

hod

used

[43]

Indi

rect

none

NL

CO

orN

LC

LS

[36]

Indi

rect

BR

DF

w/f

ewsp

atia

llyva

riat

ing

para

ms.

Dis

cret

eSa

mpl

.L

SB

RD

Ffit

ting

(fro

mph

otog

raph

s).

[36]

,[39

]D

irec

tf r

(r,ω

,ω

i)≡

f r(ω

,ω

i).P

oint

light

sour

ces.

Mea

sure

men

tfor

mph

otog

raph

s.

[62]

Mix

edL

owpa

ram

BR

DF

&Sc

ene

deco

mp.

inre

gs.w

/con

st.d

irec

tiona

lrefl

.pr

ops.

Iter

ativ

eB

RD

Ffit

ting:

NL

LS

step

s.

[9]

Indi

rect

Scen

esu

bdiv

ided

infa

r&

loca

lreg

ions

.It

erat

ive

BR

DF

fittin

g.

[44]

Dir

ect

Pure

lyL

ambe

rtia

nsu

rfac

es.

Col

or:s

ampl

ing

the

repr

oj.t

exel

sin

the

corr

esp.

imgs

.

[63]

Indi

rect

Pseu

do-B

RD

Fs.3

sour

ces:

Sun

(par

alle

l),S

ky(fi

tted)

and

Lan

dsca

pe(E

nv.M

ap).

Spec

lobe

from

sky

&la

ndsc

ape

cons

ton

larg

ere

gsD

iff

com

p:di

rect

mea

sure

.Sky

&la

ndsc

ape

spec

lobe

:LS.

Sun

spec

lobe

:Nel

der-

Mea

dw

/SA

.

[14]

,[12

]D

irec

tD

iffu

seSu

rfac

es.D

oes

notd

istin

guis

hbe

twee

nco

lour

and

shad

ows.

Refl

ecta

nce

prop

ortio

nalt

oav

erag

era

dios

ity

[32]

Dir

ect

Ass

umes

know

nex

acte

mitt

ance

from

surf

aces

As

[12]

,but

refl.

isav

rgov

erim

gs.c

onfid

ence

wei

ghts

.

[31]

Indi

rect

Dif

fuse

surf

As

[12]

+bl

ocke

dlig

htfr

omra

d.so

l.+

heur

istic

corr

.bas

edon

illum

.on

ref.

patc

h.

[41]

Dir

ect

No

high

light

sin

mos

tim

ages

Mul

tires

olut

ion

text

ures

byre

curs

ive

repr

ojec

tion

[10]

,[60

]In

dire

ctA

ssum

essp

ecco

lor

=lig

htco

lor.

No

chan

ges

inin

d.ill

um.

Sepa

ratio

nof

diff

&sp

ecpa

rts.

Dir

ectr

ecov

ofB

RD

Fpa

ram

s

[51]

Indi

rect

Firs

tTer

ms

inSp

hH

arm

seri

esSi

mpl

exm

etho

dfo

rno

nlin

ear

para

ms

&lin

ear

fittin

gfo

rre

st

[3]

Indi

rect

Hye

rarc

hica

llyte

sts

sim

ple

toco

mpl

exB

RD

Fm

odel

sSi

mpl

eM

odel

s:It

erat

ions

with

imag

e-ba

sed

erro

r.C

ompl

ex:S

impl

exM

etho

d.

[26]

Indi

rect

Con

stan

tBR

DF

over

regi

ons

(res

tric

ted

Torr

ance

–Spa

rrow

BR

DF)

Iter

ativ

eL

Sfit

ting

[28]

Indi

rect

Req

uire

sSe

vera

lPho

togr

aphs

BR

DF

sam

ple

inve

rsio

n(n

ohig

hlig

ht:L

S;hi

ghlig

ht:s

epar

able

LS

with

Reg

)

[54]

Mix

edSe

para

ble

diff

use

&sp

ecul

arpa

rts

Sepa

ratio

nof

diff

&sp

ecan

dfit

ting

ofTo

rran

ce–S

parr

owby

Lev

enbe

rg–M

arqu

ardt

[60]

Mix

edL

ocal

illum

.Con

vex

Obj

ects

.Int

erpo

late

dB

RD

Fpa

ram

s1s

tmea

sure

d,2n

dfit

ted

Torr

ance

–Spa

rrow

[13]

Dir

ect

Tota

llykn

own

light

ing

Mea

sure

men

tsby

Spec

trop

hoto

met

er

[34]

Indi

rect

Full

know

ledg

eof

Lig

hts,

cam

era

&ob

ject

geom

1stB

RD

Fsa

mpl

ecl

assi

ficat

ion,

2nd

Lev

enbe

rg–M

arqu

ardt

met

hod

for

each

sam

ples

clus

ter


“sur200pag

✐

✐ ✐

✐

✐ ✐


Tabl

e7:

Sum

mar

yof

the

mat

hem

atic

alas

pect

sof

the

inve

rse

ligh

ting

appr

oach

espr

esen

ted:

inve

rse

ligh

ting

.SV

D=

sing

ular

valu

ede

com

posi

tion

.M

TSV

D=

mod

ified

trun

cate

dSV

D.

MC

=M

onte

Car

lo.P

DE

=pa

rtia

ldiff

eren

tial

equa

tion

.

Con

stra

ints

impo

sed

Initi

algu

ess

/Ini

tiala

ppro

x.O

bjec

tive

func

tion

(?)

/dir

ecti

nver

sion

met

hod

Test

sPe

rfor

med

[52]

(CL

S)A

llw

eigh

ts≥

0.A

nyva

lue

Min

imiz

e:||∑ i

ωiφ

i−

gesp||w

here

φi

isth

eef

fect

ofth

ei-

thlig

htso

urce

.C

ompl

exen

v.w

ith19

000

poly

s,27

000

vert

.and

12(1

4)lig

htso

urce

s.

[40]

Non

eIn

itial

emitt

ance

dist

ribu

tion

prop

osed

for

unkn

owns

.In

vers

eM

Cm

etho

d(r

equi

res

high

num

ber

ofsa

mpl

es)

Rec

tang

ular

2Den

v.w

ithon

esu

rfw

ithflu

x&

heat

unkn

own.

[23]

#su

rfac

esw

/flux

and

emis

ivity

know

n=

#su

rfac

esw

/bot

hun

know

n.U

sein

itial

SVD

solu

tion

togu

ide

algo

rith

m.

Inve

rtal

gebr

aic

syst

emus

ing

MT

SVD

.R

ecta

ngul

ar2D

env.

with

one

surf

with

flux

&he

atun

know

n.

[15]

#of

vol.

elem

.with

both

emis

iviti

esan

dso

urce

stre

ngth

unkn

own

=#

ofsu

rf.e

lem

.w

ithkn

own

flux

and

emis

ivity

Use

initi

alSV

Dso

lutio

nto

guid

eal

gori

thm

.M

TSV

Dfo

rre

sulti

ngal

gebr

aic

syst

em(b

oth

med

iaan

dsu

rfac

esdi

scre

tized

).R

ecta

ngul

ar2D

env.

fille

dw

ithis

otro

pic

hom

.m

edia

,with

one

surf

with

flux

&he

atun

know

n.

[38]

Non

ePD

Eof

med

iaw

ith“c

old

wal

l”bo

unda

ryco

nditi

ons.

Use

SVD

solu

tion

togu

ide

MT

SVD

algo

rith

m.

Solv

eV

olum

ePD

Eby

Fini

teV

olum

em

etho

d.In

vert

resu

lting

alge

brai

csy

stem

for

surf

aces

only

usin

gM

TSV

D.

Rec

tang

ular

2Den

v.fil

led

with

isot

ropi

cho

m.

med

ia,w

ithon

esu

rfw

ithflu

x&

heat

unkn

own.

[37,

36]

Posi

tive

coef

ficie

nts.

Any

valu

eL

inea

rco

mbi

natio

nof

basi

sim

ages

tom

atch

agi

ven

phot

ogra

ph.

Rel

ight

ing:

Phot

os+

3Dm

odel

tom

odif

ya

give

nph

oto.

[14,

12],

[31]

�em

ittan

ces

fits

estim

tota

lint

ensi

tyR

adio

sitie

sfr

omim

ages

Fitti

ngof

the

sour

ces

pow

erA

quir

edre

alvi

deo

scen

es.

[42]

Non

eA

nyva

lue

(N·H

)n,f

ordi

rL

and

glos

sine

ssn.

Use

shad

owvo

lum

esfo

rfin

alpo

sitio

n.Si

mpl

epr

imiti

ves.

e.g.

:con

eun

der

tria

ngul

arlig

ht.

[47]

(NL

CO

)L

ight

sfr

om·sh

adsk

etch

·inco

nede

fined

byea

chpo

int

Abo

veel

lipso

idce

nter

edon

occl

uder

and

radi

usr

alon

gdi

rect

ion

from

cent

erof

mas

sof

the

sket

ched

poin

tsto

ellip

soid

cent

er.

Max

imiz

e:∑ (

p l−

p i)2

=di

stan

cefr

omev

ery

sket

ched

poin

tsp i

tolig

htpo

sitio

np l

.Se

vera

lpri

miti

ves:

0.00

3se

c/Sk

etch

edpo

int-

inte

ract

ive

spee

d.·sa

me

side

ofno

rmal

atpo

int

·right

side

ofha

lf-c

one

WR

Tco

neax

is

·umbr

a/pe

num

bra

·Um

bra:

∀Lig

htV

erte

x(L

V),

Sket

ched

Poin

t(S

P),L

V∈c

one

SPJu

stab

ove

anel

lipso

idE

cent

ered

onoc

clud

eral

ong

dir.

from

mas

sce

nter

ofth

esk

etch

edpo

ints

toE

cent

er

Max

imiz

e:∑ (

p l−

p i)2

=di

stan

cefr

omev

ery

sket

ched

poin

tsSe

vera

lpri

miti

ves:

0.01

sec/

Sket

ched

poin

twhe

nso

urce

isla

rge

WR

Tco

nes.

·Penu

mbr

a:∀L

V,∃

SP,L

V∈c

one

SP.

·hig

hlig

hts

·with

inco

nede

fined

byea

chsk

etch

edpo

int.

Any

valu

eM

inim

ize:

Val

ueof

Poin

tLig

htw

ithin

the

impl

icit

cone

equa

tion.

0.01

5se

c/sk

etch

edpo

int.

·sam

esi

deof

norm

alat

sket

ched

poin

t.·rig

htsi

deof

half

-con

eW

RT

cone

axis

.

[7]

used

-defi

ned

(add

edas

pena

lties

inth

eob

ject

ive

func

tion)

Use

r-pr

ovid

edU

ser-

prov

ided

bya

scri

pt.

1)ro

om,l

umin

arie

&ta

ble

with

hom

ogen

eous

desi

red

illum

.2)

offic

e(1

0550

obj)

,goa

l:re

duce

glar

e,sh

adow

boun

dari

es&

redu

ndan

cy.

[58,

59]

Posi

tive

coef

s.L

ambe

rtia

nco

ef.e

stim

.for

shad

owsu

rfac

eL

inea

rSy

stem

ofE

qs:s

ourc

esvs

.pix

els

inph

otog

raph

ssi

mpl

epo

lyhe

dra

onta

blec

loth

.

[57]

Non

eA

nyV

alue

Find

brig

htfe

atur

esin

each

onm

i-im

age

and

corr

elat

e.M

apre

mai

ning

asim

age.

Indo

or&

outd

oor

real

scen

es

[55]

none

Lig

htdi

rs=

cent

roid

ofno

rmal

sof

view

edsu

rfac

es.I

nten

sitie

s=

aver

age

desi

red

tovi

ewed

inte

nsiti

esra

tio[5

3]

Perc

eptio

n-ba

sed

imag

equ

ality

met

ric

(ext

enti

nco

mun

icat

ion

scen

ein

fo)

isol

ated

com

plex

geom

teri

calo

bjec

ts

[35]

Non

eU

ser-

defin

edin

itial

light

ing

Man

ualu

ser

optim

izat

ion

Com

plex

scen

es

[51]

Non

eA

nyva

lue

1)L

Sfo

rSp

heri

calH

arm

onic

sco

effs

.2.1

)su

bstr

actd

iff

com

pfr

omre

flfie

ld2.

2)re

cove

rhi

gh-r

esan

gula

r-sp

ace

illum

Com

plex

Obj

ects

[18]

Sour

ces

insi

dere

gion

sIn

ters

ectio

nsof

mdi

rect

iona

llig

hts

calc

.fr

omdi

ffus

esu

rfac

esL

east

-squ

ares

erro

rbe

twee

nth

esu

rfac

era

dian

ces

and

the

mea

sure

dra

d.Si

mpl

eC

orne

llB

ox-l

ike

scen

es


“sur200pag

✐

✐ ✐

✐

✐ ✐


Tabl

e8:

Sum

mar

yof

the

mat

hem

atic

alas

pect

sof

the

inve

rse

ligh

ting

appr

oach

espr

esen

ted:

inve

rse

refle

ctom

etry

.SV

D=

sing

ular

valu

ede

com

posi

tion

.MT

SVD

=m

odifi

edtr

unca

ted

SVD

.M

C=

Mon

teC

arlo

.PD

E=

part

iald

iffer

enti

aleq

uati

on.

Wor

kC

onst

rain

tsim

pose

dIn

itial

gues

s/I

nitia

lapp

rox.

Obj

ectiv

efu

nctio

n(?

)/d

irec

tinv

ersi

onm

etho

dTe

sts

Perf

orm

ed

[43]

·with

n≤

poin

tsV

alue

s≥

0.a)

w/s

mal

lglo

ssin

ess

(n).

b)co

nvso

las

init

gues

sfo

rne

xtit.

Use

r-de

fined

term

sin

the

BR

DF.

Phon

g-B

linn

BR

DF:

1stA

mbi

ent,

2nd

Dif

fuse

,3rd

spec

ular

[43]

·with

n≥

3po

ints

Val

ues≥

0.W

eigh

tpoi

nts

with

mos

tim

port

antc

ontr

ibut

ion.

Any

valu

eD

ista

nce

betw

een

BR

DF

and

user

-pro

vide

dsa

mpl

es.

Phon

g-B

linn

BR

DF.

[36]

Non

eA

nyva

lue

Wei

ghte

dSq

uare

der

ror

betw

een

BR

DF

and

mea

sure

dsa

mpl

es.

Sint

hetic

sphe

rew

/tex

ture

sfo

rca

lib.

Squa

sh.R

ock.

[36,

39]

Non

eA

nyva

lue

Dir

ectm

easu

rem

ento

fω

,ωi,

obta

inin

gf r

(ω,ω

i)G

ray

cylf

orve

rif.

Squa

sh.H

uman

skin

.(3

0000

0sa

mpl

e/ch

anne

l)

[62]

Non

ero

ugh

estim

ate

ofB

RD

FD

iffe

rent

BR

DF

isot

ropi

can

dan

isot

ropi

cca

ses.

Plas

tics

diff

eren

tfro

mm

etal

s.Sy

nthe

ticte

st:r

oom

w/m

utua

lillu

m.R

eal

test

:roo

mw

/fur

nitu

re,m

etal

licba

llsan

dco

lour

edpo

ster

s.

[9]

Non

eIn

itial

gues

s(u

ser)

Dif

f.be

twee

nre

nder

edlo

cals

cene

and

phot

os.

Rea

lcom

plex

scen

esw

/sim

ple

synt

hetic

obje

cts.

[44]

Non

eA

nyva

lue

All

poly

ste

xtur

esfit

ted

toon

ean

othe

r,co

rres

p.co

lors

com

bine

dW

RT

seto

fcr

it.to

form

aun

ique

text

.32

0×

400

text

ures

for

book

from

real

phot

ogra

phs.

[63]

Non

eA

nyva

lue

Dif

f:m

easu

re,L

ands

cape

spec

:LS

diff

pseu

do-B

RD

F&

reco

rded

rad.

Sun

spec

:rob

ustp

aram

reco

very

[49]

.R

elig

htin

gof

exis

ting,

real

tow

er.

[14]

,[12

]N

one

Any

valu

eA

vrg

refle

ctpr

opor

tiona

lto

radi

osity

Aqu

ired

vide

oim

age

scen

e

[31]

Non

eA

nyV

alue

1st:

eval

.as

i[12

].2n

d:A

ddbl

ocke

dlig

htfr

omra

d.so

l.3r

d:H

eur

corr

ec:fi

ndun

occl

uded

ref.

patc

hfo

rco

lor

ref.

Aqu

ired

real

vide

oIm

age

scen

e

[41]

Non

eA

nyV

alue

Mul

tires

olut

ion

quad

tree

text

ures

reco

vere

dby

recu

rsiv

ere

proj

ectio

n.Pi

ctur

es&

sim

ple

text

ured

obje

cts

[10]

Non

eSp

ecco

lor

=so

urce

colo

r.D

iff

colo

r=

med

ian

RG

Bm

easu

rem

ents

.D

irec

tfor

spec

.Lam

bert

ian

fit.

Cha

nge

view

poin

t,ill

um&

mat

chin

gre

al-w

orld

illum

.

[60]

Non

eSp

ecco

lor

=so

urce

colo

r.D

iff

colo

r=

sepa

rate

d&

fitte

dto

Torr

ance

-Sp

arro

w.

Dir

ectf

orsp

ec.L

ambe

rtia

nfit

.C

ofee

cup.

[51]

Non

eA

nyva

lue

Firs

tTer

ms

inSp

hH

arm

seri

es:B

RD

F=

Dif

fuse

+sl

ow-v

aryi

ngte

rms

+fa

st-v

aryi

ngsp

ecul

arlo

be.

Rea

l-w

orld

mod

els.

[3]

Non

eD

iffu

seSu

rfac

esus

edfir

st.T

hen

mor

eco

mpl

exB

RD

Fs.

Rat

ioof

aver

age

ofob

ject

imag

era

dian

ces

toob

ject

rend

ered

imag

e.O

ffice

Scen

e

[26]

Con

stan

tBR

DF

over

regi

ons

Inte

nsity

pixe

lval

ues

with

norm

alin

cide

ntlig

htR

MS:

1)ob

serv

edbr

ight

ness

and

orie

ntat

ion

from

rang

efin

der,

2)L

og(s

pec

com

p)an

dlo

g(B

RD

Fsp

ecpa

rt)

Sim

ple

Obj

ects

[28]

Dif

fuse

coef

smoo

thon

rang

e-de

ffici

ent(

rd)

regi

ons,

spec

ular

over

non-

spec

&rd

,ro

ughn

ess

ever

ywhe

re

Any

Lin

ear

LS

erro

rin

BR

DF

valu

esTe

xtur

edsp

here

s

[54]

Non

eA

nyno

tspe

cifie

d(fi

tof

spec

ular

com

p)C

ompl

exob

ject

s

[13]

Non

eA

nyM

easu

rem

ento

fB

RD

Fsa

mpl

esw

ithdi

ff.c

omb.

ofvi

ewin

g&

light

ing

Dat

abas

eof

mat

eria

ls

[34]

Non

ePa

ram

sfr

omav

gB

RD

FR

MSE

b/sa

mpl

ed&

pred

icte

dva

lues

text

ure

reco

very

from

com

plex

obje

cts


“sur200pag

✐

✐ ✐

✐

✐ ✐


Of the reviewed papers that deal with inverse emittanceor combined problems, only [37, 36] and [29] present view-dependent goals, especially the first one because it deals withinput given by photographs, while the second can also beused with view-independent goals.

From our survey we may derive that several inverseproblems remain open:

• The application of Monte Carlo methods to inverserendering problems has remained a vastly untouchedtopic with only one exception, and is an option thatshould be thoroughly researched and exploited in thefuture.

• The problem of inverse emittance in the presence ofparticipating media has only been superficially touched.More research and development is needed.

• General, illumination-independent, inverse reflectome-try approaches were also barely touched, both in thesingle-image and in the multiple-image variants.

• Combined problems, especially those with non-localillumination models, remain vastly untouched beyondthe initial efforts analyzed in section 5. This is especiallytrue for non-lambertian environments, since the seminalwork by Kawai, Painter and Cohen [29] covers theradiosity problem in a quite satisfactory way. We canalso see that studying combined inverse reflectometryand lighting problems in the context of the full-radianceequation is a very promising and open line of research,since local illumination entails a rough approximation.

• Combining lighting reconstruction for different points[51] across a scene, and performing correlations of theobtained information (a generalization of the work bySato et al. [59]) seems an efficient and unexplored wayto obtain the complete lighting of the scene, whichcould be very efficiently stored in some sort of lightingtextures [21]. This would allow the absolute positioningof light sources in the scene to be obtained.

• Choosing the luminaries’ shapes among a small set ofpossible shapes is another option that is suggested bythe results of Costa et al. [6].

• Letting an algorithm automatically choose among sev-eral possibilities the more suited material for a reflectorto get a given illumination at a given position in thescene.

7. Acknowledgments

We would like to thank Pierre Poulin for his insights andJack Howell for his reprints and for proofreading an earlierversion of this paper, as well to the reviewers for theiruseful comments. This work was partially done under grantTIC2001-2392-C03-01 of CICYT and 2001SGR-0296 ofGeneralitat de Catalunya.

A. Appendix

In this appendix we analyze, for each surveyed paper themathematical aspects of its solution method. Tables 7 and 8explain, for inverse emittance and reflectometry respectively,the constraints imposed either on the solution method (foroptimization approaches) or on the system of equationsfor direct inversion based approaches. In the first case therestrictions are incorporated into the optimization algorithm,while in the second the constraints are used in the system’sbuilding process, generally in a manual way. The thirdcolumn presents the starting point used by the differentmethods when the used technique requires it. The fourthcolumn presents either the objective function to optimize inthe case of optimization approaches, or the method used tosolve the direct problem presented.

References

1. James Arvo. Analytic Methods for SimulatedLight Transport, Ph.D. thesis, 1995.

2. James Arvo. The Role of Functional Analysisin Global Illumination. In P. M. Hanrahanand W. Purgathofer (eds), Rendering Tech-niques ’95 (Proceedings of the Sixth Euro-graphics Workshop on Rendering), Springer-Verlag, New York, pp. 115–126. 1995.

3. Samuel Boivin and Andre Gagalowicz. Image-based rendering of diffuse, specular and glossysurfaces from a single image. In ComputerGraphics Proceedings, Annual Conference Se-ries (SIGGRAPH 2001), pp. 107–116. 2001.

4. James F. Blinn. Models of light reflectionfor computer synthesized pictures. In Com-puter Graphics Proceedings, Annual Confer-ence Series (ACM SIGGRAPH ’77 Proceed-ings), pp. 192–198. 1977.

5. P. Bui-T. Illumination for computer gener-ated pictures. Communications of the ACM,18(6):311–317, 1975.

6. Antonio Cardoso Costa, A. Sousa and F. Fer-reira. Design de iluminacao. In 8th Portugueseworkshop on Computer Graphics. Coimbra,Portugal, 1998 in Portuguese

7. A. Costa, A. Sousa and F. Ferreira. Lightingdesign: A goal based approach using optimiza-tion. In D. Lichinski and G. Ward Larson (eds),Rendering Techniques ’99 (Proceedings of the10th Eurographics Workshop on Rendering),Springer-Verlag, New York, NY, pp. 317–328.1999.


“sur200pag

✐

✐ ✐

✐

✐ ✐


8. Michael F. Cohen and John R. Wallace. Radios-ity and Realistic Image Synthesis. AcademicPress Professional, Boston, MA, 1993.

9. Paul Debevec. Rendering synthetic objects intoreal scenes: Bridging traditional and image-based graphics with global illumination andhigh dynamic range photography. In Com-puter Graphics Proceedings, Annual Confer-ence Series (ACM SIGGRAPH ’98 Proceed-ings), pp. 189–198. 1998.

10. P. Debevec, T. Hawkins, C. Tchou, H-P Duiker,W. Sarokin and M. Sagar. Acquiring the re-flectance field of a human face. In ComputerGraphics Proceedings, Annual Conference Se-ries (SIGGRAPH 2000), pp. 145–156. 2000.

11. Paul E. Debevec and Jitendra Malik. Recover-ing high dynamic range radiance maps fromphotographs. In Computer Graphics Proceed-ings, Annual Conference Series (ACM SIG-GRAPH ’97 Proceedings), pp. 369–378. 1997.

12. George Drettakis, Luc Robert and SylvainBugnoux. Interactive common illumination forcomputer augmented reality. In Julie Dorseyand Philipp Slussalek (eds), Rendering Tech-niques ’97 (Proc. Eighth Eurographics Work-shop on Rendering), Springer Wien, New York,NY, pp. 45–56. 1997.

13. Kristin J. Dana, Bram van Ginneken, Shree K.Nayar and Jan. J. Koenderink. Reflectance andtexture of real-world surfaces. ACM Transac-tions on Graphics, 18(1):1–34, 1999.

14. Alain Fournier, Atjeng S. Gunawan and ChrisRomanzin. Common Illumination BetweenReal and Computer Generated Scenes. In Pro-ceedings of Graphics Interface ’93, MorganKaufmann, San Francisco, CA, pp. 254–262.1993.

15. F. Franca, M. Oguma and J. R. Howell.Inverse radiative heat transfer with nongray,nonisothermal participating media. In R. A.Nelson, T. Chopin and S. T. Thynell (eds),ASME HTD, vol. 361-5, IMECE, pp. 145–151.1998.

16. James D. Foley, Andries van Dam,Steven K. Feiner and John F. Hughes.Computer Graphics, Principles and Practice,Second Edition. Addison-Wesley, Reading,Massachusetts, 1990.

17. Andrew S. Glassner. Principles of Digital Im-age Synthesis. Morgan Kaufmann, San Fran-cisco, CA, 1995.

18. Erwan Guillou. Simulation d’environnementscomplexes non lambertiens a patir d’images:Application a la realite augmentee, PhDthesis, Mathematiques. Informatique, Signalet electronique et Telecomunications. IF-SIC/IRISA, 2000 in french

19. J. Hadamard. Sur les problemes aux deriveesparitelles et leur signification physique. In Bull.Univ. Of Princeton, pp. 49–52. 1902 In French

20. Berthold K. P. Horn and Michael J. Brooks.Shape from Shading. MIT Press, Cambridge,MA, 1989.

21. Paul Heckbert. Adaptive Radiosity Texturesfor Bidirectional Ray Tracing. In ComputerGraphics (ACM SIGGRAPH ’90 Proceedings),vol. 24, pp. 145–154. 1990.

22. P. Hanrahan and P. Haeberli. Direct wysiwygpainting and texturing on 3d shapes. Com-puter Graphics Proceedings, Annual Confer-ence Series (ACM SIGGRAPH ’90 Proceed-ings), 24:215–223, 1990.

23. V. Harutunian, J. C. Morales and J. R. How-ell. Radiation exchange within an enclosureof diffuse-gray surfaces: The inverse prob-lem. In Inverse Problems in Heat Transfer,ASME/AIChE National Heat Transfer Confer-ence. Portland, 1995.

24. Pat Hanrahan, David Salzman and Larry Aup-perle. A Rapid Hierarchical Radiosity Al-gorithm. In Computer Graphics (ACM SIG-GRAPH ’91 Proceedings), vol. 25, pp. 197–206. 1991.

25. P. C. Hansen, T. Sekii and H. Shibahashi. Themodified truncated svd method for regular-ization in general form. SIAM - J. Sci. StatComput., 13(5):1142–1150, 1992.

26. Katsushi Ikeuchi and K. Sato. Determiningreflectance properties of an object using rangeand brightness images. IEEE Transactions onPattern Analysis and Machine Intelligence,13(11):1139–1153, 1991.

27. Michael Kass. Inverse problems in computergraphics. In Nadia Magnenat Thalmann andDaniel Thalmann (eds), Creating and Ani-mating the Virtual World (Computer Animation


“sur200pag

✐

✐ ✐

✐

✐ ✐


’92, Fourth Workshop on Computer Anima-tion). Springer-Verlag, Tokyo, 1992.

28. G. Kay and T. Caelli. Inverting an illuminationmodel for range and intensity maps. ImageUnderstanding, 59:183–201, 1994.

29. John K. Kawai, James S. Painter and MichaelF. Cohen. Radiooptimization - Goal BasedRendering. In Computer Graphics Proceed-ings, Annual Conference Series, 1993 (ACMSIGGRAPH ’93 Proceedings), pp. 147–154.1993.

30. C. Loscos and G. Drettakis. Low-cost photo-metric calibration for interactive relighting. InFirst French-British International Workshop onVirtual Reality. Brest, France, 2000.

31. Celine Loscos, George Drettakis and LucRobert. Interactive virtual relighting of realscenes. IEEE Transactions on Visualizationand Computer Graphics, 6(4):289–305, 2000.

32. C. Loscos, M. C. Frasson, G. Drettakis, B. Wal-ter, X. Grainer and P. Poulin. Interactive virtualrelighting and remodeling of real scenes. InRendering Techniques ’99 (Proceedings of the10th Eurographics Workshop on Rendering),Springer Wien, New York, NY, pp. 329–340.1999.

33. Eric P. Lafortune, Sing-Choong Foo, KennethE. Torrance and Donald P. Greenberg. Non-linear approximation of reflectance functions.In Computer Graphics (ACM SIGGRAPH ’97Proceedings), vol. 31, pp. 117–126. 1997.

34. Hendrik P. A. Lensch, Jan. Kautz, MichaelGoesele, Wolfgang Heidrich and Hans-Peter Seidel. Image-based reconstruction ofspatially varying materials. Research ReportMPI-I-2001-4-001, Max-Planck-Institut furInformatik, Stuhlsatzenhausweg 85, 66123Saarbrucken, Germany, 2001.

35. J. Marks, B. Andalman, P. A. Beardsley, W.Freeman, S. Gibson, J. Hodgins, T. Kang, B.Mirtich, H. Pfister, W. Ruml, K. Ryall, J. Seimsand S. Shieber. Design galleries: A generalapproach to setting parameters for computergraphics and animation. In Computer GraphicsProceedings, Annual Conference Series (ACMSIGGRAPH ’97 Proceedings), pp. 389–400.1997.

36. Stephen R. Marschner. Inverse Rendering inComputer Graphics, PhD thesis, Program of

Computer Graphics, Cornell University, Ithaca,NY, 1998.

37. Stephen R. Marschner and Donald P. Green-berg. Inverse lighting for photography. In Pro-ceedings of the IS &T/SID Fifth Color Imag-ing Conference, Society for Imaging Scienceand Technology, Scottsdale, AZ, pp. 262–265.1997.

38. J. C. Morales, M. Matsumura, M. Oguma andJ. R. Howell. Computation of inverse radiativeheat transfer within enclosures. In Proc. 1997ASME National Heat Transfer Conference.Baltimore, 1997.

39. Stephen R. Marschner, Stephen H. Westin,Eric P. F. Lafortune, Kennneth E. Torranceand Donald P. Greenberg. Image-based brdfmeasurement including human skin. In D.Lichinski and G. Ward Larson (eds), RenderingTechniques ’99 (Proceedings of the 10th Eu-rographics Workshop on Rendering), Springer-Verlag, New York, NY, pp. 131–144. 1999.

40. M. Oguma and J. R. Howell. Solution ofthe two-dimensional blackbody inverse radia-tion problem by inverse monte carlo method.In ASME/JSME Thermal Engineering Confer-ence, vol. 3, ASME, pp. 243–250. 1995.

41. E. Ofek, E. Shilat, A. Rappoport and M.Werman. Multiresolution textures from imagesequences. IEEE Computer Graphics and Ap-plications, 17(2):18–29, 1997.

42. Pierre Poulin and Alain Fournier. Lights fromhighlights and shadows. Computer Graphics,25(2):31–38, 1992.

43. P. Poulin and A. Fournier. Painting surfacecharacteristics. In P. M. Hanrahan and W. Pur-gathofer (eds), Rendering Techniques ’95 (Pro-ceedings of the Sixth Eurographics Workshopon Rendering), Springer-Verlag, New York,NY, pp. 119–129. 1995.

44. Pierre Poulin, Mathieu Ouimet and Marie-Claude Frasson. Interactively modeling withphotogrammetry. In Rendering Techniques’98 (Proceedings of Eurographics RenderingWorkshop ’98), Springer Wien, New York, NY,pp. 93–104. 1998.

45. Pierre Poulin. Shading and Inverse Shadingfrom Direct Illumination, PhD thesis, De-partment of Computer Science, University ofBritish Columbia, 1993 Available from http:

//www.cs.ubc.ca/nest/imager/th.html


“sur200pag

✐

✐ ✐

✐

✐ ✐


46. G. Patow and X. Pueyo. A Survey on ReflectorDesign Problems. Technical Report TR-IIiA00-16-RR, Universitat de Girona, 2000.

47. Pierre Poulin, Karim Ratib and Marco Jacques.Sketching shadows and highlights to positionlights. In Proceedings of Computer Graph-ics International 97, IEEE Computer Society,pp. 56–63. 1997.

48. W. H. Press and S.A. Teukolsky. Simulatedannealing optimization over continuous spaces.Computers in Phisics, 426–429, 1991.

49. W. H. Press, S. A. Teukolsky, W. T. Vetterlingand B. P. Flannery. Numerical Recipes in C: The Art of Scientific Computing. CambridgeUniversity Press, 1992.

50. Ravi Ramamoorthi and Pat Hanrahan. On therelationship between radiance and irradiance:Determining the illumination from images ofa convex Lambertian object. Journal of theOptical Society of America (JOSA A), 2448–2459, 2001.

51. Ravi Ramamoorthi and Pat Hanrahan. A signal-processing framework for inverse rendering. InComputer Graphics Proceedings, Annual Con-ference Series (SIGGRAPH 2001), pp. 117–128. 2001.

52. Chris Schoeneman, Julie Dorsey, Brian Smits,James Arvo and Donald Greenberg. PaintingWith Light. In Computer Graphics Proceed-ings, Annual Conference Series, 1993 (ACMSIGGRAPH ’93 Proceedings), pp. 143–146.1993.

53. R. Shacked. Automatic lighting design us-ing a perceptual quality metric, PhD the-sis, The Hebrew University of Jerusalem,Israel, 2001 http://www.cs.huji.ac.il/

~danix/ldesign

54. Y. Sato and K. Ikeuchi. Reflectance analysisfor 3d computer graphics model generation.CVGIP Graphical Models and Image Process-ing, 58(5):437–451, 1996.

55. R. Shacked and D. Lischinski. Automatic light-ing design using a perceptual quality met-ric. Computer Graphics Forum (Eurographics2001), 20(3):215–227, 2001.

56. Francois Sillion and Claude Puech. Radiosityand Global Illumination. Morgan Kaufmann,San Francisco, CA, 1994.

57. I. Sato, Y. Sato and K. Ikeuchi. Acquiringa radiance distribution to superimpose virtualobjects onto a real scene. IEEE Transactions onVisualization and Computer Graphics, 5(1):1–12, 1999.

58. I. Sato, Y. Sato and K. Ikeuchi. Illumina-tion distribution from brightness in shadows:adaptive estimation of illumination distributionwith unknown reflectance properties in shadowregions. In Proceedings of IEEE ICCV’99,pp. 875–882. 1999.

59. I. Sato, Y. Sato and K. Ikeuchi. Illuminationdistribution from shadows. In Proceedings ofIEEE Conf. on Computer Vision and PatternRecognition’99, pp. 306–312. 1999.

60. Y. Sato, M. D. Wheeler and K. Ikeuchi. Objectshape and reflectance modeling from obser-vation. In Computer Graphics Proceedings,Annual Conference Series (ACM SIGGRAPH’97 Proceedings), pp. 379–387. 1997.

61. Alan Watt and Mark Watt. Advanced Renderingand Animation Techniques: Theory and Prac-tice. Addison-Wesley, 1992.

62. Yizhou Yu, Paul Debevec, Jitendra Malik andTim Hawkins. Inverse global illumination: re-covering reflectance models of real scenes fromphotographs. In Computer Graphics Proceed-ings, Annual Conference Series (Proc. ACMSIGGRAPH ’99 Proceeding), pp. 215–224.1999.

63. Yizhou Yu and Jitendra Malik. Recoveringphotometric properties of architectural scenesfrom photographs. In Michael Cohen (ed),Computer Graphics Proceedings, Annual Con-ference Series (ACM SIGGRAPH ’98 Proceed-ings), Addison Wesley, pp. 207–218. 1998.


Documents

A Survey of Inverse Rendering Problemsima.udg.es/~dagush/papers/surveyInvLighting.pdf“sur 200 pag Volume 22 (2003), number 4 pp. 663–687 COMPUTER GRAPHICS forum A Survey of Inverse