14
Mathematics and Computers in Simulation 55 (2001) 79–92 Hierarchical Monte Carlo image synthesis Alexander Keller * Department of Computer Science, Kaiserslautern University, Postfach 3049, D-67653 Kaiserslautern, Germany Abstract A fundamental variance reduction technique for Monte Carlo integration in the framework of integro-approximation problems is presented. Using the method of dependent tests a successive hierarchical function approximation al- gorithm is developed, which captures discontinuities and exploits smoothness in the target function. The general mathematical scheme and its highly efficient implementation are illustrated for image generation by ray tracing, yielding new and much faster image synthesis algorithms. © 2001 IMACS. Published by Elsevier Science B.V. All rights reserved. Keywords: Image synthesis; Integro-approximation; Monte Carlo integration; Quasi-Monte Carlo integration 1. Introduction Image synthesis usually is performed on a pixel by pixel basis [3,5,9] using variance reduced Monte Carlo integration for antialiasing. In [17] the convergence rate of the Monte Carlo method with N samples per pixel was shown to be of the order O(N (1/2)-(1/2d) ), where d is the dimension of the integration domain. This is also the rate obtained by quasi-Monte Carlo integration as found by arguments in [19]. The computational cost can be reduced by interpolating information instead of sampling, which is indicated by coherence heuristics (see [7] for a survey, and [23]). In our new approach, we consider the array of pixels forming an image as the samples of a function taken at the positions of the regular rectangular pixel grid. Instead of computing individual pixel values, image synthesis now consists in determining that function, i.e. to calculate all pixel functionals at once. In this context image synthesis belongs to the class of integro-approximation problems, where Monte Carlo integration generalizes to the method of dependent tests [6,20]. In [11] a multilevel version of the method of dependent tests is developed, and it is shown that for smooth model problems up to O(n 2r/d ) sampling work can be saved, where n is the number of grid points, i.e. pix- els, r the degree of smoothness of the target function and d again the dimension of the integration domain. The main idea of the algorithm is similar to principles in image analysis for compression algorithms: If there is correlation, i.e. coherence or smoothness, in an image, e.g. wavelet or Fourier coefficients become * Tel.: +49-631-205-3345; fax: +49-631-205-3270. E-mail address: [email protected] (A. Keller). 0378-4754/01/$20.00 © 2001 IMACS. Published by Elsevier Science B.V. All rights reserved. PII:S0378-4754(00)00248-2

Hierarchical Monte Carlo image synthesis

Embed Size (px)

Citation preview

Page 1: Hierarchical Monte Carlo image synthesis

Mathematics and Computers in Simulation 55 (2001) 79–92

Hierarchical Monte Carlo image synthesis

Alexander Keller∗Department of Computer Science, Kaiserslautern University, Postfach 3049, D-67653 Kaiserslautern, Germany

Abstract

A fundamental variance reduction technique for Monte Carlo integration in the framework of integro-approximationproblems is presented. Using the method of dependent tests a successive hierarchical function approximation al-gorithm is developed, which captures discontinuities and exploits smoothness in the target function. The generalmathematical scheme and its highly efficient implementation are illustrated for image generation by ray tracing,yielding new and much faster image synthesis algorithms. © 2001 IMACS. Published by Elsevier Science B.V. Allrights reserved.

Keywords:Image synthesis; Integro-approximation; Monte Carlo integration; Quasi-Monte Carlo integration

1. Introduction

Image synthesis usually is performed on a pixel by pixel basis [3,5,9] using variance reduced MonteCarlo integration for antialiasing. In [17] the convergence rate of the Monte Carlo method withN samplesper pixel was shown to be of the orderO(N(1/2)−(1/2d)), whered is the dimension of the integration domain.This is also the rate obtained by quasi-Monte Carlo integration as found by arguments in [19]. Thecomputational cost can be reduced by interpolating information instead of sampling, which is indicatedby coherence heuristics (see [7] for a survey, and [23]).

In our new approach, we consider the array of pixels forming an image as the samples of a functiontaken at the positions of the regular rectangular pixel grid. Instead of computing individual pixel values,image synthesis now consists in determining that function, i.e. to calculate all pixel functionals at once.In this context image synthesis belongs to the class of integro-approximation problems, where MonteCarlo integration generalizes to the method of dependent tests [6,20].

In [11] a multilevel version of the method of dependent tests is developed, and it is shown that for smoothmodel problems up toO(n2r/d) sampling work can be saved, wheren is the number of grid points, i.e. pix-els,r the degree of smoothness of the target function andd again the dimension of the integration domain.The main idea of the algorithm is similar to principles in image analysis for compression algorithms: Ifthere is correlation, i.e. coherence or smoothness, in an image, e.g. wavelet or Fourier coefficients become

∗ Tel.: +49-631-205-3345; fax:+49-631-205-3270.E-mail address:[email protected] (A. Keller).

0378-4754/01/$20.00 © 2001 IMACS. Published by Elsevier Science B.V. All rights reserved.PII: S0378-4754(00)00248-2

Page 2: Hierarchical Monte Carlo image synthesis

80 A. Keller / Mathematics and Computers in Simulation 55 (2001) 79–92

small and can be neglected. Transferring this to image synthesis means that coefficients providing onlysmall contribution can be computed at a much smaller accuracy, allowing to improve convergence.

Based on these ideas and [11] we develop a simple hierarchical variance reduction scheme for computergraphics using hierarchical interpolation, which synthesizes images much faster than contemporary algo-rithms, and which is able to handle the difficult problem of discontinuities by coherence heuristics [7,23].We obtain lower sampling rates than adaptive sampling algorithms [16,18], which in smooth environmentssample at a fixed base rate, whereas our approach can exploit that smoothness to further reduce the sam-pling work. This superior performance is obtained by calculating coefficients of a hierarchical function rep-resentation, where each level only has adecreasing contributionwith adecreasing variance. Reconstruct-ing the image function from that representation has a considerably reduced variance as compared to a pixelby pixel approach, enormously increasing the efficiency of the Monte Carlo algorithm. The resulting hier-archical Monte Carlo image synthesis procedure can be used as a front end suitable for any shader based raytracing kernel. However the general underlying mathematical procedure indicates many other applications.

In Section 2 we will briefly introduce the mathematical concepts needed for the hierarchical MonteCarlo image synthesis algorithm presented in Section 3. Then the superior practical performance isillustrated by a numerical reality check in Section 4. The new scheme gives raise to many extensions andapplications which are pointed out in Section 5. The final section draws the conclusions and indicatesdirections of future research.

2. Image synthesis

An image is a rectangular matrix of color values. In the sequel we consider a single scanline(Lk)nm−1k=0

containingnm := 2m + 1 pixels withm ∈ N0. The colorLk of thekth pixel is determined by the meanvalue integral

Lk := 1

|D|∫

D

LS(xk, t) dt,

wherexk := k/(nm − 1). The functionLS : [0, 1] × D → R+0 is the radiance1 returned by a shader rou-

tine for the ray starting from the eye through the locationt of the pixelk anchored atxk. In the simplestcase the integration domainD is the two dimensional areaP of a pixel. Including depth of field andmotion blur, we getd := dimD = 5 by additionally integrating over lens area and aperture time. Furtheradding direct illumination by area light sources or daylight simulation, the dimension becomes 7 [5]. Thedimension even can get infinite when using random walk approaches to simulate global illumination likebidirectional path tracing or the Metropolis light transport algorithm [25], whereD becomes the infinitedimensional path space.

2.1. Integro-approximation

The pixels(Lk)nm−1k=0 of a scanline also can be considered as samples of a continuous function.

L(x) := 1

|D|∫

D

LS(x, t) dt ∈ C([0, 1]).

1 For the sake of simplicity we neglect filtering and tone mapping.

Page 3: Hierarchical Monte Carlo image synthesis

A. Keller / Mathematics and Computers in Simulation 55 (2001) 79–92 81

whereL is an integral dependent on the parameterx and fulfils the interpolation constraint

L(xk) = Lk.

Thus, image synthesis can be interpreted as integro-approximation problem which can be solved byconstructing an interpolation operatorPm of the kind thatL ≈ PmL in the sense that

(PmL)(xk) = Lk

2.2. The method of dependent tests

In the sequel we will apply the method of dependent tests [6,20] which is a generalization of the MonteCarlo method: Instead of estimating a single function value (i.e. a pixel), the whole function (i.e. thewhole scanline) is estimated by

L(x) ≈ 1

N

N−1∑i=0

LS(x, ti). (1)

The very important thing to note is thatonly oneset ofN ∈ N independent, identically distributed(iid) random samples(ti)

N−1i=0 ⊂ D is chosen and then used forall values ofx ∈ [0, 1]. This is similar to

approaches as in [9,14], where in the case ofD = P in every pixelk we find the same supersamplingpattern. So estimates ofL for different values ofx are not statistically independent, since in fact thesamplesLS(xk, ti) for a fixedti and different values ofxk are aligned on the grid of the scanline, whichallows for interpolation and thus to reduce variance by exploiting correlation similar to [11].

2.3. Multilevel function representation

For an arbitrary sequence(Pl)ml=0 of interpolation operators, we can write

PmL = P0L +m∑

l=1

[Pl − Pl−1]L

yielding a multilevel representation ofPm, which is the sum of the base interpolationP0 and the differencelevels [Pl − Pl−1].

2.3.1. Piecewise linear hierarchical interpolationWe now select a dyadic hierarchy ofm + 1 grids each usingnl := 2l + 1 grid points in the levell,

where 0≤ l ≤ m and choose the piecewise linear Lagrange interpolation

(PlL)(x) :=nl−1∑k=0

Lk2m−lΛ(2lx − k),

whereΛ(x) := max{0, 1 − |x|} is the hat function. Using the dilated and translated hat function

Λlk(x) := Λ(2lx − 2k − 1)

Page 4: Hierarchical Monte Carlo image synthesis

82 A. Keller / Mathematics and Computers in Simulation 55 (2001) 79–92

in level l = 0 we get

(P0L)(x) = L0(1 − x) + Lnm−1x

and

([Pl − Pl−1]L)(x) =2l−1−1∑k=0

λlkΛ

lk(x)

for the remaining difference levels wherel > 0 with thenl − nl−1 = 2l−1 coefficients

λlk = L(2k+1)2m−l − Lk2m−(l−1) + L(k+1)2m−(l−1)

2, (2)

where the current integer step width at levell is 2m−l. Seen in the context of the lifting scheme [21],λl

k corresponds to the correction term (or detail coefficient) for linear prediction, however our functiondecomposition misses the lifting step. As illustrated in Fig. 1, a scanline(Lk)

nm−1k=0 can be represented in

the hierarchical hat basis by the two outmost color valuesL0, Lnm−1, and the coefficients(λlk)

2l−1−1k=0 for

0 < l ≤ m.

Fig. 1. Hierarchical synthesis of a scanline (see Section 2.3.1): (a) the coefficientsλlk for scaling (b) the basis vectorsΛl

k ofthe hierarchical hat basis (c) yielding the difference levels [Pl − Pl−1]L and (d) resulting in the reconstructed scanline where(PmL)(xk) = Lk.

Page 5: Hierarchical Monte Carlo image synthesis

A. Keller / Mathematics and Computers in Simulation 55 (2001) 79–92 83

3. The new algorithm

The hierarchical Monte Carlo image synthesis algorithm is a hierarchical variance reduction schemesimilar to [10,11], where a coarser function approximation is used to separate the main part of a finerapproximation level. Since correlation in an image causes the finer levels to have less contribution to thereconstructed function (see [21]), more work is spent to compute the predictor levels than for the corrector(i.e. the detail) levels. However, the prediction may be unreliable due to discontinuities in the integrandand hence it may be necessary to avoid large errors by localization.

3.1. Hierarchical sampling

The large support of the basis function in the coarser levels requires very exact computation, while thecoefficients in the finer levels have small support and small contribution, and as such need not to be ofthat high accuracy to obtain an overall reduced variance. According to the theoretical results in [11], weselect a geometrically decaying sampling rate

Nl := Nnm

nl

2αl 2α − 1

2α(m+1) − 1

for the integrals at levell, whereα ∈ R+ is a positive parameter controlling the decay of the number ofsamples in the finer levels for the dyadic hierarchy from Section 2.3.1. Larger values ofα concentratemore samples in the finer levels. The total sampling cost then is the sum over them + 1 levels of thesamples used per grid point:

m∑l=0

Nlnl =m∑

l=0

Nnm

nl

2αl 2α − 1

2α(m+1) − 1nl

= Nnm

2α − 1

2α(m+1) − 1

m∑l=0

(2α)l = Nnm

2α − 1

2α(m+1) − 1

(2α)(m+1) − 1

2α − 1= Nnm

Obviously,Nl has been chosen the way that the total cost is equal to the cost of the usual approach,where we would useN samples for each pixel of a scanline with resolutionnm. For the real samplingroutine the truncated integer valuesbNlc are used, whereasN now can be chosen fromR+.

3.2. Hierarchical refinement

We use the method of dependent tests (1) to estimate coefficients (see Fig. 1a) of the hierarchicalfunction representation of Section 2.3.1. Therefore, a sample set(ti)

Nm−1i=0 is fixed once in advance forall

computations and we denote

Llk := 1

Nl

Nl−1∑i=0

LS(xk, ti) (3)

The algorithm now is recursively defined by starting in levelP0, where we approximate the boundaryby

L0 ≈ L0 := L00

Page 6: Hierarchical Monte Carlo image synthesis

84 A. Keller / Mathematics and Computers in Simulation 55 (2001) 79–92

and

Lnm−1 ≈ Lnm−1 := L0nm−1

Then the subsequent refinement is derived from Eq. (2):

L(2k+1)2m−l = Lk2m−(l−1) + L(k+1)2m−(l−1)

2︸ ︷︷ ︸predictor

+ λlk︸︷︷︸

corrector

≈ L(2k+1)2m−l := Lk2m−(l−1) + L(k+1)2m−(l−1)

2+ Ll

(2k+1)2m−l −Ll

(2k)2m−l + Ll(2k+2)2m−l

2︸ ︷︷ ︸=:λl

k

(4)

where the predictor is the function linearly interpolated at the coarser levell − 1 and the correctorλlk

is computed at the finer levell to yield the approximationL(2k+1)2m−l for the pixel valueL(2k+1)2m−l .This hierarchical refinement procedure, which is illustrated in Fig. 2, in fact is a hierarchical separationof the main part: Functionals with large support are calculated at high precision and used as controlvariate for the next finer level, where the functionals with smaller support are estimated at reduced

Fig. 2. Hierarchical refinement: (a) Monte Carlo estimation ofLl

(2k+1)2m−l , Ll

(2k)2m−l , andLl

(2k+2)2m−l in level l with Nl samples

then; (b) computation of the correctorλlk and (c) prediction by linear interpolation ofLk2m−(l−1) , andL(k+1)2m−(l−1) , which were

already computed on the coarser levell − 1 usingNl−1 > Nl samples, finally yielding the new pixel valueL(2k+1)2m−l on thefiner levell.

Page 7: Hierarchical Monte Carlo image synthesis

A. Keller / Mathematics and Computers in Simulation 55 (2001) 79–92 85

precision. If correlation is present, it will be detected using the method of dependent tests resulting insmall contributionsλl

k.

3.3. To localize or not to localize. . .

Implementing the unbiased algorithm from Section 3.2 yields superior results in smooth areas of theimage while introducing disturbing interpolation artifacts as can be seen in Fig. 5a. These artifacts raisefrom locally high variance in the estimator (3) onD: Looking at direct illumination calculations, thepredictor is calculated at sufficient accuracy. The corrector then is calculated at a lower sampling rate andit can happen, that the estimator dramatically over- or underestimates the true contribution, since, e.g. bychance all rays happen to see or do not happen to see the light source, although it is partially occluded.Then, the linear interpolation in the prediction step propagates this large error over all successive levels,resulting in the striping artifacts.

This artifact is easily prevented by localization, i.e. by replacing the predictor–corrector valueL(2k+1)2m−l

by the direct estimateLl(2k+1)2m−l . A robust indicator for localization must depend only on the set of samples

(LS(xk, ti))Nl−1i=0 in order to be general, and cannot rely on statistical tests, sinceNl might be too small to

fulfil the assumptions of the law of large numbers. Localization is indicated2 by the following heuristics:

1. If the radiance in Eq. (4) exits the monitor gamut, i.e.

Lk /∈ [0, 1].

2. If the contrast between the predictor–corrector value and the direct estimate in Eq. (4) is too large, i.e.

C(Lk, Llk) > T1,

whereT1 is the contrast threshold and

C(Lk, Llk) := |Ll

k − Lk|Ll

k + Lk

∈ [0, 1] for Lk, Llk ≥ 0.

3. If any estimatorLlk in Eq. (4) has inherent high contrast, i.e.

C(min Llk, maxLl

k) > T2,

where minLlk := min0≤i<Nl

LS(xk, ti) and maxLlk is defined analogously.

AlthoughLS is assumed to be non-negative and bounded, the first case can occur due to estimationerrors in the corrector. Compared to empirical variance, which is not very reliable at low sampling rates,contrast is a much sharper indicator, is easily determined while computing Eq. (4), and senses similar toa human eye. The second case clearly indicates striping, since either the predictor or the corrector causea large deviation from the direct pixel estimate. By the third condition we prevent prediction errors in thelevels of small sample size, where a large contrast in a large sample set indicates a potentially large errorcaused by taking only a fraction of that sample set at a finer level. Fig. 5b shows the effect of localization,

2 The different criteria have to be applied to each wavelength, e.g. red, green, blue, of the color vectorsL. This is omitted herefor the sake of simplicity.

Page 8: Hierarchical Monte Carlo image synthesis

86 A. Keller / Mathematics and Computers in Simulation 55 (2001) 79–92

Fig. 3. On the fly addition of intermediateLl′k for m ≥ l′ > l while computingLl

k.

and Fig. 5c illustrates that localization rarely occurs, so that we can take almost full advantage of the newsampling scheme.

3.4. Implementation

Hierarchical Monte Carlo image synthesis is implemented as an in-place reconstruction algorithm,requiring only the storage for the scanline. It initializes the scanline to black and computes the outmostpixelsL0

0 andL0nm−1 by Eq. (3). Looking at the recursion (4), we see that all estimatesLl′

k for m ≥ l′ > l,

implicitly computed during the calculation ofLlk, are needed for the predictor calculation in the finer levels.

Therefore, these values are added on the fly to the corresponding pixels in the scanline as illustrated inFig. 3. The set of samples(ti)

Nm−1i=0 must be incremental, so that every subset of the firstNl (0 < l ≤ m)

samples is a quadrature, too.The subsequent refinement first estimates the central valueLl

(2k+1)2m−l using the nested integration

scheme. The sum of the left and right valueLl(2k)2m−l andLl

(2k+2)2m−l has already been stored in the pixel

(2k + 1)2m−l by the nested integration scheme, so thatλlk and finallyL(2k+1)2m−l can be computed by

Eq. (4).Before storingL(2k+1)2m−l , we check the conditions from Section 3.3. In case of localizationL(2k+1)2m−l

is replaced byLl(2k+1)2m−l . If the sample rateNl is below some minimum rateNfail < N0, at leastNfail

samples are drawn to estimateL. The localized pixels are predicted to be integrals of high variance, soNfail should be chosen large as compared with the average pixel sampling rateN . Once a grid point hasbeen localized, it cannot be used for interpolation any longer, and so all subsequent calculations requiringsamples of that grid point must become local calculations. Since localization usually is a rare event, thisdoes not influence the performance of the algorithm very much.

The complete algorithm consists of two nestedfor -loops, one controlling the levell, the other per-forming the reconstruction Eq. (4) with integer stepwidth 2m−l and the nested integration in Fig. 3.

3.4.1. Arbitrary image sizeThe algorithm derived so far works on scanlines withnm = 2m + 1 pixels. By choosingLk = 0 (i.e.

black) forn ≤ k < nm, it is easily extended to an arbitrary scanline resolutionn, wherenm is chosen

Page 9: Hierarchical Monte Carlo image synthesis

A. Keller / Mathematics and Computers in Simulation 55 (2001) 79–92 87

minimal. In a more elaborate approach, we could apply extrapolation in the same way as it is used inthe lifting scheme [22] to treat the image border. For the scope of this article however, we chose the firstapproach in order to clearly present the basic algorithm. For parallel computation, it is useful to dividethe image in small blocks and to apply the algorithm to each of the subimages.

4. Numerical results and discussion

The set of parameters is robust and easily chosen. A good choice for the parameter of decay isα ≈ 0.3.The two contrast thresholdsT1 andT2 are multiples of the usual thresholdT = (Tred, Tgreen, Tblue) =(0.4, 0.3, 0.6), where we choseT1 = 0.8T andT2 = 2T . Then, we fix the average sampling rateN andthe failsafe sampling rateNfail . For the measurements we fixedN = Nfail/2. If Nfail is small, so isN , andit might happen that some levels become interpolated in the regions where no localization event preventshierarchical reconstruction. That way even sampling rates of less than one sample per pixel are achieved.The performance of hierarchical Monte Carlo image synthesis is illustrated in the graph in Fig. 4. Areference solution of the test scene in Fig. 5 has been calculated using the pixel by pixel approach with2500 samples per pixel. One sample consists in shooting a primary ray from the eye through a sampleposition in the pixel under consideration and sending one shadow ray to a sample point on the extendedlight source. Then the‖ · ‖2-distance of the pixel by pixel approach and the new algorithm with the sameaverage sample number per pixel (i.e. the same computational effort) to that master were compared. Theaverage sampling rate per pixel has been determined by counting all samples used by the new algorithmand dividing this figure by the number of pixels in the image. It is strikingly obvious, that the new scanlineintegro-approximation scheme is much more efficient than a simple pixel by pixel integration; so almostup to a factor 5 of work can be saved. The main part of the error is caused by the small number of directestimators enforced by the localization criterion (see the red and yellow areas in Fig. 5–7). This justifies to

Fig. 4. Convergence graph: comparison of the usual pixel per pixel approach to the new hierarchical image synthesis algorithm.

Page 10: Hierarchical Monte Carlo image synthesis

88 A. Keller / Mathematics and Computers in Simulation 55 (2001) 79–92

Fig. 5. Localization: (a) striping effects in the basic algorithm without localization due to high variance. (b) Striping removedby the localization criterion and (c) its impact, where green indicates the use of the predictor–corrector resultL(2k+1)2m−l , yellowindicates the use of the local estimateLl

(2k+1)2m−l , and red is used wheneverLl

(2k+1)2m−l has been upsampled due toNl < Nfail inthe local case.

select a high failsafe supersampling rateNfail , nevertheless yielding a much higher efficiency as comparedto a pixel by pixel sampling scheme.

4.1. Antialiasing

The method of dependent tests is used to detect correlation. Using the same sampling pattern per pixelas in [9], may cause objectionable aliasing artifacts. However, the hierarchical approach uses a muchhigher sampling rate in the lower levels of interpolation, whereas higher levels sampled at lower rateonly have small contribution. In consequence, e.g. area light sources do not appear discretized as pointlight sources, but present the expected smooth transitions between dark and light parts of the image (seeFigs. 5–7). For an optimal sampling, a hierarchical poisson disk pattern (see the sampling survey in [7])would be suited best.

Fig. 6. (a) Overcast sky daylight architectural visualization and (b) localization (see Fig. 5 for the explanation of the colors).

Page 11: Hierarchical Monte Carlo image synthesis

A. Keller / Mathematics and Computers in Simulation 55 (2001) 79–92 89

Fig. 7. (a) Overcast daylight simulation for a very complex model and (b) localization (see Fig. 5 for the explanation of thecolors).

4.2. Splitting

The localization criterion is sensitive to the overall contrast of the samples returned by the shaderfunction. In order to reduce the localization events, it is useful to apply splitting [1] in the shaders.Considering an extended light source, this means to span multiple stratified shadow rays for one primaryray. This prefiltering of the light incident in the point seen by the primary ray reduces the overall contrastof the shader return values, which in turn causes less localization events and increases the efficiency ofthe algorithm. Splitting should be applied for all dimensions of the integration domainD which are proneto high variance.

Daylight simulation (see [7]) incorporates an expensive hemispherical integration. Splitting is illustratedfor the overcast sky model for a relatively smooth scene in Fig. 6 and very complex model in Fig. 7.For each primary ray the shaders performed a prefiltering by sending 49 stratified shadow rays into thesky. The average pixel supersampling rate was 4.9 and 5.1, respectively. The new algorithm detects theareas where the target function is highly correlated as, e.g. the ground or the background sky map, anddramatically reduces the number of samples, as can be seen in the false color images, where green indicatesareas of hierarchical reconstruction. As compared with a pixel by pixel approach, image generation wasaccelerated by a factor 2 for the same quality.

5. Applications and extensions

Hierarchical Monte Carlo image synthesis is a fundamental procedure which can be applied wheneverthe shader functionLS can be provided. In consequence, it directly applies to distribution ray tracing [5],the photon map [13] with all its extensions, and bidirectional path tracing [15,24].

In order to take advantage of the speed-up for multiple light sources, the new algorithm is appliedseparately for each light source by separating the integrals. Otherwise, artifacts may appear due to thehigh variance in pixels with low sampling rate that randomly include only a fraction of the light sources.

Page 12: Hierarchical Monte Carlo image synthesis

90 A. Keller / Mathematics and Computers in Simulation 55 (2001) 79–92

Sky illumination computations are similar to expensive final gather operations over the cosine weightedhemisphere as they are required for smooth radiosity or photon map images. Here, the new algorithm canbe applied to first calculate the irradiance image function, which usually is a very correlated function [26],and then convolve it with the textured scene sampled by primary rays. This way the concept of irradiancecaching is captured by a much faster and more precise algorithm. However, the separation of irradiancecomputation and texture filtering results in a biased algorithm. The prediction in Eq. (4) also can be usedfor fast incremental display of the image while it is being computed or transferred by a render server.

5.1. Rendering animations

It is straightforward to extend the algorithm from the scanline to the two-dimensional image, or even toan animation over the additional dimension of time. This way it is possible to exploit further correlation,bearing a huge potential for increasing rendering efficiency for animations as already indicated by theconvergence graph in Fig. 4.

5.2. Quasi-Monte Carlo integration

The core sampling routine (3) is nothing else than pixel sampling. So the random sampling pattern(ti)

Nm−1i=0 simply can be replaced by an incremental low discrepancy sampling pattern like, e.g. the Halton

sequence, or the even more elaborate jittered low discrepancy sampling pattern, making the method saveabout another 30% of work [12,14], which is only possible, because the localization criterion is not basedon statistical items like, e.g. empirical variance, which are not available for deterministic sampling.

6. Conclusion and future research

We presented a very fast and efficient hierarchical Monte Carlo image synthesis algorithm. Due to itsgenerality the new variance reduction scheme is easily adapted to any shader based ray tracer saving almosta factor 5 in rendering time, and has much potential for further extensions and even more speed-up. Thenew algorithm is based on the concept of integro-approximation, exploiting correlation exposed by thetarget function, which is a typical property of image data.

After comparing our approach to other high speed rendering algorithms as, e.g. [8], we will investigatethe application for directly synthesizing quantitized and compressed images as presented in e.g. [2] inthe context of [4].

Future research will concentrate on developing an efficient hierarchical algorithm for the lifting scheme,i.e. basis functions with overlapping support and higher order predictors, refining and investigating thelocalization criterion in more depth, and to implement obvious applications as, e.g. hierarchical formfactor calculation in wavelet radiosity, hierarchical Monte Carlo radiosity, bidirectional path tracing, orthe Metropolis light transport algorithm.

Acknowledgements

The author would like to thank Stefan Heinrich for the profound discussions, Gerald Maitschke fromDigital Equipment GmbH for providing a DEC alpha station and Rolf Herken from mental images.

Page 13: Hierarchical Monte Carlo image synthesis

A. Keller / Mathematics and Computers in Simulation 55 (2001) 79–92 91

Special thanks go to the team of architects Christa Marx, Sandra Mozew, Marc Wilke, and Martina Zangfor providing their nice models of the city of Kaiserslautern. The remaining models are courtesy of GregLarson and Peter Shirley.

References

[1] J. Arvo, D. Kirk, Particle Transport and Image Synthesis, in: Proceedings of the Conference on Computer Graphics,SIGGRAPH 90, 1990, pp. 63–66.

[2] M. Bolin, G. Meyer, A frequency based ray tracer, in: R. Cook (Ed.), Proceedings of the SIGGRAPH 95 Conference,Annual Conference Series, ACM SIGGRAPH, Addison-Wesley, Reading, MA, August 1995, pp. 409–418.

[3] R. Cook, L. Carpenter, E. Catmull, The Reyes image rendering architecture, in: M.C. Stone (Ed.), Proceedings of theSIGGRAPH 87 on Computer Graphics, July 1987, pp. 95–102.

[4] R. Claypoole, G. Davis, W. Sweldens, R. Baraniuk, Nonlinear wavelet transform for image coding, in: Proceedings of the31st Asilomar Conference on Signals, Systems, and Computers, Vol. 1, 1997, pp. 662–667.

[5] R. Cook, T. Porter, L. Carpenter, Distributed Ray Tracing, in: Proceedings of the Conference on Computer Graphics,SIGGRAPH 84, 1984, pp. 137–145.

[6] A. Frolov, N. Chentsov, On the calculation of certain integrals dependent on a parameter by the Monte Carlo method, Zh.Vychisl. Mater. Fiz. 2 (4) (1962) 714–717 (in Russian).

[7] A. Glassner, Principles of Digital Image Synthesis, Morgan Kaufmann, Los Atlos, CA, 1995.[8] B. Guo, Progressive radiance evaluation using directional coherence maps, in: M. Cohen (Ed.), Proceedings of the

SIGGRAPH 98 Conference, Annual Conference Series, ACM SIGGRAPH, Addison-Wesley, Reading, MA, July 1998, pp.255–266.

[9] P. Haeberli, K. Akeley, The Accumulation Buffer: Hardware Support for High-Quality Rendering, in: Proceedings of theConference on Computer Graphics, SIGGRAPH 90, 1190, pp. 309–318.

[10] S. Heinrich, A Multilevel Version of the Method of Dependent Tests, in: Proceedings of the Third St. Petersburg Workshopon Simulation, St. Petersburg University Press, 1998, pp. 31–35.

[11] S. Heinrich, Monte Carlo Complexity of Global Solution of Integral Equations, J. Complexity 14 (1998) 151–175.[12] S. Heinrich, A. Keller, Quasi-Monte Carlo Methods in Computer Graphics, Part I: the QMC-buffer, Interner Bericht 242/94,

University of Kaiserslautern, 1994.[13] H. Jensen, P. Christensen, Efficient Simulation of Light Transport in Scenes with Participating Media using Photon

Maps, in: M. Cohen (Ed.), Proceedings of the SIGGRAPH 98 Conference, Annual Conference Series, ACM SIGGRAPH,Addison-Wesley, Reading, MA, July 1998, pp. 311–320.

[14] A. Keller, Instant Radiosity, in: Proceedings of the SIGGRAPH 97 Conference, Annual Conference Series, 1997,pp. 49–56.

[15] E. Lafortune, Mathematical Models and Monte Carlo Algorithms for Physically Based Rendering, Ph.D. thesis, KatholiekeUniversitiet Leuven, Belgium, 1996.

[16] M. Lee, R. Redner, S. Uselton, Statistically Optimized Sampling for Distributed Ray Tracing, in: Proceedings of theConference on Computer Graphics, SIGGRAPH 85, 1985, pp. 61–67.

[17] D. Mitchell, Consequences of Stratified Sampling in Graphics, in: Proceedings of the SIGGRAPH 96 Conference, AnnualConference Series, 1996, pp. 277–280.

[18] J. Painter, K. Sloan, Antialiased Ray Tracing by Adaptive Progressive Refinement, in: Proceedings of the Conference onComputer Graphics, SIGGRAPH 89, 1989, pp. 281–288.

[19] H. Press, S. Teukolsky, T. Vetterling, B. Flannery, Numerical Recipes inC, Cambridge University Press,Cambridge, 1992.

[20] I. Sobol, The use ofw2-distribution for error estimation in the calculation of integrals by the Monte Carlo method, USSRComput. Math., Math. Phys. 2 (1962) 717–723.

[21] W. Sweldens, Wavelets and the lifting scheme: A 5 min tour, Z. Angew. Math. Mech. 76 (Suppl. 2) (1996) 41–44.[22] W. Sweldens, The lifting scheme: A construction of second generation wavelets, SIAM J. Math. Anal. 29 (2) (1997)

511–546.[23] S. Teller, K. Bala, J. Dorsey, Conservative Radiance Interpolants for Ray Tracing, in: Proceedings of the Seventh

Eurographics Workshop on Rendering, Rendering Techniques 96, 1996, pp. 257–268.

Page 14: Hierarchical Monte Carlo image synthesis

92 A. Keller / Mathematics and Computers in Simulation 55 (2001) 79–92

[24] E. Veach, L. Guibas, Bidirectional Estimators for Light Transport, in: Proceedings of the Fifth Eurographics Worshop onRendering, Darmstadt, Germany, June 1994, pp. 147–161.

[25] E. Veach, L.Guibas, Metropolis light transport, in: T. Whitted (Ed.), Proceeding of the SIGGRAPH 97 Conference, AnnualConference Series, ACM SIGGRAPH, Addison-Wesley, Reading, MA, August 1997, pp. 65–76.

[26] G.J. Ward, P. Heckbert, Irradiance gradients, in: Proceedings of the Third Eurographics Workshop on Rendering, Bristol,UK, May 1992.