Probabilistic illumination-aware filteringfor Monte Carlo rendering

Ian C. Doidge Mark W. Jones

Swansea University, Wales, UK

Wednesday 12th JuneComputer Graphics International 2013

Monte Carlo Integration

2

− Path Tracing− Unbiased

− Robust

16384 samples 10 hrs

Monte Carlo Integration

3

− Path Tracing− Unbiased

− Robust

− Progressive

− High variance:− Noise

− Converges with

1 sample 2 secs2 samples 4 secs4 samples 8 secs8 samples 17 secs16 samples 34 secs32 samples 1 min 10 secs64 samples 2 mins 30 secs128 samples 5 mins256 samples 10 mins512 samples 20 mins1024 samples 40 mins2048 samples 1 hr 20 mins4096 samples 2 hrs 30 mins8192 samples 5 hrs16384 samples 10 hrs

Motivation

− Can exploit pixel correlation

− But... often breaks down:− Texture detail− Illumination discontinuities− Glossy reflections

− Filter but detect & preserve edges− Use data from rendering− Already available

Path Tracing Reference

Related WorkOutlier rejection:

− For ‘cleaning up’ images− Decoro et al. ‘10, Pajot et al. ’11

General image denoising:− Kalantari & Sen ‘13

Sample reconstruction:− Sen & Darabi ’12− Impressive quality, few samples− Unsuitable for progressive rendering

(Ir)radiance caching:− Ward ‘88, Kontkanen ’04, Krivanek ‘05, Gassenbauer ’09− Interpolation bias, limited to low frequency detail

Contributions

− Per-pixel clustering framework− Incident radiance− Layers to separate vertex contributions− Aids detection of discontinuities

− High intensity noise removal− Pre-process to filtering

− Probabilistic filtering− Preserves texture detail− Preserves illumination edges− Suitable for progressive rendering 6

Path Tracing

Our Method

Algorithm Overview

7

Trace paths

Performclustering

High intensitynoise removal

Filtered image

Iterate

Low variancebuffers

Probabilisticfiltering

Unbiased imageCompute vertex

contributions

RadianceRadianceRadiancelayers

IncidentradianceIncident

radianceIrradiancelayers

Layer based approach

8

1 2 3

= + +

= + +

Radiance

Irradiance

Layer based approach

9

1 2 3

= + +

− Irradiance contributions:− Ignores visible texture and BRDF

− Already available!

= + +

− Separate vertex contributions:− Minimise interference

− Simplifies feature detection

Radiance

Irradiance

− Utilise texture buffer:− Incident radiance radiance

Radiance separation

, , , ·

, , … , ,

, · , , ·

Clustering

11

v = 1 v = 2 v 3

− 1D Luminance domain:− Improves performance/storage

− H.V.S. sensitive to luminance

− Preserves visual discontinuities

Clustering

12

v = 1 v = 2 v 3

− Clusters at each layer:− Aggregate samples min max

− Represents (ir)radiance for sub-integral

− Contiguous, non-overlapping in 1D

Clustering

13

v = 1 v = 2 v 3Cluster{

float[3] radiancefloat[3] irradiancefloat minfloat maxint numSamples : 31 bitsint outlier : 1 bit

}

Clustering

14

v = 1 v = 2 v 3

− Heuristic based on:− Absolute sample difference− Cluster gradient

− Add new contributions− Extend existing clusters

Clustering

Clustering

− Based on:− Absolute sample difference− Cluster gradient

− Add new contributions− Extend existing clusters

Clustering

− Based on:− Absolute sample difference− Cluster gradient

− Add new contributions− Extend existing clusters

− Merge clusters− Reduce memory costs− Dynamic approximation of integral− Always preserves largest discontinuities

Algorithm Overview

18

Trace paths

Performclustering

High intensitynoise removal

Filtered image

Iterate

Low variancebuffers

Probabilisticfiltering

Unbiased imageCompute layercontributions

RadianceRadianceRadiancelayers

IncidentradianceIncident

radianceIrradiancelayers

− Key differences:− Ignore texture variation− Layer based rejection

Lower rejection rates Energy preservation More data

Noise Removal

19

− Density estimation approach− Image space neighbourhood

− DeCoro et al. ’10

− Pixel-wise estimates− Pajot et al. ’11

− Filtering Pre-process:− Prefer sparse data to no data− Accept if isolated and low energy

Illumination-aware filtering

− Extends cross-bilateral filter:− Eisemann & Durand ’04− Petschnigg et al. ’04− Low variance range buffer

− Poisson probability distribution:

− Derive weights from cluster statistics− Probabilistic detection of illumination edges

!

Illumination-aware filtering

21

Filtered image

Find similarclusters

RadianceLow var.buffers

Calculatefilter weights

Scale clustercontributions

Final pixelradiance

Incidentradiances

− Extend min and max of cluster from pixel p

− Find overlapping clusters in neighbouring contributing pixels, c

− Compute poisson probabilities using sample counts

− Repeat for each cluster in pixel p

Find similar clusters

!

6

5

1

− Product of Gaussian, geometric and poissonweights:

Calculate filter weights

23

RadianceRadianceLow var.buffers

Geometricweights

Computegeom. weights

RadianceRadiancePoissonweights Filter

kernelweights

· ·

Gaussian GeometricPoisson

Gaussianweights

Scale cluster contributions− Radiance is a product of:

− Texture , BRDF coefficient , incident radiance ,

− Preserve detail in from pixel

− Filter and using neighbouring pixels

− Weight with kernel · ·

· · ·

Results

Path Tracing Our Method25

Cross-Bilateral

Reference

− Multiple coloured lights

− Hard and soft gradients

− Chromatic and luminance noise

16 samples per pixel

ResultsPath Tracing (22 samples) Reference

Equal time comparison after 45 seconds26

Our method (16 samples)

ResultsReference

27

Cross-Bilateral(Smooth noise)

Cross-Bilateral(Preserve texture)Our method

Equal time comparison after 45 seconds

− Self-shadowing

− Complex geometry

ResultsPath Tracing

Our method

Cross-Bilateral

Equal time comparison after 45 seconds28

Reference

Conclusion

− Fast image-based incident radiance filtering− Preserve illumination edges and gradients− Preserve texture detail− Remove high intensity noise

− Future Work− Perceptual clustering heuristic− Layer specific parameters− Automatic kernel radius

29

Thank you for listening!

Questions?

(and Answers)

30

Thank you for listening!

Questions?

(and Answers)

31

32

Future Improvements

− Aliasing:− Yang et al. ’11, Anti-Aliasing Recovery, ACM Trans. Graph.

− Convergence:− Variance based adaptive filter widths

− Noise removal parameters:− Difficult to establish for low sample counts− What is the pixels normal distribution?

33