Importance sampling Framework at MPC

Importance Sampling

Framework at MPC Guillaume Francois

Kraken character from Clash of the Titans. © Warner Bros

Importance Sampling

Framework at MPC

• Lighting solution for movies such as

– Clash of the Titans

– Harry Potter and the Deathly Hallows

– The Chronicles of Narnia: The voyage of the Dawn Treader

• Image Based Lighting using Importance Sampling

became a standard solution in a majority of shots of

Clash of the Titans

Image Based Lighting Constraints

• Energy Conserving BRDF and Albedo Pump-up

• Adjusting With Number of Samples

• Raytracing heavy scenes such as the the ones

in Clash of the Titans

• Extras

Energy Conserving BRDF

• Why?

– Avoid or reduce light leakage / Edge darkening

– Ensure that independently lookdeved objects work

together

Energy conserving BRDF

• Different BRDF tests:

– Ward Brdf [War92][Wal05]

• Anisotropic gaussian shaped Brdf, visible edge darkening

– Ashikhmin-Shirley Brdf [AS00]

• Anisotropic phong-like Brdf, small yet visible edge darkening

– Halfway Vector Disk Brdf [EBJ06]

• Designed to be energy conserving, anisotropic, no edge

darkening, straightforward retroreflection

Energy conserving BRDF

• Comparison

Ward Ashikmin-Shirley Halfway Vector Disk

Energy Conserving BRDF

• Ashikhmin-Shirley Brdf

– Well known Phong-like behaviour

– Anisotropic

– No mirror reflections at grazing angles like the

Halfway Vector Disk Brdf

Albedo Pump up

• How to make Ashikhmin-Shirley Brdf “more” energy conservative?

– Analytical Albedo pump up [Neu99]

– One simple method: multiply by a precomputed factor

Factor(ωo,roughness) = 1 / ∫Ω f(ωo, ωi, roughness) cos(θi)dωi

Albedo Pump up

• This pump up reduced the quality of the Filtered

Importance Sampling technique, overestimating

the filtering kernel for low pdf directions

• Replace the pdf in the mipmap equation by:

pow(pdf, α) with α<1

Importance Sampling = Number of

Samples

• Not very intuitive

• Very important to adjust these numbers to save

memory and time (or even just to get the renders

to finish in some cases)

• Needs of tools or automatic setup methods to

simplify the life of the technical directors

Adjusting with Number of Samples

• Low number of samples

needed to estimate the

reflection of glossy

objects

• Large number of samples

needed for rough

materials


• Non linear influence of

the number of samples

on the picture quality

• Automatic solution:

remapping of the number

of samples


• Let the TDs find the best adjustement?

• If the specular roughness varies over the surface

it becomes impossible to manually control and

optimize the number of samples

• Solution: Experimental mapping of the number

of samples

• Depends on chosen brdf / level of quality needed

– No trivial derivation of the number of samples while

looking at the brdf equations

– Render references spheres with varying roughness for

varying environments and pick the lower number of

samples that gives us an acceptable result

Use tabulated number of samples for each type of brdf

(overall quite similar number of samples)

Experimental mapping of the

Number of Samples


Number of Samples

Images from debevec.org

http://ict.debevec.org/~debevec/Probes/stpeters_probe.hdr

http://ict.debevec.org/~debevec/Probes/uffizi_probe.hdr

http://ict.debevec.org/~debevec/Probes/grace_probe.hdr

• Automatic remapping of the number of samples

depending on the specular roughness (2-64 samples)

Kraken character from Clash of the Titans. © The Moving Picture Company


Number of Samples



Number of Samples

• Limitations:

– Depends on the IBL used to define the automatic number of

samples

– Not accounting occlusion or reflections from objects

• Possible solutions:

– Get some occluders in the remapping experimentations?

– Have a dedicated UI to give extra control to the TDs


Number of Samples

• Raytracing is memory and computationally prohibitive

• Solution: trace less rays!

Camera

Adjusting the Number of Traced Rays

• Project the raytraced visibility into an appropriate basis

• Read the visibility function for new rays

Camera


• Interpolating in between raytraced rays

– Spherical or Zonal Harmonics

• Works well but needs fast projection and reconstruction

• Memory needs to be well managed

• SH in production: Nvidia/Weta Pantaray [PFHA10]

– Our trick

• Very simple and intuitive basis: Sort the raytraced rays by

quadrants



• Sort the raytraced rays by quadrants

– Quadrants around the direction of the mirror reflection

ωr

– Build T and B so that (B,T, ωr) is an orthonormal basis


• Simple Basis:

B1(ωi)=(α-1)(β-1)/4

B2(ωi)=-(α+1)(β-1)/4

B3(ωi)=(α+1)(β+1)/4

B4(ωi)=-(α-1)(β+1)/4

with

α=sign(ωi.T)

β=sign(ωi.B)

B

T

ωr


• Occlusion of a quadrant K:

Occk = ∑N Occ(ωi)Bk(ωi)

• For a non traced direction ωj:

Occ(ωj) = Occ1B1(ωj) + Occ2B2(ωj) + Occ3B3(ωj) + Occ4B4(ωj)


Algorithm: vector randomVector = normalize(vector(random(),random(),random()));

vector T = randomVector^wr ;

vector B = wr^T;

color visibilityQuadrant[4] = {0,0,0,0};

int nSampleQuadrant[4] = {0,0,0,0};

int inQuadrant[4] = {0,0,0,0};

//Raytraced samples

for( int i=0; i<nRaytracedSamples; i+=1 ){

alpha = sign(wi[i].T); beta = sign(wi[i].B);

visibility = rayTraceVisibility(wi[i]);

inQuadrant[0] = 0.25*(alpha-1)*(beta-1);

inQuadrant[1] = -0.25*(alpha+1)*(beta-1);

inQuadrant[2] = 0.25*(alpha+1)*(beta+1);

inQuadrant[3] = -0.25*(alpha-1)*(beta+1);

for( j=0; j<4; j+=1 ){

nSampleQuadrant[j] += inQuadrant[j]; //save the number of raytraced rays for normalization

visibilityQuadrant[j] += inQuadrant[j]*visibility;

}

//do the other computations…

}

//Normalize values within each quadrants

for(int j = 0; j<4; j+=1 ){

if(nSampleQuadrant[j]>0){

visibilityQuadrant[j] /= nSampleQuadrant[j];

}

else{

…

}

}

//Non raytraced samples

for( int i=nRaytracedSamples; i<nSamples; i+=1 ){

alpha = sign(wi[i].T);

beta = sign(wi[i].B);

color visibility = visibilityQuadrant[0]*0.25*(alpha-1)*(beta-1)

+ visibilityQuadrant[1]*-0.25*(alpha+1)*(beta-1)

+ visibilityQuadrant[2]*0.25*(alpha+1)*(beta+1)

+ visibilityQuadrant[3]*-0.25*(alpha-1)*(beta+1);

//do the other computations...

}


• Images rendered with 64 samples for the environment lookup

100% of traced rays for

occlusion computation

60% of traced rays for

occlusion computation

Difference

x5

Raytracing Strategies in Clash of the

Titans

• Shading and lighting scenario:

– Large city

– Large Creature

– Plenty of reflections

• No miraculous recipe: combination of multiple

techniques to reduce memory, optimize

raytracing, avoid recomputations when possible


Titans

• Cache Occlusion and IS intermediaries

• Use of multiple IBL

– Give the ability to the TDs to add or separate their IBL into

multiple ones for better control of their lighting without massive

additional cost

– ~6 IBL used in Clash of the Titans + Reflection cards

• Solution: Store an array of directions at the surface

shader level and enhance it when needed

Occlusion Caching

Camera

• Sample the Brdf

Occlusion Caching

Camera

• Randomly choose traceable samples

• Estimate 1st IBL

• Trace if needed

Occlusion Caching

Camera

? ? ?

• Trace if needed

• Estimate 2nd IBL

Occlusion Caching

Camera

? ? ?

• Estimate Reflection cards

Occlusion Caching

Camera


Titans

• Diffuse estimation with Importance Sampling is

expensive

• Irradiance Caching [SRKG07][Chr09]

– Image Based Lighting of the city of Argos

• Many samples for accurate result

– Bake the diffuse contribution for key frames in the

sequence and merge the computed point clouds

• Renderman based ptc color bleeding

Irradiance Caching

Irradiance Caching

• Ptc used for the city illumination itself but also for the

reflections of the city onto the Kraken and Water


Titans

• Trace Distance and Trace Sets

Infinite Trace Distance – 3 hours 60 meters – 32 minutes

City Rendering with diffuse computed by cosine weighted

importance sampling of a Lambertian - 512 samples per pixel


Titans

• Trace Distance and Trace Sets

– Second / Third Trace Sets …

2

1

Kraken and Argos city from Clash of the Titans. © Warner Bros


Titans

Raytracing

• Raytracing within Pixar RenderMan

• Stay at the primitive level as much as possible

• Occlusion just need the hit information

• “Raytrace reflection” vs “Raytrace reflection from bake”

• Raytrace reflection execute the shader at each hit points

(Even with degraded number of samples for secondary rays it costs a lot)

• Raytrace reflection from bake

Usually these ptc are already computed for GI

Gather the hit point and normal to lookup within point clouds

Drawback: Specular contribution of reflected objects either not taken into

account or incorrect (but believable [SRKG07])


Titans

• Raytracing Proxy Objects

– Shooting from/against polycages

• High res: Kraken ~ 8M polygons

• Low res ~ 2M polygons

• No raytracing of displaced surfaces!

• Careful if you compute anisotropic Brdf based on the surface

derivatives!


Titans

• Raytracing Proxy Objects

High Resolution Geometry

Displaced Geometry

Poly Cage of low Resolution Geometry


Titans

• Effective ray origin [TL04]

– If the ray hits before a

given offset : new origin

P

• Primitive Variable Pcage


Titans

Pcage

P

Clash of the Titans - Kraken






A few more things

• Banding artifacts reduction

• Iridescence and IS

Sampling Strategies

• Many different sampling techniques [KK02]

– Jittered

– N-rooks

– Hammersley sequence

– …

• FIS uses Hammersley

– How to reduce the banding artifacts?

Sampling Strategies

Camera

Camera

Sampling Strategies

FIS and Hammersley

sampling with no

occlusion

IS and N-rook sampling

with ray-traced

occlusion

FIS and Hammersley

sampling with ray-

traced occlusion

FIS and Hammersley

shifted sampling with

ray-traced occlusion

Quick Implementation of Iridescence

and Color Shifts with FIS

Quick Implementation of Iridescence

and Color Shifts with FIS

θi

• Straightforward filtering of ramps

Aknowledgements

• Anders Langlands, Pascal Gautron, Jean Colas

Prunier, Peter Seager, Douglas Mc Hale,

Sheldon Stopsack, Adrien Saint Girons, all my

former colleagues from MPC

Bibliography

• [AS00] - Michael Ashikhmin and Peter Shirley - An Anisotropic Phong BRDF Model

• [Wal05] - Bruce Walter - Notes on the Ward BRDF

• [EBJ06] - Dave Edwards, Solomon Boulos, Jared Johnson, Peter Shirley, Michael Ashikhmin,

Michael Stark and Chris Wyman - The halfway vector disk for BRDF modeling

• [Neu99] - László Neumann, Attila Neumann and László Szirmay-Kalos -Reflectance Models by

Pumping up the Albedo Function

• [PFHA10] - Jacopo Pantaleoni, Luca Fascione, Martin Hill and Timo Aila – Pantaray: Fast Ray-

traced Occlusion Caching of Massive Scenes

• [SRKG07] - Apurva Shah, Justin Ritter, Chris King and Stefan Gronsky - Fast, soft reflections

using radiance caches

• [Chr09] - Per Christensen - Point Based Approximate Color Bleeding

• [TL04] - Eric Tabellion and Arnauld Lamorlette - An approximate global illumination system for

computer generated films

• [KK02] - Thomas Kollig and Alexander Keller - Efficient Multidimensional Sampling

Documents

Importance sampling Framework at MPC