Lecture 16 -Bits & Pieces - Universiteit Utrecht

𝑰 𝒙, 𝒙′ = 𝒈(𝒙, 𝒙′) 𝝐 𝒙, 𝒙′ + න𝑺

𝝆 𝒙, 𝒙′, 𝒙′′ 𝑰 𝒙′, 𝒙′′ 𝒅𝒙′′

INFOMAGR – Advanced GraphicsJacco Bikker - November 2018 - February 2019

Lecture 16 - “Bits & Pieces”

Welcome!

Today’s Agenda:

▪ On Offsetting Shadows and Reflections

▪ Consistent Normal Interpolation

▪ Path Classification

▪ Packing Normals

▪ Multiple Importance Sampling, One More Time

▪ Research Directions

You have been told…

…To offset your shadow rays by ‘epsilon times R or L’.

▪ How should be chose epsilon?▪ Is epsilon the same everywhere?▪ Is this always sufficient?

Offsets

Advanced Graphics – Bits & Pieces 3

p

𝜔𝑜

𝜔𝑖𝑁

SafeOrigin

When R (or L) is almost parallel to the surface, the epsilon offset fails.

Alternative:

𝑂 += cos 𝜃3 𝜀 𝑅 + 1 − cos 𝜃3 𝜀 𝑁

Offsets

Advanced Graphics – Bits & Pieces 4

p

𝜔𝑜

𝜔𝑖𝑁

SafeOrigin

When R (or L) is almost parallel to the surface, the epsilon offset fails.

Alternative:

𝑂 += cos 𝜃3 𝜀 𝑅 + 1 − cos 𝜃3 𝜀 𝑁

Offsets

Advanced Graphics – Bits & Pieces 5

p

𝜔𝑜

𝜔𝑖

𝑁

Today’s Agenda:

▪ On Offsetting Shadows and Reflections

▪ Consistent Normal Interpolation

▪ Path Classification

▪ Packing Normals

▪ Multiple Importance Sampling, One More Time

▪ Research Directions

The Problem

Vertex normal: the average of the normals of all polygons connected to the vertex.

Generating / updating vertex normals in O(N):

▪ set each vertex normal to (0,0,0);▪ loop over polygons;▪ add normal of each polygon to each polygon vertex;▪ normalize all vertex normals.

Normals

Advanced Graphics – Bits & Pieces 7

The Problem

Vertex normal: the average of the normals of all polygons connected to the vertex.

Using vertex normals:

▪ Möller-Trumbore yields ‘u,v’ coordinates: these are barycentric coordinates;▪ calculate 𝑤 = 1 – (𝑢 + 𝑣);▪ now 𝑁𝑖 = 𝑤 ∗ 𝑁0 + 𝑢 ∗ 𝑁1 + 𝑣 ∗ 𝑁2.

Mind the side!

Advanced Graphics – Bits & Pieces 8

Normals

The Problem

Vertex normal: the average of the normals of all polygons connected to the vertex.

For grazing directions, the reflection may go into the surface. Now what?

▪ Use geometric normal instead of interpolated normal?▪ Clamp dot between R and N to 0?▪ Just return black?

Advanced Graphics – Bits & Pieces 9

Normals

Advanced Graphics – Bits & Pieces 10

Normals

Advanced Graphics – Bits & Pieces 11

Normals

Consistent Normal Interpolation*

Can we come up with an interpolated normal 𝑛𝑐 that behaves properly?

▪ It is smooth▪ Reflections in 𝑛𝑐 point away from the surface▪ If all vertex normals are equal to 𝑛, 𝑛𝑐 = 𝑛▪ If 𝑑𝑜𝑡 𝑖, 𝑛 = 1, 𝑛𝑐 = 𝑛𝑝

It turns out that this is indeed possible.

The paper outlines how to calculate, for a given 𝑖,a vector 𝑟 that satisfies the constraints.

𝑛𝑐 now is simply 𝑛𝑜𝑟𝑚𝑎𝑙𝑖𝑧𝑒 𝑖 + 𝑟 .

*: Reshetov et al., Consistent Normal Interpolation. ACM Transactions on Graphics, 2010.

Advanced Graphics – Bits & Pieces 12

Normals

Advanced Graphics – Bits & Pieces 13

Normals

Advanced Graphics – Bits & Pieces 14

Normals

Advanced Graphics – Bits & Pieces 15

Normals

Advanced Graphics – Bits & Pieces 16

Normals

Normal Mapping

A normal map can be used to modify normals.

Advanced Graphics – Bits & Pieces 17

Normals

Normal Mapping

A normal map can be used to modify normals.

This is typically done in tangent space.

Problem: the orientation of tangent space matters.

We need to align 𝑇 and 𝐵 with the texture,in other words: it needs to be based on theU and V vectors over the surface. This is non-trivial. For a derivation of the calculation ofT and B, see:

learnopengl.com/Advanced-Lighting/Normal-Mapping

Advanced Graphics – Bits & Pieces 18

Normals

Today’s Agenda:

▪ On Offsetting Shadows and Reflections

▪ Consistent Normal Interpolation

▪ Path Classification

▪ Packing Normals

▪ Multiple Importance Sampling, One More Time

▪ Research Directions

Paths

Advanced Graphics – Bits & Pieces 20

Splitting Light Transport

Global illumination consists of:

▪ Direct illumination▪ Indirect illumination

Or:

▪ Direct illumination▪ Indirect illumination with one diffuse bounce▪ Indirect illumination with multiple bounces

Paths

Advanced Graphics – Bits & Pieces 21

Splitting Light Transport

Light transport consists of:

▪ All paths of all lengths between the camera and the light.

Or:

▪ All paths with zero or one diffuse vertices, plus▪ all paths with at least two diffuse vertices.

Or:

▪ All paths that start with ES (e.g. ESDL), plus▪ all paths that do not start with ES (e.g. EDDL).

If we can isolate a certain class of paths, we can handle this class with a dedicated rendering algorithm.

Paths

Advanced Graphics – Bits & Pieces 22

Heckbert’s Path Notation*

Vertex types:

▪ L: light▪ E: eye▪ S: specular▪ D: diffuse

(D)+ one or more diffuse vertices(D)* zero or more diffuse vertices(D)? zero or one diffuse vertices(DS’) a diffuse or a specular vertex

*: Heckbert, Adaptive radiosity textures for bidirectional ray tracing. SIGGRAPH 1990.

Paths

Advanced Graphics – Bits & Pieces 23

Splitting Light Transport

If we can isolate a certain class of paths, we can handle this class with a dedicated rendering algorithm.

ESDL: Whitted-styleEDL: Whitted-style, with next event estimationEDSL: Photon mapping, direct visualization of the photon mapEDD(S*)L: Photon mapping, with final gather

We can also do this to parts of the path:

▪ Camera ray: rasterization?▪ Path segments that gather direct light: NEE▪ Path segments that gather indirect light: baked?

Paths

Advanced Graphics – Bits & Pieces 24

Splitting Light Transport

If we can isolate a certain class of paths, we can handle this class with a dedicated rendering algorithm.

ESDL: Whitted-styleEDL: Whitted-style, with next event estimationEDSL: Photon mapping, direct visualization of the photon mapEDD(S*)L: Photon mapping, with final gather

We can also do this to parts of the path:

▪ Camera ray: rasterization?▪ Path segments that gather direct light: NEE + filter, small kernel▪ Path segments that gather indirect light: bounce + filter, large kernel

Paths

Advanced Graphics – Bits & Pieces 25

Splitting Light Transport

If we can evaluate a certain path length using multiple techniques, we can use Multiple Importance Sampling to automatically select the best one.

E.g. Next Event Estimation: EDL via explicit or implicit technique.

Or, BDPT(book, latest edition, 16.3).

Paths

Advanced Graphics – Bits & Pieces 26

Paths

Advanced Graphics – Bits & Pieces 27

Today’s Agenda:

▪ On Offsetting Shadows and Reflections

▪ Consistent Normal Interpolation

▪ Path Classification

▪ Packing Normals

▪ Multiple Importance Sampling, One More Time

▪ Research Directions

Efficiently Storing Normals

A float color pixel contains r, g and b. Wouldn’t it be convenient if the fourth component could store the normal?

➔ Can we store a normal accurately in 32 bits?

Observation:

For a normal, we need two components (plus a sign).

➔ Can we store the two components in 16 bits each?

Packing

Advanced Graphics – Bits & Pieces 29

Efficiently Storing Normals

For an extensive study on this topic, see:

aras-p.info/texts/CompactNormalStorage.html

Bottom line:

We can encode a normal in 32-bit, and the precision will be excellent: expect approximately four digits of precision.

Pack:

uint PackNormal( float3 N )

{

float f = 65535.0f / sqrtf( 8.0f * N.z + 8.0f );

return (uint)(N.x * f + 32767.0f) + ((uint)(N.y * f + 32767.0f) << 16);

}

Packing

Advanced Graphics – Bits & Pieces 30

Efficiently Storing Normals

For an extensive study on this topic, see:

aras-p.info/texts/CompactNormalStorage.html

Bottom line:

We can encode a normal in 32-bit, and the precision will be excellent: expect approximately four digits of precision.

Pack:

float3 UnpackNormal( uint p )

{

float4 nn = make_float4( (float)(p & 65535) * (2.f / 65535.f), (float)(p >> 16) * (2.f / 65535.f), 0, 0 );

nn += make_float4( -1, -1, 1, -1 );

float l = dot( make_float3( nn.x, nn.y, nn.z ), make_float3( -nn.x, -nn.y, -nn.w ) );

nn.z = l, l = sqrtf( l ), nn.x *= l, nn.y *= l; return make_float3( nn ) * 2.0f + make_float3( 0, 0, -1 );

}

Packing

Advanced Graphics – Bits & Pieces 31

Today’s Agenda:

▪ On Offsetting Shadows and Reflections

▪ Consistent Normal Interpolation

▪ Path Classification

▪ Packing Normals

▪ Multiple Importance Sampling, One More Time

▪ Research Directions

MIS

Method 1: direct light sampling

▪ Samples only the part of the hemisphere occupied by unoccluded light sources.

▪ Does so by checking the visibility of a random point on a random light.

Method 2: BRDF sampling

▪ Samples only the part of the hemisphere not occupied by light sources

▪ Does so by rejecting samples that happen to hit a light source.

Method 1 and 2 can (should!) be summed to sample the full hemisphere exactly once.

Advanced Graphics – Bits & Pieces 33

MIS

−𝜋 𝜋

−𝜋 𝜋

𝑝𝑑𝑓 = ቐ

1

𝑠𝑜𝑙𝑖𝑑 𝑎𝑛𝑔𝑙𝑒0

angle

𝑎𝑟𝑒𝑎 = න−𝜋

𝜋

𝑐𝑜𝑠𝜃

𝑝𝑑𝑓 =cos 𝜃

𝜋

Advanced Graphics – Bits & Pieces 34

MIS

න 𝑓 𝑥 𝑑𝑥 = 𝐸 𝑓 𝑥 ≈1

𝑁

𝑖=1

𝑁

𝑓 𝑋𝑖

𝐸 𝑓 𝑥 ≈1

𝑁

𝑖=1

𝑁𝑓 𝑋𝑖

𝑝 𝑋𝑖

𝑙𝑎𝑟𝑔𝑒 𝑣𝑎𝑙𝑢𝑒

𝑠𝑚𝑎𝑙𝑙 𝑣𝑎𝑙𝑢𝑒= 𝑣𝑒𝑟𝑦 𝑙𝑎𝑟𝑔𝑒

𝑙𝑎𝑟𝑔𝑒 𝑣𝑎𝑙𝑢𝑒

𝑙𝑎𝑟𝑔𝑒 𝑣𝑎𝑙𝑢𝑒≈ 1

Advanced Graphics – Bits & Pieces 35

Using two pdfs:

randomly pick one.

MIS

𝐸 𝑓 𝑋 ≈1

𝑁

𝑖=1

𝑁𝑓 𝑋𝑖

𝑝 𝑋𝑖

𝐸 𝑓 𝑋 ≈

𝑖=1

𝑁1

𝑁𝑖

𝑗=1

𝑁𝑖

𝜔𝑖(𝑋𝑖,𝑗)𝑓 𝑋𝑖,𝑗

𝑝𝑖 𝑋𝑖,𝑗

This is what we did before.

Now, we combine the sampling techniques.

Weights: σ 𝜔𝑖 = 1

Most basic weight: 1

𝑁

(i.e., a weighted average of the techniques).

iterate over the available techniques / pdfs

draw 𝑁𝑖 samples using technique 𝑖

apply a weightto each sample

Advanced Graphics – Bits & Pieces 36

Weights: σ 𝜔𝑖 = 1, most basic weight: 1

𝑁. Can we do better?

Objective: minimize variance in the estimator.

How about this one:

This time, the weight is proportional to the pdf (!).

MIS

𝐸 𝑓 ҧ𝑥 ≈

𝑖=1

𝑁1

𝑁𝑖

𝑗=1

𝑁𝑖

𝜔𝑖(𝑋𝑖,𝑗)𝑓 𝑋𝑖,𝑗

𝑝𝑖 𝑋𝑖,𝑗

iterate over the available techniques

draw 𝑁𝑖 samples using technique 𝑖

apply a weightto each sample

𝜔𝑖(𝑋) =𝑁𝑖𝑝𝑖(𝑋)

σ𝑘 𝑁𝑘𝑝𝑘(𝑋)

… =

𝑖=1

𝑁

𝜔𝑖(𝑋𝑖)𝑓 𝑋𝑖

𝑝𝑖 𝑋𝑖

𝜔𝑖(𝑋) =𝑝𝑖(𝑋)

σ𝑘 𝑝𝑘(𝑋)

For two pdfs:

𝜔𝐴(𝑋) =𝑝𝐴(𝑋)

𝑝𝐴(𝑋) +𝑝𝐵 (𝑋)

𝜔𝐵(𝑋) =𝑝𝐵(𝑋)

𝑝𝐴(𝑋) +𝑝𝐵 (𝑋)

Let’s set 𝑁𝑖 to 1 to simplify things:

We are drawing 𝑋 from 𝑝𝐴, and we are looking up the same 𝑋 in 𝑝𝐵.

Advanced Graphics – Bits & Pieces 37

Summarizing:

𝐹 =

𝑖=1

𝑁

𝜔𝑖(𝑋𝑖)𝑓 𝑋𝑖

𝑝𝑖 𝑋𝑖= 𝜔𝐴 𝑋𝐴

𝑓 𝑋𝐴

𝑝𝐴 𝑋𝐴+ 𝜔𝐵 𝑋𝐵

𝑓 𝑋𝐵

𝑝𝐵 𝑋𝐵

where

𝜔𝐴(𝑋) =𝑝𝐴(𝑋)

𝑝𝐴 𝑋 +𝑝𝐵 𝑋

and

𝜔𝐵(𝑋) =𝑝𝐵(𝑋)

𝑝𝐴(𝑋) +𝑝𝐵 (𝑋)

So: for both sampling strategies, we add to the pdf the pdf of the other strategy for the same point on the hemisphere.

MIS

𝜔𝐴 𝑋𝐴

𝑓 𝑋𝐴

𝑝𝐴 𝑋𝐴

𝜔𝐵 𝑋𝐵

𝑓 𝑋𝐵

𝑝𝐵 𝑋𝐵=

𝑝𝐵(𝑋𝐵)

𝑝𝐴(𝑋𝐵) +𝑝𝐵 (𝑋𝐵)

𝑓 𝑋𝐵

𝑝𝐵 𝑋𝐵=

𝑓 𝑋𝐵

𝑝𝐴 𝑋𝐵 +𝑝𝐵 (𝑋𝐵)

=𝑝𝐴(𝑋𝐴)

𝑝𝐴(𝑋𝐴) +𝑝𝐵 (𝑋𝐴)

𝑓 𝑋𝐴

𝑝𝐴 𝑋𝐴=

𝑓 𝑋𝐴

𝑝𝐴 𝑋𝐴 +𝑝𝐵 (𝑋𝐴)

=𝑓 𝑋𝐴

𝑝𝐴 𝑋𝐴 + 𝑝𝐵(𝑋𝐴)+

𝑓 𝑋𝐵

𝑝𝐴(𝑋𝐵) + 𝑝𝐵 𝑋𝐵

Advanced Graphics – Bits & Pieces 38

Today’s Agenda:

▪ On Offsetting Shadows and Reflections

▪ Consistent Normal Interpolation

▪ Path Classification

▪ Packing Normals

▪ Multiple Importance Sampling, One More Time

▪ Research Directions

Topics

Path tracingBVHGPGPU

---------------

FilteringBlue noiseMaterials

---------------

Large scenesSubdivision surfaces, hair, displacement mapsMany lightsGeometry compression

Future Work

Advanced Graphics – Bits & Pieces 40

Today’s Agenda:

▪ On Offsetting Shadows and Reflections

▪ Consistent Normal Interpolation

▪ Path Classification

▪ Packing Normals

▪ Multiple Importance Sampling, One More Time

▪ Research Directions

INFOMAGR – Advanced GraphicsJacco Bikker - November 2018 – February 2019

END of “Bits & Pieces”next lecture: “Grand Recap”