Queensland University of Technology CRICOS No. 000213J
INB382/INN382 Real-Time Rendering Techniques Lecture 9: Global
Illumination Ross Brown
Slide 2
CRICOS No. 000213J a university for the world real R Lecture
Contents Global Illumination Radiosity Pre-computed Radiosity
Transfer Spherical Harmonics
Slide 3
CRICOS No. 000213J a university for the world real R Global
Illumination There are one-story intellects, two-story intellects,
and three-story intellects with skylights. All fact collectors with
no aim beyond their facts are one-story men. Two-story men compare,
reason and generalize, using labours of the fact collectors as well
as their own. Three- story men idealize, imagine, and predict.
Their best illuminations come from above through the skylight. -
Oliver Wendell Holmes
Slide 4
CRICOS No. 000213J a university for the world real R Lecture
Context As mentioned before, we have covered the direct forms of
illumination on object surfaces Last week we moved further into
more indirect illumination issues by modelling effects such as
reflection and refraction We then presented a more generalised
approach known as ray tracing simple but costly not practical for
real-time just yet We now cover two relevant forms of global
illumination in real-time rendering Radiosity and Pre- computed
Radiance Transfer
Slide 5
CRICOS No. 000213J a university for the world real R Global
Illumination In most rendering, the use of a local lighting model
is the norm Meaning that only the surface data at the visible point
is used in the lighting calculations This is useful for
object-precision GPU systems you can easily parallelise SIMD
instructions on a stream of vertex and pixel data Data is discarded
once rendered Problematic for global illumination as we showed last
week
Slide 6
CRICOS No. 000213J a university for the world real R Global
Illumination Beginning We have covered reflection, refraction and
area lighting, which are global illumination techniques because the
rest of the scene influences the lighting on the object Lighting
needs to be thought of as the paths that the photons take from
light sources to the eye Local only acounts for one reflection from
the light source to the object, onto the eye
Slide 7
CRICOS No. 000213J a university for the world real R Global
Illumination Beginning Global illumination calculates the results
of the paths of photons from any reflecting object in the room
specular, transparent or diffuse The contributions are summed for
the object being rendered This process is contained within the
rendering equation developed by Kajiya [2] more later You can thus
sum all the possible light paths to the surface the more paths, the
greater the level of realism
Slide 8
CRICOS No. 000213J a university for the world real R Radiosity
and Ray Tracing Thus global illumination is divided into two main
approaches Radiosity calculation of light as radiation transfer
through volume of air Ray Tracing calculation of light as a set of
discrete ray samples through a volume We have covered the general
idea of ray tracing Now we illustrate the main principles of
Radiosity using a series of lectures notes from Brown University
[1]
Slide 9
CRICOS No. 000213J a university for the world real R Color
Bleeding Soft Shadows No Ambient Term View Independent Used in
other areas of Engineering Radiosity for Inter-Object Diffuse
Reflection
Slide 10
CRICOS No. 000213J a university for the world real R Pretty
Pictures Reality (actual photograph) Minus Radiosity Rendering
Equals the difference (or error) image
http://www.graphics.cornell.edu/online/box/compare.html Mostly due
to mis-calibration
Slide 11
CRICOS No. 000213J a university for the world real R 1.Model
scene as patches 2.Each patch has an initial luminance value: all
but luminaires (light sources) are probably zero 3.Iteratively
determine how much luminance travels from each patch to each other
patch until entire system converges to stable values We can then
render scene from any angle without recomputing these final patch
luminances they are viewer independent! The Radiosity Technique: An
Overview
Slide 12
CRICOS No. 000213J a university for the world real R Overview
of Radiosity The radiometric term radiosity means rate at which
energy leaves a surface Sum of rates at which surface emits energy
and reflects (or transmits) energy received from all other surfaces
Radiosity simulations are based on thermal engineering model of
emission and reflection of radiation using Finite Element Analysis
(FEA) First determine all light interactions in a view-independent
way, then render one or more views Consider room with only floor
and ceiling: floor ceiling
Slide 13
CRICOS No. 000213J a university for the world real R Overview
of Radiosity Suppose ceiling is actually a fluorescent drop- panel
ceiling which emits light Floor gets some of this light and
reflects it back Ceiling gets some of this reflected light and
sends it back Simulation mimics these successive bounces through
progressive refinement each iteration contributes less energy so
process converges
Slide 14
CRICOS No. 000213J a university for the world real R Kajiyas
[2] Rendering Equation Light energy travelling from point i to j is
equal to light emitted from i to j, plus the integral over S (all
points on all surfaces) of reflectance from point k to i to j,
times the light from k to i, all attenuated by a geometry factor
L(i j) is the amount of light travelling along the ray from point i
to point j L e is the amount of light emitted by the surface
(luminance) f(k i j, ) is the Bidirectional Reflectance
Distribution Function (BRDF) of the surface. Describes how much of
the light incident on the surface at i from the direction of k
leaves the surface in direction of j is wavelength of light, use a
different function for R, G, & B G(i j) is a geometry term
which involves occlusion, distance, and the angle between the
surfaces
Slide 15
CRICOS No. 000213J a university for the world real R Kajiyas
Rendering Equation More complete model of light transport than
either raytracing or polygonal rendering, but does omit some
things, eg., subsurface scattering How do we evaluate this
function? very difficult to solve complicated integral equations
analytically
Slide 16
CRICOS No. 000213J a university for the world real R A scene
has: geometry luminaires (light sources) observation point Light
transport to camera must be computed for every incoming angle to
observation point, bouncing off all geometry (in all directions
combined) This is far too hard Rendering a Scene
Slide 17
CRICOS No. 000213J a university for the world real R Rendering
a Scene Both raytracing and radiosity are crude approximations to
rendering equation Raytracing: consider only a very small, finite
number of rays and ignore diffuse inter-object reflections, perhaps
using ambient hack to approximate that lost component. Can use
either a BRDF or a simpler (e.g., Phong) model for luminaires
Radiosity: approximate integral over differential source areas
dA(i) with finite sum over finite areas; consider light transport
not on basis of individual rays but on basis of energy transport
between finite patches (e.g., quads or triangles that result from
(adaptive) meshing) remember Lecture 5???
Slide 18
CRICOS No. 000213J a university for the world real R Adaptive
Meshes for Radiosity
Slide 19
CRICOS No. 000213J a university for the world real R What Can
We Do? Design an alternative simulation of real transfer of light
energy With any luck, will be more accurate, but accuracy is
relative hall of mirrors is specular raytracing museum with
latex-painted walls is diffuse radiosity Best solutions are hybrid
techniques: use raytracing for specular components and radiosity
for diffuse components
Slide 20
CRICOS No. 000213J a university for the world real R What Can
We Do? Radiosity approximates global diffuse inter- object
reflection by considering how each pair of surface elements
(patches) in scene send and receive light energy, an O(n 2 )
operation best accomplished by progressive refinement
Slide 21
CRICOS No. 000213J a university for the world real R Energy =
light energy = radiosity for our purposes it should really be rate,
ie., energy/unit time E i : initial amount of energy radiating from
i th patch B i : final amount of energy radiating from i th patch B
j : final amount of energy radiating from j th patch F j-i :
fraction of energy B i emitted by j th patch that is gathered by i
th patch (relationship between i th and j th patches based on
geometry: distance, relative angles) F j-i B j : total amount of
patch j 's energy sent to patch i i : fraction of incoming energy
to a patch that is then exported in next iteration Some Important
Symbols
Slide 22
CRICOS No. 000213J a university for the world real R Amount of
light/radiosity/energy a patch finally emits is initial emission
plus sum of emissions due to other n-1 patches in scene emitting to
this patch - recursive definition. Note: F 1-1 is zero only for
planar patches! Why is it zero for only planar patches? or Sender 1
Sender 2 Sender n Receiver patch i Lets Arrange Those Symbols
Slide 23
CRICOS No. 000213J a university for the world real R Arranging
Those Symbols Thus: Rewrite as a vector product: And the whole
system:
Slide 24
CRICOS No. 000213J a university for the world real R Decompose
the first matrix as: Can be rewritten: (I D( )F)B = E where D( ) is
diagonal matrix with i as its ith diagonal entry, and F is called a
Form Factor Matrix and is based on geometry between patches If we
know E, D( ), and F, we can determine B If we let A = I D( )F
Arranging Those Symbols Some More
Slide 25
CRICOS No. 000213J a university for the world real R Arranging
Those Symbols Even More Then we are solving (for B ) the equation
AB = E This is a linear system, and methods for solving these are
well-known, e.g. Gaussian elimination or Gauss-Seidel iteration
(although which method is best depends on nature of matrix A) part
of introductory linear algebra courses Typically want B, knowing E
and A
Slide 26
CRICOS No. 000213J a university for the world real R Let us
look at the floor ceiling problem again Both ceiling and floor act
as lights emitting and reflecting light uniformly over areas (all
surfaces considered such in radiosity) C emits 12 F gets 1/3 C F
reflects 50% C gets 1/3 F C reflects 75% Progressive
Refinement
Slide 27
CRICOS No. 000213J a university for the world real R
Progressive Refinement Let ceiling emit 12 units of light per
second Let floor reflect get 1/3 of light from ceiling (based on
geometry) reflect 50% of what it gets Let ceiling get 1/3 of floors
light (based on geometry), and reflect 75% of what it gets Writing
B 1 for ceilings total light, and B 2 for floors, and E 1 and E 2
for light generated by each: Ceiling: Floor:
Slide 28
CRICOS No. 000213J a university for the world real R
Progressive Refinement thus: B 1 = E 1 + 1 (F 2 -1(E 2 + 2 (F 1 -2B
1 ))) becoming: B 1 = E 1 + 1 F 2 -1E 2 + 1 2 F 1 -2F 1 -2B 1 which
simplifies to: In general, this algebra is too complex, but we can
find the solution iteratively using progressive refinement
Slide 29
CRICOS No. 000213J a university for the world real R Iterative
method 1, gathering energy: send out light from emitters
everywhere, accumulate it, resend from all patches Each iteration
uses radiosity values from previous iteration as estimates for
recursive form. Iterate by rows. B k = E + D(r)FB k-1 B 1 = E B 2 =
E + D(r)FB 1 B 3 = E + D(r)FB 2 Where B k is your k th guess at
radiosity values B 1, B 2 Progressive Refinement
Slide 30
CRICOS No. 000213J a university for the world real R Results
for our example (notice what happens to values): {12, 0} = {B 1, B
2 }= {E 1, E 2 } {12, 2} = {12.5, 2} = {12.5, 2.08373} = {12.5208,
2.08373} {12.5208, 2.08681} {12.5217, 2.08681} {12.5217, 2.08695}
Progressive Refinement tolerance; // ie. Matrix has not changed
Radiosity Pseudocode">
CRICOS No. 000213J a university for the world real R Algorithm
for fast progressive refinement through shooting: U 0 = e; // Set
Matrix to initial emission values U = unshot energy B 0 = e; t = 1;
do { i = index_of(MAX(u t-1 )); // Max row precalculate hemicube i
; // Shoot the radiance to each row for (j = 1; j < n; ++) { b j
t = u i t-1 F i-j A i /A j + b j t-1 ; u j t = u i t-1 F i-j A i /A
j + u j t-1 ; } u i t = u i t-1 F j-j ; ++t; } while B t -B t-1
> tolerance; // ie. Matrix has not changed Radiosity
Pseudocode
Slide 49
CRICOS No. 000213J a university for the world real R Assumption
that radiation is uniform in all directions Assumption that
radiosity is piecewise constant usual renderings make this
assumption, but then interpolate cheaply to fake a nice-looking
answer this introduces quantifiable errors Computation of form
factors F i-j can be tough especially with intervening surfaces,
etc. Assumption that reflectivity is independent of directions to
source and destination Limitations of Radiosity
Slide 50
CRICOS No. 000213J a university for the world real R
Limitations of Radiosity No volumetric objects (though there are
equations and algorithms for calculating surface-to-volume form
factors) No transparency or translucency Independence from
wavelength no fluorescence or phosphorescence Independence from
phase no diffraction Enormity of matrices! For large scenes, 10K x
10K matrices are not uncommon (shooting reduces need to have it all
memory resident)
Slide 51
CRICOS No. 000213J a university for the world real R More
Comments Even with these limitations, it produces great pictures
For n surface patches, we have to build an n x n matrix and solve
Ax = b, which takes O(n 2 ), this gets rather expensive for large
scenes Could we do it in O(n) instead? The answer, for lots of nice
scenes, is Yes The Google search engine uses an system much like
radiosity to rank its pages Site rankings are determined not only
by the number of links from various sources, but by the number of
links coming into those sources (and so on) After multiple
iterations through the link network, site rankings stabilize Site
importance is like luminance, and every site is initially
considered an emitter
Slide 52
CRICOS No. 000213J a university for the world real R One
approach is importance driven radiosity: if I turn on a bright
light in the graphics lab with the door open, itll lighten my
office a little but not much By taking each light source and asking
whats illuminated by this, really? we can follow a shooting
strategy in which unshot radiosity is weighted by its importance,
i.e., how likely it is to affect the scene from my point of view No
longer a view-independent solutionbut much faster Making Radiosity
Fast
Slide 53
CRICOS No. 000213J a university for the world real R
Precomputed Radiance Transfer PRT Imagine that time is stationary.
Picture a volume of space filled with photons each cube of space
can be said to have a constant photon density. Picture this field
of photons in linear motion We need to find out how many photons
collide with a stationary surface for each unit of time, a value
called flux which measured in joules/second or watts Divide the
flux by the differential area, we get a value called the
irradiance, measured in watts/meter 2 We thus model this using
Spherical Harmonic functions rest of PRT based on paper - Spherical
Harmonic Lighting: The Gritty Details, Robin Green[4]
Slide 54
CRICOS No. 000213J a university for the world real R Spherical
Harmonics (SH) SH lighting paper assumes knowledge of the use of
Basis Functions. Basis Functions are small pieces of signal that
can be scaled and combined to produce an approximation to original
function Process of working out how much of each basis function to
sum is called Projection. To approximate a function using basis
functions we must work out a scalar value that represents how much
the original function f(x) is like the each Basis Function B i (x).
We do this by integrating the product f(x)B i (x) over the full
domain of f use summation for an approximation
Slide 55
CRICOS No. 000213J a university for the world real R Legendre
Polynomials The associated Legendre polynomials are at the heart of
the Spherical Harmonics, a mathematical system analogous to the
Fourier transform but defined across the surface of a sphere SH
functions in general are defined on Imaginary Numbers but we are
only interested in approximating real functions over the sphere
(i.e. light intensity fields) Lecture will be working only with the
Real Spherical Harmonics
Slide 56
CRICOS No. 000213J a university for the world real R SH series
for varying values of l and m
Slide 57
CRICOS No. 000213J a university for the world real R Projecting
into SH Space Note how the first band is just a constant positive
value If you render a self-shadowing model using just the 0- band
coefficients the resulting looks just like an accessibility shader
with points deep in crevices (high curvature) shaded darker than
points on flat surfaces See Lecture 8 The l = 1 band coefficients
cover signals that have only one cycle per sphere and each one
points along the x, y, or z-axis Linear combinations of just these
functions give us very good approximations to the cosine term in
the diffuse surface reflectance model
Slide 58
CRICOS No. 000213J a university for the world real R Projection
Process Process for projecting a spherical function into SH
coefficients is simple Calculate a single coefficient for a
specific band you just integrate the product of your function f and
the SH function y Working out how much your function is like the
basis function Slide 58
Slide 59
CRICOS No. 000213J a university for the world real R Example
Complex SH Function Approximations
Slide 60
CRICOS No. 000213J a university for the world real R
Approximating Lighting using SH To project a function into SH
coefficients we want to integrate the product of the function and
an SH function We must evaluate this integral using Monte Carlo
integration where x j is our array of pre- calculated samples
(Slide 59) and the function f is the product f(x j ) = light(x j )y
i (x j ). An example lighting function displayed as a color (left)
and a spherical plot (right) Example (left) of a physically sampled
scene using a spherical silver ball and a camera
Slide 61
CRICOS No. 000213J a university for the world real R Spherical
Harmonic Sampling Basis Uses a set of ortho-normal basis functions
on the surface of a sphere just like X,Y,Z axes Rendering equation
that we want to integrate over the surface of a sphere So all we
need to do is generate evenly distributed points (more technically
called unbiased random samples) over the surface of a sphere.
Taking a pair of independent canonical random numbers x and y we
can map this square of random values into spherical coordinates
using the transform This forms a light sample basis for the later
calculations
Slide 62
CRICOS No. 000213J a university for the world real R Generating
SH Coefficients Applying this process to the light source we
defined earlier with 10,000 samples over 4 bands gives us this
vector of coefficients: [ 0.39925, - 0.21075, 0.28687, 0.28277, -
0.31530, - 0.00040, 0.13159, 0.00098, - 0.09359, - 0.00072,
0.12290, 0.30458, - 0.16427, - 0.00062, - 0.09126 ] Reconstructing
the SH functions for checking purposes from these 15 coefficients
is simply a case of calculating a weighted sum of the basis
functions: An example approximated lighting function displayed as a
color and a spherical plot
Slide 63
CRICOS No. 000213J a university for the world real R Can also
model shadows with SH The transfer function (occlusion factor) in
the lighting model can also be stored as a 16 coefficient SH
function As you can see this is a different way of recording
shadowing at a point on a model without forcing us to do the final
integral Can rotate object and still get correct values for shadows
on surface without recasting the rays an improvement on last
lecture approach Light Distribution Final Light Distribution
Transfer (Occlusion)
Slide 64
CRICOS No. 000213J a university for the world real R Indirect
Lighting Steps Geometrically, the idea of interreflected light is
simple Each point on the model already knows how much direct
illumination it has, encoded in the form of an SH transfer function
Fire rays to find sample points that can reflect light back onto
our position and add a cosine weighted copy of that transfer
function back into our own For example, point A in the illustration
above has fired a ray and hit point B
Slide 65
CRICOS No. 000213J a university for the world real R Shadowed
Indirect Lighting A rendering using 5th order diffuse shadowed SH
transfer functions Note the soft shadowing from the constant
hemisphere light source NB: This is how light probes work in Unity
5
Slide 66
CRICOS No. 000213J a university for the world real R Real-Time
Rendering using SH Now we have a set of SH coefficients for each
vertex, how do we build a renderer using current graphics hardware
that will give us real-time frame rates? Basic calculation for SH
lighting is the dot product between an SH projected light source
and the SH transfer function vertex Approximate the complete
solution over an object by filling in the gaps between vertices
using Gouraud shading Typically only need 4 coefficients per vertex
one extra 4 value colour
Slide 67
CRICOS No. 000213J a university for the world real R Real-Time
Rendering using SH Can rotate the SH coefficients so that the
object is able to have its first pass lighting updated in real time
Now have a cheap rendering technique for generalised area light
sources in a scene Can add a specular term and/or environment maps
to give required appearance
Slide 68
CRICOS No. 000213J a university for the world real R Direct X
Browser PRT Demonstration Within the DirectX SDK is a demo browser
Run the PRT demo to see similar to the movie on the left Choose
scene 4 to obtain the head figure
Slide 69
CRICOS No. 000213J a university for the world real R Radiosity
and Unity 5 For years this has been a grand challenge of CG, to run
radiosity in real time Enlightens implementation [3] performs a
partial version in real time on a tablet in Unity 5!!! See here
https://www.youtube.com/w
atch?v=Wrt5aLHI8MEhttps://www.youtube.com/w atch?v=Wrt5aLHI8ME
Slide 70
CRICOS No. 000213J a university for the world real R Unity 5 GI
Basics [5] Unity GI performed in background and baked on for static
objects Precomputed Realtime GI encodes all possible bounces into
lightmap data structure texture [6] Directional Light is bounced
into the scene via this data structure in real time due to no
position requirements
Slide 71
CRICOS No. 000213J a university for the world real R Dynamic
Objects and GI Dynamic objects cannot cast light into the scene But
they can use light probes to sample the bouncing light in the scene
to be lit themselves Light probes are placed where dynamic objects
are moving and are a spherical panoramic view of the
environment
Slide 72
CRICOS No. 000213J a university for the world real R Dynamic
Objects and GI Light probes are stored as Spherical Harmonic
approximations Interpolated internally to light dynamic object at
that point Indirect, probe and direct light blended to provide
final lighting model in scene
Slide 73
CRICOS No. 000213J a university for the world real R Unity 5
Demonstration Video Video: https://youtu.be/G-dpRCdfFc8
Slide 74
CRICOS No. 000213J a university for the world real R References
1.John F. Hughes, Andries van Dam, Introduction to Radiosity,
www.cs.brown.edu/courses/cs123/lectures.shtml 29/04/2007 2.J
Kajiya. The Rendering Equation. SIGGRAPH 1984, pp. 143- 150
3.www.geomerics.com 29/04/2007 4.Robin Green, Spherical Harmonic
Lighting: The Gritty Details -
www1.cs.columbia.edu/~cs4162/slides/spherical-harmonic-
lighting.pdf
www1.cs.columbia.edu/~cs4162/slides/spherical-harmonic-
lighting.pdf 5.http://docs.unity3d.com/Manual/GIIntro.html
04/05/2015http://docs.unity3d.com/Manual/GIIntro.html 04/05/2015
6.http://www.geomerics.com/wp-
content/uploads/2014/03/radiosity_architecture.pdfhttp://www.geomerics.com/wp-
content/uploads/2014/03/radiosity_architecture.pdf