Enhancing Traditional Rasterization Graphics with Ray Tracingcdn.imgtec.com/sdk-presentations/IDC2015_UK... · Ray tracing is emerging as the next big thing in real-time graphics

Enhancing Traditional Rasterization Graphics with Ray Tracing

October 2015

James Rumble Developer Technology Engineer, PowerVR Graphics

3 @PowerVRInsider │ #idc15 facebook.com/imgtec

Overview

• Ray Tracing Fundamentals

• PowerVR Ray Tracing Pipeline

• Using Ray Tracing


Overview





Introduction to Ray Tracing

Ray Tracing is not a new concept!

• Synonymous with photorealistic (offline) renders

• Traditionally too resource intensive for broad industry applications

Ray tracing is emerging as the next big thing in real-time graphics

• Used in a limited fashion i.e. reflections and shadows

• Often limited to very few rays and low resolution/course grained scene representations

• The benefits ray tracing can offer are obvious


Ray Tracing Fundamentals

RAY = {origin, direction, clipping distance}

• Triangles

• Rays

• Intersection Testing

• Process of determining if a given ray

has intersected a given triangle


Tracing Rays

• Commonly visualised and explained as a

process of modelling light

• Rays used to model real-world behaviour

of light

• Forward Ray Tracing:

• Rays traced from the light source

• Many rays never reach our eyes

• Backward Ray Tracing:

• Rays traced from the “eye” towards light

Forward Ray Tracing vs Backward Ray Tracing


What is Ray Tracing? Backwards Ray Tracing

• How is ray tracing

commonly used?

• ‘Primary’ rays fired from

“eye” through pixels of

the screen

• Finds closest triangle

intersection

• ‘Secondary’ Rays used for

additional effects


Ray Tracing Basics

• Now using a concrete example…

• Cast primary rays out into the world

through each pixel

• Find the closest triangle intersection

• ‘Secondary’ Rays used for

additional effects (direct lighting)

Backwards Ray Tracing


What is Ray Tracing?

• Can be used in a more abstract

way than ‘simply’ modelling light

• Can be used to calculate spatial

queries between rays and

objects/triangles

• i.e. Does ray x intersect object 1

• A ray can be fired as long it has

been given an origin and direction

Fire a ray from a box towards a

neighbouring object. It intersects

the plane in between

Fire a ray from a box towards a

neighbouring object. It reaches

the other box and is marked as

intersected

Spatial Queries


What is Ray Tracing?

• Ray Tracing provides a mechanism for determining inter-object

visibility using spatial queries

• Objects can be aware of both their own properties and position but

now also those of other objects in the scene through the use of

rays

• Provides elegant solutions to many traditionally difficult to render

or process effects


Overview





Introducing PowerVR Ray Tracing

• PowerVR Ray Tracing GPUs are built on top of the latest

generation of PowerVR GPUs

• Additional ray tracing specific hardware designed to integrate

efficiently and communicate perfectly with the rest of the GPU

architecture

• Real-time, interactive ray traced graphics that can comfortably

scale from mobile and embedded platforms to gaming consoles

and mainstream gaming PCs


PowerVR Ray Tracing Pipeline

Scene Hierarchy Building Phase

Vertex

Processing

Hierarchy

Generation

Geometry

Data

Memory

Scene

Hierarchy

Ray Traversal Phase

Primary Ray

Generation

Intersection

TestsRay Shaders Accumulation


Scene Hierarchy Organises triangles based on their size and position


Vertex

Processing

Hierarchy

Generation

Memory

Scene

Hierarchy

Geometry Submission

Scene

ComponentsScenes

Geometry

Data


Scene Hierarchy Generation Geometry submission


Vertex

Processing

Hierarchy

Generation

Memory

Scene

Hierarchy

Ray Traversal Phase

Primary Ray

Generation

Intersection


Geometry Submission

Scene

ComponentsScenes

Geometry

Data


Scene Hierarchy Generation Scene Components

Windshield

Scene Component:

- Program:

- Vertex Shader

- Ray Shader

- Uniforms

- Geometry Data

- Buffers

- Textures

Car Body

Scene Component:

- Program:

- Vertex Shader

- Ray Shader

- Uniforms

- Geometry Data

- Buffers

- Textures

Headlights

Scene Component:

- Program:

- Vertex Shader

- Ray Shader

- Uniforms

- Geometry Data

- Buffers

- Textures

Tyres

Scene Component:

- Program:

- Vertex Shader

- Ray Shader

- Uniforms

- Geometry Data

- Buffers

- Textures

Chrome

Scene Component:

- Program:

- Vertex Shader

- Ray Shader

- Uniforms

- Geometry Data

- Buffers

- Textures


Scene Hierarchy Generation

• Contain groups of scene components

• Ray traversal occurs within scene

World Scenes

Windshield

Scene Component:

- Program:

- Vertex Shader

- Ray Shader

- Uniforms

- Geometry Data

- Buffers

- Textures

Car Body

Scene Component:

- Program:

- Vertex Shader

- Ray Shader

- Uniforms

- Geometry Data

- Buffers

- Textures

Headlights

Scene Component:

- Program:

- Vertex Shader

- Ray Shader

- Uniforms

- Geometry Data

- Buffers

- Textures

Tyres

Scene Component:

- Program:

- Vertex Shader

- Ray Shader

- Uniforms

- Geometry Data

- Buffers

- Textures

Chrome

Scene Component:

- Program:

- Vertex Shader

- Ray Shader

- Uniforms

- Geometry Data

- Buffers

- Textures

Windshield

Scene Component:

- Program:

- Vertex Shader

- Ray Shader

- Uniforms

- Geometry Data

- Buffers

- Textures

Car Body

Scene Component:

- Program:

- Vertex Shader

- Ray Shader

- Uniforms

- Geometry Data

- Buffers

- Textures

Headlights

Scene Component:

- Program:

- Vertex Shader

- Ray Shader

- Uniforms

- Geometry Data

- Buffers

- Textures

Tyres

Scene Component:

- Program:

- Vertex Shader

- Ray Shader

- Uniforms

- Geometry Data

- Buffers

- Textures

Chrome

Scene Component:

- Program:

- Vertex Shader

- Ray Shader

- Uniforms

- Geometry Data

- Buffers

- Textures

Windshield

Scene Component:

- Program:

- Vertex Shader

- Ray Shader

- Uniforms

- Geometry Data

- Buffers

- Textures

Car Body

Scene Component:

- Program:

- Vertex Shader

- Ray Shader

- Uniforms

- Geometry Data

- Buffers

- Textures

Headlights

Scene Component:

- Program:

- Vertex Shader

- Ray Shader

- Uniforms

- Geometry Data

- Buffers

- Textures

Tyres

Scene Component:

- Program:

- Vertex Shader

- Ray Shader

- Uniforms

- Geometry Data

- Buffers

- Textures

Chrome

Scene Component:

- Program:

- Vertex Shader

- Ray Shader

- Uniforms

- Geometry Data

- Buffers

- Textures


Scene Hierarchy Generation Vertex processing


Hierarchy

Generation

Memory

Scene

Hierarchy

Ray Traversal Phase

Primary Ray

Generation

Intersection


Geometry

Data

Vertex

Processing


Scene Hierarchy Generation

• Vertex shader used to transform geometry to world space

Vertex processing


Scene Hierarchy Generation Hierarchy generation


Memory

Scene

Hierarchy

Ray Traversal Phase

Primary Ray

Generation

Intersection


Geometry

Data

Vertex

Processing

Hierarchy

Generation


Scene Hierarchy Generation Hierarchy generation

• Vertex shader output is passed to a fixed function scene generator which

calculates the scene hierarchy


Scene Ray Traversal


Vertex

Processing

Hierarchy

Generation

Geometry

Data

Memory

Scene

Hierarchy

Ray Traversal Phase

Primary Ray

Generation

Intersection



Scene Ray Traversal Primary ray generation


Vertex

Processing

Hierarchy

Generation

Geometry

Data

Memory

Scene

Hierarchy

Ray Traversal Phase

Intersection


Primary Ray

Generation


Scene Ray Traversal

• Frame Shaders are run once

per pixel to emit primary rays

into the scene

Primary ray generation


Scene Ray Traversal Intersection tests


Vertex

Processing

Hierarchy

Generation

Geometry

Data

Memory

Scene

Hierarchy

Ray Traversal Phase

Ray Shaders AccumulationPrimary Ray

Generation

Intersection

Tests


Scene Ray Traversal Intersection tests

• Simple approach – each ray tested

against all triangles:

• Brute force

• m * n intersection tests (m – rays, n – objects)

• Use accelerated data structure to

reduce intersection tests

• Application developer does not need to worry

about handling this

• Accelerated triangle and box testing


Scene Ray Traversal

• Simple approach – each ray tested

against all triangles:

• Brute force

• m * n intersection tests (m – rays, n – objects)

• Use accelerated data structure to

reduce intersection tests

• Fixed function hardware accelerated triangle and

box testing

Intersection tests


Scene Ray Traversal Ray shaders and accumulation


Vertex

Processing

Hierarchy

Generation

Geometry

Data

Memory

Scene

Hierarchy

Ray Traversal Phase

Intersection

Tests

Primary Ray

GenerationRay Shaders Accumulation


Scene Ray Traversal

• Ray Shaders:

• Defined for each scene

component and invoked when

a ray intersects a triangle

• Defines material properties of

component (fully

programmable)

• Can emit further rays causing

additional intersection tests

Ray shaders and accumulation

accumulate(0.85, 0.85, 0.75);

reflect(incidentDirection, normal)


Types of Rays are important

• Rays are a communication medium making inter-object visibility

possible

• An engine will likely use multiple “types” of ray

• Primary Rays – If it hits anything shade it / emit additional rays

• Shadow Rays – if it doesn’t hit a light – kill it

• etc.


Coherence-Gathering

• What if these rays could be automatically coalesced?


Pipeline Summary


Vertex

Processing

Hierarchy

Generation

Memory

Scene

Hierarchy

Ray Traversal Phase

Primary Ray

Generation

Intersection


Geometry Submission

Scene

ComponentsScenes

Geometry

Data


Overview





How do we use it?

Hybrid Shadows,

Reflections, etc.

Production-

Quality Renders Ambient

Occlusion

Physics &

Collision

Detection

A.I. & Line of

Sight

Calculations

Global Illumination Virtual Reality

Lens correction, Ultra-low latency

rendering, Lenticular Displays

Asset creation /

compression

Rapid photo-

quality output

Order-Independent

Transparency

Augmented

Reality


Render the scene

into a deep frame

buffer (G-Buffer)

Compute Lighting on

each G-Buffer pixel

(No Overdraw)

Assemble the final

frame and perform

screen-space effects

G-Buffer

Material IDs

Normals

World-Space Positions or depth buffer

Final Result Frame

Lighting Buffer(s)

Lighting calculations executed in screen

space

Modern Raster-Based Game Engine Deferred shading used in many modern game engines


Ray Trace Using the G-Buffer Who needs primary rays when we have rasterization

G-Buffer

Material IDs

Normals

World Space Positions or Depth values

(1.0,0.0,0.0)

(0.5,0.6,-0.2)

(14.3,81.6,-0.4)


Render the scene

into a deep frame

buffer (G-Buffer)

Trace Rays to

determine lighting

Assemble the final

frame and perform

screen-space effects

Hybrid Ray Tracing Use the G-Buffer to set up your rays

G-Buffer

Material IDs

Normals

World-Space Positions or depth buffer

Final Result Frame

Lighting Buffer(s)


Ray traced Cascaded shadow maps

Ray Traced Techniques Analytical 1 Ray Per Pixel Soft Shadows

http://cdn.imgtec.com/sdk-presentations/idc15_SF_EnhancingTraditionalRasterizationGraphicsWithRayTracing.pdf

http://cdn.imgtec.com/sdk-presentations/idc15_SF_EnhancingTraditionalRasterizationGraphicsWithRayTracing.pdfhttp://cdn.imgtec.com/sdk-presentations/idc15_SF_EnhancingTraditionalRasterizationGraphicsWithRayTracing.pdfhttp://cdn.imgtec.com/sdk-presentations/idc15_SF_EnhancingTraditionalRasterizationGraphicsWithRayTracing.pdfhttp://cdn.imgtec.com/sdk-presentations/idc15_SF_EnhancingTraditionalRasterizationGraphicsWithRayTracing.pdf


Direct Lighting Only Ray Traced Global Illumination

Ray Traced Techniques Low Overhead Probe – based Global Illumination

http://cdn.imgtec.com/sdk-presentations/idf15_SF_LowOverheadProbeBasedGlobalIlluminationUsingRayTracing.pdf

http://cdn.imgtec.com/sdk-presentations/idf15_SF_LowOverheadProbeBasedGlobalIlluminationUsingRayTracing.pdfhttp://cdn.imgtec.com/sdk-presentations/idf15_SF_LowOverheadProbeBasedGlobalIlluminationUsingRayTracing.pdfhttp://cdn.imgtec.com/sdk-presentations/idf15_SF_LowOverheadProbeBasedGlobalIlluminationUsingRayTracing.pdfhttp://cdn.imgtec.com/sdk-presentations/idf15_SF_LowOverheadProbeBasedGlobalIlluminationUsingRayTracing.pdf


Summary

• An introduction to ray tracing

• The PowerVR Ray Tracing Pipeline

• Hybrid Ray Tracing

• Ray Traced Technique Teaser

Documents

Enhancing Traditional Rasterization Graphics with Ray Tracingcdn.imgtec.com/sdk-presentations/IDC2015_UK... · Ray tracing is emerging as the next big thing in real-time graphics