OMNIVERSE RTX REAL-TIME RAY TRACER - NVIDIA · OMNIVERSE RTX REAL-TIME Lighting + Denoising based...

IGNACIO LLAMAS, 07/29/2019

OMNIVERSE RTX

REAL-TIME RAY TRACER

2

AGENDA

What is OMNIVERSE?

OMNIVERSE RTX

OMNIVERSE RTX Real-Time Ray Tracer

SIMULATING REALITY

Design Robotics Autonomous Vehicles Media & Entertainment

NVIDIA TECHNOLOGIES TOOLS

NVIDIA TECHNOLOGIES TOOLS

NVIDIA OMNIVERSE

MDLRealtime

Collaboration

Top Industry ToolsOpen Standards

CONNECTING ALL THE TOOLS TOGETHER THROUGH USD

USD HYDRA

RTX

RTX

60+ FPS multi-GPU Photoreal Rendering

Push limits of RTX, multi-GPU and Cloud

Omniverse Reference Renderer: USD+MDL

Target Platforms:

APIs: DX12/DXR, Vulkan/VKR, CUDA/OptiX

GPU: Pascal+ (DXR/VKR-enabled)

OS: Windows 10 1809+, Linux

Customers:

NV Robotics: Isaac Sim, Research

NV Auto: Drivesim, DriveAR

External Partners

OMNIVERSE RTX

What makes OMNIVERSE RTX unique?

Real-Time Gaming Renderers

Typically target wide range of devices

High end and multi-GPU not primary targets

Offline renderers

Minutes to hours per frame

Not designed for real-time animation and simulation

Traditionally two types of renderers

What makes OMNIVERSE RTX unique?

Target high-end Turing RTX GPUs

Multi-GPU across RTX Servers

Real-time animation and simulation

Hydra-based: run on any tool using Hydra

Built-in streaming even on headless Linux systems

Real-Time Photoreal Scalable Rendering

OMNIVERSE RTX

Lower CPU overhead [vs raster-based renderer]

Update only what changes every frame

Render 100s of small views (e.g.: for Robotics research)

MDL: Material Definition Language

Represent wide variety of materials

Facilitate material exchange across tools

Supports auto-differentiation: generates derivatives for texture sampling

Other Advantages

OMNIVERSE RTX

Render Modes

Real-Time Path Traced

OMNIVERSE RTX

Render Modes

Real-Time Path Traced

OMNIVERSE RTX

Render Modes

Real-Time Path Traced

OMNIVERSE RTX PATH TRACING

Reference Path Tracer

Using NVIDIA MDL SDK & OptiX AI Denoiser

Unidirectional

Next Event Estimation

Multiple-Importance Sampling

Support all light Primitives in USD

BSDF sampling & evaluation by MDL SDK

Volume Scattering + Absorption / SSS

Depth of Field

Our Reference

OMNIVERSE RTX REAL-TIME

Based on our work with Epic Games in UE4(see Ray Tracing Gems Chapter 19)

Ray Tracing and Denoising split light paths

Ambient Occlusion

Direct Light Visibility (Shadows)

Indirect Diffuse

Indirect Specular (Reflections)

Translucency

Splitting Light Paths

REAL-TIME RAY TRACINGSPLITTING THE LIGHT PATHS

Diffuse Reflectance * Ambient Light Ray Traced and Denoised Analytic LTC

REAL-TIME RAY TRACINGSPLITTING THE LIGHT PATHS

+ (Ray Traced and Denoised Ray Traced and Denoised Ray Traced and Denoised

REAL-TIME RAY TRACINGTRANSLUCENCY

OMNIVERSE RTX REAL-TIME

Real-Time target constrains rays and shading invocations per pixel

Naïve Path Tracing with 1-2 samples per pixel is hard to denoise

Splitting light paths

Easier to denoise a cleaner signal

Easier temporal reuse

Improved coherence

Why Split Light Paths?

OMNIVERSE RTX REAL-TIME

Lighting + Denoising based on our past work in UE4

Cinematic Rendering in UE4 with Real-Time Ray Tracing and Denoising, Edward Liu, Ignacio Llamas, Juan Cañada, Patrick Kelly

Direct Lighting based on Linear Transform of Cosines (LTC)(Heitz et al. 2016, 2017)

Ray Traced Reflections & Denoising borrows from Heitz et al 2018

Combining Analytic Direct Illumination and Stochastic Shadows, Eric Heitz, Stephen Hill and Morgan McGuire

Optional use of DDGI for Diffuse Indirect GI

Dynamic Diffuse Global Illumination, Majercik et al. 2019

Related Work

OMNIVERSE RTX REAL-TIME

Cinematic Lighting in Unreal EngineJerome Plateau, Ignacio Llamas, Marcus Wassmer, Arne Schober, GDC 2018

Ray Tracing in Games with NVIDIA RTXIgnacio Llamas, Edward Liu, GDC 2018

Ray Tracing with Low Sample CountsEdward Liu, SIGGRAPH 2018

Real-Time Global Illumination and CausticsJacopo Pantaleoni, SIGGRAPH 2018

Related Past Presentations

OMNIVERSE RTX DENOISING

NVRTD = NVIDIA Ray Tracing Denoising library

Evolution of GameWorks Ray Tracing Denoising libraries

Our real-time Ray Tracing Denoising algorithms in one library

DXBC and SPIR-V bytecode

Graphics API agnostic:

Target your graphics API abstraction, DX12 or Vulkan

Available to license for binary distribution in Q4 2019

Get source code access too: HLSL + C++

NVRTD

POST PROCESSING

DLSS or Temporal Anti-Aliasing

Depth of Field

FFT Bloom & Lens Flares

Chromatic Aberration

Tonemapping

ACES Color Correction

Film Grain

Finishing the Frame

REAL-TIME RAY TRACINGANIMATION

RAL-TIME RAY TRACINGANIMATION

OMNIVERSE RTX MULTI GPU

Real-Time Path Tracing with high SPP and OptiX AI Denoising

RTX Servers4 to 8 Quadro RTX GPUs per serverMellanox Infiniband NICs (e.g.: 100 Gbit)

Distribute Path Tracing work across many GPUs

Composite, AI Denoise and Post Process on master

Leverage NVIDIA HPC and DL scaling expertise

Near-linear scaling

SCALING ACROSS MULTIPLE RTX SERVERS

OMNIVERSE RTX NEXT

Very High Quality Real-Time PreVis with Multi-GPU Path Tracing

Multi-GPU (10-100x spp) + Temporal reuse (10x spp) + AI Denoising 2.0 (10x)

Continue improving sampling and denoising

Efficient many-light support: NVResearch (Pharr & Clarberg, 2019)

Dynamic Many-Light Sampling for Real-Time Ray Tracing

Fully compliant MDL support in real-time: lift limitations on number of BSDF layers

Post processing: Motion Blur, Reshade integration

And a lot more…

FEATURES AND PERF

OMNIVERSE & RTX AT SIGGRAPH 2019

Tuesday 11am - 12:15pm, Room 501AB, NVIDIA Presents:NVIDIA Omniverse: An Open, USD Based Collaboration Platform for Constructing and Simulating Virtual Worlds, Michael Kass, Frank DeLise, Tae-Yong Kim (NVIDIA)

Tuesday 2pm - 3pm, Room 501AB, NVIDIA Presents: Optimizing and Deploying Ray-Traced Dynamic Global Illumination, Morgan McGuire (NVIDIA)

Tuesday 2pm - 5:15pm, Room 408AB, SIGGRAPH Course: Open Problems in Real-Time Rendering, moderators: Natalya Tatarchuk (Unity Technologies), Aaron Lefohn (NVIDIA)

Wednesday 9am - 10:45am, Room 501AB, NVIDIA Presents: Ray Tracing Special Topics, Latest Research

Wednesday 10:45am - 12:15pm, Room 501AB, NVIDIA Presents: Advanced Real-Time Ray Tracing Tutorial, Pawel Koslowski, Evan Hart, Alex Dunn, Aurelio Reis (NVIDIA)

Wednesday noon - 12:25pm, NVIDIA booth #1313, Booth Talk: Collaborative Workflow for Media and Entertainment, Boris Ustaev (NVIDIA)

Wednesday 2pm - 5:15pm, Room 501AB, NVIDIA Presents: Ray Tracing Gems 1.1

Too Many more to list! See Eric Haine’s page for all the Ray Tracing content at SIGGRAPH 2019: http://www.realtimerendering.com/raytracing/roundup.html

FOUR OMNIVERSE-BASED DEMOS AT THE NVIDIA BOOTHAND DON’T MISS THESE TALKS

THANK YOU!