NVIDIA CLOUDXR - on-demand.gputechconf.comDecode HEVC Frame Sensor & input data RGB Frame Audio...

David Weinstein, 19 December 2019

NVIDIA CLOUDXR™

2

Gaming

Movies

Sports

Concerts

Travel Retail

PERSONAL ENTERTAINMENT

VR

3NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.

QUADRO PROFESSIONAL VR/AR

MEDICAL ARCHITECTURE

DESIGN MANUFACTURING COLLABORATION

DIGITAL CONTENT CREATION

4

5

6

7

FUNDAMENTAL VR GRAPHICS CHALLENGES

TRADITIONAL = 60 MP/S(1920 X 1080 @ 30 FPS)

1920

1080

VIRTUAL REALITY = 450 MP/S(3024 X 1680* @ 90 FPS)

1512

1680

1512 Motion to Photon: ≤ 20 ms

Very High Pixel Rendering Throughput Must Maintain Very Low Latency

Pixel Throughput and Perceived Latency

HMD updates pose

CPU triggers redraw

GPU computes new image

HMD flashes display

8

WANTED: THE BEST OF BOTH WORLDS

Highest Quality Graphics + Mobile Freedom

9

CLOUDXR VISION

RTX ServersContent Network

Clients

XR Streaming to Any Device

10

MOBILE WORLD CONGRESS (LA), OCT 2019

CloudXR Announcement

11

ARCHITECTURE OVERVIEWServer-Side Driver, Client-Side Application and SDK

OpenVR Runtime (Valve)

VR App

Virtual HMD Driver

Cloud

Late & Lens Warp

VR Runtime

Video Decoder

Cloud

HMD

Video Encoder

Virtual Audio Driver

Audio Encoder Audio Decoder

InternetSERVER driver (Nvidia) CLIENT app (Nvidia)

audio, video, haptics

pose, control inputs, video

12

ARCHITECTUREWIN SERVER CLIENT

INST

ALL

ER

Audio/Video

Data (sensors, inputs, QoS)

Encode

RGB Frame

Display

VR App

Decode

Vid

eo P

ara

ms

(e.g

. RO

I, o

pti

on

al)

HEVC Frame HEVC Frame

VR Runtime

Controller & inputs

Sensor,Pose &Controller Data

RGB Frame

Sensor & input data

Sensor & input data

Audio

(e.g., OpenVR)

Ap

p

Clie

nt

app

Serv

er d

rive

r

Time Warp

Lens Warp

RGB Frame

Driver API

VR Runtime

Runtime API

OpenVR Interfaces

HMD

Au

dio

Pla

ybac

k

Audio Capture / Encode

13

QOS

Relevant Internet problems:

• Jitter

• Missed packets

• Bandwidth variation

Reliability & Resilience

14

QOS

Reliability & Resilience

Challenges Solutions Control parameters

Latency Custom HEVC profile NVENC knobs

Jitter De-jitter buffer Buffer size

Missed packets Forward error correction Correction strength

Bandwidth variation Video compression Compression rate

Server has no display Frame pacing to match

client display rate

Server render rate

15

WIN SERVER

VR App

Vid

eo P

ara

ms

(e.g

. RO

I, o

pti

on

al)

Ap

p

Driver API

VR Runtime

Runtime API

ARCHITECTURE

INST

ALL

ER

Audio / Video / Data (sensors, inputs, QoS)

Encode

RGB Frame

Decode

HEVC Frame

Sensor,Pose &Controller data

Sensor & input data

Audio

Clie

nt

app

Serv

er d

rive

r

Audio Capture / Encode

Au

dio

Pla

ybac

k

Frame Pacing

Jitter FIFO

CLIENT

Display

VR Runtime

Controller & inputs

(e.g., OpenVR)Time Warp

Lens Warp

HMD

OpenVR Interfaces

16

LATENCY

Using:

• HelloVR sample modified to change color on button press

• Latency meter (instrumented switch + photodiode)

• Local Cloud with Client on WiFi (average ping time 4 ms)

Button to Photons

17

LATENCY

Results:

Server Driver + Client App latency = 12.1 ms.

The Jitter FIFO can be downsized at the cost of occasional missed frames.

Button to Photons

Latency [ms] Stdev

Local VR 38.3 3.4

CloudXR 81.6 3.3

Diff 43.3 -0.1

Jitter FIFO + Ping 31.2

Diff – Above 12.1

*Average of 40 clicks

18

WIN SERVER

VR App

Vid

eo P

ara

ms

(e.g

. RO

I, o

pti

on

al)

Ap

p

Driver API

VR Runtime

Runtime API

LATENCY

INST

ALL

ER

Audio / Video / Data (sensors, inputs, QoS)

Encode

RGB Frame

Decode

HEVC Frame

RGB FrameSensor & input data

Audio

Clie

nt

app

Serv

er d

rive

r

Audio Capture / Encode

Au

dio

Pla

ybac

k

Frame Pacing

Jitter FIFO

CLIENT

Display

VR Runtime

Controller & inputs

(e.g., OpenVR)Time Warp

Lens Warp

HMD

3-10 ms

10-50 ms(depending on network)

1-3 ms

3-10 ms

OpenVR Interfaces < 0.5 ms

19

20

Total Flexibility

CLOUDXR CONFIGURATIONS

5G

Ethernet

WiFi

21

EARLY ACCESS PROGRAMTest CLOUDXR for VR

@ https://developer.nvidia.com/nvidia-cloudxr-sdk-early-access-program

22

NVIDIA 5G MEC XR DEVKIT

23

THE OPPORTUNITY: VR/AR/VGPU OVER 5GA Convergence of Visual Compute + 5G and a New Edge Topology

Powerful GPUsIncredible compute capabilities redefining

graphics

Bandwidth and Low LatencyBoth are required for VR/AR

Compute is at the EdgeVery few hops – Low Latency

24

ENTERPRISE MARKETSSuccess Across Verticals

ManufacturingEngineering, Design, Marketing/Sales

AECEntire Design Process,

Sales, Moving to Construction

M&EVirtual Production

TrainingPossibly the Largest

Market

Enscape

Nurulize IC.IDO

Autodesk

Location-Based EntertainmentEnterprise/Consumer

ESI

25

ENTERPRISE AR

DIGITAL MODELS ANNOTATEDINSTRUCTION/COLLABORATION

Driving Value Propositions

26

THE FUTURE OF MOBILE XR

• Ultra Low Latency

• High Bandwidth

• Cutting-edge Compute

• Cost Effective Delivery

• Valuable Use Cases

Requirements:

5G + VR/AR + vGPU at the Edge

27

RTX Performance in a Server

• Powered by Quadro RTX 8000/6000

• Ray Traced Global Illumination up to 96 GB Scenes

• Remoting and Multi-GPU Virtualization with Quadro virtual Datacenter Work Station (vDWS)

• All the technical advantages of Turing

NVIDIA RTX SERVER

28

XR/VGPU 5G EDGE STACKSoftware/Hardware Stack Demands a Diverse Effort

Cloud Orchestration

Containerized Edge Applications

Low Latency Streaming

Visual Computing and AI

5G transmission

5G Network

Virtualization

Enabling SDKs

Fundamental GPUDrivers

29

COME TRY THE DEMO &REGISTER ONLINE FOR EARLY ACCESS

https://developer.nvidia.com/nvidia-cloudxr-sdk-early-access-program

Visit the VR Village on the Expo Floor