UNCLASSIFIED

GPU RAYTRACING FOR REAL-TIME SENSOR-BAND PHENOMENOLOGY MODELING

JRM Technologies, Inc.

Spectral Sciences, Inc.

Army Research Laboratory

WIDA 2012

UNCLASSIFIED

Personnel

JRM •Christopher E. Fink, Ph.D.•Daniel Bybee, BSCS •Joseph Russ Moulton, Jr., MSEE•Karl Leodler, BSAE

SSI•Dave Robertson, Ph.D.

ARL•Richard Shirkey, Ph.D.

UNCLASSIFIED

Goal : A fast, radiometrically-correct sensor-band scene renderer.

Typical Application : NVG target-to-background contrast assessments in highly-cluttered urban scenes. Could provide input to TAWS.

Solution : GPU-based raytracing

1. Start with nVidia Scenix/Optix engines (CUDA language)2. Add On-the-Fly Scene Geometry Generator3. Support loading OpenFlight models & textures4. Ephemeris & Natural Irradiance Prediction (solar, lunar, stellar, airglow, etc.)5. Planckian & Gas Discharge Local Lighting6. Modtran Atmospherics & Local Atmospheric Scattering7. Sandford-Robertson BRDF Reflection with measured material data8. Sensor Effects Processing (optics, detector, electronics)

OVERVIEW

UNCLASSIFIED

Why GPU Raytracing?

Wireframe/Polygonal

Raster Graphics

Raytraced

Raytracing allows for :

High spatial resolution

Local and area sources

Specularity

Refraction/Transmission

Shadowing

Multiple reflections

Atmospheric Scattering

GPU processing brings speed.

UNCLASSIFIED

Backtracing vs. Forward-Tracing

Back-Tracing

Pros: Rays only generated where they contribute to viewpoint.Cons: Poor sampling of sources.

Have to regenerate rays for every viewpoint change.

Forward-Tracing

Pros: Good source samplingCons: Extra bookkeeping needed to direct rays to observer.

Photon Mapping Hybrid

• Forward to deposit photon energy from sources onto surfaces, scatter, and repeat. Produces global illumination solution.

• Backward to sample the distribution from multiple arbitrary viewpoints, without recalculating global solution.

“Backward”

“Forward”

Hybrid : Photon Map

UNCLASSIFIED

Additional Radiometric Optimizations

Separation of “Scene-External” and “Scene-Internal” Atmospherics:

Outside SkyDome : Offline pre-processing of multi-layer/multi-path models to form hemispherical radiance map for inward raycasts.

Inside SkyDome : Atmospheric photon-map based scattering & propagation.

iL

Atmospheric Photons : Allowing photons to “stick” in atmosphere, not just on surfaces, allows prediction of atmospheric scattering, without expensive volumetric gridding.

Importance Sampling : Reduces number of raycasts required for sufficient sampling of BRDF, BSDF, and SkyDome Irradiance functions.

Bounding Volume Hierarchy (BVH) : Advanced photon-map storage/retrieval technique.

Progressive Refinement : Iterative photon buffering & re-use technique.

UNCLASSIFIED

Building Spec

Road Spec

Wall Spec

Story Spec

Window Spec

MMLS Spec

Grid Spec

Option 1 : User-defined on-the-fly geometry creation

Scene Graph / Geometry Loading

UrbanSceneSimple UrbanScenePhaseI

1 ROAD_ROW

1 ROAD

1 BUILDING_ROW

1 BUILDING

1 STORY

1 ROAD_ROW

1 ROAD

2 BUILDING_ROWs

2 BUILDINGs per row

1 STORY per building

Examples

UNCLASSIFIED

Scene Graph / Geometry Loading

Option 2 : Pre-generated, reusable 3D CAD models

• OpenFlight 3D terrain and entity models & associated material-encoded textures (MCMs, Emat fractions)

• CSG CMMW terrains (height field + material code + RF sigma0) • Wavefront OBJ • Collada

Use of material-encoded textures (rather than just polygons) allows for higher spatial resolution.

These all required creation of database format converters to feed Nvidia’s Scenix v.7.2 scenegraph traverser.

UNCLASSIFIED

Sensor-Band Requirements

• Ephemeris model & stellar data

• Atmospherics model & input specification or data

• Irradiance model

• Man-Made local sources & spectral power density data

• Thermal modeling & material data

• Reflection modeling & surface BRDF data

• Sensor Effects (optics, detector, electronics)

CSE Matls 1-8

0

10

20

30

40

50

60

0 6 12 18 24 30 36 42 48 54 60 66 72

ToD

Tem

p

TOAETOA

nllayer 1,0

dVd

d

)(rT

),( TOArrT

111 ,,

222 ,,

nnn ,,

UNCLASSIFIED

RF 10GHz LWIR

MWIR

EO

MWIR

LWIR

RF 10GHz “Noise”

RF 10GHz LWIR

MWIR

EO

MWIR

LWIR

RF 10GHz “Noise”

Seamless Modtran/Radtran Integration

Atmospheric Radiance, Scattering & Transmission Loss

Scene-External Contributions

•Direct Lunar

•Diffuse Lunar (single + multiply-scattered)

•Diffuse Skyshine (thermal)

•Diffuse (aggregate) stellar

•Airglow/Aurorae (via SAMM/SHARC model)

•Nearby Cityglow

Scene-Internal Data

• Extinction Coefficient

• Scattering Albedo

• Henyey-Greenstein Parameter

• Thermal Emission per unit volume parameter

UNCLASSIFIED

NVG-band Backtraced Example Scenes

Default VIS-band Sensor-specific NVG band

Note shadows, transmission, refraction, local lights, and sensor effects.

UNCLASSIFIED

Forward-traced Photon mapping combined with material properties and local irradiance.

Single-reflection-only caseTwo reflections : Note appearance of

human threat in the corner!

Forward-Traced Multiple Surface Scattering Example

UNCLASSIFIED

Atmospheric Scattering & Transmission Loss

Note :• shadows

• direct local radiance

• surface reflection

• atmospheric scattering

• atmospheric transmission loss

N

iiriatmii

isatm

S

ss

xPrxW

LxdrxWxdL

1

3

ˆ,ˆ,,~

ˆˆ,ˆ,,ˆ3

isHGsisatm gfx ˆˆ,~ˆ,ˆ,

UNCLASSIFIED

User Interface

OSVPhysics-enabledRaster Graphics

OSV/RT GUI

Image/Video OutputScenario/Sensor Control

Materially-encoded3D OpenFlight

Database

Optix/JRM/SSIPhysics-enabled

Raytraced Graphics

ScenixScenegraph Converter

SigSimPre-processing

•Atmospherics•Thermal•MMLSOTF Scene Geometry

Spec

UNCLASSIFIED

User Interface

Waveband & Resolution / FOV Optics Parameters

UNCLASSIFIED

User InterfaceDetector Params Electronics Params

UNCLASSIFIED

User InterfaceEnvironmental Params Raytrace Control

UNCLASSIFIED

Performance

BACKTRACING

640x480 @ 80 Hz for : 27000 polys & 95 textures

6 secondary raycasts per pixel

FORWARD-TRACED PHOTON MAPPING

640x480 @ 23Hz for : 27000 polys & 95 textures

1024 x 1024 photons reflecting 3 times each

UNCLASSIFIED

Goals for Follow-Up Funding

• Validation against SSI’s MC-Scene and DARPA field data• Extension to Infrared regime• Extension to RF regime• Addition of localized clouds, dust, smokes/obscurants• GUI additions, e.g. for On-the-fly Geometry Generator• TAWS Integration

UNCLASSIFIED

ValidationOptions

•Analytical Calculations for simple scenes

•SSI’s MC-Scene NRT CPU Backtracer

•DARPA field data

P

G

H

0

MCScene Simulation Tool : SSI’s Monte Carlo Based Scene Simulation

• UV to LWIR • 3D atmospheres & surfaces • Molecular absorption• Rayleigh scattering • Aerosol absorption & scattering• Multiple scattering • Thermal emission• Reflections from topographic

terrain• Scattering, emission, and

transmission by 3D clouds

UNCLASSIFIED

Thermal-band Backtraced Implementation

Frictional BC on treads

Diffraction blur

Thermal noise

Internal heat generation

Horizon & earthshine-loading on vertical surfaces

MWIR 4pm

MWIR 4pm

LWIR 11pm

UNCLASSIFIED

Vi

Hi

VVVH

HVHH

Vr

Hr

ii

ii

ii

ii

ir

ir

eV

eH

eVVeVH

eHVeHH

eV

eH

Each ray now carries polarized, complex components.

RCS reflection is now a complex Jones matrix multiplication :

RF-Implementation : Coherence & Polarization

cnfi 2expEach ray path now also has to carry a propagation phase factor :

SAR with horizontal field SAR with vertical field

UNCLASSIFIED

RF Implementation : Correlated Local Clutter Maps

Original Uncorrelated Map Single-angle Correlated

Multiply-Scattered Correlated with 4x4 angle-averaging

Multiply-Scattered Correlated with 18x72 angle-averaging

Improving RF clutter maps by including effects of multiple scattering

UNCLASSIFIED

RF Implementation : Bistatic Scatter Center Sets

Bistatic scatter centers compress the “internal” multiple reflections into a small set of localized transfer functions for later composition into a scene, thus saving a lot of run-time processing / raycasting.