Saarbr ü cker IT-Dialog Automobilindustrie A Cave System for Interactive Modeling of Global Illumination in Car Interior Kirill Dmitriev, Thomas Annen,

Saarbrücker IT-Dialog Automobilindustrie

A Cave System for Interactive Modeling of Global Illumination in

Car Interior

Kirill Dmitriev, Thomas Annen, Grzegorz Krawczyk, Rafal Mantiuk,

Karol Myszkowski, and Hans-Peter Seidel

Max-Planck-Institut für Informatik,Saarbrücken, Germany

Karol Myszkowski


[log cd/m^2]Luminance

-6 -4 -2 0 2 4 6 8

Bright projector

LCD Monitor

CRT Monitor

Human eye

High Dynamic RangeHigh Dynamic Range

Human eye adjusts comfortably up to 12 orders of magnitude and can see simultaneously up to 4 orders of magnitude


HDR Pipeline: Acquisition• Global illumination• Products: HDR cameras

– Lars III (Silicon Vision), Autobrite (SMal Camera Technologies), HDRC (IMS Chips), LM9628 (National), Digital Pixel System (Pixim)

• Technology [Nayar 2003]:– Trading-off spatial resolution

• Multiple sensor elements within pixel

• Spatially varying pixel exposures• Mosaicing with spatially varying

filter– Trading-off temporal resolution

• Multi-exposure LDR capture– Multiple image detectors (beam

splitters)– Smart pixels


HDR Pipeline: Storage• Several formats for still images

– Radiance– OpenEXR (new industrial standard)– logLuv tiff– HDR JPEG

• HDR MPEG


HDR Pipeline: Display• LDR displays: luminance compression required

– Solution: tone mapping – Important factors in tone mapping selection

• Dynamic range of display device• Video display conditions

– Background lighting• Application

– Just nice looking video– Visually plausible results– Optimizing visibility of details– Improving contrast …

• HDR displays start to appear – Sunnybrook Technologies and University of British

Columbia


HDR Pipeline: Applications

IMS ChipsIMS Chips


Light Reflection

emissiongeometryBRDF

radiometricvalues

displayedimage

Visual DisplayLight TransportSimulation

27

15

13

22

Display Observer

Realistic Image Synthesis

Greenberg et al. Siggraph’97. Cornell UniversityGreenberg et al. Siggraph’97. Cornell University

HDRHDR LDR


Acquisition: Materials• Shift variant Bi-

directional Reflectance Distribution Function (BRDF)– Lensch et al.

• Transluscency – Goesele et al.


Acquisition: Luminaires• Luminaire spatial power distribution (Goesele et al.)

– Near field photometry– Emitted energy represented as 4D light field

• Relevant for light sources installed in the car interior

measurement

VR: rendering


Acquisition: Luminaires• Natural lighting

– Light probes and multi-exposure techniques– SpheroCam from Spheron VR– Emitted energy represented as an environment map

• Relevant for external car illumination

VR: rendering

Light probe

Paul Debevec


Lighting Simulation + Rendering

1) Photographs of mirror sphere at varying exposure times

2) High-dynamicrange environment map

3) Use as light source in Monte Carlo radiosity algorithm

Philippe Bekaert


Our Goal• CAVE system for car

interior rendering– Real-time lighting simulation– Dynamic real world lighting

• HDR video environment maps– Free observer position

• Head tracking

• Special focus: – Predict impact of quickly changing lighting conditions on the visibility

of information displayed at the LCD panel• Precise modeling of light reflections from the LCD panel

• Predicting effective contrast and information readability for various viewing angles

• Taking into account light adaptation conditions for the human eye


Requirements for Rendering Algorithm• 5x2 full screen resolution frames at interactive rates• LCD panel illumination must be computed precisely• Higher error tolerance for the car interior illumination

– Only low spatial frequencies in reconstructed lighting acceptable

• Distant lighting assumption holds– HDR environment maps acceptable

• Lighting can change quickly– Relying on temporal coherence between frames

impractical• Car interior geometry static


Rendering Algorithm Selection• Requirements

– Exploit all computational resources available in the CAVE: 20 CPUs + 10 GPUs

• Rendering solution– Use Precomputed Radiance Transfer method for car

interior: 10 GPUs + 10 CPUs– Use Final Gathering for the LCD panel: 10 CPUs


Spherical Harmonics• Spherical Harmonics:

– Orthonormal basis over the sphere– Analogous to Fourier transform

• Projection:

• Reconstruction:

• Integration:

sdsysaa ii

)()(

)(syi

n

iii syasa )()(~

n

iiibasdsbsa

1

)(~

)(~


Rendering Equation

sdnssVsLvL ppinout

p

)0,max()()()(

Incident LightIncident Light VisibilityVisibility CosineCosine

ReflectedReflectedLightLight

sdsTsLvL inoutp

)()()(

Jan Kautz


Precomputed Radiance Transfer

sdsTsLvL inoutp

)()()(

light function:light function:

into SHinto SH

transfer:transfer:

into SHinto SH

TiniL

n

ii

ini

outp TLvL )(

"light vector""light vector"

"transfer vector""transfer vector"

Jan Kautz

Such a dot product must be computed for each mesh vertex: n = 25


PRT for Arbitrary Meshes

BeforeBefore AfterAfter


Handling Two-Sided Geometry• Car geometry is two sided

– Need to store SH coefficients for both sides and render geometry twice with back face culling

– This leads to lower performance and twice larger memory consumption

• Better store SH coefficients only for one side of geometry:– Only the rays going through the windows contribute to SH

coefficients– User has to point out the mesh parts representing car windows


LCD Panel Modeling• Emission characteristic of the LCD-sandwich:

– Spectral emission-function of the backlight– Transmission characteristic of the polarizer and other

layers (e.g. BEF, d-BEF, LCF, rgb-filters, …)– Transmission characteristic of the LC-cell– Transmission of other optical elements – e.g. glass

• Reflective characteristic in case of incoming light– Surface coating (e.g. AR/AG)– Reflection characteristic of the LCD sandwich

observer

LCD-sandwich

Incoming lighting

backlight

glassplate


Computing LCD Panel Lighting• DIMOS or SPECTER systems can be used for lighting simulation

within the LCD panel– We use just external spectral emissivity and reflectance data

provided with a very high angular resolution

• Algorithm– Cover geometry of LCD panel with texture– For each texel

• Compute energy emitted in the observer direction

– Use tabulated emissivity data

– Modulate emissivity as a function of the displayed information

• Compute energy reflected in the observer direction

– Use the final gathering method

– Improve performance using BRDF-weighted importance sampling

» If traced ray hits the car window, query the environment map

» If traced ray hits the car interior, query PRT data structures


System Architecture


Distributing the Computation

Camera parametersCamera parameters

• Use “Lightning” system that takes care of OpenGL synchronization between different PCs

• LCD display computation is distributed in an asynchronous way


Distributing the Computation• Use “Lightning” system that

takes care of OpenGL synchronization between different PCs

• LCD display computation is distributed in an asynchronous way

Broadcast the display image as it is readyBroadcast the display image as it is ready


Tone Mapping in the CAVE• Visual adaptation is influenced by light projected on a

small area around the center of retina • Head tracking enables precise estimation of the gaze

direction– We assume that the adaptation is affected by scene luminances

within the region of 10° surrounding the gaze direction– This region can be mapped to 1-3 walls in the CAVE– Luminance data within this region is collected and send to the

master

• Master computes common tone mapping parameters and broadcasts them to all computers in the cluster– There is a small delay in illumination update, but it is hidden by

temporal adaptation model anyway


Rendering with Tone Mapping• 3 Passes:

– Compute tone mapping parameters

• Preview rendering of 128x128 HDR images for the CPU processing

– Select visible geometry for tone mapping

• Lighting OFF• Render geometry to z-buffer

– Final rendering with tone mapping

• Lighting ON• Use z-buffer test to tone map

only visible fragments


Results (Car Interior Part)• Car model contains approx. 500K triangles• Preprocessing of SH coefficients takes about 100

minutes• Full model size with SH coefficients is about 60 MB• Rendering frame rate is about 10 FPS

– For each frame current environment map is projected into SH basis and lighting in every vertex is computed

– Full image tone mapping is applied


Results (LCD Panel Part)• One processor on each PC is

busy with sending data to OpenGL

• Another processor computes draft images (40 samples per pixel) of LCD panel and sends them to other PCs

• Draft image display is available almost immediately after each head movement or environment maps rotation, converged image is computed in approx. 2 seconds


Conclusions• We proposed efficient global illumination and tone

mapping solutions for CAVE VR applications involving static geometry and dynamic environment lighting

• We successfully applied those solutions to the car interior modeling, efficiently utilizing all CPUs and GPUs resources on the CAVE cluster

• We proposed an accurate algorithm for the LCD panel simulation and rendering based on measured BRDF and emission data.


Future work• Use HDR video stream as dynamic environment map

lighting– Trivial to do, we are just waiting for a fish-eye lens suitable for our

HDR camera

• Use HDR display (0.05-3,000 cd/m2) to render the LCD panel with luminance levels similar to real world driving conditions– Display-in-display rendering problem

• Consider recently proposed techniques for all frequency PRT lighting


Acknowledgements

• We would like to thank the following people– Michael Arnold (Virtual Reality Center, DaimlerChrysler AG)– Thomas Ganz (Research & Technology Displays and

Controls (RBP/BM), DaimlerChrysler AG)– Matthias Bues (VR Lab., Fraunhofer IAO)


Adaptive Logarithmic Tone Mapping


Tone Mapping: Time-Dependent Visual Adaptation

1. Light adaptation

2. Dark adaptation

Documents

Saarbr ü cker IT-Dialog Automobilindustrie A Cave System for Interactive Modeling of Global Illumination in Car Interior Kirill Dmitriev, Thomas Annen,