Volume Stylizer: Tomography-based Volume Painting

Volume Stylizer:Tomography-based Volume PaintingOliver Klehm1, Ivo Ihrke1,2, Hans-Peter Seidel1, Elmar Eisemann3

1 Max-Planck-Institut für Informatik2 Saarland University3 TU Delft

I3D 201323/03/2012

Oliver Klehm, 2

Oliver Klehm, 3

Want to influence the appearance of volumes (voxels)• Editing on a fine-grained level is non-intuitive• No suitable tools available

o A Programmable System For Artistic Volumetric Lighting [Nowrouzezahrai2011]

Approach:• Inverse rendering: editing by painting

Motivation

Oliver Klehm, 4

What do we modify• Volume parameters (per voxel)

o Emissiono Albedoo Extinction

Our Goal

Oliver Klehm, 5

What do we modify• Volume parameters (per voxel)

From what do we edit the parameters• Arbitrary input images (target views)• In practice: rendered images that were painted

Our Goal

Oliver Klehm, 6

Our Goal: Volume Painting

Oliver Klehm, 7

Our Goal: Volume Painting

Oliver Klehm, 8

What do we modify• Volume parameters (per voxel)

From what do we edit the parameters• Arbitrary input images (target views)• In practice: rendered images that were painted

Additional required information• Per image: camera parameters (viewing position, direction)• Lighting, scene

How do we optimize• Per-pixel difference to input image• Linear problem• Inconsistent input -> least-squares solution

Our Goal

Oliver Klehm, 9

Given: each pixel k of target view as a constraint pixel Data associated:

ray origin, direction, target radiance

Try to reconstruct volume properties, such thatcomputed radiance matches target radiance(per pixel k)

Specific Goal

Oliver Klehm, 10

A single pixel may influence many voxels And vice versa…

Problem overview

Oliver Klehm, 11

Volume Rendering

Oliver Klehm, 12

Represent the target field (e.g. ) as linear combination of basis functions• Common for stored volumes (== voxels: box/triangle basis fct.)

Reconstruct coefficient vector

, such that

Volume Reconstruction

Includes target Includes volume rendering

Oliver Klehm, 13

Emission Reconstruction

Oliver Klehm, 14

W is huge, ill-conditioned, …!=> iterative conjugate gradient method

We need to compute:• Objective Function:• Derivative:

The trick:

=> this is rendering / back-projection!

Optimization

=

Oliver Klehm, 15

Pre-computation• Cache incoming light in volume per voxel• Sub-sample environment map, deep shadow maps

Optimization• 1. objective function evaluation

o Voxel traversal, gathering pre-cached incoming light

Implementation

Oliver Klehm, 16

Pre-computation• Cache incoming light in volume per voxel• Sub-sample environment map, deep shadow maps

Optimization• 1. objective function evaluation• 2. gradient of objective function

Implementation

Oliver Klehm, 17

Pre-computation• Cache incoming light in volume per voxel• Sub-sample environment map, deep shadow maps

Optimization• 1. objective function evaluation• 2. gradient of objective function• Additional vector arithmetic (addition, norm2, …)

Implementation

Oliver Klehm, 18

Results - Video

Oliver Klehm, 19

How to handle inconsistent inputs? How to define intermediate views? How to generalize? – requires different optimization

Optimize for volume and lighting

Limitations / Future Work

Oliver Klehm, 20

3D volume properties by 2D painting Standard Rendering Model Fast GPU Optimization for interactive sessions

Conclusion

Thank you

Oliver Klehm, 21

All OpenGL Vectors are actually 3D Textures Operations as shaders• Matrix multiplication is volume rendering• Arithmetic operations are “simple” shaders

=> Reconstruction in order of seconds

Fast Implementation

Oliver Klehm, 22

Results

Oliver Klehm, 23

Results

Oliver Klehm, 24

Results - Extinction

Oliver Klehm, 25

Light Editing• Lighting with paint [Pellacini2007]

o Non-linear light optimization• BendyLights [Kerr2010]

o Editing of spot lights• Optimizing Environment Maps for Material Depiction [Bousseau2011]

o Optimization with adhoc metrics Volume editing

• A Programmable System For Artistic Volumetric Lighting [Nowrouzezahrai2011]o Derivation of volume properties for single light beam onlyo Replacing physically-motivated parts of rendering with shaderso Homogeneous media onlyo Treat light beam as string of light (e.g. flow of particle)

• Automatically Adjusting Parameters for Rendering Clouds using Photographs [Dobashi2012]

Related Work

Oliver Klehm, 26

Tomography• Ihrke• Wetzstein• Lanmann

Fabrication• Shadow Art [Pauly2009]• Manufactoring Layered Attenuators for Multiple Prescribed Shadow Images

[Baran2012]

Related Work

Oliver Klehm, 27

Properties to reconstruct:• Extinction (overall density of volume)• Albedo (amount of scattering)• Emission (self glow)

Assumption: Single Scattering!• Light scatters ONCE in volume

Volume Properties

Oliver Klehm, 28

Emission reconstruction

Background / first surface, attenuated by volume

Attenuation by volume

Integration along view ray

In-scattered light

Number of particles (extinction)