WetBrush: GPU-based 3D Painting Simulation at the Bristle ...on-demand.gputechconf.com/gtc/2016/presentation/s... · Thick “Impasto”Style A lot of overhang. Thinner painting style

GTC 2016

WetBrush: GPU-based 3D Painting Simulation at the Bristle Level

Zhili Chen Byungmoon Kim

Adobe Research

GTC 2016 2

Oil painting with brush

Complex interactions

• Bristle-Bristle

• Bristle-Paint

• Paint-Paint

Motivation:

• Exhaustive simulation?

GTC 2016 3

Oil Painting Simulation

Brush Sim RendererUser Input

Brush head

movement

Two-way

Bristle-Paint Liquid

Interaction

Paint Liquid

GTC 2016 4

Brush Model

Previous works on brush model

2D Stamps 2D Surface wrapped

around skeleton

Individual bristles

Chu et al., 2002 Expresii DiVerdi et al., 2010 D-Brush in PhotoshopNatural brushes in Photoshop

• No variations from

brush dynamics

• No details from individual

bristles with large

deformation

• Brush shape not realistic without

inter-bristle interactions

GTC 2016 5

Brush Model

Our model

Bristle vertices

Bristle samples

50-100 bristles

~10 vertices per bristle

𝐩𝑖

𝐩𝑖+1

𝑙0

𝐩𝑖𝐩𝑖+1

𝐩𝑖−1

𝜃

Bending

Stretching

GTC 2016 6

Brush Model

Inter-bristle interactions

• Collision

• Friction

GTC 2016 7

Paint Liquid Model

Height field Volumetric grid Particles

Chu, et al. 2010 Microsoft FreshPaint Overhang

Multi-layer color mixing

?• Track volume and mass better

• Less blurring from grid-sampling

• Model paint carried by brush

GTC 2016 8

Hybrid Fluid Representation

Brush

Fluid

particles

Adaptive Hybrid Fluid Representation based on

• Distance to brush

• Velocity

GTC 2016 9


Brush

Fluid

particles

Particles



• Velocity

Close to the brush

OR Fast Moving

Cover smaller region

GTC 2016 10


Brush

Fluid

particles

Particles

Density

Grid



• Velocity

Close to the brush

OR Fast Moving

Cover smaller region

Further away from the brush

AND Slow moving

Cover larger region

GTC 2016 11

Fluid Conversions

Grid & Particles

Visualized

Only Particles

Visualized

FLIP• Particles sample from velocity grid

• Avoid expensive neighbor searching

Grid Fluid

Particle Fluid

• Sample new

particles

• Convert and delete

stationary particles

GTC 2016 12

Hybrid Fluid

Grid Only Grid+Particle

• Less volume loss

• Better surface tracking

• Sharper surface detail

• Sharper color mixing

GTC 2016 13

Brush-Paint Interaction

Brush Sim

Grid Fluid

Particle Fluid

• Brush emit new pigment particles

• Brush carries and pushes existing fluid particles

• Brush pick up color from paint on canvas

• Thick paint affect brush motion and shape

GTC 2016 14

Implementation

CUDA implementation

• Both brush and paint liquid in CUDA

• CUDA+OpenGL interop. for rendering

Rendering

• Ray casting on paint density grid and pigment grid

• Screen space particle rendering

GTC 2016 15

Implementation

k_ForwardPosition

Hash

SolveSelfCollision

k_ProjectConstraints

k_UpdateFinalSim

k_UpdateRendering

k_UpdateSorted

FindBrushBoundingBox

k_InterpolatePaintNodes

k_UpdatePaintNodesRendering

k_VoxelizeBrush

k_FilterBrushVoxelization

k_BrushDistanceMap

k_Advection_Particles

k_RasterizeParticleTrilinear

k_NormalizeRasterization

k_MixBrushVelocity

Diffusion

PostProcessVelocity

k_AdvectPaintDensityGrid

SpawnNewPaticles

k_BrushPickupPigment

k_BrushPaintNodesDistanceToPaintSurface

k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint

k_PIC

k_FindConvertParticlesToPaintDensity

k_ExtendPigment

SaveActiveDomain

Forward

Brush

Inter-Bristle

Collision

Bristle

Stretching

and Bending

Update Brush

Attributes for

Rendering

Update Brush

Paint Nodes

that holds

pigment

Voxelize

Brush

Create Brush

Distance Field

Find Brush

Bounding Box

Forward FLIP

Particles

Rasterize

Particles

onto Velocity

Grid

Diffusion and

Dissapation

Mix with

Brush

Velocity Field

Grid Fluid

Simulation

Brush Pickup Pigment

Sample New Particles

from Density Grid

Brush Vertices emit

new Paint Particles

Resample FLIP Particle

Velocities

Convert inactive

particles to density

grid

Save current

window back to full-

canvas texture

GTC 2016 16

Performance

GTX Titan X

Benchmark sequence: 46 fps, 92K Particles (~2M at maximum)

5%

30%

24%

24%

8%

9%

Grid-based liquid simulation(6.5ms)

Particle-based

liquid simulation (5.2ms)

Grid-particle transfer(5.2ms)

Brush simulation(1.1ms)

Rendering(2.0ms)

Bristle-particle transfer(1.6ms)

GTC 2016 17

Performance

Faster stroke requires higher framerate• Faster stroke => more stroke samples per unit time

• Maximum distance between stroke sample ~= 3-5 pixels

Stroke Sample

distance 2px

Stroke Sample

distance 30px

GTC 2016 18

Optimization

GTC 2016 19

Optimization

Adaptive simulation window (vs fixed simulation window)

Axis-aligned bounding

box for brush

Active simulation

window

thrust::minmax_element()

For grid fluid simulation46 fps-> 75 fps (63%)

GTC 2016 20

Optimization

Particle insertion

Num_Particles

Particle Array

AtomicInc(Num_Particles)

GTC 2016 21

Optimization

New Particle Array

• Pre-allocate maximum #

particle space for each cell• No more than 27 particles within one grid cell

• No more than one particle in one subcell

Marker Array

• 1 => a new particle is

added for the element

Cell 𝑖

Cell 𝑖 + 1

Optimized Particle insertion

Cell 𝑖 − 11

0

1

0

00

00

00

01

0

1

1

10

GTC 2016 22

Optimization

New Particle Array• Pre-allocate maximum #

particle space for each cell

Marker Array• 1 => a new particle is

added for the element

Cell 𝑖

Cell 𝑖 + 1

Optimized Particle insertion

Cell 𝑖 − 11

0

1

0

00

00

00

01

0

1

1

10

Compact New

Particle Array

thrust::remove_if()

Shrink List

Append

Particle Array

75 fps-> 92 fps (22%)

GTC 2016 23

Kernel Pipeline

k_ForwardPosition

Hash

SolveSelfCollision


k_UpdateFinalSim

k_UpdateRendering

k_UpdateSorted




k_VoxelizeBrush


k_BrushDistanceMap




k_MixBrushVelocity

Diffusion

PostProcessVelocity


SpawnNewPaticles




k_PIC


k_ExtendPigment

SaveActiveDomain

GTC 2016 24

Kernel Pipeline

k_ForwardPosition

Hash

SolveSelfCollision


k_UpdateFinalSim

k_UpdateRendering

k_UpdateSorted




k_VoxelizeBrush


k_BrushDistanceMap




k_MixBrushVelocity

Diffusion

PostProcessVelocity


SpawnNewPaticles




k_PIC


k_ExtendPigment

SaveActiveDomain

Low occupancy, GPU almost idle

GTC 2016 25

Dependencies

k_ForwardPosition

Hash

SolveSelfCollision


k_UpdateFinalSim

k_UpdateRendering

k_UpdateSortedFindBrushBoundingB

ox



k_VoxelizeBrush k_FilterBrushVoxelization




k_MixBrushVelocity

Diffusion

PostProcessVelocity


k_PIC


k_ExtendPigment

SaveActiveDomain

k_BrushDistanceMap


SpawnNewPaticles



GTC 2016 26

Dependencies

k_ForwardPosition

Hash

SolveSelfCollision


k_UpdateFinalSim

k_UpdateRendering


ox




k_BrushDistanceMap




k_MixBrushVelocity

Diffusion

PostProcessVelocity


k_PIC


k_ExtendPigment

SaveActiveDomain

• Brush # Vertices << Grid Dimension, Particle #

• Low occupancy

• First half of fluid simulation do not depend on brush

k_BrushDistanceMap


SpawnNewPaticles



GTC 2016 27

Dependencies

k_ForwardPosition

Hash

SolveSelfCollision


k_UpdateFinalSim

k_UpdateRendering


ox




k_BrushDistanceMap




k_MixBrushVelocity

Diffusion

PostProcessVelocity


SpawnNewPaticles



k_PIC


k_ExtendPigment

SaveActiveDomain

• Only for rendering

• Can take place anytime

in a frame


GTC 2016 28

Dependencies

k_ForwardPosition

Hash

SolveSelfCollision


k_UpdateFinalSim

k_UpdateRendering


ox




k_BrushDistanceMap




k_MixBrushVelocity

Diffusion

PostProcessVelocity


SpawnNewPaticles




k_PIC


k_ExtendPigment

SaveActiveDomain

• More parallelization

GTC 2016 29

Optimization

k_ForwardPosition

Hash SolveSelfCollisionk_ProjectConstraints

k_UpdateFinalSim

k_UpdateRendering

k_UpdateSorted



k_VoxelizeBrush k_FilterBrushVoxelization k_BrushDistanceMap

k_Advection_Particles k_RasterizeParticleTrilinear k_NormalizeRasterization k_MixBrushVelocity k_AdvectPaintDensityGrid

SpawnNewPaticles




k_PICk_FindConvertParticle

sToPaintDensityk_ExtendPigment SaveActiveDomain

0

1

2

3

4

Sync

0->1,3,4Sync

0->3Sync

0->2

Sync

2->0

cudaEventRecord()

cudaStreamWaitEvent()


PostProcessVelocityDiffusion

Using CUDA Streams

92 fps-> 117 fps (27%)

GTC 2016 30

Optimization

46 fps -> 117 fps (154%)

Shared memory, Loop unrolling, Adjusting block size

100~200fps on GTX Titan X

30~50 fps on Surface Book

GTC 2016 31

Results

GTC 2016 32

Results

Better 3D shape

Finer surface details

More pigment variations along strokes

Thick “Impasto” Style

A lot of overhang

Thinner

painting

style

Thank you!

Documents

WetBrush: GPU-based 3D Painting Simulation at the Bristle ...on-demand.gputechconf.com/gtc/2016/presentation/s... · Thick “Impasto”Style A lot of overhang. Thinner painting style