04/04/05© 2005 University of Wisconsin NPR Today "Processing Images and Video for an Impressionist Effect", Peter Litwinowicz, Proceedings of SIGGRAPH

04/04/05 © 2005 University of Wisconsin

NPR Today

• "Processing Images and Video for an Impressionist Effect", Peter Litwinowicz, Proceedings of SIGGRAPH 97

• "Painterly Rendering for Animation", Barbara J. Meier, Proceedings of SIGGRAPH 96

• "Cartoon Rendering of Smoke Animations", Andrew Selle, Alex Mohr and Stephen Chenney, Proceedings of NPAR 2004

April. 4, 2005 © 2005 University of Wisconsin

Feng Liu

Department of Computer Science

University of Wisconsin-Madison

Processing Images and Video for an Impressionist Effect

by Peter Litwinowicz, Siggraph 97


Problem Statement

• Transform images/video into animation with Impressionist effect,

particularly, with hand-painted style


Previous work

• Interactive computer-assisted techniques [Haeberli 90]

• Interactive pen-and-ink illustrations making method [Salisbury 94]– Scale-dependent image reproduction method [Salisbury 96]

• 2-1/2D animtion method [Hsu 94].

• System for transforming 3D-gemetry into animations [Meier 96]


Stroke rendering

• Stroke generation– Size, position, length

– color

– Orientation

• Random perturbation

• Clipping and rendering– Edge preservation

• Using brush textures


Example

Without clipping With clipping


Brush stroke orientation

• Draw stroke in direction of constant color– the normal to the gradient direction

• Area with small magnitude of gradient ?– Interpolate surrounding “good” gradient


Frame-to-frame coherence

• How to move strokes across frames– Using Optical flow [Bergen 90] as stroke displacement

• How to avoid over-sparse and over dense stroke distribution?– Delaunay triangulation

– Maximal area

– Minimal distance

(a) (b) (c) (d) (e)


Video


Conclusion

• An algorithm for producing painterly animation from video

• Highlights– Use optical flow to move strokes across frames to keep temporal

coherence

– Orient strokes using gradient-based methods

– Methods to redistribute strokes

– Edge preservation strategy

• Drawback– jittering


Future work

• Other artistic style

• Apply methods to 3D objects

• Reducing jittering



Painterly Rendering

• Goals– Avoid “shower-door” effect

– Provide for frame-to-frame coherence

• Previous techniques achieved one or the other


Painterly Rendering

• How to achieve goals:– Use object geometry, color to decide where to place strokes

– Distribute particles on object surface

– Paint in screen space whereever a particle is placed

• Randomness adds character– Store random seed in “particle”

– Perturb color, orientation, scale based on user-selectable parameters


Previous Work

• "Paint by Numbers", Paul Haeberli, SIGGRAPH 90, 207-214– Similar painterly style, brush strokes stuck to view-plane

• "Comprehensible Rendering of 3-D Shapes", Saito and Takahashi, SIGGRAPH 90, 197—206.

– Used reference images, G-Buffers to generate illustrative image

• “Cellular Texture Generation”, Fleischer, et al. SIGGRAPH 95, 239-248.

– Elements rendered in model space

• Every oil-painting book ever made...


The Algorithm in Detail

Step 1: Create particles to represent geometry


The Algorithm cont...

Step 2: For each frame of animation...– create reference pictures using

geometry, surface attributes, and lighting


The Algorithm cont...

Step 3: Also for each frame of the animation... – transform particles based on animation parameters– sort particles by distance from viewpoint– for each particle, starting with furthest from viewpoint

• transform particle to screen space• determine brush stroke attributes from reference pictures or

particles and randomly perturb them based on user-selected parameters

• composite brush stroke into paint buffer– end (for each particle)


Putting it all together


Creative techniques

• Like real painting, render the scene in layers– Paint each object with multiple layers, each shrunk in more. Outside layers

are painted sparsely, inner layers painted thicker.

– Isolate highlights, shadows using image processing techniques and paint in a separate layer

– Each object or group of objects in a scene can be given its own layer

• Painting parameters can be chosen per-layer

• Semi-transparent layers allow compositing of styles


Technical considerations

• Brush strokes may jitter in size and orientation slightly between frames

– So blur the size and orientation reference images before sampling

• Rendering of back-facing particles– Useful so previously obscured strokes don't pop in when animating

– Can cause visual problems when layering

– Their solution culls back-facing particles, but fades them in as they get close to front-facing


Future Directions

• Combining painterly look with traditional renderer

• Automatically handling changing object size

• Improving particle-placement algorithm to cover geometric surfcace and screen space more evenly

• Implementing longer, deformable brushes that can follow curves on a surface


Questions

Questions?



Cartoon Rendering of Smoke AnimationsAndrew Selle, Alex Mohr, Stephen Chenney

NPAR 2004

Presented by Jared Sohn


Cartoon Smoke

• Uses physically-based simulation to drive nonphotorealistic rendering

• Draws silhouette edges based on depth differences technique

• Maintains temporal coherence


Related Work

• Depth differences technique [Deussen and Strothotte 2000]

• Nonphotorealistic billboards [Lamorlette and Foster 2002]

• Advected non-photorealistic textures [Witting 1999 and Neyret 2003]

• Procedurally-modeled still images of smoke [Yu, et. al]

• Applying NPR to visualizing fluids [Kirby, Marmanis, Laidlaw 1999]


Smoke Simulation

• Particle systems– Intuitive parameterized control

– Unclear how to create realistic rules to capture all possible effects while retaining usability and need to track scalar fields

• Physically-based– More stable (Stam 1999). Fedkiw/Stam/Jensen 2001 improves on

approach.

– Layering approach (Rasmussen 2003) improves efficiency for large simulations

– Key-framed control (Treuille 2003) of smoke simulation


Smoke Simulation

• Only need the output

• Introduce massless marker particles periodically at the source

• Position, velocity, and density are linearly interpolated from the simulation grid

• Easy to implement because semi-Lagrangian solvers already include particle tracing as part of the simulation


Rendering

Several approaches examined:

• Apply silhouette rendering to interesting particles

• Work with isosurfaces derived from simulation

• Use global algorithm to determine placement of silhouettes


Rendering

PASS 1: Primitive rendered to depth and color buffers.

PASS 2: Depth values compared and silhouette edges drawn.

Coherence maintained because particles are advected through simulation.


Simulation-Rendering Interface

• Size determined by density around particle

• Color determined by temperature or density

• Rotation and amount of stretch determined by particle’s velocity


Results

• Smoke simulation requires max of 3 secs/frame on 40x80x40 grid on 3 GHZ Pentium IV

• 2D layered representation could improve performance (maybe real-time?)

• Rendering 640x480 image takes ~ 1 frame/second. Could be reduced by implementing depth difference algorithm in hardware

• More results in video


Conclusions

• Parameters need tuning

• Can’t vary viewpoint

• Smoke simulation is slow, potentially unnecessary

• Can apply same technique to water

• Questions?


Documents

04/04/05© 2005 University of Wisconsin NPR Today "Processing Images and Video for an Impressionist Effect", Peter Litwinowicz, Proceedings of SIGGRAPH