Binary Shading using Geometry and Appearance

Binary Shading using Geometry and Appearance

Bert Buchholz Tamy Boubekeur Doug DeCarlo Marc Alexa

Telecom ParisTech – CNRS Rutgers University TU Berlin

Computer Graphics Forum Vol. 29, N. 6, 2010Presented at Eurographics 2011

Binary Drawing

Depicting scenes using 2 colors

Conversion

Binary Rendering

3 D S c e n e

Lighting Reflectance

Geometry

Camera

Related Work

Line Drawing [DeCarlo 2003][Judd 2007]

Image Binarization [Mould and Kaplan 2008]

Local shading operators [Vergne 2008]

Half-Toning [Floyd and Steinberg 1976][Ostromoukhov and Hersch 1995]

Analysis

Variational Rendering Model

Per-pixel decision

Contradictory criteria

Geometry enhancement

Low shading depiction power

VARIATIONAL BINARY RENDERING

Variational Binary Shading

• Deferred shading framework• Rendering data structured as a ST image graph• Edges energies derived from geometry and appearance• Final rendering as a min cut in the image graph

Deferred Shading

Generate a set of arrays (render buffers)– Geometric Properties– Appearance Properties nm

iA 1,0

nmiG

1,0

Rasterization

Ray Tracing

Graph construction

Standard Source-to-Sink Image Graph {V,E}

TiSi

pi eeeE

TSpV i

ImagePixels

Source(white)

Sink(black)

ImageConnectivity

Edges to Source

Edges toSInk

Appearance Contribution to the Graph

• Through terminal weights on • Appearance initiate B/W

segmentation• Experiments using :– Diffuse component– Specular component– Headlight component– Ambient Occlusion/Accessibility

component

TiSi ee

yxWi ,

yxBi ,TerminalWeights

Appearance Graph Terms

Global to local feature control using spatial averages

2

22)(

,

,, ih

vyux

vuii evuAyxA

Support

sizeLocal term: yxAyxAyxA iii ,,,'

Sign:locally lighter or darker

Weighted combination to the global measure.

yxAgyxAgyxB

yxAgyxAgyxW

iiiii

iiiii

,',0max1,1,

,',0max1,,

Global versus Local Thresholding

Global Local

Geometry Contribution to the Graph

• Modelled with neighbor edge weights• Tailors anisotropic, non-local B/W diffusion• Redistributes B/W values to enhance

geometric features• Based on:– View depth values – View dependent curvature, estimated as screen

space normal derivatives [Judd 2007]

pie 1100 ,,, yxyxN

yxz ,

11001100 ,,,,,, 2 yxnyxndyxyxS

Geodesic distance on the Gauss sphereUnit surface normal gathered at 00 , yx

Geometric Term

Local curvature normalization:

2

22

,1100 ,,,,,, c

vyux

vu

evuyxyxyx

Support

size

Yields neighbor edge weights:

2

211001100 ,,,,,,

11001 ,,, k

yxyxyxyx

eyxyxN

Geometry Contribution Modulation

Geometry Contribution Modulation

Small components are successively connected

Graph Minimum Cut

• Max Flow Min Cut Theorem– Shortest split path in the graph→Feature size control through

graph energy• Separate the graph in two components• Boykov and Kolmogorov

implementation [2004]

Cut performance

• Rendering buffers can be speed up using rasterization• Measured on a Core2Duo 1.83GHz (single thread)

RESULTS

Binary Shading

• Global to local features• Appearance & geometry depiction• Large variety of style• Interactive control

Comparison to Line Drawing

Line Drawing Binary Shading

Combination with Line Drawing

Combination with Line Drawing

RGBN Picture Rendering [Toler-Franklin 2007]

Combination with Line Drawing

Comparison to Thresholding

Diffuse Component

Thresholding ThresholdingOver Gaussian Filtering

Binary Rendering

Comparison to Thresholding

Comparison to Image Binarization

(b,e) equivalent to [Mould and Kaplan 2008]

INTERACTIVE CONTROLOn-going work

Interactive Control

• Solution 1: slider-based, for experts– Accurate control– « Too much » control for novice users

• Solution 2: painting interface

Paint Interface

• Supervised sparse B/W contraints• Derive the graph energy structure

Paint Interface

LIMITATIONS AND FUTURE WORK

Animation

… most of straightforward solutions have some temporal coherency failure cases.

Works most of the time but…

K-color Rendering

• Extend to multi-label cuts• Alternative energy minimizer– Lloyd relaxation/k-means– Mean Shift

• k-Component cut• Vector Rendering

CONCLUSION

Binary Shading as a Single Cut

• Global solution• Local to global control• Appearance vs geometry control• High level control

• For automatic binary drawing from 3D Scenes, decals, cut-out, etc…

Thank you

Binary Shading Using Geometry and Appearance.Bert Buchholz, Tamy Boubekeur, Doug DeCarlo and Marc Alexa

Computer Graphics Forum Vol. 29, Nb. 6, 2010Presented at Eurographics 2011