Connecting Graph Cuts and Level Sets

European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)

ECCV 2006 tutorial on

Graph Cuts vs. Level Sets

part III

Connecting Graph Cuts and Level Sets

Vladimir Kolmogorov

University College London

Yuri Boykov

University of Western Ontario

Daniel Cremers

University of Bonn


Graph Cuts versus Level Sets

Part I: Basics of graph cuts

Part II: Basics of level-sets

Part III: Connecting graph cuts and level-sets

Part IV: Global vs. local optimization algorithms


Graph Cuts versus Level Sets

Part III: Connecting graph cuts and level sets

• Minimal surfaces, global and local optima (VK)

• Integral and differential approaches (YB)

• Metrics on the space of contours, learning and shape

prior in graph cuts and level-sets (DC)


Discrete vs. continuous functionals

Geodesic active contours

∑∑ +=qp

qppq

p

pp xxExEE,

),()()(x ∫=

C

dsNsCgCE )),(()(r

Graph cuts

Both can incorporate basic segmentation cues

• Image contrast

• Regional bias

• Alignment (flux)


Incorporating image contrast:

graph cuts [Boykov&Jolly’01]

n-links

s

t a cuthard constraint

hard constraint

∆−=

22exp

σ

pq

pq

Iw

pqI∆

σ


Riemannian metric

(space varying, tensor D(·))

Euclidian metric

(constant)

distance map distance

map

Incorporating image contrast:

geodesic active contours [Caselles,Kimmel,Sapiro’97]

∫ ⋅=

C

dsgCE )()(

Define Riemannian metric from image gradient

Compute geodesics

• shortest curve between two points

A

B

A

B


Metrication errors on graphs

discrete metric ???

Minimum cost cut

(standard 4-neighborhoods)

Continuous metric space

(no geometric artifacts!)

Minimum length geodesic contour

(image-based Riemannian metric)


C

“Cut metrics” and “Riemannian metrics” allow to compute

contour “length” in 2D (or “area” in 3D)

Cut Metrics :

cuts impose metric properties on graphs

∑∈

=Ce

ewC ||||

contour’s “length”


C ∑∈

=Ce

ewC ||||

Geo-cuts [Boykov,Kolmogorov’03]:

Combining graph cuts and geodesic active contours

Given geometric functional, e.g. ∫=

C

dsNsCgCE )),(()(r

construct graph such that ∑∈

≡≈Ce

ewCCE ||||)(


“Distance maps” for cut metric

“standard”

4-neighborhoods

(Manhattan metric)

256-neighborhoods8-neighborhoods

Euclidean metric

Riemannianmetric

D(p)=const


What metrics can be approximated?

Question: What continuous functionals can be approximated with geo-cuts?

[Kolmogorov,Boykov’05]:

• Geometric length (e.g. Riemannian)– Distance map for g() is convex & symmetric

• Regional bias

• Flux of a given vector field

∫ ⋅=

C

dsgCE )()(

∫∫Ω

+ daf

∫ ⋅+

C

dsN )(rr

v

Ω


Geometric measures used in level set segmentation[Acknowledgement: Ron Kimmel’s presentation]

weighted arc-length ∫ ⋅=

C

dsgCE )()(

functional evolution equation

( )NNggCt

rr ⋅∇−=

weighted area ∫∫Ω

= dafCE )( NfCt

r =

alignment(flux) ∫ ⋅=

C

dsNCE )()(rr

v NCt

rr )(div v−=

robust alignment ∫ ⋅−=

C

dsNCE ||)(rr

v NNCt

rrrr ))(divsign( vv ⋅=

Ω


Flux

vector field: some vector defined at each point p

• “stream of water” with a given speed at each location

pvr

pvr

C1 ∫ ⋅=

C

dsNCflux )()(rr

v

flux: “amount of water” passing through a given contour

flux(C1) > flux(C2)

C2

n

n

• Changes sign with orientation


Segmentation of thin objects [Vasilevskiy,Siddiqi’02]

• Vector field: I∇=vr

pvr


Riemannian length + Flux [Kimmel,Bruckstein’03]

Riemannianlength

Flux of

Riemannianlength+Flux

I∇


Robust alignment

∫ ⋅∇=

C

dsNICE )()(r

assumes bright object, dark background

∫ ⋅∇−=

C

dsNICE ||)(r

no such assumption

“Robust alignment”

[Kimmel,Bruckstein’03]


Geometric measures used in level set segmentation[Acknowledgement: Ron Kimmel’s presentation]

weighted arc-length ∫ ⋅=

C

dsgCE )()(

functional evolution equation

( )NNggCt

rr ⋅∇−=

weighted area ∫∫Ω

= dafCE )( NfCt

r =

alignment(flux) ∫ ⋅=

C

dsNCE )()(rr

v NCt

rr )(div v−=

robust alignment ∫ ⋅−=

C

dsNCE ||)(rr

v NNCt

rrrr ))(divsign( vv ⋅=

Ωsame as in geo-cuts

non-submodular


Graph cuts vs. level sets

for geodesic active contours

Graph Cuts Level Sets

Minimize the same functional(Geo-cuts)

global minimum local minimum

for the class of functionalsdiscussed previously

• Connection only approximate: E(C) ≈ ||C||

• Even stronger connection: continuous maxflow


Continuous maxflow

[Iri’79],[Strang’83],[Appleton,Talbot’03]

• Analogue of discrete maxflow

• Solves continuous problem in subset of Rn

• Flow = vector field


pqpq wf ≤

0=∑q

pqf

qppq ff −=

Discrete maxflow

Flow defined on graph edges

•

Capacity constraint:

Flow conservation:

(for p ≠ s,t)

Maximize flow out of the source(s):max→∑

q

sqf

4-neighbourhood system


pqpq wf ≤

0=∑q

pqf

qppq ff −=

Discrete maxflow

Flow defined on graph edges

•


Flow conservation:

(for p ≠ s,t)

[Ford&Fulkerson theorem]:

Maximum flow saturates minimum cut


Continuous maxflow

Flow = vector field


Flow conservation:

(for p ∉ s,t)

Maximize flow out of the source:

gf p ≤||r

0div =pfr

max )(div →∫s

p dafr


Continuous maxflow

gf p ≤||r

Flow = vector field


Flow conservation:

(for p ∉ s,t)

Maximum flow saturates minimum cut

Ngf pp

rr=

0div =pfr

( for p ∈ C* )


Solving continuous maxflow

[Appleton,Talbot’03,06]: numerical algorithm

• Vector field stored on edges

– Horizontal edges => x-component

– Vertical edges => y-component

• Flow conservation similar to the discrete case

• But - capacity constraint:

• Report 0.1 pixels accuracy (on average)

xf1

yf2

xf4

yf3

222gff yx ≤+

Documents

Connecting Graph Cuts and Level Sets