Expander Flows, Graph Spectra and Graph Separators Umesh Vazirani U.C. Berkeley Based on joint work...

Expander Flows, Graph Spectra and Graph Separators

Umesh VaziraniU.C. Berkeley

Based on joint work with Khandekar and Rao and with Orrechia, Schulman and Vishnoi

Graph Separators

S T

Sparsest Cut/Edge Expansion: S

EG

TS

VS

,

2

||||

min)(

c-Balanced Separator: S

EG

TS

VSVc

,

2

||||||

min)(

Applications

• Clustering• Image segmentation• VLSI layout

• Underlie many divide-and-conquer graph algorithms

Interesting Techniques

• Spectral methods. Connection to differential

geometry, discrete isoperimetric inequalities.

• Linear/semidefinite programming

• Measure concentration

• Metric embeddings

Geometrical view

• Map vertices to points in some abstract space: - points well-spread - edges short

Geometrical view

• Map vertices to points in some abstract space: - points well-spread - edges short

• “Good bisection” of the space yields sparse cut in graph

Spectral Method

1)(

)(min

2

,

2

,

jji

i

jEji

i

xx

xx

Cut at random

Minimize sum of “edge lengths”:

Spread out vertices:

[Cheeger’70] [Alon, Milman ’85][Jerrum, Sinclair’89]

4

)()(

2

10

GG

Leighton-Rao ‘89

mw

w

jiij

Ejiij

,

,

min

22

22

2

5

4

5

5

4

11

1

1

1

1

Cut along ball of random radius

Distances form a metric: satisfy triangle inequality.wij + wjk >= wik

Minimize sum of “edge lengths”:

Spread out vertices:

O(log n) approximation: Approximate max-flow min-cut thm for multi-commodity flows.

ARV ‘04

Triangle inequality:222 )()()( kikjji vvvvvv

Unit sphere in Rd

Unit L22 embedding:

No anglesobtuse

Minimize sum of “edge lengths”

Spread out vertices

22

,

2

,

)1(4)(

)(min

nccvv

vvW

jji

i

jEji

i

)log)(( nGO Procedure to recover cut of size

ARV Procedure to recover cut

• Slice a randomly oriented “fat”-hyperplane of width

d

O1

Unit sphere in Rd

ARV Procedure to recover cut

• Slice a randomly oriented “fat”-hyperplane of width

• Discard pairs of points (u,v):

• Arrange points according to distance from S• Cut along ball of random radius r:

d

O1

Unit sphere in Rd

nvu

log

12

S

r0

Metric EmbeddingsFinite Metric Space (X, d)

x

y

Rk with L2 norm

f(x)

f(y)

Distortion of f is min c: ),()()(),(2

yxdcyfxfyxd

[Bourgain ’85] Every finite metric space can be embedded in L2 with distortion O(log n).

Longstanding open question: Better bound for L1?

[Enflo ’69] [Arora, Lee, Naor ’05] Any finite L1 metric can be embedded in

L2 with distortion nnO logloglog

f()

nlog

Today’s Talk

• Leighton-Rao: multi-commodity flow O(n2).• Arora, Hazan, Kale: O*(n2) ARV implementation

based on expander-flow formalism

• Much faster in practice. • [Khandekar-Rao-V] : O*(min{n1.5, n/α(G)}) single commodity

flow based algorithm. O(log2 n) approx. ratio.• [Arora, Kale]: matrix multiplicative weights algorithm based O(log n) approx• [Orrechia, Schulman, V, Vishnoi] O(log n) approx using

KRV style algorithm

Multi-commodity flow:

Single commodity flow:

Expander Flows

• Any algorithm for approximating sparse cuts must find a good cut, of expansion say β

• Must also certify no cut is much smaller.

• To give a k-approximation must certify that no cut has expansion less than β/k.

• Problem: there are exponentially many cuts.

ST

Expander Flows

G = H =

• For each edge of H, route one unit of flow through G

Expander Flows

G = H =

• For each edge of H, route one unit of flow through G

• Must route Ώ(|S|) units of flow from S to T.

• Therefore |ES,T| = Ώ(|S|/c) expansion = Ώ(1/c)

• Ideally c = O(1/α(G))

ST

max congestion = c expansion = Ώ(1/c)

Expander Flows

• max congestion = c. expansion = Ώ(1/c).

• ARV: max congestion =

• Leighton-Rao: H = complete graph. max cong = O(logn/α(G))tight example: G = expander graph.

• Motivating idea for ARV: write LP to find best embedding of H in G + exponentially many constraints saying H expander

eigenvalue bound gives efficient test for expansion!Therefore poly time using Ellipsoid algorithm.

• [Arora, Hazan, Kalle] O*(n2) implementation of ARV

)(

log

G

nO

• Know large number of vertices on each side of cut.

• A max-flow, min-cut computation should reveal sparse cut.

• But this is circular…

KRV

s t

• H Φ

• Embed candidate expander H in G with small congestion.

• Test whether H is expander (if so done!)

• Else non-expanding cut in H gives a bipartition of G; route a flow in G across this bipartition.

• Decompose flow into flow paths and add the resulting matching to H.

Outline of Algorithm

Cut-Matching Game

H Φ

Cut Player• Find bad 50-50 cut in H

• Goal: min # iterations until H is an expander

Matching Player• Pick a perfect matching

across cut

Goal: max # iterations untilH is an expander

Claim: There is a cut player strategy that succeeds in O(log 2 n) rounds.

Finding a cut: Spectral-like-method

= +1 charge

= –1 charge

Mix the charges alongthe matchings { M1, M2, …, Mt }

Random assignment of charge

V: Vertex setx

y

(x+y)/2

(x+y)/2

After t iterations, H = { M1, M2, …, Mt }.

Finding a cut: Spectral-like-method

Order the vertices according to the final charge presentand cut in half.

n/2 n/2

S S

But how to formalize intuition?

Lift to Rn

• Cannot directly formalize previous intuition Therefore lift random walk to Rn – walk embedding of H.

• n-dimensional vector associated with each vertex

• In each step, replace vectors at endpoints of matched edge by their average vector.

• Potential function to measure progress of this process.

• Potential function small implies H expander.

• Relate lifted process to original random walk: each successive matching decreases potential function.

Walk Embedding

H Rn,

Vertex i mapped to Pi = (pi1, …, pin)

pij = P[walk started at j ends at i]

Ht = { M1, M2, …, Mt }.

Small cut in graph shows up as clusters in walk embedding. (1/n, …, 1/n)

P1

P3P2

Pn

2

,

2)/1)((/)()( ntpntPt

jiij

ii 1Potential:

Claim: ψ(t) ≤1/4n2 implies α(Ht)≥ ½

Will show potential reduces by (1 – 1/log n) in each iteration.

Ψ(0) = n-1

(1/n, …, 1/n)

P1

P3P2

Pn

Main Question: How to augment Ht = { M1, M2, …, Mt }

by Mt+1 so H closer to expander?2

,

2)/1)((/)()( ntpntPt

jiij

ii 1Potential:

If Mt+1 matches vertex u to vertex v,

then potential reduction in t+1-st step

Since each of Pu and Pv replaced by

So potential reduction =

2vu PP

22

22

2

1

22 vu

vuvu PP

PPPP

The Lifted Walk

2

2

1vu PP

Potential Reduction

Pv

= v |Pv1/n|2

Reduction in = |green|2

1-d: reduction = ()n-d 1-d: log n stretch

Actual potential reduction = /log n

Original random walk = projection of lifted walk on random vector

Running time

• Number of iterations = O(log2 n)• Each iteration = 1 max-flow + O*(n) work

= O*(m3/2)

• [Benczur-Karger’96] In O*(m) time, we can transform any graph G on n vertices into G’ on same vertices:– G’ has O(n log (n)/ε2) edges– All cuts in G’ have size within (1 ± ε) of those in G

• Overall running time = O*(m + n3/2)

Improving to O(log n) approximation

• [Arora, Kale]: matrix multiplicative weights algorithm based combinatorial primal-dual schema for semidefinite progs

• [Orrechia, Schulman, V, Vishnoi]: simple KRV style algorithm

Idea: To find Mt+1 perform t steps of natural random walk

(instead of round-robin walk) on Ht = { M1, M2, …, Mt }

Brief Sketch

• Instead of showing that H has constant edge expansion after O(log2 n) steps, will show that the spectral gap of H is at least 1/log n, and therefore the conductance of H is at least 1/log n.

• Since degree of H is log2 n, this means its edge edge expansion is at least log n.

Why natural walk?

Suppose round robin walk on M1, … , Mk mixes perfectly

on each of S, T. Now a single averaging step on Mk+1

ensures perfect mixing on entire graph!

S T

Mk+1

Matrix inequality: tttt ABAABA )(

Question: Replace ½ self-loop with a ¾ self-loopin round-robin random walk!

x

y

(3x/4+y/4)

(3y/4 + x/4

Gives a way of relating round robin walk to time independent walk.

Conclusions and Open Questions

• Our algorithm is very similar to some heuristics.

• [Lang’04] similar to one iteration of our algorithm.

• METIS [Karypis-Kumar’99]– collapses random edges– finds a good partition in collapsed graph– induces it up to original graph, using local

search

• Connections with these heuristics? Rigorous analysis?

When the Expansion is large …

• Could have used [Spielman-Teng’04] “nibble” algorithm instead of walk-embedding. But:

Algorithm Output sparsity

Running Time

Spectral 1/2 n2/2

Spielman-Teng

1/3 log3 n n/3

KRV log2 n min {n3/2,n/}

• Conjecture: A single iteration of round-robin walk + max-flow should give a sparse cut.

• [Khot, Vishnoi] Ώ(loglog n) integrality gap

• [Orrechia, Schulman, V, Vishnoi] Ώ(√logn)bound on cut-matching game.

• Is it possible to obtain a O(√log n) approximation algorithm using single commodity flows via the cut-matching game?

Limits to these methods