Power and limits of local algorithms for sparse

random graphs

David Gamarnik

MIT

Joint work with

David Goldberg, Theophane Weber and Madhu Sudan

Workshop on Random Graphs and Their Applications

Яндекс

October, 2013

• Real life natural, technological communication networks are large

(WWW 109, neurons in a human brain 1011)

• This makes computational models traditionally deemed scalable

(polynomial time), too impractical

• Local algorithms – promising framework

• Correlation Decay – property which validates accuracy of local

algorithms

Some high level thoughts…

CD arises in statistical phyisics -- uniqueness of Gibbs measures on infinite

graphs, Dobrushin [60-70s]

Correlation Decay Property

Correlation Decay Property

Decisions based on local neighborhood ¼ decisions based on the entire

network:

Electrical Engineering: Belief Propagation (BP) algorithm

Wainwright & Jordan [2008], Mezard & Montanari [2009]

Theoretical Computer Science:

Nguyen & Onak [2008], Rubinfeld, Tamir, Vardi & Xie [2011], Suomella

[2011]

Mathematics:

Hatami, Lovasz & Szegedy [2012], Elek & Lippner [2010], Lyons &

Nazarov [2011], Czoka & Lippner [2012], Aldous [2012].

Context – graph limits.

Local Algorithms

Local algorithms and CD for combinatorial optimization.

Maximum Independent Set (MIS) problem

Local algorithms and CD for combinatorial optimization.

Maximum Independent Set (MIS) problem

Berman and Karpinski [1999]. NP-hard to approximate for

• Extensions to arbitrary constant degree graphs with weights which are

mixtures of exponentials.

• Proof technique – local algorithm (Cavity Expansion) and correlation decay

Maximum Weight Independent Set (MWIS) problem

Computational results.

G & Yedidia [2013]

Instance I

UB: 569

LB: 568

GREEDY: 420

Instance II

UB: 542

LB: 540

GREEDY: 397

Instance III

UP: 514

LB: 513

GREEDY 392

Instance IV

UB: 499

LB: 498

GREEDY: 384

Cavity Expansion Algorithm for general (non-tree) graphs

Cavity Expansion

Cavity Expansion

Problem: node cavity ! set cavity

Solution inspired by Weitz [2006] – expand set cavity as a sum of node cavities

Cavity Expansion

Theorem. Operator

is contracting – correlation decay takes place.

Cavity Expansion Algorithm

1. Compute using depth-m computation tree

2. Include iff

- Computation tree

Correlation decay property:

Probability of error

Computation time PTAS

Generalization:Optimization on networks with random rewards Economic theory of teams

Theorem. [G, Goldberg and Weber 2009]. Suppose

Then the model exhibits the correlation decay property. As a result the

Cavity Expansion algorithm provides an asymptotically optimal solution.

Generalization: Optimization on networks with random rewards

Random n-node d-regular graph

Local algorithms as i.i.d. factors. Mathematicians

perspective

Local algorithms – i.i.d. factors

1. Fix r>0 and

2. Generate U1,…,Un uniformly at random and apply at every node

i=1,…,n of the graph

3. Output

Local algorithms – i.i.d. factors

Conjecture. Hatami, Lovasz & Szegedy [2012] Max-Independent-Set

on random regular graph problem can be solved by means of local

algorithms. Namely there exists a sequence of functions such that

Conjecture holds for Max-Matching:

Lyons & Nazarov [2011]

Abert, Csoka, Lippner & Terpai [2012]

Best known bound 0.4361 for 3-regular graph is obtained by i.i.d. factors

Csoka, Gerencsér, Harangi, & Virag [2013]

Asymptotics of optimal solutions

Theorem. Bayati,G,Tetali 2010 problem exists w.h.p.

Theorem. Achlioptas & Peres [2003] K-SAT problem. Satisfiability

threshold

Theorem. Frieze & Luczak 1992

Best known algorithms

Greedy algorithms for Max Ind Set problem

Greedy algorithms for K-SAT problem. Finds satisfying assignment when

the clauses to variables ratio is at most

Theorem. [G & Sudan 2012] HLS conjecture is not valid. No local algorithm

can produce an independent set larger than factor of the

optimal for large enough d.

Proof relies on the Spin Glass Theory - clustering (shattering) phenomena

Main Result

Mezard & Parisi [1980s]

Achlioptas & Ricci-Tersenghi [2004]

Mezard, Mora & Zecchina [2005]

Krzakala, Montanari, Ricci-Tersenghi, Semerjian, Zdeborova [2007]

Clustering Phenomena. K-SAT

Clustering Phenomena. Independent Sets

Theorem. [Coja-Oghlan & Efthymiou 2010, G & Sudan 2012]

For every large enough d and every there exist

such that no two independent sets of size

have intersection size between and

Proof of non-existence of local algorithms: if an algorithm exists then

one can construct two independent sets with intersection in the non-

existent region

First direct link between clustering and algorithmic hardness

Proof of non-existence of local algorithms: if an algorithm exists then

one can construct two independent sets with intersection in the non-

existent region

Suppose exists such that

Construct two independent sets I and J using independent sources

U1,…,Un and V1,…,Vn

Then

Proof of non-existence of local algorithms: if an algorithm exists then

one can construct two independent sets with intersection in the non-

existent region

Continuously interpolate between U1,…,Un and V1,…,Vn : let

For each vertex , let with probability p and with

probability 1-p.

Consider independent set K obtained from

K=I when p=1 and K=J when p=0.

Fact: is continuous in p. Then we obtain intersection sizes for

all points in

- contradiction.

Theory is good not when it is correct but when it is interesting…

Ongoing work

Local algorithms (Belief Propagation and Survey Propagation) to solve

random instance of NAE-K-SAT problem

This would refute a conjecture put forward by physicists regarding the

power of these algorithms

Thank you