Limits of Local Algorithms for Randomly Generated Constraint Satisfaction Problems

Limits of local algorithms for random graphsLimits of local algorithms for random graphs

David Gamarnik

Joint work with Madhu Sudan (Microsoft Research, New England)

Yandex workshop on extremal combinatorics

June 24, 2014

Local Algorithms

Computer Science. Applications: hardware, fault-tolerant models of computation, sensor networks.

Lynch (book) [1996]Nguyen & Onak [2008]Rubinfeld, Tamir, Vardi, Xie [2011]Suomela (survey) [2011]

Wireless communicationsApplications: communication protocols with low overhead

Shah [2008] Shin & Shah [2012]

Who is interested in local algorithms?

Network Science. Economic team modelingApplications: models of social interaction, structure of social networks

Rusmevichientong, Van Roy [2003]Judd, Kearns & Vorobeychik [2010] Borgs, Brautbar, Chayes, Khanna & Lucier [2012]

Signal Processing. Machine learning.Applications: image processing, coding theory, wireless communication, gossip algorithms.

Wainwright & Jordan [2008], Mezard & Montanari [2009], Shah [2008] Shin & Shah [2012]

Physics.Applications: Spin glass theoryMezard & Montanari [2009]

Who is interested in local algorithms?

Mathematics/Combinatorics.Applications: graph limits.

Lovasz (book) [2012] Hatami, Lovasz & Szegedy [2012] Elek & Lippner [2010] Lyons & Nazarov [2011] Czoka & Lippner [2012] Aldous [2012].

Who is interested in local algorithms?

Random n-node d-regular graph

I. Local algorithms for random graphs : i.i.d. factors

locally regular tree like

Local algorithms: i.i.d. factors

Hatami-Lovasz-Szegedy [2012] framework

Generate i.i.d. U[0,1] weights

Apply some local rule f: decorated tree ! {0,1} for every node

f

0 or 1

Conjecture. Hatami, Lovasz & Szegedy [2012] There exists a local rule f which produces a nearly largest independent set in a random d-regular graph

Example. f =1 iff the weight of the node is larger than the weights of all of its neighbors

IN OUT

far from optimal !

Local algorithms: i.i.d. factors

Algorithms for independent sets in random graphs

Frieze & Luczak [1992]

Best known algorithm: Greedy

Hatami-Lovasz-Szegedy Conjecture: there exists f such that

Theorem. [G & Sudan ] Hatami-Lovasz-Szegedy conjecture is not valid: No

local rule f can produce an independent set larger than factor of the optimal, for large enough d.

Theorem. [Rahman & Virag 2014] No local rule f can produce an independent set larger than factor of the optimal, for large enough d.

Notes:

1. Rahman & Virag’s result is the best possible: factor ½ can be achieved by local algorithms, Lauer & Wormald [2007], G & Goldberg [2010]

2. Proof technique. Spin Glass theory: geometry of large independent sets

Main result

Geometry of solutions: clustering phenomena

Coja-Oghlan & Efthymiou [2010], G & Sudan [2012] Every two ‘’large’’ independent sets either have a very ‘’small’’ or a very ‘’large’’ intersection:

Geometry of solutions: clustering phenomena

Coja-Oghlan & Efthymiou [2010], G & Sudan [2012] Every two ‘’large’’ independent sets either have a very ‘’small’’ or a very ‘’large’’ intersection:

Proof sketch: argue by contradiction. Suppose rule f exists

?

(a) Generate two i.i.d. sequences

Proof sketch: argue by contradiction. Suppose rule f exists

(b) Interpolate between the two sequences

Proof sketch: argue by contradiction. Suppose rule f exists

(b) Interpolate between the two sequences

Proof sketch: argue by contradiction. Suppose rule f exists

(b) Interpolate between the two sequences

Proof sketch: argue by contradiction. Suppose rule f exists

(b) Interpolate between the two sequences

Proof sketch: argue by contradiction. Suppose rule f exists

(b) Interpolate between the two sequences

Proof sketch: argue by contradiction. Suppose rule f exists

(b) Interpolate between the two sequences

Proof sketch: argue by contradiction. Suppose rule f exists

(b) Interpolate between the two sequences

Proof sketch: argue by contradiction. Suppose rule f exists

(b) Interpolate between the two sequences

Intersection size belongs to the ‘’non-existent interval’’

Theory is great not when it is correct but when it is interesting… Murray Davis, 1 9 7 1

II. Sequential local algorithms for random NAE-K-SAT problem

Not-All-Equal-K-Satisfiability formula on n boolean variables and m clauses.Each clause contains K (i.e. K=3)

Assignment is satisfiable if every clause is satisfied by at least one variable, and is not satisfied by at least some other variable

Formula is satisfiable if there exists at least one satisfying assignment.

NP-Complete.

Sequential local algorithms for random NAE-K-SAT problem

Random NAE-K-SAT problem

Generate m clauses independently uniformly at random from the space of all possible clauses (with replacement). d=m/n – clause density.

Theorem. [Coja-Oglan and Panagiotou 2014] The threshold for probability that the formula is satisfiable

Best algorithm Unit Clause succeeds only when

Achlioptas et al. 2001

K gap!

Sequential local algorithms for the random NAE-K-SAT problem

Sequential r-local algorithms

1.Fix a “local rule” which maps formula © with variable x to probability value.

2.For i=1,2,…,n apply rule ¿ to variables xi and depth r sub-formulas ©i rooted at xi : fix xi to be 1 with probability ¿(©i, xi), and 0 with probability 1- ¿(©i, xi)

Notes:

Belief Propagation (BP) guided decimation and Survey Propagation (SP) guideddecimation algorithms are sequential local algorithms when the number of iterations is bounded by a constant independent of the number of variables

Unit Clause algorithm is a sequential local algorithm

Belief Propagation and Survey Propagation algorithms – message passing type algorithms proposed by statistical physicists. Work remarkably well for low values of K (K=3,4,5).

Mezard, Parisi & Zecchina [2003]Krzakala, Montanari, Ricci-Tersenghi, Semerjian, Zdeborova [2007]Mezard & Montanari (book) [2009]

Theorem. G & Sudan [2013] Every sequential local algorithm fails to find a satisfying assignment when

simple algorithm (Unit Clause) exist

no sequential local algorithms exist

no solutions exist

Sequential local algorithms cannot bridge the 1/K gap

Proof Technique

(a) Clustering property for random NAE-K-SAT(b) Interpolation between m randomly generated solutions(c) Decisions for variables are localized

UNSAT

Proof Technique

(a) Clustering property for random NAE-K-SAT

Theorem. Let

With probability approaching unity as n increases, there does not exist m satisfying assignments such that all pairwise Hamming distances are

Proof: first moment method.

Some further thoughts

• The approach should apply to other problems with “symmetry”, for example coloring of graphs. But applying the approach to random K-SAT is problematic.

• In practice Survey Propagation algorithm is run for many iterations. Challenge: establish similar result when there is no bound on the number of iterations. Coja-Oghlan [2010] – true for Belief Propagation for K-SAT.

• Establish limits for the performance of local algorithms for random constraint satisfaction problems using the most general (Computer Science) definition of local algorithms.

Mick can’t get satisfaction above (from Sequential Local Algorithms)

Chvatal & Reed [1998] ’’Mick gets some (the odds are on his side) [satisfiability]’’

Parting thoughts …

Thank you