View
224
Download
3
Category
Preview:
Citation preview
Phase transitions in discrete structures
Amin Coja-Oghlan
Goethe University Frankfurt
Overview
1 The physics approach. [following Mezard, Montanari ’09]I Basics.I Replica symmetry (“Belief Propagation”).I The 1RSB cavity method.
2 “Classical” rigorous work.I The second moment method.I “Quiet planting”.
3 A physics-inspired rigorous approach.I The Kauzmann transition.I The free entropy in the 1RSB phase.
4 Random k-SAT.I A rigorous Belief Propagation-based approach.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 2 / 40
The Boltzmann distribution
Let X be a finite set of spins and let N be a “large” integer.
Consider a probability distribution on XN defined by
µ(x) =1
Z
M∏a=1
ψa(x), with Z =∑x∈XN
M∏a=1
ψa(x).
Assume that ψa depends only on the components ∂a ⊂ [N] of x .
Thus, ψa(x) = ψa(x∂a).
We call µ the Boltzmann distribution and Z the partition function.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 3 / 40
The factor graph
Suppose we are given the Boltzmann distribution
µ(x) =1
Z
M∏a=1
ψa(x∂a).
We set up the bipartite factor graph.I Square vertices: the factors ψa, a = 1, . . . ,M.I Round vertices: the variables x1, . . . , xM .I Conncect each factor ψa with all variables xi , i ∈ ∂a.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 4 / 40
The factor graph
Example: the 1-dimensional Ising model
There are N “sites” with spins in X = −1, 1.For any a ∈ 1, . . . ,N − 1 we define
ψa(x) = exp(βxaxa+1) (β > 0).
Hence, ∂a = a, a + 1.
This yields µ(x) ∝∏M
a=1 exp(βxaxa+1) = exp[β∑N−1
a=1 xaxa+1
].
Amin Coja-Oghlan (Frankfurt) Random k-SAT 5 / 40
The factor graph
Example: the Potts antiferromagnet
Let X = [K ] = 1, 2, . . . ,K.Let G = (V ,E ) be a graph on V = [N].
For an edge e joining vertices i , j in G let
ψe(x) = exp[−β · 1xi=xj
].
This gives rise to
µ(x) =∏e∈E
ψe(x) = exp [−β ·#monochromatic edges] .
Amin Coja-Oghlan (Frankfurt) Random k-SAT 6 / 40
The factor graph
Example: the Potts antiferromagnet
Let X = [K ] = 1, 2, . . . ,K.Let G = (V ,E ) be a graph on V = [N].
For an edge e joining vertices i , j in G let
ψe(x) = exp[−β · 1xi=xj
].
This gives rise to
µ(x) =∏e∈E
ψe(x) = exp [−β ·#monochromatic edges] .
Amin Coja-Oghlan (Frankfurt) Random k-SAT 6 / 40
The free entropy
Generally, the partition function Z scales exponentially with N.
Therefore, we are interested in the free entropy 1N ln Z .
In fact, we care mostly about the theormdynamic limit N →∞, i.e.,
limN→∞
1
Nln Z ,
the free entropy density.
I Does this limit exist?I If so, can we compute/approximate it?I Is the limit an analytic function, or are there phase transitions?
Example: the Ising model
1-dim: the free entropy density is analytic, i.e., no phase transition.
2-dim: there is a phase transition. [Onsager 1949]
Amin Coja-Oghlan (Frankfurt) Random k-SAT 7 / 40
The free entropy
Generally, the partition function Z scales exponentially with N.
Therefore, we are interested in the free entropy 1N ln Z .
In fact, we care mostly about the theormdynamic limit N →∞, i.e.,
limN→∞
1
Nln Z ,
the free entropy density.I Does this limit exist?I If so, can we compute/approximate it?I Is the limit an analytic function, or are there phase transitions?
Example: the Ising model
1-dim: the free entropy density is analytic, i.e., no phase transition.
2-dim: there is a phase transition. [Onsager 1949]
Amin Coja-Oghlan (Frankfurt) Random k-SAT 7 / 40
The free entropy
Generally, the partition function Z scales exponentially with N.
Therefore, we are interested in the free entropy 1N ln Z .
In fact, we care mostly about the theormdynamic limit N →∞, i.e.,
limN→∞
1
Nln Z ,
the free entropy density.I Does this limit exist?I If so, can we compute/approximate it?I Is the limit an analytic function, or are there phase transitions?
Example: the Ising model
1-dim: the free entropy density is analytic, i.e., no phase transition.
2-dim: there is a phase transition. [Onsager 1949]
Amin Coja-Oghlan (Frankfurt) Random k-SAT 7 / 40
Disordered systems
Two levels of randomness
The Boltzmann distribution itself is random.
Hence, the free entropy density becomes
limN→∞
E
[1
Nln Z
].
Example: the Edwards-Anderson model
With x as in the 1-dim Ising model, we let
µ(x) ∝ exp
[β
N−1∑i=1
Jixixi+1
].
The Ji are independent Gaussian random variables.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 8 / 40
Disordered systems
Two levels of randomness
The Boltzmann distribution itself is random.
Hence, the free entropy density becomes
limN→∞
E
[1
Nln Z
].
Example: the Edwards-Anderson model
With x as in the 1-dim Ising model, we let
µ(x) ∝ exp
[β
N−1∑i=1
Jixixi+1
].
The Ji are independent Gaussian random variables.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 8 / 40
Disordered systems
Example: the diluted mean-filed Potts antiferromagnet
Let X = 1, . . . ,K for some “small” K ≥ 2.
Consider a random graph G (N,M) on N vertices with M edges.
For any edge e of G (N,M) joining vertices i , j , define
ψe(x) = exp[−β · 1xi=xj
].
The Boltzmann distribution
µ(x) ∝∏e
ψe(x) = exp [−β ·#monochromatic edges]
is random as it depends on the graph G (N,M).
New parameter: the denisty α = M/N.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 9 / 40
Disordered systems
Example: the diluted mean-filed Potts antiferromagnet
Let X = 1, . . . ,K for some “small” K ≥ 2.
Consider a random graph G (N,M) on N vertices with M edges.
For any edge e of G (N,M) joining vertices i , j , define
ψe(x) = exp[−β · 1xi=xj
].
The Boltzmann distribution
µ(x) ∝∏e
ψe(x) = exp [−β ·#monochromatic edges]
is random as it depends on the graph G (N,M).
New parameter: the denisty α = M/N.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 9 / 40
Disordered systems
Example: the diluted mean-filed Potts antiferromagnet
Let X = 1, . . . ,K for some “small” K ≥ 2.
Consider a random graph G (N,M) on N vertices with M edges.
For any edge e of G (N,M) joining vertices i , j , define
ψe(x) = exp[−β · 1xi=xj
].
The Boltzmann distribution
µ(x) ∝∏e
ψe(x) = exp [−β ·#monochromatic edges]
is random as it depends on the graph G (N,M).
New parameter: the denisty α = M/N.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 9 / 40
Disordered systems
Example: the diluted mean-filed Potts antiferromagnet
Let X = 1, . . . ,K for some “small” K ≥ 2.
Consider a random graph G (N,M) on N vertices with M edges.
For any edge e of G (N,M) joining vertices i , j , define
ψe(x) = exp[−β · 1xi=xj
].
The Boltzmann distribution
µ(x) ∝∏e
ψe(x) = exp [−β ·#monochromatic edges]
is random as it depends on the graph G (N,M).
New parameter: the denisty α = M/N.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 9 / 40
Disordered systems
Example: coloring random graphs
Let X = 1, . . . ,K for some “small” K ≥ 3.
For any edge e of G (N,M) joining vertices i , j , define
ψe(x) = exp[−∞ · 1xi=xj
]= 1xi 6=xj [“zero temperature”].
Thenµ(x) ∝
∏e
ψe(x) = 1no edge is monochromatic.
is the uniform distribution over proper K -colorings of G (N,M).
Amin Coja-Oghlan (Frankfurt) Random k-SAT 10 / 40
Disordered systems
Question [Erdos, Renyi 1960]
Is there a phase transition in this model (in terms of α = M/N)?
Amin Coja-Oghlan (Frankfurt) Random k-SAT 10 / 40
Disordered systems
Example: random k-SAT
Let X = 0, 1 and fix some “small” K ≥ 3.
Think of x1, . . . , xN as Boolean variables.
Let Φ be an expression of the form
(x1 ∨ x17 ∨ · · · ∨ x29)︸ ︷︷ ︸k-clause Φ1
∧ (x11 ∨ x2 ∨ · · · ∨ x1)︸ ︷︷ ︸k-clause Φ2
∧ · · ·
with M “clauses”, chosen uniformly at random.
For i = 1, . . . ,M let
ψi (x) = 1clause Φi evaluates to true.
Then µ(x) =∏M
i=1 ψi (x) is uniform over satisfying assignments.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 11 / 40
Belief Propagation
Goals
To compute the free entropy on trees.I the Belief Propagation equations.I the Bethe free entropy.
The replica symmetric ansatz.I Belief Propagation as a distributional fixed point equation.I application to “diluted mean-field models”.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 12 / 40
Example: the 1-dimensional Ising model
Let X = ±1, x ∈ XN and
µ(x) ∝ exp
[β
N−1∑i=1
xixi+1 + βBN∑i=1
xi
].
Goal: to compute the marginal µ(xi ) of xi .
Amin Coja-Oghlan (Frankfurt) Random k-SAT 13 / 40
Example: the 1-dimensional Ising model
For j ∈ [N] define two distributions ν→j , νj→ on X by
ν→j(xj) ∝∑
x1,...,xj−1∈Xexp
[β
j−1∑i=1
xixi+1 + βB
j−1∑i=1
xi
],
νj←(xj) ∝∑
xj+1,...,xN∈Xexp
β N−1∑i=j
xixi+1 + βBN∑
i=j+1
xi
.
Thenµ(xj) ∝ ν→j(xj) · exp(βBxj) · νj←(xj).
Moreover, we have the recurrence
ν→j+1(xj+1) =∑xj∈X
ν→j(xj) exp [βxjxj+1 + βBxj ] .
Amin Coja-Oghlan (Frankfurt) Random k-SAT 14 / 40
Example: the 1-dimensional Ising model
For j ∈ [N] define two distributions ν→j , νj→ on X by
ν→j(xj) ∝∑
x1,...,xj−1∈Xexp
[β
j−1∑i=1
xixi+1 + βB
j−1∑i=1
xi
],
νj←(xj) ∝∑
xj+1,...,xN∈Xexp
β N−1∑i=j
xixi+1 + βBN∑
i=j+1
xi
.Then
µ(xj) ∝ ν→j(xj) · exp(βBxj) · νj←(xj).
Moreover, we have the recurrence
ν→j+1(xj+1) =∑xj∈X
ν→j(xj) exp [βxjxj+1 + βBxj ] .
Amin Coja-Oghlan (Frankfurt) Random k-SAT 14 / 40
Example: the 1-dimensional Ising model
For j ∈ [N] define two distributions ν→j , νj→ on X by
ν→j(xj) ∝∑
x1,...,xj−1∈Xexp
[β
j−1∑i=1
xixi+1 + βB
j−1∑i=1
xi
],
νj←(xj) ∝∑
xj+1,...,xN∈Xexp
β N−1∑i=j
xixi+1 + βBN∑
i=j+1
xi
.Then
µ(xj) ∝ ν→j(xj) · exp(βBxj) · νj←(xj).
Moreover, we have the recurrence
ν→j+1(xj+1) =∑xj∈X
ν→j(xj) exp [βxjxj+1 + βBxj ] .
Amin Coja-Oghlan (Frankfurt) Random k-SAT 14 / 40
Example: the 1-dimensional Ising model
We have ν→j+1(xj+1) =∑
xj∈X ν→j(xj) exp [βxjxj+1 + βBxj ] .
Setting
u→j =1
2βln
ν→j(1)
ν→j(−1),
we can rephrase the above as
u→j+1 = f (u→j + B) with f (x) = β−1 atanh(tanh(β) tanh(βx)).
The function f has a unique fixed point u∗.
Hence, for j sufficiently far from the boundary, we find
µ(xj) ∼ tanh(β(2u∗ + B)).
Amin Coja-Oghlan (Frankfurt) Random k-SAT 15 / 40
Belief Propagation on trees
Suppose that the factor graph of µ(x) is a tree.
Generalising the above example, we define messages recursively by
ν(t+1)j→a (xj) ∝
∏b∈∂j\a
ν(t)b→j(xj),
ν(t)b→j(xj) ∝
∑x∂b\j
ψb(x∂b)∏
i∈∂b\j
ν(t)i→b(xi ).
I t counts the number of recursive steps,I ∂j is the neighborhood of j ,I the sum ranges over X ∂b\j , with xj is given upfront.
Define the Belief Propagation marginal of variable xj as
ν(t)(xj) ∝∏b∈∂j
ν(t−1)b→j (xj).
Amin Coja-Oghlan (Frankfurt) Random k-SAT 16 / 40
Belief Propagation on trees
Suppose that the factor graph of µ(x) is a tree.
Generalising the above example, we define messages recursively by
ν(t+1)j→a (xj) ∝
∏b∈∂j\a
ν(t)b→j(xj),
ν(t)b→j(xj) ∝
∑x∂b\j
ψb(x∂b)∏
i∈∂b\j
ν(t)i→b(xi ).
I t counts the number of recursive steps,I ∂j is the neighborhood of j ,I the sum ranges over X ∂b\j , with xj is given upfront.
Define the Belief Propagation marginal of variable xj as
ν(t)(xj) ∝∏b∈∂j
ν(t−1)b→j (xj).
Amin Coja-Oghlan (Frankfurt) Random k-SAT 16 / 40
Belief Propagation on trees
Theorem
Assume that the factor graph is a tree of diameter T .
1 The BP equations have a unique fixed point ν∗.
2 For t > T we have ν(t) = ν∗, regardless of the initialisation.
3 For all variables xj we have
ν∗(xj) = µ(xj),
i.e., the BP marginals coincide with the Boltzmann marginals.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 17 / 40
Belief Propagation on trees
Theorem (the Bethe free entropy)
Given the messages ν, define
F (ν) =M∑a=1
Fa(ν) +N∑i=1
Fi (ν)−∑(i ,a)
Fia(ν), where
Fa(ν) = ln∑x∂a
ψa(x∂a)∏i∈∂a
νi→a(xi ),
Fi (ν) = ln∑xi
∏b∈∂i
νb→i (xi ),
Fia(ν) = ln∑xi
νi→a(xi )νa→i (xi ).
If the factor graph is a tree, then ln Z = F (ν∗).
Amin Coja-Oghlan (Frankfurt) Random k-SAT 18 / 40
Belief Propagation on trees
On trees, BP converges to its unique fixed point.
This fixed point yieldsI the correct marginals,I the free entropy.
Example: the 1-dim Ising model.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 19 / 40
Belief Propagation on infinite trees
Question
Does this formalism survive the thermodynamic limit?
That is, does it extend to infinite trees?
Examples of infinite trees
The infinite regular tree.I Each variable/factor node has the same number of outgoing edges.I The factors ψa are identical.
Infinite random trees.I The numbers of outgoing edges is are independent Poisson variables.I The factors ψa might have some randomness, too.
Generally, we assume that sub-trees are identically distributed.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 20 / 40
Belief Propagation on infinite trees
Question
Does this formalism survive the thermodynamic limit?
That is, does it extend to infinite trees?
Examples of infinite trees
The infinite regular tree.I Each variable/factor node has the same number of outgoing edges.I The factors ψa are identical.
Infinite random trees.I The numbers of outgoing edges is are independent Poisson variables.I The factors ψa might have some randomness, too.
Generally, we assume that sub-trees are identically distributed.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 20 / 40
Belief Propagation on infinite trees
The distributional BP equations
We interpret the Belief Propagation equations
νj→a(xj) ∝∏
b∈∂j\a
νb→j(xj),
νb→j(xj) ∝∑x∂b\j
ψb(x∂b)∏
i∈∂b\j
νi→b(xi ).
as distributional fixed point equations.
That is, ifI ψb, |∂b|, |∂j | are i.i.d. from the distribution defining the tree,I the νi→b are i.i.d. from a distribution ν∗ over distributions,
then νj→a has distribution ν∗.
The Bethe free entropy is determined by ν∗.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 21 / 40
Belief Propagation on infinite trees
Example: coloring regular trees
Consider the graph K -coloring problem on the d-ary tree.
By symmetry, all the marginals are identical, i.e.,
νj→a(xj) = 1/K for all j , xj .
Thus, ν∗ is the measure concentrated on the uniform distribution.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 22 / 40
Belief Propagation on infinite trees
Example: random K -SAT
Each variable node has a Po(α) number of children.
The factors ψa are i.i.d. random clauses of length K .
The fixed point distribution ν∗I is not known to exist,I but is conjectured to be highly non-trivial and not discrete,I based on numerical evidence (“population dynamics”).
This mirrors the asymmetric combinatorics of the problem.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 23 / 40
The replica symmetric ansatz
Diluted mean-field models
Think of the factor graphs in models such asI random graph K -coloring,I random K -SAT.
The factor graph is an Erdos-Renyi-like sparse random graph.
Thus, roughly speaking,I the vertices are bounded degree,I the shortest cycle has length Ω(ln N).
In other words, locally the graph looks like the infinite random tree!
Amin Coja-Oghlan (Frankfurt) Random k-SAT 24 / 40
The replica symmetric ansatz
The replica symmetric “solution”
Let ν∗ be the BP fixed point on the ∞ tree.
Let φ∗ be the resulting Bethe free entropy.
Then the replica symmetric prediction is that
φ∗ = limN→∞
E[ln Z ]
N,
with Z the partition function in, e.g., random K -SAT.
Hypotheses
The fixed point ν∗ exists and is only one “relevant” fixed point.
There is correlation decay.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 25 / 40
The replica symmetric ansatz
The replica symmetric “solution”
Let ν∗ be the BP fixed point on the ∞ tree.
Let φ∗ be the resulting Bethe free entropy.
Then the replica symmetric prediction is that
φ∗ = limN→∞
E[ln Z ]
N,
with Z the partition function in, e.g., random K -SAT.
Hypotheses
The fixed point ν∗ exists and is only one “relevant” fixed point.
There is correlation decay.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 25 / 40
The replica symmetric ansatz
Example: ferromagnetic Ising on sparse random graphs [DM ’08]
The replica symmetric solution is correct.
Paramagnetic/ferromagnetic phase transition.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 26 / 40
The replica symmetric ansatz
Example: coloring random graphs
Let K = #colors, α = M/N density of the graph.
Then Z = #K -colorings of G (N,M).
According to the replica symmetric solution,
1
NE[ln Z ] ∼ 1
NlnE[Z ] = ln K + α ln(1− 1/K ).
The col/uncol transition occurs at α = (K − 12) ln K + oK (1).
Rigorous work: [ACO, Vilenchik ’13]I The r.s. solution is correct only for
α < (K − 1
2) ln K − ln 2 + oK (1).
I The col/uncol transition occurs at α ≤ (K − 12 ) ln K − 1
2 + oK (1).
Amin Coja-Oghlan (Frankfurt) Random k-SAT 27 / 40
Multiple Belief Propagation fixed points
Random factor graphs
In random graph K -coloring, the factor graph is similar to G (N,M).
Thus, it’s far from being a tree.
In effect, there can be multiple BP fixed points.
These mirror long range correlations.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 28 / 40
Multiple Belief Propagation fixed pointsxi
w
The reconstruction problem
Suppose µ is defined by a factor graph G .
Pick a variable xi and let Gω be the depth-ω neighborhood.
What is the impact of the boundary on xi?
Amin Coja-Oghlan (Frankfurt) Random k-SAT 29 / 40
Multiple Belief Propagation fixed points
Correlation decay: Gibbs uniqueness
Let yω be a boundary configuration.
The worst-case influence of the boundary can be cast as
E[supyω|µ (xi |yω)− µ(xi )|].
Here the expectation is over the choice of µ.
Iflim
ω→∞lim
N→∞E[sup
yω|µ (xi |yω)− µ(xi )|] = 0,
we say there is Gibbs uniqueness.
Observe that this is a purely local property.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 30 / 40
Multiple Belief Propagation fixed points
Correlation decay: non-reconstruction
The average influence of the boundary configuration can be cast as
E∑yω
µ(yω)|µ (xi |yω)− µ(xi )|.
Iflim
ω→∞lim
N→∞E∑yω
µ(yω)|µ (xi |yω)− µ(xi )| = 0,
we say there is non-reconstruction.
Otherwise reconstruction is possible.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 31 / 40
Multiple Belief Propagation fixed points
Theorem [Achlioptas, ACO ’08]
If the density satisfies
α = M/N >
(1
2+ ε
)K ln K ,
then in random graph K -coloring reconstruction is possible.
In fact, suppose we initialise BP with one random K -coloring.
Then for Ω(n) variables xi the BP marginals satisfy ν∗(xi ) = 1.
Thus, these variables are “locally frozen”.
Interpretation: µ decomposes into a large number of clusters.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 32 / 40
Multiple Belief Propagation fixed points
Theorem [Achlioptas, ACO ’08]
If the density satisfies
α = M/N >
(1
2+ ε
)K ln K ,
then in random graph K -coloring reconstruction is possible.
In fact, suppose we initialise BP with one random K -coloring.
Then for Ω(n) variables xi the BP marginals satisfy ν∗(xi ) = 1.
Thus, these variables are “locally frozen”.
Interpretation: µ decomposes into a large number of clusters.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 32 / 40
The 1RSB cavity method
1-step replica symmetry breaking
In the above scenario, µ is a convex combination
µ =N∑i=1
wiµi
of probability measures µi (corresponding to “clusters”).
Here wi ∝ exp(Fi · N), with Fi the free entropy of µi .
Key idea: perform Belief Propagation on the level of the µi !
The parameter β is replaced by the Parisi parameter y .
Amin Coja-Oghlan (Frankfurt) Random k-SAT 33 / 40
The 1RSB cavity method
Idea
A Boltzmann distribution on the µi .
This distribution is defined by
Py (µi ) ∝ w yi .
In terms of the complexity function
Σ(φ) =1
Nln # i ∈ [N ] : Fi ∼ φ ,
the partition function Ξ(y) of ξy satisfies
Ξ(y) = yφ+ Σ(φ), with φ s.t.∂Σ
∂φ= −y .
Amin Coja-Oghlan (Frankfurt) Random k-SAT 34 / 40
The 1RSB cavity method
Belief Propagation one level up
What is the factor graph of Py?I The “old” messages νi→a, νa→i are the variables.I There are factor nodes enforcing the BP equations.I Additional factor nodes implement the Parisi parameter y .I (This is possible due to Bethe’s formula.)
On this factor graph, write the Belief Propagation equations.I 1RSB cavity equations.I Complication: the spins are continuous.
The BP fixed point for Py yields Σ(φ).I Analogous to the Bethe free entropy.
The 1RSB equations can be viewed as distributional equations.I the thermodynamic limit.
Setting y = 0 yields the Survey Propagation equations.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 35 / 40
The 1RSB cavity method
Belief Propagation one level up
What is the factor graph of Py?I The “old” messages νi→a, νa→i are the variables.I There are factor nodes enforcing the BP equations.I Additional factor nodes implement the Parisi parameter y .I (This is possible due to Bethe’s formula.)
On this factor graph, write the Belief Propagation equations.I 1RSB cavity equations.I Complication: the spins are continuous.
The BP fixed point for Py yields Σ(φ).I Analogous to the Bethe free entropy.
The 1RSB equations can be viewed as distributional equations.I the thermodynamic limit.
Setting y = 0 yields the Survey Propagation equations.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 35 / 40
The 1RSB cavity method
Belief Propagation one level up
What is the factor graph of Py?I The “old” messages νi→a, νa→i are the variables.I There are factor nodes enforcing the BP equations.I Additional factor nodes implement the Parisi parameter y .I (This is possible due to Bethe’s formula.)
On this factor graph, write the Belief Propagation equations.I 1RSB cavity equations.I Complication: the spins are continuous.
The BP fixed point for Py yields Σ(φ).I Analogous to the Bethe free entropy.
The 1RSB equations can be viewed as distributional equations.I the thermodynamic limit.
Setting y = 0 yields the Survey Propagation equations.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 35 / 40
The 1RSB cavity method
Belief Propagation one level up
What is the factor graph of Py?I The “old” messages νi→a, νa→i are the variables.I There are factor nodes enforcing the BP equations.I Additional factor nodes implement the Parisi parameter y .I (This is possible due to Bethe’s formula.)
On this factor graph, write the Belief Propagation equations.I 1RSB cavity equations.I Complication: the spins are continuous.
The BP fixed point for Py yields Σ(φ).I Analogous to the Bethe free entropy.
The 1RSB equations can be viewed as distributional equations.I the thermodynamic limit.
Setting y = 0 yields the Survey Propagation equations.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 35 / 40
The 1RSB cavity method
Table 1. Critical connectivities for the dynamical, con-densation and satisfiability transitions in k-SAT and q-COLSAT !d !c !s[11] COL ld[16] lc ls[14]k = 4 9.38 9.547 9.93 q = 4 9 10 10k = 5 19.16 20.80 21.12 q = 5 14 14 15k = 6 36.53 43.08 43.4 q = 6 18 19 20
unsatisfiable with high probability2 [10]. For k-SAT, it is known that!s(2) = 1. A conjecture based on the cavity method was put forwardin [6] for all k ! 3 that implied in particular the values presented inTa-ble 1 and !s(k) = 2k log 2" 1
2(1+log 2)+O(2!k) for large k [11].
Subsequently it was proved that !s(k) ! 2k log 2"O(k) confirmingthis asymptotic behavior [12]. An analogous conjecture for q-coloringwas proposed in [13] yielding, for regular random graphs [14], the val-ues reported inTable 1 and ls(q) = 2q log q"log q"1+o(1) for largeq (according to our convention, random graphs are whp uncolorable ifl ! ls(q)). It was proved in [15, 12] that ls(q) = 2q log q"O(log q).
Evenmore interesting and challenging are phase transitions in thestructure of the set S # X N of solutions of rCSP’s (‘structural’ phasetransitions). Assuming the existence of solutions, a convenient wayof describing S is to introduce the uniform measure over solutionsµ(x):
µ(x) =1
Z
MY
a=1
"a(xia(1), . . . , xia(k)) , [1]
where Z ! 1 is the number of solutions. Let us stress that, since Sdepends on the rCSP instance, µ( · ) is itself random.
We shall now introduce a few possible ‘global’ characterizationsof the measure µ( · ). Each one of these properties has its counter-part in the theory of Gibbs measures and we shall partially adopt thatterminology here [17].
In order to define the first of such characterizations, we let i $ [N ]be a uniformly random variable index, denote as x! the vector of vari-ables whose distance from i is at least #, and by µ(xi|x!) the marginaldistribution of xi given x!. Then we say that the measure [1] satisfiesthe uniqueness condition if, for any given i $ [N ],
E supx!,x!
!
X
xi"X
˛µ(xi|x!) " µ(xi|x#
!)˛% 0 . [2]
as # % & (here and below the limitN % & is understood to be takenbefore # % &). This expresses a ‘worst case’ correlation decay con-dition. Roughly speaking: the variable xi is (almost) independent ofthe far apart variables x! irrespective is the instance realization and thevariables distributionoutside the horizonof radius #. The threshold foruniqueness (above which uniqueness ceases to hold) was estimatedin [9] for random k-SAT, yielding !u(k) = (2 log k)/k[1 + o(1)](which is asymptotically close to the threshold for the pure literalheuristics) and in [18] for coloring implying lu(q) = q for q largeenough (a ‘numerical’ proof of the same statement exists for small q).Below such thresholds BP can be proved to return good estimates ofthe local marginals of the distribution [1].
Notice that the uniqueness threshold is far below the SAT-UNSATthreshold. Furthermore, several empirical studies [19, 20] pointed outthat BP (as well as many other heuristics [4, 5]) is effective up tomuchlarger values of the clause density. In a remarkable series of papers[21, 6], statistical physicists argued that a second structural phasetransition is more relevant than the uniqueness one. Following thisliterature, we shall refer to this as the ‘dynamic phase transition’ (DPT)and denote the corresponding threshold as !d(k) (or ld(q)). In orderto precise this notion, we provide here two alternative formulations
!d,+ !d !c !s
Fig. 2. Pictorial representation of the different phase transitions in the set ofsolutions of a rCSP. At!d,+ some clusters appear, but for!d,+ < ! < !d theycomprise only an exponentially small fraction of solutions. For!d < ! < !c thesolutions are split among about eN!" clusters of size eNs" . If !c < ! < !s
the set of solutions is dominated by a few large clusters (with strongly fluctuatingweights), and above !s the problem does not admit solutions any more.
corresponding to two distinct intuitions. According to the first one,above !d(k) the variables (x1, . . . , xN) become globally correlatedunder µ( · ). The criterion in [2] is replaced by one in which far apartvariables x! are themselves sampled from µ (‘extremality’ condition):
EX
x!
µ(x!)X
xi
|µ(xi|x!) " µ(xi)| % 0 . [3]
as # % &. The infimum value of ! (respectively l) such thatthis condition is no longer fulfilled is the threshold !d(k) (ld(k)).Of course this criterion is weaker than the uniqueness one (hence!d(k) ! !u(k)).
According to the second intuition, above !d(k), the measure[1] decomposes into a large number of disconnected ‘clusters’.This means that there exists a partition Ann=1...N of X N (de-pending on the instance) such that: (i) One cannot find n suchthat µ(An) % 1; (ii) Denoting by $"A the set of configurationsx $ X N\Awhose Hamming distance fromA is at mostN%, we haveµ($"An)/µ(An)(1 " µ(An)) % 0 exponentially fast inN for all nand % small enough. Notice that the measure µ can be decomposed as
µ( · ) =NX
n=1
wn µn( · ) , [4]
where wn ' µ(An) and µn( · ) ' µ( · |An). We shall always referto An as the ‘finer’ partition with these properties.
The above ideas are obviously related to the performance of algo-rithms. For instance, the correlation decay condition in [3] is likely tobe sufficient for approximate correctness of BP on random formulae.Also, the existence of partitions as above implies exponential slowingdown in a large class of MCMC sampling algorithms3.
Recently, some important rigorous results were obtained support-ing this picture [22, 23]. However, even at the heuristic level, severalcrucial questions remain open. The most important concern the dis-tribution of the weights wn: are they tightly concentrated (on anappropriate scale) or not? A (somewhat surprisingly) related ques-tion is: can the absence of decorrelation above !d(k) be detected byprobing a subset of variables bounded in N?
SP [6] can be thought as an inference algorithm for a modifiedgraphical model that gives unit weight to each cluster [24, 20], thustilting the original measure towards small clusters. The resulting per-formances will strongly depend on the distribution of the cluster sizeswn. Further, under the tilted measure, !d(k) is underestimated be-cause small clusters have a larger impact. The correct value was neverdetermined (but see [16] for coloring). The authors of [25] undertook
3One possible approach to the definition of a MCMC algorithm is to relax the constraints bysetting#a(· · · ) = " instead of 0 whenever the a-th constraint is violated. Glauber dynamicscan then be used to sample from the relaxed measure µ"( · ).
2 www.pnas.org — — Footline Author
The geometry of the Boltzmann distribution
The replica symmetric phase.
Dynamic 1RSB.
Static 1RSB (aka condensation, Kauzmann transition).
Amin Coja-Oghlan (Frankfurt) Random k-SAT 36 / 40
The 1RSB cavity method
Example: for what d is the random d-regular graph K -colorable?
Setting y = 0, we consider the distribution
P0(µi ) =1
N, N = #clusters.
Suppose that each µi is characterized by frozen variables.I Encode µi as a map ζi : [N]→ 1, 2, . . . ,K , ∗.I ∗ = unfrozen (“joker color”).
This gives rise to SP messages:I Qi→j(xi ) =P0-probability that w/out j , i is frozen to color xi .I Qi→j(xi ) =P0-probability that w/out j , i is unfrozen.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 37 / 40
The 1RSB cavity method
Example: for what d is the random d-regular graph K -colorable?
Setting y = 0, we consider the distribution
P0(µi ) =1
N, N = #clusters.
Suppose that each µi is characterized by frozen variables.I Encode µi as a map ζi : [N]→ 1, 2, . . . ,K , ∗.I ∗ = unfrozen (“joker color”).
This gives rise to SP messages:I Qi→j(xi ) =P0-probability that w/out j , i is frozen to color xi .I Qi→j(xi ) =P0-probability that w/out j , i is unfrozen.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 37 / 40
The 1RSB cavity method
Example: for what d is the random d-regular graph K -colorable?
Setting y = 0, we consider the distribution
P0(µi ) =1
N, N = #clusters.
Suppose that each µi is characterized by frozen variables.I Encode µi as a map ζi : [N]→ 1, 2, . . . ,K , ∗.I ∗ = unfrozen (“joker color”).
This gives rise to SP messages:I Qi→j(xi ) =P0-probability that w/out j , i is frozen to color xi .I Qi→j(xi ) =P0-probability that w/out j , i is unfrozen.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 37 / 40
The 1RSB cavity method
Example: for what d is the random d-regular graph K -colorable?
We have the fixed point equations
Qi→j(xi ) =
∑x∂i\j∈N (xi )
∏k∈∂i\j Qk→i (xk)∑
x∂i\j∈D∏
k∈∂i\j Qk→i (xk)(xi ∈ [K ]).
I D contains all (c1, . . . , cd−1) such that [K ] 6⊂ c1, . . . , cd−1.I N (x) contains all (c1, . . . , cd−1) ∈ D with [K ] \ x ⊂ c1, . . . , cd−1.
Let Qi→j(∗) = 1−∑
xiQi→j(xi ).
Simplification: it might seem reasonable to assume thatI Qi→j(xi ) = a for some fixed 0 ≤ a ≤ 1/q and all xi ∈ [K ].I Qi→j(∗) = 1− Ka.I The above equation becomes
a =
∑q−1r=0 (−1)r
(K−1r
)(1− (r + 1)a)d−1∑q−1
r=0 (−1)r(
qr+1
)(1− (r + 1)a)d−1
.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 38 / 40
The 1RSB cavity method
Example: for what d is the random d-regular graph K -colorable?
We have the fixed point equations
Qi→j(xi ) =
∑x∂i\j∈N (xi )
∏k∈∂i\j Qk→i (xk)∑
x∂i\j∈D∏
k∈∂i\j Qk→i (xk)(xi ∈ [K ]).
I D contains all (c1, . . . , cd−1) such that [K ] 6⊂ c1, . . . , cd−1.I N (x) contains all (c1, . . . , cd−1) ∈ D with [K ] \ x ⊂ c1, . . . , cd−1.
Let Qi→j(∗) = 1−∑
xiQi→j(xi ).
Simplification: it might seem reasonable to assume thatI Qi→j(xi ) = a for some fixed 0 ≤ a ≤ 1/q and all xi ∈ [K ].I Qi→j(∗) = 1− Ka.I The above equation becomes
a =
∑q−1r=0 (−1)r
(K−1r
)(1− (r + 1)a)d−1∑q−1
r=0 (−1)r(
qr+1
)(1− (r + 1)a)d−1
.
Amin Coja-Oghlan (Frankfurt) Random k-SAT 38 / 40
The 1RSB cavity method
Example: for what d is the random d-regular graph K -colorable?
We obtain the complexity ( 1N ln #clusters)
Σ = ln
[q−1∑r=0
(−1)r(
q
r + 1
)(1− (r + 1)a)d
]− d
2ln(1− Ka2)
with a the solution to
a =
∑q−1r=0 (−1)r
(K−1r
)(1− (r + 1)a)d−1∑q−1
r=0 (−1)r( qr+1
)(1− (r + 1)a)d−1
.
Now, K -colorability should occur iff Σ > 0.
Asymptotically, this yields a threshold of
dK−col = (2K − 1) ln K − 1 + oK (1).
Amin Coja-Oghlan (Frankfurt) Random k-SAT 39 / 40
Conclusion
The cavity method as a non-rigorous formalism.I free entropy,I phase transitions.
There are two variants:I the replica symmetric version (“Belief Propagation”).I the 1RSB version (“Survey Propagation”).
Can we develop a rigorous foundation?
Amin Coja-Oghlan (Frankfurt) Random k-SAT 40 / 40
Recommended