arX

iv:c

ond-

mat

/960

6215

v1 [

cond

-mat

.dis

-nn]

28

Jun

1996

Statistical Mechanics of the Random K-SAT Model

Remi Monasson∗ and Riccardo Zecchina†

∗ Laboratoire de Physique Theorique de l’ENS, 24 rue Lhomond, 75231 Paris cedex 05, France

† Dipartimento di Fisica, Politecnico di Torino, C.so Duca degli Abruzzi 24, I-10129 Torino, Italy

Abstract

The Random K-Satisfiability Problem, consisting in verifying the exis-

tence of an assignment of N Boolean variables that satisfy a set of M = αN

random logical clauses containingK variables each, is studied using the replica

symmetric framework of disordered systems. The detailed structure of the an-

alytical solution is discussed for the different cases of interest K = 2, K ≥ 3

and K ≫ 1. We present an iterative scheme allowing to obtain exact and

systematically improved solutions for the replica symmetric functional or-

der parameter. The caculation of the number of solutions, which allowed us

[Phys. Rev. Lett. 76, 3881 (1996)] to predict a first order jump at the thresh-

old where the Boolean expressions become unsatisfiable with probability one,

is thoroughly displayed. In the case K = 2, the (rigourously known) critical

value (= 1) of the number of clauses per Boolean variable is recovered while

for K ≥ 3 we show that the system exhibits a replica symmetry breaking

transition. The annealed approximation is proven to be exact for large K.

PACS Numbers : 05.20 - 64.60 - 87.10

Typeset using REVTEX

1

I. INTRODUCTION

The emergent collective behaviour observed in a variety of models of statistical mechanics

and in particular in frustrated disordered systems, have been recognized to play a relevant

role also in apparently distant fields such as theoretical computer science, discrete mathe-

matics and complex systems theory [1–3]. Computationally hard problems, characterized by

exponential running time scaling of their algorithms or memory requirements, the so called

NP–complete problems [4], are known to be in one–to–one correspondence with the ground

state properties of spin–glass like models (see [3] and references therein) and the tools and

concepts of statistical physics have shed some new light onto the notion of complexity of

NP-complete problems and have lead to the definition of new search algorithms as the Sim-

ulated Annealing algorithm based on the introduction of an artificial temperature and some

cooling procedures [5].

Very recently, other techniques inspired from statistical mechanics, namely finite size

scaling analysis, have been applied [6] also to the study of universal behaviour in the com-

putational cost (running time) of some classes of algorithms in the course of searching for

solutions of random realizations of the prototype of NP–complete problems, the satisfiability

(SAT) problem we shall discuss. More generally, phase transition concepts are starting to

play a relevant role in theoretical computer science (see [7] and references therein), where the

analysis of general search methods applied to various classes of hard computational prob-

lems characterized by a large number of relevant variables is of crucial importance. In turn,

computer science is a source of highly non–trivial models containing all the paradigms nec-

essary to a deeper undestanding of the physical properties of disordered frustrated systems,

in particular diluted models for which the theoretical framework is still to be completed [8].

One of the major theoretical open questions in this context would be to understand how

algorithmic complexity and spin-glass transition, the so–called replica symmetry breaking

transition [3], are related.

Among the known NP–complete problems, the SAT problem is at the same time the

2

root problem of complexity theory [4] and a prototype model for phase transition in random

combinatorial structures [1,9]. SAT was the first problem proved to be NP–complete by

S. Cook in 1971 [10] and opened the way to the identification of a vast family of other

NP–complete problems for which a polynomial reduction to SAT became available [4]. In

particular the K–satisfiability (K–SAT) problem, a version of SAT we shall discuss in great

detail in what follows, beside playing a central role in NP–completeness proving procedures

[4], is a widespread test for the evaluation of performance of combinatorial search algorithms,

due the intractability concentration phenomena taking place in random samples generated

near criticality.

In a recent work [2], we have shown that the methods of statistical mechanics of random

systems allow to compute some algorithmically relevant quantities such as the typical entropy

of the problem, i.e. the typical number of its solutions, and to clarify the nature of the

threshold behaviour. The scope of this paper is twofold. On the one hand, we aim at

giving a thorough discussion of the analytical derivation of the above results, mainly the

calculation of the entropy jump at the transition. On the other hand, we expose in details

the replica symmetric theory of the K–SAT problem and show how to go beyond the simplest

solution proposed in our previous work [2]. The paper is organized as follows. Section II is

devoted to the presentation of the K–SAT problem and of the known exact results. Sections

III contains an outline of the statistical mechanics approach whereas the replica symmetric

solutions are exposed in Section IV. In the successive sections, fromV toVIII, the outcomes

of the analytical calculations are exposed in detail for the different values of K of interest.

Conclusions and new perspectives will be briefly discussed in Section IX.

II. THE K–SAT PROBLEM AND A BRIEF SURVEY OF KNOWN RESULTS

Given a set of N Boolean variables {xi = 0, 1}i=1,...,N , we first randomly choose K among

the N possible indices i and then, for each of them, a literal zi that is the corresponding xi

or its negation xi with equal probabilities one half. A clause C is the logical OR of the K

3

previously chosen literals, that is C will be true (or satisfied) if and only if at least one literal

is true. Next, we repeat this process to obtain M independently chosen clauses {Cℓ}ℓ=1,...,M

and ask for all of them to be true at the same time, i.e. we take the logical AND of the

M clauses thus obtaining a Boolean expression in the so called Conjunctive Normal Form

(CNF). The resulting K–CNF formula F may be written as

F =M∧

ℓ=1

Cℓ =M∧

ℓ=1

(

K∨

i=1

z(ℓ)i

)

, (1)

where∧

and∨

stand for the logical AND and OR operations respectively.

A logical assignment of the {xi}’s satisfying all clauses, that is evaluating F to true, is

called a solution of the K–satisfiability problem. If no such assignment exists, F is said to

be unsatisfiable.

When the number of clauses becomes of the same order as the number of variables

(M = α N) and in the large N limit – indeed the case of interest also in the fields of

computer science and artificial intelligence [9,11] – the K–SAT problem exhibits a striking

threshold phenomena. Numerical experiments have shown that the probability of finding a

correct Boolean assignment falls abruptly from one down to zero when α crosses a critical

value αc(K) of the number of clauses per variable. Above αc(K), all clauses cannot be

satisfied any longer and one gets interested in minimizing the number of unsatisfiable clauses,

which is the optimization version of K–SAT also referred to as MAX–K–SAT. Moreover, near

αc(K), heuristic search algorithms get stuck in non–optimal solutions and a slow down effect

is observed (intractability concentration). On the contrary, far from criticality heuristic

processes are typically rather efficient [6].

Very schematically, the known results on K–SAT which have been obtained in the frame-

work of complexity theory may be summarized as follows.

– For K = 2, 2–SAT belongs to the class P of polynomial problems [12]. P is defined as

the set of computational problems whose best solving algorithms have running times

increasing polynomially with the number of relevant variables [4]. For α > αc, MAX–

2–SAT is in NP–complete [4] : NP–complete problems are the hardest nondeterministic

4

polynomial problems, whose solutions may be found by the exhaustive inspection of a

decision tree of logical depth growing in a polynomial way with the number of relevant

variables; it is generally thought that the running times of their best solving algorithms

scale exponentially with the number of relevant variables [4]. The mapping of 2–SAT

on directed graph theory [13] allows to derive rigorously the threshold value αc = 1

and an explicit 2–SAT polynomial algorithm working for α < αc has been developed

[12].

– For K ≥ 3, both K–SAT and MAX–K–SAT belong to the NP–complete class. Only

upper and lower bounds on αc(K) are known from a rigorous point of view. Remark-

able application of finite size scaling techniques has, recently, allowed to find precise

numerical values of αc for K = 3, 4, 5, 6 [1].

– For K ≫ 1, clauses become decoupled and the asymptotic expression αc ≃ 2K ln 2 can

be easily found.

For brevity, we do not discuss here the results concerning the algorithmic approaches to

K–SAT and MAX–K–SAT [12,14,15]. We just mention that MAX–K–SAT belongs to the

subclass of NP–complete problems which allows for a Polynomial Approximation Scheme

for quasi–optimal solutions [14]. A recent numerical study of the critical behaviour in the

computational cost of satisfiability testing can be found in [6].

For α = MN

> 0, K–SAT can be cast in the framework of statistical mechanics of random

diluted systems by the identification of an energy–cost function E(K,α) equal to the number

of violated clauses [9,2]. The study of its ground state allows then to address the optimization

version of the K–SAT problem as well as to characterize the space of solutions by its typical

entropy, i.e. the degeneracy of the ground state. The vanishing condition on the ground

state energy for a given K, corresponds to the existence of a solution to the K–SAT problem

and thus identifies a critical value αc(K) of α below which random formulas are satisfiable

with probability one. For α > αc(K), the ground state energy becomes non zero and

5

gives information on the maximum number of satisfiable clauses, i.e. on the MAX–K–

SAT problem. Previous works on the statistical mechanics of combinatorial optimization

problems - like Traveling Salesman, Graph Partitioning or Matching problems [3,16–18] -

focused mainly on the comparison between the typical cost of optimal configurations and

the algorithmic results. The issues arising in K–SAT are of different nature, and the key

quantity to be discussed [19] is rather the typical number of existing solutions, i.e. the

ground state typical entropy SK(α).

A crucial rigorous result on which the whole statistical mechanics approach is founded

concernes the self-averageness taking place in MAX–K–SAT. For any K, independently of

the particular but randomly chosen sample of M clauses, the minimal fraction of violated

clauses is narrowly peaked around its mean value when N → ∞ at fixed α [15].

III. STATISTICAL MECHANICS OF THE K–SAT AND MAX–K–SAT COST

FUNCTION

As discussed above, we map the random SAT problem onto a diluted spin energy–cost

function through the introduction of spin variables, Si = 1 if the Boolean variable xi is true,

Si = −1 if xi is false. The clauses structure is taken into account by a M × N quenched

random matrix ∆ where ∆ℓ,i = −1 (respectively +1) if clause Cl contains xi (resp. xi), 0

otherwise. Then the function

E[∆, S] =M∑

ℓ=1

δ

[

N∑

i=1

∆ℓ,i Si;−K

]

(2)

where δ[i; j] denotes the Kronecker symbol, turns out to be equal to the number of violated

clauses in that the quantity∑N

i=1 ∆ℓ,i Si equals −K if and only if all Boolean variables in

the ℓ–th clause take the values opposite to the desired ones, i.e. iff the clause itself is false.

Moreover, we impose the constraints

N∑

i=1

∆2ℓ,i = K, ∀ℓ = 1, ..., M (3)

6

to ensure that the number of Boolean variables in any clause is exactly equal toK. Then, the

ground state (GS) properties of the cost function (2) will reflect those of K–SAT (EGS = 0)

and MAX–K–SAT (EGS > 0). In (2), one may interpret K as the number of “neighbours”

to which each spin is coupled inside a clause. To study the ground state properties fo the

cost function (2), we follow the replica approach in the framework of diluted models which is

indeed much more complicated than that of long–range fully connected disordered models.

As we shall see below, replica theory must be formulated in a functional form involving not

only interactions between pairs of replicas but all multi–replicas overlaps.

To compute the GS energy, we first introduce a fictitious temperature 1/β to regularize

all mathematical expressions and send β → ∞ at the end of the calculation. Note that the

introduction of a finite temperature also greatly helps to understand the physical proper-

ties of the model. We procede by computing the model “free–energy” density at inverse

temperature β, averaged over the clauses distribution

F (β) = − 1

βNlnZ[∆] , (4)

where Z[∆] is the partition function

Z[∆] =∑

{Si}

exp (−βE[∆, S]) . (5)

As already mentioned, the energy (2) is self-averaging and should therefore be obtained from

the above free–energy. The overline denotes the average over the random clauses matrices

satisfying the constraint (3) and is performed using the replica trick lnZ = limn→0Zn−1

n,

starting from integer values of n. The typical properties of the ground state, i.e. the internal

energy and the entropy, will then be recovered in the β → ∞ limit.

The averaged integer moments of the partition function are given by the following formula

Z[∆]n =∫ n/2∏

r=1

∏

a1<...<a2r

dQa1,...,a2rdQa1,...,a2reNF [Q,Q] , (6)

where

7

F [Q, Q] =∑

r,{ai}

Qa1,...,a2rQa1,...,a2r + ln

∑

{Sa=±1}

exp

−∑

r,{ai}

Qa1,...,a2rSa1 . . . Sa2r

+ α ln∑

{σa}

∫ 2π

0

n∏

a=1

dσa

2πcos(

∑

a

σaσa)∏

a

(

1 + (e−β − 1)δ[σa;−K])

×

∏

a

cos(σa)K

1 +∑

r,{ai}

(−1)rQa1,...,a2r2r∏

i=1

tan(σai)

K

, (7)

in which∑

r,{ai} ≡∑n/2

r=1

∑

a1<...<a2r and where the sums over all the σa’s run over the integers

between −K and K. The order parameters Qa1,...,a2r are the typical overlaps between spins

configurations belonging to different replicas

Qa1,...,a2r =1

N

N∑

i=1

Sa1i . . . Sa2r

i (8)

and the Qa1,...,a2r are their conjugated Lagrange multipliers.

In the large N,M limit (with fixed α = M/N), the partition function (6) may be

evaluated by taking the saddle-point over all order parameters Q, Q. Since the function

F [Q, Q] is invariant under permutation of replicas, a possible natural saddle-point can be

sought within the so called replica symmetric (RS) Ansatz [17,18]

Qa1,...,a2r = Qr , Qa1,...,a2r = Qr , (9)

which preserves invariance.

Very schematically, the computation of all the three terms appearing on the right hand

side of equation (7) procedes as follows.

i) First term :

n/2∑

r=1

∑

a1<...<a2r

Qa1,...,a2rQa1,...,a2r =n/2∑

r=1

(

n

2r

)

Qr Qr , (10)

ii) Second term : we compute the trace over the set {Sa} by means of the following

relation

(

n∑

a=1

Sa

)2r

=r∑

j=0

C(j, r, n)∑

a1<...<a2j

Sa1 . . . Sa2j , (11)

8

where C(j, r, n) is given by

C(j, r, n) =∂2r

∂x2rcoshn(x) tanh2j(x)|x=0 . (12)

We then find

∑

{Sa=±1}

exp

−∑

r,{ai}

Qa1,...,a2rSa1 . . . Sa2r

= exp(−D0)× (13)

∫ ∞

−∞

dxdx

2πcos(xx) (2 cosh(x))n exp

−n/2∑

i=1

(−1)iDix2i

(14)

where D0 = −∑nj=1C(0, j, n)Dj and all order parameters Dj (j = 1, . . . , n/2) are

solutions of the linear equations

Qr =n/2∑

j=r

C(r, j, n)Dj , ∀ 1 ≤ r ≤ n/2 (15)

iii) Third term : we rewrite the argument of the logarithm as

1

(2π)K

∫ 2π

0dx1 . . . dxK

n/2∑

r1=0

. . .n/2∑

rK=0

Qr1 . . . QrK exp

(

4iK∑

l=1

xlrl

)

×(

1 + (e−β − 1)K∏

l=1

e−ixl cos(xl)

)n

(16)

where Q0 ≡ 1. In the limit n → 0, we expand the logarithm stemming from the last

factor (elevated to the power n) in the above expression.

Putting all three terms together, the RS value of the free–energy reads

1

nF [{Qr, Qr}] =

∞∑

j=1

ajQj −∞∑

i,j=1

bijQiDj − α∞∑

l=1

(1− e−β)l

l 2Kl

1 +l/2∑

i=1

Qi

(

l

2i

)

K

+

∫ ∞

−∞

dxdx

2πcos(xx) ln (2 cosh(x)) exp

(

−∞∑

i=1

(−1)iDix2i

)

, (17)

where

aj = (−1)j∂2j

∂x2jln(cosh(x))|x=0 , bij =

1

(2i)

∂2j

∂x2jtanh2i(x)|x=0 . (18)

9

Within the replica symmetric Ansatz, equations (8) and (9) teach us that Qi is the disorder–

averaged 2ith-moment of the local magnetization of spin Si : Qi = 〈S〉2 i . It results conve-

nient to introduce the probability distribution P (m) of the Boolean magnetization, m = 〈S〉,

such that

Qi =∫ 1

−1dm P (m) m2 i , (19)

together with a similar function P (y) relative to the conjugate parameters

P (y) =∞∑

i=1

Di∂2iδ

∂y2i(y) . (20)

The free energy then reads

− βF (β) = ln 2 +∫ ∞

−∞

dudv

2πcos(uv) ln (cosh v) exp

(

−∫ ∞

−∞dyP (y) cos(uy)

)

+

∫ 1

−1dx∫ ∞

−∞dyP (x)P (y) ln (1 + x tanh y) +

∫ ∞

−∞dyP (y) ln (cosh y) +

α∫ 1

−1

K∏

ℓ=1

dxℓP (xℓ) ln

[

1 + (e−β − 1)K∏

ℓ=1

(

1 + xℓ

2

)

]

(21)

where by definition P (x) and P (y) are even functions (more precisely distributions) satisfying

the saddle point functional equations

P (x) =1

1− x2

∫ ∞

−∞

du

2πcos

[

u

2ln(

1 + x

1− x

)]

exp[

−∫

dyP (y) cos(uy)]

, (22)

and

P (y) = αKδ(y)− αK∫ 1

−1

K−1∏

ℓ=1

dxℓP (xℓ)1

2

[

δ(

y +1

2lnA(K−1)

)

+ δ(

y − 1

2lnA(K−1)

)]

,

(23)

with

A(K−1) ≡ A(K−1)({xℓ} , β) = 1 + (e−β − 1)K−1∏

ℓ=1

(

1 + xℓ

2

)

. (24)

Eliminating P (y) leads to a self–consistent equation for the magnetization distribution

P (x) =1

1− x2

∫ ∞

−∞du cos

[

u

2ln(

1 + x

1− x

)]

×

exp

[

−αK + αK∫ 1

−1

K−1∏

ℓ=1

dxℓP (xℓ) cos(

u

2lnA(K−1)

)

]

(25)

10

and to the following expression of the free–energy

− βF (β) = ln 2 + α(1−K)∫ 1

−1

K∏

ℓ=1

dxℓP (xℓ) lnA(K) +

αK

2

∫ 1

−1

K−1∏

ℓ=1

dxℓP (xℓ) lnA(K−1) −1

2

∫ 1

−1dxP (x) ln(1− x2) . (26)

Note that in eq.(26) A(K) si given by a formula similar to (24), where the upper bound of

the product is replaced by K. To end with, let us remark that equation (25) can in turn be

transformed into an integro–differential equation

∂P (x)

∂α= −KP (x) +K

∫ 1

−1

K−1∏

ℓ=1

dxℓ

[

P (x1) + α(K − 1)∂P (x1)

∂α

]

P (x2) . . . P (xK−1)×

1

2

[

∂η(x)

∂xP (η(x)) +

∂η(−x)

∂xP (η(−x))

]

(27)

where η(x) = [(x + 1)A(K−1) + x − 1]/[(1 + x)A(K−1) + 1 − x] and for which the boundary

condition is given by the solution of (25) in α = 0:

P (x)|α=0 = δ(x) . (28)

IV. A TOY MODEL : THE K = 1 CASE

The K = 1 case can be solved either by a direct combinatorial method or within our

statistical mechanics approach. Though this particular case does not present any critical

behaviour, its study will turn out to be useful in understanding the K > 1 models in which

we are interested. Moreover, the K = 1 toy model allows to check the correctness of the

statistical mechanics results.

In this case, a sample of M clauses is completely described by giving directly the num-

bers ti and fi of clauses imposing that a certain Boolean variable Si must be true or false

respectively. Therefore the partition function corresponding to a given sample reads

Z[{t, f}] =N∏

i=1

(e−βti + e−βfi) , (29)

and the average over the disorder gives

11

1

NlnZ[{t, f}] = 1

N

∑

{ti,fi}

M !∏N

i=1(ti!fi!)lnZ[{t, f}]

= ln 2− αβ

2+

∞∑

l=−∞

e−αIl(α) ln

(

cosh

(

βl

2

))

, (30)

where Il denotes the lth modified Bessel function. The zero temperature limit gives the

ground state energy

EGS(α) =α

2[1− e−αI0(α)− e−αI1(α)] (31)

and the ground state entropy

SGS(α) = e−αI0(α) ln 2 . (32)

One may notice that for any α > 0, the ground–state energy is positive. Therefore, the

clauses are never satisfiable all together and the overall function (1) is false with probability

one. Nonetheless, the entropy is finite, implying an exponential degeneracy of the ground–

state describing the minimum number N.EGS(α) of unsatisfiable clauses. Such a degeneracy

is due to the presence of a finite fraction of variables e−αI0(α) which are subject to equal

opposite constraints ti = fi, and whose corresponding spins may be chosen up or down

indifferently without changing the energy.

The above results are recovered in our approach leading to the conclusion that the RS

Ansatz is exact for all β and α when K = 1. Equation (25) can be explicitly solved at any

temperature 1/β and the solution reads

P (x) =∞∑

ℓ=−∞

e−αIℓ(α) δ

(

x− tanh

(

βℓ

2

))

, (33)

which, in the limit of physical interest β → ∞, becomes

P (x) = e−αI0(α)δ(x) +1

2(1− e−αI0(α)) (δ(x− 1) + δ(x+ 1)) . (34)

The finite value of the ground state entropy may be ascribed to the existence of unfrozen

spins whose fractional number is simply the weight of the δ–function in x = 0. At the same

time, it appears that the non zero value of the ground state energy is due to the presence

12

of completely frozen spins of magnetizations x = ±1. This is an important feature of the

problem which remains valid for any K, as we shall see in the following. In Fig. 1 we report

the plots of the above energy and entropy at zero temperature.

V. REPLICA SYMMETRIC SOLUTIONS FOR ALL K

A relevant general mechanism for the comprehension of the overcoming critical behaviour

in K–SAT is the accumulation of Boolean magnetizations 〈S〉 = ±(1−O(e−|z|β)), (z = O(1)),

in the limit of zero temperature and for α → αc. The emergence of Dirac peaks in x = ±1

signals that a freezing process has just occurred and that a further increase of α beyond αc

would cause the appearance of unsatisfiable clauses. This scenario – which can be verified

by inspection of eq. (33) for K = 1 – is true also for K > 1. In fact, by computing the

fraction of violated clauses through

E = − 1

N

∂

∂βlnZ[∆] , (35)

at temperature 1/β, one sees that the ground state energy depends only upon the magne-

tizations of order ±(1 − O(e−|z|β)), if any, and that such contributions can be described by

the introduction of the new rescaled function

R(z) = limβ→∞

[

P

(

tanh

(

βz

2

))

∂

∂ztanh

(

βz

2

)]

, (36)

which, from (25), fulfills the saddle–point equation

R(z) =∫ ∞

−∞

du

2πcos(uz) exp

[

− αK

2K−1+ αK

∫ ∞

0

K−1∏

ℓ=1

dzℓR(zℓ) cos(u min(1, z1, . . . , zK−1))

]

.

(37)

The corresponding ground state energy reads, see (26) and (36),

EGS(α) = α(1−K)∫ ∞

0

K∏

ℓ=1

dzℓR(zℓ)min(1, z1, . . . , zK) +

αK

2

∫ ∞

0

K−1∏

ℓ=1

dzℓR(zℓ)min(1, z1, . . . , zK−1)−∫ ∞

0dzR(z)z . (38)

13

It is easy to see that the saddle–point equation (37) is in fact a self–consistent identity for

R(z) in the range z ∈ [0, 1] only. Outside this interval, equation (37) is merely a definition

of the functional order parameter R. This remark will be useful in the following.

To start with, R(z) = δ(z) is obviously a solution of (37) for all values of α and K, giving

a zero ground state energy since no spin are frozen with magnetizations ±1. Let us assume

that R(z) includes another Dirac peak in 0 < z0 ≤ 1. Then, inserting this distribution in the

exponential term on the r.h.s. of (37), we find that R(z) on the l.h.s. necessarily includes all

Dirac peaks centered in kz0, where k = 0,±1,±2,±3, . . .. We now plunge again the whole

series in the r.h.s. of (37). For large enough k, kz0 is larger than one and the exponentiated

term includes a cosu contribution, which causes the presence of Dirac distributions centered

in all (positive and negative) integers. Therefore, as soon as R(z) is different from δ(z),

it contains an infinite set of Dirac functions peaked around all integer numbers. Clearly,

the simplest self–consistent solution to (37) will be obtained for z0 = 1 since the process

described above closes after one iteration. This solution reads [2]

R(z) =∞∑

ℓ=−∞

e−γ1Iℓ(γ1)δ(z − ℓ) , (39)

where γ1 depends on K and α and fulfills the implicit equation

γ1 = αK

[

1− e−γ1I0(γ1)

2

]K−1

. (40)

The physical meaning of γ1 may be understood by looking at the definition of the rescaled

function order parameter (36). Turning back to the magnetization distribution, we indeed

find in the zero temperature limit

P (x) = e−γ1I0(γ1)Pr(x) +1

2(1− e−γ1I0(γ1)) (δ(x− 1) + δ(x+ 1)) , (41)

where Pr(x) is a regular (i.e. without Dirac peaks in x = ±1) magnetization distribution

normalized to unity. The above identity is a straigthforward extension of the expression (34)

(when K = 1, γ1 = α from (40) and Pr(x) = δ(x)) to any value of K. Inserting eq.(39) in

(38) gives t the value of the cost–energy

14

EGS(α) =γ12K

(

1− e−γ1I0(γ1)−Ke−γ1I1(γ1))

. (42)

It is therefore clear that, in the RS context, the SAT to UNSAT transition corresponds to

the emergence of peaks centered in x = ±1 with finite weights, that is to a transition from

γ1 = 0 to γ1 > 0. This simplest solution was presented in ref. [2].

In addition to (39), there exist other RS solutions to the saddle–point equations. For

instance, if we choose z0 =12, the insertion process ends up after two iterations and generates

Dirac peaks centered in all integer and half–integer numbers. More generally, for any integer

p ≥ 1, we may define the solution to (37)

R(z) =∞∑

l=−∞

rℓ δ

(

z − ℓ

p

)

, (43)

having exactly p peaks in the interval [0, 1[, whose centers are zℓ =ℓp, ℓ = 0, . . . , p− 1. The

coefficients rℓ of these distributions are self–consistently found through

rℓ =∫ 2π

0

dθ

2πcos(ℓθ) exp

p∑

j=1

γj(cos(jθ)− 1)

(44)

for all ℓ = 0, . . . , p− 1 where

γj = αK

1

2− r0

2−

j−1∑

ℓ=1

rℓ

K−1

−

1

2− r0

2−

j∑

ℓ=1

rℓ

K−1

, ∀j = 1, . . . , p− 1

γp = αK

1

2− r0

2−

p−1∑

ℓ=1

rℓ

K−1

. (45)

The corresponding energy reads, from (43) and (38),

EGS =α(1−K)

p

(

1− r02

)K

+p−1∑

j=1

1− r02

−j∑

l=1

rl

K

+

αK

2p

(

1− r02

)K−1

+p−1∑

j=1

1− r02

−j∑

l=1

rl

K−1

−p∑

j=1

j

pγj

r02+

rj2+

j−1∑

l=1

rl

. (46)

Note that the last term of (46) includes the coefficient rp, which may be computed using

identity (44). It is easy to check that the first non trivial solution (39) corresponds to p = 1.

Though there might be continuous solutions to (37), we believe they can be reasonably

15

approximated by the large p solutions we have presented here. In the following sections,

we shall therefore analyze which are the physical implications of the above solutions in the

different cases of interest, K = 2, K ≥ 3 and K >> 1.

VI. THE K = 2 CASE

The case K = 2 is the first relevant instance of K–SAT. Graph theory has allowed [13]

to show that for α = αc(2) = 1 the problem undergoes a satisfiability transition which can

be also viewed as a P/ NP–complete transition, from 2–SAT to MAX–2–SAT.

Let us first consider the simplest p = 1 RS solution [2]. Self–consistency equation (40)

leads to the solution γ1 = 0 for any α. However, for α > 1 one finds another solution

γ1(α) > 0 which maximizes the free–energy (and EGS) and therefore must be chosen (this is

a well known peculiar aspect of the replicas formalism [3]). When approaching the threshold

from upside, we indeed find

EGS(α|p = 1) =4

27(α− 1)3 +O

(

(α− 1)4)

≃ 0.1481 (α− 1)3 . (47)

As expected, the p = 1 RS theory predicts EGS = 0 for α ≤ 1 and EGS > 0 when α > 1,

giving back the rigorous result αc(2) = 1 : for α > 1 the fraction of violated clauses becomes

finite and the corresponding CNF formulas turn out to be false with probability one. The

transition taking place at αc is of second order with respect to the order parameter γ1 and

is accompanied by the progressive appearance of two Dirac peaks for P (x) in x = ±1 with

equal amplitudes (1− e−γ1I0(γ1))/2.

It is straightforward to verify that RS solutions with p ≥ 2 are not present below α = 1.

However, above the threshold, one has to check whether their ground state energy are larger

than the one of the p = 1 solution, that is if they can be relevant for MAX–K–SAT. For

p = 2, resolution of equations (44) and (45) close to αc(2) leads to (discarding the choice

r1 = 0 which amounts to the p = 1 solution)

r0 = 1− 8 + 2√2

7(α− 1) +O

(

(α− 1)2)

16

r1 =3−

√2

7(α− 1) +O

(

(α− 1)2)

(48)

for the coefficients of the Dirac peaks in z = 0 and z = 12respectively. Inserting these

expansion into the energy (46), one finds

EGS(α|p = 2) =9 + 4

√2

98(α− 1)3 +O

(

(α− 1)4)

≃ 0.1496 (α− 1)3 , (49)

which is slightly larger than the p = 1 result (47). Numerical calculations for higher values

of p ≥ 3 confirm that the energy increases very slowly with p. We have found that for

large p’s the ground state energy is almost stationary, so that the p = 10 solution can be

considered as a very fair approximation of the optimal p → ∞ RS solution. The coefficients

rℓ of the distributions present in the order parameter R(z) (43) are displayed Fig. 2 for

different values of α and in the cases p = 1, p = 5 and p = 10.

The ground state energy predicted by the p = 10 RS solution is compared to numerical

exhaustive simulations carried out for small sized systems on Fig 3. For α > αc = 1, the

theoretical estimate of EGS seems to sligthly deviates from the numerical findings, which

would signal the occurrence of a Replica Symmetry Breaking (RSB) transition at the thresh-

old. If it were so, the situation would be reminiscent of the case of neural networks with

continuous weights, where RS theory is able to localize the storage capacity but not to

predict the minimal fraction of errors beyond the transition [20,21]. The extrapolation of

the simulations results for finite size systems to N → ∞ is shown Fig. 4 for the particular

choice α = 3. Data seem in favor of RSB but one cannot exclude that 1/N2 effects could

make coincide both numerics and theory. Moreover, for α ≫ 1, the exact asymptotic scaling

of the ground state energy EGS ≃ α/4 [15] is compatible with the RS prediction. In this

work we are primarily concerned with the K–SAT problem and therefore we shall leave the

detailed analysis of the validity of RS theory above αc for further studies [22].

From the above discussion, it is however reasonable to believe that RS theory is exact

in the region 0 ≤ α ≤ αc = 1 at least [23]. As already mentioned, the key quantity to study

in this range is the typical number of solution to the problem, i.e. the typical ground state

17

entropy SGS(α) given by eq.(26) in the β → ∞ limit. Notice that a simpler expression of

the ground state entropy, more precisely of its derivative, may be obtained by differentiating

(26) with respect to α and using the saddle-point equation (27). The result reads

∂SGS

∂α(α) =

∫ 1

−1

K∏

ℓ=1

dxℓ P (xℓ) ln

[

1−K∏

ℓ=1

(

1 + xℓ

2

)

]

, (50)

and is valid for any α and K. Using the initial value SGS|α=0 = ln 2 and the above equation

(50), one can in principle compute the ground state entropy for any value of α. However,

due to the difficulty of finding a solution of the integral equation (25), it turns out to be

convenient to develop a systematic expansion of the entropy in the parameter α. We now

briefly present the procedure to be employed for a generic value of K.

Inserting P (x)|α=0 = δ(x) into formula (50), we obtain the slope of the entropy at the

origin

∂SGS

∂α

∣

∣

∣

∣

∣

α=0

= ln(

1− 1

2K

)

, (51)

which coincides with the annealed result [1,19]. Then, we use eq.(27) to compute the first

derivative of the magnetizations distribution in α = 0,

∂P (x)

∂α

∣

∣

∣

∣

∣

α=0

= −αKδ(x) +αK

2δ(

x+1

2K − 1

)

+αK

2δ(

x− 1

2K − 1

)

. (52)

Now, we differentiate eq.(50) with respect to α and inject the above result, which is needed

to obtain the second derivative of the ground state entropy at α = 0,

∂2SGS

∂α2

∣

∣

∣

∣

∣

α=0

= −K2 ln(

1− 1

2K

)

+K2

2ln(

1− 1

2K − 1

)

+K2

2ln

(

1− 2K−1 − 1

2K−1(2K − 1)

)

,

(53)

which is negative as required since the entropy is expected to be a concave function of α.

The whole procedure, consisting in successive differentiations of eqs.(27) and (50) can then

be iterated to compute symbolically all the derivatives of P (x) and SGS(α) with respect to

α in α = 0.

18

In the K = 2 case, we have calculated this way the power expansion of SGS(α) up to the

seventh order in α (which implies an uncertainty less than one percent with respect to the

sixth order Taylor expansion on the range α ∈ [0; 1]). The result reads

SGS(α) = ln 2− 0.28768207 α− 0.01242252 α2 − 0.0048241588 α3 − 0.0023958362 α4 −

0.0013119155 α5 − 0.00081617226 α6 − 0.00053068034 α7 − . . . , (54)

in which, for simplicity, we have reported only few significant digits of the coefficients. The

latter are computed symbollically and have the form of a logarithm of rational number. At

the transition we find SGS(αc) ≃ 0.38 which is indeed very high as compared to SGS(0) = ln 2.

A plot of the entropy versus α is shown Fig. 3. For completness, we stress that the ground

state entropy and the logarithm of the number of solutions, which coincide below αc, have

different meanings (and values) above the treshold. In this region, the latter equals to −∞

since all solutions have disappeared while the former quantity reflects the degeneracy of the

lowest state (with strictly positive energy) and is continuous at the transition as shown by

simulations.

Since, for α > αc, there do not exist anymore sets of Si’s such that the energy (2) remains

non extensive, the vanishing of the exponentially large number of solutions that were present

below the threshold is surprisingly abrupt. We then conclude that the transition itself is due

to the appearance, with probability one, of contradictory logical loops in all the solutions

and not to a progressive disappearance of the number of these solutions down to zero.

This perfectly agrees with the graph–theoretical derivation of the critical α which is indeed

based on a probabilistic calculation of appearance of contradictory cycles in oriented random

graphs representing Boolean formulas.

VII. THE K ≥ 3 CASE

The K = 3 case is the first NP–complete instance of K–SAT. The resolution of the RS

equations leads to a scenario different from the previous K = 2 case. We shall see below

19

that RS theory does not allow to derive the value of the threshold αc(3) ≃ 4.17 ± 0.05,

which was estimated by means of finite–size scaling techniques [1]. This is due to the fact

that the calculation of αc(3) requires the introduction of Replica Symmetry Breaking (RSB),

leading to very complicated equations we have not yet succeeded in solving. However, it is

a remarkable fact that, in the relevant region for 3–SAT, i.e. for α ranging from zero up to

αc(3), the ground state entropy computed using RS theory seems to be exact.

Let us start with the p = 1 RS solution (39). Solving eq. (40) leads to the following

scenario (see Fig. 5). For α < αm(3) ≃ 4.667, there exists the solution γ1 = 0 only. At

αm(3), a non zero solution γ1(α) 6= 0 discontinuously appears. The corresponding ground

state energy is negative in the range αm(3) ≤ α < αs(3) = 5.181, meaning that the new

solution is metastable and that EGS = 0 up to αs(3). For α > αs(3) the γ1(α) 6= 0 solution

becomes thermodynamically stable. ¿From the above scheme one is tempted to conclude

that αs(3) corresponds to the desired threshold αc(3). However, this prediction is wrong

since the experimental value αc(3) = 4.17 ± 0.05 is lower than both αm(3) and αs(3). The

failure of the above p = 1 RS prediction is also confirmed by the large K limit. One finds

αm(K) ∼ K2K/16/π and αs(K) ∼ K2K/4/π which are larger than the exact asymptotic

value αc(K) ∼ 2K ln 2. It is worth noticing that though the scaling of αc(K) for large K

is wrong within the p = 1 RS Ansatz, the asymptotic value for large α (and any K) of the

ground state energy for MAX–K–SAT is correctly predicted : EGS(α) ∼ α/2K [15].

We now turn to improved RS solutions by looking at larger values of p. When p = 2, the

previous transition scenario remains qualitatively unaltered, but the precise values of the

spinodal and the threshold points are quantitatively modified. One finds, see Fig. 5. that

αm(3|p = 2) ≃ 4.45 while αs(3|p = 2) ≃ 4.82. The ground state energy curve is similar to

the p = 1 curve but is shifted to the right. Though still incorrect, the p = 2 prediction is

thus closer to the real threshold value. For larger integers p, we have found that αm(3|p) and

αs(3|p) still decrease but quickly converge to the values 4.428 and 4.605 respectively (the

maximum value of p we have tried is p = 20, but in practice convergence was already reached

for p = 10). In Fig. 6, we have plotted the values of the coefficients rℓ (ℓ = 0, . . . , p − 1)

20

entering (43) for p = 1, p = 5 and p = 10. The departure of the coefficient curves for

p = 5 from the p = 10 curves displaying r0, r2, r4, r6 and r8 is clearly visible as soon as the

remaining coefficients of the p = 10 solution, namely r1, r3, r5, r7 and r9 which are implicitly

set to zero in the p = 5 solution, acquire a non negligible value.

Therefore, we may conclude from the above analysis that RS theory is unable to correctly

predict the value of the transition threshold. When crossing the latter, a first order Replica

Symmetry Breaking transition presumably takes place. Note that the relevance of RSB for

the random SAT problem has already been suggested from a simple evaluation of the first

few moments of the partition function [24]. The calculation of the threshold value would

require the introduction of a Replica Symmetry Broken Ansatz to replace (9). However,

the issue of RSB in diluted models is largely an open one [8], due to the complex structure

of the saddle–point equations involved, and we shall not attempt here at pursuing in this

direction.

In the following, we shall rather show that RS theory still provide a consistent and very

precise analysis of the behaviour of the random K–SAT problem below its threshold. This

requires the inspection of the ground state entropy in the region where R(z) = δ(z). Using

the method exposed in the previous Section, we have computed SGS to the 8th order in α

and found that

SGS(α) = ln 2− 0.13353139 α− 0.00093730474 α2 − 0.00011458425 α3 −

0.000016252451 α4 − 2.4481877 10−6 α5 − 3.9910735 10−7 α6 −

6.5447303 10−8 α7 − 1.167915 10−8 α8 − . . . , (55)

in which, again, we have reported only few sufficient digits of the (exactly known) coefficients.

The entropy curve is displayed Fig. 7 in the range 0 ≤ α ≤ αc(3). By computing the zero

entropy points (αze) given by the ℓ − th order entropy expansion, one finds a convergent

succession of values toward αze(3) = 4.75 (within one percent of precision), definitely outside

the range of validity 0 ≤ α ≤ αs(3|p → ∞) ≃ 4.605 of the expansion (55). Notice that

αze|ℓ=1(3) = 5.1909 corresponds to the annealed theory. A similar calculation for the cases

21

K = 4, 5, 6 yields qualitatively similar results which show an even quicker convergence

towards a zero entropy point such that αc(K) < αze(K) (see next Section for the analysis

of the large K limit where both values coincide).

Therefore, SGS is always positive below αs(3|p → ∞). In contradistinction with the

p = 1 RS solution [2], the large p RS solution cannot be ruled out by a simple inspection

of their corresponding entropy. A more important consequence of the previous calculation

of the entropy is that, at the threshold αc, the RS entropy is still extensive. The crucial

point is now to understand whether such value of the entropy is exact up to αc or whether

Replica Symmetry Breaking (RSB) effects have come into play. This issue may be clarified

by resorting to exhaustive numerical simulation. As reported in [2], simulations in the

range N = 12, ..., 28 lead to the conclusion that not only the entropy is indeed finite at the

transition but also that our analytical solution appears exact up to αc. In particular the

1/N extrapolation of the entropy value at α = 4.17 shows a remarkable agreement between

the numerical trend and the RS prediction SGS(αc) ≃ 0.1 (see the inset of the figure in

[2]). RSB corrections to the RS theory seem thus to be absent below αc, which leads us to

conjecture that the RSB transition could occur at αc exactly. In this sense the situation

would be partially similar to the binary network case [25] : the RS entropy would be exact

up to αc (but does not vanish) that would also coincide with the symmetry breaking point.

VIII. THE ASYMPTOTIC CASE OF LARGE K

In the large K limit, the saddle point equations lead to a closed form for the probability

distribution P (x). In fact, in terms of the quantity

Q(A) =∫ 1

−1

K−1∏

ℓ=1

dxℓP (xℓ)δ(

A− A(K−1)

)

, (56)

the differential equation (27) reads

∂P (x)

∂α= −KP (x) +K

∫ ∞

−∞dA

(

Q(A) + α∂Q(A)

∂α

)

×

1

2

[

∂η(x)

∂xP (η(x)) +

∂η(−x)

∂xP (η(−x))

]

, (57)

22

where η(x) has been defined in Section III. For K ≫ 1 , we may expand Q(A) as

Q(A) ≃ δ(A− 1) +1

2K−1δ′(A− 1) +

1

2

(

1

4+

1

4

∫ 1

−1dxP (x)x2

)K−1

δ′′(A− 1) + . . . (58)

Under the changes of variables G(y, α) = (1− tanh2 y)P (tanh y) and

V (α) = αK(

1

4+

1

4

∫ 1

−1dxP (x)x2

)K−1

, (59)

equations (57) and (58) simplify into the celebrated heat equation

∂G(y, V )

∂V= 2

∂2G(y, V )

∂y2. (60)

whose normalized solution is G(y, V ) = exp(−y2/2V )/√2πV . Turning back to P (x), we

find

P (x) ≃ 1√

2πV (α)(1− x2)exp

(

− 1

8V (α)ln2

(

1 + x

1− x

)

)

, (K ≫ 1) , (61)

where V (α) is given by the self–consistency equation (59). The latter may be easily estimated

for large K : V (α) ≃ αK/4K−1. Therefore, when α < αc(K) ≃ 2K ln 2, V (α) is vanishingly

small, that is P (x) → δ(x), proving that the replicas become uncoupled in the large K

limit [1]. In addition, it can be checked that the zero entropy point αze(K) reaches the

threshold αc(K) from above. Another way of looking at the entropy is provided by equation

(53) : it is a simple check the fact that αc(K)2 ∂2SGS

∂α2 |α=0 → 0 for large K. We may then

conclude that the annealed approximation becomes exact when K ≫ 1. As said above, K

may be understood as the connectivity of our model and, in the asymptotic regime K ≫ 1,

RS theory includes only Gaussian interactions as in long–range spin–glasses models [25]. In

Fig. 8 we report some instances of the probability distribution, calculated for different values

of K and α. Notice that since the critical point coincides, in this large K limit, with the

zero entropy point (which is far below the point where the RS energy becomes positive -

see previous Section), the probability distribution of the Boolean magnetization is far from

being concentrated in ±1.

23

IX. CONCLUSION AND PERSPECTIVES

In this paper, we have presented the Replica Symmetric theory of the random K–SAT

problem. We have shown that, in this context, the natural quantity emerging from the

analytical study is the distribution of the average values of the Boolean variables, indicating

to what extent the latters are determined by the constraints imposed by the clauses. The

knowledge of this probability distribution requires the resolution of a functional saddle–point

equation, for which we have presented an iterative sequence of improved solutions. The most

surprising result we have derived is the fact that the entropy is finite just below the transition,

i.e. that the latter is characterized by an abrupt disappearance of all exponentially numerous

solutions due to the emergence of contradictory loops.

Some numerical simulations we have performed for K = 2 as well as in the K = 3 case

are in remarkable quantitative agreement with our RS calculations of the entropy jump at

the threshold [2]. It seems therefore reasonable to infer that RS is correct up to the critical

ratio of clauses per Boolean variable. Would it be so the physical picture of the space of

solutions would not necessarily be simple. Replica Symmetry can indeed hide a non trivial

structure of the solutions, as has been shown for long range spin-glasses [26] models and

in the (closer to K–SAT) case of neural networks [27]. This issue is probably of crucial

importance to understand the performances of local search algorithms.

As for the values of the critical thresholds themselves, RS gives the correct prediction

αc = 1 for K = 2 but fails in estimating the critical αc for K ≥ 3. The study of the

NP–complete instances K ≥ 3 of the K–SAT problem requires to break replica symmetry.

As a consequence, their direct study will surely reveal far from being easy and will require

non trivial analytical efforts.

Another route which one can follow to reach a better understanding of the K > 2 case

consists in starting from the relatively well understood 2–SAT case and modifying it to

get closer to the 3–SAT problem. Such a pertubative approach can be implemented by

considering a mixed model, which one may refer to as (2 + ǫ)–SAT model, composed of

24

(1− ǫ)M clauses of length three and ǫM clauses of length two. As a consequence, we have

at our disposal a model interpolating smoothly between 2–SAT (ǫ = 0) and 3–SAT (ǫ = 1),

which one can attempt at solving for small ǫ at least. Preliminary investigations suggest

that, both from analytical and numerical points of view, the above model could be of interest

for exploring the connection between the order of the RS to RSB phase transition and the

computational cost transition observed numerically near criticality. Work is in progress

along these lines [22].

Acknowledgments : We thank S. Kirkpatrick and B. Selman for useful discussions.

25

REFERENCES

∗ Email: monasson@physique.ens.fr ; preprint LPTENS 96/35

LPTENS is a unite propre du CNRS, associee a l’Ecole Normale Superieure et a

l’Universite de Paris–Sud.

† Email: zecchina@to.infn.it; preprint POLFIS-TH 96/15

also at INFN sez. Torino.

[1] S. Kirkpatrick and B. Selman, Science 264, 1297 (1994)

[2] R. Monasson and R. Zecchina, Phys. Rev. Lett. 76, 3881 (1996)

[3] M. Mezard, G. Parisi, M.A. Virasoro Spin Glass Theory and Beyond, World Scientific,

Singapore, 1987

[4] M. Garey and D.S. Johnson, Computers and Intractability; A guide to the theory of

NP–completness, W.H. Freeman and Co., San Francisco, 1979

[5] S. Kirkpatrick, C.D. Gelatt Jr. and M.P. Vecchi, Science 220, 339 (1983)

[6] B. Selman and S. Kirkpatrick, Artificial Intelligence 81, 273 (1996)

[7] Issue 1–2, Artificial Intelligence 81, T. Hogg, B.A. Huberman and C. Williams Eds.,

(1996)

[8] C. De Dominicis and P. Mottishaw, J. Phys. A: Math. Gen.20, L1267 (1987)

K.Y.M. Wong and D. Sherrington, J. Phys. A: Math. Gen.21, L459 (1988)

[9] S. Kirkpatrick, G. Gyorgyi, N. Tishby and L. Troyansky, in Advances in Neural Infor-

mation Processing Systems 6, J. Cowan, G. Tesauro and J. Alspector Eds., (1994)

[10] S.A. Cook, The complexity of theorem–proving procedures, in Proc. 3rd Ann. ACM

Symp. on Theory of Computing, Assoc. Comput. MAch., New York, 151 (1971)

[11] A. Kamath, R. Motwani, K. Palem and P. Spirakis, Random Structures and Algorithms

26

7, 59 (1995)

[12] B. Aspvall, M.F. Plass and R.E. Tarjan, Inf. Process. Lett.8, 121 (1979)

[13] A. Goerdt, in Proc. 7th Int. Symp. on Mathematical Foundations of Computer Science,

264 (1992)

V. Chvatal and B. Reed, in Proc. 33rd IEEE Symp. on Foundations of Computer Sci-

ence, 620 (1992)

[14] M. Yannakakis, in Proc. 4th Annual ACM–SIAM Symp. on Discrete Algorithms, 1

(1993).

[15] A.Z. Broder, A.M. Frieze and E. Upfal, in Proc. 4th Annual ACM–SIAM Symp. on

Discrete Algorithms, 322 (1993)

[16] G. Baskaran, Y. Fu, P.W. Anderson, J. Stat. Phys. 45, 1 (1986)

[17] M. Mezard, G. Parisi, J. Physique Lett. 46, L771 (1985); J. Physique 47, 1285 (1986)

[18] H. Orland, J. Physique 46, L763 (1985)

[19] O. Dubois, Theoretical Computer Science 81, 49 (1991)

O. Dubois and J. Carlier, Theoretical Computer Science 81, 65 (1991)

[20] J.A. Hertz, A. Krogh, R.G. Palmer, Introduction to the theory of neural computation,

Addison–Wesley, Redwood City (CA), 1991

[21] E. Gardner, J. Phys. A 21, 257 (1988)

E. Gardner and B. Derrida, J. Phys. A 21, 271 (1988)

[22] S. Kirkpatrick, R. Monasson, B. Selmann and R. Zecchina, in preparation

[23] It would be worth studying the local stability of the RS saddle–point. This is however

a difficult task, that can not exclude the occurrence of first order RSB. See :

M. Mezard and G. Parisi, J. Physique 48, 1451 (1987)

27

[24] L. Troyansky, N. Tishby, proceedings of PhysComp96, T. Toffoli, M. Biafore, J. Leao

eds., Boston (1996)

[25] W. Krauth and M. Mezard, J. Physique 50 , 3057 (1989)

[26] G. Parisi and M. Virasoro, J. Phys. (Paris) 50, 3317 (1986); T. R. Kirkpatrick and D.

Thirumalai, Phys. Rev. B 36, 5388 (1987);

[27] R. Monasson and R. Zecchina, Phys. Rev. Lett. 75, 2432 (1995); (Erratum) 76, 2205

(1996)

28

FIGURES

FIG. 1. Ground state cost–energy (bold line), or fraction of violated clauses, and entropy (thin

line) versus α for K = 1.

FIG. 2. Order parameters ri (i = 0, ..., p− 1) corresponding to the different RS solutions p = 1

(dashed line), p = 5 (dashed–dotted lines) and p = 10 (continuous lines), for K = 2 and α ∈ [1, 3].

The curves representing ri for i = 1..., p − 1 overlap and therefore are hardly distinguishable.

FIG. 3. RS ground state entropy (left curve, left scale) and RS ground state energy (right scale)

for p = 1, 10 versus α for K = 2. The dashed lines interpolate the numerical data of exhaustive

simulations on systems of size N = 16, 20, 24 and averaged over 15000, 7500, 2500 samples respec-

tively. Errors bars are within 10% for the entropy and even smaller for the energy and thus not

reported explicitly.

FIG. 4. 1/N extrapolation of the minimal fraction of violated clauses (i.e. ground state

cost–energy) for α = 3 and N = 18, 20, 22, 24, 26 averaged over 20000, 15000, 10000, 7500 and

5000 samples respectively. The extrapolated value appears to be different from the value 0.14472

toward which the RS solutions with increasing p rapidly converge.

FIG. 5. RS ground state energy for K = 3 (continuous lines) computed for p = 1, ..., 10 and

compared with the results of numerical simulations on systems of size N = 16, 20, 24 and averaged

over 15000, 7500, 2500 samples respectively (error bars are of the order of the size of the dots).

The RS ground state energy becomes positive (for p >> 1) at αs ≃ 4.605 whereas the value at

which the unstable solution appears is αm ≃ 4.428. Scope of the dashed line is to help the eye in

following the expected, yet unkown, RSB behaviour.

FIG. 6. Order parameters ri (i = 0, ..., p− 1) corresponding to the different RS solutions p = 1

(dashed line), p = 5 (dashed–dotted lines) and p = 10 (continuous lines), for K = 3.

29

FIG. 7. RS entropy for K = 3 (continuous line) compared with the results of exhaustive

numerical simulations for N = 16, 20, 24 and averaged over 15000, 7500, 2500 samples respectively

(see also ref.[4]). Errors bars are within 10% and not reported explicitly.

FIG. 8. Probability distributions P (x) as functions of the magnetization x, calculated for

α = 2K ln 2 (critical threshold in the K >> 1 limit) and for K = 10, 12, 14, 16, 18.

30

0.25 0.5 0.75 1 1.25 1.5 1.75 2

M/N

0

0.1

0.2

0.3

0.4

0.5

0.6

1.25 1.5 1.75 2 2.25 2.5 2.75 3

M/N

0

0.2

0.4

0.6

0.8

1

p=10p=5

p=1

0.5 1 1.5 2 2.5 3

M/N

0

0.2

0.4

0.6

0.8

1

0.04

0.08

0.12

Ln(2)

p=1,10

0.01 0.02 0.03 0.04 0.05 0.06

1/N

0.13

0.14

0.15

0.16

0.17

K=2, M/N=3, N=18,20,22,24,26

p=1

p>>1 RS-values

4.5 5 5.5 6 6.5 7 7.5 8

M/N

0

0.025

0.05

0.075

0.1

0.125

p=1

p=10

4.6 4.8 5 5.2 5.4 5.6 5.8 6

M/N

0

0.1

0.2

0.3

0.4

0.5

p=10

p=5

p=1

1 2 3 4 5 6

M/N

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Ln(2)

-0.75 -0.5 -0.25 0 0.25 0.5 0.75 1x

0

5

10

15

20

25

K=10

K=12

K=14

K=16

K=18

M/N=Log(2) 2^K