Rounding Semidefinite Programming Hierarchies via GlobalCorrelation

Boaz Barak Prasad Raghavendra David Steurer

Abstract— We show a new way to round vectorsolutions of semidefinite programming (SDP) hierar-chies into integral solutions, based on a connectionbetween these hierarchies and the spectrum of the in-put graph. We demonstrate the utility of our methodby providing a new SDP-hierarchy based algorithmfor constraint satisfaction problems with 2-variableconstraints (2-CSP’s).

More concretely, we show for every 2-CSP in-stance �, a rounding algorithm for r rounds ofthe Lasserre SDP hierarchy for � that obtains anintegral solution which is at most ε worse than therelaxation’s value (normalized to lie in [0, 1]), aslong as

r > k · rank�θ(�)/ poly(ε) ,where k is the alphabet size of �, θ = poly(ε/k), andrank�θ(�) denotes the number of eigenvalues largerthan θ in the normalized adjacency matrix of theconstraint graph of �.

In the case that � is a UniqueGames instance, thethreshold θ is only a polynomial in ε, and is indepen-dent of the alphabet size. Also in this case, we cangive a non-trivial bound on the number of roundsfor every instance. In particular our result yieldsan SDP-hierarchy based algorithm that matches theperformance of the recent subexponential algorithmof Arora, Barak and Steurer (FOCS 2010) in theworst case, but runs faster on a natural family ofinstances, thus further restricting the set of possiblehard instances for Khot’s Unique Games Conjecture.

Our algorithm actually requires less than thenO(r) constraints specified by the rth level of theLasserre hierarchy, and in some cases r rounds ofour program can be evaluated in time 2O(r) poly(n).

1. Introduction

This paper is concerned with hierarchiesof semi-definite programs (SDP’s). Semidef-inite programs are an extremely useful toolin algorithms and in particular approximationalgorithms (e.g., [15], [18]). Approximationalgorithms based on SDP’s typically involvefinding an integral (say 0/1) solution for someoptimization problem, by using convex pro-gramming to find a fractional/high-dimensionalsolution and then rounding it into an integralsolution. Sherali and Adams [34], Lovász and

Microsoft Research New England, Cambridge MA.Georgia Institute of Technology, Atlanta GA.Microsoft Research New England, Cambridge MA.

Schrijver [28], and, later Lasserre [26], pro-posed systematic techniques, known as hierar-chies, to make this convex relaxation tighter,thus ensuring that the fractional solution iscloser to an integral one. These hierarchies areparameterized by a number r, called the levelor number of rounds of the hierarchy. Givena program on n variables, optimizing over therth level of the hierarchy can be done in timenO(r). The gap between integral and fractionalsolutions decreases with r, and reaches zeroat the nth level. The paper [27] surveys andcompares the different hierarchies proposed inthe literature, (see recent survey [10]).

These semidefinite programming hierarchieshave been of some interest in recent years,since they provide natural candidate algorithmsfor many computational problems. In particu-lar, whenever the basic semidefinite or linearprogram provides a suboptimal approximationfactor, it makes sense to ask how many roundsof the hierarchy are required to significantlyimprove upon this factor. Unfortunately, takingadvantage of these hierarchies has often beendifficult, and while some algorithms (e.g., [5])can be encapsulated in, say, level 3 or 4 ofsome hierarchies, there have been relativelyfew results (e.g. [9], [8]) that use higher levelsto obtain new algorithmic results. In fact, therehas been more success in showing that highlevels of hierarchies do not help for many com-putational problems [3], [33], [14], [31], [22].In particular for 3SAT and several other NP-hard problems, it is known that it takes Ω(n)rounds of the strongest SDP hierarchy (i.e.,Lasserre) to improve upon the approximationratio achieved by the basic SDP (or sometimeseven simpler algorithms) [32], [36].

Semidefinite hierarchies are of particular in-terest in the case of problems related to Khot’sUnique Games Conjecture (UGC) [19]. Severalworks have shown that for a wide varietyof problems, the UGC implies that (unlessP = NP) the basic semidefinite program cannotbe improved upon by any polynomial-timealgorithm [20], [29], [30]. Thus in particular

2011 52nd Annual IEEE Symposium on Foundations of Computer Science

0272-5428/11 $26.00 © 2011 IEEE

DOI

470

2011 52nd Annual IEEE Symposium on Foundations of Computer Science

0272-5428/11 $26.00 © 2011 IEEE

DOI 10.1109/FOCS.2011.95

482

2011 52nd Annual IEEE Symposium on Foundations of Computer Science

0272-5428/11 $26.00 © 2011 IEEE

DOI 10.1109/FOCS.2011.95

472

2011 IEEE 52nd Annual Symposium on Foundations of Computer Science

0272-5428/11 $26.00 © 2011 IEEE

DOI 10.1109/FOCS.2011.95

472

the UGC predicts that for all these problems, itwill require a super-constant (and in fact poly-nomial, under widely believed assumptions)number of hierarchy rounds to improve uponthe basic SDP. Investigating this prediction,particularly for the Unique Games problem it-self and other related problems such as MaxCut, Sparsest Cut and Small-Set Expansion,has been the focus of several works, and it isknown that at least (log log n)Ω(1) rounds arerequired for a non-trivial approximation by anatural (though not strongest possible) SDPhierarchy [31], [22]. However, no non-trivialupper bound was known prior to the currentwork, and so it was conceivable that theselower bounds can be improved to Ω(n).

Recently, Arora, Barak and Steurer [2] gavea 2npoly(ε)

-time algorithm for solving the UniqueGames and Small-Set Expansion problems(where ε is the completeness parameter, seebelow). However, their algorithm did not usesemidefinite programming hierarchies, and sodoes not immediately imply an upper bound onthe number of rounds needed.

1.1. Our results

Our main contribution is a new method toanalyze and round SDP hierarchies. The highlevel description is that it uses global correla-tions inside the high-dimensional SDP solution,combined with the hierarchy constraints, toobtain a better rounding of this solution intoan integral one. We believe this method can beof general utility, and in particular we use ithere to give new algorithms for approximatingconstraint satisfaction problem on two-variableconstraints (2-CSP’s), that run faster than thepreviously known algorithms for a natural fam-ily of instances. To state our results we needthe notion of a threshold rank. Threshold rank

of graphs and 2-CSPs. The τ-threshold rankof a regular graph G, denoted rank�τ(G), isthe number of eigenvalues of the normalizedadjacency matrix of G that are larger than τ.1

An instance � of a Max 2-Csp problem consistsof a regular graph G�, known as the constraint

1In this paper we only consider regular undirectedgraphs, although we allow non-negative weights and/orparallel edges. Every such graph can be identified withits normalized adjacency matrix, whose (i, j)th entry isproportional to the weight of the edge (i, j), with all rowand column sums equalling one. Similarly, we restrict ourattentions to 2-CSP’s whose constraint graphs are regular.However, our definitions and results can be appropriatelygeneralized for non-regular graphs and 2-CSPs as well.

graph of � over a vertex set [n], where everyedge (i, j) in the graph is labeled with a relationΠi, j ⊆ [k]× [k] (k is known as the alphabet sizeof �). The value of an assignment x ∈ [k]n tothe variables of �, denoted val�(x), is equalto the probability that (xi, x j) ∈ Πi, j where(i, j) is a random edge in G�. The objectivevalue of � is the maximum val�(x) over allassignments x. We say that � is c-satisfiableif �’s objective value is at least c. We definerank�τ(�) = rank�τ(G�). Our main result is,

Theorem 1.1. There is a constant c such thatfor every ε > 0, and every Max 2-Csp instance� with objective value v, alphabet size k thefollowing holds: the objective value sdpopt(�)of the r-round Lasserre hierarchy for r � k ·rank�τ(�)/εc is within ε of the objective valuev of �, i.e., sdpopt(�) � v+ ε. Moreover, thereexists a polynomial time rounding scheme thatfinds an assignment x satisfying val�(x) > v−εgiven an optimal SDP solution as input.

Results for Unique Games constraints.We obtain quantitatively stronger results for

the important special case of Max 2-Csp, theUnique Games problem. A Max 2-Csp instanceis a Unique Games instance if all the relationsΠi, j have the form (a, b) ∈ Πi, j iff a = πi, j(b)where πi, j is a permutation of [k]. First, weshow that for Unique Games instances thethreshold τ in Theorem 1.1 does not need todepend on the alphabet size. Namely, we provethe following.

Theorem 1.2. There is an algorithm, basedon rounding r rounds of the Lasserre hierarchyand a constant c, such that for every ε > 0 andinput Unique Games instance � with objectivevalue v, alphabet size k, satisfying rank�τ(�) �εcr/k, where τ = εc, the algorithm outputs anassignment x satisfying val�(x) > v − ε.

The Unique Games Conjecture concerns theapproximability of the Unique Games problemin a specific regime, namely, given a UniqueGames instance with optimal value 1 − ε, thegoal is to find an assignment with value at least1/2. We also show that a sublinear number ofrounds suffice to get such an approximationin the worst case, regardless of the thresholdrank of the instance. Moreover, we also showthat such an approximation can be obtainedin a number of rounds that depends on theτ-threshold rank for τ that is close to 1 (asopposed to the small value of τ needed for

471483473473

Theorems 1.1 and 1.2).

Theorem 1.3. There is an algorithm, basedon rounding r rounds of the Lasserre hier-archy and a constant c, such that for everyε > 0 and input Unique Games instance �with objective value 1 − ε and alphabet size k,satisfying r � ck ·min{ncε1/3 , rank�1−cε(�)}, thealgorithm outputs an assignment x satisfyingval�(x) > 1/2.

Examples of graphs with small thresholdrank. Many interesting graph families havesmall τ-threshold rank for some small constantτ. Random degree d graphs have τ-thresholdequal to 1 for any τ > c/

√d for an absolute

constant c. More generally, the class of small-set expanders – graphs where the expansionof small subsets of vertices is lower-bounded– also have small threshold rank. For instance,if every set of size o(n) expands by at leastpoly(ε) in a graph G, then rank1−ε(G) is atmost npoly(ε) [2]. Generalizing this result, [35]showed that if in a graph G every set of sizeo(n) vertices has near-perfect expansion, thenit implies upper bounds on rankτ(G) for τ closeto 0.

Also, as noted in [2], hypercontractivegraphs (i.e., graphs whose 2 to 4 operatornorm is bounded) have at most polylogarithmicτ-threshold rank for every constant τ > 0.For several 2-CSP’s such as Max Cut, UniqueGames, Small-Set Expansion, Sparsest Cut,the constraint graphs for the canonical “prob-lematic instances” (i.e., integrality gap exam-ples [12], [23], [22], [31]) are all hypercontrac-tive, since they are based on either the noisyGaussian graph or noisy Boolean cube.

Hard instances for our algorithms aregraphs with a large threshold rank. For theUnique Games and Max Cut problems it is triv-ial to construct instances with large thresholdrank by taking many disjoint copies of thesame instance. For other 2-CSPs such as La-bel Cover, several natural hard instances havelinear threshold-rank. For example, the natural“clause vs. variable” or “clause vs. clause”2-CSPs obtained from random instances of3SAT have linear threshold rank. This is notsurprising since a non-trivial approximationfor random 3SAT requires Ω(n) levels of theLasserre hierarchy [32].

However, for the Small-Set Expansion prob-lem all the existing constructions are based onthe noisy boolean cube or the noisy Gaussian

graph, and hence have only polylogarithmicthreshold rank. A subsequent work addressingthis issue, exhibits small set expanders withalmost-polynomial threshold rank [6].

Algorithm efficiency. Our algorithm actuallydoes not require the full power of the Lasserrehierarchy. First, we can use the relaxed variantwith approximate constraints studied in [22],[31], [21]. Second, the proof of Theorem 1.3can be carried out without utilizing the con-straints on all

(nr

)r-sized subsets of n variables,

but rather just sufficiently many random sets.As a result, our r-round algorithm can beimplemented in time 2O(r) poly(n). Due to lackof space, the details of this improvement aredeferred to the full version [7].

1.2. Related works

Subspace enumeration algorithms. ForUnique Games and related problems,previous works [25], [24], [2] used subspaceenumeration to give algorithms with similarrunning time to Theorem 1.3 on instanceswhere the threshold rank of the label extendedgraph is small. This is known to be a stricterrequirement on the instances than boundingthe threshold rank of the constraint graph. Theonly known bound on the 1− ε threshold rankof the label extended graph in terms of the1 − ε threshold rank of the constraint graphloses a factor of about nε [2]. These subspaceenumeration algorithms are applicable only tonearly satisfiable instances (whose objectivevalue is close to 1), and therefore do notyield guarantees comparable to Theorems 1.1and 1.2. Moreover, SDP-based algorithmsare more robust and malleable than spectraltechniques. For instance, it is easy to seethat SDP hierarchies yield polynomial-timeapproximation scheme for 2CSPs whoseconstraint graphs are bounded tree widthgraphs or regular planar graphs (or moregenerally any hyperfinite family of graphs,see e.g. [17] and the references therein),although these classes of graphs could havehigh threshold degree.

Approximation schemes for (pseudo) denseCSP’s. Previously, polynomial-time approxi-mation schemes have been designed for gen-eral 2CSP’s on dense and pseudo-dense in-stances [13], [1], [11]. This work generalizesthese results, since pseudo-density is a strictercondition than having a constraint graph of

472484474474

low threshold rank. Furthermore, for an ε-approximation, these polynomial time approx-imation schemes require the degree of theinstance to be exponential in 1

ε, while our

results hold even on random graphs of degreepoly(1/ε).

Analyzing SDP hierarchy. Using very differ-ent techniques, Chlamtac [9] and Bhaskara etal [8] gave LP/SDP-hierarchy based algorithmsfor graph coloring and the densest subgraphproblem respectively. As mentioned above, sev-eral works gave lower bounds for LP/SDPhierarchies. In particular [31], [22] showed thatapproximation such as those achieved in The-orem 1.3 for Unique Games problem requirelog logΩ(1) n rounds of a relaxed variant of theLasserre hierarchy. This relaxed variant cap-tures our hierarchy as well. Schoenebeck [32]proved that achieving a non-trivial approx-imation for 3SAT on random instances re-quires Ω(n) rounds in the Lasserre hierarchy,while Tulsiani [36] showed that Lasserre lowerbounds are preserved under common types ofNP-hardness reductions.

In a concurrent and independent work, Gu-ruswami and Sinop [16] obtained very similarresults as this work. Using the Lasserre hier-archy, they obtain an approximation schemewith similar performance guarantees as theone in Theorem 1.1 for 2-CSPs, and in facteven consider generalizations involving addi-tional (approximate) global linear constraints.Moreover, they obtain essentially the sameresults as Theorem 1.3 for the case of UniqueGames. Furthermore, the rounding scheme in[16] is identical to ours. However, there areseveral differences both in results and the proof.First, although Guruswami and Sinop [16] usea notion similar to local-to-global correlationused here, they formalize it differently, andinterestingly relate it to the problem of columnselection for low rank approximations of matri-ces. Also, apart from the special case of uniqueconstraints, [16] use a bound on the thresholdrank of the label extended graph, as opposed tothe constraint graph. The analysis in [16] relieson the full power of the Lasserre hierarchy,whereas we show that a weaker hierarchy issufficient in the case of Unique Games, andit can be implemented more efficiently (i.e.,exp(r) poly(n) vs nO(r)).

2. Preliminaries

We will use capital letters X,Y to denoterandom variables, and lower-case letters todenote assignments to these random variables.CP(X) denotes the collision probability of arandom variable. For a random variable X andan element a in its domain, Xa will denote theindicator variable that equals 1 if X = a andequals 0 otherwise. For a random variable Xwith range [k], we define the variance of X asVar[X]

def=

∑a∈[k] Var[X1a] = 1 − CP(X), where

CP(X) denotes the collision probability of X.

Unique games. An instance of UniqueGames consists of a graph G = (V, E), alabel set [k] = {1, . . . , k} and a bijectionπi j : [k] → [k] for every edge (i, j) ∈ E.A labelling � : V → [k] is said to satisfyan edge (i, j) if πi j(�(i)) = �( j). The goalis to find a labeling � : V → [k] that satis-fies the maximum number of edges namely,maximize �(i, j)∈E

{πi j(�(i)) = �( j)

}Local distributions.. Let V = [n] be a set

of vertices and let [k] be a set of labels. Anm-local distribution is a distribution μT overthe set of assignments [k]T of the verticesof some set T ⊆ V of size at most m + 2.(The choice of m + 2 is immaterial but willbe convenient later on.) A collection of m-local distributions {μT }T⊆V, |T |�m+2 is consistentif for all T,T ′ ⊆ V with |T |, |T ′| � m + 2, thedistributions μT and μT ′ are consistent on theirintersection T ∩ T ′. We sometimes will viewthese distributions as random variables, hencewriting X(T )

i for the random variable over [k]that is distributed according to the label thatμT∪{i} assigns to i, and refer to a collectionX1, . . . , Xn of m-local random variables. How-ever, we stress that these are not necessarilyjointly distributed random variables, but ratherfor any subset of at most m + 2 of them, onecan find a sample space on which they arejointly distributed. For succinctness, we omitthe superscript for variables Xi

(S ) whenever itis clear from the context. For example, we willuse {Xi | XS } is short for the random variableobtained by conditioning X(S∪{i})

i on the vari-ables {X(S∪{i})

j } j∈S ;2 and use �{Xi = Xj | XS

}is short for the [0, 1]-valued random variable�

{X(S∪{i, j})

i = X(S∪{i, j})j | X(S∪{i, j})

S

}.

2Strictly speaking, the range of the random variable {Xi |XS } are random variables with range [k]. For every possiblevalue xS for XS , one obtains a [k]-valued random variable{Xi | XS = xS }.

473485475475

Lasserre hierarchy. Let U be a UniqueGames instance with constraint graph G =

(V, E), label set [k] = {1, . . . , k}, and bisections{πi j}i j∈E . An m-round Lasserre solution con-sists of m-local random variables X1, . . . , Xn

and vectors vS ,α for all vertex sets S ⊆ Vwith |S | � m + 2 and all local assignmentsα ∈ [k]S . A Lasserre solution is feasible ifthe local random variables are consistent withthe vectors, in the sense that for all S ,T ⊆ Vand α ∈ [k]S , β ∈ [k]T with |S ∪ T | � m + 2,we have 〈vS ,α, vT,β〉 = � {XS = α, XT = β} .Theobjective is to maximize the following expres-sion �i j∈E �

{Xj = πi j(Xi)

}.An important con-

sequence of the existence of the vectors vS ,αis that for every set S ⊆ V with |S | � mand local assignment xS ∈ [k]S , the matrix{Cov(Xia, Xjb | XS = xS )

}i, j∈V, a,b∈[k] is positive

semidefinite.

3. Warmup – MaxCut Example

For the sake of exposition, we first presentan algorithm for the Max Cut problem on low-rank graphs. In the MaxCut problem, the inputconsists of a graph G = (V, E) and the goal isto find a cut S ∪ S̄ = V of the vertices thatmaximizes the number of edges crossing, i.e.,maximizes |E(S , S̄ )|.

The Goemans-Williamson SDP relaxationfor the problem assigns a unit vector vi forevery vertex i ∈ V , so as to maximize theaverage squared length Ei, j∈E‖vi − v j‖2 of theedges. Formally, the SDP relaxation is givenby,

maximize �i, j∈E

‖vi−v j‖2 subject to ‖vi‖2 = 1 ∀i ∈ V

Stronger SDP relaxations produced by hier-archies such as Sherali-Adams and Lasserrehierarchy also yield probability distributionsover local assignments. More precisely, givena m-round Lasserre SDP solution, it can beassociated with a set of m-local random vari-ables X1, . . . , Xn taking values in {−1, 1}. For anedge (i, j), its contribution to the SDP objectivevalue (‖vi−v j‖2) is equal to the probability thatthe edge (i, j) is cut under the distribution oflocal assignments μi j, namely, �μi j[Xi � Xj] =‖vi − v j‖2 . Consequently, in order to obtaina cut with value close to the SDP objective,it is sufficient to jointly sample X1, . . . , Xn,such that on every edge (i, j) the distributionof Xi and Xj is close to the correspondinglocal distribution μi j. However, the variables

X1, . . . , Xn are not jointly distributed, and hencecannot all be sampled together.

As a first attempt, let us suppose we sam-ple each Xi independently from its associatedmarginal μi. If on most edges (i, j), the dis-tribution of the resulting samples Xi, Xj isclose to μi j, then we are done. On an edge(i, j), the local distribution μi j is far from theindependent sampling distribution μi × μ j onlyif the random variables Xi, Xj are correlated.Henceforth, these correlations across the edgeswould be refered to as “local correlations".A natural measure for correlations that wewill utilize here is defined as Cov(Xi, Xj) =

�[XiXj]−�[Xi]�[Xj]. Using this measure, thestatistical distance between independent sam-pling (μi × μ j) and correlated sampling (μi j) isgiven by

‖μi j − μi × μ j‖1 � |Cov(Xi, Xj)| . (3.1)

(See Lemma 5.3 for a more general version ofthe above bound).

Proof: Under the distribution {XiXj}, theevent {Xi = a, Xj = b} has probability �(a −Xi)(b − Xj)/4. On the other hand, under theproduct distribution {Xi}{Xj}, this event hasprobability �(a − Xi)�(b − Xj)/4. Hence, thedifference of these probabilities is equal to14 (� XiXj − � Xi � Xj) = |Cov(Xi, Xj)|/4. Sum-ming up over the four different assignmentsyields the desired bound.

On the flip side, the existence of correlationsmakes the problem of sampling X1, . . . , Xn

easier! If two variables Xi, Xj are correlated,then sampling/fixing the value of Xi reduces theuncertainty in the value of Xj. More precisely,conditioning on the value of Xi reduces thevariance of Xj as shown below:

�{Xi}

Var[Xj|Xi] = Var[Xj]− 1Var[Xi]

[Cov(Xi, Xj)

]2.

(3.2)Proof: Set A = Xi−� Xi and B = Xj−� Xj.

The random variables A, B have expectationzero, and have the same variance and covari-ance as Xi, Xj. Set B̃ = B/(�[B2])1/2.

The set of random variables {1, B̃} is anorthonormal basis for the subspace of functionsof B. Let ρ = � AB̃. Then, ρB̃ is the orthogonalprojection of A to the subspace of functionsof B. (Here, we use the assumption � A = 0.)Hence, using the previous lemma,

�{B}

Var[A | B] = � A2 − �(ρB̃)2 = � A2 − ρ2 ,

474486476476

which is the desired identity because � A2 =

Var A and ρ2 = Cov(A, B)2/Var B.Therefore, if we pick an i ∈ V at random

and fix its value then the expected decrease inthe variance of all the other variables is givenby,

�i∈V,{Xi}

[�j∈V

Var[Xj|Xi]

]− �

j∈VVar[Xj]

= �i, j∈V

Cov(Xi, Xj)2 · 1

2

(1

Var[Xi]+

1Var[Xj]

).

The above bound is proven in a more generalsetting in Lemma 5.4. As all random variablesinvolved have variance at most 1, we canrewrite the above expression as,

�i∈V,{Xi}

[�j∈V

Var[Xj|Xi]

]− �

j∈VVar[Xj]

� �i, j∈V

|Cov(Xi, Xj)|2 .The decrease in the variance is directly relatedto the global correlations between randompairs of vertices i, j ∈ V .

Recall that, the failure of independent sam-pling yields a lower bound on the aver-age local correlations on the edges namely,Ei, j∈E |Cov(Xi, Xj)|. The crucial observation isthat if the graph G is a good expander ina suitable sense, then these local correlationstranslate in to non-negligible global correla-tions. Formally, we show the following (inSection 4):

Lemma 3.1. Let 𝒗1, . . . ,𝒗n be vectors in theunit ball. Suppose that the vectors are corre-lated across the edges of a regular n-vertexgraph G,

�i j∼G

〈𝒗i,𝒗 j〉 � ρ .Then, the global correlation of the vectors islower bounded by

�i, j∈V

|〈𝒗i,𝒗 j〉| � Ω(ρ)/rank�Ω(ρ)(G) .

where rank�ρ(G) is the number of eigenvaluesof adjacency matrix of G that are larger thanρ.

As random variables Xi arise from the solu-tion to a SDP, the matrix

(Cov(Xi, Xj)

)i, j∈V

ispositive semidefinite, i.e., there exists vectorsui such that 〈ui, u j〉 = Cov(Xi, Xj) ∀i, j ∈ V . Letus consider the vectors vi = u⊗2i . Suppose thelocal correlation �i, j∈E |Cov(Xi, Xj)| is at leastε then we have,

�i, j∈E

〈vi, v j〉 = �i, j∈E

|Cov(Xi, Xj)|2 � ε2 ,

and �i[‖vi‖2] � 1. If the graph G is low-rank,then by Lemma 3.1 we get a lower bound onthe global correlation of the vectors vi, namely

�i, j∈V

|Cov(Xi, Xj)|2 = �i, j∈V

〈vi, v j〉 � Ω(ε2)/rank�ε2 (G) .

Summarizing, if the independent samplingis on average ε-far from correlated samplingover the edges, then conditioning on the valueof a random vertex i ∈ V reduces the averagevariance by ε2/rank�ε2 (G) in expectation. Thesame argument can now be applied on thevariables obtained after conditioning on i. Infact, starting with an SDP solution to m-roundLasserre hierarchy, the local distributions re-main consistent and their covariance matricesremain semidefinite as long as we condition onat most m − 2 vertices. Observe that averagevariance is at most 1. Hence, after at mostrank�ε2 (G)/ε2 steps, the independent samplingdistribution will be within average distance εfrom the correlated sampling on the edges.

The rounding scheme will iteratively con-dition the SDP solution on the value of avertex for at most rank�ε2 (G)/ε2 iterations. Ineach iteration, a uniformly randomly chosenvertex Xi is fixed to a value sampled from itscorresponding marginal distribution. When theglobal correlation is less than ε after condition-ing, the algorithm just rounds the solution byindependent sampling, and outputs the assign-ment.

Rounding SDP’s using a small basis. Thereis an alternate way to view the above roundingscheme. Note that in the case that the vectorsv1, . . . , vn are one dimensional unit vectors (i.e.,vi ∈ {±1}), V exactly corresponds to a cut inthe graph, and the objective value measures thefraction of edges cut. Now, suppose that youcould find r vectors vi1 , . . . , vir ∈ V, whomwe’ll call the basis vectors, such that everyother v ∈ V has some significant projectionρ into the span of vi1 , . . . , vir . That is, if we letP be the projection operator corresponding tothis space, then for every v ∈ V , ‖Pv‖2 � ρ.It turns out that in this case, if ρ is suffi-ciently close to 1 and the vector solution Vsatisfied r + 2 rounds of an appropriate SDPhierarchy, then we can round V to achieve avery good cut. The intuition behind this is thefollowing: the constraints of r + 2 hierarchyrounds allow us to essentially assume withoutloss of generality that the vectors vi1 , . . . , virare one-dimensional. That is, after applying anappropriate rotation, we can think of each one

475487477477

of them as a vector of the form (±1, 0, . . . , 0).Moreover, our assumption implies that everyother vector in v has a magnitude of at least ρin its first coordinate. Now one can show thatsimply rounding each vector to the sign of itsfirst coordinate will result in a ±1 assignmentto the vertices corresponding to a good cut.

Hence, the problem of rounding reduces tofinding a small basis of vectors vi1 , . . . , vir , suchthat the SDP solution has a significant projec-tion or lies almost completely in their span.Again, if the local correlation on the edges�i∼ j〈vi, v j〉2 is tiny, then independent sam-pling yields a good assignment. Otherwise byLemma 3.1, the global correlation �i, j〈vi, v j〉2is non-negligible (Ω(ε2)/rank�ε2 (G)). In partic-ular, there exists at least one vector vi1 suchthat � j〈vi1 , v j〉2 � Ω(ε2)/rank�ε2 (G). We cannow replace each vector v ∈ V with itsprojection into the orthogonal space to vi1 andcontinue. Eventually, either the local correla-tion becomes negligible making independentsampling a good rounding, or we find a basisvi1 , . . . , vir such that (almost all) vectors v ∈ Vhave most of their mass in Span{vi1 , . . . , vir },in which case we can successfully round thesolution.

Threshold rank vs global correlation. When-ever the graph has small number of largeeigenvalues, the condition that local correlationimplies global correlation holds. This is usefulto simulate eigenspace enumeration algorithmssuch as used by [25], [24], [2], [35] sincein the case of Unique Games (and other re-lated problems), a good SDP solution mustbe locally well correlated. But the notion oflocal to global correlation is somewhat moregeneral and robust than having small thresholdrank. For example, adding

√n isolated vertices

to a graph will increase correspondingly thenumber of eigenvectors with value 1, but willactually not change by much the local to globalcorrelation. This captures to a certain extent thefact that SDP-based solutions are more robustthan the spectral based algorithms. (A similarexample of this phenomenon is that adding atiny bipartite disjoint graph to the input graphmakes the smallest eigenvalue become −1, butdoes not change by much the value of theGoemans-Williamson SDP.) We hope that thisrobustness of the SDP-based approach willenable further improvements in the future.

4. Local Correlation implies GlobalCorrelation in Low-Rank Graphs

In this section, we present the crucial ingredi-ent for our algorithm – the translation of local-correlations to non-trivial global correlations(Lemma 3.1) in expanders and more generallylow-rank graphs. Let G be a regular graph withvertex set V = {1, . . . , n}. We identify G withits normalized adjacency matrix, a symmetricstochastic matrix. Let λ1 � . . . � λn ∈ [−1, 1]be the eigenvalues of G in non-increasing or-der.

The following lemma shows that a violationof the local vs global correlation conditionimplies that the graph has high threshold rank.

Lemma 4.1. Suppose there exist vectorsv1, . . . , vn ∈ �n such that

�i j∼G

〈vi, v j〉 � 1 − ε , �i, j∈V

〈vi, v j〉2 � 1m ,

�i∈V‖vi‖2 = 1 .

Then for all C > 1, λ(1−1/C)m � 1 − C · ε. Inparticular, λm/2 > 1 − 2ε.

Proof: Let X = (xr,s)r,s∈[n] be the Gram ma-trix (〈𝒗i,𝒗 j〉)i, j∈V represented in the eigenbasisof G, so that

�i j∼G

〈vi, v j〉 =∑r∈[n]λr xr,r , �

i, j∈V〈vi, v j〉2 =

∑r,s∈[n]

x2r,s ,

�i∈V‖vi‖2 =

∑r∈[n]

xr,r .

Let m′ be the largest index such that λm′ �1 − C · ε. Notice that the numbers p1 =

x1,1, . . . , pn = xn,n form a probability distri-bution over r ∈ [n]. Let q =

∑m′i=1 pi be the

probability of the event r � m′. Using Cauchy–Schwarz, we can bound this probability interms of m, q =

∑m′r=1 pr � m′ ∑n

r=1 p2r �

m′m . On

the other hand, we can bound the expectationof λr with respect to the probability distribution(p1, . . . ,n ) in terms of this probability q,

1 − ε �n∑

r=1

λr pr �m′∑

r=1

pr + (1 −C · ε)m∑

r=m′+1

pr

= 1 − (1 − q)C · ε � 1 −(1 − m′

m

)C · ε .

It follows that m′ � (1 − 1/C) · m, which givesthe desired conclusion that G has at least(1 − 1/C) · m eigenvalues λr � −C · ε.

Note that Lemma 3.1 follows directlyfrom the previous lemma by picking C =

476488478478

(1−ρ/100)(1−ρ) and observing that �i, j∈V |〈𝒗i,𝒗 j〉| ��i, j∈V |〈𝒗i,𝒗 j〉|2 since |〈𝒗i,𝒗 j〉| � 1 for alli, j ∈ V . A weaker converse for Lemma 3.1is also true (see full version [7]).

5. General 2-CSP on Low Rank Graphs

Let � be a (general) Max 2-Csp instancewith variable set V = [n] and label set [k].(We represent � as a distribution over triples(i, j,Π), where i, j ∈ V and Π ⊆ [k] × [k] isan arbitrary binary predicate. The goal is tofind an assignment x ∈ [k]V that maximizesthe probability �(i, j,Π)∼�

{(xi, x j) ∈ Π

}.)

For simplicity,3 we will assume that theconstraint graph of � is regular, i.e., everyvariable i ∈ V appears in the same numberof constraints. (Since we allow the constraintsto be weighted, the precise condition is thatthe total weight of the constraints incident to avertex is the same for every vertex.)

The Lasserre program for � gives rise to m-local random variables X1, . . . , Xn with range[k]. We write Xia to denote the {0, 1}-indicatorof the event Xi = a. Notice that {Xia}i∈V, a∈[k] arealso m-local random variables.

The rounding algorithm and its analysis area somewhat straight-forward generalization ofthe algorithm for MaxCut described in Sec-tion 3.

Algorithm 5.1 (Propagation Sampling).Input: r-local random variables X1, . . . , Xn

over [k]Output:(global) distribution over assign-

ments x ∈ [k]V .

1) Choose m ∈ {1, . . . , r} at random.2) Sample a random set of “seed vertices”

S ∈ Vm. (Repeated vertices are allowed.)3) Sample a assignment xS ∈ [k]S for S

according to its local distribution {XS }.4) For every other vertex i ∈ V \ S , sample

a label xi ∈ [k] according to the localdistribution for S ∪ {i} conditioned onthe assignment xS for S .

Our main Theorem of this section, whichimmediately implies Theorem 1.1, is the fol-lowing:

Theorem 5.2. Let ε > 0 and r = O(k) ·rank�Ω(ε/k)2 (G)/ε4. Suppose that the r-roundLasserre value of the Max 2-Csp instance � is

3If the constraint graph is not regular, all of our resultsstill hold for an appropriate definition of threshold rank.

σ. Then, given an optimal r-round Lasserre so-lution, Algorithm 5.1 (Propagation Sampling)outputs an assignment with expected value atleast σ − ε for � .

The proof of the theorem follows the generaloutline in Section 3. Specifically, show that aslong as for a random pair (i, j) of indices thereis noticeable correlation between Xi and Xj,conditioning on Xi “makes progress” and re-duces the global variance of the system, whileonce there is no correlation between almost allpairs (i, j) involved in constraints, then we mayas well use independent rounding to come upwith a solution. The fact that the graph has lowrank, allows us to pass between correlations fora random pair (i, j), and correlations for a ran-dom pair (i, j) that is involved in a constraint.

To implement this approach, we need toformalize the appropriate measures of correla-tion, and show an equivalence between them.One measure is the statistical distance betweencorrelated and independent sampling of twovariables Xi, Xj. If this measure is low on theedges of the constraint graph, then an indepen-dent sampling of the variables would give agood assignment. As in (3.1), we relate thismeasure to the covariances of the correspond-ing variables {Xia}, {Xjb}.Lemma 5.3. ] Let Xi and Xj be the tworandom variables over [k] from the Lasserresolution, then∥∥∥{XiXj} − {Xi}{Xj}

∥∥∥1=

∑(a,b)∈[k]2

∣∣∣Cov(Xia, Xjb)∣∣∣ .

where the left-hand side denotes the �1 (i.e.total variation) distance between the joint dis-tribution of Xi and Xj and the distribution whenthey are sampled independently.

We omit the (easy) proof from this extendedabstract. Next, we lower-bound the decreasein variance during the conditioning process interms of the covariances (analogue of (3.2)).

Lemma 5.4. For any two vertices i, j ∈ V,

Var Xi − �{Xj}Var

[Xi

∣∣∣ Xj

]�

1k

∑a,b∈[k]

�{XiaX jb}

Cov(Xia, Xjb)2/Var Xjb

We defer the easy proof to the full ver-sion [7]. We now have two natural measuresof correlation between Xi and Xj— one isthe statistical distance between sampling Xi, Xj

477489479479

jointly and independently, and the other is theamount the variance of Xi decreases after con-ditioning on Xj, with both ways relating to thecovariances of Xia and Xib. By using convexityarguments it can be shown that, in our casewhere the local random variables come froma semidefinite program, both notions can beapproximated by inner products of vectors:

Lemma 5.5. Suppose that the matrix(Cov(Xia, Xjb)

)i∈V, a∈[k] is positive semidefinite.

Then, there exists vectors 𝒗1, . . . ,𝒗n in theunit ball such that for all vertices i, j ∈ V,

1k2

( ∑(a,b)∈[k]2

∣∣∣Cov(Xia, Xjb)∣∣∣)2 � 〈𝒗i,𝒗 j〉 �

1k

∑(a,b)∈[k]2

12 (

1Var Xia

+ 1Var Xjb

) Cov(Xia, Xjb)2 .

With the equivalence between these notionsof correlations, it is easy to finish the argumentusing Lemma 3.1. Specifically, Lemma 3.1implies that if the variables are globally un-correlated, then they are locally uncorrelated.Therefore, after a small number of condition-ings, the independent sampling of the variablesyields an assignment whose value is close tothat of the SDP solution.

6. Results for Unique Games

As mentioned earlier, in case the CSP is ofthe Unique Games type, the threshold for theeigenvalues does not need to depend on thealphabet size k. The key technical step is thefollowing variant of Lemma 5.5:

Lemma 6.1. Let X1, . . . , Xn be r-local randomvariables over [k] and let Xia be the indicatorof the event Xi = a. Suppose that the matrix(Cov(Xia, Xjb)

)i∈V, a∈[k] is positive semidefinite.

Then, there exists vectors 𝒗1, . . . ,𝒗n in the unitball such that for all vertices i, j ∈ V andpermutations π of [k],

( ∑a∈[k]

∣∣∣Cov(Xia, Xj π(a))∣∣∣)4 � 〈𝒗i,𝒗 j〉 �

∑(a,b)∈[k]2

12 (

1Var Xia

+ 1Var Xjb

) Cov(Xia, Xjb)2 .

In fact, for unique games we can also obtainresults when the threshold is a constant closeto 1 as opposed to close to zero, or equiv-alently, only requiring that the Laplacian ofthe constraint graph has few eigenvalues closeto 0. Formally, this is stated in the followingtheorem:

Theorem 6.2. For every positive integer m,there exists an algorithm running in timenO(mk2) that given a unique games instance Γ

over alphabet [k] with value 1 − η, finds alabelling satisfying 1 − O( η

λm) fraction of the

edges. Here λm is the mth smallest eigenvalueof the Laplacian of the constraint graph Γ.

The (omitted) proof follows similar lines tothe proof of Theorem 5.2, but is not identical.In particular, at the moment our analysis re-quires to use a variant of propagation roundingwhere we enumerate over all possible seedsets of a certain size rather than choosing arandom one. We can then use this to obtain asubexponential algorithm for every instance ofUnique Games, thus proving Theorem 1.3. Theproof follows by combining Theorem 6.2 andthe decomposition theorem of [2] that showedhow to break up any graph into pieces of lowrank. The main observation is that one does notneed to use the decomposition first, and thenrun the SDP on each part, but rather we can usethe decomposition in order to round the SDPsolution, by applying propagation rounding toeach part independently.

Conclusions

We have shown that nO(ε1/3) rounds of anSDP hierarchy suffice for solving the UniqueGames problem on (1− ε)-satisfiable instances.The best lower bound known for the hierarchywe used is log logΩ(1) n [31], [22], and findingthe tight bound has obvious relevance to theunique games conjecture.

It is our hope that the correlation basedrounding technique presented here, will yieldmore approximation algorithms based on SDPhierarchies. Indeed, Arora and Ge (personalcommunication) recently used ideas from thiswork to obtain improved algorithms for 3-coloring on an interesting families of instances.

References

[1] N. Alon, A. Coja-Oghlan, H. Hàn, M. Kang,V. Rödl, and M. Schacht, “Quasi-randomnessand algorithmic regularity for graphs with gen-eral degree distributions,” SIAM J. Comput.,vol. 39, no. 6, pp. 2336–2362, 2010.

[2] S. Arora, B. Barak, and D. Steurer, “Subex-ponential algorithms for unique games andrelated problems,” in FOCS, 2010, pp. 563–572.

[3] S. Arora, B. Bollobás, L. Lovász, andI. Tourlakis, “Proving integrality gaps withoutknowing the linear program,” Theory of Com-puting, vol. 2, no. 1, pp. 19–51, 2006.

478490480480

[4] S. Arora, S. Khot, A. Kolla, D. Steurer, M. Tul-siani, and N. K. Vishnoi, “Unique games on ex-panding constraint graphs are easy,” in STOC,2008, pp. 21–28.

[5] S. Arora, S. Rao, and U. V. Vazirani, “Ex-pander flows, geometric embeddings and graphpartitioning,” J. ACM, vol. 56, no. 2, 2009.

[6] B. Barak, P. Gopalan, J. Hastad, R. Meka,P. Raghavendra, and D. Steurer, “Making thelong code shorter, with applications to uniquegames conjecture,” 2011, manuscript.

[7] B. Barak, P. Raghavendra, and D. Steurer,“Rounding semidefinite programming hierar-chies via global correlation,” Electronic Collo-quium on Computational Complexity (ECCC),vol. 18, p. 65, 2011.

[8] A. Bhaskara, M. Charikar, E. Chlamtac,U. Feige, and A. Vijayaraghavan, “Detectinghigh log-densities: an O(n1/4) approximationfor densest k-subgraph,” in STOC, 2010, pp.201–210.

[9] E. Chlamtac, “Approximation algorithms usinghierarchies of semidefinite programming relax-ations,” in FOCS, 2007, pp. 691–701.

[10] E. Chlamtac and M. Tulsiani, “Convex relax-ations and integrality gaps,” 2010, chapter inHandbook on Semidefinite, Cone and Polyno-mial Optimization.

[11] A. Coja-Oghlan, C. Cooper, and A. M. Frieze,“An efficient sparse regularity concept,” SIAMJ. Discrete Math., vol. 23, no. 4, pp. 2000–2034, 2010.

[12] U. Feige and G. Schechtman, “On the op-timality of the random hyperplane roundingtechnique for MAX CUT,” Random Struct.Algorithms, vol. 20, no. 3, pp. 403–440, 2002.

[13] A. M. Frieze and R. Kannan, “Quick approx-imation to matrices and applications,” Combi-natorica, vol. 19, no. 2, pp. 175–220, 1999.

[14] K. Georgiou, A. Magen, T. Pitassi, andI. Tourlakis, “Integrality gaps of 2 − o(1) forvertex cover sdps in the Lovász–Schrijver hi-erarchy,” in FOCS, 2007, pp. 702–712.

[15] M. X. Goemans and D. P. Williamson, “Im-proved approximation algorithms for maxi-mum cut and satisfiability problems usingsemidefinite programming,” J. ACM, vol. 42,no. 6, pp. 1115–1145, 1995.

[16] V. Guruswami and A. K. Sinop, “Lasserre hier-archy, higher eigenvalues, and approximationschemes for quadratic integer programmingwith psd objectives,” fOCS 2011.

[17] A. Hassidim, J. A. Kelner, H. N. Nguyen, andK. Onak, “Local graph partitions for approxi-mation and testing,” in FOCS, 2009, pp. 22–31.

[18] D. R. Karger, R. Motwani, and M. Sudan,“Approximate graph coloring by semidefiniteprogramming,” J. ACM, vol. 45, no. 2, pp. 246–265, 1998.

[19] S. Khot, “On the power of unique 2-prover 1-

round games,” in STOC, 2002, pp. 767–775.[20] S. Khot, G. Kindler, E. Mossel, and

R. O’Donnell, “Optimal inapproximabilityresults for Max-Cut and other 2-variableCSPs?” in FOCS, 2004, pp. 146–154.

[21] S. Khot, P. Popat, and R. Saket, “Approximatelasserre integrality gap for unique games,” inAPPROX-RANDOM, 2010, pp. 298–311.

[22] S. Khot and R. Saket, “SDP integrality gapswith local �1-embeddability,” in FOCS, 2009,pp. 565–574.

[23] S. Khot and N. K. Vishnoi, “The unique gamesconjecture, integrality gap for cut problemsand embeddability of negative type metricsinto �1,” in FOCS, 2005, pp. 53–62.

[24] A. Kolla, “Spectral algorithms for uniquegames,” in IEEE CCC, 2010, pp. 122–130.

[25] A. Kolla and M. Tulsiani, “Playing randomand expanding unique games,” 2007, unpub-lished manuscript available from the authors’webpages, to appear in journal version of [4].

[26] J. B. Lasserre, “Global optimization with poly-nomials and the problem of moments,” SIAMJournal on Optimization, vol. 11, no. 3, pp.796–817, 2001.

[27] M. Laurent, “A comparison of the sherali-adams, lovász-schrijver, and lasserre relax-ations for 0-1 programming,” Math. Oper. Res.,vol. 28, no. 3, pp. 470–496, 2003.

[28] L. Lovász and A. Schrijver, “Cones of matricesand set-functions and 0-1 optimization,” SIAMJ. Optim., vol. 1, no. 2, pp. 166–190, 1991.

[29] E. Mossel, R. O’Donnell, and K. Oleszkiewicz,“Noise stability of functions with low influ-ences invariance and optimality,” in FOCS,2005, pp. 21–30.

[30] P. Raghavendra, “Optimal algorithms and inap-proximability results for every CSP?” in STOC,2008, pp. 245–254.

[31] P. Raghavendra and D. Steurer, “Integralitygaps for strong SDP relaxations of uniquegames,” in FOCS, 2009, pp. 575–585.

[32] G. Schoenebeck, “Linear level Lasserre lowerbounds for certain k-CSPs,” in FOCS, 2008,pp. 593–602.

[33] G. Schoenebeck, L. Trevisan, and M. Tul-siani, “A linear round lower bound for lovasz-schrijver sdp relaxations of vertex cover,” inIEEE CCC, 2007, pp. 205–216.

[34] H. D. Sherali and W. P. Adams, “A hierar-chy of relaxations between the continuous andconvex hull representations for zero-one pro-gramming problems,” SIAM J. Discrete Math.,vol. 3, no. 3, pp. 411–430, 1990.

[35] D. Steurer, “Subexponential algorithms for d-to-1 two-prover games and for certifying al-most perfect expansion,” 2010, manuscript.

[36] M. Tulsiani, “CSP gaps and reductions in theLasserre hierarchy,” in STOC, 2009, pp. 303–312.

479491481481