Semidefinite Programming Semidefinite Programming Based Approximation Based Approximation

AlgorithmsAlgorithms

Uri ZwickUri Zwick

Tel Aviv University

UKCRC’02, Warwick UKCRC’02, Warwick University, University,

May 3, 2002.May 3, 2002.

Outline of talkOutline of talk

Semidefinite programming

MAX CUT (Goemans, Williamson ’95)

MAX 2-SAT and MAX DI-CUT (FG’95, MM’01, LLZ’02)

MAX 3-SAT (Karloff, Zwick ’97)

-function (Lovász ’79)

MAX k-CUT (Frieze, Jerrum ’95)

Colouring k-colourable graphs (Karger, Motwani, Sudan ’95)

Positive Semidefinite MatricesPositive Semidefinite Matrices

A symmetric nn matrix A is PSDPSD iff:

• xTAx 0 , for every xRn.

• A=BTB , for some mn matrix B.

• All the eigenvalues of A are non-negative.

Notation: A 0 iff A is PSD

Linear Linear ProgrammiProgrammi

ngngmax c x

s.t. ai x bi

x 0

Semidefinite Semidefinite ProgrammingProgramming

max CX

s.t. Ai X bi

X 0

Can be solved exactly

in polynomial time

Can be solved almost exactly

in polynomial time

LP/SDP algorithmsLP/SDP algorithms

• Simplex method (LP only)

• Ellipsoid method

• Interior point methods

Algorithms work well in practice, not only in theory!

Semidefinite Semidefinite ProgrammingProgramming(Equivalent formulation)

max cij (vi vj)

s.t. aij(k) (vi vj) b(k)

vi Rn

X ≥ 0 iff X=BTB. If B = [v1 v2 … vn] then xij = vi · vj .

Lovász’s Lovász’s -function-function(one of many formulations)

max JX

s.t. xij = 0 , (i,j)E

I X = 1

X 0

Orthogonal representation

of a graph:vi vj = 0 ,

whenever (i,j)E

The Sandwich TheoremThe Sandwich Theorem(Grötschel-Lovász-Schrijver ’81)

)G()G()G(

Size of max clique

Chromaticnumber

The The MAX CUTMAX CUT problem problem

Edges may be weighted

The The MAX CUTMAX CUT problem: problem:

motivationmotivation Given: n activities, m persons.

Each activity can be scheduled either in the morning or in the afternoon.

Each person interested in two activities.

Task: schedule the activities to maximize the number of persons that can enjoy both activities.

If exactly n/2 of the activities have to be held in the morning, we get MAX BISECTIONMAX BISECTION..

The The MAX CUTMAX CUT problem: problem: statusstatus

• Problem is NP-hard

• Problem is APX-hard (no PTAS unless P=NP)

• Best approximation ratio known, without SDP, is only ½. (Choose a random cut…)

• With SDP, an approximation ratio of 0.878 can be obtained! (Goemans-Williamson ’95)

• Getting an approximation ratio of 0.942 is NP-hard! (PCP theorem, …, Håstad’97)

A quadratic integer A quadratic integer programming formulation of programming formulation of

MAX CUTMAX CUT

}1,1{ s.t.2

1Max

i

jiij

x

xxw

An SDP Relaxation of An SDP Relaxation of MAX MAX CUTCUT

(Goemans-Williamson ’95)

1||||, s.t.

2

1Max

in

i

jiij

vRv

vvw

An SDP Relaxation of An SDP Relaxation of MAX CUT – MAX CUT –

Geometric intuitionGeometric intuition

Embed the vertices of the graph on the unit sphere such that vertices that are joined by edges are far apart.

Random hyperplane Random hyperplane roundingrounding

(Goemans-Williamson ’95)(Goemans-Williamson ’95)

To choose a random hyperplane,

choose a random normal vector

r

If r = (r1 , r2 , …, rn), andr1, r2 , … , rn N(0,1), then the direction of r

is uniformly distributed over the n-dimensional

unit sphere.

The probability that two vectors are separated

by a random hyperplane

vi

vj

Analysis of the Analysis of the MAX CUTMAX CUT Algorithm Algorithm (Goemans-Williamson (Goemans-Williamson

’95)’95)

1

1

1

1

exp

sdp

ratio min

1

2

2

1

cos

0.8785

( )

cos (.

)6. .

ii

jj

ijj

x

i

w

v v

x

v

w

v

x

Is the analysis tight?Is the analysis tight?

Yes!Yes!

(Karloff ’96) (Feige-Schechtman ’00)

The The MAX Directed-CUTMAX Directed-CUT problemproblem

Edges may be weighted

The The MAX 2-SATMAX 2-SAT problemproblem

747354

435362

435221

xxxxxx

xxxxxx

xxxxxx

A Semidefinite A Semidefinite Programming Relaxation Programming Relaxation

of of MAX 2-SATMAX 2-SAT(Feige-Lovász ’92, Feige-Goemans ’95)

1

1

201s.t.4

3Max 00

||||,

,

,,,

in

i

iin

kjkiji

jijiij

vRv

nivv

nkjivvvvvv

vvvvvvw

Triangle constraints

The probability that a clause xi xj is satisfied is :

2ijj0i0

ij

iv

jv

0v

Pre-rounding Pre-rounding rotationsrotations

(Feige-Goemans ‘95)iv

0v

iv

iv

i0 ivi0

)()(

],0[],0[:

)( 00

ff

f

f ii

Skewed hyperplanesSkewed hyperplanes(Feige-Goemans ’95, Matuura-Matsui ’01)

Choose a random vector r that is skewed toward v0.

Without loss of generality v0 = (1,0, …,0).

Let r = (r1 , r2 , …, rn), where r2 , …, rn ~ N(0,1).Choose r1 according to a different distribution.

““Threshold” roundingThreshold” rounding(Lewin-Livnat-Zwick ’02)

Choose a random vector r perpendicular to v0.

Set xi=1 iff vi · r ≥ T( v0· vi ).

Results for Results for MAX 2-SATMAX 2-SATAuthorsTechniqueBound

Goemans-Williamson ‘95Random

hyperplane0.878

Feige-Goemans ‘95Pre-rounding

rotations0.931

Matuura-Matsui ‘01Skewed

hyperplanes0.935

Lewin-Livnat-Zwick ‘02Threshold rounding0.941

Integrality ratio *0.945

Inapproximability0.954

The The MAX 3-SATMAX 3-SAT problem problem(Karloff-Zwick ’97 Zwick ’02)(Karloff-Zwick ’97 Zwick ’02)

A performance ratio of 7/8 is obtained using:

A more complicated SDP relaxation The simple random hyperplane rounding. A much more complicated analysis. Computer assisted proof. (Z’02)

Approximability and Approximability and Inapproximability resultsInapproximability results

ProblemApprox.

RatioInapprox.

RatioAuthors

MAX CUT0.87816/17 0.941

Goemans Williamson ’95

MAX DI-CUT0.87412/13 0.923

GW’95, FW’95 MM’01, LLZ’01

MAX 2-SAT0.94121/22 0.954

GW’95, FW’95 MM’01, LLZ’01

MAX 3-SAT7/87/8Karloff

Zwick ’97

What else can we What else can we do with do with SDPSDPs?s?

• MAX BISECTION MAX BISECTION (Frieze-Jerrum ’95)

• MAX MAX kk-CUT-CUT (Frieze-Jerrum ’95)

• (Approximate) Graph colouring (Karger-Motwani-Sudan’95)

(Approximate) Graph (Approximate) Graph colouringcolouring

• Given a 3-colourable graph, colour it, in polynomial time,

using as few colours as possible.• Colouring using 4 colours is still NP-hard. (Khanna-Linial-Safra’93 Khanna-Guruswami’01)

• A simple combinatorial algorithm can colour, in polynomial time, using about n1/2 colours.

(Wigderson’81)

• Using SDP, can colour (in poly. time) using n1/4 colours (KMS’95), or even n3/14 colours (BK’97).

Vector Vector kk-Coloring-Coloring((Karger-Motwani-Sudan ’95)

A vector k-coloring of a graph G = (V,E) is a sequence of unit vectors v1 , v2 , … , vn

such that if (i,j)E then vi · vj = -1/(k-1).

The minimum k for which G is vector k-colorable is ( )G

A vector k-coloring, if one exists, can be found using SDP.

Lemma: If G = (V,E) is k-colorable, then it is also vector k-colorable.

Proof: There are k vectors v1 ,v2 , … , vk

such that vi · vj = -1/(k-1), for i ≠ j.

k = 3 :

Finding large independent Finding large independent setssets

((Karger-Motwani-Sudan ’95)Let r be a random normally distributed vector in Rn. Let .

I’ is obtained from I by removing a vertex from each edge of I.

lnlnln 31

32c

}|{ crvViI i

Constructing a large Constructing a large ISIS

riv

jv

Colouring Colouring kk-colourable -colourable graphsgraphs

Colouring k-colourable graphs using min{ Δ1-2/k , n1-3/(k+1) } colours.

(Karger-Motwani-Sudan ’95)

Colouring 3-colourable graphs using n3/14 colours.

(Blum-Karger ’97)

Colouring 4-colourable graphs using n7/19 colours.

(Halperin-Nathaniel-Zwick ’01)

Open problemsOpen problems

• Improved results for the problems considered.

• Further applications of SDP.