Linear Programming and Approximation

Seminar in Approximation Algorithms

Linear Programming

Linear objective function and Linear constraints Canonical form Standard form

bAxxc :min T

0xbAxxc ,:min T

0, Unbounded

0

1

1

1

11

jj

jjjj

n

jjj

n

jjjn

jjj

n

jjj

n

jjj

xx

xxxx

bxa

bxa

bxa

s

bsxabxa

Linear Programming – Example

0,,

625

103..

57min

321

321

321

321

xxx

xxx

xxxts

xxx

x=(2,1,3) is a feasible solution 7 * 2 + 1 + 5 * 3 = 30 is an upper bound the optimum

Lower Bound

How can we find a lower bound? For example,

10357 321321 xxxxxx

0,,

625

103..

57min

321

321

321

321

xxx

xxx

xxxts

xxx

Lower Bound

Another example:16)25()3(57 321321321 xxxxxxxxx

0,,

625

103..

57min

321

321

321

321

xxx

xxx

xxxts

xxx

Lower Bound

Assign a non-negative coefficient yi to every primal inequality such that

Lower bound 10y1+6y2

0,,

)(625

)(103..

57min

321

2321

1321

321

xxx

yxxx

yxxxts

xxx

32132123211 57)25()3( xxxxxxyxxxy

LP Duality

The problem of finding the best lower bound is a linear program

0,

53

12

75..

610max

Dual

21

21

21

21

21

yy

yy

yy

yyts

yy

0,,

625

103..

57min

Primal

321

321

321

321

xxx

xxx

xxxts

xxx

LP Duality

0y

cyA

yb

T

T

ts ..

max

For every x and y: cTx bTy Thus, Opt(primal) Opt(dual) The dual of the dual is the primal

0x

bAx

xc

..

min

ts

T

Example

0,,

625

103..

57min

Primal

321

321

321

321

xxx

xxx

xxxts

xxx

0,

53

12

75..

610max

Dual

21

21

21

21

21

yy

yy

yy

yyts

yy

x=(7/4,0,11/4) and y=(2,1) are feasible solutions

7*7/4 + 0 + 5*11/4 = 10*2 + 6*1 = 26

Thus, x and y are optimal

Max Flow vs. Min s,t-cut

Path(s,t) - the set of directed paths from s to t.

Max Flow:

Dual:

Implicit Min s,t-cut Opt(Dual) = Opt(Min s,t-cut)

),(Path0

s.t.

max

:

1

tsPf

Eecf

f

ii

ePePi

m

ii

ii

Eed

tsPd

dc

e

iPe

e

Eeee

i

0

),(Path1s.t.

min

1ed 1,0ed

LP-duality Theorem

Theorem: Opt(primal) is finite Opt(dual) is finite If x* and y* are optimal

Conclusion: LP NPcoNP

m

iii

n

jjj ybxc

1

*

1

*

Weak Duality Theorem

If x and y are feasible

Proof:

m

iii

n

jjj ybxc

11

m

iii

m

ii

n

jjij

n

jj

m

iiij

n

jjj ybyxaxyaxc

11 11 11

)()(

Complementary Slackness Conditions

x and y are optimal

Primal conditions:

Dual conditions:j

m

iiijj cyaxj

1

or 0either ,

i

n

jjiji bxayi

1

or 0either ,

Complementary Slackness Conditions - Proof

j

m

iiijj cyax

1

or 0Either

By the duality theorem

Similar arguments work for y For other direction read the slide upwards

m

iii

m

ii

n

jjij

n

jj

m

iiij

n

jjj ybyxaxyaxc

11 11 11

)()(

01 1

n

jj

m

iiijj xyac

0 and 01

j

m

iiijj xyac

Algorithms for Solving LP

Simplex [Dantzig 47] Ellipsoid [Khachian 79] Interior point [Karmakar 84]

Open question: Is there a strongly polynomial time algorithm for LP?

Integer Programming

NP-hard Branch and Bound

LP relaxation Opt(LP) Opt(IP) Integrality Gap – Opt(IP)/Opt(LP)

nNx

0x

bAx

xc

s.t.

min T

Using LP for Approximation

Finding a good lower bound - Opt(LP) Opt(IP) Integrality gap is the best we can hope for

Techniques: Rounding

– Solve LP-relaxation and then round solution.

Primal Dual– Find a feasible dual solution y, and a feasible integral primal

solution x such that cTx r * bTy

Minimum Vertex Cover

LP-relaxation and its dual: (P)

(D)

Vux

Evuexx

xc

u

vu

Vuuu

0

),(1s.t.

min

Eey

Vucy

y

e

ueuee

Eee

0

s.t.

max

:

Solving the Dual

Algorithm:

1. Solve (D)

2. Define:

Analysis:

CD is a cover:

ueueeD cyuC

:

:

eevDuD yyCxCxvue 0)()(),,(

Solving the Dual

CD is r-approx:

Weak duality The. *2

2

)(

)(

)(

:

x

y

Cxy

yCx

Cxc

Eee

Ee euuDe

Vu eueeuD

VuuDu

eueeuD ycCu

:

,

eue

DCx:

2)(

Rounding

Algorithm:

1. Solve (P)

2. Define:

Analysis:

x* is not feasible Due to the complementary conditions:

Conclusion: x(CP) is 2-approx

0: * uP xuC

)()( DP CxCx

0)()(),,( vPuP CxCxvue

Rounding

Algorithm:

1. Solve (P)

2. Define:

Analysis:

x* is not feasible Clearly, Conclusion: x(C½) is 2-approx

21*:2

1 uxuC

)()(2

1 PCxCx

0)()(),,(2

12

1 vu CxCxvue

The Geometry of LP Claim:

Let x1,x2 be feasible solutions

Then, x1+(1-)x2 is a feasible solution

Definition:x is a vertex if

Theorem:If a finite solution exists, there exists anoptimal solution x* which is a vertex

2121 )1( ,0,, xxxx

Half IntegralityTheorem: If x is a vertex then x{0,1/2,1}n

Proof: Let x be a vertex such that x {0,1/2,1}n

Both solutions are feasible = shortest distance to {0,1/2,1}

Conclusion: 1,,0,such that 21** jxjx

21*

*2

1

0:

1:

v

v

xvV

xvV

Otherwise

'*

*

*

v

v

v

v

x

Vvx

Vvx

x

Otherwise

"*

*

*

v

v

v

v

x

Vvx

Vvx

x

)"'(21* xxx

Half Integrality

Algorithm: Construct the following bipartite graph G’:

Find a vertex cover C’ in G’

(This can be done by using max-flow.)

EvuvuE

VuuV

VuuV

),(:)",'('

:""

:''

'"or '': CuCuuC

Primal Dual

Construct an integral feasible solution x and a feasible solution y such that cTx r bTy

Weak duality theorem bTy Opt(LP) cTx r Opt(LP) r Opt(IP) x is r-approx

Greedy H()-Approximation Algorithm While there exists an uncovered edge:

– – – Remove u and incident edges

C

}{uCC )(degminarg v

c

VvG

vu

Primal Dual Schema

Modified version of the primal dual method for solving LP

Construct integral primal solution and feasible dual solution simultaneously.

We impose the primal complementary slackness conditions, while relaxing the dual conditions

Relaxed Dual Conditions

Primal: Relaxed dual:

Claim: If x,y satisfy the above conditions

Proof:

j

m

iiijj cyaxj

1

or 0either ,

i

n

jjiji bxayi

1

or 0either ,

m

iii

n

jjj ybxc

11

m

iii

m

ii

n

jjij

n

jj

m

iiij

n

jjj ybyxaxyaxc

11 11 11

)()(

Vertex Cover

Algorithm: For every edge e=(u,v) do:

–

–

Output C(We can construct C at the end.)

eve

eveueeue ycycy

::

,min

0 ; yC

eve

eveueeu ycycCC

::

,minarg

Vertex Cover

There are no uncovered edges, or over packed vertices at the end. So x=x(C) and y are feasible.

Since x and y satisfy the relaxed complementary slackness conditions with =2, x is a 2-approx solution.– Primal:

– Relaxed Dual:

ueuee cyCu

:

2 0 ),,( vue xxyvue

Eee

Eevue

Ee euue

Vuu

euee

Vuuu yxxyxyxyxc 2

:

Generalized Vertex Cover

LP-relaxation and its dual: (P)

(D)

EVtx

Evuexxx

xcxc

t

evu

Eeee

Vuuu

0

),(1s.t.

min

Eecy

Vucy

y

ee

ueuee

Eee

0

s.t.

max

: