Approximation Algorithms:

Approximation Algorithms:

Éva TardosCornell University

problems, techniques, and their use in game theory

FOCS 2002 2

What is approximation?Find solution for an optimization

problem guaranteed to have value close to the best possible.

How close?• additive error: (rare)

– E.g., 3-coloring planar graphs is NP-complete, but 4-coloring always possible

• multiplicative error: -approximation: finds solution for

an optimization problem within an factor to the best possible.

FOCS 2002 3

Why approximate?

• NP-hard to find the true optimum

• Just too slow to do it exactly

• Decisions made on-line

• Decisions made by selfish players

FOCS 2002 4

Outline of talk

Techniques: • Greedy• Local search• LP techniques:• rounding• Primal-dual

Problems:

• Disjoint paths• Multi-way cut and labeling•network design, facility location

Relation to Games– local search price of anarchy– primal dual cost sharing

FOCS 2002 5

Max disjoint paths problem

Given graph G, n nodes, m edges, and source-sink pairs.

Connect as many as possible via edge-disjoint path.

t

s t

s s

t

t

s

FOCS 2002 6

Greedy AlgorithmGreedily connect s-t pairs

via disjoint paths, if there is a free path using at most m½ edges:

m½ 4

s t

s s

t

t

s t

If there is no short path at all, take a single long one.

FOCS 2002 7

Greedy AlgorithmTheorem: m½ –approximation.

Kleinberg’96Proof: One path used can block

m½ better paths m½ 4

s t

s s

t

t

s t

Essentially best possible: m½- lower bound unless P=NP by [Guruswami, Khanna, Rajaraman, Shepherd, Yannakakis’99]

FOCS 2002 8

Disjoint paths:open problem

Connect as many as pairs possible via paths where 2 paths may share any edge

t

s t

s s

t

t

s

• Same practical motivation• Best greedy algorithm: n½ - (and also m1/3 -) approximation: Awerbuch, Azar, Plotkin’93.

• No lower bound …

FOCS 2002 9

Outline of talk


Problems:


Relation to Games– local search Price of anarchy– primal dual Cost sharing

FOCS 2002 10

Multi-way Cut ProblemGiven:

– a graph G = (V,E) ;– k terminals {s1, …, sk}– cost we for each edge e

Goal: Find a partition that separates terminals, and minimizes the cost

{e separated} weSeparated edgess1

s2

s3

s4

FOCS 2002 11

Greedy Algorithm

For each terminal in turn– Find min cut separating si

from other terminals The first cut

The next cut

s2

s1

s4

s3

s2

s1

s4

s3

FOCS 2002 12

Theorem: Greedy is a2-approximation

Proof: Each cut costs at most the optimum’s cut [Dahlhaus, Johnson, Papadimitriou, Seymour, and Yannakakis’94]

Cuts found by algorithm:

Optimum partition

Selected cuts, cheaper than optimum’s cut, but

each edge in optimum is counted twice.

s4

s3

s2

s1

FOCS 2002 13

Multi-way cuts extension

Given: – graph G = (V,E), we0 for e E– Labels L={1,…,k} – Lv L for each node v

Objective: Find a labeling of nodes such that each node v assigned to a label in Lv and it minimizes cost {e separated} we

Separated edges

part 1

part 2

part 3

part 4

FOCS 2002 14

Example

Does greedy work? For each terminal in turn

– Find min cut separating si from other terminals

Blue or greenRed

or g

reen

Red or blue

cheapmediumexpensive

s3

s1

s2

FOCS 2002 15

Greedy doesn’t workGreedyFor each terminal in turn

– Find min cut separating si from other terminals

The first two cuts:

Remaining part not valid!

Blue or greenRed

or g

reen

Red or blues2

s1

s3

FOCS 2002 16

Local search

[Boykov Veksler Zabih CVPR’98] 2-approximation

1. Start with any valid labeling.

2. Repeat (until we are tired):a. Choose a color c.b. Find the optimal move where a

subset of the vertices can be recolored, but only with the color c.

(We will call this a c-move.)

FOCS 2002 17

A possible -move

Thm [Boykov, Vekler, Zabih] The best -move can be found via an (s,t) min-cut

FOCS 2002 18

Idea of the flow networkfor finding a -move

s = all other terminals: retain current color

sc = change color to c =

G

FOCS 2002 19

Theorem: local optimum is a 2-approximation

Partition found by algorithm:

Cuts used by optimum

The parts in optimum each give a possible local move:

FOCS 2002 20

Theorem: local optimum is a 2-approximation

Partition found by algorithm:

Possible move using the optimum

Changing partition does not help current cut cheaperSum over all colors:

Each edge in optimum counted twice

FOCS 2002 21

Metric labeling classification

open problemGiven:

– graph G = (V,E); we0 for e E– k labels L– subsets of allowed labels Lv – a metric d(.,.) on the labels.

Objective: Find labeling f(v)Lv for each node v to minimizee=(v,w) we d(f(v),f(w))

Best approximation known: O(ln k ln ln k) Kleinberg-T’99

FOCS 2002 22

Outline of talk


Problems:



FOCS 2002 23

Using Linear Programs for multi-way cuts

Using a linear program = fractional cut probabilistic assignment of

nodes to parts

?

Idea: Find “optimal” fractional labeling via linear programming

Label ? as : ½ + ½

FOCS 2002 24

Fractional Labeling

Variables:0 xva 1 p=node, a=label in Lv

– xva fraction of label a used on node v

Constraints:

xva = 1aLv

for all nodes v V

– each node is assigned to a label

cost as a linear function of x: we ½ |xua - xva |

e=(u,v) aL

FOCS 2002 25

From Fractional x to multi-way cut

The Algorithm (Calinescu, Karloff, Rabani, ’98, Kleinberg-T,’99)

While there are unassigned nodes• select a label a at random

xva

1

u v Unassigned nodes

FOCS 2002 26

The Algorithm (Cont.)

While there are unassigned nodes– select a label a at random

xva

1

u vUnassigned nodes

select 0 1 at randomassign all unassigned nodes v to selected label a if xva

FOCS 2002 27

Why Is This Choice Good?

select 0 1 at randomassign all unassigned nodes v to

selected label a if xva

Note:• Probability of assigning node v to

label a is xva • Probability of separating nodes u

and v in this iteration is |xua – xva |

xpa

1

p qUnassigned nodes

FOCS 2002 28

From Fractional x to Multi-way cut (Cont.)

Theorem: Given a fractional x, we find multi-way cut with expected

separation cost 2 (LP cost of x)

Corollary: if x is LP optimum . 2-approximation

Calinescu, Karloff, Rabani, ’98 1.5 approximation for multi-way cut

(does not work for labeling)

Karger, Klein, Stein, Thorup, Young’99 improved bound 1.3438..

FOCS 2002 29

Outline of talk


Problems:



FOCS 2002 30

Metric Facility Location

F is a set of facilities (servers).D is a set of clients.

cij is the distance between any i and j in D F.

Facility i in F has cost fi.

clientfacility5

4

23

FOCS 2002 31

Problem Statement

We need to:1) Pick a set S of facilities to open. 2) Assign every client to an open

facility (a facility in S).

Goal: Minimize cost of S + p dist(p,S).

clientfacility5

4

23

openedfacility

FOCS 2002 32

What is known?

All techniques can be used:• Clever greedy [Jain, Mahdian,

Saberi ’02]

• Local search [starting with Korupolu, Plaxton, and Rajaraman ’98], can handle capacities

• LP and rounding: [starting with Shmoys, T, Aardal ’97]

Here: primal-dual [starting with Jain-Vazirani’99]

FOCS 2002 33

What is the primal-dual method?

• Uses economic intuition from cost sharing– For each requirement, like

aLv xva = 1, someone has to pay to make it true…

• Uses ideas from linear programming:– dual LP and weak duality– But does not solve linear

programs

FOCS 2002 34

Dual Problem: Collect Fees

Client p has a fee αp (cost-share)

Goal: collect as much as possible max p αp

Fairness: Do no overcharge: for any subset A of clients and any possible facility i we must have p A [αp – dist(p,i)] fi

amount client p would contribute to building facility i.

FOCS 2002 35

Exact cost-sharing

• All clients connected to a facility

• Cost share αp covers connection costs for each client p

• Costs are “fair”• Cost fi of selecting a facility i is

covered by clients using it

p αp = f(S)+ p dist(p,S) , and

both facilities are fees are optimal

FOCS 2002 36

Approximate cost-sharing

Idea 1: each client starts unconnected, and with fee αp=0

Then it starts raising what it is willing to pay to get connected

• Raise all shares evenly αExample:

= client= possible facility with its cost

4 4

4

FOCS 2002 37

Primal-Dual Algorithm (1)

• Each client raises his fee α evenly what it is willing to pay

α = 1

Its α =1 share could be used towards building a connection to either facility

4 4

FOCS 2002 38


• Each client raises evenly what it is willing to pay

Starts contributing towards facility cost

α = 2

4 4

FOCS 2002 39


• Each client raises evenly what it is willing to pay

Three clients contributing

α = 3

4 4

FOCS 2002 40


Open facility, when cost is covered by contributions

4

clients connected to open facility

Open facility

α = 3

4

FOCS 2002 41

Primal-Dual Algorithm: Trouble

Trouble: – one client p connected to

facility i, but contributes to also to facility j

4

Open facility

α = 3

4i j

p

FOCS 2002 42


Close facility j: will not open this facility.

Will this cause trouble?• Client p is close to both i and j

facilities i and j are at most 2α from each other.

4

Open facility

α = 3

4i j

p

ghost

FOCS 2002 43

4


Not yet connected clients raise their fee evenly

Until all clients get connected

4

no not need to pay more than 3

Open facility

α =6 α =3

α =3 α =3

ghost

FOCS 2002 44

Feasibility + fairness ?? All clients connected to

a facility Cost share αp covers

connection costs of client p

Cost fi of opening a facility i is covered by clients connected to it

• ?? Are costs “fair” ??

FOCS 2002 45

a set of clients A, and any possible facility i we have

p A [αp – dist(p,i)] fi

– Why? we open facility i if there is enough contribution, and do not raise fees any further

But closed facilities are ignored! and may violate fairness

Are costs “fair”??

44

open facility

closed facility, ignored

FOCS 2002 46

Fair till it reaches a “ghost” facility.

Let α’q αq be the fee till a ghost facility is reached

Are costs “fair”??

44

open facility

Closed facility, ignored

cause of closing

ji

p

α’q=4

FOCS 2002 47

Feasibility + fairness ?? All clients connected to a

facility Cost share αp covers

connection costs for client p Cost αp also covers cost of

selected a facilities Costs α’p are “fair”How much smaller is α’ α ??

44

p

FOCS 2002 48

How much smaller is α’ α?

q client met ghost facility j j became a ghost due to client p

qi

p stopped raising its share first αp α’q αq

Recall dist(i,j) 2 αp, soαq α’q +2 αp 3α’q

44

p

j

FOCS 2002 49

Primal-dual approximation

The algorithm is a 3-approximation algorithm for the facility location problem

[Jain-Vazirani’99, Mettu-Plaxton’00]Proof: Fairness of the α’p fees

p α’p min cost [max min]

cost-recovery:f(S) + p dist(p,S) = p αp

α 3α’q

3-approximation algorithm

FOCS 2002 50

Outline of talk


Problems:


Relation to Games– primal dual Cost sharing– local search Price of anarchy

FOCS 2002 51

primal dual Cost sharing

Dual variables αp are natural cost-shares:

Recall: fair = no set is overcharged

= core allocationp Aαp – dist(p,i) fi for all A and i.

[Chardaire’98; Goemans-Skutella’00] strong connection between core cost-allocation and linear programming dual solutions

See also Shapley’67, Bondareva’63 for other games

FOCS 2002 52

Primal-Dual Cost-sharing

Primal dual = for each requirement someone willing to pay to make it true

Cost-sharing: only players can have shares.

• Not all requirements are naturally associated with individual players.

• Real players need to share the cost.

FOCS 2002 53

primal dual Cost sharing

Fair no subset is overcharged

Stronger desirable property: population monotone (cross-monotone):

Extra clients do not increase cost-shares.

• Spanning-tree game: [Kent and Skorin-Kapov’96 and Jain Vazirani’01]

• Facility location, single source rent-or-buy [Pal-T’02]

FOCS 2002 54

Local search (for facility location)

Local search: simple search steps to improve objective:

• add(s) adds new facility s• delete(t) closes open facility t• swap(s,t) replaces open facility

s by a new facility t

Key to approximation bound:How bad can be a local optima?3-approximation [Charikar, Guha’00]

FOCS 2002 55

Local search Price of anarchy in games

Price of anarchy: facilities are operated by separate selfish agents

Agents open/close facilities when it benefits their own objective.

Agent’s “best response” dynamic:• Simple local steps analogous to

local search.Price of anarchy: • How bad can be a stable state?• 2-approximation in a related

maximization game: [Vetta’02]

FOCS 2002 56

Conclusions for approximation

Greedy, Local search• clever greedy/local steps can

lead to great resultsPrimal-dual algorithms• Elegant combinatorial methods• Based on linear programming

ideas, but fast, avoids explicitly solving large linear programs

Linear programming• very powerful tool, but slow to

solveInteresting connections to

issues in game theory

Documents

Approximation Algorithms: