Algorithms for Max-min Optimization

Anupam GuptaCarnegie Mellon University

based on works with Viswanath Nagarajan (IBM Watson) and R.Ravi (CMU),

Aaron Roth (MSR/Penn), Grant Schoenebeck (Princeton), and Kunal Talwar (MSR)

maximization problems

Given: universe U of elementscollection I of “independent” subsets “value” function f:2U non-negative values

When can we solve this?

Example #1:

max-weight independent set

f(S)maxs.t. S in I

Universe = vertices of graph“independent” set = no edge between two verticesf(S) = v in S wv

maximization problems

Given: universe U of elementscollection I of “independent” subsets “value” function f:2U non-negative values

When can we solve this?

Example #2:

max-weight spanning tree

f(S)maxs.t. S in I

Universe = edges of graph“independent” set = no cycles within the edgesf(S) = e in S we

A quick survey on linear and submodular maximizationI.e., when f(S) is either linear or submodular.

What is max-min optimization?I.e., cases where f(S) is defined by a minimization (covering) problem.

A couple of general theorems about these topics, the ideas behind some of the algorithms, and a couple of simple proofs.

general outline of this talk

f(S)maxs.t. S in I

case I: linear fn maximization

linear case: f(S) = e in S we

e.g., max-weight spanning tree

If I forms a matroid, we can solve this exactly!

f(S)maxs.t. S in I

In fact, the greedy algorithm suffices…

recall: a family of independent sets I forms a matroid if a) it is closed under taking subsets b) if A and B independent and |A| < |B|,

there is an element e in B such that A+e independent.

case I: linear fn maximization

linear case: f(S) = e in S we

e.g., max-weight bipartite matching

If I is the intersection of two matroids, also solvable

f(S)maxs.t. S in I

The greedy algorithm is not the best any more…

max-weight arborescence

… but it is a 2-approximation

case I: linear fn maximization

linear case: f(S) = e in S we

If I is the intersection of p > 2 matroids, NP hard in general

f(S)maxs.t. S in I

But the greedy algorithm is still a p-approximation!

can do slightly better via a local-search algorithm…

case II: submodular maximization

Recall f is submodular if

f(S)maxs.t. S in I

f(subset + e) – f(subset)

submodular function f

f(superset + e) – f(superset) ≥

f is monotone if f(subset) f(superset) ≤

two quick examples

Given a collection of sets A1, A2, …, Am

universe = {1, 2, …, m}f({1, 3, 7}) = | A1 [ A3 [ A7 |

Given an undirected graph G = (V,E)universe = Vf(S) = # of edges going from S to V\S in the graph.

The cut function in graphs is (non-monotone) submodular.

The “set-coverage” function is monotone submodular.

f(S)maxs.t. S in Imonotone submodular function f

If I is the intersection of p matroids

the greedy algorithm is a (p+1)-approximation!

case II: submodular maximization

S = { }While e such S + e is independent pick the element e* which maximizes f(S+e*) – f(S)

the greedy algorithm: a proof

Suppose we want to maximize monotone submodular f(S)such that I = { S ½ U | S contains at most k elements }

Theorem: f(Sfinal) ≥ ½ f(Sfinal [ OPT)

Proof: Suppose not.

Then f(Sfinal) < ½ f(Sfinal [ OPT)

f(Sfinal [ OPT) – f(Sfinal) > f(Sfinal)

e in OPT\S s.t. f(Sfinal + e) – f(Sfinal) >

at every step of greedy, f(S + e) – f(S) >

kf(Sfinal)

So greedy’s improvement must have been at least as much.

kf(Sfinal)

Contradiction

Cardinality constraint

n.b. use monotonicityonly in the last step.

≥ ½ f(OPT)

f(S)maxs.t. S in Inon-monotone submodular function f

If I is 2U (there are no constraints at all)already the problem is APX-hard!

However, the following algorithm is an 4-approximation:

case II: submodular maximization

S = a random subset of the universe U.

again, can do better (won’t talk about it today)…

f(S)maxs.t. S in Inon-monotone submodular function f

If I is the intersection of p matroidsthe greedy algorithm is not good!

However, the following algorithm is an O(p) approximation:

case II: submodular maximization

S1 ← Greedy(U)Find the subset T of S1 maximizing f(T)

S2 ← Greedy(U\S1)

Return the better of T and S2

the greedy algorithm: non-monotone proof

S1 ← Greedy(U)Find the subset T of S1 maximizing f(T)

S2 ← Greedy(U\S1)

Return the better of T and S2

f(S1) ≥ ½ f(S1 [ OPT)

f(S2) ≥ ½ f(S2 [ (OPT \ S1))

f(T) ≥ ¼ f(S1 \ OPT)

≥ f(OPT)

max{ f(T) , f(S2) } ≥ f(OPT)/12

for cardinality constraints

(from analysis of monotone case)

(from unconstrained max.)

(from monotone case again)

quick recap

f(S)maxs.t. S in I

If f is submodular

If I is the intersection of p matroids

then we can get an O(p) approximation.

what can we do for more general functions f( )?

for the rest of the talk, just focus on cardinality constraints I = { S ½ U, |S| ≤ k }

(some of our results will extend to more general constraints)

max-min covering problems

Fix a graph Guniverse = set of nodesfST(S) = min-cost Steiner tree on S

fMC(S) = min-cost cut separating S from root r

Fix a collection of sets A1, A2, …, Am

fSC(S) = size of min-cost set cover on S using these sets Ai

f(S)maxs.t. |S|≤ k

“Which k nodes is most expensive to connect up”?

“Which k elements are most expensive to cover”?

“Which k nodes are most expensive to disconnect from r”?

Of course, if f(S) is submodular (or close), we can use previous results…Can prove, e.g., fSC is very far from any submodular function

max-min covering: results (1)

f(S)maxs.t. |S|≤ k

For general covering problems…

Theorem:if you can solve some covering problem offlineand you can solve the problem online (deterministically)

you can solve the max-min version of the problem.

Usually offline × online approximation.

These algos extend to intersections of p matroids and q knapsacks…

max-min covering: results (2)

f(S)maxs.t. |S|≤ kFor the cardinality max-min:

O(log m + log n)-approximation for set coverO(1)-approximation for Steiner tree/forestO(log2 n)-approximation for multicut

all using the “same” algorithmic idea…

what’s the algorithm?

Recall: want to find S with |S| ≤ k that maximizes f(S)

Generic Idea:

call this value T*

For each value T do the following:

1. While some element costs “much more” than T/k to satisfy,add it to a set XT

Talg = smallest T such that cost(XT) not “much more” than T

For proof, need to show that a) for T ≈ T*, we have cost(XT) ≈ T*b) for T a bit smaller than T*, XT contains S of size k, such that cost(S) > T.

“cost(S)”

all k-sets cost < O(T*)

exists a k-set with cost ≥ (T*)

rest of the talk

Generic Algorithm:

Show how generic algorithm applies to:a) maxmin Steiner treeb) (time permitting) maxmin set cover

For each value T do the following:

1. While some element costs “much more” than T/k to satisfy,add it to a set XT

Talg = smallest T such that cost(XT) not “much more” than T

Steiner tree

Given a metric space (V, ℓ) and root rand a “scenario” S of terminals

find the least cost network connecting S to r

Results:MST is a 2-approximation.polytime 1.39-approx. algo [Byrka Grandoni Rothvoss Sanita ’10]

APX-hard [Bern Plassmann ‘89]

maxmin Steiner tree

Given a metric space (V, ℓ) and root r

find S approximately achieving

SteinerTree(S)maxs.t. |S|≤ k

the precise algorithm

Algorithm:For each value T do:1. add the root r to XT.

2. while exists a vertex v such that distance(v, XT) > 4T/kadd v to XT

find smallest T such that MST(XT) < 4T.

Fact: cost of connecting any set of k guys to MST(XT) ≤ 4T.

Corollary: cost of connecting any set of k guys ≤ 4T + MST(XT).

the analysis(1)

Fact: cost of connecting any k guys ≤ 4T + MST(XT).

Theorem 1: For T = T*, cost of MST(XT) < 4T.proof that

any k-subset can beconnected

at cost < 8T*

Lemma 2: For T = T*/8, cost of MST(XT) ≥ 4T.

if |XT| ≤ k, it is a set with cost ≥ 4T = T*/2 and small size

if |XT| is large, pick any k points from it.

their mutual distance > 4T/k, so Steiner-tree cost > 2T = T*/4.

Proof: if not, we could connect up any k points at cost < 8T = T*.

have found k-subsetwith cost > T*/4

remains to prove…

Theorem 1: For T = T*, cost of MST(XT) < 4T.

|XT| × 4T*/k × ½ ≤ ≤ |XT|/k × T*

≤ (|XT|/k + 1) × T*

|XT| ≤ k

optimal treeon XT

|XT| × T*/k ≤ T*≤ 2T*

MST is a 2-approximation, hence the 4T*

wrapping up maxmin Steiner

Algorithm:For each value T do:1. add the root r to XT.

2. while exists a vertex v such that distance(v, XT) > 4T/kadd v to XT

Talg = smallest T such that MST(XT) < 4T.

This value Talg shows all k-subsets can be connected at cost O(Talg)

the set XT for Talg/2 gives a k-subset of cost (Talg)

so, constant approximation.

rest of the talk

Generic Algorithm:

Show how generic algorithm applies to:a) maxmin Steiner treeb) (time permitting) maxmin set cover

For each value T do the following:

1. While some element costs “much more” than T/k to satisfy,add it to a set XT

Talg = smallest T such that cost(XT) not “much more” than T

to conclude

studied constrained maximization problemsgood algorithms for submodular cases

algorithms for max-min problems (f = minimization covering problem)O(log m + log n)-approximation for Set Coveralso for Steiner tree, Steiner forest, min-cut, multicut“same” simple algorithm“dual-rounding” analysis

for which other functions can we do constrained maximization well?papers are on the arxivthanks!!

f(S)maxs.t. S in I