Approximation Algorithms for NP-hard Combinatorial Problems

Magnús M. HalldórssonReykjavik University

www.ru.is/~mmh/ewscs13/lec1.ppt

Post-doc & Ph.D. Positions available

• Part of research project titled

“Algorithms for Wireless Scheduling”

Why? Motivation

• Instead of looking for the best solution, we may want to find a solution that is good enough– Instead of an optimal solution, we seek approximations

Many computational problems are NP-hard

Most

Why study approximation algorithms?

• Study valuable, general-purpose problem solving techniques

• Interesting combinatorics, from a CS viewpoint

• Related to various other paradigms:– Online algorithms– Streaming algorithms– Distributed algorithms

What I expect that you already know

• Discrete structures and problems– Graphs and networks

• Basic knowledge of algorithms• Some experience with analysis of algorithms

– E.g. why Dijkstra’s algorithm works• Bonus:

– Mathematical programming– Basic probability theory

Plan today

• Get comfortable with our main problems• Overview the algorithmic strategies that we

will examine• Look at the most naive approximation

algorithms

Fundamental problems

• There are millions of optimization problems• We focus on a few fundamental problems:

– The techniques developed can be relatively easily transferred to other problems

• The classic graph problems, w/o connectivity

Problems we shall consider

• Independent set• Vertex cover• Chromatic number• Dominating set• Domatic number• Max Cut

• And sometimes weighted versions

Graphs and Notation

• n=9• m=13• =4• =2• d=26/9

Independent sets

• Find: An independent set S of vertices• Objective: Maximize |S|

Independent sets

• Find: An independent set of vertices• Maximize: Size

& corresponding interval graph

Intervals

& corresponding interval graph

Interval selection

& corresponding interval graph

Interval selection

= independent set in the power graph Gk

k=2

distance-k-independent set in graph G

(u,v) E(Gk) distG(u,v) k

16

Matching

• Input: Graph.• Goal. Find maximum cardinality matching.

C

1

5

2

A

E

3

B

D 4

Vertex Cover

• S is a vertex cover if every edge has at least one endpoint in S

• Objective: Minimize |S|

Vertex Cover

• S is a vertex cover if every edge has at least one endpoint in S

• Objective: Minimize |S|

Map coloring

Austria and its neighbors

?

Graph Coloring

Chromatic Number

(G) = 3

Where to locate icecream stands?• The Icecream Stands problem:• Every kid should have

access to an icecreamstand in the next street. How many stands are needed?

• A dominating set for a graph is a set of vertices whose neighbors, along with themselves, constitute all the vertices in the graph.

Domatic number

• Find: A partition (coloring) of the verticessuch that each color is a dominating set

• Objective: Maximize the number of colors

C

A

F

E

B

The The MAX CUTMAX CUT problem problem

Input. A graph.Feasible solution. A set S of vertices.Value of solution. Number of edges cut.Objective. Maximize.

Weighted versions

• In Weighted Independent Set:– Given a graph G with weights on vertices– Find an independent set with maximum total

weight

• Weights on vertices:– Ind.set, Vertex cover, Dominating set

• Weights on edges– Max Cut

Algorithmic strategies

• Greedy• Subgraph removal• Local search• Probabilistic method• Local ratio• Linear programming rounding• Semi-definite programming

Core issue: Bounding OPT

• We want to compare „our“ solution, ALG, to OPT, the optimal solution– I.e., show that ALG * OPT– But OPT is hard to get a handle on (NP-hard!)

• Instead, we compare ALG to an easier property that bounds OPT:– (I) OPT(I), and– ALG(I) (I)

Greedy

• Previously known as „myopic“• In each round, the algorithm makes the action

that is best according to some criteria

Subgraph removal

• An independent set can contain at most 1 vertex from a clique

• If we remove all big cliques, the problem may become easier.

• If OPT was large, it is still fairly large after removing the cliques

Local search

• Can I make small, „local“ changes to my solution, to get a better solution?– Ex. 1: „Shortcuts“– Ex. 2: If I throw out 1 vertex, can I replace it with

2?

Probabilistic method

• A „random“ solution is often pretty good– Sometimes close to best possible– Sometimes it needs minor changes

• Basic facts from probability:– Product rule– Union bound– Linearity of expectation

LP rounding• Linear programming is a powerful hammer

– The most general purpose solving method• All of our problems can be written as Integer

Programming problems– X_i = 1, if v_i is in the independent set,– X_i = 0, if v_i is not in the set

• LP: Allow any value in the range [0,1]– Solvable in polynomial time

• „Rounding“: Use LP-value to choose the binary 0/1 value

Semi-definite programming

• „Vector programming“ assigns each vertex a vector in n-dimensional space– LP value = 1-dimensional vector

• Use nearness or farness to decide on rounding

Performance ratio

• An algorithm A is -approximate if, for every input instance I, A(I) OPT(I) (maximization)or A(I) OPT(I) (minimization)

VC - Approximation Algorithm

• C • E’ E• while E’

– do let (u,v) be an arbitrary edge of E’– C C {u,v}– remove from E’ every edge incident to

either u or v.• return C.

Demo

Compare this cover to the one from the example

Polynomial Time

• C • E’ E• while E’ do

– let (u,v) be an arbitrary edge of E’– C C {u,v}– remove from E’ every edge incident to

either u or v• return C

O(n2

)O(1)

O(n)

O(n2)

Correctness

The set of vertices our algorithm returns is clearly a vertex-cover, since we iterate until every edge is covered.

How Good an Approximation is it?Observe the set of edges our algorithm chooses

any VC contains 1 in any VC contains 1 in eacheach

our VC contains both, hence at most twice as large

no common vertices!

no common vertices!

Another algorithm

• Given a graph G=(V,E), a subset of the edges M E is a matching if no two edges of M share an endpoint.

• A matching is maximal if no more edges can be added

• The previous solution is a maximal matching!

Algorithm 1.2 (Cardinality vertex cover) Find a maximal matching in G and output the set of

matched vertices

Linking ALG to OPT(relating the approximation to the optimal)

Approximatesolution

Optimalsolution?Maximal

matching

Maximal matching

Lower bounding OPT• OPT : an optimal solution (vertex cover) (also its size)• ALG : solution found by our algorithm• M : maximal matching

• We argued a lower bound for OPT in terms of M:OPT ≥ |M|

• Thus, maximal matching is the combinatorial structure that is relevant here.

• Performance analysis follows from this:– Since, ALG = 2 |M|, the performance ratio is at most 2

Can we improve this approximation?

• Can we improve the analysis for this algorithm?

• Can we make this algorithm more clever?– What if we carefully choose the maximal

matching?

• Can we find a better algorithm?

Tight example #1

But, this is only one small graph.

We want a family of graphs, of infinite size

OPT picks 1 vertexALG selected 2

Tight example #2

• Complete bipartite graph Kn,n (here n=4)

Any maximal matching contains all the 2n vertices So the algorithm solution always contains 2n verts.

A matching and the algorithm solution

Optimal solution contains only n vertices

An optimal solution

“This was a too simple solution”

• It is simple, but not stupid. Many seemingly smarter heuristics can give a far worse performance, in the worst case.

• It is easy to complicate heuristics by adding more special cases or details. For example, if you just picked the edge whose endpoints had highest degree, it doesn’t improve the worst case. It just makes it harder to analyze.

Naive approximations

• Bound ALG and OPT, independently• Forms the baseline, with which we compare

Naïve approximation of Dominating Set

• ALG n • OPT n/(+1)

Naïve approximation of Domatic Partition

• ALG 1 • OPT +1

Naïve approximation of Coloring

• ALG +1• OPT 2

Naïve approximation of Independent Set

• ALG n/(+1)• OPT n

Acknowledgment

• Much of the presentations are adapted from slides on the web

• In this talk, material on Vertex Cover was taken from:– www.cs.tau.ac.il/~safra/Complexity/appalgo.ppt

Problem #1: Min-degree greedy

• What is the most natural algorithm for the Independent Set problem?

• „Min-degree greedy“:– 1. Find a vertex of minimum degree;

2. delete its neighbors; 3. repeat (until the graph is empty)

• A) How good/bad is it on general graphs?– Construct a bad instance for that algorithm– As a function of n

• B) Does it do better than +1 ?.

Problem #2: Max Cut

• What (easy) approximations can we give for Max Cut?