Approximation Algorithms for Combinatorial Problems

Presented by Dajiang Zhu09/20/2011

David S. Johnson 1973

OUTLINE

Motivation Introduction & Conclusion Pre – Definition • Approximation Algorithms

Two problems as examples • SUBSET-SUM• MAXIMUM SATISFIABLITY

Motivation

Can we do a classification of polynomial complete (NP complete?) optimization problems?

Metric: Worst Case BehaviorWorst solution value / optimal value

Related publicationsThe complexity of theorem-proving procedures , S.

A. Cook, 1971Reducibility among combinatorial problems , R. M.

Karp, 1972Bounds on multiprocessing anomalies and related

packing algorithms , R. L. Graham, 1972

Introduction – Problem 1

SUBSET-SUM

Given a finite set of positive numbers and another positive number called the “goal”, find that subset whose sum is closest to, without exceeding, the goal.

Introduction – Problem 2

BIN-PACKING

Given a finite list of numbers between 0 and 1 and a sequence of unit-capacity bins, find a packing of the numbers into the bins such that no bin contains a total exceeding 1 and the number of nonempty bins is minimized.

Introduction – Problem 3

MAXIMUM SATISFIABILITY

Given a set S of disjunctive form clauses, all of whose disjuncts are either literals or their negations, find a truth assignment to the literals which satisfies(make true) the most clauses.

Introduction – Problem 4

SET CONVERING I

Given a finite cover of a finite set, find that sub cover which uses the fewest sets.

Introduction – Problem 5

SET CONVERING II

Given a finite cover of a finite set, find that sub cover which has the least overlapping.

Introduction – Problem 6

GRAPH COLORING

Given a graph, find a coloring of the nodes so that no two adjacent nodes have the same color, and the total number of colors used is minimized.

Introduction – Problem 7

MAXIMUM CLIQUE

Given a graph, find the maximum sub graph all of whose points are mutually adjacent.

Conclusion – Class I

Some algorithm that they guarantee the solution they generate will be no worse than a constant times the optimal and the constant can be quite close to 1.

• FIRST FIT DECREASING (11/9)• SUBSET-SUM• MAXIMUM SATISFIABLITY

Conclusion – Class II

Some algorithm that there is no fixed bound on the ratio of the worst case results to the optimal, but the ratio can grow no faster than the log of the problem size.

• SET COVERING

Conclusion – Class III

Some algorithm that the ratio grows at least as O() when n is the problem size and Ɛ>0 depends on the algorithm.

• GRAPH COLORING• MAXIMUM CLIQUE

Pre – Definition I

An optimization problem P consists of1. A set of possible inputs,2. A map which maps each u∈ to a finite

set approximate solutions,3. A measure : ( ) → defined for all possible

approximate solutions.

Pre – Definition II

The optimal measure is defined by = BEST{(x) : x∈ (u)},

The performance is defined by = WORST{(x) : x∈ (u) and x is choosable by A on input u},

Pre – Definition III As a measure of the relative worst case

behavior of algorithm A on input u, we use the ratio:

(A,u)= Note: (A , u) ≥ 1 Overall worst case behavior of algorithm A with

respect to problem P will be a function: R[A, P](n) = MAX{(A,u):u∈ and |u|≤n}.

Example I – SUBSET SUM SUBSET – SUM (SS)={<T,s,b>: T is a finite set, s: T → is a map which assign to each x∈ T a “size” s(x), and b>0 is a single rational

number}. (<T,s,b>)={⊆T: ≤b}. ()= ,

The optimal measure for input <T,s,b> is :=MAX{m(): ⊆T and m() ≤b}.

Example I – SUBSET SUM a series of approximation algorithms

{: k≥1} and is defined as:

1. Let SUB be that subset of { x ∈ T: s(x)>b/(k+1)} whose measure is closest to without exceeding b. Let SUM be this measure, and set LEFT=T-SUB.

2. If for all x ∈ LEFT, s(x) + SUB > b, halt and return SUB.3. Let y be an element of LEFT for which s(y) + SUB is closest to,

without exceeding, b.4. Set LEFT = LEFT - {y}. SUB = SUB ∪ {y}, SUM = SUM + s(y).5. Go to 2.

Example I – SUBSET SUM Some observation

Algorithm A1 is closely related to the FIRST FIT DECREASING algorithm for BIN-PACKING.

The majority of the effort is concentrated in the first step. – O()

Example I – SUBSET SUM Claim and Proof

Claim: for k ≥ 1 and n > 0,R[](n) ≤ (k+1)/k, = (k+1)/k

Example I – SUBSET SUMProof - R[](n) ≤ (k+1)/k• ⊆ T where is choosable by algorithm . be an optimal solution for the given input.• Partition for i ∈ {0,1} as followed:

= U , where = {x∈ : s(x)>b/(k+1)}, and = - .so, for i ∈ {0,1} we have m() = m() + m()• By step 1 of , we have m() ≥ m().• If ⊒ , we have m() ≥ m(), so = .• If some x ∈ were excluded from , then by step2 of , we have s(x) + m() >b , so m()>b-s(x)≥b-b/(k+1)=kb/(k+1)≥[k/(k+1)]

Example I – SUBSET SUMProof - = (k+1)/kWe consider the input <T, s, b> where T = {, ,…, }, S() = b=k+1.Clearly, = k+1 and (<T, s, b>) = k+ .ƐThus r(,<T, s, b>) = (k+1)/(k+ ).Ɛ

Note: the above result shows that the SUBSET-SUM optimization problem has the desirable property: for any >0, there is a Ɛpolynomial-time algorithm s.t. R[] ≤ 1 + Ɛ

Example II – MAXIMUM SATISFIABILITY

MAXIMUM SATISFIABILITY(MS)

={S: S is a finite set {, ,…, } of clauses}, (T)={⊆S: there exists a truth assignment T which satisfies every clause C∈}, ()=||.

Example II – MAXIMUM SATISFIABILITY

algorithm B1:

1. Set SUB = ∅, TRUE = ∅, LEFT = S, LIT = L.2. If no literal in LIT is contained in any clause of LEFT, halt and

return SUB.3. Let y be the literal in LIT which is contained in the most clauses

of LEFT, and YT be the set of clauses in LEFT which contain y.4. Set SUB = SUB U YT, LEFT = LEFT – YT, TRUE = TRUE U {y},

and LIT = LIT - {y,} ,5. Go to 2.

Example II – MAXIMUM SATISFIABILITY

Claim and ProofClaim: for all k ≥ 1 and n > 0,

R[B1, MS(k)](n) ≤ (k+1)/k, with equality for all sufficiently large n.

Example II – MAXIMUM SATISFIABILITY

Proof - R[B1, MS(k)](n) ≤ (k+1)/k• Each time a literal is added to TRUE, the number of clauses

added to SUB at least as large as the number of clauses remaining in LEFT which are wounded (have one of their literals removed from LIT without being added to TRUE).

• When the algorithm halts, the only clauses left in LEFT are those which have been wounded as many times as they contain literals. That means when the algorithm halts there are at least k*|LEFT| wounds, so |SUB| ≥ k*|LEFT|.

Thus S* ≤ |S| = |SUB|+|LEFT| ≤[(k+1)/k] |SUB|

Example II – MAXIMUM SATISFIABILITY

algorithm B2:

1. Assign to each clause c ∈ S a weight w(c) = . Set SUB = TRUE = ∅, LIT = L, and LEFT = S.

2. If no literal in any clause of LEFT is in LIT, halt and return SUB.3. Let y be any literal occurring in both LIT and a clause of LEFT.

Let YT be the set of clauses in LEFT and containing y, YF the set of clauses in LEFT containing .

4. If ≥ , set TRUE = TRUE U {y}, SUB = SUB U YT, LEFT = LEFT – YT, and for each , set w(c) = 2w(c). Otherwise, set TRUE = TRUE U {}, SUB = SUB U YF, LEFT = LEFT – YF, and for each , set w(c) = 2w(c).

5. Set LIT = LIT – {y, }, and go to 2.

Example II – MAXIMUM SATISFIABILITY

Claim and ProofClaim: for all k ≥ 1 and n > 0,

R[B2, MS(k)](n) ≤ /(-1), with equality for all sufficiently large n.

Example II – MAXIMUM SATISFIABILITY

Proof - R[B2, MS(k)](n) ≤ /(-1)• At begging, the total weight of all the clauses in LEFT

cannot exceed |S|/.• During each iteration, the weight of clauses removed from

LEFT is at least as large as the weight added to those remaining clauses which receive new wounds. Thus the total weight can never increase even the algorithm halts.

• When algorithm halts, each clauses in LEFT have been wounded as many times as it had literals, hence must have has its weight doubled that many times, and so must have a final weight of 1.

• Therefore |LEFT| ≤ |S|/. So |SUB| ≥ |S| (1-1/).

Questions and discussion

Questions?

Thanks!