AdvancedAlgorithms(I) - Chihao Zhangchihaozhang.com/teaching/AA2019spring/slides/lec1-slides.pdf ·...

Advanced Algorithms (I)

Chihao Zhang

Shanghai Jiao Tong University

Feb. 25, 2019

Advanced Algorithms (I) 1/14

Advanced Algorithms

In the course, we will learn Approximation Algorithms

▶ linear programming, semi-definite programming▶ spectral method▶ random walks▶ …

We will emphasize on▶ tools for designing approximation algorithms▶ rigorous analysis of algorithms

Advanced Algorithms

Course Info

▶ Instructor: Chihao Zhang▶ Course Homepage:

http://chihaozhang.com/teaching/AA2019spring/▶ Office Hour: every Monday, 7:00pm - 9:00pm

Grading Policy▶ Homework 30%▶ Mid-term Exam 30%▶ Course Project 40%

Course Info

MaxSAT

Given a CNF formula φ, is it satisfiable?

φ = (x1 ∨ x3 ∨ x̄29) ∧ (x̄3 ∨ x7) ∧ · · · ∧ (x̄33 ∨ x̄34 ∨ x90 ∨ x126)

NP-hard, we look at its optimization version.

MaxSATInput: A CNF formula φ = C1 ∧ C2 · · · ∧ Cm.

Problem: Compute an assignment that satisfies maximumnumber of clauses.

Harder than SAT, so we look for an approximate solution.

MaxSAT

φ = (x1 ∨ x3 ∨ x̄29) ∧ (x̄3 ∨ x7) ∧ · · · ∧ (x̄33 ∨ x̄34 ∨ x90 ∨ x126)

MaxSAT

φ = (x1 ∨ x3 ∨ x̄29) ∧ (x̄3 ∨ x7) ∧ · · · ∧ (x̄33 ∨ x̄34 ∨ x90 ∨ x126)

MaxSAT

φ = (x1 ∨ x3 ∨ x̄29) ∧ (x̄3 ∨ x7) ∧ · · · ∧ (x̄33 ∨ x̄34 ∨ x90 ∨ x126)

MaxSAT

φ = (x1 ∨ x3 ∨ x̄29) ∧ (x̄3 ∨ x7) ∧ · · · ∧ (x̄33 ∨ x̄34 ∨ x90 ∨ x126)

MaxSAT

φ = (x1 ∨ x3 ∨ x̄29) ∧ (x̄3 ∨ x7) ∧ · · · ∧ (x̄33 ∨ x̄34 ∨ x90 ∨ x126)

Tossing a Coin

An instance φ▶ The variable sets V = {x1, x2, . . . , xn}▶ The set of clauses C = {C1,C2, . . . ,Cm}▶ Each clause Ci contains ℓi literals

We first cosnider the following simple algorithm:

▶ For each variable xi, toss an independent fair coin.▶ If the coin goes HEAD, we set xi true, otherwise we

set xi false.

Tossing a Coin

set xi false.

Tossing a Coin

set xi false.

Tossing a Coin

▶ For each variable xi, toss an independent fair coin.

▶ If the coin goes HEAD, we set xi true, otherwise weset xi false.

Tossing a Coin

set xi false.

Analysis

The outcome of the algorithm is random, we are interested in itsexpectation.

For this particular algorithm, it can be derandomized.

E [X] =m∑i=1

Pr [Ci is satisfied] =m∑i=1

(1 − 2−ℓi

)≥ m

On the otherhand,OPT ≤ m.

Therefore,

E [X] ≥ 1

2· OPT.

Analysis

E [X] =m∑i=1

(1 − 2−ℓi

)≥ m

Therefore,

E [X] ≥ 1

2· OPT.

Analysis

E [X] =m∑i=1

(1 − 2−ℓi

)≥ m

Therefore,

E [X] ≥ 1

2· OPT.

Analysis

E [X] =m∑i=1

Pr [Ci is satisfied]

=m∑i=1

(1 − 2−ℓi

)≥ m

Therefore,

E [X] ≥ 1

2· OPT.

Analysis

E [X] =m∑i=1

(1 − 2−ℓi

)≥ m

Therefore,

E [X] ≥ 1

2· OPT.

Analysis

E [X] =m∑i=1

(1 − 2−ℓi

)≥ m

Therefore,

E [X] ≥ 1

2· OPT.

Analysis

E [X] =m∑i=1

(1 − 2−ℓi

)≥ m

Therefore,

E [X] ≥ 1

2· OPT.

Can we improve the previous algorithm?

Observations▶ the worst case happens when for some singleton clause, i.e.,ℓi = 1;

▶ for a singleton C = x, if there is no C′ = x̄, then we canincrease the probability of x to be true;

▶ otherwise, we can improve the upper bound for OPT! (x and x̄cannot be both satisfied)

Tossing a Biased Coin

Let the p-biased coin be with probability p to HEAD, 1 − p to TAIL.

▶ For each variable xi, toss an independent p-biasedcoin.

Analysis

Assumption for Clauses of φ▶ More positive singletons than negative singletons.▶ There are t pairs of x and x̄.▶ p ≥ 1

E [X] ≥ t+ (m − 2t)min{p, 1 − p2

The new upper bound for OPT:

OPT ≤ m − t

Combine them and obtain

E [X] ≥ α · OPT

where α ≈ 0.618.

AnalysisAssumption for Clauses of φ▶ More positive singletons than negative singletons.

▶ There are t pairs of x and x̄.▶ p ≥ 1

E [X] ≥ t+ (m − 2t)min{p, 1 − p2

OPT ≤ m − t

E [X] ≥ α · OPT

where α ≈ 0.618.

AnalysisAssumption for Clauses of φ▶ More positive singletons than negative singletons.▶ There are t pairs of x and x̄.

▶ p ≥ 12 .

E [X] ≥ t+ (m − 2t)min{p, 1 − p2

OPT ≤ m − t

E [X] ≥ α · OPT

where α ≈ 0.618.

AnalysisAssumption for Clauses of φ▶ More positive singletons than negative singletons.▶ There are t pairs of x and x̄.▶ p ≥ 1

E [X] ≥ t+ (m − 2t)min{p, 1 − p2

OPT ≤ m − t

E [X] ≥ α · OPT

where α ≈ 0.618.

E [X] ≥ t+ (m − 2t)min{p, 1 − p2

OPT ≤ m − t

E [X] ≥ α · OPT

where α ≈ 0.618.

E [X] ≥ t+ (m − 2t)min{p, 1 − p2

OPT ≤ m − t

E [X] ≥ α · OPT

where α ≈ 0.618.Advanced Algorithms (I) 9/14

Can we further improve the algorithm?

We can use different coins for each variable, e.g., in

φ = x1 ∧ (x1 ∨ x̄2),

we prefer to set x1 to true.

How can we make use of the information?

Linear Programming helps.

φ = x1 ∧ (x1 ∨ x̄2),

Tossing Clever Coins

We introduce the following variables.

▶ for every i ∈ [n], yi indicates whether xi is true;▶ for every j ∈ [m], zj indicates whether Cj is satisfied.

maxm∑j=1

subject to∑i∈Pj

yi +∑k∈Nj

(1 − yk) ≥ zj, ∀j ∈ [m] s.t. Cj =∨i∈Pj

xi ∨∨k∈Nj

zj ∈ {0, 1}, ∀j ∈ [m]

yi ∈ {0, 1}, ∀i ∈ [n]

This integer program is equivalent to MaxSAT.

Tossing Clever CoinsWe introduce the following variables.

maxm∑j=1

subject to∑i∈Pj

yi +∑k∈Nj

(1 − yk) ≥ zj, ∀j ∈ [m] s.t. Cj =∨i∈Pj

xi ∨∨k∈Nj

zj ∈ {0, 1}, ∀j ∈ [m]

yi ∈ {0, 1}, ∀i ∈ [n]

maxm∑j=1

subject to∑i∈Pj

yi +∑k∈Nj

(1 − yk) ≥ zj, ∀j ∈ [m] s.t. Cj =∨i∈Pj

xi ∨∨k∈Nj

zj ∈ {0, 1}, ∀j ∈ [m]

yi ∈ {0, 1}, ∀i ∈ [n]

maxm∑j=1

subject to∑i∈Pj

yi +∑k∈Nj

(1 − yk) ≥ zj, ∀j ∈ [m] s.t. Cj =∨i∈Pj

xi ∨∨k∈Nj

zj ∈ {0, 1}, ∀j ∈ [m]

yi ∈ {0, 1}, ∀i ∈ [n]

maxm∑j=1

subject to∑i∈Pj

yi +∑k∈Nj

(1 − yk) ≥ zj, ∀j ∈ [m] s.t. Cj =∨i∈Pj

xi ∨∨k∈Nj

zj ∈ {0, 1}, ∀j ∈ [m]

yi ∈ {0, 1}, ∀i ∈ [n]

Relaxation

There is no efficient algorithm for integer programming in general

Therefore, we relax its non-linear constriants

maxm∑j=1

subject to∑i∈Pj

yi +∑k∈Pj

(1 − yk) ≥ zj, ∀j ∈ [m] s.t. Cj =∨i∈Pj

xi ∨∨k∈Nj

0 ≤ zj ≤ 1, ∀j ∈ [m]

0 ≤ yi ≤ 1, ∀i ∈ [n]

We can solve this LP in poly-time

Relaxation

maxm∑j=1

subject to∑i∈Pj

yi +∑k∈Pj

(1 − yk) ≥ zj, ∀j ∈ [m] s.t. Cj =∨i∈Pj

xi ∨∨k∈Nj

0 ≤ zj ≤ 1, ∀j ∈ [m]

0 ≤ yi ≤ 1, ∀i ∈ [n]

Relaxation

maxm∑j=1

subject to∑i∈Pj

yi +∑k∈Pj

(1 − yk) ≥ zj, ∀j ∈ [m] s.t. Cj =∨i∈Pj

xi ∨∨k∈Nj

0 ≤ zj ≤ 1, ∀j ∈ [m]

0 ≤ yi ≤ 1, ∀i ∈ [n]

Relaxation

maxm∑j=1

subject to∑i∈Pj

yi +∑k∈Pj

(1 − yk) ≥ zj, ∀j ∈ [m] s.t. Cj =∨i∈Pj

xi ∨∨k∈Nj

0 ≤ zj ≤ 1, ∀j ∈ [m]

0 ≤ yi ≤ 1, ∀i ∈ [n]

Relaxation

maxm∑j=1

subject to∑i∈Pj

yi +∑k∈Pj

(1 − yk) ≥ zj, ∀j ∈ [m] s.t. Cj =∨i∈Pj

xi ∨∨k∈Nj

0 ≤ zj ≤ 1, ∀j ∈ [m]

0 ≤ yi ≤ 1, ∀i ∈ [n]

Algorithm

Let({y∗i}i∈[n] ,

{z∗j

}j∈[m]

)be an optimal solution of the LP.

▶ For each variable xi, toss an independent y∗i -biasedcoin.

Algorithm

{z∗j

}j∈[m]

Algorithm

{z∗j

}j∈[m]

Algorithm

{z∗j

}j∈[m]

Algorithm

{z∗j

}j∈[m]

Analysis

A typical upper bound of OPT for LP based algorithms is

OPT ≤ OPT(LP) =m∑j=1

We can further establish

E [X] ≥(1 − 1

) m∑j=1

z∗j .

Therefore, the LP rounding is a(1 − 1

)-approximation algorithm for

MaxSAT

Analysis

E [X] ≥(1 − 1

) m∑j=1

z∗j .

MaxSAT

Analysis

E [X] ≥(1 − 1

) m∑j=1

z∗j .

MaxSAT

Analysis

E [X] ≥(1 − 1

) m∑j=1

z∗j .

MaxSAT

AdvancedAlgorithms(I) - Chihao Zhangchihaozhang.com/teaching/AA2019spring/slides/lec1-slides.pdf ·...

Documents

LinearProgramming II

Che1031 lec1

Lec1 Introduction.unlocked

Lec1 Introduction

Vishy Lec1

Research Lec1

Unit 3. LinearProgramming. The simplex method › pluginfile.php › 11553 › mod_resource › cont… · Unit 3. LinearProgramming. The simplex method Operations Research is a branch

It101 lec1

Lec1 Intro

Techrisk Lec1

Bio108 lec1

lec1 FOJJFDLJ

ec247 lec1

Biotech Lec1

20160314 LinearProgramming(Upload)

Lec1 monsoon

TutorialonCPLEX LinearProgramming - cs.upc.eduerodri/webpage/cps/lab/lp/tutorial-cplex-slides... · TutorialonCPLEX LinearProgramming CombinatorialProblemSolving(CPS) Enric Rodr´ıguez-Carbonell

Week11 lec1

SUPREME COURT’S DECISIONS By: Chihao Tran APUSH- Per. 3

Lec1-10min - Purdue University College of Engineeringlinx/ECE201/notes_2012/lec1.pdf · lec1 Page 15 . lec1 Page 16 . lec1 Page 17 . Summary Tuesday, January 03, 2012 4:20 PM lec1