Upload
others
View
13
Download
0
Embed Size (px)
Citation preview
Algorithm Design and ImplementationPrinciple of Algorithms XII
NP Problem I
Guoqiang Li
May 14, 2020
School of Software, Shanghai Jiao Tong University
1
Poly-Time Reductions
1
Algorithm design patterns and antipatterns
Algorithm design patterns.
• Greedy.
• Divide and conquer.
• Dynamic programming.
• Duality.
• Reductions.
• Local search.
• Randomization.
Algorithm design antipatterns.
• NP-completeness. O(nk) algorithm unlikely.
• PSPACE-completeness O(nk) certification algorithm unlikely.
• Undecidability No algorithm possible.
2
Classify problems according to computational requirements
Q. Which problems will we be able to solve in practice?
A working definition. Those with poly-time algorithms.
Theory. Definition is broad and robust.
• Turing machine, word RAM, uniform circuits, . . .
Practice. Poly-time algorithms scale to huge problems.
3
Classify problems according to computational requirements
Q. Which problems will we be able to solve in practice?
A working definition. Those with poly-time algorithms.
yes probably no
shortest path longest path
min cut max cut
2-satisfiability 3-satisfiability
planar 4-colorability planar 3-colorability
bipartite vertex cover vertex cover
2D-matching 3D-matching
primality testing factoring
linear programming integer linear programming
4
Classify problems
Requirement. Classify problems according to those that can be solved in
polynomial time and those that cannot.
Provably requires exponential time.
• Given a constant-size program, does it halt in at most k steps?
• Given a board position in an n-by-n generalization of checkers,
can black guarantee a win?
Frustrating news. Huge number of fundamental problems have defied
classification for decades.5
Poly-time reductions
Suppose we could solve problem Y in polynomial time. What else could we
solve in polynomial time?
Reduction. Problem X polynomial-time (Cook) reduces to problem Y if
arbitrary instances of problem X can be solved using:
• Polynomial number of standard computational steps, plus
• Polynomial number of calls to oracle that solves problem Y .
6
Poly-time reductions
Suppose we could solve problem Y in polynomial time. What else could we
solve in polynomial time?
Reduction. Problem X polynomial-time (Cook) reduces to problem Y if
arbitrary instances of problem X can be solved using:
• Polynomial number of standard computational steps, plus
• Polynomial number of calls to oracle that solves problem Y .
Notation. X ≤P Y .
Note. We pay for time to write down instances of Y sent to oracle ⇒instances of Y must be of polynomial size.
Novice mistake. Confusing X ≤P Y with Y ≤P X .
7
Quiz 1
Suppose that X ≤P Y . Which of the following can we infer?
A. If X can be solved in polynomial time, then so can Y .
B. X can be solved in poly time iff Y can be solved in poly time.
C. If X cannot be solved in polynomial time, then neither can Y .
D. If Y cannot be solved in polynomial time, then neither can X .
8
Quiz 2
Which of the following poly-time reductions are known?
A. FIND-MAX-FLOW ≤P FIND-MIN-CUT.
B. FIND-MIN-CUT ≤P FIND-MAX-FLOW.
C. Both A and B.
D. Neither A nor B.
9
Poly-time reductions
Design algorithms. If X ≤P Y and Y can be solved in polynomial time,
then X can be solved in polynomial time.
Establish intractability. If X ≤P Y and X cannot be solved in polynomial
time, then Y cannot be solved in polynomial time.
Establish equivalence. If both X ≤P Y and Y ≤P X , we use notation
X ≡P Y . In this case, X can be solved in polynomial time iff Y can be.
Bottom line. Reductions classify problems according to relative difficulty.
10
Packing and Covering Problems
10
Independent set
Independent Set. Given a graph G = (V ,E) and an integer k, is there a
subset of k (or more) vertices such that no two are adjacent?
Example. Is there an independent set of size ≥ 6?
Example. Is there an independent set of size ≥ 7?
11
Vertex cover
Vertex Cover. Given a graph G = (V ,E) and an integer k, is there a subset
of k (or fewer) vertices such that each edge is incident to at least one vertex
in the subset?
Example. Is there a vertex cover of size ≤ 4?
Example. Is there a vertex cover of size ≤ 3?
12
Quiz 3
Consider the following graph G . Which are true?
A. The white vertices are a vertex cover of size 7.
B. The black vertices are an independent set of size 3.
C. Both A and B.
D. Neither A nor B.
13
Vertex cover and independent set reduce to one another
Independent Set ≡P Vertex Cover.
Theorem
Proof. We show S is an independent set of size k iff V − S is a vertex cover
of size n − k.
14
Vertex cover and independent set reduce to one another
Independent Set ≡P Vertex Cover.
Theorem
Proof. We show S is an independent set of size k iff V − S is a vertex cover
of size n − k.
⇒• Let S be any independent set of size k.
• V − S is of size n − k.
• Consider an arbitrary edge (u, v) ∈ E .
•S independent ⇒ either u /∈ S, or v /∈ S, or both.
⇒ either u ∈ V − S, or v ∈ V − S, or both..
• Thus, V − S covers (u, v).
15
Vertex cover and independent set reduce to one another
Independent Set ≡P Vertex Cover.
Theorem
Proof. We show S is an independent set of size k iff V − S is a vertex cover
of size n − k.
⇐• Let V − S be any independent set of size n − k.
• S is of size k.
• Consider an arbitrary edge (u, v) ∈ E .
•
V − S is a vertex cover ⇒ either u ∈ V − S, or v ∈ V − S, or both.
⇒ either u /∈ S, or v /∈ S, or both.
.• Thus, S is an independent set.
16
Set cover
Set Cover. Given a set U of elements, a collection S of subsets of U, and an
integer k, are there ≤ k of these subsets whose union is equal to U?
Sample application.• m available pieces of software.
• Set U of n capabilities that we would like our system to have.
• The i th piece of software provides the set Si ⊆ U of capabilities.
• Goal: achieve all n capabilities using fewest pieces of software.
U = {1, 2, 3, 4, 5, 6, 7}Sa = {3, 7} Sb = {2, 4}Sc = {3, 4, 5, 6} Sd = {5}Se = {1} Sf = {1, 2, 6, 7}k = 2
a set cover instance17
Quiz 4
Given the universe U = {1, 2, 3, 4, 5, 6, 7} and the following sets, which is the
minimum size of a set cover?
A. 1
B. 2
C. 3
D. None of the above.
U = {1, 2, 3, 4, 5, 6, 7}Sa = {1, 4, 6} Sb = {1, 6, 7}Sc = {1, 2, 3, 6} Sd = {1, 3, 5, 7}Se = {2, 6, 7} Sf = {3, 4, 5}
18
Vertex cover reduces to set cover
Vertex Cover ≤P Set Cover.
Theorem
Proof. Given a Vertex Cover instance G = (V ,E) and k, we construct a Set
Cover instance (U,S, k) that has a set cover of size k iff G has a vertex cover
of size k.
Construction.• Universe U = E .
• Include one subset for each node
v ∈ V : Sv = {e ∈ E : e incident to v }.
vertex cover instance
(k = 2)
U = {1, 2, 3, 4, 5, 6, 7}Sa = {3, 7} Sb = {2, 4}Sc = {3, 4, 5, 6} Sd = {5}Se = {1} Sf = {1, 2, 6, 7}
set cover instance
(k = 2)19
Vertex cover reduces to set cover
G = (V ,E) contains a vertex cover of size k iff (U,S, k) contains a set
cover of size k.
Lemma
Proof. ⇒
Let X ⊆ V be a vertex cover of size k in G .
• Then Y = {Sv : v ∈ X} is a set cover of size k.
vertex cover instance
(k = 2)
U = {1, 2, 3, 4, 5, 6, 7}Sa = {3, 7} Sb = {2, 4}Sc = {3, 4, 5, 6} Sd = {5}Se = {1} Sf = {1, 2, 6, 7}
set cover instance
(k = 2)
20
Vertex cover reduces to set cover
G = (V ,E) contains a vertex cover of size k iff (U,S, k) contains a set
cover of size k.
Lemma
Proof. ⇐
Let Y ⊆ S be a set cover of size k in (U,S, k).
• Then X = {v : Sv ∈ Y } is a vertex cover of size k in G .
vertex cover instance
(k = 2)
U = {1, 2, 3, 4, 5, 6, 7}Sa = {3, 7} Sb = {2, 4}Sc = {3, 4, 5, 6} Sd = {5}Se = {1} Sf = {1, 2, 6, 7}
set cover instance
(k = 2)
21
Constraint Satisfaction Problems
21
Satisfiability
Literal. A Boolean variable or its negation: xi or x i .
Clause. A disjunction of literals: Cj = x1 ∨ x2 ∨ x3
Conjunctive normal form (CNF): Φ = C1 ∧ C2 ∧ C3 ∧ C4
SAT. Given a CNF formula Φ, does it have a satisfying truth assignment?
3-SAT. SAT where each clause contains exactly 3 literals
(and each literal corresponds to a different variable).
Φ = (x1 ∨ x2 ∨ x3) ∧ (x1 ∨ x2 ∨ x3) ∧ (x1 ∨ x2 ∨ x4)
yes instance: x1= true, x2= true, x3= false x4= false
Key application. Electronic design automation (EDA).
22
Satisfiability is hard
Scientific hypothesis. There does not exists a poly-time algorithm for 3-SAT.
P vs. NP This hypothesis is equivalent to P 6= NP conjecture.
23
3-satisfiability reduces to independent set
3-SAT ≤P Independent Set.
Theorem
Proof.
Given an instance Φ of 3-SAT, we construct an instance (G , k) of
Independent Set that has an independent set of size k = |Φ| iff Φ is satisfiable.
Construction.• G contains 3 nodes for each clause, one for each literal.
• Connect 3 literals in a clause in a triangle.
• Connect literal to each of its negations.
Φ = (x1 ∨ x2 ∨ x3) ∧ (x1 ∨ x2 ∨ x3) ∧ (x1 ∨ x2 ∨ x4)
24
3-satisfiability reduces to independent set
3-SAT ≤P Independent Set.
Theorem
Proof.
⇒ Consider any satisfying assignment for Φ.
• Select one true literal from each clause/triangle.
• This is an independent set of size k = |Φ|.
Φ = (x1 ∨ x2 ∨ x3) ∧ (x1 ∨ x2 ∨ x3) ∧ (x1 ∨ x2 ∨ x4)
25
3-satisfiability reduces to independent set
3-SAT ≤P Independent Set.
Theorem
Proof.
⇐ Let S be independent set of size k.
• S must contain exactly one node in each triangle.
• Set these literals to true and remaining literals consistently.
• All clauses in Φ are satisfied.
Φ = (x1 ∨ x2 ∨ x3) ∧ (x1 ∨ x2 ∨ x3) ∧ (x1 ∨ x2 ∨ x4)
26
Review
Basic reduction strategies.
• Simple equivalence: Independent Set ≡P Vertex Cover
• Special case to general case: Vertex Cover ≤P Set Cover.
• Encoding with gadgets: 3-SAT ≤P Independent Set.
Transitivity. If X ≤P Y and Y ≤P Z , then X ≤P Z .
Proof idea. Compose the two algorithms.
Example. 3-SAT ≤P Independent Set ≤P Vertex Cover ≤P Set Cover.
27
Decision, search and optimization problems
Decision problem. Does there exist a vertex cover of size ≤ k?
Search problem. Find a vertex cover of size ≤ k.
Optimization problem. Find a vertex cover of minimum size.
Goal. Show that all three problems poly-time reduce to one another.
28
Search problems VS. Decision problems
Vertex cover. Does there exist a vertex cover of size ≤ k?
Find vertex cover. Find a vertex cover of size ≤ k.
Theorem. Vertex cover ≡P Find vertex cover.
Proof.
≤P . Decision problem is a special case of search problem.
≥P . To find a vertex cover of size ≤ k:
• Determine if there exists a vertex cover of size ≤ k.
• Find a vertex v such that G − {v} has a vertex cover of size ≤ k − 1.
(any vertex in any vertex cover of size ≤ k will have this property)
• Include v in the vertex cover.
• Recursively find a vertex cover of size ≤ k − 1 in G − {v}.
29
Optimization problems VS. Search problems VS. Decision problems
Find vertex cover. Find a vertex cover of size ≤ k.
Find min vertex cover. Find a vertex cover of minimum size.
Theorem. Find vertex cover ≡P Find min vertex cover.
Proof.
≤P . Search problem is a special case of optimization problem.
≥P . To find vertex cover of minimum size:
• Binary search (or linear search) for size k∗ of min vertex cover.
• Solve search problem for given k∗.
30
Sequencing Problems
30
Hamilton cycle
Hamilton cycle. Given an undirected graph G = (V ,E), does there exist a
cycle Γ that visits every node exactly once?
yes
31
Hamilton cycle
Hamilton cycle. Given an undirected graph G = (V ,E), does there exist a
cycle Γ that visits every node exactly once?
no
32
Directed Hamilton cycle reduces to Hamilton cycle
Directed Hamilton cycle. Given a directed graph G = (V ,E), does there exist
a directed cycle Γ that visits every node exactly once?
Directed Hamilton cycle ≤P Hamilton cycle.
Theorem
Proof. Given a directed graph G = (V ,E), construct a graph G ′ with 3n
nodes.
33
Directed Hamilton cycle reduces to Hamilton cycle
G has a directed Hamilton cycle iff G ′ has a Hamilton cycle.
Lemma
Proof.
⇒• Suppose G has a directed Hamilton cycle Γ.
• Then G ′ has an undirected Hamilton cycle (same order).
⇐
• Suppose G ′ has an undirected Hamilton cycle Γ′.
• Γ′ must visit nodes in G ′ using one of following two orders:
. . . , black,white, blue, black,white, blue, black,white, blue, . . .
. . . , black, blue,white, black, blue,white, black, blue,white, . . .
• Black nodes in Γ′ comprise either a directed Hamilton cycle Γ in G , or
reverse of one.
34
3-satisfiability reduces to directed Hamilton cycle
3-SAT ≤P Directed Hamilton cycle.
Theorem
Proof.
Given an instance Φ of 3-SAT, we construct an instance G of Directed
Hamilton cycle that has a Hamilton cycle iff Φ is satisfiable.
Construction overview. Let n denote the number of variables in Φ. We will
construct a graph G that has 2n Hamilton cycles, with each cycle
corresponding to one of the 2n possible truth assignments.
35
3-satisfiability reduces to directed Hamilton cycle
Construction. Given 3-SAT instance Φ with n variables xi and k clauses.
• Construct G to have 2n Hamilton cycles.
• Intuition: traverse path i from left to right ⇔ set variables xi =true
36
Quiz 5
Which is truth assignment corresponding to Hamilton cycle below?
A. x1 = true, x2 = true, x3 = true
B. x1 = true, x2 = true, x3 = false
C. x1 = false, x2 = false, x3 = true
D. x1 = false, x2 = false, x3 = false
37
3-satisfiability reduces to directed Hamilton cycle
Construction. Given 3-SAT instance Φ with n variables xi and k clauses.
• For each clause: add a node and 2 edges per literal.
38
3-satisfiability reduces to directed Hamilton cycle
Construction. Given 3-SAT instance Φ with n variables xi and k clauses.
• For each clause: add a node and 2 edges per literal.
39
3-satisfiability reduces to directed Hamilton cycle
Φ is satisfiable iff G has a Hamilton cycle.
Lemma
Proof.
⇒• Suppose 3-SAT instance Φ has satisfying assignment x∗.
• Then, define Hamilton cycle Γ in G as follows:
- if x∗i = true, traverse row i from left to right.
- if x∗i = false, traverse row i from right to left.
- for each clause Cj , there will be at least one row i in which we are going in
“correct” direction to splice clause node Cj into cycle (and we splice in Cjexactly once)
40
3-satisfiability reduces to directed Hamilton cycle
Φ is satisfiable iff G has a Hamilton cycle.
Lemma
Proof.
⇐• Suppose G has a Hamilton cycle Γ.
• If Γ enters clause node Cj , it must depart on mate edge.
- nodes immediately before and after Cj are connected by an edge e ∈ E .- removing Cj from cycle, and replacing it with edge e yields Hamilton cycle
on G − {Cj}.
• Continuing in this way, we are left with a Hamilton cycle Γ′ in
G − {C1,C2, . . . ,Ck}.
• Set x∗i = true if Γ′ traverses row i left-to-right; otherwise, set x∗i = false.
• traversed in “correct” direction, and each clause is satisfied.
41
Graph Coloring
41
Home reading!
42
Numerical Problems
42
Subset sum
Subset sum. Given n natural numbers w1, . . . ,wn and an integer W , is there a
subset that adds up to exactly W ?
Example. { 215, 215, 275, 275, 355, 355, 420, 420, 580, 580, 655, 655 },W = 1505.
Yes. 215 + 355 + 355 + 580 = 1505.
Remark. With arithmetic problems, input integers are encoded in binary.
Poly-time reduction must be polynomial in binary encoding.
43
Subset sum
3-SAT ≤P Subset sum.
Theorem
Proof. Given an instance Φ of 3-SAT, we construct an instance of Subset
sum that has solution iff Φ is satisfiable.
44
3-satisfiability reduces to subset sum
Construction. Given 3-SAT instance Φ with n variables and k clauses, form
2n + 2k decimal integers, each having n + k digits:
• Include one digit for each variable xi and one digit for each clause Cj .
• Include two numbers for each variable xi .
• Include two numbers for each clause Cj .
• Sum of each xi digit is 1;
sum of each Cj digit is 4.
Key property. No carries possible ⇒each digit yields one equation.
3-SAT instanceSubset sum instance
45
3-satisfiability reduces to subset sum
Φ is satisfiable iff there exists a subset that sums to W .
Lemma
Proof.⇒ Suppose 3-SAT instance Φ has satisfying assignment x∗.
• If x∗i = true, select integer in row xi ;
otherwise, select integer in row ¬xi .
• Each xi digit sums to 1.
• Since Φ is satisfiable, each Cj digit sums.
to at least 1 from xi and ¬xi rows.
• Select dummy integers to make
Cj digits sum to 4.
3-SAT instance
Subset sum instance46
3-satisfiability reduces to subset sum
Φ is satisfiable iff there exists a subset that sums to W .
Lemma
Proof.⇐ Suppose there exists a subset S∗ that sums to W .
• Digit xi forces subset S∗ to select either row xi or row ¬xi (but not both).
• If row xi selected, assign x∗i = true; otherwise, assign x∗i = false.
Digit Cj forces subset S∗ to select
at least one literal in clause.
3-SAT instance
Subset sum instance47
Subset sum reduces to knapsack
Subset sum. Given n natural numbers w1, . . . ,wn and an integer W , is there a
subset that adds up to exactly W ?
Knapsack. Given a set of items X , weights ui ≥ 0, values vi ≥ 0, a weight
limit U, and a target value V , is there a subset S ⊆ X such that:∑i∈S
ui ≤ U,∑i∈S
vi ≥ V
Recall. O(nU) dynamic programming algorithm for knapsack.
Challenge. Prove subset sum ≤P Knapsack.
48
Poly-time reductions
3-SAT
constraint satisfaction
3-SAT poly-time reduces
to INDEPENDENT-SET
INDEPENDENT-SET SUBSET-SUMDIR-HAM-CYCLE 3-COLOR
VERTEX-COVER KNAPSACKHAM-CYCLE
SET-COVER
packing and covering numericalsequencing partitioning
Karp’s 20 poly-time reductions from satisfiability
50