Algorithms and Theory of Computation Lecture 21: NP ... · Algorithms and Theory of Computation Lecture 21: NP-Completeness Xiaohui Bei MAS 714 November 6, 2017 Nanyang Technological

Algorithms and Theory of Computation

Lecture 21: NP-Completeness

Xiaohui Bei

MAS 714

November 6, 2017

Nanyang Technological University MAS 714 November 6, 2017 1 / 32

NP-Completeness

P: set of decision problems that have polynomial time algorithms.

NP: set of decision problems that have polynomial time non-deterministic

algorithms.

Many natural problems we would like to solve are in NP.

P ⊆ NP

Some problems in NP are in P (e.g. shortest path problem)

Big Question

Does every problem in NP have an efficient algorithm?

Same as asking whether P = NP.


“Hardest” Problems

Question

What is the hardest problem in NP? How do we define it?

Towards a definition:

hardest problem must be in NP

hardest problem must be at least as “difficult” as every other problem

in NP.

NP-Completeness

A problem X is said to be NP-Complete if

X ∈ NP, and

(hardness) for any Y ∈ NP, Y ≤P X.


Solving NP-Complete Problems

Proposition

Suppose X is NP-Complete. Then X can be solved in polynomial time if

and only if P = NP.

Proof.

=⇒: Suppose X can be solved in polynomial time

Let Y ∈ NP, we know Y ≤P X.

Then Y can be solved in polynomial time.

Thus every problem Y ∈ NP is such that Y ∈ P; NP ⊆ P

Also P ⊆ NP, we have P = NP.⇐=: Since P = NP, and X ∈ NP, we know X ∈ P.


NP-Hard Problems

NP-Hardness

A problem X is said to be NP-Hard if

for any Y ∈ NP, we have that Y ≤P X.

An NP-Hard problem need not to be in NP.

Example: Halting problem is NP-Hard, but not NP-Complete.


Consequences of proving NP-Completeness

If X is NP-Complete

since we believe P 6= NP,

and solving X efficiently implies P = NP.

X is unlikely to be efficiently solvable.

At the very least, many smart people before you have failed to find an

efficient algorithm for X.

Question

Are there any problems that are NP-Complete?

Yes! There are many of them.


Cook-Levin Theorem

Theorem (Cook-Levin)

SAT is NP-Complete.

First, need to show SAT is in NP.

Easy task. What should be our “proof/solution” string?

Second, need to prove that every language L ∈ NP, L ≤P SAT.

Difficult task. Infinite number of languages in NP. Must

simultaneously show a generic reduction strategy.

Steve Cook won the Turing award for this theorem.


High-level Plan

What does it mean that L ∈ NP?

L ∈ NP implies that there is a non-deterministic TM M and polynomial

p() such that

L = {x ∈ Σ∗ |M accepts x in at most p(|x|) steps}

We will describe a reduction fM that depends on M,p such that:

fM takes as input a string x and outputs a SAT formula fM(x)

fM runs in time polynomial in |x|

x ∈ L if and only if fM(x) is satisfiable


Plan continued

fM(x) is satisfiable ⇐⇒ x ∈ L ⇐⇒ non-det M accepts x in p(|x|) steps.

Big Idea:

fM(x) will express “M on input x accepts in p(|x|) steps”

fM(x) will encode a computation history of M on x

fM(x) will be a carefully constructed CNF formula s.t if we have a

satisfying assignment to it, then we will be able to see a complete

accepting computation of M on x down to the last detail of where the

head is, what transition is chosen, what the tape contents are, at each

step.


Tableau of Computation

Entire computation of M on x can be represented by a “tableau”

1 0 0 1 blanks state q00 0 0 1 blanks state q1

......

Row i gives contents of all cells at time i.

At time 0 tape has input x followed by blanks.

Each row long enough to hold all cells M might ever have scanned.


Variable of fM(x)

Four types of variables to describe computation of M on x

T(b, h, i): tape cell at position h holds symbol b at time i.

1 ≤ h ≤ p(|x|), b ∈ Γ, 0 ≤ i ≤ p(|x|)H(h, i): read/write head is at position h at time i.

1 ≤ h ≤ p(|x|), 0 ≤ i ≤ p(|x|)S(q, i) state of M is q at time i

q ∈ Q, 0 ≤ i ≤ p(|x|)I(j, i): instruction number j is executed at time i

Since M is non-det, need to specify transitions in some way.

Assume there are ` possible transitions, where j’th transition is

〈qj, bj, q ′j , b ′j , dj〉, indicating (q ′j , b′j , dj) ∈ δ(qj, bj),

direction dj ∈ {−1, 0, 1}

Number of variables is O(p(|x|)2).


Notation

Some abbreviations for ease of notation

∨mk=1 xk means x1 ∨ x2 ∨ . . .∨ xm∧mk=1 xk means x1 ∧ x2 ∧ . . .∧ xm⊕(x1, x2, . . . , xk) is a formula that means exactly one of

x1, x2, . . . , xm is true. Can be converted to CNF form.


Clauses of fM(x)

fM(x) is the conjunction of 8 clause groups

fM(x) = ϕ1 ∧ϕ2 ∧ϕ3 ∧ϕ4 ∧ϕ5 ∧ϕ6 ∧ϕ7 ∧ϕ8

where each ϕi is a CNF formula. Described in subsequent slides.

Property

fM(x) is satisfiable iff there is a truth assignment to the variables that

simultaneously satisfy ϕ1, . . . , ϕ8.


ϕ1

ϕ1 asserts (is true iff) the variables are set T/F indicating that M starts in

state q0 at time 0 with tape contents containing x followed by blanks.

Let x = a1a2 . . . an

ϕ1 = S(q0, 0) state at time 0 is q0

∧nh=1 T(ah, h, 0) at time 0 cells 1 to n have a1 to an

∧p(|x|)h=n+1 T(B, h, 0) at time 0 cells n + 1 to p(|x|) have blanks

∧H(1, 0) head at time 0 is in position 1


ϕ2

ϕ2 asserts M in exactly one state at any time i

ϕ2 =

p(|x|)∧i=0

(⊕(S(q0, i), S(q1, i), . . . , S(q|Q|, i)

))


ϕ3

ϕ3 asserts that each tape cell holds a unique symbol at any given time.

ϕ3 =

p(|x|)∧i=0

p(|x|)∧h=1

(⊕(T(b1, h, i), T(b2, h, i), . . . , T(b|Γ |, h, i)

))

For each time i and for each cell position h exactly one symbol b ∈ Γ at

cell position h at time i


ϕ4

ϕ4 asserts that the read/write head of M is in exactly one position at any

time i.

ϕ4 =

p(|x|)∧i=0

(⊕(H(1, i), H(2, i), . . . , H(p(|x|), i))

)


ϕ5

ϕ5 asserts that M accepts, i.e., enters accepting state at some time

let qa be unique accept state of M

ϕ5 =

p(|x|)∨i=1

S(qa, i)


ϕ6

ϕ6 asserts M executes a unique instruction at each time

ϕ6 =

p(|x|)∧i=0

(⊕(I(1, i), I(2, i), . . . , I(m, i))

)

where m is the max instruction number.


ϕ7

ϕ7 ensures that variables don’t allow tape to change from one moment to

next if the read/write head was not there.

ϕ7 =

p(|x|)∧i=0

p(|x|)∧h=1

∧b6=c

(H(h, i) =⇒ T(b, h, i)∧ T(c, h, i+ 1)

)“If head is not at position h at time i then at time i + 1 the symbol at cell h must be

unchanged”

(A =⇒ B) can be formulated as (A∨ B)


ϕ8

ϕ8 asserts that changes in tableau/tape correspond to transitions of M

Let j’th instruction be 〈qj, bj, q ′j , b ′j , dj〉

ϕ8 =∧i

∧j(I(j, i) =⇒ S(qj, i))

If instr j executed at time i then state must be correct to do j∧i

∧j(I(j, i) =⇒ S(q ′j , i+ 1))

and at next time unit, state must be the proper next state for instr j∧i

∧h

∧j [(I(j, i)∧H(h, i)) =⇒ T(bj, h, i)]

if j was executed and head was at position h, then cell h has correct symbol for j∧i

∧h

∧j

[(I(j, i)∧H(h, i)) =⇒ T(b ′j , h, i+ 1)

]if j was done at time i with head at h then at next step b ′

j was written in position h∧i

∧h

∧j [(I(j, i)∧H(h, i)) =⇒ H(h+ dj, i+ 1)]

head is moved properly according to instr j.


Proof of Correctness (sketch)

Given M,x, fM can be constructed in polynomial time

if fM(x) is satisfiable then the truth assignment completely specifies

an accepting computation of M on x

if M accepts x then the accepting computation leads to an “obvious”

truth assignment to fM(x). Simply assign the variables according to

the state of M and cells at each time i.

Thus M accepts x if and only if fM(x) is satisfiable.


Proving that a Problem X is NP-Complete

To prove X is NP-Complete, we need to

show that X is in NP

give a polynomial-time reduction from a known NP-Complete problem

such as SAT to X.

SAT ≤P X implies that every NP problem Y ≤P X. Why?

transitivity of reductions, Y ≤P SAT and SAT ≤P X implies Y ≤P X

3SAT

3SAT is NP-Complete.


NP-Completeness via Reductions

SAT is NP-Complete due to Cook-Levin Theorem.

SAT ≤P 3SAT

3SAT ≤P Independent Set

Independent Set ≤P Clique

Independent Set ≤P Vertex Cover

3SAT ≤P 3-Color

3SAT ≤P Hamiltonian Cycle

Hundreds and thousands of different problems from many areas of science

and engineering have been shown to be NP-Complete.

A surprisingly frequent phenomenon.


More Reductions

We have shown NP-completeness of SAT and 3SAT.

We will show some reductions from 3SAT to a few problems.

We can use reductions from any of the problems for which we alreadyhave a reduction from 3SAT

I due to transitivityI often this is easierI you can also do this in your exams


Reducing 3SAT to Independent Set

Problem: Independent Set

Instance: A graph G and an integer k.

Question: Is there an independent set in G of size k?

The reduction 3SAT ≤P Independent Set

Input: Given a 3CNF formula ϕ

Goal: Construct a graph Gϕ and number k such that Gϕ has an

independent set of size at least k if and only if ϕ is satisfiable.

Gϕ should be constructible in time polynomial in size of ϕ

Notice: We handle only 3CNF formulas


Interpreting 3SAT

There are two ways to think about 3SAT

1 Find a way to assign 0/1 (false/true) to the variables such that the

formula evaluates to true, that is each clause evaluates to true.

2 Pick a literal from each clause and find a truth assignment to make

all of them true. You will fail if two of the literals you pick are in

conflict, i.e., you pick both xi and xi.

We will take the second view of 3SAT to construct the reduction.


The Reduction

1 Gϕ will have one vertex for each literal in a clause.2 Connect the 3 literals in a clause to form a triangle; the independent

set will pick at most one vertex from each clause, which will

correspond to the literal to be set to true.3 Connect 2 vertices if they label complementary literals; this ensures

that the literals picked do not have a conflict.4 Take k to be the number of clauses.

x2 x3

x1

x1 x3

x2

x2 x4

x1

Figure: Graph for ϕ =(x1 ∨ x2 ∨ x3

)∧

(x1 ∨ x2 ∨ x3

)∧

(x1 ∨ x2 ∨ x4

)Nanyang Technological University MAS 714 November 6, 2017 28 / 32

Correctness

Proposition

ϕ is satisfiable iff Gϕ has an independent set of size k (= number of

clauses in ϕ).

Proof.⇒: Let a be a truth assignment satisfying ϕ

Pick one of the vertices, corresponding to true literals under a, from

each triangle. This is an independent set of the appropriate size.⇐: Let S be an independent set of size k

S must contain exactly one vertex from each clause

S cannot contain vertices labeled by conflicting literals

Thus, it is possible to obtain a truth assignment that makes literals in

S true; such an assignment satisfies one literal in every clause


Vertex Cover

For an undirected graph G, a vertex cover is a subset S of the vertices

such that every edge has at least one endpoint in S

Problem: Vertex Cover

Instance: A graph G and an integer k

Question: Does G has a vertex cover of size at most k?

Theorem

Vertex Cover is NP-complete.

Part 1: Vertex Cover ∈ NP

easy, guess S and check all edges

Part 2: we reduce from Independent Set


Key Observation

Theorem

if G = (V, E) is a graph, then S is an independent set of G if and only if

V − S is a set cover.

Proof.=⇒:

Suppose S is an independent set, and let e = (u, v) be some edge.

Only one of u, v can be in S. Hence, at least one of u, v is in V − S.

So, V − S is a vertex cover.⇐=:

Suppose V − S is a vertex cover, and let u, v ∈ S. There can’t be an

edge between u and v (otherwise, that edge wouldn’t be covered in

V − S). So, S is an independent set.


Reducing Independent Set to Vertex Cover

Reducing Independent Set to Vertex Cover

Input: Given a graph G and an integer k

Goal: Construct a graph G ′ and k ′ such that G has an independent set of

size at least k if and only G ′ has a vertex cover of size at most k ′

Reduction algorithm: output G ′ = G, k ′ = |V |− k.

correctness directly followed from our previous theorem


Documents

Algorithms and Theory of Computation Lecture 21: NP ... · Algorithms and Theory of Computation Lecture 21: NP-Completeness Xiaohui Bei MAS 714 November 6, 2017 Nanyang Technological