1 The Theory of NP-Completeness 2 Cook ’ s Theorem (1971) Prof. Cook Toronto U. Receiving Turing...

1

The Theory of NP-Completeness

2

Cook’s Theorem (1971) Prof. Cook Toronto U. Receiving Turing Award (1982) Discussing difficult problems: worst

case lower bound seems to be in the order of an exponential function

NP-complete (NPC) Problems

3

Finding lower bound by problem transformation

Problem A reduces to problem B (AB) iff A can be solved by using any algorithm

which solves B. If AB, B is more difficult (B is at least as hard

as A)

Since (A) (B) +T(tr1) + T(tr2), we have(B) (A) –(T(tr1) + T(tr2))

We have (B) (A) if T(tr1) + T(tr2) (A)

instance of A

transformation T(tr1)

instance of B

solver of B

answer of A

transformation

T(tr2)

answer of B

4

The lower bound of the convex hull problem sorting convex hull A B an instance of A: (x1, x2,…, xn)

↓transformation

an instance of B: {( x1, x12), ( x2, x2

2),…, ( xn, xn

2)}assume: x1 < x2 < …< xn

5

The lower bound of the convex hull problem If the convex hull problem can be

solved, we can also solve the sorting problem, but not vice versa.

We have that the convex hull problem is harder than the sorting problem.

The lower bound of sorting problem is (n log n), so the lower bound of the convex hull problem is also (n log n).

6

NP

PNPC

NP: Non-deterministic Polynomial

P: Polynomial

NPC: Non-deterministic Polynomial Complete

P=NP?

7

Nondeterministic algorithms A nondeterministic

algorithm is an algorithm consisting of two phases: guessing and checking.

Furthermore, it is assumed that a nondeterministic algorithm always makes a correct guessing.

8

Nondeterministic algorithms

Machines for running nondeterministic algorithms do not exist and they would never exist in reality. (They can only be made by allowing unbounded parallelism in computation.)

Nondeterministic algorithms are useful only because they will help us define a class of problems: NP problems

9

NP algorithm If the checking stage of a

nondeterministic algorithm is of polynomial time-complexity, then this algorithm is called an NP (nondeterministic polynomial) algorithm.

10

NP problem If a decision problem can be solved by a

NP algorithm, this problem is called an NP (nondeterministic polynomial) problem.

NP problems : (must be decision problems)

11

Decision problems The solution is simply “Yes” or “No”. Optimization problem : harder

Decision problem : easier E.g. the traveling salesperson problem

Optimization version:Find the shortest tour

Decision version:Is there a tour whose total length is less than or equal to a constant C ?

12

Decision version of sorting

Given a1, a2,…, an and c, is there a

permutation of ais ( a1

, a2 , … ,an

)

such that∣a2–a1

∣+∣a3–a2

∣+ … +∣an–

an-1∣＜ C ?

13

Decision vs Original Version

We consider decision version problem

D rather than the original problem O

because we are addressing the lower

bound of a problem

and D ∝ O

14

To express Nondeterministic Algorithm

Choice(S) : arbitrarily chooses one of the elements in set S

Failure : an unsuccessful completion

Success : a successful completion

15

Nondeterministic searching Algorithm :

input: n elements and a target element xoutput: success if x is found among the n elements; failure, otherwise.

j ← choice(1 : n) /* guessif A(j) = x then success /* check else failure

A nondeterministic algorithm terminates unsuccessfully iff there exist no set of choices leading to a success signal.

The time required for choice(1 : n) is O(1).

16

Relationship Between NP and P

It is known PNP. However, it is not known whether P=NP or

whether P is a proper subset of NP It is believed NP is much larger than P

We cannot find a polynomial-time algorithm for many NP problems.

But, no NP problem is proved to have exponential lower bound. (No NP problem has been proved to be not in P.)

So, “does P = NP?” is still an open question! Cook tried to answer the question by

proposing NPC.

17

NP-complete (NPC)A problem A is NP-complete (NPC) if A∈NP and every NP problem reduces to A.

18

SAT is NP-complete Every NP problem can be solved by an

NP algorithm Every NP algorithm can be transformed

in polynomial time to an SAT problem Such that the SAT problem is satisfiable

iff the answer for the original NP problem is “yes”

That is, every NP problem SAT SAT is NP-complete

19

Cook’s theorem (1971)

NP = P iff SAT NP = P iff SAT P P NP = P iff the satisfiability NP = P iff the satisfiability

(SAT) problem is a P problem(SAT) problem is a P problem SAT is NP-complete It is the first NP-complete problem Every NP problem reduces to SAT

20

Proof of NP-Completeness To show that A is NP-complete

(I) Prove that A is an NP problem (II) Prove that B NPC, B A

A NPC Why ?

Transitive property of polynomial-time reduction

21

0/1 Knapsack problem

Given M (weight limit) and V, is there is a solution with value larger than V?

This is an NPC problem.

P1 P2 P3 P4 P5 P6 P7 P8

Value 10 5 1 9 3 4 11 17

Weight

7 3 3 10 1 9 22 15

22

Traveling salesperson problem Given: A set of n planar points and

a value LFind: Is there a closed tour which includes all points exactly once such that its total length is less than L?

This is an NPC problem.

23

Partition problem

Given: A set of positive integers SFind: Is there a partition of S1 and S2 such that S1S2=, S1S2=S, iS1i=iS2 i(partition S into S1 and S2 such that element sum of S1 is equal to that of S2)

e.g. S={1, 7, 10, 9, 5, 8, 3, 13} S1={1, 10, 9, 8} S2={7, 5, 3, 13}

This problem is NP-complete.

24

Art gallery problem: *Given a constant C, is there a guard placement such that the number of guards is less than C and every wall is monitored?*This is an NPC problem.

25

Karp R. Karp showed several NPC problems, such as 3-STA, node (vertex) cover, and Hamiltonian cycle, etc.

Karp received Turing Award in 1985

26

NP-Completeness Proof: Reduction

Vertex Cover

Clique 3-SAT

SAT

Chromatic Number

Dominating Set

All NP problems

27

NP-Completeness “NP-complete problems”: the

hardest problems in NP Interesting property

If any one NP-complete problem can be solved in polynomial time, then every problem in NP can also be solved in polynomial time (i.e., P=NP)

Many believe P≠NP

28

Importance of NP-Completeness

NP-complete problems: considered “intractable”

Important for algorithm designers & engineers Suppose you have a problem to solve

Your colleagues have spent a lot of time to solve it exactly but in vain

See whether you can prove that it is NP-complete If yes, then spend your time developing an

approximation (heuristic) algorithm Many natural problems can be NP-complete

29

Some concepts Up to now, none of the NPC problems can be

solved by a deterministic polynomial time algorithm in the worst case.

It does not seem to have any polynomial time algorithm to solve the NPC problems.

The lower bound of any NPC problem seems to be in the order of an exponential function.

The theory of NP-completeness always considers the worst case.

30

Caution !

If a problem is NP-complete, its special cases may or may not be of exponential time-complexity.

We consider worst case lower bound in NP-complete.

31

Some concepts Not all NP problems are difficult. (e.g.

the MST problem is an NP problem.) (But NPC problem is difficult.)

If A, B NPC, then A B and B A. Theory of NP-completeness

If any NPC problem can be solved in If any NPC problem can be solved in polynomial time, then all NP problems can be polynomial time, then all NP problems can be solved in polynomial time. solved in polynomial time. (NP = P)(NP = P)

32

Undecidable Problems

They cannot be solved by guessing and checking.

They are even more difficult than NP problems.

E.G.: Halting problem: Given an arbitrary program with an arbitrary input data, will the program terminate or not? It is not NP It is NP-hard (SAT Halting problem)

33

NP : the class of decision problem which can be solved by a non-deterministic polynomial algorithm.

P: the class of problems which can be solved by a deterministic polynomial algorithm.

NP-hard: the class of problems to which every NP problem reduces. (It is “at least as hard as the hardest problems in NP.”)

NP-complete: the class of problems which are NP-hard and belong to NP.

34

The satisfiability (SAT) problem Def : Given a Boolean formula,

determine whether this formula is satisfiable or not.

  A literal : xi or -xi

A clause : x1 v x2 v -x3 ci A formula : conjunctive normal form

C1& c2 & … & cm

35

The satisfiability (SAT) problem

The satisfiability problem The logical formula :

x1 v x2 v x3

& - x1

& - x2 the assignment :

x1 ← F , x2 ← F , x3 ← Twill make the above formula true

(-x1, -x2 , x3) represents

x1 ← F , x2 ← F , x3 ← T

36

The satisfiability problem If there is at least one assignment which

satisfies a formula, then we say that this formula is satisfiablesatisfiable; otherwise, it is unsatisfiableunsatisfiable.

An unsatisfiable formula : x1 v x2

& x1 v -x2

& -x1 v x2

& -x1 v -x2

37

Resolution principle c1 : -x1 v -x2 v x3

c2 : x1 v x4

c3 : -x2 v x3 v x4 (resolvent) If no new clauses can be deduced

satisfiable -x1 v -x2 v x3 (1)

x1 (2)

x2 (3)

(1) & (2) -x2 v x3 (4)

(4) & (3) x3 (5)

(1) & (3) -x1 v x3 (6)

The satisfiability problemx1 cannot satisfyc1 and c2 at thesame time, so itis deleted..

38

The satisfiability problem If an empty clause is deduced unsatisfiable - x1 v -x2 v x3 (1)

x1 v -x2 (2)

x2 (3)

- x3 (4)  deduce

  (1) & (2) -x2 v x3 (5)

(4) & (5) -x2 (6) (6) & (3) □ (7)

39

Nondeterministic SAT Guessing for i = 1 to n do

xi ← choice( true, false )

if E(x1, x2, … ,xn) is true

Checking then success

else failure

40

Transforming the NP searching algorithm to the SAT problem

Does there exist a number in { x(1), x(2), …, x(n) }, which is equal to 7?

Assume n = 2

41

Transforming searching to SAT i=1 v i=2& i=1 → i≠2 & i=2 → i≠1 & x(1)=7 & i=1 → SUCCESS & x(2)=7 & i=2 → SUCCESS & x(1)≠7 & i=1 → FAILURE & x(2)≠7 & i=2 → FAILURE & FAILURE → -SUCCESS & SUCCESS (Guarantees a successful termination) & x(1)=7 (Input Data) & x(2)≠7

42

Transforming searching to SAT CNF (conjunctive normal form) :

i=1 v i=2 (1) i≠1 v i≠2 (2) x(1)≠7 v i≠1 v SUCCESS (3) x(2)≠7 v i≠2 v SUCCESS (4) x(1)=7 v i≠1 v FAILURE (5) x(2)=7 v i≠2 v FAILURE (6) -FAILURE v -SUCCESS (7) SUCCESS (8) x(1)=7 (9) x(2)≠7 (10)

43

Transforming searching to SAT Satisfiable at the following assignment :

i=1 satisfying (1) i≠2 satisfying (2), (4) and (6) SUCCESS satisfying (3), (4) and (8) -FAILURE satisfying (7) x(1)=7 satisfying (5) and (9) x(2)≠7 satisfying (4) and (10)

44

Searching in CNF with inputs

Searching for 7, but x(1)7, x(2)7 CNF :

i=1 v i=2 (1) i1 v i2 (2) x(1)7 v i1 v S U C C E S S ( 3 ) x(2)7 v i2 v S U C C E S S ( 4 ) x(1)=7 v i1 v FAILURE (5 ) x(2)=7 v i2 v FAILURE (6 ) SUCCESS (7) -SUCCESS v -FAILURE (8) x(1) 7 (9) x(2) 7 (10)

45

Searching in CNF with inputs

Apply resolution principle :

(9) & (5) i1 v FAILURE (11) (10) & (6) i2 v FAILURE (12) (7) & (8) -FAILURE (13) (13) & (11) i1 (14) (13) & (12) i2 (15) (14) & (1) i=2 (11) (15) & (16) □ (17)

We get an empty clause unsatisfiable 7 does not exit in x(1) or x(2).

46

Searching in CNF with inputs

Searching for 7, where x(1)=7, x(2)=7 CNF : i=1 v i=2 (1) i1 v i2 (2) x(1)7 v i1 v S U C C E S S ( 3 ) x(2)7 v i2 v S U C C E S S ( 4 ) x(1)=7 v i1 v F A I L U R E ( 5 ) x(2)=7 v i2 v F A I L U R E ( 6 ) SUCCESS (7) -SUCCESS v -FAILURE (8) x(1)=7 (9) x(2)=7 (10)

47

Q&A