Upload
randell-baker
View
225
Download
0
Embed Size (px)
Citation preview
1
UNIT -6
P, NP, and NP-Complete
2
Tractability
Some problems are intractable: as they grow large, we are unable to solve them in reasonable time
What constitutes reasonable time? » Standard working definition: polynomial time» On an input of size n the worst-case running time is O(nk)
for some constant k» O(n2), O(n3), O(1), O(n lg n), O(2n), O(nn), O(n!)» Polynomial time: O(n2), O(n3), O(1), O(n lg n) » Not in polynomial time: O(2n), O(nn), O(n!)
3
Polynomial-Time Algorithms
Are some problems solvable in polynomial time?» Of course: many algorithms we’ve studied provide
polynomial-time solutions to some problems
Are all problems solvable in polynomial time?» No: Turing’s “Halting Problem” is not solvable by any
computer, no matter how much time is given
Most problems that do not yield polynomial-time algorithms are either optimization or decision problems.
4
Optimization/Decision Problems
Optimization Problems» An optimization problem is one which asks, “What is
the optimal solution to problem X?”» Examples:
0-1 Knapsack Fractional Knapsack Minimum Spanning Tree
Decision Problems» An decision problem is one with yes/no answer» Examples:
Does a graph G have a MST of weight W?
5
Optimization/Decision Problems
An optimization problem tries to find an optimal solution A decision problem tries to answer a yes/no question Many problems will have decision and optimization
versions» Eg: Traveling salesman problem
optimization: find hamiltonian cycle of minimum weight decision: is there a hamiltonian cycle of weight k
Some problems are decidable, but intractable: as they grow large, we are unable to solve them in reasonable time» Is there a polynomial-time algorithm that solves the problem?
6
The Class P
P: the class of decision problems that have polynomial-time deterministic algorithms. » That is, they are solvable in O(p(n)), where p(n) is a polynomial on n
» A deterministic algorithm is (essentially) one that always computes the correct answer
Why polynomial?» if not, very inefficient
» nice closure properties the sum and composition of two polynomials are always polynomials too
7
Sample Problems in P
Fractional Knapsack MST Sorting Others?
8
The class NP
NP: the class of decision problems that are solvable in polynomial time on a nondeterministic machine (or with a nondeterministic algorithm)
(A determinstic computer is what we know) A nondeterministic computer is one that can “guess” the right
answer or solution » Think of a nondeterministic computer as a parallel machine that can
freely spawn an infinite number of processes Thus NP can also be thought of as the class of problems
» whose solutions can be verified in polynomial time Note that NP stands for “Nondeterministic Polynomial-time”
9
Sample Problems in NP
Fractional Knapsack MST Sorting Others?
» Hamiltonian Cycle (Traveling Salesman)
» Graph Coloring
» Satisfiability (SAT) the problem of deciding whether a given
Boolean formula is satisfiable
10
NP-complete problems
A decision problem D is NP-complete iff1. D NP
2. every problem in NP is polynomial-time reducible to D
11
NP-Hard and NP-Complete
If R is polynomial-time reducible to Q, we denote this R p Q
Definition of NP-Hard and NP-Complete: » If all problems R NP are polynomial-time reducible to
Q, then Q is NP-Hard
» We say Q is NP-Complete if Q is NP-Hard and Q NP
If R p Q and R is NP-Hard, Q is also NP-Hard (why?)
12
Boolean satisfiability problem
In computer science, the Boolean Satisfiability Problem (sometimes called Propositional Satisfiability Problem and abbreviated as SATISFIABILITY or SAT) is the problem of determining if there exists an interpretation that satisfies a given Boolean formula.
In other words, it asks whether the variables of a given Boolean formula can be consistently replaced by the values TRUE or FALSE in such a way that the formula evaluates to TRUE.
If this is the case, the formula is called satisfiable.
13
On the other hand, if no such assignment exists, the function expressed by the formula is identically FALSE for all possible variable assignments and the formula is unsatisfiable.
For example, the formula "a AND NOT b" is satisfiable because one can find the values a = TRUE and b = FALSE, which make (a AND NOT b) = TRUE. In contrast, "a AND NOTa" is unsatisfiable.
14
SAT is one of the first problems that was proven to be NP-complete.
This means that all problems in the complexity classNP, which includes a wide range of natural decision and optimization problems, are in a technical sense at most that difficult to solve as SAT.
There is no known algorithm that efficiently solves SAT, and it is generally believed that no such algorithm exists; yet this belief has not been proven mathematically, and resolving the question whether SAT has an efficient algorithm is equivalent to the P versus NP problem, which is the most famous open problem in the theory of computing.
15
Despite the fact that no algorithms are known that solve SAT efficiently, correctly, and for all possible input instances, many instances of SAT that occur in practice, such as in artificial intelligence, circuit design and automatic theorem proving, can actually be solved rather efficiently using heuristical (intelligent guesswork) SAT-solvers.
Such algorithms are not believed to be efficient on all SAT instances, but experimentally these algorithms tend to work well for many practical applications.
16
Basic definitions and terminology
A propositional logic formula, also called Boolean expression, is built from variables, operators AND (conjunction, also denoted by ), OR (disjunction, ), NOT (negation, ¬), and parentheses. ∧ ∨A formula is said to be satisfiable if it can be made TRUE by assigning appropriate logical values (i.e. TRUE, FALSE) to its variables.
The Boolean satisfiability problem (SAT) is, given a formula, to check whether it is satisfiable.
This decision problem is of central importance in various areas of computer science, including theoretical computer science, complexitytheory, algorithmics, cryptography and artificial intelligence.
17
There are several special cases of the Boolean satisfiability problem in which the formulas are required to have a particular structure.
A literal is either a variable, then called positive literal, or the negation of a variable, then called negative literal. A clause is a disjunction of literals (or a single literal).
A clause is called Horn clause if it contains at most one positive literal.
A formula is in conjunctive normal form (CNF) if it is a conjunction of clauses (or a single clause).
18
For example, "x1" is a positive literal, "¬x2" is a negative literal, "x1 ¬∨ x2" is a clause, and "(x1 ¬∨ x2) (¬∧ x1 ∨ x2 ∨ x3) ¬∧ x1" is a formula in conjunctive normal form, its 1st and 3rd clause are Horn clauses, but its 2nd clause is not.
The formula is satisfiable, choosing x1=FALSE, x2=FALSE, and x3 arbitrarily, since (FALSE ¬FALSE) (¬FALSE FALSE ∨ ∧ ∨ ∨ x3) ¬FALSE ∧evaluates to (FALSE TRUE) (TRUE FALSE ∨ ∧ ∨ ∨ x3) TRUE, and in ∧turn to TRUE TRUE TRUE (i.e. to TRUE). In contrast, the CNF ∧ ∧formula a ¬∧ a, consisting of two clauses of one literal, is unsatisfiable, since for a=TRUE anda=FALSE it evaluates to TRUE ¬TRUE (i.e. to FALSE) ∧and FALSE ¬FALSE (i.e. again to FALSE), respectively.∧
19
The Satisfiability (SAT) Problem
Satisfiability (SAT):» Given a Boolean expression on n variables, can we
assign values such that the expression is TRUE?
» Ex: ((x1 x2) ((x1 x3) x4)) x2
» Seems simple enough, but no known deterministic polynomial time algorithm exists
» Easy to verify in polynomial time!
20
Example: CNF satisfiability This problem is in NP. Nondeterministic algorithm:
» Guess truth assignment» Check assignment to see if it satisfies CNF formula
Example: (A¬B ¬C ) (¬A B) (¬ B D F ) (F ¬ D)
Truth assignments: A B C D E F 0 1 1 0 1 0 1 0 0 0 0 1 1 1 0 0 0 1 ... (how many more?)
Checking phase: Θ(n)
21Complexity©D.Moshkovitz
21
Conclusion: SAT is NP-Complete
For any language A in NP,
testing membership in A
can be reduced to...
satisfiability problem
22Complexity©D.Moshkovitz
22
Co-NP
Revisiting the Map
NPP
NPC
SAT
23Complexity©D.Moshkovitz
23
Looking Forward
From now on, in order to show some NP problem is NP-Complete, we merely need to reduce SAT to it.
any NP problem
can be reduced to...
SAT
new NP problem
can be reduced to...We’ve already shown this
will imply the new problem is in NPC
24Complexity©D.Moshkovitz
24
... and Beyond!
Moreover, every NP-Complete problem we discover, provides us with a new way for showing problems to be NP-Complete.
any NP problem
can be reduced to...
Known NP-hard problem
new NP problem
can be reduced to...
25
26
Review: P And NP Summary
P = set of problems that can be solved in polynomial time» Examples: Fractional Knapsack, …
NP = set of problems for which a solution can be verified in polynomial time» Examples: Fractional Knapsack,…, Hamiltonian Cycle, CNF
SAT, 3-CNF SAT Clearly P NP Open question: Does P = NP?
» Most suspect not» An August 2010 claim of proof that P ≠ NP, by Vinay
Deolalikar, researcher at HP Labs, Palo Alto, has flaws
27
8- 27
Chapter 8
The Theory of NP-Completeness
28
KINDS OF PROBLEMS Decision problems Optimization problems
8- 28
29
8- 29
Decision problems The solution is simply “Yes” or “No”. Optimization problems are more difficult. e.g. the traveling salesperson problem
» Optimization version:
Find the shortest tour
» Decision version:
Is there a tour whose total length is less than or equal to a constant c ?
30
8- 30
Solving an optimization problem by a decision algorithm :
Solving TSP optimization problem by a decision algorithm :» Give c1 and test (decision algorithm)
Give c2 and test (decision algorithm)
Give cn and test (decision algorithm)
» We can easily find the smallest ci
31
8- 31
CATEGORIES OF PROBLEMS
Undecidable problems
Intractable problems
Tractable problems
32
Non deterministic Algorithms
Deterministic Algorithms : Result of every operation is uniquely defined.
Nondeterministic Algorithms : Results of operations are not uniquely defined but are limited to specified sets of possibilities.
To specify nondeterministic algorithms, we introduce three new functions:
Choice(S) – arbitrarily chooses one of the elements of set S. Failure() – signals an unsuccessful completion. Success() – signals a successful completion.
Nondeterministic machine : A machine capable of executing a non
deterministic algorithm.
8- 32
33
Nondeterministic search
j := Choice(1, n);
if A[j] = x then
{
write(j);
success();
}
write(0);
Failure();
8- 33
34
Class P
A decision problem D is solvable in polynomial time or in the class P, if there exists an algorithm A such that
A takes instances of D as inputs. A always outputs the correct answer “Yes” or “No”. There exists a polynomial p such that the execution of A
on inputs of size n always terminates in p(n) or fewer step
8- 34
35
Class NP
A decision problem solvable by non deterministic algorithms in polynomial Time or in the class NP if there exists an algorithm A such that
A takes as inputs potential witnesses for “yes” answers to problem D. A correctly distinguishes true witnesses from false witnesses. There exists a polynomial p such that for each potential witnesses of
each instance of size n of D, the execution of the algorithm A takes at most p(n) steps.
8- 35
36
P and NP
P is the set of all decision problems solvable by deterministic algorithms in polynomial time.
NP is the set of all decision problems solvable by nondeterministic algorithms in polynomial time.
P is subset of NP
8- 36
37
Polynomial Time Reduction
Problem L1 reduces to L2 (L1 α L2) if and only if there is a way to solve L1 by a deterministic polynomial time algorithm using a deterministic algorithm that solves L2 in polynomial time.
If there is a polynomial time algorithm for L2, then we can solve L1 in polynomial time. α is a transitive relation.
Example: The Hamiltonian cycle problem is polynomial-time reducible
to the decision version of TSP.
8- 37
38
NP-Hard and NP-Complete
NP-HARD : Problem P is said to be NP-hard, If all problems Q Є NP, are reducible to P. Informally, P is “as hard as” all the problems in NP.
NP-COMPLETE : Problem P is said to be NP-Complete, If P is NP-hard, and P Є NP.
Informally, P is one of the hardest problems in NP.
If P α R, and P is NP-Complete, then R is also NP-Complete.
8- 38
39
8- 39
P: the class of problems which can be solved by a deterministic polynomial algorithm.
NP : the class of decision problem which can be solved by a non-deterministic polynomial algorithm.
NP-hard: the class of problems to which every NP problem reduces.
NP-complete (NPC): the class of problems which are NP-hard and belong to NP.
40
8- 40
Some concepts of NPC
Definition of reduction: Problem A reduces to problem B (A B) iff A can be solved by a deterministic polynomial time algorithm using a deterministic algorithm that solves B in polynomial time.
Up to now, none of the NPC problems can be solved by a deterministic polynomial time algorithm in the worst case.
It does not seem to have any polynomial time algorithm to solve the NPC problems.
41
8- 41
The theory of NP-completeness always considers the worst case.
The lower bound of any NPC problem seems to be in the order of an exponential function.
Not all NP problems are difficult. (e.g. the MST problem is an NP problem.)
If A, B NPC, then A B and B A.
Theory of NP-completeness:If any NPC problem can be solved in polynomial time, then all NP problems can be solved in polynomial time. (NP = P)
42
8- 42
The satisfiability problem
The satisfiability problem» The logical formula :
x1 v x2 v x3
& - x1
& - x2 the assignment :
x1 ← F , x2 ← F , x3 ← Twill make the above formula true .
(-x1, -x2 , x3) represents x1 ← F , x2 ← F , x3 ← T
43
8- 43
If there is at least one assignment which satisfies a formula, then we say that this formula is satisfiable; otherwise, it is unsatisfiable.
An unsatisfiable formula : x1 v x2
& x1 v -x2
& -x1 v x2
& -x1 v -x2
44
8- 44
Definition of the satisfiability problem: Given a Boolean formula, determine whether this formula is satisfiable or not.
A literal : xi or -xi A clause : x1 v x2 v -x3 Ci A formula : conjunctive normal form (CNF)
C1& C2 & … & Cm
45
8- 45
Nondeterministic algorithms A nondeterminstic algorithm consists of phase 1: guessing phase 2: checking If the checking stage of a nondeterministic algorithm is
of polynomial time-complexity, then this algorithm is called an NP (nondeterministic polynomial) algorithm.
NP problems : (must be decision problems)» e.g. searching, MST
sortingsatisfiability problem (SAT)traveling salesperson problem (TSP)
46
8- 46
Decision problems
Decision version of sorting:
Given a1, a2,…, an and c, is there a permutation of ai
s ( a1, a2
, … ,an ) such that a∣ 2
–a1 + a∣ ∣ 3
–a2 + … ∣
+ a∣ n–an-1
∣< c ? Not all decision problems are NP problems
» E.g. halting problem : Given a program with a certain input data, will the program
terminate or not? NP-hard Undecidable
47
8- 47
Nondeterministic operations and functions
[Horowitz 1998] Choice(S) : arbitrarily chooses one of the elements in set S Failure : an unsuccessful completion Success : a successful completion Nonderministic searching algorithm:
j ← choice(1 : n) /* guessing */
if A(j) = x then success /* checking */
else failure
48
8- 48
A nondeterministic algorithm terminates unsuccessfully iff there does not exist a set of choices leading to a success signal.
The time required for choice(1 : n) is O(1). A deterministic interpretation of a non-deterministic
algorithm can be made by allowing unbounded parallelism in computation.
49
8- 49
Nondeterministic sorting
B ← 0 /* guessing */for i = 1 to n do
j ← choice(1 : n) if B[j] ≠ 0 then failure B[j] = A[i]
/* checking */for i = 1 to n-1 do
if B[i] > B[i+1] then failure success
50
8- 50
Nondeterministic SAT
/* guessing */
for i = 1 to n do
xi ← choice( true, false )
/* checking */
if E(x1, x2, … ,xn) is true then success
else failure
51
8- 51
Cook’s theorem
NP = P iff the satisfiability problem is a P problem.
SAT is NP-complete. It is the first NP-complete
problem. Every NP problem reduces to
SAT.
Stephen Arthur Cook
(1939~)