View
215
Download
0
Category
Preview:
Citation preview
Lecture 17:
NP-Complete
Problems
(Reading: AM&O Appendix B)
Recognition Problems
recognition problem: these problems are char-
acterized by an
instance: describing the input for the prob-
lem, and a
question: a yes-no question concerning a prop-
erty about the given instance.
Example: NODE COVER
Instance: Network G = (N,A), positive in-
teger k
Question: Does G contain a node cover of
size k or less?
7
8
4
65
1 2
3
Instance with k = 5
Optimization vs. Recognition Problems:
Typically the objective function value z is given
explicitly as part of the input, as in NODE
COVER. An optimization problem with opti-
mal solution value z∗ can actually be described
as two recognition problems:
1. Does there exist a feasible solution with
objective function z∗?
2. Does there exist a feasible solution with
objective function value z∗ − ϵ?
Since z∗ and ϵ are usually not known in ad-
vance, it would be necessary to use some sort
of binary search repeatedly asking the above
question in order to determine the optimal value
of z.
Theorem: For a given optimization problem
P, if
(a) The absolute objective function value of
any solution to P is bounded above by M ,
where logM is polynomial in the size of
the input for P;
(b) The absolute difference between the ob-
jective function values of any two solu-
tions to P is bounded below by ϵ, where
log(1/ϵ) is polynomial in the size of the
input to P;
(c) The recognition problem for P can be an-
swered in polynomial time.
then P has a polynomial time solution.
Proof: Use binary search.
Most optimization problems encountered in prac-
tice satisfy (a) and (b).
The Class of NP
polynomial certificate: an object associated
with the instance which, if given, allows
you to verify a “yes” answer in polynomial
time.
example: for NODE COVER, any set of nodes
can be checked as to being a node cover
of size at most k in polynomial (in fact,
linear) time.
NP problem: Any recognition problem that has
an associated polynomial certificate
Problem Reduction
(Turing Machine) reducibility: Let A and Bbe two classes of problems. We say that
A is reducible to B, A ∝ B, if there is a
polynomial-time algorithm that solves any
instance I of A by using as a “subroutine”
a 1-time-unit solution of any (polynomial
size) instance of B.
Special case — problem reducibility:
A ∝ B if for any instance IA of A an
instance IB for problem B can be con-
structed in polynomial time, such that the
answer to the question in A for instance
IA is identical to the answer in B for in-
stance IB.
Problems A and B are polynomially equiva-
lent if A ∝ B and B ∝ A
Example
Consider the problem CLIQUE
Instance: Network G = (N,A), positive in-
teger k
Question: Does there exist a clique (set of
mutually pairwise adjacent set of nodes)
in G of cardinality at least k?
CLIQUE ∝ NODE COVER: For any instance
G = (N,A), k for CLIQUE, construct instance
G′, k′ for NODE COVER by letting G′ be the
complementary graph to G (i.e. G′ has an
edge exactly when G does not have an edge)
and setting k′ = |N | − k. Now consider a node
cover C in G′ of size k′. the complement C̄
of C has no edges in G′ connecting any pair
of nodes in C̄, and therefore the same set C̄ in
G is therefore a clique in G.
7
8
4
65
1 2
3
Problem Reduction
CLIQUE ∝ NODE COVER
NP-Complete Problems
A recognition problem P is called NP-complete
if it is the “hardest” of the NP-problems, that
is
1. P is an NP problem
2. for every NP problem A, A ∝ P.
What this means is that we can use the poly-
nomial time solution to any one NP-complete
problem to solve all of the problems in NP!
But are there any NP-complete problems at
all?
3SAT: The Mother of All
NP-Complete Problems
Instance: set C = C1, C2, . . . , Cm of clauses,
each clause made up of exactly 3 Boolean
variables of the form xj or x̄j = NOTxj,
j ∈ {1, . . . , n}
Question: Does there exist a set of T-F as-
signments to the variables x1, . . . , xn so
that every clause Ci has at least one true
variable?
Example instance:
C = {x1, x̄2, x3}, {x̄1, x̄3, x4}, {x2, x3, x̄4}
Cook’s Theorem: 3SAT is an NP-complete
problem.
Establishing Other NP-complete
Problems
Once we have one NP-complete problem we
can show that any other problem P is NP-
complete by showing
1. P is an NP problem
2. for any one NP-complete problem A,
A ∝ P.
A Sample Reduction
We show NODE COVER is NP-complete. Since
we have already shown that NODE COVER is
in NP, we need now to reduce 3SAT to NODE
COVER.
Let C = C1, . . . , Cm be an instance of 3SAT,
with Ci = {ui1, ui2, ui3}, i = 1, . . . ,m, and uij ∈{x1, x̄1, . . . , xn, x̄n}. The associated graph G is
made up of
“variable gadgets”: edges (xj, x̄j), j = 1, . . . , n
“clause gadgets”: edges (ui1, ui2), (ui2, ui3),
(ui3, ui1), i = 1, . . . ,m
“connector gadgets”: (xj, uil) if uil = xj and
(x̄j, uil) if uil = x̄j.
Finally, set k = n+2m
Example
Associated graph for the 3SAT expression
C = {x1, x̄2, x3}, {x̄1, x̄3, x4}, {x2, x3, x̄4}
x x11
_
u u
u11
12 13
x x_
u u
u
x x_
u u
u
x x_
2 2 3 3 4 4
21
22 23
31
32 33
Node cover of size k = 10 is equivalent to as-
signment satisfying C.
Some Well-Known NP-Complete
Problems
HAMILTONIAN CIRCUIT
Given: Network G = (N,A)
Question: Does there exist a Hamiltonian cir-
cuit in G, that is, a closed walk passing
through each vertex of G exactly once?
3-DIMENSIONAL MATCHING
Given: “tripartite hypergraph” consisting of
disjoint sets X, Y , and Z of n elements
each, and m triples (xi, yi, zi), xi ∈ X,
yi ∈ Y , zi ∈ Z, i = 1, . . . ,m
Question: Does there exist a subset of n triples
such that each element of X ∪ Y ∪ Z ap-
pears in exactly one triple?
PARTITION
Given: Set S of m integers
Question: Does there exist a subset A of S
such that ∑x∈A
x =∑
y∈S\Ay ?
Optimization Problems
TRAVELING SALESMAN
Given: set N of “cities”, integer “distances”
w(i, j), i, j ∈ N , and integer “tour target”
W
Question: Does there exist “traveling sales-
man tour”, that is, an ordering v1, v2, . . . , vn
of the elements of N such that
w(v1, v2)+. . .+w(vn−1, vn)+w(vn, v1) ≤ W?
KNAPSACK
Given: “item weights” a1, . . . , an, “item val-
ues” c1, . . . , cn, and “knapsack weight limit”
W and “target value” V
Question: Does there exist a set of items i1, . . . , im
whose total weight is at most W and whose
total value is at least V ?
QUADRATIC PROGRAMMING
Given: System of integer-coefficient inequali-
ties
a11x1 + . . . + a1nxn ≤ b1(∗) ... ... ...
am1x1 + . . . + amnxn ≤ bm
in the variables x1, . . . , xn, integer cost co-
efficients ci, i = 1, . . . , n and qij, i, j =
1, . . . , n, and integer target value Z
Question: Does there exist a set of values
x̂1, . . . , x̂n for x1, . . . , xn satisfying (∗) such
that∑
i,j qijx̂ixj +∑
j cjx̂j ≥ Z?
Hard part: proving this is in NP.
Interesting note: If the values x̂1, . . . , x̂n are
required in addition to be integer, then
there is provably no algorithm which can
solve this problem!
NP-hard Problems
A problem P is called NP-hard if only thesecond part of the NP-complete definition istrue, namely, some known NP-complete prob-lem can be (Turing-machine) reduced to them.
• optimization problems, or problems requir-ing an explicit solution: These are oftenonly NP-hard in a technical sense, sincethey are generally polynomially equivalentto the recognition-problem versions.
• A truly NP-hard problem:
COUNTING (s, t)-PATHS
Given: Directed graph G = (N,A), nodess and t.
Question: How many (s, t)-paths are there?
This is in the important class of #P-complete
problems, of which a large number of re-liability and stochastic problems lie.
A Weird Problem
EUCLIDEAN SPANNING TREE
Given: Set V = {(x1, y1), . . . , (xn.yn)} of integer-
coordinate points in the plane, integer tar-
get value Z
Question: Does there exist a spanning tree
for V whose total length is at most Z?
Optimization problem: Find the minimum cost
spanning tree for V
optimization problem has a polynomial-time
solution
recognition problem not known to be in NP!
(Due to difficulty in comparing sums of
square roots.)
Coping with NP-hard Problems
Relaxations: Polynomial-time algorithms that find
a solution that satisfy some subset of the
conditions for feasibility. An optimal solution
for relaxed problem that happens also to sat-
isfy the remaining conditions will be optimal
to the original problem, and in any case its ob-
jective function value will always constitute
lower bound on the optimal solution value.
Example: The following linear program is a relax-
ation for the Traveling Salesman Problem:
min z =∑
(i,j)∈Acijxij∑
j∈A(i)
xij = 2, i ∈ N
0 ≤ xij ≤ 1, (i, j) ∈ A
It produces a 0-1 solution that identifies a dis-
joint set of tours covering all cities at mini-
mum cost.
Algorithms for Special Cases: Sometimes poly-
nomial-time algorithms can be found to solve
the problem when the particular instance has
special structure. Example: NODE COVER
can be solved in polynomial time on bipartite
graphs.
Guaranteed Approximations: These are poly-
nomial-time algorithms that can guarantee in
advance that the objective function value of
the resulting solution will be sufficiently close
to the actual optimal value. In particular, Let
P be a problem class whose solutions all have
positive objective values, and let ρ ≥ 1 be a
given ratio. An algorithm is called a ρ-approxi-
mation for P if, for every instance I of P hav-
ing optimal solution value z∗, the algorithm will
find feasible solution X0 for I whose objective
function value z0 satisfies
z0
z∗≤ ρ
Example: A 2-Approximation to
NODE COVER
Input: Graph G = (N,A)
Output: A node cover for G which is at most twiceas large as the minimum cardinality node cover.
Procedure:
do while there are edges left in G
1. Choose edge e = (u, v) in G
2. Place both u and v into the cover3. Remove u and v (together with all
adjacent edges) from G.
Proof of approximation: The set of nodes cho-sen clearly constitute a cover for G. Further,the edges chosen in the procedure constitutea matching in G, and hence any node covermust have at least one vertex on each edge ofthis matching. It follows that the cover chosenis at most twice the size of any vertex cover inG.
A Non-Approximable Problem
TRAVELING SALESMAN (optimization ver-
sion) has no ρ-approximation for any ρ (unless
P = NP ). For let G = (N,A) be any instance
for HAMILTONIAN CIRCUIT. Construct TSP
instance by giving each pair i and j of vertices
distance
w(i, j) =
{1 (i, j) ∈ A
nρ+2 otherwise
Then a Hamiltonian circuit in G will have TSP
circuit value n, and any other ordering of nodes
will have TSP circuit value at least nρ + 2 +
(n− 1) > nρ. Thus
z0
z∗> ρ
So that the only ρ-approximations for the TSP
are the Hamiltonian circuits in G.
Note that TSP does has a 3/2-approximation
when the distances satisfy the triangle inequal-
ity.
Linear-Programming-Based
Heuristics for Solving NP-hard
Optimization Problems
Some of the most powerful heuristic for solv-
ing NP-hard problems involve solving a series
of linear programs which either find an opti-
mal solution to the problem or give more ac-
curate information about where such solutions
can be found. We give the highlights common
to many of these procedures.
1. Relaxation: Formulate the problem as an in-
teger linear program and then drop the in-
tegrality conditions.
2. If the optimal LP solution happens to be inte-
ger, then it will be the optimal solution to the
original problem. If not, its objective func-
tion value constitutes a lower bound on the
optimal solution value.
3. Heuristic #1: If the current solution is non-
integral, then we can still use this solution to
construct a solution to the problem. This can
be done by rounding the solution to an inte-
ger, or looking at the structure of the solution
to guess at a feasible solution to the original
problem. This solution, together with the LP
lower bound, gives a guaranteed approxima-
tion (though specific to the instance) on the
goodness of this approximation.
4. To try to obtain a better solution to the prob-
lem, we need to restrict the LP.
Heuristic #2: Branch-and-bound. Choose a
nonintegral component of the solution, and
split the problem by forcing that variable
up or down to its next integer value. The
best solution for these two subproblems will
be the solution for the original problem. The
relaxation bound in 3 can be used to deter-
mine the appropriate subproblems to solve.
Heuristic #3: Cutting planes. For many in-
dividual classes of problems, sophisticated
inequalities have been developed which are
significantly stronger than those defining the
original LP. They are specialized and hard to
find, but often placing enough of them on a
particular problem will result in the LP so-
lution becoming integer, and hence again
optimal.
Heuristic #4: Lagrangian relaxation (Chapter 16).
This is another relaxation technique, which re-
moves linear constraints which are difficult to
deal with, and places them in the objective
function, using Lagrangian multipliers to de-
termine how much weight to give to solutions
that violate these constraints. What remains
are the “easy” constraints (e.g., a simple min
cost flow problem), for which integer solutions
can be found quickly. The Lagrangian objec-
tive function provides a lower bound on the
actual objective value, and the trick is to find
the multipliers which give you the best lower
bound, or the best possibility of a feasible (and
hence optimal) solution.
Problem-specific LP-based heuristics have been
developed for a large number of important NP-
hard problems.
Recommended