Lecture 17:

NP-Complete

Problems

(Reading: AM&O Appendix B)

Recognition Problems

recognition problem: these problems are char-

acterized by an

instance: describing the input for the prob-

lem, and a

question: a yes-no question concerning a prop-

erty about the given instance.

Example: NODE COVER

Instance: Network G = (N,A), positive in-

teger k

Question: Does G contain a node cover of

size k or less?

7

8

4

65

1 2

3

Instance with k = 5

Optimization vs. Recognition Problems:

Typically the objective function value z is given

explicitly as part of the input, as in NODE

COVER. An optimization problem with opti-

mal solution value z∗ can actually be described

as two recognition problems:

1. Does there exist a feasible solution with

objective function z∗?

2. Does there exist a feasible solution with

objective function value z∗ − ϵ?

Since z∗ and ϵ are usually not known in ad-

vance, it would be necessary to use some sort

of binary search repeatedly asking the above

question in order to determine the optimal value

of z.

Theorem: For a given optimization problem

P, if

(a) The absolute objective function value of

any solution to P is bounded above by M ,

where logM is polynomial in the size of

the input for P;

(b) The absolute difference between the ob-

jective function values of any two solu-

tions to P is bounded below by ϵ, where

log(1/ϵ) is polynomial in the size of the

input to P;

(c) The recognition problem for P can be an-

swered in polynomial time.

then P has a polynomial time solution.

Proof: Use binary search.

Most optimization problems encountered in prac-

tice satisfy (a) and (b).

The Class of NP

polynomial certificate: an object associated

with the instance which, if given, allows

you to verify a “yes” answer in polynomial

time.

example: for NODE COVER, any set of nodes

can be checked as to being a node cover

of size at most k in polynomial (in fact,

linear) time.

NP problem: Any recognition problem that has

an associated polynomial certificate

Problem Reduction

(Turing Machine) reducibility: Let A and Bbe two classes of problems. We say that

A is reducible to B, A ∝ B, if there is a

polynomial-time algorithm that solves any

instance I of A by using as a “subroutine”

a 1-time-unit solution of any (polynomial

size) instance of B.

Special case — problem reducibility:

A ∝ B if for any instance IA of A an

instance IB for problem B can be con-

structed in polynomial time, such that the

answer to the question in A for instance

IA is identical to the answer in B for in-

stance IB.

Problems A and B are polynomially equiva-

lent if A ∝ B and B ∝ A

Example

Consider the problem CLIQUE

Instance: Network G = (N,A), positive in-

teger k

Question: Does there exist a clique (set of

mutually pairwise adjacent set of nodes)

in G of cardinality at least k?

CLIQUE ∝ NODE COVER: For any instance

G = (N,A), k for CLIQUE, construct instance

G′, k′ for NODE COVER by letting G′ be the

complementary graph to G (i.e. G′ has an

edge exactly when G does not have an edge)

and setting k′ = |N | − k. Now consider a node

cover C in G′ of size k′. the complement C̄

of C has no edges in G′ connecting any pair

of nodes in C̄, and therefore the same set C̄ in

G is therefore a clique in G.

7

8

4

65

1 2

3

Problem Reduction

CLIQUE ∝ NODE COVER

NP-Complete Problems

A recognition problem P is called NP-complete

if it is the “hardest” of the NP-problems, that

is

1. P is an NP problem

2. for every NP problem A, A ∝ P.

What this means is that we can use the poly-

nomial time solution to any one NP-complete

problem to solve all of the problems in NP!

But are there any NP-complete problems at

all?

3SAT: The Mother of All

NP-Complete Problems

Instance: set C = C1, C2, . . . , Cm of clauses,

each clause made up of exactly 3 Boolean

variables of the form xj or x̄j = NOTxj,

j ∈ {1, . . . , n}

Question: Does there exist a set of T-F as-

signments to the variables x1, . . . , xn so

that every clause Ci has at least one true

variable?

Example instance:

C = {x1, x̄2, x3}, {x̄1, x̄3, x4}, {x2, x3, x̄4}

Cook’s Theorem: 3SAT is an NP-complete

problem.

Establishing Other NP-complete

Problems

Once we have one NP-complete problem we

can show that any other problem P is NP-

complete by showing

1. P is an NP problem

2. for any one NP-complete problem A,

A ∝ P.

A Sample Reduction

We show NODE COVER is NP-complete. Since

we have already shown that NODE COVER is

in NP, we need now to reduce 3SAT to NODE

COVER.

Let C = C1, . . . , Cm be an instance of 3SAT,

with Ci = {ui1, ui2, ui3}, i = 1, . . . ,m, and uij ∈{x1, x̄1, . . . , xn, x̄n}. The associated graph G is

made up of

“variable gadgets”: edges (xj, x̄j), j = 1, . . . , n

“clause gadgets”: edges (ui1, ui2), (ui2, ui3),

(ui3, ui1), i = 1, . . . ,m

“connector gadgets”: (xj, uil) if uil = xj and

(x̄j, uil) if uil = x̄j.

Finally, set k = n+2m

Example

Associated graph for the 3SAT expression

C = {x1, x̄2, x3}, {x̄1, x̄3, x4}, {x2, x3, x̄4}

x x11

_

u u

u11

12 13

x x_

u u

u

x x_

u u

u

x x_

2 2 3 3 4 4

21

22 23

31

32 33

Node cover of size k = 10 is equivalent to as-

signment satisfying C.

Some Well-Known NP-Complete

Problems

HAMILTONIAN CIRCUIT

Given: Network G = (N,A)

Question: Does there exist a Hamiltonian cir-

cuit in G, that is, a closed walk passing

through each vertex of G exactly once?

3-DIMENSIONAL MATCHING

Given: “tripartite hypergraph” consisting of

disjoint sets X, Y , and Z of n elements

each, and m triples (xi, yi, zi), xi ∈ X,

yi ∈ Y , zi ∈ Z, i = 1, . . . ,m

Question: Does there exist a subset of n triples

such that each element of X ∪ Y ∪ Z ap-

pears in exactly one triple?

PARTITION

Given: Set S of m integers

Question: Does there exist a subset A of S

such that ∑x∈A

x =∑

y∈S\Ay ?

Optimization Problems

TRAVELING SALESMAN

Given: set N of “cities”, integer “distances”

w(i, j), i, j ∈ N , and integer “tour target”

W

Question: Does there exist “traveling sales-

man tour”, that is, an ordering v1, v2, . . . , vn

of the elements of N such that

w(v1, v2)+. . .+w(vn−1, vn)+w(vn, v1) ≤ W?

KNAPSACK

Given: “item weights” a1, . . . , an, “item val-

ues” c1, . . . , cn, and “knapsack weight limit”

W and “target value” V

Question: Does there exist a set of items i1, . . . , im

whose total weight is at most W and whose

total value is at least V ?

QUADRATIC PROGRAMMING

Given: System of integer-coefficient inequali-

ties

a11x1 + . . . + a1nxn ≤ b1(∗) ... ... ...

am1x1 + . . . + amnxn ≤ bm

in the variables x1, . . . , xn, integer cost co-

efficients ci, i = 1, . . . , n and qij, i, j =

1, . . . , n, and integer target value Z

Question: Does there exist a set of values

x̂1, . . . , x̂n for x1, . . . , xn satisfying (∗) such

that∑

i,j qijx̂ixj +∑

j cjx̂j ≥ Z?

Hard part: proving this is in NP.

Interesting note: If the values x̂1, . . . , x̂n are

required in addition to be integer, then

there is provably no algorithm which can

solve this problem!

NP-hard Problems

A problem P is called NP-hard if only thesecond part of the NP-complete definition istrue, namely, some known NP-complete prob-lem can be (Turing-machine) reduced to them.

• optimization problems, or problems requir-ing an explicit solution: These are oftenonly NP-hard in a technical sense, sincethey are generally polynomially equivalentto the recognition-problem versions.

• A truly NP-hard problem:

COUNTING (s, t)-PATHS

Given: Directed graph G = (N,A), nodess and t.

Question: How many (s, t)-paths are there?

This is in the important class of #P-complete

problems, of which a large number of re-liability and stochastic problems lie.

A Weird Problem

EUCLIDEAN SPANNING TREE

Given: Set V = {(x1, y1), . . . , (xn.yn)} of integer-

coordinate points in the plane, integer tar-

get value Z

Question: Does there exist a spanning tree

for V whose total length is at most Z?

Optimization problem: Find the minimum cost

spanning tree for V

optimization problem has a polynomial-time

solution

recognition problem not known to be in NP!

(Due to difficulty in comparing sums of

square roots.)

Coping with NP-hard Problems

Relaxations: Polynomial-time algorithms that find

a solution that satisfy some subset of the

conditions for feasibility. An optimal solution

for relaxed problem that happens also to sat-

isfy the remaining conditions will be optimal

to the original problem, and in any case its ob-

jective function value will always constitute

lower bound on the optimal solution value.

Example: The following linear program is a relax-

ation for the Traveling Salesman Problem:

min z =∑

(i,j)∈Acijxij∑

j∈A(i)

xij = 2, i ∈ N

0 ≤ xij ≤ 1, (i, j) ∈ A

It produces a 0-1 solution that identifies a dis-

joint set of tours covering all cities at mini-

mum cost.

Algorithms for Special Cases: Sometimes poly-

nomial-time algorithms can be found to solve

the problem when the particular instance has

special structure. Example: NODE COVER

can be solved in polynomial time on bipartite

graphs.

Guaranteed Approximations: These are poly-

nomial-time algorithms that can guarantee in

advance that the objective function value of

the resulting solution will be sufficiently close

to the actual optimal value. In particular, Let

P be a problem class whose solutions all have

positive objective values, and let ρ ≥ 1 be a

given ratio. An algorithm is called a ρ-approxi-

mation for P if, for every instance I of P hav-

ing optimal solution value z∗, the algorithm will

find feasible solution X0 for I whose objective

function value z0 satisfies

z0

z∗≤ ρ

Example: A 2-Approximation to

NODE COVER

Input: Graph G = (N,A)

Output: A node cover for G which is at most twiceas large as the minimum cardinality node cover.

Procedure:

do while there are edges left in G

1. Choose edge e = (u, v) in G

2. Place both u and v into the cover3. Remove u and v (together with all

adjacent edges) from G.

Proof of approximation: The set of nodes cho-sen clearly constitute a cover for G. Further,the edges chosen in the procedure constitutea matching in G, and hence any node covermust have at least one vertex on each edge ofthis matching. It follows that the cover chosenis at most twice the size of any vertex cover inG.

A Non-Approximable Problem

TRAVELING SALESMAN (optimization ver-

sion) has no ρ-approximation for any ρ (unless

P = NP ). For let G = (N,A) be any instance

for HAMILTONIAN CIRCUIT. Construct TSP

instance by giving each pair i and j of vertices

distance

w(i, j) =

{1 (i, j) ∈ A

nρ+2 otherwise

Then a Hamiltonian circuit in G will have TSP

circuit value n, and any other ordering of nodes

will have TSP circuit value at least nρ + 2 +

(n− 1) > nρ. Thus

z0

z∗> ρ

So that the only ρ-approximations for the TSP

are the Hamiltonian circuits in G.

Note that TSP does has a 3/2-approximation

when the distances satisfy the triangle inequal-

ity.

Linear-Programming-Based

Heuristics for Solving NP-hard

Optimization Problems

Some of the most powerful heuristic for solv-

ing NP-hard problems involve solving a series

of linear programs which either find an opti-

mal solution to the problem or give more ac-

curate information about where such solutions

can be found. We give the highlights common

to many of these procedures.

1. Relaxation: Formulate the problem as an in-

teger linear program and then drop the in-

tegrality conditions.

2. If the optimal LP solution happens to be inte-

ger, then it will be the optimal solution to the

original problem. If not, its objective func-

tion value constitutes a lower bound on the

optimal solution value.

3. Heuristic #1: If the current solution is non-

integral, then we can still use this solution to

construct a solution to the problem. This can

be done by rounding the solution to an inte-

ger, or looking at the structure of the solution

to guess at a feasible solution to the original

problem. This solution, together with the LP

lower bound, gives a guaranteed approxima-

tion (though specific to the instance) on the

goodness of this approximation.

4. To try to obtain a better solution to the prob-

lem, we need to restrict the LP.

Heuristic #2: Branch-and-bound. Choose a

nonintegral component of the solution, and

split the problem by forcing that variable

up or down to its next integer value. The

best solution for these two subproblems will

be the solution for the original problem. The

relaxation bound in 3 can be used to deter-

mine the appropriate subproblems to solve.

Heuristic #3: Cutting planes. For many in-

dividual classes of problems, sophisticated

inequalities have been developed which are

significantly stronger than those defining the

original LP. They are specialized and hard to

find, but often placing enough of them on a

particular problem will result in the LP so-

lution becoming integer, and hence again

optimal.

Heuristic #4: Lagrangian relaxation (Chapter 16).

This is another relaxation technique, which re-

moves linear constraints which are difficult to

deal with, and places them in the objective

function, using Lagrangian multipliers to de-

termine how much weight to give to solutions

that violate these constraints. What remains

are the “easy” constraints (e.g., a simple min

cost flow problem), for which integer solutions

can be found quickly. The Lagrangian objec-

tive function provides a lower bound on the

actual objective value, and the trick is to find

the multipliers which give you the best lower

bound, or the best possibility of a feasible (and

hence optimal) solution.

Problem-specific LP-based heuristics have been

developed for a large number of important NP-

hard problems.