DESIGN AND ANALYSIS OF ALGORITHMS (DAA 2019) · NP = decision problems that can be verified in O(nk) time k constant n input length (with appropriate encoding) NP-complete = decision

Master’s Programme in Computer Science

Juha Kärkkäinen

Based on slides by Veli Mäkinen

DESIGN AND ANALYSIS OF ALGORITHMS (DAA 2019)

09/10/2019 1 Design and Analysis of Algorithms 2019 week 6

SAT is NP-complete & encodings

Week VI

09/10/2019 Design and Analysis of Algorithms 2019 week 6 2

SAT is NP-complete

Cook-Levin theorem, proof using Turing machine



REDUCTION TREE COVERED IN THIS COURSE

SAT (now)

3CNF-SAT (last week)

CLIQUE (last week) SUBSET-SUM (today)

VERTEX-COVER (last week)

HAM-CYCLE (see the book, this shows how complex reduction gadgets can be)

Independent set (study group)

Multi-LCS (study group)

Inapproximability of TSP (last week) Hamiltonian path (study group)


Unbounded Knapsack (exercises)


RECALL: COMPLEXITY CLASSES

P = decision problems that can be solved in O(nk) time

NP = decision problems that can be verified in O(nk) time

k constant

n input length (with appropriate encoding)

NP-complete = decision problem L s.t.

1. L is in NP

2. L’ ≤p L for every L’ in NP

Design and Analysis of Algorithms 2019 week 5 5

Next

week

09/10/2019


RECALL: BOOLEAN SATISFIABILITY SAT


Example: Φ=((x1˄x2)˅¬x3) (x4→x5)


BACK TO TURING

Proposition: Computation in RAM model can be simulated in

polynomial time with Turing machine, and vice versa. (proof

omitted here)

Corollary. Problem L is in NP iff it can be solved in O(nc) time on

a non-deterministic Turing machine (NDTM), where c is a

constant.



NON-DETERMINISTIC TURING MACHINE (NDTM)

q

p

start state accepting state

a

Head in an input / working tape

Transition: Read ”a” on tape, replace it with ”b”, and

move head to the right (+1), move state p->q


Nondeterministic:

accept input if

there exists a path

from the start state

to an accepting state.


MORE FORMALLY…



THEOREM: SAT IS NP-COMPLETE




a

b

… p(n)

2p(n)




SAT variables: Hi,k = 1 if head is at position i at step k

Qq,k = 1 if state q is active at step k

Ti,c,k = 1 if tape contains symbol c at index i at step k

09/10/2019 Design and Analysis of Algorithms 2019 week 6

a

b

… p(n)

2p(n)

i

k c

…



Initialization (step 0): H0,0=1

Qs,0=1

Ti,c,0=1 if c is initial content of cell i


a

b

… p(n)

2p(n)

i c




a

b

… p(n)

2p(n)




a

b

… p(n)

2p(n)



a

b

… p(n)

2p(n)

c

i

k

blackboard/exercise


˅



a

b

p(n)

2p(n)

c

c’

…

…

k

k+1 p

q

i


˄ ˄



Enforcing ”one head position at a time” creates a O(p(n)3) long

expression, and other parts create shorter expressions.

Thus, an instance of any problem in NP can be casted in

polynomial time to a boolean expression, such that a ”yes”

instance maps into a satisfying variable assignment where the

variables set to ”true” reveal a state-path transition in the NDTM

to an accepting state.




SAT (done)









(Un)bounded Knapsack (exercises)

Encodings

Pseudopolynomiality of dynamic programming solutions to

subset sum and knapsack



ENCODING INPUT P = problems that can be solved in O(nk) time

NP = problems that can be verified in O(nk) time

k constant

n input length

The size of input depends on its encoding:

• Any encoding of polynomial size in n is ok

• Encoding integers can be tricky

• Binary encoding: 6=110=00000110. How many bits to use?

• Unary encoding: 6=0000001=061. Exponentially larger than binary.

• Gamma encoding: 6=0001 110. (0001 is number of bits in unary)


Encoding?


EXAMPLE: HAMILTONIAN CYCLE PROBLEM

Given an undirected graph G=(V,E), is there a cycle visiting each vertex exactly once?

Assume |V|≤|E|.

Encoding:

vertices numbered 1..|V|.

Input = <|E|,(v1,w1),(v2,w2),…, (v|E|,w|E|)>

000000..0 1 bin(|E|) 00..0bin(v1) 00..0bin(w1) …

Input size ~ 2(|E|+1)log |E| bits

Unary encoding for all values, ≤ (2|E|+1)(|V|+1) bits, would be polynomial too.

number of bits in unary 0 0

1 1

2 10

3 11

4 100

5 101

6 110

7 111

…

x bin(x)



EXAMPLE: UNBOUNDED KNAPSACK

Unbounded Knapsack: Given integers W and V and m pairs of (weight,value) item types, is there a collection of items with total weight ≤W and total value ≥V.

In study groups we saw an O(mW) dynamic programming algorithm which makes Knapsack pseudopolynomially solvable: there is a fast algorithm if input parameters are suitably bounded.

However, Unbounded Knapsack is NP-hard (exercise). This does not contradict pseudopolynomiality because of encodings.

• Input can be encoded in O(m log N) bits, where N is the largest integer in input.

• Assume N=W=2m. Then input size is O(m2) bits but the time complexity is O(m2m), which is exponential in input size.



EXAMPLE: SUBSET-SUM

SUBSET-SUM: Given an integer t and a set S of integers, is there

a subset S’ of S with a total sum of exactly t?

SUBSET-SUM is another pseudopolynomially solvable but NP-

complete problem.

• There is an O(|S|t) time dynamic programming algorithm.

• NP-hardness proof by reduction from 3CNF-SAT (blackboard,

book Sect. 34.5.5).




SAT (today)









(Un)bounded Knapsack (exercises)


RECAP: PROVING NP-COMPLETENESS


Prove NP-completeness of L by reduction from L’

1. Prove L is in NP

• Certificate that can be verified in polynomial time

2. Describe conversion from L’ to L

• L’ is known NP-complete problem

• Conversion of input to input, not solution to solution

3. Prove that “yes”-instance maps to “yes”-instance

• Conversion of solution to solution

4. Prove that “no”-instance maps to “no”-instance

• Often proof by contradiction: “yes”-instance maps to “yes”-instance in

opposite direction (which contradicts “no”-instance mapping to “yes”-

instance).


SOLVING NP-HARD PROBLEMS


1. Exact algorithms

• Always optimal solution

• May take exponential time

• Can be practical for small instances and important special cases

2. Approximation algorithms

• Guarantee of being “close” to optimal solution

• Polynomial time

3. Heuristic algorithms

• No guarantee of even close-to-optimality

• Sometimes best one can do in practice

Documents

DESIGN AND ANALYSIS OF ALGORITHMS (DAA 2019) · NP = decision problems that can be verified in O(nk) time k constant n input length (with appropriate encoding) NP-complete = decision