Time Complexity (2)cobweb.cs.uga.edu/~potter/theory/7_time_complexity_II.pdfCSCI 2670 Time Complexity (2) A tableau A computation history of N on input w. CSCI 2670 Time Complexity

Time Complexity (2)

CSCI 2670

Original Slides were written by Dr. Frederick W Maier

Fall 2013

CSCI 2670 Time Complexity (2)

Polynomial Time Reducibility

I In Chapter 5, we defined the concept of mapping reducibility:

I A is mapping reducible to B, written A ≤m B, if and only if there is acomputable function f such that w ∈ A if and only if f (w) ∈ B.

I A function f is computable if there is a Turing machine that takes w asinput and halts with f (w) on its tape.

I This doesn’t consider the time it takes to compute f (w).

I But when talking about time complexity, this is important.

I Suppose A ≤m B and B ∈ P. This doesn’t imply A ∈ P.

I Another sort of reduction would: polynomial time reducibility.



Definition

A function f : Σ∗ → Σ∗ is a polynomial time computable function ifsome polynomial time Turing machine M exists that halts with just f (w)on its tape, when started on any input w .

Definition

Language A is polynomial time (mapping) reducible to language B,written A ≤P B, if there exists a polynomial time computable functionf : Σ∗ → Σ∗ such that for all w ,

w ∈ A if and only if f (w) ∈ B.

The function f is a polynomial time reduction of A to B.



Theorem

If A ≤P B and B ∈ P, then A ∈ P.

Proof.

Suppose A ≤P B and B ∈ P. Since A ≤P B, there is a polynomial timecomputable function f such that w ∈ A iff f (w) ∈ B. Since B ∈ P, ithas a polynomial time decider M. We construct a decider N for A.

On input w :

I Use f to convert w to f (w).

I Run M on f (w). Accept if M accepts/Reject if M rejects.

Computing f (w) takes a polynomial amount of time relative to the sizeof w . And running M with f (w) as input takes a polynomial amount oftime relative to f (w). The composition of two polynomials is stillpolynomial, so N runs in polynomial time.


The SAT Problem

The satisfiability problem, or SAT, is of central importance whendiscussing NP and polynomial time reducibility.

SAT = {φ|φ is a satisfiable Boolean formula}.

I Here we assume φ is a formula using only propositional variables, ∧(AND), ∨ (OR), and ¬ (NOT).

I φ is satisfiable iff there is a truth value assignment (TVA) to the variablessuch that φ evaluates to true.

I SAT is clearly in NP–A TVA satisfying φ can be used as certificate.

I Given a TVA, we can quickly determine whether the TVA makes φ true.

I E.g., {x = 1, y = 0, z = 0} makes (x ∨ ¬x) ∧ (¬y ∧ ¬z) true.

I But no TVA makes (x ∧ ¬x) true, and so (x ∧ ¬x) /∈ SAT .


In-class Questions???

Explain your answer for each question:

1. Is the following formula satisfiable?(x ∨ y) ∧ (x ∨ y) ∧ (x ∨ y) ∧ (x ∨ y) No

2. Is (a1 ∨ a2 ∨ a3 ∨ a4) = (a1 ∨ a2 ∨ y) ∧ (y ∨ a3 ∨ a4)? No

3. Is (a1 ∨ a2 ∨ a3 ∨ a4 ∨ a5) = (a1 ∨ a2 ∨ y1)∧ (y1 ∨ a3 ∨ y2)∧ (y2 ∨ a4 ∨ a5)?No

I Note for question 2 and 3, they are not correct equations, however, basedon the polynomial time reducibility, the boolean formula on the left sidecan be replaced by the formula on the right side with extra literals,because we can easily verify that the new formula on the right issatisfiable iff the original formula on the left was.


3SAT ≤P CLIQUE

Definition

A formula is in 3CNF (conjunctive normal form) if it has the form

(x1 ∨ x2 ∨ x3) ∧ (x3 ∨ x4 ∨ x6) ∧ . . . ∧ (xn−2 ∨ xn1 ∨ xn)

where each xi is a literal (a propositional variable or its negation); eachvariable might appear multiple times. A disjunction of literals is a clause.

3SAT = {φ|φ is a satisfiable Boolean formula in 3-CNF}.

I As an example of polynomial reducibility, we will show the following.

Theorem

3SAT ≤P CLIQUE .


3SAT ≤P CLIQUE

Theorem

3SAT ≤P CLIQUE .

I We map a 3CNF formula φ with k clauses to 〈G , k〉, where G is a graph.

I Each literal in φ becomes a node of G .

I Nodes are grouped into triples—each representing a clause.

I Edges exist between all pairs of nodes, with the following exceptions.

I No edge exists between nodes in the same triple.I No edge connects a node labelled x to the one labelled with x .

I φ is satisfiable iff G has a k-clique.

I If φ satisfiable, we can pick a literal from each clause to be true.

I If we pick a literal from each clause, and no contradictory pairs exist, then thecorresponding node set constitutes a k-clique.

I In the reverse direction, any k-clique corresponds to a consistent set of kliterals, each literal from a distinct clause.


3SAT ≤P CLIQUE

Map φ below to a graph G .

(x1 ∨ x1 ∨ x2) ∧ (x1 ∨ x2 ∨ x2) ∧ (x1 ∨ x2 ∨ x2)

I Each literal in φ becomes a node of G .

I Nodes are grouped into triples—each representing a clause.

I Edges exist between all pairs of nodes, with the following exceptions.

I No edge exists between nodes in the same triple.I No edge connects a node labelled x to the one labelled with x .


3SAT ≤P CLIQUE

I Note that the reduction of φ to 〈G , k〉 is straightforward.

I It can clearly be done in polynomial time relative to the size of φ.

I Consider representing φ above as: $x1x1x2#·x1

·x2

·x2#

·x1x2x2$.

I Given the reduction, if CLIQUE is in P (is it???), then so is 3SAT.



Show how the following 3SAT formula φ can be reduced to acorresponding graph G , so that φ is satisfiable if and only if G has ak-clique, where k is the number of clauses in φ.

φ = (x1 ∨ x2 ∨ x3) ∧ (x1 ∨ x2 ∨ x3) ∧ (x1 ∨ x2 ∨ x3)


NP-Complete Problems

I SAT and CLIQUE belong to a special class called NP-Complete problems.

Definition

A language B is NP-Complete if

1. B ∈ NP, and

2. for any language A, if A ∈ NP, then A ≤P B

I If 2 holds, then B is said to be NP-hard.

I A direct result of this definition (and ≤P) is the following.

Theorem

If B is NP-complete and B ∈ P, then P = NP.



Theorem

If B is NP-complete and B ∈ P, then P = NP.

I If any NP-complete problem is decidable in polynomial time, all NPproblems are.

I The development of NP-Completeness is very significant.

I To attack the P = NP problem, we look at NP-complete problems.

I If we find a polynomial algorithm for an NP-complete problem, we knowP=NP.

I Also, if we prove that a problem is NP-complete, we know that it isunlikely to have a P-time algorithm (because no one’s found one for anyNP-Complete problem).



Theorem

If B is NP-complete, C ∈ NP, and B ≤P C , then C is NP-Complete.

Proof?



Theorem


I C ∈ NP.

I Since B is NP-complete, then for all A ∈ NP, A ≤P B.

I And B ≤P C .

I So A ≤P C (polynomial functions are closed under composition).



I The theory of NP-Completeness was developed in the 1970s.

I Stephen Cook is one of the original developers.

I Once we have one NP-complete problem, we may obtain others bypolynomial time reduction from it.

I However, establishing the first NP-complete problem is more difficult.

I The proof that SAT is NP-complete is called “The Cook-Levin Theorem”after Cook and Leonid Levin.


The Cook-Levin Theorem

Theorem

SAT is NP-complete. Equivalently, SAT ∈ P iff P = NP.

I First, to show that SAT is in NP. (not difficult)

I The hard part is to show that any language in NP is polynomial timereducible to SAT .

First, we show that SAT is in NP. Construct a polynomial time NTM N forSAT as follows.N = “On input w of length n:

1. Nondeterministically generate an assignment to a given formula φ.

2. If the assignment satisfies φ, accept; otherwise, reject.”



Theorem

SAT is NP-complete. Equivalently, SAT ∈ P iff P = NP.

I Let A be any problem in the class NP.

I As such, A has a decider NTM N running in time nk for someconstant k .

I Since N runs in time nk , N goes through at most nk configurationswhen determining whether to accept/reject w .

I Similarly, the configurations are of size at most nk (even if N movesright every time, it can’t use more tape than that).

I As such, we can represent the computation history of N on anyinput w as an nk × nk table (called a tableau).


A tableau

A computation history of N on input w .



I There’s a 1-1 correspondence between the computation histories and thetableaus.

I So, if there’s an accepting computation history for N on w , then there’sa corresponding accepting tableau.

I The proof idea is to map input w (with N fixed) to a Boolean φ such thatφ is satisfiable if and only if an accepting tableau exists.

I But how do we construct φ?



I The variables of φ are based on the contents of a tableau cell.I For each position i , j and each tape symbol s, variable xi,j,s represents

whether or not s is in cell[i,j].I xi,j,s = 1 iff s ∈ cell [i , j ].

I Each cell (i,j) of a tableau should contain exactly one tape symbol. Toensure this holds in φ, we use a formula of the form

(xi,j,s1 ∨ xi,j,s2 ∨ . . . xi,j,sn ) ∧ (∧s 6=t

(xi,j,s ∨ xi,j,t))

I The first part ensures at least one symbol s1, s2, . . ., is in cell (i,j).I The second part ensures at most one symbol is in cell (i,j).I Call the conjunction of such formulas (one for each i , j)

φcell =∧

1≤i ,j≤nk

[(xi ,j ,s1∨ xi ,j ,s2

∨ . . . xi ,j ,sn) ∧

(∧s 6=t

(xi ,j ,s ∨ xi ,j ,t))]

I φcell is true if and only if exactly one symbol appears in each cell.



We construct additional subformulas: φstart , φmove and φaccept .

I φstart is a subformula of φ encoding the starting configuration:

[x1,1,# ∧ x1,2,q0 ∧ x1,3,w1 ∧ x1,3,w2 . . . ∧ x1,n+2,wn . . . ∧ x1,nk ,#]

I φstart is true iff row 1 of the tableau is the start configuration.

I φaccept (below) is true iff the accept state appears somewhere in thetableau:

[x1,1,qaccept ∨ x1,2,qaccept ∨ . . . ∨ xnk ,nk ,qaccept]

I φmove is used to guarantee that each row of the table corresponds to aconfiguration that legally follows the preceding row’s configurationaccording to N’s rules. It does so by ensuring that each 2× 3 “window”of cells is legal.



I Each 2× 3 “window” of cells is legal if that window does not violate theactions specified by N’s transition function.

I For example, a, b, c ∈ Γ, and q1, q2 ∈ Q of N.

I N’s transition functions: δ(q1, a) = {(q1, b,R)} andδ(q1, b) = {(q2, c , L), (q2, a,R)}.

I These are examples of legal windows.




I For example, a, b, c ∈ Γ, and q1, q2 ∈ Q of N.

I N’s transition functions: δ(q1, a) = {(q1, b,R)} andδ(q1, b) = {(q2, c , L), (q2, a,R)}.

I These are examples of illegal windows.




I These window formulas assert that the block is consistent with δ; theyencode legal moves and portions of configurations unaffected by moves.

I We must have a window formula for each 6 cell block of the tableau.

I Call the collection of the window formulas φmove . (we skip the details).


A tableau

Given the start configuration, the valid window values are constrained.



I Given the previous constructions, φ is

φcell ∧ φstart ∧ φmove ∧ φaccept .

I The claim is that if φ is satisfiable, then there exists a tableau that

1. begins with the start configuration,2. ends in an accepting configuration, and3. each row of the tableau follows from the previous one according to the

transition function of N.

I If φ is not satisfiable, no such tableau exists.

I Crucially, φ can be constructed in polynomial time relative to |w |.We skip these details....



Theorem


Theorem

SAT is NP-complete.

I Every NP problem is polynomial time reducible to SAT .

I Equivalently, SAT ∈ P iff P = NP.

I To show that a language is NP-Complete, our general strategy is toexhibit a polynomial time reduction from one already known to beNP-complete language to the language in question.



Theorem

3SAT is NP-complete.

I The proof that SAT is NP-Complete can be modified to show that 3SATis NP-complete.

I It is known that a formula φ can be converted in polynomial time to aequisatisifiable 3CNF formula φ′.

I Equisatisifiable means that φ is satisfiable if and only if φ′ is satisfiable.

I In general, if the clause contains k literals, (x1 ∨ x2 ∨ · · · ∨ xk), we canreplace it with the (k − 2) clauses(x1 ∨ x2 ∨ z1) ∧ (z1 ∨ x3 ∨ z2) ∧ (z2 ∨ x4 ∨ z3) ∨ · · · ∨ (zk−3 ∨ xk−1 ∨ xk)


Other NP-Complete Problems

I Let G = (V ,E) be an undirected graph.

I C ⊆ V is a vertex cover of G iff for each {u, v} ∈ E , u ∈ C or v ∈ C .

I VERTEX-COVER = {(G , k)|G is an undirected graph with a k-nodevertex cover}.

Theorem

VERTEX-COVER is NP-Complete.

I VERTEX-COVER is in NP: A certificate would be a set of k nodes. We canquickly verify that the set covers all the edges of G .

I To prove VERTEX-COVER NP-hard, we reduce 3SAT to it:

I Given a 3CNF formula φ, we produce a graph and integer k such that φ issatisfiable if and only if G has a k-node vertex cover.


VERTEX-COVER is NP-Complete

I VERTEX-COVER is in NP: A certificate would be a set of k nodes (here,shown in red).

I We can quickly verify that the set covers the edges of G .

I Given a 3CNF formula φ, we produce a graph and integer k such that φ issatisfiable if and only if G has a k-node vertex cover.



(Converting φ to G —in polynomial time)

I For each variable x , add two nodes vx and vx and an edge {vx , vx}to G . Call these “variable gadgets”.

I In any cover, either vx or vx must appear (because of the edge).This corresponds to either vx or vx being true.

I For each clause c with literals x ,y ,z , add nodes cx ,cy ,cz and edges{cx , cy}, {cx , cz}, and {cy , cz} to G . I.e., add a triangle withvertices cx ,cy ,cz . Call these triangles “clause gadgets”.

I For each literal x , connect each node cx from a clause gadget to thecorresponding node vx in the variable gadget.

I For example, φ = (x1 ∨ x1 ∨ x2) ∧ (x1 ∨ x2 ∨ x2) ∧ (x1 ∨ x2 ∨ x2)



I Graph with variable gadgets (top) and clause gadgets (bottom).



Theorem

Let φ be a 3SAT formula of m variables and l clauses, and let G be itscorresponding graph.φ is satisfiable iff G has a vertex cover of k = m + 2l nodes.

(Left to Right)

I Suppose there is an assignment that satisfies φ. For each true literal x ,add vx to the cover. This takes care of edge {vx , vx}.

I For each clause c (with literals x ,y ,z), pick a variable x that is true andadd cy and cz to the vertex cover.

I Since x is true, vx is in the cover and so the edge {vx , cx} is covered.

I Since cy and cz are in the cover, all edges of the c gadget are covered,and we cover the remaining edges from the c-gadget to variable gadgets.

I So all edges are covered, and the size of the cover is m + 2l = k.



I Here, x1 = 0 and x2 = 1 is a satisfying assignment.

I We add the “true” variable gadget nodes to the cover.

I Then we add two clause gadget nodes to the cover to take care of the clauseedges.



Theorem

I Let φ be a 3SAT formula of m variables and l clauses, and let G be itscorresponding graph.

I φ is satisfiable iff G has a vertex cover of k = m + 2l nodes.

(Right to Left)

I Suppose G has a cover C of size k = m + 2l .

I At least one node in each variable gadget must appear in C .

I To cover clause gadget edges, ≥ two nodes from each gadget must be in C .

I Since the cover is size m + 2l , it must be that exactly one node from the mvariable gadgets and 2 from the l clause gadgets are in the cover.

I For each clause c, if cy , cz are in the cover, it must be that vx is in the cover(to cover {cx , vx}).

I And so each clause c has a variable x such that vx is in the cover.

I The variable gadget nodes in the cover form a consistent TVA.

I And so φ is satisfied.



Show how the following 3SAT formula φ can be reduced to acorresponding graph G , so that φ is satisfiable if and only if G has avertex cover of k = m + 2l nodes, where m is the number of variablesand l is the number of clauses in φ.

φ = (x1 ∨ x2 ∨ x3) ∧ (x1 ∨ x2 ∨ x3) ∧ (x1 ∨ x2 ∨ x3)


HAMPATH is NP-Complete

Theorem


I We show 3SAT ≤P HAMPATH.

I From φ, we produce a directed graph G such that φ is satisfiable iff G has aHamiltonian path (from top node to bottom node).

I The idea here is to use two types of “gadgets”.

I Diamond shaped clusters of nodes represent variables.

I Solitary nodes represent clauses.

I How we connect them achieves the desired result.



I The idea here is we can zig or zag through the graph.

I The choices made correspond to making a variable true or false.



I Traversing the graph defines a truth value assignment.

I How the clause nodes (right) are connected is explained in the next slide.



I In a diamond, a pair of nodes is used as an exit/entry point to a clause node.

I There’s also a buffer node (in red) before and after each pair.

I If there are k clauses, we need 3k + 1 interior diamond nodes.



I If xi appears in a clause cj , we connect the j ’th pair in the diamond to cj .

I The direction of the edges depends on whether xi is positive or negated in cj .

I Here, xi is negated.

I If xi were positive, we would change the directions of the arrows (so that cj

would be entered from the left).



I That’s the construction.

I The claim is that once G has been constructed from φ, there’s a Hamiltonianpath through G iff φ is satisfiable.

I It should be clear that if φ is satisfiable, then a Hamiltonian path exists.

I The assignment determines how we traverse the graph.

I If xi is true, we zig to the left, making a detour through all of the clause nodesin which xi appears positively.

I Similarly, if xi is false, we zag to the right.

I Importantly, we need not revisit a clause if we’ve done so before.



I In the reverse direction....A Hamiltonian path must traverse the graphtop-down, and hit the nodes of every diamond in order.

I As situation like the above can’t happen.

I a2 (and other nodes) necessarily would be skipped.

I And so any Hamiltonian path maps out a truth value assignment satisfying φ.


UHAMPATH is NP-Complete

Theorem


I An undirected version of the Hamiltonian path problem, called UHAMPATH.

I We show HAMPATH ≤P UHAMPATH.

I The reduction takes a directed graph G with nodes s and t and constructs anundirected graph G ’ with nodes s’ and t’.

I G has a Hamiltonian path from s to t iff G ’ has a Ham-path from s’ to t’.

I Each node u ∈ G , where u 6= s and u 6= t, u =⇒ (uin, umid , uout) in G ’.

I s =⇒ sout , and t =⇒ t in.

I Edges of two types appear in G ’.

1. edges connect umid with uin and uout

2. an edge connects uout with v in if an edge goes from u to v in G .





Theorem


I G has a Hamiltonian path from s to t iff G ’ has a Hamiltonian path from sout

to t in.

I A Hamiltonian path P in G , s, u1, u2, . . . , uk , t, has a correspondingHamiltonian path P’ in G ’, sout , uin

1 , umid1 , uout

1 , uin2 , u

mid2 , uout

2 , . . . , t in.

I Claim: A Hamiltonian path P’ in G ’ from sout to t in in G ’ must go from atriple of nodes to a triple of nodes, except for the start and finish.

I The path starts at sout , the next node must be uini for some i because only

those nodes are connected to sout .I The next node must be umid

i . After umidi comes uout

i .I The next node must be uin

j for some j because no other available node isconnected to uout

i .


SUBSET-SUM is NP-Complete

Theorem


I SUBSET-SUM is in NP. (already proved)

I We show 3SAT ≤P SUBSET-SUM.

I Let φ be a Boolean formula with variables x1, x2, . . . , xl and clausesc1, c2, . . . , ck .

I The reduction converts φ to an instance of the SUBSET-SUM problem 〈S , t〉,wherein the elements of S and the number t are the rows in the table shown inthe next slide, expressed in ordinary decimal notation.

I The rows above the double line are labeled y1, z1, y2, z2, . . . , yl , zl andg1, h1, g2, h2, . . . , gk , hk and comprise the elements of S .

I The row below the double line is t.


(x1 ∨ x2 ∨ x3) ∧ (x2 ∨ x3 ∨ · · · ) ∧ · · · ∧ (x3 ∨ · · · ∨ · · · )



I S contains one pair of numbers, yi , zi , for each variable xi in φ.

I The decimal representation of these numbers is in two parts: the LH partcontains a 1 followed by (l − i) 0s; the RH part contains one digit foreach clause, where the jth digit of yi is 1 if clause cj contains literal xi

and the jth digit of zi is 1 if clause cj contains literal xi .

I Digits not specified to be 1 are 0.

I S contains one pair of numbers, gj , hj , for each clause cj . These twonumbers are equal and consists of a 1 followed by (k − j) 0s.

I The target number t, consists of l of 1s followed by k of 3s.



I φ is satisfiable iff some subset of S sums to t.

I Suppose that φ is satisfiable.

I Select yi if xi is TRUE in TVA, and zi if xi is FALSE.The sum of the first ldigits, each contains single 1.

I The sum of the last k digits, each is a number between 1 and 3.I Select either one or two of the g and h numbers to bring each of the last

k digits up to 3I Thus the sum of the numbers in S is equal to t.



I φ is satisfiable iff some subset of S sums to t.

I Suppose that a subset of S sums to t.

1. All the digits in members of S are either 0 or 1.2. Each column in the table describing S contains at most five 1s.3. Hence a “carry” into the next column never occurs when a subset of S is

added.4. To get a 1 in each of the first l columns the subset must have either yi or

zi for each i , but not both.5. If the subset contains yi , we assign xi TRUE; otherwise, we assign it

FALSE.6. In each of the last k columns the sum is always 3. In column cj , at most 2

can come from gj and hj , so at least one in this column must come fromcj , which is TRUE from some yi or zi in the subset.



I Can the reduction be carried out in polynomial time?

I The table has a size of (2k + 2l + 1)× (k + l), O((k + l)2).

I Each entry of the table can be easily calculated for any φ.

I The total time is in polynomial time.



Show how the following 3SAT formula φ can be reduced to acorresponding number set S , so that φ is satisfiable if and only if somesubset of S sums to t, where t contains l of 1s, followed by k of 3s. Andl is the number of variables and k is the number of clauses in φ.

φ = (x1 ∨ x2 ∨ x3) ∧ (x1 ∨ x2 ∨ x3) ∧ (x1 ∨ x2 ∨ x3)


DOUBLE-SAT is NP-Complete

Let DOUBLE-SAT = {〈φ〉| φ has at least two satisfying assignments }.Show that DOUBLE-SAT IS NP-complete.

Proof.

???




Proof.

I On input φ, a nondeterministic polynomial time Turing machine cannondeterministically generate two different TVAs and accept if bothassignments satisfy φ.

I Thus DOUBLE-SAT is in NP.

I We show SAT ≤P DOUBLE-SAT.




Proof.

The following TM F computes the polynomial time reduction f .F = “On input 〈φ〉, a Boolean formula with variables x1, x2, · · · , xm:

1. Let φ’ be φ ∧ (x ∨ x), where x is a new variable.

2. Output 〈φ′〉.”If φ ∈ SAT , then φ’ has at least two satisfying assignments that can beobtained from the original satisfying assignment of φ by changing thevalue of x . If φ′ ∈ DOUBLE-SAT, then φ is also satisfiable, because xdoes not appear in φ. Therefore φ ∈ SAT iff f (φ) ∈ DOUBLE-SAT.


Questions for group discussion

Let G represent an undirected graph. Also letSPATH = {〈G , a, b, k〉|G contains a simple path of length at most kfrom a to b}, andLPATH = {〈G , a, b, k〉|G contains a simple path of length at least kfrom a to b}.

1. Show that SPATH ∈ P.

2. Show that LPATH is NP-complete. You may assume theNP-completeness of UHAMPATH.

Let CNFk = {〈φ〉}|φ is a satisfiable cnf-formula where each variableappears in at most k places }.

1. Show that CNF2 ∈ P.

2. Show that CNF3 is NP-complete.


Documents

Time Complexity (2)cobweb.cs.uga.edu/~potter/theory/7_time_complexity_II.pdfCSCI 2670 Time Complexity (2) A tableau A computation history of N on input w. CSCI 2670 Time Complexity