REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

COMP36111: Advanced Algorithms I

Lecture 9: Savitch’s Theorem and the Immerman-SzelepcsenyiTheorem

Ian Pratt-Hartmann

Room KB2.38: email: ipratt@cs.man.ac.uk

2017–18

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• Reading for this lecture:• Sipser, Ch. 8 (Space Complexity).

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

Outline

REACHABILITY again

Savitch’s theorem

The Immerman-Szelepcsenyi Theorem

Standard hierarchyTime and spaceThe big picture

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• Recall that a directed graph is a pair G = (V ,E ), where V is

a set and E ⊆(V2

)a set of ordered pairs of distinct elements

of V .v0

v1

v2

v3

v4

v5

v6 v7

• We assume G is encoded on the input tape of a TM asfollows:

7 : 0, 1; 0, 2; 0, 3; 1, 0; 1, 3; 1, 4; 2, 5; 3, 6; 5, 1; 7, 2; 7, 5; 7, 6

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• If G = (V ,E ) is a directed graph, and u, v ∈ V , we say thatv is reachable from u if there exists a sequenceu = u0, . . . , um = v from V with m ≥ 0 such that, for each i(0 ≤ i < m) (ui , ui+1) ∈ E .

• In our graph G , v6 is reachable from v0

v0

v1

v2

v3

v4

v5

v6 v7

• since we have the sequence

v0 → v1 → v3 → v6.

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• However, v7 is not reachable from v0.

v0

v1

v2

v3

v4

v5

v6 v7

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• We then have the problem:

REACHABILITYGiven: A directed graph G = (V ,E ) and nodes s, t ∈ VReturn: Yes if t is reachable from s in G , No otherewise.

• In an earlier lecture, we gave an algorithm showing thatREACHABILTY is in Time(O(n)).

• A bit of revision won’t hurt . . .

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• An algorithm for solving REACHABILITY:

begin DFS-directed((V ,E ), u, v)DFS-aux((V ,E ), u)if v is marked

return Yreturn N

end DFS-directed

begin DFS-aux((V ,E ), u)mark ufor each e ∈ edges(u) do

if e = (u,w) with w unmarked doDFS-aux((V ,E ), w)

end DFS-aux

• We saw an essentially identical algorithm in Lecture 1a. Itruns in linear time (and hence linear space).

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

Outline

REACHABILITY again

Savitch’s theorem

The Immerman-Szelepcsenyi Theorem

Standard hierarchyTime and spaceThe big picture

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• The following deterministic algorithm outputs YES iff v isreachable from u in G = (V ,E ) in at most 2h steps:

begin isReachableNum(u,v ,G ,h)if h = 0

if u = v or (u, v) ∈ E return Yeselse return No

for w ∈ Vif (isReachableNum(u,w ,G ,h − 1) and

isReachableNum(w ,v ,G ,h − 1)) return Yesreturn No

• Thus, we can then solve the reachability problem by callingisReachableNum(u,v ,G ,dlogV e)

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• To see how this works, we note that any path from u to v oflength ≤ 2h must have a midpoint w

u

w

v

• Hence, there is a path from u to w of length ≤ 2h−1 and apath from w to v of length ≤ 2h−1

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• How do we implement this on a Turing machine?

• Answer: by keeping the triples 〈u, v , h〉 on a work-tape:

〈u, v , h〉

begin isReachableNum(u,v ,G ,h)if h = 0

if u = v or (u, v) ∈ E return Yeselse return No

for w ∈ Vif (isReachableNum(u,w ,G ,h − 1) and

isReachableNum(w ,v ,G ,h − 1)) return Yesreturn No

• We see that this algorithm requires at most O(h · log |V |)space.

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• How do we implement this on a Turing machine?

• Answer: by keeping the triples 〈u, v , h〉 on a work-tape:

〈u, v , h〉〈u,w1, h − 1〉

begin isReachableNum(u,v ,G ,h)if h = 0

if u = v or (u, v) ∈ E return Yeselse return No

for w ∈ Vif (isReachableNum(u,w ,G ,h − 1) and

isReachableNum(w ,v ,G ,h − 1)) return Yesreturn No

• We see that this algorithm requires at most O(h · log |V |)space.

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• How do we implement this on a Turing machine?

• Answer: by keeping the triples 〈u, v , h〉 on a work-tape:

〈u, v , h〉〈u,w1, h − 1〉〈u,w2, h − 2〉

begin isReachableNum(u,v ,G ,h)if h = 0

if u = v or (u, v) ∈ E return Yeselse return No

for w ∈ Vif (isReachableNum(u,w ,G ,h − 1) and

isReachableNum(w ,v ,G ,h − 1)) return Yesreturn No

• We see that this algorithm requires at most O(h · log |V |)space.

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• How do we implement this on a Turing machine?

• Answer: by keeping the triples 〈u, v , h〉 on a work-tape:

〈u, v , h〉〈u,w1, h − 1〉〈u,w2, h − 2〉 · · · 〈u,w`, h − `〉

begin isReachableNum(u,v ,G ,h)if h = 0

if u = v or (u, v) ∈ E return Yeselse return No

for w ∈ Vif (isReachableNum(u,w ,G ,h − 1) and

isReachableNum(w ,v ,G ,h − 1)) return Yesreturn No

• We see that this algorithm requires at most O(h · log |V |)space.

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• How do we implement this on a Turing machine?

• Answer: by keeping the triples 〈u, v , h〉 on a work-tape:

〈u, v , h〉〈u,w1, h − 1〉〈u,w2, h − 2〉 · · · 〈w`, v , h − `〉

begin isReachableNum(u,v ,G ,h)if h = 0

if u = v or (u, v) ∈ E return Yeselse return No

for w ∈ Vif (isReachableNum(u,w ,G ,h − 1) and

isReachableNum(w ,v ,G ,h − 1)) return Yesreturn No

• We see that this algorithm requires at most O(h · log |V |)space.

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• Hence the call isReachableNum(u,v ,G ,dlogV e) requiresO(log2 |V |) space.

• This proves:

Theorem (Savitch, first form)

REACHABILITY is in Space(log2 n).

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• Suppose we have a Turing machine M over an alphabet withc symbols, having just one tape, and running in space f (n).

• By a configuration of M we mean a triple 〈s,w , i〉, where:• s is a state of M;• w is a word over the alphabet of M (tape contents);• 1 ≤ i ≤ |w | (head position).

• Writing w = a1 . . . a`, we can conveniently encode thisconfiguration on a second (work-) tape as

a1 . . . ah−1sah . . . a`

• We can think of this as the label of a node in a graph, G .

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• Consider again the configuration

a1 . . . ah−1sah . . . a`

and suppose s, ah 7→ b, right, t is a transition of M, wherea = ah.

• Then we can easily compute the subsequent configuration

a1 . . . ah−1btah+1 . . . a`

(on another tape if you like).

• We can think of any pair of such configurations as an edge inG

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• Determining whether M has an accepting run (in time f (n))now amounts to determining whether there is a path from theinitial state of M to any accepting state of M

s0x

s∗Y

• The number of nodes of G is bounded by S · f (n) · c f (n), whereS is the number of states and c the size of the alphabet;

• Note that we can compute the edges of the G on the fly: weoften do not need the whole graph.

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

Theorem (Savitch (second form))

If f is a proper complexity function and f (n) ≥ log n, thenNSpace(f ) ⊆ Space(f 2).

Proof.Suppose P is a problem in NSpace(f ). Let M be anondeterministic TM running in Space(f ), and accepting P.To determine whether x ∈ P, determine whether configurationgraph of M has a path of length at most 2O(f (|x |)) from the initialnode to an accepting node. This can be done inSpace(O(f (n))2).

Corollary

NPSpace = PSpace; NExpSpace = ExpSpace, . . .

Corollary

NPSpace = Co-NPSpace; NExpSpace = Co-NExpSpace,. . .

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

Outline

REACHABILITY again

Savitch’s theorem

The Immerman-Szelepcsenyi Theorem

Standard hierarchyTime and spaceThe big picture

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• The algorithm DFS-directed showed that REACHABILITYis in Time(n) and hence in Space(n).

• The algorithm isReachableNum showed thatREACHABILITY is in Space((log n)2).

• We now present a very simple algorithm to show that, withnon-determinism, we can get the space requirements downstill further.

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• The following non-deterministic algorithm has a run whichoutputs YES iff v is reachable from u in G = (V ,E ):

begin isReachable(u,v ,G )let current = ulet counter = 0until current = v or counter = |V |

guess a node u′ ∈ Vif (current, u′) 6∈ E return NOlet current = u′

increment counterif current = v return YESelse return NO

• The only things we need to store are current and counter,which require only log |V | bits.

• Thus we see that REACHABILITY is in NSpace(log n)—i.e. NLogSpace.

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

TheoremREACHABILITY is NLogSpace-complete.

Proof.We showed above that REACHABILITY is in NLogSpace.

Suppose L is a language recognized by a non-deterministic TM, M,running in time O(log n). Given an input x , let G be theconfiguration graph for M with input x . Let u be the noderepresenting the initial configuration. We may assume this graphhas a single acepting node v . Now, x ∈ L if and only if (G , u, v) isan instance of REACHABILITY. The mapping x 7→ (G , u, v) caneasily be constructed in space bounded by log n.

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• It was very easy to see that REACHABILITY is inNLogSpace.

• However, let us now consider its converse:

UNREACHABILITYGiven: A directed graph G = (V ,E ) and nodes s, t ∈ VReturn: Yes if t is not reachable from s in G , No otherewise.

• We shall now show that UNREACHABILITY is inNLogSpace too.

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• Fix a directed graph G = (V ,E ), and a node u ∈ V .

• The trick is to use a very simple non-deterministic subroutine:

begin reachableLossy(u,v ,k)set u′ := uuntil k = 0

guess any node v ′

if u′ 6= v ′ and (u′, v ′) 6∈ E return Noset u′ := v ′

decrement kif u′ = v return Yesreturn No

• reachableLossy(u,v ,k) has a run returning Yes iff v isreachable from u in k or fewer steps.

• Nothing is said about runs of reachableLossy(u,v ,k)returning No.

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• Assume we have an algorithm isReachableFail(u, v , k)which, for 1 ≤ k < n, either returns ⊥, Yes or No:

• isReachableFail has a run returning Yes, iff v is reachablefrom u in at most k steps;

• isReachableFail has a run returning No, iff v is notreachable from u in at most k steps;

• Then the following algorithm returns the number of nodesreachable from u in k steps or fewer, or just returns ⊥:

begin numReachableFail(u, k)if k = 0 return 1set m = 0for i = 0, . . . , n − 1

let Q= isReachableFail(u, ui , k)if Q = ⊥, then return ⊥if Q = Yes, then increment m

return m

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• Now for the definition of isReachableFail (assume1 ≤ k < n):

begin isReachableFail(u, v , k)let s = numReachableFail(u, k − 1)if s = ⊥ then return ⊥let m = 0for i = 0, . . . , n − 1

if reachableLossy(u, ui , k − 1) = Yesif ui = v or (ui , v) ∈ E then return Yesincrement m

if m < s then return ⊥return No

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• Now for the our non-deterministic algorithm acceptingUNREACHABILITY:

begin isUnreachable(u, v , (V ,E ))if isReachableFail(u, v , |V | − 1) = No then return Yesreturn No

• It is easy to see that this algorithm requires only logarithmicspace, and has a run returning Yes if and only if v is notreachable from u in G = (V ,E ).

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

Theorem (Immerman-Szelepcsenyi, first form)

UNREACHABILITY is in NLogSpace.

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

Theorem (Immerman-Szelepcsenyi (second form))

If f is a proper complexity function and f (n) ≥ log(n), thenNSpace(f ) = Co-NSpace(f ).

Proof.Suppose P is a problem in NSpace(f ), and let P be itscomplement problem. Let M be a nondeterministic TM running inSpace(f ), and accepting P, and let x be an input string. Denoteby G be the configuration graph of M with input x . Then x is apositive instance of P if and only if the node of G representing asuccessful run is unreachable from the start node of G .

Corollary

NLogSpace= Co-NLogSpace.

Corollary

KROM-SAT (= 2-SAT) is in NLogSpace.

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

TheoremThe problem KROM-SAT (= 2-SAT) is NLogSpace-complete.

Proof.We have just shown that KROM-SAT is in NLogSpace.For NLogSpace-hardness, we reduce the problemUNREACHABILITY (=Co-REACHABILITY) to KROM-SAT. LetG = (V ,E ) be a directed graph, and u0, v0 ∈ V be vertices.Treating the vertices V as propositional variables, define the set ofclauses ΓG

{u0,¬v0} ∪ {u → v | (u, v) ∈ E}.

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

Proof.If u0, . . . , um = v0 is a path through G , then any truth-valueassignment marking u0 and {u → v | (u, v) ∈ E} true must makev0 true. Hence ΓG is unsatisfiable.

Conversely, if v0 is not reachable from u0, let θ be the truth-valueassignment

θ(v) =

{T if v is reachable from u0 in G ;

F otherwise.

Then θ evidently satisfies ΓG .

NLogSpace-hardness of KROM-SAT then follows from theNLogSpace-hardness of UNREACHABILITY.

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

Outline

REACHABILITY again

Savitch’s theorem

The Immerman-Szelepcsenyi Theorem

Standard hierarchyTime and spaceThe big picture

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• Consider again the classes Time(f ), NTime(f ), Space(f ).

• How are these related?

• First of all, it is trivial that

Time(f ) ⊆ NTime(f ).

• What about NTime(f ) and Space(f )?• Consider a TM, M, running in NTime(f ), and acting on input

x of length n.• M can make at most f (n) non-deterministic choices.• Represent these choices as a digits as a string s of length f (n).• We can run through all possible strings, guiding the choices of

M using s, succeeding if M ever succeeds.• The resulting deterministic TM takes no more space, and

recognizes the same language as M.

• Thus, we have

NTime(f ) ⊆ Space(f ).

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• To relate Space(f ) to a time-complexity class, consider aTM, M, running in Space(f ), and operating on an input x oflength n.

• We may suppose M has K tapes (K − 2 work-tapes) and usesan alphabet Σ.

• A configuration is given by K − 2 strings of length at mostf (n), together with one of |Q| states, and a K -tuple ofintegers (≤ f (n)) indicating the head position on each tape.

• The total number of such configurations is|Q|.|Σ|(K−2)f (n)+K log(f (n)), and reachability in such a graphcan be decided in time c ′ · |Q|.|Σ|(K−2)f (n)+K log(f (n)) for someconstant c ′.

• Thus, we have

Space(f (n)) ⊆ ∪a>0Time(2af (n)).

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• Applying these results to the larger classes PTime, NPTime,PSpace etc, a nice picture emerges:

PTime ⊆ NPTime ⊆ PSpace ⊆ ExpTime . . .

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• What about the non-deterministic stuff?

• We know from Savitch’s theorem that PSpace = NPSpace,ExpSpace = NExpSpace etc., so we can henceforth ignorelarge non-deterministic space-classes.

• Warning: it does not follow from Savitch’s theorem thatLogSpace = NLogSpace, and we do not know whetherthis equation holds.

• Warning: it does not follow from Savitch’s theorem thatPTime = NPTime, etc, and we do not know whether theseequations hold.

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• Thus, we have:

LogSpace ⊆ NLogSpace ⊆ PTime ⊆ NPTime ⊆PSpace ⊆ ExpTime ⊆

NExpTime ⊆ ExpSpace ⊆ · · ·

• We showed earlier that PTime ( ExpTime, and hence thatat least one of the inequalities

PTime ⊆ NPTime, NPTime ⊆ PSpace, PSpace ⊆ ExpTime

is strict; but it is not know which.

REACHABILITY again Savitch’s theorem The Immerman-Szelepcsenyi Theorem Standard hierarchy

• How do the complements of these classes fit into the picture?

• We have already establiashed the following:• deterministic classes (time or space) are always equal to their

complement classes;• non-deterministic space classes from NPSpace upwards are

equal to their deterministic variants (Savitch) and hence totheir complement classes;

• NLogSpace is equial to its complement class(Immerman-Szelepcsenyi).

• We do not know whether common non-deterministic timeclasses, such as NPTime, NExpTime etc., are equal to theircomplements.