Computational Models — Lecture 81tau-cm2017a.wdfiles.com/local--files/course-schedule/TM2_h.pdf · Computational Models — Lecture 81 Handout Mode Iftach Haitner. Tel Aviv University

Computational Models — Lecture 81

Handout Mode

Iftach Haitner.

Tel Aviv University.

December 19, 2016

1Based on frames by Benny Chor, Tel Aviv University, modifying frames by Maurice Herlihy, Brown University.

Iftach Haitner (TAU) Computational Models, Lecture 8 December 19, 2016 1 / 52

Talk Outline

I Non-deterministic Turing machines

I Enumerators

I Decidability vs. Enumerability

I Encoding of Turing Machines and Universal Turing Machines

I The Halting/Acceptance problem

I Beyond Enumerable and co-Enumerable

Sipser’s book, 3.2, 3.3, 4.1 and 4.2.


Part I

Non-deterministic Turing machines(NTMs)


Non-Deterministic Turing Machines (NTMs)

NTM N = (Q,Σ, Γ, δ,q0,qa,qr ),

δ : Q × Γ 7→ P(Q × Γ× {L,R})

The yield relation (for NTM): configuration C yields D, if it yields it, accordingto the deterministic definition, for some deterministic restriction of δ.

Configuration C = (x ′qx ′′) yields D = (y ′py ′′), if(p, (y ′y ′′)head(C),X )∈δ(q, (x ′x ′′)head(C)) and . . .

The computation tree of N on input w :

I Root is the starting configuration (with respect to w)

I The children of a node are all configurations it (directly) yields.# of children is at most: |Q| · |Γ| · 2.

Valid sequences of configurations with respect to N and w , are defined as inthe deterministic case.

Any rooted finite path in the computation tree of N and w , it is a validsequence of configurations (with respect to N and w).


Accepting a word

N accepts w ∈ Σ∗, if ∃ an accepting path (i.e., accepting sequence ofconfigurations) in its computation tree of N on w .(Equivalently, ∃ an accepting sequence of configurations, with respect to Nand w .)

N halts on w ∈ Σ∗, if it accepts it, or the computation tree of N on w is finite:∃k ∈ N, such that there is no valid sequence of length k .


Equivalence of TM and NTM

It is clear that NTM is Turing complete.

Theorem 1For any non-deterministic TM there exists a deterministic TM that emulates it.

Corollary 2

A language is enumerable [resp., decidable], if and only if there is somenon-deterministic Turing machine that accepts [resp., decides] it.

We will prove a slightly simpler to prove result.

Theorem 3For any NTM N there exists TM D with L(N) = L(D).


Basic idea

I D tries all possible branches in N computation tree

I If D finds any accepting path, it accepts.

Question 4How to traverse this tree?

I depth-first search?

I breadth-first search?


D’s tapes

11 ... ba ... 1 ...#

input tape(never altered)

simulationtape

addresstape

2 3 3

The machine D has three tapes:

I Input tape is never altered (only read from),

I Emulation tape serves as N ’s tape,

I Address tape keeps track of D’s location in N ’s computation tree.


Address tape

Let b be bound on the # of children of node in N ’s computation tree.

The address tape contains a pointer into the configuration tree.Concretely, a string in Σ∗b, for Σb = {1, . . . ,b}.

Definition 5By incrementing the value of the address tape, we mean replace its contentwith the next string in Σ∗b, according to the lexicographic order.

Example (for b = 2): ε 7→ 1 7→ 2 7→ 11 7→ 12 7→ 21 7→ 22 7→ 111 . . .

Question 6Can a TM implement the increment function?

Question 7Can a (deterministic) TM compute the value of the node indexed by theaddress tape (with respect to TM N and input w)?


TM D

Algorithm 8 (TM D (pseudocode))

1. Compute the configuration of N indexed by the address tape :

1.1 Copy input tape (i.e., w) to emulation tape.1.2 Use emulation tape to emulate the run of N on w , using the

address tape to resolve non-deterministic choices.Break current emulation, if

F End of path (i.e., symbols on address tape are exhausted)F Accepting/Rejecting configuration is reachedF Non-deterministic choice is invalid

2. Accept, if an accepting configuration was reached.

3. Increment the value of address tape.

4. Go back to Step 1.

Question 9Change D to emulate N.


Part II

Enumerators


Enumerators

A language is enumerable, if it is accepted by some Turing Machine.

Question 10But why enumerable?

1 010

aa

abba

A TM is an enumerator for a language L, if on the empty input strings, itoutputs all the strings in L and nothing else.


Enumerators, formal definition

1 010

aa

abba

Definition 11 (enumerator)

A deterministic TM M is an enumerator for a language L ⊆ Σ∗, if on the emptyinput string it does as follows:

I On its first tape (i.e., output tape), M

I Writes only elements from Σ ∪ $ (we assume wlg. $ /∈ Σ)I Never alters cell more than once (i.e., write only tape)

I Every word in L appears after a finite number of steps on the output tap(i.e., between two $′s)

I Word not in L never appears on the output tape


Having Enumerator⇔ Being in RE

Theorem 12A language is in RE iff it has an enumerator.

Will show

I If E enumerates language L, then some TM M accepts L.

I If M accepts L, then some enumerator E enumerates it.


Having enumerator =⇒ being in RE

Claim 13If a TM E enumerates a language L, then some TM M accepts L.

Proof:

Algorithm 14 (TM M)

On input w , run E .

Every time E outputs a string v :

If v = w , accept.

Otherwise continue

♣


Being in RE =⇒ Having EnumeratorClaim 15If a TM M accepts L, then some enumerator TM E enumerates L.

Proof: Let s1, s2, s3, . . . be a list of all strings in Σ∗ (e.g., strings inlexicographic order).

Algorithm 16 (TM E)

Repeat the following for i = 1,2,3, . . .

I Run M for i steps on each input s1, s2, . . . , si .

For any accepting computation, output the corresponding s.

♣Note that with this procedure, each output is duplicated infinitely often.Question 17Can this duplication be avoided?

Question 18Can we do the enumeration in (lexicographic) order?


Having “In order" Enumerator⇔ Being in R

Theorem 19A language L is decidable iff ∃ an enumerator that enumerates L inlexicographic order.

Proof: ? Left as an exercise. ♣


Part III

Decidability vs. Enumerability


Decidability vs. enumerability (R = RE ∩ co−RE)

I RE – the class of enumerable languages

I co−RE – the class of languages whose complement is enumerable:co−RE = {L : L̄ ∈ RE}

I R – the class of decidable languages.

I L ∈ R =⇒ L ∈ REI L ∈ R =⇒ L ∈ RI Thm (next slide): R = RE ∩ co−RE


Theorem R = RE ∩ co−RE

Theorem 20R = RE ∩ co−RE .

Proof:I R ⊆ RE ∩ co−RE

I L ∈ R =⇒ L ∈ REI L ∈ R =⇒ L ∈ R =⇒ L ∈ RE =⇒ L ∈ co−RE

I R ⊇ RE ∩ co−REFor L ∈ RE ∩ co−RE , let M1 be a TM that accepts L, and let M2 bea TM that accepts L.

Algorithm 21 (M - a decider for L)

Input: w .

I Run both M1 and M2 in “in parallel".

I Accept if M1 accepts

I Reject if M2 accepts


M decides L

Claim 22M decides L

Proof:

I Every string is in either L or in L (of course not in both).

I Thus either M1 or M2 accepts the input w .

I Since M halts whenever M1 or M2 accepts, M always halts (and hence isa decider).

I Moreover, M accepts strings in L and rejects strings in L.

♣


Emulating TM’s in parallel

Question 23What does it mean to emulate M1,M2 in parallel?

Answer:M has two tapes, one for each machine.

Algorithm 24 (TM M)

Do (forever)

1. Emulate the next step of M1

2. Emulate the next step of M2

3. If this is accepting configuration for some Mi , halt and return i .


Revised view of the world of languages

Question 25How do we know that CFL (and thus regular languages) are in R?


Part IV

Encodings and Universal TM


Encodings

I Input to a Turing machine is a string of symbols.

I We want algorithms that work on graphs, matrices, polynomials, Turingmachines, etc.

I Need to choose an encoding for objects (can often be done in manyreasonable ways).

I Sometimes it is helpful to distinguish between X , the object, and 〈X 〉, itsencoding.


Encoding of Turing Machines

Turing machines can be encoded as strings.

Such encoding will enable us

I To check (by an algorithm) that a given string is a legal encoding of a TM.(Similar to a compiler checking for syntax errors.)

I To build a universal machine that can read such encoding and emulatesthe encoded TM on any input string.(Similar to running an interpreter.)


Standard Encoding of Turing Machines

Definition 26 (Encoding of 〈M〉 of a TM M)

Let M = (Q,Σ = {σ1, . . . , σk}, Γ, δ,q0,qa,qr ) be a TM. Assume wlg. that

I Q = {q1, . . .qm}, where q0, qa and qr are indicated by q1, q2 and q3.

I Γ = {γ1, . . . , γs}, where σ1, . . . , σk , are indicated by γ1, . . . , γk+1 .

I The directions L and R be indicated by D1 and D2.

To encode M, we only encode the transition function δ. For each ruleδ(qi , γj ) = (qk , γ`,Db), we add the string 0i10j10k 10`10b.Different rules are separated by 11.

Fact 27There exists a TM (called universal TM) that on input 〈M,w〉 (encoded by(1100∗100∗100∗100∗100∗)∗111(0 ∪ 1)∗, can check that 〈M〉 encoded a TM,and can emulates M(w).


The Universal Turing Machine


Universal Turing Machines

Algorithm 28 (Universal TM U)

On input 〈M,w〉, where 〈M〉 and 〈w〉 are binary strings separated by 111.

I Checks that 〈M,w〉 is a proper encoding of a TM.

I Emulate M(w) (how?)

I Accept, if M enters its accept stateI Reject, if M enters its reject state

Notice that as a consequence, if M on input w enters an infinite loop, so doesU on input 〈M,w〉.


Universal Turing Machines (2)

I The universal machine U obviously has a fixed number of states (100should do).

I Despite this, it can simulate machines M with many more states.

I Universal machines inspired the development of stored-programcomputers in the 40s and 50s.

I Most of you have seen a universal machine, and have even used one!


Universal Turing Machines (3)

I For example, Dr. Scheme (interpreter) is a universal Scheme machine.

I It accepts a two part input: “Above the line” – the program(corresponding to 〈M〉), and “below the line” the input to run it on(corresponding to w).


Part V

The Acceptance & Halting Problems


The Acceptance & Halting Problems

I Of the most philosophically important theorems of the theory ofcomputation.

I Computers (and computation) are not omnipotent – they are limited in avery fundamental way.

I Many common problems are unsolvable, e.g., does a program sort anarray of integers?

Note that these problems are well defined: both program andspecification are precise mathematical objects.

Hey, proving program ∼= specification should be just like proving thattriangle 1 ∼= triangle 2 . . .

Well, this is not the case!


CFG, NFA, DFA Reminders

We saw that the following language are decidable:

I ADFA = {〈M,w〉 : M is a DFA accepting the string w}.I ANFA = {〈M,w〉 : M is an NFA accepting the string w}.I ACFG = {〈M,w〉 : M is a PDA accepting the string w}.I EMPTYCFG = {〈G〉 : G is a CFG ∧ L(G) = ∅} .

What would happen with Turing Machines?

ATM = {〈M,w〉 : M is a TM that accepts w}

Theorem 29 (The Acceptance Problem is undecidable)

ATM is undecidable (i.e., no TM decides it).


The Acceptance problem

ATM = {〈M,w〉 : M is a TM that accepts w}

Before approaching the proof of undecidability, we first prove

Theorem 30ATM is recursively enumerable (namely in RE).

Proof: The universal machine accepts ATM. ♣


Proving Thm 29

Suppose a TM, H, is a decider for ATM. Namely,

H(〈M,w)〉) =

{accepts if M accepts wrejects if M does not accept w

Algorithm 31 (D)

On input 〈M〉I Run H on input 〈M,M〉.I Output the opposite of what H outputs:

I Reject if H accepts, andI Accept if H rejects.

What happens if we run D on its own description?

D(〈D〉) =

{reject if D accepts 〈D〉accept if D rejects 〈D〉

Oh, oh...Or, more accurately, a contradiction (to what?) ♣


Self reference

Don’t be confused by the notion of running a machine on its own description!

Actually, you should get used to it.

I Notion of self-reference comes up again and again in diverse areas.

I This notion of self-reference is the basic idea behind Gödel’srevolutionary result.

I Compilers do this all the time . . . .


A non-enumerable language

I We already saw a non-decidable language: ATM.

I We now display a language that is not even recursively enumerable . . . .

Corollary 32

If L /∈ R, then either L /∈ RE or L /∈ RE .

Proof: Assume otherwise, by Thm 20 L is decidable. ♣

Corollary 33

ATM /∈ RE .


The Halting Problem

HTM = {〈M,w〉 : M is a TM and M halts on input w}

Theorem 34HTM is undecidable.

Proof idea:

I Similar to ATM.

I Alternatively, by a reduction to ATM (?).


The World as we (currently) Know It

enumerableco-enumerable decidable

ATM

ADFA

ATM

???

Question 35Are there any languages in the area marked ??? ?

Yes, for instance

L3 = {〈M1〉,w1, 〈M2〉,w2 : w1 ∈ L(M1) ∧ w2 /∈ L(M2)}


Claim: L3 = {〈M1〉,w1, 〈M2〉,w2 : w1 ∈ L(M1) ∧ w2 /∈ L(M2)} /∈ RE .

Proof: Assume L3 ∈ RE . Hence, ∃ TM A with L(A) = L3.

Algorithm 36 (TM B for ATM)

input 〈M,w〉

1. Let C be a TM s.t. L(C) = {c}

2. Run A on < C, c,M,w >.

3. Accept, if A accepts.

B accepts ATM, contradiction.♣


Claim: L3 = {〈M1〉,w1, 〈M2〉,w2 : w1 ∈ L(M1) ∧ w2 /∈ L(M2)} /∈ co−RE .

Proof: Assume L3 ∈ co−RE . Hence, ∃ TM A with L(A) = L3.

Algorithm 37 (TM B for ATM)

input 〈M,w〉

1. Let C be a TM s.t. L(C) = {c} and let d 6= c.

2. Run A on < M,w ,C,d >.

3. Accept, if A accepts.

B accepts ATM, contradiction.♣


Part VI

Beyond Enumerable and co-Enumerable


Comparing sizes of sets

I Suppose A and B are two sets, and we wish to compare their sizes.

I If both A and B are finite, we can count how many elements each ofthem has, and compare the numbers.

I This method does not generalize to infinite sets.

I Alternatively, we can pair the elements of A and B. If they pair perfectly,they have equal sizes.


Correspondence

Question 38What does it mean to say that two infinite sets are of the same size?

Answered by Georg Cantor in 1873: Pair them off.

Definition 39A map f : A 7→ B is a correspondence, if f satisfies:

I One-to-one: if a1 6= a2 then f (a1) 6= f (a2).

I Onto: for every b ∈ B, there is an a ∈ A such that f (a) = b.

Definition 40A and B are of the same size, if there is a correspondence from A to B.


Correspondence – Example

Claim 41The set N of natural numbers has the same size as the set E of evennumbers.

Proof: The mapping f (i) = 2i is a correspondence from N to E. ♣

Remark 42The set E is a proper subset of the set N, yet they are the same size!


Countable sets

Definition 43A set A is countable if

I It is finite, or

I has the same size as N.

We have just seen that E is countable.

An infinite countable set is sometimes said to have size ℵ0.

Claim 44Assuming ∃ a one-to-one mapping from a set S to a countable set, then S iscountable.

Proof: Exercise. . .♣


The set of all words is countable

Theorem 45For every finite Σ, the set Σ∗ is countable.

Proof: Consider Σ = {0,1}.Define f : Σ∗ 7→ N by f (w) = bin(1w), for

bin(σn, . . . , σ1) = |Σ| · bin(σn, . . . , σ2) + bin(σ1)

(i.e., , bin(w) is the integer corresponding to w).

Therefore, f is one-to-one and onto. ♣

Why do we need the leading 1?


The set of all TM’s is countable

Claim 46The set of all TM’s is countable.

Proof:

I Each TM M has an encoding as a string 〈M〉.I Define f : TM 7→ Σ∗ as f (M) = 〈M〉.I Therefore f is a one-to-one mapping from the set of all TMs into (but not

onto) Σ∗.

I Since Σ∗ is countable, so is the set of all TMs.

♣


The set of all infinite binary strings is uncountable

Theorem 47Let B be the set of infinite binary sequences. Then B is uncountable.

Proof: Diagonalization argument, essentially identical to the proof that R isuncountable.

I Assume B is countable using f : B 7→ N.

I Let bi ∈ B be the string with f (bi ) = i .

I Define the infinite string w ∈ B, by wi = 1− (bi )i

I Assume f (w) = k . What is the value of wk ?

♣


The set of all languages is uncountable

Theorem 48Let L be the set of all languages over Σ. Then ∃ correspondence from L to B

Hence L is uncountable. Proof:

Definition 49 (The characteristic function of L)

Let si be the i ’th word of Σ∗ (in lexicographic order).Define χ(L) ∈ B by χ(L)i = 1 if si ∈ L, and 0 otherwise.

Example 50

Σ∗ = {ε, 0, 1, 00, 01, 10, 11, 000 . . .}L = { 0, 00, 01, 000 . . .}χ(L) = {0, 1, 0, 1, 1, 0, 0, 1 . . .}

Claim 51χ : L 7→ B is one-to-one and onto.

Hence, χ is a correspondence, yielding that L is uncountable.Iftach Haitner (TAU) Computational Models, Lecture 8 December 19, 2016 51 / 52

TMs vs. Languages

I The set of all TM is countable.

I The set of all languages is uncountable.

I Therefore, there are languages outside RE (why?).

I Moreover, there are languages outside RE ∪ co−RE (why?).

I This is an existential proof – it does not explicitly show any suchlanguage.


Documents

Computational Models — Lecture 81tau-cm2017a.wdfiles.com/local--files/course-schedule/TM2_h.pdf · Computational Models — Lecture 81 Handout Mode Iftach Haitner. Tel Aviv University