CP405 Theory of...

CP405Theory of Computation

Test

Language: a set of stringsComputation as a test if a string is in a languageDFA: limited memory automatonRegular Language: a language accepted by a DFARegular Operations (closed): A B, AB, A∪ *, A, A∩BNFA: non-deterministic finite automatonDFA == NFA == regex (computational power)GNFA: NFA with regex transitionsProve lang. is non-regular with pumping lemmaPOSIX extended regexes are not formally regular (no corr. DFA)CFG: context-free grammar generates CFLSome languages are CF and not regularCFL are used for programming languagesAll regular languages are CFEvery CFL can be converted to Chomsky Normal Form (CNF)

- Rules are two vars, one terminal, or start->epsilonCNF string expansions require 2n-1 stepsL-Systems: formal grammars for modeling morphogenesis PDA == CFG (computational power)Non-deterministic PDA > Deterministic PDA (comp. power)Context-sensitive grammar > Context-free grammar (comp power)Prove lang. is not context-free with pumping lemmaTM > PDA (comp. power)TM can accept, reject, or loop foreverTM match intuitive definition of algorithmJava, Python, etc. are == TM (comp power)Linear Bounded Automata (LBA) == TM with finite tapeLBA == context-sensitive grammars (comp power)Post Tag System chops and concatenates strings (== TM)Queue Machine like PDA with queue in place of stack (== TM)PDA with two stacks == TMCounter Machines (pebbles in holes) with two registers (== TM)Random Access Machines provide indirect addressing (== TM)

Decidability

● Is everything computable?

● Do TMs always terminate?

Finite Languages

Regular Languages DFA/NFA

Deterministic Context-Free Languages

DPDA

Context-Sensitive LanguagesCSG/LBA

Context-Free LanguagesCFG/PDA

Something with TMs...

Decidable Languages

A language, L, is decidable (or recursive) if there exists a TM, M, that terminates correctly on every input string, w.

Decidable Languages

A language, L, is decidable (or recursive) if there exists a TM, M, that terminates correctly on every input string, w.

if w L, M terminates in the accept state ∈

if w L, M terminates in the reject state∉

Notice: M always terminates!

Decidable Languages

A language, L, is decidable (or recursive) if there exists a TM, M, that terminates correctly on every input string, w.

if w L, M terminates in the accept state ∈

if w L, M terminates in the reject state∉

A language that is not decidable is called undecidable.

Encoding Computational Models

a0

0

a1

1

a2

0,1DFA Number states: 3

Start state: 0Accept states: 2Transitions 0: 2 1Transitions 1: 1 1Transitions 2: 2 2

0,1

Encoding Computational Models

a0

0

a1

1

a2

0,1DFA Number states: 3

Start state: 0Accept states: 2Transitions 0: 2 1Transitions 1: 1 1Transitions 2: 2 2

0,13#0#2#2,1#1,1#2,2$

Encoding Computational Models

a0

0

a1

1

a2

0,1DFA Number states: 3

Start state: 0Accept states: 2Transitions 0: 2 1Transitions 1: 1 1Transitions 2: 2 2

0,13#0#2#2,1#1,1#2,2$

3#0#2#2,1#1,1#2,2$011With input:

Encoding Computational Models

a0

0

a1

1

a2

0,1DFA Number states: 3

Start state: 0Accept states: 2Transitions 0: 2 1Transitions 1: 1 1Transitions 2: 2 2

0,13#0#2#2,1#1,1#2,2$

11#0#10#10,1#1,1#10,10$011To binary:

3#0#2#2,1#1,1#2,2$011With input:

Encoded Model

Given a model M and a string input w:

⟨M, w⟩ = the binary encoding of M followed by the binary encoding of w.

ADFA = { M, w : M is a DFA that accepts w}⟨ ⟩

Is ADFA decidable?

ADFA = { M, w : M is a DFA that accepts w}⟨ ⟩

Is ADFA decidable?

Yes.

If the input does not have a legal encoding of a DFA, then reject. (No infinite loop)

Otherwise a TM can reconstruct M and then run M on w. (No infinite loop)

ANFA = { M, w : M is an NFA that accepts w}⟨ ⟩

Is ANFA decidable?

ANFA = { M, w : M is an NFA that accepts w}⟨ ⟩

Is ANFA decidable?

Yes.

First convert the NFA into an equivalent DFA. (No infinite loop)

Run the DFA and return its result. (No infinite loop)

ACFG = { G, w : G is a CFG with w L(G)}⟨ ⟩ ∈

Is ACFG decidable?

ACFG = { G, w : G is a CFG with w L(G)}⟨ ⟩ ∈

Is ACFG decidable?

???

We can have non-terminating expansions:

CFGS AB→A BB | a | → εB AAA | b | → ε

ACFG = { G, w : G is a CFG with w L(G)}⟨ ⟩ ∈

Is ACFG decidable?

???

We can have non-terminating expansions:

CFGS AB→A BB | a | → εB AAA | b | → ε

ExpansionSABBBBAAABBBBAABBAAABAABB...

ACFG = { G, w : G is a CFG with w L(G)}⟨ ⟩ ∈

Is ACFG decidable?

Yes.

First convert the CFG into CNF. (No infinite loop)

Try all derivations of length: 2n-1 (n = |w|). (No infinite loop)

ALBA = { M, w : M is an LBA that accepts w}⟨ ⟩

Is ALBA decidable?

ALBA = { M, w : M is an LBA that accepts w}⟨ ⟩

Is ALBA decidable?

Yes.

Run M on w for at most the following # of steps:

ALBA = { M, w : M is an LBA that accepts w}⟨ ⟩

Is ALBA decidable?

Yes.

Run M on w for at most the following # of steps:

QCΣC

where Q is the number of statesC is the number of tape cellsΣ is the alphabet

ATM = { M, w : M is a TM that accepts w}⟨ ⟩

Is AATM decidable?

ATM = { M, w : M is a TM that accepts w}⟨ ⟩

Is AATM decidable?

NO. �

ATM = { M, w : M is a TM that accepts w}⟨ ⟩

Assume AATM is decidable and show a contradiction:

ATM = { M, w : M is a TM that accepts w}⟨ ⟩

Assume AATM is decidable and show a contradiction:

There exists decider(M, w) always halts

Assume AATM is decidable and show a contradiction:

There exists decider(M, w) always halts

def rudeReverser(M):'''Input is a machine that takes an

encoding of a machine as input.'''return not decider(M, M)

ATM = { M, w : M is a TM that accepts w}⟨ ⟩

ATM = { M, w : M is a TM that accepts w}⟨ ⟩

Assume AATM is decidable and show a contradiction:

There exists decider(M, w) always halts

def rudeReverser(M):'''Input is a machine that takes an

encoding of a machine as input.'''return not decider(M, M)

print rudeReverser(rudeReverser)

ATM = { M, w : M is a TM that accepts w}⟨ ⟩

Assume AATM is decidable and TM D decides it.

Define TM R that takes input M :⟨ ⟩R runs D on M, M⟨ ⟨ ⟩⟩R returns the opposite result

Run R on input R ?⟨ ⟩

This sentence is false.

The barber shaves every man who does not shave himself.

Who shaves the barber?

This difficulty with ATM is called the acceptance problem

For example this means we cannot write a Python program that will determine if any other Python program will produce the correct output or loop forever!

HALT = { M, w : M is a TM that terminates on w}⟨ ⟩

This related language is even a bit simpler.

Does the TM even complete its execution on a given string?

This is called the halting problem.

Connection with Countable/Uncountable Infinities

Recall: ℕ: set of natural numbers {1, 2, 3, ...}

Any set that is finite or has a bijection with is ℕsaid to be countable.

ℤ is Countableℕ: set of natural numbers {1, 2, 3, ...}ℤ: set of integers {...,-3, -2, -1, 0, 1, 2, 3, ...}:

We need a function ƒ that is a bijection between and .ℕ ℤ

ℤ is Countableℕ: set of natural numbers {1, 2, 3, ...}ℤ: set of integers {...,-3, -2, -1, 0, 1, 2, 3, ...}:

We need a function ƒ that is a bijection between and .ℕ ℤ

Arrange elements of in this order:ℤ0, 1, -1, 2, -2, 3, -3, ...

ƒ(x) = x/2 if x is even= -(x-1)/2 if x is odd

ℚ is Countableℕ: set of natural numbers {1, 2, 3, ...}ℚ: set of rational numbers. x/y where x and y are both integers and y is not 0

We need a function ƒ that is a bijection between and .ℕ ℚ

ℚ is Countableℕ: set of natural numbers {1, 2, 3, ...}ℚ: set of rational numbers. x/y where x and y are both integers and y is not 0

We need a function ƒ that is a bijection between and .ℕ ℚ

Arrange elements of in a table:ℚ

0 1 2 3 ...1 0/1 1/1 2/1 3/1 ...2 0/2 1/2 2/2 3/2 ...3 0/3 1/3 2/3 3/3 ...

... ... ... ... ... ...

ℚ is Countableℕ: set of natural numbers {1, 2, 3, ...}ℚ: set of rational numbers. x/y where x and y are both integers and y is not 0

We need a function ƒ that is a bijection between and .ℕ ℚ

Arrange elements of in a table:ℚ

0 1 2 3 ...1 0/1 1/1 2/1 3/1 ...2 0/2 1/2 2/2 3/2 ...3 0/3 1/3 2/3 3/3 ...

... ... ... ... ... ...

ℝ is Uncountableℕ: set of natural numbers {1, 2, 3, ...}ℝ: set of real numbers.

There is no function ƒ that is a bijection between and ℕ. And no bijection between and a subset of , {x: 0 ≤ ℝ ℕ ℝ

x ≤ 1}

ℝ is Uncountableℕ: set of natural numbers {1, 2, 3, ...}ℝ: set of real numbers.

There is no function ƒ that is a bijection between and ℕ. And no bijection between and a subset of , {x: 0 ≤ ℝ ℕ ℝ

x ≤ 1}

Assume leading to contradiction that there is a bijection ƒ from to {x: 0 ≤ x ≤ 1}:ℕ

ℝ is UncountableAssume that there is a bijection ƒ from to {x: 0 ≤ x ≤ 1}:ℕ

Then we can write out each corresponding value ƒ(x) in decimal form:

f(1) = 0.3871643948827364...f(2) = 0.7738395004947231...f(3) = 0.9248472021125333......

ℝ is UncountableAssume that there is a bijection ƒ from to {x: 0 ≤ x ≤ 1}:ℕ

Then we can write out each corresponding value ƒ(x) in decimal form:

f(1) = 0.3871643948827364...f(2) = 0.7738395004947231...f(3) = 0.9248472021125333......

Since ƒ is a bijection, each x from {x: 0 ≤ x ≤ 1} should appear exactly once on the right-hand side.

ℝ is Uncountablef(1) = 0.3871643948827364...f(2) = 0.7738395004947231...f(3) = 0.9248472021125333......

Let's create a number, y, in the following way:

y =

ℝ is Uncountablef(1) = 0.3871643948827364...f(2) = 0.7738395004947231...f(3) = 0.9248472021125333......

Let's create a number, y, in the following way:

y = 0.4

different!

ℝ is Uncountablef(1) = 0.3871643948827364...f(2) = 0.7738395004947231...f(3) = 0.9248472021125333......

Let's create a number, y, in the following way:

y = 0.44

different!

ℝ is Uncountablef(1) = 0.3871643948827364...f(2) = 0.7738395004947231...f(3) = 0.9248472021125333......

Let's create a number, y, in the following way:

y = 0.445

different!

f(1) = 0.3871643948827364...f(2) = 0.7738395004947231...f(3) = 0.9248472021125333......

Let's create a number, y, in the following way:

y = 0.445...

So y is in {x: 0 ≤ x ≤ 1}, but it does not appear on the right-hand side of the bijection.

ℝ is Uncountable

f(1) = 0.3871643948827364...f(2) = 0.7738395004947231...f(3) = 0.9248472021125333......

Let's create a number, y, in the following way:

y = 0.445...

So y is in {x: 0 ≤ x ≤ 1}, but it does not appear on the right-hand side of the bijection. Contradiction!

So we cannot have a bijection and {x: 0 ≤ x ≤ 1} must be bigger.

ℝ is Uncountable

Diagonalization for Halting ProblemEvery TM can be encoded in a finite length binary string <M>.

We can build a table with both rows and columns corresponding to particular TMs:

⟨TM0⟩ ⟨TM1⟩ ⟨TM2⟩ ⟨TM3⟩ ...TM0 1 1 0 1 ...TM1 0 0 0 1 ...TM2 1 0 0 0 ...TM3 0 0 0 1 ...

... ... ... ... ... ...

⟨TM0⟩ ⟨TM1⟩ ⟨TM2⟩ ⟨TM3⟩ ...TM0 1 1 0 1 ...TM1 0 0 0 1 ...TM2 1 0 0 0 ...TM3 0 0 0 1 ...

... ... ... ... ... ...

Diagonalization for Halting ProblemEvery TM can be encoded in a finite length binary string <M>.

We can build a table with both rows and columns corresponding to particular TMs:

Does TM1 halt when given ⟨TM2⟩ as input?

⟨TM0⟩ ⟨TM1⟩ ⟨TM2⟩ ⟨TM3⟩ ...TM0 1 1 0 1 ...TM1 0 0 0 1 ...TM2 1 0 0 0 ...TM3 0 0 0 1 ...

... ... ... ... ... ...

Diagonalization for Halting ProblemEvery TM can be encoded in a finite length binary string <M>.

We can build a table with both rows and columns corresponding to particular TMs:

If we assume HALT is decidable, then there must exist a TM that returns these table values.

Diagonalization for Halting ProblemEvery TM can be encoded in a finite length binary string <M>.

We can build a table with both rows and columns corresponding to particular TMs:

⟨TM0⟩ ⟨TM1⟩ ⟨TM2⟩ ⟨TM3⟩ ...TM0 1 1 0 1 ...TM1 0 0 0 1 ...TM2 1 0 0 0 ...TM3 0 0 0 1 ...

... ... ... ... ... ...

Consider a TM, RudeReverser:Given x = TM⟨ n, TM⟨ n ,⟩⟩Run decider on x.Return the opposite value.

Diagonalization for Halting ProblemEvery TM can be encoded in a finite length binary string <M>.

We can build a table with both rows and columns corresponding to particular TMs:

⟨TM0⟩ ⟨TM1⟩ ⟨TM2⟩ ⟨TM3⟩ ... ⟨RR⟩

TM0 1 1 0 1 ...TM1 0 0 0 1 ...TM2 1 0 0 0 ...TM3 0 0 0 1 ...

... ... ... ... ... ...RR

Create a TM, RudeReverser:Given x = TM⟨ n, TM⟨ n ,⟩⟩Run decider on x.Return the opposite value.

RudeReverser is a TM so it must be somewhere in the table.

Diagonalization for Halting ProblemEvery TM can be encoded in a finite length binary string <M>.

We can build a table with both rows and columns corresponding to particular TMs:

Create a TM, RudeReverser:Given x = TM⟨ n, TM⟨ n ,⟩⟩Run decider on x.Return the opposite value.

RudeReverser is a TM so it must be somewhere in the table.

⟨TM0⟩ ⟨TM1⟩ ⟨TM2⟩ ⟨TM3⟩ ... ⟨RR⟩

TM0 1 1 0 1 ...TM1 0 0 0 1 ...TM2 1 0 0 0 ...TM3 0 0 0 1 ...

... ... ... ... ... ...RR 0 1 1 0

⟨TM0⟩ ⟨TM1⟩ ⟨TM2⟩ ⟨TM3⟩ ... ⟨RR⟩

TM0 1 1 0 1 ...TM1 0 0 0 1 ...TM2 1 0 0 0 ...TM3 0 0 0 1 ...

... ... ... ... ... ...RR 0 1 1 0 0

Diagonalization for Halting ProblemEvery TM can be encoded in a finite length binary string <M>.

We can build a table with both rows and columns corresponding to particular TMs:

Create a new TM, RudeReverser:Given x = TM⟨ n, TM⟨ n ,⟩⟩Run decider on x.Return the opposite value.

If the decider returns 1, then RudeReverser must reverse it to 0.

⟨TM0⟩ ⟨TM1⟩ ⟨TM2⟩ ⟨TM3⟩ ... ⟨RR⟩

TM0 1 1 0 1 ...TM1 0 0 0 1 ...TM2 1 0 0 0 ...TM3 0 0 0 1 ...

... ... ... ... ... ...RR 0 1 1 0 1

Diagonalization for Halting ProblemEvery TM can be encoded in a finite length binary string <M>.

We can build a table with both rows and columns corresponding to particular TMs:

Create a new TM, RudeReverser:Given x = TM⟨ n, TM⟨ n ,⟩⟩Run decider on x.Return the opposite value.

But if this value is 0, then the decider should return 0 and RudeReverser must reverse it to 1!

⟨TM0⟩ ⟨TM1⟩ ⟨TM2⟩ ⟨TM3⟩ ... ⟨RR⟩

TM0 1 1 0 1 ...TM1 0 0 0 1 ...TM2 1 0 0 0 ...TM3 0 0 0 1 ...

... ... ... ... ... ...RR 0 1 1 0 0

Diagonalization for Halting ProblemEvery TM can be encoded in a finite length binary string <M>.

We can build a table with both rows and columns corresponding to particular TMs:

And we're left with the zero-one contradiction monster.

1

Rice's Theorem

All non-trivial properties of Turing machines (algorithms) are undecidable!

P1 = { M : M is a TM and L(M) = {1, 10}}⟨ ⟩P2 = { M : M is a TM and L(M) = }⟨ ⟩ ∅P3 = { M : L(M) is finite}⟨ ⟩

Rice's Theorem Definition

T = the set of all binary-encoded TMs

For all P T where:⊂● P ≠ and∅● encodings of any two TMs that have the same language are either both in P or both not in P

Then, P is undecidable.

Rice's Theorem

P4 = { M : M has 42 states}⟨ ⟩

Does P4 fit Rice's Theorem criteria?

Rice's Theorem

P4 = { M : M has 42 states}⟨ ⟩

Does P4 fit Rice's Theorem criteria?

No, because we could have:

TMA and TMB with L(TMA) = L(TMB)and TMA with 42 states and TMB with 43

states

Proof of Rice's Theorem

Show a connection between a language with a certain property, LP, and the halting problem.

In particular, if we can show that LP being decidable makes the halting problem decidable, then we have a contradiction.

Proof of Rice's Theorem

M∅ is a TM that accepts . Assume M∅ ⟨ ∅ ⟩ ∉ LP

Myes is a TM such that M⟨ yes ⟩ ∈ LP

Proof of Rice's Theorem

M∅ is a TM that accepts . Assume M∅ ⟨ ∅ ⟩ ∉ LP

Myes is a TM such that M⟨ yes ⟩ ∈ LP

Note that L(M∅) ≠ L(Myes)

Assume leading to contradiction that there is a decider for the language LP.

x Recognizerfor M

yes

keithswarz.com

x

Recognizerfor M

yes

x

Recognizerfor M

yes

SimulateM on w

x

Recognizerfor M

yes

SimulateM on w

x

Recognizerfor M

yes

x

SimulateM on w

x

Recognizerfor M

yes

x

Machine M'

SimulateM on w

x

Recognizerfor M

yes

x

M' = “On input x:Run M on w.If M halts, run M

yes on x.

Accept if Myes

accepts x.

Reject if Myes

rejects x.”

Machine M'

SimulateM on w

x

Recognizerfor M

yes

x

M' = “On input x:Run M on w.If M halts, run M

yes on x.

Accept if Myes

accepts x.

Reject if Myes

rejects x.”

If M loops on w, what does M' do

on x?

Machine M'

SimulateM on w

x

Recognizerfor M

yes

x

M' = “On input x:Run M on w.If M halts, run M

yes on x.

Accept if Myes

accepts x.

Reject if Myes

rejects x.”

If M loops on w, what does M' do

on x?

Machine M'

SimulateM on w

x

Recognizerfor M

yes

x

M' = “On input x:Run M on w.If M halts, run M

yes on x.

Accept if Myes

accepts x.

Reject if Myes

rejects x.”

If M loops on w, what does M' do

on x?

M' always loops.

Machine M'

SimulateM on w

x

Recognizerfor M

yes

x

M' = “On input x:Run M on w.If M halts, run M

yes on x.

Accept if Myes

accepts x.

Reject if Myes

rejects x.”

If M loops on w, what does M' do

on x?

L(M') = Ø

Machine M'

SimulateM on w

x

Recognizerfor M

yes

x

M' = “On input x:Run M on w.If M halts, run M

yes on x.

Accept if Myes

accepts x.

Reject if Myes

rejects x.”

Machine M'

SimulateM on w

x

Recognizerfor M

yes

x

M' = “On input x:Run M on w.If M halts, run M

yes on x.

Accept if Myes

accepts x.

Reject if Myes

rejects x.”

If M halts on w, what does M' do

on x?

Machine M'

SimulateM on w

x

Recognizerfor M

yes

x

M' = “On input x:Run M on w.If M halts, run M

yes on x.

Accept if Myes

accepts x.

Reject if Myes

rejects x.”

If M halts on w, what does M' do

on x?

Machine M'

SimulateM on w

x

Recognizerfor M

yes

x

M' = “On input x:Run M on w.If M halts, run M

yes on x.

Accept if Myes

accepts x.

Reject if Myes

rejects x.”

If M halts on w, what does M' do

on x?

M' accepts x ifMyes accepts x

Machine M'

SimulateM on w

x

Recognizerfor M

yes

x

M' = “On input x:Run M on w.If M halts, run M

yes on x.

Accept if Myes

accepts x.

Reject if Myes

rejects x.”

If M halts on w, what does M' do

on x?

L(M') = L(Myes)

Machine M'

The Complete Construction

Decider for L

P

keithswarz.com

The Complete Construction

Decider for L

P

⟨M, w ⟩

The Complete Construction

Decider for L

P

ConstructM' from⟨M, w⟩

⟨M, w ⟩

The Complete Construction

SimulateM on w

x

Recognizerfor M

yes

x

Decider for L

P

ConstructM' from⟨M, w⟩

⟨M, w ⟩

The Complete Construction

SimulateM on w

x

Recognizerfor M

yes

x

Decider for L

P

ConstructM' from⟨M, w⟩

⟨M'⟩ ⟨M, w ⟩

The Complete Construction

SimulateM on w

x

Recognizerfor M

yes

x

Decider for L

P

ConstructM' from⟨M, w⟩

⟨M'⟩ ⟨M, w ⟩

What happens if M halts on w?

The Complete Construction

SimulateM on w

x

Recognizerfor M

yes

x

Decider for L

P

ConstructM' from⟨M, w⟩

⟨M'⟩ ⟨M, w ⟩

What happens if M halts on w?

L(M') L∈P

The Complete Construction

SimulateM on w

x

Recognizerfor M

yes

x

Decider for L

P

ConstructM' from⟨M, w⟩

⟨M'⟩ ⟨M, w ⟩

What happens if M halts on w?

L(M') L∈P

The Complete Construction

SimulateM on w

x

Recognizerfor M

yes

x

Decider for L

P

ConstructM' from⟨M, w⟩

⟨M'⟩ ⟨M, w ⟩

The Complete Construction

SimulateM on w

x

Recognizerfor M

yes

x

Decider for L

P

ConstructM' from⟨M, w⟩

⟨M'⟩ ⟨M, w ⟩

What happens if M loops on w?

The Complete Construction

SimulateM on w

x

Recognizerfor M

yes

x

Decider for L

P

ConstructM' from⟨M, w⟩

⟨M'⟩ ⟨M, w ⟩

What happens if M loops on w?

L(M') L∉P

The Complete Construction

SimulateM on w

x

Recognizerfor M

yes

x

Decider for L

P

ConstructM' from⟨M, w⟩

⟨M'⟩ ⟨M, w ⟩

What happens if M loops on w?

L(M') L∉P

The Complete Construction

SimulateM on w

x

Recognizerfor M

yes

x

Decider for L

P

ConstructM' from⟨M, w⟩

⟨M'⟩ ⟨M, w ⟩

The Complete Construction

SimulateM on w

x

Recognizerfor M

yes

x

Decider for L

P

ConstructM' from⟨M, w⟩

⟨M'⟩ ⟨M, w ⟩

This machine decides HALT!