1

CDT314

FABER

Formal Languages, Automata and Models of Computation

Lecture 12

Mälardalen University

2012

2

Content

Chomsky’s Language Hierarchy Turing Machines DeterminismHaltingTM ExamplesStandard TMComputing Functions with TMCombining TMsTuring Thesis

3

Regular Languages

**ba

Context-free languagesPush-down Automata

Recursively enumerable languagesTuring Machines

}0:{ ncba nnn

}0:{ nba nn

}},{:{ bawww

}*},{:{ bawwwR

}0:{ ! nan }0:{2

nba nn

Finite Automata

ba *

4

http://www.turing.org.uk/turing/

Who was Alan Turing?

Founder of Computability Theory,

mathematician,

philosopher,

codebreaker,

visionary man before his time.

http://www.cs.usfca.edu/www.AlanTuring.net/turing_archive/index.html- Jack

Copeland and Diane Proudfoot http://www.turing.org.uk/turing/ The Alan Turing Home Page

Andrew Hodges

5

Alan Turing

1912 (23 June): Birth, London

1926-31: Sherborne School

1930: Death of friend Christopher Morcom

1931-34: Undergraduate at King's College, Cambridge University

1932-35: Quantum mechanics, probability, logic

1935: Elected fellow of King's College, Cambridge

1936: The Turing machine, computability, universal machine

1936-38: Princeton University. Ph.D. Logic, algebra, number theory

1938-39: Return to Cambridge. Introduced to German Enigma cipher machine

1939-40: The Bombe, machine for Enigma decryption

1939-42: Breaking of U-boat Enigma, saving battle of the Atlantic

6

Alan Turing

1943-45: Chief Anglo-American crypto consultant. Electronic work.

1945: National Physical Laboratory, London

1946: Computer and software design leading the world.

1947-48: Programming, neural nets, and artificial intelligence

1948: Manchester University

1949: First serious mathematical use of a computer

1950: The Turing Test for machine intelligence

1951: Elected FRS. Non-linear theory of biological growth

1952: Arrested as a homosexual, loss of security clearance

1953-54: Unfinished work in biology and physics

1954 (7 June): Death (suicide) by cyanide poisoning, Wilmslow, Cheshire.

7

Hilbert’s Program, 1900

Hilbert’s hope was that mathematics would be reducible to finding proofs (manipulating the strings of symbols) from a fixed system of axioms, axioms that everyone could agree were true.

Can all of mathematics be made algorithmic, or will there always be new problems that outstrip any given algorithm, and so require creative acts of mind to solve?

8

Turing Machines and Computability

The question Hilbert raised was whether there could be a general method or process by which one could decide whether a mathematical proposition could be proved. But what exactly was meant by a 'method' or 'process'? People had already used the concept of a 'mechanical' process, and Turing had an idea which made this quite precise: computability. Turing wrote in his first sentences:

“The "computable" numbers may be described briefly as the real numbers whose expressions as a decimal are calculable by finite means... a number is computable if its decimal can be written down by a machine. “

http://www.turing.org.uk/turing/scrapbook/machine.html

9

Turing Machines and Computability

The Turing machine concept involves specifying a very restricted set of logical operations, but Turing showed how other more complex mathematical procedures could be built out of these atomic components.

Turing argued that his formalism was sufficiently general to encompass anything that a human being could do when carrying out a definite method.

Turing's famous 1936-7 paper On computable numbers, with an application to the Entscheidungsproblem,  which worked out the theory of Turing machines and the definition of computability, is available as a PDF file on-line:

http://www.cs.ox.ac.uk/activities/ieg/e-library/sources/tp2-ie.pdf

10

Turing Machines

Based on C Busch, RPI, Models of Computation

Turing Machines

Turing’s "Machines". These machines are humans who calculate. (Wittgenstein)

A man provided with paper, pencil, and rubber, and subject to strict discipline, is in effect a universal machine. (Turing)

11

12

............Tape

Read-Write head

Control Unit

Turing Machine

13

............

Read-Write head

No boundaries -- infinite length

The head moves Left or Right

The Tape

14

............

Read-Write head

1. Reads a symbol

2. Writes a symbol

3. Moves Left or Right

The head at each time step:

15

ExampleTime 0

............ a a cb

Time 1............ a b k c

1. Reads a2. Writes k3. Moves Left

16

Time 1............ a b k c

Time 2............ a k cf

1. Reads b2. Writes f

3. Moves Right

17

Head starts at the leftmost position

of the input string

............

Blank symbol

head

a b ca

Input string

The Input String

#####

18

1q 2qLba ,

Read WriteMove Left

1q 2qRba ,

Move Right

States & Transitions

19

Example

1q 2qRba ,

............ a b ca

Time 1

1qcurrent state

#####

20

............ # a b caTime 1

1q 2qRba ,

............ a b cbTime 2

1q

2q

# # # #

# # # # #

21

............ a b caTime 1

1q 2qLba ,

............ a b cbTime 2

1q

2q

Example

# # # # #

# # # # #

22

............ a b caTime 1

1q 2qRg,#

............ ga b cbTime 2

1q

2q

Example

# # # # #

# # # #

23

Determinism

1q

2qRba ,

Allowed Not Allowed

3qLdb ,

1q

2qRba ,

3qLda ,

No lambda transitions allowed in TM!

Turing Machines are deterministic

24

Determinism

Note the difference between state indeterminismwhen not even possible future states are known in advance

and choice indeterminismwhen possible future states are known,but we do not know which state will be taken.

25

Partial Transition Function

1q

2qRba ,

3qLdb ,

............ a b ca

1q

Example

No transition

for input symbol c

Allowed

# # # # #

26

Halting

The machine halts if there are

no possible transitions to follow

27

Example

............ a b ca

1q

1q

2qRba ,

3qLdb ,

No possible transition

HALT!

# # # # #

28

Final States

1q 2q Allowed

1q 2q Not Allowed

• Final states have no outgoing transitions

• In a final state the machine halts

29

Acceptance

Accept InputIf machine halts

in a final state

Reject Input

If machine halts

in a non-final state

or

If machine enters

an infinite loop

30

Turing Machine Example

A TM that accepts the language *aa

0q

Raa ,

L,##1q

31

Time 0 aa

0q

a

0q

Raa ,

1qL,##

# # # #

32

Time 1 aa

0q

a

0q

Raa ,

1q

# # # #

L,##

33

Time 2 aa

0q

a

0q

Raa ,

1q

# # # #

L,##

34

Time 3aa

0q

a

0q

Raa ,

1qL,##

# # # #

35

Time 4 aa

1q

a

0q

Raa ,

1q

Halt & Accept

L,##

# # # #

36

Rejection Example

Time 0 ba

0q

a

0q

Raa ,

1qL,##

# # # #

37

0q

Raa ,

1q

Time 1 ba

0q

a

No possible Transition

Halt & Reject

L,##

# # # #

38

Infinite Loop Example

0q

Raa ,

1q

Lbb ,

Another TM for language *aa

L,##

39

Time 0 ba

0q

a

0q

Raa ,

1q

Lbb ,

L,##

# # # #

40

Time 1 ba

0q

a

0q

Raa ,

1q

Lbb ,

L,##

# # # #

41

Time 2 ba

0q

a

0q

Raa ,

1q

Lbb ,

L,##

# # # #

42

Time 2 ba

0q

a

baTime 3

0q

a

baTime 4

0q

a

baTime 5

0q

a

... infinite loop…

# # # #

# # # #

# # # #

# # # #

43

Because of the infinite loop:

•The final state cannot be reached

•The machine never halts

•The input is not accepted

44

Another Turing Machine Example

Turing machine for the language }{ nnba

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

45

Time 0 ba

0q

a b

Ryy ,

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

4q

L,##

# # #}{ nnba

46

bx

1q

a bTime 1

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

47

bx

1q

a bTime 2

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

48

yx

2q

a bTime 3

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

49

yx

2q

a bTime 4

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

50

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

yx

0q

a bTime 5

L,##

# # #}{ nnba

51

yx

1q

x bTime 6

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

52

yx

1q

x bTime 7

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

53

yx x y

2q

Time 8

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

54

yx x y

2q

Time 9

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

55

Ryy ,

yx

0q

x yTime 10

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

4q

L,##

# # #}{ nnba

56

yx

3q

x yTime 11

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

57

yx

3q

x yTime 12

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

58

yx

4q

x y

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

Halt & Accept

Time 13

L,##

# # #}{ nnba

59

If we modify the

machine for the language }{ nnba

we can easily construct

a machine for the language }{ nnn cba

Observation

60

Formal Definitions for

Turing Machines

61

Transition Function

1q 2qRba ,

),,(),( 21 Rbqaq

62

1q 2qLdc ,

),,(),( 21 Ldqcq

Transition Function

63

Turing Machine

),#,,,,,( 0 FqQM

Transition

functionInitial

stateblank

Final

states

States

Input

alphabetTape

alphabet

64

Instantaneous description:

ba

1q

ac

baqca 1

Configuration

# # # #

65

Time 4 Time 5

yx

2q

a b yx

0q

a b

A move:aybqxxaybq 02

# # # # # #

66

Time 4 Time 5

yx

2q

a b yx

0q

a b

bqxxyybqxxaybqxxaybq 1102

Time 6 Time 7

yx

1q

x b yx

1q

x b

# # # # # #

# # # # # #

67

bqxxyxaybq 12

Equivalent notation:

bxxqybxxqaybxqxaybq 1102

68

Initial configuration: wq0

ba

0q

a b

w

Input string

# # #

69

For any Turing Machine M

}:{)( 210 xqxwqwML f

Initial state Final state

The Accepted Language

70

Standard Turing Machine

• Deterministic

• Infinite tape in both directions

•Tape is the input/output file

The machine we described is the standard:

71

Computing Functionswith

Turing Machines

72

A function )(wf has:

Domain

(domän):

DDw

Range

(värdemängd):

S Swf )(

73

A function may have many parameters:

yxyxf ),(

Example: Addition function

74

Integer Domain

Binary: 101

Decimal: 5

We prefer unary representation:

easier to manipulate

Unary: 11111

75

Definition

A function is computable if

there is a Turing Machine such that:

fM

Initial configuration Final configuration

Dw Domain

fq

)(wf

final state0q

w

initial state

For all

# # # #

76

)(0 wfqwq f

Initial

Configuration

Final

Configuration

In other words

Dw DomainFor all

A function is computable if

there is a Turing Machine such that

fM

77

Example (Addition)

The function yxyxf ),( is computable

Turing Machine:

Input string: yx0 unary

Output string: 0xy unary

yx, are integers

78

Start

Finish 0

fq

11

yx

11

final state

0

0q

1 11 1

x y

1

initial state

# #

# #

79

0q 1q 2q 3qL,## L,01

L,11

R,##

R,10

R,11

4q

R,11

Turing machine for function yxyxf ),(

80

Execution Example:

11x

11y

Time 0

0

0q

1 11 1x y

Final Result

0

4q

1 11 1

yx

(2)

(2)

# #

# #

81

Time 0 0

0q

1 11 1

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

82

0q

01 11 1Time 1

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

83

0

0q

1 11 1Time 2

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

84

1q

1 11 11Time 3

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

85

1q

1 11 11Time 4

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

86

1q

1 11 11Time 5

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

87

2q

1 11 11Time 6

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

88

3q

1 11 01Time 7

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

89

3q

1 11 01Time 8

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

90

3q

1 11 01Time 9

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

91

3q

1 11 01Time 10

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

92

3q

1 11 01Time 11

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

93

4q

1 11 01

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

HALT & accept

Time 12 # #

L,##

R,##

yxyxf ),(

94

Another Example (Multiplication)

The function xxf 2)( is computable

Turing Machine

Input string: x unary

Output string: xx unary

x is integer

95

Start

Finish 1

fq

11

x2

11

final state

0q

11

x

1

initial state

# #

# #

96

Turing Machine Pseudocode for xxf 2)(

• Replace every 1 with $

• Repeat:

• Find rightmost $, replace it with 1

• Go to right end, insert 1

Until no more $ remain

97

0q 1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

Turing Machine for xxf 2)(

98

0q

11

3q

1 11 1

Start Finish

# # # #

0q 1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

xxf 2)(

99

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

0q

1Time 0 # # # #

0q

1 xxf 2)(

100

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

0q

1$Time 1 # # # #

0q

xxf 2)(

101

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

0q

Time 2 # # # #

0q

$ $ xxf 2)(

102

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

1q

Time 3 # # # #

0q

$ $ xxf 2)(

103

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

2q

1Time 4 # # # #

0q

$ xxf 2)(

104

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

1q

1 1Time 5 # # #

0q

$ xxf 2)(

105

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

1q

1 1Time 6 # # #

0q

$ xxf 2)(

106

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

2q

1 1Time 7 # # #

0q

xxf 2)( 1

107

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

2q

1 1Time 8 # # #

0q

xxf 2)( 1

108

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

2q

1 1Time 9 # # #

0q

xxf 2)( 1

109

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

11Time 10 # #

0q

1q

1 1 xxf 2)(

110

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

11Time 11 # #

0q

1q

1 1 xxf 2)(

111

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

11Time 12 # #

0q

1q

1 1 xxf 2)(

112

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

11Time 13 # #

0q

1q

1 1 xxf 2)(

113

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

11Time 14 # #

0q

3q

1 1 xxf 2)(

114

Another Example

The function is computable

),( yxf0

1 yx

yx

if

if

Turing Machine for

Output: 1 0or

Input: yx0

115

Turing Machine Pseudocode

Match a 1 from with a 1 from x y

• Repeat

Until all of or are matchedx y

• If a 1 from is not matched

erase tape, write 1

else

erase tape, write 0

x

116

Combining Turing Machines

117

Block Diagram

Turing

Machineinput output

118

Example

),( yxf0

yx yx

yx

if

if

Comparer

Adder

Eraser

yx, yx

yx

yx

0

119

Turing’s Thesis

120

Do Turing machines have

the same power with

a digital computer?

Intuitive answer: Yes

There is no formal answer!

Question:

121

Turing’s thesis

Any computation carried out

by mechanical means

can be performed by a Turing Machine

(1930)

http://www.engr.uconn.edu/~dqg/papers/myth.pdf The Origins of the Turing Thesis Myth Goldin & Wegner

122

And, the other way round:

A computation is mechanical

if and only if

it can be performed by a Turing Machine

Strong Turing Thesis: There is no known model of computation

more powerful than Turing Machines

Wrong! Asynchronous parallel computation is more powerful than TM model, and cannot be reduced to TM.

123

Definition of Algorithm

An algorithm for function

is a Turing Machine which computes

)(wf)(wf

124

When we say

There exists an algorithm

Algorithms are Turing Machines

It means

There exists a Turing Machine

125

Language Grammar Machine

Chomsky Hierarchy

Non-computable ??

Recursively enumerable (RE)

Unrestricted Turing machines 0

Recursive

Turing machines that always halt 0

Context-sensitive

Context-sensitive grammar

Linear-bounded automata 1

Context-free

Context-free grammar

Non-deterministic pushdown automata 2

Regular

Regular expressions Finite state automata 3

properly inclusive

Languages, Grammars, Machines

126

Regular Languages

FINITE AUTOMATA

**ba

Context-free languages

PUSH-DOWN AUTOMATA

Unrestricted grammar languages

TURING MACHINES

}0:{ ncba nnn

}0:{ nba nn

}},{:{ bawww

}*},{:{ bawwwR

}0:{ ! nan }0:{2

nba nn

Interesting further reading

Computational and evolutionary aspects of language

http://www.nature.com/nature/journal/v417/n6889/full/nature00771.html

Turing’s thesis

http://www.ams.org/notices/200610/fea-feferman.pdf

http://www.mathcomp.leeds.ac.uk/turing2012/give-page.php?408

2012 The Alan Turing World

https://sites.google.com/site/naturalcomputingaisbiacap2012 Symposium on Natural/Unconventional Computing AISB/IACAP World Congress

127