©Brooks/Cole, 2003 Chapter 17 Theory of Computation

©Brooks/Cole, 2003

Chapter 17

Theoryof

Computation

©Brooks/Cole, 2003

Understand how a simple language with limited Understand how a simple language with limited statements can solve any problem.statements can solve any problem.Understand how the Turing machine can solve any Understand how the Turing machine can solve any problem that can be solved by a computer.problem that can be solved by a computer.Understand the Godel number and its importanceUnderstand the Godel number and its importancein the theory of computation.in the theory of computation.

After reading this chapter, the reader should After reading this chapter, the reader should be able to:be able to:

OOBJECTIVESBJECTIVES

Understand the Understand the halting problemhalting problem as an example of as an example of a large set of problems that cannot be solved by a a large set of problems that cannot be solved by a computer. computer.

©Brooks/Cole, 2003

SIMPLESIMPLELANGUAGELANGUAGE

SIMPLESIMPLELANGUAGELANGUAGE

17.117.1

©Brooks/Cole, 2003

Figure 17-1

Statements in simple language

Increment statement: Increment statement: incr Xincr XDecrement statement: Decrement statement: decr Xdecr XLoop statement: Loop statement: while Xwhile X

{{ ActionsActions }}

©Brooks/Cole, 2003

Power of the Simple LanguagePower of the Simple Language

The simple language with only three The simple language with only three statements is statements is as powerful asas powerful as any any sophisticated language in use today, such sophisticated language in use today, such as C.as C.Here we show how we can Here we show how we can simulatesimulate several statements found in some popular several statements found in some popular languages.languages.We call each simulation a We call each simulation a macromacro and use and use it in other simulations without the need to it in other simulations without the need to repeat code.repeat code.

©Brooks/Cole, 2003

Macros in the Simple LanguageMacros in the Simple Language

First Macro: X First Macro: X 0 0

Second Macro: X Second Macro: X n n

while X{ decr X}

X 0incr Xincr X…incr X

©Brooks/Cole, 2003


Third Macro:Third Macro:

Y Y X X

Y 0TEMP 0while X{ incr Y decr X incr TEMP}while TEMP{ decr TEMP incr X}

©Brooks/Cole, 2003

Macros in the Simple LanguageMacros in the Simple LanguageFourth Macro: Fourth Macro: Z Z X + Y X + Y

Z XTEMP Ywhile TEMP{ incr Z decr TEMP}

Fifth Macro:Fifth Macro:

Z Z X * Y X * YZ 0TEMP Ywhile TEMP{ Z Z + X decr TEMP}

©Brooks/Cole, 2003


Sixth Macro: Z Sixth Macro: Z X ** Y X ** Y

Z 1TEMP Ywhile TEMP{ Z Z * X decr TEMP}

©Brooks/Cole, 2003

Macros in the Simple LanguageMacros in the Simple LanguageSeventh Macro:Seventh Macro:

comp ( X )comp ( X )– complementscomplements

If the value of X is 0, If the value of X is 0, change it to 1.change it to 1.

If it is not 0, change it If it is not 0, change it to 0.to 0.

TEMP 1while X{ X 0 TEMP 0}while TEMP{ incr X decr TEMP}

©Brooks/Cole, 2003

Macros in the Simple LanguageMacros in the Simple LanguageEighth Macro:Eighth Macro:

if X then A1 else A2if X then A1 else A2TEMP Xwhile TEMP{ A1 TEMP 0}TEMP Xcomp (TEMP)while TEMP{ A2 TEMP 0}

©Brooks/Cole, 2003

TURINGTURINGMACHINEMACHINETURINGTURING

MACHINEMACHINE

17.217.2

©Brooks/Cole, 2003

Figure 17-2

Turing machine

Turing machine componentsTuring machine components

Tape, read/write head, controllerTape, read/write head, controller

©Brooks/Cole, 2003

Figure 17-3

TapeTape

The Turning machine’s memory isThe Turning machine’s memory is infinite infinite..

The The tapetape holds a sequence of characters holds a sequence of characters form the set of characters acceptable by form the set of characters acceptable by the machine.the machine.

The machine can accept only a few The machine can accept only a few symbolssymbols: #, &, 1, and blank.: #, &, 1, and blank.

©Brooks/Cole, 2003

Read/write headRead/write head

The read/write head points to one symbol The read/write head points to one symbol ((the current symbolthe current symbol) on the tape.) on the tape.

The read/write head reads and writes The read/write head reads and writes oneone symbol at a time from the tape.symbol at a time from the tape.

After reading and writing, it moves to the After reading and writing, it moves to the leftleft, to the, to the right right, or , or staysstays in place. in place.

©Brooks/Cole, 2003

ControllerController

The controller is the theoretical counterpart The controller is the theoretical counterpart of the of the CPUCPU in modern computers. in modern computers.

The controller is The controller is a finite state automatona finite state automaton, a , a machine that has a machine that has a predetermined finite predetermined finite number of statesnumber of states and moves from one state and moves from one state to another based on the input.to another based on the input.

At any moment, it can be in At any moment, it can be in one one of these of these states.states.

©Brooks/Cole, 2003

Figure 17-4

Transition state

Four states : A, B, C, D.Four states : A, B, C, D.Table 17.1Table 17.1

©Brooks/Cole, 2003

Table 17.1Table 17.1 Transitional tableTransitional tableCurrentCurrent

StateState-----AABBCCDD

WriteWrite

----------------#&1

same as read blank

1same as read

1

NewNewStateState-----

BCCABDBD

ReadRead

-------------1 or blank

# or &1

not 11

not 1not a blank

blank

MoveMove

--------

A A programprogram is a modern implementation of is a modern implementation of the the transition table.transition table.

©Brooks/Cole, 2003

Simulation of Simple LanguageSimulation of Simple Language

Write programs (Write programs (create transition tablescreate transition tables) ) that implement the statements of the that implement the statements of the Simple Language.Simple Language.

– Increment statementIncrement statement

– Decrement statementDecrement statement

– Loop statementLoop statement

©Brooks/Cole, 2003

Figure 17-5

Increment statementTransition diagram for incr X

©Brooks/Cole, 2003

Table 17.2Table 17.2 Transitional table for Transitional table for incr Xincr X statement statement

CurrentCurrentStateState

---------StartIncrForwardForwardAdded

BackwardBackward

WriteWrite

----------------#11&

same as read#

NewNewStateState

----------ForwardForwardAdded

BackwardBackwardStopIncr

ReadRead

-------------#1&

anynot #

#

MoveMove

--------

©Brooks/Cole, 2003

Figure 17-6

Steps in incr X statement

Figure 17-7

Decrement statementTransition diagram for decr X

©Brooks/Cole, 2003

Table 17.3Table 17.3 Transitional table for Transitional table for decr Xdecr X statement statement


---------StartDecrForwardForward

DeleteBackwardBackward

WriteWrite

----------------#1

blank&

same as read#

NewNewStateState

----------ForwardForward

DeleteBackwardBackwardStopDecr

ReadRead

-------------#1&1

not ##

MoveMove

--------

©Brooks/Cole, 2003

Figure 17-8

Transition diagram for the loop statement

©Brooks/Cole, 2003

Table 17.4Table 17.4 Transitional table for the Transitional table for the looploop statement statement


---------StartLoop

CheckCheck

ForwardForward

……

EndProcessBackwardBackward

ReadRead

-------------#

not 11

not &&……

anynot #

#

MoveMove

--------

none……

none

NewNewStateState

---------Check

StopLoopForwardForward

StartProcess……

BackwardBackward

Check

WriteWrite

-------------#

same as read1

same as read&……

same as readsame as read

#

©Brooks/Cole, 2003

ConclusionConclusion

Any problem that can be solved by the Any problem that can be solved by the Simple Language can also be solved by Simple Language can also be solved by the Turing machine.the Turing machine.

Is there a problem solvable by the Turing Is there a problem solvable by the Turing machine that is not solvable by the Simple machine that is not solvable by the Simple Language? We can Language? We can notnot prove it now. prove it now.

Church thesis:Church thesis: the Simple Language and the Simple Language and the Turing machine are the Turing machine are equivalentequivalent. We . We believebelieve it. it.

©Brooks/Cole, 2003

GODELGODELNUMBERSNUMBERS

GODELGODELNUMBERSNUMBERS

17.317.3

©Brooks/Cole, 2003

Godel numbersGodel numbers

In theoretical computer science, an In theoretical computer science, an unsigned number is assigned to every unsigned number is assigned to every program that can be written in a specific program that can be written in a specific language.language.– ProgramsPrograms can be used as a single data item can be used as a single data item

as inputas input to another program. to another program.– Programs can be referred to by just their Programs can be referred to by just their

integer representationsinteger representations..– The numbering can be used toThe numbering can be used to prove prove that that

some problems can not be solved by a some problems can not be solved by a computer.computer.

©Brooks/Cole, 2003

Table 17.5Table 17.5 Code for symbols used in the Code for symbols used in the Simple Language Simple Language

Symbol---------

12345678

Hex CodeHex Code-------------

12345678


12345678

Symbol---------

9incrdecrwhile

{}x


9ABCDEF

©Brooks/Cole, 2003

Representing a programRepresenting a program

Example 1Example 1 What is the Godel number for the program What is the Godel number for the program incr Xincr X??

SolutionSolution

incr Xincr X

A FA F

175 (in decimal)175 (in decimal)

©Brooks/Cole, 2003

Interpreting a numberInterpreting a number

Example 1Example 1

Interpret 3058 as a program.Interpret 3058 as a program.

SolutionSolution

30583058

B F 2B F 2

decr X 2decr X 2

©Brooks/Cole, 2003

HALTINGHALTINGPROBLEMPROBLEMHALTINGHALTINGPROBLEMPROBLEM

17.417.4

©Brooks/Cole, 2003

Halting problemHalting problem

In programs, a repetition construct may In programs, a repetition construct may never terminate (never terminate (halthalt).).

For example, the following programFor example, the following program never never terminates. terminates.

X 1while X{ }

©Brooks/Cole, 2003

Can you write a program that testsCan you write a program that testswhether or not any program, whether or not any program, represented by its Godel number, represented by its Godel number, will terminate? will terminate? NoNo..

A Classical Programming Question:

Halting problemHalting problem

©Brooks/Cole, 2003

Figure 17-9

Halting problem is not solvableHalting problem is not solvable

Proof by contradiction (Proof by contradiction ( 矛盾証法矛盾証法 ))

Step 1: Assume that a program, called Step 1: Assume that a program, called TestTest, , exists.exists.

Figure 17-10

Step 2: Create another Step 2: Create another program called program called Strange Strange that is made of two that is made of two parts:parts:– a copy of Test at the a copy of Test at the

beginningbeginning– an empty loop at the an empty loop at the

endend

©Brooks/Cole, 2003

Figure 17-11

Now having made the program Strange, Now having made the program Strange, we test this program with itself as input.we test this program with itself as input.

©Brooks/Cole, 2003

ContradictionContradiction

If we assume that If we assume that Test Test exists, we have the exists, we have the following contradictions:following contradictions:– StrangeStrange does not terminate if does not terminate if StrangeStrange

terminates.terminates.– StrangeStrange terminates if terminates if StrangeStrange does not does not

terminate.terminate.

This proves that the This proves that the TestTest program cannot program cannot exist.exist.The halting problem is unsolvable.The halting problem is unsolvable.

©Brooks/Cole, 2003

SOLVABLE AND SOLVABLE AND UNSOLVABLE PROBLEMSUNSOLVABLE PROBLEMS

SOLVABLE AND SOLVABLE AND UNSOLVABLE PROBLEMSUNSOLVABLE PROBLEMS

17.517.5

©Brooks/Cole, 2003

Figure 17-12

Taxonomy of problems

©Brooks/Cole, 2003

Complexity of solvable problemsComplexity of solvable problems

One way to measure the complexity of a One way to measure the complexity of a solvable problem is to find solvable problem is to find the number of the number of operations executedoperations executed by the computer by the computer when it runs the program.when it runs the program.Big-O notationBig-O notation– For example:For example: nn : the number of input data : the number of input data– Polynomial problemsPolynomial problems: O(1), O(log: O(1), O(lognn), O(), O(nn), ),

O(O(nnloglognn), O(), O(nn22),…, O(),…, O(nnkk),…),…– Non-polynomial problemsNon-polynomial problems: O(2: O(2nn), O(), O(nn!),…!),…

©Brooks/Cole, 2003

Key termsKey terms

ControllerController

Decrement statementDecrement statement

Godel numberGodel number

Hexadecimal digitHexadecimal digit

Increment statementIncrement statement

Loop statementLoop statement

macromacro

Non-polynomial Non-polynomial problemproblem

Polynomial problemPolynomial problem

Solvable problemSolvable problem

Tape Tape

Turing machineTuring machine

Unsolvable problemUnsolvable problem

Documents

©Brooks/Cole, 2003 Chapter 17 Theory of Computation