Alan Turing and the Foundations of Computer … Turing and the Foundations of Computer Science Introduction Formalism Around the turn of the 19th century, there was a turn towards

Alan Turing and the Foundations of Computer Science

Alan Turing and the Foundations ofComputer Science

Waseda University, SILS,History of Mathematics

Alan Turing and the Foundations of Computer ScienceOutline

Introduction

Hilbert’s Program

Foundations and Crisis

Alan Turing and the Decidability Problem

Computation and Computers

Alan Turing and the Foundations of Computer ScienceIntroduction

FormalismAround the turn of the 19th century, there was a turn towardsformalism among mathematicians. G. Frege and G. Peanobegan to formalize logic and arithmetic, E. Zermelo establishedaxiomatic set theory, and all of this was continued byWhitehead and Russell in their Principia Mathematica.

D. Hilbert gave an address titled, “Axiomatic thought,” inwhich he argued that mathematicians should use mathematicallogic to develop proof theory and metamathematics in order todetermine whether or not a given set of axioms are independentand consistent, and whether or not the theorems developedfrom them are complete and decidable.

In the following years, mathematicians like K. Gödel, A.Church and A. Turing began to address these questions usingnew mathematical approaches.


Giuseppe Peano’s axioms of arithmetic (1889)

Unity: 1. 1 P N.Equality: 2. n P N ñ n “ n (reflexive).

3. n,m P N,n “ m ñ m “ n (symmetric).4. n,m, o P N,n “ m,m “ o ñ n “ o (transitive).5. n P N,n “ m ñ m P N (closed).

Successor: Where Spnq “ n ` 1,6. n P N ñ Spnq P N.7. n,m P N,n “ m ðñ Spnq “ Spmq (injection).8. n P N ñ Spnq ‰ 1.

Induction: 9. K ..“ t1 P K,n P N,n P K ñ Spnq P Ku ñ K “ N.

[Peano, did not consider 0 to be a member of the naturalnumbers. In those places where he used 1, we now use 0.]


Independence

For a set of axioms to be independent means that there should beno superfluous axioms. That is, no individual axiom shoulddemonstrable with the use of any set of the remaining axioms.

For example, for many centuries some mathematicians believedthat Euclid’s 5th (parallel) postulate, and tried to prove it fromthe other postulates and, hence, tried to prove it from the other“axioms.” Eventually, with the development of non-Euclideangeometry, and a reinvestigation of the previous attempts toprove the 5th postulate, it was realized that all of the previousproofs of the postulate relied on some assumption that wasmathematically equivalent to the 5th postulate. Hence, Hilbertset out an equivalent axiom as independent and necessary forEuclidean geometry.


Consistency

For an axiomatic system to be consistent, there must be nocontradictions among the axioms and the theorems that can bederived from them. That is, it cannot be possible to use someset of axioms to show a contraction with another axiom, or twoderive two theorems which contradict one another.

If you can use an axiomatic system to derive both P and not P –a contraction – then the system is inconsistent, andmathematically worthless.

Hence, the question arises, given some set of axioms, is thereany way that we can determine whether or not they areconsistent.


Completeness

When an axiomatic system is complete it means that allstatements that can be made up of the terms defined in thesystem can be shown to be either valid or invalid – that is can beproved to follow from the axioms, or proved to contradict oneor more of the axioms. More loosely, it means that every truetheorem can be derived from the axioms.

That is, a system of axioms, A, is complete when given anystatement P, it is either the case that

A ñ P, or A ñ not P.

This leans to the final issue, having to do with the question ofwhether or not we can know whether or not A ñ P, orA ñ not P.


Decidability

An axiomatic system is decidable when we can state a generalprocedure that can be used to determine whether or not A ñ P,or A ñ not P. That is, irregardless of whether or not a systemis complete, does a decidability procedure exist?

For example, if we could state some procedure that would showthat A œ P, or A œ not P, then the system would beincomplete, but it might still be decidable.

Might there be such a decidability procedure that coulddetermine whether or not P or not P are true, even if they arenot provable?

In the early decades of the 20th century, a number ofmathematicians, at the urging of D. Hilbert, began to examinethese metamathematical questions.

Alan Turing and the Foundations of Computer ScienceHilbert’s Program

David Hilbert (1862–1943)

§ Born to a middle class Prussianfamily.

§ Educated at Göttingen, and aftersome years at Königsberg, wasprofessor of mathematics atGöttingen for most of his life.

§ He and Keine developed animportant school of mathematicians,of which he was the leading figure.

§ Developed and worked in a broadrange of areas.

§ Advocated for formalism and rigorin mathematics.


The 1900 International Congress of Mathematics

At the 2nd ICM, Paris, Hilbert was asked to give one of themajor general addresses. At the suggestion of his colleague H.Minkoski, he chose to talk about what he imagined would bemajor goals of mathematics in the 20th century. He set out a listof 10 problems (out of 23 published later) that are known asHilbert’s problems and became deeply influential amongmathematicians.

Hilbert’s Address, 1900“[The] conviction of the solvability of every mathematicalproblem is a powerful incentive to the worker. We hear withinus the perpetual call: There is the problem. Seek its solution.You can find it by pure reason, for in mathematics there is noignorabimus.”


Hilbert’s Problem 10 (the Entscheidungsproblem)

Problem 1 Show that the cardinality of the continuum is thenext transfinite number after that of the naturalnumbers.

Problem 2 Show that the axioms of arithmetic consistent.Problem 8 Demonstrate the Riemann hypothosis, Goldbach’s

conjecture, the twin prime conjecture, etc.

10. Entscheindung der Lösbarkeit einer diophantischenGleichung. (Determination of the Solvability of a DiophantineEquation.)

Given a diophantine equation . . . : To devise a process according towhich it can be determined by a finite number of operation whether theequation is solvable in rational integers.

Alan Turing and the Foundations of Computer ScienceFoundations and Crisis

Foundations and Crisis

The goal of the foundations of mathematics was to set out asystem of logical or mathematical axioms from which all thevalid theorems of mathematics could be derived. In order toshow that this had been done it would then be necessary toshow that these axioms were independent and consistent, andthat the system was complete and all theorems were decidable.

As the decades progressed, however, it began to becomeincreasingly clear that there might be problems with thisproject. Russel discovered a paradox concerning the set of allsets that are not members of themselves, R ..“ tx | x R xu, andwrote the Principia Mathematica with Whitehead, in which theyspent hundreds of pages to prove that 1 ` 1 “ 2. Finally, Gödeland Turing showed that even relatively simple sets of axiomsare not complete and not decidable.


Gödel’s Completeness Theorem

Kurt Gödel (1906–1978) was Austrian mathematician who latersettled in the US and worked at the Princeton Institute forAdvanced Studies.

In 1929, in his PhD thesis, Gödel demonstrated that first-orderpredicate logic – a logic with relations and quantifiers – iscomplete. This means that every semantically valid theorem issyntactically derivable from the axioms. A number of simplerversions of this proof were given over the following decades.

Although this was a significant result, it was not unexpected.For mathematics, however, the main issue was that even themost basic set of axioms – like Peano’s axioms of arithmetic, orZermelo’s axioms of set theory, required extra, purelymathematical axioms.


Gödel’s Incompleteness Theorems

Gödel announced, in 1930, that he had shown that if sufficientaxioms were added to first-order logic to allow the derivation ofarithmetic, then the system was incomplete. In 1931, hepublished his two incompleteness theorems.

The first theorem states that no consistent set of axioms that canbe listed by an effective procedure is capable of proving allstatements that are true of the natural numbers. The secondincompleteness theorem, which is a generalization, states thatsuch a system cannot demonstrate its own consistency.

The key to the proofs was an argument analogous to Cantor’sdiagonal argument. We assume a complete listing, and showthat it is incomplete. It is often claimed that these theoremsshow that all axiomatic systems are incomplete, but this is false.

Alan Turing and the Foundations of Computer ScienceAlan Turing and the Decidability Problem

Alan Turing (1912–1954)

§ Turing was born to a British colonialfamily and his father was often away inIndia.

§ Took a first class in mathematics fromKing’s College, Cambridge.

§ Became fellow at Kings, worked withA. Church in Princeton.

§ During WWII, played a lead role in thebreaking of German codes at BletchleyPark.

§ Worked on early electronic computers.§ Was arrested for homosexuality,

medicated, and died mysteriously.


“On Computable Numbers. . . ” (1936)

In 1935, in the first year of his fellowship at King’s College, hewrote a paper called “On Computable Numbers, with anApplication to the Entsheidungsproblem.”

§ He defines a computable number as a real number whosedigits can be computed by finite means.

§ Defines a idealized computing machine (Turing Machine,TM), a provides and introduces the idea of an enumerationof computing machines.

§ Argues that there is a universal computing machine(UTM).

§ Uses the diagonal argument – as developed by Cantor andGödel – to show that some numbers are not computable,which is another way of saying that they are undecidable.


Abstracting the Idea of Computation

“On Computable Numbers. . . ,” 1936Computing is normally done by writing certain symbols onpaper. We may suppose this paper divided into squares like achild’s arithmetic book. . . I think it can be agreed that thetwo-dimensional character of the paper is no essential ofcomputation. I assume then that the computation is carried outon one-dimensional paper, i.e. on a tape divided into squares. Ishall suppose that the number of symbols which may beprinted is finite.. . . The behavior of the computer at anymoment is determined by the symbols that he is computing,and his ‘state of mind’ at that moment.. . . Let us imagine thatthe operations performed by the computer to be split up into‘simple operations’ which are so elementary that it is not easyto imagine them further divided.


The Turing Machine

In order to address the question of computability, Turingdevised a conceptual model of a computing device, which waslater called a “Turing machine” (TM). It consists of a limitlesstape, a head, and an instruction table.

§ The tape is divided up into cells, each of which can beblank, or contain a symbol from some finite set. The tapemay also contain instructions for the machine.

§ The head can read and write symbols in the cells of thetape, and can move.

§ The instruction table issues the following possibleinstructions: move right, move left, print, change state, halt.

No mechanisms for performing any of these functions aredescribed in the paper – the machine is purely conceptual.

. . . s1 s1 s2 s2 s1 s1 s1 s2 s1 . . . Tape

dc

b

a . . .

e

Instruction Table

c

Head (reads, writes and movesin either direction)


A Basic Program

As an example, Turing gives the following “instruction table”:

State Scanned Square Operation Next statea blank P[0], R bb blank R cc blank P[1], R dd blank R a

This table starts out with a blank tape – actually, it will fail if thetape is not blank – and prints the following sequence:

. . . 1 0 1 0 1 0 1 . . .


The Universal Turing Machine

He then gives an idealized construction of a general machine –that is, a general instruction table – that can read and executeinstructions coded onto the tape. In this way, the machine canbe controlled by some set of instructions coded onto the tapeitself, which acts as a stored-program.

Since any program that we like may be coded onto the tape, thegeneral machine can carry out any computation that anindefinitely large set of instruction tables can carry out. That is,the set of programs that can be run the general machine is thedenumerable set, ℵ0.

This conceptual machine was later called a “universal Turingmachine” (UTM), or simply a Turning machine.


The Entscheidungsproblem (Decidability Problem)

Turing approached the Entscheidungsproblem by an argumentthat was explicitly analogous to the diagonal argumentdeveloped by Cantor to show that the reals arenon-denumerable. He argued that if we set all of thecomputable numbers out in a denumerable table, – for example,we have a TM to compute 2, another for

?2, another for π, and

so on – then we can define a number by the diagonal process.Then we can ask whether or not another TM can be writtenwhich can compute that number. That is, whether or not it ispossible to decide if the process of defining that diagonalnumber halts. Turing showed that this is not always possible.

The generalization of this claim involves the followingassumption.


The Church-Turing Thesis

The Church-Turing thesis is the claim that any computationthat can be carried out by a rote method – a well-definedeffective procedure – can be carried out by a UTM. That is, aUTM “can do anything that could be described as ‘rule ofthumb’ or ‘purely mechanical.’”

This is a transformation of the concept of “computable” tomean anything that can be carried out on a UTM. An extensionof this is the claim that any process that can be carried out onone UTM can also be carried out on any other UTM.

It is sometimes claimed that this means that a UTM can carryout any computation that can be carried out by any machine, butthis is not true – a quantum computer or analog machine thatincorporates a random mechanism are clear counter examples.

Alan Turing and the Foundations of Computer ScienceComputation and Computers

The Concept of the Stored-Program Computer

Contained in Turing’s 1936 paper was a concept that wouldplay a fundamental role in the development of computers astechnological objects – namely, the idea of the stored-programcalculator, or computer. The key idea is that the logical controland arithmetical functions could be written into the core of theUTM, and any auxiliary program could be written out on thetape and read in as necessary.

This idea was popularized by John von Neumann (1903–1957),a Jewish-Hungarian mathematician who immigrated to the US.He worked closely with American engineers and encouragedthem to read Turing’s 1936 paper. Von Neumann later went onto write a government report called “First Draft of a Report onthen EDVAC.” This report introduced the idea of thestored-program computer to a wider readership of engineers.


British Codebreaking in WWIIStarting in 1926, the German military adopted the use of theEnigma encoding machine, made by the firm of Scherbius &Ritter. The Dutch, Japanese, and Italians followed them.

Before the outbreak of war, the Poles bought an Enigma andtheir mathematicians designed a machine, the Bomba, todecode it. When they realized the Germans were going toattack, they gave the machines to the British and the Polishmathematicians taught the Brits what they had learned.

The British govenment set up a code breaking division atBletchley Park, where they built “Bombes” and eventuallyColossus computers, to decode the new German Lorenzencoders. The first Colossus used 1,500 vacuum tubes and readpaper tape at 5,000 characters per second. The Colossidecrypted around 63,000,000 German characters of code.


The Enigma Machines

The Enigma machines were a series of rotor cipher machinesthat combined mechanical rotors with electrical sub-systems.Mechanical: Keyboard for the input letters, rotating disks on a

spindle, a stepping mechanism to rotate the disks.Electrical: A circuit that varied with different rotor positions,

a plugboard that could be reconfigured by theoperator, and lamps for the output cipher letters.

Both German operators had a series of daily keys that theywould use to set up the machines. Then the transmitter wouldenter letters on the keyboard and the lamp would indicate theoutput characters. This message would be transmittednormally, and as long as the receiving operator had the sameset-up, they could ender the cyphers and receive normalGerman output.

A German Enigma Machine


Turing at Bletchley ParkBletchley Park was a manor house that had been converted tothe use of the Government Code and Cypher School. HereTuring worked on a team with a number of othermathematicians and engineers. Based on the previous work ofthe Poles, in 1939, Turing and Gordon Welchman designed aBombe that could elecro-mechanically search for a certainstring – called a crib – at a rate of 20 positions per second.When the Bombe stopped, the positions would be tried on anEngima to see if it produced German. If not, they would keepgoing, if so, the messages for that section of the Germanmilitary could be read in near real time for the rest of the day.The following day thw whole process would begin again.In 1943, T. Flowers built Colossus, a large-scale electroniccomputer based on designs by Max Newman, one of Turing’smathematics professors, to decrypt the new Lorentz machines.


Turing’s Version of the Bombe (Reconstruction)


A Bombe machine in the US


The Colossus at Bletchley Park


Turing at the National Physical Laboratory (NPL)

At the NPL in Bushy Park, London, Turing set out the designsfor an electronic stored-program computer, called theAutomatic Computing Engine (ACE). In 1946, he wrote a reportcalled “Proposed Electronic Calculator” that included many ofvon Neumann’s ideas, but set out detailed designs for logiccontrol and programming.

However, because of the secrecy about the work at BletchleyPark, Turing was not able to tell the engineers about theColossus, and they did not believe that it was possible to builda fully electronic devise – also Flowers could not be employedto do the job. Hence, Turing’s full design was not implementedand first only a pilot ACE was built. In 1948, Turing left theNPL and moved to Manchester.


Turing at Manchester

At Manchester, Turing joined Newman’s group, including theengineers Williams and Kilburn, who were working on theManchester Baby, and the Mark I, which werefully-implemented, electronic, stored-program computers –having worked with and been influenced by the people at bothBletchley Park and the Moore School in the US.

When he arrived at Manchester, Turing finally had a fullyfunctional general-purpose electronic computer to work with.He designed the input mechanism, the programming systemand wrote a programming manual. Here, he frustrated theengineers by working directly in the base-32 of the machine,since he found it more efficient to program in this base systemthan to convert everything back and forth between decimal.


Artificial Intelligence

Before he started at Manchester, Turing took a sabbatical atKing’s College and wrote a report called “IntelligentMachinery” (IM). This was a highly philosophical paper thatsummarized many of his ideas on machine intelligence inwhich he introduced the concepts of logic-based problemsolving and genetic or evolutionary search. He claims in thispaper that “intellectual activity consists mainly of various kindsof search.” (At Manchester, Turing wrote one of the first chestprograms – purely on paper.)

A major concept of the IM paper was that unorganizedmachines could act like neurons and by following simple rulesorganize themselves into networks – now called Turing nets –and eventually carry out general computation.


Artificial LifeIn the final years of his life, Turing worked in the field that isnow known as artificial life (A-life). The goal of A-life is toproduce a theoretical understanding of the way that organismsself-organize.

Turing used the early Ferranti computers (the commercialversions of the Mark I) to study the spontaneous formation ofstructure related to early cell division in embryos; symmetricalstructures in shell fish, starfish and flowers; the arrangement ofleaves and branches in plants; and color patterns, such as spotsand striping on animals and fish.

Unfortunately, in the middle of this work, he died. It isgenerally believed that he committed suicide, but there areproblems with this interpretation (coroner’s report mentionsviolence, no evidence of depression, etc.).


The Development of Computer ScienceThe Turing machine notion of computability, the Church-Turingthesis, and the stored-program concept of the computer wereall developed originally to show that certain mathematicalproblems were undecidable – that is, to show mathematical andlogical limits to what is possible in mathematics.

To Turing himself, however, and to the first generation ofcomputer programers – on paper – and engineers, moreinteresting questions soon began to circulate around whatcould actually be done with these Turing machines.

The new computer science stimulated work in mathematics invarious ways – attention to detail around computation and basesystems, new logical theories to study the structural similaritiesbetween programs and proofs, new experimental methods forinvestigating mathematical claims, and so on.

Documents

Alan Turing and the Foundations of Computer … Turing and the Foundations of Computer Science Introduction Formalism Around the turn of the 19th century, there was a turn towards