The Power and the Limits of Computation

Elaine Rich

The Earliest Digital Computers

1945 ENIAC

Stored 20 10-digit decimal numbers

The IBM 7090A dual 7090 system at NASA in about 1962.

Could store 32,768 36-bit words.

That’s about .00015 gigabytes.

Cost:

about $3,000,000.

or $19,794,000 2005 dollars

The Earth Simulator

The Earth Simulator (ES) is a project of Japanese agencies to develop a 40 TFLOPS system for climate modeling.

The ES site is a new location in an industrial area of Yokohama, an hour drive west of Tokyo. The facility became operational in late 2001.

Hardware

The ES is based on: 5,120 (640 8-way nodes) 500 MHz NEC CPUs

8 GFLOPS per CPU (41 TFLOPS total) 2 GB (4 512 MB FPLRAM modules) per CPU (10 TB total)

shared memory inside the node 640 × 640 crossbar switch between the nodes

16 GB/s inter-node bandwidth 20 kVA power consumption per node

The iPod Nano

2005

Can store 4 gigabytes (1000 songs).

Cost: about $250.

Compute Power Increases Over Time

From Hans Moravec, Robot Mere Machine to Transcendent Mind 1998.

Moore’s Law

What Can We Do With All that Power?

Overheard in ACES last week: “Genomics has turned biology into an information science.”

Is there anything we can’t do?

The Traveling Salesman Problem

Finding a Solution to the TSP

Given n cities:

•n choices for a starting point.•n-1 for the next city•n-2 for the next city

For a total of n! paths to be considered.

Finding a Solution to the TSP

Given n cities:

•n choices for a starting point.•n-1 for the next city•n-2 for the next city

For a total of n! paths to be considered.

We notice that it doesn’t matter where we start (since we need to make a loop).

And the cost is the same forward or backward. So we can cut the number of paths down to:

(n – 1)!/2

The Growth Rate of n!n! = n(n-1)(n-2)…(1)

2 2 11 479001600

3 6 12 6227020800

4 24 13 87178291200

5 120 14 1307674368000

6 720 15 20922789888000

7 5040 16 355687428096000

8 40320 17 6402373705728000

9 362880 18 121645100408832000

10 3628800 19 2432902008176640000

11 39916800 36 3.6 1041

Putting that Rate into Perspective

Speed of light: 3 108 m/sec

Width of a proton: 10-15 m

If we perform one operation in the time light crosses a proton, we can perform: 3 1023 ops/sec

Seconds since the big bang: 3 1017

Operations since the big bang: 9 1040

Compared to 36! 3.6 1041

The Post Correspondence Problem

i X Y

1 b bbb

2 babbb ba

3 ba a

4 bbbaa babbb

The Post Correspondence Problem

i X Y

1 b bbb

2 babbb ba

3 ba a

4 bbbaa babbb

2

X b a b b b

Y b a

The Post Correspondence Problem

i X Y

1 b bbb

2 babbb ba

3 ba a

4 bbbaa babbb

2 1

X b a b b b b

Y b a b b b

The Post Correspondence Problem

i X Y

1 b bbb

2 babbb ba

3 ba a

4 bbbaa babbb

2 1 1

X b a b b b b b

Y b a b b b b b b

The Post Correspondence Problem

i X Y

1 b bbb

2 babbb ba

3 ba a

4 bbbaa babbb

2 1 1 3

X b a b b b b b b a

Y b a b b b b b b a

The Post Correspondence Problem

i X Y

1 10 101

2 011 11

3 101 011

i X Y

1 1101 1

2 0110 11

3 1 110

The Post Correspondence ProblemA program to solve this problem:

Until a solution is found do:Generate the next candidate solution.Test it. If it is a solution, halt and report yes.

So, if there are say 4 rows in the table, we’ll try:1 2 3 4 1,1 1,2 1,3 1,4 1,5 2,1 …… 1,1,1 ….

The Post Correspondence ProblemA program to solve this problem:

Until a solution is found do:Generate the next candidate solution.Test it. If it is a solution, halt and report yes.

So, if there are say 4 rows in the table, we’ll try:1 2 3 4 1,1 1,2 1,3 1,4 1,5 2,1 …… 1,1,1 ….

But what if there is no solution?

A Tiling Problem

A Tiling Problem

A Tiling Problem

A Tiling Problem

A Tiling Problem

A Tiling Problem

A Tiling Problem

A Tiling Problem

A Tiling Problem

A Tiling Problem

Programs Debug Programs

read nif 2*int(n/2) = n then print “even” else print “odd”

read nresult = 1for i = 2 to n do

result = result * iprint result

Given an arbitrary program, can it be guaranteed to halt?

A Problem That Cannot be Solved in Any Amount of Time

read nif 2*int(n/2) = n then print “even” else print “odd”

read nresult = 1for i = 2 to n do

result = result * iprint result

Given an arbitrary program, can it be guaranteed to halt?

result1

i= 2 2 i= 3 6i= 4 24i= 5 120

Other Kinds of Loops

result = 0count = 0until result > 100 do

read n count = count + 1 result = result+nprint count

Other Kinds of Loops

result = 0count = 0until result > 100 do

read n count = count + 1 result = result+nprint count

Suppose all the inputs are positive integers.

Other Kinds of Loops

result = 0count = 0until result > 100 do

read n count = count + 1 result = result+nprint count

Suppose some of the integers are negative.

The Halting Problem Is Not Solvable

Suppose that the following program existed:

Halts(program, string) returns:•True if program halts on string•False otherwise

The Halting Problem is Not Solvable

Consider the following program:

Trouble(string) = If Halts(string, string) then loop forever else halt.

Now we invoke Trouble(<Trouble>).

What should Halts(<Trouble>, <Trouble>) say? If it:•Returns True (Trouble will halt): Trouble loops•Returns False (Trouble will not halt): Trouble halts

So there is no answer that Halts can give that does not lead to a contradiction.

Other Unsolvable Problems

PCP:

•We can encode a <program>,<input> pair as an instance of PCP so that the PCP problem has a solution iff <program> halts on <input>.

•So if PCP were solvable then Halting would be.

•But Halting isn’t. So neither is PCP.

Other Unsolvable Problems

Tiling:

•We can encode a <program>,<input> pair as an instance of a tiling problem so that there is an infinite tiling iff <program> halts on <input>.

00010000111000000111110000000000000 0001000011101000011111000000000000000010000111011000111110000000000000 …

•So if the tiling problem were solvable then Halting would be.

•But Halting isn’t. So neither is the tiling problem.