1

Ethics of Computing MONT 113G, Spring 2012

Session 13Limits of Computer Science

2

Limitations of Computer Science

In practice, we cannot solve every problem with a program because...

1) The execution time of a program may be too long.

2) The problem may be non-computable.

3) We may not know how to solve the problem.E.g. Computer Vision

Artificial IntelligenceModeling complex systems such as weatheretc.

3

Execution Time

The execution time of a program can be a limitation in what we can compute.

Examples:

Taxes: How long would it take to sort taxpayers by Social Security number?

Electricity: What is the most efficient routing of power lines through a region or neighborhood?

Tractable problems can be solved in a reasonable amount of time.

Intractable problems require huge amounts of time to solve.

4

A Tractable Problem

Suppose we have a list of patients and an associated list of patient weights.

We want to write a program that prints out the names of patients with weight above a given value.

The names are stored in a list of strings.The weights are stored in a parallel list of real numbers.

Amy Ted Ann Tom

name

1 2 3 4 5 . . . n

115 145 132 224

weight

1 2 3 4 5 . . . n

. . .

. . .

5

The solution and its running time

The program examines each patient in turn.The running time is proportional to the number of patients in the list (n).

n (# patients) time (seconds) 2500 1.275 5000 2.550 7500 3.82510000 5.100

time

n

5.1

10000

Python solution:for i in range(n):

if weight[i] > targetWeight:print name[i]

6

Linear Running time

The graph of the running time of the previous problem is a line.

t = 5.1 x 10-4 n

We say the the running time is linear.

Problems that have linear running times can be solved in a reasonable amount of time. These problems are tractable.

7

Another example

Suppose we want to sort the patients by their weight. I.e. we would like to rearrange the list so that the weights go from lowest to highest.

Many sorting algorithms run as a function of n2. For example:t = 3.2 x 10-3 n2

Others run as a function of n log2 n. For example:t = 2.1 x 10-3 n log2 n

Logarithms: If 2x = n, then log2 n = x23= 8, log2 8 = 3

8

Graphing the running time for a sorting function

# people (n) t (seconds) 2500 59.261 5000 129.021 7500 202.745 10000 279.042

time

n

n

n2

n log n

Sorting a list takes more time than searching a list, but it still can be done in a reasonable amount of time.

9

Polynomial and Polylogarithmic running times

In general, tractable problems are problems that have running times that are a polynomial function of n (the size of the input) or a polylogarithmic function of n.

Examples:

t = 4n3 + 5n2 -3

t = n2(log2(n))3

t = 17 n6 - log2 n + 6n

10

Intractable Computations

A computation is intractable if the running time increases faster than a polynomial of the input size (n).

For example: Running times that increase exponentially:

t = 3n t = 2n + n3

Running times that increase like the factorial of n:

t = n! t = 4n3 + 3n!

(Recall that n! = 1x2x3x4 ...x(n-1)xn )

11

Example of an intractable problem:

Find all possible orderings (permutations) of n objects.n = 3; A, B, C (we will work this out in class)

ABC BAC CAB Total of 6 possibilities.ACB BCA CBA

In general, for n objects, their are n! possible orderings.

n time n time 5 < 1 sec 20 Can you guess? 6 3.19 sec 21 ?? 7 22.57 sec 22 ?? 8 3.05 min 9 27.87 min 10 4.64 hours

12

The Traveling Salesman problem

Suppose a salesperson wants to visit a series of n cities, beginning and ending with the home city. He wants to visit each city only once (except the home city where he begins and ends).

What order of cities will give him the shortest possible trip?

1521

4345

2716 14

23

52

15

12

13

A

B

C D

E

F3237

19

13

Solving Traveling Salesman problem

To solve the traveling salesman problem, we must compute the distance for every possible path:

ABCDEFAACBDEFAADBDEFA ...

How many possibilities are there?(n - 1)!

Therefore, this is an intractable problem.

14

Alternative solutions for intractable problems

Many intractable problems involve solving for the best solution.

We can often speed up the solution if we are willing to accept a good solution.

For example, in the traveling salesman problem:1) Find the shortest path from the current city to a city that

has not yet been visited.2) If no cities are still unvisited, go home. Otherwise,

repeat step 1.This will lead to a good solution (e.g. a reasonably short path),

but not necessarily the best solution (i.e. the shortest path).

15

Limitations of Computer Science

In practice, we cannot solve every problem with a program because...

1) The execution time of a program may be too long.

2) The problem may be non-computable.

3) We may not know how to solve the problem.E.g. Computer Vision

Artificial IntelligenceModeling complex systems such as weatheretc.

16

The Halting Problem

One problem that frequently arises when programming is when the program contains a loop that never terminates. These are called infinite loops.

It would be a nice compiler feature if the compiler could detect the presence of an infinite loop in a program and inform the programmer that one exists.

The Halting Problem: Can we write a program that will read in other programs and output a message saying whether or not the program will halt on all inputs (i.e. it does not have any infinite loops)?

17

Proof by Contradiction

• The halting problem assertion is as follows:No finite program can be written that will check other

programs and halt in a finite time giving a solution to the halting problem.

• We can prove the halting problem assertion using a technique known as proof by contradiction.

The idea is as follows:1) Assume that a program solving the halting problem exists.2) Show that this assumption leads to impossible

consequences.3) This implies that the assumption must be false and

therefore the program cannot exist.

18

Other non-computable problems

Almost every problem related to the behavior of a program is non-computable. For example:

1) Showing that 2 programs are equivalent (for each input you always get the same output)

2) Determining whether a given output will occur

3) Determining the correctness of a program

etc.

19

Limitations in integer representation

The circuitry of computers leads to limitations in what they can compute.

Integer storage limitations:

Because we use a limited number of bits to store integers, there is a maximum integer that can be represented:

16 bits: Maximum = 32,767 (if storing both positive and negative integers)

32 bits: Maximum = 2 x 109 (approximately)

Is this big enough?

20

Limits of Real number representation

The format of real numbers leads to limitations because of:

1) Limited precision of the representation.

2) Rounding error

3) Limits in the smallest number that can be represented (underflow)

4) Limits in the largest number that can be represented (overflow).

21

Other hardware limitations

1. Components can fail (disk crash, computer crash, etc).

2. Components can have design flaws so they don't work as they should (Intel pentium chip, 1994)

3. Electronic noise can change 1's to 0's and vice versa when information is sent over wires or cables (or through the air for wireless).There are error checking and error correcting techniques, but these are not 100% accurate.