CS221Week 13 - Monday

Last time

What did we talk about last time? B-trees Hamiltonian tour Traveling Salesman Problem

Questions?

Project 4

Assignment 6

TSP

Traveling Salesman Problem:Hamiltonian Cycle Meets Shortest Path

Given a graph, find the shortest tour that visits every node and comes back to the starting point

Like a lazy traveling salesman who wants to visit all possible clients and then return to his home

What’s the Shortest Tour? Find the shortest TSP tour:

Starts anywhere Visits all nodes Has the shortest length of such a tour

E

A

DB

FC

56

44

35

27

3

3

How Can We Always Find the Shortest Tour? Strategies:

Always add the nearest neighbor Randomized approaches Try every possible combination

Why are we only listing strategies?

Best Solution to TSP

So far we haven’t given a foolproof solution to TSP

Let’s look at two possibilities: Greedy Solution: Pick the closest

neighbor Brute Force: Try all possibilities

Greedy Doesn’t Work

We are tempted to always take the closest neighbor, but there are pitfalls Greedy Optimal

Brute Force is Brutal

In a completely connected graph, we can try any sequence of nodes

If there are n nodes, there are (n – 1)! tours

For 30 cities, 29! = 8841761993739701954543616000000

If we can check 1,000,000,000 tours in one second, it will only take about 20,000 times the age of the universe to check them all

We will (eventually) get the best answer!

What’s the Best Time for TSP? No one knows how to find the best

solution to TSP in an efficient amount of time

For a general graph, no good approximation even exists

For a graph with the triangle inequality, there is an approximation that yields a tour no more than 3/2 the optimal

Some variations on the problem are easier than others

Hard Problems

Is TSP the hardest problem there is? Are there other problems equally

hard? How do we compare the difficulty of

one problem to another?

NP-Completeness

TSP is one of many problems which are called NP-complete

All of these problems share the characteristic that the only way we know to find best solution uses brute force

All NP-complete problems are reducible to all other NP-complete problems

An efficient solution to one would guarantee an efficient solution to all

NP-complete Problems on Graphs Traveling Salesman Problem Hamiltonian Cycle (and Path) Let’s see just a couple more…

Graph Coloring

Find the smallest number of colors for coloring nodes such that no two adjacent nodes have the same color

E

A

DB

FC

E

A

DB

FC

Graph Coloring

What about this graph?

Graph Coloring

Like TSP and Hamiltonian Cycle, large graphs get really hard to find the smallest coloring for

It might not be obvious, but graph coloring has practical applications

Perhaps the most common example is register allocation inside of a compiler

Maximum Clique

Find the largest complete subgraph in a given graph

This graph has a clique of size 3

E

A

DB

FC

Maximum Clique

What about this graph? This graph has a clique of size 4 As before, a large graph can be very

difficult

E

A

DB

FC

Knapsack problem Not all NP-complete problems are graph

problems The knapsack problem is the following:

Imagine that you are Indiana Jones You are the first to open the tomb of some long-lost

pharaoh You have a knapsack that can hold m pounds of loot,

but there's way more than that in the tomb Because you're Indiana Jones, you can instantly tell

how much everything weighs and how valuable it is You want to find the most valuable loot that weighs

less than or equal to m pounds

Subset sum

This one is a little bit mathematical Say you have a set of numbers Somebody gives you a number k

Is there any subset of the numbers in your set that add up to exactly k?

Example: Set: { 3, 9, 15, 22, 35, 52, 78, 141} Is there a subset that adds up to exactly

100? What about 101?

Fabulous Cash Prizes

These NP-complete problems are very hard

Many of them are really useful Especially if you are a lazy traveling

salesman Clay Mathematics Institute has

offered a $1,000,000 prize You can do one of two things to

collect it: Find an efficient solution to any of the

problems Prove that one cannot have an efficient

solution

A Few Finer Points…

Turing machine A Turing machine is a mathematical model

for computation It consists of a head, an infinitely long tape,

a set of possible states, and an alphabet of characters that can be written on the tape

A list of rules saying what it should write and should it move left or right given the current symbol and state

1 0 1 1 1 1 0 0 0 0

A

Turing machine example

3 state, 2 symbol “busy beaver” Turing machine:

Starting state A

Tape Symbo

l

State A State B State C

Write Move Next Write Move Next Write Move Next

0 1 R B 0 R C 1 L C

1 1 R HALT 1 R B 1 L A

Church-Turing thesis

If an algorithm exists, a Turing machine can perform that algorithm

In essence, a Turing machine is the most powerful model we have of computation

Power, in this sense, means the ability to compute some function, not the speed associated with its computation

NP

NP is actually a class of problem Problems in NP can be solved in

polynomial time on a non-deterministic computer

A deterministic computer is the kind you know: First it has to consider possibility A, then,

it can consider possibility B

Deterministic vs. non-deterministic

A non-deterministic computer (which, as far as we know, only exists in our imagination) can consider both possibility A and possibility B at the same timeDeterministi

c

A B C D E F

A

B

C

D

E

F

Non-Deterministi

c

P P is the class of decision problems that can

be solved in polynomial time by a deterministic computer

Lots of great problems are in P: Is this list sorted? Is this number prime? Is the largest number in this B-tree equal to 38?

Many problems are unknown: Does this number have exactly two factors? Is this graph equivalent to this other graph?

NP-complete

Everything in P is also in NP Some problems are the “hardest”

problems in NP This means that any problem in NP

can be converted into one of these problem in polynomial time

These problems make up the class NP-complete

Decisions, decisions

Notice that P, NP, and NP-complete are all decision problems

So, the TSP we stated is not technically NP-complete

The NP-complete version is: Is there a tour of length less than or

equal to 24 in this graph? The optimization versions of NP-

complete problems are called NP-hard

Complexity hierarchy EXPSPACE

Exponential space EXP

Exponential time on a deterministic Turing machine

PSPACE Polynomial space

NP Polynomial time on a non-

deterministic Turing machine P

Polynomial time on a deterministic Turing machine

L Logarithmic space on a

deterministic Turing machine

EXPSPACE

P

L

RE Co-RE

But wait it gets worse!

There are also problems that cannot be solved by any computer (or algorithm) in a finite amount of time

Problems in R can be solved by an algorithm in a finite amount of time Problems in RE can reach a "yes" answer in a finite amount of time, but no

algorithm is guaranteed to reach completion if the answer is "no" Problems in Co-RE can reach a "no" answer in a finite amount of time, but

no algorithm is guaranteed to reach completion if the answer is "yes"

R

EXPSPACE

R is the set of recursive decision problems

RE is the set of recursively enumerable decision problems

Co-RE is the complement of set RE

Halting problem

Given a Turing machine and input x, does it reach the halt state?

It turns out that this problem is undecidable

That means that there is no algorithm that can be to determine if any Turing machine will go into an infinite loop

Consequently, there is no algorithm that can take any program and check to see if it goes into an infinite loop

Post correspondence problem Given two finite lists of words A and B, can you pick k words

(repetitions allowed) from A and k words (repetitions allowed) from B so that the words from A concatenated are exactly the same string as the words from B concatenated

Example:

Solution: aa + bb + bb = aabbbb = a + a + bbbb

The Post correspondence problem (PCP) is undecidable (there is no algorithm that can solve all instances of it)

A Baa bbbbaba abb abba

Upcoming

Next time…

Elementary sorts Merge sort Quicksort Priority queues Heapsort

Reminders

Work on Project 4Work on Assignment 6 Read sections 2.1 through 2.4