COS 126: General Computer Science • http://www.Princeton.EDU/~cos126

Lecture 21: Intractability

Princeton CS Building West Wall, Circa 2001

2

Overview

What is an algorithm? Turing machine.

What problems can be solved on a computer? Computability.

What ALGORITHMS will be useful in practice? Analysis of algorithms.

Which PROBLEMS can be solved in practice? Intractability.

3

Which ALGORITHMS are useful in practice?

A broad and robust definition: (Cobham 1960s, Jack Edmonds, 1962)

! Efficient = polynomial time for ALL inputs.Ex: mergesort requires at most N log2N steps.Ex: insertion sort requires at most N2 / 2 steps.

! Inefficient = "exponential time" for SOME inputs.Ex: brute force TSP takes N! > 2N steps.

In practice, huge gulf between polynomial and exponential algorithms.

Properties of Algorithms

4

Exponential Growth

Exponential growth dwarfs technological change.

! Suppose each electron in the universe had power of today'ssupercomputers . . .

! And each works for the life of the universe in an effort to solveTSP problem via brute force N! algorithm.

! Will not help solve 1,000 city TSP problem.

1000! >> 101000 >> 1079 ! 1017 ! 1012

Quantity

Supercomputer instructions per second

Age of universe in seconds †

Number

1012

1017

Electrons in universe † 1079 † Estimated

5

P

Which PROBLEMS will we be able to solve in practice?

! Those with efficient (polynomial-time) algorithms.

Definition of P: Set of all decision problems solvable in polynomial timeon a deterministic Turing machine.

34, 5134, 39Euclid (300 BCE)Are the integers x and yrelatively prime?

RELPRIME

5351Agarwal-Kayal-Saxena (2002)

Does the integer x have afactor other than 1 and x?

COMPOSITE

niether

another

neither

niether

neither

another

DynamicProgramming

Is x closer to y or z interms of edit distance?

EDIT-DISTANCE

Is the Kevin Baconnumber of x less than 6?

Description

Breadth FirstSearch

Algorithm

BACON

Problem

Akbar

Abdi

Julia

Roberts

NoYes

Some problems in P

6

Extended Church-Turing Thesis

Extended Church-Turing thesis:

! P = decision problems solvable in polynomial time on real computers.

! If function is computable by a piece of hardware in time T(n) forinput of size n, then computable by TM in time (T(n)) k for some k.

Evidence supporting thesis:

! True for all physical computers.

! k = 2 for random access machines.

Implication: to make future computers more efficient, only need tofocus on improving implementation of existing designs.

Possible exception: quantum computers.

! Schor's factoring algorithm runs in poly-time on quantum computer.

! No algorithm currently known for classical computers.

7

Properties of Problems

Which PROBLEMS won't we be able to solve in practice?

! No easy answers, but theory helps.

Two hard problems.

! Factorization: Given an integer, find its prime factorization.

! CIRCUIT-SAT: Is there a way to assign inputs to a givencombinational circuit that makes its output true?

YES NO

4901 = 132 ! 29

8

More Hard Problems

More hard computational problems.! Biology: protein folding.

! Chemical engineering: heat exchanger network synthesis.

! Civil engineering: equilibrium of urban traffic flow.

! Economics: computation of arbitrage in financial markets with friction.

! Financial engineering: find minimum risk portfolio of given return.

! Genomics: phylogeny reconstruction, sequence assembly.

! Electrical engineering: VLSI layout.

! Mechanical engineering: structure of turbulence in sheared flows.

! Medicine: reconstructing 3-D shape from biplane angiocardiogram.

! Operations research: optimal resource allocation.

! Physics: anti-ferromagnetic Potts model.

! Politics: Shapley-Shubik voting power.

! Pop culture: Minesweeper consistency, playing optimal Tetris.

! Statistics: optimal experimental design.

How can we justify our belief that these problems are difficult?

9

Reduction

Problem X reduces to problem Y if given an efficient subroutine for Y, youcan devise an efficient algorithm for X.

! Cost of solving X " cost of solving Y + cost of reduction.

! May call subroutine for Y more than once.

Consequences:

! Classify problems: establish relative difficulty between two problems.

! Design algorithms: given algorithm for Y, can also solve X.

! Establish intractability: if X is hard, then so is Y.

Example.

! X = COMPOSITE.

! Y = Factorization.

static boolean isComposite (int x) {

int[] factors = factorize(x);

return (factors.length > 1);

}

10

More Reductions

CIRCUIT-SAT

3SAT

3DM VERTEXCOVER

HAMILTONIANCIRCUIT

CLIQUE

INDEPENDENTSET

GRAPH3-COLOR

PLANAR3-COLOR

EXACTCOVER

TSP

SUBSET-SUM

PARTITION INTEGERPROGRAMMING

KNAPSACK

Dick Karp (1972)

CIRCUIT-SAT reduces to 3SAT

11

Some Hard Problems

TSP: What is the shortest tour that visits all N cities?

12

Some Hard Problems

PLANAR-3-COLOR: Given a planar map, can it be colored using 3 colors

so that no adjacent regions have the same color?

YES instance.

13

Some Hard Problems

PLANAR-3-COLOR: Given a planar map, can it be colored using 3 colors

so that no adjacent regions have the same color?

NO instance.

14

Minesweeper Consistency Problem

Minesweeper.

! Start: Blank grid of squares, some of which conceal mines.

! Goal: Find location of all mines without detonating any.

! Choose a square.

– if mine underneath, it detonates and you lose

– otherwise, computer tells you # mines in neighboring squares

! Repeat.

15

Minesweeper Consistency Problem

MINESWEEPER: Given a state of what purports to be a N-by-NMinesweeper game, is it logically consistent?

YES

NO

16

Reduction

Problem X reduces to problem Y if given an efficient subroutine for Y, youcan devise an efficient algorithm for X.

! Cost of solving X " cost of solving Y + cost of reduction.

! May call subroutine for Y more than once.

Consequences:

! Classify problems: establish relative difficulty between two problems.

! Design algorithms: given algorithm for Y, can also solve X.

! Establish intractability: if X is hard, then so is Y.

Example.

! X = CIRCUIT-SAT.

! Y = MINESWEEPER.

17

Minesweeper Consistency Problem

Claim. SAT reduces to MINESWEEPER.

! Build circuit by laying out appropriate minesweeper configurations.

! Minesweeper game is consistent if and only if circuit is satisfiable.

A Minesweeper Wire

A Minesweeper NOT Gate

Exactly one of xand x' is a bomb.

18

Minesweeper Consistency Problem

Claim. SAT reduces to MINESWEEPER.

! Build circuit by laying out appropriate minesweeper configurations.

! Minesweeper game is consistent if and only if circuit is satisfiable.

A Minesweeper AND Gate

19

One More Hard Problem

CIRCUIT-SAT

3SAT

3DM VERTEXCOVER

HAMILTONIANCIRCUIT

CLIQUE

INDEPENDENTSET

GRAPH3-COLOR

PLANAR3-COLOR

EXACTCOVER

TSP

SUBSET-SUM

PARTITION INTEGERPROGRAMMING

KNAPSACK

MINE-SWEEPER

CIRCUIT-SAT reduces to 3SAT

20

Nondeterminism

Nondeterministic machine. One that can GUESS the right answer andCHECK that the guessed answer is correct.

Ex 1: COMPOSITE.

! Guess a factor p of x.

! Computer must check that p divides x.

Ex 2: MINESWEEPER.

! Guess an assignment of mines to squares.

! Computer must check that each squareadjacent to the required number of mines.

Checking seems easy compared to solving from scratch. Is it really?

running time = # steps to check

Input: x = 437,669Guess: p = 809

Input Guess

21

P and NP

P: Set of decision problems solvable in polynomial time on adeterministic Turing machine.

NP: Set of decision problems solvable in polynomial time on a non-deterministic Turing machine.

Cook-Levin Theorem (1960s). ALL problems in NP reduce to SAT.

! If efficient algorithm for SAT, then P = NP.

! Intellectual milestone in CS.

Proof idea:

! Given problem X in NP, consider nondeterministic TMthat solves X in nondeterministic polynomial time.

! For each time step t, use Boolean variables to model whichsymbol occupies cell i, location of read head, state of DFA, etc.

! Use logic gates to ensure TM makes legal moves, etc.

! SAT instance is satisfiable if and only if TM outputs YES.

Stephen Cook

if we can solve SAT, we can solve any of them

22

Cook's Theorem: Implications

CIRCUIT-SAT

3SAT

3DM VERTEXCOVER

HAMILTONIANCIRCUIT

CLIQUE

INDEPENDENTSET

GRAPH3-COLOR

PLANAR3-COLOR

EXACTCOVER

TSP

SUBSET-SUM

PARTITION INTEGERPROGRAMMING

KNAPSACK

All of these problems reduceto each other.

CIRCUIT-SAT reduces to 3SAT

23

The Main Question

Does P = NP?

! Can problems like SAT be solved in a polynomial number of steps?

! Is the decision problem as easy as checking a proposed solution?

! Does nondeterminism help you solve problems faster?

! Most important open problem in computer science.

! Clay Foundation $1 million prize.

EXP NP

P

If P # NP If P = NP

EXP

P = NP

Jack Edmonds, 1962

24

Does P = NP?

Does P = NP?

! If yes, then efficient algorithms for 3-COLOR, TSP, FACTOR.

! If no, then can't hope to write efficient algorithm for TSP.

What is the consensus opinion on P = NP?

! Probably no, since thousands of researchers have spent fourdecades in search of polynomial algorithms without success.

! But yes is possible since no success in proving P # NP either.

would break modern cryptography andcollapse ecommerce and banking system

27

NP-Complete

NP-complete: A problem in NP is NP-complete if SAT reduces to it.

! Ex: SAT, MINESWEEPER, all Karp problems, thousands more.

! Note: FACTOR is in NP, but not known to be NP-complete.

Links together a huge and diverse number of fundamental problems:

! Efficient algorithms for all Karp problems or none of them.

! If efficient algorithm for 3-COLOR, then efficient algorithm for:

– any other NP-complete problem, including all Karp problems

– any other problem in NP, including FACTOR

Notorious complexity class.

! All known algorithms for these problems are exponential.

! Unlikely that they can be solved given limited computing resources.

! Called intractable.

no one has found a way to reduce SAT to FACTOR

28

NP-Complete

Definition of NP-complete:

! A problem in NP with the property that if it can be solvedefficiently, then it can be used as a subroutine to solve any otherproblem in NP efficiently.

! "Hardest computational problems" in NP.

EXP NP

P

If P # NP If P = NP

NP-complete

EXP

P = NP

29

Coping With Intractability

Relax one of desired features.

! Solve the problem in polynomial time.

! Solve the problem to optimality.

! Solve arbitrary instances of the problem.

Complexity theory deals with worst case behavior.

! Instance(s) you want to solve may be "easy."

! Concorde algorithm solved 13,509 US city TSP problem.

(Cook et. al., 1998)

30

Coping With Intractability

Relax one of desired features.

! Solve the problem in polynomial time.

! Solve the problem to optimality.

! Solve arbitrary instances of the problem.

Develop a heuristic, and hope it produces a good solution.

! TSP assignment.

! Metropolis algorithm, simulating annealing, genetic algorithms.

Design an approximation algorithm: algorithm that guarantees to find ahigh-quality solution in polynomial time.

! Ex: Euclidean TSP tour within 1% of optimal.

! Active area of research, but not always possible!

Sanjeev Arora (1997)

31

Coping With Intractability

Relax one of desired features.

! Solve the problem in polynomial time.

! Solve the problem to optimality.

! Solve arbitrary instances of the problem.

Exploit intractability.

! Cryptography.

Keep trying to prove P = NP.

see next lecture

can do any 2 of 3

32

Summary

Many fundamental problems are NP-complete.

! TSP, CIRCUIT-SAT, 3-COLOR.

Theory says we probably won't be able to design efficient algorithmsfor NP-complete problems.

! You will surely run into these problems in your scientific life.

! If you know about NP-completeness, you can identify them and avoidwasting time and energy.