Computer Science 365: Design and Analysis of Algorithms

Computer Science 365: Design and Analysis of Algorithms

Last lecture

Topics Where to learn more My favorite problem My favorite heuristic Topics we’ve neglected

Exciting trends Lies I’ve told What else to take

TexPoint fonts used in EMF.

Read the TexPoint manual before you delete this

box.: AAAAA

Where to learn more:

Major conferences: ACM STOC (Symposium on Theory of Computing) IEEE FOCS (Foundations of Computer Science) ACM/SIAM SODA (Symposium on Discrete Algorithms) ICALP (European Association for Theoretical CS) Blogs/arXiv: http://feedworld.net/toc/ Surveys: in Communications of the ACM Simon’s Foundation MPS Articles (esp. Klarreich) Quanta Magazine

Favorite Problem(s): Graph bisection: divide vertices V into two sets A and B of equal size,

cutting as few edges as possible

A

4

Sparsity of cut (A,B) !

Sparsity of G !

Favorite Problem: Sparsest Cut

5

Sparsity

A! A!A!

6

Sparsity

A

7

A

Why Sparsest Cut:

Clustering in graphs Organizing data

8

Why Sparsest Cut:

Dividing computation over processors.

9

Why Sparsest Cut:

Dividing computation over processors.

10

Why Sparsest Cut:

Generalized Divide and Conquer (3-SAT) Variables -> vertices Clauses -> cliques (edges on all pairs)

x1

x2

x3

x4

x5

x6

x7

x8

x9

x10

11

Why Sparsest Cut:

Generalized Divide and Conquer (3-SAT) Variables -> vertices Clauses -> cliques (edges on all pairs)

x1

x2

x3

x4

x5

x6

x7

x8

x9

x10

Cut vertices instead of edges

Complexity of Sparsest Cut

Exact solution NP hard. If (1+ε)-approximation, then NP is in time , α < 1 Can approximate within Really good heuristics (Chaco, Metis, Scotch) by multilevel methods.

2O(n↵)

O(

plog n)

Multilevel methods

1. Approximate problem by smaller problem. 2. Solve smaller problem (recursively). 3. Lift solution of smaller problem to original. 4. Refine lifted solution.

Multilevel methods


1. Match nodes at random, and merge

Multilevel methods



Multilevel methods



Multilevel methods


2. Solve sub-problem

Multilevel methods


3. Lift solution

Multilevel methods


3. Lift solution

Multilevel methods


4. Refine solution

Multilevel methods


4. Refine solution

Multilevel methods


4. Refine solution

Multilevel methods


4. Refine solution

Multilevel methods


4. Refine solution

Very fast.

Works well.

No analysis, yet.

Neglected topics

Neglected topic: Data structures

Fancy data structures enable fast algorithms.

Hashing Splay trees (Sleator-Tarjan) practical binary search trees van Emde Boas trees Amortized analysis, rather than worst case.

Do not bound cost of every operation Bound cost of sum of operations

Neglected topic: Dynamic Algorithms

Maintain answers on changing inputs.

Recent breakthroughs: Maintaining components in dynamic graphs. Time O(log5 n) per edge insertion and deletion. [Kapron-King-Mountjoy ‘13] Cannot beat time n1/3 for node insertions unless are faster algorithms for 3SUM [Abboud-Vassilavska ‘14]

Neglected topic: Geometric Algorithms

Convex Hulls Voronoi Diagrams and Delaunay Triangulations Meshing Visibility Point Location Geometric Data Structures

Neglected topic: Primality Testing

Miller ‘76: in polynomial time, if Extended Riemann Hypothesis true

Rabin ‘80: In randomized polynomial time,

detect composite with probability ½. Adelman-Huang ’92: Expected polynomial time,

when stops zero probability of error. Agarwal-Kayal-Saxena ’04: Polynomial time,

by derandomization.

Hard Problems? Factoring Integers

For b-bit integers, best-known complexity is

If NP-hard, would have NP = co-NP.

(assuming conjectures in number theory)

Pollard, Arjen and Hendrik Lenstra,

Manasse, Odlyzko, Adelman, Pomerance

2O(b1/3 log

2/3 b)

Hard Problems? Graph Isomorphism.

Given two labeled graphs, can vertex sets be relabeled so are same?


Given two labeled graphs, can vertex sets be relabeled so are same? a

b

c d

e f g

h i

j

a e

b g

d

j

h

f

i

c


Complexity [Babai-Luks ’83] Polynomial time if constant degree Are counter-examples to most naïve algorithms

Given two labeled graphs, can vertex sets be relabeled so are same?

nO(pn)

Quantum Computing

Quantum computers have different operations

Factoring in polynomial time (Shor ’94)

and breaking Diffie-Helman (‘76) key exchange

Can they solve NP hard problems?

Graph isomorphism?

Numerical Algorithms

Solving systems of linear equations, quickly.

Doing Gaussian elimination correctly.

Graph algorithms and data structures.

Gaussian Elimination with Partial Pivoting!

pivot largest in column to top

Fast heuristic for maintaining precision. Tries to keep entries small.

Laplacian Linear Equations/ Electrical Flows

Solve for current when fix voltages

1V

0V



Can solve in 9me

O(m log

c m)

(S-Teng ’04)



Can solve in 9me

O(m log

c m)

(S-Teng ’04)

O(mp

logm)

(Cohen-Kyng-Pachocki-Peng-Rao ‘14)

Maximum flow problem, electrical approach

1.  Try the electrical flow.

s t

[Christiano-Kelner-Madry-S-Teng ‘11]

0V

2.18 V

0.82 0.82 0.55

1.09

1.09

1.09

1.09

0.27 0.27


1.  Try the electrical flow. 2.  Increase resistance when too much flow

s t


0V

2.18 V

0.82 0.82 0.55

1.09

1.09

1.09

1.09

0.27 0.27


1.  Try the electrical flow. 2.  Increase resistance when too much flow

s t


0V

2.10 V

0.90 0.90 0.60

1.05

1.05

1.05

1.05

0.30 0.30

Maximum Flow in Undirected Graphs

Approximately optimal solution:

(Mądry ’13)

Exact solution:

O(m10/7log

c n)

O(m2plogn) (Sherman ‘13,

Kelner-‐Lee-‐Orecchia-‐Sidford ‘13)

Lies

Very few algorithms are both elegant and useful. Most algorithms papers are merely evidence for the existence of useful algorithms.

Lies

Polynomial-time = efficient

Big-O notation.

Worst-case analysis.

Smoothed Analysis

Want theorems that explain why algorithms

and heuris9cs work in prac9ce

May be contrived examples on which they fail.

take too long

return wrong answer

Worst-‐case analysis is not sa9sfactory.

Attempted fix: Average-case analysis

Measure expected performance on random inputs.

random graphs

random point sets

random signals

random matrices

random preferences

Random is not typical

Assume randomness/noise in low-‐order bits

randomly perturb problem

Problem comes through noisy channel

Smoothed Analysis

measurement error

random sampling

arbitrary circumstances

(managers)


“Gaussian elimination with partial pivoting is utterly stable in practice. In fifty years of computing, no matrix problems that excite an explosive instability are know to have arisen under natural circumstances … Matrices with large growth factors are vanishingly rare in applications.” Nick Trefethen

Complexity of algorithm : inputs -‐> 9me

Hybrid of worst and average case

Worst case:

Ave case:

Smoothed:

Gaussian perturbation of std dev

Interpolates between worst and average case

Considers neighborhood of every input

If low, all high complexity is unstable

Smoothed Complexity

Complexity Landscape

run

time

input space

worst case

average

case

Smoothed Complexity Landscape

run

time

input space

smoothed complexity


“Gaussian elimination with partial pivoting is utterly stable in practice. In fifty years of computing, no matrix problems that excite an explosive instability are know to have arisen under natural circumstances … Matrices with large growth factors are vanishingly rare in applications.” Nick Trefethen

[Sankar-S-Teng]

Theorem:

Simplex Method for Linear Programming!

max s.t.!

Worst-Case: exponential Average-Case: polynomial

Widely used in practice

Smoothed Analysis of Simplex Method!

max s.t. !

Theorem: For all A, b, c, simplex method takes expected time polynomial in

max s.t. !

G is Gaussian

Related Courses

Optimization AMTH/OPRS 235 AMTH 437 ENAS 530

Linear programming.

Convex programming.

Newton’s method.

Maximize something subject to constraints.

Machine Learning STAT 365 CPSC 463

Boosting

Support Vector Machines

Genetic Algorithms

Neural Nets

Belief Propagation

Numerical Methods: CPSC 440 ENAS 440/441

Numerical problems that arise in sciences.

Solving linear equations.

Computing eigenvectors and eigenvalues.

Solving differential equations

Interpolation.

Integration.

More Algorithms

CPSC 465: Advanced Algorithms

CPSC 469: Randomized Algorithms

Hashing

Random Walks

Generating Random Objects

Streaming Algorithms

CPSC 468: Complexity Theory

P = NP?

NP = co-NP? short proofs of unsatisfiability?

Poly Time = Poly Space?

Interactive proofs.

Probabilistically checkable proofs.

Hardness of approximation.

Pseudo-random generators, and

Derandomizaton.

Interactive proofs:

Colors exist:

give colorblind two balls of different colors

shown one, we can always tell which it was

Interactive proofs:

Colors exist:

give colorblind two balls of different colors

shown one, we can always tell which it was

Graphs non-isomorphic:

pick one at random,

draw it at random,

can I tell you which it was?

Graph non-isomorphism.

pick one


pick one


scramble it


scramble it


scramble it


scramble it


randomly located vertices

If were same, no one can tell which it was

Probabilistically Checkable Proofs

A proof you can check by examining a few

bits chosen carefully at random.

For every “yes” instance of every

problem in NP, there is a PCP.

Hardness of Approximation

Max 3-SAT: Satisfy as many clauses as possible.

Hard to do better than 7/8+ε

Set-Cover: Hard to do better than c log n

Max IS: Hard to do better than n1-ε

VC: Hard to do better than 1.36 (and maybe 2-ε)

CPSC 467: Cryptography

RSA

One-way functions

Zero-knowledge proofs

Digital signatures

Playing poker over the phone

Good luck

Documents

Computer Science 365: Design and Analysis of Algorithms