Computer Science 5 Studies the powers of machines. Fundamental
Question CS asks: What is computable by machines? Turing, along
with Church and Godel, was the person who made computation
something we can mathematically study.
Slide 6
Turings Answer To What Computation Is (1936) 6 On Computable
Numbers, with an Application to the Entscheidungsproblem: We may
compare a man in the process of computing a real number to a
machine which is only capable of a finite number of conditions
Slide 7
Turings Answer To What Computation Is 7 Computing is normally
done by writing certain symbols on paper. We may suppose this paper
is divided into squares like a child's arithmetic book. The
behavior of the computer at any moment is determined by the symbols
which he is observing, and his " state of mind " at that moment. We
may suppose that there is a bound B to the number of symbols or
squares which the computer can observe at one moment. If he wishes
to observe more, he must use successive observations. We will also
suppose that the number of states of mind which need be taken into
account is finite.
Slide 8
Turings Answer To What Computation Is 8 Let us imagine the
operations performed by the computer to be split up into "simple
operations" which are so elementary that it is not easy to imagine
them further divided. Every such operation consists of some change
of the physical system consisting of the computer and his tape. We
may suppose that in a simple operation not more than one symbol is
altered.
Slide 9
Turings Answer To What Computation Is 9 Besides these changes
of symbols, the simple operations must include changes to the
observed squares. I think it is reasonable to suppose that they can
only be squares whose distance from the closest of the immediately
previously observed squares does not exceed a certain fixed
amount.
Slide 10
The Turing Machine 10 Its paradoxical that as humans in our
quest to understand what machines can do, we have been studying an
abstract machine that in essence imitates a human being.
Slide 11
Church-Turing Thesis 11 Central Dogma of Computer Science:
**Whatever is computable is computable by the Turing machine.**
There is no proof of this claim.
Slide 12
Turing In Defense Of His Claim: 12 All arguments which can be
given are bound to be, fundamentally, appeals to intuition, and for
this reason rather unsatisfactory mathematically. The arguments
which I shall use are of three kinds. (a) A direct appeal to
intuition. (b) A proof of the equivalence of two definitions (in
case the new definition has a greater intuitive appeal). (c) Giving
examples of large classes of numbers which are computable.
Slide 13
Algorithm = Turing Machine 13 When we say there is an algorithm
computing shortest paths of a graph in O(mlog(n)) times we really
mean: There is a Turing Machine that computes the shortest paths of
a graph in O(mlog(n)) operations.
Slide 14
Turing Machine Is A Very Powerful Machine 14 CS tries to
understand the limits of TM. We limit/extend TM and try to
understand what can be computed by it. Limit the # times its head
is allowed to move left/right and it changes states to poly-time.
=> poly-time algs What if the machine had access to a random
source => randomized algorithms. What if there were multiple
heads on the tape => parallel algorithms Limit the length of its
tape. => space-efficient algs What if the head was only allowed
to move right => streaming algorithms
Online Algorithms 16 Takes as input a possibly infinite stream.
At each point in time t make a decision based on what has been seen
so far but without knowing the rest of the input Type of Optimality
Analysis: Competitive Ratio Worst (Cost of online algorithm)/(Cost
of OPT) ratios against any input stream Where OPT is the best
solution possible if we knew the entire input in advance
Slide 17
Caching 17 Slow Disk O.w (miss), send request to disk, put the
page into cache. Q: Which page to evict? If page is in cache (hit)
reply directly from cache
Slide 18
Caching 18 Input: N pages in disk, and stream of infinite page
requests. Online Algorithm: Decide which page to evict from cache
when its full and theres a miss. Goal: minimize the number of
misses. Idea: LRU: Remove the Least Recently Used page
Slide 19
LRU with k = 3 19 412153441132451 miss
Slide 20
LRU with k = 3 20 412153441132451
Slide 21
LRU with k = 3 21 412153441132451 miss
Slide 22
LRU with k = 3 22 412153441132451
Slide 23
LRU with k = 3 23 412153441132451 miss
Slide 24
LRU with k = 3 24 412153441132451
Slide 25
LRU with k = 3 25 412153441132451 hit
Slide 26
LRU with k = 3 26 412153441132451 miss
Slide 27
LRU with k = 3 27 412153441132451
Slide 28
LRU with k = 3 28 412153441132451 miss
Slide 29
LRU with k = 3 29 412153441132451
Slide 30
LRU with k = 3 30 412153441132451 miss
Slide 31
LRU with k = 3 31 412153441132451
Slide 32
LRU with k = 3 32 412153441132451 hit
Slide 33
LRU with k = 3 33 412153441132451 miss
Slide 34
LRU with k = 3 34 412153441132451 so and so forth
Slide 35
Competitive Ratio Claim 35 Claim: If the optimal sequence of
choices for a size-h cache causes m misses. Then, for the same
sequence of requests, LRU for a size-k cache causes misses
Interpretation: If LRU had twice as much cache size as an algorithm
OPT that knew the future, it would have at most twice the misses of
OPT. Note will prove the claim for
Slide 36
Proof of Competitive Ratio 36 Recursively break the sequence of
inputs into phases. Let t be the time when we see the (k+1)st
different request. Phase 1: a 1 a t-1 Let t` be the time we see the
(k+1)st different element starting from a t Phase 2: a t a t-1
412153441132451
Slide 37
Proof of Competitive Ratio 37 412153441132451 k=3 Phase 1 Phase
2 Phase 3 Phase 4 By construction, each phase has k distinct
requests. Q: At most how many misses does LRU have in each phase?
A: k b/c even if it evicted everything in the k+1 st item, it would
have at most k misses.
Slide 38
Proof of Competitive Ratio 38 412153441132451 Phase 1 Phase 2
Phase 3 Phase 4 Q: Whats the minimum misses that any size-h cache
must have in any phase? A: k-h b/c k distinct items will be in the
cache at different points during the phase, so at least k-h of them
must trigger misses. Therefore the CR: k/k-h Q.E.D.
40 Parallel Algorithms Question: Which problems are
parallelizable, which are inherently sequential? Parallelizable:
Connected components, sorting, selection, many computational
geometry problems all have parallel algorithms (Believed To Be)
Inherently Sequential (P-complete): DFS Horn-satisfiability Conways
Game of Life, and others.
Slide 41
41 2 Common Computational Models Model 1: Shared Memory (PRAM):
Single Machine Memory CPU 1 CPU 2 CPU k Each Time Step, each
processor: Can read a location of memory Can write to a location of
memory Q: How much time does it take to solve a computational
problem with polynomial # processors?
Slide 42
42 Example 1: Finding the min in an array Memory CPU (1,2) CPU
(1,3) CPU (4, 5) There are n(n-1) processors, one for each pair (i,
j) Initially allocate an array of size n all 0 Step 1: Each cpu (i,
j) compares i and j If i < j, then write 1 to j o.w. write 1 to
location i Step 2: return the item whose output memory location is
0
Slide 43
43 Example 1: Finding the min in an array Memory CPU (1,2) CPU
(1,3) CPU (4, 5)
Slide 44
44 Model 2: Distributed Memory Machine 1 Machine 2 Machine k
Input Partition 1 Input Partition 2 Input Partition k Each machine
performs local computation send/receives messages to/from other
machines can be synchronously or asynchronously Q: How much
communication is necessary? Q: How many synchronizations is
necessary?
Slide 45
Recap: Bellman-Ford 45 v, and for i={1, , n} P (v, i) :
shortest s v path with i edges (or null) L (v, i) : w(P (v, i) )
(and + for null paths) L (v, i) = min L (v, i-1) min u: (u,v) E : L
(u, i-1) + c (u,v)
Slide 46
46 Example: Distributed Bellman-Ford A C D F B E 5 3 -6 2
2
Slide 47
47 Example: Distributed Bellman-Ford 0 0 A C D F B E 5 3 -6 2
2
Slide 48
48 Example: Distributed Bellman-Ford 0 0 5 5 A C D F B E 5 3 -6
2 2
Slide 49
49 Example: Distributed Bellman-Ford 0 0 5 5 8 8 7 7 A C D F B
E 5 3 -6 2 2
Slide 50
50 Example: Distributed Bellman-Ford 1 1 0 0 5 5 8 8 7 7 A C D
F B E 5 3 -6 2 2
Slide 51
51 Example: Distributed Bellman-Ford 1 1 0 0 5 5 0 0 7 7 A C D
F B E 5 3 -6 2 2
Slide 52
52 Example: Distributed Bellman-Ford 1 1 0 0 5 5 0 0 7 7 A C D
F B E 5 3 -6 2 2 Termination When No Vertex Value Changes
Slide 53
53 Parallel Algorithms Parallel Computing: CS 149 A systems
course. Teaches parallel technologies but also algorithms.
55 Linear Programming (CS 261/361) Optimization Problem of
following structure: maximize x 1 + x 2 subject to x 1 + 2x 2 1 2x
1 + x 2 1 x 1 0 x 2 0
Slide 56
56 Geometric Interpretation Constraint 2= 2x 1 + x 2 1 x1x1
x2x2 Constraint 1= x 1 + 2x 2 1 Feasible Solution Set **Opt
Solution**
Slide 57
57 Linear Programming Applications Tons! Lots and lots of
problems can be solved or approximated with LP! Vertex Cover Set
Cover Load Balancing Lots of problems in manufacturing/operations
research/finance, etc.. Covered in CS 261/361 **Invented By George
Dantzig.**
59 Approximation Techniques (including LP, SP: Semidefinite
Programming) Many more Approximation Algorithms Approximation
Algorithms (CS 261/CS 361)
Slide 60
60 Commonly appearing general randomization techniques.
Probabilistic Method Markov Chains **Chernoff Bounds** Very very
cool/elegant algorithms! Randomized Algorithms (CS 365)
Slide 61
61 Competitive Ratio Average-Case Analysis Instance Optimality
Smoothed Analysis, and others Beyond Worst-Case (CS 369)
Slide 62
62 Resource Lower Bounds on Computational Problems Algorithms
study what can be computed and with how much resource? Complexity
studies what cannot be computed with how much resource? Turing
Machines/P-NP/Circuit- Complexity/Randomized-Complexity/Polynomial
Hierarchy/Space Complexity/PCP/One-way Functions Complexity Theory
(CS 254)
Slide 63
63 Power of Quantum Computers over Classic Computers Quantum
Teleportation Quantum Circuits Quantum Algorithms: Fourier
Transform/Factoring/Search, etc Quantum Error Correction Quantum
Computing (CS 259Q)
65 Geometric modeling of physical objects Search in
high-dimensional spaces Triangulation of a set of points Image
Processing/Graphics/Vision Computational Geometry (CS 268)
Slide 66
66 Design and Analysis of Computational Problems In Strategic
Environments Key Analysis Concept: Price of Anarchy Also Studies
Complexities of Finding Equilibria Auctions Traffic Routing
Multi-agent Systems, and others Algorithmic Game Theory (CS
364)
Slide 67
67 Randomized Algorithms (CS 365) (Chernoff Bounds) Linear
Programming (CS 261/361) Complexity Theory (CS 254) My
Recommendations For Future Classes
Slide 68
68 Tim Roughgarden Keith Schwarz Billy, Mike & Yiming
Acknowledgements
