of 69 /69
Caching Parallel Computational Models Other Topics in Algorithms Wednesday, August 13 th 1

# Caching Parallel Computational Models Other Topics in Algorithms Wednesday, August 13 th 1

Tags:

Embed Size (px)

### Text of Caching Parallel Computational Models Other Topics in Algorithms Wednesday, August 13 th 1

• Slide 1
• Caching Parallel Computational Models Other Topics in Algorithms Wednesday, August 13 th 1
• Slide 2
• Announcements 2 1.PS#6 due tonight at midnight 2.Winners of the Competition: 1.Shir Aharon 2.Rasoul Kabirzadeh 3.Alice Yeh & Marie Feng 3.Extra Office Hours on Thursday & Friday Semih: Th 10am-12pm (Gates 424) Billy: Friday 1pm-5pm (Gates B24) Mike: Th 3pm-5pm (Gates B24) Yiming: Fr: 5pm-7pm (Gates B24)
• Slide 3
• Outline For Today 1.Caching 2.Other Algorithms & Algorithmic Techniques Beyond CS 161 1.Parallel Algorithms (Finding Min, Bellman-Ford) 2.Linear Programming 3.Other Topics & Classes 3
• Slide 4
• Alan Turing: Father of Computer Science 4
• Slide 5
• Computer Science 5 Studies the powers of machines. Fundamental Question CS asks: What is computable by machines? Turing, along with Church and Godel, was the person who made computation something we can mathematically study.
• Slide 6
• Turings Answer To What Computation Is (1936) 6 On Computable Numbers, with an Application to the Entscheidungsproblem: We may compare a man in the process of computing a real number to a machine which is only capable of a finite number of conditions
• Slide 7
• Turings Answer To What Computation Is 7 Computing is normally done by writing certain symbols on paper. We may suppose this paper is divided into squares like a child's arithmetic book. The behavior of the computer at any moment is determined by the symbols which he is observing, and his " state of mind " at that moment. We may suppose that there is a bound B to the number of symbols or squares which the computer can observe at one moment. If he wishes to observe more, he must use successive observations. We will also suppose that the number of states of mind which need be taken into account is finite.
• Slide 8
• Turings Answer To What Computation Is 8 Let us imagine the operations performed by the computer to be split up into "simple operations" which are so elementary that it is not easy to imagine them further divided. Every such operation consists of some change of the physical system consisting of the computer and his tape. We may suppose that in a simple operation not more than one symbol is altered.
• Slide 9
• Turings Answer To What Computation Is 9 Besides these changes of symbols, the simple operations must include changes to the observed squares. I think it is reasonable to suppose that they can only be squares whose distance from the closest of the immediately previously observed squares does not exceed a certain fixed amount.
• Slide 10
• The Turing Machine 10 Its paradoxical that as humans in our quest to understand what machines can do, we have been studying an abstract machine that in essence imitates a human being.
• Slide 11
• Church-Turing Thesis 11 Central Dogma of Computer Science: **Whatever is computable is computable by the Turing machine.** There is no proof of this claim.
• Slide 12
• Turing In Defense Of His Claim: 12 All arguments which can be given are bound to be, fundamentally, appeals to intuition, and for this reason rather unsatisfactory mathematically. The arguments which I shall use are of three kinds. (a) A direct appeal to intuition. (b) A proof of the equivalence of two definitions (in case the new definition has a greater intuitive appeal). (c) Giving examples of large classes of numbers which are computable.
• Slide 13
• Algorithm = Turing Machine 13 When we say there is an algorithm computing shortest paths of a graph in O(mlog(n)) times we really mean: There is a Turing Machine that computes the shortest paths of a graph in O(mlog(n)) operations.
• Slide 14
• Turing Machine Is A Very Powerful Machine 14 CS tries to understand the limits of TM. We limit/extend TM and try to understand what can be computed by it. Limit the # times its head is allowed to move left/right and it changes states to poly-time. => poly-time algs What if the machine had access to a random source => randomized algorithms. What if there were multiple heads on the tape => parallel algorithms Limit the length of its tape. => space-efficient algs What if the head was only allowed to move right => streaming algorithms
• Slide 15
• Outline For Today 1.Caching 2.Other Algorithms & Algorithmic Techniques Beyond CS 161 1.Parallel Algorithms (Finding Min, Bellman-Ford) 2.Linear Programming 3.Other Topics & Classes 15
• Slide 16
• Online Algorithms 16 Takes as input a possibly infinite stream. At each point in time t make a decision based on what has been seen so far but without knowing the rest of the input Type of Optimality Analysis: Competitive Ratio Worst (Cost of online algorithm)/(Cost of OPT) ratios against any input stream Where OPT is the best solution possible if we knew the entire input in advance
• Slide 17
• Caching 17 Slow Disk O.w (miss), send request to disk, put the page into cache. Q: Which page to evict? If page is in cache (hit) reply directly from cache
• Slide 18
• Caching 18 Input: N pages in disk, and stream of infinite page requests. Online Algorithm: Decide which page to evict from cache when its full and theres a miss. Goal: minimize the number of misses. Idea: LRU: Remove the Least Recently Used page
• Slide 19
• LRU with k = 3 19 412153441132451 miss
• Slide 20
• LRU with k = 3 20 412153441132451
• Slide 21
• LRU with k = 3 21 412153441132451 miss
• Slide 22
• LRU with k = 3 22 412153441132451
• Slide 23
• LRU with k = 3 23 412153441132451 miss
• Slide 24
• LRU with k = 3 24 412153441132451
• Slide 25
• LRU with k = 3 25 412153441132451 hit
• Slide 26
• LRU with k = 3 26 412153441132451 miss
• Slide 27
• LRU with k = 3 27 412153441132451
• Slide 28
• LRU with k = 3 28 412153441132451 miss
• Slide 29
• LRU with k = 3 29 412153441132451
• Slide 30
• LRU with k = 3 30 412153441132451 miss
• Slide 31
• LRU with k = 3 31 412153441132451
• Slide 32
• LRU with k = 3 32 412153441132451 hit
• Slide 33
• LRU with k = 3 33 412153441132451 miss
• Slide 34
• LRU with k = 3 34 412153441132451 so and so forth
• Slide 35
• Competitive Ratio Claim 35 Claim: If the optimal sequence of choices for a size-h cache causes m misses. Then, for the same sequence of requests, LRU for a size-k cache causes misses Interpretation: If LRU had twice as much cache size as an algorithm OPT that knew the future, it would have at most twice the misses of OPT. Note will prove the claim for
• Slide 36
• Proof of Competitive Ratio 36 Recursively break the sequence of inputs into phases. Let t be the time when we see the (k+1)st different request. Phase 1: a 1 a t-1 Let t` be the time we see the (k+1)st different element starting from a t Phase 2: a t a t-1 412153441132451
• Slide 37
• Proof of Competitive Ratio 37 412153441132451 k=3 Phase 1 Phase 2 Phase 3 Phase 4 By construction, each phase has k distinct requests. Q: At most how many misses does LRU have in each phase? A: k b/c even if it evicted everything in the k+1 st item, it would have at most k misses.
• Slide 38
• Proof of Competitive Ratio 38 412153441132451 Phase 1 Phase 2 Phase 3 Phase 4 Q: Whats the minimum misses that any size-h cache must have in any phase? A: k-h b/c k distinct items will be in the cache at different points during the phase, so at least k-h of them must trigger misses. Therefore the CR: k/k-h Q.E.D.
• Slide 39
• Outline For Today 1.Caching 2.Other Algorithms & Algorithmic Techniques Beyond CS 161 1.Parallel Algorithms (Finding Min, Bellman-Ford) 2.Linear Programming 3.Other Topics & Classes 39
• Slide 40
• 40 Parallel Algorithms Question: Which problems are parallelizable, which are inherently sequential? Parallelizable: Connected components, sorting, selection, many computational geometry problems all have parallel algorithms (Believed To Be) Inherently Sequential (P-complete): DFS Horn-satisfiability Conways Game of Life, and others.
• Slide 41
• 41 2 Common Computational Models Model 1: Shared Memory (PRAM): Single Machine Memory CPU 1 CPU 2 CPU k Each Time Step, each processor: Can read a location of memory Can write to a location of memory Q: How much time does it take to solve a computational problem with polynomial # processors?
• Slide 42
• 42 Example 1: Finding the min in an array Memory CPU (1,2) CPU (1,3) CPU (4, 5) There are n(n-1) processors, one for each pair (i, j) Initially allocate an array of size n all 0 Step 1: Each cpu (i, j) compares i and j If i < j, then write 1 to j o.w. write 1 to location i Step 2: return the item whose output memory location is 0
• Slide 43
• 43 Example 1: Finding the min in an array Memory CPU (1,2) CPU (1,3) CPU (4, 5)
• Slide 44
• 44 Model 2: Distributed Memory Machine 1 Machine 2 Machine k Input Partition 1 Input Partition 2 Input Partition k Each machine performs local computation send/receives messages to/from other machines can be synchronously or asynchronously Q: How much communication is necessary? Q: How many synchronizations is necessary?
• Slide 45
• Recap: Bellman-Ford 45 v, and for i={1, , n} P (v, i) : shortest s v path with i edges (or null) L (v, i) : w(P (v, i) ) (and + for null paths) L (v, i) = min L (v, i-1) min u: (u,v) E : L (u, i-1) + c (u,v)
• Slide 46
• 46 Example: Distributed Bellman-Ford A C D F B E 5 3 -6 2 2
• Slide 47
• 47 Example: Distributed Bellman-Ford 0 0 A C D F B E 5 3 -6 2 2
• Slide 48
• 48 Example: Distributed Bellman-Ford 0 0 5 5 A C D F B E 5 3 -6 2 2
• Slide 49
• 49 Example: Distributed Bellman-Ford 0 0 5 5 8 8 7 7 A C D F B E 5 3 -6 2 2
• Slide 50
• 50 Example: Distributed Bellman-Ford 1 1 0 0 5 5 8 8 7 7 A C D F B E 5 3 -6 2 2
• Slide 51
• 51 Example: Distributed Bellman-Ford 1 1 0 0 5 5 0 0 7 7 A C D F B E 5 3 -6 2 2
• Slide 52
• 52 Example: Distributed Bellman-Ford 1 1 0 0 5 5 0 0 7 7 A C D F B E 5 3 -6 2 2 Termination When No Vertex Value Changes
• Slide 53
• 53 Parallel Algorithms Parallel Computing: CS 149 A systems course. Teaches parallel technologies but also algorithms.
• Slide 54
• Outline For Today 1.Caching 2.Other Algorithms & Algorithmic Techniques Beyond CS 161 1.Parallel Algorithms (Finding Min, Bellman-Ford) 2.Linear Programming 3.Other Topics & Classes 54
• Slide 55
• 55 Linear Programming (CS 261/361) Optimization Problem of following structure: maximize x 1 + x 2 subject to x 1 + 2x 2 1 2x 1 + x 2 1 x 1 0 x 2 0
• Slide 56
• 56 Geometric Interpretation Constraint 2= 2x 1 + x 2 1 x1x1 x2x2 Constraint 1= x 1 + 2x 2 1 Feasible Solution Set **Opt Solution**
• Slide 57
• 57 Linear Programming Applications Tons! Lots and lots of problems can be solved or approximated with LP! Vertex Cover Set Cover Load Balancing Lots of problems in manufacturing/operations research/finance, etc.. Covered in CS 261/361 **Invented By George Dantzig.**
• Slide 58
• Outline For Today 1.Caching 2.Other Algorithms & Algorithmic Techniques Beyond CS 161 1.Parallel Algorithms (Finding Min, Bellman-Ford) 2.Linear Programming 3.Other Topics & Classes 58
• Slide 59
• 59 Approximation Techniques (including LP, SP: Semidefinite Programming) Many more Approximation Algorithms Approximation Algorithms (CS 261/CS 361)
• Slide 60
• 60 Commonly appearing general randomization techniques. Probabilistic Method Markov Chains **Chernoff Bounds** Very very cool/elegant algorithms! Randomized Algorithms (CS 365)
• Slide 61
• 61 Competitive Ratio Average-Case Analysis Instance Optimality Smoothed Analysis, and others Beyond Worst-Case (CS 369)
• Slide 62
• 62 Resource Lower Bounds on Computational Problems Algorithms study what can be computed and with how much resource? Complexity studies what cannot be computed with how much resource? Turing Machines/P-NP/Circuit- Complexity/Randomized-Complexity/Polynomial Hierarchy/Space Complexity/PCP/One-way Functions Complexity Theory (CS 254)
• Slide 63
• 63 Power of Quantum Computers over Classic Computers Quantum Teleportation Quantum Circuits Quantum Algorithms: Fourier Transform/Factoring/Search, etc Quantum Error Correction Quantum Computing (CS 259Q)
• Slide 64
• 64 Sequencing Genome assembly Gene sampling Gene finding Gene comparison Gene regulation, etc Computational Genomics (CS 262)
• Slide 65
• 65 Geometric modeling of physical objects Search in high-dimensional spaces Triangulation of a set of points Image Processing/Graphics/Vision Computational Geometry (CS 268)
• Slide 66
• 66 Design and Analysis of Computational Problems In Strategic Environments Key Analysis Concept: Price of Anarchy Also Studies Complexities of Finding Equilibria Auctions Traffic Routing Multi-agent Systems, and others Algorithmic Game Theory (CS 364)
• Slide 67
• 67 Randomized Algorithms (CS 365) (Chernoff Bounds) Linear Programming (CS 261/361) Complexity Theory (CS 254) My Recommendations For Future Classes
• Slide 68
• 68 Tim Roughgarden Keith Schwarz Billy, Mike & Yiming Acknowledgements
• Slide 69
• 69 Thank you! Good luck in the final!

Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Education
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents