Upload
roxanne-bates
View
218
Download
0
Embed Size (px)
Citation preview
1
CPSC 320: Intermediate AlgorithmDesign and Analysis
July 16, 2014
2
Course Outline
• Introduction and basic concepts
• Asymptotic notation
• Greedy algorithms
• Graph theory
• Amortized analysis
• Recursion
• Divide-and-conquer algorithms
• Randomized algorithms
• Dynamic programming algorithms
• NP-completeness
3
Union-Find Data Structure
4
Union-Find Data Structure
• Assume any graph algorithm that needs to maintain connected components
• Operations:
• Create: build a new set where each node is in its own component
• Find: return which component a node is in
• Union: join two components
• Objective: create should run in , find and union should run in or better
5
Tree implementation
• Idea: each node points to a “parent”, nodes are identified by the parent
• Create: pointers are initialized to nothing
• Find: return the root of the tree that contains the node
• Union: set the parent of one node to be the parent of the other
• Problem: union is , but find is , can we improve?
• Union: the shallowest becomes child of the deepest
• Each node keeps track of its depth (“rank”)
• Path compression:
• When find is performed, every node in the path is updated to point to root
6
Cost claims
• Claim: Find takes in average over calls
• is the (binary) iterated logarithm function
• Number of times lg needs to be applied until 1 (or less) is reached
• Example: (because , lg was applied 5 times)
7
Amortized Analysis
8
Amortized cost
• Although the worst case for one call has a large bound, we are interested in the total cost of a sequence of calls
• In other words, calls to a function may be better than
• The amortized cost of the function takes the total amount in consideration
9
Potential Method
• Each operation receives a potential cost, which if larger than actual cost can be “saved” towards future operations
• Operations that take longer can use “saved” value
• Total cost of sequence of operations is the sum of potential costs
• Properties:
• Assume is state of algorithm after iterations
• (you cannot use more than what you saved)
• (you start with no saved cost – usually)
• Alternative notation:
• for real cost, for amortized cost
10
Union-Find with Path Compression
• Property: For every node N, rank(N)<rank(parent(N))
• Initial update: trivial; after compression: property is maintained
• Property: A node of rank has at least nodes in its tree
• Proof by induction on
• Property: If there are elements, then there are at most nodes of rank
• This implies the maximum rank is
• Theorem: The amount of time required for find+union operations on a union-find structure with elements is
11
Proof – Potential Function
• Potential function depends on depth, but also on comparing with parents
• Assume is a node in the tree, is its parent, and is its rank
• Divide nodes based on the comparison of ranks between node and parent:
• root: no parent
• close:
• far: (but not close)
• really far: everything else
• Observation:
• if is close
• if is far
12
Proof – Potential Function
• Based on previous observations, , and thus
• Now let’s analyse the cost of each operation
13
Proof – Create operation
• Create runs in linear time, as expected.
14
Proof – Union operation
• If we add tree rooted in to tree rooted in :
• decreases (changes from root to some other rule)
• may increase by 3 if rank is incremented
• If rank of increases, if then may decrease
• All other nodes are unaltered
• So , and by consequence
15
Proof – Find operation
• Suppose that we are looking for node , and to find its root we look at (and update) nodes
• Then real cost of find is
• What about ? Let’s track for which nodes changes
• does not change
• does not change
• nodes not in do not change
• For all nodes new parent rank is now higher, so may change from:
• really far to really far (does not change)
• far to far or really far
• close to close, far or really far
16
Proof – Find operation
• If changes from close or far to really far:
• in , in , so decreases
• If changes from close to close:
• increases by at least , so decreases
• If changes from close to far:
• In ,
• In , , so decreases
• If changes from far to far:
• For at most one node, , so may be the same
• For all other nodes, , so decreases
17
Proof – Find operations
• So, which nodes did not decrease?
• , , at most one far node, and nodes that were really far before
• In total: at most
• So, how many nodes did decrease (by at least 1)?
• at least
18
Proof – Conclusion
• Since and , a sequence of operations is
• CQD
19
Recurrence Relations
20
Recursive Functions
• Complexity analysis of iterative functions has been covered
• How can we analyse the complexity of a recursive function?
• Example:
Algorithm MergeSort(, , )
If () Then
MergeSort(, , )
MergeSort(, , )
Merge(, , , )
21
Recurrence Relations
• The complexity of MergeSort can be computed as:
• How can we figure out what the complexity is for the function above?
22
Guess and Test
• First approach: guess one expression and test if it works
• Test usually involves strong induction
• Example: prove that
• Induction step:
23
Guess and Test
• Base case:
• :
doesn’t work!
• :
2 is not enough, since
24
Guess and Test
• Base case (cont.):
• :
• So, for and ,
25
Guess and Test
• Prove that
• Induction step:
• Prove didn’t reach expected result, disproved
• Careful, even though c+d is a constant, we needed the constant to be c only.
26
Guess and Test
• Prove that
• Induction step:
• Prove didn’t reach expected result, but we got close
• Let’s try a stronger claim:
27
Guess and Test
• Prove that
• Induction step:
• Base case,
• Proved that for and