Upload
bruce-murphy
View
61
Download
0
Tags:
Embed Size (px)
DESCRIPTION
COSC 2P03. Lecture #5 – Trees Part III, Heaps. Today. Take up the quiz Assignment Questions Red-Black Trees Binary Heaps Heap sort D-Heaps, Leftist Heaps Brief: Skew, Binomial, Fibonacci. Review From Last Class. Why balance trees? How to DSW balanced tree? - PowerPoint PPT Presentation
Citation preview
1
COSC 2P03
Lecture #5 – Trees Part III, Heaps
2
Today
Take up the quiz Assignment Questions Red-Black Trees Binary Heaps Heap sort D-Heaps, Leftist Heaps Brief: Skew, Binomial, Fibonacci
3
Review From Last Class
Why balance trees? How to DSW balanced tree? What is the idea behind AVL balancing? What is the idea behind splay trees? When should we use AVL? Splay? What is amortized analysis?
4
Red Black Trees
Is really a 2-4 tree, that has been represented as a binary tree Comparing with 2-4 tree
Still has O(lgN) performance Simpler to implement!
2 5 8 5
2 8
5
Red-Black Trees
Is a BST where Every node is either colored “Red” or “Black” Every null pointer is Black
Meaning that every “real” node has 2 children If a node is Red, then its children are Black
So, can’t have 2 consecutive reds on a path Every path from root to null contains the same # of Black nodes
Meaning that all leaves have the same “Black Height” The root is always black
The BLACK HEIGHT of a node N is the number of Black Nodes on any path to null, not including N – but including null as black
6
The Black-Height
Black Height of the root? 3
Of “X” 2
X
7
16
14 10
8 7 9 3
2 4 1
Coloring an RBT
1. Every node is either red or black2. Every leaf (null) is black3. If a node is red, then both its children are
black4. All paths from node to descendants contain
the same number of black nodes5. The root is black
8
16
14 10
8 7 9 3
2 4 1
Coloring an RBT
1. Every node is either red or black2. Every leaf (null) is black3. If a node is red, then both its children are
black4. All paths from node to descendants contain
the same number of black nodes5. The root is black
9
16
14 10
8 7 9 3
2 4 1
Coloring an RBT
1. Every node is either red or black2. Every leaf (null) is black3. If a node is red, then both its children are
black4. All paths from node to descendants contain
the same number of black nodes5. The root is black
10
16
14 10
8 7 9 3
2 4 1
Coloring an RBT
1. Every node is either red or black2. Every leaf (null) is black3. If a node is red, then both its children are
black4. All paths from node to descendants contain
the same number of black nodes5. The root is black
11
16
14 10
8 7 9 3
2 4 1
Coloring an RBT
1. Every node is either red or black2. Every leaf (null) is black3. If a node is red, then both its children are
black4. All paths from node to descendants contain
the same number of black nodes5. The root is black
12
16
14 10
8 7 9 3
2 4 1
Coloring an RBT
H = 4B = 2
H = 2B = 2
H = 1B = 1
H = 1B = 1
H = 2B = 1
H = 3B = 2
H = 2B = 1 H = 1
B = 1
1. Every node is either red or black2. Every leaf (null) is black3. If a node is red, then both its children are
black4. All paths from node to descendants
contain the same number of black nodes5. The root is black
13
Theorem 1
Any RBT (Red-Black Tree) rooted at X has n ≥ 2bh(X) – 1 nodes bh(X) is black height of node X
Proof: (idea behind inductive proof) Consider a node with 2 children Each child has a bh(X) or bh(X)-1, depending on if it is red or
black Since height of child < height of X, can apply 2bh(X) – 1 for the
children node: 2bh(X)-1 – 1 So, subtree rooted at X has at least:
(2bh(X)-1 – 1)+(2bh(X)-1 – 1) + 1= 2bh(X) – 1 internal nodes
14
Theorem 2
In an RBT at least ½ the nodes on any path from the root to null must be Black
Proof: If there is a red node, by definition of RBT there must exist a
black node Meaning that bh(X) ≥h/2
15
Theorem 3
There does not exist a path from any X to null that is more than twice as long as any other path from X to any another null
Proof: Any path from X to any null contains the same number of black
nodes From Theorem 2, we know that at least ½ of the nodes on any
path are black This means that the length of any path from X is no more than
twice as long as any other
16
Theorem 4
A RBT with N nodes has a height of h ≤ 2 lg(N + 1) Proof:
Let h be the height of an RBT rooted at X From Theorem 2: bh(X) ≥ h/2 From Theorem 1: N ≥ 2bh(X) – 1 Thus,
N ≥ 2 h/2 – 1 N + 1 ≥ 2h/2
lg(N + 1) ≥ h/22lg(N + 1) ≥ h
Implying that the height of an RBT is O(lgN)
17
Insertion
Insert a node Z uses the same insertion algorithm as BST, where Z is given the color red (unless it is the root) Preserve root, external and depth properties If parent of Z is black, also preserves internal property If parent of Z is red, it causes a violation of internal property and
the tree must be reorganized How to fix this?
18
Fixing the Double Red
Let Z be the newly inserted node, P its parent and S its sibling (uncle/aunt of Z), then there are 2 cases that may arise
Case 1: S is red Double red corresponds to an overflow Recolor the nodes In 2-4 tree same as a split
Case 2: S is black Double red is an incorrect placement of a 4-node Restructure the 4-node
19
Re-coloring
Fixes the case where the parent node P and its sibling S are both red
P and S become black and parent G of P and S becomes red, unless it is the root
It is possible that this can propagate back to the grandparent of G
2
5
9
7
PS
G
2
5
9
7
PS
G
20
Restructuring
Fixes the case where the Parent (P) is red and its sibling (S) is black
Same as restoring the correct location of a 4-node Restores internal property There are 4 cases to consider, depending on structure of
subtree Same as the ones used in AVL trees, but slightly modified to
keep the color properties
21
Ex. Restructuring
1 2 3 4
=>
3
6
83
6
83
8
6
8
3
6
3 8
6
22
Insertion Algorithm
Search for location to insert node ZAdd Z to the tree and color it redwhile doubleRed(Z)
if sibling of Parent(Z) is black Z = restructure(Z)else Z = recolor(Z)
Searching is O(lgN) Adding Z is O(1) At most 1 restructure: O(lgN), O(lgN) recolorings each is O(1) So, insertion is O(lgN)
23
Insertion Example
Insert 10, 3, 15, 20, 1
10
5 15
201
24
Insertion Example
Insert 5
10
3 15
201
.
10
3 15
201 5
25
Insertion Example
Insert 7
10
3 15
201 5
Red red parent has red child
10
3 15
201 5
7
Recolor
7
Color red, then fix by recoloring.
10
3 15
201 5
7
Recolor parent, uncle to black.Recolor grandparent to red.
26
Insertion Example10
3 15
201 5
7
Insert 8
10
3 15
201 5
7
8
Previous recoloring doesn’t work.Uncle (a leaf) already black.
Recolor & Rotate
10
3 15
201 7
85
Recolor parent black.Recolor grandparent red.Rotate grandparent.
27
Insertion Example10
3 15
201 7
85
Insert 9
10
3 15
201 7
85
9
Uncle is so…Recolor parent, uncle to black.Recolor grandparent to red.
Recolor
10
3 15
201 7
85
9
Grandparent & its parent might now both be red. Continue.
28
Insertion Example10
3 15
201 7
85
9
Uncle of current node is black, so…
Need to recolor parent & grandparent.
But current node is a right child while parent is a left child. Want both to be same side.
Rotate
10
3
15
20
1
7
8
5 9
29
Insertion Example
Recolor & Rotate
10
3
15
20
1
7
8
5 9
103
15
20
1
7
85
9
30
Deletion
Basically the same as insertion Delete as you would in a BST, then check to maintain
properties Read about this in Weiss (12.2) and/or Cormen (13)
31
Conclusion
Red Black vs Splay Trees Fewer Rotations 1 extra color bit
Where are they used? Generally for look-up tables (key, value) data In Java: The TreeMap and TreeSet classes in the java.util
package When would you use?
AVL? B-Tree? Red-Black? Splay?
32
33
Priority Queues
Unordered Linked List Insert O(1), delete O(n)
Ordered Linked List Insert O(n), delete O(1)
Balanced BST Insert, delete, remove O(log n) Merge O(n)
A better way exists…Heaps!!!
34
Binary Heaps
Binary Heaps Are Complete and Partially ordered trees Complete: Filled on all levels, except the last one where it is
filled left to right (so between 2h and 2h+1-1 nodes) Partially ordered: Key of each node is <= parent (for min-heap)
or >= parent (max-heap) Note: left child is not necessarily less than right child
Since they are complete, can be easily stored as an array
35
Binary Heaps
Min/Max element is in the root Heap with N elements has height )lg(N
6
14
78 18
81 7791
45
5347
6484 9983
36
Insertion
Idea: Put into next available leaf node Recursively, if the new node is < parent then swap (if min-heap)
This may “bubble up” to the root, also called “sifting up”
Complexity? O(lgN), but on average O(1)
37
Insertion
Insert element X into heap Insert into next available slot Bubble up until obeys heap ordering rule (min/max heap)
6
14
78 18
81 7791
45
5347
6484 9983 42 next free slot
38
Insertion
Insert element X into heap Insert into next available slot Bubble up until obeys heap ordering rule (min/max heap)
6
14
78 18
81 7791
45
5347
6484 9983 4242
swap with parent
39
Insertion
Insert element X into heap Insert into next available slot Bubble up until obeys heap ordering rule (min/max heap)
6
14
78 18
81 7791
45
4247
6484 9983 4253
swap with parent
40
Insertion
Insert element X into heap Insert into next available slot Bubble up until obeys heap ordering rule (min/max heap)
6
14
78 18
81 7791
42
4547
6484 9983 53
stop: heap ordered
41
Deletion
Idea: Remove the root from the tree Remove the last (rightmost) leaf and place it at the root Perform the “sift down” operation
If the key is larger then swap with the smallest child, repeat.
Complexity? O(lgN), on average O(lgN)
42
Binary Heap: Delete Min
Delete minimum element from heap. Exchange root with rightmost leaf. Bubble root down until it's heap ordered.
6
14
78 18
81 7791
42
4547
6484 9983 53
43
Binary Heap: Delete Min
Delete minimum element from heap. Exchange root with rightmost leaf. Bubble root down until it's heap ordered.
53
14
78 18
81 7791
42
4547
6484 9983 6
44
Binary Heap: Delete Min
Delete minimum element from heap. Exchange root with rightmost leaf. Bubble root down until it's heap ordered.
53
14
78 18
81 7791
42
4547
6484 9983
exchange with left child
45
Binary Heap: Delete Min
Delete minimum element from heap. Exchange root with rightmost leaf. Bubble root down until it's heap ordered.
14
53
78 18
81 7791
42
4547
6484 9983
exchange with right child
46
Binary Heap: Delete Min
Delete minimum element from heap. Exchange root with rightmost leaf. Bubble root down until it's heap ordered.
14
18
78 53
81 7791
42
4547
6484 9983
stop: heap ordered
47
Building a Heap
Start at the root Recursively travel to each of the leaves. From each leaf, perform the “sift up” operations
Pseudo-code: for i= downto 1
minHeapify(A,i) //or maxHeapify
Where A is an array representation of the heap, and i is the current node being inspected for the heap property
2/)(Alength
48
Fix the bottom level
Fix the next to bottom level
Fix the top level
49
Building a Heap
50
Complexity of Heap Building
Theorem: Given a perfect binary tree of height h, containing 2h+1-1 nodes the sum of the heights of the nodes is 2h+1-1-(h+1)
What is the complexity? O(n) Why?
Sum of the heights of the nodes is N
Proof:
)()1(1222...842
)1(2...)2(8)1(42)(22
)1(2...)3(8)2(4)1(2)(2
11
10
10
NOhhS
hhhihS
hhhhihS
hhh
hihi
hihi
51
Sorting
How would you use heaps to sort?
52
Heap Sort
First Build the heap Continually call the Delete method until no more items are in
the heap Complexity?
O(N lgN)
53
H1 H214
78 18
81 7791
11
6253
6484 9941
Union
Union Combine two binary heaps H1 and H2 into a single heap.
No easy solution - (N) operations apparently required Can support fast union with fancier heaps.
54
55
D-heaps
Same as a binary heap except each node has d children So, a binary heap is a d-heap or order 2 A 3-heap could look like:
56
D-Heaps
Since more children per node it is shallower than a 2-heap Improves insert running time O(logdn)
If d is large then deletion becomes costly Shallower tree but must find the minimum of d children which is
a linear algorithm Result is O(dlogdn)
57
D-Heaps
A speedup is possible if the number of insertions is much greater than the number of deletions
If the heap becomes so large that it cannot entirely fit in memory
Possible that 4-heap is better than a 3-heap
58
Union
Problem: Merging 2 array implemented heaps is linear in time Why? Must copy 1 heap into another Could use a linked list, but then other operations may be slower
because we are using linear lists Is it possible to use arrays and still be able to merge heaps in
O(lgn)?
59
Leftist Heap
Ordering is same as for 2-heaps Structure is a bit more complicated
Leftist heaps not perfectly balanced, rather make a point not to be balanced!
The Null Path Length (NPL) of a node X is the length of the shortest path from X to a node without 2 children
Requirement for Leftist heap: For every node X, the NPL(X.left) >= NPL(X.right)
60
Theorem
A leftist tree with R nodes on the right path must have at least 2R-1 nodes
Proof: If R=1, then there must be >= 1 node Assume the theorem is true for 1,2,…R Consider leftist tree with R+1 nodes on the right path
So, root has a right subtree with R nodes on the right path Left subtree with at least R nodes on the right path (or not leftist) Applying 2R-1 hypothesis to each subtree results in a minimum of
2R-1 in each subtree. If we also add in the root, we get 2R+1-1 total nodes. QED.
61
Operations
Merge: main reason for existence! O(lgN) Insert: merges a 1 node heap with an existing heap Delete: removes the root then merges the subtrees
62
Merge Algorithm
Merge(H1, H2) If H1 or H2 is empty, return non-empty heap Recursively merge the heap with the larger root with the right
subheap of the other (smaller rooted) heap Make the new heap the right child of the root of H1
Satisfies heap property, but not leftist property – except the right subtree of the root is leftist
Done by swapping the root’s left and right children and updating the NPL
63
Merge Example
8 6 9
6 8 5
4 3
6
64
Merge Example
8 6 9
6 8 5
4 3
6
Merge right subtree of tree with smaller root and the other tree
65
Merge Example
8 6 9
6 8 5
4 3
6
Merge right subtree of tree with smaller root and the other tree
66
Merge Example
8 6
6 8
4 6
Merge right subtree of tree with smaller root and the other tree
67
Merge Example
8 6
Merge right subtree of tree with smaller root and all of other tree.
Right subtree of 6 is empty. So, result of melding right subtree of tree with smaller root and other tree is the other tree.
68
Merge Example
Swap left and right subtree if NPL(left) < NPL(right)
Make merged subtree right subtree of smaller root
6
8
6
8
69
Merge Example
8 6
6 6
4
88 6
6
46
8
Make merged subtree right subtree of smaller root
70
Merge Example
Make melded subtree right subtree of smaller root, then swap children
9
5
3
8 6
6 6
4
8
1 2
71
Merge Example
9
5
3
8 6
6 6
4
8
Final Result!
72
Other Types
Skew Heaps Like leftist, but no left balancing condition Always swaps children after a merge
Binomial Queues Collection of heap ordered “binomial trees” each with size of power 2 Merge is like adding integers in base 2 Very flexible
Fibonacci Heaps Variation of binomial queues “sift up” runs in O(1) amortized time Other operations in O(lgN) amortized time