View
221
Download
3
Embed Size (px)
Citation preview
CS420 lecture fivePriority Queues, Sorting
wim bohm cs csu
Heaps• Heap: array representation of a
complete binary tree– every level is completely filled except the bottom level: filled from left
to right
• Can compute the index of parent and children– parent(i) = floor(i/2) left(i)= 2i index(i)=2i+1 (for 1 based arrays)
• Heap property: A[parent(i)] >= A[i]
16
10
9 3
142
78
14
16 14 10 8 7 9 3 2 4 1
HeapifyTo create a heap at i, assuming left(i) and right(i) are heaps, bubble A[i] down: swap with max child until heap property holds
heapify(A,i){ L=left(i); R=right(i); if L<=N and A[L] > A[i] max=L else max = i; if R<=N and A[R]>A[max] max =R; if max != i { swap(A,i,max); heapify(A.max) } }
Building a heap
• heapify performs at most lg n swaps• building a heap out of an array:– The leaves are all heaps– heapify backwards starting at last internal node
buildheap(A){ for i = floor(n/2) downto 1 heapify(A,i)}
Complexity buildheap
• Suggestions? ...
Complexity buildheap
• initial thought O(nlgn), but
half of the heaps are height 1 quarter are height 2 only one is height log n
It turns out that O(nlgn) is not tight!
complexity buildheapheight
0
1
2
3
max #swaps
0 = 21-2
1 = 22-3
2*1+2 = 4 = 23-4
2*4+3 = 11 = 24-5
complexity buildheapheight
0
1
2
3
max #swaps
0 = 21-2
1 = 22-3
2*1+2 = 4 = 23-4
2*4+3 = 11 = 24-5
Conjecture: height = h max #swaps = 2h+1-(h+2)
Proof: induction height = (h+1) max #swaps: 2*(2h+1-(h+2))+(h+1) = 2h+2-2h-4+h+1 = 2h+2-(h+3) = 2(h+1)+1-((h+1)+2) n nodes O(n) swaps
Cormen et.al. complexity buildheap
€
N
2hh=0
lgN
∑ h = Nh
2hh=0
lgN
∑
h
2hh=0
∞
∑ =1/2
(1−1/2)2 = 2
Nh
2hh=0
lgN
∑ =O(n)
Differentiation trick (Cormen et.al. Appendix A)
€
x h
h=0
∞
∑ =1
1− x Infinite geometrics series | x |<1
differentiate both sides and multiply by x :
hx h
h=0
∞
∑ =x
(1− x)2
take x =1/2
Heapsort
heapsort(A){ buildheap(A); for i = n downto 2{ swap(A,1,i); n=n-1; heapify(A,1); }}
Complexity heapsort
• buildheap: O(n)• swap/heapify loop: O(nlgn)
• space: in place: n– less space than merge sort
142
78
14
1412
74
8
1482
14
7
1487
12
4
1487
41
2
1487
42
1
Priority Queues
• heaps are used in priority queues– each value associated with a key– max priority queue S (as in heapsort) has
operations that maintain the heap property of S• insert(S,x)• max(S) returning max element• Extract-max(S) extracting and returning max element• increase key(S,x,k) increasing the key value of x
Extract max: O(log n)
Extract-max(S){ // pre:N>0 max=S[1]; S[1]=S[N]; N=N-1; heapify(S) }
O(log N)
Increase key: O(log n)
Increase-key(S,i,k){ //pre: k>S[i] A[i]=k; // bubble up while(i>1 and S[parent(i)]<S[i]){ swap(S,i,parent(i)); i = parent(i) }}
Insert O(log n)
• Insert(S,x)– put x at end of S– bubble x up like in Increase-key
Decrease-key
• How would decrease key work?
• What would be its complexity?
Quicksort
• Quicksort has worst case complexity?
Quicksort
• Quicksort has worst case complexity O(n2)
• So why do we care about Quicksort?
Quicksort
• Quicksort has worst case complexity O(n2)
• So why do we care about Quicksort?– Because it is very fast in practice• Average complexity: O(nlgn)• Low multiplicative constants• Sequential array access• In place • Often faster than MergeSort and HeapSort
Partition: O(n)Partition(A,p,r){ // partition A[p..r] in-place in two sub arrays: low and hi // all elements in low < all elements in hi // return index of last element of low x=A[p]; i=p-1; j=r+1; while true repeat j=j-1 until A[j]<=x repeat i=i+1 until A[i]>x if i<j swap(A,i,j) else return j}
QuickSort
Quicksort(A,p,r){ if p<r { q=Partition(A,p,r) Quicksort(A,p,q) Quicksort(A,q+1,r) } }
Worst case complexity: when one partition is size 1: T(n)=T(n-1)+n=T(n-2)+(n-1)+n=T(n-3)+(n-2)+(n-1)+n= = O(n2)
Quicksort average case complexity
• Assume a uniform distribution: each partition index has equal probability 1/n. Thus
€
T(n) = (n +1) +1
nT(q) +T(n −q)
q=1
n−1
∑
T(0) = T(1) = 0
Quicksort average case complexity
We are summing all Ts up (T(q)) and down (T(N-q)), both are the same sums so
€
T(n) = (n +1) +1
nT(q) +T(n −q)
q=1
n−1
∑
T(0) = T(1) = 0
€
T(n) = (n +1) +2
nT(q)
q=1
n−1
∑
Quicksort average case complexity
• multiply both sides by n
• substitute n-1 for n€
T(n) = (n +1) +2
nT(q)
q=1
n−1
∑
€
nT(n) = n(n +1) + 2 T(q)q=1
n−1
∑ (2)
€
(n −1)T(n −1) = n(n −1) + 2 T(q)q=1
n−2
∑ (3)
Quicksort average case complexity
• subtract (2)-(3)€
nT(n) = n(n +1) + 2 T(q)q=1
n−1
∑ (2)
€
(n −1)T(n −1) = n(n −1) + 2 T(q)q=1
n−2
∑ (3)
€
nT(n) − (n −1)T(n −1) = 2n + 2T(n −1)
nT(n) = 2n + 2T(n −1) + (n −1)T(n −1)
nT(n) = 2n + (n +1)T(n −1)
Quicksort average case complexity
• Which technique that we have learned can help us here?
€
nT(n) = 2n + (n +1)T(n −1)
a non − linear recurrence relation :
T(n) = 2 +n +1
nT(n −1)
Quicksort average case complexity
• repeated substitution!!
€
T(n) = 2 +n +1
nT(n −1)
T(n −1) = 2 +n
n −1T(n − 2)
T(n − 2) = 2 +n −1
n − 2T(n − 3)
etc.
T(n) = 2 + 2n +1
n+ 2
(n +1)n
n(n −1)+ 2
(n +1)n(n −1)
n(n −1)(n − 2)+ ...
Quicksort average case complexity
€
T(n) = 2 + 2n +1
n+ 2
(n +1)n
n(n −1)+ 2
(n +1)n(n −1)
n(n −1)(n − 2)+ ...
simplify
T(n) = 2 + 2(n +1)1
n+ 2(n +1)
1
(n −1)+ 2(n +1)
1
(n − 2)+ ...
reorder
T(n) = 2 + 2(n +1)1
kk=1
n
∑ = 2(n +1)1
kk=1
n
∑
Harmonic series from previous lecture : 1
kk=1
n
∑ = ln(n) +O(1)
T(n) =O(n lgn)