Upload
nita
View
25
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Analysis of Algorithms CS 477/677. Instructor: Monica Nicolescu Lecture 9. Array Representation of Heaps. A heap can be stored as an array A . Root of tree is A[1] Left child of A[i] = A[2i] Right child of A[i] = A[2i + 1] Parent of A[i] = A[ i/2 ] Heapsize[A] ≤ length[A] - PowerPoint PPT Presentation
Citation preview
Analysis of AlgorithmsCS 477/677
Instructor: Monica NicolescuLecture 9
CS 477/677 - Lecture 9 2
How Fast Can We Sort?
• Insertion sort, Bubble Sort, Selection Sort
• Merge sort
• Quicksort
• What is common to all these algorithms?– These algorithms sort by making comparisons between the
input elements
• To sort n elements, comparison sorts must make
(nlgn) comparisons in the worst case
(n2)(nlgn)(nlgn)
CS 477/677 - Lecture 9 3
COUNTING-SORT
Alg.: COUNTING-SORT(A, B, n, k)1. for i ← 0 to k2. do C[ i ] ← 03. for j ← 1 to n4. do C[A[ j ]] ← C[A[ j ]] + 15. C[i] contains the number of elements equal to i6. for i ← 1 to k7. do C[ i ] ← C[ i ] + C[i -1]8. C[i] contains the number of elements ≤ i9. for j ← n downto 110. do B[C[A[ j ]]] ← A[ j ]11. C[A[ j ]] ← C[A[ j ]] - 1
1 n
0 k
A
C1 n
B
j
CS 477/677 - Lecture 9 4
Example
303203521 2 3 4 5 6 7 8
A
032021 2 3 4 5
C 10
774221 2 3 4 5
C 80
31 2 3 4 5 6 7 8
B
764221 2 3 4 5
C 80
301 2 3 4 5 6 7 8
B
764211 2 3 4 5
C 80
3301 2 3 4 5 6 7 8
B
754211 2 3 4 5
C 80
33201 2 3 4 5 6 7 8
B
753211 2 3 4 5
C 80
CS 477/677 - Lecture 9 5
Example (cont.)
303203521 2 3 4 5 6 7 8
A
332001 2 3 4 5 6 7 8
B
753201 2 3 4 5
C 80
53332001 2 3 4 5 6 7 8
B
743201 2 3 4 5
C 70
3332001 2 3 4 5 6 7 8
B
743201 2 3 4 5
C 80
533322001 2 3 4 5 6 7 8
B
CS 477/677 - Lecture 9 6
Analysis of Counting SortAlg.: COUNTING-SORT(A, B, n, k)1. for i ← 0 to k2. do C[ i ] ← 03. for j ← 1 to n4. do C[A[ j ]] ← C[A[ j ]] + 15. C[i] contains the number of elements equal to i6. for i ← 1 to k7. do C[ i ] ← C[ i ] + C[i -1]8. C[i] contains the number of elements ≤ i9. for j ← n downto 110. do B[C[A[ j ]]] ← A[ j ]11. C[A[ j ]] ← C[A[ j ]] - 1
(k)
(n)
(k)
(n)
Overall time: (n + k)
CS 477/677 - Lecture 9 7
Analysis of Counting Sort
• Overall time: (n + k)• In practice we use COUNTING sort when k =
O(n) running time is (n)
• Counting sort is stable– Numbers with the same value appear in the same order
in the output array
– Important when additional data is carried around with
the sorted keys
CS 477/677 - Lecture 9 8
Radix Sort
• Considers keys as numbers in a base-k number– A d-digit number will occupy a field of d
columns• Sorting looks at one column at a time
– For a d digit number, sort the least significant digit first
– Continue sorting on the next least significant digit, until all digits have been sorted
– Requires only d passes through the list
CS 477/677 - Lecture 9 9
RADIX-SORTAlg.: RADIX-SORT(A, d)
for i ← 1 to ddo use a stable sort to sort array A on digit i
• 1 is the lowest order digit, d is the highest-order digit
CS 477/677 - Lecture 9 10
Analysis of Radix Sort
• Given n numbers of d digits each, where each
digit may take up to k possible values, RADIX-
SORT correctly sorts the numbers in (d(n+k))– One pass of sorting per digit takes (n+k) assuming
that we use counting sort
– There are d passes (for each digit)
CS 477/677 - Lecture 9 11
Correctness of Radix sort• We use induction on the number d of passes through the
digits• Basis: If d = 1, there’s only one digit, trivial• Inductive step: assume digits 1, 2, . . . , d-1 are sorted
– Now sort on the d-th digit– If ad < bd, sort will put a before b: correct
a < b regardless of the low-order digits– If ad > bd, sort will put a after b: correct
a > b regardless of the low-order digits– If ad = bd, sort will leave a and b in the
same order and a and b are already sorted on the low-order d-1 digits
CS 477/677 - Lecture 9 12
Bucket Sort• Assumption:
– the input is generated by a random process that distributes elements uniformly over [0, 1)
• Idea:– Divide [0, 1) into n equal-sized buckets– Distribute the n input values into the buckets– Sort each bucket– Go through the buckets in order, listing elements in each one
• Input: A[1 . . n], where 0 ≤ A[i] < 1 for all i • Output: elements in A sorted• Auxiliary array: B[0 . . n - 1] of linked lists, each list
initially empty
CS 477/677 - Lecture 9 13
BUCKET-SORT
Alg.: BUCKET-SORT(A, n)
for i ← 1 to n do insert A[i] into list B[nA[i]]for i ← 0 to n - 1
do sort list B[i] with insertion sort
concatenate lists B[0], B[1], . . . , B[n -1] together in order
return the concatenated lists
CS 477/677 - Lecture 9 14
Example - Bucket Sort
.78
.17
.39
.26
.72
.94
.21
.12
.23
.68
0
1
2
3
4
5
6
7
8
9
1
2
3
4
5
6
7
8
9
10
.21
.12 /
.72 /
.23 /
.78
.94 /
.68 /
.39 /
.26
.17
/
/
/
/
CS 477/677 - Lecture 9 15
Example - Bucket Sort
0
1
2
3
4
5
6
7
8
9
.23
.17 /
.78 /
.26 /
.72
.94 /
.68 /
.39 /
.21
.12
/
/
/
/
.17.12 .23 .26.21 .39 .68 .78.72 .94 /
Concatenate the lists from 0 to n – 1 together, in order
CS 477/677 - Lecture 9 16
Correctness of Bucket Sort
• Consider two elements A[i], A[ j]• Assume without loss of generality that A[i] ≤ A[j]• Then nA[i] ≤ nA[j]
– A[i] belongs to the same group as A[j] or to a group with a lower index than that of A[j]
• If A[i], A[j] belong to the same bucket:– insertion sort puts them in the proper order
• If A[i], A[j] are put in different buckets:– concatenation of the lists puts them in the proper order
CS 477/677 - Lecture 9 17
Analysis of Bucket Sort
Alg.: BUCKET-SORT(A, n)
for i ← 1 to n
do insert A[i] into list B[nA[i]]for i ← 0 to n - 1
do sort list B[i] with insertion sort
concatenate lists B[0], B[1], . . . , B[n -1] together in order
return the concatenated lists
(n)
(n)
(n)
(n)
CS 477/677 - Lecture 9 18
Conclusion
• Any comparison sort will take at least nlgn to sort an
array of n numbers
• We can achieve a better running time for sorting if we
can make certain assumptions on the input data:– Counting sort: each of the n input elements is an integer in the
range 0 to k– Radix sort: the elements in the input are integers represented
with d digits
– Bucket sort: the numbers in the input are uniformly distributed
over the interval [0, 1)
CS 477/677 - Lecture 9 19
A Job Scheduling Application
• Job scheduling– The key is the priority of the jobs in the queue– The job with the highest priority needs to be executed
next
• Operations– Insert, remove maximum
• Data structures– Priority queues– Ordered array/list, unordered array/list
CS 477/677 - Lecture 9 20
PQ Implementations & Cost
Worst-case asymptotic costs for a PQ with N items
Insert Remove max
ordered arrayordered listunordered arrayunordered list
N
N
N
N
1
1
1
1
Can we implement both operations efficiently?
CS 477/677 - Lecture 9 21
Background on Trees• Def: Binary tree = structure composed of a finite
set of nodes that either:– Contains no nodes, or– Is composed of three disjoint sets of nodes: a root
node, a left subtree and a right subtree
2
14 8
1
16
4
3
9 10
root
Right subtreeLeft subtree
CS 477/677 - Lecture 9 22
Special Types of Trees• Def: Full binary tree = a
binary tree in which each node is either a leaf or has degree (number of children) exactly 2.
• Def: Complete binary tree = a binary tree in which all leaves have the same depth and all internal nodes have degree 2.
Full binary tree
2
14 8
1
16
7
4
3
9 10
12
Complete binary tree
2
1
16
4
3
9 10
CS 477/677 - Lecture 9 23
The Heap Data Structure• Def: A heap is a nearly complete binary tree
with the following two properties:– Structural property: all levels are full, except
possibly the last one, which is filled from left to right– Order (heap) property: for any node x
Parent(x) ≥ x
Heap
5
7
8
4It doesn’t matter that 4 in level 1 is smaller than 5 in level 22
CS 477/677 - Lecture 9 24
Definitions• Height of a node = the number of edges on a longest
simple path from the node down to a leaf• Depth of a node = the length of a path from the root to
the node• Height of tree = height of root node
= lgn, for a heap of n elements
2
14 8
1
16
4
3
9 10
Height of root = 3
Height of (2)= 1 Depth of (10)= 2
CS 477/677 - Lecture 9 25
Array Representation of Heaps• A heap can be stored as an
array A.– Root of tree is A[1]– Left child of A[i] = A[2i]– Right child of A[i] = A[2i + 1]– Parent of A[i] = A[ i/2 ]– Heapsize[A] ≤ length[A]
• The elements in the subarray A[(n/2+1) .. n] are leaves
• The root is the maximum element of the heap
CS 477/677 - Lecture 9 26
Heap Types
• Max-heaps (largest element at root), have the max-heap property: – for all nodes i, excluding the root:
A[PARENT(i)] ≥ A[i]
• Min-heaps (smallest element at root), have the min-heap property:– for all nodes i, excluding the root:
A[PARENT(i)] ≤ A[i]
CS 477/677 - Lecture 9 27
Operations on Heaps
• Maintain the max-heap property– MAX-HEAPIFY
• Create a max-heap from an unordered array– BUILD-MAX-HEAP
• Sort an array in place– HEAPSORT
• Priority queue operations
CS 477/677 - Lecture 9 28
Operations on Priority Queues
• Max-priority queues support the following
operations:– INSERT(S, x): inserts element x into set S– EXTRACT-MAX(S): removes and returns element of
S with largest key
– MAXIMUM(S): returns element of S with largest key
– INCREASE-KEY(S, x, k): increases value of element
x’s key to k (assume k ≥ current key value at x)
CS 477/677 - Lecture 9 29
Building a Heap
Alg: BUILD-MAX-HEAP(A)1. n = length[A]
2. for i ← n/2 downto 13. do MAX-HEAPIFY(A, i, n)
• Convert an array A[1 … n] into a max-heap (n = length[A])
• The elements in the subarray A[(n/2+1) .. n] are leaves• Apply MAX-HEAPIFY on elements between 1 and n/2
2
14 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
4 1 3 2 16 9 10 14 8 7A:
CS 477/677 - Lecture 9 30
Example: A 4 1 3 2 16 9 10 14 8 7
2
14 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
14
2 8
1
16
7
4
10
9 3
1
2 3
4 5 6 7
8 9 10
2
14 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 1014
2 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
14
2 8
16
7
1
4
10
9 3
1
2 3
4 5 6 7
8 9 108
2 4
14
7
1
16
10
9 3
1
2 3
4 5 6 7
8 9 10
i = 5 i = 4 i = 3
i = 2 i = 1
CS 477/677 - Lecture 9 31
Correctness of BUILD-MAX-HEAP• Loop invariant:
– At the start of each iteration of the for loop, each node i + 1, i + 2,…, n is the root of a max-heap
• Initialization:– i = n/2: Nodes n/2+1, n/2+2, …, n are leaves
they are the root of trivial max-heaps
2
14 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
CS 477/677 - Lecture 9 32
Correctness of BUILD-MAX-HEAP• Maintenance:
– MAX-HEAPIFY makes node i a max-heap root and preserves the property that nodes i + 1, i + 2, …, n are roots of max-heaps
– Decrementing i in the for loop reestablishes the loop invariant
• Termination:– i = 0 each node 1, 2, …, n is the root of
a max-heap (by the loop invariant)
2
14 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
CS 477/677 - Lecture 9 33
Running Time of BUILD MAX HEAP
It would seem that running time is O(nlgn)• This is not an asymptotically tight upper bound
Alg: BUILD-MAX-HEAP(A)1. n = length[A]
2. for i ← n/2 downto 13. do MAX-HEAPIFY(A, i, n) O(lgn)
O(n)
CS 477/677 - Lecture 9 34
Readings• Chapters 8 and 9