CPSC 311 Section 502 Analysis of Algorithm Fall 2002 Department of Computer Science Texas A&M University

CPSC 311 Section 502Analysis of Algorithm

Fall 2002Department of Computer Science

Texas A&M University

CPSC 311 O. Burchan Bayazit – Fall 02

Announcements

•There will be a quiz on next Tuesday (Sep 17)

•A new homework will be put to the web site today ( due date next Thursday (Sep 19))

•Grades for the previous quiz is available


Solving Recurrences: Master MethodThe master method provides a 'cookbook' method for solving recurrences of a certain form.

Master Theorem: Let a 1 and b > 1 be constants, let f(n) be a function, and let T(n) be defined on nonnegative integers as:

T(n) = aT(n/b) + f(n)

Then, T(n) can be bounded asymptotically as follows:

1. T(n) = (nlogba) if f(n) = O(nlogba-) for some constant > 0

2. T(n) = (nlogbalogn) if f(n) = (nlogba)

3. T(n) = (f(n)) if f(n) = (nlogba+) for some constant > 0and if af(n/b) cf(n) for some constant c < 1 and all sufficiently large n.


Case 3 T(n) = (f(n)), if f(n) = (nlogba+) for some constant > 0 and if

af(n/b) cf(n) for some constant c < 1 and all sufficiently large n.

T(n) = aT(n/b) + f(n) T(n) =3T(n/4)+nlgn

a=3,b=4,f(n)=nlgn{PART I: Is f(n) = (nlogba+) for some >0?

nlogba+ = nlog43+=n0.79+

f(n)=nlgn can we find > 0 such that nlgn= (n0.79+ )

If we choose =1-0.79 the lower bound becomes (n )And nlgn has a lower bound n i.e., f(nlgn)= (n )So the first part holds.


Case 3 T(n) = (f(n)), if f(n) = (nlogba+) for some constant > 0 and if

af(n/b) cf(n) for some constant c < 1 and all sufficiently large n.

T(n) = aT(n/b) + f(n) T(n) =3T(n/4)+nlgn

a=3,b=4,f(n)=nlgn{Part II: is there a constant <1 that satisfies af(n/b) cf(n) ?

af(n/b) = 3f(n/4) So we should find a c such3f(n/4)<=c. f(n) and c>13f(n/4)= 3(n/4)lg(n/4) and f(n)=nlgn (3/4)n lg(n)-(3/4)lg4 <=c.nlgn

If we select c=3/4 this inequality holds so T(n)= (nlgn)


Sorting Algorithms Ch. 6 - 8

Slightly modified definition of the sorting problem:input: A collection of n data items <a1,a2,...,an> where each data

item has a key drawn from a linearly ordered set (e.g., ints, chars)

output: A permutation (reordering) <a'1,a'2,...,a'n> of the input sequence such that a'1 a'2 ... a'n

• In practice, one usually sorts 'records' according to their key (the non-key data is called satellite data.)

• If the records are large, we may sort an array of pointers.


Sorting Algorithms• A sorting algorithm is in place if only a constant number of

elements of the input array are ever stored outside the array.

• A sorting algorithm is comparison based if the only operation we can perform on keys is to compare two keys.


Merge Sort

A Divide-and-Conquer algorithm

•Divide: the n-element array to be sorted into two n/2-element subarrays

•Conquer: Sort the subarrays recursively using merge sort

•Combine: Merge the two sorted subarrays to find sorted array


Merge Sort

5 2 4 6 1 3 8 7

Unsorted original array


Merge SortDivide the problem to two subproblems

5 2 4 6 1 3 8 7

Call merge sort on subarrays

5 2 4 6 1 3 8 7


Merge SortFurther divide subarrays

5 2 4 6 1 3 8 7

5 2 4 6 1 3 8 7


Merge SortUntil the smallest array is reached

5 2 4 6 1 3 8 7

5 2 4 6 1 3 8 7


Merge SortMerge results of two subarrays

2 5 4 6 1 3 7 8

5 2 4 6 1 3 8 7

5 2 4 6 1 3 8 7


Merge SortFurther merge

2 5 4 6 1 3 7 8

5 2 4 6 1 3 8 7



2 4 5 6 1 3 7 8

2 5 4 6 1 3 7 8

5 2 4 6 1 3 8 7



2 4 5 6 1 3 7 8

5 2 4 6 1 3 8 7


Merge Sort

1 2 3 4 5 6 7 8

Final merge

2 4 5 6 1 3 7 8

5 2 4 6 1 3 8 7


Merge Sort

1 2 3 4 5 6 7 8

Result

5 2 4 6 1 3 8 7


Merge Sort - Algorithm

Merge-Sort(A,start,end)

1. if start=end then {

2. return

3. }

4. half = (start+end)/2 5. Merge-Sort (A,start,half)

6. Merge-Sort (A,half+1,end)

7. Merge (A,start,half,end)


Merging

2 4 5 6 1 3 7 8


Merging

2 4 5 6 3 7 8

1


Merging

4 5 6 7 8

1 2

3


Merging

4 5 6 7 8

1 2 3


Merging

5 6 7 8

1 2 3 4


Merging

6 7 8

1 2 3 4 5


Merging

7 8

1 2 3 4 5 6


Merging

8

1 2 3 4 5 6 7


Merging

1 2 3 4 5 6 7 8


Merge AlgorithmMerge(A,start,half,end)

1. n1= half-start+1; n2 end-half;

2. for i= 1 to n1 {

3. Left[i]= A[start+i-1] // copy left subarray

4. }

5. for i= 1 to n2 {

6. R[i] A[half+i] // copy right subarray7. }

8. Left[n1+1] = Right[n2+1] =

9. i= j= 1

10. for k = start to end {

11. if Left[i] Right[j] then {12. A[k]=Left[i]

13. i= i+1 }

14. else { A[k]= Right[j]

15. j= j+1 }

16. }

Running Time = (n)


Merge Sort - Analysis

Merge-Sort(A,start,end)

1. if start=end then {

2. return

3. }

4. half = (start+end)/2 5. Merge-Sort (A,start,half)

6. Merge-Sort (A,half+1,end)

7. Merge (A,start,half,end)

(1)

T (n/2)T (n/2)

(n)

T (n)=2T(n/2)+ (n) (nlgn)using master method


Heap Sort

heap concept• complete binary tree, except may be missing some rightmost

leaves on the bottom level.• each node contains a key• values in the nodes satisfy the heap property.


Heap vs. Array

1

2 3

4

8 9

765

10

1 2 3 4 5 6 7 8 9 10

binary tree: an array implementation• root is A[1]• for element A[i]

- left child is in position A[2i]- right child is in position A[2i + 1]- parent is in A[i/2]

Left child=2i

i=2

Right child=2i+1

Parent =i/2


Max-Heap

Max-heap property: A[Parent(i)] A[i]

In the array representation of a max-heap, the root of the treeis in A[1], and given the index i of a node, Parent(i) LeftChild(i) RightChild(i) return (i/2) return (2i) return (2i + 1)

20 18 15 11 10 9 6 2 4 5 3

1 2 3 4 5 6 7 8 9 10 11 A index

keys

20

18 15

11 10 9 6

2 4 5 3

1

2 3

4

8 9

765

1110

n = 11height = 3(# edges on longest leaf toroot path)


Min-Heap

Min-heap property: A[Parent(i)] A[i]

Min-heaps are commonly used for priority queues in event-driven simulators.Parent(i) LeftChild(i) RightChild(i) return (i/2) return (2i) return (2i + 1)

2 3 5 4 6 9 11 10 15 20 18

1 2 3 4 5 6 7 8 9 10 11 A index

keys

2

3 5

4 6 9 11

10 15 20 18

1

2 3

4

8 9

765

1110

n = 11height = 3(# edges on longest leaf toroot path)


Heap Sort – How it works

18

11 10

2 4 5

Assume we have a max-heap

12 3

4 56

18 11 10 2 4 5



18

11 10

2 4 5

The largest number is the root (max-heap property)



18

11 10

2 4 5

Take the root away



18

11 10

2 4

5

Put the last one to the root



18

11 10

2 4

5

Fix the max-heap



18

5 10

2 4

11

Fix the max-heap



18

5 10

2 4

11

Take the root away



18

5 10

2

4

11




18

5 4

2

10

11

Fix the max-heap



18

5 4

2

10 11

Take the root away



18

5 4

2

10 11




18

2 4

5

10 11

Fix the max-heap



18

2 4

5 10 11

Take the root away



18

2

4

5 10 11

Put the last one to the top



18

2

4 5 10 11

Take the top away



182 4 5 10 11

Put the last one


Heap SortBasic Functions• HEAPIFY: maintain (fix) max-heap property • BUILD-HEAP: Create a max-heap from unsorted array

(requires HEAPIFY)• HEAPSORT: sorts an array (requires BUILD-HEAP and

HEAPIFY)How it works?

•Build a max-heap (the root is the largest number)

•Take the largest number away, put the last number in the heap to the root position and if max-heap property is not satisfied, fix it.

•Repeat until all the elements are processed.


Heapify (fix max-heap)

Assume one node i does not satisfy MAX-HEAP property but its children are.

Heapify function corrects this by recursively fixing inconsistencies.

20

2 15

10 11 9 6

2 8 5 3

Not satisfy max-heap




20

11 15

10 2 9 6

2 8 5 3

Switch with the largest child





20

11 15

10 2 9 6

2 8 5 3Not satisfy max-heap





20

11 15

10 5 9 6

2 8 2 3Switch with the largest child




•Find the largest child

•Switch the node i with its largest child

•Recursively HEAPIFY for the subtree where node i placed


Heapify(A,i,n)/* A: Array, I: node to be fixed, n:heap-size */1. left 2i; right 2i + 1 /*indices of left & right children of A[i] */2. largest i;3. if left n and A[left] > A[i] then {4. largest left 5. }5. if right n and A[right] > A[largest] then {6. largest right 7. }7. if largest i then8. swap(A[i], A[largest])9. Heapify(A, largest,n)

T (n)=T(2n/3)+ (1) O (lgn)using master method



Build-Heap

Given unsorted array

Starting from largest non-leaf node down to the first node, call Heapify for each node

1

2 3

4 56


Build-Heap

8

3 2

4 6 8 5

10 9 18 7

8 3 2 4 6 8 5 10 9 18 7

Build-Max-Heap(A,n)1. for i n/2 downto 1 { 2. Heapify(A, i,n)3. }


Build-Heap

8

3 2

4 6 8 5

10 9 18 7



Build-Heap

8

3 2

4 18 8 5

10 9 6 7



Build-Heap

8

3 2

4 18 8 5

10 9 6 7



Build-Heap

8

3 2

10 18 8 5

4 9 6 7



Build-Heap

8

3 2

10 18 8 5

4 9 6 7



Build-Heap

8

3 8

10 18 2 5

4 9 6 7



Build-Heap

8

3 8

10 18 2 5

4 9 6 7



Build-Heap

8

18 8

10 3 2 5

4 9 6 7



Build-Heap

8

18 8

10 7 2 5

4 9 6 3



Build-Heap

8

18 8

10 7 2 5

4 9 6 3



Build-Heap

18

8 8

10 7 2 5

4 9 6 3



Build-Heap

18

10 8

8 7 2 5

4 9 6 3



Build-Heap

18

10 8

9 7 2 5

4 8 6 3



Build-Heap


O(lgn) repeated (n/2 times)

T(n)=O(nlgn)

Can we do better?

Yes T(n)=O(n)


Heap Sort Algorithm

HeapSort(A,n)1. Build-Heap(A,n) /* put all elements in heap */2. for i = n downto 2 {3. swap A[1] A[i] /* puts max in ith array position */4. Heapify(A,1,i-1) /* restore heap property */5. }

Input: An n-element array A (unsorted).Output: An n-element array A in sorted order, smallest to largest.

Running time of HeapSort• 1 call to Build-Heap() O(n) time• n-1 calls to Heapify() each takes O(lgn) time O(nlgn) time



18

11 10

2 4 5

Assume we have a max-heap

12 3

4 56

18 11 10 2 4 5



18

11 10

2 4 5

The largest number is the root (max-heap property)

18 11 10 2 4 5



18

11 10

2 4 5

Take the root away

18 11 10 2 4 5



18

11 10

2 4

5


18 11 10 2 4 5



18

11 10

2 4

5

Fix the max-heap

18 11 10 2 4 5



18

5 10

2 4

11

Fix the max-heap

18 11 10 2 4 5



18

5 10

2 4

11

Take the root away

18 11 10 2 4 5



18

5 10

2

4

11


18 11 10 2 4 5



18

5 4

2

10

11

Fix the max-heap

18 11 10 2 4 5



18

5 4

2

10 11

Take the root away

18 11 10 2 4 5



18

5 4

2

10 11


18 11 10 2 4 5



18

2 4

5

10 11

Fix the max-heap

18 11 10 2 4 5



18

2 4

5 10 11

Take the root away

18 11 10 2 4 5



18

2

4

5 10 11

Put the last one to the top

18 11 10 2 4 5



18

2

4 5 10 11

Take the top away

18 11 10 2 4 5



182 4 5 10 11

Put the last one

18 11 10 2 4 5


Priority Queues: Application for Heaps

An application of max-priority queues is to schedule jobs on a shared processor. Need to be able to get the highest priority job Heap-Maximum(A,n) remove job from the queue Heap-Extract-Max(A,n) insert new jobs into queue Max-Heap-Insert(A, key,n) increase priority of jobs Heap-Increase-Key(A,i,key,n)


Heap-Extract-Max

Heap-Extract-Max(A,n)1. max=A[1]2. A[1]=A[n]3. n=n-1 /* puts max in ith array position */4. Heapify(A,1,n) /* restore heap property */5. return max

T(n)=O(lgn)


Heap-Increase-Key

Heap-Increase-Key(A,i,key,n)1. A[i]=key2. while I>1 and A[Parent[i]]<A[i] {3. swap A[Parent[i]] A[i] 4. i=Parent[i]

T(n)=O(lgn)


Max-Heap-Insert

Max-Heap-Insert(A,i,key,n)1. n=n+1

2. A[i]=- 3. Heap-Increase-Key(A,n,key,n)

T(n)=O(lgn)

Documents

CPSC 311 Section 502 Analysis of Algorithm Fall 2002 Department of Computer Science Texas A&M University