View
205
Download
0
Category
Tags:
Preview:
Citation preview
Design and Analysis of Algorithms – Chapter 4 1
Divide and Conquer (I)
Dr. Ying Luylu@cse.unl.edu
CSCE 310: Data Structures & CSCE 310: Data Structures & AlgorithmsAlgorithms
Design and Analysis of Algorithms – Chapter 4 2
Giving credit where credit is due:Giving credit where credit is due:• Most of the lecture notes are based on the slides from Most of the lecture notes are based on the slides from
the Textbook’s companion websitethe Textbook’s companion website– http://www.aw-bc.com/info/levitinhttp://www.aw-bc.com/info/levitin
• Some examples and slides are based on lecture notes Some examples and slides are based on lecture notes created by Dr. Ben Choi, Louisiana Technical University created by Dr. Ben Choi, Louisiana Technical University and Dr. Chuck Cusack, Hope College and Dr. Chuck Cusack, Hope College
• I have modified many of their slides and added new I have modified many of their slides and added new slides.slides.
CSCE 310: Data Structures & CSCE 310: Data Structures & AlgorithmsAlgorithms
Design and Analysis of Algorithms – Chapter 4 3
Divide and ConquerDivide and Conquer
The most well known algorithm design strategy:The most well known algorithm design strategy:
1.1. Divide instance of problem into two or more smaller Divide instance of problem into two or more smaller instancesinstances
1.1. Solve smaller instances recursivelySolve smaller instances recursively
1.1. Obtain solution to original (larger) instance by combining Obtain solution to original (larger) instance by combining these solutionsthese solutions
Design and Analysis of Algorithms – Chapter 4 4
Divide-and-conquer TechniqueDivide-and-conquer Technique
subproblem 2 of size n/2
subproblem 1 of size n/2
a solution to subproblem 1
a solution tothe original problem
a solution to subproblem 2
a problem of size n
Design and Analysis of Algorithms – Chapter 4 5
Divide and Conquer ExamplesDivide and Conquer Examples
Sorting: mergesort and quicksortSorting: mergesort and quicksort
Tree traversalsTree traversals
Binary searchBinary search
Matrix multiplication-Strassen’s algorithmMatrix multiplication-Strassen’s algorithm
Convex hull-QuickHull algorithmConvex hull-QuickHull algorithm
Design and Analysis of Algorithms – Chapter 4 6
MergesortMergesort
Algorithm:Algorithm: Split array A[1..Split array A[1..nn] in two and make copies of each half in ] in two and make copies of each half in
arrays B[1.. arrays B[1.. nn/2 ] and C[1.. /2 ] and C[1.. nn/2 ]/2 ]
Sort arrays B and CSort arrays B and C Merge sorted arrays B and C into array AMerge sorted arrays B and C into array A
Design and Analysis of Algorithms – Chapter 4 7
Using Divide and Conquer: Using Divide and Conquer: MergesortMergesort
Mergesort StrategyMergesort Strategy
Sorted
Merge
Sorted Sorted
Sort recursively by Mergesort
Sort recursively by Mergesort
first last
(first last)2
Design and Analysis of Algorithms – Chapter 4 8
MergesortMergesort
Algorithm:Algorithm: Split array A[1..Split array A[1..nn] in two and make copies of each half in ] in two and make copies of each half in
arrays B[1.. arrays B[1.. nn/2 ] and C[1.. /2 ] and C[1.. nn/2 ]/2 ]
Sort arrays B and CSort arrays B and C Merge sorted arrays B and C into array AMerge sorted arrays B and C into array A
Design and Analysis of Algorithms – Chapter 4 9
MergesortMergesort
Algorithm:Algorithm: Split array A[1..Split array A[1..nn] in two and make copies of each half in ] in two and make copies of each half in
arrays B[1.. arrays B[1.. nn/2 ] and C[1.. /2 ] and C[1.. nn/2 ]/2 ]
Sort arrays B and CSort arrays B and C Merge sorted arrays B and C into array A as follows:Merge sorted arrays B and C into array A as follows:
• Repeat the following until no elements remain in one of the arrays:Repeat the following until no elements remain in one of the arrays:– compare the first elements in the remaining unprocessed portions of compare the first elements in the remaining unprocessed portions of
the arraysthe arrays
– copy the smaller of the two into A, while incrementing the index copy the smaller of the two into A, while incrementing the index indicating the unprocessed portion of that array indicating the unprocessed portion of that array
• Once all elements in one of the arrays are processed, copy the Once all elements in one of the arrays are processed, copy the remaining unprocessed elements from the other array into A.remaining unprocessed elements from the other array into A.
Design and Analysis of Algorithms – Chapter 4 10
Algorithm: MergesortAlgorithm: Mergesort
Input:Input: Array E and indices first and last, such that the Array E and indices first and last, such that the elements E[i] are defined for first <= i <= last.elements E[i] are defined for first <= i <= last.Output:Output: E[first], …, E[last] is a sorted rearrangement of E[first], …, E[last] is a sorted rearrangement of the same elementsthe same elementsvoid mergeSort(Element[] E, int first, int last)void mergeSort(Element[] E, int first, int last)if (first < last)if (first < last)int mid = int mid = (first+last)/2(first+last)/2;;mergeSort(E, first, mid);mergeSort(E, first, mid);mergeSort(E, mid+1, last);mergeSort(E, mid+1, last);merge(E, first, mid, last);merge(E, first, mid, last);
return;return;
Design and Analysis of Algorithms – Chapter 4 11
In-Class ExerciseIn-Class Exercise
P128 Problem 6: Apply mergesort to sort the list E, X, A, P128 Problem 6: Apply mergesort to sort the list E, X, A, M, P, L, E in alphabetical order. M, P, L, E in alphabetical order.
Design and Analysis of Algorithms – Chapter 4 12
Evaluating Sort AlgorithmsEvaluating Sort Algorithms
Run-timeRun-time: The number of basic operations performed (e.g., : The number of basic operations performed (e.g., compare and swap)compare and swap)
MemoryMemory: The amount of memory used beyond what is : The amount of memory used beyond what is needed to store the data being sortedneeded to store the data being sorted• ““In place” algorithms use a constant amount of extra memory—the In place” algorithms use a constant amount of extra memory—the
constant may be zeroconstant may be zero
• Other algorithms are described as linear or exponential with Other algorithms are described as linear or exponential with respect to the space used.respect to the space used.
• Less is better, but there is often a space/time trade-off.Less is better, but there is often a space/time trade-off.
StabilityStability: An algorithm is stable if it preserves the relative : An algorithm is stable if it preserves the relative order of equal keysorder of equal keys
Design and Analysis of Algorithms – Chapter 4 13
Algorithm: MergesortAlgorithm: Mergesort
void mergeSort(Element[] E, int first, int last)void mergeSort(Element[] E, int first, int last)
if (first < last)if (first < last)int mid = int mid = (first+last)/2(first+last)/2;;mergeSort(E, first, mid);mergeSort(E, first, mid);
mergeSort(E, mid+1, last);mergeSort(E, mid+1, last);
merge(E, first, mid, last);merge(E, first, mid, last);
return;return;
Design and Analysis of Algorithms – Chapter 4 14
Mergesort complexityMergesort complexity
Mergesort always partitions the array equally.Mergesort always partitions the array equally. Thus, the recursive depth is always Thus, the recursive depth is always (lg (lg nn) ) The amount of work done at each level is The amount of work done at each level is ((nn) ) Intuitively, the complexity should be Intuitively, the complexity should be ((nn lg lg nn) )
We have, We have, • TT((nn) =2) =2TT((nn/2) /2) ((n) for n>1, T(1)=0 n) for n>1, T(1)=0 ((nn lg lg nn))
The amount of extra memory used is The amount of extra memory used is ((nn)) Note: Mergesort is stableNote: Mergesort is stable
Design and Analysis of Algorithms – Chapter 4 15
Efficiency of mergesortEfficiency of mergesort
Number of comparisons is close to theoretical minimum for Number of comparisons is close to theoretical minimum for comparison-based sorting: comparison-based sorting: • lg lg n n ! ≈ ! ≈ nn lg lg n n - 1.44- 1.44 n n
Space requirement: Space requirement: ΘΘ( ( n n ) () (NOTNOT in-place) in-place)
Can be implemented without recursion (bottom-up)Can be implemented without recursion (bottom-up)
All cases have same efficiency: All cases have same efficiency: ΘΘ( ( n n log log nn) )
Design and Analysis of Algorithms – Chapter 4 16
AnimationAnimation
http://math.hws.edu/TMCM/java/xSortLab/index.html
Design and Analysis of Algorithms – Chapter 4 17
The master theoremThe master theorem
Given: a Given: a divide and conquerdivide and conquer algorithm algorithm
Then, the Master Theorem gives us a cookbook for the Then, the Master Theorem gives us a cookbook for the algorithm’s running time:algorithm’s running time:
Design and Analysis of Algorithms – Chapter 4 18
A general divide-and-conquer A general divide-and-conquer recurrencerecurrence
TT((nn) = ) = aTaT((n/bn/b) + ) + f f ((nn)) where where f f ((nn)) ∈∈ ΘΘ((nndd))
T(1) = cT(1) = c
a < ba < bdd T T((nn) ) ∈∈ ΘΘ((nndd)) a = ba = bdd T T((nn) ) ∈∈ ΘΘ((nnd d lg lg n n )) a > ba > bdd T T((nn) ) ∈∈ ΘΘ((nnlog log b b aa))
Please refer to Appendix B (P483) for the proofPlease refer to Appendix B (P483) for the proof
Design and Analysis of Algorithms – Chapter 4 19
In-class exerciseIn-class exercise
Page 128: Problem 5Page 128: Problem 5 5. Find the order of growth for solutions of the following 5. Find the order of growth for solutions of the following
recurrences.recurrences.• a. T(n) = 4T(n/2) + n, T(1) = 1a. T(n) = 4T(n/2) + n, T(1) = 1
• b. T(n) = 4T(n/2) + nb. T(n) = 4T(n/2) + n2222, T(1) = 1, T(1) = 1
• c. T(n) = 4T(n/2) + nc. T(n) = 4T(n/2) + n3333, T(1) = 1, T(1) = 1
In-Class ExerciseIn-Class Exercise
4.1.9 Let A[0, n-1] be an array of n distinct real 4.1.9 Let A[0, n-1] be an array of n distinct real numbers. A pair (A[i], A[j]) is said to be an numbers. A pair (A[i], A[j]) is said to be an inversion if these numbers are out of order, i.e., i<j inversion if these numbers are out of order, i.e., i<j but A[i]>A[j]. Design an O(nlogn) algorithm for but A[i]>A[j]. Design an O(nlogn) algorithm for counting the number of inversions. counting the number of inversions.
Design and Analysis of Algorithms – Chapter 4 20
Design and Analysis of Algorithms – Chapter 4 21
Quicksort by Hoare (1962)Quicksort by Hoare (1962)
Select a Select a pivotpivot (partitioning element) (partitioning element) Rearrange the list so that all the elements in the positions Rearrange the list so that all the elements in the positions
before the pivot are smaller than or equal to the pivot and before the pivot are smaller than or equal to the pivot and those after the pivot are larger than or equal to the pivot those after the pivot are larger than or equal to the pivot
Exchange the pivot with the last element in the first (i.e., ≤) Exchange the pivot with the last element in the first (i.e., ≤) sublist – the pivot is now in its final positionsublist – the pivot is now in its final position
Sort the two sublists recursivelySort the two sublists recursively
p
A[i]≤p A[i]p
Design and Analysis of Algorithms – Chapter 4 22
Quicksort by Hoare (1962)Quicksort by Hoare (1962)
Select a Select a pivotpivot (partitioning element) (partitioning element) Rearrange the list so that all the elements in the positions Rearrange the list so that all the elements in the positions
before the pivot are smaller than or equal to the pivot and before the pivot are smaller than or equal to the pivot and those after the pivot are larger than or equal to the pivot those after the pivot are larger than or equal to the pivot
Exchange the pivot with the last element in the first (i.e., ≤) Exchange the pivot with the last element in the first (i.e., ≤) sublist – the pivot is now in its final positionsublist – the pivot is now in its final position
Sort the two sublists recursivelySort the two sublists recursively
p
A[i]≤p A[i]p
Design and Analysis of Algorithms – Chapter 4 23
The partition algorithmThe partition algorithm
Design and Analysis of Algorithms – Chapter 4 24
The partition algorithmThe partition algorithm
A “sentinel” at A[n] to prevent i advances beyond position n. A “sentinel” at A[n] to prevent i advances beyond position n.
Design and Analysis of Algorithms – Chapter 4 25
Quicksort ExampleQuicksort Example
Recursive implementation with the left most array Recursive implementation with the left most array entry selected as the pivot element.entry selected as the pivot element.
Design and Analysis of Algorithms – Chapter 4 26
QuicksortQuicksort
Animated Example: Animated Example: http://math.hws.edu/TMCM/java/xSortLab/index.html
Design and Analysis of Algorithms – Chapter 4 27
Quicksort AlgorithmQuicksort Algorithm
Input: Array E and indices first, and last, s.t. elements E[i] Input: Array E and indices first, and last, s.t. elements E[i] are defined for first are defined for first i i last last
Ouput: E[first], …, E[last] is a sorted rearrangement of the Ouput: E[first], …, E[last] is a sorted rearrangement of the arrayarray
Void quickSort(Element[] E, int first, int last)Void quickSort(Element[] E, int first, int last)if (first < last)if (first < last)Element pivotElement = E[first];Element pivotElement = E[first];int splitPoint = partition(E, pivotElement, first, last);int splitPoint = partition(E, pivotElement, first, last);quickSort (E, first, splitPoint –1 );quickSort (E, first, splitPoint –1 );quickSort (E, splitPoint +1, last );quickSort (E, splitPoint +1, last );return;return;
Design and Analysis of Algorithms – Chapter 4 28
Quicksort AnalysisQuicksort Analysis
Partition can be done in O(Partition can be done in O(nn) time, where ) time, where nn is is the size of the arraythe size of the array
Let Let TT((nn) be the number of compares required ) be the number of compares required by Quicksortby Quicksort
If the pivot ends up at position If the pivot ends up at position kk, then we have, then we have• TT((nn) ) TT((nnkk) ) TT((kk 1) 1) nn
To determine best-, worst-, and average-case To determine best-, worst-, and average-case complexity we need to determine the values of complexity we need to determine the values of kk that correspond to these cases.that correspond to these cases.
Design and Analysis of Algorithms – Chapter 4 29
Best-Case ComplexityBest-Case Complexity
The best case is clearly when the pivot always The best case is clearly when the pivot always partitions the array equally.partitions the array equally.
Intuitively, this would lead to a recursive depth of Intuitively, this would lead to a recursive depth of at most lg at most lg nn calls calls
We can actually prove this. In this case We can actually prove this. In this case • TT((nn) ) TT((nn/2) /2) TT((nn/2) /2) n n ((nn lg lg nn))
Design and Analysis of Algorithms – Chapter 4 30
Worst-Case and Average-Case Worst-Case and Average-Case ComplexityComplexity
The worst-case is when the pivot always ends up in The worst-case is when the pivot always ends up in the first or last element. That is, partitions the the first or last element. That is, partitions the array as unequally as possible.array as unequally as possible.
In this case In this case • TT((nn) ) TT((nn11) ) TT(1(11) 1) n n TT((nn11) ) nn
nn ( (nn11) ) … + 1 … + 1
nn((nn 1)/2 1)/2 ((nn22))
Average case is rather complex, but is where the Average case is rather complex, but is where the algorithm earns its name. The bottom line is:algorithm earns its name. The bottom line is:
)lg(lg386.1)( nnnnnA
Design and Analysis of Algorithms – Chapter 4 31
Summary of quicksortSummary of quicksort
Best caseBest case: split in the middle — : split in the middle — ΘΘ( ( n n log log nn) ) Worst caseWorst case: sorted array! — : sorted array! — ΘΘ( ( nn22) ) Average caseAverage case: random arrays — : random arrays — ΘΘ( ( n n log log nn))
Improvements:Improvements:• better pivot selection: median of three partitioning avoids worst better pivot selection: median of three partitioning avoids worst
case in sorted filescase in sorted files• switch to insertion sort on small subfilesswitch to insertion sort on small subfiles• elimination of recursionelimination of recursion
these combine to 20-25% improvementthese combine to 20-25% improvement
Considered the method of choice for internal sorting for Considered the method of choice for internal sorting for large files (large files (nn ≥ 10000) ≥ 10000)
In-class exercisesIn-class exercises
4.2.1 Apply quicksort to sort the list E, X, A, 4.2.1 Apply quicksort to sort the list E, X, A, M, P, L, E in alphabetical order. M, P, L, E in alphabetical order.
Apply quicksort to sort the list 7 2 9 10 5 4Apply quicksort to sort the list 7 2 9 10 5 4
4.2.8 Design an algorithm to rearrange 4.2.8 Design an algorithm to rearrange elements of a given array of n real numbers elements of a given array of n real numbers so that all its negative elements proceed all so that all its negative elements proceed all its positive elements. Your algorithm should its positive elements. Your algorithm should be both time- and space-efficient. be both time- and space-efficient.
Design and Analysis of Algorithms – Chapter 4 32
Recommended