Data Structures and Algorithms - Pradžiaalgis/dsax/Data-structures-1.pdf · Data Structures and Algorithms Course’s slides: Sorting Algorithms algis . ... • The resulting formula

Data Structures and

Algorithms

Course’s slides: Sorting Algorithms

www.mif.vu.lt/~algis

Sorting

ò  Card players all know how to sort …

ò  First card is already sorted

ò  With all the rest ❶  Scan back from the end until you find the first card larger

than the new one,

➋  Move all the lower ones up one slot

❸  insert it

♣%

Q

♦%

2

♦%

9 ♠%

A

♠%

K

♠%

10

♥%

J

♥%

2 ♠%

2

♦%

9 ❸

Sorting terminology and notation

ò  Records are compared to one another by means of a comparator class

ò  The key method of the comparator class is prior, which returns true when its first argument should appear prior to its second argument in the sorted list

ò  For every record type there is a swap function that can interchange the contents of two records in the array

ò  Given a set of records r1, r2, ..., rn with key values k1, k2, ..., kn, the Sorting Problem is to arrange the records into any order s such that records rs1 , rs2 , ..., rsn have keys obeying the property ks1 ≤ ks2 ≤ ... ≤ ksn


Time analysis: some algorithms are much more efficient than others.

For sorting algorithms, we’ll focus on two types of operations: comparisons and moves (swaps)

To express the time complexity of an algorithm, we’ll express the number of operations performed as a function of n

• C(n) = number of comparisons, M(n) = number of moves

Examples: C(n) = n2 + 3 n, M(n) = 2 n2 - 1


When n is large, expressions of n are dominated by their “largest” term – the term that grows fastest as a function of n.

In characterizing the time complexity of an algorithm, we’ll focus on the largest term in its operation-count expression.

• for selection sort, C(n) = n2/2 - n/2 ≈ n2/2

In addition, we’ll typically ignore the coefficient of the largest term (e.g., n2/2 à n2).


Mathematical definition of Big-O notation: f (n) = O (g (n) ) if there exist positive constants c and n0 , such that f (n) <= c g (n) for all n >= n0 example: f (n) = n2/2 – n/2 is O (n2), because n2/2–n/2 <= n2 for all n >= 0.

Mathematical Definition of Big-O Notation• f(n) = O(g(n)) if there exist positive constants c and n0

such that f(n) <= cg(n) for all n >= n0

• Example: f(n) = n2/2 – n/2 is O(n2), becausen2/2 – n/2 <= n2 for all n >= 0.

• Big-O notation specifies an upper bound on a function f(n) as n grows large.

n

f(n) = n2/2 – n/2

g(n) = n2

c = 1 n0 = 0

Big-O Notation and Tight Bounds• Big-O notation provides an upper bound, not a tight bound

(upper and lower).

• Example: • 3n – 3 is O(n2) because 3n – 3 <= n2 for all n >= 1• 3n – 3 is also O(2n) because 3n – 3 <= 2n for all n >= 1

• However, we generally try to use big-O notation to characterize a function as closely as possible – i.e., as if we were using it to specify a tight bound.• for our example, we would say that 3n – 3 is O(n)


Common classes of algorithms:

Name example expressions big-O notation

constant time 1,7,10 O(1)

logarithmic time 3 log10n, log2n + 5 O (log n)

linear time 5 n,10 n – 2 log2n O (n)

n logn time 4 n log2n, n log2n + n O (n log n)

quadratic time 2 n2 + 3 n, n2 – 1 O (n2)

exponential time 2n, 5 en + 2n2 O (c n)

For large inputs, efficiency matters more than CPU speed. e.g., an O (log n) algorithm on a slow machine will outperform an O(n) algorithm on a fast machine


big-theta notation (Θ) is used to specify a tight bound:

f (n) = Θ (g (n) ) if there exist constants c1, c2, and n0 such that c1 g (n) <= f (n) <= c2 g (n) for all n >n0

Ex: f (n) = n2/2 – n/2 is Θ(n2), because (1/4)*n2 <= n2/2 – n/2 <= n2 for all n >= 2

Big-Theta Notation• In theoretical computer science, big-theta notation (4) is used to

specify a tight bound.

• f(n) = 4(g(n)) if there exist constants c1, c2, and n0 such thatc1g(n) <= f(n) <= c2 g(n) for all n > n0

• Example: f(n) = n2/2 – n/2 is 4(n2), because(1/4)*n2 <= n2/2 – n/2 <= n2 for all n >= 2

n(1/4) * g(n) = n2/4

f(n) = n2/2 – n/2

g(n) = n2

c1 = 1/4 n0 = 2c2 = 1

Big-O Time Analysis of Selection Sort• Comparisons: we showed that C(n) = n2/2 – n/2

• selection sort performs O(n2) comparisons

• Moves: after each of the n-1 passes to find the smallest remaining element, the algorithm performs a swapto put the element in place.• n–1 swaps, 3 moves per swap• M(n) = 3(n-1) = 3n-3

• selection sort performs O(n) moves.

• Running time (i.e., total operations): ?

Sorting: Selection Sort

Selection sorting works according to the prescript:

ò  first find the smallest element in the array and exchange it with the element in the first position,

ò  then find the second smallest element and exchange it with the element in the second position, etc.

Sorting: Selection Sort 242 Chap. 7 Internal Sorting

i=0 1 2 3 4 5 64220171328142315

1320174228142315

1314174228202315

1314154228202317

1314151728202342

1314151720282342

1314151720232842

1314151720232842

Figure 7.3 An example of Selection Sort. Each column shows the array after theiteration with the indicated value of i in the outer for loop. Numbers above theline in each column have been sorted and are in their final positions.

Key = 42

Key = 5

Key = 42

Key = 5

(a) (b)

Key = 23

Key = 10

Key = 23

Key = 10

Figure 7.4 An example of swapping pointers to records. (a) A series of fourrecords. The record with key value 42 comes before the record with key value 5.(b) The four records after the top two pointers have been swapped. Now the recordwith key value 5 comes before the record with key value 42.

remember the position of the element to be selected and do one swap at the end.Thus, the number of comparisons is still ⇥(n2), but the number of swaps is muchless than that required by bubble sort. Selection Sort is particularly advantageouswhen the cost to do a swap is high, for example, when the elements are long stringsor other large records. Selection Sort is more efficient than Bubble Sort (by aconstant factor) in most other situations as well.

There is another approach to keeping the cost of swapping records low thatcan be used by any sorting algorithm even when the records are large. This isto have each element of the array store a pointer to a record rather than store therecord itself. In this implementation, a swap operation need only exchange thepointer values; the records themselves do not move. This technique is illustratedby Figure 7.4. Additional space is needed to store the pointers, but the return is afaster swap operation.


The following program is a full implementation of this process:

procedure selection; var i, j, min, t: integer; begin

for i := 1 to N -1 do

begin min := i ;

for j := i+1 to N do

if a [j] < a[min] then min := j ; t := a [min]; a [min] := a [i] := t

end; end;

Complexity: O (n2) - uses about N2/2 comparisons and N exchanges


To sort n elements, selection sort performs n - 1 passes:

on 1st pass, it performs n - 1 comparisons to find indexSmallest,

on 2nd pass, it performs n - 2 comparisons …...

on the (n-1)st pass, it performs 1 comparison

adding up the comparisons for each pass, we get:

C(n) = 1 + 2 + ... + (n - 2) + (n – 1)

Counting Comparisons by Selection Sortprivate static int indexSmallest(int[] arr, int lower,int upper){

int indexMin = lower;

for (int i = lower+1; i <= upper; i++)if (arr[i] < arr[indexMin])

indexMin = i;

return indexMin;}public static void selectionSort(int[] arr) {

for (int i = 0; i < arr.length-1; i++) {int j = indexSmallest(arr, i, arr.length-1);swap(arr, i, j);

}}

• To sort n elements, selection sort performs n - 1 passes:on 1st pass, it performs n - 1 comparisons to find indexSmalleston 2nd pass, it performs n - 2 comparisons

on the (n-1)st pass, it performs 1 comparison

• Adding up the comparisons for each pass, we get:C(n) = 1 + 2 + … + (n - 2) + (n - 1)

Counting Comparisons by Selection Sort (cont.)• The resulting formula for C(n) is the sum of an arithmetic

sequence:

C(n) = 1 + 2 + … + (n - 2) + (n - 1) =

• Formula for the sum of this type of arithmetic sequence:

• Thus, we can simplify our expression for C(n) as follows:

C(n) =

=

=

¦

1 - n

1 i

i

2

1)m(m i

m

1 i

� ¦

¦

1 - n

1 i

i

2

1)1)-1)((n-(n

�

2

1)n-(n 2n- 2n2C(n) =


Comparisons: C(n) = n2/2 – n/2, so selection sort performs O (n2) comparisons

Moves: after each of the n-1 passes to find the smallest remaining element, the algorithm performs a swap to put the element in place: n–1 swaps, 3 moves per swap:

M (n) = 3 (n-1) = 3n – 3,

selection sort performs O (n) moves.

Sorting – Insertion sort

modularity, and error handling are often ignored in order to convey the essence of the algorithm more concisely.

We start with insertion sort, which is an efficient algorithm for sorting a small number of elements. Insertion sort works the way many people sort a hand of playing cards. We start with an empty left hand and the cards face down on the table. We then remove one card at a time from the table and insert it into the correct position in the left hand. To find the correct position for a card, we compare it with each of the cards already in the hand, from right to left, as illustrated in Figure 2.1. At all times, the cards held in the left hand are sorted, and these cards were originally the top cards of the pile on the table.

Figure 2.1: Sorting a hand of cards using insertion sort.

Our pseudocode for insertion sort is presented as a procedure called INSERTION-SORT, which takes as a parameter an array A[1 � n] containing a sequence of length n that is to be sorted. (In the code, the number n of elements in A is denoted by length[A].) The input numbers are sorted in place: the numbers are rearranged within the array A, with at most a constant number of them stored outside the array at any time. The input array A contains the sorted output sequence when INSERTION-SORT is finished.

INSERTION-SORT(A) 1 for j ĸ 2 to length[A] 2 do key ĸ A[j] .Insert A[j] into the sorted sequence A[1 � j - 1] ڂ 34 i ĸ j - 1 5 while i > 0 and A[i] > key 6 do A[i + 1] ĸ A[i] 7 i ĸ i - 1 8 A[i + 1] ĸ key

Loop invariants and the correctness of insertion sort

Figure 2.2 shows how this algorithm works for A = �5, 2, 4, 6, 1, 3�. The index j indicates the "current card" being inserted into the hand. At the beginning of each iteration of the "outer" for loop, which is indexed by j, the subarray consisting of elements A[1 � j - 1] constitute the currently sorted hand, and elements A[j + 1 � n] correspond to the pile of cards still on the table. In fact, elements A[1 � j - 1] are the elements originally in positions 1 through j - 1, but now in sorted order. We state these properties of A[1 � j -1] formally as a loop invariant:


ò  Insertion sort:

ò  Input: A sequence of n numbers a1, a2, . . .,an.

ò  Output: A permutation (reordering) of the input sequence such that the numbers that we wish to sort are also known as the keys.

ò  Insertion sort is an efficient algorithm for sorting a small number of elements. Insertion sort works the way many people sort a hand of playing cards. We start with an empty left hand and the cards face down on the table. We then remove one card at a time from the table and insert it into the correct position in the left hand.

Sorting – Insertion sort 238 Chap. 7 Internal Sorting

i=1 3 4 5 64220171328142315

2042171328142315

21720421328142315

13172042281423

13172028421423

13141720284223

13141720232842

1314151720232842

7

15 15 1515

Figure 7.1 An illustration of Insertion Sort. Each column shows the array afterthe iteration with the indicated value of i in the outer for loop. Values abovethe line in each column have been sorted. Arrows indicate the upward motions ofrecords through the array.

7.2.1 Insertion Sort

Imagine that you have a stack of phone bills from the past two years and that youwish to organize them by date. A fairly natural way to do this might be to look atthe first two bills and put them in order. Then take the third bill and put it into theright order with respect to the first two, and so on. As you take each bill, you wouldadd it to the sorted pile that you have already made. This naturally intuitive processis the inspiration for our first sorting algorithm, called Insertion Sort. InsertionSort iterates through a list of records. Each record is inserted in turn at the correctposition within a sorted list composed of those records already processed. Thefollowing is a Java implementation. The input is an array of n records stored inarray A.

static <E extends Comparable<? super E>> void Sort(E[] A) {for (int i=1; i<A.length; i++) // Insert i’th record

for (int j=i; (j>0) && (A[j].compareTo(A[j-1])<0); j--)DSutil.swap(A, j, j-1);

}

Consider the case where inssort is processing the ith record, which has keyvalue X. The record is moved upward in the array as long as X is less than thekey value immediately above it. As soon as a key value less than or equal to X isencountered, inssort is done with that record because all records above it in thearray must have smaller keys. Figure 7.1 illustrates how Insertion Sort works.

The body of inssort is made up of two nested for loops. The outer forloop is executed n � 1 times. The inner for loop is harder to analyze becausethe number of times it executes depends on how many keys in positions 1 to i � 1have a value less than that of the key in position i. In the worst case, each record


Insertion sort:

procedure insertion;

var i,j,v:integer;

begin for i:=2 to N do

begin v:=a[i]; j:=i; while a[j-1]>v do begin a[j]:=a[j-1]; j:=j-1 end;

a[j]:=v end

Insertion sort

Complexity:

For each card: Scan - O(n), Shift up - O(n), Insert - O(1), Total - O(n)

First card requires O(1), second O(2), … For n card - O(n2)

ò  selection sort uses about N2/2 comparisons and N exchanges;

ò  insertion sort uses N2/4 comparisons and N2/8 exchanges on the average, twice as many in the worst case;

ò  bubble sort uses N2/2 comparisons and N2/2 exchanges on the average and in the worst case

Sorting – Shellsort

Shellsort or Shell's method, is an in-place comparison sort, it is a generalization of insertion sort. The method starts by sorting elements far apart from each other and progressively reducing the gap between them.

The list of elements considering every hth element gives a sorted list. Such a list is said to be h-sorted.

If the file is then k-sorted for some smaller integer k, then the file remains h-sorted. Following this idea for a decreasing sequence of h values ending in 1 is guaranteed to leave a sorted list in the end.


ò  An example run of Shellsort with gaps 7, 3 and 1:

3 7 9 0 5 1 6 8 4 2 0 6 1 5 7 3 4 9 8 2

3 7 9 0 5 1 6 3 3 2 0 5 1 5

8 4 2 0 6 1 5 -> 7 4 4 0 6 1 6

7 3 4 9 8 2 8 7 9 9 8 2

The new data: 3 3 2 0 5 1 5 7 4 4 0 6 1 6 8 7 9 9 8 2

are devided into columns by 3 elements


Different sequences of decreasing increments can be used.

The version uses values: 2k –1 for some k: 63, 31, 15, 7, 3, 1

get to the next lower increment using integer division: incr = incr/2

Should avoid numbers that are multiples of each other. Otherwise, elements that are sorted with respect to each other in one pass are grouped together again in subsequent passes.

Example of a bad sequence: 64, 32, 16, 8, 4, 2, 1

What happens if the largest values are all in odd positions?


Difficult to analyze precisely, typically use experiments. With a bad interval sequence, it’s O (n2) in the worst case. With a good interval sequence, it’s better than O(n2). At least O (n1.5) in the average and worst case. Some experiments have shown average-case running times of O (n1.25) or even O (n7/6). Significantly better than insertion or selection for large n: n n2 n1.5 n1.25 10 100 31.6 17.8 100 10,000 1000 316 10,000 100,000,000 1,000,000 100,000 106 1012 109 3.16 x 107


Algorithm:

h = 1

while h < n, h = 3*h + 1

while h > 0,

h = h / 3

for k = 1 : h, insertion sort a[ k : h : n]

→ invariant: each h-sub-array is sorted

end

•  Not stable

•  O(1) extra space

•  O(n3/2) time, depend on sequence h (neither tight upper bounds nor the best increment sequence are known

•  Adaptive: O(n·lg(n)) time when nearly sorted

Sorting – Bubble sort

Perform a sequence of passes through the array. On each pass: proceed from left to right, swapping adjacent elements if they are out of order.

Larger elements “bubble up” to the end of the array.

At the end of the kth pass, the k rightmost elements are in their final positions, so we don’t need to consider them in subsequent passes.

Repeat from the first to n-1. Stop when you have only one element to check.

Sorting – Bubble sort 240 Chap. 7 Internal Sorting

i=0 1 2 3 4 5 642201713281423

13422017142815

1314422017152823

1314154220172328

1314151742202328

1314151720422328

13141517202342

131415172023284223 2815

Figure 7.2 An illustration of Bubble Sort. Each column shows the array afterthe iteration with the indicated value of i in the outer for loop. Values above theline in each column have been sorted. Arrows indicate the swaps that take placeduring a given iteration.

7.2.2 Bubble Sort

Our next sort is called Bubble Sort. Bubble Sort is often taught to novice pro-grammers in introductory computer science courses. This is unfortunate, becauseBubble Sort has no redeeming features whatsoever. It is a relatively slow sort, itis no easier to understand than Insertion Sort, it does not correspond to any in-tuitive counterpart in “everyday” use, and it has a poor best-case running time.However, Bubble Sort serves as the basis for a better sort that will be presented inSection 7.2.3.

Bubble Sort consists of a simple double for loop. The first iteration of theinner for loop moves through the record array from bottom to top, comparingadjacent keys. If the lower-indexed key’s value is greater than its higher-indexedneighbor, then the two values are swapped. Once the smallest value is encountered,this process will cause it to “bubble” up to the top of the array. The second passthrough the array repeats this process. However, because we know that the smallestvalue reached the top of the array on the first pass, there is no need to comparethe top two elements on the second pass. Likewise, each succeeding pass throughthe array compares adjacent elements, looking at one less value than the precedingpass. Figure 7.2 illustrates Bubble Sort. A Java implementation is as follows:

static <E extends Comparable<? super E>> void Sort(E[] A) {for (int i=0; i<A.length-1; i++) // Bubble up i’th record

for (int j=A.length-1; j>i; j--)if ((A[j].compareTo(A[j-1]) < 0))

DSutil.swap(A, j, j-1);}

Sorting - Bubble Sort

#define swap (a,b) { int t; t=a; a=b; b=t; }

void bubble ( int a[], int n ) {

int i, j;

for (i=0; i<n; i++) { /* n passes thru the array */

/* From start to the end of unsorted part */

for (j=1; j<(n-i); j++) {

/* If adjacent items out of order, swap */

if ( a[j-1]>a[j] ) swap (a[j-1],a[j]);

}}}} }

}

Sorting - Bubble Sort

Sec. 7.2 Three ⇥(n2) Sorting Algorithms 241

Determining Bubble Sort’s number of comparisons is easy. Regardless of thearrangement of the values in the array, the number of comparisons made by theinner for loop is always i, leading to a total cost of

nX

i=1

i = ⇥(n2).

Bubble Sort’s running time is roughly the same in the best, average, and worstcases.

The number of swaps required depends on how often a value is less than theone immediately preceding it in the array. We can expect this to occur for abouthalf the comparisons in the average case, leading to ⇥(n2) for the expected numberof swaps. The actual number of swaps performed by Bubble Sort will be identicalto that performed by Insertion Sort.

7.2.3 Selection Sort

Consider again the problem of sorting a pile of phone bills for the past year. An-other intuitive approach might be to look through the pile until you find the bill forJanuary, and pull that out. Then look through the remaining pile until you find thebill for February, and add that behind January. Proceed through the ever-shrinkingpile of bills to select the next one in order until you are done. This is the inspirationfor our last ⇥(n2) sort, called Selection Sort. The ith pass of Selection Sort “se-lects” the ith smallest key in the array, placing that record into position i. In otherwords, Selection Sort first finds the smallest key in an unsorted list, then the secondsmallest, and so on. Its unique feature is that there are few record swaps. To findthe next smallest key value requires searching through the entire unsorted portionof the array, but only one swap is required to put the record in place. Thus, the totalnumber of swaps required will be n� 1 (we get the last record in place “for free”).

Figure 7.3 illustrates Selection Sort. Below is a Java implementation.

static <E extends Comparable<? super E>> void Sort(E[] A) {for (int i=0; i<A.length-1; i++) { // Select i’th recordint lowindex = i; // Remember its indexfor (int j=A.length-1; j>i; j--) // Find the least value

if (A[j].compareTo(A[lowindex]) < 0)lowindex = j; // Put it in place

DSutil.swap(A, i, lowindex);}

}

Selection Sort is essentially a Bubble Sort, except that rather than repeatedlyswapping adjacent values to get the next smallest record into place, we instead

Bubble Sort’s running time is roughly the same in the best, average, and worst cases. The number of swaps required depends on how often a value is less than the one immediately preceding it in the array. We can expect this to occur for about half the comparisons in the average case, leading to Θ(n2) for the expected number of swaps.

Distribution counting

A very special situation for which there is a simple sorting algorithm: "sort a file of N records whose keys are distinct integers between 1 and N" or

sort a file of N records whose keys are integers between 0 and M – 1.

If M is not too large, an algorithm called distribution counting can be used to solve this problem. The idea is to count the number of keys with each value and then use the counts to move the records into position on a second pass through the file

This method will work very well for the type of files postulated.

The Divide and Conquer Approach

The most well known algorithm design strategy:

1.  Divide the problem into two or more smaller subproblems.

2.  Conquer the subproblems by solving them recursively.

3.  Combine the solutions to the subproblems into the solutions for the original problem.

Sorting – Quicksort

ò  Efficient sorting algorithm

ò  Discovered by C.A.R. Hoare

ò  Example of “Divide and Conquer” algorithm

ò  Two phases

ò  Partition phase

ò  Divides the work into half

ò  Sort phase

ò  Conquers the halves!


ò  Partition

ò  Choose a pivot

ò  Find the position for the pivot so that

ò  all elements to the left are less

ò  all elements to the right are greater

< pivot > pivot pivot


ò  Conquer

ò  Apply the same algorithm to each half

< pivot > pivot

pivot < p’ p’ > p’ < p” p” > p”

It works by partitioning a file into two parts, then sorting the parts independently. The exact position of the partition depends on the file, and the algorithm has the following recursive structure:

If the partition method can be made more precise, (for example, choosing right-most element as a partition element), and the recursive call of this procedure may be eliminated, then the implementation of the algorithm looks like this: procedure quicksort (l,r: integer); var v, t, i, j: integer; begin if r>l then begin v:=a[r]; i:=l-1; j:=r; repeat repeat i:=i+1 until a[i]=>v; repeat j:=j-1 until a[j]=<v; t:=a[i]; a[i]:=a[j]; a[j]:=t; until j=<i; a[j]:=a[i]; a[i]:=a[r]; a[r]:=t; quicksort (l,i-1); quicksort (i+1,r) end end; An example of a partitioning of a larger file (choosing the right-most element):

The crux of the method is the partition procedure, which must rearrange the array to make the following conditions hold: � the element a[i] is in its final place in the array for some i; � all the elements left to the a[i] are less then or equal to it; � all the elements right to the a[i] are greater then or equal to it.

The first improvement: Removing Recursion The recursion may be removed by using an explicit pushdown stack, which is containing "work to be done" in the form of subfiles to be sorted. Any time we need a subfile to process, we pop the stack. When we partition, we create two subfiles to be processed which can be pushed on the stack. This leads to the following nonrecursive implementation:

procedure quicksort; var t, i, 1, r: integer; begin l:=1; r:=N; stackinit; push (l); push (r);

Quicksort – Partition

int partition( int *a, int low, int high ) { int left, right; int pivot_item; pivot_item = a[low]; pivot = left = low; right = high; while ( left < right ) { /* Move left while item < pivot */ while( a[left] <= pivot_item ) left++; /* Move right while item > pivot */ while( a[right] >= pivot_item ) right--; if ( left < right ) SWAP(a,left,right); } /* right is final position for the pivot */ a[low] = a[right]; a[right] = pivot_item; return right; }

Quicksort – Analysis

ò  Partition

ò  Check every item once O(n)

ò  Conquer

ò  Divide data in half O(log2n)

ò  Total

ò  Product O(n log n)

ò  Quicksort is generally faster

ò  Fewer comparisons

Quicksort – The truth!

ò  What happens if we use quicksort on data that’s already sorted (or nearly sorted)

ò  We’d certainly expect it to perform well!


ò  Sorted data

1 2 3 4 5 6 7 8 9

pivot

< pivot

?

> pivot


ò  Sorted data

ò  Each partition produces

ò  a problem of size 0

ò  and one of size n-1!

ò  Number of partitions?

1 2 3 4 5 6 7 8 9

> pivot

2 3 4 5 6 7 8 9

> pivot

pivot

pivot


ò  Sorted data

ò  Each partition produces

ò  a problem of size 0

ò  and one of size n-1!

ò  Number of partitions?

ò  n each needing time O(n) ò  Total n O(n)

or O(n2)

1 2 3 4 5 6 7 8 9

> pivot

2 3 4 5 6 7 8 9

> pivot

pivot

pivot


ò  Quicksort’s O(n log n) behaviour

ò  Depends on the partitions being nearly equal

➧  there are O( log n ) of them

ò  On average, this will nearly be the case

  and quicksort is generally O(n log n) ò  Can we do anything to ensure O(n log n) time?

ò  In general, no

ò  But we can improve our chances!!

Quicksort – Choice of the pivot

ò  Any pivot will work …

ò  Choose a different pivot …

ò  so that the partitions are equal

ò  then we will see O(n log n) time

1 2 3 4 5 6 7 8 9

pivot

< pivot > pivot

Quicksort – Median-of-3 pivot

ò  Take 3 positions and choose the median

ò  say … First, middle, last

➧  median is 5 ➧  perfect division of sorted data every time! ➧  O(n log n) time

➧  Since sorted (or nearly sorted) data is common, median-of-3 is a good strategy

ò  especially if you think your data may be sorted!

1 2 3 4 5 6 7 8 9

Quicksort – Random pivot

ò  Choose a pivot randomly

ò  Different position for every partition

➧  On average, sorted data is divided evenly

➧  O(n log n) time

ò  Key requirement

ò  Pivot choice must take O(1) time

Mergesort

ò  The merge sort algorithm is closely following the divide-and-conquer paradigm. Intuitively, it operates as follows:

ò  Divide: Divide the n-element sequence to be sorted into two subsequences of (n/2) elements each.

ò  Conquer: Sort the two subsequences recursively using mergesort.

ò  Combine: Merge the two sorted subsequences to produce the sorted answer.

ò  Suppose we have two sorted arrays a[1...N], b[1...M], and we need to merge them into one. The mergesort algorithm is based on a merge procedure, which can be presented as follows:

Mergesort

Mergesort

procedure merge (a, b);

var a,b,c: array [1..M + N)] of integer;

begin i:=1; j:=1; a[M+1]:=maxint; b[N+1]:=maxint;

for k:=1 to M+N do

if a[i]<b[j] then

begin c[k]:=a[i]; i:=i+1 end else begin c[k]:=b[j]; j:=j+1 end;

end;

Mergesort

To perform the mergesort, the algorithm for arrays can be as follows:

ò  procedure mergesort(l,r: integer); var i,j,k,m: integer; begin if r-l>0 then

ò  begin m:=(r+l) div 2; mergesort(l,m); mergesort(m+1,r); for i:=m downto l do b[i]:=a[i]; for j:=m+1 to r do b[r+m+1-j]:=a[j];

ò  for k:=l to r do if b[i]<b[j] then

ò  begin a[k]:=b[i]; i:=i+1 end else begin a[k]:=b[j]; j:=j-1 end;

ò  end;

ò  end;

Mergesort

Property 1: Mergesort requires about N lg N comparisons to sort any file of N elements.

Property 2: Mergesort is stable.

Property 3: Mergesort is insensible to the initial order of its input.

Quicksort – comparing to insert

ò  Quicksort is generally faster

ò  Fewer comparisons and exchanges

ò  Some empirical data

n Quick Heap InsertComp Exch Comp Exch Comp Exch

100 712 148 2842 581 2595 899200 1682 328 9736 9736 10307 3503500 5102 919 53113 4042 62746 21083

Comparison-Based Sorting Algorithms

Summary: Comparison-Based Sorting Algorithms

• Insertion sort is best for nearly sorted arrays.

• Mergesort has the best worst-case complexity, but requires extra memory – and moves to and from the temp array.

• Quicksort is comparable to mergesort in the average case. With a reasonable pivot choice, its worst case is seldom seen.

• Use ~cscie119/examples/sorting/SortCount.java to experiment.

algorithm best case avg case worst case extra memory

selection sort O(n2) O(n2) O(n2) O(1)

insertion sort O(n) O(n2) O(n2) O(1)

Shell sort O(n log n) O(n1.5) O(n1.5) O(1)

bubble sort O(n2) O(n2) O(n2) O(1)

quicksort O(n log n) O(n log n) O(n2) O(1)

mergesort O(n log n) O(n log n) O(nlog n) O(n)

Comparison-Based vs. Distributive Sorting• Until now, all of the sorting algorithms we have considered have

been comparison-based:• treat the keys as wholes (comparing them)• don’t “take them apart” in any way• all that matters is the relative order of the keys, not their

actual values.

• No comparison-based sorting algorithm can do better than O(nlog2n) on an array of length n. • O(nlog2n) is a lower bound for such algorithms.

• Distributive sorting algorithms do more than compare keys; they perform calculations on the actual values of individual keys.

• Moving beyond comparisons allows us to overcome the lower bound. • tradeoff: use more memory.

Comparison-Based Sorting Algorithms

Summary: Comparison-Based Sorting Algorithms

• Insertion sort is best for nearly sorted arrays.

• Mergesort has the best worst-case complexity, but requires extra memory – and moves to and from the temp array.

• Quicksort is comparable to mergesort in the average case. With a reasonable pivot choice, its worst case is seldom seen.

• Use ~cscie119/examples/sorting/SortCount.java to experiment.

algorithm best case avg case worst case extra memory

selection sort O(n2) O(n2) O(n2) O(1)

insertion sort O(n) O(n2) O(n2) O(1)

Shell sort O(n log n) O(n1.5) O(n1.5) O(1)

bubble sort O(n2) O(n2) O(n2) O(1)

quicksort O(n log n) O(n log n) O(n2) O(1)

mergesort O(n log n) O(n log n) O(nlog n) O(n)

Comparison-Based vs. Distributive Sorting• Until now, all of the sorting algorithms we have considered have

been comparison-based:• treat the keys as wholes (comparing them)• don’t “take them apart” in any way• all that matters is the relative order of the keys, not their

actual values.

• No comparison-based sorting algorithm can do better than O(nlog2n) on an array of length n. • O(nlog2n) is a lower bound for such algorithms.

• Distributive sorting algorithms do more than compare keys; they perform calculations on the actual values of individual keys.

• Moving beyond comparisons allows us to overcome the lower bound. • tradeoff: use more memory.

External Sorting and Basic Algorithm

Problem: If a list is too large to fit in main memory, the time required to access a data value on a disk or tape dominates any efficiency analysis.

1 disk access ≡ Several million machine instructions Solution: Develop external sorting algorithms that minimize

disk accesses

A Typical Disk Drive

Disk Access

Disk Access Time =

Seek Time (moving disk head to correct track)

+ Rotational Delay (rotating disk to correct block in track)

+ Transfer Time (time to transfer block of data to main memory)

Assume unsorted data is on disk at start

Let M = maximum number of records that can be stored & sorted in internal memory at one time

Basic External Sorting Algorithm

Algorithm Repeat: 1.  Read M records into main memory & sort internally. 2.  Write this sorted sub-list onto disk, until all data are

processed into runs Repeat: 1.  Merge two runs into one sorted run twice as long 2.  Write this single run back onto disk, until all runs

processed into runs twice as long Merge runs again as often as needed until only one large

run: the sorted list

Basic External Sorting

11 96 12 35 17 99 28 58 41 75 15 94 81 Unsorted Data on Disk

Assume M = 3 (M would actually be much larger, of course.) First step is to read 3 data items at a time into main memory, sort them and write them back to disk as runs of length 3.

11 94 81

96 12 35

17 99 28

58 41 75

15


Next step is to merge the runs of length 3 into runs of length 6.

11 94 81 96 12 35

17 99 28 58 41 75

15 11 94 81

96 12 35

17 99 28

58 41 75

15


Next step is to merge the runs of length 6 into runs of length 12.

11 94 81 96 12 35 17 99 28 58 41 75

15

15

11 94 81 96 12 35

17 99 28 58 41 75

CISC 235 Topic 8 58


Next step is to merge the runs of length 12 into runs of length 24. Here we have less than 24, so we’re finished.

11 94 81 96 12 35 17 99 28 58 41 75 15

11 94 81 96 12 35 17 99 28 58 41 75

15

Streaming Data through RAM

ò  An important method for sorting & other DB operations

ò  Simple case:

ò  Compute f(x) for each record, write out the result ò  Read a page from INPUT to Input Buffer ò  Write f(x) for each item to Output Buffer ò  When Input Buffer is consumed, read another page ò  When Output Buffer fills, write it to OUTPUT

ò  Reads and Writes are not coordinated

ò  E.g., if f() is Compress(), you read many pages per write. ò  E.g., if f() is DeCompress(), you write many pages per read.

f(x) RAM

Input Buffer

Output Buffer

OUTPUT INPUT

2-Way Sort: Requires 3 Buffers

ò  Pass 0: Read a page, sort it, write it.

ò  only one buffer page is used ò  Pass 1, 2, 3, …, etc.:

ò  requires 3 buffer pages ò  merge pairs of runs into runs twice as long ò  three buffer pages used.

Main memory buffers

INPUT 1

INPUT 2

OUTPUT

Disk Disk

Two-Way External Merge Sort

Each pass we read + write each page in file.

N pages in the file => the number of passes

So total cost is:

Idea: Divide and conquer: sort subfiles and merge

! "= +log2 1N

! "( )2 12N Nlog +

Input file

1-page runs

2-page runs

4-page runs

8-page runs

PASS 0

PASS 1

PASS 2

PASS 3

9

3,4 6,2 9,4 8,7 5,6 3,1 2

3,4 5,6 2,6 4,9 7,8 1,3 2

2,3 4,6

4,7 8,9

1,3 5,6 2

2,3 4,4 6,7 8,9

1,2 3,5 6

1,2 2,3 3,4 4,5 6,6 7,8

General External Merge Sort

ò  To sort a file with N pages using B buffer pages:

ò  Pass 0: use B buffer pages. Produce sorted runs of B pages each.

ò  Pass 1, 2, …, etc.: merge B-1 runs.

! "N B/

B Main memory buffers

INPUT 1

INPUT B-1

OUTPUT

Disk Disk

INPUT 2

. . . . . .

. . .

Cost of External Merge Sort

ò  Number of passes:

ò  Cost = 2N * (# of passes)

ò  E.g., with 5 buffer pages, to sort 108 page file:

ò  Pass 0: = 22 sorted runs of 5 pages each (last run is only 3 pages)

ò  Pass 1: = 6 sorted runs of 20 pages each (last run is only 8 pages)

ò  Pass 2: 2 sorted runs, 80 pages and 28 pages

ò  Pass 3: Sorted file of 108 pages

! "! "1 1+ −log /B N B

! "108 5/

! "22 4/

Formula check: ┌log4 22┐= 3 … + 1 à 4 passes √

Number of Passes of External Sort

N B=3 B=5 B=9 B=17 B=129 B=257 100 7 4 3 2 1 1 1,000 10 5 4 3 2 2 10,000 13 7 5 4 2 2 100,000 17 9 6 5 3 3 1,000,000 20 10 7 5 3 3 10,000,000 23 12 8 6 4 3 100,000,000 26 14 9 7 4 4 1,000,000,000 30 15 10 8 5 4

( I/O cost is 2N times number of passes)

I/O for External Merge Sort

ò  Do I/O a page at a time

ò  Not one I/O per record

ò  In fact, read a block (chunk) of pages sequentially!

ò  Suggests we should make each buffer (input/output) be a block of pages.

ò  But this will reduce fan-in during merge passes!

ò  In practice, most files still sorted in 2-3 passes.

Double Buffering ò  To reduce wait time for I/O request to complete, can

prefetch into `shadow block’.

ò  Potentially, more passes; in practice, most files still sorted in 2-3 passes.

OUTPUT OUTPUT'

Disk Disk

INPUT 1

INPUT k

INPUT 2

INPUT 1'

INPUT 2'

INPUT k'

block size b

B main memory buffers, k-way merge

Balanced multiway merging

ò  To begin, we'll trace through the various steps of the simplest sort-merge procedure for a small example. Suppose that we have records with the keys

AS0RTINGANDMERGINGEXAMPLE

ò  on an input tape; these are to be sorted and put onto an output tape. Using a "tape" simply means that we're restricted to reading the records sequentially: the second record can't be read until the first is read, and so on. Assume further that we have only enough room for three records in our computer memory but that we have plenty of tapes available.


ò  The first step is to read in the file three records at a time, sort them to make three-record blocks, and output the sorted blocks. Thus, first we read in A S 0 and output the block A 0 S, next we read in R T I and output the block I R T, and so forth. Now, in order for these blocks to be merged together, they must be on different tapes. If we want to do a three-way merge, then we would use three tapes.

Balanced Multiway Merging To begin, we'll trace through the various steps of the simplest sort-merge procedure for a small example. Suppose that we have records with the keys

A S 0 R T I N G A N D M E R G I N G E X A M P L E on an input tape; these are to be sorted and put onto an output tape. Using a "tape" simply means that we're restricted to reading the records sequentially: the second record can't be read until the first is read, and so on. Assume further that we have only enough room for three records in our computer memory but that we have plenty of tapes available.

The first step is to read in the file three records at a time, sort them to make three-record blocks, and output the sorted blocks. Thus, first we read in A S 0 and output the block A 0 S, next we read in R T I and output the block I R T, and so forth. Now, in order for these blocks to be merged together, they must be on different tapes. If we want to do a three-way merge, then we would use three tapes, ending up after the sorting pass with the configuration shown in figure:

Now we're ready to merge the sorted blocks of size three. We read the first record off each input tape (there's just enough room in the memory) and output the one with the smallest key. Then the next record from the same tape as the record just output is read in and, again, the record in memory with the smallest key is output. When the end of a three-word block in the input is encountered, that tape is ignored until the blocks from the other two tapes have been processed and nine records have been output. Then the process is repeated to merge the second three-word block on each tape into a nine-word block (which is output on a different tape, to get ready for the next merge). By continuing in this way, we get three long blocks configured as shown in figure next:

Now one more three-way merge completes the sort. If we had a much longer file with many blocks of size 9 on each tape, then we would finish the second pass with blocks of size 27 on tapes 1, 2, and 3, then a third pass would produce blocks of size 81 on tapes 4, 5, and 6, and so forth. We need six tapes to sort an arbitrarily large file: three for the input and three for the output of each three-way merge. (Actually, we could get by with just four tapes: the output could be put on just one tape, and then the blocks from that tape distributed to the three input tapes in between merging passes.)

Balanced multiway merging ò  We read the first record off each input tape (there's just enough

room in the memory) and output the one with the smallest key. Then the next record from the same tape as the record just output is read in and, again, the record in memory with the smallest key is output. When the end of a three-word block in the input is encountered, that tape is ignored until the blocks from the other two tapes have been processed and nine records have been output. Then the process is repeated to merge the second three-word block on each tape into a nine-word block (which is output on a different tape, to get ready for the next merge)

Balanced Multiway Merging To begin, we'll trace through the various steps of the simplest sort-merge procedure for a small example. Suppose that we have records with the keys

A S 0 R T I N G A N D M E R G I N G E X A M P L E on an input tape; these are to be sorted and put onto an output tape. Using a "tape" simply means that we're restricted to reading the records sequentially: the second record can't be read until the first is read, and so on. Assume further that we have only enough room for three records in our computer memory but that we have plenty of tapes available.

The first step is to read in the file three records at a time, sort them to make three-record blocks, and output the sorted blocks. Thus, first we read in A S 0 and output the block A 0 S, next we read in R T I and output the block I R T, and so forth. Now, in order for these blocks to be merged together, they must be on different tapes. If we want to do a three-way merge, then we would use three tapes, ending up after the sorting pass with the configuration shown in figure:

Now we're ready to merge the sorted blocks of size three. We read the first record off each input tape (there's just enough room in the memory) and output the one with the smallest key. Then the next record from the same tape as the record just output is read in and, again, the record in memory with the smallest key is output. When the end of a three-word block in the input is encountered, that tape is ignored until the blocks from the other two tapes have been processed and nine records have been output. Then the process is repeated to merge the second three-word block on each tape into a nine-word block (which is output on a different tape, to get ready for the next merge). By continuing in this way, we get three long blocks configured as shown in figure next:

Now one more three-way merge completes the sort. If we had a much longer file with many blocks of size 9 on each tape, then we would finish the second pass with blocks of size 27 on tapes 1, 2, and 3, then a third pass would produce blocks of size 81 on tapes 4, 5, and 6, and so forth. We need six tapes to sort an arbitrarily large file: three for the input and three for the output of each three-way merge. (Actually, we could get by with just four tapes: the output could be put on just one tape, and then the blocks from that tape distributed to the three input tapes in between merging passes.)


ò  Now one more three-way merge completes the sort. If we had a much longer file with many blocks of size 9 on each tape, then we would finish the second pass with blocks of size 27 on tapes 1, 2, and 3, then a third pass would produce blocks of size 81 on tapes 4, 5, and 6, and so forth. We need six tapes to sort an arbitrarily large file: three for the input and three for the output of each three-way merge.

ò  Actually, we could get by with just four tapes: the output could be put on just one tape, and then the blocks from that tape distributed to the three input tapes in between merging passes.

ò  This method is called the balanced multiway merge: it is a reasonable algorithm for external sorting and a good starting point for the implementation of an external sort.

Replacement selection

ò  It turns out that the details of the implementation can be developed in an elegant and efficient way using priority queues. First, we'll see that priority queues provide a natural way to implement a multiway merge. More important, it turns out that we can use priority queues for the initial sorting pass in such a way that they can produce sorted blocks much longer than could fit into internal memory.

ò  The basic operation needed to do P-way merging is repeatedly to output the smallest of the smallest elements not yet output from each of the P blocks to be merged. That smallest element should be replaced with the next element from the block from which it came.

ò  The replace operation on a priority queue of size P is exactly what is needed.


ò  The process of merging A 0 S with I R T and A G N (the first merge from our example above), using a heap of size three in the merging process is shown:

This method is called the balanced multiway erge: it is a reasonable algorithm for external sorting and a good starting point for the implementation of an external sort. The more sophisticated algorithms below can make the sort run a little faster, but not much. (However, when execution times are measured in hours, as is not uncommon in external sorting, even a small percentage decrease in running time can be quite significant.) Suppose that we have N words to be manipulated by the sort and an internal memory of size M. Then the "sort" pass produces about N/M sorted blocks. (This estimate assumes one-word records: for larger records, the number of sorted blocks is computed by multiplying further by the record size.) If we do P-way merges on each subsequent pass, then the number of subsequent passes is about logp (N/M), since each pass reduces the number of sorted blocks by a factor of P. For example, the formula above says that using a four-way merge to sort a 200 million-word file on a computer with a million words of memory should take a total of about five passes. A very rough estimate of the running time can be found by multiplying by five the running time for the reverse file copy implementation suggested above.

Replacement Selection

It turns out that the details of the implementation can be developed in an elegant and efficient way using priority queues. First, we'll see that priority queues provide a natural way to implement a multiway merge. More important, it turns out that we can use priority queues for the initial sorting pass in such a way that they can produce sorted blocks much longer than could fit into internal memory. The basic operation needed to do P-way merging is repeatedly to output the smallest of the smallest elements not yet output from each of the P blocks to be merged. That smallest element should be replaced with the next element from the block from which it came. The replace operation on a priority queue of size P is exactly what is needed. Specifically, to do a P-way merge we begin by filling up a priority queue of size P with the smallest element from each of the P inputs. Then we output the smallest element and replace it in the priority queue with the next element from its block. The process of merging A 0 S with I R T and A G N (the first merge from our example above), using a heap of size three in the merging process is shown in figure:

The "keys" in these heaps are the smallest (first) key in each node. For clarity, we show entire blocks in the nodes of the heap; of course, an actual implementation would be an indirect heap of pointers into the blocks. First, the A is output so that the O (the next key in its block) becomes the "key" of the root. This violates the heap condition, so that node is exchanged with the node containing A, G, and N. Then that A is output and replaced with the next key in its block, the G. This does not violate the heap condition, so no further change is necessary.


ò  In summary, the replacement selection technique can be used for both the "sort" and the "merge" steps of a balanced multiway merge.

ò  Property: A file of N records can be sorted using an internal memory capable of holding M records and (P + 1) tapes in about 1 + logP (N/2M) passes.

ò  We first use replacement selection with a priority queue of size M to produce initial runs of size about 2M (in a random situation) or more (if the file is partially ordered), then use replacement selection with a priority queue of size P for about logP (N/2M) (or fewer) merge passes.

Polyphase merging

ò  One problem with balanced multiway merging for tape sorting is that it requires either an excessive number of tape units or excessive copying. For P-way merging either we must use 2P tapes (P for input and P for output) or we must copy almost all of the file from a single output tape to P input tapes between merging passes, which effectively doubles the number of passes to be about 2 logP (N/2M). Several clever tape-sorting algorithms have been invented which eliminate virtually all of this copying by changing the way in which the small sorted blocks are merged together. The most prominent of these methods is called polyphase merging.

ò  The basic idea behind polyphase merging is to distribute the sorted blocks produced by replacement selection somewhat unevenly among the available tape units (leaving one empty) and then to apply a "merge-until-empty" strategy, at which point one of the output tapes and the input tape switch roles.

Polyphase merging

ò  For example, suppose that we have just three tapes, and we start out with the initial configuration of sorted blocks on the tapes shown at the top of figure:

(This comes from applying replacement selection to our example file with an internal memory that can only hold two records.) Tape 3 is initially empty, the output tape for the first merges. Now, after three two-way merges from tapes I and 2 to tape 3, the second tape becomes empty. Then, after two two-way merges from tapes I and 3 to tape 2, the first tape becomes empty. The sort is completed in two more steps. First, a two-way merge from tapes 2 and 3 to tape I leaves one file on tape 2, one file on tape 1. Then a two-way merge from tapes I and 2 to tape 3 leaves the entire sorted file on tape 3. This merge-until-empty strategy can be extended to work for an arbitrary number of tapes. Table below shows how six tapes might be used to sort 497 initial runs:

Tape 1 61 0 31 15 7 3 1 0 1

Tape 2 0 61 30 14 6 2 0 1 0

Tape 3 120 59 28 12 4 0 2 1 0

Tape 4 116 55 24 8 0 4 2 1 0

Tape 5 108 47 16 0 8 4 2 1 0

Tape 6 92 31 0 16 8 4 2 1 0 If we start out as indicated in the first column of table, with Tape 2 being the output tape, Tape I having 61 initial runs, Tape 3 having 120 initial runs, etc. as indicated in the first column of table, then after running a five-way "merge until empty," we have Tape I empty, Tape 2 with 61 (long) runs, Tape 3 with 59 runs, etc., as shown in the second column of table. At this point, we can rewind Tape 2 and make it an input tape, and rewind Tape I and make it the output tape. Continuing in this way, we eventually get the whole sorted file onto Tape 1. The merge is broken up into many phases which don't involve all the data, but no direct copying is involved.

The main difficulty in implementing a polyphase merge is to determine how to distribute the initial runs. It is not difficult to see how to build the table above by working backwards:

take the largest number in each column, make it zero, and add it to each of the other numbers to get the previous column.

This corresponds to defining the highest-order merge for the previous column which could give the present column. This technique works for any number of tapes (at least three):

Polyphase merging

ò  This merge-until-empty strategy can be extended to work for an arbitrary number of tapes. Table below shows how six tapes might be used to sort 497 initial runs:

(This comes from applying replacement selection to our example file with an internal memory that can only hold two records.) Tape 3 is initially empty, the output tape for the first merges. Now, after three two-way merges from tapes I and 2 to tape 3, the second tape becomes empty. Then, after two two-way merges from tapes I and 3 to tape 2, the first tape becomes empty. The sort is completed in two more steps. First, a two-way merge from tapes 2 and 3 to tape I leaves one file on tape 2, one file on tape 1. Then a two-way merge from tapes I and 2 to tape 3 leaves the entire sorted file on tape 3. This merge-until-empty strategy can be extended to work for an arbitrary number of tapes. Table below shows how six tapes might be used to sort 497 initial runs:

Tape 1 61 0 31 15 7 3 1 0 1

Tape 2 0 61 30 14 6 2 0 1 0

Tape 3 120 59 28 12 4 0 2 1 0

Tape 4 116 55 24 8 0 4 2 1 0

Tape 5 108 47 16 0 8 4 2 1 0

Tape 6 92 31 0 16 8 4 2 1 0 If we start out as indicated in the first column of table, with Tape 2 being the output tape, Tape I having 61 initial runs, Tape 3 having 120 initial runs, etc. as indicated in the first column of table, then after running a five-way "merge until empty," we have Tape I empty, Tape 2 with 61 (long) runs, Tape 3 with 59 runs, etc., as shown in the second column of table. At this point, we can rewind Tape 2 and make it an input tape, and rewind Tape I and make it the output tape. Continuing in this way, we eventually get the whole sorted file onto Tape 1. The merge is broken up into many phases which don't involve all the data, but no direct copying is involved.

The main difficulty in implementing a polyphase merge is to determine how to distribute the initial runs. It is not difficult to see how to build the table above by working backwards:

take the largest number in each column, make it zero, and add it to each of the other numbers to get the previous column.

This corresponds to defining the highest-order merge for the previous column which could give the present column. This technique works for any number of tapes (at least three):

Polyphase merging

ò  The main difficulty in implementing a polyphase merge is to determine how to distribute the initial runs. It is not difficult to see how to build the table above by working backwards:

ò  take the largest number in each column, make it zero, and add it to each of the other numbers to get the previous column.

ò  This corresponds to defining the highest-order merge for the previous column which could give the present column. This technique works for any number of tapes (at least three):

ò  the numbers which arise are "generalized Fibonacci numbers" which have many interesting properties.

Summary

ò  External sorting is important

ò  External merge sort minimizes disk I/O cost:

ò  Pass 0: Produces sorted runs of size B (# buffer pages). Later passes: merge runs.

ò  # of runs merged at a time depends on B, and block size.

ò  Larger block size means less I/O cost per page.

ò  Larger block size means smaller # runs merged.

ò  In practice, # of runs rarely more than 2 or 3.

Summary, cont.

ò  Choice of internal sort algorithm may matter:

ò  Quicksort: Quick!

ò  Heap/tournament sort: slower (2x), longer runs

ò  The best sorts are wildly fast:

ò  Despite 40+ years of research, still improving!

ò  Clustered B+ tree is good for sorting; unclustered tree is usually very bad.

Documents

Data Structures and Algorithms - Pradžiaalgis/dsax/Data-structures-1.pdf · Data Structures and Algorithms Course’s slides: Sorting Algorithms algis . ... • The resulting formula