AITI Lecture 22Sorting

Adapted from MIT Course 1.00 Spring 2003

Lecture 32

(Teachers: Please do not erase the above note)

Sorting Order

• Sorting implies that the items to be sorted are ordered. We use the same approach that we employed for binary search trees .

• That is, we will sort Objects, and whenever we invoke a sort,– we either supply a Comparator to order the elements to

be sorted or– the sort routine can assume that the objects to be

sorted belong to a class like Integer that implements Comparable and possesses a native ordering.

The Sort Interface

• There are times when we want to sort only part of a collection.

• We define an interface for sorts with this in mind. A Sort must supply two methods, one to sort an entire array of Objects and one to sort a portion of an array specified by start and end elements (a true closed interval, not half open):

public interface Sort { public void sort(Object[] d,int start,int end);

public void sort(Object[] d); }

Using the Sort Interface

• This interface makes sorting algorithms become classes.• We must create an instance of the class before we can use it

to perform a sort.• Each sort class will have two constructors. The default

constructor will sort elements that have a native order. The other constructor will take a Comparator as the only argument.

• Here is an example code fragment that creates an instance of InsertionSort to sort an array of Integers:

Integer [] iArray = new Integer[ 100 ]; . . . // initialize iArray Sort iSort = new InsertionSort(); iSort.sort( iArray );

• Our examples will be simpler than this, though.

Insertion Sort

• Insertion sorting is a formalized version of the way most of us sort cards. In an insertion sort, items are selected one at a time from an unsorted list and inserted into a sorted list.

• It is a good example of an elementary sort. It runs slowly, which is ok on small datasets but unacceptable on large ones. (See the exercise)

• To save memory and unnecessary copying we sort the array in place.

• We do this by using the low end of the array to grow the sorted list against the unsorted upper end of the array.

Insertion Sort Diagram

7 4 363Initially

Sorted Unsorted

4 7 363After 1

insertion

Sorted Unsorted

4 7 363After 3

insertions

Sorted Unsorted

. . .

Run the InsertionSort Simulation

• Download InsertionSort.jar, the simulation that you will use to examine the algorithm.

• Type in a series of numbers each followed by a return. These are the numbers to sort.

• Click start and then stepInto to single step through the code.

• reset restarts the current sort.• new allows you to specify a new sort.

InsertionSort Questions

Use the simulator to explore the following questions and let us know when you think you know the answers:– How many elements have to be moved in the inner for

loop if you run InsertionSort on an already sorted list? Does it run in O(1), O(n), or O(n2) time?

– What order of elements will produce the worst performance? Does this case run in O(1), O(n), or O(n2) time? Why?

– Does the average case have more in common with the best case or the worst case?

Insertion Sort Codepublic class InsertionSortMain {

private static class Item implements Comparable { public double key; public String value; public Item(double k, String v) { key= k; value= v; } public int compareTo(Object o) { Item other= (Item) o; if (key < other.key ) return -1; else if (key > other.key) return 1; else return 0; } }

Insertion Sort Code p.2 public static void main(String[] args) { Item a= new Item(77.0, "Al"); Item b= new Item(9.0, "Bob"); Item c= new Item(22.0, "Cindy"); Item d= new Item(5.0, "Debra"); Item[] names= {a, b, c, d}; insertionSort(names); for (int i=0; i < names.length; i++) System.out.println(names[i].key + names[i].value); } public static void insertionSort(Comparable[] data) { for (int pos= 1; pos < data.length; pos++) { Comparable v= data[pos]; int i= 0; for (i= pos; i>0 && data[i-1].compareTo(v)>0; i--) data[i]= data[i-1]; data[i]= v; } } }

Quicksort overview

• Most efficient general purpose sort, O(n lg n)• Basic strategy

– Split array (or vector) of data to be sorted into 2 subarrays so that:

• Everything in first subarray is smaller than a known value• Everything in second subarray is larger than that value

– Technique is called ‘partitioning’• Known value is called the ‘pivot element’

– Once we’ve partitioned, pivot element will be located in its final position

– Then we continue splitting the subarrays into smaller subarrays, until the resulting pieces have only one element (recursion!)

Quicksort algorithm

1. Choose an element as pivot. We use right element2. Start indexes at left and (right-1) elements3. Move left index until we find an element> pivot 4. Move right index until we find an element < pivot5. If indexes haven’t crossed, swap values and repeat

steps 3 and 46. If indexes have crossed, swap pivot and left index

values7. Call quicksort on the subarrays to the left and right

of the pivot value

Example

Quicksort(a, 0, 6)

Original 45 80 55 85 50 70 65

i i j j pivot

1st swap 45 50 55 85 80 70 65

i i j j

2nd swap 45 50 55 65 80 70 85

quicksort(a,0,2) quicksort(a,4,6)

final position

Partitioning

• Partitioning is the key step in quicksort.• In our version of quicksort, the pivot is chosen to be the

last element of the (sub)array to be sorted.• We scan the (sub)array from the left end using index low

looking for an element >= the pivot.• When we find one we scan from the right end using index

high looking for an element <= pivot.• If low < high, we swap them and start scanning for another

pair of swappable elements.• If low >= high, we are done and we swap low with the

pivot, which now stands between the two partitions.

Recursive Partitioning

8 4 579663 2

2 34 6968 75

2 43 5 8966 7

2 43 5 9766 8

current pivot

pivot swap

5partition swap

old pivot5

Quicksort Simulation

• Double click the QuickSort.jar file.• It works the same way as the InsertionSort

simulation.• Use the simulator slowly. If you click too quickly

it may lose events and show incorrect results. Sorry.

• What is the worst case behavior for quicksort?• What happens when you quicksort an already-

sorted array?

A Better Quicksort

• The choice of pivot is crucial to quicksort's performance.• The ideal pivot is the median of the subarray, that is, the middle

member of the sorted array. But we can't find the median without sorting first.

• It turns out that the median of the first, middle and last element of each subarray is a good substitute for the median. It guarantees that each part of the partition will have at least two elements, provided that the array has at least four, but its performance is usually much better. And there are no natural cases that will produce worst case behavior.

• Try running MedianQuickSort from MedianQuickSort.jar.• Other improvements:

– Convert from recursive to iterative, for efficiency, and always process shortest subarray first (limit stack size, pops, pushes)

– When subarray is small enough (5-25 elements) use insertion sort

Quicksort main(), swappublic class QuicksortMain { public static void main(String[] args) { String input= JOptionPane.showInputDialog("Enter no of

elements to sort:"); int size= Integer.parseInt(input); int[] sortdata= new int[size]; sortdata[0]= sortdata[size-1]= 500; for (int i=1; i < size-1; i++) sortdata[i]= (int) (1000*Math.random()); quicksort(sortdata, 0, size-1); System.out.println("Done"); for (int i=0; i < size; i++) System.out.println(sortdata[i]); System.exit(0); } public static void swap(int[] a, int i, int j) { int temp= a[i]; // Swaps a[i] and a[j] a[i]= a[j]; a[j]= temp; }

Quicksort, partition public static int partition(int[] a, int left, int right) { int i= left -1; int j= right; int pvalue= a[right]; // Partition element for ( ; ; ) { while ( a[++i] < pvalue) ; // Move index to right while ( a[--j] > pvalue) ; // Move index to left if (i >= j) break; // Indexes cross swap(a, i, j); // Exchange elements } swap(a, i, right); // Exchange pivot, right return i; } public static void quicksort(int[] a, int left, int right) { if (right <= left) return; int part= partition(a, left, right); quicksort(a, left, part-1); quicksort(a, part+1, right); } }

Counting sort

• O(n) sort when keys are small integers• Procedure:

– Count the number of records with a given key– Calculate the number of records with keys that

are less by adding up their counts– This tells you where the final position of your

keys will be– Move keys to final positions and you’re done

• You never compare any value pairs!– This is special case of radix sorting

Counting sort example

Input Count: 1st 2nd Output

i a 0 0 0 i b

1 3 1 0 0 1 2

2 5 2 1 1 2 3

3 2 3 2 3 3 3

4 3 4 0 3 4 5

5 1 4size

rangeMove values to output vector or array

Steps in the CountingSort Algorithm

1. Copy the numbers to be sorted to a temporary array.

2. Initialize an array indexed by key values (a histogram of keys) to 0.

3. Iterate over the array to be sorted counting the frequency of each key.

4. Now calculate the cumulative histogram for each key value, k. The first element, k=0, is the same as the frequency of key k. The second, k=1, is the sum of the frequency for k=0 and k=1. The third is the sum of the second plus the frequency for k=2. And so on.

Steps in the CountingSort Algorithm, 2

5. The first element of the cumulative histogram contains the number of elements of the original array with values <= 0. the second, those <= 1. They lay out blocks of values in the sorted array.

6. Starting with the last element in the original array and working back to the first, look up its key in the cumulative histogram to find its destination in the sorted array. It will be the histogram value – 1.

7. Decrement the entry in the cumulative histogram so the next key is not stored on top of the first.

CountingSort

4 3

range = 8

7 6 4 538 2

frequencies

0 0 1 2 2 111 1

0 1 2 3 4 765 8

cumulative histogram

0 0 1 3 5 876 9

0 1 2 3 4 765 8

There are no keys

0, 1

Key 2 occurs 1 time

and should occupy

Position 0.

Key 3 occurs 2 times

and should occupy

positions 1 and 2.

Key 4 occurs 2 times

and should occupy

positions 3 and 4.

Etc.sorted output

2 3 3 4 4 765 8

0 1 2 3 4 765 8

CountingSort Questions

• Double click the CountingSort.jar file.• Note that since CountingSort requires the specification of a

range (or two passes over the data), it can’t be implemented using our Sort interface.

• Experiment with the simulation. Type the numbers to be sorted first. Then enter the range in the range field and press the start button. Use stepinto to trace the code.

• Does counting sort have a best case or worst case?• Does it sort in O(1), O(n), or O(n log n) time? Why?

Counting Sort main()public class CountingSort { public static void main(String[] args) { int size= 10; // Size-number of items to sort int range= 100; // Range= number of keys (0-99) int[] a= new int[size+1]; // Input data-key, unsorted String[] s= new String[size+1]; // Input data-value int[] b= new int[size+1]; // Output data-key,sorted String[] t= new String[size+1]; // Output data-value,srt String input= ""; for (int i=0; i < size; i++) { input= JOptionPane.showInputDialog("Enter key:"); a[i]= Integer.parseInt(input); s[i]= JOptionPane.showInputDialog("Enter value:");} countingSort(a, s, b, t, range); for (int i=1; i <= size; i++) System.out.println(b[i] + " " + t[i]); System.exit(0); }

Counting sort code public static void countingSort(int[] a, String[] s, int[] b,

String[] t, int range) { int size= a.length -1; int[] count= new int[range]; // Count of records w/key for (int i=1; i <= size; i++) // Count keys in bucket count[a[i]]++; for (int j=1; j < range; j++) // Count keys before this count[j] += count[j-1]; // 1st position of this key for (int i=size; i >= 1; i--) { // Move keys to end spot t[count[a[i]]]= s[i]; // Move value b[count[a[i]]--]= a[i]; // Move key } }}

Applications of sorting

• Business applications are obvious:– Reports sorted by ID, department, etc.

• Many technical applications use sorting:– Satellite payload– Optimal maintenance policy (e.g., pavement)– Processor scheduling– Generating graphs (networks)

Satellite payload

Experiment Weight Value Val/Wgt

A 2 4 2.0

B 4 6 1.5

C 2 5 2.5

D 3 5 1.7

Assume satellite can carry max weight= 7

Satellite, p.2

0 2 4 6 8 10 12Weight

20

15

10

5

0

Value

x

x

x

x

C

A

D

B

Max=7

Sort in Value/Weight ratio. Maximum derivative gives optimum value.Gives solution for all maximum weights M. (‘Greedy algorithm’)

Last item in solution may be fractional. Often this is acceptable.If not, we use decision trees (‘branch and bound’) as in PS9

Processor scheduling

• You have a set of n jobs to run on a processor (CPU) or machine

• Each job i has a deadline di and profit pi

• There is one processor or machine• Each job takes 1 unit of time (simplification)• You earn the profit if and only if the job is

completed by its deadline– “Profit” can be the priority of the computer job in a

real time system that discards tasks that cannot be completed by their deadline

Processor scheduling example

Number of jobs n=5Job (i) Profit Deadline Profit/TimeA 100 2 100B 19 1 19C 27 2 27D 25 1 25E 15 3 15

Solution is {C, A, E} for profit of 142

Processor scheduling algorithm

• Sort jobs by profit/time ratio (slope or derivative)

• Place each job at latest time that meets its deadline– Nothing is gained by scheduling it earlier, and

scheduling it earlier could prevent another more profitable job from being done

0 1 2 3Time

A (100)

C (27)

D, B infeasible

E (15)