Cfe2 ch12 final

Chapter Twelve: Sorting and Searching

Slides by Evan Gallagher

• To compare the selection sort and merge sort algorithms• To study the linear search and binary search algorithms• To appreciate that algorithms for the same task can differ

widely in performance• To understand the big-Oh notation• To be able to estimate and compare the performance of

algorithms• To write code to measure the running time of a program

Chapter Goals

Selection Sort

The selection sort algorithm

sorts a sequence by repeatedly finding the smallest element

of the unsorted tail region and moving it to the front.

11 9 17 5 12

Selection Sort

11 9 17 5 12

Selection Sort

Now the first element is in the correct place.

5 9 17 11 12

Selection Sort

Now do the same for the second position.Find the minimum in the unsorted tail region.

5 9 17 11 12

Selection Sort

There is no need to swap this time.The minimum in the tail portion is already greater.

So we do nothing,and the first and second elements are in order.

5 9 17 11 12

Selection Sort

There is no need to swap this time.The minimum in the tail portion is already greater.

So we do nothing,and the first and second elements are in order.

5 9 17 11 12

Selection Sort

Repeat this process of finding the minimum in the tail portionand either swapping it into place or doing nothing.

5 9 17 11 12

Selection Sort

5 9 17 11 12

Selection Sort

5 9 17 11 12

Selection Sort

5 9 11 17 12

Selection Sort

5 9 11 17 12

Even when the unsorted region only two elements long, keep to the same successful strategy.

5 9 11 17 12

Selection Sort

5 9 11 17 12

Selection Sort

5 9 11 12 17

Selection Sort

5 9 11 12 17

Selection Sort

That leaves you with an unprocessed region of length 1,but of course a region of length 1 is always sorted.

5 9 11 12 17

Selection Sort

That leaves you with an unprocessed region of length 1,but of course a region of length 1 is always sorted.

You are done.

5 9 11 12 17

Selection Sort

#include <cstdlib>#include <ctime>#include <iostream>using namespace std;

/**Gets the position of the smallest element in an array range.@param a the array@param from the beginning of the range@param to the end of the range@return the position of the smallest element inthe range a[from]...a[to]

*/int min_position(int a[], int from, int to){

int min_pos = from;for (int i = from + 1; i <= to; i++){

if (a[i] < a[min_pos]) { min_pos = i; }}return min_pos;

ch12/selsort.cpp

Selection Sort

/**Swaps two integers.@param x the first integer to swap@param y the second integer to swap

*/void swap(int& x, int& y){

int temp = x;x = y;y = temp;

ch12/selsort.cpp

Selection Sort

/**Sorts an array using the selection sort algorithm.@param a the array to sort@param size the number of elements in a

*/void selection_sort(int a[], int size){

int next; // The next position to be set to the minimumfor (next = 0; next < size - 1; next++){

// Find the position of the minimumint min_pos = min_position(a, next, size - 1);if (min_pos != next){ swap(a[min_pos], a[next]);}

ch12/selsort.cpp

Selection Sort

/**Prints all elements in an array.@param a the array to print@param size the number of elements in a

*/void print(int a[], int size){

for (int i = 0; i < size; i++){ cout << a[i] << " ";}cout << endl;

ch12/selsort.cpp

Selection Sort

int main(){

srand(time(0));const int SIZE = 20;int values[SIZE];for (int i = 0; i < SIZE; i++){ values[i] = rand() % 100;}print(values, SIZE);selection_sort(values, SIZE);print(values, SIZE);return 0;

ch12/selsort.cpp

Selection Sort

If speed was not an issue for us,we could stop the discussion of sorting right here.

However, the selection sort algorithm shows disappointingperformance when run on large data sets,

and it is worthwhile to study better sorting algorithms.

Selection Sort

Profiling A Program Run

To measure the performance of a program,one could simply run it

and measure how long it takes by using a stopwatch.

However, most of our programs run very quickly,and it is not easy to time them accurately in this way.

Instead we use the time function.

#include <ctime>

is required

int now = time(0);

This call sets now to the number of secondsthat have elapsed since January 1, 1970.

int now = time(0); int later = time(0);

We only care about the differencebetween the start and end times.

Profiling the Selection Sort Algorithm

By measuring the time just before and after the sorting,

int before = time(0);selection_sort(values, size);int after = time(0);cout << "Elapsed time = “ << after - before << " seconds" << endl;

By measuring the time just before and after the sorting,you don’t count the time it takes to initialize the array

or the time during which the program waitsfor the user to provide inputs.

int before = time(0);selection_sort(values, size);int after = time(0);cout << "Elapsed time = “ << after - before << " seconds" << endl;

Here are the results of some sample runs.

These measurements were obtained on a Pentium processorwith a clock speed of 1.67 GHz running Linux.

On another computer, the actual numbers will differ,but the relationship between the numbers will be the same.

As you can see,doubling the size of the data set

more than doubles the time needed to sort it..

Analyzing the Performance of the Selection Sort Algorithm

Let’s do some analysis…

Let us count the number of operations that a program must carry out to sort a sequence using the selection sort algorithm.

Actually, we don’t know how many machine operationsare generated for each C++ instruction

or which of those instructions are more time-consuming than others, but we can make a simplification.

Simply count how often an element is visited.

Each visit requires about the sameamount of work by other operations,

such as incrementing subscriptsand comparing values.

Let n be the size of the array.

To set the first element from the unsorted portion of the array:

You must find the smallest of n numbers.This takes n visits.

Then you must (or might) swap the elements.This takes two visits.

To set the second element from the unsorted portion of the array:

You must find the smallest of n – 1 numbers.This takes n – 1 visits.

To set the third element from the unsorted portion of the array:

You must find the smallest of n – 2 numbers.This takes n – 2 visits.

The process continues until…

…there are only two elements in the unsorted region.

To set the last two elements:

There is no searching, there are just two elements.This takes zero visits.

Then you must (or might) swap the last two elements.This takes two visits.

Therefore, the total number of visits is

This is a quadratic equation in n.

That explains why the graph looks

approximately like a parabola.

Now simplify the analysis further:

Just ignore the lower-level terms!

We are really only interested inhow the algorithm performs

as n increases.

We need only look for the factor thatmost influences the increase in time.

We can discount constant multipliersand look for the largest exponent.

Here the most important factor is the n2

We will simply say:

“The number of visits is of order n2”.

Computer scientists often use big-Oh notation:

The number of visits is O(n2).

To turn an exact expression such as

into big-Oh notation,

locate the fastest-growing term, n2,and ignore its constant coefficient,

½ in this case, no matter how large or small it may be:

The sad fact remains thatdoubling the size of the array causes

a fourfold increase in the time required for sorting it.

When the size of the sequence increases by a factor

of 100, the sorting time increases by a factor of 10,000!

To sort a sequence of a million

entries (for example, to create a telephone directory), takes 10,000 times as long

as sorting 10,000 entries.

If 10,000 entries can be sorted in about a second (as in our

example), then sorting one million entries requires almost three hours.

day 10

day 11

day 12

day 13

day 14

A Fortnight!

day 14

Merge Sort

The basic idea behind merge sort is very simple:

wishful thinking!

Merge Sort

We will simply hope that the first half

of the array is already perfectly sorted,

and the second half is too.

Now it is an easy matter to merge the two sorted sequencesinto a sorted sequence, simply by taking a new element from

either the first or the second subarray andchoosing the smaller of the elements each time:

Merge Sort

This process continuesuntil…

Merge Sort

There is only one element left (in either side),

Merge Sort

There is only one element left (in either side),

Merge Sort

and the array is completely sorted.

Merge Sort

Recall that the basic idea behind merge sort is

wishful thinking!

Merge Sort

The two half arrays will hardly ever be already sorted.

Someone had to sort each half.

Merge Sort

But if you think about it,what really happed was:

the array was divided into halves,then each half was sorted (by magic),

Merge Sort

But if you think about it,what really happed was:

the array was divided into halves,then each half was sorted (by magic),

and then the two sorted halves were merged together.

Merge Sort

If this process were to continue,then eventually there would besubarrays of only one element

which are very easy to sort and merged together

Merge Sort

If this process were to continue,then eventually there would besubarrays of only one element

which are very easy to sort and merged together

and then the two-length subarrays could be merged together

and then the four-length subarrays could be merged together

and then the eight-length subarrays could be merged together

and then the. . .

Merge Sort

and the entire array will be sortedwhen we get back to merging the first two halves:

Merge Sort!

Yes, this is a recursive algorithm.

At the recursive step,the merge sort is applied to each half problem,

recursively sorting each half and putting them back together.

The end test is an array of only one element.

We pass the same array on each call

and use indicesas the left and right endpointsof the subarray to be sorted.

Merge Sort

Using 0 and 9 (the whole array),one of the subarrays will be:

5 to 9:

Merge Sort

Using 0 and 9 (the whole array),one of the subarrays will be:

5 to 9:

Merge Sort

The end test of a one element arrayoccurs when the endpoints are the same index:

Merge Sort

/**Sorts the elements in a range of an array.@param a is the array with the elements to sort@param from start of the range to sort@param to end of the range to sort

*/void merge_sort(int a[], int from, int to){

if (from == to) { return; }int mid = (from + to) / 2;// Sort the first half and the second halfmerge_sort(a, from, mid);merge_sort(a, mid + 1, to);merge(a, from, mid, to);

Merge Sort

The merging part is longer,but straightforward.

The code for merge sort:

Merge Sort

ch12/mergesort.cpp

Merge Sort#include <cstdlib>#include <ctime>#include <iostream>using namespace std;

/** Merges two adjacent ranges in an array. @param a the array with the elements to merge @param from the start of the first range @param mid the end of the first range @param to the end of the second range*/

void merge(int a[], int from, int mid, int to){ int n = to - from + 1; // Size of the range to be merged

// Merge both halves into a temporary array b . // We allocate the array dynamically because its size is only // known at runtime—see Section 7.4 int* b = new int[n];

ch12/mergesort.cpp

Merge Sort

int i1 = from; // Next element to consider in the first half

int i2 = mid + 1; // Next element to consider in the second half

int j = 0; // Next open position in b

ch12/mergesort.cpp

Merge Sort

// As long as neither i1 nor i2 is past the end, // move the smaller element into b while (i1 <= mid && i2 <= to) { if (a[i1] < a[i2]) { b[j] = a[i1]; i1++; } else { b[j] = a[i2]; i2++ j++; } }

ch12/mergesort.cpp

Merge Sort

// Note that the body of only one of the // two while loops below is executed

// Copy any remaining entries of the first half while (i1 <= mid) { b[j] = a[i1]; i1++; j++; }

// Copy any remaining entries of the second half while (i2 <= to) { b[j] = a[i2]; i2++; j++; }

ch12/mergesort.cpp

Merge Sort

// Copy back from the temporary array for (j = 0; j < n; j++) { a[from + j] = b[j]; }

// The temporary array is no longer needed delete[] b;}

ch12/mergesort.cpp

Merge Sort

/** Sorts the elements in a range of an array. @param a the array with the elements to sort @param from start of the range to sort @param to end of the range to sort*/void merge_sort(int a[], int from, int to){ if (from == to) { return; } int mid = (from + to) / 2; // Sort the first half and the second half merge_sort(a, from, mid); merge_sort(a, mid + 1, to); merge(a, from, mid, to);}

ch12/mergesort.cpp

Merge Sort

/** Prints all elements in an array. @param a the array to print @param size the number of elements in a*/void print(int a[], int size){ for (int i = 0; i < size; i++) { cout << a[i] << " "; } cout << endl;}

ch12/mergesort.cpp

Merge Sort

int main(){ srand(time(0)); const int SIZE = 20; int values[SIZE]; for (int i = 0; i < SIZE; i++) { values[i] = rand() % 100; } print(values, SIZE); merge_sort(values, 0, SIZE - 1) print(values, SIZE); return 0;}

Analyzing the Merge Sort Algorithm

The merge sort algorithm looksmuch more complicated

than the selection sort algorithm,

and it appears that it may welltake much longer

to carry out these repeated subdivisions.

Here are the results of some sample runscomparing selection sort and merge sort.

You’ll need to look closely – there’s merge sort.

Note that the graph does not have a parabolic shape.

Instead, it appears as if the running time grows approximately linearly with the size of the sequence.

To understand why the merge sort algorithmis such a tremendous improvement,

let us estimate the number of sequence element visits.

First, we tackle the merge process that happens

after the first and second halves have been sorted.

merge(a, from, mid, to);

Each step in the merge process adds one more element to b.

There are n elements in b.

That element may come from the first or second half of a,and in most cases the elements from the two halves

must be compared to see which one to take.

Count that as 3 visits per element(one for b and one each for the two halves of a),

or 3n visits total.

3n (for comparisons)

Then you must

copy back from b to a.

Using two visits per elementyields another 2n visits.

3n (for comparisons) 2n (for copying) -------- 5n (for the whole merging process)

Now for the merge sort portion.

merge_sort(a, from, mid);merge_sort(a, mid + 1, to);

Letting T(n) denote the number of visits required to sort a range of n elements,

the two recursive calls will each require T(n/2) visits.

But…

What if n is not an even number?

We can (and will) make the assumption thatn is some power of 2, say m

(this will help later on).

n = 2m

Now every subsequence is divisible by 2.

To understand the relationship,

evaluate T(n/2),using the same formula:

To understand the relationship,

Do that process again:

This generalizes from 2, 4, 8, to arbitrary powers of 2:

Letting k be the power of 2:

Remember that it’s-going-to-be-helpful-laterassumption we made that n was a power of 2?

n = 2m

Let k = m

Let k = mBecause k = 2m, you have m = log2(n).

To establish the growth order,you drop the lower order term n

and you are left with:

You will also drop constant term 5:

It is customary to drop the base of the logarithmbecause all logarithms are related by a constant factor.

So the Merge Sort algorithm is an

O( n log( n ) )

algorithm.

So much better than Selection Sort’s O(n2)

How much better?

To sort 1,000,000 integers,

if it took Selection Sort a bit less than three hours O(n2)

on the same machine,

it would take Merge Sort about three minutes! O( n log( n ) )

Searching

Searching for an element in a sequenceis an extremely common task.

As with sorting,the right choice of algorithmscan make a big difference.

Linear Search

What’s Bjarne Stroustrup’s phone number?

Here’s the phone book – look it up!

(He’s under the C++’s, not the S’s!)

It’s easy because the elementsin a phone book are sorted by name.

Linear Search

Whose phone number is 399-728-9011?

Here’s the phone book – look it up!

Not so easy!

Linear Search

So get started.

Until you find the number:

Open the phone book to the first page,go to the first entry,

then the second entry,then the third…

go through all of them on page 1

if it’s not there, you’ll need to turn the page

Linear Search

On page 2:

go to the first entry,then the second entry,

then the third…all of them on page 2

if it’s not there, you’ll need to turn the page

Linear Search

On page 3:

go to the first entry, then the second…

Linear Search

THIS IS GOING TO TAKE FOREVER!

Or maybe I’ll get lucky.

Linear Search

This process is called a linear or sequential search.

A linear search examines all values in a sequenceuntil it finds a match or reaches the end.

Here’s the code:

Linear Search

int linear_search(int a[], int size, int value) { for (int i = 0; i < size; i++) { if (a[i] == value) { return i; } } return -1; }

-1 is returned to indicate that the value was not found.

ch12/lsearch.cpp(excerpt)

Analyzing the Linear Search Algorithm

If you assume that the value is present in the array, then the average search visits n/2 elements.

In the worst case, when the value it is not present,then all n elements must be inspected to verify the absence.

Either way, a linear search is an O(n) algorithm.

Binary Search

If you were told that the values in an array were already sorted,

(sounds a little bit like what we assumed at first in the Merge Sort!)

would you still consider using a linear search?

Binary Search

Consider this array:

Binary Search

Is 123 in the array?

Binary Search

Is 123 in the first half of this array?

Binary Search

Consider the last element in the first half (at index 3)?What is the relationship between 123

and the last value in the first half array?

Binary Search

Consider the last element in the first half (at index 3)?What is the relationship between 123

and the last value in the first half array?

Binary Search

Given that 123 is greater than 100,then because the data is sorted,123 must be in the upper half.

Binary Search

Repeat the process on the upper half array.

Binary Search

Consider the last index in the lower halfof the array from index 4 to 7.

That’s index 5:

Binary Search

123 is less than 290so it must be in the left half of this subarray

(or not in the array at all)

Binary Search

123 is less than 290so it must be in the left half of this subarray

(or not in the array at all)

Binary Search

Consider the last index in the lower halfof the very short subarray from index 4 to 5.

That at index 4:

Binary Search

123 is greater than 115 so you must look atthe very, very short array index 5:

Binary Search

123 is greater than 115 so you must look atthe very, very short array index 5:

Binary Search

There’s only one element in this subarrayand 290 is not 123.

123 is not found.

Binary Search

There’s only one element in this subarrayand 290 is not 123.

123 is not found.

Not found

Binary Search

Did we visit all n elements? No.

Not found

Binary Search

Not found

Binary Search

Binary search cuts the search in half each time.We do not visit every element.

Of course, this is only possible when thevalues in the array already sorted.

Not found

Binary Search

Here is the code for binary search:

Linear Search

int binary_search(int a[], int from, int to, int value){ if (from > to) { return -1; }

int mid = (from + to) / 2; if (a[mid] == value) { return mid; } else if (a[mid] < value) { return binary_search(a, mid + 1, to, value); } else { return binary_search(a, from, mid - 1, value); }}

ch12/bsearch.cpp (excerpt)

Analyzing the Binary Search Algorithm

Use the same technique as in the analysis of merge sortto determine the number of element visits required.

Because you look at the middle element,which counts as one comparison,

and then search either the left or the rightsubsequence, you have

As before, to understand the relationship,

evaluate T(n/2),using the same formula, you get:

As before, to understand the relationship,

evaluate T(n/2),using the same formula, you get:

Again, evaluate T(n/4),

using the same formula, and continue…

This generalizes to:

As in the analysis of merge sort,you make the simplifying assumption

that n is a power of 2, n = 2 m,where m = log2(n).

Then you obtain

As in the analysis of merge sort,you make the simplifying assumption

that n is a power of 2, n = 2 m,where m = log2(n).

Then you obtain

So binary search is an O(log(n)) algorithm.

Binary search is much faster than linear search,

is it worthwhile to sort a sequence firstand then use a binary search?

Sorting takes time, too!

Is it worth the time to sort?

It depends.

If you will only search the sequence once,then it is more efficient to pay

for an O(n) linear search than for anO(n log(n)) sort plus an O(log(n)) binary search.

But if one must make a number of searchesin the same sequence,

then sorting before the searchesis definitely worthwhile.

Chapter Summary

End Chapter Twelve

Slides by Evan Gallagher

Cfe2 ch12 final

Technology

Ch12 implementation

Ch12 ISM.pdf

Foster Ch12

Ch12 Polymer

Hrm10e ch12

Ch12 Ready

Cfe2 ch07 final-mod-1

Ebc10 e ch12-instructor ppt - final

Ch12 Statistics

Economics ch12

Brakes Ch12

Ch12 Kinematics_of_a_Particle

ch12 (Probability Part II - general rules) - York · PDF filech12 (Probability Part II - general rules) ch12-links . Kerns probability using R Ch12 image. ch12 exercise: 12.1, 12.3,

Cfe2 ch13 final

Cengel ch12

cfe2-h-168-celtic-treasures-differentiated-reading-comprehension … · 2021. 1. 5. · Title: cfe2-h-168-celtic-treasures-differentiated-reading-comprehension-activity_ver_5.pdf

Ch12 outline

Lussier3e ch12

SSOM2 Ch12

MedTerm Ch12