45
1 Sorting Algorithms (Basic) Search Algorithms Binary Interpolation Big-O Notation Complexity Recursion Intro to Algorithms Selection Sort for int’s for string’s Templated Functions Recursion

1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

Embed Size (px)

Citation preview

Page 1: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

1

Sorting Algorithms (Basic)Search Algorithms

BinaryInterpolation

Big-O NotationComplexity

Sorting, Searching, Recursion Intro to Algorithms

Selection Sortfor int’sfor string’s

Templated Functions

Recursion

Page 2: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

2

Sorting Elementary sorting algorithms

– Bubble– Insertion*– Selection

Page 3: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

3

Bubble Sort

const size_t N = <constant>;int A[N];// ‘A’ then is populated // Sort followsfor (size_t i = N - 1; i >= 1; --i) for (size_t j = 0; j < i; ++j) if (A[j] > A[j + 1]) std::swap (A[j], A[j + 1]);

Complexity?

Page 4: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

4

Insertion Sort

for (i = 1; i < N; ++i){ // Invariant: A[0..i) sorted v = A[i]; // Element to place j = i; // Index to place ‘v’ while (j >= 1 && v < A[j–1]) { A[j] = A[j–1]; --j; } // Invariant: v >= A[j-1] or j == 0

A[j] = v;}

Page 5: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

5

Selection Sort

// Find N-1 smallest and placefor (i = 0; i < N - 1; ++i) { min = i; for (j = i+1; j < N; ++j) if (A[j] < A[min])

min = j; //if (i != min) swap (A[i], A[min]);}

Page 6: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

6

Searching Sequential

– Examine 1st element, 2nd, …– Assume int A[N];– int* pos = std::find (A, A+N, val);

Binary– Requires sequence be sorted– More efficient – quantify– int* pos = lower_bound (A, A+N, val);

// *pos >= val or pos @ end– bool found = binary_search (A, A+N, val);

Interpolation– Sorted sequence– O(lg lg (N)) average

Page 7: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

7

Binary Search If data are sorted can use Binary Search Look at middle element

– If match, return location– If smaller, search left sub-array– If larger, search right sub-array

Page 8: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

8

Binary Search

Case 1: Matchif (target == midValue)

return mid;

m idfirs t

target

C as e 1: target = m id valueS earc h is d o ne

las t-1 las t

Page 9: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

9

Binary Search

Case 2: Target smaller than middle value// Search the left sublist

if (target < midValue)

<reposition last to mid><search

sublist arr[first]…arr[mid-1]

las t-1firs t

target

C as e 2: target < m id valueS earc h lo w er s ub lis t

m id -1 las t

Page 10: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

10

Binary Search

Case 3: Target larger than middle value// Search upper sublist if (target > midValue)

<reposition first to mid+1><search sublist arr[mid+1]…arr[last-

1]>

C as e 3: target > m id valueS earc h up p er s ub lis t

las t-1new firs t = m id + 1

firs t

target

las t

Page 11: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

11

Illustrating the Binary Search- Successful Search

1. Search for target = 23Step 1: Indices first = 0, last = 9, mid = (0+9)/2 = 4.

Since target = 23 > midvalue = 12, step 2 searches the upper sublist with first = 5 and last = 9.

m id

-7 3 5 8 12 16arr

0 1 2 3 4 5

23 33 55

6 7 8 9

Page 12: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

12

Illustrating the Binary Search- Successful Search

Step 2:Indices first = 5, last = 9, mid = (5+9)/2 = 7.

Since target = 23 < midvalue = 33, step 3 searches the lower sublist with first = 5 and last = 7.

m id

-7 3 5 8 12 16arr

0 1 2 3 4 5

23 33 55

6 7 8 9

Page 13: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

13

Illustrating the Binary Search- Successful Search

Step 3: Indices first = 5, last = 7, mid = (5+7)/2 = 6.

Since target = midvalue = 23, a match is found at index mid = 6.

m id

-7 3 5 8 12 16arr0 1 2 3 4 5

23 33 55

6 7 8 9

Page 14: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

14

Illustrating the Binary Search- Unsuccessful Search

Search for target = 4. Step 1: Indices first = 0, last = 9, mid = (0+9)/2 = 4.

m id

-7 3 5 8 12 16arr

0 1 2 3 4 5

23 33 55

6 7 8 9

Since target = 4 < midvalue = 12, step 2 searches the lower sublist with first = 0 and last = 4.

Page 15: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

15

Illustrating the Binary Search- Unsuccessful Search

Step 2: Indices first = 0, last = 4, mid = (0+4)/2 = 2.

Since target = 4 < midvalue = 5, step 3 searches the lower sublist with first = 0 and last 2.

m id

-7 3 5 8 12 16arr0 1 2 3 4 5

23 33 556 7 8

Page 16: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

16

Illustrating the Binary Search- Unsuccessful Search

Step 3: Indices first = 0, last = 2, mid = (0+2)/2 = 1.

Since target = 4 > midvalue = 3, step 4 should search the upper sublist with first = 2 and last =2. However, since first >= last, the target is not in the list and we return index -1.

m id

-7 3 5 8 12 16arr

0 1 2 3 4 5

23 33 55

6 7 8 9

Page 17: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

17

Binary Search Algorithm

int binSearch (const int A[],

int first, int last,

int target)

{int midIndex;int midValue;

Page 18: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

18

Binary Search Algorithm

// Test for nonempty sublistwhile (first < last)

{midIndex = (first+last)/2;midValue = A[mid];if (target == midValue)

return midIndex;// Determine which sublist to

// search

Page 19: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

19

Binary Search Algorithm

else if (target < midValue)last = midIndex;

elsefirst = midIndex + 1;

}return -1;

}// Note half-open range convention

Page 20: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

20

Binary Search (Observation) midIndex = first + 1 * (last – first) / 2; Implicit assumption? Can we do better?

Page 21: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

21

Interpolation Searchint iSearch (const vector <int>& A, int v) {

int loc, s = 0, e = A.size () - 1;

if (v < A[s]) return (-1);

if (v >= A[e]) s = e;

while (s < e) {

loc = s + (e - s) * (v – A[s]) / (A[e] – A[s]);

if (v > A[loc]) { s = loc + 1; }

else if (v < A[loc]) { e = loc - 1; }

else return (loc);

}

if (v != A[s]) s = -1;

return (s);

}

Page 22: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

22

Algorithm Complexity Big-O notation – machine-independent means

for expressing efficiency of an algorithm Count key operations or stmts, and relate this

count to problem size using growth fn. f(N) = O(g(N)) if there are positive constants c

and n0 such that f(N) <= c * g(N) when N >= n0

Express op count as function of input size– f1(N) = 3N2 + 50N + 120– f2(N) = 1000 lg(N) + 50– f3(N) = 5

f1(N) = O(N2); n0 = 1, c = 200 3N2 + 50N + 120 <= 200N2, for N >= n0

f3(N) = O(1); n0 = 1, c = 6

Page 23: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

23

Big-O Notation

Page 24: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

24

Constant Time Algorithms

An algorithm is O(1) (constant time) when its running time is independent of input size

push_back on vector involves a simple assignment statement complexity O(1) (if capacity sufficient)

fro nt rear

D irec t In se r t a t R ear

Page 25: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

25

Linear Time AlgorithmsAn algorithm is O(N) when its running time is proportional to input size

S equential S earch fo r the Minim um E lem ent in an A rray

32 46 8 12 3

m in im u m elem en t fou n d in th e list a fter n com p a rison s

n = 51 2 3 4 5

Page 26: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

26

Polynomial AlgorithmsAlgorithms with running time O(N2) are

quadratic– practical only for relatively small values of N;

whenever N triples, the running time increases by ?

Algorithms with running time O(N3) are cubic– efficiency is generally poor; tripling N

increases the running time by ?

Page 27: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

27

Algorithm Complexity Chart

n log2n n log2n n2 n3 2n 2 1 2 4 8 4 4 2 8 16 64 16 8 3 24 64 512 256 16 4 64 256 4096 65536 32 5 160 1024 32768 4294967296 128 7 896 16384 2097152 3.4 x 1038

1024 10 10240 1048576 1073741824 1.8 x 10308

65536 16 1048576 4294967296 2.8 x 1014 Forget it!

Page 28: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

28

Logarithmic Time AlgorithmsThe logarithm of N, base 2, is commonly used when analyzing computer algorithmsEx. log2(2) = lg (2) = 1

log2(75) = lg (75) ~= 6.2288

When compared to functions N and N2, the function lg N grows very slowly

nn 2

lo g 2n

Page 29: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

29

Function Templates Same logic is appropriate for different types

– Templatize function Example: selection sort logic identical for

floats, ints, strings, etc.

Page 30: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

30

Selection Sort AlgorithmInteger Version

void selectionSort (int A[], int n) {

. . .int temp;

// int temp used for the exchangefor (pass = 0; pass < n-1; ++pass)

{. . .

if (A[j] < A[smallIndex])

// Compare integer elements . . .

}}

Page 31: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

31

Selection Sort AlgorithmString Version

void selectionSort (string A[], int n) {

. . .string temp;

// string temp used for the exchange

for (pass = 0; pass < n-1; ++pass) {

. . .if (A[j] <

A[smallIndex])// compare string elements

. . .

}

}

Page 32: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

32

Template Syntax Include keyword template followed list of formal

types enclosed in angle brackets In the argument list, each type is preceded by

the keyword typename (or class)

// Argument list with multiple template

// types template <typename T, typename U,

typename V, ...>

Page 33: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

33

Template Syntax Example

template <typename T>void selectionSort (T A[], int n){ int smallIndex;

// index of smallest element in the // sublist

int pass, j;T temp;

Page 34: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

34

Template Syntax Example

// pass has the range 0 to n-2for (pass = 0; pass < n-1; +

+pass){

// scan the sublist starting at // index pass

smallIndex = pass;

// j traverses the sublist // A[pass+1] to A[n-1]

for (j = pass+1; j < n; ++j) // update if smaller element found

Page 35: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

35

Template Syntax Example

if (A[j] < A[smallIndex])

smallIndex = j;// if smallIndex and

pass are not // the same location, exchange the

// smallest item in the sublist with // A[pass]

if (smallIndex != pass)

swap (A[pass],

A[smallIndex]);}

}

Page 36: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

36

Recursion Must ensure

– Recursive calls work on smaller problem– Eventually reach base case

Many problems have naturally recursive solutions (difficult to express iteratively)

Page 37: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

37

Recursion Examples Power function Binary search Towers of Hanoi Fibonacci numbers (*)

Page 38: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

38

Recursive Definition of the Power Function

Recursive definition– exponent n = 0 (base case)– n 1 which assumes we already know xn-1

Compute successive powers of x by multiplying the previous value by x

1,*

0,11 nxx

nx n

n

Page 39: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

39

Implementing the Recursive Power Function

Recursive power:double power (double x, int n)

// n is a non-negative integer{

if (n == 0)return 1.0; // base case

elsereturn x * power (x, n-1);

// recursive step} // runtime? how to improve?

Page 40: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

Solving Recurrence T(N) = T(N-1) + 1; T(0) = 0 Solve:

– T(N) = T(N-1) + 1– T(N-1) = T(N-2) + 1– T(N-2) = T(N-3) + 1– …– T(1) = T(0) + 1– T(0) = 0– :: T(N) = N = O(N)– (not good enough)

40

Page 41: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

41

Binary Search// Search A [s, e)int bs (int A[], int s, int e, int v){ if (s >= e) return -1; int m = (s + e) / 2; if (v == A[m]) return m; if (v < A[m]) return bs (A, s, m, v); else return bs (A, m+1, e, v);}

Page 42: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

Improvement

Reformulate definition to cut problem in half each time (?)

Suppose N = 2k for k >= 0 T(N) = T(N/2) + 1; T(0) = 0 Solve:

– T(N) = T(N/2) + 1– T(N/2) = T(N/4) + 1– T(N/4) = T(N/8) + 1– …– T(1) = T(0) + 1– T(0) = 0– :: T(N) = lg(N) + N = O(lg(N))

42

Page 43: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

43

Solving the Tower of Hanoi Puzzle using Recursion

N eed le A

. . . . . . . .

N eed le CN eed le B N eed le C

. . . . . . . .

N eed le BN eed le A

Move N-1 discs from A to B (using B as temp)Move disc N from A to CMove N-1 discs from B to C (using A as temp)

Recurrence?

Page 44: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

44

Fibonacci Numbers using Iteration

int fibIter (int n){ // Don’t use recursion!

if (n == 0 || n == 1)return n;

// Store previous two Fib #s int oneBack = 0, twoBack = 1,

current;

Page 45: 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection

45

Fibonacci Numbers using Iteration

// Compute successive terms

for (int i = 2; i <= n; ++i) {

current = oneBack + twoBack;twoBack = oneBack;

oneBack = current; }

return current;}// Counting adds: N >= 2// T(N) = N-1 = O(N)