MATH 224 – Discrete Mathematics

104/22/23

MATH 224 – Discrete MathematicsAlgorithms and Complexity

An algorithm is a precise recipe or set of instruction for solving a problem. In addition, to be considered an algorithm the set of instructions must solve the problem with a finite number of steps. In other words, if the algorithm is implemented on a computer, it must terminate with a solution. An algorithm should leave nothing to chance and must not have any infinite loops. (There are some algorithms that do use random or probabilistic techniques, and therefore may leave some things to chance.)

Algorithms solve problems such as sorting, find an object, finding the shortest path between two points, determining mathematical functions such as logarithms among others.

Specifying an Algorithms

•Informal English description•Formal English description•Pseudo-Code•Code in programming language (e.g., C++)

204/22/23

MATH 224 – Discrete MathematicsDescribing an Algorithm – Sorting an Array 0..n−1

Informal description of selection sort

•Find the smallest number and put it in the first position•Find the second smallest number and put it in the second position•Continue as above until the array is sorted

Formal Description

1. Set i = 02. Find the smallest value between positions i and n−13. Swap the value at i with the smallest value4. Set i = i+15. If i < n −1 go back to Step 2, otherwise quit

304/22/23

MATH 224 – Discrete MathematicsDescribing an Algorithm – Sorting an Array 0.. n−1

Pseudo-Code

void function sort(Array A[n]) for i in [0..n-2] →

pos := ifor j in [i+1.. n-1] →

if A[j] < A[pos] → pos := j

firofswap(A[i], A[pos])

rof end sort

C++ Version

void function sort (A_type A[ ], int n) { for (int i = 0; i < n-1; i++) {

pos = i;for (int j = i+1; j < n; j++) if (A[j] < A[pos])

pos = j; // fi// rofswap(A[i], A[pos]);

} // rof} // end sort

Note the use of “:=” for assignment as do Pascal and Ada.

404/22/23

MATH 224 – Discrete MathematicsDescribing an Algorithm – Insertion Sort Array 0.. n−1

Pseudo-Code

procedure insert_sort (A_type A[ ], int n) for i1 in [1..n-1] →

i2 := i1–1 item := A[i1]do (i2 ≥ 0) and (item < A[i2]) → A[i2+1] := A[i2]; i2 := i2 –1 odA[i2+1] := item

rofend insert_sort

C++ Version

void function insert_sort (A_type A[ ], int n) { int i1. i2; A_type item; for (int i1 = 1; i < n; i++) {

i2 = i1 –1 ;item = A[i1];while (i2 >= 0 && item < A[i2]) { A[i2+1] = A[i2]; i2 – – ;} // end whileA[i2 + 1] = item;

} // rof} // end insert_sort

504/22/23

MATH 224 – Discrete MathematicsBinary search of an Array 1.. n

Pseudo-Code – Algorithm 3 from the textbook (Page 172)

procedure binary_search(x: integer a[1..n] : integer)i := 1; j := nwhile i < j

m := └(i+j)/2┘ // just integer division in C++

if x > a[m] i := m+1else j := mfi

end whileif x = a[i] location is i // here = corresponds to else x is not in the array // C++ = =

end binary_search

Note the text uses “:=” for assignment as do Pascal and Ada.

604/22/23

MATH 224 – Discrete Mathematics

Exponentiation ap

Incremental version

int function exp(int a, int p) if p ≤ 0 → return 1 | p > 0 → val := a

for i in [1..p-1] →

val := val*a rof

fireturn val

end exp

How times does the code inside the for statement execute?

704/22/23


Exponentiation ap

Divide-and-Conquer version (using recursion)

int function exp(int a, int p) if p ≤ 0 → return 1 | p > 0 →

val := exp(a, p/2) if p is even →

return val*val

| p is odd →

return val*val*a

fifi

end exp How many times does the if statement execute?

804/22/23


Exponentiation ap

The Program (Run Time) Stack

int function exp(int a, int p) if p ≤ 0 → return 1 | p > 0 →

val := exp(a, p/2) if p is even →

return val*val| p is odd → return val*val*afi

fiend exp

MAIN a=3, p=11

exp(3, 5)

val = 9

exp(3, 2)

val = 3

exp(3, 1)

val = 1

exp(3, 0)

Return 1

exp(3, 11)

val = 243

1

3

9

243

177,147

904/22/23


Greedy Algorithm for Change

Algorithm 6 from the textbook (Page 175)

procedure change(n : integer c[1..r] : integer) // C is an array of coins sorted fromfor i := 1 to r // largest to smallest, and n is the

while n > c[i] // the total amount of change.add c[i] to change

n = n – c[i] end whilerof

end change

1004/22/23

MATH 224 – Discrete MathematicsAsymptotic Growth of Functions

Big-0 – an Upper Bound

Omega – a Lower Bound

Theta – a Tight Bound

1104/22/23


Examples Functions and their Asymptotic Growth

Polynomial Functions

0

200

400

600

800

1000

1200

1400

1600

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

3N^2 +10N+3 4N^2 3N^2

1204/22/23


T(N)O(N2)

(N2)

1 10 20

n0 = 10 for O

n0 = 0 for

3N2 3N24N2+10N +3

1304/22/23


Lg(N)

N 0.2

X axis * 105

0

5

10

15

20

25

30

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91 94 97

N^0.2 lg(N)

n0 = 5.7*106

N0.2

Log Function

1404/22/23

MATH 224 – Discrete MathematicsEvaluating Algorithms and Complexity

• Execute the algorithm on a standard computer with standard tests

• Problems: depends on the input and the computer

• Count the number of steps

• Problem: depends on the input and can be difficult to do

• Use big-O (asymptotic) evaluation

•Problem: provides an approximation and does not give the full picture

1504/22/23

MATH 224 – Discrete MathematicsSearching an Array

-4 -1 127 2321 31 45 8372 9286VALUE

INDEX 0 1 2 9 10 11

1604/22/23

MATH 224 – Discrete MathematicsLinear Search of an Array

-4 -1 127 2321 31 45 8372 9286VALUE

INDEX 0 1 2 3 4 5 6 7 8 9 10 11

Inspects 11 out of 12 elements O(N)

1704/22/23

MATH 224 – Discrete MathematicsBinary Search of an Array

-4 -1 127 2321 31 45 8372 9286VALUE

INDEX 0 1 2 5

1 2 3 4

Searches O(log n) locations (4 in this example)

Divides segments in half until a value is found

8 9 10 11

1804/22/23


Complexity of Algorithms

The complexity of an algorithm refers to how long it takes for an algorithm to solve a problem (time complexity) or how much memory the algorithm takes (space complexity). Complexity normally expressed as a function of one or more variables. For example, in the previous slides the binary search algorithm is said to take lg(n) steps, where n is the number of elements in the array being searched.

Complexity is normally express using O, Ω, or Θ notation since the precise running time is often difficult to calculate and will depend on the computer that is used.

1904/22/23


Types of Complexity

Most often when considering an algorithm either the worst-case or average-case complexity is analyzed. More rarely the best-case complexity may be considered.

Worst-case refers to the maximum for a given size problem. So for example the worst-case time complexity for binary search is Θ(lg(x)) for an array of size x.

Average-case refers to the average for an algorithm of a given size. It turns out that the average case is also Θ(lg(x)) for binary search since the average case takes one less iteration than does the worst-case.

Best-case refers to the best an algorithm can do in the ideal case. So, for example, if the item being searched is in the exact middle of an array, binary search will find it on the first iteration. What would be the best-case time complexity for binary search? Best case is rarely used.

2004/22/23

MATH 224 – Discrete MathematicsWorst-Case and Big-O

Because it is the easiest to calculate, worst-case analysis is most often used. In addition, it is often the most useful since it is often necessary to minimize the worst-case time or memory usage.

Also Big-O is most often used when stating the complexity of an algorithm since it is often easier to calculate than Θ. In most cases, people are in the habit of using Big-O when they really mean Θ. Do you have any idea why Big-O is more common than Θ?

2104/22/23


Complexity of Selection Sort


pos := ifor j in [i+1.. n−1] → if A[j] < A[pos] → pos := j firofswap(A[i], A[pos])

rof end sort

How many times does the outer for-loop execute?

The inner loop is more complex since the starting value keeps increasing. When i = 0, the inner loop executes n − 1 times, but when i = n − 2 it only executes once. So the number of steps is indicated by a sum of n − 1 + n − 2, n − 3, … 1.

How is this sum written using the summation symbol Σ?

2204/22/23

MATH 224 – Discrete MathematicsComplexity of Selection Sort

Thus the total number of steps in the worst-case is something like

Σn−1 (2i ) + 2(n−1) +1 i=1

What does this look like in simplified form?

n(n–1) + 2(n−1) + 1 = n2 + n − 1

What is this in Big-O or Θ?

Note that Σn (i) = n(n+1)/2 i=1


pos := ifor j in [i+1.. n−1] → if A[j] < A[pos] → pos := j firofswap(A[i], A[pos])

rof end sort

Documents

MATH 224 – Discrete Mathematics