Algorithms & Complexity

Algorithms &ComplexityDr. Ranette Halverson CMPS 2433 – Discrete Systems & Analysis

Chapter 1: Introduction, 1.4 1

Solving a Problem

• Existence of a Solution•How many Solutions•Optimal Solution

• Sample problems on page 1 of book.2

Algorithm

• Step-by-Step instructions to accomplish a task or solve a problem• Computer Program• Assembly instructions• Driving Directions • Recipe• Format & Detail level depends on

user 3

Efficiency of Algorithms

•Different algorithms for a single task may have different efficiency• Driving Directions – short route vs. long

route• Searching Algorithms• Sorting Algorithms•Must be able to evaluate efficiency

(speed) of computer algorithms 4

Comparing Algorithms

•Want to choose the “best” algorithm for tasks •Generally, best = fastest •But there are other considerations• Hardware (e.g. LaTex)• Size of data set • Data Structure•Need a standard measurement

5

Complexity = Speed = Efficiency

•NOT Really – but sort of• *Complexity is the number of basic

operations required by an algorithm•Will 2 algorithms with same complexity

take the same actual amount of time to run?? •Why or Why Not?* Means memorize 6

Complexity Examples

Read x, y, zx = y + zPrint x

How many operations?

Read x, y, z x = y / z Print x

Are these algorithms the same complexity?The same speed? 7

Complexity Examples

Read x, y, zx = y + zPrint x

Do 10 times Read x, y, z x = y + z Print x

8

Complexity Examples

Do 10 times Read x, y, z x = y + z Print x

Do n times Read x, y, z x = y + z Print x

9

Big Oh Notation

O(f(n)) – “Big Oh of f of n” n represents size of data setExamples• O(1) or O(c) • O(n)• O(n2)

Big Oh is an upper bound.10

Constant Complexity

•An algorithm with the same number of operations regardless of the data set is said to have CONSTANT COMPLEXITY•O(1) or 0(c)• See previous algorithms•Most significant algorithms are NOT

constant complexity11

Complexity Examples

Do 10 times Read x, y, z x = y + z Print x

Complexity?O(c) or O(1) - constant

Do n times Read x, y, z x = y + z Print x

Complexity?Depends on value of n!NOT Constant6 * n basic operationsO(6n) O(n) 12

Algorithm Complexity

•Number of operations in terms of the data set sizeDo 10 timesRead x, y, zx = y + zPrint xConstant Time Complexity – O(1) or O(c)

13

Algorithm Complexity

Do n times Read x, y, z x = y + z Print x

Time Complexity is dependent upon n 6n Operations - O(6n) O(n) 14

*Big Oh (yes, memorize)

O(f(n)) – Big Oh of f of n

A function g(n) = O(f(n)) if there exist 2 constants K and n0 such that|g(n)| <= K|f(n)| for all n >=n0

Big Oh is an upper bound.15

Evaluating xn

P = x k = 1While k < n

P = P * xk = k + 1

Print P

• Operations???

16

Evaluating xn

P = x k = 1While k < nP = P * xk = k + 1Print P

• Operations11n2(n-1)2(n-1)1

17

Total =3 + n + 2(n-1) + 2(n-1) 3 + n + 2n -2 + 2n – 2 5n – 1 = O(5n -1) O(n)

Trace: xn 54 P = x k = 1While k < n

P = P * xk = k + 1

Print P

(n=4, x=5)P = 5, k = 1P = 5*5 (25); k= 2P = 25*5 (125); k = 3P = 125*5 (625); k=4Print 625

18

Complexity & Rate of Growth• Complexity measures growth rate of

algorithm time in terms of data size• O(1) Constant• O(n) Linear• O(n2) Polynomial (any constant power)• O(5n) Exponential (any constant base)What do these functions look like when graphed?How big are real world data sets? (name some)

19

Polynomial Evaluation P(x) = anxn + an-1xn-1 + … +a1x + a0

Given values for x, a0, a1,…an

How many ops for anxn ?Operations to calculate each term in poly?

n + n-1 + n-2 +…+ 1 + 0 = (n*(n+1))/2Additions to combine the n+1 terms = nTotal Ops = (n2+n)/2 + n O(n2)What did we omit from analysis??? 20

Polynomial Evaluation (p.26) P(x) = anxn + an-1xn-1 + … +a1x + a0

Given values for x, a0, a1,…an

S = a0 , k = 1while k <=1

S = S + ak xk

k = k+ 1Print S

Can you spot any inefficiencies?

21

Polynomial Evaluation Algorithm (p.26) P(x) = anxn + an-1xn-1 + … +a1x + a0

Given values for n, x, a0, a1,…an

S = a0 , k = 1while k <=nS = S + ak xk

k = k+ 1Print S

*Slide 17: xk = 5k + 1

2n + 1#n(2+5(k+1)+1)2n1#Total = 2n +5kn + 5n + n

5kn + 8n22

Complexity of Poly. Evaluation

Total = (by line on previous slide)

=2 + (n + 1) + (5kn + 8n) + 2n + 1= 5kn + 11n + 4Worst case for k??= 5n2 + 11 n + 4 O(n2)

Note: this is for inserting code for xk

Function call adds additional overhead.23

Analysis of Polynomial

Compare Slides 20 and 22/23• Same Big Oh O(n2)• “Different details” but same results

24

Horner’s Rule (Polynomial Evaluation) P(x) = anxn + an-1xn-1 + … +a1x + a0

S = an, k = 1While k <= nS = Sx + an-k

k = k = 1Print s

2n + 1n * 3n *21

25Total = 2 + n+1 + 3n + n2 + 1 = 6n + 4 O(n)

Summary & Homework

• See table on page 31• In general, an algorithm is considered “good” if

its complexity is no more than some polynomial in n• For small n, some non-polynomial algorithms

may be “acceptable”Homework: Page 33+1 – 10, 23 – 26 – will discuss in class 27 – 30 (detail for each step as on slides, state Total & Big Oh) – Turn in 27-30 for grading

26